Skip to content

Binary test data is not fully covered by .gitattributes and can be mangled by EOL conversion #146

Description

@SkowronskiAndrew

Binary test data under TestCommon/Data is only partly covered by the binary rules in
.gitattributes, so some of it is stored as text and is exposed to end-of-line conversion.

.gitattributes sets * text=auto eol=lf and then names specific binary paths:

assetbundle binary
scenes binary
level* binary
*.dll binary
*.dylib binary
*.so binary
...

Unity data files whose names do not match those patterns fall through to text=auto, which leaves
the decision to git's heuristic — and that only looks for a NUL byte in the first 8000 bytes.

sharedassets0.assets.resS is the clearest case. It is 512 KB with just 8 NUL bytes, none of them
early, so git classifies it as text:

$ git check-attr -a TestCommon/Data/PlayerWithTypeTrees/sharedassets0.assets.resS
... text: auto
... eol: lf

$ git diff --numstat <commit-that-added-it>
1   0   TestCommon/Data/PlayerWithTypeTrees/sharedassets0.assets.resS

(A file git considered binary shows - - there, as the .assets and level* files in the same
folder do.)

Nothing is corrupted today. Both checked-in .resS files happen to contain zero CR bytes, so
eol=lf normalization is a no-op and they round-trip byte for byte — verified by comparing the
SHA-256 of the blob in git against the source file. This is a latent hazard, not a live bug.

The risk is the next binary fixture whose bytes happen to include 0d 0a. On checkout it would
have those bytes rewritten to 0a, producing a corrupt file that still looks plausible, and the
resulting test failure would point at the parser rather than at git. Anything without an extension
already covered by a binary rule is affected — .resS, .resource, .assets, .bundle,
.buildreport, .cf, and the extensionless CAB-* files.

Suggested fix: mark the data folder's binary formats explicitly, e.g.

TestCommon/Data/** -text

or per-extension binary rules for *.resS, *.resource, *.assets, *.bundle, *.buildreport,
*.cf alongside the existing ones. Worth checking afterwards that no already-committed file changes
content (they should not — the ones that would have been mangled are the ones that do not exist
yet), and that the README.md files inside TestCommon/Data are not caught by a blanket rule.

Found while adding Unity 6.7 test data in #145, where the new .resS reproduced the same
classification.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions