Support new-format (0x1000c800) zone WLDs and fix later-era parsing panics#55
Open
djhenry wants to merge 2 commits into
Open
Support new-format (0x1000c800) zone WLDs and fix later-era parsing panics#55djhenry wants to merge 2 commits into
djhenry wants to merge 2 commits into
Conversation
Loading many post-Velious zone WLDs (poknowledge, povalor, paineel, …) panicked instead of returning meshes, because the parser assumed the classic fragment set and aborted (or unwrapped a None) on anything newer. Four independent panic sites are made tolerant so terrain still loads: - WldDoc::parse: a fragment whose body can't be parsed (unknown type id, or an unsupported later-era variant such as a 0x37 DmTrackDef2 with an unexpected layout) is replaced in place by a new FragmentType::Unknown placeholder instead of failing the whole document. Fragment references are by 1-based index, so keeping the slot preserves every other fragment's index; references to the bad fragment resolve to None and the raw bytes still round-trip. Only structural (header/string-table) failures remain fatal. This also removes the `// FIXME: do not panic` in Wld::load for these files. - RenderMethod::from_u32: unknown user-defined material type codes now fall back to MaterialType::Diffuse rather than unwrapping None. - Mesh::primitives: a face-material group running past the face list is clamped to the available faces instead of `.expect()`-panicking. - Primitive::material: a material index past the palette is clamped to the last material instead of panicking in Vec::remove. Adds unit tests for the unknown-material fallback and the unknown-fragment parse tolerance. Drops the now-unused itertools dependency. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…lerating garbage
The previous commit stopped the panics on later-era (Luclin/PoP) zone
WLDs, but the extracted content was still wrong: ~27% of meshes in the
25 new-format zone files (povalor, pofire, abysmal, ...) decoded with
garbage faces and material groups without any parse error. Root causes,
confirmed against the decompiled RoF2 client (EQGraphicsDX9.dll,
FUN_100bfae0 / FUN_100c0470):
- DmSpriteDef2 (0x36) texture coordinates are 2xf32 in new-format WLDs
(header version 0x1000c800), not 2xi16. Reading them as i16 shifted
every following array (normals, colors, faces, material groups) onto
misaligned bytes. WldHeader::format() now mirrors the client's
`version & 0xffff0000 == 0x10000000` check and is threaded into
DmSpriteDef2::parse_with_format; UVs are a TextureCoordinates
Old/New enum and Mesh::texture_coordinates() handles both encodings.
- DmTrackDef2 (0x37) never had a trailing `size6: u16` field; those
bytes are 4-byte alignment padding. Parsing failed whenever the frame
data happened to end 4-aligned (the original panic, then 36 Unknown
placeholders in classic *_obj.wld files, losing animated-vertex
data). The field is removed.
- User-defined material-type codes not in the known table (0x18, 0x2a,
...) were clobbered to Diffuse on parse, destroying data. The native
client rejects these codes too ("Invalid material type.") and falls
back to a default, but the parser now preserves them losslessly as
MaterialType::Unknown(u32).
Also adds the missing FragmentType::Unknown arm in wld-cli (left
non-exhaustive by the previous commit) and updates the defensive-clamp
comments in lib.rs: well-formed files, including new-format zones,
never trip them.
Verified across all 1079 RoF2 archives with a parse -> re-serialize ->
byte-compare sweep: new-format 0x36 mismatches drop from 72,514 to 0,
0x37 Unknown placeholders from 36 to 0, and 102,673 meshes show zero
face-group or material-index overruns. Remaining diffs are 1-3 bytes of
non-zero garbage in trailing padding written by the original tools.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Later-era (Luclin/PoP-era) zone WLDs either panicked the parser or silently decoded to garbage. This PR fixes both, in two commits:
1.
fix(wld): tolerate unparseable fragments instead of panickingFragmentType::Unknownplaceholder that keeps its slot (fragment refs are by 1-based index) and round-trips its raw bytes, instead of failing the whole document.Mesh::primitives/Primitive::materialclamp out-of-range face groups and material indices instead of panicking on malformed meshes.2.
feat(wld): parse the new WLD format correctlyThe 25 Luclin/PoP zone WLDs in later clients use header version
0x1000c800, and ~27% of their meshes decoded with garbage faces/material groups without any parse error. Root causes were confirmed against the decompiled RoF2 client (EQGraphicsDX9.dll):DmSpriteDef2(0x36) texture coordinates are 2×f32 in new-format WLDs, not 2×i16. Reading them as i16 shifted every following array (normals, colors, faces, material groups) onto misaligned bytes.WldHeader::format()now mirrors the client'sversion & 0xffff0000 == 0x10000000check and is threaded intoDmSpriteDef2::parse_with_format; UVs are aTextureCoordinates::Old/Newenum andMesh::texture_coordinates()handles both encodings.DmTrackDef2(0x37) never had a trailingsize6: u16field — those bytes are 4-byte alignment padding. Parsing failed whenever the frame data ended 4-aligned (this was the original panic on files likecabeast_obj.wld). The field is removed.Diffuse, destroying data on round-trip). The native client rejects these codes too ("Invalid material type.") and falls back to a default, but the parser now preserves them losslessly asMaterialType::Unknown(u32).Breaking API changes
DmSpriteDef2::texture_coordinatesis now theTextureCoordinatesenum.DmTrackDef2::size6is removed.MaterialTypegains aUnknown(u32)variant and no longer derivesFromPrimitive/ToPrimitive(useMaterialType::from_code/u32::from).Verification
.s3darchives in a RoF2 install with a parse → re-serialize → byte-compare harness: new-format 0x36 mismatches drop from 72,514 to 0, 0x37 failures from 36 to 0; across 102,673 meshes there are zero face-group or material-index overruns. (The only remaining diffs are 1–3 bytes of non-zero garbage in trailing padding written by the original SOE tools, which the client never reads.)🤖 Generated with Claude Code