fix(memory): 64-bit page-size mask + ArenaAllocator move semantics - #731
Merged
Merged
Conversation
JeanPhilippeKernel
force-pushed
the
fix/arena-subarena-committed-size
branch
2 times, most recently
from
September 2, 2026 07:40
5448d05 to
fb54339
Compare
m_mem_page_size was declared unsigned long, which is 32-bit under Windows'
LLP64 data model (64-bit on macOS/Linux LP64). The page-align commit mask
(offset + size + m_mem_page_size - 1) & ~(m_mem_page_size - 1)
computed ~(page_size - 1) in 32-bit unsigned long, then zero-extended that
pattern to 64-bit when ANDed against the 64-bit size_t operand. Once
offset + size crossed 4 GB — exactly where this project's sub-arena budget
total lands (VFS 64MB + AssetManager 1GB + Input 4MB + ECS 512MB +
ImportPipeline 3.5GB + UIContext 128MB ~= 5.19 GB) — the zero-extended mask
cleared bit 32, collapsing commit_size below the already-committed size.
The subsequent commit_size - m_committed_size underflowed, and the garbage
size passed to VirtualAlloc(MEM_COMMIT) failed, returning nullptr, which
secure_memset then dereferenced — the access violation seen on Windows at
the first allocation past the 4 GB mark (RenderResourceManager::m_upload_pool).
Changes:
- m_mem_page_size is size_t everywhere: the Allocator.h field, Initialize's
parameter, and the local in MemoryManager.cpp. All arithmetic sites
(ArenaAllocateRaw, Resize, CreateSubArena) now operate in pure 64-bit.
- Both VirtualAlloc(MEM_COMMIT) call sites commit the full [base,
commit_size) range each time instead of the delta from m_committed_size —
an already-committed page is a documented Windows no-op to re-commit, so
this removes the only other place a stale m_committed_size could produce
an invalid size.
- CreateSubArena page-aligns the sub-arena start on every platform (was
Windows-only): correctness-required on Windows for clean VirtualAlloc
boundaries; on macOS/Linux it costs at most one page of padding per
sub-arena in exchange for a single code path instead of two.
- ArenaAllocator and PoolAllocator delete their copy constructor/assignment
— a shallow copy would double-VirtualFree/munmap the same pointer at
destruction. ArenaAllocator gains move construction/assignment so it can
still be relocated safely. Updated the one call site that copied
MemoryManager::MainArena by value (allocator_test.cpp) to use a reference.
Tests: renamed ArenaSubArenaPageAlignedOnWindows to ArenaSubArenaPageAligned
and removed its platform guard so it runs everywhere. All 531 existing
tests pass on macOS.
JeanPhilippeKernel
force-pushed
the
fix/arena-subarena-committed-size
branch
from
September 2, 2026 07:47
fb54339 to
6152858
Compare
JeanPhilippeKernel
added a commit
that referenced
this pull request
Sep 3, 2026
…mpleted/ Verified every doc's completion claim against the actual codebase — file existence, key symbols, checklist items — rather than trusting the Status line alone. Two categories of finding: Checklist-hygiene gaps (real implementation, boxes just never ticked): render-resource-manager.md, vfs-ticket2/4/5. Ticked every item after confirming the referenced file/symbol/test exists. render-resource-manager.md also got a naming-divergence note — the doc's proposed GPUResource.h/ GPUBuffer/GPUImage shipped as GpuAllocator.h's BufferView/BufferImage instead; functionally identical, different names. Genuine correctness gap: fly-camera-redesign.md claimed 'Implemented' with every one of its own checklist items unchecked, and describes an entirely different architecture (FlyCameraInput/FlyCameraState/EditorCameraController) than what's actually in FlyCameraController.h (CamState enum, SetViewportRect self-gating). Corrected the status to flag this and pointed at the real design so a future reader isn't misled. memory-allocator-audit.md — added a scope note: its 13 bugs (#497/#531) are genuinely all fixed, but later, unrelated allocator bugs were found and fixed independently this cycle (#680-683, #697, #728, #731) — pointed to the wiki's Memory Management page for the current picture. Moved to ZEngine/docs/completed/ (verified, no open items): asset-manager.md, memory-allocator-audit.md, vfs-design.md, vfs-ticket2/3/4/5/6, gpu-allocator-rearchitecture.md, render-resource-manager.md, system-scheduler.md, ui-system.md. Left in place — genuinely partial: tlsf-allocator-integration.md (Phase 3 blocked), logging-policy.md (real outstanding verification/benchmark tasks), fly-camera-redesign.md (needs a content rewrite, not just a status fix), and everything already marked Design/Planning/In Progress/Partially implemented. Cross-checked every backtick-quoted reference to the 12 moved filenames across the rest of the docs tree — all are informal textual mentions, not markdown hyperlinks, so nothing broke.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root cause
The Windows crash (
m_upload_poolallocation, access violation writing at a null pointer) traces tom_mem_page_sizebeing declaredunsigned long, which is 32-bit under Windows' LLP64 data model (64-bit on macOS/Linux LP64).The page-align mask:
computes
~(page_size - 1)in 32-bitunsigned long, then zero-extends that 32-bit pattern to 64-bit when ANDed against the 64-bitsize_toperand. Onceoffset + sizecrosses 4 GB — exactly where this project's sub-arena budget total lands (VFS 64MB + AssetManager 1GB + Input 4MB + ECS 512MB + ImportPipeline 3.5GB + UIContext 128MB ≈ 5.19 GB) — the zero-extended mask clears bit 32, collapsingcommit_sizeto a value smaller than what's already committed. The subsequentcommit_size - m_committed_sizeunderflows, and the garbage size passed toVirtualAlloc(MEM_COMMIT)fails, returningnullptr— whichsecure_memsetthen dereferences.Fix
m_mem_page_sizeis nowsize_teverywhere: theAllocator.hfield,Initialize's parameter, and the local inMemoryManager.cpp. All arithmetic sites (ArenaAllocateRaw,Resize,CreateSubArena) now do pure 64-bit arithmetic.VirtualAlloc(MEM_COMMIT)call sites now commit the full[base, commit_size)range each time instead of the delta fromm_committed_size— committing an already-committed page is a documented Windows no-op, so this removes the only other place a stale/incorrectm_committed_sizecould produce an invalid commit size.CreateSubArenapage-aligns the sub-arena start on every platform (was Windows-only): correctness-required on Windows for cleanVirtualAllocboundaries; on macOS/Linux it costs at most one page of padding per sub-arena in exchange for a single code path instead of two.ArenaAllocatorandPoolAllocatornow delete their copy constructor/assignment — a shallow copy would double-VirtualFree/munmapthe same pointer at destruction.ArenaAllocatorgains move construction/assignment so it can still be relocated safely. Updated the one call site that copiedMemoryManager::MainArenaby value (allocator_test.cpp) to use a reference instead.Test plan
m_upload_poolallocation without access violation