fix(memory): lazy VirtualAlloc commit on Windows + mmap overcommit on macOS/Linux - #728
Merged
Merged
Conversation
… macOS/Linux Windows: CreateSubArena no longer calls Allocate(size) which triggered VirtualAlloc(MEM_COMMIT) for the full sub-arena budget upfront. With ImportPipeline (3.5 GB) + AssetManager (1 GB) + VulkanDevice (1 GB) committed before UIContext, pagefile quota was exhausted and the UIContext VirtualAlloc returned NULL. Sub-arenas now start with m_committed_size = 0 and commit pages lazily in ArenaAllocateRaw. Sub-arena m_memory is aligned to m_mem_page_size (4 KB) so commit boundaries never cross into adjacent sub-arenas' pages. macOS/Linux: Replace mmap(PROT_NONE) + per-allocation mprotect with mmap(PROT_READ|PROT_WRITE) + m_committed_size = size. The OS uses overcommit — physical pages are only backed on first write. This eliminates every mprotect syscall from the allocation hot path. Sub-arenas inherit m_committed_size = size (parent mmap covers their range). The mprotect branches in ArenaAllocateRaw and Resize are removed entirely for macOS/Linux. Tests: updated ArenaSubArenaLifecycle to assert platform-specific m_committed_size; added ArenaSubArenaMultipleLargeSubArenas (regression for the UIContext Windows bug) and ArenaSubArenaPageAlignedOnWindows.
JeanPhilippeKernel
added
area-linux
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CreateSubArenacalledAllocate(size)which triggeredVirtualAlloc(MEM_COMMIT)for the full sub-arena budget upfront. By the time UIContext was allocated, earlier sub-arenas (ImportPipeline 3.5 GB + AssetManager 1 GB + VulkanDevice 1 GB) had already consumed pagefile quota and the UIContext commit returned NULL.m_committed_size = 0and commit pages lazily inArenaAllocateRaw. Sub-arenam_memoryis page-aligned (m_mem_page_size) soVirtualAlloc(MEM_COMMIT)boundaries never cross into adjacent sub-arenas' pages.mmap(PROT_NONE)+ per-allocationmprotectwithmmap(PROT_READ|PROT_WRITE)+m_committed_size = size. The OS uses overcommit — physical pages are backed on first write only. Eliminates everymprotectsyscall from the allocation hot path. Themprotectbranches inArenaAllocateRawandResizeare removed entirely.Test plan
ArenaSubArenaLifecycle— assertsm_committed_size = 0on Windows,= sizeon macOS/Linux; allocation into sub-arena still worksArenaSubArenaMultipleLargeSubArenas— regression: 3 x 1 MB sub-arenas from a 4 MB parent all succeed and have non-overlapping rangesArenaSubArenaPageAlignedOnWindows— Windows only: sub-arenam_memoryis a multiple ofdwPageSize