Complete roadmap foundations and optimize chunk streaming

.github/workflows/tests.yml

@@ -10,11 +10,8 @@ name: PR tests
 # only becomes available once the file is on the default branch.
 #
 # Known follow-ups before making this required:
-#   - tools/stream_full_verify.gd caps its wait at 240 * 0.25s (60s); slow
-#     (4 vCPU) runners can exceed that and fail spuriously. Raise MAX_WAIT_TICKS.
 #   - The parse check only walks scripts the editor loads; an unreferenced
 #     broken script needs a dedicated scanner (load() + can_instantiate()).
-#   - Unit tests for pure logic still need a runner (ROADMAP "Unit tests").
 on:
   pull_request:
     types: [opened, synchronize, reopened, ready_for_review]
@@ -61,11 +58,19 @@ jobs:
           CHECK_TIMEOUT_SECONDS: "120"
         run: |
           bash .github/scripts/run_checks.sh \
+            tools/unit_tests.gd \
+            tools/chunk_priority_verify.gd \
+            tools/lod_batch_verify.gd \
+            tools/mesh_parity_verify.gd \
+            tools/asset_license_verify.gd \
             tools/audio_verify.gd \
             tools/weather_verify.gd \
             tools/ui_flow_verify.gd \
             tools/ui_scale_verify.gd \
             tools/input_rebind_verify.gd \
+            tools/loading_progress_verify.gd \
+            tools/first_run_hints_verify.gd \
+            tools/motion_camera_verify.tscn \
             tools/photo_mode_verify.gd \
             tools/player_target_verify.tscn \
             tools/player_survival_verify.gd \
@@ -108,6 +113,8 @@ jobs:
         run: |
           bash .github/scripts/run_checks.sh \
             tools/worldgen_verify.gd \
+            tools/population_benchmark.gd \
+            tools/population_pruning_verify.gd \
             tools/worldgen_water_verify.gd \
             tools/worldgen_ore_verify.gd \
             tools/worldgen_structure_verify.gd \
@@ -175,11 +182,13 @@ jobs:
         run: |
           bash .github/scripts/run_checks.sh \
             tools/stream_full_verify.gd \
+            tools/stream_transition_verify.gd \
             tools/stream_soak_verify.gd \
             tools/lod_mode_verify.gd \
             tools/chunk_data_compression_verify.gd \
             tools/world_storage_verify.gd \
-            tools/light_invalidation_verify.gd
+            tools/light_invalidation_verify.gd \
+            tools/block_physics_verify.tscn
 
       - name: Upload check logs
         if: failure()

AGENTS.md

@@ -26,6 +26,7 @@ RedotCraft: a Minecraft-like voxel sandbox built with **Redot Engine** (Godot 4
 - `voxel_defs.gd`: CHUNK_SIZE 16, WORLD_HEIGHT 192, SEA_LEVEL 48, collision layer 1, padded-neighbor face/AO tables. Shared by the mesher and clouds.
 - `voxel_world.gd`: streams chunks around the player (default render_distance 10, unload +2). Terrain Detail defaults to Full Detail (`lod_distance = render_distance`) at every distance, including Extreme; opt-in Balanced LOD caps the full-detail radius at `COLLISION_DISTANCE + 2` (8) and uses compact chunks outside it. Extreme Full Detail can take minutes and several GB but never silently enables LOD. Distant full chunks omit collision until approach. Player movement is swept through resident chunk coordinates and stops just inside the truly unloaded boundary; visible LOD/full chunks remain traversable while their approach collision rebuild catches up. All movement pauses while the current chunk's collision is unavailable, and downward motion is suppressed on the entry tick so boosted flight cannot descend through a pending floor. Missing collision is discovered before scheduling, ordered nearest-first, and may use one bounded urgent worker slot so stale distant jobs cannot occupy every normal slot. `Chunk.lod` tracks the mode; jobs carry it and stale transitions are requeued (also checked against `_stream_center` on commit, like edit versions). Chunk generation and meshing run on `WorkerThreadPool` threads that may only read immutable state (`BlockRegistry`, `TerrainGenerator`, `ChunkMesher`); normal worker concurrency is half the logical cores (4-8), plus the one urgent near-player slot, and only the main thread touches scene nodes. LOD chunks store compact per-column top/sub/water arrays (~3 KB) instead of full voxel data; edits refuse LOD chunks. The full-chunk palette/RLE codec remains verifier-covered, but live cold-chunk compaction is disabled: repeated neighbor decoding caused flight stutter and could feed a stale short snapshot to remeshing. Balanced LOD is the supported memory-saving path.
 	- Stream-center rebuilds enumerate nearest-first rings without sorting the full square, completed collision results commit before distant visuals, and a single-cell edit queues its owner ahead of neighbor-light remeshes. Physics bodies/shapes exist only inside the collision radius; distant visual chunks do not register empty bodies with the physics server.
+	- `begin_initial_stream()` keeps player/photo input locked until `initial_stream_ready` confirms the authoritative 3x3 collision ring. `VoxelWorld` uses `PROCESS_MODE_ALWAYS`, so worker collection and commits continue under menus while water, fire, and gravity simulation stay paused.
 	- Persistence: `world/world_storage.gd` stores versioned metadata and deterministic sparse edits in ZSTD region files. Region I/O and edit hydration stay on the main thread; workers receive immutable per-chunk snapshots. The clean-region cache is capped at 16, dirty regions cannot be evicted, unloaded disk-backed edit buckets are released, and temp/backup replacement preserves a valid recovery backup. Save metadata includes player/inventory/slot/spawn/time/moon/weather; `Main` autosaves every 30 seconds and flushes on exit/new world.
   - Spawning: `TerrainGenerator.find_spawn_position()` only knows terrain height, so `VoxelWorld.find_safe_spawn()` searches outward in the generated chunk data for a column whose topmost block is solid (not leaves) with two air cells above; `Main` calls it after `setup_player()`. It depends on which chunk jobs have committed, so the result varies run to run — pin `GameConfig.world["seed"]` and use `get_spawn_position()` (deterministic) for reproducible captures.
 - `terrain_generator.gd`: FastNoiseLite climate biomes, caves, data-driven ores, and tree/structure stamping. Deterministic per seed, no mutable state after `configure()`. `WorldGenConfig.CURRENT_VERSION` is 14: v8 keeps legacy cave classification, v9 adds dripstone caves/highland boulders, v10 preserves the original default-off spline terrain/elevated hydrology experiments, v11 adds climate variants, 2D spline profiles, routed elevated hydrology, organic cave regions, large trees, improved lava basins, region POIs, and soft ceiling behavior, v12 replaces the frequent artificial cobblestone outcrop props with low stone pebbles, v13 removes rigid fallen-log/driftwood decorations, and v14 generates ores independently of cave density in non-flat worlds (older worlds retain the old gate). `generate_data(pos, edits, lod)` returns either full voxel data or compact LOD columns; both modes consume the same authoritative inland-water field.
@@ -38,6 +39,7 @@ RedotCraft: a Minecraft-like voxel sandbox built with **Redot Engine** (Godot 4
   - Cutout blocks (leaves, glass, torch) are antialiased with `alpha_to_coverage`, so their edge quality depends on MSAA being enabled.
 - Water: `BLOCK_WATER` (16) is a source; IDs 28-34 are flowing levels 7..1, resolved through `BlockRegistry.is_water_id()`/`water_level()`/`water_id_for_level()`. `VoxelWorld` runs a 4 Hz cellular flow tick (`_water_tick`, 1024 cells/tick) seeded by breaks/places near water; flow replaces only air or weaker flowing water, falls when the cell below is air, and dries when unfed. Every change goes through `_record_edit()`, so the settled state is reapplied on chunk regeneration; `_edited_blocks` is mirrored into `_edits_by_chunk` so job snapshots and `TerrainGenerator.generate_data()` only see the chunk's own edits. Meshes are non-opaque, non-breakable, no collision and get the same `ARRAY_CUSTOM0` light attribute as blocks; flowing surfaces render at `_water_top(level)` and set vertex-color alpha so `water.gdshader` can calm the waves. `water.gdshader` samples `hint_screen_texture` for screen-space refraction (wave-distorted background with a faint chromatic split, carried through `EMISSION` so the refracted scene is not lit twice), `hint_depth_texture` for shore blending (shallow water is lighter, more transparent, and foams at the edge), and block light for warm glow.
 - Fire: burning a block never replaces it. A `BlockRegistry.FLAG_FLAMMABLE` log/leaf/dry plant/mushroom enters the `_burning` state for `FUEL_BURN_TICKS` with its texture and collision intact, and `FireOverlay` (`world/fire_overlay.gd` + `fire_overlay.gdshader`/`smoke_overlay.gdshader`) clusters flame and smoke billboards on it — one MultiMesh per effect, animated entirely in-shader from per-instance phase/progress and rebuilt only on the 4 Hz `VoxelWorld._fire_tick`. Burning blocks crawl into flammable neighbours one at a time, paced by a per-block `FIRE_IGNITION_DELAY_TICKS` (4), `FIRE_SPREAD_PER_TICK` (1) on every `FIRE_SPREAD_PERIOD_TICKS` (2), plus a per-cell `FIRE_SPREAD_INTERVAL_TICKS` cooldown (~2 blocks/s), and can chain-detonate adjacent TNT/nukes. When the timer expires the block crumbles to air and that edit goes through `_record_edit()`, so the settled result survives chunk regeneration (mid-burn state does not persist). `BLOCK_FIRE` (70) is only the standalone cutout cross flame that flint-and-steel leaves on a non-flammable face (`FLAG_CUTOUT | FLAG_CROSS | FLAG_EMISSIVE`, animated by `block.gdshader`); it decays when starved/unsupported and can catch adjacent fuel. `use_flint_and_steel()` detonates an explosive, starts the clicked flammable block burning, or lights the clicked face. `tools/fire_verify.tscn` covers the lot.
+- Gravity: sand, red sand, and gravel use a deduplicated 4 Hz queue seeded by edits and once per full-detail chunk residency after persisted edits hydrate. Moves record both cells through `_record_edit()` and refuse unloaded/LOD destinations; cross blocks support falling blocks because replacement has no item-drop owner. `tools/block_physics_verify.tscn` covers persistence and lifecycle behavior.
 - Underwater biomes: `OCEAN`/`DEEP_OCEAN` are split with `KELP_FOREST`, `SEAGRASS_MEADOW`, `CORAL_REEF`, `FROZEN_OCEAN` (BiomeCatalog IDs 15-18), chosen in `TerrainSampler._underwater_biome()` from depth, temperature, and patch noise; `TerrainSampler._ocean_floor_at()` shapes the shelf, slope, and abyss. `VoxelPopulator._decorate_underwater()` grows seabed plants from `DecorationCatalog._underwater_sets` (new blocks 52-58: coral substrate, seagrass, kelp, coral fan/branch, sponge, anemone) and only replaces water; the water mesh culls faces against `FLAG_CROSS` plants and `get_water_ambience()` treats a camera inside one as submerged, so foliage does not sit in glassy air pockets. `VoxelWorld.get_water_ambience()` + `Main._update_underwater()` grade the overlay, `DayNightCycle.set_underwater()` the fog/light/caustics global, and `AudioManager.set_underwater()` a Master low-pass; keep their throttled world queries out of per-frame hot paths.
 
 ## Weather / atmosphere
@@ -93,13 +95,14 @@ RedotCraft: a Minecraft-like voxel sandbox built with **Redot Engine** (Godot 4
 
 - Parse check after editing scripts or scenes: `redot --editor --headless --path . --quit` (a nonzero "error" count means a script/scene fails to load).
 - Headless verifiers, from the repo root (`redot --headless --path . --script res://tools/<file>`):
-  `worldgen_verify.gd` (parity, seams, continuity, roughness), `worldgen_biome_verify.gd` (region size, vegetation density), `worldgen_river_verify.gd` (river width, banks, floodplain, bed), `worldgen_island_verify.gd` (detached large/small island landmasses), `worldgen_lod_verify.gd` (LOD parity, canopies, patches, shading), `worldgen_water_verify.gd` (aquifers), `worldgen_ore_verify.gd`, `worldgen_structure_verify.gd`, `worldgen_poi_verify.gd`, `worldgen_audit_verify.gd`, `worldgen_config_sweep_verify.gd`, `worldgen_spline_verify.gd`, `worldgen_elevated_hydrology_verify.gd`, `worldgen_tree_verify.gd`, `worldgen_cactus_verify.gd`, `world_storage_verify.gd`, `lod_mode_verify.gd`, `chunk_data_compression_verify.gd`, `stream_soak_verify.gd`, `light_invalidation_verify.gd`, `audio_verify.gd`, `weather_verify.gd`, `ui_flow_verify.gd`, `ui_scale_verify.gd`, `input_rebind_verify.gd`, `stream_full_verify.gd`, `photo_mode_verify.gd`, `frame_pacing_verify.gd`, `shadow_proxy_verify.gd`.
+  `unit_tests.gd`, `worldgen_verify.gd` (parity, seams, continuity, roughness), `worldgen_biome_verify.gd` (region size, vegetation density), `worldgen_river_verify.gd` (river width, banks, floodplain, bed), `worldgen_island_verify.gd` (detached large/small island landmasses), `worldgen_lod_verify.gd` (LOD parity, canopies, patches, shading), `worldgen_water_verify.gd` (aquifers), `worldgen_ore_verify.gd`, `worldgen_structure_verify.gd`, `worldgen_poi_verify.gd`, `worldgen_audit_verify.gd`, `worldgen_config_sweep_verify.gd`, `worldgen_spline_verify.gd`, `worldgen_elevated_hydrology_verify.gd`, `worldgen_tree_verify.gd`, `worldgen_cactus_verify.gd`, `world_storage_verify.gd`, `lod_mode_verify.gd`, `chunk_data_compression_verify.gd`, `stream_soak_verify.gd`, `stream_transition_verify.gd`, `chunk_priority_verify.gd`, `mesh_parity_verify.gd`, `population_pruning_verify.gd`, `lod_batch_verify.gd`, `light_invalidation_verify.gd`, `asset_license_verify.gd`, `audio_verify.gd`, `weather_verify.gd`, `ui_flow_verify.gd`, `ui_scale_verify.gd`, `input_rebind_verify.gd`, `loading_progress_verify.gd`, `first_run_hints_verify.gd`, `stream_full_verify.gd`, `photo_mode_verify.gd`, `frame_pacing_verify.gd`, `shadow_proxy_verify.gd`.
 - `redot --headless --path . res://tools/player_target_verify.tscn` is scene-based because autoload identifiers are unavailable to `--script` harnesses; use a `.tscn` whenever a test references `GameConfig`/`AudioManager`.
 - `redot --headless --path . res://tools/explosives_verify.tscn` checks flint-and-steel activation, TNT/nuke radii, spherical edit persistence, unbreakable blocks, and item-only placement rejection.
 - `redot --path . res://tools/shadow_proxy_measure.tscn` needs a rendering display: it loads the real scene with a flat canopy, sweeps the sun, and prints the shadow-edge temporal MAD for the leaf shadow proxy off/off-control/on with and without TAA.
 - `redot --headless --path . res://tools/fire_verify.tscn` checks flammability flags, fire ignition/spread/decay, chain detonation of explosives, the flint-and-steel face-light path, persistence reseeding, and the fire shader wiring. Scene-based for the same class_name reason as above.
+- `redot --headless --path . res://tools/block_physics_verify.tscn` checks persisted falling blocks; `motion_camera_verify.tscn` checks autoload-backed settings and camera behavior.
 - `tools/icon_check.gd` needs a rendering display and fails headless.
-- Benchmarks: `worldgen_benchmark.gd`, `worldgen_mesh_benchmark.gd`, and `worldgen_stream_benchmark.gd` (the last is minutes long at 32 chunks).
+- Benchmarks: `population_benchmark.gd`, `worldgen_benchmark.gd`, `worldgen_mesh_benchmark.gd`, and `worldgen_stream_benchmark.gd` (the last is minutes long at 32 chunks). Rendered profiles are `lod_batch_render_profile.tscn` and `gameplay_render_profile.tscn`.
 
 ## Conventions