Speed up authoritative chunk streaming

ROADMAP.md

@@ -15,14 +15,14 @@ Property names and file references are included so each item is easy to find.
 - [x] Full per-option menu — `ui/graphics_panel.tscn` ("Advanced Graphics..." in Settings); sections: Lighting, Shadows, Sky & Atmosphere, Post-Processing, Performance
 - [x] Per-option overrides on top of a preset with "Custom" indicator + "Reset to Preset"
 - [x] Graphics settings persist to `user://settings.cfg` (`[graphics] values` dictionary)
-- [x] Render distance slider 4-32 chunks — `ui/settings_category_panel.gd` (Display category) raises the cap from 16; full-detail chunks now extend to the render distance itself (`lod_distance = min(render_distance, VoxelWorld.MAX_FULL_DETAIL_DISTANCE)`). An "Extreme" toggle raises the slider to 100 chunks with an inline load-time/memory warning; only distances past the 32-chunk full-detail cap fall back to compact LOD. Collision shapes are built only within `VoxelWorld.COLLISION_DISTANCE` and added on approach, so distant full chunks hold data/mesh but no physics. Measured RD 32 (4225 full chunks, 8 workers) ~155 s CPU-bound; RD 16 ~37 s
+- [x] Render distance slider 4-32 chunks — `ui/settings_category_panel.gd` (Display category) raises the cap from 16; full-detail chunks now extend to the render distance itself (`lod_distance = min(render_distance, VoxelWorld.MAX_FULL_DETAIL_DISTANCE)`). An "Extreme" toggle raises the slider to 100 chunks with an inline load-time/memory warning; only distances past the 32-chunk full-detail cap fall back to compact LOD. Collision shapes are built only within `VoxelWorld.COLLISION_DISTANCE` and added on approach, so distant full chunks hold data/mesh but no physics. Authoritative streaming now generates data first and meshes once the neighbor ring is ready, avoiding temporary preview geometry and redundant seam regeneration; RD 10 (441 full chunks, 8 workers) completes in ~11.5 s in `stream_full_verify.gd`. The older pre-pipeline RD 32 ~155 s / RD 16 ~37 s measurements need refreshing
 
 ## Interface
 - [x] Deepslate & Ember UI overhaul — shared `UITheme`, bundled Iosevka typography, reusable motion and generated controls, dusk voxel title screen, sectioned settings/world creation, responsive inventory, texture-derived isometric block icons, compact telemetry HUD, item hotbar, focus restoration, and nested-modal cancel handling. Architecture notes live in `ui/README.md`
 - [x] Play and Settings information architecture — Play opens a world setup screen (world type, seed, clipboard import, Create Game) whose Advanced button opens the tunables-only world gen screen; Settings opens a category hub (Display, Graphics, Sound, Gameplay) where Advanced Graphics is itself a hub split into Lighting, Shadows, Sky & Atmosphere, Post-Processing, and Performance section panels. Cancel unwinds one layer at a time with focus restoration, and the pause menu reuses the hub. `tools/ui_flow_verify.gd` covers the flows
 - [x] Compact HUD — hotbar slots 76 -> 48 px with smaller icons, key hints, and counts, coords/stats chips reduced to ~150 px through the shared compact chip style, and status/selection labels moved above the smaller hotbar
 - [x] E opens the inventory — `inventory` action rebound from I; main-menu footer and docs updated
-- [ ] Loading and world-creation progress — show generation/streaming progress between menu and gameplay; high render distances block with no feedback today (RD 32 measured ~155 s CPU-bound)
+- [ ] Loading and world-creation progress — show generation/streaming progress between menu and gameplay; high render distances still need feedback while the refreshed RD 16/32 authoritative-stream measurements are collected
 - [ ] Chat / command console — `/time`, `/weather`, `/tp`, `/give`, `/seed`, and `/help` for testing and moderation; reuses `DayNightCycle.set_time()`, `WeatherSystem.toggle()`, and `GameConfig.world`
 - [ ] Death and respawn flow — prompts respawn at `Player.spawn_position` or quit to menu; pairs with the survival layer in Gameplay
 - [x] Photo mode and HUD hiding — F1 hides the HUD, F2 saves a HUD-free PNG to `user://screenshots/` (every CanvasLayer, so modals cannot leak into the frame), and P swaps in a detached free camera (`game/photo_mode.gd`, built on the `game/shadow_capture.gd` precedent: opt-in node added by `Main`, own key handling, `status_requested` feedback, `user://` output). The free camera inherits the player camera pose/FOV, flies with WASD/SPACE/SHIFT/CTRL, zooms with the wheel, and P/Esc return to the player. While active, `Main` follows it with chunk streaming (entry streams around the camera asynchronously; exit sync-commits the player's 3x3 ring before unfreezing so a long flight cannot drop the player), rain/cover, underwater grading, and the coords readout, and `Player.set_photo_mode()` freezes input and hides the targeting highlight and held block; entering photo mode hides the HUD, minimap, and F3 worldgen overlay, and exiting restores the pre-entry HUD state (an F1 press inside photo mode persists). `tools/photo_mode_verify.gd` covers HUD state, camera handoff, look/movement, player artifact and UI-layer hiding, and screenshot naming; `_get_shadow_capture_state()` records HUD/camera mode
@@ -56,7 +56,7 @@ Property names and file references are included so each item is easy to find.
 ## Geometry / materials
 - [x] `Texture2DArray` instead of the block atlas — `BlockRegistry` now builds one 64px layer per texture (per-image tint, alpha edge fix-up, mipmaps) and the mesher writes each face's layer into `ARRAY_CUSTOM1` (`ARRAY_CUSTOM_R_FLOAT`); `block.gdshader` samples `vec3(UV, layer)`, so there is no atlas inset and no cross-tile UV/mip bleeding
 - [ ] Chunk merging / MultiMesh — fewer draw calls for distant chunks
-- [x] Chunk LOD meshes — `ChunkMesher.build_lod()` distance meshes (top quad per column + exposed side runs, no light volume/AO/collision) for chunks beyond `lod_distance` (the render distance, capped at 32 chunks for Extreme distances); full detail covers the configured render distance and rebuilds LOD<->full on approach/retreat with the same version checks as edits. Distance chunks carry compact per-column top/sub/water arrays (~3 KB) instead of a full 192-block data array (~50 KB), and full chunks expand compact neighbors on worker threads for correct light and culling. Pinned benchmark: full mesh CPU averages ~181 ms vs ~13 ms for LOD, and LOD generation ~12 ms vs ~41 ms full
+- [x] Chunk LOD meshes — `ChunkMesher.build_lod()` distance meshes (top quad per column + merged exposed material runs, no light volume/AO/collision) for chunks beyond `lod_distance` (the render distance, capped at 32 chunks for Extreme distances); full detail covers the configured render distance and rebuilds LOD<->full on approach/retreat with the same version checks as edits. Distance chunks carry compact per-column top/sub/water arrays (~3 KB) instead of a full 192-block data array (~50 KB), and full chunks expand compact neighbors on worker threads for correct light and culling. Current pinned 5x5/eight-worker benchmark: 34.1 full chunks/s vs 180 compact LOD chunks/s, with ~93 ms full mesh vs ~3.6 ms merged LOD mesh average per concurrent job
 - [x] LOD tree canopies — compact columns bake the actual tree crowns using the same stamp functions as full chunks. Distance collection keeps only anchors inside the padded field and skips site-validity probes, which leave the field and re-enter the sampler (full parity cost ~19 ms per forest chunk, nearly full population). The top two tree blocks become the column's solid/sub pair, so distance forests read as real trees. Measured: LOD generation ~12.2 ms vs ~28.4 ms full, with the canopy pass ~1.2 ms; grove sample 142 canopy columns, open-ocean sample 0. `worldgen_lod_verify.gd` checks forest coverage and ocean exclusion
 - [x] LOD occlusion shading — distance meshes carry no AO or block-light volume, so far terrain read flat and cliffs showed a single soil stripe to the ground. Top faces now darken under taller cardinal neighbors (`LOD_AO_PER_BLOCK` 0.14, floor 0.52), side faces darken with depth, and sides below `LOD_SOIL_DEPTH` switch to stone (leaf columns stay leaf-textured). Pinned verification: flat spread 0.0000, hilly spread 1.5036, 5273 stone side faces; LOD mesh stays ~14 ms vs ~183 ms full
 - [x] LOD ground cover — compact columns now carry the same floor patches as full chunks (dirt/mud/gravel from the shared patch pass) plus low scrub bumps on grove edges and clearings (grove strength sampled on a stride-8 lattice, 12% of columns in the 0.45-0.85 band). `worldgen_lod_verify` checks patch propagation (2408 patch columns across a 7x7 grove area) alongside canopy coverage. Full generation ~30.8 ms, LOD ~12.5 ms

tools/chunk_loading_benchmark.gd

@@ -0,0 +1,157 @@
+extends SceneTree
+
+## Repeatable phase benchmark for chunk workers and detached main-thread mesh
+## resources. It reports full-detail and compact-LOD latency,
+## throughput, generation phase costs, mesh cost, and ArrayMesh/shape commit
+## cost. Keep the seed, positions, and collision policy pinned so before/after
+## runs are comparable.
+
+const SEED := 123456789
+const RADIUS := 2
+const CONFIG := {
+	"seed": SEED,
+	"world_type": 0,
+	"terrain_scale": 1.0,
+	"tree_density": 1.0,
+	"macro_scale": 384.0,
+	"river_density": 1.0,
+	"erosion_strength": 0.55,
+	"regional_erosion": 0.5,
+	"hydraulic_erosion": false,
+	"cave_density": 1.0,
+	"decoration_density": 1.0,
+}
+
+
+func _initialize() -> void:
+	var blocks := BlockRegistry.new()
+	var generator := TerrainGenerator.new()
+	generator.configure(CONFIG)
+	var mesher := ChunkMesher.new(blocks)
+	var positions := _positions()
+	var concurrency := clampi(OS.get_processor_count() / 2, 4, 8)
+	_warm_up(generator, mesher, blocks)
+	print("CHUNK LOAD BENCH seed=", SEED, " chunks=", positions.size(),
+		" concurrency=", concurrency)
+	_run_mode("full", false, generator, mesher, blocks, positions, concurrency)
+	_run_mode("lod", true, generator, mesher, blocks, positions, concurrency)
+	quit()
+
+
+func _warm_up(generator: TerrainGenerator, mesher: ChunkMesher, blocks: BlockRegistry) -> void:
+	var generated := generator.generate_data(Vector2i(3, 3), {}, false)
+	var result := mesher.build(generated.data, generated.max_y, generated.heights,
+		generated.foliage_tints, generated.water_tints, ChunkMesher.NeighborSet.new(), true)
+	_warm_resources(result, blocks)
+	generated = generator.generate_data(Vector2i(4, 3), {}, true)
+	result = mesher.build_lod(generated.lod_solid_y, generated.lod_solid_id, generated.lod_sub_id,
+		generated.lod_water_y, generated.lod_water_level, generated.max_y,
+		generated.foliage_tints, generated.water_tints, ChunkMesher.LodNeighbors.new())
+	_warm_resources(result, blocks)
+
+
+func _warm_resources(result: ChunkMesher.MeshResult, blocks: BlockRegistry) -> void:
+	ChunkMesher.arrays_to_mesh(result.verts, result.normals, result.uvs, result.colors,
+		result.indices, blocks.material, result.light, result.layers)
+	ChunkMesher.arrays_to_mesh(result.water_verts, result.water_normals, result.water_uvs,
+		result.water_colors, result.water_indices, blocks.water_material, result.water_light)
+	if not result.collision.is_empty():
+		var shape := ConcavePolygonShape3D.new()
+		shape.set_faces(result.collision)
+
+
+func _positions() -> Array[Vector2i]:
+	var out: Array[Vector2i] = []
+	for z in range(-RADIUS, RADIUS + 1):
+		for x in range(-RADIUS, RADIUS + 1):
+			out.append(Vector2i(x, z))
+	out.sort_custom(func(a: Vector2i, b: Vector2i) -> bool:
+		return a.length_squared() < b.length_squared()
+	)
+	return out
+
+
+func _run_mode(label: String, lod: bool, generator: TerrainGenerator, mesher: ChunkMesher,
+		blocks: BlockRegistry, positions: Array[Vector2i], concurrency: int) -> void:
+	var start := Time.get_ticks_usec()
+	var next_position := 0
+	var tasks: Array[Dictionary] = []
+	var completed: Array[Dictionary] = []
+	while next_position < positions.size() or not tasks.is_empty():
+		while tasks.size() < concurrency and next_position < positions.size():
+			var slot: Dictionary = {}
+			var pos := positions[next_position]
+			next_position += 1
+			var task := WorkerThreadPool.add_task(
+				_job.bind(generator, mesher, pos, lod, slot), true, "chunk_load_bench")
+			tasks.append({"task": task, "slot": slot})
+		var found := false
+		for index in range(tasks.size() - 1, -1, -1):
+			var entry: Dictionary = tasks[index]
+			if not WorkerThreadPool.is_task_completed(int(entry["task"])):
+				continue
+			WorkerThreadPool.wait_for_task_completion(int(entry["task"]))
+			completed.append(entry["slot"])
+			tasks.remove_at(index)
+			found = true
+		if not found:
+			OS.delay_usec(200)
+	var worker_wall_us := Time.get_ticks_usec() - start
+	var commit_start := Time.get_ticks_usec()
+	var triangles := 0
+	for slot in completed:
+		var result: ChunkMesher.MeshResult = slot["result"]
+		var mesh := ChunkMesher.arrays_to_mesh(result.verts, result.normals, result.uvs,
+			result.colors, result.indices, blocks.material, result.light, result.layers)
+		var water := ChunkMesher.arrays_to_mesh(result.water_verts, result.water_normals,
+			result.water_uvs, result.water_colors, result.water_indices, blocks.water_material,
+			result.water_light)
+		var shape: ConcavePolygonShape3D = null
+		if not result.collision.is_empty():
+			shape = ConcavePolygonShape3D.new()
+			shape.set_faces(result.collision)
+		triangles += result.indices.size() / 3 + result.water_indices.size() / 3
+		# Keep resources alive through the measured call.
+		slot["mesh"] = mesh
+		slot["water"] = water
+		slot["shape"] = shape
+	var commit_us := Time.get_ticks_usec() - commit_start
+	var terrain_us := 0
+	var populate_us := 0
+	var heightmap_us := 0
+	var generation_us := 0
+	var mesh_us := 0
+	for slot in completed:
+		var timings: Dictionary = slot["timings"]
+		terrain_us += int(timings["terrain_us"])
+		populate_us += int(timings["populate_us"])
+		heightmap_us += int(timings["heightmap_us"])
+		generation_us += int(timings["generation_us"])
+		mesh_us += int(slot["mesh_us"])
+	var count := float(completed.size())
+	var wall_ms := float(worker_wall_us) / 1000.0
+	print(label.to_upper(), " wall_ms=", snappedf(wall_ms, 0.01),
+		" chunks/s=", snappedf(count * 1000.0 / wall_ms, 0.1),
+		" avg_ms={terrain:", snappedf(float(terrain_us) / count / 1000.0, 0.01),
+		", populate:", snappedf(float(populate_us) / count / 1000.0, 0.01),
+		", heightmap:", snappedf(float(heightmap_us) / count / 1000.0, 0.01),
+		", generation:", snappedf(float(generation_us) / count / 1000.0, 0.01),
+		", mesh:", snappedf(float(mesh_us) / count / 1000.0, 0.01),
+		", commit:", snappedf(float(commit_us) / count / 1000.0, 0.01),
+		"} triangles/chunk=", roundi(float(triangles) / count))
+
+
+func _job(generator: TerrainGenerator, mesher: ChunkMesher, pos: Vector2i, lod: bool,
+		slot: Dictionary) -> void:
+	var generated := generator.generate_data(pos, {}, lod)
+	var mesh_start := Time.get_ticks_usec()
+	if lod:
+		slot["result"] = mesher.build_lod(generated.lod_solid_y, generated.lod_solid_id,
+			generated.lod_sub_id, generated.lod_water_y, generated.lod_water_level,
+			generated.max_y, generated.foliage_tints, generated.water_tints,
+			ChunkMesher.LodNeighbors.new())
+	else:
+		slot["result"] = mesher.build(generated.data, generated.max_y, generated.heights,
+			generated.foliage_tints, generated.water_tints, ChunkMesher.NeighborSet.new(), true)
+	slot["timings"] = generated.timings
+	slot["mesh_us"] = Time.get_ticks_usec() - mesh_start

tools/chunk_loading_benchmark.gd.uid

@@ -0,0 +1 @@
+uid://cgvm04h6uukjb

tools/stream_full_verify.gd

@@ -7,7 +7,10 @@ extends SceneTree
 
 const RENDER_DISTANCE := 10
 const CONFIG := {"seed": 918273, "tree_density": 1.0, "decoration_density": 1.0}
-const MAX_WAIT_TICKS := 240
+# Shared CI runners can take just over 60 seconds to finish the 441-chunk pass
+# under concurrent shards. Keep the assertion strict, but leave enough wall
+# time to distinguish a real stalled queue from runner contention.
+const MAX_WAIT_TICKS := 480
 const WAIT_TICK := 0.25
 
 var _failures := 0
@@ -28,10 +31,25 @@ func _run() -> void:
 
 	var expected := (RENDER_DISTANCE * 2 + 1) * (RENDER_DISTANCE * 2 + 1)
 	var waited := 0
-	while (world._chunks.size() < expected or not world._pending.is_empty()) and waited < MAX_WAIT_TICKS:
+	var stream_start := Time.get_ticks_usec()
+	var visible_ms := -1.0
+	while (world._chunks.size() < expected or not world._pending.is_empty() \
+			or not world._gen_queue.is_empty() or not world._mesh_queue.is_empty() \
+			or not world._generated.is_empty() or not world._commit_queue.is_empty()) \
+			and waited < MAX_WAIT_TICKS:
 		await create_timer(WAIT_TICK).timeout
 		waited += 1
-	print("STREAM FULL: chunks=%d/%d wait_ticks=%d" % [world._chunks.size(), expected, waited])
+		if visible_ms < 0.0 and world._chunks.size() >= expected:
+			visible_ms = float(Time.get_ticks_usec() - stream_start) / 1000.0
+	print("STREAM FULL: chunks=%d/%d visible_ms=%.1f full_ms=%.1f wait_ticks=%d" % [
+		world._chunks.size(), expected, visible_ms,
+		float(Time.get_ticks_usec() - stream_start) / 1000.0, waited])
+	if world._chunks.size() != expected:
+		_fail("stream timed out with %d/%d chunks" % [world._chunks.size(), expected])
+	if not world._pending.is_empty() or not world._gen_queue.is_empty() \
+			or not world._mesh_queue.is_empty() or not world._generated.is_empty() \
+			or not world._commit_queue.is_empty():
+		_fail("stream timed out with unfinished generation, mesh, or commit work")
 
 	for pos in world._chunks.keys():
 		if (world._chunks[pos] as VoxelWorld.Chunk).lod:
@@ -59,7 +77,9 @@ func _run() -> void:
 		await create_timer(WAIT_TICK).timeout
 		waited += 1
 		if world._stream_center != center or world._pending.size() > 0 \
-				or world._gen_queue.size() > 0 or world._commit_queue.size() > 0:
+				or world._gen_queue.size() > 0 or world._mesh_queue.size() > 0 \
+				or world._generated.size() > 0 \
+				or world._commit_queue.size() > 0:
 			continue
 		var chunk: VoxelWorld.Chunk = world._chunks.get(center)
 		if chunk != null and chunk.shape.shape != null:

tools/worldgen_lod_verify.gd

@@ -46,6 +46,7 @@ func _initialize() -> void:
 		_verify_chunk(clean, decorated, cave_free, mesher, pos)
 	_verify_canopy_coverage(decorated)
 	_verify_lod_shading(mesher)
+	_verify_unloaded_water_edges(mesher)
 	if _failures == 0:
 		print("WORLDGEN LOD VERIFY: PASS")
 	else:
@@ -249,6 +250,35 @@ func _verify_compact_neighbors(generator: TerrainGenerator, mesher: ChunkMesher,
 		_fail("compact neighbors did not cull boundary faces at %s" % pos)
 
 
+## Missing distance neighbors must not turn a flat ocean into four transparent
+## chunk-edge curtains. Only the 16x16 water top should be emitted.
+func _verify_unloaded_water_edges(mesher: ChunkMesher) -> void:
+	var solid_y := PackedInt32Array()
+	var solid_id := PackedByteArray()
+	var sub_id := PackedByteArray()
+	var water_y := PackedInt32Array()
+	var water_level := PackedByteArray()
+	var tints := PackedColorArray()
+	solid_y.resize(VoxelDefs.CHUNK_AREA)
+	solid_y.fill(12)
+	solid_id.resize(VoxelDefs.CHUNK_AREA)
+	solid_id.fill(BlockRegistry.BLOCK_SAND)
+	sub_id.resize(VoxelDefs.CHUNK_AREA)
+	sub_id.fill(BlockRegistry.BLOCK_SAND)
+	water_y.resize(VoxelDefs.CHUNK_AREA)
+	water_y.fill(VoxelDefs.SEA_LEVEL)
+	water_level.resize(VoxelDefs.CHUNK_AREA)
+	water_level.fill(8)
+	tints.resize(VoxelDefs.CHUNK_AREA)
+	tints.fill(Color.WHITE)
+	var mesh := mesher.build_lod(solid_y, solid_id, sub_id, water_y, water_level,
+		VoxelDefs.SEA_LEVEL, tints, tints, ChunkMesher.LodNeighbors.new())
+	var expected_top_indices := VoxelDefs.CHUNK_AREA * 6
+	if mesh.water_indices.size() != expected_top_indices:
+		_fail("unloaded ocean edge emitted water curtains: %d indices != %d" % [
+			mesh.water_indices.size(), expected_top_indices])
+
+
 func _top_face_heights(mesh: ChunkMesher.MeshResult) -> Dictionary:
 	var tops := {}
 	var index := 0

world/block.gdshader

@@ -1,7 +1,7 @@
 shader_type spatial;
 render_mode diffuse_burley, specular_schlick_ggx, alpha_to_coverage;
 
-uniform sampler2DArray albedo_texture : source_color, filter_linear_mipmap_anisotropic, repeat_disable;
+uniform sampler2DArray albedo_texture : source_color, filter_linear_mipmap_anisotropic, repeat_enable;
 uniform float block_emission : hint_range(0.0, 2.0) = 0.45;
 uniform float min_light : hint_range(0.0, 1.0) = 0.05;
 // Camera-submerged strength (0 above water), driven by DayNightCycle. Caustics