[#120] Move the schema documentation to where it is useful - #136
Merged
Merged
Conversation
SQLite only keeps the text of the CREATE statement itself in sqlite_master, so the 58 comment lines documenting the schema in Init.sql reached nobody: .schema on a produced database showed bare column names. Comments now go inside the CREATE statements, where SQLite preserves them, so a produced .db is self-describing. Only facts needed to write a correct query and not guessable from the column name are kept there; the prose moved into the documentation, which ships in the release zip since #118. The same fix applies to AssetBundle.sql, the ContentLayout*.sql files (which already had good in-paren column notes plus a discarded prose block above each statement) and ContentSummary.sql. AGENTS.md records the convention. Documentation/analyzer.md was 81% schema reference, so the schema moved to a new strictly formatted Documentation/analyzer-schema.md and analyzer.md became the overview. contentlayout-database.md gained the same markdown-table format. The schema version history moved out of the Init.sql comment into a table on the new page, and serialized_files, types, property_names and property_types gained documentation they never had. No table, view or column changed, so PRAGMA user_version stays at 7. Also fixed while here: the ResX file-reference encoding for Init.sql and Finalize.sql (Windows-1252 since the initial commit, every other .sql is utf-8), the duplicated DDL in the Commands/SerializedFile classes (AddPreloadDependency declared a primary key the real DDL never had), and nine pre-existing broken documentation links.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #120.
SQLite only keeps the text of the
CREATEstatement itself insqlite_master, so a comment placedabove a statement is discarded. All 58 comment lines documenting the core schema in
Init.sqlwere above their statements, so none of them reached a produced database:
.schema objectsshowedbare column names, and anyone working from a
.dbartifact never saw the documentation.This takes both halves of the fix offered in the issue. Concise key facts move inside the
CREATEstatements, where SQLite preserves them verbatim, so a produced database is self-describing. The
detailed prose moves into the documentation, which has shipped in the release zip since #118, so it
travels with the binary. Every comment outside a
CREATEis gone, so the old practice stops ratherthan being repeated by the next table.
Only facts that are needed to write a correct query and are not guessable from the column name went
into the schema, for example that
serialized_files.archiveandobjects.game_objecthold''rather than NULL, that
refs.referenced_objectcan be adangling_refs.id, and that--skip-crcsets every
crc32to 0..schema serialized_filesnow reads:No table, view or column changed — verified mechanically that all 15 changed
.sqlfiles arebyte-identical to their previous versions once comments are stripped.
PRAGMA user_versionthereforestays at 7; bumping it would falsely signal an incompatibility to
find-refs.Changes
SQL resources
Init.sql: all 58 comments outsideCREATEstatements removed. Table-level notes and per-columnnotes added inside the parentheses; each of the 7 views gained a one-line purpose comment as the
first line of its body (5 of them had no comment at all before). The
INSERT INTO typescommentcould not survive on its own, so the
-1Scene sentinel is documented ontypes.idinstead.AssetBundle.sqlhad the identical bug — 19 lines of good comments, all discarded. Same treatment.ContentLayout*.sqlfiles were half converted: good in-paren column notes plus a proseblock above each statement that was thrown away. The prose blocks are gone and anything
query-critical that lived only there (e.g. why
serialized_fileis NULL for a layout-onlyanalyze) was promoted into the parentheses.
ContentLayoutViews.sqlandContentSummary.sql: per-view comments moved inside the view bodies.ContentLayoutIndexes.sqlkeeps one comment outside the statements, trimmed to the "why late"rationale.
CREATE INDEXhas no body to hold a note, and that comment explains the code ratherthan the schema.
AGENTS.mdrecords the convention so new.sqlfiles follow it.Documentation
Documentation/analyzer-schema.md: the schema reference, in one uniform format(
| Column | Type | Description |per entity, plus an at-a-glance index). Replaces the fourdifferent styles previously in use across the schema pages.
Documentation/analyzer.mdwas 81% schema reference; it is now a 76-line overview (intro, a map ofwhere each part of the schema is documented, and the Advanced/extending sections).
Documentation/contentlayout-database.mdreformatted to the same markdown-table shape, absorbingthe prose evicted from the
ContentLayout*.sqlfiles.serialized_files,types,property_namesandproperty_typesgained documentation they neverhad — the SerializedFile naming rules per build pipeline (
CAB-,BuildPlayer-,levelN) existedonly in a discarded
Init.sqlcomment.Init.sqlcomment into a markdown table on the newpage;
AGENTS.mdpoints the bump rule at it.agent-guide.mdtold agents that "the produced database's schema shows bare table definitions",which this change makes false. Rewritten.
Incidental fixes
Analyzer/Properties/Resources.resx:Init.sqlandFinalize.sqlwere declaredWindows-1252while the other 49
.sqlentries areutf-8. Both date from the initial commit and there is nopurpose to it; all
.sqlfiles are pure ASCII, which decodes identically under either encoding, sothe switch to
utf-8is a no-op on current content and removes a trap for future comment edits./* TABLE DEFINITION: */blocks in the nineCommands/SerializedFile/*.csclasses hand-copiedthe DDL and had to be kept in sync manually. They are replaced with a one-line pointer to the
owning
.sqlfile. The drift was already real:AddPreloadDependencydeclaredPRIMARY KEY (object, dependency), which the actual DDL has never had.Notably
analyzer.mddescribed anIWriterinterface that no longer exists in the codebase.Testing
dotnet build -c Release— clean, 0 warnings.dotnet test -c Release— full suite green: 787 passed, 0 failed (Analyzer.Tests 62,UnityFileSystem.Tests 435, UnityDataTool.Tests 290). Any
.sqlsyntax error fails essentiallyevery analyze test at
Begin(), so this is the real regression net.analyzeagainstTestCommon/Data/AssetBundlesandagainst the ContentDirectory build in
TestCommon/Data/LeadingEdgeBuilds, then confirmed withsqlite3 ... ".schema <table>"that the notes are present and readable for core tables,AssetBundle tables, ContentLayout tables and views; that queries against
object_viewandview_breakdown_by_typestill work; and thatPRAGMA user_versionstill reports 7. Re-ran afterthe ResX encoding change to confirm the resource still loads.
.sqlfiles are byte-identical ignoring comments, so the schema itself isuntouched.
(0 problems), since there is no CI link checker.
.sqlfiles are pure ASCII and that the only remaining comment outside aCREATEstatement is the intended
ContentLayoutIndexes.sqlone.🤖 Generated with Claude Code