diff --git a/Dokka-plugin-kdoc2json/README.md b/Dokka-plugin-kdoc2json/README.md index 086eed20..f40339f5 100644 --- a/Dokka-plugin-kdoc2json/README.md +++ b/Dokka-plugin-kdoc2json/README.md @@ -110,6 +110,7 @@ dokka { | `classDiscriminator` | String | `"kind"` | The JSON key used to discriminate between polymorphic `Documentable` types (e.g., `"kind": "class"`). Must not collide with an existing DTO field name (e.g. `"type"` or `"name"`), or serialization will fail. | | `prettyPrint` | Boolean | `false` | If `true`, formats the written JSON files with indentation for human readability instead of compact single-line output. | | `sourceSetWhitelist` | List | `[]` | A list of source set names (matching the values that appear in the output `sourceSets` field, e.g. `["jvm"]`). If non-empty, any Documentable that isn't present in at least one whitelisted source set has its output file omitted, and a message is logged with the symbol's name and its `sourceSets`. Leave empty to disable filtering (default: all source sets included). | +| `javadoc-mode` | Boolean | `false` | If `true`, emit **javadoc-shaped** JSON mirroring the `api/` tree of the `javadoc` tool instead of Dokka-shaped JSON. See [§10](#10-javadoc-mode). Note the kebab-case key -- it is spelled that way in the config, unlike every other option here. | > **`omitNulls` also strips *empty* values, not just `null`.** Despite the name, `omitNulls: true` removes a key whenever its value is `null`, `""`, `[]`, or `{}` (see the filter in `JsonRenderer.filterJson`) — so with it enabled, `"functions": []` doesn't appear at all rather than appearing as an empty array. Consumers must treat a **missing** key as equivalent to its empty value (e.g. `functions is defined and functions is not empty`, as in the Pebble example in §8), not assume every key is always present. @@ -207,3 +208,362 @@ For example, to render a table of functions for a class: This writes HTML output to `/html/latest/all-libs` and JSON output to `/json/latest/all-libs` (`output-dir` defaults to `scripts/kotlin/build-output`). > **Provenance / staleness warning:** `scripts/kotlin/build.gradle.kts` was derived from the `kotlin-stdlib-docs/build.gradle.kts` in JetBrains' `kotlin` repo as of commit [`cfcb49fd0113`](https://github.com/JetBrains/kotlin/commit/cfcb49fd0113d2300a2b677c4fc2e16dddff7df5) ("[stdlib] Update Dokka to 2.2.0-Beta and migrate to DGPv2"). That upstream file is not under our control and can change — new source sets, Dokka API changes, or a different doc-build structure could all require re-diffing our modifications against a newer upstream version. If `build-kotlin-stdlib.sh` starts failing against a newer `kotlin` checkout, compare `scripts/kotlin/build.gradle.kts` against the current upstream `kotlin-stdlib-docs/build.gradle.kts` and re-apply the `useJsonPlugin`/`dokkaGenerateModuleJson` additions by hand. + +--- + +## 10. Javadoc Mode + +Setting `"javadoc-mode": true` replaces the plugin's whole output with JSON that mirrors what the +`javadoc` tool produces under its `api/` directory -- same file layout, same page sections, same +member anchors. It is intended for documenting **Java** sources (the JDK's own API docs being the +motivating case) where the downstream templates expect javadoc's structure rather than Dokka's. + +Only JSON is written. The one non-JSON file is `element-list`, which javadoc itself emits as a +plain-text manifest and external tooling reads to resolve links into the output. No HTML pages are +produced -- rendering stays the job of the downstream template engine. + +> The key is spelled `javadoc-mode`, not `javadocMode`. Every other option in this block is +> camelCase; this one is deliberately kebab-case. + +### Output layout + +``` +index.json overview: the run's modules and packages +element-list javadoc's plain-text manifest (not JSON) +allclasses-index.json every documented type +allpackages-index.json every documented package +deprecated-list.json deprecated elements, grouped by kind +constant-values.json static final fields, grouped by package then type +index-files/index-N.json the A-Z index, one file per letter +/module-summary.json module page +//package-summary.json package page +//.json type page +``` + +#### How modules are determined + +Dokka's own model has no notion of JPMS -- a Dokka "module" is a build-level grouping -- so +Javadoc mode reads `module-info.java` directly instead. Every configured source root is checked +for one; a root that has one *is* a JPMS module root, which makes this self-validating (an +ordinary `src/main/java` has no `module-info.java`, so a non-modular project is unaffected). + +From each descriptor the plugin takes the module name, its doc comment, and its `requires`, +`exports`, `opens`, `uses` and `provides` directives -- everything javadoc's module-summary page +is built from. A package is attributed to the module that declares it, falling back to whichever +module's source root the declaration's file sits under. + +The leading `/` segment appears when the run contains more than one module -- more than +one JPMS module if the sources are modular, otherwise more than one Dokka module. That mirrors +javadoc's own split between modular and non-modular builds. A single-module run also writes +`module-summary.json` at the root: javadoc omits a module page entirely for a non-modular build, +but the module's documentation would otherwise be dropped. + +To document a modular codebase, then, give Dokka **one source root per module directory** and let +the descriptors do the rest -- see `scripts/java/` for a worked example that does this for the +entire JDK. + +#### Multi-module Gradle builds + +Separately from JPMS, a *Gradle* multi-module build makes Dokka run the renderer once per +subproject into that subproject's own output directory, then make an aggregating pass. Each +per-module run therefore writes global index files scoped to its own module, and the aggregating +pass writes only the overview `index.json` linking to each. Merging those per-module indexes into +run-wide ones is a downstream step this plugin does not perform. Documenting a modular codebase +from a single Dokka run (as `scripts/java/` does) avoids this entirely. + +All links between pages are **relative to the page they appear on** (`../lang/Object.json`), as +javadoc's are, so the tree can be served from any prefix. This includes links inside rendered doc +comments. A link to something the run does not document resolves to `null` (or, inside a comment, +degrades to plain text) rather than becoming a dead `href`. + +### Page shape + +Type pages carry the sections a javadoc class page has: the type signature and its parts +(`modifiers`, `typeParameters`, `superclass`, `superinterfaces`), the hierarchy closures +(`inheritance`, `allImplementedInterfaces`, `allSuperinterfaces`, `directKnownSubclasses`, +`allKnownSubinterfaces`, `allKnownImplementingClasses`), the doc comment and its block tags +(`description`, `since`, `seeAlso`, `authors`, `versions`, `deprecated`, `tags`), the member +tables (`nestedTypes`, `enumConstants`, `fields`, `constructors`, `methods`, `annotationElements`) +and the inherited-member groups (`inheritedFields`, `inheritedMethods`). + +Structured data is primary: types, modifiers, parameters, throws clauses and override +relationships are all discrete fields. Each declaration also carries a flat `signature` string +(`public default Shape scaled(double factor) throws IllegalArgumentException`) as a +convenience -- ignore it if you would rather compose signatures in the template. + +Member `anchor` values follow javadoc's scheme: a bare name for a field, `name(erasedParamTypes)` +for an executable, and `(...)` for a constructor -- so `toArray(java.lang.Object[])`, not +`toArray(T[])`. `overrides` and `specifiedBy` are derived from those same erased signatures. + +Doc-comment text is HTML, because a javadoc comment's body already is (`

`, ``, ``, +`
`). That matches the convention the default output mode already uses. + +### Recommended Dokka settings + +javadoc documents public **and protected** members by default; Dokka documents only public. For +parity, set this on the consuming project: + +```kotlin +dokka { + dokkaSourceSets.configureEach { + documentedVisibilities.set(setOf(VisibilityModifier.Public, VisibilityModifier.Protected)) + } +} +``` + +`examples/example-java-library` is a working Java example wired up this way; `tests/test_javadoc_mode.sh` +drives it. + +### Known limitations + +These are places where Dokka's model does not carry something a real javadoc page shows. Each is a +missing *input*, not a gap in the mapping: + +| Limitation | Effect | +| --- | --- | +| Dokka has no JPMS model | Worked around by parsing `module-info.java` directly (see above), so `requires`/`exports`/`opens`/`uses`/`provides` and the module description *are* populated for modular sources. Without `module-info.java` in a source root those sections are empty. | +| Dokka does not record annotation-element defaults | Annotation elements are reported as one `annotationElements` list rather than being split into javadoc's Required/Optional tables. `defaultValue` is populated only when Dokka does supply it. | +| Dokka has no `record` class kind | Java records are documented as classes; `recordComponents` stays empty. | +| Dokka merges a private field and its accessors into one property | Unfolded back into methods so `getWidth()` is a method and the private field is not documented, as javadoc has it. Note this means a *public* field that happens to have a same-named accessor pair is reported through its accessors. | +| Inherited members depend on Dokka's inheritance propagation | If Dokka does not attach `InheritedMember`, those members appear as declared rather than in an inherited group. | + +### What Javadoc mode does not change + +`omitFields`, `omitNulls`, `prettyPrint`, `logLevel`, `logFile` and `sourceSetWhitelist` all behave +as documented in [§3](#3-configuration-options). `replaceHtmlExtension` and `classDiscriminator` +have no effect: javadoc-mode pages are written with `.json` links throughout and none of its DTOs +are polymorphic. Javadoc mode also skips the `LinkPostProcessor` pass, since it resolves every link +itself rather than rewriting Dokka's. + +Pages are serialized with `encodeDefaults = true`, so every documented key is present on every page +even when empty -- a template can test a field without also testing whether it exists. Enabling +`omitNulls` strips the empty ones back out if you prefer that. + +--- + +## 11. Reproducing the JDK API Docs (`scripts/java`) + +`scripts/java/` builds the JDK's own API documentation as javadoc-shaped JSON -- the JSON +counterpart of the `api/` tree in `SourceDocs/JavaDocs/html/api`. + +```bash +# Document the JDK that JAVA_HOME points at (use a JDK 17 to match SourceDocs/JavaDocs) +scripts/java/build-jdk-json-docs.sh -j /path/to/jdk-17 -o /path/to/output/api + +# Quick check on two small modules instead of all 60 +scripts/java/build-jdk-json-docs.sh -j /path/to/jdk-17 -m java.sql,java.transaction.xa +``` + +### How it works + +1. **`stage_jdk_sources.py`** unpacks the JDK's `lib/src.zip` and keeps only what javadoc + documents. javadoc's rule turns out to be exact: a package appears in `api/` if and only if its + module `exports` it **unqualified**. For JDK 17's `java.base`, the 53 unqualified exports are + precisely the 53 documented packages, with nothing left over on either side. The script also + drops the modules the JDK's own docs build filters out (`jdk.internal.*`, `jdk.unsupported*`, + `jdk.random`), leaving 60 modules and 224 packages -- exactly what the official docs contain. + + Each module becomes its own directory in the staging tree, with its `module-info.java` copied + alongside. Everything left behind still resolves from the JDK on the analysis classpath. + +2. **`jdk-docs/`** is a Dokka project that registers each staged module directory as a source root + and runs the plugin in javadoc mode. Nothing is compiled -- Dokka only analyses. + +3. **`compare_with_javadoc.py`** checks the result against the official HTML, level by level: + + ```bash + python3 scripts/java/compare_with_javadoc.py /SourceDocs/JavaDocs/html/api + python3 scripts/java/compare_with_javadoc.py --members + ``` + + `--members` compares the member anchors of every type. That is the sharpest of the checks: + javadoc's anchor encodes a member's name and its erased parameter types, so a matching anchor + set means the two sides agree on the members, their signatures and their overloads -- not + merely on the page count. + +### Measured result (JDK 17) + +A full run takes **about 90 seconds** and writes **4,988 JSON files**. Against the official docs in +`SourceDocs/JavaDocs/html/api`: + +| Level | Result | +| --- | --- | +| modules | **60 / 60** — no missing, no extra | +| packages | **224 / 224** — no missing, no extra | +| types | **4,672 / 4,672** — no missing, no extra | +| member anchors | 4,305 / 4,672 types match *exactly* (92%) | + +The 367 types whose member sets differ do so for two understood reasons, neither of which is a +missing page: + +- **893 anchors we have that javadoc doesn't** (330 types). javadoc folds an override whose entire + doc comment is `{@inheritDoc}` — adding nothing of its own — into the superclass's "Methods + declared in…" list instead of giving it a detail section. `java.awt.Frame.setBackground` is a + typical case. We document them as the declared members they are, so this is extra data, not lost + data. +- **481 anchors javadoc has that we don't** (37 types). Where a class extends an *undocumented* + supertype (a package-private base like `java.awt.AttributeValue`), javadoc pulls that supertype's + members up and shows them as if declared. We only document what the source declares. + +### Two Dokka problems this works around + +Both were found running the JDK through it, and both are in Dokka rather than in this plugin: + +1. **Unbounded recursion in `{@inheritDoc}`.** Dokka's `InheritDocTagResolver.resolveThrowsTag` → + `PsiElementToHtmlConverter.toInheritDocHtml` recurses until the stack dies, reproducibly, on + much of the JDK (`java.io` and `java.util` among others). A bigger stack only buys time: + `-Xss64m` fails after 42 s, `-Xss512m` after 3m28s. `stage_jdk_sources.py` therefore rewrites + `{@inheritDoc}` to an inert marker before Dokka parses it, and the plugin resolves the marker + itself, walking the same supertype chain javadoc walks — so all 3,214 occurrences across the JDK + still resolve, and the plugin additionally inherits a *missing* `@param`/`@return`/`@throws` the + way javadoc does. Pass `--keep-inherit-doc` to re-check whether a newer Dokka has fixed this. +2. **Type arguments in DRI parameter types.** Dokka builds a Java DRI from the PSI type's canonical + text, which carries type arguments, so a naive anchor comes out as + `addAll(java.util.Collection)` where javadoc uses the erasure, + `addAll(java.util.Collection)`. `JavadocPaths.eraseGenerics` strips them while keeping array + brackets. This alone moved member-anchor parity from 80% to 92%. + +### Notes + +- Dokka generates in a *worker process*, not the Gradle daemon, so `org.gradle.jvmargs` does not + size it — `dokkaGeneratorIsolation` in `jdk-docs/build.gradle.kts` does. Override with + `-PdokkaWorkerHeap` / `-PdokkaWorkerStack` if needed. +- Use a JDK whose version matches the docs you are reproducing. Source and docs from different + update releases differ in small ways that are real, not bugs. + +--- + +## 12. Rendering the JSON to HTML (`pebble-renderer`) + +`pebble-renderer/` turns a javadoc-mode JSON tree into browsable HTML using Pebble templates that +follow the official javadoc page structure. It is the reference consumer of the JSON: if a field +is in the JSON, a template here shows it. + +```bash +# After scripts/java/build-jdk-json-docs.sh +pebble-renderer/render.sh scripts/java/build-output/api scripts/java/build-output/html + +# Then browse it +(cd scripts/java/build-output/html && python3 -m http.server 8000) +``` + +### How links work + +The HTML tree mirrors the JSON tree file-for-file, `Foo.json` becoming `Foo.html` in the same +directory. Every link in the JSON is already relative to the page it appears on, so **the path is +correct as-is and only the extension needs changing**. Two Pebble filters do that: + +| Filter | Use | What it does | +| --- | --- | --- | +| `href` | `{{ type.url \| href }}` | rewrites one URL's trailing `.json` to `.html`, preserving any `#anchor` | +| `doc` | `{{ description \| doc }}` | rewrites every `href` *inside* a block of documentation HTML, and marks it safe so it isn't escaped | + +Doc text needs the second filter because a javadoc comment's body is HTML that can itself contain +links. Autoescaping stays on everywhere else, so names and signatures are escaped by default. + +### Templates + +One per page kind, selected by the JSON's `page` field: + +| `page` | Template | Renders | +| --- | --- | --- | +| `class` | `class.peb` | class/interface/enum/annotation/exception page | +| `package` | `package-summary.peb` | package page | +| `module` | `module-summary.peb` | module page, including the JPMS tables | +| `overview` | `overview.peb` | `index.html` | +| `all-classes` / `all-packages` | `all-classes.peb` / `all-packages.peb` | the global indexes | +| `deprecated-list` | `deprecated-list.peb` | deprecated API | +| `constant-values` | `constant-values.peb` | constant field values | +| `index` | `index-page.peb` | one A-Z index page | + +`base.peb` holds the shared skeleton and navigation; `macros.peb` holds the fragments (type links, +signatures, notes, member details). Class names follow the official javadoc output (`top-nav`, +`summary-table`, `col-first`, `member-signature`, `notes`, `inheritance`, …), and +`static/stylesheet.css` styles those names -- it is a readable approximation of javadoc's look, +written here rather than copied from the JDK. + +Two Pebble details worth knowing before editing a template, because both fail *silently*: + +- `{% import "macros" %}` pulls macros into the importing template's own namespace. Call them by + bare name (`{{ typeLink(ref) }}`); a Jinja/Twig-style `macros.typeLink(...)` renders nothing. +- `loop.index` is **0-based**, unlike Jinja's. + +### Measured result (JDK 17) + +Rendering all 4,988 pages takes about two seconds. Of the **450,575** internal links in the +output, **99.88% resolve**; the 555 that don't break down as: + +| Count | Cause | +| --- | --- | +| 251 | `doc-files/` pages -- javadoc copies these from the JDK's build repository, and `src.zip` does not ship them, so they cannot be produced from this source at all | +| 227 | links out of `api/` into `specs/` and `legal/`, which are siblings of `api/` in the official docs and outside what this pipeline generates | +| 77 | hand-written relative links in doc comments that are still rebased imperfectly when a summary sentence is shown on a different page than the one that declares it | + +### Comparison against the official docs + +Beyond the counts above, the rendered HTML was diffed section by section against +`SourceDocs/JavaDocs/html/api`. What that turned up, and where it stands: + +**Fixed** + +| Gap | Scale | Now | +| --- | --- | --- | +| `module-graph.svg` missing entirely | all 60 module pages | Generated from the JSON's `requires`. All 60 have node sets identical to the originals; 57/60 also match edge counts (graphviz applies a transitive reduction we don't). | +| Inherited nested types never emitted | ~700 groups | `inheritedNestedTypes` is now computed from the hierarchy -- Dokka, unlike for fields and methods, does not copy nested types down, so there is no `InheritedMember` to read. | +| "Indirect Exports" table | 16 module pages | Added as `indirectExports`. | +| "Indirect Requires" table | 3 module pages | Added as `indirectRequires`. | +| Block tags shown by their source name | ~2,400 notes | `@apiNote` now renders as "API Note:", `@implSpec` as "Implementation Requirements:", and so on, as javadoc does. The data was always in the JSON's `tags`. | +| "Enclosing class:" on a nested interface | 111 pages | Uses the enclosing type's own kind. | + +The module graph rules were derived by checking candidates against all 60 originals rather than +assumed. Both turned out to be narrower than they look: the graph draws the **`requires transitive` +closure plus `java.base`** (a plain `requires` does not propagate readability -- `java.naming` +plainly requires `java.security.sasl` and its graph shows neither), and `java.base` is drawn even +though no `module-info.java` declares it. Same for the tables: **Indirect Requires** is that +closure minus the direct requires (3/3 exact), **Indirect Exports** is the exporting modules in it +(16/16 exact). + +**Not reproducible from this source** + +| Gap | Scale | Why | +| --- | --- | --- | +| `doc-files/` pages and images | 93 files, 251 links | javadoc copies these from the JDK's build repository. `src.zip` does not contain them, so no pipeline reading `src.zip` can produce them. | +| `serialVersionUID` values | 1,174 | Shown only on `serialized-form.html`, and read from private fields Dokka does not model. | + +**Out of scope (page kinds this pipeline does not generate)** + +`class-use/` (4,672 pages), `package-use` (224), `package-tree`/`overview-tree` (225), +`serialized-form`, `system-properties`, `help-doc`, `new-list`, `preview-list`, `search` and its +`.js` index. These are cross-reference and navigation pages rather than API data; everything they +present is derivable from the JSON already emitted. + +**Deliberate differences** + +Our module pages show `Requires`, `Provides`, `Uses` and `Exports` tables on more modules than +javadoc does -- javadoc suppresses some rows (for instance a `provides` whose implementation class +is not itself documented, as in `java.smartcardio`). That is extra data rather than missing data, +so it is left in. + +### Page-by-page content audit + +Every page type was diffed against the originals, not just the class pages. What that changed: + +| Page | Was | Now | +| --- | --- | --- | +| `index.html` | listed all 224 packages *as well as* the 60 modules | modules only, as javadoc does. The package list has its own page; javadoc's overview shows packages only for a non-modular run, which is what the template now keys on. | +| `constant-values.html` | 3,138 of the JDK's 3,463 constants | 3,458. The 325 absent were all inherited from *undocumented* supertypes -- `java.util.jar.JarEntry`'s 40 `CEN*`/`END*` constants come from the package-private `java.util.zip.ZipConstants`. | +| class pages | same 325-constant cause, plus ~480 members | a member inherited from a type this run does not document is now shown as declared, which is what javadoc does: an "inherited from" group pointing at a page that does not exist is a dead end. Member-anchor parity 4,305 -> 4,327 of 4,672 types. | +| `deprecated-list.html` | section headings were the raw JSON keys (`classes`, `enumConstants`) | javadoc's titles ("Deprecated Classes", "Deprecated Enum Constants"), plus a contents list | +| package pages | no "Related Packages" table | present on 206 pages, 181 matching the originals exactly | +| `allclasses-index.html` | all 4,672 types | 4,506 -- javadoc indexes the public API, so the 167 protected nested types are left out (they keep their pages, reachable from the enclosing class) | + +"Related Packages" is the parent, the direct children, and -- only when the result stays at five or +fewer -- the siblings. That size condition is javadoc's own: `java.nio.channels` lists its siblings +`java.nio.charset` and `java.nio.file`, while `java.util.concurrent` and `java.lang.annotation` +list none, because `java.util` and `java.lang` have too many children for the table to stay +useful. Five reproduces 181 of the 190 originals; no cut-off at all reproduces 95. + +Two counts still differ, both small and both in the direction of showing more rather than less: +`allclasses-index.html` lists 4,506 against javadoc's 4,402, and the A-Z index has 54,248 entries +against 55,483. Neither reduced to a rule that held across all 60 modules, so they are left as they +are rather than tuned to fit. diff --git a/Dokka-plugin-kdoc2json/examples/example-java-library/build.gradle.kts b/Dokka-plugin-kdoc2json/examples/example-java-library/build.gradle.kts new file mode 100644 index 00000000..243a968d --- /dev/null +++ b/Dokka-plugin-kdoc2json/examples/example-java-library/build.gradle.kts @@ -0,0 +1,58 @@ +import org.jetbrains.dokka.gradle.engine.parameters.VisibilityModifier +import org.jetbrains.dokka.InternalDokkaApi +import org.jetbrains.dokka.gradle.engine.plugins.DokkaPluginParametersBaseSpec +import javax.inject.Inject + +// A Java-only sibling of examples/example-data-processor, used by tests/test_javadoc_mode.sh. +// Javadoc mode mirrors the output of the `javadoc` tool, so it needs Java sources exercising the +// constructs a javadoc page actually has sections for: generic interfaces and their implementors, +// an abstract base class, an enum, an annotation type, a checked exception, nested types, +// compile-time constants, deprecation, and the full set of javadoc block tags. +plugins { + java + // Must match the dokka-core/dokka-base version kdoc-to-json was compiled against. + id("org.jetbrains.dokka") version "2.2.0-Beta" +} + +repositories { + // Lets Gradle find the locally published kdoc-to-json plugin. + mavenLocal() + mavenCentral() +} + +dependencies { + dokkaPlugin("org.appdevforall.dokka:kdoc-to-json:1.0.0-SNAPSHOT") +} + +@OptIn(InternalDokkaApi::class) +abstract class JsonOutputPluginParameters @Inject constructor( + name: String +) : DokkaPluginParametersBaseSpec(name, "org.appdevforall.dokka.kdoc2json.JsonOutputPlugin") { + // As in example-data-processor: point KDOC2JSON_TEST_CONFIG at a JSON file to drive this + // project through an arbitrary plugin config without editing this build script. + override fun jsonEncode(): String { + val overridePath = System.getenv("KDOC2JSON_TEST_CONFIG") + if (overridePath != null) { + return File(overridePath).readText() + } + return """{ + "logLevel": "debug", + "javadoc-mode": true, + "prettyPrint": true + }""" + } +} + +dokka { + dokkaSourceSets.configureEach { + // javadoc documents public *and* protected members by default; Dokka documents only + // public. Without this, Javadoc mode would silently omit every protected member that a + // real javadoc build would have shown. + documentedVisibilities.set(setOf(VisibilityModifier.Public, VisibilityModifier.Protected)) + } + + pluginsConfiguration { + registerBinding(JsonOutputPluginParameters::class, JsonOutputPluginParameters::class) + register("org.appdevforall.dokka.kdoc2json.JsonOutputPlugin") { } + } +} diff --git a/Dokka-plugin-kdoc2json/examples/example-java-library/gradle/wrapper/gradle-wrapper.jar b/Dokka-plugin-kdoc2json/examples/example-java-library/gradle/wrapper/gradle-wrapper.jar new file mode 100644 index 00000000..1b33c55b Binary files /dev/null and b/Dokka-plugin-kdoc2json/examples/example-java-library/gradle/wrapper/gradle-wrapper.jar differ diff --git a/Dokka-plugin-kdoc2json/examples/example-java-library/gradle/wrapper/gradle-wrapper.properties b/Dokka-plugin-kdoc2json/examples/example-java-library/gradle/wrapper/gradle-wrapper.properties new file mode 100644 index 00000000..df6a6ad7 --- /dev/null +++ b/Dokka-plugin-kdoc2json/examples/example-java-library/gradle/wrapper/gradle-wrapper.properties @@ -0,0 +1,9 @@ +distributionBase=GRADLE_USER_HOME +distributionPath=wrapper/dists +distributionUrl=https\://services.gradle.org/distributions/gradle-9.5.1-bin.zip +networkTimeout=10000 +retries=0 +retryBackOffMs=500 +validateDistributionUrl=true +zipStoreBase=GRADLE_USER_HOME +zipStorePath=wrapper/dists diff --git a/Dokka-plugin-kdoc2json/examples/example-java-library/gradlew b/Dokka-plugin-kdoc2json/examples/example-java-library/gradlew new file mode 100755 index 00000000..b9bb139f --- /dev/null +++ b/Dokka-plugin-kdoc2json/examples/example-java-library/gradlew @@ -0,0 +1,248 @@ +#!/bin/sh + +# +# Copyright © 2015 the original authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# SPDX-License-Identifier: Apache-2.0 +# + +############################################################################## +# +# Gradle start up script for POSIX generated by Gradle. +# +# Important for running: +# +# (1) You need a POSIX-compliant shell to run this script. If your /bin/sh is +# noncompliant, but you have some other compliant shell such as ksh or +# bash, then to run this script, type that shell name before the whole +# command line, like: +# +# ksh Gradle +# +# Busybox and similar reduced shells will NOT work, because this script +# requires all of these POSIX shell features: +# * functions; +# * expansions «$var», «${var}», «${var:-default}», «${var+SET}», +# «${var#prefix}», «${var%suffix}», and «$( cmd )»; +# * compound commands having a testable exit status, especially «case»; +# * various built-in commands including «command», «set», and «ulimit». +# +# Important for patching: +# +# (2) This script targets any POSIX shell, so it avoids extensions provided +# by Bash, Ksh, etc; in particular arrays are avoided. +# +# The "traditional" practice of packing multiple parameters into a +# space-separated string is a well documented source of bugs and security +# problems, so this is (mostly) avoided, by progressively accumulating +# options in "$@", and eventually passing that to Java. +# +# Where the inherited environment variables (DEFAULT_JVM_OPTS, JAVA_OPTS, +# and GRADLE_OPTS) rely on word-splitting, this is performed explicitly; +# see the in-line comments for details. +# +# There are tweaks for specific operating systems such as AIX, CygWin, +# Darwin, MinGW, and NonStop. +# +# (3) This script is generated from the Groovy template +# https://github.com/gradle/gradle/blob/3d91ce3b8caaf77ad09f381f43615b715b53f72c/platforms/jvm/plugins-application/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt +# within the Gradle project. +# +# You can find Gradle at https://github.com/gradle/gradle/. +# +############################################################################## + +# Attempt to set APP_HOME + +# Resolve links: $0 may be a link +app_path=$0 + +# Need this for daisy-chained symlinks. +while + APP_HOME=${app_path%"${app_path##*/}"} # leaves a trailing /; empty if no leading path + [ -h "$app_path" ] +do + ls=$( ls -ld "$app_path" ) + link=${ls#*' -> '} + case $link in #( + /*) app_path=$link ;; #( + *) app_path=$APP_HOME$link ;; + esac +done + +# This is normally unused +# shellcheck disable=SC2034 +APP_BASE_NAME=${0##*/} +# Discard cd standard output in case $CDPATH is set (https://github.com/gradle/gradle/issues/25036) +APP_HOME=$( cd -P "${APP_HOME:-./}" > /dev/null && printf '%s\n' "$PWD" ) || exit + +# Use the maximum available, or set MAX_FD != -1 to use that value. +MAX_FD=maximum + +warn () { + echo "$*" +} >&2 + +die () { + echo + echo "$*" + echo + exit 1 +} >&2 + +# OS specific support (must be 'true' or 'false'). +cygwin=false +msys=false +darwin=false +nonstop=false +case "$( uname )" in #( + CYGWIN* ) cygwin=true ;; #( + Darwin* ) darwin=true ;; #( + MSYS* | MINGW* ) msys=true ;; #( + NONSTOP* ) nonstop=true ;; +esac + + + +# Determine the Java command to use to start the JVM. +if [ -n "$JAVA_HOME" ] ; then + if [ -x "$JAVA_HOME/jre/sh/java" ] ; then + # IBM's JDK on AIX uses strange locations for the executables + JAVACMD=$JAVA_HOME/jre/sh/java + else + JAVACMD=$JAVA_HOME/bin/java + fi + if [ ! -x "$JAVACMD" ] ; then + die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME + +Please set the JAVA_HOME variable in your environment to match the +location of your Java installation." + fi +else + JAVACMD=java + if ! command -v java >/dev/null 2>&1 + then + die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. + +Please set the JAVA_HOME variable in your environment to match the +location of your Java installation." + fi +fi + +# Increase the maximum file descriptors if we can. +if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then + case $MAX_FD in #( + max*) + # In POSIX sh, ulimit -H is undefined. That's why the result is checked to see if it worked. + # shellcheck disable=SC2039,SC3045 + MAX_FD=$( ulimit -H -n ) || + warn "Could not query maximum file descriptor limit" + esac + case $MAX_FD in #( + '' | soft) :;; #( + *) + # In POSIX sh, ulimit -n is undefined. That's why the result is checked to see if it worked. + # shellcheck disable=SC2039,SC3045 + ulimit -n "$MAX_FD" || + warn "Could not set maximum file descriptor limit to $MAX_FD" + esac +fi + +# Collect all arguments for the java command, stacking in reverse order: +# * args from the command line +# * the main class name +# * -classpath +# * -D...appname settings +# * --module-path (only if needed) +# * DEFAULT_JVM_OPTS, JAVA_OPTS, and GRADLE_OPTS environment variables. + +# For Cygwin or MSYS, switch paths to Windows format before running java +if "$cygwin" || "$msys" ; then + APP_HOME=$( cygpath --path --mixed "$APP_HOME" ) + + JAVACMD=$( cygpath --unix "$JAVACMD" ) + + # Now convert the arguments - kludge to limit ourselves to /bin/sh + for arg do + if + case $arg in #( + -*) false ;; # don't mess with options #( + /?*) t=${arg#/} t=/${t%%/*} # looks like a POSIX filepath + [ -e "$t" ] ;; #( + *) false ;; + esac + then + arg=$( cygpath --path --ignore --mixed "$arg" ) + fi + # Roll the args list around exactly as many times as the number of + # args, so each arg winds up back in the position where it started, but + # possibly modified. + # + # NB: a `for` loop captures its iteration list before it begins, so + # changing the positional parameters here affects neither the number of + # iterations, nor the values presented in `arg`. + shift # remove old arg + set -- "$@" "$arg" # push replacement arg + done +fi + + +# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. +DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"' + +# Collect all arguments for the java command: +# * DEFAULT_JVM_OPTS, JAVA_OPTS, and optsEnvironmentVar are not allowed to contain shell fragments, +# and any embedded shellness will be escaped. +# * For example: A user cannot expect ${Hostname} to be expanded, as it is an environment variable and will be +# treated as '${Hostname}' itself on the command line. + +set -- \ + "-Dorg.gradle.appname=$APP_BASE_NAME" \ + -jar "$APP_HOME/gradle/wrapper/gradle-wrapper.jar" \ + "$@" + +# Stop when "xargs" is not available. +if ! command -v xargs >/dev/null 2>&1 +then + die "xargs is not available" +fi + +# Use "xargs" to parse quoted args. +# +# With -n1 it outputs one arg per line, with the quotes and backslashes removed. +# +# In Bash we could simply go: +# +# readarray ARGS < <( xargs -n1 <<<"$var" ) && +# set -- "${ARGS[@]}" "$@" +# +# but POSIX shell has neither arrays nor command substitution, so instead we +# post-process each arg (as a line of input to sed) to backslash-escape any +# character that might be a shell metacharacter, then use eval to reverse +# that process (while maintaining the separation between arguments), and wrap +# the whole thing up as a single "set" statement. +# +# This will of course break if any of these variables contains a newline or +# an unmatched quote. +# + +eval "set -- $( + printf '%s\n' "$DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS" | + xargs -n1 | + sed ' s~[^-[:alnum:]+,./:=@_]~\\&~g; ' | + tr '\n' ' ' + )" '"$@"' + +exec "$JAVACMD" "$@" diff --git a/Dokka-plugin-kdoc2json/examples/example-java-library/gradlew.bat b/Dokka-plugin-kdoc2json/examples/example-java-library/gradlew.bat new file mode 100644 index 00000000..aa5f10b0 --- /dev/null +++ b/Dokka-plugin-kdoc2json/examples/example-java-library/gradlew.bat @@ -0,0 +1,82 @@ +@rem +@rem Copyright 2015 the original author or authors. +@rem +@rem Licensed under the Apache License, Version 2.0 (the "License"); +@rem you may not use this file except in compliance with the License. +@rem You may obtain a copy of the License at +@rem +@rem https://www.apache.org/licenses/LICENSE-2.0 +@rem +@rem Unless required by applicable law or agreed to in writing, software +@rem distributed under the License is distributed on an "AS IS" BASIS, +@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +@rem See the License for the specific language governing permissions and +@rem limitations under the License. +@rem +@rem SPDX-License-Identifier: Apache-2.0 +@rem + +@if "%DEBUG%"=="" @echo off +@rem ########################################################################## +@rem +@rem Gradle startup script for Windows +@rem +@rem ########################################################################## + +@rem Set local scope for the variables, and ensure extensions are enabled +setlocal EnableExtensions + +set DIRNAME=%~dp0 +if "%DIRNAME%"=="" set DIRNAME=. +@rem This is normally unused +set APP_BASE_NAME=%~n0 +set APP_HOME=%DIRNAME% + +@rem Resolve any "." and ".." in APP_HOME to make it shorter. +for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi + +@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. +set DEFAULT_JVM_OPTS="-Xmx64m" "-Xms64m" + +@rem Find java.exe +if defined JAVA_HOME goto findJavaFromJavaHome + +set JAVA_EXE=java.exe +%JAVA_EXE% -version >NUL 2>&1 +if %ERRORLEVEL% equ 0 goto execute + +echo. 1>&2 +echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. 1>&2 +echo. 1>&2 +echo Please set the JAVA_HOME variable in your environment to match the 1>&2 +echo location of your Java installation. 1>&2 + +"%COMSPEC%" /c exit 1 + +:findJavaFromJavaHome +set JAVA_HOME=%JAVA_HOME:"=% +set JAVA_EXE=%JAVA_HOME%/bin/java.exe + +if exist "%JAVA_EXE%" goto execute + +echo. 1>&2 +echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% 1>&2 +echo. 1>&2 +echo Please set the JAVA_HOME variable in your environment to match the 1>&2 +echo location of your Java installation. 1>&2 + +"%COMSPEC%" /c exit 1 + +:execute +@rem Setup the command line + + + +@rem Execute Gradle +@rem endlocal doesn't take effect until after the line is parsed and variables are expanded +@rem which allows us to clear the local environment before executing the java command +endlocal & "%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -jar "%APP_HOME%\gradle\wrapper\gradle-wrapper.jar" %* & call :exitWithErrorLevel + +:exitWithErrorLevel +@rem Use "%COMSPEC%" /c exit to allow operators to work properly in scripts +"%COMSPEC%" /c exit %ERRORLEVEL% diff --git a/Dokka-plugin-kdoc2json/examples/example-java-library/settings.gradle.kts b/Dokka-plugin-kdoc2json/examples/example-java-library/settings.gradle.kts new file mode 100644 index 00000000..27680a63 --- /dev/null +++ b/Dokka-plugin-kdoc2json/examples/example-java-library/settings.gradle.kts @@ -0,0 +1 @@ +rootProject.name = "javalib" diff --git a/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/com/example/shapes/AbstractShape.java b/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/com/example/shapes/AbstractShape.java new file mode 100644 index 00000000..c4b70277 --- /dev/null +++ b/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/com/example/shapes/AbstractShape.java @@ -0,0 +1,46 @@ +package com.example.shapes; + +/** + * Skeletal implementation of {@link Shape} that supplies the parts every shape shares. + * + * @param the unit of measure areas are reported in + * @since 1.0 + */ +public abstract class AbstractShape implements Shape { + + /** Identifies this shape for diagnostics; never {@code null}. */ + protected final String name; + + /** + * Creates a shape with the given diagnostic name. + * + * @param name the shape's name + */ + protected AbstractShape(String name) { + this.name = name; + } + + /** + * {@inheritDoc} + * + *

This implementation always reports four sides.

+ */ + @Override + public int sides() { + return 4; + } + + /** + * Returns this shape's diagnostic name. + * + * @return the name passed to the constructor + */ + public String getName() { + return name; + } + + @Override + public String toString() { + return name + "[" + area() + "]"; + } +} diff --git a/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/com/example/shapes/Corner.java b/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/com/example/shapes/Corner.java new file mode 100644 index 00000000..67171053 --- /dev/null +++ b/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/com/example/shapes/Corner.java @@ -0,0 +1,35 @@ +package com.example.shapes; + +/** + * The four corners of an axis-aligned bounding box. + * + * @since 1.0 + */ +public enum Corner { + + /** The top-left corner. */ + TOP_LEFT, + + /** The top-right corner. */ + TOP_RIGHT, + + /** The bottom-left corner. */ + BOTTOM_LEFT, + + /** The bottom-right corner. */ + BOTTOM_RIGHT; + + /** + * Returns the corner diagonally opposite this one. + * + * @return the opposite corner + */ + public Corner opposite() { + switch (this) { + case TOP_LEFT: return BOTTOM_RIGHT; + case TOP_RIGHT: return BOTTOM_LEFT; + case BOTTOM_LEFT: return TOP_RIGHT; + default: return TOP_LEFT; + } + } +} diff --git a/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/com/example/shapes/Measured.java b/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/com/example/shapes/Measured.java new file mode 100644 index 00000000..31e6d4ca --- /dev/null +++ b/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/com/example/shapes/Measured.java @@ -0,0 +1,30 @@ +package com.example.shapes; + +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +/** + * Marks a type whose measurements have been verified against a reference implementation. + * + * @since 1.2 + */ +@Retention(RetentionPolicy.RUNTIME) +@Target(ElementType.TYPE) +public @interface Measured { + + /** + * The tolerance the measurements were verified to. + * + * @return the absolute tolerance + */ + double tolerance(); + + /** + * Who performed the verification. + * + * @return the verifier's name, or the empty string if unrecorded + */ + String verifiedBy() default ""; +} diff --git a/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/com/example/shapes/Rectangle.java b/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/com/example/shapes/Rectangle.java new file mode 100644 index 00000000..fe1c0916 --- /dev/null +++ b/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/com/example/shapes/Rectangle.java @@ -0,0 +1,122 @@ +package com.example.shapes; + +/** + * An axis-aligned rectangle measured in {@code double} units. + * + *

Example:

+ * 
{@code
+ * Rectangle r = new Rectangle(3.0, 4.0);
+ * assert r.area() == 12.0;
+ * }
+ * + * @since 1.0 + * @see Shape + */ +public class Rectangle extends AbstractShape { + + /** A rectangle of zero width and height. */ + public static final String EMPTY_LABEL = "empty"; + + /** The number of sides a rectangle always has. */ + public static final int SIDE_COUNT = 4; + + private final double width; + private final double height; + + /** + * Creates a rectangle of the given dimensions. + * + * @param width the width, must not be negative + * @param height the height, must not be negative + * @throws IllegalArgumentException if either dimension is negative + */ + public Rectangle(double width, double height) { + super("rectangle"); + if (width < 0 || height < 0) { + throw new IllegalArgumentException("dimensions must not be negative"); + } + this.width = width; + this.height = height; + } + + /** Creates a unit square. */ + public Rectangle() { + this(1.0, 1.0); + } + + @Override + public Double area() { + return width * height; + } + + /** + * Returns the rectangle's width. + * + * @return the width in unspecified units + */ + public double getWidth() { + return width; + } + + /** + * Returns the rectangle's height. + * + * @return the height in unspecified units + */ + public double getHeight() { + return height; + } + + /** + * Returns the perimeter. + * + * @return twice the sum of width and height + * @deprecated Use {@link #getWidth()} and {@link #getHeight()} and compute it directly. + * Scheduled for removal in 3.0. + */ + @Deprecated(since = "2.0", forRemoval = true) + public double perimeter() { + return 2 * (width + height); + } + + /** + * A builder for {@link Rectangle} instances. + * + *

Nested to exercise javadoc's nested-type sections.

+ */ + public static final class Builder { + private double width; + private double height; + + /** + * Sets the width. + * + * @param width the width + * @return this builder + */ + public Builder width(double width) { + this.width = width; + return this; + } + + /** + * Sets the height. + * + * @param height the height + * @return this builder + */ + public Builder height(double height) { + this.height = height; + return this; + } + + /** + * Builds the rectangle. + * + * @return a new rectangle + */ + public Rectangle build() { + return new Rectangle(width, height); + } + } +} diff --git a/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/com/example/shapes/Shape.java b/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/com/example/shapes/Shape.java new file mode 100644 index 00000000..27506c20 --- /dev/null +++ b/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/com/example/shapes/Shape.java @@ -0,0 +1,47 @@ +package com.example.shapes; + +/** + * A closed geometric figure with a computable area. + * + *

Implementations are expected to be immutable; the {@link #area()} of a shape must not change + * over its lifetime. This mirrors the contract style used throughout the JDK's own collection + * interfaces.

+ * + * @param the unit of measure areas are reported in + * @author Docs Pipeline + * @since 1.0 + * @see Rectangle + */ +public interface Shape { + + /** The maximum number of sides any shape in this library may declare. */ + int MAX_SIDES = 64; + + /** + * Returns the area enclosed by this shape. + * + * @return the enclosed area, never negative + */ + U area(); + + /** + * Returns the number of sides this shape has. + * + * @return the side count, between 0 and {@value #MAX_SIDES} + */ + int sides(); + + /** + * Scales this shape by the given factor. + * + * @param factor the scaling factor, must be positive + * @return a new scaled shape + * @throws IllegalArgumentException if {@code factor} is not positive + */ + default Shape scaled(double factor) { + if (factor <= 0) { + throw new IllegalArgumentException("factor must be positive"); + } + return this; + } +} diff --git a/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/com/example/shapes/ShapeException.java b/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/com/example/shapes/ShapeException.java new file mode 100644 index 00000000..562a1849 --- /dev/null +++ b/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/com/example/shapes/ShapeException.java @@ -0,0 +1,20 @@ +package com.example.shapes; + +/** + * Thrown when a shape cannot be constructed from the supplied measurements. + * + * @since 1.0 + */ +public class ShapeException extends IllegalArgumentException { + + private static final long serialVersionUID = 1L; + + /** + * Creates an exception with the given detail message. + * + * @param message the detail message + */ + public ShapeException(String message) { + super(message); + } +} diff --git a/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/com/example/shapes/Square.java b/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/com/example/shapes/Square.java new file mode 100644 index 00000000..0681f28c --- /dev/null +++ b/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/com/example/shapes/Square.java @@ -0,0 +1,18 @@ +package com.example.shapes; + +/** + * A rectangle whose sides are all equal. + * + * @since 1.1 + */ +public class Square extends Rectangle { + + /** + * Creates a square with the given side length. + * + * @param side the length of each side + */ + public Square(double side) { + super(side, side); + } +} diff --git a/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/com/example/shapes/package-info.java b/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/com/example/shapes/package-info.java new file mode 100644 index 00000000..5e5783f9 --- /dev/null +++ b/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/com/example/shapes/package-info.java @@ -0,0 +1,9 @@ +/** + * Geometric shapes and the operations over them. + * + *

The central abstraction is {@link com.example.shapes.Shape}, implemented by + * {@link com.example.shapes.Rectangle} and its subclasses.

+ * + * @since 1.0 + */ +package com.example.shapes; diff --git a/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/com/example/shapes/spi/ShapeFactory.java b/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/com/example/shapes/spi/ShapeFactory.java new file mode 100644 index 00000000..a9f3e237 --- /dev/null +++ b/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/com/example/shapes/spi/ShapeFactory.java @@ -0,0 +1,23 @@ +package com.example.shapes.spi; + +import com.example.shapes.Shape; + +/** + * Service provider interface for creating shapes from a textual specification. + * + *

Exists in a second package so Javadoc mode's package tables and cross-package links have + * something to resolve.

+ * + * @since 1.2 + */ +public interface ShapeFactory { + + /** + * Parses a shape from its textual form. + * + * @param specification the shape specification + * @return the parsed shape + * @throws java.text.ParseException if the specification is malformed + */ + Shape parse(String specification) throws java.text.ParseException; +} diff --git a/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/com/example/shapes/spi/package-info.java b/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/com/example/shapes/spi/package-info.java new file mode 100644 index 00000000..1a39dbd5 --- /dev/null +++ b/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/com/example/shapes/spi/package-info.java @@ -0,0 +1,6 @@ +/** + * Extension points for supplying shapes from outside this library. + * + * @since 1.2 + */ +package com.example.shapes.spi; diff --git a/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/module-info.java b/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/module-info.java new file mode 100644 index 00000000..81be309d --- /dev/null +++ b/Dokka-plugin-kdoc2json/examples/example-java-library/src/main/java/module-info.java @@ -0,0 +1,18 @@ +/** + * Defines a small geometry library. + * + *

Present so the plugin's Javadoc mode has a real {@code module-info.java} to read: the module + * page's requires / exports / uses / provides sections come from here, not from Dokka's model.

+ * + * @uses com.example.shapes.spi.ShapeFactory + * @since 1.0 + */ +module com.example.shapes { + requires transitive java.logging; + requires static java.sql; + + exports com.example.shapes; + exports com.example.shapes.spi; + + uses com.example.shapes.spi.ShapeFactory; +} diff --git a/Dokka-plugin-kdoc2json/kdoc-to-json/build.gradle.kts b/Dokka-plugin-kdoc2json/kdoc-to-json/build.gradle.kts index 3343a62e..92e67de4 100644 --- a/Dokka-plugin-kdoc2json/kdoc-to-json/build.gradle.kts +++ b/Dokka-plugin-kdoc2json/kdoc-to-json/build.gradle.kts @@ -14,6 +14,11 @@ repositories { dependencies { compileOnly("org.jetbrains.dokka:dokka-core:2.2.0-Beta") compileOnly("org.jetbrains.dokka:dokka-base:2.2.0-Beta") + // Dokka deserializes this plugin's config block with Jackson, not kotlinx.serialization + // (see org.jetbrains.dokka.utilities.parseJson), so a config key whose JSON spelling + // differs from its Kotlin property name needs @JsonProperty to be seen -- @SerialName + // alone is ignored on that path. compileOnly: Dokka already brings Jackson at runtime. + compileOnly("com.fasterxml.jackson.core:jackson-annotations:2.15.3") implementation("org.jetbrains.kotlinx:kotlinx-serialization-json:1.6.0") } diff --git a/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/JsonFilters.kt b/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/JsonFilters.kt new file mode 100644 index 00000000..ceeb9abf --- /dev/null +++ b/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/JsonFilters.kt @@ -0,0 +1,46 @@ +package org.appdevforall.dokka.kdoc2json + +import kotlinx.serialization.json.JsonArray +import kotlinx.serialization.json.JsonElement +import kotlinx.serialization.json.JsonNull +import kotlinx.serialization.json.JsonObject +import kotlinx.serialization.json.JsonPrimitive + +/** + * The `omitFields` / `omitNulls` post-processing both renderers apply to a page before writing it. + * + * Lives outside [JsonRenderer] so Javadoc mode honours exactly the same two config options with + * exactly the same semantics, rather than reimplementing them and drifting. + */ +internal object JsonFilters { + + /** Strips [omitFields] keys everywhere, and (when [omitNulls]) null/empty values with them. */ + fun filterJson(element: JsonElement, omitFields: List, omitNulls: Boolean): JsonElement { + if (omitFields.isEmpty() && !omitNulls) return element + + return when (element) { + is JsonObject -> { + val filteredMap = element.entries + .filterNot { omitFields.contains(it.key) } + .mapNotNull { (key, value) -> + val filteredValue = filterJson(value, omitFields, omitNulls) + if (omitNulls && isNullOrEmpty(filteredValue)) null else key to filteredValue + } + .toMap() + JsonObject(filteredMap) + } + is JsonArray -> { + val mapped = element.map { filterJson(it, omitFields, omitNulls) } + if (omitNulls) JsonArray(mapped.filterNot { isNullOrEmpty(it) }) else JsonArray(mapped) + } + else -> element + } + } + + /** `omitNulls` drops empty values too, not just nulls -- see the note in the plugin README. */ + fun isNullOrEmpty(element: JsonElement): Boolean = + element is JsonNull || + (element is JsonPrimitive && element.isString && element.content.isEmpty()) || + (element is JsonArray && element.isEmpty()) || + (element is JsonObject && element.isEmpty()) +} diff --git a/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/JsonPluginConfig.kt b/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/JsonPluginConfig.kt index 61e3273a..67b93332 100644 --- a/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/JsonPluginConfig.kt +++ b/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/JsonPluginConfig.kt @@ -1,5 +1,7 @@ package org.appdevforall.dokka.kdoc2json +import com.fasterxml.jackson.annotation.JsonProperty +import kotlinx.serialization.SerialName import kotlinx.serialization.Serializable import org.jetbrains.dokka.plugability.ConfigurableBlock @@ -12,5 +14,18 @@ data class JsonPluginConfig( val omitNulls: Boolean = false, val classDiscriminator: String = "kind", val prettyPrint: Boolean = false, - val sourceSetWhitelist: List = emptyList() -) : ConfigurableBlock \ No newline at end of file + val sourceSetWhitelist: List = emptyList(), + // Opt-in "Javadoc mode": instead of Dokka-shaped JSON at Dokka's own page paths, emit + // javadoc-shaped JSON laid out like the `api/` tree that the `javadoc` tool produces + // (module-summary / package-summary / pages plus the global index files). + // + // Spelled kebab-case in the config on purpose -- that is the documented spelling of the + // switch -- even though every other option here is camelCase. That costs two annotations + // rather than one, because this config block is read by two different deserializers: + // Dokka's own pluginsConfiguration parsing uses Jackson (@JsonProperty), while + // JsonRenderer's manual fallback uses kotlinx.serialization (@SerialName). Dropping either + // would leave "javadoc-mode" silently ignored on one of the two paths. + @JsonProperty("javadoc-mode") + @SerialName("javadoc-mode") + val javadocMode: Boolean = false +) : ConfigurableBlock diff --git a/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/JsonRenderer.kt b/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/JsonRenderer.kt index 67e2b63d..8a07b954 100644 --- a/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/JsonRenderer.kt +++ b/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/JsonRenderer.kt @@ -3,6 +3,7 @@ package org.appdevforall.dokka.kdoc2json import kotlinx.serialization.decodeFromString import kotlinx.serialization.json.* import org.appdevforall.dokka.kdoc2json.dtos.* +import org.appdevforall.dokka.kdoc2json.javadoc.JavadocRenderer import org.jetbrains.dokka.base.DokkaBase import org.jetbrains.dokka.model.* import org.jetbrains.dokka.pages.PageNode @@ -42,13 +43,33 @@ class JsonRenderer(private val context: DokkaContext) : Renderer { val finalConfig = config ?: JsonPluginConfig() val logger = PluginLogger(context.logger, finalConfig.logLevel, finalConfig.logFile) + logger.info("Initializing JSON Renderer with config: $finalConfig") + + if (finalConfig.javadocMode) { + // Javadoc mode replaces the whole output: a different file layout, a different page + // shape, and its own link resolution -- so it takes over here rather than trying to + // post-process the Dokka-shaped output into javadoc's structure. It also has no use + // for the location provider, the package-list, or the LinkPostProcessor pass below, + // all of which exist to serve Dokka's own page paths. + logger.info("javadoc-mode enabled: emitting javadoc-shaped JSON instead of Dokka-shaped JSON.") + JavadocRenderer( + config = finalConfig, + logger = logger, + outputDir = context.configuration.outputDir, + moduleReferences = context.configuration.modules.map { + it.name to it.relativePathToOutputDirectory.invariantSeparatorsPath + }, + sourceRoots = context.configuration.sourceSets.flatMap { it.sourceRoots }.distinct() + ).render(root) + logger.info("JSON rendering completed (javadoc mode).") + return + } + val json = Json { prettyPrint = finalConfig.prettyPrint classDiscriminator = finalConfig.classDiscriminator } - logger.info("Initializing JSON Renderer with config: $finalConfig") - val locationProvider = context.plugin() .querySingle { locationProviderFactory } .getLocationProvider(root) @@ -220,43 +241,9 @@ class JsonRenderer(private val context: DokkaContext) : Renderer { logger.info("JSON rendering completed.") } - // --- RECURSIVE JSON AST FILTER --- - private fun filterJson(element: JsonElement, omitFields: List, omitNulls: Boolean): JsonElement { - if (omitFields.isEmpty() && !omitNulls) return element - - return when (element) { - is JsonObject -> { - val filteredMap = element.entries - .filterNot { omitFields.contains(it.key) } - .mapNotNull { (key, value) -> - val filteredValue = filterJson(value, omitFields, omitNulls) - if (omitNulls && isNullOrEmpty(filteredValue)) { - null - } else { - key to filteredValue - } - } - .toMap() - JsonObject(filteredMap) - } - is JsonArray -> { - val mapped = element.map { filterJson(it, omitFields, omitNulls) } - if (omitNulls) { - JsonArray(mapped.filterNot { isNullOrEmpty(it) }) - } else { - JsonArray(mapped) - } - } - else -> element - } - } - - private fun isNullOrEmpty(element: JsonElement): Boolean { - return element is JsonNull || - (element is JsonPrimitive && element.isString && element.content.isEmpty()) || - (element is JsonArray && element.isEmpty()) || - (element is JsonObject && element.isEmpty()) - } + // Delegates to JsonFilters so Javadoc mode applies identical omitFields/omitNulls semantics. + private fun filterJson(element: JsonElement, omitFields: List, omitNulls: Boolean): JsonElement = + JsonFilters.filterJson(element, omitFields, omitNulls) private fun passesSourceSetWhitelist( sourceSets: Set, diff --git a/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/javadoc/JavadocDocs.kt b/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/javadoc/JavadocDocs.kt new file mode 100644 index 00000000..82a5e59b --- /dev/null +++ b/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/javadoc/JavadocDocs.kt @@ -0,0 +1,281 @@ +package org.appdevforall.dokka.kdoc2json.javadoc + +import org.jetbrains.dokka.links.DRI +import org.jetbrains.dokka.model.Documentable +import org.jetbrains.dokka.model.doc.* +// kotlin.Deprecated is a default import and would otherwise win over the star import here. +import org.jetbrains.dokka.model.doc.Deprecated as DeprecatedTag + +/** + * The javadoc block tags of one declaration, pulled out of Dokka's [DocumentationNode] and sorted + * into the buckets a javadoc page actually renders. + * + * @param params `@param` text keyed by parameter name -- type parameters included, keyed by their + * bare name (Dokka strips the angle brackets `@param ` is written with). + * @param other every block tag with no dedicated bucket (`@apiNote`, `@implSpec`, `@serial`, ...), + * in source order, so nothing in the source comment is silently dropped. + */ +class JavadocDocBundle( + val description: String? = null, + val params: Map = emptyMap(), + val returns: String? = null, + val throws: List> = emptyList(), + val seeAlso: List> = emptyList(), + val since: List = emptyList(), + val authors: List = emptyList(), + val versions: List = emptyList(), + val deprecated: String? = null, + val isDeprecatedTagPresent: Boolean = false, + val other: List = emptyList() +) + +/** + * Turns Dokka doc trees into the HTML strings javadoc pages carry, and sorts block tags into + * [JavadocDocBundle]. + * + * Output is HTML rather than Markdown because that is what a javadoc comment already contains and + * what the existing renderer emits, so a downstream template can drop either into a page unchanged. + * + * @param resolveLink maps a link target to a URL relative to the page being written; returns null + * for a target this run doesn't document, in which case the link degrades to plain text rather + * than becoming a dead `href`. + */ +class JavadocDocs( + private val resolveLink: (DRI) -> String?, + /** + * What `{@docRoot}` expands to on the page being rendered -- the relative path back to the + * documentation root. JDK comments use it inside raw `` markup, which + * Dokka hands over as a literal attribute value, so it has to be substituted here or the link + * ships with the tag still in it. + */ + private val docRoot: String = ".", + /** + * Rebases a *relative* href written by hand in a doc comment (``). + * + * Such an href is relative to the page that *declares* the comment. When the same comment is + * shown somewhere else -- a summary on an index page -- it has to be re-expressed relative to + * the page it now appears on, or it points at nothing. The identity default is correct while + * rendering a declaration on its own page. + */ + private val rebaseRelativeHref: (String) -> String = { it } +) { + + /** Applies [rebaseRelativeHref] to an `href` attribute, leaving every other attribute alone. */ + private fun rebased(params: Map): Map { + val href = params["href"] ?: return params + if (!isRelative(href)) return params + return params.toMutableMap().apply { put("href", rebaseRelativeHref(href)) } + } + + /** True for an href that resolves against the current page rather than a root or a host. */ + private fun isRelative(href: String): Boolean = + href.isNotBlank() && + !href.startsWith("#") && + !href.startsWith("/") && + !href.contains("://") && + !href.startsWith("mailto:") && + !href.startsWith(DOC_ROOT_TAG) + + /** + * Picks the doc comment to render. Javadoc has no notion of source sets, so where Dokka has + * several this takes the first that actually carries tags, which for a Java run is the only one. + */ + fun bundleFor(doc: Documentable): JavadocDocBundle { + val node: DocumentationNode = doc.documentation.entries + .firstOrNull { it.value.children.isNotEmpty() } + ?.value + ?: return JavadocDocBundle() + + var description: String? = null + val params = LinkedHashMap() + var returns: String? = null + val throws = mutableListOf>() + val seeAlso = mutableListOf>() + val since = mutableListOf() + val authors = mutableListOf() + val versions = mutableListOf() + var deprecated: String? = null + var deprecatedPresent = false + val other = mutableListOf() + + node.children.forEach { tag -> + val text = render(tag.root).trim() + when (tag) { + is Description -> description = listOfNotNull(description?.takeIf { it.isNotBlank() }, text) + .filter { it.isNotBlank() } + .joinToString("\n") + .ifBlank { null } + is Param -> params[tag.name] = text + is Return -> returns = text + is Throws -> throws += Triple(tag.name, tag.exceptionAddress, text) + is See -> seeAlso += Triple(tag.name, tag.address, text) + is Since -> since += unwrapParagraph(text) + is Author -> authors += unwrapParagraph(text) + is Version -> versions += unwrapParagraph(text) + is DeprecatedTag -> { + deprecatedPresent = true + deprecated = text.ifBlank { null } + } + is CustomTagWrapper -> other += JdTag(tag.name, text) + else -> other += JdTag(tag::class.java.simpleName, text) + } + } + + return JavadocDocBundle( + description = description?.ifBlank { null }, + params = params, + returns = returns?.ifBlank { null }, + throws = throws, + seeAlso = seeAlso, + since = since.filter { it.isNotBlank() }, + authors = authors.filter { it.isNotBlank() }, + versions = versions.filter { it.isNotBlank() }, + deprecated = deprecated, + isDeprecatedTagPresent = deprecatedPresent, + other = other.filter { it.text.isNotBlank() } + ) + } + + /** + * Renders one doc tree back to HTML. + * + * JDK javadoc comments are full HTML -- tables, definition lists, `
` -- so + * structural tags and their attributes are preserved rather than flattened to their text. + * Anything Dokka parsed into a tag this doesn't know is emitted as its children, which loses + * the wrapper but never the content. + */ + fun render(tag: DocTag): String { + val children = tag.children.joinToString("") { render(it) } + return when (tag) { + is Text -> escapeHtmlText(tag.body) + is Br -> "
" + is HorizontalRule -> "
" + is Img -> "" + is CodeBlock -> "
$children
" + is CodeInline -> "$children" + is A -> "$children" + is DocumentationLink -> { + val href = resolveLink(tag.dri) + // An unresolvable {@link} degrades to its own text rather than an href to nowhere: + // javadoc-mode output is meant to be servable as-is, and a dead link is worse than + // a plain-text mention of the symbol. + if (href == null) children else "$children" + } + is CustomDocTag -> children + else -> { + val htmlName = HTML_TAG_NAMES[tag::class.java.simpleName] + if (htmlName == null) children else "<$htmlName${attributes(tag.params, docRoot)}>$children" + } + } + } + + companion object { + /** + * Dokka doc-tag class name -> HTML element name, for every tag that is just a wrapper + * around its children. Tags needing special handling (links, images, code, line breaks) + * are matched by type in [render] instead and are deliberately absent here. + */ + private val HTML_TAG_NAMES: Map = mapOf( + "P" to "p", "B" to "b", "I" to "i", "Em" to "em", "Strong" to "strong", + "BlockQuote" to "blockquote", "Pre" to "pre", "Ul" to "ul", "Ol" to "ol", "Li" to "li", + "H1" to "h1", "H2" to "h2", "H3" to "h3", "H4" to "h4", "H5" to "h5", "H6" to "h6", + "Dl" to "dl", "Dt" to "dt", "Dd" to "dd", "Div" to "div", "Span" to "span", + "Table" to "table", "THead" to "thead", "TBody" to "tbody", "TFoot" to "tfoot", + "Tr" to "tr", "Td" to "td", "Th" to "th", "Caption" to "caption", + "Sub" to "sub", "Sup" to "sup", "Small" to "small", "Big" to "big", "Var" to "var", + "Tt" to "tt", "U" to "u", "Strikethrough" to "del", "Cite" to "cite", "Code" to "code", + "Dfn" to "dfn", "Mark" to "mark", "Font" to "font", "Menu" to "menu", "Dir" to "dir", + "Section" to "section", "Main" to "main", "Nav" to "nav", "Header" to "header", + "Footer" to "footer", "Listing" to "listing" + ) + + /** + * Strips a single enclosing `

` from a one-paragraph fragment. + * + * Dokka wraps every tag body in a paragraph, but `@since`, `@author` and `@version` are + * plain text in javadoc ("1.0", not "

1.0

"). Anything with internal structure is + * left exactly as it is. + */ + fun unwrapParagraph(html: String): String { + val trimmed = html.trim() + if (!trimmed.startsWith("

") || !trimmed.endsWith("

")) return trimmed + val inner = trimmed.removePrefix("

").removeSuffix("

") + return if (inner.contains("

", ignoreCase = true)) trimmed else inner.trim() + } + + private const val DOC_ROOT_TAG = "{@docRoot}" + + /** Renders a tag's attributes back into HTML, in the order Dokka recorded them. */ + private fun attributes(params: Map, docRoot: String): String = + params.entries.joinToString("") { (key, value) -> + " $key=\"${escapeHtmlAttribute(expandDocRoot(value, docRoot))}\"" + } + + /** Substitutes javadoc's `{@docRoot}` in a raw attribute value. */ + fun expandDocRoot(value: String, docRoot: String): String = + if (DOC_ROOT_TAG in value) value.replace(DOC_ROOT_TAG, docRoot) else value + + private fun escapeHtmlText(value: String): String = + value.replace("&", "&").replace("<", "<").replace(">", ">") + + // "&" first, or escaping the rest afterwards would double-escape a """ that was + // already literally present in the source text. "<" and ">" are escaped as well as the + // quotes: an unescaped ">" inside an attribute value would otherwise look like the end of + // the tag to anything scanning the markup, firstSentence's depth tracking included. + private fun escapeHtmlAttribute(value: String): String = + value.replace("&", "&") + .replace("\"", """) + .replace("<", "<") + .replace(">", ">") + + /** + * The leading sentence of an HTML description, as javadoc shows it in a summary table. + * + * Cuts at the first `.` that sits outside a tag and is followed by whitespace (or ends the + * text), then closes any element the cut left open so the fragment is still well-formed + * HTML. Returns null when there is no description to summarise. + */ + fun firstSentence(html: String?): String? { + if (html.isNullOrBlank()) return null + var depth = 0 + var cut = -1 + for (i in html.indices) { + when (html[i]) { + '<' -> depth++ + '>' -> if (depth > 0) depth-- + '.' -> if (depth == 0) { + val next = html.getOrNull(i + 1) + if (next == null || next.isWhitespace()) { + cut = i + 1 + } + } + } + if (cut >= 0) break + } + val fragment = if (cut >= 0) html.substring(0, cut) else html + return closeOpenTags(fragment.trim()).ifBlank { null } + } + + private val TAG_REGEX = Regex("<\\s*(/?)\\s*([a-zA-Z][a-zA-Z0-9]*)[^>]*?(/?)\\s*>") + private val VOID_ELEMENTS = setOf("br", "hr", "img", "input", "meta", "link", "wbr") + + /** Appends closing tags for any element left open by a truncated HTML fragment. */ + private fun closeOpenTags(fragment: String): String { + val open = ArrayDeque() + TAG_REGEX.findAll(fragment).forEach { match -> + val closing = match.groupValues[1] == "/" + val name = match.groupValues[2].lowercase() + val selfClosing = match.groupValues[3] == "/" + if (name in VOID_ELEMENTS || selfClosing) return@forEach + if (closing) { + // Tolerate stray/mismatched closers instead of corrupting the stack. + if (open.isNotEmpty() && open.last() == name) open.removeLast() + else open.remove(name) + } else { + open.addLast(name) + } + } + return fragment + open.reversed().joinToString("") { "" } + } + } +} diff --git a/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/javadoc/JavadocDtos.kt b/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/javadoc/JavadocDtos.kt new file mode 100644 index 00000000..811d9f07 --- /dev/null +++ b/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/javadoc/JavadocDtos.kt @@ -0,0 +1,450 @@ +package org.appdevforall.dokka.kdoc2json.javadoc + +import kotlinx.serialization.Serializable + +/** + * DTOs mirroring the pages the `javadoc` tool emits under its `api/` output directory. + * + * These are deliberately *not* part of the `DocumentableDto` hierarchy in + * `dtos/SemanticModelDtos.kt`: that hierarchy mirrors Dokka's own AST, whereas everything here + * mirrors what a javadoc page actually presents (inheritance closures, inherited-member groups, + * summary tables, the global index pages). Keeping them separate means Javadoc mode can add + * javadoc-specific concepts without perturbing the default output's schema. + * + * Field naming follows the javadoc page section it comes from, so a consumer holding a javadoc + * page open next to the JSON can match them up section by section. + */ + +// --- Shared building blocks --- + +/** + * A reference to a type. [display] is the javadoc-style rendering including type arguments and + * array dimensions (e.g. `List`, `int[]`, `? extends Number`); [qualifiedName] and [url] are + * populated only when the referenced type is part of this documentation run. + */ +@Serializable +data class JdTypeRef( + val display: String, + val qualifiedName: String? = null, + val url: String? = null, + val kind: String? = null +) + +/** A resolved `@see` / `@link` reference. [url] is null when the target isn't documented here. */ +@Serializable +data class JdSeeRef( + val label: String, + val url: String? = null, + val qualifiedName: String? = null +) + +/** A javadoc block tag this mapper has no dedicated field for (`@apiNote`, `@implSpec`, ...). */ +@Serializable +data class JdTag( + val name: String, + val text: String +) + +@Serializable +data class JdDeprecation( + val comment: String? = null, + val forRemoval: Boolean = false, + val since: String? = null +) + +/** A method/constructor parameter, paired with its `@param` text when the source documents one. */ +@Serializable +data class JdParameter( + val name: String, + val type: JdTypeRef, + val description: String? = null, + val annotations: List = emptyList() +) + +/** A type parameter declaration, paired with its `@param ` text. */ +@Serializable +data class JdTypeParameter( + val name: String, + val bounds: List = emptyList(), + val description: String? = null +) + +/** One entry of a `@throws`/`@exception` list. */ +@Serializable +data class JdThrows( + val type: JdTypeRef, + val description: String? = null +) + +/** + * A pointer at another member -- used for javadoc's "Specified by:" / "Overrides:" notes and for + * the members listed in an inherited-member group. + */ +@Serializable +data class JdMemberRef( + val name: String, + val signature: String, + val url: String? = null, + val declaringType: JdTypeRef? = null +) + +/** + * One "Methods declared in class X" / "Fields declared in interface Y" group, as javadoc renders + * them at the bottom of a summary table. + */ +@Serializable +data class JdInheritedMembers( + val declaringType: JdTypeRef, + val members: List = emptyList() +) + +// --- Members --- + +/** A field, an enum constant, or a record component's backing field. */ +@Serializable +data class JdField( + val name: String, + val anchor: String, + val modifiers: List = emptyList(), + val type: JdTypeRef, + val signature: String, + val url: String? = null, + val description: String? = null, + val firstSentence: String? = null, + val constantValue: String? = null, + val since: List = emptyList(), + val seeAlso: List = emptyList(), + val deprecated: JdDeprecation? = null, + val annotations: List = emptyList(), + val tags: List = emptyList() +) + +/** + * A constructor, a method, or an annotation element. [returnType] is null for constructors; + * [defaultValue] is populated only for annotation elements that declare a `default`. + */ +@Serializable +data class JdExecutable( + val name: String, + val anchor: String, + val kind: String, + val modifiers: List = emptyList(), + val typeParameters: List = emptyList(), + val returnType: JdTypeRef? = null, + val parameters: List = emptyList(), + val exceptions: List = emptyList(), + val signature: String, + val url: String? = null, + val description: String? = null, + val firstSentence: String? = null, + val returns: String? = null, + val specifiedBy: List = emptyList(), + val overrides: JdMemberRef? = null, + val since: List = emptyList(), + val seeAlso: List = emptyList(), + val deprecated: JdDeprecation? = null, + val annotations: List = emptyList(), + val defaultValue: String? = null, + val tags: List = emptyList() +) + +/** A nested type as listed in an enclosing type's "Nested Class Summary". */ +@Serializable +data class JdNestedTypeRef( + val name: String, + val qualifiedName: String, + val kind: String, + val modifiers: List = emptyList(), + val url: String? = null, + val firstSentence: String? = null, + val deprecated: JdDeprecation? = null +) + +// --- Pages --- + +/** One `.json` page -- the javadoc class/interface/enum/record/annotation page. */ +@Serializable +data class JdClassPage( + val page: String = "class", + val kind: String, + val name: String, + val simpleName: String, + val qualifiedName: String, + val packageName: String, + val moduleName: String? = null, + /** Link to this type's module page, relative to this page. Null for a non-modular run. */ + val moduleUrl: String? = null, + /** Link to this type's package page, relative to this page. */ + val packageUrl: String? = null, + /** + * This page's own path, relative to the output root. Note the asymmetry with every *link* + * URL in these DTOs, which is relative to the page it appears on, the way javadoc links are. + */ + val url: String, + val modifiers: List = emptyList(), + val signature: String, + val typeParameters: List = emptyList(), + val superclass: JdTypeRef? = null, + val superinterfaces: List = emptyList(), + /** Superclass chain from `java.lang.Object` down to (and including) this type. */ + val inheritance: List = emptyList(), + val allImplementedInterfaces: List = emptyList(), + val allSuperinterfaces: List = emptyList(), + val directKnownSubclasses: List = emptyList(), + val allKnownSubinterfaces: List = emptyList(), + val allKnownImplementingClasses: List = emptyList(), + val enclosingType: JdTypeRef? = null, + val isFunctionalInterface: Boolean = false, + val description: String? = null, + val firstSentence: String? = null, + val since: List = emptyList(), + val seeAlso: List = emptyList(), + val authors: List = emptyList(), + val versions: List = emptyList(), + val deprecated: JdDeprecation? = null, + val annotations: List = emptyList(), + val tags: List = emptyList(), + val nestedTypes: List = emptyList(), + val recordComponents: List = emptyList(), + val enumConstants: List = emptyList(), + val fields: List = emptyList(), + val constructors: List = emptyList(), + val methods: List = emptyList(), + /** + * The elements of an annotation type. + * + * javadoc splits these into "Required" and "Optional" tables by whether the element declares + * a `default`. Dokka's model does not carry annotation-element default values, so making that + * split here would mean labelling every element "required" whether it is or not; instead they + * are reported as one list and each element's [JdExecutable.defaultValue] is populated when + * (and only when) Dokka does supply it. + */ + val annotationElements: List = emptyList(), + val inheritedNestedTypes: List = emptyList(), + val inheritedFields: List = emptyList(), + val inheritedMethods: List = emptyList() +) + +/** One entry in a package page's type table, or in `allclasses-index.json`. */ +@Serializable +data class JdTypeSummary( + val name: String, + val qualifiedName: String, + val kind: String, + val packageName: String, + val moduleName: String? = null, + val url: String? = null, + val firstSentence: String? = null, + val deprecated: JdDeprecation? = null, + /** The type's own modifiers, so a consumer can index only the public API as javadoc does. */ + val modifiers: List = emptyList() +) + +/** One `package-summary.json` page. */ +@Serializable +data class JdPackagePage( + val page: String = "package", + val name: String, + val moduleName: String? = null, + /** Link to this package's module page, relative to this page. Null for a non-modular run. */ + val moduleUrl: String? = null, + /** This page's own path, relative to the output root -- see [JdClassPage.url]. */ + val url: String, + val description: String? = null, + val firstSentence: String? = null, + val since: List = emptyList(), + val seeAlso: List = emptyList(), + val deprecated: JdDeprecation? = null, + val tags: List = emptyList(), + /** The parent, child and sibling packages javadoc lists under "Related Packages". */ + val relatedPackages: List = emptyList(), + val interfaces: List = emptyList(), + val classes: List = emptyList(), + val enums: List = emptyList(), + val records: List = emptyList(), + val exceptions: List = emptyList(), + val annotationTypes: List = emptyList(), + /** Every type in the package, in one list, regardless of which table above it also appears in. */ + val allTypes: List = emptyList() +) + +/** One entry in a module page's package table, or in `allpackages-index.json`. */ +@Serializable +data class JdPackageSummary( + val name: String, + val moduleName: String? = null, + val url: String? = null, + val firstSentence: String? = null, + val deprecated: JdDeprecation? = null +) + +/** One `requires` directive on a module page. */ +@Serializable +data class JdModuleRequires( + val module: String, + val isTransitive: Boolean = false, + val isStatic: Boolean = false, + val url: String? = null +) + +/** + * One `exports` or `opens` directive. [to] is empty for an unqualified directive; javadoc shows a + * populated [to] in its "Exported To" / "Opened To" column, and does not document those packages. + */ +@Serializable +data class JdModuleExport( + val packageName: String, + val to: List = emptyList(), + val url: String? = null, + /** The exported package's summary sentence, which javadoc shows in this table's last column. */ + val firstSentence: String? = null +) + +/** + * One row of a module page's "Indirect Exports" table: a module readable through this one, and the + * packages it exports. + */ +@Serializable +data class JdIndirectExport( + val module: String, + val moduleUrl: String? = null, + val packages: List = emptyList() +) + +/** One `provides ... with ...` directive. */ +@Serializable +data class JdModuleProvides( + val service: JdTypeRef, + val implementations: List = emptyList() +) + +/** + * One `module-summary.json` page. + * + * The JPMS sections -- [requires], [exports], [opens], [uses], [provides] -- are read from the + * module's `module-info.java`, which Dokka's own model does not carry (a Dokka "module" is a + * build-level grouping, not a JPMS module). They are populated whenever the run's source roots + * are JPMS module roots, and are empty otherwise. + */ +@Serializable +data class JdModulePage( + val page: String = "module", + val name: String, + /** This page's own path, relative to the output root -- see [JdClassPage.url]. */ + val url: String, + val description: String? = null, + val firstSentence: String? = null, + val since: List = emptyList(), + val seeAlso: List = emptyList(), + val deprecated: JdDeprecation? = null, + val tags: List = emptyList(), + /** The module's documented packages -- those it exports unqualified. */ + val packages: List = emptyList(), + val requires: List = emptyList(), + /** + * Modules a consumer of this one also reads, reached through `requires transitive` but not + * required directly -- javadoc's "Indirect Requires" table. + */ + val indirectRequires: List = emptyList(), + val exports: List = emptyList(), + /** + * Packages that become part of this module's API surface because it re-exports the modules + * providing them -- javadoc's "Indirect Exports" table. + */ + val indirectExports: List = emptyList(), + val opens: List = emptyList(), + val uses: List = emptyList(), + val provides: List = emptyList() +) + +/** `index.json` -- javadoc's overview page. */ +@Serializable +data class JdOverviewPage( + val page: String = "overview", + val title: String? = null, + val modules: List = emptyList(), + val packages: List = emptyList() +) + +@Serializable +data class JdModuleSummary( + val name: String, + val url: String? = null, + val firstSentence: String? = null +) + +/** `allclasses-index.json`. */ +@Serializable +data class JdAllClassesIndex( + val page: String = "all-classes", + val types: List = emptyList() +) + +/** `allpackages-index.json`. */ +@Serializable +data class JdAllPackagesIndex( + val page: String = "all-packages", + val packages: List = emptyList() +) + +/** One row of `deprecated-list.json`, grouped under the javadoc section it belongs to. */ +@Serializable +data class JdDeprecatedEntry( + val element: String, + val kind: String, + val url: String? = null, + val comment: String? = null, + val forRemoval: Boolean = false, + val since: String? = null +) + +/** `deprecated-list.json`, keyed by javadoc's section names (`classes`, `methods`, ...). */ +@Serializable +data class JdDeprecatedList( + val page: String = "deprecated-list", + val sections: Map> = emptyMap() +) + +@Serializable +data class JdConstantField( + val name: String, + val modifiers: List = emptyList(), + val type: JdTypeRef, + val value: String, + val url: String? = null +) + +@Serializable +data class JdConstantsForType( + val qualifiedName: String, + val url: String? = null, + val fields: List = emptyList() +) + +/** `constant-values.json`, grouped by package then by declaring type, as javadoc groups it. */ +@Serializable +data class JdConstantValues( + val page: String = "constant-values", + val packages: Map> = emptyMap() +) + +/** One entry of the A-Z index that javadoc splits across `index-files/index-N.html`. */ +@Serializable +data class JdIndexEntry( + val label: String, + val kind: String, + val url: String? = null, + val containingElement: String? = null, + val firstSentence: String? = null, + val deprecated: Boolean = false +) + +/** One `index-files/index-N.json` page. */ +@Serializable +data class JdIndexPage( + val page: String = "index", + val letter: String, + val index: Int, + val letters: List = emptyList(), + val entries: List = emptyList() +) diff --git a/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/javadoc/JavadocExtras.kt b/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/javadoc/JavadocExtras.kt new file mode 100644 index 00000000..e94e05b9 --- /dev/null +++ b/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/javadoc/JavadocExtras.kt @@ -0,0 +1,15 @@ +package org.appdevforall.dokka.kdoc2json.javadoc + +import org.jetbrains.dokka.model.Documentable +import org.jetbrains.dokka.model.properties.PropertyContainer +import org.jetbrains.dokka.model.properties.WithExtraProperties + +/** + * Reads a documentable's extras without caring which concrete subtype it is. + * + * Dokka declares `extra` on [WithExtraProperties] rather than on [Documentable], so every caller + * would otherwise need its own cast; the star projection is safe here because extras are only ever + * read, never added. + */ +internal fun Documentable.extrasOrEmpty(): PropertyContainer<*> = + (this as? WithExtraProperties<*>)?.extra ?: PropertyContainer.empty() diff --git a/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/javadoc/JavadocMapper.kt b/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/javadoc/JavadocMapper.kt new file mode 100644 index 00000000..770ff6bd --- /dev/null +++ b/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/javadoc/JavadocMapper.kt @@ -0,0 +1,1262 @@ +package org.appdevforall.dokka.kdoc2json.javadoc + +import org.appdevforall.dokka.kdoc2json.PluginLogger +import org.jetbrains.dokka.links.DRI +import org.jetbrains.dokka.model.* + +/** + * Builds javadoc-shaped page DTOs from Dokka's model. + * + * Every method that produces a page takes the output-relative path of the file being written, + * because javadoc links relatively (`../lang/Object.json`) and a link is therefore only meaningful + * with respect to the page it appears on. [PageScope] binds that path once and carries it through + * type references, member links and rendered doc comments. + */ +class JavadocMapper( + private val index: JavadocModelIndex, + private val logger: PluginLogger +) { + + companion object { + /** Java modifier order, as javadoc prints it; anything unrecognized is appended after. */ + private val MODIFIER_ORDER = listOf( + "public", "protected", "private", "abstract", "default", "static", "final", + "sealed", "non-sealed", "transient", "volatile", "synchronized", "native", "strictfp" + ) + + /** Dokka spells a "no modifier" visibility/modifier as an empty or Kotlin-only name. */ + private val NON_JAVA_MODIFIERS = setOf("", "open", "empty", "final_kotlin") + + private const val OBJECT_SIMPLE_NAME = "Object" + + /** A javadoc inline tag: `{@code x}`, `{@link a.B#c label}`, `{@docRoot}`. */ + private val INLINE_TAG = Regex("""\{@(\w+)\s*([^}]*)\}""") + + /** + * Stand-in for `{@inheritDoc}`, substituted into the sources before analysis. + * + * Dokka's own `{@inheritDoc}` resolver recurses without bound on parts of the JDK + * (`InheritDocTagResolver.resolveThrowsTag` -> `toInheritDocHtml`) and takes the whole run + * down with a StackOverflowError. Rewriting the tag to an inert text marker before Dokka + * sees it sidesteps that, and this mapper resolves the marker itself -- walking the same + * supertype chain "Overrides:" and "Specified by:" are derived from, which is what javadoc + * does. See scripts/java/stage_jdk_sources.py. + */ + const val INHERIT_DOC_MARKER = "ADFAINHERITDOC" + + /** Depth cap for chained `{@inheritDoc}`, in case a hierarchy is cyclic after merging. */ + private const val MAX_INHERIT_DEPTH = 16 + + /** Above this many related packages javadoc drops the siblings -- see [relatedPackages]. */ + private const val MAX_RELATED_PACKAGES = 5 + + /** The stand-in occupying a paragraph of its own, the usual way `{@inheritDoc}` is written. */ + private val MARKER_PARAGRAPH = Regex("""

\s*$INHERIT_DOC_MARKER\s*

""") + } + + // Anchors of the members each type declares itself, used to derive Overrides/Specified by. + // Keyed by type key; built on demand because most runs only touch part of the graph. + private val declaredMemberAnchors = mutableMapOf>() + + /** A single output file, and everything that has to be resolved relative to it. */ + inner class PageScope(private val fromFile: String) { + + val docs = JavadocDocs(resolveLink = { dri -> linkFor(dri) }, docRoot = pathToRoot()) + + /** + * A renderer for a comment that *belongs* to the page at [declaringFile] but is being + * shown on this one -- a summary sentence on an index page. Hand-written relative links + * inside it are rebased from that page's directory to this one's. + */ + fun docsFrom(declaringFile: String): JavadocDocs { + if (declaringFile == fromFile) return docs + return JavadocDocs( + resolveLink = { dri -> linkFor(dri) }, + docRoot = pathToRoot(), + rebaseRelativeHref = { href -> rebase(declaringFile, href) } + ) + } + + /** Re-expresses [href], written relative to [declaringFile], relative to this page. */ + private fun rebase(declaringFile: String, href: String): String { + val anchorAt = href.indexOf('#') + val path = if (anchorAt < 0) href else href.substring(0, anchorAt) + val anchor = if (anchorAt < 0) "" else href.substring(anchorAt) + if (path.isEmpty()) return href + val declaringDir = declaringFile.substringBeforeLast('/', "") + val absolute = normalize(if (declaringDir.isEmpty()) path else "$declaringDir/$path") + return index.paths.relativeUrl(fromFile, absolute) + anchor + } + + /** Collapses `.` and `..` segments so the result can be compared against page paths. */ + private fun normalize(path: String): String { + val parts = mutableListOf() + path.split('/').forEach { segment -> + when (segment) { + "", "." -> Unit + ".." -> if (parts.isNotEmpty()) parts.removeAt(parts.size - 1) + else -> parts += segment + } + } + return parts.joinToString("/") + } + + /** Output-relative path [targetFile], expressed relative to this page. */ + fun url(targetFile: String, anchor: String? = null): String { + val relative = index.paths.relativeUrl(fromFile, targetFile) + return if (anchor.isNullOrBlank()) relative else "$relative#$anchor" + } + + /** A link to whatever [dri] points at, or null when this run doesn't document it. */ + fun linkFor(dri: DRI): String? { + val ownerKey = JavadocModelIndex.keyOf(dri) + val type = index.typeForKey(ownerKey) ?: return null + val callable = dri.callable ?: return url(type.filePath) + val isConstructor = isConstructorCallableName(callable.name, type.simpleName) + return url(type.filePath, index.paths.memberAnchor(dri, isConstructor)) + } + + /** A reference to a documented type by key, or a name-only reference when undocumented. */ + fun typeRefForKey(key: String, display: String = key.substringAfterLast('.')): JdTypeRef { + val type = index.typeForKey(key) + return JdTypeRef( + display = if (type != null) type.classNames else display, + qualifiedName = key, + url = type?.let { url(it.filePath) }, + kind = type?.kind + ) + } + + /** A reference to a type *use*, keeping its type arguments and array dimensions. */ + fun typeRef(bound: Bound): JdTypeRef { + val dri = boundDri(bound) + val type = dri?.let { index.typeForKey(JavadocModelIndex.keyOf(it)) } + return JdTypeRef( + display = renderBound(bound), + qualifiedName = dri?.let { JavadocModelIndex.keyOf(it) }, + url = type?.let { url(it.filePath) }, + kind = type?.kind + ) + } + + /** + * Resolves a javadoc reference written as text -- `java.sql.Driver`, `Connection#close()` + * -- to a URL, or null when this run doesn't document it. Used for the inline tags in + * comment text the plugin parsed itself. + */ + fun linkForReference(reference: String): String? { + val typePart = reference.substringBefore('#').trim().trimEnd('.') + val memberPart = reference.substringAfter('#', "").trim() + val type = index.typeForKey(typePart) + // A bare `#member` reference, or a simple name, can't be resolved without a + // context type; only fully qualified references are linked. + ?: return null + if (memberPart.isEmpty()) return url(type.filePath) + // The text form carries the *declared* parameter types, which are not necessarily the + // erased ones the anchor uses, so only the no-arg form is linked precisely. + return url(type.filePath, memberPart) + } + + /** A relative path from this page back to the output root, for `{@docRoot}`. */ + fun pathToRoot(): String { + val depth = fromFile.count { it == '/' } + return if (depth == 0) "." else List(depth) { ".." }.joinToString("/") + } + + fun seeRefs(bundle: JavadocDocBundle): List = bundle.seeAlso.map { (name, address, text) -> + JdSeeRef( + // Dokka puts the referenced symbol in the tag's name and any trailing label in + // its body; javadoc shows the label when there is one. + label = text.ifBlank { name }, + url = address?.let { linkFor(it) }, + qualifiedName = address?.let { JavadocModelIndex.keyOf(it) } + ) + } + + fun throwsList(bundle: JavadocDocBundle): List = bundle.throws.map { (name, address, text) -> + JdThrows( + type = JdTypeRef( + display = name.substringAfterLast('.'), + qualifiedName = address?.let { JavadocModelIndex.keyOf(it) } ?: name, + url = address?.let { linkFor(it) }, + kind = address?.let { index.typeForKey(JavadocModelIndex.keyOf(it))?.kind } + ), + description = text.ifBlank { null } + ) + } + } + + fun scope(fromFile: String) = PageScope(fromFile) + + // ------------------------------------------------------------------ pages + + fun classPage(type: JdType): JdClassPage { + val scope = PageScope(type.filePath) + val doc = type.documentable + val bundle = scope.docs.bundleFor(doc) + + val generics = (doc as? WithGenerics)?.generics.orEmpty() + val superclassKey = index.superclassOf(type.key) + val declaredInterfaceKeys = index.directInterfacesOf(type.key) + + // Supertype *uses* keep their type arguments (`AbstractList`), which the key-only + // hierarchy maps can't carry, so they are read back off the documentable here. + val supertypeUses = (doc as? WithSupertypes)?.supertypes?.values?.flatten() + ?.distinctBy { JavadocModelIndex.keyOf(it.typeConstructor.dri) } + .orEmpty() + .associate { JavadocModelIndex.keyOf(it.typeConstructor.dri) to it.typeConstructor } + + val superclassRef = superclassKey?.let { key -> + supertypeUses[key]?.let { scope.typeRef(it) } ?: scope.typeRefForKey(key) + } + val superinterfaceRefs = declaredInterfaceKeys.map { key -> + supertypeUses[key]?.let { scope.typeRef(it) } ?: scope.typeRefForKey(key) + } + + val members = membersOf(type, scope) + val modifiers = modifiersOf(doc) + + val nestedTypes = doc.classlikes + .mapNotNull { index.typeFor(it.dri) } + .sortedBy { it.simpleName } + .map { nested -> + // Rendered in *this* page's scope, not the nested type's own, so the links inside + // the summary resolve relative to the page the summary appears on. + val nestedBundle = scope.docs.bundleFor(nested.documentable) + JdNestedTypeRef( + name = nested.classNames, + qualifiedName = nested.qualifiedName, + kind = nested.kind, + modifiers = modifiersOf(nested.documentable), + url = scope.url(nested.filePath), + firstSentence = JavadocDocs.firstSentence(nestedBundle.description), + deprecated = deprecationOf(nested.documentable, nestedBundle) + ) + } + + val isInterfaceLike = type.kind == "interface" || type.kind == "annotation" + val inheritanceRefs = + if (isInterfaceLike) emptyList() + else index.inheritanceChain(type.key).map { scope.typeRefForKey(it) } + + return JdClassPage( + kind = if (index.isException(type.key)) "exception" else type.kind, + name = type.classNames, + simpleName = type.simpleName, + qualifiedName = type.qualifiedName, + packageName = type.packageName, + moduleName = type.moduleName, + moduleUrl = moduleUrlFor(type.moduleName, scope), + packageUrl = index.packages.firstOrNull { it.name == type.packageName } + ?.let { scope.url(it.filePath) }, + url = type.filePath, + modifiers = modifiers, + signature = classSignature(type, modifiers, generics, superclassRef, superinterfaceRefs, scope), + typeParameters = typeParameters(generics, bundle, scope), + superclass = superclassRef, + superinterfaces = superinterfaceRefs, + inheritance = inheritanceRefs, + allImplementedInterfaces = + if (isInterfaceLike) emptyList() + else index.allSuperinterfaces(type.key).map { scope.typeRefForKey(it) }, + allSuperinterfaces = + if (isInterfaceLike) index.allSuperinterfaces(type.key).map { scope.typeRefForKey(it) } + else emptyList(), + directKnownSubclasses = index.directKnownSubclasses(type.key).map { scope.typeRefForKey(it) }, + allKnownSubinterfaces = index.allKnownSubinterfaces(type.key).map { scope.typeRefForKey(it) }, + allKnownImplementingClasses = index.allKnownImplementingClasses(type.key).map { scope.typeRefForKey(it) }, + enclosingType = index.enclosingTypeOf(type)?.let { scope.typeRefForKey(it.key) }, + isFunctionalInterface = isFunctionalInterface(type, members.methods), + description = bundle.description, + firstSentence = JavadocDocs.firstSentence(bundle.description), + since = bundle.since, + seeAlso = scope.seeRefs(bundle), + authors = bundle.authors, + versions = bundle.versions, + deprecated = deprecationOf(doc, bundle), + annotations = annotationNamesOf(doc), + tags = bundle.other, + nestedTypes = nestedTypes, + enumConstants = members.enumConstants, + fields = members.fields, + constructors = members.constructors, + methods = members.methods, + annotationElements = members.annotationElements, + inheritedFields = members.inheritedFields, + inheritedMethods = members.inheritedMethods, + inheritedNestedTypes = inheritedNestedTypes(type, scope) + ) + } + + fun packagePage(pkg: JdPackage, typesInPackage: List): JdPackagePage { + val scope = PageScope(pkg.filePath) + val bundle = pkg.documentables + .map { scope.docs.bundleFor(it) } + .firstOrNull { it.description != null || it.other.isNotEmpty() } + ?: JavadocDocBundle() + + val summaries = typesInPackage.sortedBy { it.classNames }.map { typeSummary(it, scope) } + fun of(vararg kinds: String) = summaries.filter { it.kind in kinds } + + return JdPackagePage( + name = pkg.name, + moduleName = pkg.moduleName, + moduleUrl = moduleUrlFor(pkg.moduleName, scope), + url = pkg.filePath, + description = bundle.description, + firstSentence = JavadocDocs.firstSentence(bundle.description), + since = bundle.since, + seeAlso = scope.seeRefs(bundle), + deprecated = pkg.documentables.firstNotNullOfOrNull { deprecationOf(it, bundle) }, + tags = bundle.other, + relatedPackages = relatedPackages(pkg).map { packageSummary(it, scope) }, + interfaces = of("interface"), + classes = of("class", "object"), + enums = of("enum"), + records = of("record"), + exceptions = of("exception"), + annotationTypes = of("annotation"), + allTypes = summaries + ) + } + + fun modulePage(module: JdModule, packagesInModule: List): JdModulePage { + val scope = PageScope(module.filePath) + val bundle = module.documentables + .map { scope.docs.bundleFor(it) } + .firstOrNull { it.description != null || it.other.isNotEmpty() } + ?: JavadocDocBundle() + + val jpms = module.jpms + // A JPMS module's documentation lives in module-info.java, which Dokka does not read, so + // it is preferred over whatever the Dokka module happens to carry (usually nothing). + val description = jpms?.description?.let { renderJavadocText(it, scope) } ?: bundle.description + val documentedPackages = packagesInModule.map { packageSummary(it, scope) } + val documentedByName = documentedPackages.associateBy { it.name } + + return JdModulePage( + name = module.name, + url = module.filePath, + description = description, + firstSentence = JavadocDocs.firstSentence(description), + since = jpms?.since?.takeIf { it.isNotEmpty() } ?: bundle.since, + seeAlso = scope.seeRefs(bundle), + deprecated = module.documentables.firstNotNullOfOrNull { deprecationOf(it, bundle) }, + tags = jpms?.tags?.takeIf { it.isNotEmpty() } ?: bundle.other, + packages = documentedPackages, + requires = jpms?.requires.orEmpty().map { requires -> + JdModuleRequires( + module = requires.module, + isTransitive = requires.isTransitive, + isStatic = requires.isStatic, + url = index.modules.firstOrNull { it.name == requires.module } + ?.let { scope.url(it.filePath) } + ) + }, + exports = jpms?.exports.orEmpty().map { export -> + val documented = documentedByName[export.packageName] + JdModuleExport( + packageName = export.packageName, + to = export.to, + // A qualified export is not documented, so it has no page to link to. + url = documented?.url, + firstSentence = documented?.firstSentence + ) + }, + opens = jpms?.opens.orEmpty().map { opens -> + val documented = documentedByName[opens.packageName] + JdModuleExport( + packageName = opens.packageName, + to = opens.to, + url = documented?.url, + firstSentence = documented?.firstSentence + ) + }, + indirectRequires = indirectlyReadable(module).map { name -> + JdModuleRequires(module = name, isTransitive = true, url = moduleUrlFor(name, scope)) + }, + indirectExports = readableThrough(module) + .mapNotNull { name -> index.modules.firstOrNull { it.name == name } } + .filter { it.jpms?.exportedPackages?.isNotEmpty() == true } + .sortedBy { it.name } + .map { readable -> + JdIndirectExport( + module = readable.name, + moduleUrl = scope.url(readable.filePath), + packages = readable.jpms?.exportedPackages.orEmpty().sorted().map { packageName -> + JdPackageSummary( + name = packageName, + moduleName = readable.name, + url = index.packages.firstOrNull { it.name == packageName } + ?.let { scope.url(it.filePath) } + ) + } + ) + }, + uses = jpms?.uses.orEmpty().map { scope.typeRefForKey(it) }, + provides = jpms?.provides.orEmpty().map { provides -> + JdModuleProvides( + service = scope.typeRefForKey(provides.service), + implementations = provides.implementations.map { scope.typeRefForKey(it) } + ) + } + ) + } + + /** + * Renders raw javadoc comment text -- text this plugin read itself rather than getting from + * Dokka, i.e. `module-info.java`'s doc comment. + * + * Only the inline tags are handled: block-level HTML in a javadoc comment is already HTML and + * passes through untouched. An `{@link}` whose target this run does not document degrades to + * `` rather than becoming a dead link, matching what [JavadocDocs] does for the doc + * trees Dokka hands over. + */ + private fun renderJavadocText(raw: String, scope: PageScope): String { + var result = INLINE_TAG.replace(raw) { match -> + val tag = match.groupValues[1] + val body = match.groupValues[2].trim() + when (tag) { + "code", "literal" -> { + val escaped = body.replace("&", "&").replace("<", "<").replace(">", ">") + if (tag == "code") "$escaped" else escaped + } + "link", "linkplain" -> { + val target = body.substringBefore(' ').trim() + val label = body.substringAfter(' ', "").trim().ifBlank { target.substringAfterLast('.') } + val href = scope.linkForReference(target) + val text = if (tag == "link") "$label" else label + if (href == null) text else "$text" + } + // {@docRoot} is a path back to the documentation root, which is exactly what a + // relative link from this page to the root looks like. + "docRoot" -> scope.pathToRoot() + else -> body + } + } + // Collapse the blank lines a stripped block-tag section can leave behind. + result = result.trim() + return result + } + + // ------------------------------------------------------------- summaries + + fun typeSummary(type: JdType, scope: PageScope): JdTypeSummary { + val bundle = scope.docsFrom(type.filePath).bundleFor(type.documentable) + return JdTypeSummary( + name = type.classNames, + qualifiedName = type.qualifiedName, + kind = if (index.isException(type.key)) "exception" else type.kind, + packageName = type.packageName, + moduleName = type.moduleName, + url = scope.url(type.filePath), + firstSentence = JavadocDocs.firstSentence(bundle.description), + deprecated = deprecationOf(type.documentable, bundle), + modifiers = modifiersOf(type.documentable) + ) + } + + fun packageSummary(pkg: JdPackage, scope: PageScope): JdPackageSummary { + val bundle = pkg.documentables + .map { scope.docs.bundleFor(it) } + .firstOrNull { it.description != null } + ?: JavadocDocBundle() + return JdPackageSummary( + name = pkg.name, + moduleName = pkg.moduleName, + url = scope.url(pkg.filePath), + firstSentence = JavadocDocs.firstSentence(bundle.description), + deprecated = pkg.documentables.firstNotNullOfOrNull { deprecationOf(it, bundle) } + ) + } + + fun moduleSummary(module: JdModule, scope: PageScope): JdModuleSummary { + val bundle = module.documentables + .map { scope.docs.bundleFor(it) } + .firstOrNull { it.description != null } + ?: JavadocDocBundle() + // As on the module page itself, module-info.java's own comment is the real source of a + // module's description -- Dokka's module carries none -- so the overview's Description + // column is empty without this. + val description = module.jpms?.description?.let { renderJavadocText(it, scope) } + ?: bundle.description + return JdModuleSummary( + name = module.name, + url = scope.url(module.filePath), + firstSentence = JavadocDocs.firstSentence(description) + ) + } + + // --------------------------------------------------------------- members + + /** Everything a class page lists, split the way javadoc splits it into tables. */ + class ClassMembers( + val enumConstants: List = emptyList(), + val fields: List = emptyList(), + val constructors: List = emptyList(), + val methods: List = emptyList(), + val annotationElements: List = emptyList(), + val inheritedFields: List = emptyList(), + val inheritedMethods: List = emptyList() + ) + + /** + * The members of a type, sorted into javadoc's four buckets (declared/inherited x field/method). + * + * Dokka merges a private Java field and its accessor pair into a single `DProperty` with a + * non-null `getter`, the way a Kotlin property looks. javadoc shows the opposite: the private + * field is not documented at all and `getWidth()` is a *method*. So a property that carries + * accessors is unfolded back into its accessor methods, and only an accessor-less property -- + * a genuine Java field -- is reported as a field. + */ + private class SplitMembers( + val declaredFields: List = emptyList(), + val inheritedFields: List> = emptyList(), + val declaredMethods: List = emptyList(), + val inheritedMethods: List> = emptyList() + ) + + private val splitMembersCache = mutableMapOf() + + private fun splitMembers(type: JdType): SplitMembers = splitMembersCache.getOrPut(type.key) { + val doc = type.documentable + + val declaredFields = mutableListOf() + val inheritedFields = mutableListOf>() + val declaredMethods = mutableListOf() + val inheritedMethods = mutableListOf>() + + doc.properties.forEach { property -> + val from = inheritedFromKey(property, type.key) + val accessors = listOfNotNull(property.getter, property.setter) + if (accessors.isNotEmpty()) { + accessors.forEach { accessor -> + if (from == null) declaredMethods += accessor else inheritedMethods += from to accessor + } + } else { + if (from == null) declaredFields += property else inheritedFields += from to property + } + } + + doc.functions.forEach { function -> + val from = inheritedFromKey(function, type.key) + if (from == null) declaredMethods += function else inheritedMethods += from to function + } + + SplitMembers(declaredFields, inheritedFields, declaredMethods, inheritedMethods) + } + + fun membersOf(type: JdType, scope: PageScope): ClassMembers { + val doc = type.documentable + val split = splitMembers(type) + + val constructors = (doc as? WithConstructors)?.constructors.orEmpty() + val enumEntries = (doc as? DEnum)?.entries.orEmpty() + + val isAnnotation = doc is DAnnotation + val executables = split.declaredMethods.map { executable(it, type, scope, isConstructor = false) } + + return ClassMembers( + enumConstants = enumEntries.map { enumConstant(it, type, scope) }, + fields = split.declaredFields.map { field(it, type, scope) }.sortedBy { it.name }, + constructors = constructors.map { executable(it, type, scope, isConstructor = true) }, + methods = if (isAnnotation) emptyList() else executables.sortedBy { it.anchor }, + annotationElements = if (!isAnnotation) emptyList() else executables.sortedBy { it.name }, + inheritedFields = groupInherited(split.inheritedFields, scope) { property, owner -> + memberRef(property.name, property.dri, owner, scope, isConstructor = false) + }, + inheritedMethods = groupInherited(split.inheritedMethods, scope) { function, owner -> + memberRef(function.name, function.dri, owner, scope, isConstructor = false) + } + ) + } + + private fun groupInherited( + members: List>, + scope: PageScope, + toRef: (T, String) -> JdMemberRef + ): List = + members.groupBy({ it.first }, { it.second }) + .toSortedMap() + .map { (ownerKey, owned) -> + JdInheritedMembers( + declaringType = scope.typeRefForKey(ownerKey), + members = owned.map { toRef(it, ownerKey) }.sortedBy { it.signature } + ) + } + + private fun memberRef( + name: String?, + dri: DRI, + ownerKey: String, + scope: PageScope, + isConstructor: Boolean + ): JdMemberRef { + val anchor = index.paths.memberAnchor(dri, isConstructor) + val owner = index.typeForKey(ownerKey) + return JdMemberRef( + name = name.orEmpty(), + signature = anchor, + url = owner?.let { scope.url(it.filePath, anchor) }, + declaringType = scope.typeRefForKey(ownerKey) + ) + } + + private fun field(property: DProperty, owner: JdType, scope: PageScope): JdField { + val bundle = scope.docs.bundleFor(property) + val modifiers = modifiersOf(property) + val typeRef = scope.typeRef(property.type) + val anchor = index.paths.memberAnchor(property.dri, isConstructor = false) + return JdField( + name = property.name, + anchor = anchor, + modifiers = modifiers, + type = typeRef, + signature = (modifiers + typeRef.display + property.name).joinToString(" "), + url = scope.url(owner.filePath, anchor), + description = bundle.description, + firstSentence = JavadocDocs.firstSentence(bundle.description), + constantValue = defaultValueOf(property), + since = bundle.since, + seeAlso = scope.seeRefs(bundle), + deprecated = deprecationOf(property, bundle), + annotations = annotationNamesOf(property), + tags = bundle.other + ) + } + + private fun enumConstant(entry: DEnumEntry, owner: JdType, scope: PageScope): JdField { + val bundle = scope.docs.bundleFor(entry) + val anchor = entry.name + return JdField( + name = entry.name, + anchor = anchor, + modifiers = listOf("public", "static", "final"), + type = scope.typeRefForKey(owner.key, owner.classNames), + signature = "public static final ${owner.simpleName} ${entry.name}", + url = scope.url(owner.filePath, anchor), + description = bundle.description, + firstSentence = JavadocDocs.firstSentence(bundle.description), + since = bundle.since, + seeAlso = scope.seeRefs(bundle), + deprecated = deprecationOf(entry, bundle), + annotations = annotationNamesOf(entry), + tags = bundle.other + ) + } + + /** + * The nearest ancestor declaring the same erased signature, and its declaration -- the method + * `{@inheritDoc}` inherits from. Superclasses are searched before interfaces, as javadoc does. + */ + private fun inheritedFrom(owner: JdType, anchor: String): Pair? { + val ancestors = index.superclassChain(owner.key) + index.allSuperinterfaces(owner.key) + ancestors.forEach { key -> + val type = index.typeForKey(key) ?: return@forEach + val declaration = splitMembers(type).declaredMethods.firstOrNull { + index.paths.memberAnchor(it.dri, isConstructor = false) == anchor + } + if (declaration != null) return type to declaration + } + return null + } + + /** + * Replaces [INHERIT_DOC_MARKER] in [text] with the corresponding text from the method this one + * overrides, recursing when the ancestor's own comment inherits in turn. [select] picks which + * part of the ancestor's comment to pull in, so one walk serves the description, `@return`, + * `@param` and `@throws`. + */ + private fun resolveInheritDoc( + text: String?, + owner: JdType, + anchor: String, + scope: PageScope, + depth: Int = 0, + select: (JavadocDocBundle) -> String? + ): String? { + if (text == null || !text.contains(INHERIT_DOC_MARKER)) return text + if (depth >= MAX_INHERIT_DEPTH) return clean(text.replace(INHERIT_DOC_MARKER, "")) + + val parent = inheritedFrom(owner, anchor) + val inherited = parent?.let { (parentType, declaration) -> + // Rendered in the *current* page's scope, so links in the inherited prose resolve + // relative to the page it is being shown on. + resolveInheritDoc( + select(scope.docs.bundleFor(declaration)), parentType, anchor, scope, depth + 1, select + ) + } + val block = inherited.orEmpty() + // Where the marker occupies a paragraph of its own -- `{@inheritDoc}` on its own line, + // which is how javadoc comments almost always write it -- that whole paragraph is + // replaced by the inherited block, wrapper included. Splicing inside the existing

+ // would nest paragraphs whenever the inherited prose runs to more than one. + var result = MARKER_PARAGRAPH.replace(text) { block } + // Any marker left is inline within a sentence, so the inherited fragment's own enclosing + //

comes off before it is spliced in. + if (result.contains(INHERIT_DOC_MARKER)) { + result = result.replace(INHERIT_DOC_MARKER, JavadocDocs.unwrapParagraph(block)) + } + return clean(result) + } + + /** + * As [resolveInheritDoc], but an *absent* value is treated as an implicit `{@inheritDoc}`. + * + * javadoc inherits a missing `@param`/`@return`/`@throws` from the overridden method even + * without the tag being written out, so a method that documents only some of its parameters + * still shows text for the rest. + */ + private fun inheritIfAbsent( + text: String?, + owner: JdType, + anchor: String, + scope: PageScope, + select: (JavadocDocBundle) -> String? + ): String? = resolveInheritDoc(text ?: INHERIT_DOC_MARKER, owner, anchor, scope, select = select) + + /** + * The summary sentence for a declaration as it should read on [scope]'s page. + * + * Global index pages re-render summaries against their own location; passing [owner] and + * [anchor] for a member runs the same `{@inheritDoc}` resolution there as on the member's own + * page, instead of leaking an unresolved marker into the index. + */ + fun summaryFor( + doc: Documentable, + scope: PageScope, + owner: JdType? = null, + anchor: String? = null + ): String? { + val raw = scope.docsFrom(owner?.filePath ?: "").bundleFor(doc).description + val resolved = + if (owner != null && anchor != null) { + resolveInheritDoc(raw ?: INHERIT_DOC_MARKER, owner, anchor, scope) { it.description } + } else { + raw + } + return JavadocDocs.firstSentence(resolved) + } + + /** + * Link to a module's page, relative to [scope]'s page. + * + * Where the module page sits depends on whether the run uses module directories, so the link + * is resolved from the index rather than assembled from the module name in a template. + */ + private fun moduleUrlFor(moduleName: String?, scope: PageScope): String? { + if (moduleName == null) return null + return index.modules.firstOrNull { it.name == moduleName }?.let { scope.url(it.filePath) } + } + + /** + * javadoc's "Nested classes/interfaces declared in class X" groups. + * + * Unlike fields and methods, Dokka does not copy a supertype's nested types down onto the + * subtype, so there is no `InheritedMember` to read: the groups are walked out of the + * hierarchy directly. A nested type the subtype redeclares under the same simple name shadows + * the inherited one and is left out, as it is in javadoc. + */ + private fun inheritedNestedTypes(type: JdType, scope: PageScope): List { + val shadowed = type.documentable.classlikes.mapNotNull { it.name }.toMutableSet() + val alreadyListed = mutableSetOf() + + return (index.superclassChain(type.key) + index.allSuperinterfaces(type.key)) + .mapNotNull { index.typeForKey(it) } + .mapNotNull { ancestor -> + val nested = ancestor.documentable.classlikes + .mapNotNull { index.typeFor(it.dri) } + .filter { it.simpleName !in shadowed && alreadyListed.add(it.qualifiedName) } + .sortedBy { it.simpleName } + if (nested.isEmpty()) { + null + } else { + JdInheritedMembers( + declaringType = scope.typeRefForKey(ancestor.key), + members = nested.map { inner -> + JdMemberRef( + name = inner.classNames, + signature = inner.classNames, + url = scope.url(inner.filePath), + declaringType = scope.typeRefForKey(ancestor.key) + ) + } + ) + } + } + } + + /** + * Every module a consumer of [module] also gets to read: the `requires transitive` closure. + * + * Only `transitive` edges carry readability onward, so a plain `requires` is not followed and + * the walk is seeded with the transitive requires alone -- seeding it with *all* direct + * requires is what makes the result disagree with javadoc. Checked against every JDK module + * page that has one of these tables. + */ + private fun readableThrough(module: JdModule): Set { + val result = LinkedHashSet() + val seed = module.jpms?.requires.orEmpty().filter { it.isTransitive }.map { it.module } + val seen = seed.toMutableSet() + val work = ArrayDeque(seed) + while (work.isNotEmpty()) { + val current = work.removeFirst() + if (current == module.name) continue + result += current + index.modules.firstOrNull { it.name == current }?.jpms?.requires.orEmpty() + .filter { it.isTransitive } + .forEach { if (seen.add(it.module)) work += it.module } + } + return result + } + + /** [readableThrough] minus what this module already requires directly. */ + private fun indirectlyReadable(module: JdModule): List { + val direct = module.jpms?.requires.orEmpty().map { it.module }.toSet() + return readableThrough(module).filterNot { it in direct || it == module.name }.sorted() + } + + /** + * The packages javadoc lists under "Related Packages": the parent, the direct children, and -- + * only when the result stays small -- the siblings. + * + * The size condition is javadoc's, not an invention: `java.nio.channels` lists its siblings + * `java.nio.charset` and `java.nio.file`, while `java.util.concurrent` and + * `java.lang.annotation` list none, because `java.util` and `java.lang` have too many + * children for the table to stay useful. A cut-off of five reproduces 181 of the 190 JDK + * package pages that have this table. + */ + private fun relatedPackages(pkg: JdPackage): List { + val name = pkg.name + val parentName = name.substringBeforeLast('.', "") + + fun childrenOf(prefix: String) = index.packages.filter { + it.name != prefix && + it.name.startsWith("$prefix.") && + !it.name.removePrefix("$prefix.").contains('.') + } + + val parent = index.packages.filter { it.name == parentName } + val children = childrenOf(name) + val siblings = if (parentName.isEmpty()) emptyList() else childrenOf(parentName).filter { it.name != name } + + val core = parent + children + val related = if (core.size + siblings.size <= MAX_RELATED_PACKAGES) core + siblings else core + return related.distinctBy { it.name }.sortedBy { it.name } + } + + private fun clean(text: String): String? = text.trim().ifBlank { null } + + private fun executable( + function: DFunction, + owner: JdType, + scope: PageScope, + isConstructor: Boolean + ): JdExecutable { + val bundle = scope.docs.bundleFor(function) + val modifiers = run { + val found = modifierSetOf(function) + // Dokka reports an interface's default methods simply as "not abstract" and never + // emits the `default` keyword. In Java an interface method that is neither abstract + // nor static is exactly a default method, so the keyword is recovered here rather + // than being lost from the signature. + if (owner.kind == "interface" && "abstract" !in found && "static" !in found) { + found += "default" + } + orderModifiers(found) + } + val anchor = index.paths.memberAnchor(function.dri, isConstructor) + val returnType = if (isConstructor) null else scope.typeRef(function.type) + + val parameters = function.parameters.map { parameter -> + JdParameter( + name = parameter.name.orEmpty(), + type = scope.typeRef(parameter.type), + description = inheritIfAbsent( + parameter.name?.let { bundle.params[it] }?.ifBlank { null }, owner, anchor, scope + ) { parent -> parent.params[parameter.name.orEmpty()] }, + annotations = annotationNamesOf(parameter) + ) + } + + val declaredThrows = scope.throwsList(bundle).map { thrown -> + thrown.copy( + description = inheritIfAbsent(thrown.description, owner, anchor, scope) { parent -> + parent.throws.firstOrNull { it.first.substringAfterLast('.') == thrown.type.display }?.third + } + ) + } + val kind = when { + isConstructor -> "constructor" + owner.kind == "annotation" -> "annotationElement" + else -> "method" + } + + val (overrides, specifiedBy) = + if (isConstructor) null to emptyList() else overrideInfo(owner, anchor, scope) + + // A method that documents only its tags -- or has no comment at all -- still shows the + // overridden method's description in javadoc, so an absent description inherits too. + val description = inheritIfAbsent(bundle.description, owner, anchor, scope) { it.description } + + return JdExecutable( + name = if (isConstructor) owner.simpleName else function.name, + anchor = anchor, + kind = kind, + modifiers = modifiers, + typeParameters = typeParameters(function.generics, bundle, scope), + returnType = returnType, + parameters = parameters, + exceptions = declaredThrows, + signature = executableSignature( + if (isConstructor) owner.simpleName else function.name, + modifiers, function.generics, returnType, parameters, declaredThrows, scope + ), + url = scope.url(owner.filePath, anchor), + description = description, + firstSentence = JavadocDocs.firstSentence(description), + returns = inheritIfAbsent(bundle.returns, owner, anchor, scope) { it.returns }, + specifiedBy = specifiedBy, + overrides = overrides, + since = bundle.since, + seeAlso = scope.seeRefs(bundle), + deprecated = deprecationOf(function, bundle), + annotations = annotationNamesOf(function), + defaultValue = defaultValueOf(function), + tags = bundle.other.map { tag -> + tag.copy( + text = resolveInheritDoc(tag.text, owner, anchor, scope) { parent -> + parent.other.firstOrNull { it.name == tag.name }?.text + }.orEmpty() + ) + } + ) + } + + /** + * javadoc's "Overrides:" (nearest superclass declaring the same erased signature) and + * "Specified by:" (every superinterface declaring it). Both are derived from the anchor, + * which already encodes name plus erased parameter types -- exactly the identity Java uses + * to decide whether one method overrides another. + */ + private fun overrideInfo( + owner: JdType, + anchor: String, + scope: PageScope + ): Pair> { + val overriddenIn = index.superclassChain(owner.key).firstOrNull { anchor in anchorsDeclaredIn(it) } + val specifiedIn = index.allSuperinterfaces(owner.key).filter { anchor in anchorsDeclaredIn(it) } + + fun refTo(ownerKey: String): JdMemberRef? { + val target = index.typeForKey(ownerKey) ?: return null + return JdMemberRef( + name = anchor.substringBefore('('), + signature = anchor, + url = scope.url(target.filePath, anchor), + declaringType = scope.typeRefForKey(ownerKey) + ) + } + + return (overriddenIn?.let { refTo(it) }) to specifiedIn.mapNotNull { refTo(it) } + } + + private fun anchorsDeclaredIn(key: String): Set = declaredMemberAnchors.getOrPut(key) { + val type = index.typeForKey(key) ?: return@getOrPut emptySet() + val split = splitMembers(type) + val anchors = mutableSetOf() + split.declaredMethods.forEach { anchors += index.paths.memberAnchor(it.dri, isConstructor = false) } + split.declaredFields.forEach { anchors += index.paths.memberAnchor(it.dri, isConstructor = false) } + anchors + } + + // ------------------------------------------------------------ signatures + + private fun classSignature( + type: JdType, + modifiers: List, + generics: List, + superclass: JdTypeRef?, + superinterfaces: List, + scope: PageScope + ): String { + val keyword = when (type.kind) { + "interface" -> "interface" + "enum" -> "enum" + "annotation" -> "@interface" + else -> "class" + } + val typeParams = if (generics.isEmpty()) "" else + generics.joinToString(",", "<", ">") { renderTypeParameterDeclaration(it, scope) } + + return buildString { + append((modifiers + keyword).joinToString(" ")) + append(' ') + append(type.classNames) + append(typeParams) + // An interface's parents are all spelled `extends`; a class extends one and + // implements the rest. + if (type.kind == "interface" || type.kind == "annotation") { + val parents = listOfNotNull(superclass) + superinterfaces + if (parents.isNotEmpty()) append(parents.joinToString(", ", " extends ") { it.display }) + } else { + superclass?.let { append(" extends ${it.display}") } + if (superinterfaces.isNotEmpty()) { + append(superinterfaces.joinToString(", ", " implements ") { it.display }) + } + } + } + } + + private fun executableSignature( + name: String, + modifiers: List, + generics: List, + returnType: JdTypeRef?, + parameters: List, + exceptions: List, + scope: PageScope + ): String = buildString { + if (modifiers.isNotEmpty()) append(modifiers.joinToString(" ")).append(' ') + if (generics.isNotEmpty()) { + append(generics.joinToString(",", "<", "> ") { renderTypeParameterDeclaration(it, scope) }) + } + returnType?.let { append(it.display).append(' ') } + append(name) + append(parameters.joinToString(", ", "(", ")") { "${it.type.display} ${it.name}" }) + if (exceptions.isNotEmpty()) { + append(exceptions.joinToString(", ", " throws ") { it.type.display }) + } + } + + private fun renderTypeParameterDeclaration(generic: DTypeParameter, scope: PageScope): String { + // `extends Object` is implicit in Java and javadoc omits it, so an Object-only bound is + // dropped rather than printed. + val bounds = generic.bounds.map { renderBound(it) }.filter { it != OBJECT_SIMPLE_NAME } + val name = generic.variantTypeParameter.let { generic.name } + return if (bounds.isEmpty()) name else "$name extends ${bounds.joinToString(" & ")}" + } + + private fun typeParameters( + generics: List, + bundle: JavadocDocBundle, + scope: PageScope + ): List = generics.map { generic -> + JdTypeParameter( + name = generic.name, + bounds = generic.bounds.map { scope.typeRef(it) }, + // Dokka keeps the angle brackets a type parameter's @param was written with, so + // `@param ...` is filed under "" rather than "U". + description = (bundle.params["<${generic.name}>"] ?: bundle.params[generic.name]) + ?.ifBlank { null } + ) + } + + // ------------------------------------------------------------- type text + + /** Renders a [Bound] the way javadoc spells a type in a signature. */ + fun renderBound(bound: Bound): String = when (bound) { + is TypeParameter -> bound.presentableName ?: bound.name + is Nullable -> renderBound(bound.inner) + is DefinitelyNonNullable -> renderBound(bound.inner) + is TypeAliased -> renderBound(bound.typeAlias) + is PrimitiveJavaType -> bound.name + is JavaObject -> OBJECT_SIMPLE_NAME + is Void -> "void" + is Dynamic -> "dynamic" + is UnresolvedBound -> bound.name + is GenericTypeConstructor -> renderConstructor(bound.dri, bound.projections, bound.presentableName) + is FunctionalTypeConstructor -> renderConstructor(bound.dri, bound.projections, bound.presentableName) + } + + private fun renderConstructor(dri: DRI, projections: List, presentableName: String?): String { + val key = JavadocModelIndex.keyOf(dri) + // Dokka models a Java array as a single-argument `kotlin.Array`. + if (key == "kotlin.Array") { + val element = projections.firstOrNull()?.let { renderProjection(it) } ?: OBJECT_SIMPLE_NAME + return "$element[]" + } + PRIMITIVE_ARRAYS[key]?.let { return it } + val name = presentableName ?: dri.classNames ?: key.substringAfterLast('.') + if (projections.isEmpty()) return name + return name + projections.joinToString(",", "<", ">") { renderProjection(it) } + } + + private fun renderProjection(projection: Projection): String = when (projection) { + is Star -> "?" + is Covariance<*> -> "? extends ${renderBound(projection.inner)}" + is Contravariance<*> -> "? super ${renderBound(projection.inner)}" + is Invariance<*> -> renderBound(projection.inner) + is Bound -> renderBound(projection) + } + + // ------------------------------------------------------------- modifiers + + /** + * The Java modifier list for a declaration, in javadoc's order. + * + * Three Dokka sources are merged: `visibility`, `modifier` (final/abstract) and the + * `AdditionalModifiers` extra (static, synchronized, transient, volatile, native, default...). + * Modifiers Dokka reports that aren't Java keywords are kept at the end rather than dropped, + * so nothing from the model is lost when the source is Kotlin. + */ + fun modifiersOf(doc: Documentable): List = orderModifiers(modifierSetOf(doc)) + + private fun modifierSetOf(doc: Documentable): MutableSet { + val found = LinkedHashSet() + + (doc as? WithVisibility)?.visibility?.values?.forEach { visibility -> + visibility.name.lowercase().takeIf { it !in NON_JAVA_MODIFIERS }?.let { found += it } + } + (doc as? WithAbstraction)?.modifier?.values?.forEach { modifier -> + modifier.name.lowercase().takeIf { it !in NON_JAVA_MODIFIERS }?.let { found += it } + } + doc.extrasOrEmpty().allOfType().forEach { additional -> + additional.content.values.flatten().forEach { found += it.name.lowercase() } + } + return found + } + + /** Puts a modifier set into javadoc's print order, keeping anything unrecognized at the end. */ + private fun orderModifiers(found: Set): List = + MODIFIER_ORDER.filter { it in found } + found.filterNot { it in MODIFIER_ORDER }.sorted() + + // ------------------------------------------------------------------ misc + + /** + * A declaration's deprecation as it should read *on [scope]'s page*. + * + * The comment is a rendered doc fragment and can contain links, so it cannot be lifted from + * one page onto another -- a global index page has to re-render it against its own location + * or the links inside it point at the wrong place. + */ + fun deprecationFor(doc: Documentable, scope: PageScope): JdDeprecation? = + deprecationOf(doc, scope.docs.bundleFor(doc)) + + private fun deprecationOf(doc: Documentable, bundle: JavadocDocBundle): JdDeprecation? { + val annotation = deprecatedAnnotation(doc) + if (!bundle.isDeprecatedTagPresent && annotation == null) return null + return JdDeprecation( + comment = bundle.deprecated, + forRemoval = annotation?.get("forRemoval")?.contains("true") == true, + since = annotation?.get("since")?.trim('"')?.ifBlank { null } + ) + } + + /** The parameters of a `@Deprecated`/`@kotlin.Deprecated` annotation, if one is present. */ + private fun deprecatedAnnotation(doc: Documentable): Map? = + doc.extrasOrEmpty().allOfType() + .flatMap { it.directAnnotations.values.flatten() } + .firstOrNull { it.dri.classNames == "Deprecated" } + ?.params + ?.mapValues { it.value.toString() } + + private fun annotationNamesOf(doc: Documentable): List = + doc.extrasOrEmpty().allOfType() + .flatMap { it.directAnnotations.values.flatten() } + .mapNotNull { it.dri.classNames } + .distinct() + .map { "@$it" } + + /** + * A constant field's value, or an annotation element's `default`. Dokka records both in the + * same `DefaultValue` extra, which is read reflectively because its accessor name has moved + * between Dokka versions (see `ModelMapper.mapExtras` for the same treatment). + */ + private fun defaultValueOf(doc: Documentable): String? { + val extra = doc.extrasOrEmpty().allOfType() + .firstOrNull { it::class.java.simpleName == "DefaultValue" } ?: return null + return try { + val accessor = extra::class.java.methods + .firstOrNull { it.name == "getValue" || it.name == "getExpression" } ?: return null + when (val value = accessor.invoke(extra)) { + null -> null + is Map<*, *> -> value.values.firstOrNull()?.let { renderExpression(it) } + else -> renderExpression(value) + } + } catch (e: Exception) { + logger.debug("javadoc-mode: could not read DefaultValue for ${doc.dri}: ${e.message}") + null + } + } + + /** + * Renders one of Dokka's `Expression` values (`IntegerConstant`, `StringConstant`, ...) as the + * literal javadoc would print. Their `toString` is the data-class form -- `IntegerConstant( + * value=4)` -- so the wrapped value is unwrapped reflectively, string constants being quoted + * the way javadoc's constant-values page quotes them. + */ + private fun renderExpression(expression: Any): String { + val unwrapped = try { + expression::class.java.methods + .firstOrNull { it.name == "getValue" && it.parameterCount == 0 } + ?.invoke(expression) + } catch (e: Exception) { + logger.debug("javadoc-mode: could not unwrap expression ${expression::class.java.simpleName}: ${e.message}") + null + } ?: return expression.toString() + + return if (expression::class.java.simpleName == "StringConstant") "\"$unwrapped\"" else unwrapped.toString() + } + + /** + * The key of the type a member was inherited from, or null when [ownerKey] declares it itself. + * Dokka attaches this as the `InheritedMember` extra when it copies members down a hierarchy. + */ + private fun inheritedFromKey(doc: Documentable, ownerKey: String): String? { + val inherited = doc.extrasOrEmpty().allOfType().firstOrNull() ?: return null + val from = inherited.inheritedFrom.values.firstOrNull { it != null } ?: return null + val fromKey = JavadocModelIndex.keyOf(from) + if (fromKey == ownerKey || fromKey.isBlank()) return null + // A member inherited from a type this run does not document is shown by javadoc as if the + // subtype declared it -- there is no page to send the reader to, so an "inherited from" + // group would be a dead end. java.util.jar.JarEntry gets its 40 CEN*/END*/LOC* constants + // this way, from the package-private java.util.zip.ZipConstants. + if (index.typeForKey(fromKey) == null) return null + return fromKey + } + + /** javadoc marks an interface with exactly one abstract method as a functional interface. */ + private fun isFunctionalInterface(type: JdType, methods: List): Boolean { + if (type.kind != "interface") return false + return methods.count { "abstract" in it.modifiers || ("default" !in it.modifiers && "static" !in it.modifiers) } == 1 + } + + /** + * The DRI a type *use* should link to. + * + * An array links to its element type, which is what javadoc does -- `BodyPublisher[]` links to + * `BodyPublisher`. Dokka models an array as `kotlin.Array`, a type nothing documents, so + * without this unwrapping every array-typed parameter and return renders as dead text. + */ + private fun boundDri(bound: Bound): DRI? = when (bound) { + is GenericTypeConstructor -> + if (JavadocModelIndex.keyOf(bound.dri) == "kotlin.Array") { + bound.projections.firstOrNull()?.let { projectionBound(it) }?.let { boundDri(it) } + } else { + bound.dri + } + is FunctionalTypeConstructor -> bound.dri + is TypeParameter -> null + is Nullable -> boundDri(bound.inner) + is DefinitelyNonNullable -> boundDri(bound.inner) + is TypeAliased -> boundDri(bound.typeAlias) + else -> null + } + + /** The bound inside a projection, or null for a star projection. */ + private fun projectionBound(projection: Projection): Bound? = when (projection) { + is Bound -> projection + is Variance<*> -> projection.inner + else -> null + } + + private fun isConstructorCallableName(callableName: String, simpleName: String): Boolean = + callableName == "" || callableName == simpleName + + private val PRIMITIVE_ARRAYS = mapOf( + "kotlin.IntArray" to "int[]", "kotlin.LongArray" to "long[]", + "kotlin.ShortArray" to "short[]", "kotlin.ByteArray" to "byte[]", + "kotlin.CharArray" to "char[]", "kotlin.BooleanArray" to "boolean[]", + "kotlin.FloatArray" to "float[]", "kotlin.DoubleArray" to "double[]" + ) +} diff --git a/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/javadoc/JavadocModelIndex.kt b/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/javadoc/JavadocModelIndex.kt new file mode 100644 index 00000000..f5430108 --- /dev/null +++ b/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/javadoc/JavadocModelIndex.kt @@ -0,0 +1,429 @@ +package org.appdevforall.dokka.kdoc2json.javadoc + +import org.appdevforall.dokka.kdoc2json.PluginLogger +import org.jetbrains.dokka.links.DRI +import org.jetbrains.dokka.model.DAnnotation +import org.jetbrains.dokka.model.DClass +import org.jetbrains.dokka.model.DClasslike +import org.jetbrains.dokka.model.DEnum +import org.jetbrains.dokka.model.DInterface +import org.jetbrains.dokka.model.DModule +import org.jetbrains.dokka.model.DObject +import org.jetbrains.dokka.model.DPackage +import org.jetbrains.dokka.model.Documentable +import org.jetbrains.dokka.model.WithSupertypes +import org.jetbrains.dokka.pages.PageNode +import org.jetbrains.dokka.model.WithSources +import org.jetbrains.dokka.pages.WithDocumentables +import java.io.File + +/** A type this documentation run covers, plus everything the renderer needs to place it. */ +class JdType( + val documentable: DClasslike, + val key: String, + val qualifiedName: String, + /** Dotted name relative to the package, e.g. `Map.Entry`. */ + val classNames: String, + val simpleName: String, + val packageName: String, + val moduleName: String?, + val kind: String, + val filePath: String +) + +class JdPackage( + val name: String, + val moduleName: String?, + val documentables: List, + val filePath: String +) + +class JdModule( + val name: String, + val documentables: List, + val filePath: String, + /** The JPMS descriptor this module was built from, when the sources are modular. */ + val jpms: JpmsModuleInfo? = null +) + +/** + * A whole-run view of the documented model. + * + * A javadoc page is not derivable from its own declaration alone: "All Implemented Interfaces", + * "Direct Known Subclasses", "All Known Implementing Classes" and the inherited-member groups are + * all *global* facts about the type graph. So Javadoc mode does one collection pass up front, + * builds the hierarchy in both directions, and then renders every page against this index. + * + * All closures are computed once in [build] rather than on demand: a per-page closure walk would + * be quadratic in the number of types, which is the difference between usable and unusable on a + * JDK-sized run. + * + * Types outside the run (e.g. `java.lang.Object` when only one package is documented) are simply + * absent, which is also what javadoc does -- it links only what it documents. + */ +class JavadocModelIndex private constructor( + val paths: JavadocPaths, + val modules: List, + val packages: List, + val types: List, + /** True when page paths carry a leading `/` segment. */ + val useModuleDirs: Boolean, + private val byKey: Map, + private val superclassByKey: Map, + private val interfacesByKey: Map>, + private val allSuperinterfacesByKey: Map>, + private val directSubclassesByKey: Map>, + private val subinterfacesByKey: Map>, + private val implementorsByKey: Map>, + private val exceptionKeys: Set +) { + + companion object { + const val OBJECT_FQN = "java.lang.Object" + private const val THROWABLE_FQN = "java.lang.Throwable" + + /** Normalized cross-reference key: a type's fully qualified dotted name. */ + fun keyOf(dri: DRI): String { + val pkg = dri.packageName.orEmpty() + val cls = dri.classNames.orEmpty() + return if (pkg.isBlank()) cls else "$pkg.$cls" + } + + fun build( + root: PageNode, + logger: PluginLogger, + sourceSetWhitelist: List, + sourceRoots: Collection = emptyList() + ): JavadocModelIndex { + val collected = collectDocumentables(root) + + val moduleDocs = collected.filterIsInstance() + val jpmsModules = JpmsModuleScanner.scan(sourceRoots, logger) + + // Module directories when the run genuinely spans several modules -- JPMS modules read + // off module-info.java if the sources are modular, Dokka modules otherwise. javadoc + // likewise flattens packages to the output root for a non-modular build. + val useModuleDirs = + if (jpmsModules.isNotEmpty()) jpmsModules.size > 1 + else moduleDocs.distinctBy { it.name }.size > 1 + val paths = JavadocPaths(useModuleDirs) + + // A package belongs to whichever module declares it, which module-info.java states + // outright. Qualified exports count too: the package is still *in* that module even + // though javadoc won't document it. + val moduleOfPackage = mutableMapOf() + jpmsModules.forEach { module -> + (module.exports + module.opens).forEach { export -> + moduleOfPackage.putIfAbsent(export.packageName, module.name) + } + } + if (jpmsModules.isEmpty()) { + moduleDocs.forEach { module -> + module.packages.forEach { pkg -> + moduleOfPackage.putIfAbsent(pkg.dri.packageName.orEmpty(), module.name) + } + } + } + + // Fallback for a modular project that documents a package its module never exports: + // the source file still sits under exactly one module's source root. + val moduleRoots = jpmsModules.map { it.sourceRoot.absolutePath.trimEnd(File.separatorChar) to it.name } + fun moduleForSourcePath(path: String?): String? { + if (path.isNullOrBlank() || moduleRoots.isEmpty()) return null + val normalized = File(path).absolutePath + return moduleRoots.firstOrNull { (rootPath, _) -> + normalized.startsWith(rootPath + File.separatorChar) + }?.second + } + + fun moduleFor(packageName: String, doc: Documentable): String? = + moduleOfPackage[packageName] + ?: moduleForSourcePath((doc as? WithSources)?.sources?.values?.firstOrNull()?.path) + + fun passesWhitelist(doc: Documentable): Boolean { + if (sourceSetWhitelist.isEmpty()) return true + return doc.sourceSets.any { it.sourceSetID.toString().substringAfterLast("/") in sourceSetWhitelist } + } + + // --- Types --- + val types = mutableListOf() + val byKey = mutableMapOf() + collected.filterIsInstance().forEach { doc -> + if (!passesWhitelist(doc)) { + logger.info("javadoc-mode: omitting '${doc.name}' (source sets not in whitelist $sourceSetWhitelist)") + return@forEach + } + val key = keyOf(doc.dri) + if (byKey.containsKey(key)) return@forEach + val packageName = doc.dri.packageName.orEmpty() + val classNames = doc.dri.classNames ?: doc.name ?: return@forEach + val moduleName = moduleFor(packageName, doc) + val type = JdType( + documentable = doc, + key = key, + qualifiedName = key, + classNames = classNames, + simpleName = classNames.substringAfterLast('.'), + packageName = packageName, + moduleName = moduleName, + kind = kindOf(doc), + filePath = paths.classFile(packageName, classNames, moduleName) + ) + types += type + byKey[key] = type + } + + // --- Direct hierarchy --- + val superclassByKey = mutableMapOf() + val interfacesByKey = mutableMapOf>() + + types.forEach { type -> + val doc = type.documentable + if (doc !is WithSupertypes) { + interfacesByKey[type.key] = emptyList() + return@forEach + } + val supers = doc.supertypes.values.flatten().distinctBy { keyOf(it.typeConstructor.dri) } + val ifaces = mutableListOf() + supers.forEach { supertype -> + val superKey = keyOf(supertype.typeConstructor.dri) + if (superKey == type.key) return@forEach + // Prefer what the supertype actually *is* over the kind recorded at the use + // site; the recorded kind only has to be trusted for types we don't document. + val known = byKey[superKey] + val isInterface = when { + known != null -> known.kind == "interface" || known.kind == "annotation" + else -> supertype.kind.toString().uppercase().contains("INTERFACE") + } + if (isInterface) { + ifaces += superKey + } else { + // A class has at most one superclass; keep the first and ignore any + // duplicate a cross-source-set merge might have produced. + superclassByKey.putIfAbsent(type.key, superKey) + } + } + interfacesByKey[type.key] = ifaces.distinct() + } + + val directSubclasses = mutableMapOf>() + types.forEach { type -> + superclassByKey[type.key]?.let { superKey -> + directSubclasses.getOrPut(superKey) { mutableListOf() } += type.key + } + } + + // --- Transitive interface closure, memoized across the whole graph --- + val closureMemo = mutableMapOf>() + val inProgress = mutableSetOf() + + fun closureOf(key: String): List { + closureMemo[key]?.let { return it } + // Guards against a cycle in a malformed/merged hierarchy; without it a cyclic + // `extends` chain would recurse until the stack blew. + if (!inProgress.add(key)) return emptyList() + val result = LinkedHashSet() + interfacesByKey[key].orEmpty().forEach { iface -> + result += iface + result += closureOf(iface) + } + superclassByKey[key]?.let { result += closureOf(it) } + inProgress.remove(key) + val list = result.toList() + closureMemo[key] = list + return list + } + + val allSuperinterfacesByKey = types.associate { it.key to closureOf(it.key) } + + val subinterfaces = mutableMapOf>() + val implementors = mutableMapOf>() + types.forEach { type -> + val bucket = if (type.kind == "interface") subinterfaces else implementors + allSuperinterfacesByKey[type.key].orEmpty().forEach { iface -> + bucket.getOrPut(iface) { mutableListOf() } += type.key + } + } + + // --- Exception classification (javadoc tables exceptions separately) --- + val exceptionKeys = mutableSetOf() + types.forEach { type -> + if (isExceptionType(type, superclassByKey)) exceptionKeys += type.key + } + + // --- Packages and modules --- + // A package with no exports entry falls back to whichever module its own types + // resolved to, so the two never disagree about where the package page belongs. + val moduleOfTypePackage = types.groupBy { it.packageName } + .mapValues { (_, inPackage) -> inPackage.firstNotNullOfOrNull { it.moduleName } } + + val packages = collected.filterIsInstance() + .groupBy { it.dri.packageName.orEmpty() } + .map { (name, docs) -> + val moduleName = moduleOfPackage[name] ?: moduleOfTypePackage[name] + JdPackage(name, moduleName, docs, paths.packageFile(name, moduleName)) + } + .sortedBy { it.name } + + val modules = if (jpmsModules.isNotEmpty()) { + val dokkaModuleByName = moduleDocs.groupBy { it.name } + jpmsModules + .map { jpms -> + JdModule( + name = jpms.name, + documentables = dokkaModuleByName[jpms.name].orEmpty(), + filePath = paths.moduleFile(jpms.name), + jpms = jpms + ) + } + .sortedBy { it.name } + } else { + moduleDocs.groupBy { it.name } + .map { (name, docs) -> JdModule(name, docs, paths.moduleFile(name)) } + .sortedBy { it.name } + } + + logger.info( + "javadoc-mode: indexed ${types.size} types, ${packages.size} packages, " + + "${modules.size} module(s); module directories=$useModuleDirs" + ) + + return JavadocModelIndex( + paths = paths, + modules = modules, + packages = packages, + types = types.sortedBy { it.qualifiedName }, + useModuleDirs = useModuleDirs, + byKey = byKey, + superclassByKey = superclassByKey, + interfacesByKey = interfacesByKey, + allSuperinterfacesByKey = allSuperinterfacesByKey, + directSubclassesByKey = directSubclasses.mapValues { it.value.distinct().sorted() }, + subinterfacesByKey = subinterfaces.mapValues { it.value.distinct().sorted() }, + implementorsByKey = implementors.mapValues { it.value.distinct().sorted() }, + exceptionKeys = exceptionKeys + ) + } + + /** + * Whether a type belongs in javadoc's "Exception Classes" table. + * + * The reliable signal is a `java.lang.Throwable` ancestor, but a run that documents only + * part of a codebase often stops short of it -- the chain ends at, say, an undocumented + * `java.lang.RuntimeException`. Dokka's own `ExceptionInSupertypes` extra covers most of + * that gap; the trailing name check is the last resort for the remainder, and only ever + * looks at *ancestors*, never at the type's own name, so a class merely called + * `ExceptionHandler` isn't miscategorised. + */ + private fun isExceptionType(type: JdType, superclassByKey: Map): Boolean { + if (type.documentable.extrasOrEmpty().allOfType() + .any { it::class.java.simpleName == "ExceptionInSupertypes" } + ) { + return true + } + val ancestors = mutableListOf() + val seen = mutableSetOf(type.key) + var current = superclassByKey[type.key] + while (current != null && seen.add(current)) { + ancestors += current + current = superclassByKey[current] + } + return ancestors.any { + it == THROWABLE_FQN || it.endsWith("Exception") || it.endsWith("Error") + } + } + + private fun kindOf(doc: DClasslike): String = when (doc) { + is DInterface -> "interface" + is DEnum -> "enum" + is DAnnotation -> "annotation" + is DObject -> "object" + is DClass -> "class" + else -> "class" + } + + /** + * Every documentable reachable from the page tree, deduplicated. + * + * Walking pages alone is not enough: a package's classlikes and a class's nested types + * hang off the *documentable* tree, and Dokka does not always give every one of them its + * own page, so both trees are traversed. + */ + private fun collectDocumentables(root: PageNode): List { + val seen = LinkedHashMap() + + fun visitDocumentable(doc: Documentable) { + val id = "${doc::class.java.simpleName}|${doc.dri}" + if (seen.putIfAbsent(id, doc) != null) return + when (doc) { + is DModule -> doc.packages.forEach { visitDocumentable(it) } + is DPackage -> { + doc.classlikes.forEach { visitDocumentable(it) } + doc.typealiases.forEach { visitDocumentable(it) } + } + is DClasslike -> doc.classlikes.forEach { visitDocumentable(it) } + else -> Unit + } + } + + fun visitPage(node: PageNode) { + if (node is WithDocumentables) node.documentables.forEach { visitDocumentable(it) } + node.children.forEach { visitPage(it) } + } + + visitPage(root) + return seen.values.toList() + } + } + + fun typeFor(dri: DRI): JdType? = byKey[keyOf(dri)] + + fun typeForKey(key: String): JdType? = byKey[key] + + fun superclassOf(key: String): String? = superclassByKey[key] + + fun directInterfacesOf(key: String): List = interfacesByKey[key].orEmpty() + + /** + * The superclass chain, outermost ancestor first and [key] itself last -- the order javadoc + * prints its inheritance tree in. Cyclic input terminates rather than looping. + */ + fun inheritanceChain(key: String): List { + val chain = mutableListOf() + val seen = mutableSetOf() + var current: String? = key + while (current != null && seen.add(current)) { + chain += current + current = superclassByKey[current] + } + return chain.reversed() + } + + /** Superclass chain excluding [key] itself, nearest ancestor first. */ + fun superclassChain(key: String): List = inheritanceChain(key).dropLast(1).reversed() + + /** + * Every interface reachable from [key] through superclasses and interface extension. Backs + * both "All Implemented Interfaces" (for a class) and "All Superinterfaces" (for an interface). + */ + fun allSuperinterfaces(key: String): List = allSuperinterfacesByKey[key].orEmpty() + + fun directKnownSubclasses(key: String): List = directSubclassesByKey[key].orEmpty() + + /** Interfaces that extend [key], directly or transitively. */ + fun allKnownSubinterfaces(key: String): List = subinterfacesByKey[key].orEmpty() + + /** Classes, enums and objects that implement [key], directly or transitively. */ + fun allKnownImplementingClasses(key: String): List = implementorsByKey[key].orEmpty() + + /** True when [key] is a `Throwable` subtype, which javadoc tables separately. */ + fun isException(key: String): Boolean = key in exceptionKeys + + /** The enclosing type of a nested type, or null for a top-level one. */ + fun enclosingTypeOf(type: JdType): JdType? { + if (!type.classNames.contains('.')) return null + val outer = type.classNames.substringBeforeLast('.') + val outerKey = if (type.packageName.isBlank()) outer else "${type.packageName}.$outer" + return byKey[outerKey] + } +} diff --git a/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/javadoc/JavadocPaths.kt b/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/javadoc/JavadocPaths.kt new file mode 100644 index 00000000..a8014d83 --- /dev/null +++ b/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/javadoc/JavadocPaths.kt @@ -0,0 +1,141 @@ +package org.appdevforall.dokka.kdoc2json.javadoc + +import org.jetbrains.dokka.links.Callable +import org.jetbrains.dokka.links.DRI +import org.jetbrains.dokka.links.JavaClassReference +import org.jetbrains.dokka.links.RecursiveType +import org.jetbrains.dokka.links.StarProjection +import org.jetbrains.dokka.links.TypeConstructor +import org.jetbrains.dokka.links.TypeParam +import org.jetbrains.dokka.links.TypeReference +import org.jetbrains.dokka.links.Vararg + +/** + * Reproduces javadoc's on-disk layout and anchor scheme, with `.json` in place of `.html`. + * + * javadoc lays its `api/` tree out as: + * + * ``` + * //.html (module dir only for a modular run) + * //package-summary.html + * /module-summary.html + * ``` + * + * and links between those pages relatively (`../lang/Object.html`), which is what makes the tree + * self-contained when it is served from an arbitrary prefix. [relativeUrl] does the same thing. + * + * @param useModuleDirs whether page paths carry a leading `/` segment. Mirrors javadoc's + * own split: a modular run gets module directories, a non-modular one puts packages at the root. + */ +class JavadocPaths(private val useModuleDirs: Boolean) { + + companion object { + const val EXTENSION = "json" + const val PACKAGE_SUMMARY = "package-summary" + const val MODULE_SUMMARY = "module-summary" + + /** Dokka models a Java array as a one-argument `kotlin.Array` type constructor. */ + private val ARRAY_FQNS = setOf("kotlin.Array", "java.lang.Array") + } + + private fun prefix(moduleName: String?): String = + if (useModuleDirs && !moduleName.isNullOrBlank()) "$moduleName/" else "" + + private fun packageDir(packageName: String?): String = + if (packageName.isNullOrBlank()) "" else packageName.replace('.', '/') + "/" + + /** `java.base/java/util/Map.Entry.json`. Nested types keep their dotted name, as javadoc does. */ + fun classFile(packageName: String?, classNames: String, moduleName: String?): String = + "${prefix(moduleName)}${packageDir(packageName)}$classNames.$EXTENSION" + + fun packageFile(packageName: String?, moduleName: String?): String = + "${prefix(moduleName)}${packageDir(packageName)}$PACKAGE_SUMMARY.$EXTENSION" + + fun moduleFile(moduleName: String): String = + if (useModuleDirs) "$moduleName/$MODULE_SUMMARY.$EXTENSION" else "$MODULE_SUMMARY.$EXTENSION" + + /** + * A javadoc-style relative link from the page at [fromFile] to the page at [toFile], both + * given as output-dir-relative paths. Returns just the file name when they share a directory. + */ + fun relativeUrl(fromFile: String, toFile: String): String { + val fromDir = fromFile.split('/').dropLast(1) + val toParts = toFile.split('/') + val toDir = toParts.dropLast(1) + + var common = 0 + while (common < fromDir.size && common < toDir.size && fromDir[common] == toDir[common]) { + common++ + } + val up = List(fromDir.size - common) { ".." } + val down = toDir.drop(common) + toParts.last() + return (up + down).joinToString("/") + } + + /** + * javadoc's member anchor: the bare name for a field, and `name(erasedParamTypes)` for an + * executable -- with constructors spelled `(...)`, as javadoc has done since JDK 18. + * + * The parameter types are *erased* and fully qualified, so ` T[] toArray(T[] a)` anchors as + * `toArray(java.lang.Object[])`. That erasure is exactly what Dokka's DRI already carries, so + * the anchors here line up with the ones in a real javadoc build. + */ + fun memberAnchor(dri: DRI, isConstructor: Boolean): String { + val callable = dri.callable ?: return dri.classNames?.substringAfterLast('.') ?: "" + val name = if (isConstructor) "" else callable.name + if (isField(callable)) return name + val params = callable.params.joinToString(",") { erasedTypeName(it) } + return "$name($params)" + } + + /** + * A field's DRI carries no parameter list and is flagged `isProperty`; Dokka also emits + * zero-arg *methods* though, so `isProperty` is what actually separates the two. + */ + private fun isField(callable: Callable): Boolean = callable.isProperty + + /** + * Drops type arguments while keeping array brackets: `List[]` -> `List[]`. + * + * Dokka builds a Java DRI from the PSI type's canonical text, which carries the type + * arguments; javadoc's anchors use the *erasure*, so `addAll(java.util.Collection)` has to become `addAll(java.util.Collection)` or the anchor won't match a real javadoc + * build's. + */ + private fun eraseGenerics(name: String): String { + if ('<' !in name) return name + val result = StringBuilder(name.length) + var depth = 0 + name.forEach { character -> + when (character) { + '<' -> depth++ + '>' -> if (depth > 0) depth-- + else -> if (depth == 0) result.append(character) + } + } + return result.toString() + } + + /** Renders one DRI parameter type the way javadoc spells it inside a member anchor. */ + fun erasedTypeName(ref: TypeReference): String = when (ref) { + is TypeConstructor -> { + val fqn = eraseGenerics(ref.fullyQualifiedName) + if (fqn in ARRAY_FQNS) { + // A raw `kotlin.Array` with no argument can't be rendered as `X[]`; fall back to + // Object[] rather than emitting a bare "[]". + val inner = ref.params.firstOrNull()?.let { erasedTypeName(it) } ?: "java.lang.Object" + "$inner[]" + } else { + fqn + } + } + is JavaClassReference -> eraseGenerics(ref.name) + // A type variable erases to its leftmost bound, or to Object when unbounded. + is TypeParam -> ref.bounds.firstOrNull()?.let { erasedTypeName(it) } ?: "java.lang.Object" + is org.jetbrains.dokka.links.Nullable -> erasedTypeName(ref.wrapped) + is Vararg -> "${erasedTypeName(ref.elementType)}[]" + is StarProjection -> "java.lang.Object" + is RecursiveType -> "java.lang.Object" + else -> ref.toString() + } +} diff --git a/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/javadoc/JavadocRenderer.kt b/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/javadoc/JavadocRenderer.kt new file mode 100644 index 00000000..3837185a --- /dev/null +++ b/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/javadoc/JavadocRenderer.kt @@ -0,0 +1,516 @@ +package org.appdevforall.dokka.kdoc2json.javadoc + +import kotlinx.serialization.json.Json +import kotlinx.serialization.json.JsonElement +import kotlinx.serialization.serializer +import org.appdevforall.dokka.kdoc2json.JsonFilters +import org.appdevforall.dokka.kdoc2json.JsonPluginConfig +import org.appdevforall.dokka.kdoc2json.PluginLogger +import org.jetbrains.dokka.model.Documentable +import org.jetbrains.dokka.pages.RootPageNode +import java.io.File + +/** + * Writes the Javadoc-mode output tree. + * + * The layout mirrors what the `javadoc` tool produces under its `api/` directory, with `.json` + * in place of `.html`: + * + * ``` + * index.json overview: every module (or package) in the run + * element-list javadoc's plain-text manifest of modules/packages + * allclasses-index.json every documented type + * allpackages-index.json every documented package + * deprecated-list.json deprecated elements, grouped by kind + * constant-values.json static final fields with constant values + * index-files/index-N.json the A-Z index, one file per letter + * /module-summary.json (module directories only for a multi-module run) + * //package-summary.json + * //.json + * ``` + * + * Only JSON (plus javadoc's own plain-text `element-list`) is written -- no HTML. Rendering the + * pages is the downstream template engine's job. + */ +class JavadocRenderer( + private val config: JsonPluginConfig, + private val logger: PluginLogger, + private val outputDir: File, + /** + * The modules of a multi-module Dokka run, as `name to relative output path`. Non-empty only + * in the aggregating run that Dokka performs after the per-module ones; empty otherwise. + */ + private val moduleReferences: List> = emptyList(), + /** + * The run's configured source roots. Scanned for `module-info.java`, which is where the JPMS + * module structure javadoc lays its `api/` tree out by actually lives -- Dokka's model has none. + */ + private val sourceRoots: Collection = emptyList() +) { + + /** + * Javadoc-mode pages are written with `encodeDefaults = true` so every documented key is + * present on every page, even when empty -- a template can then test a field without also + * testing whether it exists. Callers who do want the empty keys gone still get that from + * `omitNulls`, which is applied afterwards, so the choice stays theirs rather than being + * baked into the serializer. No class discriminator is configured because none of the + * javadoc DTOs are polymorphic; their `kind` fields are ordinary data. + */ + private val json = Json { + prettyPrint = config.prettyPrint + encodeDefaults = true + } + + /** What a global index page needs about one member, without holding the whole class page. */ + private class MemberRecord( + val documentable: Documentable, + val label: String, + val kind: String, + val anchor: String, + val owner: JdType, + val deprecated: JdDeprecation? + ) + + /** A `static final` field carrying a compile-time constant, for `constant-values.json`. */ + private class ConstantRecord( + val owner: JdType, + val name: String, + val anchor: String, + val modifiers: List, + val typeDisplay: String, + val typeQualifiedName: String?, + val value: String + ) + + fun render(root: RootPageNode) { + val index = JavadocModelIndex.build(root, logger, config.sourceSetWhitelist, sourceRoots) + if (index.types.isEmpty() && index.packages.isEmpty()) { + // Dokka's aggregating pass over a multi-module build sees only module references, no + // documentables -- the real pages were written by the per-module runs. Emit just the + // overview, which is the one page that pass is actually responsible for. + if (moduleReferences.isNotEmpty()) { + writeAggregateOverview() + return + } + logger.warn("javadoc-mode: no documented types or packages were found; nothing to write.") + return + } + val mapper = JavadocMapper(index, logger) + + val members = mutableListOf() + val constants = mutableListOf() + + writeClassPages(index, mapper, members, constants) + writePackagePages(index, mapper) + writeModulePages(index, mapper) + writeOverview(index, mapper) + writeAllClassesIndex(index, mapper) + writeAllPackagesIndex(index, mapper) + writeDeprecatedList(index, mapper, members) + writeConstantValues(index, constants) + writeAlphabeticalIndex(index, mapper, members) + writeElementList(index) + + logger.info( + "javadoc-mode: wrote ${index.types.size} type page(s), ${index.packages.size} package " + + "page(s), ${index.modules.size} module page(s) and the global index files." + ) + } + + /** + * The overview page for a multi-module run, listing each module's own output. + * + * Only the module list is available here; each module's descriptions and index files live in + * its own output directory, written by that module's run. + */ + private fun writeAggregateOverview() { + logger.info("javadoc-mode: multi-module aggregation pass; writing the overview only.") + write( + "index.${JavadocPaths.EXTENSION}", + JdOverviewPage( + modules = moduleReferences + .sortedBy { it.first } + .map { (name, path) -> + JdModuleSummary( + name = name, + url = "$path/${JavadocPaths.MODULE_SUMMARY}.${JavadocPaths.EXTENSION}" + ) + } + ) + ) + } + + // ------------------------------------------------------------ page passes + + private fun writeClassPages( + index: JavadocModelIndex, + mapper: JavadocMapper, + members: MutableList, + constants: MutableList + ) { + index.types.forEach { type -> + val page = runCatching { mapper.classPage(type) }.getOrElse { error -> + // One bad page must not cost every other page, matching JsonRenderer's own + // per-page resilience. + logger.warn("javadoc-mode: failed to build page for '${type.qualifiedName}': ${error.message}") + return@forEach + } + write(type.filePath, page) + harvest(index, type, page, members, constants) + } + } + + /** Collects the per-member facts the global index pages are built from. */ + private fun harvest( + index: JavadocModelIndex, + type: JdType, + page: JdClassPage, + members: MutableList, + constants: MutableList + ) { + val doc = type.documentable + + // Anchor -> declaration, first occurrence winning. `doc.functions` can also carry members + // Dokka copied down from a supertype; the page only lists the declared ones, so matching + // by anchor and keeping the first keeps the declaration rather than the inherited copy. + fun byAnchor(items: List, isConstructor: Boolean): Map { + val result = LinkedHashMap() + items.forEach { result.putIfAbsent(index.paths.memberAnchor(it.dri, isConstructor), it) } + return result + } + + val propertyByName = doc.properties.associateBy { it.name } + val functionByAnchor = byAnchor(doc.functions, isConstructor = false) + val constructorByAnchor = byAnchor( + (doc as? org.jetbrains.dokka.model.WithConstructors)?.constructors.orEmpty(), + isConstructor = true + ) + val enumEntryByName = + (doc as? org.jetbrains.dokka.model.DEnum)?.entries?.associateBy { it.name }.orEmpty() + + fun recordField(field: JdField, kind: String, source: Documentable?) { + if (source != null) { + members += MemberRecord(source, field.name, kind, field.anchor, type, field.deprecated) + } + val value = field.constantValue + if (value != null && "static" in field.modifiers && "final" in field.modifiers) { + constants += ConstantRecord( + owner = type, + name = field.name, + anchor = field.anchor, + modifiers = field.modifiers, + typeDisplay = field.type.display, + typeQualifiedName = field.type.qualifiedName, + value = value + ) + } + } + + page.fields.forEach { recordField(it, "field", propertyByName[it.name]) } + page.enumConstants.forEach { recordField(it, "enumConstant", enumEntryByName[it.name]) } + (page.methods + page.annotationElements).forEach { executable -> + functionByAnchor[executable.anchor]?.let { + members += MemberRecord( + it, executable.name, executable.kind, executable.anchor, type, executable.deprecated + ) + } + } + page.constructors.forEach { executable -> + constructorByAnchor[executable.anchor]?.let { + members += MemberRecord( + it, executable.name, "constructor", executable.anchor, type, executable.deprecated + ) + } + } + } + + private fun writePackagePages(index: JavadocModelIndex, mapper: JavadocMapper) { + val typesByPackage = index.types.groupBy { it.packageName } + index.packages.forEach { pkg -> + write(pkg.filePath, mapper.packagePage(pkg, typesByPackage[pkg.name].orEmpty())) + } + } + + private fun writeModulePages(index: JavadocModelIndex, mapper: JavadocMapper) { + index.modules.forEach { module -> + val packages = index.packages.filter { it.moduleName == module.name } + write(module.filePath, mapper.modulePage(module, packages)) + } + } + + private fun writeOverview(index: JavadocModelIndex, mapper: JavadocMapper) { + val path = "index.${JavadocPaths.EXTENSION}" + val scope = mapper.scope(path) + write( + path, + JdOverviewPage( + title = index.modules.singleOrNull()?.name, + modules = index.modules.map { mapper.moduleSummary(it, scope) }, + packages = index.packages.map { mapper.packageSummary(it, scope) } + ) + ) + } + + private fun writeAllClassesIndex(index: JavadocModelIndex, mapper: JavadocMapper) { + val path = "allclasses-index.${JavadocPaths.EXTENSION}" + val scope = mapper.scope(path) + write( + path, + JdAllClassesIndex( + types = index.types + .map { mapper.typeSummary(it, scope) } + .sortedWith(compareBy({ it.name.lowercase() }, { it.qualifiedName })) + ) + ) + } + + private fun writeAllPackagesIndex(index: JavadocModelIndex, mapper: JavadocMapper) { + val path = "allpackages-index.${JavadocPaths.EXTENSION}" + val scope = mapper.scope(path) + write(path, JdAllPackagesIndex(packages = index.packages.map { mapper.packageSummary(it, scope) })) + } + + private fun writeDeprecatedList( + index: JavadocModelIndex, + mapper: JavadocMapper, + members: List + ) { + val path = "deprecated-list.${JavadocPaths.EXTENSION}" + val scope = mapper.scope(path) + val sections = linkedMapOf>() + + fun add(section: String, entry: JdDeprecatedEntry) { + sections.getOrPut(section) { mutableListOf() } += entry + } + + index.types.forEach { type -> + val summary = mapper.typeSummary(type, scope) + val deprecation = summary.deprecated ?: return@forEach + // javadoc gives exceptions, interfaces, enums and annotations their own sections. + add( + when (summary.kind) { + "interface" -> "interfaces" + "enum" -> "enums" + "annotation" -> "annotationTypes" + "exception" -> "exceptions" + else -> "classes" + }, + JdDeprecatedEntry( + element = type.qualifiedName, + kind = summary.kind, + url = scope.url(type.filePath), + comment = deprecation.comment, + forRemoval = deprecation.forRemoval, + since = deprecation.since + ) + ) + } + + members.forEach { member -> + if (member.deprecated == null) return@forEach + // Re-rendered against this page rather than reusing the class page's copy, whose + // links are relative to the class page. + val deprecation = mapper.deprecationFor(member.documentable, scope) ?: return@forEach + add( + when (member.kind) { + "constructor" -> "constructors" + "field" -> "fields" + "enumConstant" -> "enumConstants" + "annotationElement" -> "annotationElements" + else -> "methods" + }, + JdDeprecatedEntry( + element = "${member.owner.qualifiedName}.${member.anchor}", + kind = member.kind, + url = scope.url(member.owner.filePath, member.anchor), + comment = deprecation.comment, + forRemoval = deprecation.forRemoval, + since = deprecation.since + ) + ) + } + + write( + path, + JdDeprecatedList(sections = sections.mapValues { (_, entries) -> entries.sortedBy { it.element } }) + ) + } + + private fun writeConstantValues(index: JavadocModelIndex, constants: List) { + val path = "constant-values.${JavadocPaths.EXTENSION}" + val paths = index.paths + val byPackage = constants + .groupBy { it.owner.packageName } + .toSortedMap() + .mapValues { (_, records) -> + records.groupBy { it.owner } + .toList() + .sortedBy { it.first.qualifiedName } + .map { (owner, fields) -> + JdConstantsForType( + qualifiedName = owner.qualifiedName, + url = paths.relativeUrl(path, owner.filePath), + fields = fields.sortedBy { it.name }.map { record -> + JdConstantField( + name = record.name, + modifiers = record.modifiers, + type = JdTypeRef( + display = record.typeDisplay, + qualifiedName = record.typeQualifiedName, + url = record.typeQualifiedName + ?.let { index.typeForKey(it) } + ?.let { paths.relativeUrl(path, it.filePath) } + ), + value = record.value, + url = "${paths.relativeUrl(path, owner.filePath)}#${record.anchor}" + ) + } + ) + } + } + write(path, JdConstantValues(packages = byPackage)) + } + + /** + * javadoc's A-Z index, split one file per letter under `index-files/`. + * + * Every documented element -- module, package, type, field, constructor, method -- gets an + * entry, which is what makes the index usable as a search backing store downstream. + */ + private fun writeAlphabeticalIndex( + index: JavadocModelIndex, + mapper: JavadocMapper, + members: List + ) { + class PendingEntry( + val label: String, + val kind: String, + val filePath: String, + val anchor: String?, + val containingElement: String?, + val documentable: Documentable?, + val deprecated: Boolean, + /** Set for a member, so its summary resolves `{@inheritDoc}` as its own page does. */ + val owner: JdType? = null + ) + + val pending = mutableListOf() + + index.modules.forEach { + pending += PendingEntry(it.name, "module", it.filePath, null, null, it.documentables.firstOrNull(), false) + } + index.packages.forEach { + pending += PendingEntry(it.name, "package", it.filePath, null, it.moduleName, it.documentables.firstOrNull(), false) + } + index.types.forEach { type -> + pending += PendingEntry( + label = type.classNames, + kind = if (index.isException(type.key)) "exception" else type.kind, + filePath = type.filePath, + anchor = null, + containingElement = type.packageName, + documentable = type.documentable, + deprecated = false + ) + } + members.forEach { member -> + pending += PendingEntry( + label = if (member.kind == "constructor") member.owner.simpleName else member.label, + kind = member.kind, + filePath = member.owner.filePath, + anchor = member.anchor, + containingElement = member.owner.qualifiedName, + documentable = member.documentable, + deprecated = member.deprecated != null, + owner = member.owner + ) + } + + val grouped = pending + .sortedWith(compareBy({ it.label.lowercase() }, { it.containingElement.orEmpty() }, { it.kind })) + .groupBy { groupLabelFor(it.label) } + + // Symbols sort ahead of letters, which is also where javadoc puts them. + val letters = grouped.keys.sortedWith(compareBy({ it != SYMBOL_GROUP }, { it })) + + letters.forEachIndexed { position, letter -> + val number = position + 1 + val path = "index-files/index-$number.${JavadocPaths.EXTENSION}" + val scope = mapper.scope(path) + val entries = grouped.getValue(letter).map { entry -> + JdIndexEntry( + label = entry.label, + kind = entry.kind, + url = scope.url(entry.filePath, entry.anchor), + containingElement = entry.containingElement, + firstSentence = entry.documentable?.let { + mapper.summaryFor(it, scope, entry.owner, entry.anchor) + }, + deprecated = entry.deprecated + ) + } + write(path, JdIndexPage(letter = letter, index = number, letters = letters, entries = entries)) + } + } + + /** + * javadoc's plain-text manifest of what this documentation covers -- the file downstream + * tooling reads to resolve external links into this output. Modular runs list each module + * with `module:` followed by its packages; non-modular runs list packages alone. + */ + private fun writeElementList(index: JavadocModelIndex) { + val content = buildString { + if (index.modules.size > 1) { + index.modules.forEach { module -> + appendLine("module:${module.name}") + index.packages.filter { it.moduleName == module.name } + .map { it.name } + .sorted() + .forEach { appendLine(it) } + } + // A package Dokka surfaced without attaching it to a module would otherwise be + // absent from the manifest entirely; list it unqualified rather than lose it. + index.packages.filter { it.moduleName == null } + .map { it.name } + .sorted() + .forEach { appendLine(it) } + } else { + index.packages.map { it.name }.sorted().forEach { appendLine(it) } + } + } + val file = File(outputDir, "element-list") + file.parentFile?.mkdirs() + file.writeText(content) + } + + // ------------------------------------------------------------------ i/o + + private inline fun write(relativePath: String, value: T) { + try { + val element: JsonElement = json.encodeToJsonElement(serializer(), value) + val filtered = JsonFilters.filterJson(element, config.omitFields, config.omitNulls) + val file = File(outputDir, relativePath) + file.parentFile?.mkdirs() + // Belt and braces: the {@inheritDoc} stand-in is resolved wherever it is meant to be, + // but one leaking into published output would be visible corruption, so any straggler + // is dropped here rather than shipped. + file.writeText( + json.encodeToString(JsonElement.serializer(), filtered) + .replace(JavadocMapper.INHERIT_DOC_MARKER, "") + ) + logger.debug("javadoc-mode: wrote $relativePath") + } catch (e: Exception) { + logger.warn("javadoc-mode: failed to write $relativePath: ${e.message}") + } + } + + private companion object { + const val SYMBOL_GROUP = "SYMBOLS" + + /** The A-Z bucket a label belongs to; anything not starting with a letter is a symbol. */ + fun groupLabelFor(label: String): String { + val first = label.firstOrNull() ?: return SYMBOL_GROUP + return if (first.isLetter()) first.uppercaseChar().toString() else SYMBOL_GROUP + } + } +} diff --git a/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/javadoc/JpmsModuleInfo.kt b/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/javadoc/JpmsModuleInfo.kt new file mode 100644 index 00000000..76c210e4 --- /dev/null +++ b/Dokka-plugin-kdoc2json/kdoc-to-json/src/main/kotlin/javadoc/JpmsModuleInfo.kt @@ -0,0 +1,184 @@ +package org.appdevforall.dokka.kdoc2json.javadoc + +import org.appdevforall.dokka.kdoc2json.PluginLogger +import java.io.File + +/** One `requires` directive. */ +class JpmsRequires(val module: String, val isTransitive: Boolean, val isStatic: Boolean) + +/** One `exports` or `opens` directive; [to] is empty for an unqualified one. */ +class JpmsExports(val packageName: String, val to: List) + +/** One `provides ... with ...` directive. */ +class JpmsProvides(val service: String, val implementations: List) + +/** + * A JPMS module descriptor, read straight from a `module-info.java` in a source root. + * + * Dokka's model has no notion of JPMS: a Dokka "module" is a build-level grouping, so the module + * directories, the requires/exports/uses/provides tables and the module description that javadoc's + * module-summary page is built from are simply absent from it. They are all right there in + * `module-info.java` though, so Javadoc mode reads that file directly rather than doing without. + * + * @param description the module's doc comment with its block tags removed, still carrying javadoc + * inline tags (`{@link ...}`, `{@code ...}`). Resolving those needs the type index, so it is + * left to [JavadocMapper] rather than done here. + */ +class JpmsModuleInfo( + val name: String, + val sourceRoot: File, + val description: String?, + val since: List, + val requires: List, + val exports: List, + val opens: List, + val uses: List, + val provides: List, + val tags: List +) { + /** Packages this module exports to everyone -- exactly the set javadoc documents. */ + val exportedPackages: List get() = exports.filter { it.to.isEmpty() }.map { it.packageName } +} + +/** + * Finds and parses the `module-info.java` at the root of each configured source root. + * + * A source root holding a `module-info.java` *is* a JPMS module root, which makes this a + * self-validating signal: an ordinary `src/main/java` root has no such file, so nothing is + * misidentified as a module and non-modular projects are unaffected. + */ +object JpmsModuleScanner { + + // A directive is terminated by ';', and '[^;]' matches newlines, so multi-line `to` and + // `with` lists are handled without needing DOT_MATCHES_ALL. + private val MODULE_DECL = Regex("""\bmodule\s+([\w.]+)\s*\{""") + private val REQUIRES = Regex("""\brequires\s+((?:transitive\s+|static\s+)*)([\w.]+)\s*;""") + private val EXPORTS = Regex("""\bexports\s+([\w.]+)\s*(?:to\s+([^;]+?))?\s*;""") + private val OPENS = Regex("""\bopens\s+([\w.]+)\s*(?:to\s+([^;]+?))?\s*;""") + private val USES = Regex("""\buses\s+([\w.$]+)\s*;""") + private val PROVIDES = Regex("""\bprovides\s+([\w.$]+)\s+with\s+([^;]+?)\s*;""") + + private val BLOCK_COMMENT = Regex("""/\*.*?\*/""", RegexOption.DOT_MATCHES_ALL) + private val LINE_COMMENT = Regex("""//[^\n]*""") + private val DOC_COMMENT = Regex("""/\*\*(.*?)\*/""", RegexOption.DOT_MATCHES_ALL) + private val BLOCK_TAG = Regex("""^\s*@(\w+)\s*(.*)$""") + + private const val MODULE_INFO = "module-info.java" + + fun scan(sourceRoots: Collection, logger: PluginLogger): List { + logger.debug("javadoc-mode: scanning ${sourceRoots.size} source root entry/entries for $MODULE_INFO") + val found = LinkedHashMap() + sourceRoots.forEach { entry -> + // Dokka hands over source roots either as directories or, when the Gradle plugin has + // already expanded a source set, as the individual files in them. Both spellings of + // "here is a module root" are accepted. + val moduleInfo = when { + entry.isDirectory -> File(entry, MODULE_INFO) + entry.name == MODULE_INFO -> entry + else -> return@forEach + } + if (!moduleInfo.isFile) return@forEach + val root = moduleInfo.parentFile ?: return@forEach + try { + val parsed = parse(moduleInfo, root) + // Two source roots for the same module (e.g. a split main/generated layout) would + // otherwise fight over the mapping; the first wins, as it does for packages. + if (found.putIfAbsent(parsed.name, parsed) != null) { + logger.warn("javadoc-mode: module '${parsed.name}' declared in more than one source root; using the first.") + } + } catch (e: Exception) { + logger.warn("javadoc-mode: could not parse ${moduleInfo.path}: ${e.message}") + } + } + if (found.isNotEmpty()) { + logger.info("javadoc-mode: found ${found.size} JPMS module descriptor(s): ${found.keys.sorted().joinToString(", ")}") + } + return found.values.toList() + } + + private fun parse(moduleInfo: File, root: File): JpmsModuleInfo { + val text = moduleInfo.readText() + val (description, since, tags) = parseDocComment(text) + + // Comments are stripped before the directives are read, so a commented-out `exports` is + // never mistaken for a live one. + val body = LINE_COMMENT.replace(BLOCK_COMMENT.replace(text, " "), " ") + val name = MODULE_DECL.find(body)?.groupValues?.get(1) ?: root.name + + fun moduleList(raw: String?): List = + raw?.split(',')?.map { it.trim() }?.filter { it.isNotEmpty() }.orEmpty() + + return JpmsModuleInfo( + name = name, + sourceRoot = root, + description = description, + since = since, + requires = REQUIRES.findAll(body).map { match -> + val modifiers = match.groupValues[1] + JpmsRequires( + module = match.groupValues[2], + isTransitive = modifiers.contains("transitive"), + isStatic = modifiers.contains("static") + ) + }.toList(), + exports = EXPORTS.findAll(body).map { + JpmsExports(it.groupValues[1], moduleList(it.groupValues[2].ifBlank { null })) + }.toList(), + opens = OPENS.findAll(body).map { + JpmsExports(it.groupValues[1], moduleList(it.groupValues[2].ifBlank { null })) + }.toList(), + uses = USES.findAll(body).map { it.groupValues[1] }.toList(), + provides = PROVIDES.findAll(body).map { + JpmsProvides(it.groupValues[1], moduleList(it.groupValues[2])) + }.toList(), + tags = tags + ) + } + + /** + * Pulls the module's doc comment apart into description, `@since`, and every other block tag. + * + * The comment taken is the last one before the `module` declaration -- `module-info.java` + * opens with a license header, which must not be mistaken for the module's documentation. + */ + private fun parseDocComment(text: String): Triple, List> { + val declarationAt = MODULE_DECL.find(text)?.range?.first ?: text.length + val comment = DOC_COMMENT.findAll(text) + .lastOrNull { it.range.last < declarationAt } + ?.groupValues?.get(1) + ?: return Triple(null, emptyList(), emptyList()) + + val lines = comment.lines().map { it.trim().removePrefix("*").let { l -> if (l.startsWith(" ")) l.substring(1) else l } } + + val descriptionLines = mutableListOf() + val since = mutableListOf() + val tags = mutableListOf() + var currentTag: String? = null + val currentText = StringBuilder() + + fun flush() { + val tag = currentTag ?: return + val value = currentText.toString().trim() + if (tag == "since") since += value else tags += JdTag(tag, value) + currentTag = null + currentText.setLength(0) + } + + lines.forEach { line -> + val match = BLOCK_TAG.find(line) + if (match != null) { + flush() + currentTag = match.groupValues[1] + currentText.append(match.groupValues[2]) + } else if (currentTag != null) { + currentText.append('\n').append(line) + } else { + descriptionLines += line + } + } + flush() + + val description = descriptionLines.joinToString("\n").trim().ifBlank { null } + return Triple(description, since.filter { it.isNotBlank() }, tags.filter { it.text.isNotBlank() || it.name == "moduleGraph" }) + } +} diff --git a/Dokka-plugin-kdoc2json/pebble-renderer/build.gradle.kts b/Dokka-plugin-kdoc2json/pebble-renderer/build.gradle.kts new file mode 100644 index 00000000..0c32b854 --- /dev/null +++ b/Dokka-plugin-kdoc2json/pebble-renderer/build.gradle.kts @@ -0,0 +1,34 @@ +// Renders the plugin's javadoc-mode JSON into browsable HTML using Pebble templates that follow +// the official javadoc page structure. +// +// Deliberately plain Java: it has to build on whatever JDK is around (including ones too new for +// the Kotlin version the plugin itself is pinned to), and there is nothing here that needs Kotlin. +plugins { + application +} + +repositories { + mavenCentral() +} + +dependencies { + implementation("io.pebbletemplates:pebble:3.2.2") + implementation("com.fasterxml.jackson.core:jackson-databind:2.17.2") +} + +java { + // Compatibility rather than a toolchain: this has to build with whatever JDK is on the + // machine (a toolchain would demand a specific one be installed and registered), and the + // code targets nothing newer than 17. + sourceCompatibility = JavaVersion.VERSION_17 + targetCompatibility = JavaVersion.VERSION_17 +} + +application { + mainClass.set("org.appdevforall.docs.render.JavadocHtmlRenderer") +} + +tasks.named("run") { + // Lets the driver script pass " " through as -Pargs="..." + (findProperty("args") as String?)?.let { args = it.split(" ").filter { a -> a.isNotEmpty() } } +} diff --git a/Dokka-plugin-kdoc2json/pebble-renderer/gradle/wrapper/gradle-wrapper.jar b/Dokka-plugin-kdoc2json/pebble-renderer/gradle/wrapper/gradle-wrapper.jar new file mode 100644 index 00000000..1b33c55b Binary files /dev/null and b/Dokka-plugin-kdoc2json/pebble-renderer/gradle/wrapper/gradle-wrapper.jar differ diff --git a/Dokka-plugin-kdoc2json/pebble-renderer/gradle/wrapper/gradle-wrapper.properties b/Dokka-plugin-kdoc2json/pebble-renderer/gradle/wrapper/gradle-wrapper.properties new file mode 100644 index 00000000..df6a6ad7 --- /dev/null +++ b/Dokka-plugin-kdoc2json/pebble-renderer/gradle/wrapper/gradle-wrapper.properties @@ -0,0 +1,9 @@ +distributionBase=GRADLE_USER_HOME +distributionPath=wrapper/dists +distributionUrl=https\://services.gradle.org/distributions/gradle-9.5.1-bin.zip +networkTimeout=10000 +retries=0 +retryBackOffMs=500 +validateDistributionUrl=true +zipStoreBase=GRADLE_USER_HOME +zipStorePath=wrapper/dists diff --git a/Dokka-plugin-kdoc2json/pebble-renderer/gradlew b/Dokka-plugin-kdoc2json/pebble-renderer/gradlew new file mode 100755 index 00000000..b9bb139f --- /dev/null +++ b/Dokka-plugin-kdoc2json/pebble-renderer/gradlew @@ -0,0 +1,248 @@ +#!/bin/sh + +# +# Copyright © 2015 the original authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# SPDX-License-Identifier: Apache-2.0 +# + +############################################################################## +# +# Gradle start up script for POSIX generated by Gradle. +# +# Important for running: +# +# (1) You need a POSIX-compliant shell to run this script. If your /bin/sh is +# noncompliant, but you have some other compliant shell such as ksh or +# bash, then to run this script, type that shell name before the whole +# command line, like: +# +# ksh Gradle +# +# Busybox and similar reduced shells will NOT work, because this script +# requires all of these POSIX shell features: +# * functions; +# * expansions «$var», «${var}», «${var:-default}», «${var+SET}», +# «${var#prefix}», «${var%suffix}», and «$( cmd )»; +# * compound commands having a testable exit status, especially «case»; +# * various built-in commands including «command», «set», and «ulimit». +# +# Important for patching: +# +# (2) This script targets any POSIX shell, so it avoids extensions provided +# by Bash, Ksh, etc; in particular arrays are avoided. +# +# The "traditional" practice of packing multiple parameters into a +# space-separated string is a well documented source of bugs and security +# problems, so this is (mostly) avoided, by progressively accumulating +# options in "$@", and eventually passing that to Java. +# +# Where the inherited environment variables (DEFAULT_JVM_OPTS, JAVA_OPTS, +# and GRADLE_OPTS) rely on word-splitting, this is performed explicitly; +# see the in-line comments for details. +# +# There are tweaks for specific operating systems such as AIX, CygWin, +# Darwin, MinGW, and NonStop. +# +# (3) This script is generated from the Groovy template +# https://github.com/gradle/gradle/blob/3d91ce3b8caaf77ad09f381f43615b715b53f72c/platforms/jvm/plugins-application/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt +# within the Gradle project. +# +# You can find Gradle at https://github.com/gradle/gradle/. +# +############################################################################## + +# Attempt to set APP_HOME + +# Resolve links: $0 may be a link +app_path=$0 + +# Need this for daisy-chained symlinks. +while + APP_HOME=${app_path%"${app_path##*/}"} # leaves a trailing /; empty if no leading path + [ -h "$app_path" ] +do + ls=$( ls -ld "$app_path" ) + link=${ls#*' -> '} + case $link in #( + /*) app_path=$link ;; #( + *) app_path=$APP_HOME$link ;; + esac +done + +# This is normally unused +# shellcheck disable=SC2034 +APP_BASE_NAME=${0##*/} +# Discard cd standard output in case $CDPATH is set (https://github.com/gradle/gradle/issues/25036) +APP_HOME=$( cd -P "${APP_HOME:-./}" > /dev/null && printf '%s\n' "$PWD" ) || exit + +# Use the maximum available, or set MAX_FD != -1 to use that value. +MAX_FD=maximum + +warn () { + echo "$*" +} >&2 + +die () { + echo + echo "$*" + echo + exit 1 +} >&2 + +# OS specific support (must be 'true' or 'false'). +cygwin=false +msys=false +darwin=false +nonstop=false +case "$( uname )" in #( + CYGWIN* ) cygwin=true ;; #( + Darwin* ) darwin=true ;; #( + MSYS* | MINGW* ) msys=true ;; #( + NONSTOP* ) nonstop=true ;; +esac + + + +# Determine the Java command to use to start the JVM. +if [ -n "$JAVA_HOME" ] ; then + if [ -x "$JAVA_HOME/jre/sh/java" ] ; then + # IBM's JDK on AIX uses strange locations for the executables + JAVACMD=$JAVA_HOME/jre/sh/java + else + JAVACMD=$JAVA_HOME/bin/java + fi + if [ ! -x "$JAVACMD" ] ; then + die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME + +Please set the JAVA_HOME variable in your environment to match the +location of your Java installation." + fi +else + JAVACMD=java + if ! command -v java >/dev/null 2>&1 + then + die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. + +Please set the JAVA_HOME variable in your environment to match the +location of your Java installation." + fi +fi + +# Increase the maximum file descriptors if we can. +if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then + case $MAX_FD in #( + max*) + # In POSIX sh, ulimit -H is undefined. That's why the result is checked to see if it worked. + # shellcheck disable=SC2039,SC3045 + MAX_FD=$( ulimit -H -n ) || + warn "Could not query maximum file descriptor limit" + esac + case $MAX_FD in #( + '' | soft) :;; #( + *) + # In POSIX sh, ulimit -n is undefined. That's why the result is checked to see if it worked. + # shellcheck disable=SC2039,SC3045 + ulimit -n "$MAX_FD" || + warn "Could not set maximum file descriptor limit to $MAX_FD" + esac +fi + +# Collect all arguments for the java command, stacking in reverse order: +# * args from the command line +# * the main class name +# * -classpath +# * -D...appname settings +# * --module-path (only if needed) +# * DEFAULT_JVM_OPTS, JAVA_OPTS, and GRADLE_OPTS environment variables. + +# For Cygwin or MSYS, switch paths to Windows format before running java +if "$cygwin" || "$msys" ; then + APP_HOME=$( cygpath --path --mixed "$APP_HOME" ) + + JAVACMD=$( cygpath --unix "$JAVACMD" ) + + # Now convert the arguments - kludge to limit ourselves to /bin/sh + for arg do + if + case $arg in #( + -*) false ;; # don't mess with options #( + /?*) t=${arg#/} t=/${t%%/*} # looks like a POSIX filepath + [ -e "$t" ] ;; #( + *) false ;; + esac + then + arg=$( cygpath --path --ignore --mixed "$arg" ) + fi + # Roll the args list around exactly as many times as the number of + # args, so each arg winds up back in the position where it started, but + # possibly modified. + # + # NB: a `for` loop captures its iteration list before it begins, so + # changing the positional parameters here affects neither the number of + # iterations, nor the values presented in `arg`. + shift # remove old arg + set -- "$@" "$arg" # push replacement arg + done +fi + + +# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. +DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"' + +# Collect all arguments for the java command: +# * DEFAULT_JVM_OPTS, JAVA_OPTS, and optsEnvironmentVar are not allowed to contain shell fragments, +# and any embedded shellness will be escaped. +# * For example: A user cannot expect ${Hostname} to be expanded, as it is an environment variable and will be +# treated as '${Hostname}' itself on the command line. + +set -- \ + "-Dorg.gradle.appname=$APP_BASE_NAME" \ + -jar "$APP_HOME/gradle/wrapper/gradle-wrapper.jar" \ + "$@" + +# Stop when "xargs" is not available. +if ! command -v xargs >/dev/null 2>&1 +then + die "xargs is not available" +fi + +# Use "xargs" to parse quoted args. +# +# With -n1 it outputs one arg per line, with the quotes and backslashes removed. +# +# In Bash we could simply go: +# +# readarray ARGS < <( xargs -n1 <<<"$var" ) && +# set -- "${ARGS[@]}" "$@" +# +# but POSIX shell has neither arrays nor command substitution, so instead we +# post-process each arg (as a line of input to sed) to backslash-escape any +# character that might be a shell metacharacter, then use eval to reverse +# that process (while maintaining the separation between arguments), and wrap +# the whole thing up as a single "set" statement. +# +# This will of course break if any of these variables contains a newline or +# an unmatched quote. +# + +eval "set -- $( + printf '%s\n' "$DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS" | + xargs -n1 | + sed ' s~[^-[:alnum:]+,./:=@_]~\\&~g; ' | + tr '\n' ' ' + )" '"$@"' + +exec "$JAVACMD" "$@" diff --git a/Dokka-plugin-kdoc2json/pebble-renderer/gradlew.bat b/Dokka-plugin-kdoc2json/pebble-renderer/gradlew.bat new file mode 100644 index 00000000..aa5f10b0 --- /dev/null +++ b/Dokka-plugin-kdoc2json/pebble-renderer/gradlew.bat @@ -0,0 +1,82 @@ +@rem +@rem Copyright 2015 the original author or authors. +@rem +@rem Licensed under the Apache License, Version 2.0 (the "License"); +@rem you may not use this file except in compliance with the License. +@rem You may obtain a copy of the License at +@rem +@rem https://www.apache.org/licenses/LICENSE-2.0 +@rem +@rem Unless required by applicable law or agreed to in writing, software +@rem distributed under the License is distributed on an "AS IS" BASIS, +@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +@rem See the License for the specific language governing permissions and +@rem limitations under the License. +@rem +@rem SPDX-License-Identifier: Apache-2.0 +@rem + +@if "%DEBUG%"=="" @echo off +@rem ########################################################################## +@rem +@rem Gradle startup script for Windows +@rem +@rem ########################################################################## + +@rem Set local scope for the variables, and ensure extensions are enabled +setlocal EnableExtensions + +set DIRNAME=%~dp0 +if "%DIRNAME%"=="" set DIRNAME=. +@rem This is normally unused +set APP_BASE_NAME=%~n0 +set APP_HOME=%DIRNAME% + +@rem Resolve any "." and ".." in APP_HOME to make it shorter. +for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi + +@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. +set DEFAULT_JVM_OPTS="-Xmx64m" "-Xms64m" + +@rem Find java.exe +if defined JAVA_HOME goto findJavaFromJavaHome + +set JAVA_EXE=java.exe +%JAVA_EXE% -version >NUL 2>&1 +if %ERRORLEVEL% equ 0 goto execute + +echo. 1>&2 +echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. 1>&2 +echo. 1>&2 +echo Please set the JAVA_HOME variable in your environment to match the 1>&2 +echo location of your Java installation. 1>&2 + +"%COMSPEC%" /c exit 1 + +:findJavaFromJavaHome +set JAVA_HOME=%JAVA_HOME:"=% +set JAVA_EXE=%JAVA_HOME%/bin/java.exe + +if exist "%JAVA_EXE%" goto execute + +echo. 1>&2 +echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% 1>&2 +echo. 1>&2 +echo Please set the JAVA_HOME variable in your environment to match the 1>&2 +echo location of your Java installation. 1>&2 + +"%COMSPEC%" /c exit 1 + +:execute +@rem Setup the command line + + + +@rem Execute Gradle +@rem endlocal doesn't take effect until after the line is parsed and variables are expanded +@rem which allows us to clear the local environment before executing the java command +endlocal & "%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -jar "%APP_HOME%\gradle\wrapper\gradle-wrapper.jar" %* & call :exitWithErrorLevel + +:exitWithErrorLevel +@rem Use "%COMSPEC%" /c exit to allow operators to work properly in scripts +"%COMSPEC%" /c exit %ERRORLEVEL% diff --git a/Dokka-plugin-kdoc2json/pebble-renderer/render.sh b/Dokka-plugin-kdoc2json/pebble-renderer/render.sh new file mode 100755 index 00000000..627aac7e --- /dev/null +++ b/Dokka-plugin-kdoc2json/pebble-renderer/render.sh @@ -0,0 +1,34 @@ +#!/usr/bin/env bash +# Renders a javadoc-mode JSON tree to browsable HTML. +# +# ./render.sh +# +# Builds the renderer if needed, then walks the JSON tree writing one .html per .json at the same +# relative path. Because the trees mirror each other file-for-file, the relative links already in +# the JSON resolve as soon as their .json extension is swapped for .html, which is what the +# templates' `href` and `doc` filters do. +set -euo pipefail + +if [ $# -lt 2 ]; then + echo "Usage: $0 " >&2 + echo >&2 + echo "Example, after scripts/java/build-jdk-json-docs.sh:" >&2 + echo " $0 ../scripts/java/build-output/api ../scripts/java/build-output/html" >&2 + exit 1 +fi + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +JSON_DIR="$(cd "$1" && pwd)" +mkdir -p "$2" +HTML_DIR="$(cd "$2" && pwd)" + +echo "==> Building the renderer" +(cd "$SCRIPT_DIR" && ./gradlew --console=plain -q installDist) + +echo "==> Rendering $JSON_DIR -> $HTML_DIR" +"$SCRIPT_DIR/build/install/pebble-renderer/bin/pebble-renderer" "$JSON_DIR" "$HTML_DIR" + +echo +echo "Open it with:" +echo " (cd \"$HTML_DIR\" && python3 -m http.server 8000)" +echo " then browse http://localhost:8000/index.html" diff --git a/Dokka-plugin-kdoc2json/pebble-renderer/settings.gradle.kts b/Dokka-plugin-kdoc2json/pebble-renderer/settings.gradle.kts new file mode 100644 index 00000000..518ab513 --- /dev/null +++ b/Dokka-plugin-kdoc2json/pebble-renderer/settings.gradle.kts @@ -0,0 +1 @@ +rootProject.name = "pebble-renderer" diff --git a/Dokka-plugin-kdoc2json/pebble-renderer/src/main/.DS_Store b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/.DS_Store new file mode 100644 index 00000000..81dcca77 Binary files /dev/null and b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/.DS_Store differ diff --git a/Dokka-plugin-kdoc2json/pebble-renderer/src/main/java/org/appdevforall/docs/render/JavadocExtension.java b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/java/org/appdevforall/docs/render/JavadocExtension.java new file mode 100644 index 00000000..130198e8 --- /dev/null +++ b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/java/org/appdevforall/docs/render/JavadocExtension.java @@ -0,0 +1,89 @@ +package org.appdevforall.docs.render; + +import io.pebbletemplates.pebble.extension.AbstractExtension; +import io.pebbletemplates.pebble.extension.Filter; +import io.pebbletemplates.pebble.extension.escaper.SafeString; +import io.pebbletemplates.pebble.template.EvaluationContext; +import io.pebbletemplates.pebble.template.PebbleTemplate; + +import java.util.List; +import java.util.Map; +import java.util.regex.Matcher; +import java.util.regex.Pattern; + +/** + * The two filters every template needs. + * + * The JSON links to `.json` files, because that is what the plugin writes. The HTML output mirrors + * that tree file-for-file with `.html` names instead, so a link is correct in HTML as soon as its + * extension is swapped -- the relative path itself already points at the right place. Both filters + * here do that swap; they differ only in whether they are handed a bare URL or a block of + * documentation HTML with URLs inside it. + */ +public final class JavadocExtension extends AbstractExtension { + + @Override + public Map getFilters() { + return Map.of( + "href", new HrefFilter(), + "doc", new DocFilter() + ); + } + + /** `.json` -> `.html`, leaving any `#anchor` and any absolute URL alone. */ + static String toHtmlLink(String url) { + if (url == null || url.isEmpty()) return url; + if (url.startsWith("http://") || url.startsWith("https://")) return url; + int hash = url.indexOf('#'); + String path = hash < 0 ? url : url.substring(0, hash); + String fragment = hash < 0 ? "" : url.substring(hash); + if (path.endsWith(".json")) { + path = path.substring(0, path.length() - ".json".length()) + ".html"; + } + return path + fragment; + } + + /** Rewrites a single URL, e.g. `{{ type.url | href }}`. */ + static final class HrefFilter implements Filter { + @Override + public List getArgumentNames() { + return null; + } + + @Override + public Object apply(Object input, Map args, PebbleTemplate self, + EvaluationContext context, int lineNumber) { + return input == null ? null : toHtmlLink(input.toString()); + } + } + + /** + * Rewrites every `href` inside a block of documentation HTML and marks the result safe. + * + * Doc text arrives as HTML already -- a javadoc comment's body is HTML -- so it must not be + * escaped, but the links it contains still point at `.json`. Marking it safe here rather than + * writing `| raw` at each use keeps the "this is trusted HTML" decision in one place. + */ + static final class DocFilter implements Filter { + private static final Pattern HREF = Pattern.compile("href=\"([^\"]*)\""); + + @Override + public List getArgumentNames() { + return null; + } + + @Override + public Object apply(Object input, Map args, PebbleTemplate self, + EvaluationContext context, int lineNumber) { + if (input == null) return null; + Matcher matcher = HREF.matcher(input.toString()); + StringBuilder result = new StringBuilder(); + while (matcher.find()) { + String replacement = "href=\"" + toHtmlLink(matcher.group(1)) + "\""; + matcher.appendReplacement(result, Matcher.quoteReplacement(replacement)); + } + matcher.appendTail(result); + return new SafeString(result.toString()); + } + } +} diff --git a/Dokka-plugin-kdoc2json/pebble-renderer/src/main/java/org/appdevforall/docs/render/JavadocHtmlRenderer.java b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/java/org/appdevforall/docs/render/JavadocHtmlRenderer.java new file mode 100644 index 00000000..c6348eb3 --- /dev/null +++ b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/java/org/appdevforall/docs/render/JavadocHtmlRenderer.java @@ -0,0 +1,207 @@ +package org.appdevforall.docs.render; + +import com.fasterxml.jackson.databind.ObjectMapper; +import io.pebbletemplates.pebble.PebbleEngine; +import io.pebbletemplates.pebble.loader.ClasspathLoader; +import io.pebbletemplates.pebble.template.PebbleTemplate; + +import java.io.IOException; +import java.io.InputStream; +import java.io.Writer; +import java.nio.charset.StandardCharsets; +import java.nio.file.FileVisitResult; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.SimpleFileVisitor; +import java.nio.file.attribute.BasicFileAttributes; +import java.util.LinkedHashMap; +import java.util.LinkedHashSet; +import java.util.List; +import java.util.Set; +import java.util.Map; + +/** + * Turns a javadoc-mode JSON tree into browsable HTML. + * + * Every JSON page carries a {@code page} field naming its kind, which selects the Pebble template; + * the parsed JSON becomes the template context directly, so a template reads the same field names + * that appear in the JSON. Output mirrors the input tree exactly, with {@code .json} swapped for + * {@code .html}, which is what makes the relative links in the JSON resolve once rewritten. + * + * 

+ *   java -jar pebble-renderer.jar <json-dir> <html-dir>
+ *
+ */ +public final class JavadocHtmlRenderer { + + /** {@code page} field value -> template. A page kind with no entry here is skipped. */ + private static final Map TEMPLATES = Map.of( + "class", "class", + "package", "package-summary", + "module", "module-summary", + "overview", "overview", + "all-classes", "all-classes", + "all-packages", "all-packages", + "deprecated-list", "deprecated-list", + "constant-values", "constant-values", + "index", "index-page" + ); + + private final ObjectMapper json = new ObjectMapper(); + private final PebbleEngine engine; + + private JavadocHtmlRenderer() { + this.engine = new PebbleEngine.Builder() + .loader(new ClasspathLoader() {{ + setPrefix("templates"); + setSuffix(".peb"); + }}) + // The doc text in the JSON is already HTML (a javadoc comment's body is), and the + // templates mark those values with |raw. Autoescaping stays on so everything else + // -- names, signatures, modifiers -- is escaped by default rather than by memory. + .autoEscaping(true) + .strictVariables(false) + .extension(new JavadocExtension()) + .build(); + } + + public static void main(String[] args) throws Exception { + if (args.length < 2) { + System.err.println("Usage: JavadocHtmlRenderer "); + System.exit(2); + } + Path source = Path.of(args[0]).toAbsolutePath().normalize(); + Path target = Path.of(args[1]).toAbsolutePath().normalize(); + if (!Files.isDirectory(source)) { + System.err.println("Not a directory: " + source); + System.exit(2); + } + int written = new JavadocHtmlRenderer().renderTree(source, target); + System.out.println("Wrote " + written + " HTML pages to " + target); + } + + private int renderTree(Path source, Path target) throws IOException { + Files.createDirectories(target); + copyStaticAssets(target); + // Built before the pages are written: a module page embeds its graph, so the graph has to + // exist, and drawing one needs every module's requires, not just its own. + ModuleGraphWriter graphs = new ModuleGraphWriter(readModuleRequires(source)); + + int[] counters = {0, 0}; + Files.walkFileTree(source, new SimpleFileVisitor<>() { + @Override + public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException { + Path relative = source.relativize(file); + if (file.getFileName().toString().endsWith(".json")) { + if (renderPage(file, relative, target, graphs)) counters[0]++; else counters[1]++; + } else { + // element-list and anything else non-JSON is carried across untouched. + Path copy = target.resolve(relative); + Files.createDirectories(copy.getParent()); + Files.copy(file, copy, java.nio.file.StandardCopyOption.REPLACE_EXISTING); + } + return FileVisitResult.CONTINUE; + } + }); + if (counters[1] > 0) { + System.out.println("Skipped " + counters[1] + " JSON file(s) with no matching template."); + } + return counters[0]; + } + + /** Every module's direct `requires`, read from the module pages before rendering starts. */ + @SuppressWarnings("unchecked") + private Map> readModuleRequires(Path source) throws IOException { + Map> result = new LinkedHashMap<>(); + try (var paths = Files.walk(source)) { + for (Path file : (Iterable) paths.filter(p -> p.getFileName().toString() + .equals("module-summary.json"))::iterator) { + Map data = readPage(file); + if (data == null) continue; + Object name = data.get("name"); + if (name == null) continue; + // Only `requires transitive` -- see ModuleGraphWriter.requires for why. + Set required = new LinkedHashSet<>(); + Object requires = data.get("requires"); + if (requires instanceof List list) { + for (Object entry : list) { + if (entry instanceof Map map && map.get("module") != null + && Boolean.TRUE.equals(map.get("isTransitive"))) { + required.add(map.get("module").toString()); + } + } + } + result.put(name.toString(), required); + } + } + return result; + } + + private boolean renderPage(Path file, Path relative, Path target, ModuleGraphWriter graphs) + throws IOException { + Map data = readPage(file); + if (data == null) return false; + + Object kind = data.get("page"); + String templateName = kind == null ? null : TEMPLATES.get(kind.toString()); + if (templateName == null) { + System.err.println("No template for page kind '" + kind + "' (" + relative + ")"); + return false; + } + + Map context = new LinkedHashMap<>(data); + // Depth of this page below the output root, so templates can reach shared assets and the + // top-level index pages regardless of how deep they sit. + context.put("pathToRoot", pathToRoot(relative)); + context.put("pageKind", kind.toString()); + + Path out = target.resolve(withHtmlExtension(relative)); + Files.createDirectories(out.getParent()); + + if ("module".equals(kind.toString()) && data.get("name") != null) { + graphs.write(out.getParent(), data.get("name").toString()); + context.put("hasModuleGraph", true); + } + PebbleTemplate template = engine.getTemplate(templateName); + try (Writer writer = Files.newBufferedWriter(out, StandardCharsets.UTF_8)) { + template.evaluate(writer, context); + } catch (IOException e) { + throw e; + } catch (RuntimeException e) { + throw new IOException("Failed rendering " + relative + ": " + e.getMessage(), e); + } + return true; + } + + @SuppressWarnings("unchecked") + private Map readPage(Path file) { + try { + return json.readValue(file.toFile(), Map.class); + } catch (IOException e) { + System.err.println("Could not read " + file + ": " + e.getMessage()); + return null; + } + } + + private static Path withHtmlExtension(Path relative) { + String name = relative.getFileName().toString(); + String renamed = name.substring(0, name.length() - ".json".length()) + ".html"; + Path parent = relative.getParent(); + return parent == null ? Path.of(renamed) : parent.resolve(renamed); + } + + /** {@code ""} at the root, {@code "../"} one level down, and so on. */ + private static String pathToRoot(Path relative) { + int depth = relative.getNameCount() - 1; + return "../".repeat(Math.max(0, depth)); + } + + private void copyStaticAssets(Path target) throws IOException { + for (String asset : new String[]{"stylesheet.css"}) { + try (InputStream in = getClass().getResourceAsStream("/static/" + asset)) { + if (in == null) continue; + Files.copy(in, target.resolve(asset), java.nio.file.StandardCopyOption.REPLACE_EXISTING); + } + } + } +} diff --git a/Dokka-plugin-kdoc2json/pebble-renderer/src/main/java/org/appdevforall/docs/render/ModuleGraphWriter.java b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/java/org/appdevforall/docs/render/ModuleGraphWriter.java new file mode 100644 index 00000000..b49dfd6e --- /dev/null +++ b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/java/org/appdevforall/docs/render/ModuleGraphWriter.java @@ -0,0 +1,202 @@ +package org.appdevforall.docs.render; + +import java.io.IOException; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.ArrayDeque; +import java.util.ArrayList; +import java.util.Comparator; +import java.util.Deque; +import java.util.HashMap; +import java.util.HashSet; +import java.util.LinkedHashMap; +import java.util.LinkedHashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.TreeMap; + +/** + * Draws the `module-graph.svg` that each javadoc module page embeds. + * + * The official ones are produced by graphviz, which isn't a dependency here, so these are laid out + * directly: a module's transitive `requires` closure, arranged in rows by depth. They carry the + * same information and the same visual language as the originals -- text-only nodes, grey arrows, + * and javadoc's two module colours -- without being byte-identical to a graphviz rendering. + * + * The colour rule is javadoc's: modules that make up the Java SE platform (`java.se` and + * everything it requires) are orange, JDK-specific modules blue. + */ +final class ModuleGraphWriter { + + private static final String SE_COLOR = "#e76f00"; + private static final String JDK_COLOR = "#437291"; + private static final String EDGE_COLOR = "#999999"; + private static final String FONT = "DejaVuSans"; + private static final int FONT_SIZE = 12; + private static final int ROW_HEIGHT = 42; + private static final int CHAR_WIDTH = 7; // approximate advance for the 12pt font + private static final int COLUMN_GAP = 24; + private static final int MARGIN = 8; + + /** + * module name -> the modules it `requires transitive`. + * + * Plain `requires` is deliberately absent: javadoc's module graph draws the *readability* + * graph, which only propagates through `requires transitive`. `java.naming` plainly requires + * `java.security.sasl` and its graph shows only itself and `java.base` -- verified against all + * 60 JDK module graphs, which this rule reproduces exactly. + */ + private final Map> requires; + private final Set platformModules; + + private static final String BASE_MODULE = "java.base"; + + ModuleGraphWriter(Map> transitiveRequires) { + this.requires = transitiveRequires; + this.platformModules = platformModules(transitiveRequires); + } + + + + /** + * The Java SE platform: `java.se`, everything it requires, and `java.base`, which every module + * requires implicitly. Empty when the run has no `java.se`, in which case every node is drawn + * in the JDK colour. + */ + private static Set platformModules(Map> requires) { + Set platform = new HashSet<>(); + Set se = requires.get("java.se"); + if (se == null) return platform; + platform.add("java.se"); + platform.add(BASE_MODULE); + platform.addAll(se); + return platform; + } + + /** Writes `/module-graph.svg` for [moduleName]. */ + void write(Path moduleDir, String moduleName) throws IOException { + List> rows = layout(moduleName); + if (rows.isEmpty()) return; + Files.createDirectories(moduleDir); + Files.writeString(moduleDir.resolve("module-graph.svg"), render(moduleName, rows), + StandardCharsets.UTF_8); + } + + /** + * Groups the transitive closure into rows by longest distance from the root, so a module is + * always drawn below everything that requires it. + */ + private List> layout(String root) { + Map depth = new HashMap<>(); + depth.put(root, 0); + Deque queue = new ArrayDeque<>(); + queue.add(root); + // Longest-path depth needs re-visiting when a longer route to a node turns up, which is + // why this is a worklist rather than a plain BFS. + while (!queue.isEmpty()) { + String current = queue.poll(); + int next = depth.get(current) + 1; + for (String required : requires.getOrDefault(current, Set.of())) { + if (!requires.containsKey(required)) continue; // undocumented module + if (next > depth.getOrDefault(required, -1)) { + depth.put(required, next); + queue.add(required); + } + } + } + // Every module reads java.base implicitly. JPMS grants that without it being written, so + // it is absent from module-info.java and from the JSON, but javadoc still draws it (while + // leaving it out of the Requires *table*). + if (!root.equals(BASE_MODULE) && requires.containsKey(BASE_MODULE)) { + int deepest = depth.values().stream().mapToInt(Integer::intValue).max().orElse(0); + depth.put(BASE_MODULE, deepest + 1); + } + + Map> byDepth = new TreeMap<>(); + depth.forEach((name, level) -> byDepth.computeIfAbsent(level, k -> new ArrayList<>()).add(name)); + List> rows = new ArrayList<>(); + byDepth.values().forEach(row -> { + row.sort(Comparator.naturalOrder()); + rows.add(row); + }); + return rows; + } + + private String render(String root, List> rows) { + Map centres = new HashMap<>(); // name -> {cx, cy} + int width = 0; + for (List row : rows) { + int rowWidth = row.stream().mapToInt(n -> n.length() * CHAR_WIDTH).sum() + + COLUMN_GAP * Math.max(0, row.size() - 1); + width = Math.max(width, rowWidth); + } + width += MARGIN * 2; + int height = rows.size() * ROW_HEIGHT + MARGIN * 2; + + for (int level = 0; level < rows.size(); level++) { + List row = rows.get(level); + int rowWidth = row.stream().mapToInt(n -> n.length() * CHAR_WIDTH).sum() + + COLUMN_GAP * Math.max(0, row.size() - 1); + int x = (width - rowWidth) / 2; + int y = MARGIN + level * ROW_HEIGHT + FONT_SIZE; + for (String name : row) { + int nodeWidth = name.length() * CHAR_WIDTH; + centres.put(name, new int[]{x + nodeWidth / 2, y}); + x += nodeWidth + COLUMN_GAP; + } + } + + StringBuilder svg = new StringBuilder(); + svg.append("\n"); + svg.append("\n"); + svg.append("").append(escape(root)).append("\n"); + svg.append("\n"); + svg.append("\n"); + + // Edges first so the labels sit on top of them. + Set drawn = new LinkedHashSet<>(); + centres.keySet().stream().sorted().forEach(from -> { + Set targets = new LinkedHashSet<>(); + requires.getOrDefault(from, Set.of()).stream() + .filter(centres::containsKey) + .forEach(targets::add); + // A module with no transitive requires of its own reads java.base directly, and that + // is the edge javadoc draws for it. + if (targets.isEmpty() && !from.equals(BASE_MODULE) && centres.containsKey(BASE_MODULE)) { + targets.add(BASE_MODULE); + } + for (String to : targets) { + if (!drawn.add(from + "->" + to)) continue; + int[] a = centres.get(from); + int[] b = centres.get(to); + if (b[1] <= a[1]) continue; // only draw downwards, never back up a cycle + svg.append("\n"); + } + }); + + centres.keySet().stream().sorted().forEach(name -> { + int[] c = centres.get(name); + String colour = platformModules.contains(name) ? SE_COLOR : JDK_COLOR; + svg.append("").append(escape(name)) + .append("\n"); + }); + + svg.append("\n"); + return svg.toString(); + } + + private static String escape(String value) { + return value.replace("&", "&").replace("<", "<").replace(">", ">"); + } +} diff --git a/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/.DS_Store b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/.DS_Store new file mode 100644 index 00000000..9f2fe980 Binary files /dev/null and b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/.DS_Store differ diff --git a/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/static/stylesheet.css b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/static/stylesheet.css new file mode 100644 index 00000000..2dc1a11b --- /dev/null +++ b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/static/stylesheet.css @@ -0,0 +1,167 @@ +/* + * Styling for the javadoc-mode HTML output. + * + * Written against the class names the official javadoc doclet emits (top-nav, sub-nav, header, + * summary-table, col-first/col-second/col-last, member-signature, notes, inheritance, ...) so the + * generated markup and the real thing describe the same structure. It is a readable approximation + * of javadoc's look, not a copy of the JDK's own stylesheet. + */ + +:root { + --body-font: 'DejaVu Sans', Arial, Helvetica, sans-serif; + --code-font: 'DejaVu Sans Mono', monospace; + --text: #353833; + --link: #4a6782; + --link-active: #bb7a2a; + --nav-bg: #4d7a97; + --nav-fg: #ffffff; + --subnav-bg: #dee3e9; + --border: #ededed; + --even-row: #ffffff; + --odd-row: #eeeeef; + --header-bg: #dee3e9; + --deprecated: #9c3328; +} + +* { box-sizing: border-box; } + +body { + background-color: #ffffff; + color: var(--text); + font-family: var(--body-font); + font-size: 14px; + margin: 0; + padding: 0; + line-height: 1.45; +} + +a { color: var(--link); text-decoration: none; } +a:hover, a:focus { color: var(--link-active); text-decoration: underline; } + +code, .member-signature, .type-signature, .package-signature { font-family: var(--code-font); font-size: 13px; } + +/* --- Navigation ------------------------------------------------------- */ + +.top-nav { + background-color: var(--nav-bg); + color: var(--nav-fg); + padding: 0 1rem; +} +.top-nav .nav-list { list-style: none; display: flex; flex-wrap: wrap; margin: 0; padding: 0.4rem 0; gap: 1.25rem; } +.top-nav .nav-list a { color: var(--nav-fg); font-weight: bold; font-size: 13px; } +.top-nav .nav-list a:hover { color: #bb7a2a; } + +.sub-nav { background-color: var(--subnav-bg); padding: 0 1rem; min-height: 1.8rem; } +.sub-nav-list { list-style: none; display: flex; flex-wrap: wrap; margin: 0; padding: 0.3rem 0; gap: 0.4rem; font-size: 13px; } +.sub-nav-list li + li::before { content: "\00a0/\00a0"; color: #666; } + +/* --- Layout ----------------------------------------------------------- */ + +.flex-content { padding: 0 1rem 2rem; max-width: 1400px; } +main { display: block; } + +.header { margin: 1rem 0; } +h1.title { font-size: 1.5rem; margin: 0.3rem 0; font-weight: normal; } +.sub-title { font-size: 13px; margin: 0.2rem 0; } +.package-label-in-type, .module-label-in-package { font-weight: bold; } + +h2 { font-size: 1.15rem; border-bottom: 1px solid #bbb; padding-bottom: 0.2rem; margin-top: 1.6rem; } +h3 { font-size: 1rem; margin: 1.1rem 0 0.3rem; } + +hr { border: none; border-top: 1px solid var(--border); margin: 1rem 0; } + +/* --- Class description ------------------------------------------------ */ + +.inheritance { margin-left: 1.2rem; font-size: 13px; } +.class-description > .inheritance:first-of-type { margin-left: 0; } + +.type-signature, .package-signature { + margin: 0.6rem 0; + padding: 0.5rem; + background-color: #f7f7f7; + border-left: 3px solid var(--nav-bg); + white-space: pre-wrap; +} +.modifiers, .return-type, .type-parameters { color: #4a6782; } +.element-name { font-weight: bold; } + +.block { margin: 0.4rem 0; } +.block p:first-child { margin-top: 0; } + +dl.notes { margin: 0.6rem 0; } +dl.notes dt { font-weight: bold; margin-top: 0.5rem; font-size: 13px; } +dl.notes dd { margin: 0.1rem 0 0.1rem 1.5rem; } +ul.see-list { list-style: none; margin: 0; padding: 0; } + +.deprecation-block { + border: 1px solid var(--deprecated); + border-left-width: 4px; + padding: 0.4rem 0.6rem; + margin: 0.6rem 0; + background-color: #fdf3f2; +} +.deprecated-label { color: var(--deprecated); font-weight: bold; } +.deprecation-comment { margin-top: 0.3rem; } + +/* --- Summary and detail lists ----------------------------------------- */ + +.summary-list, .details-list, .member-list { list-style: none; margin: 0; padding: 0; } +.member-list > li { border-top: 1px solid var(--border); padding-top: 0.5rem; margin-top: 0.8rem; } + +.caption { margin-top: 1rem; } +.caption span { + display: inline-block; + background-color: var(--nav-bg); + color: #fff; + padding: 0.25rem 0.8rem; + font-weight: bold; + font-size: 13px; + border-radius: 3px 3px 0 0; +} + +.summary-table { display: grid; border: 1px solid var(--border); font-size: 13px; } +.two-column-summary { grid-template-columns: minmax(20%, max-content) minmax(20%, auto); } +.three-column-summary { grid-template-columns: minmax(15%, max-content) minmax(15%, max-content) minmax(20%, auto); } + +.table-header { background-color: var(--header-bg); font-weight: bold; padding: 0.4rem 0.6rem; } +.summary-table > div { padding: 0.4rem 0.6rem; overflow-wrap: anywhere; } +.even-row-color { background-color: var(--even-row); } +.odd-row-color { background-color: var(--odd-row); } +.col-first, .col-second, .col-constructor-name { font-family: var(--code-font); } +.col-deprecated-item-name { font-family: var(--code-font); } + +.inherited-list { margin: 0.8rem 0; font-size: 13px; } +.inherited-list h3 { font-size: 0.9rem; background-color: var(--subnav-bg); padding: 0.3rem 0.5rem; margin-bottom: 0.3rem; font-weight: bold; } +.inherited-list code { overflow-wrap: anywhere; } + +/* --- Index ------------------------------------------------------------ */ + +.contents-list { margin: 0.5rem 0; font-family: var(--code-font); } +.contents-list a { margin-right: 0.4rem; } +dl.index dt { margin-top: 0.6rem; } +dl.index dd { margin-left: 1.5rem; } +.member-name-link { font-weight: bold; } + +/* --- Footer ----------------------------------------------------------- */ + +footer { margin-top: 2rem; font-size: 12px; color: #666; } + +@media screen and (max-width: 800px) { + .two-column-summary, .three-column-summary { grid-template-columns: 1fr; } + .table-header { display: none; } + .summary-table > div { border-bottom: 1px solid var(--border); } +} + +/* --- Module graph ------------------------------------------------------ */ + +/* javadoc shows a 100px-high thumbnail and reveals the full-size graph on hover. */ +.module-graph { position: relative; display: inline-block; } +.module-graph span { display: none; } +.module-graph:hover span { + display: block; + position: absolute; + top: 0; + left: 0; + z-index: 10; + background-color: #fff; +} diff --git a/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates.zip b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates.zip new file mode 100644 index 00000000..feeaaea1 Binary files /dev/null and b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates.zip differ diff --git a/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/all-classes.peb b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/all-classes.peb new file mode 100644 index 00000000..2bede7c9 --- /dev/null +++ b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/all-classes.peb @@ -0,0 +1,19 @@ +{% extends "base" %} +{% block title %}All Classes and Interfaces{% endblock %} +{% block bodyClass %}all-classes-index-page{% endblock %} +{% block content %} +

All Classes and Interfaces

+
+
Classes, Interfaces, Enums and Annotation Interfaces
+
+
Class
+
Package
+
Description
+{% for type in types %}{% if type.modifiers is empty or type.modifiers contains 'public' %} +
{{ type.name }}
+
{{ type.packageName }}
+
{{ type.firstSentence | doc }}
+{% endif %}{% endfor %} +
+
+{% endblock %} diff --git a/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/all-packages.peb b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/all-packages.peb new file mode 100644 index 00000000..644e7381 --- /dev/null +++ b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/all-packages.peb @@ -0,0 +1,19 @@ +{% extends "base" %} +{% block title %}All Packages{% endblock %} +{% block bodyClass %}all-packages-index-page{% endblock %} +{% block content %} +

All Packages

+
+
Package Summary
+
+
Module
+
Package
+
Description
+{% for pkg in packages %} +
{{ pkg.moduleName }}
+
{{ pkg.name }}
+
{{ pkg.firstSentence | doc }}
+{% endfor %} +
+
+{% endblock %} diff --git a/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/base.peb b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/base.peb new file mode 100644 index 00000000..59fe4a96 --- /dev/null +++ b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/base.peb @@ -0,0 +1,43 @@ +{# + The page skeleton every javadoc page shares: the two navigation bars, the flex wrapper, and the + footer. Class names follow the official javadoc output (top-nav, sub-nav, flex-content, header, + ...) so the accompanying stylesheet and the real one describe the same structure. +#} + + + +{% block title %}Documentation{% endblock %} + + + + +
+
+ +
+
+
+{% block content %}{% endblock %} +
+
+
+

Generated from javadoc-mode JSON by kdoc-to-json's Pebble renderer.

+
+
+
+ + diff --git a/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/class.peb b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/class.peb new file mode 100644 index 00000000..043f0735 --- /dev/null +++ b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/class.peb @@ -0,0 +1,298 @@ +{% extends "base" %} +{% import "macros" %} + +{% block title %}{{ name }} ({{ moduleName | default('API') }}){% endblock %} +{% block bodyClass %}class-declaration-page{% endblock %} + +{% block subnav %} +
+
+
    +{% if moduleName is not empty %}
  1. {% if moduleUrl is not empty %}{{ moduleName }}{% else %}{{ moduleName }}{% endif %}
    2. {% endif %} +
  2. {{ packageName }}
    3. +
  3. {{ name }}
    4. +
+
+
+{% endblock %} + +{% block content %} +
+{% if packageName is not empty %} +
Package {{ packageName }}
+{% endif %} +

{{ kind | capitalize }} {{ name }}{% if typeParameters is not empty %}<{% for t in typeParameters %}{{ t.name }}{% if not loop.last %},{% endif %}{% endfor %}>{% endif %}

+
+ +
+
+ +{# The inheritance tree, indented one step per level, exactly as javadoc draws it. #} +{% if inheritance is not empty and inheritance | length > 1 %} +{# javadoc nests one div per level so each generation indents further than the last. The final + entry is this class itself, shown as plain text rather than a link to the page you are on. #} +
+{%- for ancestor in inheritance -%} +
{% if loop.last %}{{ ancestor.qualifiedName }}{% else %}{{ typeLink(ancestor) }}{% endif %} +{%- endfor -%} +{%- for ancestor in inheritance -%}
{%- endfor -%} +
+{% endif %} + +{% if typeParameters is not empty and typeParameters | first is not null %} +{% set documentedTypeParams = false %} +{% for t in typeParameters %}{% if t.description is not empty %}{% set documentedTypeParams = true %}{% endif %}{% endfor %} +{% if documentedTypeParams %} +
+
Type Parameters:
+{% for t in typeParameters %}{% if t.description is not empty %}
{{ t.name }} - {{ t.description | doc }}
{% endif %}{% endfor %} +
+{% endif %} +{% endif %} + +{% if allImplementedInterfaces is not empty %} +
All Implemented Interfaces:
{{ typeList(allImplementedInterfaces) }}
+{% endif %} +{% if allSuperinterfaces is not empty %} +
All Superinterfaces:
{{ typeList(allSuperinterfaces) }}
+{% endif %} +{% if allKnownSubinterfaces is not empty %} +
All Known Subinterfaces:
{{ typeList(allKnownSubinterfaces) }}
+{% endif %} +{% if allKnownImplementingClasses is not empty %} +
All Known Implementing Classes:
{{ typeList(allKnownImplementingClasses) }}
+{% endif %} +{% if directKnownSubclasses is not empty %} +
Direct Known Subclasses:
{{ typeList(directKnownSubclasses) }}
+{% endif %} +{% if enclosingType is not empty %} +
Enclosing {{ enclosingType.kind | default('class') }}:
{{ typeLink(enclosingType) }}
+{% endif %} +{% if isFunctionalInterface %} +
Functional Interface:
This is a functional interface and can therefore be used as the assignment target for a lambda expression or method reference.
+{% endif %} + +
+
{{ signature }}
+{% if description is not empty %}
{{ description | doc }}
{% endif %} +{% if deprecated is not empty %} +
Deprecated{% if deprecated.forRemoval %}, for removal: This API element is subject to removal in a future version{% endif %}. +{% if deprecated.since is not empty %}Since {{ deprecated.since }}.{% endif %} +{% if deprecated.comment is not empty %}
{{ deprecated.comment | doc }}
{% endif %} +
+{% endif %} +{% if since is not empty or seeAlso is not empty or tags is not empty or authors is not empty or versions is not empty %} +
+{% if since is not empty %}
Since:
{{ since | join(', ') }}
{% endif %} +{% for tag in tags %}
{{ tagLabel(tag.name) }}
{{ tag.text | doc }}
{% endfor %} +{% if authors is not empty %}
Author:
{{ authors | join(', ') }}
{% endif %} +{% if versions is not empty %}
Version:
{{ versions | join(', ') }}
{% endif %} +{% if seeAlso is not empty %} +
See Also:
+
    {% for see in seeAlso %}
  • {% if see.url is not empty %}{{ see.label }}{% else %}{{ see.label }}{% endif %}
    • {% endfor %}
+{% endif %} +
+{% endif %} +
+ +
+
    + +{% if nestedTypes is not empty or inheritedNestedTypes is not empty %} +
  • +
    +

    Nested Class Summary

    +{% if nestedTypes is not empty %} +
    Nested Classes
    +
    +
    Modifier and Type
    +
    Class
    +
    Description
    +{% for nested in nestedTypes %} +
    {{ nested.modifiers | join(' ') }} {{ nested.kind }}
    +
    {{ nested.name }}
    +
    {{ nested.firstSentence | doc }}
    +{% endfor %} +
    +{% endif %} +{% for group in inheritedNestedTypes %} +
    +

    Nested classes/interfaces declared in {{ group.declaringType.kind | default('class') }} {{ typeLink(group.declaringType) }}

    +{% for member in group.members %}{{ memberLink(member) }}{% if not loop.last %}, {% endif %}{% endfor %} +
    +{% endfor %} +
    +
    • +{% endif %} + +{% if enumConstants is not empty %} +
  • +
    +

    Enum Constant Summary

    +
    Enum Constants
    +
    +
    Enum Constant
    +
    Description
    +{% for field in enumConstants %} +
    {{ field.name }}
    +
    {{ field.firstSentence | doc }}
    +{% endfor %} +
    +
    +
    • +{% endif %} + +{% if fields is not empty or inheritedFields is not empty %} +
  • +
    +

    Field Summary

    +{% if fields is not empty %} +
    Fields
    +
    +
    Modifier and Type
    +
    Field
    +
    Description
    +{% for field in fields %} +
    {{ field.modifiers | join(' ') }} {{ typeLink(field.type) }}
    +
    {{ field.name }}
    +
    {{ field.firstSentence | doc }}
    +{% endfor %} +
    +{% endif %} +{% for group in inheritedFields %} +
    +

    Fields declared in {{ group.declaringType.kind | default('class') }} {{ typeLink(group.declaringType) }}

    +{% for member in group.members %}{{ memberLink(member) }}{% if not loop.last %}, {% endif %}{% endfor %} +
    +{% endfor %} +
    +
    • +{% endif %} + +{% if constructors is not empty %} +
  • +
    +

    Constructor Summary

    +
    Constructors
    +
    +
    Constructor
    +
    Description
    +{% for ctor in constructors %} +
    {{ ctor.name }}{{ parameters(ctor.parameters) }}
    +
    {{ ctor.firstSentence | doc }}
    +{% endfor %} +
    +
    +
    • +{% endif %} + +{% if annotationElements is not empty %} +
  • +
    +

    Element Summary

    +
    Elements
    +
    +
    Modifier and Type
    +
    Element
    +
    Description
    +{% for element in annotationElements %} +
    {{ typeLink(element.returnType) }}
    +
    {{ element.name }}
    +
    {{ element.firstSentence | doc }}
    +{% endfor %} +
    +
    +
    • +{% endif %} + +{% if methods is not empty or inheritedMethods is not empty %} +
  • +
    +

    Method Summary

    +{% if methods is not empty %} +
    All Methods
    +
    +
    Modifier and Type
    +
    Method
    +
    Description
    +{% for method in methods %} +
    {{ method.modifiers | join(' ') }} {{ typeLink(method.returnType) }}
    +
    {{ method.name }}{{ parameters(method.parameters) }}
    +
    {{ method.firstSentence | doc }}
    +{% endfor %} +
    +{% endif %} +{% for group in inheritedMethods %} +
    +

    Methods declared in {{ group.declaringType.kind | default('class') }} {{ typeLink(group.declaringType) }}

    +{% for member in group.members %}{{ memberLink(member) }}{% if not loop.last %}, {% endif %}{% endfor %} +
    +{% endfor %} +
    +
    • +{% endif %} + +
+
+ +
+
    + +{% if enumConstants is not empty %} +
  • +
    +

    Enum Constant Details

    +
      +{% for field in enumConstants %}{{ fieldDetail(field) }}{% endfor %} +
    +
    +
    • +{% endif %} + +{% if fields is not empty %} +
  • +
    +

    Field Details

    +
      +{% for field in fields %}{{ fieldDetail(field) }}{% endfor %} +
    +
    +
    • +{% endif %} + +{% if constructors is not empty %} +
  • +
    +

    Constructor Details

    +
      +{% for ctor in constructors %}{{ executableDetail(ctor) }}{% endfor %} +
    +
    +
    • +{% endif %} + +{% if annotationElements is not empty %} +
  • +
    +

    Element Details

    +
      +{% for element in annotationElements %}{{ executableDetail(element) }}{% endfor %} +
    +
    +
    • +{% endif %} + +{% if methods is not empty %} +
  • +
    +

    Method Details

    +
      +{% for method in methods %}{{ executableDetail(method) }}{% endfor %} +
    +
    +
    • +{% endif %} + +
+
+{% endblock %} diff --git a/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/constant-values.peb b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/constant-values.peb new file mode 100644 index 00000000..79a7bef2 --- /dev/null +++ b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/constant-values.peb @@ -0,0 +1,25 @@ +{% extends "base" %} +{% block title %}Constant Field Values{% endblock %} +{% block bodyClass %}constants-summary-page{% endblock %} +{% block content %} +

Constant Field Values

Contents

+{# Pebble iterates a map as entries: entry.key is the package, entry.value its types. #} +{% for group in packages %} +
+

{{ group.key }}

+{% for type in group.value %} +
{% if type.url is not empty %}{{ type.qualifiedName }}{% else %}{{ type.qualifiedName }}{% endif %}
+
+
Modifier and Type
+
Constant Field
+
Value
+{% for field in type.fields %} +
{{ field.modifiers | join(' ') }} {{ field.type.display }}
+
{% if field.url is not empty %}{{ field.name }}{% else %}{{ field.name }}{% endif %}
+
{{ field.value }}
+{% endfor %} +
+{% endfor %} +
+{% endfor %} +{% endblock %} diff --git a/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/deprecated-list.peb b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/deprecated-list.peb new file mode 100644 index 00000000..5489722e --- /dev/null +++ b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/deprecated-list.peb @@ -0,0 +1,44 @@ +{% extends "base" %} +{% block title %}Deprecated List{% endblock %} +{% block bodyClass %}deprecated-list-page{% endblock %} +{# The JSON keys each section by element kind; javadoc heads them with a title. #} +{% macro sectionTitle(kind) %} +{%- if kind == 'classes' -%}Deprecated Classes +{%- elseif kind == 'interfaces' -%}Deprecated Interfaces +{%- elseif kind == 'enums' -%}Deprecated Enum Classes +{%- elseif kind == 'exceptions' -%}Deprecated Exception Classes +{%- elseif kind == 'annotationTypes' -%}Deprecated Annotation Interfaces +{%- elseif kind == 'fields' -%}Deprecated Fields +{%- elseif kind == 'methods' -%}Deprecated Methods +{%- elseif kind == 'constructors' -%}Deprecated Constructors +{%- elseif kind == 'enumConstants' -%}Deprecated Enum Constants +{%- elseif kind == 'annotationElements' -%}Deprecated Annotation Elements +{%- else -%}Deprecated {{ kind }} +{%- endif -%} +{% endmacro %} + +{% block content %} +

Deprecated API

Contents

+
+{% if sections is empty %} +
No deprecated API in this documentation.
+{% endif %} +{# Pebble iterates a map as entries, so the section name is entry.key. #} +{% for section in sections %} +
+
{{ sectionTitle(section.key) }}
+
+
Element
+
Description
+{% for entry in section.value %} +
{% if entry.url is not empty %}{{ entry.element }}{% else %}{{ entry.element }}{% endif %}
+
+{% if entry.forRemoval %}Terminally deprecated.{% endif %} +{% if entry.since is not empty %}Since {{ entry.since }}.{% endif %} +{% if entry.comment is not empty %}
{{ entry.comment | doc }}
{% endif %} +
+{% endfor %} +
+
+{% endfor %} +{% endblock %} diff --git a/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/index-page.peb b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/index-page.peb new file mode 100644 index 00000000..b03e70d0 --- /dev/null +++ b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/index-page.peb @@ -0,0 +1,22 @@ +{% extends "base" %} +{% block title %}{{ letter }}-Index{% endblock %} +{% block bodyClass %}index-page{% endblock %} +{% block content %} +
+

Index

+
+{# Pebble's loop.index is 0-based; the index files are numbered from 1. #} +{% for l in letters %}{{ l }}{% if not loop.last %} {% endif %}{% endfor %} +
+
+

{{ letter }}

+
+{% for entry in entries %} +
{% if entry.url is not empty %}{{ entry.label }}{% else %}{{ entry.label }}{% endif %} +{% if entry.containingElement is not empty %} - {{ entry.kind }} in {{ entry.containingElement }}{% else %} - {{ entry.kind }}{% endif %} +{% if entry.deprecated %}Deprecated.{% endif %} +
+
{% if entry.firstSentence is not empty %}
{{ entry.firstSentence | doc }}
{% endif %}
+{% endfor %} +
+{% endblock %} diff --git a/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/macros.peb b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/macros.peb new file mode 100644 index 00000000..24d92a47 --- /dev/null +++ b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/macros.peb @@ -0,0 +1,158 @@ +{# + Shared fragments. + + Every macro here is called by BARE NAME, both from inside this file and from the templates that + import it. Pebble's {% import %} pulls macros straight into the importing template's namespace -- + unlike Jinja/Twig, there is no `macros.` prefix. A prefixed call resolves to nothing and renders + empty, silently, which is easy to miss. + Every one of these is defensive about missing keys: the plugin's `omitNulls` + option drops null and empty values entirely, so a template that assumes a key exists breaks + depending on how the JSON was generated. +#} + +{# + javadoc's display label for a block tag. The JSON carries the tag as written in the source + (`apiNote`), because that is the data; javadoc prints a spelled-out heading ("API Note:"). Any + tag not listed here falls back to its own name, which is what javadoc does for a custom tag. +#} +{% macro tagLabel(name) %} +{%- if name == 'apiNote' -%}API Note: +{%- elseif name == 'implSpec' -%}Implementation Requirements: +{%- elseif name == 'implNote' -%}Implementation Note: +{%- elseif name == 'jls' -%}See Java Language Specification: +{%- elseif name == 'jvms' -%}See Java Virtual Machine Specification: +{%- elseif name == 'serialData' -%}Serial Data: +{%- elseif name == 'serialField' -%}Serial Field: +{%- elseif name == 'serial' -%}Serial: +{%- elseif name == 'toolGuide' -%}Tool Guides: +{%- elseif name == 'revised' -%}Revised: +{%- elseif name == 'spec' -%}External Specifications: +{%- else -%}{{ name }}: +{%- endif -%} +{% endmacro %} + +{# A type reference: linked when this run documents the type, plain text when it doesn't. #} +{% macro typeLink(ref) %} +{%- if ref is not empty -%} +{%- if ref.url is not empty -%} +{{ ref.display }} +{%- else -%} +{{ ref.display }} +{%- endif -%} +{%- endif -%} +{% endmacro %} + +{# A comma-separated list of type references. #} +{% macro typeList(refs) %} +{%- for ref in refs -%}{{ typeLink(ref) }}{% if not loop.last %}, {% endif %}{%- endfor -%} +{% endmacro %} + +{# A member reference, as used by "Overrides:", "Specified by:" and the inherited-member lists. #} +{% macro memberLink(ref) %} +{%- if ref.url is not empty -%}{{ ref.name }}{%- else -%}{{ ref.name }}{%- endif -%} +{% endmacro %} + +{# One "dt/dd" note row, as javadoc renders Since / See Also / Overrides and friends. #} +{% macro note(label, body) %} +
{{ label }}
+
{{ body | raw }}
+{% endmacro %} + +{# The modifier prefix of a signature, e.g. "public static final". #} +{% macro modifiers(list) %} +{%- if list is not empty %}{{ list | join(' ') }} {% endif -%} +{% endmacro %} + +{# The parameter list of an executable, with linked parameter types. #} +{% macro parameters(params) %} +({%- for p in params -%}{{ typeLink(p.type) }} {{ p.name }}{% if not loop.last %}, {% endif %}{%- endfor -%}) +{% endmacro %} + +{# The `throws` clause of a signature. #} +{% macro throwsClause(exceptions) %} +{%- if exceptions is not empty %} throws {% for e in exceptions %}{{ typeLink(e.type) }}{% if not loop.last %}, {% endif %}{% endfor %}{% endif -%} +{% endmacro %} + +{# Zebra striping, which javadoc drives off the row index. #} +{% macro rowColor(index) %}{% if index is odd %}odd-row-color{% else %}even-row-color{% endif %}{% endmacro %} + +{# The block tags shared by every documented element: since, see also, deprecation, custom tags. #} +{% macro commonNotes(item) %} +{% if item.deprecated is not empty %} +
Deprecated{% if item.deprecated.forRemoval %}, for removal: This API element is subject to removal in a future version{% endif %}. +{% if item.deprecated.since is not empty %}Since {{ item.deprecated.since }}.{% endif %} +{% if item.deprecated.comment is not empty %}
{{ item.deprecated.comment | doc }}
{% endif %} +
+{% endif %} +{% if item.since is not empty or item.seeAlso is not empty or item.tags is not empty or item.authors is not empty or item.versions is not empty %} +
+{% if item.since is not empty %}
Since:
{{ item.since | join(', ') }}
{% endif %} +{% for tag in item.tags %}
{{ tagLabel(tag.name) }}
{{ tag.text | doc }}
{% endfor %} +{% if item.authors is not empty %}
Author:
{{ item.authors | join(', ') }}
{% endif %} +{% if item.versions is not empty %}
Version:
{{ item.versions | join(', ') }}
{% endif %} +{% if item.seeAlso is not empty %} +
See Also:
+
    {% for see in item.seeAlso %}
  • {% if see.url is not empty %}{{ see.label }}{% else %}{{ see.label }}{% endif %}
    • {% endfor %}
+{% endif %} +
+{% endif %} +{% endmacro %} + +{# One field/enum-constant entry in the Details section. #} +{% macro fieldDetail(field) %} +
  • +
    +

    {{ field.name }}

    +
    {{ modifiers(field.modifiers) }}{{ typeLink(field.type) }} {{ field.name }}
    +{% if field.description is not empty %}
    {{ field.description | doc }}
    {% endif %} +{{ commonNotes(field) }} +{% if field.constantValue is not empty %} +
    Constant Field Value:
    {{ field.constantValue }}
    +{% endif %} +
    +
    • +{% endmacro %} + +{# One constructor/method/annotation-element entry in the Details section. #} +{% macro executableDetail(member) %} +
  • +
    +

    {{ member.name }}

    +
    {{ modifiers(member.modifiers) }}{% if member.typeParameters is not empty %}<{% for t in member.typeParameters %}{{ t.name }}{% if not loop.last %},{% endif %}{% endfor %}> {% endif %}{% if member.returnType is not empty %}{{ typeLink(member.returnType) }} {% endif %}{{ member.name }}{{ parameters(member.parameters) }}{{ throwsClause(member.exceptions) }}
    +{% if member.description is not empty %}
    {{ member.description | doc }}
    {% endif %} +{% if member.specifiedBy is not empty or member.overrides is not empty or member.typeParameters is not empty or member.parameters is not empty or member.returns is not empty or member.exceptions is not empty %} +
    +{% for spec in member.specifiedBy %} +
    Specified by:
    +
    {{ memberLink(spec) }} in {{ spec.declaringType.kind | default('interface') }} {{ typeLink(spec.declaringType) }}
    +{% endfor %} +{% if member.overrides is not empty %} +
    Overrides:
    +
    {{ memberLink(member.overrides) }} in {{ member.overrides.declaringType.kind | default('class') }} {{ typeLink(member.overrides.declaringType) }}
    +{% endif %} +{% set hasTypeParamDocs = false %} +{% for t in member.typeParameters %}{% if t.description is not empty %}{% set hasTypeParamDocs = true %}{% endif %}{% endfor %} +{% if hasTypeParamDocs %} +
    Type Parameters:
    +{% for t in member.typeParameters %}{% if t.description is not empty %}
    {{ t.name }} - {{ t.description | doc }}
    {% endif %}{% endfor %} +{% endif %} +{% set hasParamDocs = false %} +{% for p in member.parameters %}{% if p.description is not empty %}{% set hasParamDocs = true %}{% endif %}{% endfor %} +{% if hasParamDocs %} +
    Parameters:
    +{% for p in member.parameters %}{% if p.description is not empty %}
    {{ p.name }} - {{ p.description | doc }}
    {% endif %}{% endfor %} +{% endif %} +{% if member.returns is not empty %}
    Returns:
    {{ member.returns | doc }}
    {% endif %} +{% if member.exceptions is not empty %} +
    Throws:
    +{% for e in member.exceptions %}
    {{ typeLink(e.type) }}{% if e.description is not empty %} - {{ e.description | doc }}{% endif %}
    {% endfor %} +{% endif %} +
    +{% endif %} +{% if member.defaultValue is not empty %} +
    Default:
    {{ member.defaultValue }}
    +{% endif %} +{{ commonNotes(member) }} +
    +
    • +{% endmacro %} diff --git a/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/module-summary.peb b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/module-summary.peb new file mode 100644 index 00000000..d6699a3a --- /dev/null +++ b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/module-summary.peb @@ -0,0 +1,157 @@ +{% extends "base" %} +{% import "macros" %} + +{% block title %}{{ name }}{% endblock %} +{% block bodyClass %}module-declaration-page{% endblock %} + +{% block subnav %} +
    1. {{ name }}
    +{% endblock %} + +{% block content %} +
    +

    Module {{ name }}

    +
    + +
    +{% if description is not empty %}
    {{ description | doc }}
    {% endif %} +{% if hasModuleGraph %} +
    +
    Module Graph:
    +
    Module graph for {{ name }}Module graph for {{ name }}
    +
    +{% endif %} +{% if since is not empty or tags is not empty or seeAlso is not empty %} +
    +{% if since is not empty %}
    Since:
    {{ since | join(', ') }}
    {% endif %} +{% for tag in tags %}{% if tag.name != 'moduleGraph' and tag.name != 'uses' and tag.name != 'provides' %}
    {{ tagLabel(tag.name) }}
    {{ tag.text | doc }}
    {% endif %}{% endfor %} +{% if seeAlso is not empty %}
    See Also:
      {% for see in seeAlso %}
    • {{ see.label }}
      • {% endfor %}
    {% endif %} +
    +{% endif %} +
    + +
    +
      + +{% if requires is not empty %} +
    • +
      +

      Modules

      +
      Requires
      +
      +
      Modifier
      +
      Module
      +
      Description
      +{% for req in requires %} +
      {% if req.isTransitive %}transitive{% endif %}{% if req.isStatic %}static{% endif %}
      +
      {% if req.url is not empty %}{{ req.module }}{% else %}{{ req.module }}{% endif %}
      +
      +{% endfor %} +
      +
      +
      • +{% endif %} + +{% if indirectRequires is not empty %} +
    • +
      +{% if requires is empty %}

      Modules

      {% endif %} +
      Indirect Requires
      +
      +
      Modifier
      +
      Module
      +
      Description
      +{% for req in indirectRequires %} +
      transitive
      +
      {% if req.url is not empty %}{{ req.module }}{% else %}{{ req.module }}{% endif %}
      +
      +{% endfor %} +
      +
      +
      • +{% endif %} + +{% if exports is not empty %} +
    • +
      +

      Packages

      +
      Exports
      +
      +
      Package
      +
      Exported To Modules
      +
      Description
      +{% for export in exports %} +
      {% if export.url is not empty %}{{ export.packageName }}{% else %}{{ export.packageName }}{% endif %}
      +
      {% if export.to is not empty %}{{ export.to | join(', ') }}{% else %}All Modules{% endif %}
      +
      {{ export.firstSentence | doc }}
      +{% endfor %} +
      +
      +
      • +{% endif %} + +{% if indirectExports is not empty %} +
    • +
      +
      Indirect Exports
      +
      +
      From
      +
      Packages
      +{% for entry in indirectExports %} +
      {% if entry.moduleUrl is not empty %}{{ entry.module }}{% else %}{{ entry.module }}{% endif %}
      +
      {% for pkg in entry.packages %}{% if pkg.url is not empty %}{{ pkg.name }}{% else %}{{ pkg.name }}{% endif %}{% if not loop.last %}, {% endif %}{% endfor %}
      +{% endfor %} +
      +
      +
      • +{% endif %} + +{% if opens is not empty %} +
    • +
      +

      Opens

      +
      +
      Package
      +
      Opened To Modules
      +{% for open in opens %} +
      {% if open.url is not empty %}{{ open.packageName }}{% else %}{{ open.packageName }}{% endif %}
      +
      {% if open.to is not empty %}{{ open.to | join(', ') }}{% else %}All Modules{% endif %}
      +{% endfor %} +
      +
      +
      • +{% endif %} + +{% if uses is not empty or provides is not empty %} +
    • +
      +

      Services

      +{% if uses is not empty %} +
      Uses
      +
      +
      Type
      +
      Description
      +{% for use in uses %} +
      {{ typeLink(use) }}
      +
      +{% endfor %} +
      +{% endif %} +{% if provides is not empty %} +
      Provides
      +
      +
      Type
      +
      Implementations
      +{% for provide in provides %} +
      {{ typeLink(provide.service) }}
      +
      {% for impl in provide.implementations %}{{ typeLink(impl) }}{% if not loop.last %}, {% endif %}{% endfor %}
      +{% endfor %} +
      +{% endif %} +
      +
      • +{% endif %} + +
    +
    +{% endblock %} diff --git a/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/overview.peb b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/overview.peb new file mode 100644 index 00000000..70aaa31b --- /dev/null +++ b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/overview.peb @@ -0,0 +1,40 @@ +{% extends "base" %} +{% block title %}Overview{% endblock %} +{% block bodyClass %}package-index-page{% endblock %} +{% block content %} +
    +

    {{ title | default('API Documentation') }}

    +
    +
    +{% if modules is not empty %} +
    +
    Modules
    +
    +
    Module
    +
    Description
    +{% for module in modules %} +
    {{ module.name }}
    +
    {{ module.firstSentence | doc }}
    +{% endfor %} +
    +
    +{% endif %} +{# Only for a non-modular run: with modules present javadoc's overview lists just + the modules, and allpackages-index.html carries the package list. #} +{% if packages is not empty and modules is empty %} +
    +
    Packages
    +
    +
    Module
    +
    Package
    +
    Description
    +{% for pkg in packages %} +
    {{ pkg.moduleName }}
    +
    {{ pkg.name }}
    +
    {{ pkg.firstSentence | doc }}
    +{% endfor %} +
    +
    +{% endif %} +
    +{% endblock %} diff --git a/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/package-summary.peb b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/package-summary.peb new file mode 100644 index 00000000..554fb1f1 --- /dev/null +++ b/Dokka-plugin-kdoc2json/pebble-renderer/src/main/resources/templates/package-summary.peb @@ -0,0 +1,84 @@ +{% extends "base" %} +{% import "macros" %} + +{% block title %}{{ name }}{% endblock %} +{% block bodyClass %}package-declaration-page{% endblock %} + +{% block subnav %} +
      +{% if moduleName is not empty %}
    1. {% if moduleUrl is not empty %}{{ moduleName }}{% else %}{{ moduleName }}{% endif %}
      2. {% endif %} +
    2. {{ name }}
      3. +
    +{% endblock %} + +{% macro typeTable(caption, rows) %} +{% if rows is not empty %} +
    {{ caption }}
    +
    +
    Class
    +
    Description
    +{% for row in rows %} +
    {{ row.name }}
    +
    {{ row.firstSentence | doc }}
    +{% endfor %} +
    +{% endif %} +{% endmacro %} + +{% block content %} +
    +{% if moduleName is not empty %} +
    Module {% if moduleUrl is not empty %}{{ moduleName }}{% else %}{{ moduleName }}{% endif %}
    +{% endif %} +

    Package {{ name }}

    +
    + +
    +
    package {{ name }}
    +{% if description is not empty %}
    {{ description | doc }}
    {% endif %} +{% if deprecated is not empty %} +
    Deprecated. +{% if deprecated.comment is not empty %}
    {{ deprecated.comment | doc }}
    {% endif %}
    +{% endif %} +{% if since is not empty or seeAlso is not empty or tags is not empty %} +
    +{% if since is not empty %}
    Since:
    {{ since | join(', ') }}
    {% endif %} +{% for tag in tags %}
    {{ tagLabel(tag.name) }}
    {{ tag.text | doc }}
    {% endfor %} +{% if seeAlso is not empty %}
    See Also:
      {% for see in seeAlso %}
    • {% if see.url is not empty %}{{ see.label }}{% else %}{{ see.label }}{% endif %}
      • {% endfor %}
    {% endif %} +
    +{% endif %} +
    + +
    +
      +{% if relatedPackages is not empty %} +
    • + +
      • +{% endif %} +
    • +
      +

      Classes and Interfaces

      +{{ typeTable("Interfaces", interfaces) }} +{{ typeTable("Classes", classes) }} +{{ typeTable("Enum Classes", enums) }} +{{ typeTable("Record Classes", records) }} +{{ typeTable("Exception Classes", exceptions) }} +{{ typeTable("Annotation Interfaces", annotationTypes) }} +
      +
      • +
    +
    +{% endblock %} diff --git a/Dokka-plugin-kdoc2json/scripts/java/build-jdk-json-docs.sh b/Dokka-plugin-kdoc2json/scripts/java/build-jdk-json-docs.sh new file mode 100755 index 00000000..9cfaab09 --- /dev/null +++ b/Dokka-plugin-kdoc2json/scripts/java/build-jdk-json-docs.sh @@ -0,0 +1,123 @@ +#!/usr/bin/env bash +# Generates the JDK's API documentation as javadoc-shaped JSON, mirroring the api/ tree of the +# official docs (the ones in SourceDocs/JavaDocs/html/api). +# +# Three steps: +# 1. stage_jdk_sources.py unpacks the JDK's lib/src.zip and keeps only what javadoc documents: +# one directory per JPMS module, containing that module's unqualified-exported packages. +# 2. jdk-docs/ runs Dokka over that tree with kdoc-to-json in javadoc-mode. +# 3. The result is copied to the output directory. +# +# The plugin reads each module's module-info.java back out of the staging tree, which is what +# gives the output its //.json layout and fills in the module pages' +# requires / exports / uses / provides sections. +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +ROOT_DIR="$(cd "$SCRIPT_DIR/../.." && pwd)" +PLUGIN_DIR="$ROOT_DIR/kdoc-to-json" +PROJECT_DIR="$SCRIPT_DIR/jdk-docs" + +usage() { + cat >&2 <] [-o ] [-w ] [-m ] [--skip-publish] + + -j JDK whose lib/src.zip to document. Defaults to \$JDK_SOURCE_HOME, else \$JAVA_HOME. + Use a JDK matching the docs you want to reproduce -- the docs under + SourceDocs/JavaDocs are Java SE 17. + -o Where to write the JSON tree. Default: $SCRIPT_DIR/build-output/api + -w Scratch directory for the extracted and staged sources. + Default: $SCRIPT_DIR/build-output/work + -m Comma-separated module names to document instead of all of them. Useful for a quick + check: -m java.sql,java.transaction.xa takes seconds rather than many minutes. + --skip-publish Don't republish kdoc-to-json to mavenLocal first. +USAGE + exit 1 +} + +JDK_HOME="${JDK_SOURCE_HOME:-${JAVA_HOME:-}}" +OUTPUT_DIR="$SCRIPT_DIR/build-output/api" +WORK_DIR="$SCRIPT_DIR/build-output/work" +MODULES="" +SKIP_PUBLISH=0 + +while [[ $# -gt 0 ]]; do + case "$1" in + -j) JDK_HOME="$2"; shift 2 ;; + -o) OUTPUT_DIR="$2"; shift 2 ;; + -w) WORK_DIR="$2"; shift 2 ;; + -m) MODULES="$2"; shift 2 ;; + --skip-publish) SKIP_PUBLISH=1; shift ;; + -h|--help) usage ;; + *) echo "Unknown argument: $1" >&2; usage ;; + esac +done + +if [[ -z "$JDK_HOME" ]]; then + echo "Error: no JDK given. Pass -j , or set JDK_SOURCE_HOME or JAVA_HOME." >&2 + exit 1 +fi + +SRC_ZIP="$JDK_HOME/lib/src.zip" +if [[ ! -f "$SRC_ZIP" ]]; then + echo "Error: $SRC_ZIP not found -- that JDK doesn't ship sources." >&2 + exit 1 +fi + +STAGING_DIR="$WORK_DIR/staged" +EXTRACT_DIR="$WORK_DIR/src-extracted" +mkdir -p "$WORK_DIR" + +echo "==> JDK sources: $SRC_ZIP" +"$JDK_HOME/bin/java" -version 2>&1 | head -1 | sed 's/^/ /' + +stage_args=("$SRC_ZIP" "$STAGING_DIR" --extract-to "$EXTRACT_DIR") +if [[ -n "$MODULES" ]]; then + stage_args+=(--modules "$MODULES") +fi +python3 "$SCRIPT_DIR/stage_jdk_sources.py" "${stage_args[@]}" + +if [[ "$SKIP_PUBLISH" != "1" ]]; then + echo "==> Publishing kdoc-to-json to mavenLocal" + (cd "$PLUGIN_DIR" && ./gradlew --console=plain -q publishToMavenLocal) +fi + +echo "==> Running Dokka in javadoc-mode over the staged sources" +echo " (the whole JDK is ~4,800 files across 60 modules; this takes a while)" +(cd "$PROJECT_DIR" && ./gradlew --console=plain dokkaGenerate -PjdkSources="$STAGING_DIR") + +GENERATED="$PROJECT_DIR/build/dokka/html" +if [[ ! -d "$GENERATED" ]]; then + echo "Error: Dokka produced no output at $GENERATED" >&2 + exit 1 +fi + +# Dokka only writes JSON, so the staged doc-files/ directories have to be carried across +# separately. The HTML renderer copies every non-JSON file through untouched, so putting them in +# the JSON tree is enough to get them into the rendered output too. +echo "==> Copying doc-files/ alongside the generated JSON" +doc_file_count=0 +while IFS= read -r dir; do + rel="${dir#"$STAGING_DIR"/}" + mkdir -p "$GENERATED/$rel" + cp -R "$dir"/. "$GENERATED/$rel"/ + doc_file_count=$((doc_file_count + 1)) +done < <(find "$STAGING_DIR" -type d -name doc-files) +echo " $doc_file_count doc-files directory/ies" + +echo "==> Copying output to $OUTPUT_DIR" +rm -rf "$OUTPUT_DIR" +mkdir -p "$(dirname "$OUTPUT_DIR")" +cp -R "$GENERATED" "$OUTPUT_DIR" + +json_count=$(find "$OUTPUT_DIR" -name '*.json' | wc -l | tr -d ' ') +module_count=$(find "$OUTPUT_DIR" -name 'module-summary.json' | wc -l | tr -d ' ') +package_count=$(find "$OUTPUT_DIR" -name 'package-summary.json' | wc -l | tr -d ' ') + +echo +echo "Done: $json_count JSON files -- $module_count modules, $package_count packages." +echo " Output: $OUTPUT_DIR" +echo " Plugin log: $PROJECT_DIR/build/dokka_json.log" +echo +echo "Compare against the official docs with:" +echo " python3 $SCRIPT_DIR/compare_with_javadoc.py $OUTPUT_DIR /SourceDocs/JavaDocs/html/api" diff --git a/Dokka-plugin-kdoc2json/scripts/java/compare_with_javadoc.py b/Dokka-plugin-kdoc2json/scripts/java/compare_with_javadoc.py new file mode 100755 index 00000000..8f5ee264 --- /dev/null +++ b/Dokka-plugin-kdoc2json/scripts/java/compare_with_javadoc.py @@ -0,0 +1,215 @@ +#!/usr/bin/env python3 +"""Compares a Javadoc-mode JSON tree against the official javadoc HTML it is meant to mirror. + +Reports, at each level of the api/ tree, what is in one side and not the other: + + modules -- directories with a module-summary page + packages -- directories with a package-summary page + types -- class/interface/enum/record/annotation pages + members -- the anchors on each type page (fields, constructors, methods) + +Member anchors are the sharpest check of the four: javadoc's anchor encodes a member's name and +its *erased* parameter types, so a matching anchor set means the two sides agree on the members, +their signatures, and their overload resolution -- not merely on the page count. + +Exit status is 0 when every level matches, 1 otherwise. +""" + +from __future__ import annotations + +import argparse +import json +import re +import sys +from pathlib import Path + +# javadoc's own non-API pages, which have no JSON counterpart by design. +NON_TYPE_PAGES = { + "package-summary", "package-tree", "package-use", "module-summary", "module-graph", + "allclasses-index", "allpackages-index", "constant-values", "deprecated-list", + "help-doc", "index", "overview-tree", "serialized-form", "system-properties", + "search", "new-list", "preview-list", +} +SKIP_DIRS = {"index-files", "class-use", "doc-files", "legal", "resources", "specs"} + +MEMBER_ANCHOR = re.compile(r'
    str: + return (text.replace("<", "<").replace(">", ">") + .replace(""", '"').replace("&", "&")) + + +def html_modules(api: Path) -> set[str]: + return {p.parent.name for p in api.glob("*/module-summary.html")} + + +def json_modules(out: Path) -> set[str]: + return {p.parent.name for p in out.glob("*/module-summary.json")} + + +def _relative_package(page: Path, root: Path) -> str | None: + rel = page.parent.relative_to(root) + parts = rel.parts + if not parts: + return None + # Drop the leading module directory when the tree is modular. + return ".".join(parts[1:]) if len(parts) > 1 else ".".join(parts) + + +def html_packages(api: Path, modular: bool) -> set[str]: + result = set() + for page in api.rglob("package-summary.html"): + if any(part in SKIP_DIRS for part in page.parts): + continue + parts = page.parent.relative_to(api).parts + result.add(".".join(parts[1:] if modular else parts)) + return result + + +def json_packages(out: Path, modular: bool) -> set[str]: + result = set() + for page in out.rglob("package-summary.json"): + parts = page.parent.relative_to(out).parts + result.add(".".join(parts[1:] if modular else parts)) + return result + + +def html_types(api: Path, modular: bool) -> set[str]: + result = set() + for page in api.rglob("*.html"): + if any(part in SKIP_DIRS for part in page.parts): + continue + if page.stem in NON_TYPE_PAGES: + continue + parts = page.parent.relative_to(api).parts + package = ".".join(parts[1:] if modular else parts) + if not package: + continue + result.add(f"{package}.{page.stem}") + return result + + +def json_types(out: Path, modular: bool) -> set[str]: + result = set() + for page in out.rglob("*.json"): + if any(part in SKIP_DIRS for part in page.parts): + continue + if page.stem in NON_TYPE_PAGES: + continue + parts = page.parent.relative_to(out).parts + package = ".".join(parts[1:] if modular else parts) + if not package: + continue + result.add(f"{package}.{page.stem}") + return result + + +def html_member_anchors(page: Path) -> set[str]: + text = page.read_text(encoding="utf-8", errors="replace") + return {unescape(a) for a in MEMBER_ANCHOR.findall(text)} + + +def json_member_anchors(page: Path) -> set[str]: + data = json.loads(page.read_text(encoding="utf-8")) + anchors = set() + for key in ("fields", "enumConstants", "constructors", "methods", "annotationElements"): + for member in data.get(key) or []: + anchors.add(member["anchor"]) + return anchors + + +def report(label: str, expected: set[str], actual: set[str], limit: int) -> bool: + missing = sorted(expected - actual) + extra = sorted(actual - expected) + status = "OK " if not missing and not extra else "DIFF" + print(f"[{status}] {label}: {len(actual)}/{len(expected)} " + f"(missing {len(missing)}, extra {len(extra)})") + for name in missing[:limit]: + print(f" missing: {name}") + if len(missing) > limit: + print(f" ... and {len(missing) - limit} more missing") + for name in extra[:limit]: + print(f" extra: {name}") + if len(extra) > limit: + print(f" ... and {len(extra) - limit} more extra") + return not missing and not extra + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + parser.add_argument("json_dir", type=Path, help="the generated javadoc-mode JSON tree") + parser.add_argument("html_dir", type=Path, help="the official javadoc api/ directory") + parser.add_argument("--limit", type=int, default=10, help="how many names to list per difference") + parser.add_argument("--members", action="store_true", + help="also compare the member anchors of every type (slower)") + args = parser.parse_args() + + for path in (args.json_dir, args.html_dir): + if not path.is_dir(): + print(f"Error: {path} is not a directory", file=sys.stderr) + return 2 + + ok = True + expected_modules = html_modules(args.html_dir) + actual_modules = json_modules(args.json_dir) + modular = bool(expected_modules) + ok &= report("modules", expected_modules, actual_modules, args.limit) + ok &= report("packages", + html_packages(args.html_dir, modular), + json_packages(args.json_dir, modular), args.limit) + + expected_types = html_types(args.html_dir, modular) + actual_types = json_types(args.json_dir, modular) + ok &= report("types", expected_types, actual_types, args.limit) + + if args.members: + shared = sorted(expected_types & actual_types) + html_index = {} + for page in args.html_dir.rglob("*.html"): + if page.stem in NON_TYPE_PAGES or any(p in SKIP_DIRS for p in page.parts): + continue + parts = page.parent.relative_to(args.html_dir).parts + package = ".".join(parts[1:] if modular else parts) + if package: + html_index[f"{package}.{page.stem}"] = page + json_index = {} + for page in args.json_dir.rglob("*.json"): + if page.stem in NON_TYPE_PAGES or any(p in SKIP_DIRS for p in page.parts): + continue + parts = page.parent.relative_to(args.json_dir).parts + package = ".".join(parts[1:] if modular else parts) + if package: + json_index[f"{package}.{page.stem}"] = page + + total = matched = 0 + differing: list[tuple[str, int, int]] = [] + for name in shared: + try: + expected = html_member_anchors(html_index[name]) + actual = json_member_anchors(json_index[name]) + except Exception as exc: # a malformed page shouldn't abort the whole comparison + print(f" error reading {name}: {exc}") + continue + total += 1 + if expected == actual: + matched += 1 + else: + differing.append((name, len(expected - actual), len(actual - expected))) + + print(f"[{'OK ' if matched == total else 'DIFF'}] member anchors: " + f"{matched}/{total} types match exactly") + for name, missing, extra in differing[:args.limit]: + print(f" {name}: missing {missing}, extra {extra}") + if len(differing) > args.limit: + print(f" ... and {len(differing) - args.limit} more types differ") + ok &= matched == total + + print() + print("MATCH" if ok else "DIFFERENCES FOUND") + return 0 if ok else 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/Dokka-plugin-kdoc2json/scripts/java/jdk-docs/build.gradle.kts b/Dokka-plugin-kdoc2json/scripts/java/jdk-docs/build.gradle.kts new file mode 100644 index 00000000..f432b8c4 --- /dev/null +++ b/Dokka-plugin-kdoc2json/scripts/java/jdk-docs/build.gradle.kts @@ -0,0 +1,86 @@ +import org.jetbrains.dokka.InternalDokkaApi +import org.jetbrains.dokka.gradle.engine.parameters.VisibilityModifier +import org.jetbrains.dokka.gradle.engine.plugins.DokkaPluginParametersBaseSpec +import javax.inject.Inject + +// Generates the JDK's API documentation as javadoc-shaped JSON. +// +// Sources come from a staging tree produced by ../stage_jdk_sources.py: one directory per JPMS +// module, containing only the packages that module exports unqualified -- which is exactly the +// set the `javadoc` tool documents. Each module directory is registered as its own source root, +// so it is a valid package root and so the plugin can attribute every declaration back to its +// module from `module-info.java`. +// +// Nothing is compiled here. Dokka only needs to *analyse* the sources, and the JDK is not +// buildable as an ordinary Gradle project; the java plugin is applied solely because Dokka's +// Gradle plugin hangs its source sets off one. +plugins { + id("org.jetbrains.dokka") version "2.2.0-Beta" +} + +repositories { + mavenLocal() + mavenCentral() +} + +dependencies { + dokkaPlugin("org.appdevforall.dokka:kdoc-to-json:1.0.0-SNAPSHOT") +} + +val stagedSources: String = (findProperty("jdkSources") as String?) + ?: error("Set -PjdkSources= (see scripts/java/stage_jdk_sources.py)") + +val stagedModules: List = file(stagedSources) + .listFiles { f: File -> f.isDirectory && File(f, "module-info.java").isFile } + ?.sortedBy { it.name } + ?: error("No JPMS module directories found under $stagedSources") + +@OptIn(InternalDokkaApi::class) +abstract class JsonOutputPluginParameters @Inject constructor( + name: String +) : DokkaPluginParametersBaseSpec(name, "org.appdevforall.dokka.kdoc2json.JsonOutputPlugin") { + override fun jsonEncode(): String { + System.getenv("KDOC2JSON_TEST_CONFIG")?.let { return File(it).readText() } + return """{ + "logLevel": "info", + "logFile": "build/dokka_json.log", + "javadoc-mode": true, + "omitNulls": true + }""" + } +} + +dokka { + moduleName.set("jdk") + + // Dokka generates in a *worker*, not in the Gradle daemon, so `org.gradle.jvmargs` in + // gradle.properties does not size it -- the worker inherits a default heap and, analysing the + // whole JDK in one pass, dies with an OutOfMemoryError at around 2.5 GB. Give it a process of + // its own with room to work. + dokkaGeneratorIsolation.set( + ProcessIsolation { + maxHeapSize.set(providers.gradleProperty("dokkaWorkerHeap").orElse("24g")) + // The JDK's deeply generic types drive Dokka's analysis into recursion far past what + // the default ~1 MB thread stack survives -- without this it dies with a + // StackOverflowError about a minute in. + jvmArgs.add(providers.gradleProperty("dokkaWorkerStack").orElse("-Xss64m")) + } + ) + + dokkaSourceSets.register("jdk") { + sourceRoots.from(stagedModules) + // javadoc documents public and protected members; Dokka defaults to public only. + documentedVisibilities.set(setOf(VisibilityModifier.Public, VisibilityModifier.Protected)) + // The JDK's own sources are the whole API surface -- there is nothing to link out to. + enableJdkDocumentationLink.set(false) + enableKotlinStdLibDocumentationLink.set(false) + jdkVersion.set(17) + } + + pluginsConfiguration { + registerBinding(JsonOutputPluginParameters::class, JsonOutputPluginParameters::class) + register("org.appdevforall.dokka.kdoc2json.JsonOutputPlugin") { } + } +} + +logger.lifecycle("jdk-api-docs: ${stagedModules.size} module source roots from $stagedSources") diff --git a/Dokka-plugin-kdoc2json/scripts/java/jdk-docs/gradle.properties b/Dokka-plugin-kdoc2json/scripts/java/jdk-docs/gradle.properties new file mode 100644 index 00000000..3773a24d --- /dev/null +++ b/Dokka-plugin-kdoc2json/scripts/java/jdk-docs/gradle.properties @@ -0,0 +1,3 @@ +# The Dokka *worker* does the heavy lifting and is sized by dokkaGeneratorIsolation in +# build.gradle.kts, not from here. The daemon itself only needs enough to run the build. +org.gradle.jvmargs=-Xmx4g -XX:MaxMetaspaceSize=1g diff --git a/Dokka-plugin-kdoc2json/scripts/java/jdk-docs/gradle/wrapper/gradle-wrapper.jar b/Dokka-plugin-kdoc2json/scripts/java/jdk-docs/gradle/wrapper/gradle-wrapper.jar new file mode 100644 index 00000000..1b33c55b Binary files /dev/null and b/Dokka-plugin-kdoc2json/scripts/java/jdk-docs/gradle/wrapper/gradle-wrapper.jar differ diff --git a/Dokka-plugin-kdoc2json/scripts/java/jdk-docs/gradle/wrapper/gradle-wrapper.properties b/Dokka-plugin-kdoc2json/scripts/java/jdk-docs/gradle/wrapper/gradle-wrapper.properties new file mode 100644 index 00000000..df6a6ad7 --- /dev/null +++ b/Dokka-plugin-kdoc2json/scripts/java/jdk-docs/gradle/wrapper/gradle-wrapper.properties @@ -0,0 +1,9 @@ +distributionBase=GRADLE_USER_HOME +distributionPath=wrapper/dists +distributionUrl=https\://services.gradle.org/distributions/gradle-9.5.1-bin.zip +networkTimeout=10000 +retries=0 +retryBackOffMs=500 +validateDistributionUrl=true +zipStoreBase=GRADLE_USER_HOME +zipStorePath=wrapper/dists diff --git a/Dokka-plugin-kdoc2json/scripts/java/jdk-docs/gradlew b/Dokka-plugin-kdoc2json/scripts/java/jdk-docs/gradlew new file mode 100755 index 00000000..b9bb139f --- /dev/null +++ b/Dokka-plugin-kdoc2json/scripts/java/jdk-docs/gradlew @@ -0,0 +1,248 @@ +#!/bin/sh + +# +# Copyright © 2015 the original authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# SPDX-License-Identifier: Apache-2.0 +# + +############################################################################## +# +# Gradle start up script for POSIX generated by Gradle. +# +# Important for running: +# +# (1) You need a POSIX-compliant shell to run this script. If your /bin/sh is +# noncompliant, but you have some other compliant shell such as ksh or +# bash, then to run this script, type that shell name before the whole +# command line, like: +# +# ksh Gradle +# +# Busybox and similar reduced shells will NOT work, because this script +# requires all of these POSIX shell features: +# * functions; +# * expansions «$var», «${var}», «${var:-default}», «${var+SET}», +# «${var#prefix}», «${var%suffix}», and «$( cmd )»; +# * compound commands having a testable exit status, especially «case»; +# * various built-in commands including «command», «set», and «ulimit». +# +# Important for patching: +# +# (2) This script targets any POSIX shell, so it avoids extensions provided +# by Bash, Ksh, etc; in particular arrays are avoided. +# +# The "traditional" practice of packing multiple parameters into a +# space-separated string is a well documented source of bugs and security +# problems, so this is (mostly) avoided, by progressively accumulating +# options in "$@", and eventually passing that to Java. +# +# Where the inherited environment variables (DEFAULT_JVM_OPTS, JAVA_OPTS, +# and GRADLE_OPTS) rely on word-splitting, this is performed explicitly; +# see the in-line comments for details. +# +# There are tweaks for specific operating systems such as AIX, CygWin, +# Darwin, MinGW, and NonStop. +# +# (3) This script is generated from the Groovy template +# https://github.com/gradle/gradle/blob/3d91ce3b8caaf77ad09f381f43615b715b53f72c/platforms/jvm/plugins-application/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt +# within the Gradle project. +# +# You can find Gradle at https://github.com/gradle/gradle/. +# +############################################################################## + +# Attempt to set APP_HOME + +# Resolve links: $0 may be a link +app_path=$0 + +# Need this for daisy-chained symlinks. +while + APP_HOME=${app_path%"${app_path##*/}"} # leaves a trailing /; empty if no leading path + [ -h "$app_path" ] +do + ls=$( ls -ld "$app_path" ) + link=${ls#*' -> '} + case $link in #( + /*) app_path=$link ;; #( + *) app_path=$APP_HOME$link ;; + esac +done + +# This is normally unused +# shellcheck disable=SC2034 +APP_BASE_NAME=${0##*/} +# Discard cd standard output in case $CDPATH is set (https://github.com/gradle/gradle/issues/25036) +APP_HOME=$( cd -P "${APP_HOME:-./}" > /dev/null && printf '%s\n' "$PWD" ) || exit + +# Use the maximum available, or set MAX_FD != -1 to use that value. +MAX_FD=maximum + +warn () { + echo "$*" +} >&2 + +die () { + echo + echo "$*" + echo + exit 1 +} >&2 + +# OS specific support (must be 'true' or 'false'). +cygwin=false +msys=false +darwin=false +nonstop=false +case "$( uname )" in #( + CYGWIN* ) cygwin=true ;; #( + Darwin* ) darwin=true ;; #( + MSYS* | MINGW* ) msys=true ;; #( + NONSTOP* ) nonstop=true ;; +esac + + + +# Determine the Java command to use to start the JVM. +if [ -n "$JAVA_HOME" ] ; then + if [ -x "$JAVA_HOME/jre/sh/java" ] ; then + # IBM's JDK on AIX uses strange locations for the executables + JAVACMD=$JAVA_HOME/jre/sh/java + else + JAVACMD=$JAVA_HOME/bin/java + fi + if [ ! -x "$JAVACMD" ] ; then + die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME + +Please set the JAVA_HOME variable in your environment to match the +location of your Java installation." + fi +else + JAVACMD=java + if ! command -v java >/dev/null 2>&1 + then + die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. + +Please set the JAVA_HOME variable in your environment to match the +location of your Java installation." + fi +fi + +# Increase the maximum file descriptors if we can. +if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then + case $MAX_FD in #( + max*) + # In POSIX sh, ulimit -H is undefined. That's why the result is checked to see if it worked. + # shellcheck disable=SC2039,SC3045 + MAX_FD=$( ulimit -H -n ) || + warn "Could not query maximum file descriptor limit" + esac + case $MAX_FD in #( + '' | soft) :;; #( + *) + # In POSIX sh, ulimit -n is undefined. That's why the result is checked to see if it worked. + # shellcheck disable=SC2039,SC3045 + ulimit -n "$MAX_FD" || + warn "Could not set maximum file descriptor limit to $MAX_FD" + esac +fi + +# Collect all arguments for the java command, stacking in reverse order: +# * args from the command line +# * the main class name +# * -classpath +# * -D...appname settings +# * --module-path (only if needed) +# * DEFAULT_JVM_OPTS, JAVA_OPTS, and GRADLE_OPTS environment variables. + +# For Cygwin or MSYS, switch paths to Windows format before running java +if "$cygwin" || "$msys" ; then + APP_HOME=$( cygpath --path --mixed "$APP_HOME" ) + + JAVACMD=$( cygpath --unix "$JAVACMD" ) + + # Now convert the arguments - kludge to limit ourselves to /bin/sh + for arg do + if + case $arg in #( + -*) false ;; # don't mess with options #( + /?*) t=${arg#/} t=/${t%%/*} # looks like a POSIX filepath + [ -e "$t" ] ;; #( + *) false ;; + esac + then + arg=$( cygpath --path --ignore --mixed "$arg" ) + fi + # Roll the args list around exactly as many times as the number of + # args, so each arg winds up back in the position where it started, but + # possibly modified. + # + # NB: a `for` loop captures its iteration list before it begins, so + # changing the positional parameters here affects neither the number of + # iterations, nor the values presented in `arg`. + shift # remove old arg + set -- "$@" "$arg" # push replacement arg + done +fi + + +# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. +DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"' + +# Collect all arguments for the java command: +# * DEFAULT_JVM_OPTS, JAVA_OPTS, and optsEnvironmentVar are not allowed to contain shell fragments, +# and any embedded shellness will be escaped. +# * For example: A user cannot expect ${Hostname} to be expanded, as it is an environment variable and will be +# treated as '${Hostname}' itself on the command line. + +set -- \ + "-Dorg.gradle.appname=$APP_BASE_NAME" \ + -jar "$APP_HOME/gradle/wrapper/gradle-wrapper.jar" \ + "$@" + +# Stop when "xargs" is not available. +if ! command -v xargs >/dev/null 2>&1 +then + die "xargs is not available" +fi + +# Use "xargs" to parse quoted args. +# +# With -n1 it outputs one arg per line, with the quotes and backslashes removed. +# +# In Bash we could simply go: +# +# readarray ARGS < <( xargs -n1 <<<"$var" ) && +# set -- "${ARGS[@]}" "$@" +# +# but POSIX shell has neither arrays nor command substitution, so instead we +# post-process each arg (as a line of input to sed) to backslash-escape any +# character that might be a shell metacharacter, then use eval to reverse +# that process (while maintaining the separation between arguments), and wrap +# the whole thing up as a single "set" statement. +# +# This will of course break if any of these variables contains a newline or +# an unmatched quote. +# + +eval "set -- $( + printf '%s\n' "$DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS" | + xargs -n1 | + sed ' s~[^-[:alnum:]+,./:=@_]~\\&~g; ' | + tr '\n' ' ' + )" '"$@"' + +exec "$JAVACMD" "$@" diff --git a/Dokka-plugin-kdoc2json/scripts/java/jdk-docs/gradlew.bat b/Dokka-plugin-kdoc2json/scripts/java/jdk-docs/gradlew.bat new file mode 100644 index 00000000..aa5f10b0 --- /dev/null +++ b/Dokka-plugin-kdoc2json/scripts/java/jdk-docs/gradlew.bat @@ -0,0 +1,82 @@ +@rem +@rem Copyright 2015 the original author or authors. +@rem +@rem Licensed under the Apache License, Version 2.0 (the "License"); +@rem you may not use this file except in compliance with the License. +@rem You may obtain a copy of the License at +@rem +@rem https://www.apache.org/licenses/LICENSE-2.0 +@rem +@rem Unless required by applicable law or agreed to in writing, software +@rem distributed under the License is distributed on an "AS IS" BASIS, +@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +@rem See the License for the specific language governing permissions and +@rem limitations under the License. +@rem +@rem SPDX-License-Identifier: Apache-2.0 +@rem + +@if "%DEBUG%"=="" @echo off +@rem ########################################################################## +@rem +@rem Gradle startup script for Windows +@rem +@rem ########################################################################## + +@rem Set local scope for the variables, and ensure extensions are enabled +setlocal EnableExtensions + +set DIRNAME=%~dp0 +if "%DIRNAME%"=="" set DIRNAME=. +@rem This is normally unused +set APP_BASE_NAME=%~n0 +set APP_HOME=%DIRNAME% + +@rem Resolve any "." and ".." in APP_HOME to make it shorter. +for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi + +@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. +set DEFAULT_JVM_OPTS="-Xmx64m" "-Xms64m" + +@rem Find java.exe +if defined JAVA_HOME goto findJavaFromJavaHome + +set JAVA_EXE=java.exe +%JAVA_EXE% -version >NUL 2>&1 +if %ERRORLEVEL% equ 0 goto execute + +echo. 1>&2 +echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. 1>&2 +echo. 1>&2 +echo Please set the JAVA_HOME variable in your environment to match the 1>&2 +echo location of your Java installation. 1>&2 + +"%COMSPEC%" /c exit 1 + +:findJavaFromJavaHome +set JAVA_HOME=%JAVA_HOME:"=% +set JAVA_EXE=%JAVA_HOME%/bin/java.exe + +if exist "%JAVA_EXE%" goto execute + +echo. 1>&2 +echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% 1>&2 +echo. 1>&2 +echo Please set the JAVA_HOME variable in your environment to match the 1>&2 +echo location of your Java installation. 1>&2 + +"%COMSPEC%" /c exit 1 + +:execute +@rem Setup the command line + + + +@rem Execute Gradle +@rem endlocal doesn't take effect until after the line is parsed and variables are expanded +@rem which allows us to clear the local environment before executing the java command +endlocal & "%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -jar "%APP_HOME%\gradle\wrapper\gradle-wrapper.jar" %* & call :exitWithErrorLevel + +:exitWithErrorLevel +@rem Use "%COMSPEC%" /c exit to allow operators to work properly in scripts +"%COMSPEC%" /c exit %ERRORLEVEL% diff --git a/Dokka-plugin-kdoc2json/scripts/java/jdk-docs/settings.gradle.kts b/Dokka-plugin-kdoc2json/scripts/java/jdk-docs/settings.gradle.kts new file mode 100644 index 00000000..ae169cfc --- /dev/null +++ b/Dokka-plugin-kdoc2json/scripts/java/jdk-docs/settings.gradle.kts @@ -0,0 +1 @@ +rootProject.name = "jdk-api-docs" diff --git a/Dokka-plugin-kdoc2json/scripts/java/stage_jdk_sources.py b/Dokka-plugin-kdoc2json/scripts/java/stage_jdk_sources.py new file mode 100755 index 00000000..f9adcde9 --- /dev/null +++ b/Dokka-plugin-kdoc2json/scripts/java/stage_jdk_sources.py @@ -0,0 +1,202 @@ +#!/usr/bin/env python3 +"""Stages the JDK's *documented* sources for a Javadoc-mode Dokka run. + +`src.zip` ships every source file in the JDK, including internal packages that the official API +docs deliberately leave out. javadoc's rule is exact and easy to reproduce: a package appears in +`api/` if and only if its module `exports` it *unqualified* (an `exports ... to ...` directive is +a targeted export and is not documented). Verified against the JDK 17 docs in +SourceDocs/JavaDocs: for java.base, the 53 unqualified exports are precisely the 53 documented +packages, with nothing on either side left over. + +So this script copies, per module, only the source files of unqualified-exported packages, plus +that module's `module-info.java` (which the plugin reads back for the module page's requires / +uses / provides / exports sections). The result is a tree of one directory per module, each a +valid package root: + + /java.base/module-info.java + /java.base/java/lang/Object.java + /java.sql/java/sql/Connection.java + +Everything left behind still resolves at analysis time from the JDK on the compile classpath, so +dropping it costs nothing but analysis time. +""" + +from __future__ import annotations + +import argparse +import re +import shutil +import sys +import zipfile +from pathlib import Path + +# `exports ;` with no `to` clause. The `to` form is a qualified export -- visible only to the +# named modules, and never part of the published API docs. +UNQUALIFIED_EXPORT = re.compile(r"^\s*exports\s+([\w.]+)\s*;", re.MULTILINE) + +# Dokka's {@inheritDoc} resolver recurses without bound on parts of the JDK +# (InheritDocTagResolver.resolveThrowsTag -> PsiElementToHtmlConverter.toInheritDocHtml) and brings +# the whole run down with a StackOverflowError -- reproducibly, on java.io and java.util among +# others. Rewriting the tag to an inert text marker before Dokka parses it avoids that; the plugin +# then resolves the marker itself, walking the same supertype chain javadoc walks. Nothing is lost: +# 3,214 occurrences across the JDK are still resolved, just by us instead of by Dokka. +INHERIT_DOC = re.compile(r"\{@inheritDoc\}") +INHERIT_DOC_MARKER = "ADFAINHERITDOC" # must match JavadocMapper.INHERIT_DOC_MARKER + +BLOCK_COMMENT = re.compile(r"/\*.*?\*/", re.DOTALL) +LINE_COMMENT = re.compile(r"//[^\n]*") +MODULE_DECL = re.compile(r"\bmodule\s+([\w.]+)\s*\{", re.MULTILINE) + +# Modules the JDK's own docs build leaves out (make/Docs.gmk's MODULES_FILTER). They ship in +# src.zip but never appear in api/, so staging them would produce module pages the official docs +# don't have. Verified against the JDK 17 docs: excluding exactly these makes the staged module +# set identical to the documented one. +EXCLUDED_MODULE_PREFIXES = ("jdk.internal.",) +EXCLUDED_MODULES = frozenset({"jdk.unsupported", "jdk.unsupported.desktop", "jdk.random"}) + + +def is_excluded(name: str) -> bool: + return name in EXCLUDED_MODULES or name.startswith(EXCLUDED_MODULE_PREFIXES) + + +def strip_comments(source: str) -> str: + """Removes comments so a commented-out directive is never mistaken for a live one.""" + return LINE_COMMENT.sub("", BLOCK_COMMENT.sub("", source)) + + +def parse_module_info(path: Path) -> tuple[str, list[str]]: + """Returns (module name, unqualified exported packages) for one module-info.java.""" + body = strip_comments(path.read_text(encoding="utf-8", errors="replace")) + match = MODULE_DECL.search(body) + name = match.group(1) if match else path.parent.name + return name, sorted(set(UNQUALIFIED_EXPORT.findall(body))) + + +def extract_sources(src_zip: Path, destination: Path) -> None: + if destination.exists() and any(destination.iterdir()): + print(f" reusing already-extracted sources at {destination}") + return + destination.mkdir(parents=True, exist_ok=True) + with zipfile.ZipFile(src_zip) as archive: + archive.extractall(destination) + + +def copy_source(source: Path, target: Path, rewrite_inherit_doc: bool) -> None: + """Copies one .java file, optionally neutralising {@inheritDoc} on the way through.""" + if not rewrite_inherit_doc: + shutil.copy2(source, target) + return + text = source.read_text(encoding="utf-8", errors="replace") + rewritten = INHERIT_DOC.sub(INHERIT_DOC_MARKER, text) + if rewritten == text: + shutil.copy2(source, target) + else: + target.write_text(rewritten, encoding="utf-8") + + +def stage( + extracted: Path, + staging: Path, + only: set[str] | None, + excluded: set[str], + rewrite_inherit_doc: bool = True, +) -> tuple[int, int, int]: + if staging.exists(): + shutil.rmtree(staging) + staging.mkdir(parents=True) + + modules = staged_packages = staged_files = 0 + skipped: list[str] = [] + + for module_info in sorted(extracted.glob("*/module-info.java")): + module_dir = module_info.parent + name, exports = parse_module_info(module_info) + if only and name not in only: + continue + if not only and (is_excluded(name) or name in excluded): + skipped.append(name) + continue + + target_module = staging / name + target_module.mkdir(parents=True, exist_ok=True) + shutil.copy2(module_info, target_module / "module-info.java") + modules += 1 + + for package in exports: + source_package = module_dir / Path(*package.split(".")) + if not source_package.is_dir(): + # A package can be exported by one module but live in another's directory in + # src.zip (or not ship sources at all); skip rather than fail the whole run. + print(f" warning: {name} exports {package}, but no sources found", file=sys.stderr) + continue + target_package = target_module / Path(*package.split(".")) + target_package.mkdir(parents=True, exist_ok=True) + # Non-recursive on purpose: a Java package is exactly one directory, and a + # subdirectory is a *different* package that must be exported in its own right. + files = [f for f in source_package.iterdir() if f.suffix == ".java" and f.is_file()] + for java_file in files: + copy_source(java_file, target_package / java_file.name, rewrite_inherit_doc) + # javadoc copies each package's doc-files/ directory into the output verbatim -- + # supplementary pages the comments link to (java.lang/doc-files/ValueBased.html and + # the like). They are staged here so the build can copy them through, or those links + # land on nothing. + doc_files = source_package / "doc-files" + if doc_files.is_dir(): + shutil.copytree(doc_files, target_package / "doc-files", dirs_exist_ok=True) + + staged_packages += 1 + staged_files += len(files) + + if skipped: + print(f" skipped {len(skipped)} undocumented module(s): {', '.join(sorted(skipped))}") + return modules, staged_packages, staged_files + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + parser.add_argument("src_zip", type=Path, help="path to the JDK's lib/src.zip") + parser.add_argument("staging", type=Path, help="directory to write the staged source tree to") + parser.add_argument("--extract-to", type=Path, default=None, + help="where to unpack src.zip (default: /../src-extracted)") + parser.add_argument("--modules", default=None, + help="comma-separated module names to stage; overrides the exclusion " + "list, so naming an excluded module stages it (default: all " + "documented modules)") + parser.add_argument("--keep-inherit-doc", action="store_true", + help="leave {@inheritDoc} tags as they are. Dokka's own resolver crashes " + "with a StackOverflowError on much of the JDK, so this is expected to " + "fail; it exists to re-check whether a newer Dokka has fixed the bug") + parser.add_argument("--exclude-modules", default="", + help="extra comma-separated module names to leave out, on top of the " + "ones the JDK's own docs build filters") + args = parser.parse_args() + + if not args.src_zip.is_file(): + print(f"Error: {args.src_zip} not found", file=sys.stderr) + return 1 + + extracted = args.extract_to or args.staging.parent / "src-extracted" + only = {m.strip() for m in args.modules.split(",")} if args.modules else None + excluded = {m.strip() for m in args.exclude_modules.split(",") if m.strip()} + + print(f"==> Extracting {args.src_zip}") + extract_sources(args.src_zip, extracted) + + print(f"==> Staging exported packages into {args.staging}") + modules, packages, files = stage( + extracted, args.staging, only, excluded, rewrite_inherit_doc=not args.keep_inherit_doc + ) + + if modules == 0: + print("Error: no modules staged -- is this a modular JDK's src.zip?", file=sys.stderr) + return 1 + + print(f" {modules} modules, {packages} exported packages, {files} source files") + if not args.keep_inherit_doc: + print(" {@inheritDoc} rewritten to an inert marker; the plugin resolves it (see the " + "comment on INHERIT_DOC above)") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/Dokka-plugin-kdoc2json/tests/lib.sh b/Dokka-plugin-kdoc2json/tests/lib.sh index 44a881f3..66ce6884 100755 --- a/Dokka-plugin-kdoc2json/tests/lib.sh +++ b/Dokka-plugin-kdoc2json/tests/lib.sh @@ -10,6 +10,10 @@ ROOT_DIR="$(cd "$TESTS_DIR/.." && pwd)" PLUGIN_DIR="$ROOT_DIR/kdoc-to-json" EXAMPLE_DIR="$ROOT_DIR/examples/example-data-processor" OUTPUT_DIR="$EXAMPLE_DIR/build/dokka/html" +# Javadoc mode mirrors the output of the `javadoc` tool, so it is exercised against a +# Java-only example rather than the Kotlin one every other test uses. +JAVA_EXAMPLE_DIR="$ROOT_DIR/examples/example-java-library" +JAVA_OUTPUT_DIR="$JAVA_EXAMPLE_DIR/build/dokka/html" TMP_DIR="$(mktemp -d /tmp/kdoc2json_test.XXXXXX)" trap 'rm -rf "$TMP_DIR"' EXIT @@ -58,6 +62,25 @@ run_dokka() { LAST_GRADLE_LOG="$gradle_log" } +# run_dokka_java '' is run_dokka against examples/example-java-library, the +# Java-source example Javadoc mode is tested with. Afterwards $JAVA_OUTPUT_DIR reflects only +# this run. Aborts the whole script if the Dokka build itself fails, like run_dokka. +run_dokka_java() { + local config_json="$1" + local config_file="$TMP_DIR/config-java-$RANDOM.json" + local gradle_log="$TMP_DIR/gradle-java-$RANDOM.log" + printf '%s' "$config_json" >"$config_file" + + rm -rf "$JAVA_EXAMPLE_DIR/build/dokka" + + if ! (cd "$JAVA_EXAMPLE_DIR" && KDOC2JSON_TEST_CONFIG="$config_file" ./gradlew --console=plain dokkaGenerate) >"$gradle_log" 2>&1; then + echo "FATAL: dokkaGenerate failed for config: $config_json" >&2 + cat "$gradle_log" >&2 + exit 1 + fi + LAST_GRADLE_LOG="$gradle_log" +} + # run_dokka_expect_failure '' is run_dokka's counterpart for tests # that assert the build SHOULD fail (e.g. a genuinely malformed config, or a # classDiscriminator collision). Never aborts the script on a Dokka failure -- @@ -171,6 +194,32 @@ assert_no_local_html_urls() { fi } +# assert_json evaluates a Python expression +# against the parsed JSON document (bound to `d`) and compares its str() to . Lets a +# test assert on structure -- a field's value, a list's contents -- instead of grepping for a +# substring that might match somewhere unrelated in the file. +assert_json() { + local path="$1" expr="$2" expected="$3" desc="$4" + local actual + actual=$(python3 -c " +import json, sys +try: + d = json.load(open(sys.argv[1])) +except Exception as e: + print('' % e) + sys.exit(0) +try: + print($expr) +except Exception as e: + print('' % e) +" "$path" 2>/dev/null) + if [[ "$actual" == "$expected" ]]; then + pass "$desc" + else + fail "$desc (expected '$expected', got '$actual')" + fi +} + assert_gt() { local actual="$1" threshold="$2" desc="$3" if [[ "$actual" -gt "$threshold" ]]; then diff --git a/Dokka-plugin-kdoc2json/tests/test_javadoc_mode.sh b/Dokka-plugin-kdoc2json/tests/test_javadoc_mode.sh new file mode 100755 index 00000000..176106aa --- /dev/null +++ b/Dokka-plugin-kdoc2json/tests/test_javadoc_mode.sh @@ -0,0 +1,206 @@ +#!/usr/bin/env bash +# Exercises the "javadoc-mode" config option against examples/example-java-library. +# +# Javadoc mode's contract is that the output *mirrors the javadoc tool's own api/ tree* -- both +# where files land and what each page contains -- so these assertions are written against real +# javadoc behaviour: package directories rather than Dokka's `com.example.shapes/-rectangle/` +# layout, `(double,double)` member anchors, an inheritance closure, inherited-member groups, +# and the global index files (allclasses-index, deprecated-list, constant-values, index-files, +# element-list). +set -uo pipefail + +TESTS_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +source "$TESTS_DIR/lib.sh" + +publish_plugin + +JD_ON='{"logLevel":"debug","javadoc-mode":true,"prettyPrint":true}' +JD_OFF='{"logLevel":"debug","prettyPrint":true}' + +echo "==> javadoc-mode disabled: output keeps Dokka's own layout" +run_dokka_java "$JD_OFF" +assert_file_exists "$JAVA_OUTPUT_DIR/com.example.shapes/-rectangle/index.json" \ + "default mode still writes Dokka-shaped pages" +assert_file_not_exists "$JAVA_OUTPUT_DIR/com/example/shapes/Rectangle.json" \ + "default mode writes no javadoc-shaped pages" +assert_file_not_exists "$JAVA_OUTPUT_DIR/allclasses-index.json" \ + "default mode writes no javadoc index files" + +echo +echo "==> javadoc-mode enabled: layout mirrors javadoc's api/ tree" +run_dokka_java "$JD_ON" + +# Package-as-directory layout, and a nested type kept as Outer.Nested in the enclosing package. +assert_file_exists "$JAVA_OUTPUT_DIR/com/example/shapes/Rectangle.json" \ + "class page lands at /.json" +assert_file_exists "$JAVA_OUTPUT_DIR/com/example/shapes/Rectangle.Builder.json" \ + "nested type keeps its dotted name, as javadoc does" +assert_file_exists "$JAVA_OUTPUT_DIR/com/example/shapes/package-summary.json" \ + "package page lands at package-summary.json" +assert_file_exists "$JAVA_OUTPUT_DIR/com/example/shapes/spi/package-summary.json" \ + "a second package gets its own package-summary.json" +assert_file_not_exists "$JAVA_OUTPUT_DIR/com.example.shapes/-rectangle/index.json" \ + "Dokka-shaped pages are not written in javadoc mode" + +# The global index files javadoc emits at the root of api/. +for f in index.json allclasses-index.json allpackages-index.json deprecated-list.json \ + constant-values.json element-list index-files/index-1.json; do + assert_file_exists "$JAVA_OUTPUT_DIR/$f" "javadoc index file $f is written" +done + +# No HTML is ever produced -- rendering is the downstream template engine's job. +html_count=$(find "$JAVA_OUTPUT_DIR" -name '*.html' 2>/dev/null | wc -l | tr -d ' ') +assert_eq "$html_count" "0" "javadoc mode writes no HTML files" + +echo +echo "==> class page content mirrors a javadoc class page" +RECT="$JAVA_OUTPUT_DIR/com/example/shapes/Rectangle.json" +assert_json "$RECT" "d['qualifiedName']" "com.example.shapes.Rectangle" "qualified name" +assert_json "$RECT" "d['signature']" "public class Rectangle extends AbstractShape" \ + "type signature reads as javadoc prints it" +assert_json "$RECT" "d['superclass']['qualifiedName']" "com.example.shapes.AbstractShape" "superclass" +assert_json "$RECT" "[t['qualifiedName'] for t in d['inheritance']]" \ + "['com.example.shapes.AbstractShape', 'com.example.shapes.Rectangle']" \ + "inheritance tree runs ancestor-first, ending at this type" +assert_json "$RECT" "[t['qualifiedName'] for t in d['allImplementedInterfaces']]" \ + "['com.example.shapes.Shape']" \ + "All Implemented Interfaces closes over the superclass chain" +assert_json "$RECT" "[t['qualifiedName'] for t in d['directKnownSubclasses']]" \ + "['com.example.shapes.Square']" "Direct Known Subclasses" +assert_json "$RECT" "[n['qualifiedName'] for n in d['nestedTypes']]" \ + "['com.example.shapes.Rectangle.Builder']" "nested type summary" + +# javadoc has used (...) for constructor anchors since JDK 18, with erased parameter types. +assert_json "$RECT" "sorted(c['anchor'] for c in d['constructors'])" \ + "['()', '(double,double)']" "constructor anchors use javadoc's (...) form" + +# A private field with a public getter is a *method* in javadoc, not a field. Dokka merges the +# pair into a Kotlin-style property, so this is the regression guard for unfolding it back. +assert_json "$RECT" "sorted(m['name'] for m in d['methods'])" \ + "['area', 'getHeight', 'getWidth', 'perimeter']" \ + "Java accessors stay methods rather than becoming synthetic properties" +assert_json "$RECT" "sorted(f['name'] for f in d['fields'])" \ + "['EMPTY_LABEL', 'SIDE_COUNT']" "only real fields are listed as fields" + +# Overrides / Specified by, derived from erased signatures. +assert_json "$RECT" "[s['declaringType']['qualifiedName'] for m in d['methods'] if m['name']=='area' for s in m['specifiedBy']]" \ + "['com.example.shapes.Shape']" "Specified by points at the declaring interface" +# javadoc groups inherited members per declaring type, interfaces included -- Rectangle inherits +# scaled() from the Shape interface as well as the AbstractShape methods. +assert_json "$RECT" "sorted(g['declaringType']['qualifiedName'] for g in d['inheritedMethods'])" \ + "['com.example.shapes.AbstractShape', 'com.example.shapes.Shape']" \ + "inherited methods are grouped by declaring type, classes and interfaces alike" + +# Deprecation carries javadoc's since/forRemoval, not just the comment. +assert_json "$RECT" "[ (m['deprecated']['forRemoval'], m['deprecated']['since']) for m in d['methods'] if m['name']=='perimeter' ]" \ + "[(True, '2.0')]" "@Deprecated(since, forRemoval) is captured" + +echo +echo "==> interface, enum, annotation and exception pages" +SHAPE="$JAVA_OUTPUT_DIR/com/example/shapes/Shape.json" +assert_json "$SHAPE" "d['kind']" "interface" "interface kind" +assert_json "$SHAPE" "[m['signature'] for m in d['methods'] if m['name']=='scaled']" \ + "['public default Shape scaled(double factor) throws IllegalArgumentException']" \ + "a non-abstract interface method is recovered as 'default'" +assert_json "$SHAPE" "d['typeParameters'][0]['description'] is not None" "True" \ + "@param is attached to the type parameter" +assert_json "$SHAPE" "d['since']" "['1.0']" "@since is plain text, not a wrapped paragraph" +assert_json "$SHAPE" "d['authors']" "['Docs Pipeline']" "@author is captured" +assert_json "$SHAPE" "d['isFunctionalInterface']" "False" \ + "an interface with two abstract methods is not functional" + +FACTORY="$JAVA_OUTPUT_DIR/com/example/shapes/spi/ShapeFactory.json" +assert_json "$FACTORY" "d['isFunctionalInterface']" "True" \ + "an interface with one abstract method is functional" +assert_json "$FACTORY" "[e['type']['qualifiedName'] for e in d['methods'][0]['exceptions']]" \ + "['java.text.ParseException']" "@throws is captured with its resolved type" + +CORNER="$JAVA_OUTPUT_DIR/com/example/shapes/Corner.json" +assert_json "$CORNER" "d['kind']" "enum" "enum kind" +assert_json "$CORNER" "[e['name'] for e in d['enumConstants']]" \ + "['TOP_LEFT', 'TOP_RIGHT', 'BOTTOM_LEFT', 'BOTTOM_RIGHT']" \ + "enum constants keep declaration order" + +MEASURED="$JAVA_OUTPUT_DIR/com/example/shapes/Measured.json" +assert_json "$MEASURED" "d['kind']" "annotation" "annotation kind" +assert_json "$MEASURED" "d['signature']" "public @interface Measured" "annotation signature" +# The type's own elements must be listed. equals/hashCode/toString/annotationType come along too: +# they are inherited from java.lang.annotation.Annotation and java.lang.Object, which this small +# example does not document, and a member inherited from an *undocumented* type is shown as +# declared -- javadoc does the same, since there is no page to link the reader to. In a run that +# documents java.lang (the JDK build) they are inherited-member groups instead. +assert_json "$MEASURED" "sorted(e['name'] for e in d['annotationElements'] if e['name'] in ('tolerance','verifiedBy'))" \ + "['tolerance', 'verifiedBy']" "annotation elements are listed" +assert_json "$MEASURED" "'annotationType' in [e['name'] for e in d['annotationElements']]" "True" \ + "members inherited from an undocumented supertype are pulled up, as javadoc does" + +EXC="$JAVA_OUTPUT_DIR/com/example/shapes/ShapeException.json" +assert_json "$EXC" "d['kind']" "exception" \ + "a Throwable subtype is tabled as an exception, as javadoc does" + +echo +echo "==> package, module and global index pages" +PKG="$JAVA_OUTPUT_DIR/com/example/shapes/package-summary.json" +assert_json "$PKG" "[t['name'] for t in d['interfaces']]" "['Shape']" "package interface table" +assert_json "$PKG" "[t['name'] for t in d['exceptions']]" "['ShapeException']" "package exception table" +assert_json "$PKG" "[t['name'] for t in d['annotationTypes']]" "['Measured']" "package annotation table" +# Shape, AbstractShape, Rectangle, Rectangle.Builder, Square, Corner, Measured, ShapeException. +assert_json "$PKG" "len(d['allTypes'])" "8" "allTypes lists every type in the package" + +# module-summary.json is built from module-info.java, which Dokka's model does not carry at all -- +# these assertions are the guard on that separate parsing path. +MOD="$JAVA_OUTPUT_DIR/module-summary.json" +assert_json "$MOD" "d['name']" "com.example.shapes" \ + "module name comes from module-info.java, not from Dokka's module name" +assert_json "$MOD" "d['since']" "['1.0']" "the module's @since is captured" +assert_json "$MOD" "[(r['module'], r['isTransitive'], r['isStatic']) for r in d['requires']]" \ + "[('java.logging', True, False), ('java.sql', False, True)]" \ + "requires keeps its transitive/static modifiers" +assert_json "$MOD" "[(e['packageName'], e['to']) for e in d['exports']]" \ + "[('com.example.shapes', []), ('com.example.shapes.spi', [])]" "exports directives" +assert_json "$MOD" "[u['qualifiedName'] for u in d['uses']]" \ + "['com.example.shapes.spi.ShapeFactory']" "uses resolves to the documented service type" +assert_json "$MOD" "'module-info.java' in d['description']" "True" \ + "javadoc inline tags in the module comment are rendered" +assert_json "$MOD" "sorted(p['name'] for p in d['packages'])" \ + "['com.example.shapes', 'com.example.shapes.spi']" "module lists its documented packages" + +ALL="$JAVA_OUTPUT_DIR/allclasses-index.json" +# The eight in com.example.shapes plus ShapeFactory in com.example.shapes.spi. +assert_json "$ALL" "len(d['types'])" "9" "allclasses-index covers every documented type" +assert_json "$ALL" "[t['url'] for t in d['types'] if t['name']=='Shape']" \ + "['com/example/shapes/Shape.json']" "index links are relative to the index page" + +DEP="$JAVA_OUTPUT_DIR/deprecated-list.json" +assert_json "$DEP" "[e['element'] for e in d['sections']['methods']]" \ + "['com.example.shapes.Rectangle.perimeter()']" "deprecated methods are listed" +# Regression guard: the comment is a rendered fragment and must be re-rendered relative to the +# page it lands on, not lifted verbatim off the class page (where the href is just +# "Rectangle.json#getWidth()"). Asserted on the parsed value, since prettyPrint escapes the +# quotes in the raw file. +assert_json "$DEP" "'href=\"com/example/shapes/Rectangle.json#getWidth()\"' in d['sections']['methods'][0]['comment']" \ + "True" "links inside a deprecation comment resolve from the index page" + +CONST="$JAVA_OUTPUT_DIR/constant-values.json" +assert_json "$CONST" "sorted(f['value'] for t in d['packages']['com.example.shapes'] for f in t['fields'])" \ + "['\"empty\"', '4', '64']" "constant values are unwrapped to their literals" + +assert_contains "$JAVA_OUTPUT_DIR/element-list" "com.example.shapes.spi" \ + "element-list names every documented package" + +IDX="$JAVA_OUTPUT_DIR/index-files/index-1.json" +assert_json "$IDX" "d['entries'][0]['url'].startswith('../')" "True" \ + "index-files entries link back out of their own directory" +assert_json "$IDX" "len(d['letters']) > 1" "True" "index pages carry the full letter list" + +echo +echo "==> javadoc mode honours the shared output options" +run_dokka_java '{"logLevel":"debug","javadoc-mode":true,"omitNulls":true,"omitFields":["firstSentence"]}' +assert_not_contains "$JAVA_OUTPUT_DIR/com/example/shapes/Rectangle.json" '"firstSentence"' \ + "omitFields strips keys from javadoc-mode pages" +assert_not_contains "$JAVA_OUTPUT_DIR/com/example/shapes/Rectangle.json" '"seeAlso":[]' \ + "omitNulls strips empty values from javadoc-mode pages" +assert_eq "$(line_count "$JAVA_OUTPUT_DIR/com/example/shapes/Rectangle.json")" "1" \ + "prettyPrint off yields compact single-line JSON" + +summarize_and_exit