From 0060a3763f143148b15e937f1f4bf85683c46a14 Mon Sep 17 00:00:00 2001 From: sempervent Date: Wed, 30 Sep 2026 22:35:07 -0400 Subject: [PATCH 01/12] docs: audit portfolio and project pages Add maintainer recon for site IA, live GitHub Pages verification, and BlackLake lineage. --- .../maintainers/portfolio-recon-2026-09-30.md | 97 +++++++++++++++++++ 1 file changed, 97 insertions(+) create mode 100644 docs/maintainers/portfolio-recon-2026-09-30.md diff --git a/docs/maintainers/portfolio-recon-2026-09-30.md b/docs/maintainers/portfolio-recon-2026-09-30.md new file mode 100644 index 0000000..d902d7d --- /dev/null +++ b/docs/maintainers/portfolio-recon-2026-09-30.md @@ -0,0 +1,97 @@ +# Portfolio & site reconnaissance (2026-09-30) + +Evidence sources: this repository (`main`), live site `https://sempervent.github.io/`, +GitHub API (`sempervent/*` public repos), and HTTP checks against org GitHub Pages paths. + +## Current site architecture (before restructure) + +| Layer | Behavior | +| --- | --- | +| Stack | MkDocs 1.x + Material 9.x, GitHub Actions → GitHub Pages (`deploy-mkdocs.yml`) | +| Home (`docs/index.md`) | Hero + **Best Practices / Tutorials / Projects / Just for Fun** cards; deep “Start Here” and featured articles; stale **What I'm Working On** and **Latest Updates** | +| Nav (`mkdocs.yml`) | Many top-level tabs: Home, What's New, Tags, Profile, Projects, Technical Documentation, Doctrine, ADRs, Best Practices, Deep Dives, Tutorials, Contact | +| Projects | Single hand-maintained `docs/projects.md` — foregrounds BlackLake, ORNL OpenSAMPL, and legacy web apps | +| Freshness | Manual lists on homepage; `whats-new.md` is explicitly curated (ADR-0001) | +| Project metadata | Duplicated prose in `projects.md` / homepage; **no canonical registry** | +| Plugins installed | `git-revision-date-localized`, `tags`, `minify`; `mkdocs-redirects` / `macros` in `requirements.txt` but **not wired** in `mkdocs.yml` | + +## Stale or misleading sections + +- Homepage **Featured** projects: Final Fantasy Football, Where I've Been, This Is A Casino, DCRS — not reflected in recent `sempervent` GitHub activity; several lack repo links in `projects.md`. +- **What I'm Working On** — generic doc links, not tied to active repositories (PARQONAUT, `dots`, NUMBRANE). +- **About This Site** — “Everything here is battle-tested” / “production-ready configurations” overstates mixed content (experiments, tutorials, reference). +- **BlackLake** — `projects.md` links “Live Demo” to `s3-rust-data-portal` Pages only; separate `blacklake` Python repo also publishes docs at `/blacklake/`. +- **Footer** — presents ORNL affiliation without personal-site disclaimer. + +## Verified live GitHub Pages (HTTP 200) + +Org site base: `https://sempervent.github.io/` + +| Path | Repo | Notes | +| --- | --- | --- | +| `/` | `sempervent.github.io` | Main portfolio / docs | +| `/PARQONAUT/` | `PARQONAUT` | Homepage URL set on repo | +| `/dots/` | `dots` | Dotfiles docs | +| `/music-rig/` | `music-rig` | Studio setup docs | +| `/blacklake/` | `blacklake` | Python Blacklake docs | +| `/s3-rust-data-portal/` | `s3-rust-data-portal` | Rust portal docs (README also branded Blacklake) | +| `/gi/` | `gi` | | +| `/wildfire-smoke-risk-correlator/` | `wildfire-smoke-risk-correlator` | | +| `/agent-llm-wiki-matrix/` | `agent-llm-wiki-matrix` | | +| `/postgres-query-autopsy-tool/` | `postgres-query-autopsy-tool` | | +| `/smart-farm-wiki/` | `smart-farm-wiki` | | +| `/llm-wiki-template/` | `llm-wiki-template` | | +| `/generative-midi-workbench/` | `generative-midi-workbench` | | + +External: [OpenSAMPL docs](https://ornl.github.io/OpenSAMPL/) (`ORNL/OpenSAMPL`). + +## Pages enabled but not serving (HTTP 404 on org URL) + +GitHub Pages API reports a site; path returned 404 on 2026-09-30: + +| Repo | Configured URL | +| --- | --- | +| `mqtt-comparison` | `https://sempervent.github.io/mqtt-comparison/` | +| `embers-of-the-earth` | `https://sempervent.github.io/embers-of-the-earth/` | +| `colony` | `https://sempervent.github.io/colony/` | +| `universe` | `https://sempervent.github.io/universe/` | + +Omitted from the public “documentation sites” directory until deploys succeed. + +## Orphaned / unlinked live Pages + +Live sites above were **not** linked from the main portfolio prior to this work (except BlackLake via wrong repo URL). `PARQONAUT`, `dots`, wikis, and tooling docs were absent from `projects.md`. + +## Recently active public repos (portfolio-relevant) + +By `updatedAt` on 2026-09-30: `numbrane`, `dots`, `PARQONAUT`, `paraclete`, `music-rig`, `cosmic-architect`, `wildfire-smoke-risk-correlator`, agent/wiki tooling, `postgres-query-autopsy-tool`. + +## Recommended hierarchy + +1. **Person** — hero + professional scope (geospatial, data, distributed systems, tooling). +2. **Current work** — registry-driven cards: PARQONAUT, dots, NUMBRANE, Paraclete, Cosmic Architect. +3. **Proof** — featured engineering (Blacklake lineages, smoke correlator, autopsy tool). +4. **Technical writing** — Doctrine / Best Practices / Deep Dives / Tutorials (unchanged URLs, grouped under **Writing** tab). +5. **Lab** — creative, games, MIDI, Pi experiments. +6. **Deep docs** — existing taxonomy preserved. + +## BlackLake lineage (evidence-based) + +| Repo | Implementation | Docs URL | Last push (API) | +| --- | --- | --- | --- | +| `sempervent/blacklake` | Python; git-like S3 dataset store + semantic metadata | `/blacklake/` | 2025-10-09 | +| `sempervent/s3-rust-data-portal` | Rust/Axum ML artifact portal; README title “Blacklake” | `/s3-rust-data-portal/` | 2025-12-16 | + +Both are public, both publish MkDocs sites under the org Pages namespace. They appear to be **parallel lineages** sharing a product name rather than a simple rename; this site lists them separately and avoids implying a single merged codebase. + +## Deployment vs repository + +- CI builds with `mkdocs build --clean` (no `--strict` on `main` deploy); PR preview uses `--strict`. +- No pre-build project generator existed before this change. + +## Implementation follow-up (this branch) + +- Canonical registry: `data/projects.yaml` +- Generator: `scripts/generate_portfolio.py` +- Validation: `scripts/validate_projects.py` +- Redirect: legacy `projects.md` → `projects/index.md` From ee33ba5b78bcfd733c2b20c77956bbf8471acb88 Mon Sep 17 00:00:00 2001 From: sempervent Date: Wed, 30 Sep 2026 22:35:15 -0400 Subject: [PATCH 02/12] feat(site): add canonical project registry Introduce data/projects.yaml with validation and generation scripts for portfolio pages. --- data/projects.yaml | 351 ++++++++++++++++++++++++++++++++++ requirements.txt | 3 + scripts/generate_portfolio.py | 248 ++++++++++++++++++++++++ scripts/validate_projects.py | 126 ++++++++++++ 4 files changed, 728 insertions(+) create mode 100644 data/projects.yaml create mode 100644 scripts/generate_portfolio.py create mode 100644 scripts/validate_projects.py diff --git a/data/projects.yaml b/data/projects.yaml new file mode 100644 index 0000000..2eb09d1 --- /dev/null +++ b/data/projects.yaml @@ -0,0 +1,351 @@ +# Canonical project registry for sempervent.github.io +# Edit this file; run `python scripts/generate_portfolio.py` before `mkdocs build`. +# Validated by `python scripts/validate_projects.py`. + +schema_version: 1 + +projects: + - slug: parqonaut + name: PARQONAUT + summary: >- + Rust toolkit (`prqnt`) for scanning, streaming, transforming, diagnosing, + and repairing Parquet datasets across local paths and S3-compatible storage. + status: active + category: engineering + repo: https://github.com/sempervent/PARQONAUT + docs: https://sempervent.github.io/PARQONAUT/ + featured: true + current: true + language: Rust + tags: [parquet, rust, data-engineering] + + - slug: dots + name: dots + summary: >- + Personal workstation operating layer — shared shell tooling and bootstrap + flows across macOS and Linux (bash and zsh). + status: active + category: infrastructure + repo: https://github.com/sempervent/dots + docs: https://sempervent.github.io/dots/ + featured: true + current: true + language: Shell + tags: [dotfiles, dev-environment] + + - slug: numbrane + name: NUMBRANE + summary: >- + Polyglot generative audiovisual system driven by mathematics and procedural + rules — explicitly not LLM-based generative AI. + status: active + category: creative + repo: https://github.com/sempervent/numbrane + featured: true + current: true + language: Python + tags: [generative-art, audio, studio] + + - slug: paraclete + name: Paraclete + summary: >- + Data exploration tool; upstream lineage merged into the PARQONAUT workspace + (see PARQONAUT provenance docs). + status: maintained + category: engineering + repo: https://github.com/sempervent/paraclete + featured: false + current: true + language: Rust + tags: [parquet, exploration] + + - slug: cosmic-architect + name: Cosmic Architect + summary: >- + Web-based planet-building game (successor to earlier pygame experiments); + competitive ecosystem design. + status: experimental + category: games + repo: https://github.com/sempervent/cosmic-architect + featured: false + current: true + language: Python + tags: [games, web] + + - slug: music-rig + name: music-rig + summary: >- + Documentation and configuration for a home audio recording studio setup. + status: maintained + category: creative + repo: https://github.com/sempervent/music-rig + docs: https://sempervent.github.io/music-rig/ + featured: false + current: false + language: Markdown + tags: [audio, homelab] + + - slug: blacklake-python + name: Blacklake (Python) + summary: >- + Git-like dataset versioning on S3-compatible stores with JSON-LD semantic + metadata, SHACL validation, and Solr-backed search. + status: maintained + category: engineering + repo: https://github.com/sempervent/blacklake + docs: https://sempervent.github.io/blacklake/ + featured: true + current: false + language: Python + tags: [s3, metadata, versioning] + + - slug: s3-rust-data-portal + name: Blacklake (Rust data portal) + summary: >- + Rust/Axum Git-style artifact service for ML models and datasets on S3 with + PostgreSQL metadata search and CLI tooling (`s3-rust-data-portal` repo). + status: maintained + category: engineering + repo: https://github.com/sempervent/s3-rust-data-portal + docs: https://sempervent.github.io/s3-rust-data-portal/ + featured: false + current: false + language: Rust + tags: [s3, rust, ml-artifacts] + + - slug: wildfire-smoke-risk + name: Wildfire Smoke Risk Correlator + summary: >- + Geospatial analysis project correlating wildfire smoke exposure with health + risk indicators (public research prototype). + status: experimental + category: engineering + repo: https://github.com/sempervent/wildfire-smoke-risk-correlator + docs: https://sempervent.github.io/wildfire-smoke-risk-correlator/ + featured: false + current: false + language: Python + tags: [geospatial, health, wildfire] + + - slug: postgres-query-autopsy + name: Postgres Query Autopsy Tool + summary: >- + .NET tool for inspecting and explaining PostgreSQL query plans and workload + behavior. + status: experimental + category: engineering + repo: https://github.com/sempervent/postgres-query-autopsy-tool + docs: https://sempervent.github.io/postgres-query-autopsy-tool/ + featured: false + current: false + language: C# + tags: [postgres, diagnostics] + + - slug: gi + name: gi + summary: >- + CLI utility to combine and manage `.gitignore` patterns across polyglot + repositories. + status: maintained + category: engineering + repo: https://github.com/sempervent/gi + docs: https://sempervent.github.io/gi/ + featured: false + current: false + language: Go + tags: [git, tooling] + + - slug: agent-llm-wiki-matrix + name: Agent LLM Wiki Matrix + summary: >- + Comparison framework for agentic coding systems and repo-centered signals + for measuring assistant workflows. + status: experimental + category: documentation + repo: https://github.com/sempervent/agent-llm-wiki-matrix + docs: https://sempervent.github.io/agent-llm-wiki-matrix/ + featured: false + current: false + language: Markdown + tags: [llm, agents, documentation] + + - slug: smart-farm-wiki + name: Smart Farm Wiki + summary: >- + MkDocs knowledge base for smart-farm / homelab automation experiments. + status: experimental + category: documentation + repo: https://github.com/sempervent/smart-farm-wiki + docs: https://sempervent.github.io/smart-farm-wiki/ + featured: false + current: false + language: Markdown + tags: [wiki, iot] + + - slug: llm-wiki-template + name: LLM Wiki Template + summary: >- + Reusable MkDocs template for LLM-oriented team wikis and agent playbooks. + status: maintained + category: documentation + repo: https://github.com/sempervent/llm-wiki-template + docs: https://sempervent.github.io/llm-wiki-template/ + featured: false + current: false + language: Markdown + tags: [template, mkdocs] + + - slug: generative-midi-workbench + name: Generative MIDI Workbench + summary: >- + Experiments in algorithmic MIDI composition and performance tooling. + status: experimental + category: creative + repo: https://github.com/sempervent/generative-midi-workbench + docs: https://sempervent.github.io/generative-midi-workbench/ + featured: false + current: false + language: Python + tags: [midi, music] + + - slug: mqtt-comparison + name: MQTT Comparison + summary: >- + Experimental comparison of MQTT brokers and client patterns (Pages site + configured in GitHub but not currently serving — see maintainer recon). + status: experimental + category: engineering + repo: https://github.com/sempervent/mqtt-comparison + featured: false + current: false + language: Python + tags: [mqtt, iot] + + - slug: cockpit + name: cockpit + summary: >- + tmux-backed Go TUI homelab cockpit that orchestrates terminal tools as + persistent workspaces. + status: experimental + category: infrastructure + repo: https://github.com/sempervent/cockpit + featured: false + current: false + language: Go + tags: [tui, homelab] + + - slug: parqknife + name: parqknife + summary: >- + Parquet “swiss army knife” CLI; functionality consolidated into PARQONAUT + (`prqnt`). + status: historical + category: engineering + repo: https://github.com/sempervent/parqknife + featured: false + current: false + language: Rust + tags: [parquet, cli] + + - slug: opensampl + name: OpenSAMPL + summary: >- + ORNL open-source clock probe aggregation and timing analysis platform + (external organization repository). + status: maintained + category: engineering + repo: https://github.com/ORNL/OpenSAMPL + docs: https://ornl.github.io/OpenSAMPL/ + featured: false + current: false + language: Python + tags: [timing, ornl-external] + + - slug: whereivebeen + name: Where I've Been + summary: >- + County-level travel map and statistics web app (earlier-generation personal + project). + status: historical + category: engineering + repo: https://github.com/sempervent/whereivebeen + featured: false + current: false + language: Python + tags: [web, maps] + + - slug: thisisacasino + name: This Is A Casino + summary: >- + Experimental semantic / ML-driven stock exploration and visualization + stack (circa 2023). + status: historical + category: engineering + repo: https://github.com/sempervent/thisisacasino + featured: false + current: false + language: Python + tags: [finance, ml] + + - slug: dcrs + name: Decentralized Content Reward System (DCRS) + summary: >- + Web3 content-rewards experiment combining Flask services and blockchain + ideas (historical prototype). + status: historical + category: engineering + repo: https://github.com/sempervent/dcrs + featured: false + current: false + language: Python + tags: [web3, prototype] + + - slug: genesis-game + name: Genesis + summary: >- + pygame experiment: audio-input-driven procedural universe toy. + status: historical + category: games + repo: https://github.com/sempervent/genesis + featured: false + current: false + language: Python + tags: [games, audio] + + - slug: colony + name: colony + summary: >- + Rust colony simulation prototype (GitHub Pages enabled but not currently + serving content at the org URL). + status: historical + category: games + repo: https://github.com/sempervent/colony + featured: false + current: false + language: Rust + tags: [simulation, rust] + + - slug: embers-of-the-earth + name: Embers of the Earth + summary: >- + Pixel-art farming game concept in a post-apocalyptic steampunk setting + (Pages deploy not verified live on org site). + status: experimental + category: games + repo: https://github.com/sempervent/embers-of-the-earth + featured: false + current: false + language: Unknown + tags: [games, pixel-art] + + - slug: cosmic-garden + name: Cosmic Garden + summary: >- + Cultivation / garden simulation experiment related to the Cosmic game line. + status: experimental + category: games + repo: https://github.com/sempervent/cosmic-garden + featured: false + current: false + language: Unknown + tags: [games] diff --git a/requirements.txt b/requirements.txt index 48bc435..a9f86b5 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,3 +1,6 @@ +# Portfolio registry validation and generation +PyYAML>=6.0 + # Core MkDocs and Material Theme (pin to avoid MkDocs 2.0 and deprecated materialx) mkdocs>=1.5.0,<2.0 mkdocs-material>=9.4.0,<10.0 diff --git a/scripts/generate_portfolio.py b/scripts/generate_portfolio.py new file mode 100644 index 0000000..ecc77da --- /dev/null +++ b/scripts/generate_portfolio.py @@ -0,0 +1,248 @@ +#!/usr/bin/env python3 +"""Generate portfolio Markdown from data/projects.yaml.""" + +from __future__ import annotations + +import sys +from pathlib import Path + +import yaml + +ROOT = Path(__file__).resolve().parents[1] +REGISTRY = ROOT / "data" / "projects.yaml" +GENERATED = ROOT / "docs" / "_generated" +PROJECTS_DIR = ROOT / "docs" / "projects" + +HEADER = "\n" + + +def load_projects() -> list[dict]: + with REGISTRY.open(encoding="utf-8") as fh: + data = yaml.safe_load(fh) + return data["projects"] + + +def link_button(label: str, url: str | None, *, primary: bool = False) -> str: + if not url: + return "" + if primary: + return f"[{label}]({url}){{ .md-button .md-button--primary }}" + return f"[{label}]({url}){{ .md-button }}" + + +def project_card(project: dict) -> str: + lines = [ + '
', + f"### {project['name']}", + "", + project["summary"], + "", + ] + meta = [] + if project.get("language"): + meta.append(f"**Stack:** {project['language']}") + meta.append(f"**Status:** {project['status']}") + if meta: + lines.append(" · ".join(meta)) + lines.append("") + + actions = [] + for label, key, primary in ( + ("Docs", "docs", True), + ("Repo", "repo", False), + ("Demo", "demo", False), + ): + btn = link_button(label, project.get(key), primary=primary) + if btn: + actions.append(btn) + if actions: + lines.append(" ".join(actions)) + lines.append("") + + lines.append("
") + return "\n".join(lines) + + +def cards_grid(projects: list[dict]) -> str: + if not projects: + return "_No projects in this section._\n" + parts = ['
', ""] + for p in projects: + parts.append(project_card(p)) + parts.append("") + parts.append("
") + return "\n".join(parts) + + +def write_current_work(projects: list[dict]) -> None: + current = [p for p in projects if p.get("current")] + current.sort(key=lambda p: (not p.get("featured"), p["name"].lower())) + body = HEADER + cards_grid(current) + (GENERATED / "home-current-work.md").write_text(body, encoding="utf-8") + + +def section(title: str, intro: str, projects: list[dict]) -> str: + lines = [f"## {title}", "", intro, ""] + if projects: + lines.append(cards_grid(projects)) + else: + lines.append("_None listed._") + lines.append("") + return "\n".join(lines) + + +def write_projects_index(projects: list[dict]) -> None: + featured = [p for p in projects if p.get("featured")] + active = [p for p in projects if p.get("status") == "active" and not p.get("featured")] + maintained = [p for p in projects if p.get("status") == "maintained"] + experimental = [p for p in projects if p.get("status") == "experimental"] + historical = [p for p in projects if p.get("status") in {"historical", "archived"}] + + intro = """# Projects + +Portfolio of independent software, documentation sites, and experiments. Entries are +generated from [`data/projects.yaml`](https://github.com/sempervent/sempervent.github.io/blob/main/data/projects.yaml) +so descriptions stay in one reviewable place. + +**Status legend:** *active* — ongoing focus; *maintained* — usable with occasional updates; +*experimental* — lab/prototype; *historical* — kept for reference. + +See also **[Project documentation sites](documentation-sites.md)** for verified GitHub Pages deployments. +""" + parts = [HEADER, intro, ""] + parts.append( + section( + "Featured", + "Flagship or especially representative work.", + sorted(featured, key=lambda p: p["name"].lower()), + ) + ) + parts.append( + section( + "Active development", + "Additional projects with recent independent development.", + sorted(active, key=lambda p: p["name"].lower()), + ) + ) + parts.append( + section( + "Maintained tools & platforms", + "Libraries, portals, and utilities that remain useful.", + sorted(maintained, key=lambda p: p["name"].lower()), + ) + ) + parts.append( + section( + "Experiments & documentation", + "Wikis, comparisons, games-in-progress, and strange machinery.", + sorted(experimental, key=lambda p: p["name"].lower()), + ) + ) + parts.append( + section( + "Historical archive", + "Older web apps and prototypes — preserved, not promoted as current work.", + sorted(historical, key=lambda p: p["name"].lower()), + ) + ) + + PROJECTS_DIR.mkdir(parents=True, exist_ok=True) + (PROJECTS_DIR / "index.md").write_text("\n".join(parts), encoding="utf-8") + + +def write_lab(projects: list[dict]) -> None: + lab = [ + p + for p in projects + if p.get("category") in {"creative", "games"} + or p.get("status") == "experimental" + and p.get("category") != "documentation" + ] + lab = sorted({p["slug"]: p for p in lab}.values(), key=lambda p: p["name"].lower()) + + intro = """# Lab & Experiments + +Unusual projects, games, generative systems, and homelab oddities. Material here is +**experimental** unless explicitly marked maintained — not production guidance. + +For step-by-step creative tutorials, see **[Just for Fun](../tutorials/just-for-fun/index.md)**. +""" + body = HEADER + intro + "\n" + cards_grid(lab) + lab_dir = ROOT / "docs" / "lab" + lab_dir.mkdir(parents=True, exist_ok=True) + (lab_dir / "index.md").write_text(body, encoding="utf-8") + + +def write_documentation_sites(projects: list[dict]) -> None: + verified_slugs = { + "parqonaut", + "dots", + "music-rig", + "blacklake-python", + "s3-rust-data-portal", + "gi", + "wildfire-smoke-risk", + "agent-llm-wiki-matrix", + "postgres-query-autopsy", + "smart-farm-wiki", + "llm-wiki-template", + "generative-midi-workbench", + "opensampl", + } + with_docs = [ + p + for p in projects + if p.get("docs") and p.get("slug") in verified_slugs + ] + with_docs.sort(key=lambda p: p["name"].lower()) + + lines = [ + HEADER, + "# Project documentation sites", + "", + "Verified live documentation deployments (HTTP 200 as of the portfolio recon in", + "`docs/maintainers/portfolio-recon-2026-09-30.md`). Repositories with GitHub Pages", + "enabled but returning 404 are omitted until the deploy is fixed.", + "", + "| Project | Purpose | Site |", + "| --- | --- | --- |", + ] + for p in with_docs: + summary = p["summary"].replace("|", "\\|").replace("\n", " ") + if len(summary) > 120: + summary = summary[:117] + "..." + lines.append(f"| {p['name']} | {summary} | [{p['docs']}]({p['docs']}) |") + + lines.append("") + PROJECTS_DIR.mkdir(parents=True, exist_ok=True) + (PROJECTS_DIR / "documentation-sites.md").write_text("\n".join(lines), encoding="utf-8") + + +def main() -> int: + sys.path.insert(0, str(ROOT / "scripts")) + from validate_projects import validate_registry, load_registry # noqa: WPS433 + + try: + data = load_registry() + except (OSError, yaml.YAMLError, ValueError) as exc: + print(f"Failed to load registry: {exc}", file=sys.stderr) + return 1 + + errors = validate_registry(data) + if errors: + for err in errors: + print(err, file=sys.stderr) + return 1 + + projects = data["projects"] + GENERATED.mkdir(parents=True, exist_ok=True) + write_current_work(projects) + write_projects_index(projects) + write_lab(projects) + write_documentation_sites(projects) + print("Generated portfolio pages from data/projects.yaml") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/validate_projects.py b/scripts/validate_projects.py new file mode 100644 index 0000000..d461bc2 --- /dev/null +++ b/scripts/validate_projects.py @@ -0,0 +1,126 @@ +#!/usr/bin/env python3 +"""Validate data/projects.yaml for the portfolio registry.""" + +from __future__ import annotations + +import re +import sys +from pathlib import Path +from urllib.parse import urlparse + +import yaml + +ROOT = Path(__file__).resolve().parents[1] +REGISTRY = ROOT / "data" / "projects.yaml" + +STATUSES = frozenset({"active", "maintained", "experimental", "historical", "archived"}) +CATEGORIES = frozenset( + { + "engineering", + "documentation", + "creative", + "games", + "infrastructure", + "historical", + } +) + +URL_RE = re.compile(r"^https?://", re.I) + + +def load_registry() -> dict: + with REGISTRY.open(encoding="utf-8") as fh: + data = yaml.safe_load(fh) + if not isinstance(data, dict) or "projects" not in data: + raise ValueError("Registry must be a mapping with a 'projects' list") + if not isinstance(data["projects"], list): + raise ValueError("'projects' must be a list") + return data + + +def validate_url(field: str, url: str, errors: list[str]) -> None: + if not URL_RE.match(url): + errors.append(f"{field}: invalid URL {url!r}") + return + parsed = urlparse(url) + if not parsed.netloc: + errors.append(f"{field}: missing host in {url!r}") + + +def validate_registry(data: dict) -> list[str]: + errors: list[str] = [] + slugs: set[str] = set() + docs_urls: dict[str, str] = {} + + for idx, project in enumerate(data["projects"]): + prefix = f"projects[{idx}]" + if not isinstance(project, dict): + errors.append(f"{prefix}: must be a mapping") + continue + + slug = project.get("slug") + if not slug or not isinstance(slug, str): + errors.append(f"{prefix}: missing or invalid slug") + continue + if slug in slugs: + errors.append(f"duplicate slug: {slug}") + slugs.add(slug) + + for key in ("name", "summary", "status", "category"): + if not project.get(key): + errors.append(f"{prefix} ({slug}): missing required field '{key}'") + + status = project.get("status") + if status and status not in STATUSES: + errors.append(f"{prefix} ({slug}): invalid status {status!r}") + + category = project.get("category") + if category and category not in CATEGORIES: + errors.append(f"{prefix} ({slug}): invalid category {category!r}") + + if project.get("featured") and not str(project.get("summary", "")).strip(): + errors.append(f"{prefix} ({slug}): featured project missing summary") + + repo = project.get("repo") + if repo: + validate_url(f"{prefix} ({slug}).repo", repo, errors) + + docs = project.get("docs") + if docs: + validate_url(f"{prefix} ({slug}).docs", docs, errors) + if docs in docs_urls: + errors.append( + f"docs URL {docs} used by both {docs_urls[docs]} and {slug}" + ) + docs_urls[docs] = slug + + demo = project.get("demo") + if demo: + validate_url(f"{prefix} ({slug}).demo", demo, errors) + + return errors + + +def main() -> int: + if not REGISTRY.is_file(): + print(f"Registry not found: {REGISTRY}", file=sys.stderr) + return 1 + try: + data = load_registry() + except (OSError, yaml.YAMLError, ValueError) as exc: + print(f"Failed to load registry: {exc}", file=sys.stderr) + return 1 + + errors = validate_registry(data) + if errors: + print("Project registry validation failed:", file=sys.stderr) + for err in errors: + print(f" - {err}", file=sys.stderr) + return 1 + + print(f"OK: {len(data['projects'])} projects validated") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) From 4f97ba57c96ad9046cc2d01a6f1855f04ab23c4c Mon Sep 17 00:00:00 2001 From: sempervent Date: Wed, 30 Sep 2026 22:35:18 -0400 Subject: [PATCH 03/12] feat(site): rebuild project and docs indexes Generate portfolio, lab, and documentation-site indexes from the registry. --- docs/_generated/home-current-work.md | 59 +++++ docs/lab/index.md | 143 +++++++++++ docs/projects.md | 258 -------------------- docs/projects/documentation-sites.md | 23 ++ docs/projects/index.md | 348 +++++++++++++++++++++++++++ 5 files changed, 573 insertions(+), 258 deletions(-) create mode 100644 docs/_generated/home-current-work.md create mode 100644 docs/lab/index.md delete mode 100644 docs/projects.md create mode 100644 docs/projects/documentation-sites.md create mode 100644 docs/projects/index.md diff --git a/docs/_generated/home-current-work.md b/docs/_generated/home-current-work.md new file mode 100644 index 0000000..9043bee --- /dev/null +++ b/docs/_generated/home-current-work.md @@ -0,0 +1,59 @@ + +
+ +
+### dots + +Personal workstation operating layer — shared shell tooling and bootstrap flows across macOS and Linux (bash and zsh). + +**Stack:** Shell · **Status:** active + +[Docs](https://sempervent.github.io/dots/){ .md-button .md-button--primary } [Repo](https://github.com/sempervent/dots){ .md-button } + +
+ +
+### NUMBRANE + +Polyglot generative audiovisual system driven by mathematics and procedural rules — explicitly not LLM-based generative AI. + +**Stack:** Python · **Status:** active + +[Repo](https://github.com/sempervent/numbrane){ .md-button } + +
+ +
+### PARQONAUT + +Rust toolkit (`prqnt`) for scanning, streaming, transforming, diagnosing, and repairing Parquet datasets across local paths and S3-compatible storage. + +**Stack:** Rust · **Status:** active + +[Docs](https://sempervent.github.io/PARQONAUT/){ .md-button .md-button--primary } [Repo](https://github.com/sempervent/PARQONAUT){ .md-button } + +
+ +
+### Cosmic Architect + +Web-based planet-building game (successor to earlier pygame experiments); competitive ecosystem design. + +**Stack:** Python · **Status:** experimental + +[Repo](https://github.com/sempervent/cosmic-architect){ .md-button } + +
+ +
+### Paraclete + +Data exploration tool; upstream lineage merged into the PARQONAUT workspace (see PARQONAUT provenance docs). + +**Stack:** Rust · **Status:** maintained + +[Repo](https://github.com/sempervent/paraclete){ .md-button } + +
+ +
\ No newline at end of file diff --git a/docs/lab/index.md b/docs/lab/index.md new file mode 100644 index 0000000..24eca86 --- /dev/null +++ b/docs/lab/index.md @@ -0,0 +1,143 @@ + +# Lab & Experiments + +Unusual projects, games, generative systems, and homelab oddities. Material here is +**experimental** unless explicitly marked maintained — not production guidance. + +For step-by-step creative tutorials, see **[Just for Fun](../tutorials/just-for-fun/index.md)**. + +
+ +
+### cockpit + +tmux-backed Go TUI homelab cockpit that orchestrates terminal tools as persistent workspaces. + +**Stack:** Go · **Status:** experimental + +[Repo](https://github.com/sempervent/cockpit){ .md-button } + +
+ +
+### colony + +Rust colony simulation prototype (GitHub Pages enabled but not currently serving content at the org URL). + +**Stack:** Rust · **Status:** historical + +[Repo](https://github.com/sempervent/colony){ .md-button } + +
+ +
+### Cosmic Architect + +Web-based planet-building game (successor to earlier pygame experiments); competitive ecosystem design. + +**Stack:** Python · **Status:** experimental + +[Repo](https://github.com/sempervent/cosmic-architect){ .md-button } + +
+ +
+### Cosmic Garden + +Cultivation / garden simulation experiment related to the Cosmic game line. + +**Stack:** Unknown · **Status:** experimental + +[Repo](https://github.com/sempervent/cosmic-garden){ .md-button } + +
+ +
+### Embers of the Earth + +Pixel-art farming game concept in a post-apocalyptic steampunk setting (Pages deploy not verified live on org site). + +**Stack:** Unknown · **Status:** experimental + +[Repo](https://github.com/sempervent/embers-of-the-earth){ .md-button } + +
+ +
+### Generative MIDI Workbench + +Experiments in algorithmic MIDI composition and performance tooling. + +**Stack:** Python · **Status:** experimental + +[Docs](https://sempervent.github.io/generative-midi-workbench/){ .md-button .md-button--primary } [Repo](https://github.com/sempervent/generative-midi-workbench){ .md-button } + +
+ +
+### Genesis + +pygame experiment: audio-input-driven procedural universe toy. + +**Stack:** Python · **Status:** historical + +[Repo](https://github.com/sempervent/genesis){ .md-button } + +
+ +
+### MQTT Comparison + +Experimental comparison of MQTT brokers and client patterns (Pages site configured in GitHub but not currently serving — see maintainer recon). + +**Stack:** Python · **Status:** experimental + +[Repo](https://github.com/sempervent/mqtt-comparison){ .md-button } + +
+ +
+### music-rig + +Documentation and configuration for a home audio recording studio setup. + +**Stack:** Markdown · **Status:** maintained + +[Docs](https://sempervent.github.io/music-rig/){ .md-button .md-button--primary } [Repo](https://github.com/sempervent/music-rig){ .md-button } + +
+ +
+### NUMBRANE + +Polyglot generative audiovisual system driven by mathematics and procedural rules — explicitly not LLM-based generative AI. + +**Stack:** Python · **Status:** active + +[Repo](https://github.com/sempervent/numbrane){ .md-button } + +
+ +
+### Postgres Query Autopsy Tool + +.NET tool for inspecting and explaining PostgreSQL query plans and workload behavior. + +**Stack:** C# · **Status:** experimental + +[Docs](https://sempervent.github.io/postgres-query-autopsy-tool/){ .md-button .md-button--primary } [Repo](https://github.com/sempervent/postgres-query-autopsy-tool){ .md-button } + +
+ +
+### Wildfire Smoke Risk Correlator + +Geospatial analysis project correlating wildfire smoke exposure with health risk indicators (public research prototype). + +**Stack:** Python · **Status:** experimental + +[Docs](https://sempervent.github.io/wildfire-smoke-risk-correlator/){ .md-button .md-button--primary } [Repo](https://github.com/sempervent/wildfire-smoke-risk-correlator){ .md-button } + +
+ +
\ No newline at end of file diff --git a/docs/projects.md b/docs/projects.md deleted file mode 100644 index 283b1b2..0000000 --- a/docs/projects.md +++ /dev/null @@ -1,258 +0,0 @@ -# Projects Portfolio - -## Featured Projects - -### OpenSAMPL - Open Source Clock Probe Aggregator and Visualizer -**Advanced clock synchronization and timing analysis platform** - -A comprehensive open-source platform for clock probe aggregation and visualization, developed at Oak Ridge National Laboratory. OpenSAMPL provides advanced tools for precise clock monitoring, time synchronization analysis, and timing accuracy verification across distributed systems. - -**Technologies**: Python, NumPy, SciPy, Matplotlib, Pandas, TimeSync, Network Time Protocol (NTP), Docker - -**Key Features**: -- **Clock Probe Aggregation**: Centralized collection and analysis of clock signals -- **Time Synchronization Analysis**: Precise measurement of clock drift and synchronization -- **Visualization Tools**: Interactive dashboards for timing analysis and monitoring -- **Distributed Systems Support**: Multi-node clock monitoring and analysis -- **Performance Optimization**: High-precision timing algorithms and measurements -- **Research Applications**: Scientific computing, real-time systems, embedded development - -**Repository**: [github.com/ORNL/OpenSAMPL](https://github.com/ORNL/OpenSAMPL) -**Documentation**: [ornl.github.com/OpenSAMPL](https://ornl.github.com/OpenSAMPL) -**PyPI Package**: [pypi.org/project/opensampl](https://pypi.org/project/opensampl) - ---- - -### BlackLake - S3-based Data Portal -**Enterprise-grade data artifact management platform** - -A production-ready, enterprise-grade data artifact management platform that combines modern technology with comprehensive features for data management, search, governance, and compliance. Built with Rust and featuring a complete S3-based Git CLI and application stack for data. - -**Technologies**: Rust, Axum, React, TypeScript, PostgreSQL, Apache Solr, Redis, MinIO, Docker, Prometheus, Grafana, OpenTelemetry - -**Key Features**: -- **Multi-Tenant Architecture**: ABAC policies with tenant isolation -- **Advanced Security**: OIDC/JWT authentication, RBAC, audit trails -- **Comprehensive Governance**: Branch protection, quotas, retention policies -- **Production Operations**: Monitoring, backup, disaster recovery -- **Modern UI**: React interface with mobile support and PWA capabilities -- **Developer Experience**: CLI tools, SDKs, comprehensive documentation - -**Live Demo**: [BlackLake Documentation](https://sempervent.github.io/s3-rust-data-portal/) -**Repository**: [s3-rust-data-portal](https://github.com/sempervent/s3-rust-data-portal) - ---- - -### Final Fantasy Football -**A semantic learning algorithm for fantasy football** - -A sophisticated machine learning system that applies semantic analysis to fantasy football decision-making. The project combines web scraping, data processing, and predictive modeling to provide actionable insights for fantasy football players. - -**Technologies**: Python, scikit-learn, Pandas, NumPy, Matplotlib, Requests, BeautifulSoup, Flask, Docker, Compose, Airflow - -**Key Features**: -- Automated data collection from multiple sources -- Semantic analysis of player performance patterns -- Predictive modeling for roster decisions -- Web interface for user interaction -- Containerized deployment with orchestration - ---- - -### Where I've Been -**A web application to visualize travel history** - -An interactive web application that allows users to visualize and explore their travel history through an intuitive mapping interface. The application processes location data and presents it in an engaging, interactive format. - -**Technologies**: Python, Flask, Docker, Compose, PostgreSQL, Leaflet, Vue, Bootstrap - -**Key Features**: -- Interactive map visualization using Leaflet -- Travel timeline and statistics -- Data import/export capabilities -- Responsive design with modern UI -- Secure user authentication - ---- - -### This Is A Casino -**Trade/visualize stocks via semantic, RL, DL** - -A comprehensive stock trading platform that leverages semantic analysis, reinforcement learning, and deep learning techniques to provide intelligent trading insights and portfolio management. - -**Technologies**: Python, Flask, Docker, Compose, PostgreSQL, Leaflet, React, Bootstrap - -**Key Features**: -- Real-time market data processing -- Semantic analysis of market sentiment -- Reinforcement learning for trading strategies -- Interactive portfolio visualization -- Risk assessment and management tools - ---- - -### Decentralized Content Reward System -**A decentralized content reward system for the web** - -An innovative Web3 platform that enables content creators to monetize their work through a decentralized reward system, leveraging blockchain technology for transparent and fair compensation. - -**Technologies**: Python, Rust, Flask, Docker, Compose, PostgreSQL, Leaflet, React, Bootstrap - -**Key Features**: -- Blockchain-based reward distribution -- Smart contract integration -- Content verification system -- User reputation management -- Decentralized governance - ---- - -### Genesis -**Audio-input game to create a universe** - -An experimental game that uses audio input to procedurally generate and evolve a virtual universe. Players interact with the system through sound, creating unique and dynamic worlds. - -**Technologies**: Python, pygame - -**Key Features**: -- Real-time audio processing -- Procedural universe generation -- Interactive sound-based controls -- Dynamic visual effects -- Save/load universe states - ---- - -### Cosmic Architect -**Compete to build the best planet** - -A competitive game where players design and build planets, competing against others to create the most successful planetary ecosystem. - -**Technologies**: Python, pygame - -**Key Features**: -- Planet design interface -- Ecosystem simulation -- Multiplayer competition -- Scoring and ranking system -- Real-time gameplay mechanics - ---- - -### pygarden -**Python package for geospatial data processing and analysis** - -A comprehensive Python package for geospatial data processing, spatial analysis, and machine learning applications. Developed as part of the OpenSAMPL ecosystem at Oak Ridge National Laboratory. - -**Technologies**: Python, NumPy, Pandas, GeoPandas, Shapely, Rasterio, GDAL, scikit-learn - -**Key Features**: -- **Spatial Data Processing**: Efficient handling of vector and raster data -- **Machine Learning Integration**: Spatial ML algorithms and workflows -- **Data Format Support**: Multiple geospatial data format support -- **Performance Optimization**: High-performance spatial operations -- **Research Tools**: Advanced spatial analysis capabilities -- **Documentation**: Comprehensive API documentation and examples - -**PyPI Package**: [pypi.org/project/pygarden](https://pypi.org/project/pygarden) -**Repository**: [github.com/ORNL/pygarden](https://code.ornl.gov/pygarden/pygarden) - ---- - -### maw -**CLI for quickly combining tabular data into Parquet** - -A high-performance command-line tool written in Rust for efficiently combining and processing tabular data into Parquet format, optimized for speed and memory usage. - -**Technologies**: Rust - -**Key Features**: -- High-performance data processing -- Memory-efficient operations -- Command-line interface -- Support for multiple input formats -- Optimized Parquet output - -## Project Categories - -### Research & Open Source Platforms -- **OpenSAMPL**: Open source clock probe aggregator and visualizer -- **pygarden**: Python framework for data processing and fast project setup - -### Enterprise Data Platforms -- **BlackLake**: Production-ready data artifact management platform -- **S3-based Git CLI**: Command-line interface for data version control - -### Data Engineering & Analytics -- **Final Fantasy Football**: Machine learning and data processing -- **maw**: High-performance data processing tool - -### Web Applications -- **Where I've Been**: Travel visualization platform -- **This Is A Casino**: Financial trading platform -- **Decentralized Content Reward System**: Web3 content platform - -### Interactive Games -- **Genesis**: Audio-based universe creation -- **Cosmic Architect**: Planet building competition - -## Technical Approach - -All projects demonstrate a focus on: - -- **Scalable Architecture**: Containerized deployments with Docker -- **Modern Web Technologies**: React, Vue, and responsive design -- **Data Processing**: Efficient handling of large datasets -- **User Experience**: Intuitive interfaces and interactive visualizations -- **Performance**: Optimized code and efficient algorithms - -## Open Source Contributions - -### GitHub Profile -**Profile**: [github.com/sempervent](https://github.com/sempervent) - -All projects are available on GitHub and demonstrate: -- **Clean, well-documented code** with comprehensive README files -- **Comprehensive testing** with automated CI/CD pipelines -- **Docker containerization** for consistent deployment -- **API documentation** with OpenAPI/Swagger specifications -- **User guides and examples** for easy adoption - -### Key Repositories -- **[s3-rust-data-portal](https://github.com/sempervent/s3-rust-data-portal)**: Enterprise data management platform -- **Data Engineering Tools**: High-performance CLI utilities and data processing libraries -- **Web Applications**: Full-stack applications with modern frameworks -- **Machine Learning Projects**: ML pipelines and predictive modeling systems - -### Contribution Philosophy -- **Open Source First**: All personal projects are open source -- **Documentation Driven**: Comprehensive documentation for all projects -- **Community Focused**: Welcoming contributions and feedback -- **Production Ready**: Enterprise-grade code quality and testing - -## Future Projects - -### Advanced Geospatial Data Processing -- **Real-time IoT Data Streams**: Processing and analyzing sensor data from IoT devices -- **Spatial Machine Learning**: Applying ML techniques to geospatial datasets -- **Cloud-native Geospatial Architectures**: Scalable solutions for large-scale spatial data - -### Enterprise Data Management -- **Data Mesh Architecture**: Implementing domain-driven data architecture patterns -- **Advanced Data Governance**: Enhanced compliance and data lineage tracking -- **Federated Data Platforms**: Cross-organization data sharing and collaboration - -### AI/ML Integration -- **MLOps Pipeline**: End-to-end machine learning operations -- **Semantic Search Enhancement**: Advanced natural language processing for data discovery -- **Automated Data Quality**: AI-powered data validation and cleansing - -### Performance & Scalability -- **Edge Computing**: Distributed processing for real-time applications -- **Multi-cloud Strategies**: Hybrid cloud data management -- **Advanced Caching**: Intelligent data caching and optimization - -### Developer Experience -- **Enhanced SDKs**: Improved developer tools and APIs -- **Visualization Tools**: Interactive data exploration interfaces -- **Documentation Automation**: AI-powered documentation generation diff --git a/docs/projects/documentation-sites.md b/docs/projects/documentation-sites.md new file mode 100644 index 0000000..8cc72cb --- /dev/null +++ b/docs/projects/documentation-sites.md @@ -0,0 +1,23 @@ + + +# Project documentation sites + +Verified live documentation deployments (HTTP 200 as of the portfolio recon in +`docs/maintainers/portfolio-recon-2026-09-30.md`). Repositories with GitHub Pages +enabled but returning 404 are omitted until the deploy is fixed. + +| Project | Purpose | Site | +| --- | --- | --- | +| Agent LLM Wiki Matrix | Comparison framework for agentic coding systems and repo-centered signals for measuring assistant workflows. | [https://sempervent.github.io/agent-llm-wiki-matrix/](https://sempervent.github.io/agent-llm-wiki-matrix/) | +| Blacklake (Python) | Git-like dataset versioning on S3-compatible stores with JSON-LD semantic metadata, SHACL validation, and Solr-backed... | [https://sempervent.github.io/blacklake/](https://sempervent.github.io/blacklake/) | +| Blacklake (Rust data portal) | Rust/Axum Git-style artifact service for ML models and datasets on S3 with PostgreSQL metadata search and CLI tooling... | [https://sempervent.github.io/s3-rust-data-portal/](https://sempervent.github.io/s3-rust-data-portal/) | +| dots | Personal workstation operating layer — shared shell tooling and bootstrap flows across macOS and Linux (bash and zsh). | [https://sempervent.github.io/dots/](https://sempervent.github.io/dots/) | +| Generative MIDI Workbench | Experiments in algorithmic MIDI composition and performance tooling. | [https://sempervent.github.io/generative-midi-workbench/](https://sempervent.github.io/generative-midi-workbench/) | +| gi | CLI utility to combine and manage `.gitignore` patterns across polyglot repositories. | [https://sempervent.github.io/gi/](https://sempervent.github.io/gi/) | +| LLM Wiki Template | Reusable MkDocs template for LLM-oriented team wikis and agent playbooks. | [https://sempervent.github.io/llm-wiki-template/](https://sempervent.github.io/llm-wiki-template/) | +| music-rig | Documentation and configuration for a home audio recording studio setup. | [https://sempervent.github.io/music-rig/](https://sempervent.github.io/music-rig/) | +| OpenSAMPL | ORNL open-source clock probe aggregation and timing analysis platform (external organization repository). | [https://ornl.github.io/OpenSAMPL/](https://ornl.github.io/OpenSAMPL/) | +| PARQONAUT | Rust toolkit (`prqnt`) for scanning, streaming, transforming, diagnosing, and repairing Parquet datasets across local... | [https://sempervent.github.io/PARQONAUT/](https://sempervent.github.io/PARQONAUT/) | +| Postgres Query Autopsy Tool | .NET tool for inspecting and explaining PostgreSQL query plans and workload behavior. | [https://sempervent.github.io/postgres-query-autopsy-tool/](https://sempervent.github.io/postgres-query-autopsy-tool/) | +| Smart Farm Wiki | MkDocs knowledge base for smart-farm / homelab automation experiments. | [https://sempervent.github.io/smart-farm-wiki/](https://sempervent.github.io/smart-farm-wiki/) | +| Wildfire Smoke Risk Correlator | Geospatial analysis project correlating wildfire smoke exposure with health risk indicators (public research prototype). | [https://sempervent.github.io/wildfire-smoke-risk-correlator/](https://sempervent.github.io/wildfire-smoke-risk-correlator/) | diff --git a/docs/projects/index.md b/docs/projects/index.md new file mode 100644 index 0000000..5ea5ea1 --- /dev/null +++ b/docs/projects/index.md @@ -0,0 +1,348 @@ + + +# Projects + +Portfolio of independent software, documentation sites, and experiments. Entries are +generated from [`data/projects.yaml`](https://github.com/sempervent/sempervent.github.io/blob/main/data/projects.yaml) +so descriptions stay in one reviewable place. + +**Status legend:** *active* — ongoing focus; *maintained* — usable with occasional updates; +*experimental* — lab/prototype; *historical* — kept for reference. + +See also **[Project documentation sites](documentation-sites.md)** for verified GitHub Pages deployments. + + +## Featured + +Flagship or especially representative work. + +
+ +
+### Blacklake (Python) + +Git-like dataset versioning on S3-compatible stores with JSON-LD semantic metadata, SHACL validation, and Solr-backed search. + +**Stack:** Python · **Status:** maintained + +[Docs](https://sempervent.github.io/blacklake/){ .md-button .md-button--primary } [Repo](https://github.com/sempervent/blacklake){ .md-button } + +
+ +
+### dots + +Personal workstation operating layer — shared shell tooling and bootstrap flows across macOS and Linux (bash and zsh). + +**Stack:** Shell · **Status:** active + +[Docs](https://sempervent.github.io/dots/){ .md-button .md-button--primary } [Repo](https://github.com/sempervent/dots){ .md-button } + +
+ +
+### NUMBRANE + +Polyglot generative audiovisual system driven by mathematics and procedural rules — explicitly not LLM-based generative AI. + +**Stack:** Python · **Status:** active + +[Repo](https://github.com/sempervent/numbrane){ .md-button } + +
+ +
+### PARQONAUT + +Rust toolkit (`prqnt`) for scanning, streaming, transforming, diagnosing, and repairing Parquet datasets across local paths and S3-compatible storage. + +**Stack:** Rust · **Status:** active + +[Docs](https://sempervent.github.io/PARQONAUT/){ .md-button .md-button--primary } [Repo](https://github.com/sempervent/PARQONAUT){ .md-button } + +
+ +
+ +## Active development + +Additional projects with recent independent development. + +_None listed._ + +## Maintained tools & platforms + +Libraries, portals, and utilities that remain useful. + +
+ +
+### Blacklake (Python) + +Git-like dataset versioning on S3-compatible stores with JSON-LD semantic metadata, SHACL validation, and Solr-backed search. + +**Stack:** Python · **Status:** maintained + +[Docs](https://sempervent.github.io/blacklake/){ .md-button .md-button--primary } [Repo](https://github.com/sempervent/blacklake){ .md-button } + +
+ +
+### Blacklake (Rust data portal) + +Rust/Axum Git-style artifact service for ML models and datasets on S3 with PostgreSQL metadata search and CLI tooling (`s3-rust-data-portal` repo). + +**Stack:** Rust · **Status:** maintained + +[Docs](https://sempervent.github.io/s3-rust-data-portal/){ .md-button .md-button--primary } [Repo](https://github.com/sempervent/s3-rust-data-portal){ .md-button } + +
+ +
+### gi + +CLI utility to combine and manage `.gitignore` patterns across polyglot repositories. + +**Stack:** Go · **Status:** maintained + +[Docs](https://sempervent.github.io/gi/){ .md-button .md-button--primary } [Repo](https://github.com/sempervent/gi){ .md-button } + +
+ +
+### LLM Wiki Template + +Reusable MkDocs template for LLM-oriented team wikis and agent playbooks. + +**Stack:** Markdown · **Status:** maintained + +[Docs](https://sempervent.github.io/llm-wiki-template/){ .md-button .md-button--primary } [Repo](https://github.com/sempervent/llm-wiki-template){ .md-button } + +
+ +
+### music-rig + +Documentation and configuration for a home audio recording studio setup. + +**Stack:** Markdown · **Status:** maintained + +[Docs](https://sempervent.github.io/music-rig/){ .md-button .md-button--primary } [Repo](https://github.com/sempervent/music-rig){ .md-button } + +
+ +
+### OpenSAMPL + +ORNL open-source clock probe aggregation and timing analysis platform (external organization repository). + +**Stack:** Python · **Status:** maintained + +[Docs](https://ornl.github.io/OpenSAMPL/){ .md-button .md-button--primary } [Repo](https://github.com/ORNL/OpenSAMPL){ .md-button } + +
+ +
+### Paraclete + +Data exploration tool; upstream lineage merged into the PARQONAUT workspace (see PARQONAUT provenance docs). + +**Stack:** Rust · **Status:** maintained + +[Repo](https://github.com/sempervent/paraclete){ .md-button } + +
+ +
+ +## Experiments & documentation + +Wikis, comparisons, games-in-progress, and strange machinery. + +
+ +
+### Agent LLM Wiki Matrix + +Comparison framework for agentic coding systems and repo-centered signals for measuring assistant workflows. + +**Stack:** Markdown · **Status:** experimental + +[Docs](https://sempervent.github.io/agent-llm-wiki-matrix/){ .md-button .md-button--primary } [Repo](https://github.com/sempervent/agent-llm-wiki-matrix){ .md-button } + +
+ +
+### cockpit + +tmux-backed Go TUI homelab cockpit that orchestrates terminal tools as persistent workspaces. + +**Stack:** Go · **Status:** experimental + +[Repo](https://github.com/sempervent/cockpit){ .md-button } + +
+ +
+### Cosmic Architect + +Web-based planet-building game (successor to earlier pygame experiments); competitive ecosystem design. + +**Stack:** Python · **Status:** experimental + +[Repo](https://github.com/sempervent/cosmic-architect){ .md-button } + +
+ +
+### Cosmic Garden + +Cultivation / garden simulation experiment related to the Cosmic game line. + +**Stack:** Unknown · **Status:** experimental + +[Repo](https://github.com/sempervent/cosmic-garden){ .md-button } + +
+ +
+### Embers of the Earth + +Pixel-art farming game concept in a post-apocalyptic steampunk setting (Pages deploy not verified live on org site). + +**Stack:** Unknown · **Status:** experimental + +[Repo](https://github.com/sempervent/embers-of-the-earth){ .md-button } + +
+ +
+### Generative MIDI Workbench + +Experiments in algorithmic MIDI composition and performance tooling. + +**Stack:** Python · **Status:** experimental + +[Docs](https://sempervent.github.io/generative-midi-workbench/){ .md-button .md-button--primary } [Repo](https://github.com/sempervent/generative-midi-workbench){ .md-button } + +
+ +
+### MQTT Comparison + +Experimental comparison of MQTT brokers and client patterns (Pages site configured in GitHub but not currently serving — see maintainer recon). + +**Stack:** Python · **Status:** experimental + +[Repo](https://github.com/sempervent/mqtt-comparison){ .md-button } + +
+ +
+### Postgres Query Autopsy Tool + +.NET tool for inspecting and explaining PostgreSQL query plans and workload behavior. + +**Stack:** C# · **Status:** experimental + +[Docs](https://sempervent.github.io/postgres-query-autopsy-tool/){ .md-button .md-button--primary } [Repo](https://github.com/sempervent/postgres-query-autopsy-tool){ .md-button } + +
+ +
+### Smart Farm Wiki + +MkDocs knowledge base for smart-farm / homelab automation experiments. + +**Stack:** Markdown · **Status:** experimental + +[Docs](https://sempervent.github.io/smart-farm-wiki/){ .md-button .md-button--primary } [Repo](https://github.com/sempervent/smart-farm-wiki){ .md-button } + +
+ +
+### Wildfire Smoke Risk Correlator + +Geospatial analysis project correlating wildfire smoke exposure with health risk indicators (public research prototype). + +**Stack:** Python · **Status:** experimental + +[Docs](https://sempervent.github.io/wildfire-smoke-risk-correlator/){ .md-button .md-button--primary } [Repo](https://github.com/sempervent/wildfire-smoke-risk-correlator){ .md-button } + +
+ +
+ +## Historical archive + +Older web apps and prototypes — preserved, not promoted as current work. + +
+ +
+### colony + +Rust colony simulation prototype (GitHub Pages enabled but not currently serving content at the org URL). + +**Stack:** Rust · **Status:** historical + +[Repo](https://github.com/sempervent/colony){ .md-button } + +
+ +
+### Decentralized Content Reward System (DCRS) + +Web3 content-rewards experiment combining Flask services and blockchain ideas (historical prototype). + +**Stack:** Python · **Status:** historical + +[Repo](https://github.com/sempervent/dcrs){ .md-button } + +
+ +
+### Genesis + +pygame experiment: audio-input-driven procedural universe toy. + +**Stack:** Python · **Status:** historical + +[Repo](https://github.com/sempervent/genesis){ .md-button } + +
+ +
+### parqknife + +Parquet “swiss army knife” CLI; functionality consolidated into PARQONAUT (`prqnt`). + +**Stack:** Rust · **Status:** historical + +[Repo](https://github.com/sempervent/parqknife){ .md-button } + +
+ +
+### This Is A Casino + +Experimental semantic / ML-driven stock exploration and visualization stack (circa 2023). + +**Stack:** Python · **Status:** historical + +[Repo](https://github.com/sempervent/thisisacasino){ .md-button } + +
+ +
+### Where I've Been + +County-level travel map and statistics web app (earlier-generation personal project). + +**Stack:** Python · **Status:** historical + +[Repo](https://github.com/sempervent/whereivebeen){ .md-button } + +
+ +
From 26468d1499237d6d3a9303f9bbfefc0f867f809a Mon Sep 17 00:00:00 2001 From: sempervent Date: Wed, 30 Sep 2026 22:35:18 -0400 Subject: [PATCH 04/12] feat(site): refocus landing page on current work Lead with person and registry-driven current work; defer deep taxonomy below the fold. --- docs/assets/styles/custom.css | 27 +++++++ docs/index.md | 143 ++++++++-------------------------- 2 files changed, 61 insertions(+), 109 deletions(-) diff --git a/docs/assets/styles/custom.css b/docs/assets/styles/custom.css index 43cb57b..00a8174 100644 --- a/docs/assets/styles/custom.css +++ b/docs/assets/styles/custom.css @@ -244,6 +244,33 @@ text-decoration: underline; } +/* Project cards (registry-generated) */ +.project-card-grid { + display: grid; + grid-template-columns: repeat(auto-fit, minmax(260px, 1fr)); + gap: 1.25em; + margin: 1.5em 0; +} + +.project-card { + background: var(--md-default-bg-color); + border: 1px solid var(--md-default-fg-color--lightest); + border-radius: 8px; + padding: 1.25em 1.35em; + box-shadow: 0 1px 3px rgba(0, 0, 0, 0.05); +} + +.project-card h3 { + margin-top: 0; + font-size: 1.15em; + color: var(--md-primary-fg-color); +} + +.project-card .md-button { + margin: 0.25em 0.35em 0.25em 0; + font-size: 0.85em; +} + /* ============================================ Navigation & Sidebar ============================================ */ diff --git a/docs/index.md b/docs/index.md index db74542..4df0dd5 100644 --- a/docs/index.md +++ b/docs/index.md @@ -2,92 +2,64 @@ # Geospatial Systems, Data, and Strange Machinery -*Building production-scale systems at the intersection of geospatial data, cloud infrastructure, and distributed computing* +**Joshua N. Grant** — geospatial systems architect, data infrastructure engineer, and builder of developer tooling and experimental software.
- [Best Practices →](best-practices/index.md){ .md-button .md-button--primary } - [Tutorials →](tutorials/index.md){ .md-button .md-button--primary } - [Projects →](projects.md){ .md-button .md-button--primary } + [Current work ↓](#current-work){ .md-button .md-button--primary } + [Projects →](projects/index.md){ .md-button .md-button--primary } + [Technical writing →](documentation.md){ .md-button .md-button--primary } + [About →](about.md){ .md-button } + [GitHub →](https://github.com/sempervent){ .md-button }
--- -## Welcome +## Current work {#current-work} -This is my technical studio—a curated collection of best practices, tutorials, and experiments from building production systems at **Oak Ridge National Laboratory** and beyond. Here you'll find deep dives into geospatial data engineering, distributed systems architecture, and the practical patterns that make complex systems reliable. +Independent projects with recent public activity (from the [canonical registry](https://github.com/sempervent/sempervent.github.io/blob/main/data/projects.yaml)): -!!! tip "What to Expect" - This site is organized into **Best Practices** (conceptual guides and patterns) and **Tutorials** (step-by-step implementations). Both are written for engineers who need production-ready solutions, not just examples. +--8<-- "_generated/home-current-work.md" ---- - -## Quick Navigation +[Full project index →](projects/index.md) · [Live project docs sites →](projects/documentation-sites.md) -
- -
- -### 🎯 Best Practices - -Production-grade patterns, architectures, and methodologies for building reliable distributed systems. +--- -**[Explore Best Practices →](best-practices/index.md)** +## Technical writing -**Highlights:** -- [System Resilience & Concurrency](best-practices/operations-monitoring/system-resilience-and-concurrency.md) -- [Configuration Management](best-practices/operations-monitoring/configuration-management.md) -- [Release Management & Progressive Delivery](best-practices/operations-monitoring/release-management-and-progressive-delivery.md) -- [IAM & RBAC Governance](best-practices/security/iam-rbac-abac-governance.md) +Long-form **best practices**, **deep dives**, **ADRs**, and **tutorials** live here — production-derived patterns where noted, reference implementations elsewhere, and experiments clearly in the Lab. -
+
-### 📚 Tutorials - -Step-by-step guides with copy-paste examples for implementing key technologies and patterns. - -**[Browse Tutorials →](tutorials/index.md)** +### Start with intent -**Popular:** -- [PostGIS Geometry Indexing](tutorials/database-data-engineering/postgis-geometry-indexing.md) -- [PostgreSQL Auditing with PgAudit](tutorials/database-data-engineering/postgres-pgaudit-pgcron-auditing.md) -- [RKE2 on Raspberry Pi](tutorials/docker-infrastructure/rke2-raspberry-pi.md) -- [ONNX Browser Inference](tutorials/ml-ai/onnx-browser-inference.md) +**[Architectural compass →](start-here-architectural-paths.md)** — pick a problem domain and reading path.
-### 🏗️ Projects - -Technical implementations, experiments, and creative explorations at the edge of what's possible. - -**[View Projects →](projects.md)** +### Representative guides -**Featured:** -- Final Fantasy Football (semantic learning) -- Where I've Been (travel visualization) -- This Is A Casino (trading platform) -- Decentralized Content Reward System +- [Geospatial system architecture](best-practices/geospatial/geospatial-system-design.md) +- [PostGIS best practices](best-practices/postgres/postgis-best-practices.md) +- [Parquet](best-practices/database-data/parquet.md) · [GeoParquet](best-practices/database-data/geoparquet.md) +- [System resilience & concurrency](best-practices/operations-monitoring/system-resilience-and-concurrency.md) +- [ADR & decision governance](best-practices/architecture-design/adr-decision-governance.md) +- [Release management & progressive delivery](best-practices/operations-monitoring/release-management-and-progressive-delivery.md)
-### 🎨 Just for Fun +### Browse the corpus -Creative, experimental, and occasionally absurd implementations — from generative art to MIDI-driven servers. +**[Best practices →](best-practices/index.md)** · **[Deep dives →](deep-dives/index.md)** · **[Tutorials →](tutorials/index.md)** -**[Explore Just for Fun →](tutorials/just-for-fun/index.md)** - -**Recent:** -- [Pi Sample Library Server](tutorials/just-for-fun/pi-sample-server.md) -- [Recursive Cathedral Generator](tutorials/just-for-fun/kotlin-recursive-cathedral.md) -- [Fractal Art Explorer](tutorials/just-for-fun/fractal-art-explorer-js.md) -- [OSC + MQTT + SuperCollider](tutorials/just-for-fun/osc-mqtt-prometheus-supercollider.md) +Curated additions: **[What's New](whats-new.md)** (updated when substantial docs land — not a live feed).
@@ -95,67 +67,22 @@ Creative, experimental, and occasionally absurd implementations — from generat --- -## 📌 Start Here - -If you're new to this site, these foundational guides will give you the most value: - -1. **[ADR and Technical Decision Governance](best-practices/architecture-design/adr-decision-governance.md)** — How we make and document architectural decisions -2. **[Configuration Management](best-practices/operations-monitoring/configuration-management.md)** — Managing configs across multi-environment systems -3. **[System Resilience & Concurrency](best-practices/operations-monitoring/system-resilience-and-concurrency.md)** — Patterns for building resilient distributed systems -4. **[IAM & RBAC Governance](best-practices/security/iam-rbac-abac-governance.md)** — Identity and access management across heterogeneous stacks - ---- - -## Featured Deep Dives - -### Parquet & Data Warehouses - -**[Fast queries & cheap storage](best-practices/database-data/parquet.md)** — How to lay out partitions, size files and row groups, enable predicate pushdown & column pruning, serve efficiently over S3 byte-range, and wire up `parquet_s3_fdw` in Postgres for pushdown. - -### Geospatial Data Engineering - -**[PostGIS Best Practices](best-practices/postgres/postgis-best-practices.md)** — Production patterns for spatial indexing, raster workflows, and large-scale geospatial data management. - -### Release Management - -**[Release Management & Progressive Delivery](best-practices/operations-monitoring/release-management-and-progressive-delivery.md)** — Safe deployment strategies for coordinating changes across applications, databases, data pipelines, and ML systems. - ---- - -## What I'm Working On +## Lab & experiments -Currently focused on: +Games, generative art, MIDI, Raspberry Pi builds, and other **non-production** explorations. -- **[Advanced geospatial data warehouse architectures](best-practices/database-data/geoparquet-data-warehouses.md)** using GeoParquet and PostGIS -- **[Real-time IoT tracking systems](tutorials/database-data-engineering/kafka-timescaledb-iot.md)** with Kafka, TimescaleDB, and geospatial processing -- **[Production-grade configuration management](best-practices/operations-monitoring/configuration-management.md)** for multi-environment distributed systems -- **[ML model deployment pipelines](tutorials/ml-ai/mcp-mlflow-toolchain.md)** with ONNX, MLflow, and progressive delivery +**[Lab overview →](lab/index.md)** · **[Just for Fun tutorials →](tutorials/just-for-fun/index.md)** --- -## About This Site +## About this site -This documentation represents years of building production systems, learning from failures, and refining patterns that actually work. Everything here is battle-tested in real environments—from air-gapped clusters to cloud-native architectures. +Personal portfolio and technical notebook. Content mixes ORNL-informed engineering practice, independent open source, tutorials, and creative experiments — not everything is production-derived. -The content is organized to be **immediately useful**: copy-paste examples, production-ready configurations, and clear explanations of trade-offs. No fluff, no theory without practice. +- Production-oriented guides are labeled as patterns from real systems where that applies. +- Lab and Just for Fun material is experimental by design. -**[Learn more about me →](about.md)** - ---- - -## Latest Updates - -**Feb 2026:** -- [Recursive Cathedral Generator (Kotlin + Processing)](tutorials/just-for-fun/kotlin-recursive-cathedral.md) — L-system generative architecture -- [Pi-Based Sample Library Server](tutorials/just-for-fun/pi-sample-server.md) — WebAudio + SQLite + USB MIDI on a Raspberry Pi - -**2025:** -- [Vibe → Agentic LLMs](best-practices/ml-ai/vibe-to-agentic.md) — Production agentic LLM architecture -- [IAM & RBAC Governance](best-practices/security/iam-rbac-abac-governance.md) — Identity and access across heterogeneous stacks -- [Release Management & Progressive Delivery](best-practices/operations-monitoring/release-management-and-progressive-delivery.md) — Safe deployments at scale -- [ONNX Browser Inference](tutorials/ml-ai/onnx-browser-inference.md) — Run ML models client-side - -**[See all recent additions →](whats-new.md)** +**[Professional profile →](about.md)** · **[Contact →](getting-started.md)** --- @@ -165,5 +92,3 @@ The content is organized to be **immediately useful**: copy-paste examples, prod - **LinkedIn**: [Joshua N. Grant](https://linkedin.com/in/joshuanagrant) - **Email**: [jngrant@live.com](mailto:jngrant@live.com) - **Blog**: [Not Just a Datum](https://notjustadatum.blogspot.com) - -[Contact & Collaboration →](getting-started.md) From 5aea81adfde19a5918968cd4359cac24c4fa7a94 Mon Sep 17 00:00:00 2001 From: sempervent Date: Wed, 30 Sep 2026 22:35:18 -0400 Subject: [PATCH 05/12] refactor(site): simplify top-level navigation Group doctrine and docs under Writing; add Projects, Lab, and About tabs with redirects. --- mkdocs.yml | 883 +++++++++++++++++++++-------------------- scripts/rebuild_nav.py | 64 +++ 2 files changed, 514 insertions(+), 433 deletions(-) create mode 100644 scripts/rebuild_nav.py diff --git a/mkdocs.yml b/mkdocs.yml index f5e65c9..749683b 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -65,441 +65,452 @@ theme: nav: - Home: index.md - - What's New: whats-new.md - - Tags: tags.md - - Professional Profile: about.md - - Projects: projects.md - - Technical Documentation: documentation.md - - Doctrine: - - Start Here — Architectural Compass: start-here-architectural-paths.md - - Reading Tracks: reading-tracks.md - - Decision Frameworks: decision-frameworks.md - - Anti-Patterns: anti-patterns.md - - Philosophy: philosophy.md - - Systems Glossary: systems-glossary.md - - Diagrams: - - Diagram Style Guide: diagrams/style-guide.md - - Architecture Decisions: - - Overview: adr/index.md - - "ADR-0001: Site Architecture": adr/0001-site-architecture.md - - "ADR-0002: MkDocs + Material": adr/0002-why-mkdocs-material.md - - "ADR-0003: Best Practices vs Tutorials": adr/0003-why-best-practices-vs-tutorials.md - - "ADR-0004: Why Just for Fun Exists": adr/0004-why-just-for-fun-exists.md - - "ADR-0005: ESP32 Section Architecture": adr/0005-esp32-section-architecture.md - - "ADR-0006: Embedded Expansion (LoRa + Power)": adr/0006-embedded-expansion-lora-power.md - - "ADR-0007: Deep Dives Section": adr/0007-deep-dives-section.md - - "ADR-0008: Expand Deep Dives Section": adr/0008-expand-deep-dives-section.md - - "ADR-0009: Deep Dives Expansion": adr/0009-deep-dives-expansion.md - - "ADR-0010: Deep Dives Governance": adr/0010-deep-dives-governance.md - - "ADR-0011: Deep Dives Expansion Phase II": adr/0011-deep-dives-expansion-phase-2.md - - "ADR-0012: Deep Dives Scale Governance": adr/0012-deep-dives-scale-governance.md - - "ADR-0013: Deep Dives Curation Model": adr/0013-deep-dives-curation-model.md - - "ADR-0014: Elevate Site to Systems Doctrine": adr/0014-elevate-site-to-systems-doctrine.md - - "ADR-0015: Standardize Diagrams on Mermaid": adr/0015-diagrams-mermaid.md - - Best Practices: - - Overview: best-practices/index.md - - 📌 Start Here: - - ADR and Technical Decision Governance: best-practices/architecture-design/adr-decision-governance.md - - Configuration Management: best-practices/operations-monitoring/configuration-management.md - - System Resilience & Concurrency: best-practices/operations-monitoring/system-resilience-and-concurrency.md - - IAM & RBAC Governance: best-practices/security/iam-rbac-abac-governance.md - - Data: - - Reproducible Data Pipelines: best-practices/data/reproducible-data-pipelines.md - - Metadata as Control Plane: best-practices/data/metadata-control-plane.md - - Architecture: - - Cost-Aware System Architecture: best-practices/architecture/cost-aware-systems.md - - Geospatial: - - Geospatial System Architecture: best-practices/geospatial/geospatial-system-design.md - - Operations: - - Failure-Oriented System Design: best-practices/operations/failure-oriented-design.md - - Data Processing: - - Spark: - - When to Use Spark: best-practices/data-processing/spark/when-to-use-spark.md - - Scaling Spark Clusters: best-practices/data-processing/spark/scaling-spark.md - - Spark on Kubernetes: best-practices/data-processing/spark/spark-on-kubernetes.md - - Spark Performance Tuning: best-practices/data-processing/spark/spark-performance-tuning.md - - Spark in Modern Architectures: best-practices/data-processing/spark/spark-modern-architecture.md - - 🐍 Python Development: - - Overview: best-practices/python/index.md - - "Developer Experience (DX) as Infrastructure: Golden Paths, Tooling Ecosystems & Workflow Automation": best-practices/python/dx-architecture-and-golden-paths.md - - Python Package: best-practices/python/python-package.md - - Typing in Python: best-practices/python/typing-in-python.md - - Python Concurrency: best-practices/python/python-threading-and-multiprocessing.md - - Python Async Best Practices: best-practices/python/python-async-best-practices.md - - Pytest Best Practices: best-practices/python/pytest-best-practices.md - - API Development: best-practices/python/api-development.md - - FastAPI Geospatial: best-practices/python/fastapi-geospatial.md - - Web Performance Optimization: best-practices/python/web-performance-optimization.md - - TUI Applications: best-practices/python/tui-applications.md - - 🦀 Rust Development: - - Overview: best-practices/rust/index.md - - Rust Development Environment: best-practices/rust/rust-dev-environment.md - - TUI Applications: best-practices/rust/tui-applications.md - - 🐹 Go Development: - - Overview: best-practices/go/index.md - - Go Development Environment: best-practices/go/go-dev-environment.md - - TUI Applications: best-practices/go/tui-applications.md - - 📊 R Development: - - Overview: best-practices/r/index.md - - R Development Environment: best-practices/r/r-dev-environment.md - - 🐳 Docker & Infrastructure: - - Overview: best-practices/docker-infrastructure/index.md - - Docker & Compose: best-practices/docker-infrastructure/docker-and-compose.md - - Conda to Docker Migration: best-practices/docker-infrastructure/conda-to-docker-migration.md - - SBOMs, Trivy Scans, and Automated CVE Mitigation: best-practices/docker-infrastructure/docker-sbom-trivy-cve-mitigation.md - - Nginx Production: best-practices/docker-infrastructure/nginx-production.md - - NGINX Best Practices: best-practices/docker-infrastructure/nginx-best-practices.md - - Ansible Inventory Management: best-practices/docker-infrastructure/ansible-inventory-management.md - - Ansible Playbook Design: best-practices/docker-infrastructure/ansible-playbook-design.md - - Ansible Security Hardening: best-practices/docker-infrastructure/ansible-security-hardening.md - - Ansible Performance Optimization: best-practices/docker-infrastructure/ansible-performance-optimization.md - - Jinja Best Practices: best-practices/docker-infrastructure/jinja-best-practices.md - - Advanced Tmux Workflows: best-practices/docker-infrastructure/tmux-advanced.md - - 📝 Git & Version Control: - - Overview: best-practices/git/index.md - - Git Production: best-practices/git/git-production.md - - Git Workflows & Collaboration: best-practices/git/git-workflows-collaboration.md - - Modern GitFlow Best Practices: best-practices/git/gitflow-best-practices.md - - 🗄️ Database & Data Management: - - Overview: best-practices/database-data/index.md - - "Cross-System Data Lineage, Inter-Service Metadata Contracts & Provenance Enforcement": best-practices/database-data/data-lineage-contracts.md - - Semantic Layer Engineering, Domain Models, and Knowledge Graph Alignment: best-practices/database-data/semantic-layer-engineering.md - - "AI-Ready, ML-Enabled Geospatial Knowledge Graph": best-practices/database-data/ai-ml-geospatial-knowledge-graph.md - - Parquet: best-practices/database-data/parquet.md - - GeoParquet: best-practices/database-data/geoparquet.md - - Patroni PostgreSQL HA: best-practices/database-data/patroni-postgres-ha.md - - Database Optimization: best-practices/database-data/database-optimization.md - - Database Migrations & Schema Evolution: best-practices/database-data/database-migrations.md - - GeoParquet Data Warehouses: best-practices/database-data/geoparquet-data-warehouses.md - - AWS Serverless Geospatial: best-practices/database-data/aws-serverless-geospatial.md - - Lakes vs Lakehouses vs Warehouses: best-practices/database-data/lake-vs-lakehouse-vs-warehouse.md - - Data Lake Governance: best-practices/database-data/data-lake-governance.md - - Geospatial Data Engineering: best-practices/database-data/geospatial-data-engineering.md - - Geospatial Benchmarking: best-practices/database-data/geospatial-benchmarking.md - - 🐘 PostgreSQL Development: - - Overview: best-practices/postgres/index.md - - Core & Design: - - Development Environment: best-practices/postgres/postgres-dev-environment.md - - Database Design: best-practices/postgres/postgres-database-design.md - - Data Types: best-practices/postgres/postgres-data-types.md - - Constraints & Validation: best-practices/postgres/postgres-constraints-validation.md - - Transactions & Concurrency: best-practices/postgres/postgres-transactions-concurrency.md - - Indexing Strategies: best-practices/postgres/postgres-indexing-strategies.md - - Security Best Practices: best-practices/postgres/postgres-security-best-practices.md - - Performance & Operations: - - Performance Tuning: best-practices/postgres/postgres-performance-tuning.md - - Monitoring & Observability: best-practices/postgres/postgres-monitoring-observability.md - - Backup & Recovery: best-practices/postgres/postgres-backup-recovery.md - - Replication & High Availability: best-practices/postgres/postgres-replication-ha.md - - Maintenance & Vacuum: best-practices/postgres/postgres-maintenance-vacuum.md - - Scaling Strategies: best-practices/postgres/postgres-scaling-strategies.md - - Connection Pooling: best-practices/postgres/postgres-pooling.md - - Troubleshooting: best-practices/postgres/postgres-troubleshooting.md - - Configuration Management: best-practices/postgres/postgres-configuration-management.md - - Advanced Features: - - Extensions: best-practices/postgres/postgres-extensions.md - - JSON & JSONB: best-practices/postgres/postgres-json-jsonb.md - - Full-Text Search: best-practices/postgres/postgres-fulltext-search.md - - Partitioning: best-practices/postgres/postgres-partitioning.md - - Time Series Data: best-practices/postgres/postgres-timeseries.md - - Large Object Storage: best-practices/postgres/postgres-large-objects.md - - Foreign Data Wrappers: best-practices/postgres/fdw-postgres.md - - PostGIS Best Practices: best-practices/postgres/postgis-best-practices.md - - Integration & Deployment: - - API Development: best-practices/postgres/postgres-api-development.md - - Event-Driven Architecture: best-practices/postgres/postgres-event-driven.md - - Data Pipeline Integration: best-practices/postgres/postgres-data-pipeline-integration.md - - Cloud Integration: best-practices/postgres/postgres-cloud-integration.md - - Containerization: best-practices/postgres/postgres-containerization.md - - Deployment Strategies: best-practices/postgres/postgres-deployment-strategies.md - - Data Engineering: best-practices/database-data/data-engineering.md - - ETL Pipeline Design: best-practices/database-data/etl-pipeline-design.md - - 📊 Data Governance: - - Overview: best-practices/data-governance/index.md - - Metadata Standards, Schema Governance & Data Provenance: best-practices/data-governance/metadata-provenance-contracts.md - - Data Validation and Contract Governance: best-practices/data-governance/data-validation-and-contract-governance.md - - Data Freshness, SLA/SLO Governance, and Pipeline Reliability Contracts: best-practices/data-governance/data-freshness-sla-governance.md - - Data Retention, Archival Strategy, Lifecycle Governance & Cold Storage Patterns: best-practices/data-governance/data-retention-archival-lifecycle-governance.md - - Data Quality SLAs, Validation Layers, and Observability for Tabular, Geospatial, and ML Data: best-practices/data-governance/data-quality-sla-validation-observability.md - - 🤖 Machine Learning & AI: - - Overview: best-practices/ml-ai/index.md - - "ML Systems Architecture: Feature Stores, Model Serving, Experiment Governance, and Cross-System Reproducibility": best-practices/ml-ai/ml-systems-architecture-governance.md - - Prompting LLMs: best-practices/ml-ai/prompting-llms.md - - ONNX Model Optimization: best-practices/ml-ai/onnx-model-optimization.md - - MCP + FastAPI Full Stack: best-practices/ml-ai/mcp-fastapi-stack.md - - Embeddings & Vector Databases: best-practices/ml-ai/embeddings-and-vector-databases.md - - Vibe → Agentic LLMs: best-practices/ml-ai/vibe-to-agentic.md - - R Data Exploration: best-practices/ml-ai/r-data-exploration.md - - 🏗️ Architecture & Design: - - Overview: best-practices/architecture-design/index.md - - System-Wide Naming, Taxonomy, and Structural Vocabulary Governance: best-practices/architecture-design/system-taxonomy-governance.md - - Cost-Aware Architecture & Resource-Efficiency Governance: best-practices/architecture-design/cost-aware-architecture-and-efficiency-governance.md - - Holistic Capacity Planning, Scaling Economics, and Workload Modeling: best-practices/architecture-design/capacity-planning-and-workload-modeling.md - - Multi-Region, Multi-Cluster Disaster Recovery, Failover Topologies, and Data Sovereignty: best-practices/architecture-design/multi-region-dr-strategy.md - - Cross-Environment Configuration Strategy and Multi-Cluster State Management: best-practices/architecture-design/environment-config-governance.md - - Cognitive Load Management and Developer Experience: best-practices/architecture-design/cognitive-load-developer-experience.md - - Architectural Fitness Functions and Governance: best-practices/architecture-design/architecture-fitness-functions-governance.md - - Temporal Governance and Time Synchronization: best-practices/architecture-design/temporal-governance-and-time-synchronization.md - - Repository Standardization and Governance: best-practices/architecture-design/repository-standardization-and-governance.md - - Event-Driven Architecture: best-practices/architecture-design/event-driven-architecture.md - - "Streaming Architecture Patterns: SAGA, CQRS, and Outbox": best-practices/architecture-design/streaming-architecture-patterns.md - - API Governance, Backward Compatibility Rules, and Cross-Language Interface Stability: best-practices/architecture-design/api-governance-interface-stability.md - - API Gateway Architecture: best-practices/architecture-design/api-gateway-architecture.md - - Multi-Cloud Federation & Portability Architecture: best-practices/architecture-design/multi-cloud-federation-portability.md - - Cloud Architecture: best-practices/architecture-design/cloud-architecture.md - - Secrets Management: best-practices/architecture-design/secrets-management.md - - Testing & CI/CD Pipelines: best-practices/architecture-design/ci-cd-pipelines.md - - Documentation: best-practices/architecture-design/documentation.md - - ADR and Technical Decision Governance: best-practices/architecture-design/adr-decision-governance.md - - Caching & Performance Layers: best-practices/architecture-design/caching-performance.md - - Cache-Topology Architecture: best-practices/architecture-design/cache-topology-architecture.md - - Data Mesh Architecture: best-practices/architecture-design/data-mesh-architecture.md - - Service Decomposition Strategy: best-practices/architecture-design/service-decomposition-strategy.md - - Polyglot Interoperability Design: best-practices/architecture-design/polyglot-interoperability-design.md - - Reference Architecture Diagrams: best-practices/architecture-design/reference-architecture-diagrams.md - - RDF/OWL Metadata Automation: best-practices/architecture-design/rdf-owl-metadata-automation.md - - Protocol Buffers with Python: best-practices/architecture-design/protobuf-python.md - - 🔧 Operations & Monitoring: - - Overview: best-practices/operations-monitoring/index.md - - "Observability as Architecture: Unified Telemetry Models Across Clusters, Services, and Languages": best-practices/operations-monitoring/unified-observability-architecture.md - - Chaos Engineering, Fault Injection, and Reliability Validation: best-practices/operations-monitoring/chaos-engineering-governance.md - - Operational Risk Modeling, Blast Radius Reduction & Failure Domain Architecture: best-practices/operations-monitoring/blast-radius-risk-modeling.md - - Cross-Environment Configuration Drift Prevention, Promotion Workflows & Release Channels: best-practices/operations-monitoring/environment-promotion-drift-governance.md - - Observability-Driven Development (ODD), Telemetry-First Coding Practices, and Preemptive Debugging Architecture: best-practices/operations-monitoring/observability-driven-development.md - - Operational Resilience and Incident Response: best-practices/operations-monitoring/operational-resilience-and-incident-response.md - - Performance Monitoring: best-practices/operations-monitoring/performance-monitoring.md - - System Resilience, Rate Limiting, Concurrency Control & Backpressure: best-practices/operations-monitoring/system-resilience-and-concurrency.md - - Configuration Management, Secrets Lifecycle, and Multi-Environment Drift Control: best-practices/operations-monitoring/configuration-management.md - - Cross-Environment Configuration Drift Detection & Prevention: best-practices/operations-monitoring/configuration-drift-detection-prevention.md - - Release Management, Change Governance, and Progressive Delivery: best-practices/operations-monitoring/release-management-and-progressive-delivery.md - - Structured Logging & Observability: best-practices/operations-monitoring/logging-observability.md - - Grafana, Prometheus, Loki, and Observability: best-practices/operations-monitoring/grafana-prometheus-loki-observability.md - - Testing Best Practices: best-practices/operations-monitoring/testing-best-practices.md - - Secrets & Configuration Management: best-practices/operations-monitoring/secrets-config.md - - Grafana: best-practices/operations-monitoring/grafana.md - - 🔒 Security: - - Overview: best-practices/security/index.md - - End-to-End Secrets Management & Key Rotation: best-practices/security/secrets-governance.md - - Identity & Access Management, RBAC/ABAC, and Least-Privilege Governance: best-practices/security/iam-rbac-abac-governance.md - - "Secure-by-Design Lifecycle Architecture Across Polyglot Systems": best-practices/security/secure-by-design-polyglot.md - - Secure Computes, Sandboxing, and Multi-Tenant Isolation for Polyglot Systems: best-practices/security/secure-sandboxing-and-multi-tenant-isolation.md - - Cross-Domain Identity Federation, AuthZ/AuthN Architecture & Identity Propagation Models: best-practices/security/identity-federation-authz-authn-architecture.md - - Secret Supply Chains, Encryption Lifecycle Management & Cryptographic Rotation Strategy: best-practices/security/encryption-lifecycle-and-crypto-rotation.md - - ⚡ Performance: - - Overview: best-practices/performance/index.md - - End-to-End Caching Strategy: best-practices/performance/end-to-end-caching-strategy.md - - 🧪 Testing: - - Overview: best-practices/testing/index.md - - End-to-End Testing Strategy: best-practices/testing/end-to-end-testing-strategy.md - - 🖼️ Diagrams: - - Systems Diagramming Best Practices: best-practices/diagrams/systems-diagramming-best-practices.md - - SVG Workflow Generation: best-practices/diagrams/svg-workflow-generation.md - - 🎨 Creative & Fun: - - Overview: best-practices/creative-fun/index.md - - Time Hygiene (UTC, TZ, Clocks): best-practices/creative-fun/time-hygiene.md - - Idempotency & De-dup: best-practices/creative-fun/idempotency-and-dedup.md - - Celery Tasks: best-practices/creative-fun/celery-best-practices.md - - LaTeX Workflows: best-practices/creative-fun/latex.md - - YAML Recipe Format: best-practices/creative-fun/yaml-recipe-format.md - - 🔌 Embedded Systems & ESP32: - - Overview: best-practices/esp32/index.md - - Programming Architecture: best-practices/esp32/esp32-programming-architecture.md - - Power Management & Deep Sleep: best-practices/esp32/power-management-and-deep-sleep.md - - Hardware & Electrical Safety: best-practices/esp32/esp32-hardware-and-electrical-safety.md - - Embedded Security & OTA: best-practices/esp32/embedded-security-and-ota.md - - Sensor Integration: best-practices/esp32/sensor-integration-best-practices.md - - E-Ink Display Integration: best-practices/esp32/e-ink-display-best-practices.md - - MQTT Security: best-practices/esp32/mqtt-security-best-practices.md - - LoRa Best Practices (SX127x): best-practices/esp32/lora-best-practices-sx127x.md - - Safety Checklist (Printable): best-practices/esp32/esp32-safety-checklist-printable.md - - ESP32-S3 and C3 Notes: best-practices/esp32/esp32-s3-and-c3-architecture-notes.md - - 🔋 Power Electronics & Embedded Hardware: - - Overview: best-practices/embedded/index.md - - Power Electronics for ESP32: best-practices/embedded/power-electronics-for-esp32.md - - 🏠 Home Automation & MQTT: - - Overview: best-practices/home-automation/index.md - - Home Assistant Security: best-practices/home-automation/home-assistant-security-best-practices.md - - Deep Dives: - - Overview: deep-dives/index.md - - 📦 Data Formats & Storage: - - Parquet vs CSV vs ORC vs Avro: deep-dives/parquet-vs-csv-orc-avro.md - - Geospatial File Format Choices: deep-dives/geospatial-file-format-choices.md - - Polars vs Pandas for Geospatial Data: deep-dives/polars-vs-pandas-geospatial.md - - The Physics of Storage Systems: deep-dives/the-physics-of-storage-systems.md - - The Operational Geometry of Spatial Systems: deep-dives/the-operational-geometry-of-spatial-systems.md - - 🧩 Systems Design & Architecture: - - Why Most Microservices Should Be Monoliths: deep-dives/why-most-microservices-should-be-monoliths.md - - Appropriate Use of Microservices: deep-dives/appropriate-use-of-microservices.md - - Distributed Systems and the Myth of Infinite Scale: deep-dives/distributed-systems-myth-of-infinite-scale.md - - Distributed Systems Architecture: deep-dives/distributed-systems-architecture.md - - Event-Driven Architecture: deep-dives/event-driven-architecture.md - - Raft Consensus Explained: deep-dives/raft-consensus-explained.md - - Why Most Kubernetes Clusters Shouldn't Exist: deep-dives/why-most-kubernetes-clusters-shouldnt-exist.md - - When to Use a TUI, CLI, or WebApp: deep-dives/when-to-use-tui-cli-or-webapp.md - - Blockchain vs Hashchain: deep-dives/blockchain-vs-hashchain.md - - Merkle Trees Explained: deep-dives/merkle-trees-explained.md - - Proof of Work Explained: deep-dives/proof-of-work-explained.md - - 🔭 Operations & Reliability: - - Observability vs Monitoring: deep-dives/observability-vs-monitoring.md - - The Economics of Observability: deep-dives/the-economics-of-observability.md - - Designing Resilient Distributed Systems: deep-dives/resilient-distributed-systems.md - - 🐳 Infrastructure & Automation: - - Container Base Image Philosophy: deep-dives/container-base-image-philosophy.md - - IaC vs GitOps: deep-dives/iac-vs-gitops.md - - The Human Cost of Automation: deep-dives/the-human-cost-of-automation.md - - The Economics of GPU Infrastructure: deep-dives/the-economics-of-gpu-infrastructure.md - - Building a Bitcoin Mining Rig: deep-dives/building-a-bitcoin-mining-rig.md - - Converting Bitcoin Mining to LLM Clusters: deep-dives/converting-bitcoin-mining-to-llm.md - - The Myth of Serverless Simplicity: deep-dives/the-myth-of-serverless-simplicity.md - - 🏗️ Data Systems & Architecture: - - Lakehouse vs Warehouse vs Database: deep-dives/lakehouse-vs-warehouse-vs-database.md - - DuckDB vs PostgreSQL vs Spark: deep-dives/duckdb-vs-postgres-vs-spark.md - - Why Most Data Pipelines Fail: deep-dives/why-most-data-pipelines-fail.md - - Why Most Data Lakes Become Data Swamps: deep-dives/why-data-lakes-become-swamps.md - - Prefect vs Airflow: deep-dives/prefect-vs-airflow.md - - Metadata as Infrastructure: deep-dives/metadata-as-infrastructure.md - - The Hidden Cost of Real-Time Systems: deep-dives/the-hidden-cost-of-real-time-systems.md - - The End of the Data Warehouse?: deep-dives/the-end-of-the-data-warehouse.md - - The Hidden Cost of Metadata Debt: deep-dives/the-hidden-cost-of-metadata-debt.md - - Why Most ML Systems Fail in Production: deep-dives/why-ml-systems-fail-in-production.md - - Why Spark Clusters Fail in Production: deep-dives/why-spark-clusters-fail.md - - The Economics of Distributed Data Processing: deep-dives/economics-of-distributed-data-processing.md - - Spark vs DuckDB vs Polars at Scale: deep-dives/spark-vs-duckdb-vs-polars.md - - The Myth of Infinite Data Scale: deep-dives/myth-of-infinite-data-scale.md - - Why Most Data Lakes Become Data Swamps: deep-dives/data-lakes-become-data-swamps.md - - The Metadata Crisis in Modern Data Platforms: deep-dives/metadata-crisis-in-modern-data-platforms.md - - Why Most Data Pipelines Are Operationally Fragile: deep-dives/why-data-pipelines-are-operationally-fragile.md - - 📡 Embedded & Radio: - - LoRaWAN vs Raw LoRa: deep-dives/lorawan-vs-raw-lora.md - - MQTT vs HTTP in IoT Systems: deep-dives/mqtt-vs-http-iot.md - - ESP32 vs Raspberry Pi: deep-dives/esp32-vs-raspberry-pi.md - - Tutorials: - - Overview: tutorials/index.md - - 🚀 Quick Start: - - Overview: tutorials/quick-start/index.md - - Creating MkDocs GitHub Site: tutorials/quick-start/creating-mkdocs-github-site.md - - Monitoring with Grafana & Prometheus: tutorials/quick-start/monitoring-with-grafana-prometheus.md - - 🐍 Python Development: - - Overview: tutorials/python-development/index.md - - psycopg2 to psycopg 3 Migration: tutorials/python-development/psycopg2-to-psycopg3-migration.md - - Ruff Check Ignore in pyproject.toml: tutorials/python-development/ruff-check-ignore-pyproject.md - - Class-Based NiceGUI Pages and Integrations: tutorials/python-development/nicegui-class-based-pages.md - - Advanced NiceGUI Architecture: tutorials/python-development/advanced-nicegui-architecture.md - - Distributed NiceGUI Architecture with Redis: tutorials/python-development/distributed-nicegui-redis.md - - R Shiny Geospatial App: tutorials/python-development/r-shiny-geoapp.md - - Click CLI to FastAPI Conversion: tutorials/python-development/click-to-fastapi-conversion.md - - WebSocket Chat with FastAPI: tutorials/python-development/websocket-chat-fastapi.md - - Glitch Observatory (JS): tutorials/python-development/js-glitch-observatory.md - - Chaos Engineering with Kubernetes and Python: tutorials/python-development/chaos-engineering-k8s-python.md - - Building a Python TUI: tutorials/python-development/building-a-python-tui.md - - 🦀 Rust Development: - - Overview: tutorials/rust-development/index.md - - Rust + CSR (Parse & Build from Parquet/DB): tutorials/rust-development/rust-csr-parquet-db.md - - Event-Sourcing in Rust: tutorials/rust-development/rust-event-sourcing.md - - Building a Rust TUI: tutorials/rust-development/building-a-rust-tui.md - - 🐹 Go Development: - - Overview: tutorials/go-development/index.md - - Building a Go TUI: tutorials/go-development/building-a-go-tui.md - - 🐳 Docker & Infrastructure: - - Overview: tutorials/docker-infrastructure/index.md - - Multi-Stage Docker (Conda → scratch): tutorials/docker-infrastructure/multistage-conda-to-scratch.md - - Slim Geospatial + GPU Containers (GDAL): tutorials/docker-infrastructure/slim-geospatial-gpu-conda.md - - Slimming GPU Docker Images: tutorials/docker-infrastructure/slim-gpu-docker-images.md - - Slimming TensorFlow GPU Images: tutorials/docker-infrastructure/slim-tf-gpu-images.md - - TensorFlow GPU Slim Images - Repository Skeleton: tutorials/docker-infrastructure/slim-tf-gpu-images-skeleton.md - - Compose Profiles Polyglot Stack: tutorials/docker-infrastructure/compose-profiles-polyglot-stack.md - - RKE2 on Raspberry Pi Farm: tutorials/docker-infrastructure/rke2-raspberry-pi.md - - Building a 2×8 RKE2 Cluster with Rancher, PGO, and Prefect: tutorials/docker-infrastructure/ansible-rke2-rancher-pgo-prefect.md - - ZFS Tank with OS on NVMe: tutorials/docker-infrastructure/zfs-tank-nvme.md - - Deploy SLURM with Ansible on Raspberry Pi Cluster: tutorials/docker-infrastructure/ansible-slurm-raspberrypi.md - - Harbor Container Registry Setup: tutorials/docker-infrastructure/harbor-registry-setup.md - - Dask with Ansible (CPU + GPU): tutorials/docker-infrastructure/ansible-dask-heterogeneous.md - - 🗄️ Database & Data Engineering: - - Overview: tutorials/database-data-engineering/index.md - - PostGIS Geometry Indexing: tutorials/database-data-engineering/postgis-geometry-indexing.md - - PostGIS Raster Indexing: tutorials/database-data-engineering/postgis-raster-indexing.md - - Raster–Vector Workflows: tutorials/database-data-engineering/postgis-raster-vector-workflows.md - - Alembic Migrations: tutorials/database-data-engineering/alembic-migrations.md - - PostgreSQL Pooling (PgBouncer + FastAPI): tutorials/database-data-engineering/postgres-pooling.md - - Solr + Postgres JSONB Search: tutorials/database-data-engineering/solr-postgres-jsonb-search.md - - Auditing PostgreSQL with PgAudit and PgCron: tutorials/database-data-engineering/postgres-pgaudit-pgcron-auditing.md - - parquet_s3_fdw with Local, MinIO, Vast, and AWS: tutorials/database-data-engineering/parquet-s3-fdw.md - - Building a Postgres Lakehouse Image with pg_lake and parquet_s3_fdw: tutorials/database-data-engineering/postgres-lakehouse-pglake-parquet-fdw.md - - GeoParquet with Polars: tutorials/database-data-engineering/geoparquet-with-polars.md - - Generating Dark OpenMapTiles for the Entire US at Zoom Level 12: tutorials/database-data-engineering/openmaptiles-us-dark-z12.md - - Real-Time Data Processing: tutorials/database-data-engineering/real-time-data-processing.md - - Kafka + TimescaleDB IoT Streaming: tutorials/database-data-engineering/kafka-timescaledb-iot.md - - Go-Glue OSM → PostGIS → Tiles Pipeline: tutorials/database-data-engineering/go-osm-tiling-pipeline.md - - Apache Spark Mastery: tutorials/database-data-engineering/apache-spark-mastery.md - - Apache Iceberg Mastery: tutorials/database-data-engineering/apache-iceberg-mastery.md - - Pulsar → Flink → Pinot (Realtime OLAP) + Superset: tutorials/database-data-engineering/pulsar-flink-pinot-superset.md - - H3 + Tile38 + NATS + DuckDB: tutorials/database-data-engineering/h3-tile38-nats-duckdb.md - - H3 Raster to Hex: tutorials/database-data-engineering/h3-raster-to-hex.md - - IPFS + SurrealDB + Meilisearch + NATS + Deno + Svelte: tutorials/database-data-engineering/ipfs-surreal-meili-nats-deno-svelte.md - - Graph vs Vector Databases: tutorials/database-data-engineering/graph-vs-vector-databases.md - - Geospatial Knowledge Graph: tutorials/database-data-engineering/geospatial-knowledge-graph.md - - DuckDB Parquet Data Quality: tutorials/database-data-engineering/duckdb-parquet-data-quality.md - - 🤖 Machine Learning & AI: - - Overview: tutorials/ml-ai/index.md - - MLflow API Experiments: tutorials/ml-ai/mlflow-api-experiments.md - - Local LLM Deployments (Ollama, llama.cpp, vLLM, TGI): tutorials/ml-ai/local-llm-deployments.md - - ONNX Browser Inference: tutorials/ml-ai/onnx-browser-inference.md - - RAG with Ollama + Database: tutorials/ml-ai/rag-ollama-db.md - - MCP ↔ MLflow Toolchain: tutorials/ml-ai/mcp-mlflow-toolchain.md - - Semantic ML Training: tutorials/ml-ai/semantic-ml-training.md - - 🔧 System Administration: - - Overview: tutorials/system-administration/index.md - - iPXE Multi-System Booting: tutorials/system-administration/ipxe-multi-boot.md - - Remote Dev with tmux & screen: tutorials/system-administration/remote-dev-tmux-screen.md - - FIFO Prefect Flow with Redis: tutorials/system-administration/prefect-fifo-redis.md - - AWK Unix Text Processing: tutorials/system-administration/awk-unix-text-processing.md - - 🖼️ Diagrams: - - "Layered Systems Diagrams: Mermaid → SVG": tutorials/diagrams/layered-systems-diagrams-mermaid-to-svg.md - - Mermaid → SVG Workflow Pipeline: tutorials/diagrams/mermaid-to-svg-workflow-pipeline.md - - 📊 Data Science & Visualization: - - Overview: tutorials/data-science-visualization/index.md - - Jupyter Notebook Best Practices (Geospatial Edition): tutorials/data-science-visualization/jupyter-notebook-best-practices-geo.md - - Mermaid Diagrams in MkDocs: tutorials/data-science-visualization/mermaid-diagrams.md - - TikZ Diagrams in LaTeX: tutorials/data-science-visualization/latex-tikz-diagrams.md - - Generative Art in R: tutorials/data-science-visualization/r-generative-art.md - - 🛠️ Development Tools: - - Overview: tutorials/development-tools/index.md - - jq JSON Parsing Mastery: tutorials/development-tools/jq-json-parsing-mastery.md - - find_files for parquet_s3_fdw: tutorials/development-tools/find-files-parquet-fdw.md - - Mosquitto + Python (MQTT Best Practices): tutorials/development-tools/mosquitto-mqtt-python.md - - Python UDP Messaging: tutorials/development-tools/python-udp.md - - Python Modbus Device Communication: tutorials/development-tools/python-modbus-devices.md - - Mixing Tech with Go Glue: tutorials/development-tools/go-tech-mixer.md - - Tauri + rqlite + Syncthing: tutorials/development-tools/tauri-rqlite-syncthing.md - - 🎨 Just for Fun: - - Overview: tutorials/just-for-fun/index.md - - Terminal to GIF: tutorials/just-for-fun/terminal-to-gif.md - - Redis Streams + Web MIDI: tutorials/just-for-fun/redis-midi-music.md - - PostGIS Rasters + WebGL Art: tutorials/just-for-fun/postgis-webgl-art.md - - IoT + IPFS + GraphQL + Blender: tutorials/just-for-fun/iot-ipfs-graphql-blender.md - - MQTT + TimescaleDB + WebSockets + Three.js: tutorials/just-for-fun/mqtt-timescaledb-websockets-threejs.md - - Fastify + Kafka + ClickHouse + WASM + WebGPU: tutorials/just-for-fun/fastify-kafka-clickhouse-wasm-webgpu.md - - Gonzo Prometheus Exporter: tutorials/just-for-fun/gonzo-prometheus-exporter.md - - Git Commit Weather Station: tutorials/just-for-fun/git-weather-node-redis-ipfs-webrtc.md - - Selenium Grid (Docker) + Python: tutorials/just-for-fun/selenium-grid-docker-python.md - - Martin + PostGIS Tiling: tutorials/just-for-fun/martin-postgis-tiling.md - - Managing People in Software Development: tutorials/just-for-fun/managing-people-software-dev.md - - OSC + MQTT + Prometheus + SuperCollider: tutorials/just-for-fun/osc-mqtt-prometheus-supercollider.md - - Fractal Art Explorer (JavaScript): tutorials/just-for-fun/fractal-art-explorer-js.md - - Pi-Based Sample Library Server: tutorials/just-for-fun/pi-sample-server.md - - Go Auth Backend (scratch + Compose): tutorials/just-for-fun/go-auth-scratch-compose.md - - Recursive Cathedral Generator (Kotlin): tutorials/just-for-fun/kotlin-recursive-cathedral.md - - MIDI-Driven Particle Nebula (Kotlin): tutorials/just-for-fun/kotlin-midi-particle-nebula.md - - Cellular Automata Garden (Kotlin): tutorials/just-for-fun/kotlin-cellular-automata-garden.md - - Pi Infinite Art Frame (Kotlin): tutorials/just-for-fun/pi-infinite-art-frame-kotlin.md - - 🔌 Embedded Systems: - - Overview: tutorials/embedded/index.md - - ESP32 E-Ink Environmental Monitor: tutorials/embedded/esp32-eink-sensor-monitor.md - - ESP32 RF Room Light Controller: tutorials/embedded/esp32-rf-room-light-controller.md - - ESP32 + MQTT + Home Assistant: tutorials/embedded/esp32-mqtt-home-assistant-integration.md - - Contact & Collaboration: getting-started.md + - Projects: + - Portfolio: projects/index.md + - Documentation sites: projects/documentation-sites.md + - Writing: + - Technical overview: documentation.md + - What's New: whats-new.md + - Tags: tags.md + - Doctrine: + - Start Here — Architectural Compass: start-here-architectural-paths.md + - Reading Tracks: reading-tracks.md + - Decision Frameworks: decision-frameworks.md + - Anti-Patterns: anti-patterns.md + - Philosophy: philosophy.md + - Systems Glossary: systems-glossary.md + - Diagrams: + - Diagram Style Guide: diagrams/style-guide.md + - Architecture Decisions: + - Overview: adr/index.md + - "ADR-0001: Site Architecture": adr/0001-site-architecture.md + - "ADR-0002: MkDocs + Material": adr/0002-why-mkdocs-material.md + - "ADR-0003: Best Practices vs Tutorials": adr/0003-why-best-practices-vs-tutorials.md + - "ADR-0004: Why Just for Fun Exists": adr/0004-why-just-for-fun-exists.md + - "ADR-0005: ESP32 Section Architecture": adr/0005-esp32-section-architecture.md + - "ADR-0006: Embedded Expansion (LoRa + Power)": adr/0006-embedded-expansion-lora-power.md + - "ADR-0007: Deep Dives Section": adr/0007-deep-dives-section.md + - "ADR-0008: Expand Deep Dives Section": adr/0008-expand-deep-dives-section.md + - "ADR-0009: Deep Dives Expansion": adr/0009-deep-dives-expansion.md + - "ADR-0010: Deep Dives Governance": adr/0010-deep-dives-governance.md + - "ADR-0011: Deep Dives Expansion Phase II": adr/0011-deep-dives-expansion-phase-2.md + - "ADR-0012: Deep Dives Scale Governance": adr/0012-deep-dives-scale-governance.md + - "ADR-0013: Deep Dives Curation Model": adr/0013-deep-dives-curation-model.md + - "ADR-0014: Elevate Site to Systems Doctrine": adr/0014-elevate-site-to-systems-doctrine.md + - "ADR-0015: Standardize Diagrams on Mermaid": adr/0015-diagrams-mermaid.md + - Best Practices: + - Overview: best-practices/index.md + - 📌 Start Here: + - ADR and Technical Decision Governance: best-practices/architecture-design/adr-decision-governance.md + - Configuration Management: best-practices/operations-monitoring/configuration-management.md + - System Resilience & Concurrency: best-practices/operations-monitoring/system-resilience-and-concurrency.md + - IAM & RBAC Governance: best-practices/security/iam-rbac-abac-governance.md + - Data: + - Reproducible Data Pipelines: best-practices/data/reproducible-data-pipelines.md + - Metadata as Control Plane: best-practices/data/metadata-control-plane.md + - Architecture: + - Cost-Aware System Architecture: best-practices/architecture/cost-aware-systems.md + - Geospatial: + - Geospatial System Architecture: best-practices/geospatial/geospatial-system-design.md + - Operations: + - Failure-Oriented System Design: best-practices/operations/failure-oriented-design.md + - Data Processing: + - Spark: + - When to Use Spark: best-practices/data-processing/spark/when-to-use-spark.md + - Scaling Spark Clusters: best-practices/data-processing/spark/scaling-spark.md + - Spark on Kubernetes: best-practices/data-processing/spark/spark-on-kubernetes.md + - Spark Performance Tuning: best-practices/data-processing/spark/spark-performance-tuning.md + - Spark in Modern Architectures: best-practices/data-processing/spark/spark-modern-architecture.md + - 🐍 Python Development: + - Overview: best-practices/python/index.md + - "Developer Experience (DX) as Infrastructure: Golden Paths, Tooling Ecosystems & Workflow Automation": best-practices/python/dx-architecture-and-golden-paths.md + - Python Package: best-practices/python/python-package.md + - Typing in Python: best-practices/python/typing-in-python.md + - Python Concurrency: best-practices/python/python-threading-and-multiprocessing.md + - Python Async Best Practices: best-practices/python/python-async-best-practices.md + - Pytest Best Practices: best-practices/python/pytest-best-practices.md + - API Development: best-practices/python/api-development.md + - FastAPI Geospatial: best-practices/python/fastapi-geospatial.md + - Web Performance Optimization: best-practices/python/web-performance-optimization.md + - TUI Applications: best-practices/python/tui-applications.md + - 🦀 Rust Development: + - Overview: best-practices/rust/index.md + - Rust Development Environment: best-practices/rust/rust-dev-environment.md + - TUI Applications: best-practices/rust/tui-applications.md + - 🐹 Go Development: + - Overview: best-practices/go/index.md + - Go Development Environment: best-practices/go/go-dev-environment.md + - TUI Applications: best-practices/go/tui-applications.md + - 📊 R Development: + - Overview: best-practices/r/index.md + - R Development Environment: best-practices/r/r-dev-environment.md + - 🐳 Docker & Infrastructure: + - Overview: best-practices/docker-infrastructure/index.md + - Docker & Compose: best-practices/docker-infrastructure/docker-and-compose.md + - Conda to Docker Migration: best-practices/docker-infrastructure/conda-to-docker-migration.md + - SBOMs, Trivy Scans, and Automated CVE Mitigation: best-practices/docker-infrastructure/docker-sbom-trivy-cve-mitigation.md + - Nginx Production: best-practices/docker-infrastructure/nginx-production.md + - NGINX Best Practices: best-practices/docker-infrastructure/nginx-best-practices.md + - Ansible Inventory Management: best-practices/docker-infrastructure/ansible-inventory-management.md + - Ansible Playbook Design: best-practices/docker-infrastructure/ansible-playbook-design.md + - Ansible Security Hardening: best-practices/docker-infrastructure/ansible-security-hardening.md + - Ansible Performance Optimization: best-practices/docker-infrastructure/ansible-performance-optimization.md + - Jinja Best Practices: best-practices/docker-infrastructure/jinja-best-practices.md + - Advanced Tmux Workflows: best-practices/docker-infrastructure/tmux-advanced.md + - 📝 Git & Version Control: + - Overview: best-practices/git/index.md + - Git Production: best-practices/git/git-production.md + - Git Workflows & Collaboration: best-practices/git/git-workflows-collaboration.md + - Modern GitFlow Best Practices: best-practices/git/gitflow-best-practices.md + - 🗄️ Database & Data Management: + - Overview: best-practices/database-data/index.md + - "Cross-System Data Lineage, Inter-Service Metadata Contracts & Provenance Enforcement": best-practices/database-data/data-lineage-contracts.md + - Semantic Layer Engineering, Domain Models, and Knowledge Graph Alignment: best-practices/database-data/semantic-layer-engineering.md + - "AI-Ready, ML-Enabled Geospatial Knowledge Graph": best-practices/database-data/ai-ml-geospatial-knowledge-graph.md + - Parquet: best-practices/database-data/parquet.md + - GeoParquet: best-practices/database-data/geoparquet.md + - Patroni PostgreSQL HA: best-practices/database-data/patroni-postgres-ha.md + - Database Optimization: best-practices/database-data/database-optimization.md + - Database Migrations & Schema Evolution: best-practices/database-data/database-migrations.md + - GeoParquet Data Warehouses: best-practices/database-data/geoparquet-data-warehouses.md + - AWS Serverless Geospatial: best-practices/database-data/aws-serverless-geospatial.md + - Lakes vs Lakehouses vs Warehouses: best-practices/database-data/lake-vs-lakehouse-vs-warehouse.md + - Data Lake Governance: best-practices/database-data/data-lake-governance.md + - Geospatial Data Engineering: best-practices/database-data/geospatial-data-engineering.md + - Geospatial Benchmarking: best-practices/database-data/geospatial-benchmarking.md + - 🐘 PostgreSQL Development: + - Overview: best-practices/postgres/index.md + - Core & Design: + - Development Environment: best-practices/postgres/postgres-dev-environment.md + - Database Design: best-practices/postgres/postgres-database-design.md + - Data Types: best-practices/postgres/postgres-data-types.md + - Constraints & Validation: best-practices/postgres/postgres-constraints-validation.md + - Transactions & Concurrency: best-practices/postgres/postgres-transactions-concurrency.md + - Indexing Strategies: best-practices/postgres/postgres-indexing-strategies.md + - Security Best Practices: best-practices/postgres/postgres-security-best-practices.md + - Performance & Operations: + - Performance Tuning: best-practices/postgres/postgres-performance-tuning.md + - Monitoring & Observability: best-practices/postgres/postgres-monitoring-observability.md + - Backup & Recovery: best-practices/postgres/postgres-backup-recovery.md + - Replication & High Availability: best-practices/postgres/postgres-replication-ha.md + - Maintenance & Vacuum: best-practices/postgres/postgres-maintenance-vacuum.md + - Scaling Strategies: best-practices/postgres/postgres-scaling-strategies.md + - Connection Pooling: best-practices/postgres/postgres-pooling.md + - Troubleshooting: best-practices/postgres/postgres-troubleshooting.md + - Configuration Management: best-practices/postgres/postgres-configuration-management.md + - Advanced Features: + - Extensions: best-practices/postgres/postgres-extensions.md + - JSON & JSONB: best-practices/postgres/postgres-json-jsonb.md + - Full-Text Search: best-practices/postgres/postgres-fulltext-search.md + - Partitioning: best-practices/postgres/postgres-partitioning.md + - Time Series Data: best-practices/postgres/postgres-timeseries.md + - Large Object Storage: best-practices/postgres/postgres-large-objects.md + - Foreign Data Wrappers: best-practices/postgres/fdw-postgres.md + - PostGIS Best Practices: best-practices/postgres/postgis-best-practices.md + - Integration & Deployment: + - API Development: best-practices/postgres/postgres-api-development.md + - Event-Driven Architecture: best-practices/postgres/postgres-event-driven.md + - Data Pipeline Integration: best-practices/postgres/postgres-data-pipeline-integration.md + - Cloud Integration: best-practices/postgres/postgres-cloud-integration.md + - Containerization: best-practices/postgres/postgres-containerization.md + - Deployment Strategies: best-practices/postgres/postgres-deployment-strategies.md + - Data Engineering: best-practices/database-data/data-engineering.md + - ETL Pipeline Design: best-practices/database-data/etl-pipeline-design.md + - 📊 Data Governance: + - Overview: best-practices/data-governance/index.md + - Metadata Standards, Schema Governance & Data Provenance: best-practices/data-governance/metadata-provenance-contracts.md + - Data Validation and Contract Governance: best-practices/data-governance/data-validation-and-contract-governance.md + - Data Freshness, SLA/SLO Governance, and Pipeline Reliability Contracts: best-practices/data-governance/data-freshness-sla-governance.md + - Data Retention, Archival Strategy, Lifecycle Governance & Cold Storage Patterns: best-practices/data-governance/data-retention-archival-lifecycle-governance.md + - Data Quality SLAs, Validation Layers, and Observability for Tabular, Geospatial, and ML Data: best-practices/data-governance/data-quality-sla-validation-observability.md + - 🤖 Machine Learning & AI: + - Overview: best-practices/ml-ai/index.md + - "ML Systems Architecture: Feature Stores, Model Serving, Experiment Governance, and Cross-System Reproducibility": best-practices/ml-ai/ml-systems-architecture-governance.md + - Prompting LLMs: best-practices/ml-ai/prompting-llms.md + - ONNX Model Optimization: best-practices/ml-ai/onnx-model-optimization.md + - MCP + FastAPI Full Stack: best-practices/ml-ai/mcp-fastapi-stack.md + - Embeddings & Vector Databases: best-practices/ml-ai/embeddings-and-vector-databases.md + - Vibe → Agentic LLMs: best-practices/ml-ai/vibe-to-agentic.md + - R Data Exploration: best-practices/ml-ai/r-data-exploration.md + - 🏗️ Architecture & Design: + - Overview: best-practices/architecture-design/index.md + - System-Wide Naming, Taxonomy, and Structural Vocabulary Governance: best-practices/architecture-design/system-taxonomy-governance.md + - Cost-Aware Architecture & Resource-Efficiency Governance: best-practices/architecture-design/cost-aware-architecture-and-efficiency-governance.md + - Holistic Capacity Planning, Scaling Economics, and Workload Modeling: best-practices/architecture-design/capacity-planning-and-workload-modeling.md + - Multi-Region, Multi-Cluster Disaster Recovery, Failover Topologies, and Data Sovereignty: best-practices/architecture-design/multi-region-dr-strategy.md + - Cross-Environment Configuration Strategy and Multi-Cluster State Management: best-practices/architecture-design/environment-config-governance.md + - Cognitive Load Management and Developer Experience: best-practices/architecture-design/cognitive-load-developer-experience.md + - Architectural Fitness Functions and Governance: best-practices/architecture-design/architecture-fitness-functions-governance.md + - Temporal Governance and Time Synchronization: best-practices/architecture-design/temporal-governance-and-time-synchronization.md + - Repository Standardization and Governance: best-practices/architecture-design/repository-standardization-and-governance.md + - Event-Driven Architecture: best-practices/architecture-design/event-driven-architecture.md + - "Streaming Architecture Patterns: SAGA, CQRS, and Outbox": best-practices/architecture-design/streaming-architecture-patterns.md + - API Governance, Backward Compatibility Rules, and Cross-Language Interface Stability: best-practices/architecture-design/api-governance-interface-stability.md + - API Gateway Architecture: best-practices/architecture-design/api-gateway-architecture.md + - Multi-Cloud Federation & Portability Architecture: best-practices/architecture-design/multi-cloud-federation-portability.md + - Cloud Architecture: best-practices/architecture-design/cloud-architecture.md + - Secrets Management: best-practices/architecture-design/secrets-management.md + - Testing & CI/CD Pipelines: best-practices/architecture-design/ci-cd-pipelines.md + - Documentation: best-practices/architecture-design/documentation.md + - ADR and Technical Decision Governance: best-practices/architecture-design/adr-decision-governance.md + - Caching & Performance Layers: best-practices/architecture-design/caching-performance.md + - Cache-Topology Architecture: best-practices/architecture-design/cache-topology-architecture.md + - Data Mesh Architecture: best-practices/architecture-design/data-mesh-architecture.md + - Service Decomposition Strategy: best-practices/architecture-design/service-decomposition-strategy.md + - Polyglot Interoperability Design: best-practices/architecture-design/polyglot-interoperability-design.md + - Reference Architecture Diagrams: best-practices/architecture-design/reference-architecture-diagrams.md + - RDF/OWL Metadata Automation: best-practices/architecture-design/rdf-owl-metadata-automation.md + - Protocol Buffers with Python: best-practices/architecture-design/protobuf-python.md + - 🔧 Operations & Monitoring: + - Overview: best-practices/operations-monitoring/index.md + - "Observability as Architecture: Unified Telemetry Models Across Clusters, Services, and Languages": best-practices/operations-monitoring/unified-observability-architecture.md + - Chaos Engineering, Fault Injection, and Reliability Validation: best-practices/operations-monitoring/chaos-engineering-governance.md + - Operational Risk Modeling, Blast Radius Reduction & Failure Domain Architecture: best-practices/operations-monitoring/blast-radius-risk-modeling.md + - Cross-Environment Configuration Drift Prevention, Promotion Workflows & Release Channels: best-practices/operations-monitoring/environment-promotion-drift-governance.md + - Observability-Driven Development (ODD), Telemetry-First Coding Practices, and Preemptive Debugging Architecture: best-practices/operations-monitoring/observability-driven-development.md + - Operational Resilience and Incident Response: best-practices/operations-monitoring/operational-resilience-and-incident-response.md + - Performance Monitoring: best-practices/operations-monitoring/performance-monitoring.md + - System Resilience, Rate Limiting, Concurrency Control & Backpressure: best-practices/operations-monitoring/system-resilience-and-concurrency.md + - Configuration Management, Secrets Lifecycle, and Multi-Environment Drift Control: best-practices/operations-monitoring/configuration-management.md + - Cross-Environment Configuration Drift Detection & Prevention: best-practices/operations-monitoring/configuration-drift-detection-prevention.md + - Release Management, Change Governance, and Progressive Delivery: best-practices/operations-monitoring/release-management-and-progressive-delivery.md + - Structured Logging & Observability: best-practices/operations-monitoring/logging-observability.md + - Grafana, Prometheus, Loki, and Observability: best-practices/operations-monitoring/grafana-prometheus-loki-observability.md + - Testing Best Practices: best-practices/operations-monitoring/testing-best-practices.md + - Secrets & Configuration Management: best-practices/operations-monitoring/secrets-config.md + - Grafana: best-practices/operations-monitoring/grafana.md + - 🔒 Security: + - Overview: best-practices/security/index.md + - End-to-End Secrets Management & Key Rotation: best-practices/security/secrets-governance.md + - Identity & Access Management, RBAC/ABAC, and Least-Privilege Governance: best-practices/security/iam-rbac-abac-governance.md + - "Secure-by-Design Lifecycle Architecture Across Polyglot Systems": best-practices/security/secure-by-design-polyglot.md + - Secure Computes, Sandboxing, and Multi-Tenant Isolation for Polyglot Systems: best-practices/security/secure-sandboxing-and-multi-tenant-isolation.md + - Cross-Domain Identity Federation, AuthZ/AuthN Architecture & Identity Propagation Models: best-practices/security/identity-federation-authz-authn-architecture.md + - Secret Supply Chains, Encryption Lifecycle Management & Cryptographic Rotation Strategy: best-practices/security/encryption-lifecycle-and-crypto-rotation.md + - ⚡ Performance: + - Overview: best-practices/performance/index.md + - End-to-End Caching Strategy: best-practices/performance/end-to-end-caching-strategy.md + - 🧪 Testing: + - Overview: best-practices/testing/index.md + - End-to-End Testing Strategy: best-practices/testing/end-to-end-testing-strategy.md + - 🖼️ Diagrams: + - Systems Diagramming Best Practices: best-practices/diagrams/systems-diagramming-best-practices.md + - SVG Workflow Generation: best-practices/diagrams/svg-workflow-generation.md + - 🎨 Creative & Fun: + - Overview: best-practices/creative-fun/index.md + - Time Hygiene (UTC, TZ, Clocks): best-practices/creative-fun/time-hygiene.md + - Idempotency & De-dup: best-practices/creative-fun/idempotency-and-dedup.md + - Celery Tasks: best-practices/creative-fun/celery-best-practices.md + - LaTeX Workflows: best-practices/creative-fun/latex.md + - YAML Recipe Format: best-practices/creative-fun/yaml-recipe-format.md + - 🔌 Embedded Systems & ESP32: + - Overview: best-practices/esp32/index.md + - Programming Architecture: best-practices/esp32/esp32-programming-architecture.md + - Power Management & Deep Sleep: best-practices/esp32/power-management-and-deep-sleep.md + - Hardware & Electrical Safety: best-practices/esp32/esp32-hardware-and-electrical-safety.md + - Embedded Security & OTA: best-practices/esp32/embedded-security-and-ota.md + - Sensor Integration: best-practices/esp32/sensor-integration-best-practices.md + - E-Ink Display Integration: best-practices/esp32/e-ink-display-best-practices.md + - MQTT Security: best-practices/esp32/mqtt-security-best-practices.md + - LoRa Best Practices (SX127x): best-practices/esp32/lora-best-practices-sx127x.md + - Safety Checklist (Printable): best-practices/esp32/esp32-safety-checklist-printable.md + - ESP32-S3 and C3 Notes: best-practices/esp32/esp32-s3-and-c3-architecture-notes.md + - 🔋 Power Electronics & Embedded Hardware: + - Overview: best-practices/embedded/index.md + - Power Electronics for ESP32: best-practices/embedded/power-electronics-for-esp32.md + - 🏠 Home Automation & MQTT: + - Overview: best-practices/home-automation/index.md + - Home Assistant Security: best-practices/home-automation/home-assistant-security-best-practices.md + - Deep Dives: + - Overview: deep-dives/index.md + - 📦 Data Formats & Storage: + - Parquet vs CSV vs ORC vs Avro: deep-dives/parquet-vs-csv-orc-avro.md + - Geospatial File Format Choices: deep-dives/geospatial-file-format-choices.md + - Polars vs Pandas for Geospatial Data: deep-dives/polars-vs-pandas-geospatial.md + - The Physics of Storage Systems: deep-dives/the-physics-of-storage-systems.md + - The Operational Geometry of Spatial Systems: deep-dives/the-operational-geometry-of-spatial-systems.md + - 🧩 Systems Design & Architecture: + - Why Most Microservices Should Be Monoliths: deep-dives/why-most-microservices-should-be-monoliths.md + - Appropriate Use of Microservices: deep-dives/appropriate-use-of-microservices.md + - Distributed Systems and the Myth of Infinite Scale: deep-dives/distributed-systems-myth-of-infinite-scale.md + - Distributed Systems Architecture: deep-dives/distributed-systems-architecture.md + - Event-Driven Architecture: deep-dives/event-driven-architecture.md + - Raft Consensus Explained: deep-dives/raft-consensus-explained.md + - Why Most Kubernetes Clusters Shouldn't Exist: deep-dives/why-most-kubernetes-clusters-shouldnt-exist.md + - When to Use a TUI, CLI, or WebApp: deep-dives/when-to-use-tui-cli-or-webapp.md + - Blockchain vs Hashchain: deep-dives/blockchain-vs-hashchain.md + - Merkle Trees Explained: deep-dives/merkle-trees-explained.md + - Proof of Work Explained: deep-dives/proof-of-work-explained.md + - 🔭 Operations & Reliability: + - Observability vs Monitoring: deep-dives/observability-vs-monitoring.md + - The Economics of Observability: deep-dives/the-economics-of-observability.md + - Designing Resilient Distributed Systems: deep-dives/resilient-distributed-systems.md + - 🐳 Infrastructure & Automation: + - Container Base Image Philosophy: deep-dives/container-base-image-philosophy.md + - IaC vs GitOps: deep-dives/iac-vs-gitops.md + - The Human Cost of Automation: deep-dives/the-human-cost-of-automation.md + - The Economics of GPU Infrastructure: deep-dives/the-economics-of-gpu-infrastructure.md + - Building a Bitcoin Mining Rig: deep-dives/building-a-bitcoin-mining-rig.md + - Converting Bitcoin Mining to LLM Clusters: deep-dives/converting-bitcoin-mining-to-llm.md + - The Myth of Serverless Simplicity: deep-dives/the-myth-of-serverless-simplicity.md + - 🏗️ Data Systems & Architecture: + - Lakehouse vs Warehouse vs Database: deep-dives/lakehouse-vs-warehouse-vs-database.md + - DuckDB vs PostgreSQL vs Spark: deep-dives/duckdb-vs-postgres-vs-spark.md + - Why Most Data Pipelines Fail: deep-dives/why-most-data-pipelines-fail.md + - Why Most Data Lakes Become Data Swamps: deep-dives/why-data-lakes-become-swamps.md + - Prefect vs Airflow: deep-dives/prefect-vs-airflow.md + - Metadata as Infrastructure: deep-dives/metadata-as-infrastructure.md + - The Hidden Cost of Real-Time Systems: deep-dives/the-hidden-cost-of-real-time-systems.md + - The End of the Data Warehouse?: deep-dives/the-end-of-the-data-warehouse.md + - The Hidden Cost of Metadata Debt: deep-dives/the-hidden-cost-of-metadata-debt.md + - Why Most ML Systems Fail in Production: deep-dives/why-ml-systems-fail-in-production.md + - Why Spark Clusters Fail in Production: deep-dives/why-spark-clusters-fail.md + - The Economics of Distributed Data Processing: deep-dives/economics-of-distributed-data-processing.md + - Spark vs DuckDB vs Polars at Scale: deep-dives/spark-vs-duckdb-vs-polars.md + - The Myth of Infinite Data Scale: deep-dives/myth-of-infinite-data-scale.md + - Why Most Data Lakes Become Data Swamps: deep-dives/data-lakes-become-data-swamps.md + - The Metadata Crisis in Modern Data Platforms: deep-dives/metadata-crisis-in-modern-data-platforms.md + - Why Most Data Pipelines Are Operationally Fragile: deep-dives/why-data-pipelines-are-operationally-fragile.md + - 📡 Embedded & Radio: + - LoRaWAN vs Raw LoRa: deep-dives/lorawan-vs-raw-lora.md + - MQTT vs HTTP in IoT Systems: deep-dives/mqtt-vs-http-iot.md + - ESP32 vs Raspberry Pi: deep-dives/esp32-vs-raspberry-pi.md + - Tutorials: + - Overview: tutorials/index.md + - 🚀 Quick Start: + - Overview: tutorials/quick-start/index.md + - Creating MkDocs GitHub Site: tutorials/quick-start/creating-mkdocs-github-site.md + - Monitoring with Grafana & Prometheus: tutorials/quick-start/monitoring-with-grafana-prometheus.md + - 🐍 Python Development: + - Overview: tutorials/python-development/index.md + - psycopg2 to psycopg 3 Migration: tutorials/python-development/psycopg2-to-psycopg3-migration.md + - Ruff Check Ignore in pyproject.toml: tutorials/python-development/ruff-check-ignore-pyproject.md + - Class-Based NiceGUI Pages and Integrations: tutorials/python-development/nicegui-class-based-pages.md + - Advanced NiceGUI Architecture: tutorials/python-development/advanced-nicegui-architecture.md + - Distributed NiceGUI Architecture with Redis: tutorials/python-development/distributed-nicegui-redis.md + - R Shiny Geospatial App: tutorials/python-development/r-shiny-geoapp.md + - Click CLI to FastAPI Conversion: tutorials/python-development/click-to-fastapi-conversion.md + - WebSocket Chat with FastAPI: tutorials/python-development/websocket-chat-fastapi.md + - Glitch Observatory (JS): tutorials/python-development/js-glitch-observatory.md + - Chaos Engineering with Kubernetes and Python: tutorials/python-development/chaos-engineering-k8s-python.md + - Building a Python TUI: tutorials/python-development/building-a-python-tui.md + - 🦀 Rust Development: + - Overview: tutorials/rust-development/index.md + - Rust + CSR (Parse & Build from Parquet/DB): tutorials/rust-development/rust-csr-parquet-db.md + - Event-Sourcing in Rust: tutorials/rust-development/rust-event-sourcing.md + - Building a Rust TUI: tutorials/rust-development/building-a-rust-tui.md + - 🐹 Go Development: + - Overview: tutorials/go-development/index.md + - Building a Go TUI: tutorials/go-development/building-a-go-tui.md + - 🐳 Docker & Infrastructure: + - Overview: tutorials/docker-infrastructure/index.md + - Multi-Stage Docker (Conda → scratch): tutorials/docker-infrastructure/multistage-conda-to-scratch.md + - Slim Geospatial + GPU Containers (GDAL): tutorials/docker-infrastructure/slim-geospatial-gpu-conda.md + - Slimming GPU Docker Images: tutorials/docker-infrastructure/slim-gpu-docker-images.md + - Slimming TensorFlow GPU Images: tutorials/docker-infrastructure/slim-tf-gpu-images.md + - TensorFlow GPU Slim Images - Repository Skeleton: tutorials/docker-infrastructure/slim-tf-gpu-images-skeleton.md + - Compose Profiles Polyglot Stack: tutorials/docker-infrastructure/compose-profiles-polyglot-stack.md + - RKE2 on Raspberry Pi Farm: tutorials/docker-infrastructure/rke2-raspberry-pi.md + - Building a 2×8 RKE2 Cluster with Rancher, PGO, and Prefect: tutorials/docker-infrastructure/ansible-rke2-rancher-pgo-prefect.md + - ZFS Tank with OS on NVMe: tutorials/docker-infrastructure/zfs-tank-nvme.md + - Deploy SLURM with Ansible on Raspberry Pi Cluster: tutorials/docker-infrastructure/ansible-slurm-raspberrypi.md + - Harbor Container Registry Setup: tutorials/docker-infrastructure/harbor-registry-setup.md + - Dask with Ansible (CPU + GPU): tutorials/docker-infrastructure/ansible-dask-heterogeneous.md + - 🗄️ Database & Data Engineering: + - Overview: tutorials/database-data-engineering/index.md + - PostGIS Geometry Indexing: tutorials/database-data-engineering/postgis-geometry-indexing.md + - PostGIS Raster Indexing: tutorials/database-data-engineering/postgis-raster-indexing.md + - Raster–Vector Workflows: tutorials/database-data-engineering/postgis-raster-vector-workflows.md + - Alembic Migrations: tutorials/database-data-engineering/alembic-migrations.md + - PostgreSQL Pooling (PgBouncer + FastAPI): tutorials/database-data-engineering/postgres-pooling.md + - Solr + Postgres JSONB Search: tutorials/database-data-engineering/solr-postgres-jsonb-search.md + - Auditing PostgreSQL with PgAudit and PgCron: tutorials/database-data-engineering/postgres-pgaudit-pgcron-auditing.md + - parquet_s3_fdw with Local, MinIO, Vast, and AWS: tutorials/database-data-engineering/parquet-s3-fdw.md + - Building a Postgres Lakehouse Image with pg_lake and parquet_s3_fdw: tutorials/database-data-engineering/postgres-lakehouse-pglake-parquet-fdw.md + - GeoParquet with Polars: tutorials/database-data-engineering/geoparquet-with-polars.md + - Generating Dark OpenMapTiles for the Entire US at Zoom Level 12: tutorials/database-data-engineering/openmaptiles-us-dark-z12.md + - Real-Time Data Processing: tutorials/database-data-engineering/real-time-data-processing.md + - Kafka + TimescaleDB IoT Streaming: tutorials/database-data-engineering/kafka-timescaledb-iot.md + - Go-Glue OSM → PostGIS → Tiles Pipeline: tutorials/database-data-engineering/go-osm-tiling-pipeline.md + - Apache Spark Mastery: tutorials/database-data-engineering/apache-spark-mastery.md + - Apache Iceberg Mastery: tutorials/database-data-engineering/apache-iceberg-mastery.md + - Pulsar → Flink → Pinot (Realtime OLAP) + Superset: tutorials/database-data-engineering/pulsar-flink-pinot-superset.md + - H3 + Tile38 + NATS + DuckDB: tutorials/database-data-engineering/h3-tile38-nats-duckdb.md + - H3 Raster to Hex: tutorials/database-data-engineering/h3-raster-to-hex.md + - IPFS + SurrealDB + Meilisearch + NATS + Deno + Svelte: tutorials/database-data-engineering/ipfs-surreal-meili-nats-deno-svelte.md + - Graph vs Vector Databases: tutorials/database-data-engineering/graph-vs-vector-databases.md + - Geospatial Knowledge Graph: tutorials/database-data-engineering/geospatial-knowledge-graph.md + - DuckDB Parquet Data Quality: tutorials/database-data-engineering/duckdb-parquet-data-quality.md + - 🤖 Machine Learning & AI: + - Overview: tutorials/ml-ai/index.md + - MLflow API Experiments: tutorials/ml-ai/mlflow-api-experiments.md + - Local LLM Deployments (Ollama, llama.cpp, vLLM, TGI): tutorials/ml-ai/local-llm-deployments.md + - ONNX Browser Inference: tutorials/ml-ai/onnx-browser-inference.md + - RAG with Ollama + Database: tutorials/ml-ai/rag-ollama-db.md + - MCP ↔ MLflow Toolchain: tutorials/ml-ai/mcp-mlflow-toolchain.md + - Semantic ML Training: tutorials/ml-ai/semantic-ml-training.md + - 🔧 System Administration: + - Overview: tutorials/system-administration/index.md + - iPXE Multi-System Booting: tutorials/system-administration/ipxe-multi-boot.md + - Remote Dev with tmux & screen: tutorials/system-administration/remote-dev-tmux-screen.md + - FIFO Prefect Flow with Redis: tutorials/system-administration/prefect-fifo-redis.md + - AWK Unix Text Processing: tutorials/system-administration/awk-unix-text-processing.md + - 🖼️ Diagrams: + - "Layered Systems Diagrams: Mermaid → SVG": tutorials/diagrams/layered-systems-diagrams-mermaid-to-svg.md + - Mermaid → SVG Workflow Pipeline: tutorials/diagrams/mermaid-to-svg-workflow-pipeline.md + - 📊 Data Science & Visualization: + - Overview: tutorials/data-science-visualization/index.md + - Jupyter Notebook Best Practices (Geospatial Edition): tutorials/data-science-visualization/jupyter-notebook-best-practices-geo.md + - Mermaid Diagrams in MkDocs: tutorials/data-science-visualization/mermaid-diagrams.md + - TikZ Diagrams in LaTeX: tutorials/data-science-visualization/latex-tikz-diagrams.md + - Generative Art in R: tutorials/data-science-visualization/r-generative-art.md + - 🛠️ Development Tools: + - Overview: tutorials/development-tools/index.md + - jq JSON Parsing Mastery: tutorials/development-tools/jq-json-parsing-mastery.md + - find_files for parquet_s3_fdw: tutorials/development-tools/find-files-parquet-fdw.md + - Mosquitto + Python (MQTT Best Practices): tutorials/development-tools/mosquitto-mqtt-python.md + - Python UDP Messaging: tutorials/development-tools/python-udp.md + - Python Modbus Device Communication: tutorials/development-tools/python-modbus-devices.md + - Mixing Tech with Go Glue: tutorials/development-tools/go-tech-mixer.md + - Tauri + rqlite + Syncthing: tutorials/development-tools/tauri-rqlite-syncthing.md + - 🎨 Just for Fun: + - Overview: tutorials/just-for-fun/index.md + - Terminal to GIF: tutorials/just-for-fun/terminal-to-gif.md + - Redis Streams + Web MIDI: tutorials/just-for-fun/redis-midi-music.md + - PostGIS Rasters + WebGL Art: tutorials/just-for-fun/postgis-webgl-art.md + - IoT + IPFS + GraphQL + Blender: tutorials/just-for-fun/iot-ipfs-graphql-blender.md + - MQTT + TimescaleDB + WebSockets + Three.js: tutorials/just-for-fun/mqtt-timescaledb-websockets-threejs.md + - Fastify + Kafka + ClickHouse + WASM + WebGPU: tutorials/just-for-fun/fastify-kafka-clickhouse-wasm-webgpu.md + - Gonzo Prometheus Exporter: tutorials/just-for-fun/gonzo-prometheus-exporter.md + - Git Commit Weather Station: tutorials/just-for-fun/git-weather-node-redis-ipfs-webrtc.md + - Selenium Grid (Docker) + Python: tutorials/just-for-fun/selenium-grid-docker-python.md + - Martin + PostGIS Tiling: tutorials/just-for-fun/martin-postgis-tiling.md + - Managing People in Software Development: tutorials/just-for-fun/managing-people-software-dev.md + - OSC + MQTT + Prometheus + SuperCollider: tutorials/just-for-fun/osc-mqtt-prometheus-supercollider.md + - Fractal Art Explorer (JavaScript): tutorials/just-for-fun/fractal-art-explorer-js.md + - Pi-Based Sample Library Server: tutorials/just-for-fun/pi-sample-server.md + - Go Auth Backend (scratch + Compose): tutorials/just-for-fun/go-auth-scratch-compose.md + - Recursive Cathedral Generator (Kotlin): tutorials/just-for-fun/kotlin-recursive-cathedral.md + - MIDI-Driven Particle Nebula (Kotlin): tutorials/just-for-fun/kotlin-midi-particle-nebula.md + - Cellular Automata Garden (Kotlin): tutorials/just-for-fun/kotlin-cellular-automata-garden.md + - Pi Infinite Art Frame (Kotlin): tutorials/just-for-fun/pi-infinite-art-frame-kotlin.md + - 🔌 Embedded Systems: + - Overview: tutorials/embedded/index.md + - ESP32 E-Ink Environmental Monitor: tutorials/embedded/esp32-eink-sensor-monitor.md + - ESP32 RF Room Light Controller: tutorials/embedded/esp32-rf-room-light-controller.md + - ESP32 + MQTT + Home Assistant: tutorials/embedded/esp32-mqtt-home-assistant-integration.md + + - Lab: + - Overview: lab/index.md + - Just for Fun: tutorials/just-for-fun/index.md + - About: + - Professional Profile: about.md + - Contact & Collaboration: getting-started.md markdown_extensions: + - pymdownx.snippets: + base_path: docs + check_paths: true - pymdownx.highlight: anchor_linenums: true line_spans: __span @@ -547,6 +558,12 @@ plugins: lang: en separator: '[\s\-\_\.]+' - tags + - redirects: + redirect_maps: + projects.md: projects/index.md + - exclude: + glob: + - "_generated/**" - git-revision-date-localized: enable_creation_date: true enable_git_follow: false diff --git a/scripts/rebuild_nav.py b/scripts/rebuild_nav.py new file mode 100644 index 0000000..e85e74d --- /dev/null +++ b/scripts/rebuild_nav.py @@ -0,0 +1,64 @@ +#!/usr/bin/env python3 +"""Rebuild mkdocs nav: nest doctrine/docs under Writing tab.""" + +from __future__ import annotations + +import subprocess +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[1] +MKDOCS = ROOT / "mkdocs.yml" + + +def main() -> None: + original = subprocess.check_output( + ["git", "show", "main:mkdocs.yml"], text=True, cwd=ROOT + ) + lines = original.splitlines(keepends=True) + + nav_start = next(i for i, l in enumerate(lines) if l.startswith("nav:")) + ext_start = next(i for i, l in enumerate(lines) if l.startswith("markdown_extensions:")) + + nav_lines = lines[nav_start:ext_start] + idx = next(i for i, l in enumerate(nav_lines) if l.strip().startswith("- Doctrine:")) + body = nav_lines[idx:] + body = [l for l in body if "Contact & Collaboration" not in l] + + indented: list[str] = [] + for line in body: + if not line.strip(): + indented.append(line) + else: + indented.append(" " + line) + + header = """nav: + - Home: index.md + - Projects: + - Portfolio: projects/index.md + - Documentation sites: projects/documentation-sites.md + - Writing: + - Technical overview: documentation.md + - What's New: whats-new.md + - Tags: tags.md +""" + footer = """ - Lab: + - Overview: lab/index.md + - Just for Fun: tutorials/just-for-fun/index.md + - About: + - Professional Profile: about.md + - Contact & Collaboration: getting-started.md + +""" + new_content = ( + "".join(lines[:nav_start]) + + header + + "".join(indented) + + footer + + "".join(lines[ext_start:]) + ) + MKDOCS.write_text(new_content, encoding="utf-8") + print("Rebuilt mkdocs.yml navigation") + + +if __name__ == "__main__": + main() From e77529923571f8e4f78fc8bd4748540780ebe6d2 Mon Sep 17 00:00:00 2001 From: sempervent Date: Wed, 30 Sep 2026 22:35:18 -0400 Subject: [PATCH 06/12] docs: clarify project maturity and personal-site scope Add ORNL disclaimer, soften absolute production claims, and fix project links. --- docs/about.md | 5 +++++ docs/assets/overrides/partials/footer.html | 7 ++++++- docs/documentation.md | 2 +- docs/getting-started.md | 4 ++-- docs/tutorials/just-for-fun/pi-sample-server.md | 4 +++- 5 files changed, 17 insertions(+), 5 deletions(-) diff --git a/docs/about.md b/docs/about.md index 7c7a18b..e9d2b56 100644 --- a/docs/about.md +++ b/docs/about.md @@ -1,5 +1,10 @@ # Professional Profile +!!! note "Personal site" + This site is Joshua N. Grant's personal portfolio and technical writing. Employer + biography below is factual; open-source projects and opinions here are independent + work unless explicitly attributed to Oak Ridge National Laboratory. + ## About Me I'm a **Geospatial Systems Architect** at Oak Ridge National Laboratory, where I transform research and experiments into production-scalable solutions. My work sits at the intersection of geospatial data, cloud infrastructure, and distributed systems—building the pipelines, architectures, and tools that make complex data problems tractable. diff --git a/docs/assets/overrides/partials/footer.html b/docs/assets/overrides/partials/footer.html index e9eec9d..eb5dee7 100644 --- a/docs/assets/overrides/partials/footer.html +++ b/docs/assets/overrides/partials/footer.html @@ -43,7 +43,12 @@