Skip to content

Make the crawlable index actually reachable, and fix deep-link photo URLs - #127

Merged
ledwards merged 2 commits into
mainfrom
crawlable-index-and-deep-links
Aug 12, 2026
Merged

Make the crawlable index actually reachable, and fix deep-link photo URLs#127
ledwards merged 2 commits into
mainfrom
crawlable-index-and-deep-links

Conversation

@ledwards

Copy link
Copy Markdown
Contributor

Two fixes on the way to making root.vc's terminal legible to crawlers without a static mirror.

The crawlable index was invisible to the crawlers it was written for

The homepage block listing every portfolio company and team member was wrapped in <noscript>. That loses it twice over:

  • Text-extraction pipelines (Readability-style, trafilatura, most naive HTML-to-text passes) routinely strip <noscript> as non-content. An LLM crawler running one of those sees an empty page.
  • Googlebot indexes the rendered DOM, and <noscript> content is absent from it once JS runs.

So the one artifact meant to make the terminal readable was skipped by most of its intended audience.

It's now an offscreen div using the .visually-hidden class already in css/styles.css — whose own comment notes that display:none gets discounted by search engines. The text sits in both the raw HTML and the rendered DOM, and sighted visitors still never see it.

Photo fallback URLs broke on deep-link arrivals

js/ascii-art.js built the fallback text by concatenating onto document.location.href, so any fragment ended up in the middle of the path. Landing on /#whois-avidan rendered:

[ Photo: https://root.vc/#whois-avidanimages/avidan.png ]

Resolving against document.baseURI drops the fragment the way URL resolution is meant to. This matters more now that deep links are becoming the main way to reach specific content.

Verification

  • npm test — 313/313 passing, including a new test that locks in the no-<noscript>/no-display:none decision so it doesn't get "tidied" back
  • Built dist/ and served it in a browser: #text-version is in the rendered DOM, clips to 1×1, adds no scroll overflow, and a plain text extraction of the page returns all 6318 characters
  • Screenshotted /#whois-avidan before and after — terminal visually identical, photo URL now correct

🤖 Generated with Claude Code

ledwards and others added 2 commits August 12, 2026 11:39
The block that gives non-JS crawlers the portfolio and team was wrapped in
<noscript>, which loses it to exactly the audience it was written for.
Extraction pipelines routinely strip <noscript> as non-content, and Googlebot
indexes the rendered DOM, which drops <noscript> entirely once JS runs. So the
one artifact meant to make the terminal legible to crawlers was invisible to
most of them.

It is now an offscreen div using the .visually-hidden class already in
styles.css, whose own comment notes that display:none gets discounted. The
text is present in both the raw HTML and the rendered DOM, and sighted
visitors still never see it.

Verified in a browser against a built dist/: the element is in the rendered
DOM, clips to 1x1, adds no scroll overflow, and a plain text extraction of the
page now returns all 6318 characters.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The fallback text concatenated the image path onto document.location.href, so
any fragment in the URL landed in the middle of the result. Arriving at
/#whois-avidan rendered "[ Photo: https://root.vc/#whois-avidanimages/
avidan.png ]".

Resolving against document.baseURI drops the fragment the way URL resolution
is supposed to. Matters more now that deep links are becoming the primary way
to reach specific content.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@ledwards
ledwards merged commit f167898 into main Aug 12, 2026
6 checks passed
@ledwards
ledwards deleted the crawlable-index-and-deep-links branch August 12, 2026 18:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant