Skip to content

docs(15.9): Amazon S3 moved to the fess-storage-s3 plugin - #528

Merged
marevol merged 1 commit into
masterfrom
docs/storage-s3-plugin
Sep 10, 2026
Merged

docs(15.9): Amazon S3 moved to the fess-storage-s3 plugin#528
marevol merged 1 commit into
masterfrom
docs/storage-s3-plugin

Conversation

@marevol

@marevol marevol commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Merge after codelibs/fess#3427 and codelibs/fess#3428. The corrected notes describe the reporting #3427 adds; if that does not land in 15.9, the "silently" and "an empty list" wording has to come back.

The AWS SDK left the distribution in codelibs/fess#3428, so s3:// crawling and the s3 and s3_compat storage types come from the new fess-storage-s3 plugin. 36 files, all seven languages, 15.9 development trees only.

The new upgrade section follows the Google Cloud Storage one it sits beside, using each language's own sentence frames rather than a fresh translation — headings, the two install routes, the :: command block, and the closing "nothing to do if you don't use it" all come from the GCS section of that language, and the .. note:: bodies from the GCS note already on the same page.

storage.type still defaults to auto, which resolves to S3 when no endpoint is set, so the note says the storage page stops working until the plugin is installed even for an installation that never named s3 itself. It also says what survives an upgrade: storage.* lives in WEB-INF/conf/system.properties, and an existing crawler.file.protocols is not replaced.

The GCS section was wrong in three places, and this change falsifies a fourth

  • "rejected as an invalid protocol" was half the story. CreateForm.paths carries @UriType(protocolType = ProtocolType.FILE), and UriTypeValidator resolves that to ProtocolHelper.getFileProtocols(), so a fresh installation does refuse to save the path. An installation that upgrades keeps its own crawler.file.protocols, so there the path is accepted and no crawler client handles it. Both are now described, because the second is the one an existing installation meets.
  • "the storage page reports that no client is registered" was not true at all. AdminStorageAction.getFileItems swallowed the failure into a logger.debug line and rendered an empty file list. #3427 is what makes it report, and names the plugin.
  • "Amazon S3 and S3-compatible storage stay in the distribution and are unaffected" now points at the new section instead.
  • The shipped crawler.file.protocols was quoted as file,smb,smb1,ftp,s3, which #3428 makes file,smb,smb1,ftp. Corrected in the GCS section, the new S3 section, the removed-storage-protocol section, config/crawler-advanced.rst and en/config/properties.rst. grep -rn "file,smb,smb1,ftp,s3" over the 15.9 trees now returns nothing.

Verification

docutils 0.22.4 across 43 files, parsed before and after the way Sphinx parses — doctitle_xform=False, sectsubtitle_xform=False, file_insertion_enabled=False, with the conf.ini [replace] items prepended as the prolog so |Fess| resolves. 300 assertions, 0 failures.

  • System messages identical per file, compared as counters with path:lineno: prefixes stripped so that line shifts do not register.
  • No new unparsed markup in the rendered text. The residue that remains is exactly the two pre-existing defects: the nested literal in config/crawler-advanced.rst and the |Fess| inside bold in install/upgrade.rst.
  • One document-level section per file throughout; total section count +1 on exactly the seven install/upgrade.rst and +0 on the other 36, with the expected value stated up front rather than read off the output. The added section is printed by name for each language.
  • Every inline literal introduced parses as a literal node — the check that catches an inline start-string sitting against a CJK character, which docutils drops silently. 11 asserted on each install/upgrade.rst, fewer on the other pages.
  • .. note:: counts up by exactly the notes added, +0 for ko/admin/fileconfig-guide.rst which has no example sections. This is what catches a mis-indented note body.
  • Heading rules compared by display width, counting East Asian wide and fullwidth characters as two. The seven new headings match their titles exactly, as the seven GCS headings do. Korean pads to display width as well — Hangul is W — so its rules are padded.

Non-vacuity: 14/14. Confirmed against known-broken inputs — a literal and **bold** jammed against CJK text produce zero literal/strong nodes and leave backtick/star residue, a list-table row with one cell instead of two produces a new system message, an unindented .. note:: body produces zero note nodes and a directive error, a short heading rule is detected — and against the correct controls, which produce none of those.

Korean

ko/15.9/admin/storage-guide.rst is an earlier translation with no per-backend subsections, so its note goes after the storage-server list, and it omits the sentence about the Auto type resolving to S3 — that page never introduces the Type control, and pointing at a control the reader has not met would be worse than silence.

Still out of step there, unchanged by this PR: the overview names only MinIO where the others name Amazon S3, Google Cloud Storage and S3-compatible storage; there are no Common/S3/GCS/MinIO settings subsections, so Region, Project ID and Credentials Path are undocumented; and admin/fileconfig-guide.rst has no bucket-crawling example sections, so it carries neither the S3 nor the GCS example note. #526 skipped this file for GCS as well.

Deliberately not changed

One figure is knowingly stale

install/upgrade.rst still says the ZIP drops from 438.5 MiB to 204.7 MiB, measured for the Node.js removal alone. The GCS, S3 and SSO extractions each shrink it further, so the final 15.9 number has to be measured from a built distribution before release. No guess was substituted.

The AWS SDK left the distribution, so s3:// crawling and the s3 and s3_compat
storage types come from the fess-storage-s3 plugin. The new upgrade section
follows the Google Cloud Storage one it sits beside, with each language's own
sentence frames rather than a fresh translation.

storage.type still defaults to auto, which resolves to S3 when no endpoint is
set, so the note says the storage page stops working until the plugin is
installed even for an installation that never named s3 itself. It also says
what survives: storage.* lives in WEB-INF/conf/system.properties, and an
existing crawler.file.protocols is not replaced by an upgrade.

The GCS section is corrected in the same pass. Three of its statements were
wrong, and one has been falsified by this change:

- "rejected as an invalid protocol" was half the story. CreateForm.paths
  carries @UriType(protocolType = ProtocolType.FILE), and UriTypeValidator
  resolves that to ProtocolHelper.getFileProtocols(), so a fresh installation
  does refuse to save the path. An installation that upgrades keeps its own
  crawler.file.protocols, so there the path is accepted and no crawler client
  handles it. Both behaviours are now described.
- "the storage page reports that no client is registered" was not true at all:
  AdminStorageAction.getFileItems swallowed the failure into a debug line and
  rendered an empty file list. codelibs/fess#3427 is what makes it report.
- "Amazon S3 and S3-compatible storage stay in the distribution and are
  unaffected" is now false, and points at the new section instead.
- The shipped crawler.file.protocols was quoted as file,smb,smb1,ftp,s3, which
  this change makes file,smb,smb1,ftp. Corrected in the GCS section, the new S3
  section, the removed-storage-protocol section, config/crawler-advanced.rst
  and en/config/properties.rst. No occurrence of the old value is left in the
  15.9 trees.

storage-guide, fileconfig-guide, crawler-advanced and general-guide gain the
plugin prerequisite in the position the GCS note already occupies in each. The
one note under S3 Settings names S3-compatible storage as well, rather than
being duplicated under the MinIO subsection that the same plugin serves.

Verified with docutils 0.22.4 across 43 files, before and after, parsed the way
Sphinx parses (doctitle_xform=False, the conf.ini substitutions prepended):
system messages identical per file as counters with line prefixes stripped, no
new unparsed markup in the rendered text, one document-level section per file
throughout, the section count up by exactly one on the seven upgrade.rst and
unchanged on the other 36, the note count up by exactly the notes added, and
every inline literal introduced parsed as a literal node -- which is the check
that catches an inline start-string sitting against a CJK character. Heading
rules were compared by display width, counting East Asian wide and fullwidth
characters as two; the seven new headings match their titles exactly, as the
seven GCS headings do. Korean pads to display width as well, so its rules are
padded.

The checks were confirmed non-vacuous against known-broken inputs -- literals
and bold markup jammed against CJK text, a list-table row with the wrong cell
count, an unindented note body, a short heading rule -- and against correct
controls, so they are not simply always-red.

ko/15.9/admin/storage-guide.rst is an earlier translation with no per-backend
subsections, so its note goes after the storage-server list and omits the
sentence about the Auto type, which that page never introduces. What is still
out of step there is unchanged by this commit.
@marevol
marevol added this pull request to stack #530 September 10, 2026 04:17
@marevol marevol self-assigned this Sep 10, 2026
@marevol
marevol merged commit d87cd3e into master Sep 10, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant