docs(15.9): Amazon S3 moved to the fess-storage-s3 plugin - #528
Merged
Conversation
The AWS SDK left the distribution, so s3:// crawling and the s3 and s3_compat storage types come from the fess-storage-s3 plugin. The new upgrade section follows the Google Cloud Storage one it sits beside, with each language's own sentence frames rather than a fresh translation. storage.type still defaults to auto, which resolves to S3 when no endpoint is set, so the note says the storage page stops working until the plugin is installed even for an installation that never named s3 itself. It also says what survives: storage.* lives in WEB-INF/conf/system.properties, and an existing crawler.file.protocols is not replaced by an upgrade. The GCS section is corrected in the same pass. Three of its statements were wrong, and one has been falsified by this change: - "rejected as an invalid protocol" was half the story. CreateForm.paths carries @UriType(protocolType = ProtocolType.FILE), and UriTypeValidator resolves that to ProtocolHelper.getFileProtocols(), so a fresh installation does refuse to save the path. An installation that upgrades keeps its own crawler.file.protocols, so there the path is accepted and no crawler client handles it. Both behaviours are now described. - "the storage page reports that no client is registered" was not true at all: AdminStorageAction.getFileItems swallowed the failure into a debug line and rendered an empty file list. codelibs/fess#3427 is what makes it report. - "Amazon S3 and S3-compatible storage stay in the distribution and are unaffected" is now false, and points at the new section instead. - The shipped crawler.file.protocols was quoted as file,smb,smb1,ftp,s3, which this change makes file,smb,smb1,ftp. Corrected in the GCS section, the new S3 section, the removed-storage-protocol section, config/crawler-advanced.rst and en/config/properties.rst. No occurrence of the old value is left in the 15.9 trees. storage-guide, fileconfig-guide, crawler-advanced and general-guide gain the plugin prerequisite in the position the GCS note already occupies in each. The one note under S3 Settings names S3-compatible storage as well, rather than being duplicated under the MinIO subsection that the same plugin serves. Verified with docutils 0.22.4 across 43 files, before and after, parsed the way Sphinx parses (doctitle_xform=False, the conf.ini substitutions prepended): system messages identical per file as counters with line prefixes stripped, no new unparsed markup in the rendered text, one document-level section per file throughout, the section count up by exactly one on the seven upgrade.rst and unchanged on the other 36, the note count up by exactly the notes added, and every inline literal introduced parsed as a literal node -- which is the check that catches an inline start-string sitting against a CJK character. Heading rules were compared by display width, counting East Asian wide and fullwidth characters as two; the seven new headings match their titles exactly, as the seven GCS headings do. Korean pads to display width as well, so its rules are padded. The checks were confirmed non-vacuous against known-broken inputs -- literals and bold markup jammed against CJK text, a list-table row with the wrong cell count, an unindented note body, a short heading rule -- and against correct controls, so they are not simply always-red. ko/15.9/admin/storage-guide.rst is an earlier translation with no per-backend subsections, so its note goes after the storage-server list and omits the sentence about the Auto type, which that page never introduces. What is still out of step there is unchanged by this commit.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The AWS SDK left the distribution in codelibs/fess#3428, so
s3://crawling and thes3ands3_compatstorage types come from the newfess-storage-s3plugin. 36 files, all seven languages, 15.9 development trees only.The new upgrade section follows the Google Cloud Storage one it sits beside, using each language's own sentence frames rather than a fresh translation — headings, the two install routes, the
::command block, and the closing "nothing to do if you don't use it" all come from the GCS section of that language, and the.. note::bodies from the GCS note already on the same page.storage.typestill defaults toauto, which resolves to S3 when no endpoint is set, so the note says the storage page stops working until the plugin is installed even for an installation that never nameds3itself. It also says what survives an upgrade:storage.*lives inWEB-INF/conf/system.properties, and an existingcrawler.file.protocolsis not replaced.The GCS section was wrong in three places, and this change falsifies a fourth
CreateForm.pathscarries@UriType(protocolType = ProtocolType.FILE), andUriTypeValidatorresolves that toProtocolHelper.getFileProtocols(), so a fresh installation does refuse to save the path. An installation that upgrades keeps its owncrawler.file.protocols, so there the path is accepted and no crawler client handles it. Both are now described, because the second is the one an existing installation meets.AdminStorageAction.getFileItemsswallowed the failure into alogger.debugline and rendered an empty file list. #3427 is what makes it report, and names the plugin.crawler.file.protocolswas quoted asfile,smb,smb1,ftp,s3, which #3428 makesfile,smb,smb1,ftp. Corrected in the GCS section, the new S3 section, the removed-storage-protocol section,config/crawler-advanced.rstanden/config/properties.rst.grep -rn "file,smb,smb1,ftp,s3"over the 15.9 trees now returns nothing.Verification
docutils 0.22.4 across 43 files, parsed before and after the way Sphinx parses —
doctitle_xform=False,sectsubtitle_xform=False,file_insertion_enabled=False, with theconf.ini[replace]items prepended as the prolog so|Fess|resolves. 300 assertions, 0 failures.path:lineno:prefixes stripped so that line shifts do not register.config/crawler-advanced.rstand the|Fess|inside bold ininstall/upgrade.rst.install/upgrade.rstand +0 on the other 36, with the expected value stated up front rather than read off the output. The added section is printed by name for each language.literalnode — the check that catches an inline start-string sitting against a CJK character, which docutils drops silently. 11 asserted on eachinstall/upgrade.rst, fewer on the other pages... note::counts up by exactly the notes added, +0 forko/admin/fileconfig-guide.rstwhich has no example sections. This is what catches a mis-indented note body.W— so its rules are padded.Non-vacuity: 14/14. Confirmed against known-broken inputs — a literal and
**bold**jammed against CJK text produce zeroliteral/strongnodes and leave backtick/star residue, alist-tablerow with one cell instead of two produces a new system message, an unindented.. note::body produces zeronotenodes and a directive error, a short heading rule is detected — and against the correct controls, which produce none of those.Korean
ko/15.9/admin/storage-guide.rstis an earlier translation with no per-backend subsections, so its note goes after the storage-server list, and it omits the sentence about the Auto type resolving to S3 — that page never introduces the Type control, and pointing at a control the reader has not met would be worse than silence.Still out of step there, unchanged by this PR: the overview names only MinIO where the others name Amazon S3, Google Cloud Storage and S3-compatible storage; there are no Common/S3/GCS/MinIO settings subsections, so Region, Project ID and Credentials Path are undocumented; and
admin/fileconfig-guide.rsthas no bucket-crawling example sections, so it carries neither the S3 nor the GCS example note. #526 skipped this file for GCS as well.Deliberately not changed
api/admin/api-admin-fileconfig.rst— docs(15.9): Google Cloud Storage moved to the fess-lib-gcs plugin #526 left it alone for GCS and it still listsgcs://with no plugin note. Annotatings3://but notgcs://on a path-format reference would read as an inconsistency; both rows or neither.dev/plugin-architecture.rst— thefess-storage-*row already says what it needs to.admin/general-guide.rstgains the prerequisite on the S3 bullet only, matching what docs(15.9): Google Cloud Storage moved to the fess-lib-gcs plugin #526 did for GCS. Auto needs the plugin too when the endpoint is blank; that trap is carried by the upgrade note and the storage-guide note instead.One figure is knowingly stale
install/upgrade.rststill says the ZIP drops from 438.5 MiB to 204.7 MiB, measured for the Node.js removal alone. The GCS, S3 and SSO extractions each shrink it further, so the final 15.9 number has to be measured from a built distribution before release. No guess was substituted.