fix(crawler): use the Markdown front matter title as the document title - #3511
Merged
Merged
Conversation
The Markdown extractor in fess-crawler reports each YAML front matter key as "frontmatter.<key>", so a "title:" in the front matter arrives as frontmatter.title. crawler.metadata.name.mapping only mapped title, Title and dc:title to the title field, so the value was never used and the file transformer fell back to the file name (for example getting-started.md) as the title. Add frontmatter.title=title:string to the default mapping in fess_config.properties and FessConfig. A Markdown file without a front matter title keeps the file name as before.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Use the
title:from a Markdown file's YAML front matter as the document title.When a
.mdfile that hastitle:in its front matter is crawled, the indexed title is the file name (for examplegetting-started.md), not the front matter title.MarkdownExtractorin fess-crawler already parses the front matter and reports each key asfrontmatter.<key>, so the title arrives in the extract data asfrontmatter.title(MarkdownExtractor#extractFrontMatterMetadata). On the Fess side,AbstractFessFileTransformercopies extract metadata into the document only throughcrawler.metadata.name.mapping. That setting maps onlytitle,Titleanddc:title, sofrontmatter.titlewas dropped,hasTitlefound no title, and the transformer fell back to the file name.Changes
fess_config.properties/FessConfig: addfrontmatter.title=title:stringto the defaultcrawler.metadata.name.mapping.A Markdown file without a front matter title keeps the file name as its title, as before. No other front matter keys are mapped.
Tests
FessPropTest: with the shipped configuration,frontmatter.titlemaps totitleasstring, the existingdc:titlemapping is still present, and other front matter keys stay unmapped.mvn -o formatter:format license:formatmvn -o test -Dtest='FessPropTest,FessConfigTest,AbstractFessFileTransformerTest,FessFileTransformerTest,FessXpathTransformerTest': 209 tests, 0 failures, 0 errors