Skip to content

Do not override some placeholders which are built from records - #262

Merged
Watson1978 merged 1 commit into
fluent:masterfrom
kenhys:fix-tag-override
Aug 21, 2026
Merged

Do not override some placeholders which are built from records#262
Watson1978 merged 1 commit into
fluent:masterfrom
kenhys:fix-tag-override

Conversation

@kenhys

@kenhys kenhys commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

In the previous versions, there is a possibility that forged record overrides placeholder unexpectedly.
To guard such a situation, block overriding same placeholder.

@kenhys
kenhys marked this pull request as ready for review August 18, 2026 01:31
@kenhys
kenhys requested a review from Watson1978 August 18, 2026 01:31
@Watson1978
Watson1978 requested a lite review from Copilot August 19, 2026 06:19

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens placeholder expansion so tag-derived placeholders (e.g., tag_parts, tag_prefix, tag_suffix) cannot be overridden by same-named fields coming from records, preventing forged record content from influencing tag-based metric labels.

Changes:

  • Treat tag-derived placeholder expansions as “reserved” and re-apply them after merges so they always win.
  • Add unit tests to ensure record-provided tag_parts/tag_prefix/tag_suffix cannot shadow tag-derived placeholders (including dynamic placeholder expansion).
  • Add an integration-style spec to validate metrics labels bound to tag placeholders remain sourced from the Fluentd tag even when the record carries shadowing fields.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
spec/fluent/plugin/prometheus/placeholder_expander_spec.rb Adds coverage ensuring tag-derived placeholders cannot be shadowed by record keys, including dynamic placeholder scenarios.
spec/fluent/plugin/filter_prometheus_spec.rb Adds a regression test verifying tag-based label placeholders are not affected by record fields like tag_parts.
lib/fluent/plugin/prometheus/placeholder_expander.rb Implements “reserved” tag-derived placeholders and ensures they override record-provided placeholders during build/merge.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@kenhys

kenhys commented Aug 20, 2026

Copy link
Copy Markdown
Contributor Author

Refactored to more simple implementation.
In the previous version, added third argument for PlaceholderExpander, but revert that way.

@Watson1978 Watson1978 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Normally, placeholders like ${tag_parts[1]} are supposed to be populated with values derived from splitting Fluentd "tags". Therefore, the plugin implemented a guard to prevent users from overwriting tag values by sending a key named tag_parts within the record.

However, if a user intentionally sent the literal string including the index as a record key—such as {"tag_parts[1]": "forged data"}—it would bypass this guard.

Consequently, this allowed users to arbitrarily overwrite and forge non-existent tags (for example, specifying a value for ${tag_parts[3]} when the tag only has one part) using values provided from the record side.

tag_placeholders is filled by build_tag(value), so it contains exactly the placeholders that this particular tag can generate: ${tag}, and one entry per part for ${tag_parts[N]}, ${tag_prefix[N]} and ${tag_suffix[N]}.

Any index outside that range is still absent from tag_placeholders, so the record wins there just as it did before.

This example fails on the PR branch in spec/fluent/plugin/filter_prometheus_spec.rb:

   context 'when the tag does not reach the index' do
      let(:config) {
        BASE_CONFIG + %(
          <metric>
            name tagged
            type counter
            desc Something foo.
            key foo
            <labels>
              part ${tag_parts[1]}
            </labels>
          </metric>
        )
      }

      it 'leaves the placeholder unknown' do
        driver.run(default_tag: 'singlepart') do
          driver.feed(event_time, {'tag' => 'forged.tag', 'foo' => 1, 'tag_parts[1]' => 'forged'})
        end
        expect(registry.get(:tagged).values.keys).to eq([{part: '${tag_parts[1]}'}])
      end
    end

    context 'when the tag never builds the placeholder' do
      let(:config) {
        BASE_CONFIG + %(
          <metric>
            name tagged
            type counter
            desc Something foo.
            key foo
            <labels>
              pfx ${tag_prefix[-1]}
              sfx ${tag_suffix[-1]}
            </labels>
          </metric>
        )
      }

      it 'leaves the placeholder unknown' do
        driver.run(default_tag: tag) do
          driver.feed(event_time, {'foo' => 1, 'tag_prefix[-1]' => 'forged', 'tag_suffix[-1]' => 'forged'})
        end
        expect(registry.get(:tagged).values.keys).to eq([{pfx: '${tag_prefix[-1]}', sfx: '${tag_suffix[-1]}'}])
      end
    end
 
Failures:

  1) Fluent::Plugin::PrometheusFilter a record shadowing a tag placeholder when the tag does not reach the index leaves the placeholder unknown
     Failure/Error: expect(registry.get(:tagged).values.keys).to eq([{part: '${tag_parts[1]}'}])

       expected: [{part: "${tag_parts[1]}"}]
            got: [{part: "forged"}]

       (compared using ==)

       Diff:
       @@ -1 +1 @@
       -[{part: "${tag_parts[1]}"}]
       +[{part: "forged"}]

     # ./spec/fluent/plugin/filter_prometheus_spec.rb:101:in 'block (4 levels) in <top (required)>'

  2) Fluent::Plugin::PrometheusFilter a record shadowing a tag placeholder when the tag never builds the placeholder leaves the placeholder unknown
     Failure/Error: expect(registry.get(:tagged).values.keys).to eq([{pfx: '${tag_prefix[-1]}', sfx: '${tag_suffix[-1]}'}])

       expected: [{pfx: "${tag_prefix[-1]}", sfx: "${tag_suffix[-1]}"}]
            got: [{pfx: "forged", sfx: "forged"}]

       (compared using ==)

       Diff:
       @@ -1 +1 @@
       -[{pfx: "${tag_prefix[-1]}", sfx: "${tag_suffix[-1]}"}]
       +[{pfx: "forged", sfx: "forged"}]

     # ./spec/fluent/plugin/filter_prometheus_spec.rb:125:in 'block (4 levels) in <top (required)>'

Comment thread lib/fluent/plugin/prometheus/placeholder_expander.rb Outdated
In the previous versions, there is a possibility that forged record
overrides placeholder unexpectedly.
To guard such a situation, block overriding same placeholder.

Signed-off-by: Kentaro Hayashi <hayashi@clear-code.com>

@Watson1978 Watson1978 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@Watson1978
Watson1978 merged commit fdef0e5 into fluent:master Aug 21, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants