Filter secrets from logged query strings and Referer/Location URLs

[PetrHeinz]

HTTPEvents logs the raw query string, so ?password=hunter2&token=… lands in http_request_received.query_string verbatim. The queries of the URLs in the Referer request header and the Location response header are stored raw in headers_json too. Yesterday's red-team found password=hunter2 and token= values in plain text in Rack, Sinatra and on all seven Rails versions, while Rails' own params on the same request showed [FILTERED].

What changes:

• New HTTPEvents.query_string_filters setting, a class-level list like http_header_filters. It defaults to DEFAULT_QUERY_STRING_FILTERS = %w[passw secret token _key crypt salt certificate otp ssn cvv cvc], which is the list Rails puts in new apps, minus email.
• Matching works like Rails' filter_parameters: a String or Symbol filters every parameter whose name contains it, ignoring case, and a Regexp filters every name it matches (=~). Other entries, such as the Procs Rails allows, are ignored.
• Names are URL-decoded before matching, and nested names are matched as the full decoded name, so user[password] and user%5Bpassword%5D are both filtered.
• A matching parameter's value becomes [FILTERED]. The rest of the string stays byte for byte unchanged, including the parameter name, separators, empty values and parameters without a value.
• The filters apply to query_string, and to the query part of Referer and Location in headers_json, whatever the header's casing. That includes the single event of collapsed mode (collapse_into_single_event / logrageify!).
• Setting [] disables the filtering.

Logtail::Integrations::Rack::HTTPEvents.query_string_filters =
  Logtail::Integrations::Rack::HTTPEvents::DEFAULT_QUERY_STRING_FILTERS + ["code", /\Aapi_/]

Behaviour and compatibility:

• Logged query strings and Referer/Location URLs now carry [FILTERED] for matching parameters, where they had the raw value before. Setting the list to [] restores the old output.
• Like Rails' filtered_query_string (and Rack before 3.0), both & and ; separate parameters. A filtered value runs up to the next separator, so token=abc=def becomes token=[FILTERED].
• The fragment of a Location URL isn't touched, and neither are paths, which never contain the query in these events.
• In Rails apps, logtail-rails will use the app's config.filter_parameters instead of this default (separate PR, targeting the logtail-rails 0.3.0 minor).
• No new dependencies, and the code runs on Ruby 2.5 and Rack 1.2 to 3.x.

Targets the 0.2.9 patch release. The logtail-rails PR that applies filter_parameters to URLs runs its CI against this branch until 0.2.9 is released.

The first commit only adds the tests and is expected to fail on CI; the second commit makes them pass.

🤖 Generated with Claude Code