Skip to content

Wire up API rate limiting in shadow mode - #205

Merged
brianglass merged 1 commit into
mainfrom
api-rate-limit-shadow-mode
Aug 20, 2026
Merged

Wire up API rate limiting in shadow mode#205
brianglass merged 1 commit into
mainfrom
api-rate-limit-shadow-mode

Conversation

@brianglass

Copy link
Copy Markdown
Owner

Summary

  • ORTHOCAL_API_RATELIMIT/BURST_RATE (calendarium/api.py:26) was defined but never actually wired to anything -- the rate-string format ("5/s", matching django-ninja's SimpleRateThrottle expectations exactly) makes clear it was meant to drive throttling; that wiring was just never finished.
  • Rather than flip straight to enforcing 429s, this adds ShadowAnonRateThrottle: runs the exact same per-IP check ninja's built-in AnonRateThrottle would, logs a warning for every request that would have been rejected (ip, path, user-agent), but always lets the request through. No client can be broken by this PR.
  • Once the logs show it's safe (no legitimate client trips it), swap ShadowAnonRateThrottle for AnonRateThrottle directly in throttle=[...] -- same rate, same per-IP identity behavior, only the enforcement changes.

Context / investigation

  • Confirmed via a Cloud Logging query that a rtdx-calendar-labeled client (self-described as an "offline Orthodox calendar" build tool, referencing a GitHub repo that returns 404) accounts for ~9.5% of recent traffic, spread across 127 distinct IPs -- worth noting that per-IP throttling won't meaningfully curb that specific pattern (each IP gets its own allowance), but it's still valuable protection against any single misbehaving client.
  • Confirmed Firebase Hosting has no built-in rate limiting/WAF at all (checked against Firebase's own docs) -- Cloud Armor would be the GCP-native option but requires an external Load Balancer this deploy topology doesn't have, so application-level throttling is the practical lever here.
  • Read ninja 1.6.3's actual installed throttling source (not docs, to avoid version drift) to confirm: on throttle, it returns a bare 429 with {"detail": "Too many requests."} and no Retry-After header by default -- worth adding a custom exception handler for that later if/when this moves to enforcing.
  • Confirmed client identification behind Cloud Run's proxy layer works correctly by default (get_ident() uses the full X-Forwarded-For value when NINJA_NUM_PROXIES isn't set, which Cloud Run populates reliably) -- no additional settings needed.

Verification

  • Local dev's local_settings.py swaps CACHES to DummyCache (a deliberate, unrelated dev convenience -- keeps local testing always-fresh), which silently no-ops set()/get() and would have hidden any real bug in the throttle logic. Re-verified with a real cache backend (LocMemCache) swapped in for testing only: history correctly accumulates, and the shadow warning fires exactly on the requests exceeding the configured rate (5/s) -- confirmed 5 allowed, then 5 consecutive "would rate-limit" warnings, all 10 requests still returning success.
  • Full test suite: docker compose run --rm tests -- 162 tests, all passing.

Test plan

  • Verified shadow logic end-to-end against a real cache backend (described above)
  • Full test suite passes
  • After deploy, watch Cloud Logging for Would rate-limit warnings for a while to confirm no legitimate client would be broken by enforcing
  • Once confirmed safe, follow up with a PR swapping ShadowAnonRateThrottleAnonRateThrottle (and ideally a Retry-After header on the real 429)

🤖 Generated with Claude Code

https://claude.ai/code/session_01Hf6j2xXQXywHVh3HAVRxB3

ORTHOCAL_API_RATELIMIT / BURST_RATE (calendarium/api.py) was defined
but never actually passed to NinjaAPI's throttle= parameter -- the
rate string format (django-ninja's SimpleRateThrottle expects exactly
"N/period") makes clear this was meant to be wired up, just never
finished.

Rather than flip straight to enforcing (returning real 429s), this
adds ShadowAnonRateThrottle: runs the same per-IP check ninja's
built-in AnonRateThrottle would, logs a warning for every request
that would have been rejected (ip, path, user-agent), but always lets
the request through. No client can be broken by this. Once the logs
show it's safe -- no legitimate client would trip it -- swap
ShadowAnonRateThrottle for AnonRateThrottle directly; same rate, same
per-IP behavior, only the enforcement changes.

Investigated ninja 1.6.3's actual installed throttling source
(SimpleRateThrottle.get_ident, NINJA_NUM_PROXIES) to confirm client
identification behind Cloud Run's proxy layer works correctly by
default without needing NINJA_NUM_PROXIES configured, since Cloud Run
reliably sets X-Forwarded-For.

Verified the shadow logic directly with a real cache backend (local
dev's local_settings.py swaps CACHES to DummyCache, which silently
no-ops set()/get() and would have hidden any real bug here) --
confirmed history accumulates correctly and the warning fires exactly
on the requests that exceed the rate, while every request still
returns 200. Full test suite (162 tests) passes.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Hf6j2xXQXywHVh3HAVRxB3
@brianglass
brianglass merged commit c2cb195 into main Aug 20, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant