Skip to content

Count free-tier delivery on the earnings and slots pages too - #225

Merged
ralyodio merged 1 commit into
masterfrom
worktree-ads-free-tier-reporting
Sep 1, 2026
Merged

Count free-tier delivery on the earnings and slots pages too#225
ralyodio merged 1 commit into
masterfrom
worktree-ads-free-tier-reporting

Conversation

@ralyodio

@ralyodio ralyodio commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

What was wrong

/dashboard/ads/earnings and /dashboard/ads/slots read impressions and
clicks off ad_slot_stats / ad_campaign_stats. Those columns are the
tier-'paid' halves of the views — the free halves live in free_impressions /
free_clicks, which neither page selected.

Once every slot and every campaign belonged to one account, ad_charge_click
took its self-deal branch on every fill and paid delivery stopped:

Month Paid impressions Free impressions
Jul 2026 17,303 1,503
Aug 2026 3 260,905
Sep 2026 0 84,274

So both pages have read zero ever since, while the campaigns dashboard — which
sums both halves after #199 — showed six figures over the same window.
rssamplifier.com delivered 116,071 impressions and its row said 0.

#199 fixed this measure on the campaigns dashboard and never reached here.

Two further defects, found in the same place:

  • The views are lifetime, with no window at all, while the page and the PDF
    header both promise "last N days". The figures were wrong twice over: the
    wrong tier, for the wrong period. "Total earned $5.30" was all-time July money
    presented as a 30-day figure.
  • 57,060 clicks were visible to nothing. A click we refuse to bill is
    recorded with valid = false, and resolveClick's insert left tier at its
    'paid' default, so the row matched neither the billed bucket (valid) nor
    the free bucket (not valid and tier = 'free').

What this does

  • Adds ad_slot_totals(p_since) — the publisher-side twin of the existing
    ad_campaign_totals, same shape and same rules (security invoker,
    owner_id = auth.uid(), one row per slot so it cannot hit PostgREST's
    1000-row cap). Plus an invalid_clicks column, which has no advertiser-side
    equivalent.
  • loadEarnings reads the two RPCs instead of the two views, so the tables and
    the impression/click totals cover the window they claim and count both tiers.
    This fixes the PDF report at the same time.
  • The slots page reads the same RPC with a null window, keeping its lifetime
    semantics and losing the zeros.
  • Money stays lifetime on purpose, and the page now says so. "Available to
    withdraw" is lifetime earnings minus lifetime payouts; scoping either side to
    30 days would under-report a balance the account is owed. Tiles are grouped
    under Balance · all time, tables under Delivery · last 30 days, instead of
    one heading claiming a period for both.
  • Invalid clicks stay out of delivery — a bot click is not delivery, and folding
    it in would put a 16% CTR on the page — but the count is now reported in a
    line of its own, and the insert writes tier explicitly so the bucket is a
    decision rather than a column default.

Verification

pnpm install is still broken at HEAD (the @profullstack/autoblog pin is a
dead commit), so nothing could be typechecked or tested locally. CI ran and is
green
, including test + typecheck (tsc --noEmit plus the full vitest
suite, 1m24s) — the Actions billing lock from 2026-08-31 has been cleared.

Also verified directly against prod:

  • ad_slot_totals over the last 30 days returns 174,959 impressions / 5,550
    clicks / 57,063 invalid
    , matching a raw count over ad_impressions and
    ad_clicks exactly, in ~360ms warm against an 8s statement timeout.
    Lifetime is ~380ms.
  • rssamplifier.com, which the slots page rendered as 0, returns 116,110
    lifetime delivered impressions.
  • New tests/ads-earnings-free-tier.test.ts covers both tiers, the invalid
    bucket staying out of delivery, the bigint-as-string coercion, RPC failure
    falling back to zeros rather than throwing, and the UTC-day window alignment.

Migration

20260901120000_ad_slot_totals.sql is already applied to prod. It adds a
function and alters nothing else, so it was safe to land before the code; the
new grants match ad_campaign_totals' ACL exactly. No data was modified.

🤖 Generated with Claude Code

https://claude.ai/code/session_0147H2VoYJS2WUz4JQmaLQKV

The earnings page and the slots page read `impressions` and `clicks` off
ad_slot_stats / ad_campaign_stats. Those are the tier-'paid' halves of the
views; the free halves sit in `free_impressions` / `free_clicks`, which neither
page ever selected. Once every slot and every campaign belonged to one account,
ad_charge_click took its self-deal branch on every fill and paid delivery
stopped: 17,303 paid impressions in July, 3 in August, 0 in September. So both
pages have read zero ever since, while the campaigns dashboard — which sums both
halves after #199 — showed six figures over the same window. rssamplifier.com
delivered 116,071 impressions and its row said 0.

#199 fixed this measure on the campaigns dashboard and did not reach here.

The same views are also lifetime, with no window at all, while the page and the
PDF header both promise "last N days" — so the figures were wrong twice over:
the wrong tier, for the wrong period. loadEarnings now reads ad_campaign_totals
and a new ad_slot_totals, both windowed and both returning each tier, and the
window is the same whole-UTC-day span the chart above the tables already drew.

Money stays lifetime on purpose, and the page now says so. "Available to
withdraw" is lifetime earnings minus lifetime payouts; scoping either side to 30
days would under-report a balance the account is actually owed. The tiles are
grouped under "Balance · all time" and the tables under "Delivery · last 30
days" rather than one heading claiming a period for both.

Invalid clicks were the third gap. A click we refuse to bill is recorded with
valid = false, and resolveClick's insert left `tier` at its 'paid' default, so
the row matched neither the billed bucket (valid) nor the free bucket (not valid
and tier = 'free'). 57,060 clicks had collected there, visible to nothing. They
stay out of the delivery figures deliberately — a bot click is not delivery, and
folding it in would put a 16% CTR on the page — but ad_slot_totals returns the
count and the page reports it in a line of its own. The insert now writes `tier`
explicitly, so the bucket is a decision rather than a default.

Verified against prod: ad_slot_totals over the last 30 days returns 174,959
impressions / 5,550 clicks / 57,063 invalid, matching a raw count over
ad_impressions and ad_clicks exactly, in ~360ms against an 8s statement timeout.

The migration adds a function and alters nothing, so it is already applied.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0147H2VoYJS2WUz4JQmaLQKV
@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown

ThreatCrush Security Scan

39 finding(s)

HIGH/CRITICAL: 2 | MEDIUM: 28 | LOW: 9

Severity Rule Location
HIGH tls-verification-disabled lib/onion.ts:47
HIGH secret-generic-credential lib/sp/platforms/facebook.ts:32
MEDIUM js-unescaped-html-sink app/(app)/dashboard/admin/email-broadcast/EmailBroadcastForm.tsx:125
MEDIUM js-unescaped-html-sink app/(app)/dashboard/projects/[id]/autoblog/articles/[articleId]/page.tsx:214
MEDIUM js-unescaped-html-sink app/(marketing)/blog/[slug]/page.tsx:67
MEDIUM js-unescaped-html-sink app/(marketing)/blog/[slug]/page.tsx:97
MEDIUM js-unescaped-html-sink app/(marketing)/blog/[slug]/page.tsx:104
MEDIUM js-unescaped-html-sink app/(marketing)/blog/[slug]/page.tsx:110
MEDIUM js-unescaped-html-sink app/(marketing)/recent/page.tsx:186
MEDIUM js-unescaped-html-sink app/(marketing)/recent/page.tsx:190
MEDIUM js-unescaped-html-sink app/c/[project]/[slug]/page.tsx:77
MEDIUM js-unescaped-html-sink app/c/[project]/page.tsx:57
MEDIUM js-unescaped-html-sink app/careers.js/route.ts:228
MEDIUM js-unescaped-html-sink app/careers.js/route.ts:285
MEDIUM js-unescaped-html-sink app/layout.tsx:129
MEDIUM js-open-redirect app/login/form.tsx:39
MEDIUM js-unescaped-html-sink app/r/[token]/page.tsx:176
MEDIUM js-open-redirect app/signup/form.tsx:43
MEDIUM js-open-redirect components/billing/buy-credits-modal.tsx:98
MEDIUM js-unescaped-html-sink components/json-ld.tsx:8
MEDIUM js-unescaped-html-sink components/report/markdown-view.tsx:15
MEDIUM js-unescaped-html-sink lib/careers/page-templates.ts:198
MEDIUM redos-nested-quantifier lib/emailMarkdown.ts:41
MEDIUM redos-nested-quantifier lib/emailMarkdown.ts:324
MEDIUM redos-nested-quantifier lib/lx/articleGen.ts:98
MEDIUM redos-nested-quantifier lib/tracker/agent-gate.ts:61
MEDIUM sh-remote-script-execution prober/deploy/provision.sh:30
MEDIUM sql-template-interpolation scripts/detect-slot-themes.ts:31
MEDIUM sql-template-interpolation scripts/purge-constructed-keywords.ts:163
MEDIUM sql-template-interpolation scripts/purge-offniche-keywords.ts:124
LOW secret-generic-credential app/(marketing)/docs/autoblog-webhook/page.tsx:145
LOW secret-generic-credential lib/sp/platforms/linkedin.ts:25
LOW js-dynamic-code-execution tests/careers-page-templates.test.ts:21
LOW js-dynamic-code-execution tests/careers-widget-script.test.ts:19
LOW js-dynamic-code-execution tests/careers-widget-script.test.ts:69
LOW js-dynamic-code-execution tests/contract/ad-visitor-id.test.ts:51
LOW js-dynamic-code-execution tests/contract/ad-visitor-id.test.ts:52
LOW secret-generic-credential tests/contract/posthog-integration.test.ts:13
LOW secret-generic-credential tests/lead-campaign.test.ts:16

Snippets are redacted; ThreatCrush never prints matched credential material.

@ralyodio
ralyodio merged commit 4386845 into master Sep 1, 2026
10 checks passed
@ralyodio
ralyodio deleted the worktree-ads-free-tier-reporting branch September 1, 2026 14:54
ralyodio added a commit that referenced this pull request Sep 1, 2026
)

/dashboard/ads read 0 for everything, intermittently, while the account
was delivering 176,264 impressions over the window. The measure was
right this time -- #199 and #225 both hold -- and the data was there.
The RPCs were being cancelled.

ad_account_series, ad_campaign_totals and the two daily-series functions
are security invoker, so the RLS policy on ad_impressions ("slot is mine
OR campaign is mine") joins the plan. With it the planner abandons the
hash join for a nested loop: one index scan per owned campaign, 139
loops, ~176k random heap fetches, 401,791 buffers (~3GB) touched per
page load. ad_impressions passed 364k rows / 154MB and traffic ran 10x
baseline on 2026-09-01, which tipped it over the 8s statement_timeout on
`authenticated` -- 34 cancellations in two hours, surfacing as HTTP 500
on three RPCs.

Each function already did its own authorisation and never relied on RLS
for it: every read is gated by `<x>_id in (select id from owned)` where
owned is `owner_id = auth.uid()`. Running them as definer drops the RLS
subplans and the planner picks the hash join again: 11,818 buffers /
208ms against 401,791 / 932ms, byte-identical output. Verified with a
stranger's JWT that all five still return 0 rows. Note the guard is `in`
and not `not in`, so an anon caller gets an empty `owned` rather than
everything.

The second half is why this took a log dive to find. Every loader
swallowed the error into a zero-filled result, so a cancelled query and
a genuinely quiet range produced identical output and the page reported
four confident zeros over a live network. The zero-fill stays -- one bad
panel should not take the page down -- but the loaders now return
Loaded<T> carrying `failed`, log the error instead of discarding it, and
the four ad surfaces render "couldn't load" in place of the zeros. The
PDF report says so too: that document goes to accountants, where a
silent zero is read as fact.

Migration is already applied to prod.


Claude-Session: https://claude.ai/code/session_01318XDMF7H8AtH7h4ZjweTS

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant