test: strengthen SDK regression coverage

[marandaneto]

💡 Motivation and Context

Several tests could pass without checking the behavior they named. Some assertions were unreachable after an expected exception. Other tests used missing flag keys instead of the intended fixtures, accepted both null and false, or only checked that an event was queued. Older consumer tests also contacted live PostHog endpoints.

This audit replaces live requests with a loopback server and verifies the actual requests from all three network consumers. The gzip cases caught a production bug in fork-curl, which was fixed separately in #250 and is now on main. This branch has merged main and removed the duplicate change intent. There are no remaining production code or changeset differences.

The remaining changes strengthen flag results, local-only routing, group rollout bucketing, cache round trips, timestamp serialization, exception capture and context isolation. Shared test helpers now reject unexpected HTTP routes and restore nested clocks. Python adapter tests use ephemeral ports and restore their environment. CI now runs the adapter fidelity tests alongside PHPUnit.

The restored invalid-date tests retain PHP's existing exception behavior. That behavior differs from the current local-evaluator spec, which calls for no-match on malformed property dates and supports integral Unix seconds. Aligning those semantics remains a separate compatibility change.

💚 How did you test it?

The original audit comparison below was measured before merging #250, using PHP 8.5.10, PHPUnit 11.5.55 and Xdebug 3.5.0 on macOS. Both runs used the same source filter and blocked external networking while allowing loopback.

Metric	Before	After
PHP test cases	696	729
Assertions	4,638	5,379
Line coverage	86.64% (2,360/2,724)	90.35% (2,461/2,724)
Method coverage	71.65% (187/261)	73.95% (193/261)

The baseline had one failure and one error under network isolation. The repaired suite passes all 729 PHP cases and 16 Python cases. No previously covered production lines were lost, and no coverage exclusions were added.

• The full PHP suite also passed in randomized order with warnings and risky tests treated as failures.
• Six controlled defects were detected, including the original gzip bug, wrong group bucketing, empty flag results, payload-access side effects, zero-valued Retry-After dates and broken nested-clock restoration.
• Composer validation, the public API snapshot, workflow lint and whitespace checks passed.
• PHP_CodeSniffer reported zero errors and the same 55 existing warnings. Existing PHP and PHPUnit deprecation notices remain.
• After merging main at 5451f4e, all 731 PHP cases (5,389 assertions) and 16 Python cases pass. The randomized PHP run also passes with warnings and risky tests treated as failures. The two additional PHP cases came from fix: preserve escaped characters in fork-curl gzip payloads #250.
• Composer validation, the public API snapshot, actionlint and whitespace checks passed after the merge.
• Autoreview passed at 7ce6cdf8f77e3807c531d959f8de1fad6bae4f3b with no actionable findings.
• Local coverage was not remeasured after the merge because the temporary Xdebug module is no longer available; the table above retains the original measurements.

Linux, PHP 8.2-8.4 and the external Docker compliance harness were not run locally. Coverage does not include manually launched PHP subprocesses or Python implementation code.

📝 Checklist

• I reviewed the submitted code.
• I added tests to verify the changes.
• I updated the docs if needed.
• No breaking change or entry added to the changelog.

If releasing new changes

Not applicable: test-only changes. The production fix and its patch change intent were merged separately in #250.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Pi performed the audit using file, shell and subagent tools. Two read-only subagents reviewed separate flag suites. Pi applied the fixes and ran the tests, coverage comparison and mutation checks. The isolated autoreview helper reviewed the committed branch. No session was published.

The audit kept the existing regression vectors and test layers. It added a loopback transport fixture instead of mocking the code that sends requests. The detailed audit ledger and logs remain local rather than adding machine-specific files to the repository. Human review is required.

[marandaneto]

Test coverage comparison

Baseline: c2c9dd294477b10bb513a93101ef3e394e076489
PR head: b3b2860dd5226b90cf2ee9bd8f8327ad10e46df4

Metric	Before	After	Change
Line coverage	86.64% (2,360/2,724)	90.35% (2,461/2,724)	+3.71 percentage points, +101 lines
Method coverage	71.65% (187/261)	73.95% (193/261)	+2.30 percentage points, +6 methods
Fully covered classes	40.00% (10/25)	40.00% (10/25)	No change
PHP test cases	696	729	+33
Assertions	4,638	5,379	+741
PHP failures / errors	1 / 1	0 / 0	Both resolved
Python tests	16 passing	16 passing	No change

Both measurements used PHP 8.5.10, PHPUnit 11.5.55 and Xdebug 3.5.0 on macOS, with the same Composer lockfile and unchanged coverage filter covering all PHP files under lib, including uncovered files. External networking was blocked and loopback was allowed.

The baseline was not green under network isolation. Its file-sender test depended on live delivery, and its socket production-error test installed a throwing error callback while expecting no exception. The repaired tests exercise local delivery and failure behavior without live PostHog requests.

No previously covered production lines were lost, and no coverage exclusions were added. All 729 PHP tests also passed in randomized order. Six targeted mutations were detected by the strengthened assertions.

These figures measure SDK line/method coverage, not branch coverage or an exhaustive mutation score. They exclude manually launched PHP subprocesses and Python implementation coverage. Existing PHP/PHPUnit deprecation notices remain. Linux, PHP 8.2-8.4 and the external Docker compliance harness were not run locally.

[greptile-apps]

[Medium risk] Refactors transport tests and fixes gzip payload handling.

The PR appears safe to merge; no actionable regression was identified.

_{Reviews (1) · Last reviewed commit: "fix: strengthen SDK tests and preserve f..."}

[github-actions]

posthog-php-fork_curl Compliance Report

Date: 2026-09-28T13:08:07.543939+00:00
Duration: 111812ms

⚠️ Some Tests Failed

36/47 tests passed, 11 failed

---

Capture Tests

⚠️ 19/30 tests passed, 11 failed

View Details

Test	Status	Duration
Format Validation.Event Has Required Fields	✅	25ms
Format Validation.Event Has Uuid	✅	520ms
Format Validation.Event Has Lib Properties	✅	521ms
Format Validation.Distinct Id Is String	✅	521ms
Format Validation.Token Is Present	✅	521ms
Format Validation.Custom Properties Preserved	✅	523ms
Format Validation.Event Has Timestamp	✅	522ms
Format Validation.Non Utc Event Timestamp Is Converted To Utc	✅	521ms
Retry Behavior.Retries On 503	❌	5526ms
Retry Behavior.Does Not Retry On 400	✅	2524ms
Retry Behavior.Does Not Retry On 401	✅	2523ms
Retry Behavior.Respects Retry After Header	❌	5526ms
Retry Behavior.Implements Backoff	❌	15528ms
Retry Behavior.Retries On 500	❌	5037ms
Retry Behavior.Retries On 502	❌	5526ms
Retry Behavior.Retries On 504	❌	5526ms
Retry Behavior.Max Retries Respected	❌	15537ms
Deduplication.Generates Unique Uuids	✅	526ms
Deduplication.Preserves Uuid On Retry	❌	5524ms
Deduplication.Preserves Uuid And Timestamp On Retry	❌	10530ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry	❌	5528ms
Deduplication.No Duplicate Events In Batch	✅	527ms
Deduplication.Different Events Have Different Uuids	✅	522ms
Compression.Sends Gzip When Enabled	✅	523ms
Batch Format.Uses Proper Batch Structure	✅	519ms
Batch Format.Flush With No Events Sends Nothing	✅	510ms
Batch Format.Multiple Events Batched Together	✅	515ms
Error Handling.Does Not Retry On 403	✅	2522ms
Error Handling.Does Not Retry On 413	✅	2521ms
Error Handling.Retries On 408	❌	5528ms

Failures

retry_behavior.retries_on_503

Expected at least 3 requests, got 1

retry_behavior.respects_retry_after_header

Expected at least 2 requests, got 1

retry_behavior.implements_backoff

Expected at least 3 requests, got 1

retry_behavior.retries_on_500

Expected at least 2 requests, got 1

retry_behavior.retries_on_502

Expected at least 2 requests, got 1

retry_behavior.retries_on_504

Expected at least 2 requests, got 1

retry_behavior.max_retries_respected

Expected 4 requests, got 1

deduplication.preserves_uuid_on_retry

Need at least 2 requests to check retry

deduplication.preserves_uuid_and_timestamp_on_retry

Expected at least 3 requests, got 1

deduplication.preserves_uuid_and_timestamp_on_batch_retry

Expected at least 2 requests, got 1

error_handling.retries_on_408

Expected at least 2 requests, got 1

Feature_Flags Tests

✅ 17/17 tests passed

View Details

Test	Status	Duration
Request Payload.Request With Person Properties Device Id	✅	515ms
Request Payload.Flags Request Uses V2 Query Param	✅	513ms
Request Payload.Flags Request Hits Flags Path Not Decide	✅	513ms
Request Payload.Flags Request Omits Authorization Header	✅	513ms
Request Payload.Token In Flags Body Matches Init	✅	513ms
Request Payload.Groups Round Trip	✅	514ms
Request Payload.Groups Default To Empty Object	✅	513ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False	✅	512ms
Request Payload.Disable Geoip Omitted Defaults To False	✅	513ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key	✅	514ms
Request Lifecycle.No Flags Request On Init Alone	✅	511ms
Request Lifecycle.No Flags Request On Normal Capture	✅	510ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests	✅	516ms
Request Lifecycle.Mock Response Value Is Returned To Caller	✅	514ms
Retry Behavior.Retries Flags On 502	✅	614ms
Retry Behavior.Retries Flags On 504	✅	614ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event	✅	522ms

[github-actions]

posthog-php-lib_curl Compliance Report

Date: 2026-09-28T13:08:19.635942+00:00
Duration: 118650ms

✅ All Tests Passed!

47/47 tests passed

---

Capture Tests

✅ 30/30 tests passed

View Details

Test	Status	Duration
Format Validation.Event Has Required Fields	✅	24ms
Format Validation.Event Has Uuid	✅	518ms
Format Validation.Event Has Lib Properties	✅	521ms
Format Validation.Distinct Id Is String	✅	521ms
Format Validation.Token Is Present	✅	520ms
Format Validation.Custom Properties Preserved	✅	521ms
Format Validation.Event Has Timestamp	✅	521ms
Format Validation.Non Utc Event Timestamp Is Converted To Utc	✅	521ms
Retry Behavior.Retries On 503	✅	5830ms
Retry Behavior.Does Not Retry On 400	✅	2523ms
Retry Behavior.Does Not Retry On 401	✅	2522ms
Retry Behavior.Respects Retry After Header	✅	8531ms
Retry Behavior.Implements Backoff	✅	16244ms
Retry Behavior.Retries On 500	✅	5632ms
Retry Behavior.Retries On 502	✅	5629ms
Retry Behavior.Retries On 504	✅	5630ms
Retry Behavior.Max Retries Respected	✅	17045ms
Deduplication.Generates Unique Uuids	✅	228ms
Deduplication.Preserves Uuid On Retry	✅	5629ms
Deduplication.Preserves Uuid And Timestamp On Retry	✅	10837ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry	✅	5635ms
Deduplication.No Duplicate Events In Batch	✅	527ms
Deduplication.Different Events Have Different Uuids	✅	523ms
Compression.Sends Gzip When Enabled	✅	523ms
Batch Format.Uses Proper Batch Structure	✅	522ms
Batch Format.Flush With No Events Sends Nothing	✅	518ms
Batch Format.Multiple Events Batched Together	✅	511ms
Error Handling.Does Not Retry On 403	✅	2524ms
Error Handling.Does Not Retry On 413	✅	2526ms
Error Handling.Retries On 408	✅	5630ms

Feature_Flags Tests

✅ 17/17 tests passed

View Details

Test	Status	Duration
Request Payload.Request With Person Properties Device Id	✅	521ms
Request Payload.Flags Request Uses V2 Query Param	✅	521ms
Request Payload.Flags Request Hits Flags Path Not Decide	✅	520ms
Request Payload.Flags Request Omits Authorization Header	✅	520ms
Request Payload.Token In Flags Body Matches Init	✅	520ms
Request Payload.Groups Round Trip	✅	520ms
Request Payload.Groups Default To Empty Object	✅	520ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False	✅	520ms
Request Payload.Disable Geoip Omitted Defaults To False	✅	520ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key	✅	520ms
Request Lifecycle.No Flags Request On Init Alone	✅	515ms
Request Lifecycle.No Flags Request On Normal Capture	✅	509ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests	✅	524ms
Request Lifecycle.Mock Response Value Is Returned To Caller	✅	521ms
Retry Behavior.Retries Flags On 502	✅	623ms
Retry Behavior.Retries Flags On 504	✅	622ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event	✅	523ms

[github-actions]

posthog-php-socket Compliance Report

Date: 2026-09-28T13:08:45.825213+00:00
Duration: 148825ms

⚠️ Some Tests Failed

36/47 tests passed, 11 failed

---

Capture Tests

⚠️ 19/30 tests passed, 11 failed

View Details

Test	Status	Duration
Format Validation.Event Has Required Fields	✅	22ms
Format Validation.Event Has Uuid	✅	517ms
Format Validation.Event Has Lib Properties	✅	519ms
Format Validation.Distinct Id Is String	✅	518ms
Format Validation.Token Is Present	✅	519ms
Format Validation.Custom Properties Preserved	✅	518ms
Format Validation.Event Has Timestamp	✅	518ms
Format Validation.Non Utc Event Timestamp Is Converted To Utc	✅	520ms
Retry Behavior.Retries On 503	❌	9227ms
Retry Behavior.Does Not Retry On 400	✅	2523ms
Retry Behavior.Does Not Retry On 401	✅	2519ms
Retry Behavior.Respects Retry After Header	❌	9227ms
Retry Behavior.Implements Backoff	❌	19238ms
Retry Behavior.Retries On 500	❌	9224ms
Retry Behavior.Retries On 502	❌	8731ms
Retry Behavior.Retries On 504	❌	9226ms
Retry Behavior.Max Retries Respected	❌	19236ms
Deduplication.Generates Unique Uuids	✅	523ms
Deduplication.Preserves Uuid On Retry	❌	9224ms
Deduplication.Preserves Uuid And Timestamp On Retry	❌	14232ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry	❌	9231ms
Deduplication.No Duplicate Events In Batch	✅	524ms
Deduplication.Different Events Have Different Uuids	✅	518ms
Compression.Sends Gzip When Enabled	✅	518ms
Batch Format.Uses Proper Batch Structure	✅	518ms
Batch Format.Flush With No Events Sends Nothing	✅	514ms
Batch Format.Multiple Events Batched Together	✅	510ms
Error Handling.Does Not Retry On 403	✅	2519ms
Error Handling.Does Not Retry On 413	✅	2521ms
Error Handling.Retries On 408	❌	5524ms

Failures

retry_behavior.retries_on_503

Expected at least 3 requests, got 1

retry_behavior.respects_retry_after_header

Expected at least 2 requests, got 1

retry_behavior.implements_backoff

Expected at least 3 requests, got 1

retry_behavior.retries_on_500

Expected at least 2 requests, got 1

retry_behavior.retries_on_502

Expected at least 2 requests, got 1

retry_behavior.retries_on_504

Expected at least 2 requests, got 1

retry_behavior.max_retries_respected

Expected 4 requests, got 1

deduplication.preserves_uuid_on_retry

Need at least 2 requests to check retry

deduplication.preserves_uuid_and_timestamp_on_retry

Expected at least 3 requests, got 1

deduplication.preserves_uuid_and_timestamp_on_batch_retry

Expected at least 2 requests, got 1

error_handling.retries_on_408

Expected at least 2 requests, got 1

Feature_Flags Tests

✅ 17/17 tests passed

View Details

Test	Status	Duration
Request Payload.Request With Person Properties Device Id	✅	518ms
Request Payload.Flags Request Uses V2 Query Param	✅	516ms
Request Payload.Flags Request Hits Flags Path Not Decide	✅	516ms
Request Payload.Flags Request Omits Authorization Header	✅	525ms
Request Payload.Token In Flags Body Matches Init	✅	516ms
Request Payload.Groups Round Trip	✅	515ms
Request Payload.Groups Default To Empty Object	✅	514ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False	✅	514ms
Request Payload.Disable Geoip Omitted Defaults To False	✅	515ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key	✅	514ms
Request Lifecycle.No Flags Request On Init Alone	✅	511ms
Request Lifecycle.No Flags Request On Normal Capture	✅	506ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests	✅	520ms
Request Lifecycle.Mock Response Value Is Returned To Caller	✅	517ms
Retry Behavior.Retries Flags On 502	✅	619ms
Retry Behavior.Retries Flags On 504	✅	616ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event	✅	519ms