Skip to content

test(e2e): cover platform binary patch updates - #156

Draft
floyd-soomgo wants to merge 1 commit into
feature/binary-patch-ios-applyfrom
feature/binary-patch-e2e
Draft

test(e2e): cover platform binary patch updates#156
floyd-soomgo wants to merge 1 commit into
feature/binary-patch-ios-applyfrom
feature/binary-patch-e2e

Conversation

@floyd-soomgo

Copy link
Copy Markdown
Member

Summary

Sixth PR of the binary differential OTA series (stacked on feat(ios): apply binary patch updates — merge that first).

The appliers landed with unit-level coverage (Android) and compile gates (iOS); this PR is the first behavioral end-to-end gate: real patch installs and failure-driven fallbacks on a physical Android device and an iOS simulator, built as a new phase of the existing Maestro E2E harness. No library code changes — the diff is e2e/ only.

The central verification problem: a patch install and a fallback-then-full install produce identical UI and identical packageHash, so the outcome cannot be told apart from the screen. The harness now answers it with a download-order log on the mock server: every scenario asserts the exact sequence of archive fetches — [patch] (patch applied), [patch, full] (fell back exactly once), [full] (legacy path). The assertion is order- and length-sensitive and rejects an empty log, so no scenario can pass vacuously.

The embedded base bundle is extracted from the installed artifact itself (APK assets via adb shell pm path + unzip; main.jsbundle from the simulator's app container) — byte-identical with the binary by construction, which is the invariant the whole feature rests on.

Scenarios

Scenario Expected downloads
Patch installs on top of the app binary [patch]
Patch carrying assets installs (archive asserted to contain assets) [patch]
Patch built against a stale base bundle [patch, full]
Corrupt patch body [patch, full]
Intact patch, manifest describing a different bundle [patch, full]
Corrupt patch header [patch, full]
Patch archive of the other platform [patch, full]
History without a patch URL (legacy regression, pinned) [full]
One pre-built bundle, released with and without a patch [patch] / [full]
App stays responsive while a patch installs timing-sensitive flow

Two scenarios deserve their design notes:

  • Corrupt body vs. target-hash defense. On-device measurement showed that randomly corrupting the compressed patch body is usually caught by the zstd decoder itself (patch_apply_failed), not by the final hash. The target-hash defense — the only protection when a patch applies "successfully" but produces wrong bytes — is therefore exercised by its own scenario: an intact patch whose manifest describes a different bundle, which provably fails at exactly target_verification_failed and nowhere earlier.
  • One bundle, patch on one target only. The brief's two-binaryVersion shape would require two app binaries, so the harness uses two identifiers instead: bundle once, release --skip-bundle twice (once with the base, once without). Both histories are asserted at file level — same packageHash, exactly one carries binaryPatchDownloadUrl — then each is served to the same binary in turn and installs via [patch] and [full] respectively. The cross-platform archive is likewise synthesized by rewriting a real archive into the other platform's shape, so an Android run never requires an iOS build (and vice versa).

Changes

  • e2e/mock-server/server.ts + e2e/helpers/download-order.ts — request logging on the archive routes and the order-sensitive assertion helpers (reset inside the retry closure, so a retried scenario re-baselines instead of accumulating).
  • e2e/helpers/binary-patch-fixtures.ts — base-bundle extraction from the installed artifact, patch-release helpers, and archive mutators (body/header corruption, manifest retarget, cross-platform rewrite) that mirror the real CLI archive layout field-for-field.
  • e2e/helpers/binary-patch-phase.ts + e2e/run.ts — the new phase, wired through the existing retry/cleanup/config conventions; storage layout (bundles/{platform}/{identifier}, histories/{platform}/{identifier}/{binaryVersion}.json) pinned by assertion.
  • e2e/flows-binary-patch/*.yaml — Maestro flows asserting real UI outcomes (updated text + metadata label, not just app-didn't-crash); the responsiveness flow is gated behind the existing --exclude-timing-sensitive convention.
  • e2e/helpers/artifact-storage.ts, config template, READMEs (EN/KO) updated alongside.

Test plan

Local device runs (E2E does not run in CI):

npm run e2e -- --app RN0840 --platform android   # physical device — 34 assertions, passed
npm run e2e -- --app RN0840 --platform ios       # simulator — 34 assertions, passed
# variant: --exclude-timing-sensitive green on both platforms (responsiveness scenario skipped)
npm run typecheck

Scope note: E2E ran against the RN0840 example app on both platforms. Other example apps (older RN, Expo) share the same harness via --app but were not exercised in this PR.

A release published with a binary patch offers two archives of the same
update, and a client that cannot use the patch has to end up with exactly
the update it would have downloaded in full. Both artifacts install the
same contents, so the screen cannot tell a patch install apart from a
fallback - which is why nothing covered the patch path end to end until
now.

The mock server records every request it answers, and the runner reads
that log back as the update archives the app downloaded, in order: one
patch for a patch install, a patch followed by the full archive for a
fallback, and the full archive alone for a release published without a
patch. The local CLI config records where it stored each artifact, so the
storage layout is asserted from what the CLI asked for rather than from
paths the test re-derives.

On top of that, a phase installs a patch update on the device and works
through every way a patch can fail: a stale base bundle, a corrupt patch
body, a corrupt header, a restored bundle the manifest does not describe,
and an archive built for the other platform. Each of them has to fall
back exactly once and leave the app running the update it advertises. The
base bundle a patch is computed against is taken out of the app that is
installed on the device, because a patch applies to those bytes and no
others.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant