diff --git a/.gitignore b/.gitignore index 1dc084d97..2c2272d00 100644 --- a/.gitignore +++ b/.gitignore @@ -170,6 +170,9 @@ bin/* # E2E mock server data e2e/mock-server/data/ +# E2E scratch files (extracted binary bundles, artifact records) +e2e/.work/ + # maestro-runner report directory e2e/reports/ diff --git a/e2e/README.ko.md b/e2e/README.ko.md index 438d38fd4..b02872137 100644 --- a/e2e/README.ko.md +++ b/e2e/README.ko.md @@ -78,6 +78,22 @@ npm run e2e -- --app Expo55Beta --framework expo --platform ios --maestro-only - `03-optional-update-on-resume-after-20s` — 앱이 백그라운드에 20초 이상 머문 뒤 포그라운드로 돌아올 때 `ON_NEXT_RESUME`으로 업데이트가 적용되는지 확인합니다. `--exclude-timing-sensitive`를 주지 않으면 실행됩니다. - `04-optional-update-on-suspend-after-20s` — 앱이 백그라운드에 20초 이상 머무는 동안 `ON_NEXT_SUSPEND`로 업데이트가 적용되고, 다음 포그라운드 진입 시 반영된 번들이 보이는지 확인합니다. `--exclude-timing-sensitive`를 주지 않으면 실행됩니다. +### Phase 6 — 바이너리 패치 업데이트 (`flows-binary-patch/`) + +14. **베이스 번들 추출** — 기기에 설치된 앱에서 JS 번들을 꺼냅니다(Android는 APK의 `assets/`, iOS는 `.app`). 바이너리 패치는 바이너리에 실린 바로 그 바이트에만 적용되므로 다른 것으로 대체할 수 없습니다. +15. **시나리오별 릴리스와 설치** — 각 시나리오는 `--binary-bundle-path`로 릴리스한 뒤 필요한 지점만 고장 내고, `01-install-update`로 업데이트를 설치해 `UPDATED!`와 `METADATA_V`을 확인합니다. + - `1.3.1` — 앱 바이너리 위에 patch 업데이트가 설치됩니다. + - `1.3.2` — 이미지 asset을 포함한 patch 업데이트가 설치됩니다. + - `1.3.3` — 낡은 베이스 번들로 만든 patch는 full 업데이트로 fallback합니다. + - `1.3.4` — 압축 본문이 손상된 patch는 fallback합니다. + - `1.3.9` — manifest가 설명하지 않는 번들을 복원하는 patch는 fallback합니다. + - `1.3.5` — 헤더가 손상된 patch는 fallback합니다. + - `1.3.6` — 다른 플랫폼용으로 만들어진 patch archive는 fallback합니다. + - `1.3.7` — 한 번 만든 번들(`bundle` 1회 + `release --skip-bundle` 2회, 한쪽에만 베이스 번들 전달)이 patch URL이 실린 히스토리에서는 patch로, 실리지 않은 히스토리에서는 full로 설치됩니다. + - `1.3.8` — `02-ui-responsive-during-install`: patch를 내려받아 적용하는 동안에도 앱이 탭에 반응합니다. `--exclude-timing-sensitive`를 주지 않으면 실행됩니다. + +patch 설치와 full archive fallback은 같은 내용을 설치하므로 화면만으로는 구분되지 않습니다. 둘을 가르는 것은 앱이 서버에 요청한 archive의 순서이며, 모든 시나리오가 이를 검증합니다: patch 설치는 `[patch]`, fallback은 `[patch, full]`, patch 없이 배포된 릴리스는 `[full]`입니다. + ## 아키텍처 ``` @@ -86,17 +102,22 @@ e2e/ ├── config.ts # 경로, 포트, 호스트 설정 ├── tsconfig.json ├── mock-server/ -│ └── server.ts # Express 정적 파일 서버 (포트 18081) +│ └── server.ts # Express 정적 파일 서버 (포트 18081), 모든 요청 기록 ├── templates/ │ └── code-push.config.local.ts # 파일시스템 기반 CodePush 설정 ├── helpers/ │ ├── prepare-config.ts # App.tsx 패치(호스트 + 임시 E2E 버튼), 설정 복사 │ ├── prepare-bundle.ts # code-push CLI로 번들 생성 -│ └── build-app.ts # iOS/Android Release 빌드 +│ ├── build-app.ts # iOS/Android Release 빌드 +│ ├── artifact-storage.ts # CLI가 번들과 릴리스 히스토리를 저장한 위치 검증 +│ ├── download-order.ts # 서버 요청 기록을 앱이 내려받은 archive 순서로 변환 +│ ├── binary-patch-fixtures.ts # 베이스 번들 추출과 손상된 patch archive +│ └── binary-patch-phase.ts # 바이너리 패치 시나리오 매트릭스 ├── flows/ # Phase 1: 기본 플로우 ├── flows-rollback/ # Phase 2: 바이너리로 롤백 ├── flows-partial-rollback/ # Phase 3: 부분 롤백 (v1.0.2 → v1.0.1) ├── flows-optional/ # Phase 4: optional 설치 모드 검증 +├── flows-binary-patch/ # Phase 6: 바이너리 패치 설치와 fallback └── scripts/ └── sleep.js # Maestro runScript 대기 헬퍼 ``` @@ -107,7 +128,11 @@ e2e/ - **번들**: `mock-server/data/bundles/{platform}/{identifier}/` - **릴리스 히스토리**: `mock-server/data/histories/{platform}/{identifier}/{version}.json` -`code-push.config.local.ts` 템플릿은 모든 CLI 작업(업로드, 히스토리 읽기/쓰기)을 로컬 파일시스템으로 라우팅하며, 앱의 `CODEPUSH_HOST`는 mock 서버를 가리키도록 패치됩니다. +`code-push.config.local.ts` 템플릿은 모든 CLI 작업(업로드, 히스토리 읽기/쓰기)을 로컬 파일시스템으로 라우팅하며, 앱의 `CODEPUSH_HOST`는 mock 서버를 가리키도록 패치됩니다. 바이너리 패치와 함께 배포된 릴리스는 full archive 옆에 `{packageHash}-patch.zip` 이름으로 두 번째 archive를 저장합니다. + +서버는 응답한 모든 요청을 기록합니다. 러너는 이 기록을 되읽어, 화면으로는 구분할 수 없는 것 — 앱이 어떤 업데이트 archive를 어떤 순서로 내려받았는지 — 를 검증합니다. + +`E2E_ARTIFACT_LOG_PATH`가 주어지면 템플릿은 저장한 모든 artifact도 (서빙 디렉터리 바깥에) 기록하며, 러너는 이를 되읽어 번들과 릴리스 히스토리가 계속 `{platform}/{identifier}` 아래에 저장되는지 검증합니다. ### 릴리스 마커 @@ -117,5 +142,5 @@ e2e/ - **iOS 빌드 시 서명 오류**: setup 스크립트가 `SUPPORTED_PLATFORMS = iphonesimulator`를 설정하고 코드 서명을 비활성화합니다. `scripts/setupExampleApp`으로 예제 앱이 설정되었는지 확인하세요. - **Maestro/maestro-runner가 앱을 찾지 못함**: 실행 전에 시뮬레이터/에뮬레이터가 부팅되어 있는지 확인하세요. iOS의 경우 스크립트가 부팅된 시뮬레이터를 자동 감지합니다. -- **Android 네트워크 오류**: Android 에뮬레이터는 호스트 머신의 localhost에 접근하기 위해 `10.0.2.2`를 사용합니다. 설정에서 자동으로 처리됩니다. +- **Android 네트워크 오류**: Android 에뮬레이터는 호스트 머신의 localhost에 접근하기 위해 `10.0.2.2`를 사용합니다. 설정에서 자동으로 처리됩니다. adb로 연결한 실기기에는 이 별칭이 없으므로, 러너가 mock 서버 포트를 기기로 포워딩(`adb reverse`)하고 앱이 기기 자신의 localhost를 보도록 합니다. 두 기본값 모두 `E2E_ANDROID_MOCK_SERVER_HOST`로 덮어쓸 수 있습니다. - **업데이트가 적용되지 않음**: Mock 서버가 실행 중인지(포트 18081), `mock-server/data/`에 예상되는 번들과 히스토리 파일이 있는지 확인하세요. diff --git a/e2e/README.md b/e2e/README.md index 066c47f21..6f86dcce9 100644 --- a/e2e/README.md +++ b/e2e/README.md @@ -78,6 +78,22 @@ The test runner (`e2e/run.ts`) executes these phases in order: - `03-optional-update-on-resume-after-20s` — Verifies `ON_NEXT_RESUME` applies the update when the app returns to foreground after staying in background for at least 20 seconds. Runs unless `--exclude-timing-sensitive` is passed. - `04-optional-update-on-suspend-after-20s` — Verifies `ON_NEXT_SUSPEND` applies the update while the app stays in background for at least 20 seconds, so the updated bundle is visible on the next foreground. Runs unless `--exclude-timing-sensitive` is passed. +### Phase 6 — Binary Patch Updates (`flows-binary-patch/`) + +14. **Extract the base bundle** — Copies the JS bundle out of the app that is installed on the device (the APK's `assets/` on Android, the `.app` on iOS). A binary patch only applies to the exact bytes that shipped in the binary, so nothing else can stand in for it. +15. **Publish and install one release per scenario** — Each scenario releases with `--binary-bundle-path`, breaks the published patch where it wants it broken, and installs the update with `01-install-update`, which verifies the app shows `UPDATED!` and `METADATA_V`: + - `1.3.1` — Patch update installs on top of the app binary. + - `1.3.2` — Patch update carrying an image asset installs. + - `1.3.3` — Patch computed against a stale base bundle falls back to the full update. + - `1.3.4` — Patch whose compressed body is corrupt falls back. + - `1.3.9` — Patch that restores a bundle its manifest does not describe falls back. + - `1.3.5` — Patch whose header is corrupt falls back. + - `1.3.6` — Patch archive built for the other platform falls back. + - `1.3.7` — One pre-built bundle (`bundle` once, `release --skip-bundle` twice, with the base bundle passed to only one of the two) installs as a patch from the history that carries a patch URL, and in full from the history that does not. + - `1.3.8` — `02-ui-responsive-during-install`: the app answers taps while the patch is being downloaded and applied. Runs unless `--exclude-timing-sensitive` is passed. + +A patch install and a fallback to the full archive install the same contents, so they look identical on screen. What tells them apart is which archives the app asked the mock server for, which every scenario asserts: `[patch]` for a patch install, `[patch, full]` for a fallback, `[full]` for a release published without a patch. + ## Architecture ``` @@ -86,17 +102,22 @@ e2e/ ├── config.ts # Paths, ports, host configuration ├── tsconfig.json ├── mock-server/ -│ └── server.ts # Express static file server (port 18081) +│ └── server.ts # Express static file server (port 18081), records every request ├── templates/ │ └── code-push.config.local.ts # Filesystem-based CodePush config ├── helpers/ │ ├── prepare-config.ts # Patches App.tsx (host + temporary E2E buttons), copies config │ ├── prepare-bundle.ts # Runs code-push CLI to create bundles -│ └── build-app.ts # Builds iOS/Android in Release mode +│ ├── build-app.ts # Builds iOS/Android in Release mode +│ ├── artifact-storage.ts # Asserts where the CLI stored bundles and release histories +│ ├── download-order.ts # Turns the server's request log into the archives the app downloaded +│ ├── binary-patch-fixtures.ts # Base bundle extraction and broken patch archives +│ └── binary-patch-phase.ts # Binary patch scenario matrix ├── flows/ # Phase 1: basic flows ├── flows-rollback/ # Phase 2: rollback to binary ├── flows-partial-rollback/ # Phase 3: partial rollback (v1.0.2 → v1.0.1) ├── flows-optional/ # Phase 4: optional install mode verification +├── flows-binary-patch/ # Phase 6: binary patch install and fallback └── scripts/ └── sleep.js # Maestro runScript helper for deterministic waits ``` @@ -107,7 +128,11 @@ Instead of a real CodePush server, tests use a local Express server that serves: - **Bundles**: `mock-server/data/bundles/{platform}/{identifier}/` - **Release history**: `mock-server/data/histories/{platform}/{identifier}/{version}.json` -The `code-push.config.local.ts` template routes all CLI operations (upload, history read/write) to this local filesystem, and the app's `CODEPUSH_HOST` is patched to point at the mock server. +The `code-push.config.local.ts` template routes all CLI operations (upload, history read/write) to this local filesystem, and the app's `CODEPUSH_HOST` is patched to point at the mock server. A release published with a binary patch stores a second archive next to the full one, named `{packageHash}-patch.zip`. + +The server records every request it answers. Reading that log back is how the runner tells apart cases the screen cannot: which update archives the app downloaded, and in which order. + +When the config template is given `E2E_ARTIFACT_LOG_PATH`, it also records every artifact it stores (outside the served directory), which the runner reads back to assert that bundles and release histories keep landing under `{platform}/{identifier}`. ### Release Markers @@ -117,5 +142,5 @@ When creating multiple releases with identical source code (e.g. v1.0.1 and v1.0 - **Build fails with signing error (iOS)**: The setup script sets `SUPPORTED_PLATFORMS = iphonesimulator` and disables code signing. Make sure the example app was set up with `scripts/setupExampleApp`. - **Maestro/maestro-runner can't find the app**: Ensure the simulator/emulator is booted before running. For iOS, the script auto-detects the booted simulator. -- **Android network error**: Android emulators use `10.0.2.2` to reach the host machine's localhost. This is handled automatically by the config. +- **Android network error**: Android emulators use `10.0.2.2` to reach the host machine's localhost. This is handled automatically by the config. A phone connected over adb has no such alias, so the runner forwards the mock server port onto the device (`adb reverse`) and points the app at its own localhost instead. Set `E2E_ANDROID_MOCK_SERVER_HOST` to override either default. - **Update not applying**: Check that the mock server is running (port 18081) and that `mock-server/data/` contains the expected bundle and history files. diff --git a/e2e/config.ts b/e2e/config.ts index d367f0f61..f7fd9c7ff 100644 --- a/e2e/config.ts +++ b/e2e/config.ts @@ -4,6 +4,19 @@ export const MOCK_SERVER_PORT = 18081; export const EXAMPLES_DIR = path.resolve(__dirname, "../Examples"); export const MOCK_DATA_DIR = path.resolve(__dirname, "mock-server/data"); +/** + * Scratch directory for files a run builds but never serves, such as the JS bundle + * extracted from the installed app binary. + */ +export const WORK_DIR = path.resolve(__dirname, ".work"); + +/** + * Where the local CLI config records every artifact it stores. Kept outside the served + * data directory so the record is not itself downloadable, and so wiping the mock data + * between scenarios does not decide when the record is cleared. + */ +export const ARTIFACT_LOG_PATH = path.join(WORK_DIR, "artifact-log.jsonl"); + export function getMockServerHost(platform: "ios" | "android"): string { const host = platform === "android" ? process.env.E2E_ANDROID_MOCK_SERVER_HOST ?? "10.0.2.2" diff --git a/e2e/flows-binary-patch/01-install-update.yaml b/e2e/flows-binary-patch/01-install-update.yaml new file mode 100644 index 000000000..078f9eebd --- /dev/null +++ b/e2e/flows-binary-patch/01-install-update.yaml @@ -0,0 +1,49 @@ +appId: ${APP_ID} +--- +- launchApp +- runFlow: + when: + platform: Android + file: ../flows/shared/android-dismiss-overlays.yaml +- runFlow: + when: + platform: iOS + file: ../flows/shared/ios-dismiss-overlays.yaml +- assertVisible: "React Native.*" + +# Start from the app binary, so the update is downloaded and installed from scratch. +# A binary patch is computed against the bundle inside the binary, so this is also the +# only state in which it can be applied. +- tapOn: "Clear updates" +- tapOn: "Restart app" +- waitForAnimationToEnd: + timeout: 15000 +- tapOn: + text: "(?i)^wait$" + optional: true +- assertVisible: "React Native.*" +- assertNotVisible: "UPDATED!" + +# Mandatory update: downloaded, installed and applied without asking +- tapOn: "Check for updates" +- waitForAnimationToEnd: + timeout: 30000 +# An update that cannot be installed from its patch is downloaded again in full, which +# takes a second pass over the network before the app restarts. +- runScript: + file: ../scripts/sleep.js + env: + WAIT_MS: "5000" + +- launchApp: + stopApp: false +- tapOn: + text: "(?i)^wait$" + optional: true +- assertVisible: "React Native.*" +- assertVisible: "UPDATED!" + +- tapOn: "Get update metadata" +- waitForAnimationToEnd: + timeout: 10000 +- assertVisible: "METADATA_V${RELEASE_LABEL}" diff --git a/e2e/flows-binary-patch/02-ui-responsive-during-install.yaml b/e2e/flows-binary-patch/02-ui-responsive-during-install.yaml new file mode 100644 index 000000000..918ee37e9 --- /dev/null +++ b/e2e/flows-binary-patch/02-ui-responsive-during-install.yaml @@ -0,0 +1,57 @@ +appId: ${APP_ID} +--- +# Applying a binary patch reads a whole JS bundle, rebuilds it and hashes it twice. None +# of that may happen where it can hold up the app: the screen has to keep answering taps +# while the update is being downloaded and applied. +# +# This scenario is timing sensitive - it only means something while that work is still in +# flight - so it is skipped when timing-sensitive scenarios are excluded. +- launchApp +- runFlow: + when: + platform: Android + file: ../flows/shared/android-dismiss-overlays.yaml +- runFlow: + when: + platform: iOS + file: ../flows/shared/ios-dismiss-overlays.yaml +- assertVisible: "React Native.*" + +- tapOn: "Clear updates" +- tapOn: "Restart app" +- waitForAnimationToEnd: + timeout: 15000 +- tapOn: + text: "(?i)^wait$" + optional: true +- assertVisible: "React Native.*" +- assertNotVisible: "UPDATED!" +- assertVisible: "METADATA_IDLE" + +# An optional update is installed without restarting the app, so the app stays on screen +# for the whole download and patch apply. +- tapOn: "Check for updates" + +# No wait in between: this tap has to be answered while that work is in flight, and the +# indicator has to move off its initial state. +- tapOn: "Get update metadata" +- assertVisible: "METADATA_NULL" + +- waitForAnimationToEnd: + timeout: 30000 +- assertVisible: "Result: UPDATE_INSTALLED" +- assertNotVisible: "UPDATED!" + +- tapOn: "Restart app" +- waitForAnimationToEnd: + timeout: 15000 +- tapOn: + text: "(?i)^wait$" + optional: true +- assertVisible: "React Native.*" +- assertVisible: "UPDATED!" + +- tapOn: "Get update metadata" +- waitForAnimationToEnd: + timeout: 10000 +- assertVisible: "METADATA_V${RELEASE_LABEL}" diff --git a/e2e/helpers/artifact-storage.ts b/e2e/helpers/artifact-storage.ts new file mode 100644 index 000000000..2a8355fdf --- /dev/null +++ b/e2e/helpers/artifact-storage.ts @@ -0,0 +1,101 @@ +import fs from "fs"; +import path from "path"; +import { ARTIFACT_LOG_PATH, MOCK_DATA_DIR } from "../config"; + +/** + * What the CLI stored through the local config, read back from the record the config + * template writes. + * + * The point of reading it back is that the storage layout is a contract between the CLI + * and whatever hosts the artifacts: bundles live under `{platform}/{identifier}`, and a + * release history under `{platform}/{identifier}/{binaryVersion}.json`. Publishing a + * binary patch adds a second archive per release, and it has to land in the same place + * as the full one rather than somewhere of its own. + */ +export interface StoredBundleArtifact { + kind: "bundle"; + platform: string; + identifier: string; + fileName: string; + storedPath: string; + downloadUrl: string; +} + +export interface StoredHistoryArtifact { + kind: "history"; + platform: string; + identifier: string; + binaryVersion: string; + storedPath: string; +} + +export type StoredArtifact = StoredBundleArtifact | StoredHistoryArtifact; + +/** Appended to the full archive name so the two artifacts of a release stay paired. */ +export const PATCH_ARCHIVE_SUFFIX = "-patch.zip"; + +export function clearArtifactLog(): void { + fs.rmSync(ARTIFACT_LOG_PATH, { force: true }); +} + +export function readArtifactLog(): StoredArtifact[] { + if (!fs.existsSync(ARTIFACT_LOG_PATH)) { + return []; + } + + return fs.readFileSync(ARTIFACT_LOG_PATH, "utf8") + .split("\n") + .filter((line) => line.trim().length > 0) + .map((line) => JSON.parse(line) as StoredArtifact); +} + +/** + * Asserts that everything released so far was stored where a client looks for it, and + * that a release published with a binary patch stored both of its archives together. + * + * @return the artifacts that were checked, so a caller can go on to assert something + * about a specific one. + */ +export function assertArtifactStorageLayout(scenario: string): StoredArtifact[] { + const artifacts = readArtifactLog(); + if (artifacts.length === 0) { + throw new Error(`${scenario}: no artifacts were stored`); + } + + const bundles = artifacts.filter((artifact): artifact is StoredBundleArtifact => artifact.kind === "bundle"); + + for (const artifact of artifacts) { + const expectedPath = artifact.kind === "bundle" + ? path.join("bundles", artifact.platform, artifact.identifier, artifact.fileName) + : path.join("histories", artifact.platform, artifact.identifier, `${artifact.binaryVersion}.json`); + + if (artifact.storedPath !== expectedPath) { + throw new Error( + `${scenario}: ${artifact.kind} was stored at "${artifact.storedPath}" instead of "${expectedPath}"`, + ); + } + + if (!fs.existsSync(path.join(MOCK_DATA_DIR, artifact.storedPath))) { + throw new Error(`${scenario}: ${artifact.kind} is missing from the served data at "${artifact.storedPath}"`); + } + } + + // The full archive of a release is stored under its package hash, and the patch archive + // next to it under the same hash with the patch suffix. + for (const patchArchive of bundles.filter((bundle) => bundle.fileName.endsWith(PATCH_ARCHIVE_SUFFIX))) { + const packageHash = patchArchive.fileName.slice(0, -PATCH_ARCHIVE_SUFFIX.length); + const fullArchive = bundles.find((bundle) => + bundle.platform === patchArchive.platform + && bundle.identifier === patchArchive.identifier + && bundle.fileName === packageHash); + + if (!fullArchive) { + throw new Error( + `${scenario}: patch archive "${patchArchive.storedPath}" has no full archive of the same release beside it`, + ); + } + } + + console.log(`[assert] ${scenario}: ${artifacts.length} artifacts stored under {platform}/{identifier}`); + return artifacts; +} diff --git a/e2e/helpers/binary-patch-fixtures.ts b/e2e/helpers/binary-patch-fixtures.ts new file mode 100644 index 000000000..699daa645 --- /dev/null +++ b/e2e/helpers/binary-patch-fixtures.ts @@ -0,0 +1,409 @@ +import { execFileSync } from "child_process"; +import crypto from "crypto"; +import fs from "fs"; +import path from "path"; +import { MOCK_DATA_DIR, WORK_DIR } from "../config"; +import { PATCH_ARCHIVE_SUFFIX } from "./artifact-storage"; + +/** + * Fixtures for the binary patch scenarios: the base bundle a patch is computed against, + * and the broken patch archives a client has to survive. + * + * The base bundle is taken out of the app that is installed on the device rather than + * out of a build directory. A patch only applies to the exact bytes that shipped in the + * binary, so taking them from anywhere else would test a patch against a bundle no user + * is running. + * + * The broken archives are built by rewriting a real patch archive, so everything except + * the one fault under test is exactly what the CLI produces. + */ + +/** Manifest a patch archive carries so a client knows how to rebuild the JS bundle. */ +const PATCH_MANIFEST_NAME = "codepush-binary-patch.json"; + +export interface BinaryPatchManifest { + formatVersion: number; + algorithm: string; + bundlePath: string; + patchFile: string; + baseBundleHash: string; + targetBundleHash: string; + targetBundleSize: number; +} + +export type Platform = "ios" | "android"; + +/** JS bundle name react-native writes, and the name a client looks for in an update. */ +export function getJsBundleName(platform: Platform): string { + return platform === "ios" ? "main.jsbundle" : "index.android.bundle"; +} + +export function getOtherPlatform(platform: Platform): Platform { + return platform === "ios" ? "android" : "ios"; +} + +export function sha256OfFile(filePath: string): string { + return crypto.createHash("sha256").update(fs.readFileSync(filePath)).digest("hex"); +} + +/** + * Copies the JS bundle out of the app binary that is installed on the device. + * + * @param appId {string} Android package name or iOS bundle identifier + * @return path to the extracted bundle, ready to pass to `--binary-bundle-path` + */ +export function extractBinaryBundle(platform: Platform, appId: string): string { + const bundleName = getJsBundleName(platform); + const destDir = path.join(WORK_DIR, "binary-bundle", platform); + fs.rmSync(destDir, { recursive: true, force: true }); + fs.mkdirSync(destDir, { recursive: true }); + const destPath = path.join(destDir, bundleName); + + if (platform === "android") { + extractAndroidBinaryBundle(appId, bundleName, destPath, destDir); + } else { + extractIosBinaryBundle(appId, bundleName, destPath); + } + + const size = fs.statSync(destPath).size; + if (size === 0) { + throw new Error(`The JS bundle extracted from the installed ${platform} app is empty`); + } + console.log(`[binary-patch] base bundle extracted from the installed app: ${destPath} (${size} bytes)`); + + return destPath; +} + +function extractAndroidBinaryBundle( + appId: string, + bundleName: string, + destPath: string, + workDir: string, +): void { + const paths = execFileSync("adb", ["shell", "pm", "path", appId], { encoding: "utf8" }) + .split("\n") + .map((line) => line.trim()) + .filter((line) => line.startsWith("package:")) + .map((line) => line.slice("package:".length)); + + const apkPath = paths.find((candidate) => candidate.endsWith("base.apk")) ?? paths[0]; + if (!apkPath) { + throw new Error(`Could not find the installed APK of "${appId}". Install the app before running this phase.`); + } + + const localApkPath = path.join(workDir, "installed.apk"); + execFileSync("adb", ["pull", apkPath, localApkPath], { stdio: "ignore" }); + try { + const bundle = execFileSync("unzip", ["-p", localApkPath, `assets/${bundleName}`], { + maxBuffer: 512 * 1024 * 1024, + }); + fs.writeFileSync(destPath, bundle); + } finally { + fs.rmSync(localApkPath, { force: true }); + } +} + +function extractIosBinaryBundle(appId: string, bundleName: string, destPath: string): void { + const appContainer = execFileSync("xcrun", ["simctl", "get_app_container", "booted", appId, "app"], { + encoding: "utf8", + }).trim(); + + const bundlePath = path.join(appContainer, bundleName); + if (!fs.existsSync(bundlePath)) { + throw new Error(`The installed app at "${appContainer}" does not contain "${bundleName}"`); + } + + fs.copyFileSync(bundlePath, destPath); +} + +export function getHistoryFilePath(platform: Platform, identifier: string, binaryVersion: string): string { + return path.join(MOCK_DATA_DIR, "histories", platform, identifier, `${binaryVersion}.json`); +} + +export function readReleaseHistory( + platform: Platform, + identifier: string, + binaryVersion: string, +): Record { + return JSON.parse(fs.readFileSync(getHistoryFilePath(platform, identifier, binaryVersion), "utf8")); +} + +/** + * Asserts that a release offers a binary patch, and returns the patch download URL. + * + * A patch is only published when it is smaller than the full archive, so a release that + * was meant to exercise the patch path but silently went out without one would otherwise + * turn into a scenario that quietly tests nothing. + */ +export function assertReleaseOffersPatch( + scenario: string, + platform: Platform, + identifier: string, + binaryVersion: string, + releaseVersion: string, +): string { + const release = readReleaseHistory(platform, identifier, binaryVersion)[releaseVersion]; + if (!release) { + throw new Error(`${scenario}: v${releaseVersion} is missing from the "${identifier}" release history`); + } + + const patchDownloadUrl = release.binaryPatchDownloadUrl; + if (!patchDownloadUrl || !patchDownloadUrl.endsWith(PATCH_ARCHIVE_SUFFIX)) { + throw new Error( + `${scenario}: v${releaseVersion} was published without a binary patch (binaryPatchDownloadUrl: ${String(patchDownloadUrl)})`, + ); + } + + console.log(`[assert] ${scenario}: v${releaseVersion} offers a binary patch (${patchDownloadUrl})`); + return patchDownloadUrl; +} + +/** Asserts that a release says nothing about a binary patch, so a client downloads it in full. */ +export function assertReleaseOffersNoPatch( + scenario: string, + platform: Platform, + identifier: string, + binaryVersion: string, + releaseVersion: string, +): void { + const release = readReleaseHistory(platform, identifier, binaryVersion)[releaseVersion]; + if (!release) { + throw new Error(`${scenario}: v${releaseVersion} is missing from the "${identifier}" release history`); + } + + if ("binaryPatchDownloadUrl" in release) { + throw new Error( + `${scenario}: v${releaseVersion} carries a binaryPatchDownloadUrl (${String(release.binaryPatchDownloadUrl)}), but it was released without a base bundle`, + ); + } + + console.log(`[assert] ${scenario}: v${releaseVersion} offers the full archive only`); +} + +/** + * Asserts that two releases published the same update. + * + * Releasing one pre-built bundle twice is only a fair comparison of the two histories if + * both of them describe the same package: the difference between them then has to be the + * base bundle that was passed to one release and not the other. + */ +export function assertSameReleasedPackage( + scenario: string, + platform: Platform, + identifier: string, + otherIdentifier: string, + releaseVersion: string, +): void { + const packageHash = readReleaseHistory(platform, identifier, "1.0.0")[releaseVersion]?.packageHash; + const otherPackageHash = readReleaseHistory(platform, otherIdentifier, "1.0.0")[releaseVersion]?.packageHash; + + if (!packageHash || packageHash !== otherPackageHash) { + throw new Error( + `${scenario}: "${identifier}" released ${String(packageHash)} but "${otherIdentifier}" released ${String(otherPackageHash)}`, + ); + } + + console.log(`[assert] ${scenario}: both releases published ${packageHash}`); +} + +/** + * Serves one identifier's release history to the app, which reads only its own. + * + * This is how a single installed binary gets to install the same pre-built bundle twice: + * once from a history that carries a patch URL and once from a history that does not. + */ +export function serveReleaseHistoryOf( + platform: Platform, + fromIdentifier: string, + toIdentifier: string, + binaryVersion: string, +): void { + const source = getHistoryFilePath(platform, fromIdentifier, binaryVersion); + const destination = getHistoryFilePath(platform, toIdentifier, binaryVersion); + + fs.mkdirSync(path.dirname(destination), { recursive: true }); + fs.copyFileSync(source, destination); + console.log(`[binary-patch] serving the "${fromIdentifier}" release history as "${toIdentifier}"`); +} + +export function findPatchArchive(platform: Platform, identifier: string): string { + return findArchive(platform, identifier, (fileName) => fileName.endsWith(PATCH_ARCHIVE_SUFFIX)); +} + +/** The full archive of a release is stored under its package hash, with no extension. */ +export function findFullArchive(platform: Platform, identifier: string): string { + return findArchive(platform, identifier, (fileName) => !fileName.endsWith(PATCH_ARCHIVE_SUFFIX)); +} + +function findArchive(platform: Platform, identifier: string, matches: (fileName: string) => boolean): string { + const bundleDir = path.join(MOCK_DATA_DIR, "bundles", platform, identifier); + const fileNames = fs.existsSync(bundleDir) ? fs.readdirSync(bundleDir).filter(matches) : []; + + if (fileNames.length !== 1) { + throw new Error( + `Expected exactly one matching archive in "${bundleDir}", found ${fileNames.length}: [${fileNames.join(", ")}]`, + ); + } + + return path.join(bundleDir, fileNames[0]); +} + +export function readPatchManifest(archivePath: string): BinaryPatchManifest { + const manifest = execFileSync("unzip", ["-p", archivePath, `*/${PATCH_MANIFEST_NAME}`], { encoding: "utf8" }); + return JSON.parse(manifest) as BinaryPatchManifest; +} + +/** Every file inside an archive, relative to the directory the archive wraps them in. */ +export function listArchiveContents(archivePath: string): string[] { + return execFileSync("unzip", ["-Z", "-1", archivePath], { encoding: "utf8" }) + .split("\n") + .map((line) => line.trim()) + .filter((line) => line.length > 0 && !line.endsWith("/")) + .map((line) => line.split("/").slice(1).join("/")); +} + +/** Copies the JS bundle out of a full update archive, without installing anything. */ +export function extractBundleFromArchive(archivePath: string, bundleName: string, destPath: string): void { + const bundle = execFileSync("unzip", ["-p", archivePath, `*/${bundleName}`], { + maxBuffer: 512 * 1024 * 1024, + }); + + fs.mkdirSync(path.dirname(destPath), { recursive: true }); + fs.writeFileSync(destPath, bundle); +} + +/** + * Rewrites an archive in place, leaving its name - and so the download URL the release + * history already points at - untouched. + */ +function rewriteArchive(archivePath: string, mutate: (contentsDir: string) => void): void { + fs.mkdirSync(WORK_DIR, { recursive: true }); + const stagingDir = fs.mkdtempSync(path.join(WORK_DIR, "archive-")); + + try { + execFileSync("unzip", ["-q", "-o", archivePath, "-d", stagingDir]); + mutate(resolveContentsDir(stagingDir)); + + fs.rmSync(archivePath); + execFileSync("zip", ["-q", "-r", "-X", archivePath, ...fs.readdirSync(stagingDir)], { cwd: stagingDir }); + } finally { + fs.rmSync(stagingDir, { recursive: true, force: true }); + } +} + +/** An archive wraps its files in a single directory, and that directory is the contents root. */ +function resolveContentsDir(extractDir: string): string { + const entries = fs.readdirSync(extractDir, { withFileTypes: true }); + if (entries.length === 1 && entries[0].isDirectory()) { + return path.join(extractDir, entries[0].name); + } + return extractDir; +} + +function readManifestFile(contentsDir: string): BinaryPatchManifest { + return JSON.parse(fs.readFileSync(path.join(contentsDir, PATCH_MANIFEST_NAME), "utf8")) as BinaryPatchManifest; +} + +function writeManifestFile(contentsDir: string, manifest: BinaryPatchManifest): void { + fs.writeFileSync(path.join(contentsDir, PATCH_MANIFEST_NAME), JSON.stringify(manifest, null, 2)); +} + +/** + * Corrupts the compressed data of the patch, leaving its header intact. + * + * This is the fault nothing else catches: the header still describes a patch this client + * supports, and neither the diff format nor its compressed streams carry a checksum of + * what they produce, so the applier can report success over wrong bytes. Only the hash + * of the restored bundle stands between this archive and a broken update. + */ +export function corruptPatchBody(archivePath: string): void { + rewriteArchive(archivePath, (contentsDir) => { + const patchPath = path.join(contentsDir, readManifestFile(contentsDir).patchFile); + const patch = fs.readFileSync(patchPath); + + if (patch.length < 32) { + throw new Error(`The patch at "${patchPath}" is too small to corrupt without touching its header`); + } + + for (let offset = patch.length - 4; offset < patch.length; offset += 1) { + patch[offset] = patch[offset] ^ 0xff; + } + fs.writeFileSync(patchPath, patch); + }); +} + +/** + * Leaves the patch intact but makes the manifest describe a different bundle than the one + * it restores. + * + * Applying a patch is not self-verifying: the applier reports success over whatever bytes + * it produced. This isolates the check that stands behind it - the hash of the restored + * bundle - by giving it a bundle that applies cleanly and still is not the promised one. + */ +export function breakRestoredBundleExpectation(archivePath: string): void { + rewriteArchive(archivePath, (contentsDir) => { + const manifest = readManifestFile(contentsDir); + writeManifestFile(contentsDir, { ...manifest, targetBundleHash: manifest.baseBundleHash }); + }); +} + +/** Corrupts the header of the patch, so the applier cannot even read what it is. */ +export function corruptPatchHeader(archivePath: string): void { + rewriteArchive(archivePath, (contentsDir) => { + const patchPath = path.join(contentsDir, readManifestFile(contentsDir).patchFile); + const patch = fs.readFileSync(patchPath); + + patch.fill(0, 0, Math.min(8, patch.length)); + fs.writeFileSync(patchPath, patch); + }); +} + +/** + * Turns a patch archive into the archive the other platform's release would have + * published: it rebuilds the other platform's JS bundle, out of the bundle that shipped + * in the other platform's binary. + * + * Serving it to this platform is the mistake a release pipeline makes when the two + * platforms' artifacts are crossed, and the client has to refuse it and download the + * full archive instead. + */ +export function retargetPatchArchiveToOtherPlatform( + archivePath: string, + platform: Platform, + otherPlatformBaseBundleHash: string, +): void { + const otherBundleName = getJsBundleName(getOtherPlatform(platform)); + + rewriteArchive(archivePath, (contentsDir) => { + const manifest = readManifestFile(contentsDir); + const otherPatchFile = `${otherBundleName}.patch`; + + fs.renameSync(path.join(contentsDir, manifest.patchFile), path.join(contentsDir, otherPatchFile)); + writeManifestFile(contentsDir, { + ...manifest, + bundlePath: otherBundleName, + patchFile: otherPatchFile, + baseBundleHash: otherPlatformBaseBundleHash, + }); + }); +} + +/** + * Asserts that the patch archive carries the update's assets as they are. + * + * Only the JS bundle is sent as a patch; everything else in an update travels in the + * patch archive untouched, which is what lets the restored contents hash to the same + * package as the full archive. + */ +export function assertPatchArchiveCarriesAssets(scenario: string, archivePath: string): void { + const contents = listArchiveContents(archivePath); + const assets = contents.filter((entry) => /(^|\/)(assets|drawable[^/]*|raw)\//.test(entry)); + + if (assets.length === 0) { + throw new Error( + `${scenario}: the patch archive carries no assets, so this release does not exercise them. Contents: [${contents.join(", ")}]`, + ); + } + + console.log(`[assert] ${scenario}: patch archive carries ${assets.length} asset file(s)`); +} diff --git a/e2e/helpers/binary-patch-phase.ts b/e2e/helpers/binary-patch-phase.ts new file mode 100644 index 000000000..2b8bd1e0f --- /dev/null +++ b/e2e/helpers/binary-patch-phase.ts @@ -0,0 +1,301 @@ +/** + * Installs binary patch updates on a device, and every way a patch can go wrong. + * + * A release published with a binary patch offers two archives of the same update, and a + * client that cannot use the patch has to end up with exactly the update it would have + * downloaded in full. The two are indistinguishable on screen, so what each scenario + * asserts is which archives the app asked the server for, on top of the update it ended + * up running. + */ + +import path from "path"; +import { WORK_DIR } from "../config"; +import { assertArtifactStorageLayout } from "./artifact-storage"; +import { + assertPatchArchiveCarriesAssets, + assertReleaseOffersNoPatch, + assertReleaseOffersPatch, + assertSameReleasedPackage, + breakRestoredBundleExpectation, + corruptPatchBody, + corruptPatchHeader, + extractBinaryBundle, + extractBundleFromArchive, + findFullArchive, + findPatchArchive, + getJsBundleName, + retargetPatchArchiveToOtherPlatform, + serveReleaseHistoryOf, + sha256OfFile, +} from "./binary-patch-fixtures"; +import { + assertDownloadedArchives, + startRecordingDownloads, + type DownloadedArchive, +} from "./download-order"; +import { + clearReleaseMarker, + getCodePushReleaseArgs, + prepareBundle, + runCodePushCommand, + setReleaseMarker, + setReleasingBundle, +} from "./prepare-bundle"; + +/** Binary version every example app release targets, and the name of its history file. */ +const BINARY_VERSION = "1.0.0"; + +export interface BinaryPatchPhaseContext { + appPath: string; + platform: "ios" | "android"; + framework?: "expo"; + /** Identifier the app reads its release history under. */ + releaseIdentifier: string; + /** Android package name or iOS bundle identifier of the installed app. */ + appId: string; + excludeTimingSensitive: boolean; + /** Empties the served mock data, so each scenario starts from an empty server. */ + cleanMockData: () => void; + runMaestro: (flowPath: string, flowEnv: Record) => Promise; + withRetry: (label: string, action: () => Promise) => Promise; +} + +interface BinaryPatchScenario { + name: string; + releaseVersion: string; + flowPath: string; + /** Publishes the release this scenario installs, and breaks it where the scenario needs it broken. */ + prepare: () => Promise; + /** The archives the app is expected to download, in order. */ + expectedDownloads: DownloadedArchive[]; + /** Only meaningful while the update is still being installed, so it is excluded with the other timing-sensitive scenarios. */ + timingSensitive?: boolean; +} + +export async function runBinaryPatchPhase(context: BinaryPatchPhaseContext): Promise { + const { appPath, platform, framework, releaseIdentifier, appId } = context; + + const installUpdateFlow = path.resolve(__dirname, "../flows-binary-patch/01-install-update.yaml"); + const uiResponsiveFlow = path.resolve(__dirname, "../flows-binary-patch/02-ui-responsive-during-install.yaml"); + const jsBundleName = getJsBundleName(platform); + + // A patch only applies to the exact bundle that shipped inside the app binary, so the + // base is taken out of the app that is installed on the device rather than out of a + // build directory. + const binaryBundlePath = extractBinaryBundle(platform, appId); + // Filled in by the first scenario: a real JS bundle of this app that is not the one in + // the binary, which is what a release built against a stale binary patches against. + const staleBaseBundlePath = path.join(WORK_DIR, "stale-base", jsBundleName); + + const releasePatchUpdate = ( + releaseVersion: string, + extraOptions: { assetMarkerVersion?: string; mandatory?: boolean; binaryBundlePath?: string } = {}, + ) => prepareBundle(appPath, platform, releaseIdentifier, framework, { + releaseVersion, + releaseMarkerVersion: releaseVersion, + binaryBundlePath, + ...extraOptions, + }); + + const installUpdate = ( + scenarioName: string, + flowPath: string, + releaseVersion: string, + expectedDownloads: DownloadedArchive[], + ) => context.withRetry(`run-maestro: binary patch (${scenarioName})`, async () => { + startRecordingDownloads(); + await context.runMaestro(flowPath, { RELEASE_LABEL: releaseVersion }); + assertDownloadedArchives(scenarioName, expectedDownloads); + }); + + const scenarios: BinaryPatchScenario[] = [ + { + name: "patch update installs on top of the app binary", + releaseVersion: "1.3.1", + flowPath: installUpdateFlow, + expectedDownloads: ["patch"], + prepare: async () => { + await releasePatchUpdate("1.3.1"); + // The bundle this release ships is a real bundle of this app that is not the one + // in the binary, which is exactly what a later scenario needs as a stale base. + extractBundleFromArchive(findFullArchive(platform, releaseIdentifier), jsBundleName, staleBaseBundlePath); + }, + }, + { + name: "patch update carrying assets installs", + releaseVersion: "1.3.2", + flowPath: installUpdateFlow, + expectedDownloads: ["patch"], + prepare: async () => { + await releasePatchUpdate("1.3.2", { assetMarkerVersion: "1.3.2" }); + assertPatchArchiveCarriesAssets( + "patch update carrying assets installs", + findPatchArchive(platform, releaseIdentifier), + ); + }, + }, + { + name: "patch against a stale base bundle falls back to the full update", + releaseVersion: "1.3.3", + flowPath: installUpdateFlow, + expectedDownloads: ["patch", "full"], + prepare: () => releasePatchUpdate("1.3.3", { binaryBundlePath: staleBaseBundlePath }), + }, + { + name: "corrupt patch body falls back to the full update", + releaseVersion: "1.3.4", + flowPath: installUpdateFlow, + expectedDownloads: ["patch", "full"], + prepare: async () => { + await releasePatchUpdate("1.3.4"); + corruptPatchBody(findPatchArchive(platform, releaseIdentifier)); + }, + }, + { + name: "restored bundle that the manifest does not describe falls back to the full update", + releaseVersion: "1.3.9", + flowPath: installUpdateFlow, + expectedDownloads: ["patch", "full"], + prepare: async () => { + await releasePatchUpdate("1.3.9"); + breakRestoredBundleExpectation(findPatchArchive(platform, releaseIdentifier)); + }, + }, + { + name: "corrupt patch header falls back to the full update", + releaseVersion: "1.3.5", + flowPath: installUpdateFlow, + expectedDownloads: ["patch", "full"], + prepare: async () => { + await releasePatchUpdate("1.3.5"); + corruptPatchHeader(findPatchArchive(platform, releaseIdentifier)); + }, + }, + { + name: "patch archive of the other platform falls back to the full update", + releaseVersion: "1.3.6", + flowPath: installUpdateFlow, + expectedDownloads: ["patch", "full"], + prepare: async () => { + await releasePatchUpdate("1.3.6"); + retargetPatchArchiveToOtherPlatform( + findPatchArchive(platform, releaseIdentifier), + platform, + sha256OfFile(staleBaseBundlePath), + ); + }, + }, + { + name: "app stays responsive while a patch update installs", + releaseVersion: "1.3.8", + flowPath: uiResponsiveFlow, + expectedDownloads: ["patch"], + timingSensitive: true, + prepare: () => releasePatchUpdate("1.3.8", { mandatory: false }), + }, + ]; + + for (const scenario of scenarios) { + if (scenario.timingSensitive && context.excludeTimingSensitive) { + console.log(`\n=== [phase 6] skipping timing-sensitive scenario (${scenario.name}) ===`); + continue; + } + + console.log(`\n=== [prepare-bundle: binary patch ${scenario.releaseVersion} (${scenario.name})] ===`); + context.cleanMockData(); + await scenario.prepare(); + assertArtifactStorageLayout(scenario.name); + assertReleaseOffersPatch(scenario.name, platform, releaseIdentifier, BINARY_VERSION, scenario.releaseVersion); + + await installUpdate(scenario.name, scenario.flowPath, scenario.releaseVersion, scenario.expectedDownloads); + } + + await runPublishedTwiceScenario(context, binaryBundlePath, installUpdateFlow, installUpdate); +} + +/** + * One pre-built bundle, released twice with the base bundle passed to only one of the two + * releases, so only one of the two histories carries a patch URL. + * + * The releases are told apart by identifier rather than by binary version: a second binary + * version would need a second app binary, while a second identifier is the same release + * axis - it stores its own history under `{platform}/{identifier}` - and can be installed + * by the binary that is already on the device. The app reads the history of its own + * identifier, so each of the two is served to it in turn. + */ +async function runPublishedTwiceScenario( + context: BinaryPatchPhaseContext, + binaryBundlePath: string, + installUpdateFlow: string, + installUpdate: ( + scenarioName: string, + flowPath: string, + releaseVersion: string, + expectedDownloads: DownloadedArchive[], + ) => Promise, +): Promise { + const { appPath, platform, framework, releaseIdentifier } = context; + const scenario = "one pre-built bundle released with and without a patch"; + const releaseVersion = "1.3.7"; + const fullOnlyIdentifier = `${releaseIdentifier}-full-only`; + + console.log(`\n=== [prepare-bundle: binary patch ${releaseVersion} (${scenario})] ===`); + context.cleanMockData(); + setReleasingBundle(appPath, true); + const { entryFile, frameworkArgs } = getCodePushReleaseArgs(appPath, framework); + try { + setReleaseMarker(appPath, releaseVersion); + await runCodePushCommand(appPath, platform, [ + "bundle", + ...frameworkArgs, + "-p", platform, + "-e", entryFile, + "--binary-bundle-path", binaryBundlePath, + ]); + + for (const identifier of [releaseIdentifier, fullOnlyIdentifier]) { + await runCodePushCommand(appPath, platform, [ + "create-history", + "-c", "code-push.config.local.ts", + "-b", BINARY_VERSION, + "-p", platform, + "-i", identifier, + ]); + } + + // The bundle is left in place for the second release, which reuses it untouched. + await runCodePushCommand(appPath, platform, [ + "release", + "-c", "code-push.config.local.ts", + "-b", BINARY_VERSION, "-v", releaseVersion, + ...frameworkArgs, + "-p", platform, "-i", releaseIdentifier, + "-e", entryFile, "-m", "true", + "--skip-bundle", "true", + "--skip-cleanup", "true", + "--binary-bundle-path", binaryBundlePath, + ]); + await runCodePushCommand(appPath, platform, [ + "release", + "-c", "code-push.config.local.ts", + "-b", BINARY_VERSION, "-v", releaseVersion, + ...frameworkArgs, + "-p", platform, "-i", fullOnlyIdentifier, + "-e", entryFile, "-m", "true", + "--skip-bundle", "true", + ]); + } finally { + clearReleaseMarker(appPath); + setReleasingBundle(appPath, false); + } + + assertArtifactStorageLayout(scenario); + assertReleaseOffersPatch(scenario, platform, releaseIdentifier, BINARY_VERSION, releaseVersion); + assertReleaseOffersNoPatch(scenario, platform, fullOnlyIdentifier, BINARY_VERSION, releaseVersion); + assertSameReleasedPackage(scenario, platform, releaseIdentifier, fullOnlyIdentifier, releaseVersion); + + await installUpdate(`${scenario} — history with a patch URL`, installUpdateFlow, releaseVersion, ["patch"]); + + serveReleaseHistoryOf(platform, fullOnlyIdentifier, releaseIdentifier, BINARY_VERSION); + await installUpdate(`${scenario} — history without a patch URL`, installUpdateFlow, releaseVersion, ["full"]); +} diff --git a/e2e/helpers/download-order.ts b/e2e/helpers/download-order.ts new file mode 100644 index 000000000..680005a00 --- /dev/null +++ b/e2e/helpers/download-order.ts @@ -0,0 +1,51 @@ +import { clearRequestLog, getRequestLog } from "../mock-server/server"; +import { PATCH_ARCHIVE_SUFFIX } from "./artifact-storage"; + +/** + * Which archive of an update the app asked the mock server for. + * + * A release published with a binary patch offers two archives that install to the same + * contents, so the app looks and behaves identically whether it applied the patch or + * downloaded the full update after the patch failed. What the two cases do not share is + * which archives were requested: a patch install fetches the patch alone, a fallback + * fetches the patch and then the full archive, and a release without a patch fetches the + * full archive alone. + */ +export type DownloadedArchive = "patch" | "full"; + +export function startRecordingDownloads(): void { + clearRequestLog(); +} + +/** The update archives the app downloaded, in the order it asked for them. */ +export function getDownloadedArchives(): DownloadedArchive[] { + return getRequestLog() + .filter((request) => request.method === "GET" && request.url.startsWith("/bundles/")) + .map((request) => (request.url.endsWith(PATCH_ARCHIVE_SUFFIX) ? "patch" : "full")); +} + +export function assertDownloadedArchives(scenario: string, expected: DownloadedArchive[]): void { + const actual = getDownloadedArchives(); + + if (actual.length !== expected.length || actual.some((archive, index) => archive !== expected[index])) { + throw new Error( + `${scenario}: expected the app to download [${expected.join(", ")}], but it downloaded [${actual.join(", ")}]`, + ); + } + + console.log(`[assert] ${scenario}: downloaded [${actual.join(", ")}]`); +} + +/** Asserts that no patch archive was offered to the app, let alone downloaded. */ +export function assertNoPatchDownloads(scenario: string): void { + const archives = getDownloadedArchives(); + + if (archives.includes("patch")) { + throw new Error(`${scenario}: expected full archives only, but the app downloaded [${archives.join(", ")}]`); + } + if (archives.length === 0) { + throw new Error(`${scenario}: expected at least one full archive download, but nothing was downloaded`); + } + + console.log(`[assert] ${scenario}: downloaded [${archives.join(", ")}]`); +} diff --git a/e2e/helpers/prepare-bundle.ts b/e2e/helpers/prepare-bundle.ts index e6c14f114..4d8e0a944 100644 --- a/e2e/helpers/prepare-bundle.ts +++ b/e2e/helpers/prepare-bundle.ts @@ -1,13 +1,17 @@ import fs from "fs"; import path from "path"; import { spawn } from "child_process"; -import { MOCK_DATA_DIR, getMockServerHost } from "../config"; +import { ARTIFACT_LOG_PATH, MOCK_DATA_DIR, getMockServerHost } from "../config"; interface PrepareBundleOptions { releaseVersion?: string; mandatory?: boolean; releaseMarkerVersion?: string; crashOnStartVersion?: string; + /** Releases a binary patch against this JS bundle alongside the full bundle. */ + binaryBundlePath?: string; + /** Adds an image asset to the released bundle, so the update carries more than JS. */ + assetMarkerVersion?: string; } export function setReleasingBundle(appPath: string, value: boolean): void { @@ -73,6 +77,49 @@ export function clearCrashOnStartMarker(appPath: string): void { fs.writeFileSync(appTsxPath, content, "utf8"); } +const ASSET_MARKER_PATTERN = /^global\.__E2E_ASSET__ = require\("\.\/e2e-asset-.*\.png"\);$/m; +const ASSET_MARKER_FILE_PREFIX = "e2e-asset-"; +// Smallest valid PNG, so Metro reads its dimensions and packs it like any other image. +const ASSET_MARKER_PNG_BASE64 = + "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg=="; + +/** + * Adds an image asset to the released bundle. + * + * An update is not only its JS bundle, and a binary patch only replaces the bundle: the + * assets have to travel in the patch archive untouched for the restored contents to be + * the update. Requiring the image is enough to put it in the update - Metro copies every + * asset the module graph reaches, whether or not the app draws it. + */ +export function setAssetMarker(appPath: string, version: string): void { + const assetFileName = `${ASSET_MARKER_FILE_PREFIX}${version}.png`; + fs.writeFileSync(path.join(appPath, assetFileName), Buffer.from(ASSET_MARKER_PNG_BASE64, "base64")); + + const appTsxPath = path.join(appPath, "App.tsx"); + let content = fs.readFileSync(appTsxPath, "utf8"); + // Assigned to a global so that no minifier can decide the asset is unused. + const marker = `global.__E2E_ASSET__ = require("./${assetFileName}");`; + if (ASSET_MARKER_PATTERN.test(content)) { + content = content.replace(ASSET_MARKER_PATTERN, marker); + } else { + content = `${marker}\n${content}`; + } + fs.writeFileSync(appTsxPath, content, "utf8"); +} + +export function clearAssetMarker(appPath: string): void { + const appTsxPath = path.join(appPath, "App.tsx"); + let content = fs.readFileSync(appTsxPath, "utf8"); + content = content.replace(ASSET_MARKER_PATTERN, "").replace(/^\n+/, ""); + fs.writeFileSync(appTsxPath, content, "utf8"); + + for (const fileName of fs.readdirSync(appPath)) { + if (fileName.startsWith(ASSET_MARKER_FILE_PREFIX) && fileName.endsWith(".png")) { + fs.rmSync(path.join(appPath, fileName), { force: true }); + } + } +} + export async function prepareBundle( appPath: string, platform: "ios" | "android", @@ -84,6 +131,7 @@ export async function prepareBundle( const mandatory = options.mandatory ?? true; const releaseMarkerVersion = options.releaseMarkerVersion; const crashOnStartVersion = options.crashOnStartVersion; + const assetMarkerVersion = options.assetMarkerVersion; setReleasingBundle(appPath, true); @@ -94,6 +142,9 @@ export async function prepareBundle( if (crashOnStartVersion) { setCrashOnStartMarker(appPath, crashOnStartVersion); } + if (assetMarkerVersion) { + setAssetMarker(appPath, assetMarkerVersion); + } await runCodePushCommand(appPath, platform, [ "create-history", @@ -109,6 +160,7 @@ export async function prepareBundle( releaseVersion, mandatory, framework, + options.binaryBundlePath, ); } finally { if (releaseMarkerVersion) { @@ -117,6 +169,9 @@ export async function prepareBundle( if (crashOnStartVersion) { clearCrashOnStartMarker(appPath); } + if (assetMarkerVersion) { + clearAssetMarker(appPath); + } setReleasingBundle(appPath, false); } } @@ -128,6 +183,7 @@ function runCodePushRelease( releaseVersion: string, mandatory: boolean, framework?: "expo", + binaryBundlePath?: string, ): Promise { const { frameworkArgs, entryFile } = getCodePushReleaseArgs(appPath, framework); return runCodePushCommand(appPath, platform, [ @@ -140,6 +196,7 @@ function runCodePushRelease( "-i", appName, "-e", entryFile, "-m", mandatory ? "true" : "false", + ...(binaryBundlePath ? ["--binary-bundle-path", binaryBundlePath] : []), ]); } @@ -195,6 +252,7 @@ export function runCodePushCommand( ...process.env, E2E_MOCK_DATA_DIR: MOCK_DATA_DIR, E2E_MOCK_SERVER_HOST: getMockServerHost(platform), + E2E_ARTIFACT_LOG_PATH: ARTIFACT_LOG_PATH, }, }); child.on("error", reject); diff --git a/e2e/mock-server/server.ts b/e2e/mock-server/server.ts index b4d435e71..04ffcac3c 100644 --- a/e2e/mock-server/server.ts +++ b/e2e/mock-server/server.ts @@ -4,12 +4,38 @@ import type { Server } from "http"; let server: Server | null = null; +/** One request the app made to the mock server, in the order it arrived. */ +export interface MockServerRequest { + method: string; + url: string; + receivedAt: number; +} + +const requestLog: MockServerRequest[] = []; + +/** + * Every request the server has answered since the log was last cleared. + * + * The order matters more than the count: an update that is published as both a full and + * a patch archive installs to the same contents either way, so which archives were asked + * for, and in which order, is what tells a patch install apart from a fallback to the + * full archive. + */ +export function getRequestLog(): MockServerRequest[] { + return [...requestLog]; +} + +export function clearRequestLog(): void { + requestLog.length = 0; +} + export function startMockServer(): Promise { return new Promise((resolve, reject) => { const app = express(); app.use((req: express.Request, _res: express.Response, next: express.NextFunction) => { console.log(`[mock-server] ${req.method} ${req.url}`); + requestLog.push({ method: req.method, url: req.url, receivedAt: Date.now() }); next(); }); diff --git a/e2e/run.ts b/e2e/run.ts index bcab6509e..6d07d9671 100644 --- a/e2e/run.ts +++ b/e2e/run.ts @@ -1,12 +1,16 @@ import { Command } from "commander"; -import { spawn } from "child_process"; +import { spawn, spawnSync } from "child_process"; import path from "path"; import fs from "fs"; -import { getAppPath, MOCK_DATA_DIR } from "./config"; +import { getAppPath, MOCK_DATA_DIR, MOCK_SERVER_PORT, WORK_DIR } from "./config"; import { prepareConfig, restoreConfig } from "./helpers/prepare-config"; import { prepareBundle, runCodePushCommand, setReleasingBundle, setReleaseMarker, clearReleaseMarker, getCodePushReleaseArgs } from "./helpers/prepare-bundle"; import { buildApp } from "./helpers/build-app"; import { startMockServer, stopMockServer } from "./mock-server/server"; +import { assertArtifactStorageLayout, clearArtifactLog } from "./helpers/artifact-storage"; +import { assertNoPatchDownloads, startRecordingDownloads } from "./helpers/download-order"; +import { assertReleaseOffersNoPatch } from "./helpers/binary-patch-fixtures"; +import { runBinaryPatchPhase } from "./helpers/binary-patch-phase"; interface CliOptions { app: string; @@ -96,6 +100,7 @@ async function main() { try { // 1. Prepare config console.log("\n=== [prepare] ==="); + prepareAndroidMockServerAccess(options.platform); prepareConfig(appPath, options.platform); // 2. Build (unless --maestro-only) @@ -115,12 +120,20 @@ async function main() { await startMockServer(); await resetAppStateBeforeFlows(options.platform, appId); + // A release published without a base bundle says nothing about a binary patch, and + // has to keep being downloaded in full exactly as it was before patches existed. + const fullOnlyRelease = "release without a binary patch"; + assertArtifactStorageLayout(fullOnlyRelease); + assertReleaseOffersNoPatch(fullOnlyRelease, options.platform, releaseIdentifier, "1.0.0", "1.0.1"); + // 5. Run Maestro — Phase 1: main flows console.log("\n=== [run-maestro: phase 1] ==="); const flowsDir = path.resolve(__dirname, "flows"); - await withRetry("run-maestro: phase 1", options.retryCount, retryDelayMs, () => - runMaestro(flowsDir, options.platform, appId), - ); + await withRetry("run-maestro: phase 1", options.retryCount, retryDelayMs, async () => { + startRecordingDownloads(); + await runMaestro(flowsDir, options.platform, appId); + assertNoPatchDownloads(fullOnlyRelease); + }); // 6. Disable release for rollback test console.log("\n=== [disable-release] ==="); @@ -316,6 +329,20 @@ async function main() { ); } + // 13. Run Maestro — Phase 6: binary patch updates + console.log("\n=== [run-maestro: phase 6 (binary patch updates)] ==="); + await runBinaryPatchPhase({ + appPath, + platform: options.platform, + framework: options.framework, + releaseIdentifier, + appId, + excludeTimingSensitive: options.excludeTimingSensitive ?? false, + cleanMockData, + runMaestro: (flowPath, flowEnv) => runMaestro(flowPath, options.platform, appId, flowEnv), + withRetry: (label, action) => withRetry(label, options.retryCount, retryDelayMs, action), + }); + console.log("\n=== E2E tests passed ==="); } catch (error) { const message = error instanceof Error ? error.message : String(error); @@ -335,6 +362,49 @@ function cleanMockData(): void { fs.rmSync(MOCK_DATA_DIR, { recursive: true }); } fs.mkdirSync(MOCK_DATA_DIR, { recursive: true }); + // The record of what was stored describes the data that was just thrown away, so it is + // cleared with it and every assertion over it is scoped to one scenario. + clearArtifactLog(); + fs.mkdirSync(WORK_DIR, { recursive: true }); +} + +/** + * Points an Android device at the mock server. + * + * An emulator reaches the host through a loopback alias, which is what the mock server + * host defaults to. A phone connected over adb has no such alias, so the server port is + * forwarded onto the device and the app is pointed at the device's own localhost. + */ +function prepareAndroidMockServerAccess(platform: "ios" | "android"): void { + if (platform !== "android" || process.env.E2E_ANDROID_MOCK_SERVER_HOST) { + return; + } + + const listed = spawnSync("adb", ["devices"], { encoding: "utf8" }); + if (listed.status !== 0) { + return; + } + + const serials = listed.stdout + .split("\n") + .slice(1) + .map((line) => line.split("\t")) + .filter((columns) => columns.length === 2 && columns[1].trim() === "device") + .map((columns) => columns[0].trim()); + + if (serials.length === 0 || serials.some((serial) => serial.startsWith("emulator-"))) { + return; + } + + const port = String(MOCK_SERVER_PORT); + console.log(`[command] adb reverse tcp:${port} tcp:${port}`); + const forwarded = spawnSync("adb", ["reverse", `tcp:${port}`, `tcp:${port}`], { stdio: "inherit" }); + if (forwarded.status !== 0) { + throw new Error(`adb reverse tcp:${port} failed; the device cannot reach the mock server`); + } + + process.env.E2E_ANDROID_MOCK_SERVER_HOST = "localhost"; + console.log(`[android] physical device detected (${serials.join(", ")}); mock server forwarded to its localhost`); } // npx code-push release/create-history must use the same identifier that the app uses when fetching history. @@ -449,12 +519,20 @@ function runMaestro( flowsDir: string, platform: "ios" | "android", appId: string, + flowEnv: Record = {}, ): Promise { + // Scenarios that differ only in what the installed update should say reuse one flow and + // are told the difference through the flow environment. maestro-runner refuses a mix of + // the long and the short flag, so each runner is passed the form it is already given. + const flowEnvArgs = (flag: string) => + Object.entries(flowEnv).flatMap(([name, value]) => [flag, `${name}=${value}`]); + if (platform === "ios") { const args = [ "test", "--platform", "ios", "-e", `APP_ID=${appId}`, + ...flowEnvArgs("-e"), flowsDir, ]; console.log(`[command] maestro ${args.join(" ")}`); @@ -473,7 +551,7 @@ function runMaestro( const reportRootDir = path.resolve(__dirname, "reports"); fs.mkdirSync(reportRootDir, { recursive: true }); const args = ["--platform", "android"]; - args.push("test", "--output", reportRootDir, "--env", `APP_ID=${appId}`, flowsDir); + args.push("test", "--output", reportRootDir, "--env", `APP_ID=${appId}`, ...flowEnvArgs("--env"), flowsDir); console.log(`[command] maestro-runner ${args.join(" ")}`); diff --git a/e2e/templates/code-push.config.local.ts b/e2e/templates/code-push.config.local.ts index 1f2fc78af..9fe8cd3da 100644 --- a/e2e/templates/code-push.config.local.ts +++ b/e2e/templates/code-push.config.local.ts @@ -15,12 +15,25 @@ if (!MOCK_SERVER_HOST) { throw new Error("E2E_MOCK_SERVER_HOST environment variable is required"); } +// Optional: when set, every stored artifact is appended here as one JSON object per +// line, so the runner can assert where the CLI asked for its artifacts to be stored +// instead of re-deriving the paths it expects. +const ARTIFACT_LOG_PATH = process.env.E2E_ARTIFACT_LOG_PATH; + function ensureDir(dir: string) { if (!fs.existsSync(dir)) { fs.mkdirSync(dir, { recursive: true }); } } +function recordArtifact(entry: Record) { + if (!ARTIFACT_LOG_PATH) { + return; + } + ensureDir(path.dirname(ARTIFACT_LOG_PATH)); + fs.appendFileSync(ARTIFACT_LOG_PATH, `${JSON.stringify(entry)}\n`); +} + const Config: CliConfigInterface = { bundleUploader: async ( source: string, @@ -36,6 +49,14 @@ const Config: CliConfigInterface = { const downloadUrl = `${MOCK_SERVER_HOST}/bundles/${platform}/${identifier}/${fileName}`; console.log("Bundle copied to:", destPath); console.log("Download URL:", downloadUrl); + recordArtifact({ + kind: "bundle", + platform, + identifier, + fileName, + storedPath: path.relative(MOCK_DATA_DIR, destPath), + downloadUrl, + }); return { downloadUrl }; }, @@ -65,6 +86,13 @@ const Config: CliConfigInterface = { const destPath = path.join(destDir, `${targetBinaryVersion}.json`); fs.copyFileSync(jsonFilePath, destPath); console.log("Release history saved to:", destPath); + recordArtifact({ + kind: "history", + platform, + identifier, + binaryVersion: targetBinaryVersion, + storedPath: path.relative(MOCK_DATA_DIR, destPath), + }); }, };