Skip to content

ci: run the compiler PHPT suite under UBSan - #139

Open
Giandonn wants to merge 1 commit into
swoole:masterfrom
Giandonn:ci/run-tests-sanitize
Open

Giandonn wants to merge 1 commit into
swoole:masterfrom
Giandonn:ci/run-tests-sanitize

Conversation

@Giandonn

@Giandonn Giandonn commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

The compiler rejects every statically detectable undefined operation, but undefined behavior that only shows up at runtime in the generated C++ is compiled into whatever the CPU happens to do. tpc already supports --sanitize, but nothing in CI used it, so a regression of this kind is invisible to the test suite: the binary prints some value, and if the PHPT expectation was written from that binary, the test passes.

The change

  • run-tests.php --sanitize <list> (undefined, address, or both): every test binary is compiled with --sanitize <list>, and UBSAN_OPTIONS=halt_on_error=1 turns a report into a failing test instead of a line in its output. The option reaches the parallel workers together with the other globals. An unknown sanitizer is rejected up front.
  • The Linux PHPT job gets one extra matrix entry, PHP 8.5 with sanitize: undefined. sanitize: [""] in the base matrix makes GitHub add it as a separate job rather than merging the key into the existing PHP 8.5 entry. Its artifacts get a -undefined suffix, and release packaging still comes only from the unsanitized entries.

It catches what it should

A PHPT whose expectation was written from the binary (shiftLeft(1, 64) prints 1) passes without the option and fails with it, with the location in the generated C++:

$ php run-tests.php tmp/ub-shift.phpt                       -> PASS
$ php run-tests.php --sanitize undefined tmp/ub-shift.phpt  -> FAIL
ub-shift.cc:21:34: runtime error: shift exponent 64 is too large for 64-bit type 'long int'

The same trap fails when it runs among other tests with -j4, so the option does reach the workers and is not bypassed by the build cache (sanitize is part of the native command options).

The current suite is clean

PHPT tests/compiler with --sanitize undefined
Local, PHP 8.4.26 ZTS, GCC 11 1247 passed, 0 failed, 27 skipped
This branch on GitHub Actions (PHPT - PHP 8.5 ZTS (undefined sanitizer)) 1272 passed, 0 failed, 2 skipped

So the job starts green; from here on, new undefined behavior in generated code fails CI instead of silently changing results. All 9 jobs of that run passed, and actionlint reports no issues for the workflow.

The compiler rejects every statically detectable undefined operation, but
undefined behavior that only appears at runtime in the generated C++ (a
shift count >= 64, a zero divisor, signed overflow, misaligned access, ...)
is silently compiled into whatever the CPU happens to do. tpc already
supports --sanitize, but nothing in CI used it.

run-tests.php gains --sanitize <list>: every test binary is compiled with the
given sanitizers, and UBSAN_OPTIONS=halt_on_error=1 turns a report into a
failing test instead of a line in its output. The option travels to the
parallel workers with the other globals.

The Linux PHPT job gets one more matrix entry, PHP 8.5 with
sanitize=undefined. The current suite is clean under UBSan (1247 passed,
27 skipped locally), so the job starts green and guards against new
undefined behavior in generated code. Release packaging still comes only
from the unsanitized entries.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant