feat(crashtracking): retrieve c assert message for linux when __assert_fail is dynamically loaded#2268
Open
gyuheon0h wants to merge 1 commit into
Open
feat(crashtracking): retrieve c assert message for linux when __assert_fail is dynamically loaded#2268gyuheon0h wants to merge 1 commit into
__assert_fail is dynamically loaded#2268gyuheon0h wants to merge 1 commit into
Conversation
Contributor
Author
This stack of pull requests is managed by Graphite. Learn more about stacking. |
gyuheon0h
changed the title
Contributor
📚 Documentation Check Results📦
|
Contributor
🔒 Cargo Deny Results📦
|
gyuheon0h
force-pushed
the
gyuheon0h/PROF-15482-c-assert-msg
branch
from
July 23, 2026 16:42
f3e6979 to
6e7f47d
Compare
BenchmarksComparisonBenchmark execution time: 2026-07-23 18:36:40 Comparing candidate commit 7568fed in PR branch Found 0 performance improvements and 1 performance regressions! Performance is the same for 0 metrics, 0 unstable metrics.
|
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 7568fed | 1784831316 | gyuheon0h/PROF-15482-c-assert-msg |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| receiver_entry_point/report/2644 | execution_time | 3.488ms | 3.528ms ± 0.033ms | 3.518ms ± 0.010ms | 3.529ms | 3.604ms | 3.650ms | 3.693ms | 4.96% | 2.409 | 6.445 | 0.94% | 0.002ms | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| receiver_entry_point/report/2644 | execution_time | [3.524ms; 3.533ms] or [-0.131%; +0.131%] | None | None | None |
Baseline
Baseline benchmark details
Group 1
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 15899df | 1784828029 | main |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| receiver_entry_point/report/2644 | execution_time | 3.311ms | 3.345ms ± 0.034ms | 3.334ms ± 0.009ms | 3.345ms | 3.433ms | 3.449ms | 3.467ms | 4.02% | 1.981 | 3.034 | 1.00% | 0.002ms | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| receiver_entry_point/report/2644 | execution_time | [3.340ms; 3.349ms] or [-0.139%; +0.139%] | None | None | None |
🎉 All green!🧪 All tests passed 🎯 Code Coverage (details) 🔗 Commit SHA: 7568fed | Docs | Datadog PR Page | Give us feedback! |
gyuheon0h
force-pushed
the
gyuheon0h/PROF-15482-c-assert-msg
branch
2 times, most recently
from
July 23, 2026 17:13
13ad896 to
28b571f
Compare
gyuheon0h
changed the title
__assert_fail is dynamically loaded
gyuheon0h
changed the title
__assert_fail is dynamically loaded__assert_fail is dynamically loaded
gyuheon0h
force-pushed
the
gyuheon0h/PROF-15482-c-assert-msg
branch
from
July 23, 2026 18:19
28b571f to
6b2f473
Compare
gyuheon0h
force-pushed
the
gyuheon0h/PROF-15482-c-assert-msg
branch
from
July 23, 2026 18:28
6b2f473 to
7568fed
Compare
Contributor
Artifact Size Benchmark Reportaarch64-alpine-linux-musl
aarch64-unknown-linux-gnu
libdatadog-x64-windows
libdatadog-x86-windows
x86_64-alpine-linux-musl
x86_64-unknown-linux-gnu
|
gyuheon0h
marked this pull request as ready for review
July 24, 2026 20:45
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7568fed720
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| let guard_ptr = &mut guard as *mut PageProtGuard; | ||
| let patched_ptr = &mut patched_any as *mut bool; | ||
|
|
||
| iterate_libraries(|info| { |
There was a problem hiding this comment.
In runtimes that dlopen native extensions after crashtracker::init, this one-time dl_iterate_phdr pass only patches the libraries that are already loaded, and there is no dlopen hook or assert-hook update path elsewhere in the repo. A later-loaded extension gets its own __assert_fail PLT/GOT entry pointing directly at libc, so its C assert() failures still report as a bare SIGABRT instead of carrying the assertion text; the hook needs to be applied on later loads as well. This matters for libdatadog’s FFI runtime integrations.
AGENTS.md reference: AGENTS.md:L70-L74
Useful? React with 👍 / 👎.
| Receiver::update_stored_config(receiver_config)?; | ||
| register_crash_handlers(&config)?; | ||
| register_panic_hook()?; | ||
| install_assert_hook()?; |
There was a problem hiding this comment.
When callers configure crashtracker without SIGABRT (for example, only SIGSEGV), this still installs a process-wide GOT hook for __assert_fail even though the registered signal handlers above will not handle the abort. That changes every loaded module's C assert path and allocates in the hook while providing no crash report, so the hook should be guarded on config.signals().contains(&libc::SIGABRT) or a separate explicit opt-in.
AGENTS.md reference: AGENTS.md:L70-L74
Useful? React with 👍 / 👎.
Comment on lines
+132
to
+133
| if strtab.is_null() || symtab.is_null() || gnu_hash.is_null() { | ||
| return None; |
There was a problem hiding this comment.
When a loaded object was linked with --hash-style=sysv and has DT_HASH but no DT_GNU_HASH, this early return skips its relocation tables entirely. Such objects can still have __assert_fail PLT relocations, so their C asserts bypass the hook and keep producing a bare SIGABRT even when the library was loaded before init; please parse DT_HASH or use another bounded dynsym fallback instead of requiring GNU hash.
AGENTS.md reference: AGENTS.md:L70-L74
Useful? React with 👍 / 👎.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
When a process crashes due to a C assert() failure, the crash report now includes the assertion expression string (
Assertion failed: (x > 0), function foo, file bar.c, line 42) in error.message, instead of just reporting a bare SIGABRT.When a shared library calls an external function like
__assert_fail, it doesn't jump directly to the target. Instead, it jumps through the global offset table. Every shared library has its own GOT with its own slot for each external function it calls.We overwrites the slots at runtime for
__assert_failso calls are redirected to our hook function. My initial thought was to useLD_PRELOADbut this approach works no matter the load order.I was inspired by
libdd-profiling-heap-gotterwhich does something similar to interceptmalloc/freefor heap profiling.During
crashtracker::init(),install_assert_hook()PT_DYNAMICsegment to locate its relocation tablesWhen a C
assert()fails:The hook captures the assertion expression, file name, line no, func name, and stores it through an atomic pointer:
__assert_fail()implementation.abort(), which raises SIGABRT.Unfortunately, this doesn't work everywhere
__assert_failhas no GOT entry to patch. The hook installs silently as a no-op.install_assert_hook()compiles to an empty function.Motivation
What inspired you to submit this pull request?
Additional Notes
Anything else we should know when reviewing?
How to test the change?