Skip to content

Add thapi_start() and thapi_stop() - #454

Open
thilinarmtb wants to merge 54 commits into
argonne-lcf:develfrom
thilinarmtb:thapi_on_and_off_v2
Open

Add thapi_start() and thapi_stop()#454
thilinarmtb wants to merge 54 commits into
argonne-lcf:develfrom
thilinarmtb:thapi_on_and_off_v2

Conversation

@thilinarmtb

@thilinarmtb thilinarmtb commented Nov 17, 2025

Copy link
Copy Markdown
Contributor

I created lttng_ust_toggle event category and implemented thapi_start() and thapi_stop()
user APIs (in a similar spirit to cudaProfilerStart() etc.). The library which the user has to link
against is named libThapi.so.

Usage of the API is as follows:

#include <thapi.h>
//
// code
//
thapi_start();
// code to be profiled
thapi_stop();
//
// more code
//

If libThapi.so is linked, thapi_auto_stop() (which is not part of the user API) is called automatically
during the library load time using __attribute__((constructor)). I added a check in configure.ac to
check if the compiler supports __attribute__((constructor)).

If the user runs iprof with --toggle-on, the traces generated by thapi_auto_stop(), thapi_start()
and thapi_stop() will be respected by babeltrace (otherwise there is no change to iprof/babeltrace
behavior). Since thapi_auto_stop() is called automatically, babeltrace will filter out all of the traces
till it see a thapi_start(). Trace filtering out will start again once the user call thapi_stop(). This API
only affects the trace collection of the calling process/thread.

P.S: This feature requires lttng-ust >= 2.12.8. Specifically, this fix.

TODO

  • Respect lttng_ust_profiler_* lttng_ust_toggle_* events when calculating statistics
  • Fix failing checks
  • Add the filter.toggle.toggle to babeltrace graph

@thilinarmtb
thilinarmtb force-pushed the thapi_on_and_off_v2 branch 2 times, most recently from 8120018 to 7a184b7 Compare December 5, 2025 17:38
@thilinarmtb
thilinarmtb force-pushed the thapi_on_and_off_v2 branch 5 times, most recently from 19954ee to 9817061 Compare January 27, 2026 21:32
@thilinarmtb

thilinarmtb commented Jan 29, 2026

Copy link
Copy Markdown
Contributor Author

@TApplencourt : Two tests fail out of the three tests used to check the toggle API. Those two fail
because we have not integrated the toggle plugin to babeltrace_thapi. Once we do that, I think
those two will pass as well.

@TApplencourt

Copy link
Copy Markdown
Collaborator

I can show you how to do it tomorrow : )

@thilinarmtb

Copy link
Copy Markdown
Contributor Author

I can show you how to do it tomorrow : )

Works for me.

@thilinarmtb
thilinarmtb force-pushed the thapi_on_and_off_v2 branch 9 times, most recently from 889b0d5 to 7ae17f3 Compare February 5, 2026 22:49
@thilinarmtb
thilinarmtb force-pushed the thapi_on_and_off_v2 branch 6 times, most recently from fa70166 to c4ebb89 Compare July 16, 2026 17:50
This is done by adding `__attribute((constructor))__` to
`thapi_stop()`. Added an autotools check to see if the
compiler supports `__attribute((constructor))__`.
and install `thapi_profiler.h` header files to provied declarations
of `thapi_start()` and `thapi_stop()`.
@thilinarmtb
thilinarmtb force-pushed the thapi_on_and_off_v2 branch from c4ebb89 to 547ce95 Compare July 16, 2026 18:13
@TApplencourt

Copy link
Copy Markdown
Collaborator

@thapi-local-reviewer review

@thapi-local-reviewer thapi-local-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Defects

1. push_downstream uses a global hostname instead of the per-event hostname

File: utils/thapi_toggle_callbacks.cpp, function push_downstream

auto key = ToggleKey{std::string(hostname_s), vpid_v};

hostname_s is a file-scope static that is only written by thapi_start_callback. In a multi-host MPI run (the test toggle_plugin_mpi_np_2 uses mpirun -n 2), processes on different hosts produce events with different hostnames. push_downstream is invoked for every downstream message, but it always looks up the toggle map with the hostname of the last thapi_start it saw, not the hostname of the event currently being filtered.

Trigger: Two MPI ranks on different hosts. Rank 0 on hostA calls thapi_start() (sets hostname_s = "hostA"). A regular API-call event from rank 1 on hostB then arrives at push_downstream; the key is constructed as ("hostA", vpid_rank1) instead of ("hostB", vpid_rank1), so the lookup either misses (event incorrectly dropped) or hits the wrong entry (event incorrectly passed).

The fix is to extract the hostname from the current event/stream context inside push_downstream rather than relying on the global.


2. No null-check on the vpid field before dereferencing

File: utils/thapi_toggle_callbacks.cpp, function push_downstream

const bt_field *ccf = bt_event_borrow_common_context_field_const(event);
const bt_field *vpid = bt_field_structure_borrow_member_field_by_name_const(ccf, "vpid");
uint64_t vpid_v = bt_field_integer_signed_get_value(vpid);

If the event's common context is absent (ccf == NULL) or does not contain a member named "vpid" (e.g., a stream whose context layout differs, or a malformed/legacy trace), vpid will be NULL and bt_field_integer_signed_get_value(NULL) is undefined behaviour (in practice a segfault in babeltrace2).

Trigger: Any event that flows through the filter whose common-context structure does not include a vpid member. The YAML in btx_thapi_toggle.yaml declares vpid in the context, but the filter sits after the muxer and sees events from all backends; a backend whose stream class omits vpid from its common context will hit this path.


3. Toggle granularity is per-process, not per-thread, despite the stated intent

File: utils/thapi_toggle_callbacks.cpp (all three callbacks and push_downstream)

The PR description says:

This API only affects the trace collection of the calling process/thread.

But every key is (hostname, vpid) — the virtual process id. vtid is received by the callbacks but never used in the key. If two threads in the same process call thapi_start() / thapi_stop() concurrently (or in sequence), the second thread's thapi_stop() will turn off tracing for the first thread as well, because they share the same vpid key.

Trigger: Thread A calls thapi_start(), thread B calls thapi_stop(). Thread A's subsequent API calls are silently dropped because the shared (hostname, vpid) entry is now false.

If per-process granularity is the intended design, the description should be corrected; if per-thread is intended, the key must include vtid.

Reviewed 2e63a99e — local model qwen3.8:27b. Verify before acting.

@TApplencourt

Copy link
Copy Markdown
Collaborator

sorry @thilinarmtb just playing with some bot :p

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants