Skip to content

feat(web-watcher): set audible on browser events from media sessions - #299

Draft
0xbrayo wants to merge 3 commits into
ActivityWatch:masterfrom
0xbrayo:feat/web-watcher-audible
Draft

0xbrayo wants to merge 3 commits into
ActivityWatch:masterfrom
0xbrayo:feat/web-watcher-audible

Conversation

@0xbrayo

@0xbrayo 0xbrayo commented Sep 20, 2026

Copy link
Copy Markdown
Member

WebWatcher hard-codes audible=false on every aw-watcher-android-web event (// TODO). aw-webui uses audible browser events as not-AFK evidence (include_audible in queries.ts), so the field was worthless on Android.

Changes

  • BrowserAudibleDetector — reports whether the browser package owns a media session in STATE_PLAYING via MediaSessionManager.getActiveSessions(). This is the precise per-app signal (Chrome/Firefox publish a media session for page audio/video) but needs the existing MediaWatcher notification-listener access. Without that access it falls back to AudioManager.isMusicActive(), which is global and can't tell browser audio from a background music app. Result is cached for 1s since accessibility events arrive many times a second while scrolling.
  • BrowserSessionTracker — carries audible per session and splits the session when it flips, so each logged event has the right value for its stretch of time (desktop aw-watcher-web likewise starts a new event when data changes). Url, browser and title carry over across the split.
  • WebWatcher — samples audible on every browser accessibility event and feeds it to the tracker.

Known limitations

  • incognito is still hard-coded to false; detecting it needs per-browser accessibility-tree heuristics and is out of scope here.
  • Audible is sampled on accessibility events, so if the user starts a video and then doesn't touch the screen, the split lands on the next event rather than at the exact play/pause moment.
  • Chrome only publishes a media session for media longer than ~5s, so short sound effects won't register (same as the notification-based MediaWatcher).

Addresses the audible half of #106.

Validation

:mobile:testStandardDebugUnitTest --tests 'net.activitywatch.android.watcher.*': 39 tests pass (4 new tracker tests, 2 new detector tests). Not yet exercised on a device — draft until I've checked Chrome + Firefox with and without notification access.

@greptile-apps

greptile-apps Bot commented Sep 20, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 5/5

[Medium impact] The reviewed changes appear safe to merge, with no blocking findings.

Summary

The PR records browser playback in audible and splits sessions when that value changes. The latest update prevents audio from one browser from changing another browser's session.

  • Browser events now record when their browser is playing media.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Browser accessibility event] --> B[Read URL and browser playback]
  B --> C{URL available?}
  C -->|Yes| D[Update URL and audible state]
  C -->|No| E{Browser matches current session?}
  E -->|No| F[Leave session unchanged]
  E -->|Yes| G[Update audible state]
  D --> H[Write completed session when state changes]
  G --> H
Loading

Reviews (4) · Last reviewed commit: "fix(web-watcher): apply audible changes ..." · Reviewed by Greptile

Comment thread mobile/src/main/java/net/activitywatch/android/watcher/BrowserAudibleDetector.kt Outdated
@0xbrayo

0xbrayo commented Sep 20, 2026

Copy link
Copy Markdown
Member Author

@greptile review

@ErikBjare

ErikBjare commented Sep 21, 2026 •

Copy link
Copy Markdown
Member

@0xbrayo I'm wondering if we should just keep a separate "audible" bucket and not make it browser-only. There is no "AFK" bucket/state on Android anyway to filter by, so it doesn't affect filtering.

@TimeToBuildBob

Copy link
Copy Markdown
Contributor

Agree — this does not change Android filtering. include_audible in aw-webui only unions browser_events.audible==true into not_afk under isDesktopParams. Android queries use bid_android / activityQueryAndroid and never load browser buckets.

The non-browser-only source already exists: aw-watcher-android-media (MediaWatcher, same getActiveSessions this PR samples, all apps). I would treat that as the audible bucket rather than copying the desktop audible field onto web events.

URL-level "this page was playing" is a join (web URL ∩ browser-package media session), not a reason to duplicate the flag. Filling the field is schema-complete for #106, but it is not the Android audible design.

WebWatcher hard-coded audible=false on every aw-watcher-android-web event.
aw-webui uses audible browser events as not-AFK evidence, so the field
was worthless on Android.

BrowserAudibleDetector reports whether the browser package owns a media
session in STATE_PLAYING via MediaSessionManager, which is the precise
per-app signal but needs the MediaWatcher notification-listener access.
Without that access it falls back to AudioManager.isMusicActive(), which
is global and can't tell browser audio from a background music app. The
result is cached for one second because accessibility events arrive many
times a second while scrolling.

BrowserSessionTracker now carries audible per session and splits the
session when it flips, so each logged event has the right value for its
stretch of time (the desktop web watcher likewise starts a new event
when data changes). Url, browser and title carry over across the split.

incognito is still hard-coded to false; detecting it needs per-browser
accessibility-tree heuristics and is left for a follow-up.
… fallback

handleUrl returned early when the url and browser were unchanged, so an
audible transition sampled on a same-page event was discarded and the
session logged with a stale value. Same-page calls now route through
handleAudible.

The AudioManager.isMusicActive() fallback is removed: it is device-wide,
so a background music app would have marked silent browser sessions
audible, which aw-webui counts as not-AFK evidence. Without notification
access audible is now always false, as before this change.
@0xbrayo
0xbrayo force-pushed the feat/web-watcher-audible branch from ba6bde0 to a8797a8 Compare October 11, 2026 19:28
@0xbrayo

0xbrayo commented Oct 11, 2026

Copy link
Copy Markdown
Member Author

🤖 Claude, on behalf of @0xbrayo

@greptile review

Comment thread mobile/src/main/java/net/activitywatch/android/watcher/WebWatcher.kt Outdated
… session

An event from a browser just switched to can arrive before its url is
extractable. Its audible state was applied to the previous browser's
session, so a silent page could be logged as playing. handleAudible now
takes the browser and ignores any other than the current session's.
@0xbrayo

0xbrayo commented Oct 11, 2026

Copy link
Copy Markdown
Member Author

🤖 Claude, on behalf of @0xbrayo

@greptile review

@TimeToBuildBob

Copy link
Copy Markdown
Contributor

Checked the new head f858af3: it addresses Greptile's cross-browser finding. WebWatcher passes the source browser through, and BrowserSessionTracker.handleAudible rejects updates from a different browser. The added regression test covers Firefox playback arriving while Chrome still owns the tracked URL.

This is a code inspection, not a local test run; CI and the new Greptile round are still running. The PR remains draft, with the Chrome/Firefox device checks still required. Erik's separate/all-app audible-bucket design question also remains for the author to settle; this correctness fix does not decide that direction.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants