Skip to content

Repository navigation

Immersive album art in the full player (mirror, extend or card), Android and desktop - #59

Merged
ingoau merged 22 commits into
mainfrom
claude/fullscreen-album-art-anb77k
Oct 10, 2026
Merged

Immersive album art in the full player (mirror, extend or card), Android and desktop#59
ingoau merged 22 commits into
mainfrom
claude/fullscreen-album-art-anb77k

Conversation

@ingoau

@ingoau ingoau commented Oct 10, 2026

Copy link
Copy Markdown
Owner

Full-bleed album art in the full player, like Apple Music and Cider, on Android and desktop. Each cover is carried on past its edge, by a reflection or by its own colours, or stays a card on a moving blurred background when that would look wrong. Which of the three is decided automatically, per cover, by one classifier in the Rust core.

How it decides

Style What it draws When
Mirror The art flipped past its edge, blurring more with distance. Nothing near the edge looks wrong flipped: no close-up face, no text or logo in the reflection zone, not too busy.
Extend The edge's colours carried on, softening with distance. An edge already in one colour continues with a crisp edge (a white cover stays white). A plain edge (the last row, or most of the edge strip, in one colour), or columns that each hold their colour (a horizon, a gradient).
Card The rounded art on its own moving blurred colours. Everything else: close-up portraits, busy covers, text across the bottom, non-square art.
  • hocket_core::artwork::layout decides from the cover's pixels (resampled to 128 px, in OKLab) plus the faces the platform found, for both edges at once: the bottom (phones, tall windows) and the right (wide windows). It takes about 7 ms per cover.
  • It also picks light or dark controls. They follow the app theme unless the generated part is almost all dark or light, and the core sizes a scrim for 4.5:1 contrast.
  • The art never cuts over to the generated part. Its last stretch fades onto a blurred copy of itself (with a vignette on desktop), and the continuation starts from that same blur. From the controls down, the player fades partway to black or white.
  • New setting: Display → Immersive artwork (Automatic / Always / Never).

Platforms

  • Core: new artwork module and API types (shared with the apps through the generated bindings); the display.immersiveArtwork setting; exposed to Android (UniFFI) and desktop (napi).
  • Android:
    • Full-bleed art in the Now Playing sheet, with faces from the framework FaceDetector.
    • The moving LyricsBackground behind card covers.
    • The art flying back to the mini player carries its fade, and its corners round to the thumbnail's.
  • Desktop:
    • The continuation is drawn on a canvas.
    • Wide windows (3:2 or wider) put the art at full height on the left, with the controls beside it. Tall windows (2:3 or narrower) put it full width on top. Shapes in between show the card.
    • On a track change, the art, its layout and the continuation switch together once the new cover is ready (the next track's is prepared ahead of time), and the whole picture crossfades.

Testing

  • cargo fmt, cargo clippy -D warnings, cargo test --workspace (746 passed, including 22 classifier tests).
  • Desktop: typecheck, lint, unit tests and build pass. End-to-end: 52 passed, and 2 failed that fail identically on main (see Known gaps).
  • Android: testDebugUnitTest assembleDebug :app:lintDebug pass. I couldn't cross-compile the native core here (no NDK), but the Rust it builds from is tested on the host.
  • Ran the classifier over a real 504-cover library and reviewed contact sheets of every cover, in dark and light themes:
    • Android: rendered with Robolectric.
    • Desktop: wide and tall windows, with Playwright.
    • Both screenshot tests are opt-in (ImmersiveScreenshotTest, e2e/immersive-screenshots.spec.ts). They run only when pointed at a local folder of covers, and no album art is committed.
  • scripts/artwork-eval.py runs the classifier over a folder of covers for tuning.
  • Not yet tried on a physical Android device.

Known gaps

  • The desktop has no face detection yet. On Android, the framework detector misses many faces, so some portraits mirror where a better detector would pick a card.
  • Over a full-width cover, the header can overlap text at the top of the art. That's left to the player redesign (Revamp mobile now playing #40, Revamp desktop now playing #47).
  • The desktop card background doesn't follow light mode yet. That code predates this branch.
  • e2e/keyboard.spec.ts ("space toggles play…") and e2e/undo.spec.ts ("replacing the queue shows a toast…") fail the same way on main in this environment: a shortcut pressed after clicking the content area does nothing. They aren't caused by this PR.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LDCDUaQsSzQpKRVfkiE1FK

Generated by Claude Code

claude added 22 commits October 10, 2026 00:23
@claude
core: immersive artwork classifier (mirror, extend or card per edge)
68b6045 
A pure function of a cover's pixels and the faces the platform found:
Mirror where a reflection shows nothing that looks wrong flipped (no
close-up faces, no text or logos near the edge, not too busy), Extend
where the edge is one colour or its columns are, Card otherwise. Both
edges (bottom for portrait, right for landscape), the colours to carry
on, and light/dark controls with the scrim they need for 4.5:1.

Tuned on 504 covers from a real library with scripts/artwork-eval.py
and the artwork_layout example.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LDCDUaQsSzQpKRVfkiE1FK
@claude
Immersive artwork: FFI, setting, and the desktop fullscreen player
ad15761 
- artwork_layout over UniFFI (Android) and napi (desktop): RGBA8 pixels
  and an ArtworkLayoutRequest in, an ArtworkLayout out.
- display.immersiveArtwork (automatic | always | never), device-local.
- Desktop: a wide window puts the artwork at its full height on the left
  and carries it on to the right; a tall one at its full width on top,
  carried on below; anything in between keeps the card. The continuation
  is drawn once per cover and window size on a canvas over the fluid
  background (reflection or the edge's colours, softening with distance),
  with the scrim the core sized and dark controls over light covers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LDCDUaQsSzQpKRVfkiE1FK
@claude
Immersive artwork: smooth seams, crisp flat edges, pure blacks count …
74d2cc4 
…as flat

- The seam no longer cuts over: the artwork fades out over its last 18%
  onto a blurred copy of itself, and the continuation starts at that same
  blur (Mirror) or bridges from a short blurred reflection into the edge
  colours (Extend). A flat extension skips it: the artwork already ends in
  that colour, so it stays crisp and the colour simply carries on.
- One-colour checks measure lightness through OKLab's toe: plain OKLab
  made sRGB 0..2 look noisy, so a circle on pure black was mirrored.
- docs/design.md: the immersive artwork decision.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LDCDUaQsSzQpKRVfkiE1FK
@claude
Android: immersive artwork in the full player
b4a8d4f 
- The core classifies each cover (decoded to 256 px, faces from the
  framework's FaceDetector) through ArtworkLayouts; cached per cover.
- Mirror or Extend: the artwork runs edge to edge from the top of the
  screen, square corners, no shadow, no paused shrink; below it the
  reflection or the edge's colours, with the seam softened (the artwork's
  last 18% fades onto a blurred copy of itself) except for a flat
  extension, which stays crisp. Dark controls over light continuations.
- The player's backdrop is the lyrics' moving blurred artwork in every
  mode (Android 13+), the still blur before. Its frame loop is an
  infinite animation, so test rules stop it instead of waiting on it.
- Settings > Appearance: Immersive artwork (Automatic, Always, Never).
- ImmersiveScreenshotTest: opt-in screenshots of the player with a
  folder of real covers and the host-built core.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LDCDUaQsSzQpKRVfkiE1FK
@claude
Android: import withInfiniteAnimationFrameNanos from animation.core
ba212bb 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LDCDUaQsSzQpKRVfkiE1FK
@claude
Immersive artwork: fade to black, light-player colours, seamless coll…
4f1ec31 
…apse, header on top

- Under light controls the continuation darkens to near black towards
  the bottom of the screen (Android and desktop).
- A light continuation (dark controls) uses the one dark the core checked
  contrast for, and translucent dark tonal buttons over the cover's colour.
- The artwork flying between the mini player and the full player carries
  the seam fade in proportion to how far open the sheet is, with its
  corners going from the thumbnail's to square: no snap on landing.
- The header is drawn over the immersive artwork (the page keeps its
  room), and artwork decodes at its requested size so a resize keeps the
  image instead of reloading it.
- The classifier skips the coarse text pass once the fine one has ruled
  a mirror out (same decisions, ~25% faster).
- ImmersiveScreenshotTest preloads covers and advances Robolectric's
  clock; SettingsCategoriesTest knows the new row.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LDCDUaQsSzQpKRVfkiE1FK
@claude
Immersive artwork: a one-colour bottom always extends; softer reflect…
259593d 
…ion under the title

- A bottom edge that is one colour (or just its last row) extends before
  a mirror is considered, so the choice no longer hangs on whether small
  text near the edge was detected (Halsey mirrored on Android, extended
  on desktop).
- Only a thin band at the seam stays near-sharp in a mirror, and the far
  blur is stronger: the title sits right under the artwork, on soft
  colour instead of a busy reflection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LDCDUaQsSzQpKRVfkiE1FK
@claude
Immersive artwork: no seam line, scrim at the seam, two-way bottom fa…
bbc7f04 
…de, header from the top

- Android draws the artwork and its reflection as one blurred image, so
  both sides of the seam come from one blur (two separately blurred
  copies each faded at their own edge: a dark line at the seam).
- The controls' scrim is at full strength where they start, right under
  the artwork: it comes in under the artwork's fading edge.
- Towards the bottom the continuation fades to near black under light
  controls and to near white under dark ones.
- The header over the artwork takes its colours from the artwork's top,
  with a soft dark scrim under light text; no more white haze over it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LDCDUaQsSzQpKRVfkiE1FK
@claude
Immersive artwork: a 60% bottom fade from the title, to the app theme…
14ac563 
…'s black or white

The fade starts where the title is (the seam) instead of halfway down,
stops at 60% instead of 85%, and goes to black in dark mode and white in
light mode (the app's theme, not the controls'). ImmersiveScreenshotTest
takes HOCKET_SCREENSHOT_THEME=light for light-mode renders.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LDCDUaQsSzQpKRVfkiE1FK
@claude
Immersive artwork: controls follow the theme unless the cover is almo…
0684c1d 
…st all dark or light

The request carries the app's theme. Over a mixed cover the controls
follow it (light in dark mode, dark in light mode) and the core sizes the
scrim for them; a cover that is at least 80% very dark keeps light
controls, and one at least 80% very light keeps dark controls, in either
theme (a white cover stays white, a black one black). The bottom fade
goes to the controls' opposite, so they always stand out. Metrics carry
the dark and light shares.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LDCDUaQsSzQpKRVfkiE1FK
@claude
Immersive artwork: the controls follow what sits behind them, not the…
b89deac 
… whole cover

The light/dark override now looks at the continuation (the edge colours
an extension carries on, or the lower part a mirror reflects): when that
is almost all very light, dark controls and a fade to white; almost all
very dark, light controls and a fade to black; else the theme. A big
subject on a white cover no longer greys its white continuation in dark
mode (27 covers of a 504-cover library change in dark mode, all light
extensions).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LDCDUaQsSzQpKRVfkiE1FK
@claude
ImmersiveScreenshotTest: placeholder song, artist and album names
d404e39 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LDCDUaQsSzQpKRVfkiE1FK
@claude
Desktop: screenshot the fullscreen player with real covers
e027084 
- With HOCKET_FAKE_COVERS naming a folder of covers, the fake core serves
  them as album artwork (copied into its image cache, the only place the
  artwork protocol serves from), and main loads the addon for its
  immersive-artwork classifier even under the fake core.
- e2e/immersive-screenshots.spec.ts (skipped unless HOCKET_FAKE_COVERS and
  HOCKET_SCREENSHOT_DIR are set) plays each album, opens the fullscreen
  player and shoots it in a wide and a tall window, with placeholder names.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LDCDUaQsSzQpKRVfkiE1FK
@claude
Desktop: legible controls over a light continuation; screenshot spec …
14973f1 
…hides toasts, takes a theme

Over a light immersive continuation the fullscreen player's icons, skip
buttons, tonal and mode pills and the seek bar were white on white: they
now take the dark ink, mirroring their light-on-dark styles.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LDCDUaQsSzQpKRVfkiE1FK
@claude
Make the immersive screenshot run resumable and drop its trace
d321492 
A run over a whole library outgrew the retained trace and died partway;
covers already shot are now skipped, and a dead app is relaunched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LDCDUaQsSzQpKRVfkiE1FK
@claude
Smooth the desktop seam between the artwork and its continuation
dd69744 
The bottom fade beside a full-height artwork started hard at its edge;
it now comes in under the artwork's fading edge. The artwork's feather,
the scrim and the bottom fade follow a smoothstep instead of a straight
ramp, whose ends read as edges, and beside the artwork the scrim reaches
full strength a little past the seam, before the controls start.

The screenshot run waits for a new continuation's fade-in before
shooting, and gives up only when relaunches stop making progress.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LDCDUaQsSzQpKRVfkiE1FK
@claude
Smoother desktop continuation: wider blur, vignette, fade over the co…
9be3ffa 
…ntrols, no seam

The seam beside a full-height artwork was the blurred copy coming back
from the canvas blur a few percent see-through: under the artwork the
background showed through it, past the seam the farther blurs beneath
made it opaque. The blur is now drawn twice, which makes it opaque
without changing its colour, and the blurred copy runs under the whole
artwork and on past the seam as one image.

The artwork fades over its last 30% instead of 18%, with a vignette on
that side so the fade comes in sooner towards the corners rather than
along a straight line, onto a stronger blur (a thin dark strip along a
cover's edge no longer doubles into a band with its own reflection),
with a middle blur between the near and far ones. The fade towards the
bottom of the window now covers all of the controls: it comes in just
above them and is at full strength by their bottom.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LDCDUaQsSzQpKRVfkiE1FK
@claude
Cut crisp only into truly one-colour edges; fade plain but textured o…
47db033 
…nes in

A plain edge strip (85% one colour) carried on in one flat colour with
a crisp edge, which shows where low-contrast texture stops: dark ground,
grain, a gradient. Crisp now needs the outermost row in one colour or
95% of the strip; between that and 85% the edge still extends
(extend:plain), in its own, much smoothed column colours, faded in like
any other extension. Over the library that is 21 bottom edges and 17
right ones of 504 covers.

The apps take crispness from the core's reason (extend:flat) rather
than from the edge being one colour, which an evenly textured edge also
smooths into.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LDCDUaQsSzQpKRVfkiE1FK
@claude
Screenshot run: wait for every finite animation, not just the continu…
cc248c4 
…ation's

The player's crossfade to the next track could still be running when a
shot was taken, leaving its controls and continuation half faded in.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LDCDUaQsSzQpKRVfkiE1FK
@claude
Desktop fullscreen: switch artwork, layout and continuation together,…
51e10a6 
… crossfaded

On a track change the artwork swapped at once (through a placeholder),
the player fell back to a card until the new cover was classified, then
snapped to full bleed while the continuation faded in on its own over
the background, and the controls changed colour on their own time: for
half a second a new cover sat beside a grey, half-faded continuation.

Each cover now has a scene (the artwork decoded at full size, and its
layout), prepared once and cached, the next track's ahead of time. The
player switches to a cover only when its scene is ready, so the artwork,
the layout and the continuation (drawn before paint) change in one
commit, and the new scene crossfades in over the last; when the two are
laid out differently (a card and an immersive cover), the new one fades
in from the background on its own.

The screenshot run waits for the playing track's own scene to be on
screen and settled, and only for the player's animations (lyrics
animate for as long as a line lasts).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LDCDUaQsSzQpKRVfkiE1FK
@claude
Merge remote-tracking branch 'origin/main' into claude/fullscreen-alb…
cb768fb 
…um-art-anb77k
@claude
docs: immersive artwork as it ended up
cfd783f 
The soft seam (and where an extension is crisp instead), the controls
following the theme unless the continuation is almost all dark or
light, the fade over the controls, and how a track change and the
collapse to the mini player keep the artwork and its continuation
together.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LDCDUaQsSzQpKRVfkiE1FK
@ingoau
ingoau merged commit e19e79c into main Oct 10, 2026
4 checks passed
@ingoau ingoau mentioned this pull request Oct 10, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@ingoau @claude