diff --git a/.llms/learnings.md b/.llms/learnings.md index 9443e174..3c25c157 100644 --- a/.llms/learnings.md +++ b/.llms/learnings.md @@ -9,6 +9,16 @@ Format: brief heading + explanation + (optional) relevant file paths. +## Comment style: keep them on definitions, not inside logic + +This repo prefers minimal comments. Write them on **function, prop, or data-structure +definitions** (docstrings / JSDoc, a one-liner over a type field, an interface member) — +they describe intent and behavior that outlives the implementation. Avoid comments +*inside* function bodies that narrate the process step-by-step, restate the code, or +tag "Phase N"/PR history; that knowledge belongs in the definition's summary, the +relevant skill (e.g. the pyodide skill), or git history. A component's top-of-file +comment should concisely state what it is and does — not a diagram of every branch. + ## Markers: device-agnostic injection via the EEGDriver interface Marker injection used to be Muse-only and lived in the UI (`RunComponent.eventCallback` diff --git a/TODOS.md b/TODOS.md index 1896c35a..168dd44e 100644 --- a/TODOS.md +++ b/TODOS.md @@ -15,12 +15,15 @@ Deferred and in-flight work. Keep this current — when something ships, delete - [ ] External / generic LSL stream support (any EEG device). - [ ] Lab.js cleanup — remove jspsych, type lab.js data. Not on the content critical path; users never see it. - [ ] Lesson surface beyond markdown — block-based programming, embedded notebooks (the CLAUDE.md extensibility horizon). +- [ ] **Epoch reviewer Phase 3 — onboarding layer.** Plan: `docs/epoch-review-ui-plan.md` §9. Plain-language explanations of epochs + each artifact type, a **guided mode** (step through auto-flagged epochs with "why we flagged this," student confirms/overrides), channel legend tied to head position (Muse 10-20), student-facing tone. Builds on the Phase 0-2 reviewer (PRs #223/#224/#225). **Open question OQ3 (onboarding depth) is still unresolved** — how much curriculum (tooltips only vs. a real walkthrough), guided-mode-as-default? This is product-shaped, not architecture — take it through `/office-hours` or `/plan-ceo-review` before forging. Note it overlaps the "in-app lesson surface" critical-path work above; sequence deliberately. +- [ ] **Epoch reviewer Phase 4 — polish & generalize.** Plan: `docs/epoch-review-ui-plan.md` §9. N-channel devices (Neurosity 8-ch, external LSL 32-64-ch) — the renderer already windows/downsamples but needs real testing at scale and possibly the WebGL swap behind `drawEpochs` (OQ1 left that a later swap). Accessibility, keyboard-first flow, performance. Also OQ7 (keep a read-only static-SVG fallback for environments where the interactive UI can't run?) is still open. ## Known issues / tech debt - [ ] **(Optional) Full Pyodide worker RPC** — the analysis/Clean pipeline crash is now **fixed** (harvested from PR #194): a `dataKey` routing pattern parallel to `plotKey` — the worker echoes `dataKey` + PyProxy-converted results, and `pyodideMessageEpic` routes `epochsInfo`→`SetEpochInfo` / `channelInfo`→`SetChannelInfo`; the info epics are fire-and-forget. This unblocks the pipeline without the bigger refactor. The deeper latent issue remains, though: `worker.postMessage` returns `undefined` on *post*, so the `await`s in `webworker/index.ts` are no-ops and cross-message sequencing still relies on worker FIFO. A true `runPython(worker, code, ctx?)` RPC — `Map` + one `message` listener, worker echoes `id` — would let epics `await` real results and delete the `plotKey`/`dataKey` switch entirely. Only worth doing if the FIFO sequencing ever actually bites; not urgent now. - [ ] Pyodide-fidelity smoke test — analysis pipeline is tested against native MNE, not yet under Pyodide/WASM (see `.llms/learnings.md`). **In progress:** the epoch-review Phase 0 (see `docs/epoch-review-ui-plan.md` §0) adds a *narrow* Pyodide test for the `get_epochs_arrays` float32 buffer path (byteLength, decode-vs-native, transfer detaches source); the full-pipeline Pyodide job remains deferred. - [ ] Pre-existing TypeScript errors (not regressions): `experimentEpics.ts` (RxJS operator types), `routes.tsx` (Redux container prop types). +- [ ] **Epoch reviewer Phase 2 polish (from PR #225 review).** Three non-blocking behaviors flagged during the Phase 2 forge: (1) `get_epochs_arrays`/`suggest_rejections` use `pick_types(eeg=True)` whose MNE default is `exclude='bads'`, so after a Clean that flags a bad channel the re-fetched reviewer omits that channel from the display (saved `.fif` is unaffected) — decide whether to keep bad channels visible-but-greyed (`exclude=[]`) instead of vanishing; (2) re-running "Auto-flag" with the same threshold re-adds suggestions the user had manually unclicked (additive union merge in `CleanComponent.componentDidUpdate`); (3) the auto-flag threshold `` has no min guard (0 µV flags everything). All in `src/renderer/components/CleanComponent/` + `webworker/utils.py`. ## Done recently diff --git a/docs/epoch-review-ui-plan.md b/docs/epoch-review-ui-plan.md index 43d4ee95..f1bd788f 100644 --- a/docs/epoch-review-ui-plan.md +++ b/docs/epoch-review-ui-plan.md @@ -45,9 +45,32 @@ temper→forge; Phases 2–4 remain roadmap. - **Delivery → two sequential PRs:** Phase 0 (transport + static render), then Phase 1 (interaction + apply + live ERP). +### Phase 2 decisions (2026-07-07) + +- **OQ4 (auto-flag UX) → one-click button + expandable settings, dev-gated.** An + "Auto-flag artifacts" button a newcomer can just press; an expandable settings + panel exposes the peak-to-peak threshold(s) for users who want to control/learn + more. The button is **dev-configurable per experiment** (a constant set keyed by + the `EXPERIMENTS` enum, gated on `CleanComponent`'s `type` prop) so devs opt + experiments in/out. +- **New-A (where auto-flag runs) → Python/MNE.** A worker round-trip + `suggest_rejections(epochs, threshold)` computes peak-to-peak per epoch in Python + and returns suggested indices + reasons (new `suggestedRejections` dataKey, + extends the OQ2 pattern). Suggestions **pre-mark** epochs in the reviewer and are + fully **overridable**; the actual drop still goes through the Phase-1 + `apply_rejection` path so the saved `.fif` stays MNE-exact. +- **OQ5 (bad channels on low-channel devices) → enabled everywhere, with a guard.** + Click a channel label to toggle it bad; bad channels flow to `apply_rejection`'s + `bad_channels` param (already built in Phase 1). On a 4-channel dataset, a + **warning Dialog** appears when the user marks a 2nd bad channel, nudging them to + re-collect if signal quality is that poor (informational — they can proceed). +- **New-B (condition legend) → human-readable labels via `codeToLabel`.** Plumb the + marker registry's `codeToLabel` (built from `state.experiment.params.stimuli`) + into the reviewer + Live ERP legends instead of raw numeric codes. +- **Delivery → one PR (Phase 2)**, stacked on Phase 1. + Remaining open (not forge-blocking; decide when their phase comes): OQ3 (onboarding -depth), OQ4 (auto-reject UX/thresholds), OQ5 (bad channels on 4-ch Muse), OQ7 -(static fallback). +depth, Phase 3), OQ7 (static fallback). --- @@ -342,10 +365,11 @@ Traced end to end against `webworker/`, `src/main/index.ts`, `src/preload/index. `runPython` RPC (reinforced by OQ6's client-side live ERP needing no round-trip). 3. **Onboarding depth** — Is guided mode the default? How much curriculum (tooltips only vs. a real walkthrough)? -4. **Auto-rejection** — Expose peak-to-peak thresholds to the user, or keep them - as invisible "suggestions"? What defaults? -5. **Bad channels on Muse** — dropping 1 of 4 channels is drastic; do we support - channel rejection for low-channel devices, or epochs-only there? +4. **Auto-rejection — RESOLVED (§0, Phase 2): one-click "Auto-flag" button + + expandable threshold settings, dev-gated per experiment; suggestions computed in + Python (peak-to-peak) and pre-marked but overridable.** +5. **Bad channels on Muse — RESOLVED (§0, Phase 2): enabled on all devices; a + warning Dialog fires when marking a 2nd bad channel on a 4-channel dataset.** 6. **Live ERP preview — RESOLVED (§0): in scope for v1 (Phase 1), computed client-side** over the Phase-0 buffer (Flavor 1 only; real-time-during-experiment is a separate future non-Python effort). @@ -396,8 +420,15 @@ Traced end to end against `webworker/`, `src/main/index.ts`, `src/preload/index. - Reaches functional parity-plus with the *essential* MNE workflow. - **Note:** the write-back bridge (§6d) already landed standalone in PR #222, so Phase 1 just points the apply action at a bridge that works. -- **Phase 2 — Full parity.** Bad-channel flagging, condition coloring/legend, - auto-flag suggestions from `drop_log`/peak-to-peak. +- **Phase 2 — Full parity (PR 4). LOCKED (§0 Phase 2 decisions).** + - **Bad-channel flagging:** click a channel label → toggle bad; bad channels flow + to the existing `apply_rejection` `bad_channels` param. Warning Dialog on the 2nd + bad channel of a 4-ch dataset (OQ5). + - **Auto-flag (OQ4 + New-A):** dev-gated "Auto-flag artifacts" button + expandable + threshold settings → `suggest_rejections(epochs, threshold)` in Python (peak-to- + peak, native-testable) → `suggestedRejections` dataKey → pre-marks epochs in the + reviewer (overridable) with reasons shown. + - **Condition legend (New-B):** real labels via `codeToLabel`. - **Phase 3 — Onboarding layer.** Explanations, guided mode, artifact tutorials, live ERP preview. - **Phase 4 — Polish & generalize.** N-channel devices (Neurosity/LSL), diff --git a/src/renderer/actions/pyodideActions.ts b/src/renderer/actions/pyodideActions.ts index 41f2d276..04b41d1c 100644 --- a/src/renderer/actions/pyodideActions.ts +++ b/src/renderer/actions/pyodideActions.ts @@ -14,6 +14,15 @@ export interface EpochArraysMeta { drop_log: string[][]; } +// Auto-flag: a single artifact suggestion returned by Python's +// suggest_rejections(epochs, threshold_uv) — one epoch/channel over threshold. +export interface SuggestedRejection { + index: number; + channel: string; + peak_uv: number; + reason: string; +} + // ------------------------------------------------------------------------- // Actions @@ -56,6 +65,13 @@ export const PyodideActions = { // eslint-disable-next-line @typescript-eslint/no-explicit-any ReceiveError: createAction('RECEIVE_ERROR'), // Worker error event — shape is dynamic SetWorkerReady: createAction('SET_WORKER_READY'), + GetSuggestedRejections: createAction( + 'GET_SUGGESTED_REJECTIONS' + ), + SetSuggestedRejections: createAction< + SuggestedRejection[], + 'SET_SUGGESTED_REJECTIONS' + >('SET_SUGGESTED_REJECTIONS'), } as const; export type PyodideActionType = ActionType< diff --git a/src/renderer/components/CleanComponent/EpochReviewer.tsx b/src/renderer/components/CleanComponent/EpochReviewer.tsx index 06184e6b..ea5bc2b8 100644 --- a/src/renderer/components/CleanComponent/EpochReviewer.tsx +++ b/src/renderer/components/CleanComponent/EpochReviewer.tsx @@ -8,26 +8,10 @@ import { epochChannelSeries, } from './epochArrays'; -// --------------------------------------------------------------------------- -// Canvas layout (matches MNE's epochs.plot): epochs run ACROSS (x), channels -// are STACKED vertically (y). One trace per (epoch, channel) cell. -// -// [◀ Prev] [Next ▶] [amp －] [amp ＋] -// epoch 0 epoch 1 epoch 2 ... -// ┌────────────┬────────────┬────────────┐ -// ch 0 │ ~~~~~~~~ │ ░░grey░░✕░░ │ ~~~~~~~~ │ channel lane -// ├────────────┼────────────┼────────────┤ (rejected column -// ch 1 │ ~~~~~~~~ │ ░░░░░░░░░░ │ ~~~~~~~~ │ is greyed out) -// ├────────────┼────────────┼────────────┤ -// ch 2 │ ~~~~~~~~ │ ░░░░░░░░░░ │ ~~~~~~~~ │ -// └────────────┴────────────┴────────────┘ -// epoch 0 epoch 1 epoch 2 (bottom index labels) -// -// Interactive (Phase 1): a transparent DOM overlay div per visible column makes -// each epoch click-to-reject (rejected epochs grey out); Prev/Next page through -// all epochs; amp ＋/－ scale trace amplitude. Rendering stays Canvas 2D + a DOM -// overlay for labels/hit-targets (no canvas hit-testing). -// --------------------------------------------------------------------------- +// Interactive epoch reviewer: epochs run across (x), channels stacked (y). +// Click an epoch column to reject it; click a channel label to flag it bad +// (its lane washes red across all epochs). Prev/Next paginate, amp +/- scales +// traces. Canvas 2D for traces, a DOM overlay for labels/click targets. interface Props { epochArrays: { buffer: ArrayBuffer; meta: EpochArraysMeta } | null; @@ -35,6 +19,12 @@ interface Props { rejected: Set; // Toggle a single ABSOLUTE epoch index in/out of the rejected set. onToggleEpoch: (index: number) => void; + // Channel names the student has flagged as "bad" (controlled by the parent). + badChannels: Set; + // Toggle a single channel name in/out of the bad-channel set. + onToggleChannel: (name: string) => void; + // Optional map from numeric event code to a human-readable condition label. + codeToLabel?: Record; } // Logical canvas size (scaled up for devicePixelRatio at draw time). @@ -56,11 +46,15 @@ const GAIN_MAX = 20; const REJECTED_TRACE_COLOR = 'rgba(120, 120, 120, 0.5)'; const REJECTED_FILL_COLOR = 'rgba(120, 120, 120, 0.15)'; +const BAD_CHANNEL_FILL_COLOR = 'rgba(200, 60, 60, 0.10)'; export default function EpochReviewer({ epochArrays, rejected, onToggleEpoch, + badChannels, + onToggleChannel, + codeToLabel, }: Props): JSX.Element { const canvasRef = useRef(null); // First epoch of the current page (absolute index). @@ -100,7 +94,7 @@ export default function EpochReviewer({ ctx.clearRect(0, 0, CANVAS_WIDTH, CANVAS_HEIGHT); const { buffer } = epochArrays; - const { n_epochs, n_channels, n_times, event_codes } = meta; + const { n_epochs, n_channels, n_times, event_codes, ch_names } = meta; const plotWidth = CANVAS_WIDTH - LABEL_GUTTER; const plotHeight = CANVAS_HEIGHT - BOTTOM_GUTTER; @@ -120,6 +114,20 @@ export default function EpochReviewer({ } } + // Translucent red wash over flagged bad-channel lanes — spans every epoch + // column (drawn under traces) so bad channels read at a glance. + for (let ch = 0; ch < n_channels; ch += 1) { + if (badChannels.has(ch_names[ch])) { + ctx.fillStyle = BAD_CHANNEL_FILL_COLOR; + ctx.fillRect( + LABEL_GUTTER, + ch * laneHeight, + CANVAS_WIDTH - LABEL_GUTTER, + laneHeight + ); + } + } + // Faint lane dividers (channels). ctx.strokeStyle = 'rgba(0, 0, 0, 0.08)'; ctx.lineWidth = 1; @@ -235,7 +243,7 @@ export default function EpochReviewer({ ctx.stroke(); } } - }, [epochArrays, meta, rejected, clampedStart, perPage, gain]); + }, [epochArrays, meta, rejected, clampedStart, perPage, gain, badChannels]); // Empty state — friendly, brand-styled, student-facing. if (!epochArrays || !meta || meta.n_epochs === 0) { @@ -252,6 +260,10 @@ export default function EpochReviewer({ const firstShown = clampedStart + 1; const lastShown = clampedStart + visibleCount; + const uniqueSortedCodes = [...new Set(meta.event_codes)].sort( + (a, b) => a - b + ); + return (
@@ -306,22 +318,41 @@ export default function EpochReviewer({ style={{ width: CANVAS_WIDTH, height: CANVAS_HEIGHT }} /> - {/* Channel labels (left gutter). */} - {meta.ch_names.map((name, ch) => ( -
- {name} -
- ))} + {/* Channel labels (left gutter) double as click-to-flag bad-channel + toggles. A flagged channel renders struck-through + red and its lane + is washed red across every epoch column (see canvas draw). */} + {meta.ch_names.map((name, ch) => { + const isBad = badChannels.has(name); + return ( +
onToggleChannel(name)} + onKeyDown={(e) => { + if (e.key === 'Enter' || e.key === ' ') { + e.preventDefault(); + onToggleChannel(name); + } + }} + > + {name} +
+ ); + })} {/* Transparent click targets — one per visible epoch column. Clicking toggles that ABSOLUTE epoch index in/out of the rejected set. */} @@ -375,6 +406,28 @@ export default function EpochReviewer({ })}
+ {/* Condition legend: one swatch + human-readable label per unique code. */} + {uniqueSortedCodes.length > 0 && ( +
+ {uniqueSortedCodes.map((code) => ( + + + {codeToLabel?.[code] ?? `Condition ${code}`} + + ))} +
+ )} +

showing {firstShown}–{lastShown} of {meta.n_epochs} epochs {rejected.size > 0 && ` · ${rejected.size} marked for rejection`} diff --git a/src/renderer/components/CleanComponent/LiveErpPane.tsx b/src/renderer/components/CleanComponent/LiveErpPane.tsx index 0c95ddbb..b336d5a2 100644 --- a/src/renderer/components/CleanComponent/LiveErpPane.tsx +++ b/src/renderer/components/CleanComponent/LiveErpPane.tsx @@ -24,6 +24,8 @@ interface Props { epochArrays: { buffer: ArrayBuffer; meta: EpochArraysMeta } | null; // ABSOLUTE epoch indices marked for rejection (excluded from the average). rejected: Set; + // Optional map from numeric event code to a human-readable condition label. + codeToLabel?: Record; } const CANVAS_WIDTH = 640; @@ -36,6 +38,7 @@ const PAD_BOTTOM = 12; export default function LiveErpPane({ epochArrays, rejected, + codeToLabel, }: Props): JSX.Element { const canvasRef = useRef(null); const [channel, setChannel] = useState(0); @@ -249,7 +252,7 @@ export default function LiveErpPane({ ), }} /> - Condition {code} ({count}) + {codeToLabel?.[code] ?? `Condition ${code}`} ({count}) ); })} diff --git a/src/renderer/components/CleanComponent/__tests__/EpochReviewer.test.tsx b/src/renderer/components/CleanComponent/__tests__/EpochReviewer.test.tsx index df6f674b..301f9445 100644 --- a/src/renderer/components/CleanComponent/__tests__/EpochReviewer.test.tsx +++ b/src/renderer/components/CleanComponent/__tests__/EpochReviewer.test.tsx @@ -43,6 +43,8 @@ describe('EpochReviewer', () => { epochArrays={makeEpochArrays()} rejected={new Set()} onToggleEpoch={onToggleEpoch} + badChannels={new Set()} + onToggleChannel={vi.fn()} /> ); @@ -61,6 +63,8 @@ describe('EpochReviewer', () => { epochArrays={makeEpochArrays()} rejected={new Set([0])} onToggleEpoch={onToggleEpoch} + badChannels={new Set()} + onToggleChannel={vi.fn()} /> ); diff --git a/src/renderer/components/CleanComponent/index.tsx b/src/renderer/components/CleanComponent/index.tsx index 7abd4638..e338358d 100644 --- a/src/renderer/components/CleanComponent/index.tsx +++ b/src/renderer/components/CleanComponent/index.tsx @@ -1,9 +1,22 @@ import React, { Component } from 'react'; import path from 'pathe'; import { Link } from 'react-router-dom'; -import { isNil, isString } from 'lodash'; +import { isNil, isString, memoize } from 'lodash'; import { Button } from '../ui/button'; -import { EXPERIMENTS, DEVICES } from '../../constants/constants'; +import { + Dialog, + DialogContent, + DialogHeader, + DialogTitle, + DialogDescription, +} from '../ui/dialog'; +import { + EXPERIMENTS, + DEVICES, + DEFAULT_PTP_THRESHOLD_UV, +} from '../../constants/constants'; +import { ExperimentParameters } from '../../constants/interfaces'; +import { buildMarkerRegistry } from '../../utils/eeg/markerRegistry'; import { readWorkspaceRawEEGData } from '../../utils/filesystem/storage'; import CleanSidebar from './CleanSidebar'; import EpochReviewer from './EpochReviewer'; @@ -12,8 +25,15 @@ import { PyodideActions, ExperimentActions, EpochArraysMeta, + SuggestedRejection, } from '../../actions'; +// Memoized by stimuli reference so we don't rebuild the registry every render. +const codeToLabelFor = memoize( + (stimuli: ExperimentParameters['stimuli']) => + buildMarkerRegistry(stimuli).codeToLabel +); + export interface Props { type?: EXPERIMENTS; title: string; @@ -26,6 +46,8 @@ export interface Props { ExperimentActions: typeof ExperimentActions; subject: string; session: number; + params: ExperimentParameters | null; + suggestedRejections: SuggestedRejection[]; } interface DropdownOption { @@ -42,6 +64,14 @@ interface State { isSidebarVisible: boolean; // ABSOLUTE epoch indices the student has marked for rejection. rejectedEpochs: Set; + // Channel names the student has flagged as bad (dropped across all epochs). + badChannels: Set; + // Peak-to-peak threshold (µV) used by the auto-flag request. + autoFlagThreshold: number; + // Whether the auto-flag threshold settings panel is open. + showAutoFlagSettings: boolean; + // Whether the "too many bad channels" warning dialog is open. + showChannelWarning: boolean; } export default class Clean extends Component { @@ -56,12 +86,19 @@ export default class Clean extends Component { selectedSubject: props.subject, isSidebarVisible: false, rejectedEpochs: new Set(), + badChannels: new Set(), + autoFlagThreshold: DEFAULT_PTP_THRESHOLD_UV, + showAutoFlagSettings: false, + showChannelWarning: false, }; this.handleRecordingChange = this.handleRecordingChange.bind(this); this.handleLoadData = this.handleLoadData.bind(this); this.handleSidebarToggle = this.handleSidebarToggle.bind(this); this.handleSubjectChange = this.handleSubjectChange.bind(this); this.handleToggleEpoch = this.handleToggleEpoch.bind(this); + this.handleToggleChannel = this.handleToggleChannel.bind(this); + this.handleAutoFlag = this.handleAutoFlag.bind(this); + this.handleThresholdChange = this.handleThresholdChange.bind(this); this.icons = props.type === EXPERIMENTS.N170 ? ['😊', '🏠', '✕', '📖'] @@ -107,8 +144,22 @@ export default class Clean extends Component { handleLoadData() { this.props.ExperimentActions.SetSubject(this.state.selectedSubject); this.props.PyodideActions.LoadEpochs(this.state.selectedFilePaths); - // A fresh dataset invalidates any previously selected epoch indices. - this.setState({ rejectedEpochs: new Set() }); + // A fresh dataset invalidates any previously selected epoch indices and + // bad-channel selections. + this.setState({ rejectedEpochs: new Set(), badChannels: new Set() }); + } + + componentDidUpdate(prevProps: Props) { + if (prevProps.suggestedRejections !== this.props.suggestedRejections) { + const suggested = this.props.suggestedRejections; + if (suggested.length > 0) { + this.setState((prev) => { + const next = new Set(prev.rejectedEpochs); + for (const s of suggested) next.add(s.index); + return { rejectedEpochs: next }; + }); + } + } } handleToggleEpoch(index: number) { @@ -123,6 +174,39 @@ export default class Clean extends Component { }); } + handleToggleChannel(name: string) { + this.setState((prev) => { + const next = new Set(prev.badChannels); + const adding = !next.has(name); + if (adding) { + next.add(name); + } else { + next.delete(name); + } + const nCh = this.props.epochArrays?.meta.n_channels ?? 0; + // Dropping >1 of a 4-channel (Muse) recording loses a lot of signal — + // warn (informational; they can still proceed). + const warn = adding && next.size > 1 && nCh === 4; + return { + badChannels: next, + showChannelWarning: warn || prev.showChannelWarning, + }; + }); + } + + handleAutoFlag() { + this.props.PyodideActions.GetSuggestedRejections( + this.state.autoFlagThreshold + ); + } + + handleThresholdChange(e: React.ChangeEvent) { + const parsed = parseFloat(e.target.value); + if (!Number.isNaN(parsed)) { + this.setState({ autoFlagThreshold: parsed }); + } + } + handleSidebarToggle() { this.setState({ isSidebarVisible: !this.state.isSidebarVisible }); } @@ -169,6 +253,10 @@ export default class Clean extends Component { return this.state.selectedSubject === subjectFromFilepath; }); + const codeToLabel = codeToLabelFor(this.props.params?.stimuli); + const showAutoFlag = !this.props.params?.hideAutoFlagEpochs; + const { suggestedRejections } = this.props; + return (

{this.state.isSidebarVisible && ( @@ -227,16 +315,75 @@ export default class Clean extends Component { onClick={() => { this.props.PyodideActions.CleanEpochs({ dropIndices: Array.from(this.state.rejectedEpochs), - badChannels: [], + badChannels: Array.from(this.state.badChannels), }); // After Clean, raw_epochs is re-fetched with fewer epochs, // so the old absolute indices no longer apply. - this.setState({ rejectedEpochs: new Set() }); + this.setState({ + rejectedEpochs: new Set(), + badChannels: new Set(), + }); }} > Clean Data
+ {showAutoFlag && ( +
+
+ + +
+ {this.state.showAutoFlagSettings && ( +
+ +

+ Flag epochs whose peak-to-peak amplitude exceeds this. + Higher = fewer flags. +

+
+ )} + {suggestedRejections.length > 0 && ( +
+

+ Flagged {suggestedRejections.length}{' '} + {suggestedRejections.length === 1 ? 'epoch' : 'epochs'} +

+
    + {suggestedRejections.slice(0, 3).map((s, i) => ( +
  • + {s.reason} +
    • + ))} +
+
+ )} +
+ )}
{this.renderEpochLabels()} @@ -248,13 +395,40 @@ export default class Clean extends Component { epochArrays={this.props.epochArrays} rejected={this.state.rejectedEpochs} onToggleEpoch={this.handleToggleEpoch} + badChannels={this.state.badChannels} + onToggleChannel={this.handleToggleChannel} + codeToLabel={codeToLabel} />
+ this.setState({ showChannelWarning: o })} + > + + + That's a lot of channels to drop + + You've marked more than one bad channel on a 4-channel + recording. That removes a big chunk of your data — if the signal + is really this noisy, consider collecting another dataset. + + +
+ +
+ + ); } diff --git a/src/renderer/constants/constants.ts b/src/renderer/constants/constants.ts index f2e32f8d..e30d9614 100644 --- a/src/renderer/constants/constants.ts +++ b/src/renderer/constants/constants.ts @@ -9,6 +9,8 @@ export enum EXPERIMENTS { // SSVEP = 'Steady-state Visual Evoked Potential', } +export const DEFAULT_PTP_THRESHOLD_UV = 100; + export const SCREENS = { HOME: { route: '/', title: 'HOME', order: 0 }, BANK: { route: '/home', title: 'HOME', order: 0 }, diff --git a/src/renderer/constants/interfaces.ts b/src/renderer/constants/interfaces.ts index a64bfa9c..7ce9b7da 100644 --- a/src/renderer/constants/interfaces.ts +++ b/src/renderer/constants/interfaces.ts @@ -31,6 +31,7 @@ export type ExperimentParameters = { sampleType: string; selfPaced?: boolean; showProgressBar: boolean; + hideAutoFlagEpochs?: boolean; stimuli?: Stimulus[]; taskHelp?: string; trialDuration: number; diff --git a/src/renderer/containers/CleanContainer.ts b/src/renderer/containers/CleanContainer.ts index 33cd55af..8d03efab 100644 --- a/src/renderer/containers/CleanContainer.ts +++ b/src/renderer/containers/CleanContainer.ts @@ -11,6 +11,7 @@ function mapStateToProps(state: RootState) { subject: state.experiment.subject, group: state.experiment.group, session: state.experiment.session, + params: state.experiment.params, deviceType: state.device.deviceType, ...state.pyodide, }; diff --git a/src/renderer/epics/pyodideEpics.ts b/src/renderer/epics/pyodideEpics.ts index 1eb875ea..041e2182 100644 --- a/src/renderer/epics/pyodideEpics.ts +++ b/src/renderer/epics/pyodideEpics.ts @@ -3,7 +3,12 @@ import { EMPTY, from, fromEvent, Observable, of } from 'rxjs'; import { map, mergeMap, tap, pluck, filter, catchError } from 'rxjs/operators'; import { toast } from 'react-toastify'; import { isActionOf } from '../utils/redux'; -import { PyodideActions, PyodideActionType, EpochArraysMeta } from '../actions'; +import { + PyodideActions, + PyodideActionType, + EpochArraysMeta, + SuggestedRejection, +} from '../actions'; import { RootState } from '../reducers'; import { buildMarkerRegistry } from '../utils/eeg/markerRegistry'; import { @@ -15,6 +20,7 @@ import { requestEpochsInfo, requestChannelInfo, requestEpochArrays, + requestSuggestRejections, applyRejection, plotPSD, plotERP, @@ -115,6 +121,11 @@ const pyodideMessageEpic: Epic< }) ); } + if (dataKey === 'suggestedRejections') { + return of( + PyodideActions.SetSuggestedRejections(results as SuggestedRejection[]) + ); + } if (dataKey === 'savedEpochs') { const savedEpochsBuffer = e.data.buffer as ArrayBuffer; const { title, subject } = state$.value.experiment; @@ -257,6 +268,24 @@ const getChannelInfoEpic: Epic< mergeMap(() => EMPTY) ); +const getSuggestedRejectionsEpic: Epic< + PyodideActionType, + PyodideActionType, + RootState +> = (action$, state$) => + action$.pipe( + filter(isActionOf(PyodideActions.GetSuggestedRejections)), + pluck('payload'), + tap((threshold) => + requestSuggestRejections( + state$.value.pyodide.worker!, + PYODIDE_VARIABLE_NAMES.RAW_EPOCHS, + threshold + ) + ), + mergeMap(() => EMPTY) + ); + const loadPSDEpic: Epic = ( action$, state$ @@ -308,6 +337,7 @@ export default combineEpics( cleanEpochsEpic, getEpochsInfoEpic, getChannelInfoEpic, + getSuggestedRejectionsEpic, loadPSDEpic, loadTopoEpic, loadERPEpic diff --git a/src/renderer/reducers/pyodideReducer.ts b/src/renderer/reducers/pyodideReducer.ts index c3d7e3a2..e43e55ea 100644 --- a/src/renderer/reducers/pyodideReducer.ts +++ b/src/renderer/reducers/pyodideReducer.ts @@ -1,5 +1,10 @@ import { createReducer } from '@reduxjs/toolkit'; -import { PyodideActions, ExperimentActions, EpochArraysMeta } from '../actions'; +import { + PyodideActions, + ExperimentActions, + EpochArraysMeta, + SuggestedRejection, +} from '../actions'; export interface PyodideStateType { readonly epochsInfo: Array<{ @@ -25,6 +30,7 @@ export interface PyodideStateType { | null | undefined; readonly epochArrays: { buffer: ArrayBuffer; meta: EpochArraysMeta } | null; + readonly suggestedRejections: SuggestedRejection[]; readonly worker: Worker | null; readonly isWorkerReady: boolean; } @@ -36,6 +42,7 @@ const initialState: PyodideStateType = { topoPlot: null, erpPlot: null, epochArrays: null, + suggestedRejections: [], worker: null, isWorkerReady: false, }; @@ -79,8 +86,13 @@ export default createReducer(initialState, (builder) => }; }) .addCase(PyodideActions.SetEpochArrays, (state, action) => { - return { ...state, epochArrays: action.payload }; + // New epoch arrays → any prior auto-flag suggestions are stale. + return { ...state, epochArrays: action.payload, suggestedRejections: [] }; }) + .addCase(PyodideActions.SetSuggestedRejections, (state, action) => ({ + ...state, + suggestedRejections: action.payload, + })) .addCase(PyodideActions.SetWorkerReady, (state) => { return { ...state, isWorkerReady: true }; }) @@ -90,6 +102,7 @@ export default createReducer(initialState, (builder) => epochsInfo: [], channelInfo: [], epochArrays: null, + suggestedRejections: [], psdPlot: null, topoPlot: null, erpPlot: null, diff --git a/src/renderer/utils/webworker/index.ts b/src/renderer/utils/webworker/index.ts index 3c31bfd2..5ab9a36c 100644 --- a/src/renderer/utils/webworker/index.ts +++ b/src/renderer/utils/webworker/index.ts @@ -139,6 +139,17 @@ export const requestEpochArrays = (worker: Worker, variableName: string) => { }); }; +export const requestSuggestRejections = ( + worker: Worker, + variableName: string, + thresholdUv: number +) => { + worker.postMessage({ + data: `suggest_rejections(${variableName}, ${thresholdUv})`, + dataKey: 'suggestedRejections', + }); +}; + // Apply the user's rejection to raw_epochs in Python: drop the marked epoch // indices + set bad channels, mutating in place. Trailing ';' suppresses the // return value — apply_rejection returns the Epochs object, which cannot cross diff --git a/src/renderer/utils/webworker/utils.py b/src/renderer/utils/webworker/utils.py index 4d289166..4657f94d 100644 --- a/src/renderer/utils/webworker/utils.py +++ b/src/renderer/utils/webworker/utils.py @@ -330,3 +330,39 @@ def apply_rejection(epochs, drop_indices, bad_channels): if drop_indices: epochs.drop(list(drop_indices)) return epochs + + +def suggest_rejections(epochs, threshold_uv): + """Suggest artifact epochs by peak-to-peak amplitude (does NOT drop anything). + + For each epoch, compute the per-channel peak-to-peak (max-min over time) on the + EEG channels only (Marker/stim excluded), take the worst channel, and if it + exceeds threshold_uv microvolts, suggest that epoch. Advisory only — the UI + pre-marks these but the user can override; the real drop goes through + apply_rejection so the saved data stays MNE-exact. + + Returns list[dict] with keys: index (int), channel (str), peak_uv (float, + rounded 1dp), reason (str). MNE data is in volts; threshold_uv is microvolts. + """ + # NOTE: peak_uv is in microvolts; index is 0-based into the CURRENT epochs + # (same order as get_epochs_arrays produced). + picks = pick_types(epochs.info, eeg=True) + data = epochs.get_data(picks=picks) # (n_epochs, n_channels, n_times), volts + ch_names = [epochs.ch_names[i] for i in picks] + ptp = data.max(axis=2) - data.min(axis=2) # (n_epochs, n_channels), volts + thresh_v = threshold_uv * 1e-6 + suggestions = [] + for e in range(ptp.shape[0]): + worst = int(ptp[e].argmax()) + peak_v = float(ptp[e, worst]) + if peak_v > thresh_v: + peak_uv = round(peak_v * 1e6, 1) + suggestions.append({ + "index": int(e), + "channel": ch_names[worst], + "peak_uv": peak_uv, + "reason": "peak-to-peak {}µV on {}".format( + round(peak_v * 1e6), ch_names[worst] + ), + }) + return suggestions diff --git a/tests/analysis/test_suggest_rejections.py b/tests/analysis/test_suggest_rejections.py new file mode 100644 index 00000000..36bf132a --- /dev/null +++ b/tests/analysis/test_suggest_rejections.py @@ -0,0 +1,68 @@ +"""Native-MNE tests for the artifact-suggestion step (Phase 2 epoch review). + +Verifies `utils.suggest_rejections` flags epochs whose worst EEG-channel +peak-to-peak amplitude exceeds a microvolt threshold. It is advisory only — it +never drops anything (the real drop goes through `apply_rejection`) — so these +tests only check which epochs it *suggests*, and the shape of each suggestion. +""" +import utils # src/renderer/utils/webworker/utils.py (see conftest) +from synthetic import ( + generate_recording, + TARGET_CODE, + STANDARD_CODE, +) + +EVENT_ID = {"STANDARD": STANDARD_CODE, "TARGET": TARGET_CODE} +TMIN, TMAX = -0.1, 0.8 + + +def _build_epochs(): + csv, _ = generate_recording() + raw = utils.load_data(csv_strings=[csv]) + raw.filter(1, 30, method="iir", verbose=False) + return utils.get_raw_epochs(raw, EVENT_ID, TMIN, TMAX) + + +def test_threshold_gating(): + epochs = _build_epochs() + assert len(epochs) > 3 + + # Very high threshold: nothing is anomalous. + assert utils.suggest_rejections(epochs, 1e9) == [] + + # Very low threshold: every epoch is flagged. + suggestions = utils.suggest_rejections(epochs, 0) + assert len(suggestions) == len(epochs) + for s in suggestions: + assert set(s.keys()) == {"index", "channel", "peak_uv", "reason"} + assert isinstance(s["index"], int) + assert 0 <= s["index"] < len(epochs) + assert s["channel"] in epochs.ch_names + assert isinstance(s["peak_uv"], float) + assert s["peak_uv"] > 0 + assert isinstance(s["reason"], str) + + +def test_detects_injected_artifact(): + epochs = _build_epochs() + assert len(epochs) > 3 + + ep = epochs.copy() + k = 2 + # 5 mV = 5000 µV peak-to-peak bump on one EEG channel of one epoch. + data = ep._data + data[k, 0, :] += 5e-3 + + # Threshold between the normal ptp and the artifact. + suggestions = utils.suggest_rejections(ep, 1000) + indices = [s["index"] for s in suggestions] + + assert k in indices + flagged = next(s for s in suggestions if s["index"] == k) + assert flagged["peak_uv"] > 1000 + + +def test_marker_channel_excluded(): + epochs = _build_epochs() + suggestions = utils.suggest_rejections(epochs, 0) + assert all(s["channel"] != "Marker" for s in suggestions)