# Marker-only freely moving demo — methods

## Goal and recording

Show pose, estimated binaural sound and measured/predicted neural activity on one clock, without camera imagery or an animal silhouette. Single-recording illustration, not a model comparison statistic.

PRN053a04_a_NTD, batch 348, trial 70 (zero-based 69), recording 1125.420–1146.620 s. Freely moving ferret in a go/no-go tone-detection recording. Acquisition metadata specifies randomized two-speaker sound; both physical speaker banks contain stimulus.

Output: `PRN053a_trial70_full_continuous`. Artifacts: clip.mp4, data.json, index.html, validation.json. Generator: `ferret_trial_video_hrtf.py --website-out` → `website_demo/export.py`. Existing nems_cpu environment; no fitting, recaching or upload. Updated 2026-10-16.

## Input lineage

- `recording_cache` ← `PRN053a_2458f717a90eb13356d53df96157aafb9c2659e8.tgz` (SHA-256 in data.json).
- `video` ← `recording.avi` (SHA-256 in data.json).
- `pose` ← `PRN053a04_a_NTDDLC_resnet50_right_chimney_PRNfbNov30shuffle1_60000.h5` (SHA-256 in data.json).
- `trial_log` ← `trial_log.csv` (SHA-256 in data.json).
- `prediction_cache` ← `preds_PRN053a_b348_v2.npz` (SHA-256 in data.json).
- `arena_calibration` ← `recording-PRN053a04_a_NTD-dlc_arena_config.toml` (SHA-256 in data.json).
- `generic_hrtf` ← `hrtf_az.csv` (SHA-256 in data.json).
- `prediction_accuracy_cache` ← `percell_b348_v2.csv` (SHA-256 in data.json).

Recording cache ← BAPHYExperiment.get_recording. Stimulus cache ← runclass.py gammatone magnitude, followed by square-root compression. Existing predictions ← pose_task_talk.load_preds: ten-fold held-out windows, dropbank_modelnames(fidelity="none", basis="v2"); no HRTF. No upstream cache builder is run.

## Browser layout

`index.html`, `style.css` and `app.js` place the HRTF polar inside the arena, with a reserved region that leaves calibrated points and pose landmarks visible. Five signal plots are stacked with short titles and aligned time bounds; only the bottom raster displays time ticks. A single key below the visual defines signal types and sound magnitudes. Additional readouts and overlay switches are in the collapsed Options & details section. Narrow screens place the arena above the signal stack. Canvas resizing changes display geometry only; video, timestamps and signal samples are unchanged.

## Tracking and movie

`frame_map()` checks spike-corrected TRIALSTART against cached TRIAL epochs, floors PSIVIDEO anchors to the recording raster, interpolates the anchor-to-frame map, and selects nearest original frame indices for each 30 Hz movie timestamp. `load_pose()` reads those rows of existing DeepLabCut HDF: likelihood <0.8, nonfinite or out-of-image landmarks become null, before the optional fill step. `arena_geometry()` uses the recording-specific TOML calibration (translation, rotation, pixels per meter, y reflection) for pose, nosepoke and lickspout; speaker angle/distance are converted to metric positions by baphy_io. Speaker distances are not compressed. Raw pixel pose remains in JSON alongside metric pose. The exact anatomy of the training label “rear” is undocumented.

`encode_markers()` draws points and labels on a blank dark canvas and writes H.264/yuv420p, 800×800, 30 Hz; it never decodes camera images. The original video is only probed and fingerprinted. `drawPose()` draws the same metric coordinates in the browser. Playback clock is the generated MP4; overlays never replace or retime source samples.

## Speaker display and level envelope

`add_demo_metadata()` in `website_demo/export.py` computes `sound.speaker_waves` from the original cached speaker stimulus before HRTF: sqrt(sum of squared compressed gammatone band magnitudes), at native 20 ms bins. Speakers without channels receive zero. The shared threshold is 20% of the largest speaker envelope value in the clip. `drawPose()` emits a ring at 160 ms sample intervals only when the corresponding envelope exceeds that threshold. Rings expand to 0.35 m and fade over 0.6 s, with opacity proportional to level and remaining life. Ring positions use the shared video clock, so seeking reproduces the same state. These are illustrative waves, not physical propagation speed or a recovered raw pressure waveform. Simple vector speaker glyphs are centered on the calibrated speaker points; their icon direction is schematic, not a measured radiation pattern.

## Estimated binaural spectrograms and HRTF polar

`binaural_estimate()` selects source pose frames independently at the 50 Hz signal-bin centers using the same camera anchors. The existing `free_tools.dlc_headfix()` defines forward by nose minus head center and right by the right-ear projection; angles are 0° forward and positive right. A missing nose, head or right-ear estimate masks both gains and ear spectrograms unless bounded pose interpolation is enabled; remaining gaps stay masked. `free_tools.calculate_hrtf_gains()` uses the generic azimuth-only ferret HRTF (`hrtf_az.csv`) on the existing 18-channel gammatone frequency grid, linearly interpolated in angle/frequency, plus the manuscript's distance rule −6 × (distance_m − 1) dB. This rule is not inverse-square propagation. It is an estimate, not an individual HRTF measurement or calibrated SPL.

`free_tools.apply_hrtf_to_stim()` applies the exact existing transform: each ear's output is sqrt(sum over active speakers of stimulus² × 10^(gain_dB/10)). Input is the cache's square-root-compressed gammatone magnitude, so output remains an HRTF-adjusted compressed stimulus representation, not a physical pressure waveform. Speaker banks in this recording: L, R. Missing speaker channels contribute no stimulus. JSON retains original speaker values and separate left/right estimates. `drawSound()` uses the same clip-wide color range for both ears; null bins remain blank.

`drawPolar()` uses the manuscript's example frequency (nearest channel to 2.72 kHz: 2716.467 Hz). The center icon reuses `nems_lbhb/ferret_glyph.py:draw_ferret_head()` as a static SVG orientation key, nose up. Front/Back/Left/Right replace numeric bearing ticks; the view remains head-centered (0 degrees forward, positive right). The icon does not encode pose or gain. The portfolio places the static Left/Right direction words at the canvas margins to leave space for fixed-position ray letters on narrow screens. The radial curves are directional HRTF gain only, radius in dB; the fixed corner key gives the center-to-rim gain range, with 5 dB per ring; speaker rays use instantaneous head-relative bearing, carry L/R labels at a fixed inset from the ray tips plus distinct dash patterns (no collision-avoidance repositioning), and dots mark directional ear gains. Readouts show total applied gain including distance for each ear. This same total gain was applied to the displayed ear spectrogram. No gain is applied to the saved neural predictions. HRTF curves and per-bin bearings, distances and total ear gains are timestamped in JSON.

## Ear balance and tracking quality

`binaural_balance()` pools squared magnitudes across all 18 frequency bands for each ear, then computes 10 log10(sum(right²) / sum(left²)) per native 20 ms bin. `drawDifference()` renders this as one line above the two ear spectrograms. Positive dB means right stronger; negative means left stronger. Blue indicates <−0.5 dB, orange >+0.5 dB, and gray the near-equal interval ±0.5 dB. These are ratios of the estimated compressed-band representation, not measured pressure levels. No smoothing, frequency selection, or time reordering is used.

Sound-on bins require finite, positive values for both pooled ears and a larger-ear pooled magnitude above 0.1% of the clip-wide maximum. Quiet and invalid bins remain blank and do not enter the balance percentages. Fractions are counts of sound-on time bins (left/right/near equal), not fractions of frequency bins or total stimulus energy. JSON stores the line, formula, floor, neutral threshold, counts, and fractions. The full frequency-wise 20 log10(right/left) difference remains in JSON for inspection; it is not drawn. The polar shows a separate single-frequency directional gain and must not be equated with the broadband balance line.

JSON includes confidence threshold and valid-landmark/all-landmark-frame/binaural-bin fractions. No confidence threshold is lowered. Original valid acoustic-bin fraction: 90.566%; all-landmark-frame fraction: 90.252%.

## Selected neuron and all-cell raster

`response_gains()` requires matching sample rate, cell order, array shape and FILE/TRIAL epochs. It recovers a fixed zero-offset normalization gain per cell by matching cached response against raw spike counts across all finite bins; rejects residuals above 0.00001 spikes/bin. Those same gains convert cached model outputs to expected spikes/bin, preserving negative predictions and nulls.

`add_demo_metadata()` joins the sibling `percell_b348_v2.csv` by exact cell ID to each exported model. The cached scores originate from `pose_task_talk.load_arms()` and Results-table `r_test` for the same no-HRTF v2 drop-bank model family. Each model's `accuracy` object includes cell IDs, saved validation correlation values, and descending cell IDs for sorting; missing/nonfinite scores are null and sort last. These are saved model-validation scores, not correlations computed on this short clip. The caches do not retain fit timestamps, so exact fit-version identity between the score table and prediction arrays cannot be independently established. No database query, model loading or fitting is performed. The portfolio preview selects and sorts by the saved model scores described below.

`drawNeural()` sums five 20 ms bins then divides by 0.1 s. A missing constituent bin makes the entire display bin missing. The portfolio preview defaults to the cell with the highest best-of-three saved held-out r, with ties broken by recording order. Each model can be used for sorting separately; recording order is also available. Sorting selects the top cell but preserves original raster row identity. This is illustrative selection of a high-scoring example, not an unbiased model-comparison result. Scores are not percent accuracy or a clip correlation. Its raster row is highlighted in matching mint. Measured rates are solid mint; existing held-out predictions use colored dashed lines. Sound inputs use stimulus-file channels, with unused speaker banks padded with zeros. These auditory frequency-band magnitudes undergo log compression (reducing the range between quiet and loud sounds), then one global min–max rescaling across stimulus channels and masked recording samples to 0–1 in the saved pipeline, not rescaling within this clip. Pose inputs: head-center x/y position, movement speed, head-angle sine/cosine and angular change. Task inputs: time since nose contact, tone, port exit, first lick and lick-bout end; time until the next trial-initiating nose contact and first lick; tone-present timing, previous reward, nose-in-port and lick-contact indicators. Future event timing makes the task model descriptive rather than a real-time forecast. Ten-fold predictions hold out recording windows (nominally 20 s); each output is from the model that omitted that window during fitting. No head-related transfer-function (head/ear acoustic filtering) stage is used. Predictions are shown without clipping negative model outputs.

`drawRaster()` shows all 63 cells in recording order, one row each, at native 20 ms resolution. Intensity encodes spike count, dark zero; occupied bins are not exact spike times. JSON cellids maps each row to its recorded unit.

## Clock and limitations

All exported signal times are bin centers relative to clip start; video/pose times are presentation timestamps. Recording time = 1125.420 + clip time. Native signal arrays retain every bin. Browser play/pause, seek, speed and plot-click scrubbing share the MP4 clock; mobile layout stacks the two columns.

Maximum video nearest-frame difference 16.579 ms; anchor flooring <20 ms. These are mapping bounds, not independently measured physical synchronization. Per-frame exposure timestamps and microphone waveform/soundtrack are unavailable in this export. Camera timing within anchor intervals is estimated; irregular drops within an interval cannot be located exactly. Existing neural models use speaker inputs without HRTF, while the sound visualization uses the generic acoustic estimate. No retraining or individual HRTF fitting occurs.

## Linear pose interpolation

`interpolate_pose()` operates once on contiguous native DLC source frames spanning the selected video and signal samples. Frame times come from the same camera-anchor map. Each missing xy pair is linearly interpolated between that marker's nearest accepted endpoints, using elapsed time rather than frame number. All bounded gaps are eligible, with no duration cap. Leading/trailing gaps and markers with fewer than two accepted points remain missing. Accepted coordinates and original confidence scores are unchanged. This is an assumed straight path; actual motion inside a gap is unknown.

`sample_pose()` selects from this single filled source table for both the 30 Hz video and the 50 Hz sound clock. `binaural_estimate()` recomputes gains and ear magnitudes from the filled pose and original stimulus; it does not interpolate spectrogram pixels or neural data. Original unfilled pose is retained as `raw_xy`, at both clocks, with per-landmark `interpolated` flags. JSON also retains original metric pose, ear arrays, balance line and speaker gains; endpoint times and marker names for every filled source gap are in `interpolation.gaps`. `sound.interpolated` identifies bins using any filled nose/head/right-ear coordinate, even if another missing or degenerate landmark still prevents an ear estimate.

Original ear coverage: 90.566%. Coverage after fill: 100.000%. Pose-fill-dependent signal bins: 9.434%. The preview and standalone MP4 display filled pose and acoustic estimates continuously, without per-sample fill highlights or a fill toggle. This is an illustrative science demo, not a reproduction of manuscript preprocessing. Interpolation remains documented here and in the JSON flags and original arrays. Neural responses and saved predictions are unchanged.

## Whole-trial selection

Trial 70 (zero-based 69), a go/HIT trial. The clip spans recording 1125.42–1146.62 s: exactly 1 s before the target/tone epoch starts at 1126.42 s, through the end of the cached TRIAL interval. The behavioral `trial_end` event occurs earlier, at 1128.48 s; this export deliberately includes the post-response pause up to the next target onset. Tone onset is 1.00 s on the displayed clock. The source epoch times are retained under `recording.selected_events`. Reproduce with `--trial 69 --trial-pre 1 --seconds 21.2 --interpolate-pose` using the recording and prediction caches listed above.

## Portfolio playback and ranking

Playback is paused initially. After deliberate Play or Replay, Loop is enabled by default and can be switched off. The parent portfolio controls and standalone controls share the MP4 clock. Cell sorting uses only this export’s 63 exact cell IDs and three models’ saved `accuracy.values`; no scores from the previous PRN075a recording are used. Best-of-three takes the maximum finite saved r per cell; model-specific sorts use that model’s r; missing scores sort last and ties preserve recording order. Model visibility switches do not affect ranking. The original per-model score provenance and fit-version caveats remain in `data.json`.