OrcaDolittle — site-controlled killer-whale call decoding

What was tested

Core findings, every number reproducible from public data.

One frozen audio foundation model (AVES2) runs unchanged across the primary analysis; the contribution is the rigour around it — measuring the recording-site shortcut that fools naive probes, reporting only what survives explicit null baselines, and checking the two primary representation results under a second frozen encoder.

The trap

Most of the “97% accuracy” was the microphone

0.948

A frozen model “decodes ecotype” at 0.91 pooled — but hold out a whole recording site and it collapses to 0.23 (chance 0.25), while the site itself decodes at 0.948. The confound is measured instead of hidden.

Stowell 2022 · Ghani 2023 · Palmer et al. 2025

Real biology

Ecotype is decodable once the site is held constant

0.889–0.973

Within five fixed recording sites the embeddings still separate ecotypes; the hydrophone is held constant, so this is not a site shortcut.

Hagiwara 2023 (AVES) · Chen et al. 2022

Units that travel

Named call types transfer across sites

0.71 / 0.97

Catalogue call types are recovered within a site (14 SRKW types 0.71; 18 NRKW types 0.97) and — unlike ecotype — transfer to an independent site (0.64; 0.83 on unambiguous types): the control ecotype failed.

Ford 1989 · Filatova et al. 2015

More than one context

Calls predict what the whale is doing

0.770

On animal-borne tags, communicative calls decode movement-defined context (foraging vs not 0.77; three-way 0.58) with every individual held out, and specific call types occur in specific contexts (V=0.40). Controls rule out call rate and loudness as the explanation.

Holt, Tennessen et al. 2024 · Wilson et al. 2006

Structure

Sequence structure, and — in SRKW — beyond first order

+0.645 bits

Calls don’t come at random: one call predicts the next, and Southern-Resident sequences carry information a first-order model can’t explain (candidate phrase S01→S04→S01). A prerequisite for combinatorial coding — not meaning.

Sharma et al. 2024 · Berthet et al. 2025 · Girard-Buttoz et al. 2025

A measured response

Wild whales answer same-pod calls, ignore strangers

p = 0.002

Re-analysis of a published playback: free-ranging whales replied to same-pod broadcasts (6/6) and stayed silent to different-pod ones (0/6), naive animals — and the model recovers the dialect call-type space underlying the response contrast (purity 0.439 vs 0.05 null). The experiment is prior work; the response tracks dialect, not content.

Filatova et al. 2011 · FEROP catalogue · Miller et al. 2004

Second encoder

The same representation checks repeat in NatureLM-audio

0.709 / 0.800

Frozen NatureLM-audio repeats the site-controlled SRKW and NRKW call-type checks above chance, preserves VFPA→SMRU transfer (0.682 vs 0.20 chance), and separates FEROP K-type exemplars (purity 0.366 vs 0.050 chance). Cross-encoder check, not semantic meaning.

Robinson et al. 2024 · reports/naturelm_analysis_readout.json

Results at a glance

Every panel regenerates from public data and open code.

Generated by the scripts in the repository; full metrics live in reports/ and reproduction status in the environment manifest.

Biology vs. site. Provider decodes at 0.948; ecotype is decodable within five fixed sites (0.889–0.973) but collapses cross-site (0.529) — real, but local.

Call types transfer across sites. Within-site 0.71 (SRKW) / 0.97 (NRKW); a VFPA-trained classifier scores 0.64 on an independent site (0.83 unambiguous).

Behaviour from calls. Communicative calls predict movement-defined foraging context at 0.77, leave-individual-out; the label uses depth/acceleration only.

Context-specific production. Specific call types occur in specific contexts (Cramér's V = 0.40 vs a within-individual null of 0.08).

Beyond first order (SRKW). The 2-back call adds information a first-order model can't explain (Δ = 0.645 bits); NRKW is null.

Structure ↔ context. SRKW call types with similar sequential company share independent ethogram context (Mantel r = 0.33, p = 0.042); NRKW null. The distributional hypothesis tested, not assumed.

Attribution controls. The decode (0.98) is compared with matched-noise (0.54), feature-shuffle (0.50), and label-permutation (0.50) controls.

Receiver response (re-analysis). Same-pod 6/6 vs different-pod 0/6 (Fisher p = 0.002), naive animals. Prior published experiment; response tracks dialect, not content.

Second encoder: call types. NatureLM-audio recovers SRKW call types at 0.709 and NRKW call types at 0.800 under full-data, permutation-controlled checks.

Second encoder: FEROP space. NatureLM-audio separates FEROP K-type catalogue exemplars (1-NN purity 0.366 vs 0.050 chance, p = 0.000999).

Talk Orca · full model console

Run the full evidence stack.

Use the model-facing console to select or upload a candidate signal, inspect call-type/dialect structure, production-context evidence, playback-response design, and a field-planning readout from the same claim-bounded stack.

⚠

Model scope. Real FEROP K-call clips anchor the audition set from the published dialect-response re-analysis. DTAG/context findings inform the field protocol, and synthetic contours are listening sketches. The console reports structure, context association, and response design without turning the observer note into whale speech. Build the controlled protocol →

Catalogue substrate Real FEROP K-call recordings are reused as named audition references, not invented labels or field-release stimuli.

Model output Frozen embeddings support classification, separability, and context-associated probes; not semantic translation.

Audition layer Synthetic contours are optional listening sketches; real FEROP clips remain the evidence anchors.

Model does

Recover call-type identity

Site-controlled catalogue heads recover stereotyped call types and cross-site transfer.

0.709 / 0.968

Model does

Decode production context

DTAG calls predict movement-defined context with the individual held out.

0.770 / 0.577

Playback evidence

Receivers respond to dialect

Published conspecific playback re-analysis: same-pod replies, different-pod silence.

6/6 vs 0/6

Claim scope

Content-specific response

The next route is a content-controlled conspecific playback with real field responses.

Frontier

Observer note

Choose the protocol frame first; edit evidence anchors after.

Observer note Observed context Receiver frame

Protocol lens Audition tone

Energy Calm Spacing Natural gaps Water echo Soft water

All dictionary anchors are available; presets are starting sets, not context translations

Tap anchors above to build an audition set…

Select anchors, then Play

Full model readout

Real FEROP clipsK1 · K3 · K7

Expedition Playback Protocol Kit

Configure a controlled study, produce a randomized encounter allocation, and export a protocol, catalogue audit, manifest, operator sheet, and genuinely blinded scoring sheet. The website does not create field-release audio.

Build review kit Review exports update as you edit. Field audio requires the offline provenance, permit, welfare, and calibration gate.

Planned conditions

Evidence stack

Operator steps

Protocol builder

Experimental units per condition Washout minutes Baseline window Response window Study label Random seed

matched-noise control different-pod control observer-blinded scoring disturbance stop rules

Randomized encounter allocation

Deployment gate audit

Upload/listen mode

Optional browser-only preview for a collaborator-supplied sequence. It can summarize file metadata here; eligibility and acoustic QC belong in the offline validator.

Export

Review-kit JSON includes the protocol, condition table, encounter allocation, manifest, blind scoring schema, and release checklist. It contains no field-release audio.

Preview review-kit JSON

For field scientists

The experiment an archive cannot run.

Archival recordings can characterise call structure and its confounds. What an archive cannot do — even in principle — is measure how a whale responds to a signal deliberately broadcast to it. That needs a controlled in-situ playback with a permitted field collaborator. The analysis and review tooling are built; target-specific stimuli, calibration, authorization, and field execution are not.

What this project brings

A frozen-embedding pipeline for killer-whale calls with hash-frozen artifacts and per-head metrics.
A site-controlled evaluation protocol that separates acoustic structure from the recording-site shortcut and keeps field claims tied to measured controls.
Validated catalogue call-type evidence that transfers across sites, plus a behavioural-context decoder with explicitly bounded training populations.
Reusable manifests and analysis code for collaborator-supplied audio, subject to population-specific validation.

What a field partner brings

Permitted, non-invasive in-situ playback with a study population that has individual/group ID.
Per-trial response measurement (movement, vocal, behavioural) with pre-exposure and matched controls.
The design that closes the loop: a content-controlled, conspecific playback using independently context-labelled natural sequences while holding pod/caller identity and playback handling as constant as possible.
Observer-grounded, segment-level behavioural context.

Run the tool yourself

Clone it, install the analysis extras, run the tests, and reproduce the playback-response statistic from the committed trial table:

git clone https://github.com/ladyFaye1998/OrcaDolittle
cd OrcaDolittle
python -m pip install -e ".[dev,analysis]"
python -m pytest -q
python scripts/run_playback_response_stats.py
python scripts/build_field_playback_package.py --config field_playback/field_config.example.json --output field_playback/orca_playback_review_package.zip

The example build is review-only and contains no broadcast audio. See the controlled-playback protocol and release boundary.

Reach out

For permitted playback or biologging studies, include the study population, permit status, season timing, and response measures.

✉ Reach out Open GitHub issue ↗

Scope

Current evidence and what the field test can decide.

Current evidence

Killer-whale call structure is decodable from frozen embeddings once the recording site is controlled.
Validated catalogue call types are recoverable and transfer across recording sites.
Calls carry information about more than one behavioural context, with the individual held out.
Call sequences are non-random — and in Southern Residents show structure beyond first order.
A published playback re-analysis shows a measurable, dialect-selective response to broadcast conspecific calls.

Field-test questions

Whether approved, content-controlled natural sequences elicit differential responses in the target population.
That a receiver responds to a call’s content, not only to whose voice (dialect) it is.
That the decoder generalises to your study population and recording conditions.
Whether the system can support a controlled interactive exchange.
These are testable predictions, not yet confirmed by a content-controlled field trial.

Limitations & controls

What this analysis cannot see yet.

The project is intentionally acoustic and claim-bounded: it models recorded underwater audio and measured behavioural-response evidence, not the whale’s full perceptual world, not translation, and not semantic meaning. The full limitation register is in the repository.

Signal channel

Recorded audio is not the whole whale percept

Orcas may use multimodal or unrecorded cues. The analysis therefore makes acoustic claims only, and the field design pairs hydrophones with visual, group-ID, movement, and response measures.

Kershenbaum 2024

Bandwidth & equipment

Archives differ in sample rates, filters, gain, and formats

Killer whales hear across high-frequency ranges, while public recordings come from mixed hydrophones and conversion paths. Provider is exposed as a confound; positive claims are limited to within-site tests or explicit transfer tests.

Szymanski et al. 1999 · Palmer et al. 2025 · Johnson & Tyack 2003

Dialect vs content

The playback result is dialect-selective, not content-controlled

The published playback shows a receiver response to familiar dialect calls. It does not prove that whales respond to call content; that remains the permitted field playback that isolates content while holding dialect/social familiarity constant.

Filatova et al. 2011 · FEROP catalogue

Model specificity

Second-encoder agreement is not meaning

NatureLM-audio repeats the two primary representation checks, which addresses part of the single-encoder concern. It still does not turn acoustic separability into semantic interpretation.

AVES2 · BEATs · NatureLM-audio

See limitations and mitigations and data availability for the source-by-source boundary.

References & data

Everything here is sourced.

Data are public: DCLDE 2026 (NOAA/NCEI), the DTAG archive (Zenodo, CC-BY), and the FEROP catalogue. Code and the full bibliography are in the repository.

Palmer, K. J., et al. (2025). A public dataset of annotated Orcinus orca acoustic signals for detection and ecotype classification. Scientific Data 12:1137.

Szymanski, M. D., et al. (1999). Killer whale hearing: auditory brainstem response and behavioral audiograms. J. Acoust. Soc. Am. 106:1134–1141.

Johnson, M. P., Tyack, P. L. (2003). A digital acoustic recording tag for measuring the response of wild marine mammals to sound. IEEE J. Oceanic Engineering 28:3–12.

Hagiwara, M. (2023). AVES: animal vocalization encoder based on self-supervision. ICASSP 2023.

Chen, S., et al. (2022). BEATs: audio pre-training with acoustic tokenizers. arXiv.

Ford, J. K. B. (1989). Acoustic behaviour of resident killer whales off Vancouver Island, British Columbia. Canadian Journal of Zoology 67:727–745.

Foote, A. D., Osborne, R. W., Hoelzel, A. R. (2008). Temporal and contextual patterns of killer whale call type production. Ethology 114:599–606.

Filatova, O. A., et al. (2015). Cultural evolution of killer whale calls. Behaviour 152:2001–2038.

Yurk, H., et al. (2002). Cultural transmission within maternal lineages: vocal clans in resident killer whales in southern Alaska. Animal Behaviour 63:1103–1119.

Riesch, R., Ford, J. K. B., Thomsen, F. (2008). Whistle sequences in wild killer whales. J. Acoust. Soc. Am. 124:1822–1829.

Filatova, O. A., Fedutin, I. D., Burdin, A. M., Hoyt, E. (2011). Responses of Kamchatkan fish-eating killer whales to playbacks of conspecific calls. Marine Mammal Science 27:E26–E42.

Far East Russia Orca Project (FEROP). Catalogue of Kamchatkan killer whale calls.

Holt, M. M., Tennessen, J. B., et al. (2024). Calibrated movement data and variables. Zenodo, doi:10.5281/zenodo.13308835.

Tennessen, J. B., et al. (2019). Hidden Markov models reveal temporal patterns and sex differences in killer whale behavior. Scientific Reports 9:14951.

Wilson, R. P., et al. (2006). Moving towards acceleration for estimates of activity-specific metabolic rate (ODBA). J. Animal Ecology 75:1081–1090.

Berthet, M., Surbeck, M., Townsend, S. W. (2025). Extensive compositionality in the vocal system of bonobos. Science 388:104–108.

Girard-Buttoz, C., …, Crockford, C. (2025). Versatile use of chimpanzee call combinations promotes meaning expansion. Science Advances 11:eadq2879.

Sharma, P., et al. (2024). Contextual and combinatorial structure in sperm whale vocalisations. Nature Communications 15:3617.

Stowell, D. (2022). Computational bioacoustics with deep learning. PeerJ 10:e13152.

Ghani, B., et al. (2023). Global birdsong embeddings enable superior transfer learning. Scientific Reports 13:22876.

Kershenbaum, A. (2024). Why Animals Talk. Penguin Press.

Sayigh, L. S., et al. (2025). Widespread sharing of stereotyped non-signature whistle types by wild dolphins. bioRxiv.

Selbmann, A., et al. (2026). Aversive behavioural responses of killer whales to sounds of long-finned pilot whales. Scientific Reports 16:4716.

Bowers, M. T., et al. (2018). Selective reactions to different killer whale call categories in two delphinid species. J. Experimental Biology 221:jeb162479.

Miller, P. J. O., et al. (2004). Call-type matching in vocal exchanges of free-ranging resident killer whales. Animal Behaviour 67:1099–1107.

Robinson, D., et al. (2024). NatureLM-audio: an audio-language foundation model for bioacoustics. arXiv:2411.07186.

Site-controlled killer-whale call decoding.