Instruction information and spectral sensitivity in a musical model of speech and song
Instruction information, spectral intervention and native affect
Abstract
A model can retain information about vocal expression without assigning its native affect scores the expected direction. We examined this distinction in Musical Intelligence (MI), a system designed for musical listening, using an instruction factorial and a subsequent acoustic intervention. The first study analysed 342 synthetic recordings from 172 requests. Across held-out sentences, a fixed probe decoded five speech instructions from 889 C³ dimensions with 68.9% balanced accuracy, compared with 57.8% from 15 acoustic descriptors and 35.6% from two native affect outputs. Yet native valence was lower for excited than sad speech in every text. A second study selected six recordings and independently varied an instrument-derived upper-harmonic envelope and residual retention, yielding 1,596 primary full-recording evaluations. Mean native valence rose from 0.139 to 0.435 for cello-derived and 0.437 for viola-derived envelopes, while all 588 primary paired-anchor contrasts retained the inverse instruction ordering. Residual removal produced a substantially larger shift than envelope transfer alone. Restoring the source consonance-feature trajectories suppressed 94.8% and 94.2% of the mean endpoint rises. Arousal ordering depended on waveform level, and 192 near-endpoint probes exposed substantial score differences between very close waveforms. These results distinguish decodable instruction information, native affect mapping and numerical sensitivity. They identify a computational route through which speech-derived texture changes affect scores, without establishing consonant-specific causality, natural instrument equivalence or human perceptual thresholds.
Two connected computational studies of one synthetic voice and three intended texts. The intervention reuses six source clips; 1,914 full-file evaluations include 1,596 primary and 318 sensitivity/control analyses. Feature holds explain most of the absolute valence rise, while near-endpoint probes expose numerical sensitivity. Phoneme-specific causality and human perception remain untested.
Supporting materials
New findings · spectral intervention, feature attribution and numerical sensitivityFigures ↗Supplementary Information · 42 pages, complete earlier results and intervention controlsPDF ↗Companion study · From Birdsong to MusicRecord ↗Version history
The record stays at this address. Each PDF preserves the version cited.
- Research manuscript · CurrentCite this version ↓
Ten pages before References, six main figures and the complete 42-page supplement. Integrates the executed spectral intervention and adds the interactive-companion method, including frozen full-recording curves, evaluated anchors and mechanism-specific layer semantics. Numerical findings, original figures and prior editions are preserved. No human or neural measurements are reported.
- Research manuscriptCite this version ↓
Ten pages before References; six main figures and a 42-page supplement with nine figures. Adds the completed six-source spectral intervention, factorial controls, feature holds and boundary diagnostics. All previous artwork, detailed results and full original methods are preserved. No human or neural measurements are reported.
- Research manuscriptCite this version ↓
Instruction information and level-sensitive affect in a musical model of speech and song
Ten pages before References, six main figures and a separate 24-page supplement. Preserves all computational results and distinguishes affect calibration, domain-dependent cue weighting, phonetic structure and target mismatch. No participant or neural measurements are reported.