Research/Computational experiment
Computational experimentResearch manuscript · 15 October 2026 · interactive companion

Instruction information and spectral sensitivity in a musical model of speech and song

Instruction information, spectral intervention and native affect

Abstract

A model can retain information about vocal expression without assigning its native affect scores the expected direction. We examined this distinction in Musical Intelligence (MI), a system designed for musical listening, using an instruction factorial and a subsequent acoustic intervention. The first study analysed 342 synthetic recordings from 172 requests. Across held-out sentences, a fixed probe decoded five speech instructions from 889 C³ dimensions with 68.9% balanced accuracy, compared with 57.8% from 15 acoustic descriptors and 35.6% from two native affect outputs. Yet native valence was lower for excited than sad speech in every text. A second study selected six recordings and independently varied an instrument-derived upper-harmonic envelope and residual retention, yielding 1,596 primary full-recording evaluations. Mean native valence rose from 0.139 to 0.435 for cello-derived and 0.437 for viola-derived envelopes, while all 588 primary paired-anchor contrasts retained the inverse instruction ordering. Residual removal produced a substantially larger shift than envelope transfer alone. Restoring the source consonance-feature trajectories suppressed 94.8% and 94.2% of the mean endpoint rises. Arousal ordering depended on waveform level, and 192 near-endpoint probes exposed substantial score differences between very close waveforms. These results distinguish decodable instruction information, native affect mapping and numerical sensitivity. They identify a computational route through which speech-derived texture changes affect scores, without establishing consonant-specific causality, natural instrument equivalence or human perceptual thresholds.

Two connected computational studies of one synthetic voice and three intended texts. The intervention reuses six source clips; 1,914 full-file evaluations include 1,596 primary and 318 sensitivity/control analyses. Feature holds explain most of the absolute valence rise, while near-endpoint probes expose numerical sensitivity. Phoneme-specific causality and human perception remain untested.

Instrument-derived spectral intervention, absolute valence paths and excited-minus-sad contrasts for cello and viola envelopes under native and common-RMS levels.
Figure 4 · Spectral intervention raises valence without recovering instruction ordering. Across six source clips, mean native valence rises from 0.139 to 0.435 or 0.437, while every one of the 588 primary paired contrasts remains negative. These are repeated model evaluations of fixed sources, not listener judgments.

Supporting materials

New findings · spectral intervention, feature attribution and numerical sensitivityFigures ↗Supplementary Information · 42 pages, complete earlier results and intervention controlsPDF ↗Companion study · From Birdsong to MusicRecord ↗

Version history

The record stays at this address. Each PDF preserves the version cited.

  1. Research manuscript · Current

    Ten pages before References, six main figures and the complete 42-page supplement. Integrates the executed spectral intervention and adds the interactive-companion method, including frozen full-recording curves, evaluated anchors and mechanism-specific layer semantics. Numerical findings, original figures and prior editions are preserved. No human or neural measurements are reported.

    Cite this version ↓
  2. Research manuscript

    Ten pages before References; six main figures and a 42-page supplement with nine figures. Adds the completed six-source spectral intervention, factorial controls, feature holds and boundary diagnostics. All previous artwork, detailed results and full original methods are preserved. No human or neural measurements are reported.

    Cite this version ↓
  3. Research manuscript

    Instruction information and level-sensitive affect in a musical model of speech and song

    Ten pages before References, six main figures and a separate 24-page supplement. Preserves all computational results and distinguishes affect calibration, domain-dependent cue weighting, phonetic structure and target mismatch. No participant or neural measurements are reported.

    Cite this version ↓
Underlying architectureMusical Intelligence