MuDuSi: A Multi-Agent Acoustic Simulator for Full-Duplex Speech Language Models

Kyudan Jung · Jaegul Choo

KAIST AI

Published October 12, 2026 · Revised October 13, 2026 · Version 1.2 · MUDUSI-TR-2026-01

Abstract

MuDuSi (Multi-agent Duplex Simulator) is a virtual acoustic testbed for independently running full-duplex speech models. It couples simultaneous listening and speaking through a 24 kHz PCM engine with spatial attenuation, propagation delay, occlusion, reflection, and microphone noise. Aligned source, receiver, and model-input recordings support frozen-model evaluation and acoustic trajectory collection. We describe the interaction and timing contracts and report NVIDIA B200 experiments with PersonaPlex, Moshi, and Raon. Paired conversations completed at real-time input pacing. A larger experiment ran all nine ordered model pairs concurrently as eighteen isolated workers on four GPUs, recording 60 audio seconds per room over approximately 240 wall-clock seconds. All eighteen agents generated audio; routing checks passed, with no sustained underruns, input overflows, output drops, or clipping. This is a reduced-pacing collection result, not a real-time throughput claim. Offline ASR review nevertheless exposed substantial speech-quality problems in both Moshi–Raon pairings. We distinguish operational validity from conversational competence and acoustic recordings from token-policy rollouts. Prospective applications include HiPLEX-style hierarchical policy evaluation, overlap-aware benchmarks, and reinforcement learning; full speech-model policy training is not implemented.

@techreport{jung2026mudusi,
  author      = {Jung, Kyudan and Choo, Jaegul},
  title       = {{MuDuSi}: A Multi-Agent Acoustic Simulator for Full-Duplex Speech Language Models},
  institution = {MuDuSi Project},
  type        = {Technical Report},
  number      = {MUDUSI-TR-2026-01},
  year        = {2026},
  month       = oct,
  note        = {Version 1.2, October 13, 2026},
  url         = {https://kyudan1.github.io/MuDuSi/paper/}
}