Graduate researcher in speech processing and audio machine learning at CMU WavLab. Drawn to problems where signal meets meaning. ✌︎
I'm Zhuoyan Tao, Terry to friends and colleagues. I'm a master's student in CMU's Intelligent Information Systems program at the Language Technologies Institute, advised by Prof. Shinji Watanabe at WavLab.
My work sits at the boundary of signal processing, machine learning, and linguistics, and keeps circling one question: how meaning is carried in speech beyond just the words.
Before CMU I finished dual B.S. degrees at USC, and spent summer 2026 at Apple building simulation tooling for satellite connectivity. Outside of research, I enjoy exploring new places ☀︎
Non-intrusive models that predict how a listener will rate speech from partial audio. The open question is how much acoustic context a quality judgment needs, and how to stay accurate while that context is still arriving.
Spoken language diarization for conversational, code-switching speech. I study what pre-trained speech encoders already encode about language identity, and when a lightweight probe is enough to recover it.
Speech-to-speech translation that carries pragmatic intent across languages, so the output keeps what the speaker meant and not only what they said.
Merged contributions to ESPnet (16k+ stars), including a ≈50× speedup for speaker-embedding extraction and new TTS recipes. Benchmarks and evaluation code ship with the research so other groups can build on it.
Author names in order; my name in bold.
Extends ARECHO (NeurIPS 2025 Spotlight) to predict speech quality before an utterance finishes. Dual-resolution query tokens with chunk-first decoding in a shared Transformer decoder cut PLCMOS MAE by 48% on 2-second prefixes and locate a 4 to 6 second perceptual context horizon.
Introduces off-target LID posterior mass as a measure that predicts probe gains on pre-trained speech encoders. Pre-registered on unseen language pairs and validated across Whisper, MMS, and classical LID on DISPLACE and MUCS Hindi-English code-switching data.
A multi-domain benchmark for turn-taking dynamics in spoken dialogue, measuring how well models anticipate when a speaker yields the floor across conversational settings.
HuBERT- and prosody-based speech-to-speech translation systems that preserve stance, emotion, and dialog function across English and Spanish.
Email is the fastest way to reach me.