Research
Papers and projects on post-training, reinforcement learning, and multimodal learning, explored largely through music and audio.
Research interests
- Human-Aligned AI
- Self-Improving AI
- Human–AI Co-Evolution
- Reinforcement Learning
- Generative Model Post-Training
- Preference Learning
- Reward Modeling
- Adaptive Evaluation
- Multi-Agent Reinforcement Learning
- Social Learning
- Human–AI Coordination
- Adaptive Learning Environments
- World Models
- Multimodal Generative Modeling
- AI for Music
- Music Information Retrieval (MIR)
Publications
Ping Wang is shown in bold. Expand a paper for its abstract and BibTeX. Also on Google Scholar.
Ping Wang, Guang Yang, Nazif Can Tamer, Victoria Ebert, Noah A. Smith
Under review at TISMIR · 2026
Abstract›
Automatic music transcription systems produce sheet music that can be read and played back. We argue that these two targets call for complementary evaluations of notation similarity to a reference score and playback similarity to the original performance, respectively. Our study considers notation similarity metrics from the optical music recognition literature and a wide range of playback-similarity methods validated through a listening study across over 100 participants and 230 piano recordings covering 23 works, 30 performers, and six composers. We find, fortuitously, that the playback similarity metric that correlates best with human judgments, CLEWS, is also the cheapest to run. We also find that the two evaluation dimensions favor different systems among a collection of 24 pipelines formed by pairing eight audio-to-MIDI models with three MIDI-to-score converters, with the latter component systematically determining the favored objective. The complementarity between metrics also holds when adding to the pool Rubato, a new end-to-end system that offers substantially improved notation similarity while remaining competitive, though not the best, on playback similarity.
BibTeX›
@article{wang2026dual,
title = {A Dual Evaluation for Music Transcription},
author = {Wang, Ping and Yang, Guang and Tamer, Nazif Can and Ebert, Victoria and Smith, Noah A.},
journal = {arXiv preprint arXiv:2608.04511},
year = {2026},
}
Ping Wang, Guang Yang, Shao-Rong Su, Junkai Wu, Pang Wei Koh, Noah A. Smith
Under review at ICLR 2027 · 2026
Abstract›
Post-training text-to-music generation requires reward signals that capture multiple aspects of musical quality beyond what any single automatic metric can measure. We study structured, rubric-based rewards from pretrained audio-language models (ALMs) as training signals for both autoregressive and diffusion-based music generators. An ALM scores each generated clip against the rubric; we rank candidates generated for the same text prompt by their scores and convert these rankings into preference pairs for DPO on both MusicGen-small and ACE-Step v1, and additionally use the rubric scores directly as scalar rewards for DiffusionNFT on ACE-Step v1. On MusicCaps, rubric-based optimization improves CLAP, SongEval, and Audiobox-Aesthetics simultaneously, with the strongest gains obtained by DiffusionNFT on ACE-Step. By contrast, on MusicGen-small, building preferences from any one of these automatic evaluators produces clear cross-metric trade-offs: the targeted evaluator improves while other independent evaluators deteriorate. We further study tempo, key, and instrumentation, where precise objective rewards are available. Directly optimizing these specialized rewards reliably improves the target attributes, whereas ALM rubrics provide only partial transfer for tempo and instrumentation and no measurable improvement for key. Together, these results suggest a practical division of labor: ALM rubrics are effective for broad perceptual qualities that are difficult to formalize, while specialized objective rewards remain preferable when reliable measurements are available.
BibTeX›
@article{wang2026rubric,
title = {Rubric-Based Optimization for Text-to-Music Generation},
author = {Wang, Ping and Yang, Guang and Su, Shao-Rong and Wu, Junkai and Koh, Pang Wei and Smith, Noah A.},
journal = {arXiv preprint arXiv:2610.03589},
year = {2026},
}
- generation
- post-training
- reinforcement-learning
- evaluation
Ping Wang, Fang-Duo Tsai, Wei-Jaw Lee, Yi-Hsuan Yang
ISMIR 2026 Late-Breaking/Demo · 2026
Abstract›
A lead sheet is a compact, editable symbolic summary of a song. Sheet Sage showed that melody and chord symbols can be transcribed automatically from music audio, but it does not transcribe or align lyrics, and it predates several of the specialised MIR models now available. We present SheetSage-Pro, a modular audio-to-lead-sheet system that integrates recent pretrained models for separation, beat and downbeat tracking, structure analysis, melody, chords and lyric alignment, and exports a lyric-aligned lead sheet with bilingual Mandarin and English support. On public component benchmarks, scored with the same code for Sheet Sage and SheetSage-Pro, the new system improves pooled note F1@50 ms on GTSinger from 0.080 to 0.295 and maj/min chord WCSR on GuitarSet from 0.621 to 0.904, and completes every item. On MIR-ST500 its melody component reaches 0.670, comparable to dedicated singing transcribers trained on that dataset. Users can transcribe a song and inspect its melody, chords, bars, sections and lyrics aligned to the notes.
BibTeX›
@inproceedings{wang2026sheetsagepro,
title = {SheetSage-Pro: Bilingual Lyric-Aligned Lead-Sheet Transcription from Music Audio},
author = {Wang, Ping and Tsai, Fang-Duo and Lee, Wei-Jaw and Yang, Yi-Hsuan},
booktitle = {Late-Breaking/Demo Session of the 27th International Society for Music Information Retrieval Conference (ISMIR)},
year = {2026},
}
Projects
Research2025 · Completed
Cough detection, localization, and tuberculosis risk classification
Machine learning on cough audio: detecting and localizing cough events in continuous recordings, and classifying tuberculosis risk with transformer-based audio models.
Engineering2025 · Completed
Real-time gesture recognition on resource-constrained devices
Lightweight neural networks for real-time gesture recognition on embedded hardware, including quantization and deployment for on-device inference.
- embedded
- on-device-ml
- quantization