跳至主要內容
王平

研究

關於後訓練、強化學習與多模態學習的論文與專案,多半透過音樂與音訊來探索。

研究興趣

  • 與人類對齊的 AI
  • 自我改進的 AI
  • 人機共同演化
  • 強化學習
  • 生成模型後訓練
  • 偏好學習
  • 獎勵建模
  • 自適應評估
  • 多智能體強化學習
  • 社會學習
  • 人機協作
  • 自適應學習環境
  • 世界模型
  • 多模態生成建模
  • 音樂 AI
  • 音樂資訊檢索(MIR)

論文

Ping Wang 以粗體標示。展開論文可查看摘要與 BibTeX。完整列表也可見於 Google Scholar。

Ping Wang, Guang Yang, Nazif Can Tamer, Victoria Ebert, Noah A. Smith

Under review at TISMIR · 2026

摘要

Automatic music transcription systems produce sheet music that can be read and played back. We argue that these two targets call for complementary evaluations of notation similarity to a reference score and playback similarity to the original performance, respectively. Our study considers notation similarity metrics from the optical music recognition literature and a wide range of playback-similarity methods validated through a listening study across over 100 participants and 230 piano recordings covering 23 works, 30 performers, and six composers. We find, fortuitously, that the playback similarity metric that correlates best with human judgments, CLEWS, is also the cheapest to run. We also find that the two evaluation dimensions favor different systems among a collection of 24 pipelines formed by pairing eight audio-to-MIDI models with three MIDI-to-score converters, with the latter component systematically determining the favored objective. The complementarity between metrics also holds when adding to the pool Rubato, a new end-to-end system that offers substantially improved notation similarity while remaining competitive, though not the best, on playback similarity.

BibTeX
          @article{wang2026dual,
  title   = {A Dual Evaluation for Music Transcription},
  author  = {Wang, Ping and Yang, Guang and Tamer, Nazif Can and Ebert, Victoria and Smith, Noah A.},
  journal = {arXiv preprint arXiv:2608.04511},
  year    = {2026},
}
        
  • amt
  • evaluation
  • mir

Ping Wang, Guang Yang, Shao-Rong Su, Junkai Wu, Pang Wei Koh, Noah A. Smith

Under review at ICLR 2027 · 2026

摘要

Post-training text-to-music generation requires reward signals that capture multiple aspects of musical quality beyond what any single automatic metric can measure. We study structured, rubric-based rewards from pretrained audio-language models (ALMs) as training signals for both autoregressive and diffusion-based music generators. An ALM scores each generated clip against the rubric; we rank candidates generated for the same text prompt by their scores and convert these rankings into preference pairs for DPO on both MusicGen-small and ACE-Step v1, and additionally use the rubric scores directly as scalar rewards for DiffusionNFT on ACE-Step v1. On MusicCaps, rubric-based optimization improves CLAP, SongEval, and Audiobox-Aesthetics simultaneously, with the strongest gains obtained by DiffusionNFT on ACE-Step. By contrast, on MusicGen-small, building preferences from any one of these automatic evaluators produces clear cross-metric trade-offs: the targeted evaluator improves while other independent evaluators deteriorate. We further study tempo, key, and instrumentation, where precise objective rewards are available. Directly optimizing these specialized rewards reliably improves the target attributes, whereas ALM rubrics provide only partial transfer for tempo and instrumentation and no measurable improvement for key. Together, these results suggest a practical division of labor: ALM rubrics are effective for broad perceptual qualities that are difficult to formalize, while specialized objective rewards remain preferable when reliable measurements are available.

BibTeX
          @article{wang2026rubric,
  title   = {Rubric-Based Optimization for Text-to-Music Generation},
  author  = {Wang, Ping and Yang, Guang and Su, Shao-Rong and Wu, Junkai and Koh, Pang Wei and Smith, Noah A.},
  journal = {arXiv preprint arXiv:2610.03589},
  year    = {2026},
}
        
  • generation
  • post-training
  • reinforcement-learning
  • evaluation

Ping Wang, Fang-Duo Tsai, Wei-Jaw Lee, Yi-Hsuan Yang

ISMIR 2026 Late-Breaking/Demo · 2026

摘要

A lead sheet is a compact, editable symbolic summary of a song. Sheet Sage showed that melody and chord symbols can be transcribed automatically from music audio, but it does not transcribe or align lyrics, and it predates several of the specialised MIR models now available. We present SheetSage-Pro, a modular audio-to-lead-sheet system that integrates recent pretrained models for separation, beat and downbeat tracking, structure analysis, melody, chords and lyric alignment, and exports a lyric-aligned lead sheet with bilingual Mandarin and English support. On public component benchmarks, scored with the same code for Sheet Sage and SheetSage-Pro, the new system improves pooled note F1@50 ms on GTSinger from 0.080 to 0.295 and maj/min chord WCSR on GuitarSet from 0.621 to 0.904, and completes every item. On MIR-ST500 its melody component reaches 0.670, comparable to dedicated singing transcribers trained on that dataset. Users can transcribe a song and inspect its melody, chords, bars, sections and lyrics aligned to the notes.

BibTeX
          @inproceedings{wang2026sheetsagepro,
  title     = {SheetSage-Pro: Bilingual Lyric-Aligned Lead-Sheet Transcription from Music Audio},
  author    = {Wang, Ping and Tsai, Fang-Duo and Lee, Wei-Jaw and Yang, Yi-Hsuan},
  booktitle = {Late-Breaking/Demo Session of the 27th International Society for Music Information Retrieval Conference (ISMIR)},
  year      = {2026},
}
        
  • amt
  • lead-sheets
  • lyrics
  • mir

專案

研究2025 · 已完成

咳嗽音訊健康篩檢

咳嗽偵測、定位與結核病風險分類

以機器學習分析咳嗽音訊:在連續錄音中偵測並定位咳嗽事件,並使用以 Transformer 為基礎的音訊模型分類結核病風險。

  • audio
  • health
  • transformers
工程2025 · 已完成

嵌入式手勢辨識

在資源受限的裝置上即時辨識手勢

在嵌入式硬體上進行即時手勢辨識的輕量化神經網路,包含模型量化與裝置端推論部署。

  • embedded
  • on-device-ml
  • quantization