Rewriting Chinese songs with controllable content and rhyme scheme

Abstract

Although lyrics generation has achieved significant progress in recent years, it has limited practical applications. The generated lyrics cannot be performed without composing compatible melodies. In this work, we bridge this practical gap by proposing a song rewriting system which rewrites the lyrics of an existing song such that the generated lyrics are compatible with the rhythm of the existing melody and thus singable. In particular, we propose SongRewriter, a controllable Chinese lyric generation and editing system which assists users without prior knowledge of melody composition in generating performable lyrics. The system is trained by a randomized multi-level masking strategy which produces a unified model for generating entirely new lyrics or editing a fragment under optional controlled conditions such as keywords and rhyme schemes. During inference, several decoding constraints are incorporated to improve rhyme control and rhyming word diversity. While prior metrics to evaluate rhyme quality are mainly designed for rap lyrics, we propose novel rhyme evaluation metrics for lyrics of songs. We conduct extensive experiments, and both automatic and human evaluations show that the proposed model performs better than the state-of-the-art models in terms of contents and rhyme quality.

Date of Award	2022
Original language	English
Awarding Institution	The Hong Kong University of Science and Technology
Supervisor	Dit Yan YEUNG (Supervisor)