GramError: a quality metric for machine generated songs.
MetadataShow full item record
DAVIES, C., WIRATUNGA, N. and MARTIN, K. 2018. GramError: a quality metric for machine generated songs. In Bramer, M. and PETRIDIS, M. (eds.) Artificial intelligence xxxv: proceedings of the 38th British Computer Society's specialist group on artificial intelligence (SGAI) annual international artificial intelligence conference (AI-2018), 11-13 December 2018, Cambridge, UK. Lecture notes in artificial intelligence, 11311. Cham: Springer [online], pages 184-190. Available from: https://doi.org/10.1007/978-3-030-04191-5_16
This paper explores whether a simple grammar-based metric can accurately predict human opinion of machine-generated song lyrics quality. The proposed metric considers the percentage of words written in natural English and the number of grammatical errors to rate the quality of machine-generated lyrics. We use a state-of-the-art Recurrent Neural Network (RNN) model and adapt it to lyric generation by re-training on the lyrics of 5,000 songs. For our initial user trial, we use a small sample of songs generated by the RNN to calibrate the metric. Songs selected on the basis of this metric are further evaluated using 'Turinglike' tests to establish whether there is a correlation between metric score and human judgment. Our results show that there is strong correlation with human opinion, especially at lower levels of song quality. They also show that 75% of the RNN-generated lyrics passed for human-generated over 30% of the time.