热门数字音频预测技术综述

doi:10.11871/jfdc.issn.2096-742X.2021.04.007

数据与计算发展前沿 ›› 2021, Vol. 3 ›› Issue (4): 81-92.

doi: 10.11871/jfdc.issn.2096-742X.2021.04.007

A Survey on Popular Digital Audio Prediction Techniques

ZHANG Yining^1,²(),HE Hongbo^1,^*(),WANG Runqiang¹()

1. Computer Network Information Center, Chinese Academy of Sciences, Beijing 100190, China
2. University of Chinese Academy of Sciences, Beijing 100049, China

Received:2021-01-21 Online:2021-08-20 Published:2021-08-30
Contact: HE Hongbo

摘要/Abstract

摘要：

【目的】近些年网络数字音频受众愈发广泛,研究热门数字音频预测技术对于数字音频领域的发展具有重要意义。【文献范围】我们采用关键词检索和引文二次检索的方法收集了该领域相关的论文。【方法】本文通过广泛的文献查阅,总结了在该研究领域中学者们对热门指标的定义,归纳了预测热门音频常用的四大类内部特征,综述和分析了常用的预测模型,并展望了热门数字音频预测技术未来的发展趋势和研究方向。【结果】通过选取恰当的特征表示,可以成功地预测热门音乐与热门播客,其中热门音乐预测领域的研究成果更为丰富可观。【局限】国内学术界对热门音频预测领域开展的研究较少,因而所能检索到的中文文献也较为匮乏。【结论】热门数字音频预测领域仍然存在着广阔的发展空间,尤其是我国热门播客预测领域仍存在着很大的研究空白。

关键词: 数字音频, 音乐, 播客, 热度预测, 音频特征, 机器学习, 深度学习

Abstract:

[Objective] In recent years, the number of online digital audio audiences have increased greatly. It is of great significance to study the popular digital audio prediction techniques for the development of digital audio systems. [Coverage] Relevant papers in this field are collected by using keyword search and citation retrieval. [Methods] Through extensive literature review, we have summarized the definitions of popular indicators by scholars in this research field, categorized the four main types of internal features commonly used for predicting popular audio, reviewed and analyzed commonly used prediction models. We also forecast the future development trends and research directions of this field. [Results] By selecting appropriate feature representations, popular music and popular podcasts can be successfully predicted, in which the research on popular music prediction are more versatile and impressive. [Limitations] There is little domestic research in the field of popular audio prediction, so the number of retrieved Chinese literature is very small. [Conclusions] There is still huge growth potential in popular digital audio prediction, especially for podcast prediction, which is still an underdeveloped realm in China.

Key words: digital audio, music, podcast, popularity prediction, audio features, machine learning, deep learning

张怡宁,何洪波,王闰强. 热门数字音频预测技术综述[J]. 数据与计算发展前沿, 2021, 3(4): 81-92.

ZHANG Yining,HE Hongbo,WANG Runqiang. A Survey on Popular Digital Audio Prediction Techniques[J]. Frontiers of Data and Computing, 2021, 3(4): 81-92.

图/表 4

表1

表2

表3

表4

参考文献 38

[1]	艾媒大文娱产业研究中心. 2020上半年中国在线音频市场研究报告[R]. 广州:广州艾媒数聚信息咨询股份有限公司, 2020.
[2]	Paul Lee, Jeff Loucks, Duncan Stewart, et al. “听”时代来临有声读物与播客迎来新风口[R]. 美国:德勤洞察, 2019.
[3]	李明扬. 广播电视工程中数字音频技术的优势与应用发展[J]. 中国科技投资, 2014,000(004):488-488.
[4]	Singhi A, Brown D G. Hit song detection using lyric features alone[C]//ISMIR. 2014: 1-2.
[5]	Georgieva E, Suta M, Burton N. Hitpredict: Predicting Hit Songs Using Spotify Data Stanford Computer Science 229: Machine Learning[C]//Center for Computer Rese-arch in Music and Acoustics, Stanford University, USA, Department of Civil and Environmental Engineer-ing. 2018: 1-5.
[6]	Zangerle E, Vötter M, Huber R, et al. Hit Song Prediction: Leveraging Low-and High-Level Audio Features[C]//ISMIR. 2019: 319-326.
[7]	Ni Y, Santos-Rodriguez R, Mcvicar M, et al. Hit song science once again a science[C]//4th International Work-shop on Machine Learning and Music. 2011: 1-2.
[8]	Fan J, Casey M. Study of Chinese and UK hit songs prediction[C]//Proceedings of International Symposium on Computer Music Multidisciplinary Research. 2013: 640-652.
[9]	Herremans D, Martens D, Sörensen K. Dance hit song prediction[J]. Journal of New Music Research, 2014,43(3):291-302. doi: 10.1080/09298215.2014.881888
[10]	Wang K. Predicting Hit Songs with MIDI Musical Features Stanford Computer Science 229: Machine Learning[C]//Center for Computer Research in Music and Acoustics, Stanford University, USA. 2014: 1-5.
[11]	Herremans D, Bergmans T. Hit song prediction based on early adopter data and audio features[J]. arXiv preprint arXiv:2010.09489, 2020.
[12]	Dhanaraj R, Logan B. Automatic Prediction of Hit Songs[C]//ISMIR. 2005: 488-491.
[13]	Lee J, Lee J S. Music Popularity: Metrics, Character-istics, Audio-Based Prediction[J]. IEEE Transactions on Multimedia, 2018,PP(11):1-1.
[14]	Borg N, Hokkanen G. What makes for a hit pop song? What makes for a pop song Stanford Computer Science 229: Machine Learning[C]// Stanford University, USA. 2011:1-5.
[15]	Yang L C, Chou S Y, Liu J Y, et al. Revisiting the problem of audio-based hit song prediction using convolutional neural networks[C]//2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2017: 621-625.
[16]	Yu L C, Yang Y H, Hung Y N, et al. Hit song prediction for pop music by siamese CNN with ranking loss[J]. arXiv preprint arXiv:1710.10814, 2017.
[17]	Pham J, Kyauk E, Park E. Predicting song popularity Stanford Computer Science 229: Machine Learning[C]// Stanford University, USA. 2015: 26.
[18]	Spotify A B. Spotify Web API[Z]. USA: Spotify USA Inc, 2018.
[19]	Tsagkias M, Larson M, De Rijke M. Exploiting surface features for the prediction of podcast preference[C]//European Conference on Information Retrieval. Springer, Berlin, Heidelberg, 2009: 473-484.
[20]	Yang L, Wang Y, Dunne D, et al. More than just words: Modeling non-textual characteristics of podcasts[C]//Proceedings of the Twelfth ACM International Confer-ence on Web Search and Data Mining. 2019: 276-284.
[21]	Joshi B, Mittal S, Chetan A. Did You “Read” the Next Episode? Using Textual Cues for Predicting Podcast Popularity[C]//Proceedings of the 1st Workshop on NLP for Music and Audio (NLP4MusA). 2020: 13-17.
[22]	Interiano M, Kazemi K, Wang L, et al. Musical trends and predictability of success in contemporary songs in and out of the top charts[J]. Royal Society open science, 2018,5(5):1-16.
[23]	Bischoff K, Firan C S, Georgescu M, et al. Social know-ledge-driven music hit prediction[C]//International Con-ference on Advanced Data Mining and Applications. Spr-inger, Berlin, Heidelberg, 2009: 43-54.
[24]	Lee J, Lee J S. Music popularity: Metrics, characteristics, and audio-based prediction[J]. IEEE Transactions on Multimedia, 2018,20(11):3173-3182. doi: 10.1109/TMM.2018.2820903
[25]	Monterola C, Abundo C, Tugaff J, et al. Prediction of potential hit song and musical genre using artificial neural networks[J]. International Journal of Modern Physics C, 2009,20(11):1697-1718. doi: 10.1142/S0129183109014680
[26]	Porter A, Bogdanov D, Kaye R, et al. Acousticbrainz: a community platform for gathering music information obtained from audio[C]// Proceedings of the 16th ISMIR Conference. 2015: 787.
[27]	Tsagkias M, Larson M, Weerkamp W, et al. PodCred: A framework for analyzing podcast preference[C]//Proce-edings of the 2nd ACM workshop on Information credibility on the web. 2008: 67-74.
[28]	Hirjee H, Brown D G. Rhyme analyzer: An analysis tool for rap lyrics[C]//Proceedings of the 11th International Society for Music Information Retrieval Conference. 2010: 1.
[29]	Elovitz H S, Johnson R W, McHugh A, et al. Automatic translation of English text to phonetics by means of letter-to-sound rules[R]. NAVAL RESEARCH LAB WASH-INGTON DC, 1976.
[30]	Liu J Y, Yang Y H. Event localization in music auto-tagging[C]//Proceedings of the 24th ACM international conference on Multimedia. 2016: 1048-1057.
[31]	Chon S H, Slaney M, Berger J. Predicting success from music sales data: a statistical and adaptive approach[C]//Proceedings of the 1st ACM workshop on Audio and music computing multimedia. 2006: 83-88.
[32]	储中明, 肖邓杰, 乔予思, 万金宇. 机器学习在粒子加速器的应用[J]. 数据与计算发展前沿, 2019,1(2):110-120.
[33]	Kleinbaum D G, Dietz K, Gail M, et al. Logistic regression[M]. New York: Springer-Verlag, 2002: 1-39.
[34]	王彦棡, 王珏, 曹荣强. 人工智能计算与数据服务平台的研究与应用[J]. 数据与计算发展前沿, 2019,1(2):86-97.
[35]	Salganik M J, Dodds P S, Watts D J. Experimental study of inequality and unpredictability in an artificial cultural market[J]. science, 2006,311(5762):854-856. pmid: 16469928
[36]	Pachet F, Roy P. Hit Song Science Is Not Yet a Science[C]//ISMIR. 2008: 355-360.
[37]	Reiman, Minna, Philippa Örnell. Predicting Hit Songs with Machine Learning[D]. Swenden: Royal Insti-tute of Technology, 2018: 1-37.
[38]	廖方宇, 洪学海, 汪洋, 褚大伟. 数据与计算平台是驱动当代科学研究发展的重要基础设施[J]. 数据与计算发展前沿, 2019,1(1):2-10.

特征类型	特征名称	特征英文名	特征解释
音频浅层特征	节拍	Tempo	以每分钟节拍数量表示的平均节奏。
音频深层特征	可舞蹈性	Danceability	描述一个音乐是否适合跳舞。数字范围从0到1,由the Echo Nest计算,基于节拍强度、节奏稳定性、整体节奏等计算。
文本特征	词汇线索	Lexical Cues	歌词或播客文本中某关键词出现的频率。
元数据特征	作者名称	Author	音频作者名称的字符串。

音频属性	预测为热门音频	预测为非热门音频
实际为热门音频	TP	FN
实际为非热门音频	FP	TN

指标名称	英文名	计算公式	意义
准确率	Accuracy	$\frac{TP+TN}{TP+TN+FP+FN}$	所有预测类别正确的样本比例,反映预测模型的综合预测效果。
精确率	Precision	$\frac{TP}{TP+FP}$	在所有实际的热门音频中,有多少比例的热门音频被成功预测出。
召回率	Recall	$\frac{TP}{TP+FN}$	在所有预测为热门的音频中,有多少比例的音频为实际热门音频。
F1值	F1 Score	$\frac{2recallprecision}{recall+precision}$	一个综合了精确率和召回率的评价指标。
AUC	Area Under Curve	P(P_热门音频>P_{非热门音频})	随机抽出一对样本（一个实际热门音频,一个实际非热门音频）,然后用训练得到的分类器来对这两个样本进行预测,预测得到热门的概率大于非热门的概率的概率。

方法	预测类型	特征选择	方法复杂度	预测效果	备注
支持向量机	二分类	音频特征、文本特征和元数据特征	低	好	适用于小型样本库。
逻辑回归	二分类	音频特征、文本特征和元数据特征	低	好	不能解决非线性问题,依赖特征工程。
贝叶斯网络	二分类	文本特征	低	中	不能对特征变量进行筛选。
随机森林	二分类和排名预测	元数据特征	中	好	是机器学习算法中计算成本较高的方法。
XG-Boost	二分类	文本特征	中	好	算法较复杂,计算成本较高。
卷积神经网络	二分类	音频特征和元数据特征	高	好	深层结构模型效果往往更好。
Siamese-CNN	排名预测	音频特征和元数据特征	高	好	需使用歌曲对进行训练。