Fourier at the heart of computer music: From harmonic sounds to texture

Vincent Lostanlen; Joakim Andén; Mathieu Lagrange

doi:10.1016/j.crhy.2019.07.005

Fourier and the science of today / Fourier et la science d'aujourd'hui

Fourier at the heart of computer music: From harmonic sounds to texture
[Fourier au cœur de la musique par ordinateur : des sons harmoniques à la texture]

Vincent Lostanlen ¹ ; Joakim Andén ² ; Mathieu Lagrange ³

¹ Music and Audio Research Lab, New York University, New York, NY, USA
² Center for Computational Mathematics, Flatiron Institute, New York, NY, USA
³ LS2N, CNRS, École centrale de Nantes, Nantes, France

Comptes Rendus. Physique, Volume 20 (2019) no. 5, pp. 461-473.

Résumé
Abstract

Au-delà de son apport théorique dans le domaine de la conduction thermique, le mémoire de Joseph Fourier sur la Théorie analytique de la chaleur (1822) a révolutionné notre conception des ondes sonores. Ce mémoire affirme que toute fonction de période unitaire se décompose en une série de sinusoïdes, chacune représentant une propriété essentielle du phénomène périodique étudié. En acoustique, cette décomposition révèle les modes de résonance d'une corde vibrante. Ainsi, l'introduction des séries de Fourier a ouvert de nouveaux horizons en matière de modélisation du timbre musical, un sujet qui prendra une importance cruciale à partir des années 1960, avec les débuts de la musique par ordinateur. Cet article propose de thématiser l'œuvre de Joseph Fourier à la lumière de ses implications en recherche musicale. Nous retraçons d'abord le changement de paradigme que les séries de Fourier ont opéré en acoustique, supplantant un mode de pensée fondé sur les consonances du monocorde pythagoricien. Par la suite, nous soulignons l'intérêt du paradigme de Fourier à travers trois problèmes pratiques en analyse-synthèse : l'imitation d'instruments de musique, la transposition fréquentielle, et la génération de textures sonores. Chacun de ses trois problèmes convoque une perspective différente sur la dualité temps–fréquence, et suscite un dialogue multidisciplinaire entre recherche et création qui est toujours d'actualité.

Beyond the scope of thermal conduction, Joseph Fourier's treatise on the Analytical Theory of Heat (1822) profoundly altered our understanding of acoustic waves. It posits that any function of unit period can be decomposed into a sum of sinusoids, whose respective contributions represent some essential property of the underlying periodic phenomenon. In acoustics, such a decomposition reveals the resonant modes of a freely vibrating string. The introduction of Fourier series thus opened new research avenues on the modeling of musical timbre—a topic that was to become of crucial importance in the 1960s with the advent of computer-generated sounds. This article proposes to revisit the scientific legacy of Joseph Fourier through the lens of computer music research. We first discuss how the Fourier series marked a paradigm shift in our understanding of acoustics, supplanting the theory of consonance of harmonics in the Pythagorean monochord. Then, we highlight the utility of Fourier's paradigm via three practical problems in analysis–synthesis: the imitation of musical instruments, frequency transposition, and the generation of audio textures. Interestingly, each of these problems involves a different perspective on time–frequency duality, and stimulates a multidisciplinary interplay between research and creation that is still ongoing.

Publié le : 2019-07-01

DOI : 10.1016/j.crhy.2019.07.005

Keywords: Fourier analysis, Computer music, Audio signal processing
Mot clés : Analyse de Fourier, Musique par ordinateur, Traitement du signal audio-numérique

Affiliations des auteurs :

Vincent Lostanlen ¹ ; Joakim Andén ² ; Mathieu Lagrange ³

¹ Music and Audio Research Lab, New York University, New York, NY, USA
² Center for Computational Mathematics, Flatiron Institute, New York, NY, USA
³ LS2N, CNRS, École centrale de Nantes, Nantes, France

@article{CRPHYS_2019__20_5_461_0,
     author = {Vincent Lostanlen and Joakim And\'en and Mathieu Lagrange},
     title = {Fourier at the heart of computer music: {From} harmonic sounds to texture},
     journal = {Comptes Rendus. Physique},
     pages = {461--473},
     publisher = {Elsevier},
     volume = {20},
     number = {5},
     year = {2019},
     doi = {10.1016/j.crhy.2019.07.005},
     language = {en},
}

TY  - JOUR
AU  - Vincent Lostanlen
AU  - Joakim Andén
AU  - Mathieu Lagrange
TI  - Fourier at the heart of computer music: From harmonic sounds to texture
JO  - Comptes Rendus. Physique
PY  - 2019
SP  - 461
EP  - 473
VL  - 20
IS  - 5
PB  - Elsevier
DO  - 10.1016/j.crhy.2019.07.005
LA  - en
ID  - CRPHYS_2019__20_5_461_0
ER  -

%0 Journal Article
%A Vincent Lostanlen
%A Joakim Andén
%A Mathieu Lagrange
%T Fourier at the heart of computer music: From harmonic sounds to texture
%J Comptes Rendus. Physique
%D 2019
%P 461-473
%V 20
%N 5
%I Elsevier
%R 10.1016/j.crhy.2019.07.005
%G en
%F CRPHYS_2019__20_5_461_0

Vincent Lostanlen; Joakim Andén; Mathieu Lagrange. Fourier at the heart of computer music: From harmonic sounds to texture. Comptes Rendus. Physique, Volume 20 (2019) no. 5, pp. 461-473. doi : 10.1016/j.crhy.2019.07.005. https://comptes-rendus.academie-sciences.fr/physique/articles/10.1016/j.crhy.2019.07.005/

Bibliographie
Cité par

[1] M.V. Mathews The digital computer as a musical instrument, Science, Volume 142 (1963) no. 3592, pp. 553-557

[2] C. Shannon Communication in the presence of noise, Proc. Inst. Radio Eng., Volume 37 (1949) no. 1, pp. 10-21

[3] U. Zölzer DAFX: Digital Audio Effects, John Wiley & Sons, 2011

[4] F. Jedrzejewski Mathématiques des sysèmes acoustiques, L'Harmattan, Paris, 2002

[5] M. Andreatta; F. Nicolas; C. Alunni À la lumière des mathématiques et à l'ombre de la philosophie, Actes du séminaire Mamuphi, Mathématiques–Musique–Philosophie, Ircam/Delatour, France, 2012

[6] M. Kac Can one hear the shape of a drum?, Am. Math. Mon., Volume 73 (1966) no. 4, Part 2, pp. 1-23

[7] M. Mersenne Harmonie universelle, contenant la théorie et la pratique de la musique, Bibliothèque nationale de France, 1636 (No. FRBNF30932210)

[8] J. Fourier Théorie analytique de la chaleur, Bibliothèque nationale de France, 1822 (No. FRBNF30454516)

[9] R.M. Friedman The creation of a new science: Joseph Fourier's analytical theory of heat, Hist. Stud. Phys. Sci., Volume 8 (1977), pp. 73-99

[10] A. Herreman L'inauguration des séries trigonométriques dans la Théorie analytique de la chaleur de Fourier et dans la controverse des cordes vibrantes, Rev. Histoire Math., Soc. Math. Fr., Volume 19 (2013) no. 2, pp. 151-243

[11] M.V. Mathews et al. The Technology of Computer Music. Vol. 969, MIT Press, Cambridge, MA, USA, 1969

[12] N.H. Fletcher; T.D. Rossing The Physics of Musical Instruments, Springer-Verlag, 1991

[13] H. Triebel Theory of Function Spaces, Birkhäuser Verlag, 1992

[14] S. McAdams et al. Perceptual scaling of synthesized musical timbres: common dimensions, specificities, and latent subject classes, Psychol. Res., Volume 58 (1995) no. 3, pp. 177-192

[15] B. Bogert; M. Healy; J. Tukey The quefrency analysis of time series for echoes: cepstrum, pseudo-autocovariance, cross-cepstrum and saphe cracking, Proceedings of a Symposium on Time Series Analysis, John Wiley & Sons, Inc., 1963, pp. 209-243

[16] P. Schaeffer Treatise on Musical Objects: An Essay Across Disciplines, University of California Press, Berkeley, USA, 2017

[17] J. Harvey The Music of Stockhausen: An Introduction, University of California Press, Berkeley, CA, USA, 1975

[18] P. Schaeffer In Search of a Concrete Music, Vol. 15, University of California Press, Berkeley, CA, USA, 2012

[19] J.M. Chowning The synthesis of complex audio spectra by means of frequency modulation, J. Audio Eng. Soc., Volume 21 (1973) no. 7, pp. 526-534

[20] H. Berjamin et al. Time-domain numerical modeling of brass instruments including nonlinear wave propagation, viscothermal losses, and lips vibration, Acta Acust. united Ac., Volume 103 (2017) no. 1, pp. 117-131

[21] M. Castellengo Écoute musicale et acoustique, Eyrolles, Paris, 2015

[22] X. Rodet; P. Depalle Spectral envelopes and inverse FFT synthesis, Portland, OR, USA ( 29–31 May 1992 )

[23] W. Koenig; H. Dunn; L. Lacy The sound spectrograph, J. Acoust. Soc. Am., Volume 18 (1946) no. 1, pp. 19-49

[24] J.-C. Risset Synthèse de sons à l'aide de calculateurs électroniques appliquée à l’étude de sons de trompette, C. R. Acad. Sci. Paris, Ser. B, Volume 263 (1966), pp. 111-114

[25] M.R. Schroeder Auditory paradox based on fractal waveform, J. Acoust. Soc. Am., Volume 79 (1986) no. 1, pp. 186-189

[26] H. Dudley The vocoder—electrical re-creation of speech, J. Soc. Motion Pict. Eng., Volume 34 (1940) no. 3, pp. 272-278

[27] J.L. Flanagan et al. Phase vocoder, J. Acoust. Soc. Am., Volume 38 (1965) no. 5, pp. 939-940

[28] J.-C. Risset Du songe au son. Entretiens avec Matthieu Guillot, L'Harmattan, Paris, 2008

[29] J. Laroche; M. Dolson Improved phase vocoder time–scale modification of audio, IEEE Trans. Speech Audio Process., Volume 7 (1999) no. 3, pp. 323-332

[30] M. Betser et al. Estimation of frequency for AM/FM models using the phase vocoder framework, IEEE Trans. Signal Process., Volume 56 (2008) no. 2, pp. 505-517

[31] M. Liuni et al. Automatic adaptation of the time–frequency resolution for sound analysis and re-synthesis, IEEE Trans. Audio Speech Lang. Process., Volume 21 (2013) no. 5, pp. 959-970

[32] A. Cont ANTESCOFO: anticipatory synchronization and control of interactive parameters in computer music, ICMC 2008, Belfast, Ireland, 24–29 August 2008, Michigan Publishing (2008), pp. 33-40

[33] T. Wishart The composition of Vox-5, Comput. Music J., Volume 12 (1988) no. 4, pp. 21-27

[34] R. Strachan Sonic Technologies: Popular Music, Digital Culture and the Creative Process, Bloomsbury Publishing, USA, 2017

[35] X. Serra; J.O. Smith Spectral modeling synthesis: a sound analysis/synthesis system based on a deterministic plus stochastic decomposition, Comput. Music J., Volume 14 (1990) no. 4, pp. 12-24

[36] K. Fitz; L. Haken; P. Christensen A new algorithm for bandwidth association in bandwidth-enhanced additive sound modeling, Berlin ( 27 August–1 September 2000 ), pp. 384-387

[37] J.H. McDermott; E.P. Simoncelli Sound texture perception via statistics of the auditory periphery: evidence from sound synthesis, Neuron, Volume 71 (2011) no. 5, pp. 926-940

[38] W.-H. Liao; A. Roebel; A.W.-Y. Su On the modeling of sound textures based on the STFT representation, Maynooth, Ireland, 2–4 September (2013)

[39] S. Mallat A Wavelet Tour of Signal Processing: The Sparse Way, Academic Press, 2009

[40] E.S. Ottosen et al. A phase vocoder based on nonstationary Gabor frames, IEEE Trans. Audio Speech Lang. Process., Volume 25 (2017) no. 11, pp. 2199-2208

[41] R. Kronland-Martinet; J. Morlet; A. Grossmann Analysis of sound patterns through wavelet transforms, Int. J. Pattern Recognit. Artif. Intell., Volume 1 (1987) no. 2, pp. 273-302

[42] I. Waldspurger Wavelet Transform Modulus: Phase Retrieval and Scattering, École normale supérieure, Paris, 2015 (PhD thesis)

[43] A. Röbel A new approach to transient processing in the phase vocoder, London, UK, 8–11 September (2003)

[44] A. Röbel A shape-invariant phase vocoder for speech transformation, Graz, Austria, 6–10 September (2010)

[45] J. Laroche; M. Dolson New phase-vocoder techniques for real-time pitch shifting, chorusing, harmonizing and other exotic audio modifications, J. Audio Eng. Soc., Volume 47 (1999) no. 11, pp. 928-936

[46] S. Mallat Understanding deep convolutional networks, Philos. Trans. R. Soc. A, Volume 374 (2016) no. 2065

[47] D. Depireux et al. Spectro-temporal response field characterization with dynamic ripples in ferret primary auditory cortex, J. Neurophysiol., Volume 85 (2001) no. 3, pp. 1220-1234

[48] J. Andén; V. Lostanlen; S. Mallat Joint time–frequency scattering, IEEE Trans. Signal Process., Volume 67 (2019) no. 14, pp. 3704-3718

[49] J. Bruna; S. Mallat Audio texture synthesis with scattering moments, 2013 (arXiv preprint) | arXiv

[50] I. Waldspurger Exponential decay of scattering coefficients, Tallinn, Estonia, 3–7 July (2017), pp. 143-146

[51] J. Andén; V. Lostanlen; S. Mallat Joint time-frequency scattering for audio classification, 17–20 September 2015, Boston, USA (2015) (6 pp) | DOI

[52] V. Lostanlen On time–frequency scattering and computer music (V.J.M. Nicolaus Schafhausen, ed.), Florian Hecker: Halluzination, Perspektive, Synthese, Sternberg Press, Berlin, 2019

[53] V. Lostanlen; F. Hecker The shape of RemiXXXes to come: audio texture synthesis with time-frequency scattering, Birmingham, UK, 2–6 September (2019)

Cité par Sources :

Commentaires - Politique