I've been experimenting with methods for visualising harmony, intonation (tuning), and overtones in music. Ordinary spectrograms aren't very well suited for that as the harmonic relations are not intuitively visible. Let's see what could be done about this. I'll try to sprinkle the text with Wikipedia links in order to immerse (nerd snipe?) the reader in the subject.
Equal temperament cents against time
We can examine how tuning evolves during a recording by choosing a reference pitch and plotting all frequencies relative to it modulo 100 cents. This is similar to what an electronic tuner does, but instead of just showing the fundamental frequency, we'll plot the whole spectrum. Information about the absolute frequencies is lost. This "zoomed-in" plot visualises how the distribution of frequencies fits the 12-tone equal temperament system (12-TET) common in Western music.
Here's the first 20 seconds of Debussy's Clair De Lune as found on YouTube, played with a well-tuned (video) and an out-of-tune piano (video). The second piano sounds out of tune because there are relative differences in tuning between the strings. The first piano looks to be a few cents sharp as a whole, but consistently so, so it's not perceptible.
The vertical axis is the same that electronic tuners use. All the notes of a chord will appear in the middle, as long as they are well tuned against the reference pitch of, say, A = 440 Hz. The top edge of the graph is half a semitone sharp (quarter tone = 50c), and the bottom is half a semitone flat.
Overtones barely appear in the picture because the first three conveniently hit other notes in tune. But from f5 onwards the harmonic series starts deviating from 12-TET and the harmonics start to look out-of-tune (f5 = −14c, f7 = −31c, ...). These can be cut out by filtering, or hoping that they're lower in volume and setting the color range accordingly.
Six months later...
You know how you sometimes accidentally delete a project and have to rewrite it from scratch much later, and it's never exactly the same? That's what happened at this point. It's a little scuffed, but here's some piano music that utilises quarter tones (by Wyschnegradsky). I used the same scale on purpose, so the quarter tones wrap around from the top and bottom.
More examples of these "intonation plots" in a tweet.
This suits well for piano music. However, not even all western classical music is tuned to equal temperament; for instance, solo strings may be played with Pythagorean intonation[1], whereas vocal ensembles[2] and string quartets may tune some intervals closer to just intonation. Unlike the piano, these wouldn't look too good in the equal-note plot.
Octave spiral
If we instead plot the spectrum modulo 1200 cents (1 octave) we get an interesting interval view. We could even ditch the time scale and wind the spectrogram into a spiral to make it prettier and preserve the overtones and absolute frequency information. Now each note is represented by an angle in this spiral; in 12-TET, they're separated by 30 degrees. At any point in the spiral, going 1 turn outwards doubles the frequency.
Here's a C major chord on a piano. Note how the harmonic content adds several high-frequency notes on top of the C, E, and G of the triad chord, and how multiple notes can contribute to the same overtone:
I had actually hoped I would get an Ultimate Chord Viewer where any note, regardless of frequency, would have all its harmonics neatly stacked on top of it. But it's not what has happened here: the harmonic series is not a stack of octaves (2^n), but instead of integer multiples (n). Some harmonics appear at seemingly unrelated angles. But it's still a pretty interesting visualisation, and perhaps makes more sense musically.
This plot is also better at illustrating different tuning systems. Let's look at a major third interval F-A in equal temperament and just intonation, with a few more harmonics.
The intuition from this plot is that equal temperament aims for equal distances (angles) between notes and just intonation tries to make more of the harmonics match instead.
Even though the Ultimate Chord Viewer was not achieved I now have ideas for Christmas lights...
Live visualization
Here's what a cappella music with some reverb looks like on the spiral spectrogram.
A shader toy
This little real-time GLSL demo on shadertoy draws a spiral FFT from microphone audio. But don't get your hopes up: The spectrograms in this blog post were made with a 65536-point FFT. Shadertoy's 512px microphone texture offers a lot less in terms of frequency range and bins. This greatly blurs the frequency resolution, especially towars the low end. Could it be improved with the right colormap? Or a custom FFT with the waveform texture as its input?
References
- Loosen, F. (1994): Tuning of diatonic scales by violinists, pianists, and nonmusicians. Perception & Psychophysics 56(2): 221–226.
- D'Amario, S., Howard, D.M., Daffern, H., Pennill, N. (2020): A Longitudinal Study of Intonation in an a cappella Singing Quintet. Journal of Voice 34(1): 159.e13–159.e27.
![[Image: Two spectrograms labeled 'Piano in tune' and 'Piano out of tune'. The first one shows blobs of light along the center axis. In the second one, the blobs are jumping up and down the graph.]](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEitcu6_yTFzFwmNwpEV8KFKefpfG4kILVbRoBMjoYw5b0qXJnWab3aMCmvhrFGOKs3J-EvJi6DiXJI1hzzBAXSdY19HBoeYrwj35nKpqjhtVutYaMHJ4riEuE_wg5p3BbN5tfonWsQ__1F2/s500/debussy.png)
![[Image: Similar as above, but blobs are also seen at the very top and bottom of the graph.]](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg_w_F10RjlAfFh2dfoUk6n_cghhzaPR7sYG45J9gtehVncrO-nY1fL4ZGizUo4j41b781dzUBwj60-EkX9L_fpin0Ma-GnSGl_7ZFZcb6Upbzt2V-AL7HPCpy6y0NaFkeh1oJx7yVj9XkQ/s500/quartertones.png)
![[Image: 12 note names labeled around a spiral. The C, E, and G notes light up sequentially, and their frequency and harmonics are displayed on the spiral.]](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhFURiTfZMg_rFGPBD2invqT_bFHNGfpc-pstZCR6IpeZR1w7-K9MkfK6Rp-MnyBlwzypV5nDOfqsIgszWvjMXPEvVZlVGUUg8JRdYuLf85kAKjJdJ6Slgs8ipm39oEEQuZOga-nIv_Ex4h/s390/cduuri-plot.gif)
![[Image: The interval alternates between 400 and 386 cents. When it's 386, a few harmonics of the F note merge with those of the A note.]](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhXFPWrnuz599rW3Yw5TrOFUGmj9cAHK7c2eBYwS0kZzKvo1OP6uTcwPsuUu_NyD5esMk04cn4n1Xjkg7mu8bWF17ssjIgS4l-FKAKo23_2AjHmS_XC32NmbVdmlOTTRVb99BVFetsdzgqM/s500/just-equal-optimized.gif)
![[Image: Close-up of a vinyl record showing a wavy pattern.]](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgIK350lA9fMo-Z9PohL6SBuT_Xkl7mOGVsHRblgbnNBp7cLfnTWTXkWSJmiwMy5KD7PVhJGnISfBLfKrNTj8oR9-x5Qb33xGyx3Gs0Te13Aa-6jfWw1gLq5kMI936vJF30xCzpzVKtjMf8/s320/BlackPink.jpg)
![[Image: A plot showing similar patterns that were on the disc surface.]](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgjfo5UWLOUu5REWu2Pywu_t1WgP6UJZ5W9DpzWU2NBuej4TthDHygkCXWka07TwdFMIpkroeqYT15OA3S19IcNU4nzb15WZ7fyyNmI6h0CE4A7VBTSXX-DhRDaGNlIYa2AG3NKQ9NGxO6_/s1024/blackpink2.png)
![[Image: A square image of the disc video texture.]](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjPisSQ0k9f16Mkoi7KOWCajT5YBDRJbUt0QjbR7mIWw3LZD22hyphenhyphenS9o4HTOCrkbjlEr9Q8la_GWX-CUZeotP9V8yBj7tdogf4dM351kjbrt1YTsMsc1E8vUf98ZophoMLD57qxwmPMv7vwn/s512/disc-pigpen-insync.jpg)
![[Image: Screenshot of C++ code with a list of events.]](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWdBWR5I5I1mTXJs721g0ZtFf5ixXLtAqnh96HYgl_ICY1u_vgb_OmR5hqidOjSyOeo8lhomcHU6Kl4EGhHZVW7ItkV3Qx_VH0Pe8H9lThTNk0ekEKenZhRXx55glQ_mXYbsapu-uLmqGx/s479/events-screenshot.png)
![[Image: A screenshot of Blender with the device without textures.]](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZGeWmeXgvDoQaLk2DzbwzbB9gftTUJqL6fJP0KPXjMdMAtsKeFbcf9tEzkso-FJHZEnjhGU5gebXX-bJ6JOZrf7K2NrQbjKuapeGXzsfBfopsSjsl2ip6SR_iF5SqRJGEcUN-DqA4sjLA/s500/modeling-psyche4.jpg)
![[Image: A Blender texture node map.]](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgIF0nbsZgcGp5_hTtXkrSDtcjcyTQ0p072dWDuut948V5VK1KbZQJk9LRs6CrsFiM3EPLtez-2a7cPeGIfI-hG9D_VfqWXyrGRcwx7UXDUBm4_Y-p9-oWW_kQFWqokP2Q03raDENghfyB5/s500/disc-shader.jpg)
![[Image: A render of the record player.]](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgCWvYWzuSFKzHBqwtb5aICiMpF1aJNpDntyNnTZOGh9FY4aObotdk4IB4MmasuG6uLPw2pOTtfVC4xJSTlHWrUpMPsEXQKMasaKnmDXoSHdb9TTHKtWcUIK0s6tyPR_Zu3cSNatrbaiLgw/s500/psyche-render1.jpg)
![[Image: A screenshot of a Python script.]](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgoOUqggMM8fUV4EK7B4vh9P71EfJsgxbMDTzZbecqYHCO1GcFs2yPEoDCvdMP7jRljmO4wcMT8ru_mDjRZUOdFMcvj3h-UWXjvH0VX4anuZUmnHn4PYoCUTNq27zi_alHwsQeMF0etPAm_/s635/python-csv-import.jpg)
![[Image: White concentric circles on a black background, with evenly separated lines connecting them.]](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEicIPnEytmyXedVbulIXkF378vzJR4EYLKX3ygpqWbJJcn-6p_AGQUB3jt2R4vKotN_0dxB5edNDpQAe4Rc1w6SKiCOyLRbKJR4USqIxof6rgvea29M71-q_0ymcPiuUHtZBZGmOSLLlRZp/s450/spokes.png)
![[Image: A zoomed view of a circle, showing the glow effect.]](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjNAONoJIEObnIVGE-AbyecROvA4VkTER3ic2TDQ5nHXI5GYfvHEejAruFS33WyLTCruuN0lO2Mo4AWXOQL8iQYZLhpvQ9jExSwQmRz2diVCoU3jPWjjB7VqVJwD2Z2oRXRSzM2JHdaAAx7/s450/glow-zoomed.png)
![[Image: A zoomed view of a circle, showing the color overlay effect.]](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjnljOUCNrwy85yDcP0YzmqcEFu7f6W0uqegcsrFkhj4jTG9oMivCOJRrCAWgMyjYAAI0TKTXQZ-PwRMmlhrR9BpeBiFh9yFaZCUxI9MXxrIGbfCA78zvSrbs0TcLcmJwOXnCpUrBLJqWd2/s450/overlay-effect.png)
![[Image: A zoomed view of the text 'finally' with a glow and color overlay effect.]](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEirmsMDeVOsnMtHYkFwICo5_uXRRFuUEnk_Cyewpe2mqHEJkIWdWt3I3rgMvDdcy0SAM5FNFoYKxekllK3zP53uhHpMLmk_C69yUSLlc_YoR0KlOAy0eaGJb7RWFvXoPS5mpO9-zAt2IdHA/s450/finally.png)
![[Image: Detail of an Audacity screenshot, showing its oscillogram viewer zoomed in so much that only a few dozen samples, represented by points in the X-Y plane, are on screen. Adjacent samples are connected to each other by straight lines, and there's a zero-crossing between every sample.]](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEitbmvykVcbEI6GZIvV7g4eEkmAIYgL_XUUkba-EpBA8oCyfvfTTBstyPM8P2M3Au118PZyHxBaPayLz2gVanHrHla-nc0ujVFKuAIlhbg6lKXg1tAyOBftlg7hoc62adQRq90vSgAzcYG-/s500/aaltoauda4.png)
![[Image: The same samples as in the above screenshot, but instead of connecting adjacent samples to each other, the samples are connected to the X axis using a dotted line.]](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiY98zqeZfypfaV0nNALVbtIzexbbjZ2AbqyorOpd4fZE7viFCYQivh1sXRp-OFY-uD4FpumBtkBhjCaoIzV_VfvnjlXLYUCA3NMa7AaswWwFLBeoiS0JKvTPlGGdAzEDOFF5yqrLtd5UO2/s450/maybe4.png)
![[Image: The same set of samples as in the above images, but this time, a sinusoid of constant frequency and amplitude is drawn that passes though all the points.]](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhIHTLVIo3z8XMVnePrRR7Gtj2p2wJlXlNc3DI_xQzXsDqStKrt45qtnNTTf05yF77DBpzCTKhLDcoDd8igK1Us5jGdjLpkQ6ntcW2HuFDaT7McAJfoV02IXwPA8bVEyHlOJcG3TZUkuG-k/s450/maybe3a.png)
![[Image: Screenshot showing the oscillogram viewer of Audacity. 500 milliseconds of a signal is shown, beginning as a thin line that suddenly makes a positive rise, a slightly slower fall and then changes into a wide, solidly filled surface, with a lighter color in the middle.]](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHvGAVIn0TPxlkAhbMzEgJH_VjlsMobv8LUCmmv2z0hTFXNwNxDybbGzhFE67OX7Z6rJ1yXFq0P6v9wehXAdzOG7o9k1GpDJNAVvPHwsa0O7eEy3GhopjbLeooReO9W_TdFInR9wnQX5m2/s480/Screenshot+-+30.03.2013+-+11.29.15.png)

![[Image: An Audacity oscillogram zoomed in so much that only a few dozen samples span the picture. Samples are drawn as dots that are connected by straight lines. Below it, another oscillogram of the same waveform, with a smooth waveform passing through all the samples.]](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi5GEoIDdfJ6oZb9JnjBdRd1npTNP8s1A2jpbfFy-giyHRW38ZwwZ91mzR-I5gZNSRD6jgJ5rem6zf8kE9X90OCIHAwFf-73AZuOEWBHI38TyK8-zTAO_G1RhR4XlC8hp2tfriTk4aMMnG3/s480/Screenshot+-+30.03.2013+-+12.21.19.png)
![[Image: An oscillogram of a signal varying in both time and frequency, revealing inner structure as brightness variations.]](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjYTn8rADYJR1FWd985KFvq1-Ko7D7SyvvbBevuaNrQY-nd4NtKk4liVzmaKvsrbY5CvLPl40X-yNLMO2ommcD4CRViiVZ57jI0YVP_qINNsXhTcEEsjpFQ1OwE8SEgT3N0GO-IKYfKa6IY/s480/osc.png)