By now I've rambled quite a lot about RDS, the data subcarrier on the third harmonic of the 19 kHz FM stereo pilot tone. And we know the second harmonic carries the stereo information. But is there something beyond that?
Using a wideband FM receiver, like an RTL-SDR, I can plot the whole demodulated spectrum of any station, say, from baseband to 90 kHz. Most often there is nothing but silence above the third pilot harmonic (denoted 3f here), which is the RDS subcarrier. But two local stations have this kind of a peculiar spectrogram:
There seems to be lots of more data on a pretty wide band centered on the fourth harmonic (4f)!
As so often with mysterious persistent signals, I got a lead by googling for the center frequency (76 kHz). So, meet the imaginatively named Data Radio Channel, or DARC for short. This 16,000 bps data stream uses level-controlled minimum-shift keying (L-MSK), which can be thought of as a kind of offset-quadrature phase-shift keying (O-QPSK) where consecutive bits are sent alternating between the in-phase and quadrature channels.
To find out more, I'll need to detect the signal first. Detecting L-MSK coherently is non-trivial ("tricky") for a hobbyist like me. So I'm going to cheat a little and treat the signal as continuous-phase but non-coherent binary FSK instead. This should give us good data, even though the bit error probability will be suboptimally high. I'll use a band-pass filter first; then a splitter filter to split the signal band into mark and space; detect both envelopes and calculate difference; lowpass filter with a cutoff at the bitrate; recover bit timing and synchronize using a PLL; and now we can just take the bits out by hard decision at bit middle points.
Oh yeah, we have a bitstream!
Decoder software for DARC does not exist, according to search engines. To read the actual data frames, I'll have to acquire block synchronization, regenerate the pseudorandom scrambler used and reverse its action, check the data against CRCs for errors, and implement the stack of layers that make up the protocol. The DARC specification itself is luckily public, albeit not very helpfully written; in contrast to the RDS standard, it contains almost no example circuits, data, or calculations. So, a little recap of Galois field mathematics and linear feedback shift registers for me.
But what does it all mean? And who's listening to it?
Now, it's no secret (Signaali 3/2011, p. 16) that the HSL bus stop timetable displays in Helsinki get their information about the next arriving GPS positioned buses through this very FM station, YLE 1. That's the only thing they're telling though. I haven't found anything bus-related in the RDS data, so it's quite likely that they're listening to DARC.
DARC is known to be used in various other applications as well, such as DGPS correction signals. So decoding it could prove interesting.
Sniffing the packet types being sent, it seems that quite a big part of the time is being used to send a transmit cycle schedule (COT and SCOT). And indeed, the aforementioned magazine article hints that the battery-powered displays use a precise radio schedule to save power, receiving only during a short period every minute each. (This grep only lists metadata type packets, not the actual data.)
$ perl darcdec-filters.pl | grep Type: | sort | uniq -c
88 Type: 0 Channel Organization Table (COT)
8 Type: 1 Alternative Frequency Table (AFT)
8 Type: 2 Service Alternative Frequency Table (SAFT)
1 Type: 4 Service Name Table (SNT)
8 Type: 5 Time and Date Table (TDT)
112 Type: 6 Synchronous Channel Organization Table (SCOT)
$ █
At the moment I'm only getting sensible data out at the very beginning of each packet (or "block"). I do get a solid, error-free block sync but, for example, the date is consistently showing the year 1922 and all CRCs fail. Other fields are similarly weird but consistent. This could mean that I've still got the descrambler PN polynomial wrong, only succeeding when it's using bits from the initialization seed. And this can only mean many more sleepless coding nights ahead.
(Continued in the next post)
(Pseudotags for Finns: HSL:n pysäkkinäytöt eli aikataulunäytöt.)
Update 1/2019: There's now a basic DARC decoder on GitHub, it's called darc2json.
Terve!
ReplyDeleteI just wanted to tell you that I really enjoy all of your deeply-technical posts. While I only have a layman's understanding of what you're saying (you might as well be The Doctor, talking about "reversing polarity"), I still find it tremendously exciting to read this stuff. Simply as a reminder that there is still so much to learn about, should I ever get bored with what I'm doing these days :)
Thank you, I'm glad there are people who like reading my posts :)
DeleteSo we know who to blame when these bus stop displays go crazy :3
ReplyDeleteThis is interesting, if this data is being broadcasted that means that devices are listening for it possibly unidirectional. so, with that said, I wonder if the devices listening share an encryption key?
ReplyDeleteIt's in the clear; there is no encryption.
DeleteAs a former broadcast engineer this is news to me! All of my previous installs never tx'ed anything above 57KHz. It's nice to see the 4th harmonic being used for something useful.
DeleteI use Inkscape.
ReplyDeleteThe two topmost waveforms are actually Audacity screenshots, recolored in GIMP and imported into Inkscape. The bottom one was also a screenshot that I vectorized into splines.
Why use YLE1 instead of a local Helsinki station for transmission? Doesn't that prevent the rest of the country from using the same system (at least on YLE1)?
ReplyDeleteThe signal is injected at the transmitter site, not at the programme origin.
DeleteHi!
ReplyDeleteWhat software used in the first picture?
Baudline.
DeleteFound this on Hackaday and just wanted to say that you are AWESOME. I'm really jealous! I studied this stuff in school and thought it was super cool but never really played around with it. What are you using for the decoding? That is done in software right? (the filtering, splitting, mixing, comparing, etc). Any tips for those of us with technical knowledge of modulation schemes who'd like to play around with actual hardware?
ReplyDeleteAgain, really nice!
Oona, I saw your 30C3 talk, it was fantastic. Can you please share any tips on which baudline channel/input mapping operations and transforms that you use when evaluating signals with data like this? I have been trying to become more familiar with using baudline, sox, inkscape, & perl as they seem to be your toolset of choice.
ReplyDeleteKeep up the fantastic work & stay fluxy!
The top screenshot is just FFT on the demodulated FM signal. Everything else was pretty much done without baudline, using SoX and Perl.
DeleteDetecting L-MSK as binary FSK? how? L-MSK has 4 states, FSK has only 2!
ReplyDeleteBoth have 0 and 1
Deletebut.. what about other two states?
DeleteGoogle for: noncoherent GMSK detection
DeleteI have no idea of the data transmitted in the DARC channel, but according to my RTL-SDR readings, here in Kauhava, I am receiving the same 76k carrier with Yle Radio 1.. So would it be possible that it is national data channel, not regional since this supports the fact that it is present in every transmission of Radio 1?
ReplyDeleteI'd love to have a few-minute recording of the local signal.
DeleteYeah, I just have to figure out how to capture the FM demodulated part (MPX) from RTL-SDR...
Deletertl_fm -N -f 88.2M -l 0 -F -p 96 -s 180000 -g 25 > yle1kauhava.pcm
Deletereplace numbers with local YLE1 frequency (-f) and your PPM error (-p)
Oh man.. I think the Windows-version of rtl_fm is a bit crippled, since it doesn't provide a way to output to stdout, it outputs directly to a file and it's name is the PPM number without a suffix, but I will post here a suitable file, I need to examine the quality first.
DeleteHi again! Here's an file, the signal is quite noisy, and I couldn't verify that the DARC-channel is above the noise level since MPXtool shows at max to RDS. Hope you can scavenge data out of it, if not, I shall try to improve my receiving conditions, only a poor antenna was now used. http://www.modu.fi/snd/yle1kauhava.pcm (deep journalism can be heard...)
ReplyDeleteYes, my noncoherent detector requires a low-noise signal = bigger antenna with correct polarization.
Deletethe new RDS 2.0 standard is going to make use of subcarriers around 76khz, will the bus service have to be re-engineered?, great blog btw
ReplyDeleteThank You Oona,
ReplyDeletethis is what I failed exams in around 1990/91 at Huddersfield Polytechnic in Yorkshire, England
doing a Degree in Electronic & Information Engineering
This illustrates the purpose of it which we were never told
most teachers & lecturers don't explain the practical aspects for what to do in the future
e.g. in A-Level Chemistry and repetitive molecules in organic chemistry, if the teacher had said, 'it's working towards protein structure & DNA' then I would have understood
I kind of understand FFT now & FSK and such
and I enjoy coding
I was just researching RDS and came across DARC
Great Blog
Hello friend Oona Räisänen, I listened to your sounds and some what I liked, thank you for the audio materials, creative ideas, do not get bored, lose interest in life.
ReplyDeleteThank you! I really try not to!
DeleteHello Oona,
ReplyDeleteDo you know what DARC FM decoders and encoders are used for this (chips/modules)? For the end devices and for transmition encoders?
BR, Ilya.
Hmm. It seems that the HSL bus stop transmissions have moved from Yle 1 to Me Naiset (91.1 MHz) and Radio Nova (106.2 MHz). I may have already discovered this in the past and forgotten, because it rings a bell.
ReplyDelete