Redsea 0.7, a lightweight RDS decoder

I've written about redsea, my RDS decoder project, many times before. It has changed a lot lately; it even has a version number, 0.7.6 as of this writing. What follows is a summary of its current state and possible future developments.

Input formats

Redsea can decode several types of data streams. The command-line switches to activate these can be found in the readme.

Its main use, perhaps, is to demodulate an FM multiplex carrier, as received using a cheap rtl-sdr radio dongle and demodulated using rtl_fm. The multiplex is an FM demodulated signal sampled at 171 kHz, a convenient multiple of the RDS data rate (1187.5 bps) and the subcarrier frequency (57 kHz). There's a convenience shell script that starts both redsea and the rtl_fm receiver. For example, ./rtl-rx.sh -f 88.0M would start reception on 88.0 MHz.

It can also decode an "ASCII binary" stream (--input-ascii):

0001100100111001000101110000101110011000010010110010011001000000100001
1010010000011010110100010000000100000001101110000100010111000010111001
1001000010110000111111011101101011001010101110100011111101000011100010
100000011010010001011100001

Or hex-encoded RDS groups one per line (--input-hex), which is the format used by RDS Spy:

6201 01D8 E704 594C
6201 01D9 2217 4520
6201 E1C1 594C 6202
6201 01DA 1139 594B
6201 21DC 2020 2020

Output formats

The default output has changed drastically. There used to be no strict format to it, rather it was just a human-readable terminal display. This sort of output format will probably return at some point, as an option. But currently redsea outputs line-delimited JSON, where every group is a JSON object on a separate line. It is quite verbose but machine readable and well-suited for post-processing:

{"pi":"0x6201","group":"0A","tp":false,"prog_type":"Serious classical","ta":tru
e,"is_music":true,"alt_freqs":[87.9,88.5,89.2,89.5,89.8,90.9,93.2],"ps":"YLE YK
SI"}
{"pi":"0x6201","group":"14A","tp":false,"prog_type":"Serious classical","other_
network":{"pi":"0x6205","tp":false,"has_linkage":false}}
{"pi":"0x6201","group":"0A","tp":false,"prog_type":"Serious classical","ta":tru
e,"is_music":true,"partial_ps":"YL      "}
{"pi":"0x6201","group":"2A","tp":false,"prog_type":"Serious classical","partial
_radiotext":"Yöklassinen."}
{"pi":"0x6201","group":"0A","tp":false,"prog_type":"Serious classical","ta":tru
e,"is_music":true,"partial_ps":"YLE     "}
{"pi":"0x6201","group":"0A","tp":false,"prog_type":"Serious classical","ta":tru
e,"is_music":true,"partial_ps":"YLE YK  "}
{"pi":"0x6201","group":"2A","tp":false,"prog_type":"Serious classical","partial
_radiotext":"Yöklassinen."}
{"pi":"0x6201","group":"0A","tp":false,"prog_type":"Serious classical","ta":tru
e,"is_music":true,"alt_freqs":[87.9,88.5,89.2,89.5,89.8,90.9,93.2],"ps":"YLE YK
SI"}

Someone on GitHub hinted about jq, a command-line tool that can color and filter JSON, among other things:

> ./rtl-rx.sh -f 87.9M | jq -c
{"pi":"0x6201","group":"0A","tp":false,"prog_type":"Serious classical","ta":tru
e,"is_music":true,"partial_ps":"YL      "}
{"pi":"0x6201","group":"14A","tp":false,"prog_type":"Serious classical","other_
network":{"pi":"0x6202","tp":false}}
{"pi":"0x6201","group":"0A","tp":false,"prog_type":"Serious classical","ta":tru
e,"is_music":true,"partial_ps":"YLE     "}
{"pi":"0x6201","group":"0A","tp":false,"prog_type":"Serious classical","ta":tru
e,"is_music":true,"partial_ps":"YLE YK  "}
{"pi":"0x6201","group":"1A","tp":false,"prog_type":"Serious classical","prog_it
em_started":{"day":9,"time":"23:10"},"has_linkage":false}
^C

> ./rtl-rx.sh -f 87.9M | grep "\"radiotext\"" | jq ".radiotext"
"Yöklassinen."
"Yöklassinen."
"Yöklassinen."
"Yöklassinen."
"Yöklassinen."
"Yöklassinen."
"Yöklassinen."

The output can be timestamped using the ts utility from moreutils.

Additionally, redsea can output hex-endoded groups, the same format mentioned above.

Fast and lightweight

I've made an effort to make redsea fast and lightweight, so that it could be run real-time on cheap single-board computers like the Raspberry Pi 1. I rewrote it in C++ and chose liquid-dsp as the DSP library, which seems to work very well for the purpose.

Redsea now uses around 40% CPU on the Pi 1. Enough cycles will be left for the FM receiver, rtl_fm, which has a similar CPU demand. On my laptop, redsea has negligible CPU usage (0.9% of a single core). Redsea only runs a single thread and takes up 1500 kilobytes of memory.

Sensitivity

I've gotten several reports that redsea requires a stronger signal than other RDS decoders. This has been improved in recent versions, but I think it still has problems with even many local stations.

Let's examine how a couple of test signals go through the demodulator in Subcarrier::​demodulateMoreBits() and list possible problems. The test signals shall be called the good one (green) and the noisy one (red). They were recorded on different channels using different antenna setups. Here are their average demodulated power spectra:

[Image: Spectrum plots of the two signals superimposed.]

The noise floor around the RDS subcarrier is roughly 23 dB higher in the noisy signal. Redsea recovers 99.9 % of transmitted blocks from the good signal and 60.1 % from the noisy one.

Below, redsea locks onto our good-quality signal. Time is in seconds.

[Image: A graph of several signal properties against time.]

Out of the noisy signal, redsea could recover a majority of blocks as well, even though the PLL and constellations are all over the place:

[Image: A graph of several signal properties against time.]

1) PLL

There's some jitter in the 57 kHz PLL, especially pronounced when the signal is noisy. One would expect a PLL to slowly converge on a frequency, but instead it just fluctuates around it. The PLL is from the liquid-dsp library (internal PLL of the NCO object).

  • Is this an issue?
  • What could affect this? Loop filter bandwidth?
  • What about the gain, i.e. the multiplier applied to the phase error?

2) Symbol synchronizer

  • Is liquid's symbol synchronizer being used correctly?
  • What should be the correct values for bandwidth, delay, excess bandwidth factor?
  • Do we really need a separate PLL and symbol synchronizer? Couldn't they be combined somehow? Afterall, the PLL already gives us a multiple of the symbol speed (57,000 / 48 = 1187.5).

3) Pilot tone

The PLL could potentially be made to lock onto the pilot tone instead. It would yield a much higher SNR.

  • According to the specs, the RDS subcarrier is phase-locked to the pilot, but can we trust this? Also, the phase difference is not defined in the standard.
  • What about mono stations with no pilot tone?
  • Perhaps a command-line option?

4) rtl_fm

  • Are the parameters for rtl_fm (gain, filter) optimal?
  • Is there a poor-quality resampling phase somewhere, such as the one mentioned in the rtl_fm guide? Probably not, since we don't specify -r
  • Is the bandwidth (171 kHz) right?

Other features (perhaps you can help!)

Besides the basic RDS features (program service name, radiotext, etc.) redsea can decode some Open Data applications as well. It receives traffic messages from the TMC service and prints them in English. These are partially encrypted in some areas. It can also decode RadioText+, a service used in some parts of Germany to transmit such information as artist/title tags, studio hotline numbers and web links.

If there's an interesting service in your area you'd like redsea to support, please tell me! I've heard eRT (Enhanced RadioText) being in use somewhere in the world, and RASANT is used to send DGPS corrections in Germany, but I haven't seen any good data on those.

A minute or two of example data would be helpful; you can get hex output by adding the -x switch to the redsea command in rtl-rx.sh.

11 comments:

  1. RASANT has been shutdown (almost?) everywhere in Germany during 2009-2011. Those encrypted TMC messages are most probably Navteq Traffic (formerly TMCpro, Germany/Europe) data or some other private traffic service. These services have to be licensed by radio manufacturers which then get the decryption information. (You could probably extract keys from e.g. a TomTom firmware as they've licensed most services around the world, too.)

    ReplyDelete
    Replies
    1. That's a shame. I saw 11A groups with no ODA allocation, so I thought it could be RASANT.

      Delete
  2. I just found this post: http://www.mikrocontroller.net/topic/229218 (in German) where one guy wrote:

    Most important thing (about RDS) here are the location codes, which encode lots of locations, streets, exits and interchanges in a 16bit value. These are the "heart" of the messages.

    And only these codes are "encrypted" in TMCpro. That's explained in ISO 14819-6. And it's very mundane and based on a fixed code table which is built into TMCpro devices. Basically, the 16bit values are rotated and then XOR'd with a specific value.

    In the TMC data stream there's a flag that announces the encryption (Otherwise non-TMCpro devices would try to use the encrypted locations.) and tells the device which key should be used from the table.

    I ran a small attack against that and found one entry of that table (which was used by my local TMCpro station). This didn't change over a loooong time so I'm not sure if that ever changes at all.

    (So those encrypted messages you're seeing are probably something else.)

    ReplyDelete
    Replies
    1. The encrypted TMC messages I mentioned were seen in Finland, not Germany. (I've also written a post about decrypting them in 2013, titled A determined 'hacker' decrypts RDS-TMC.)

      Delete
  3. it's easy play rtl-sdr stream and decode rds with redsea same time:

    in ubuntu:
    1: sudo apt-get install nmap
    2: rtl_fm -M fm -l 0 -A std -p 0 -s 171k -g 40 -F 9 -f 89.2M | ncat -l -p 6500 -m 5 -k
    3: (in differend window1): nc localhost 6500 |./src/redsea
    4: (in differend window2): nc localhost 6500 | aplay -t raw -r 171000 -c 1 -f S16_LE

    ReplyDelete
    Replies
    1. Redsea now supports the option -e to echo the stdin signal back to stdout and print the RDS groups to stderr instead:

      rtl_fm -M fm -l 0 -A std -p 0 -s 171k -g 40 -F 9 -f 89.2M | ./src/redsea -e | aplay -t raw -r 171000 -c 1 -f S16_LE

      Delete
  4. Have a look at RT+ (RadioText Plus). Some receivers can parse out Artist/Title from RadioText into separate display fields. Newer Mercedez cars have this and even Apple iPod nano (back to 5th generation)...

    https://tech.ebu.ch/docs/techreview/trev_307-radiotext.pdf
    http://www.pira.cz/rds/show.asp?art=guide_text_11

    ReplyDelete
    Replies
    1. I wish RT+ was used in Finland as well!

      Redsea fully supports RT+; it prints the fields in separate JSON key/value pairs, like so:

      {
        "group":"12A",
        "pi":"0xD3C2",
        "prog_type":"Pop music",
        "radiotext_plus":{
          "item.artist":"Silbermond",
          "item.title":"Das Leichteste der Welt",
          "item_running":true
        },
        "tp":false
      }

      Delete
  5. Can't remember if I clicked submit yesterday. Apologies for duplicates. Anyhow, thank you so much for this. Im looking to build Radio into KODI for a Car Computer. Would be nice to get station information in DBus so that this can just sit nicely on the backend minding its own business.

    Anyhow, my curiosity is how car radio's work with scanning frequencies while listening to the radio. I presume with the RTL-SDR dongles, you can't tune into a station, while scanning the rest of the frequencies for RDS signals for Traffic, News etc... I presume I would need two RTL-SDR dongles?

    Simon

    ReplyDelete
    Replies
    1. To scan other stations in the background you would indeed need another RTL-SDR.

      For this reason some stations transmit so-called "enhanced other networks information" (EON), which is supported by redsea. It contains information about program types and traffic messages on other stations of the same network - usually this means stations owned by the same broadcaster.

      This information is spread over several messages because of the low bandwidth. Here's an example:

      {
        "group":"14A",
        "other_network":{
          "frequency":"94.0 MHz",
          "pi":"0x6203",
          "tp":true
        },
        "pi":"0x6201",
        "prog_type":"Serious classical",
        "tp":false
      }

      Now we know there's another station in this network at 94.0 MHz with PI code 6203, and audible traffic information can be expected there ("tp":true). When a traffic message is currently running on that frequency, this will be signified by both "tp" and "ta" being true.

      The other_network field may also contain the station name ("ps") and current program type ("prog_type"), whether it's news, pop music or something else.

      Delete

The comments section is pre-moderated; it will take some time for the comment to show up.

You might want to check out the FAQ first.