Feb 1, 2014

Mystery signal from a helicopter

Last night, YouTube suggested a video for me. It was a raw clip from a news helicopter filming a police chase in Kansas City, Missouri. I quickly noticed a weird interference in the audio, especially the left channel, and thought it must be caused by the chopper's engine. I turned up the volume and realized it's not interference at all, but a mysterious digital signal! And off we went again.

The signal sits alone on the left audio channel, so I can completely isolate it. Judging from the spectrogram, the modulation scheme seems to be BFSK, switching the carrier between 1200 and 2200 Hz. I demodulated it by filtering it with a lowpass and highpass sinc in SoX and comparing outputs. Now I had a bitstream at 1200 bps.

The bitstream consists of packets of 47 bytes each, synchronized by start and stop bits and separated by repetitions of the byte 0x80. Most bits stay constant during the video, but three distinct groups of bytes contain varying data, marked blue below:

What could it be? Location telemetry from the helicopter? Information about the camera direction? Video timestamps?

The first guess seems to be correct. It is supported by the relationship of two of the three byte groups. If the 4 first bits of each byte are ignored, the data forms a smooth gradient of three-digit numbers in base-10. When plotted parametrically, they form an intriguing winding curve. It is very similar to this plot of the car's position (blue, yellow) along with viewing angles from the helicopter (green), derived from the video by magical image analysis (only the first few minutes shown):

When the received curve is overlaid with the car's location trace, we see that 100 steps on the curve scale corresponds to exactly 1 minute of arc on the map!

Using this relative information, and the fact that the helicopter circled around the police station in the end, we can plot all the received data points in Google Earth to see the location trace of the helicopter:

Update: Apparently the video downlink to ground was transmitted using a transmitter similar to Nucomm Skymaster TX that is able to send live GPS coordinates. And this is how they seem to do it.

Update 2: Yes, it's 7-bit Bell 202 ASCII. I tried decoding it as 7-bit data earlier, ignoring parity, but must have gotten the bit order wrong! So I just chose a roundabout way and kept looking at the hex. When fully decoded, the stream says:

#L N390386 W09434208YJ
#L N390386 W09434208YJ
#L N390384 W09434208YJ
#L N390384 W09434208YJ
#L N390381 W09434198YJ
#L N390381 W09434198YJ
#L N390379 W09434188YJ

These are the full lat/lon pairs of coordinates (39° 3.86′ N, 94° 34.20′ W). Nucomm says the system enables viewing the helicopter "on a moving map system". Also, it could enable the receiving antenna to be locked onto the helicopter's position, to allow uninterrupted video downlink.

Thanks to all the readers for additional hints!

104 comments:

  1. Thanks for sharing your solution. Well done!

    ReplyDelete
  2. Why do I get the feeling that you are going to be contacted by a shadowy organization for an off-the-record secret mission with an infamous jewel thief (sprung from prison just for this event) and the last Real Ninja, a side show contortionist, an experimental weapons expert, a bumbling but brilliant scientist and George Clooney?

    ReplyDelete
  3. What tools did you use to decode and analyze the data?

    ReplyDelete
    Replies
    1. Perl and SoX. Also used Baudline as a simple waveform inspector.

      Delete
    2. Long live perl.

      Delete
    3. Yeah! In a world getting full of snakes it's good to have your trusty camel ;-)

      Delete
    4. Watch how you talk about "the" snake :)

      Delete
    5. Perl. The programming language for super humans.

      Delete
  4. This is fascinating. Would it be too much to ask for a detailed step-by-step of how you did this using the tools you used?

    ReplyDelete
  5. How did that single get into the video in the first place ? Telemetry from heli was in the form of audio ?

    ReplyDelete
    Replies
    1. yup. quite common. here's one in Tulsa, OK from last week:
      http://www.newson6.com/story/24559132/tulsa-police-catch-a-vehicle-after-high-speed-chase
      (the 14min video)

      Delete
  6. That's brilliant. Very interesting!

    ReplyDelete
    Replies
    1. Yes, koodoos for her. I'm mightily impressed.

      Delete
  7. I love reading about people that do this stuff. Totally awesome. Great work! HACK THE PLANET!!!

    ReplyDelete
  8. +1 for the diligent work
    +5 for the conclusive end-result
    +100 for detecting it in the first place

    ReplyDelete
  9. What software did you use for the "magical image analysis"?

    ReplyDelete
    Replies
    1. Plotting the car's position was actually all manual work. I've done that before from videos.

      Delete
  10. well done Dr. Arroway ;) http://youtu.be/uhIEfxRLiPI

    ReplyDelete
    Replies
    1. Funny you should say that, Dr. Arroway was one of the reasons I ever got into decoding signals back then.

      Delete
    2. it's not funny "Anonymous" said that; they know what you watched when younger.

      Delete
    3. Dr. Arroway was my inspiration for getting into radio as well.
      Eventually I'll live somewhere I can actually put up an antenna of decent size.

      Delete
  11. awesome work! this could be advertised to make children go into science :D

    ReplyDelete
  12. Wow, thats the most fascinating discovery since hexagon storm on Jupiter!

    ReplyDelete
  13. Oona, you're an inspiration. Keep publishing stuff like this!

    ReplyDelete
  14. Nice work!

    I'm curious what tool you used to threshold and software decode the UART stream to get the hex log?

    ReplyDelete
  15. congratulations for the lesson :-)

    ReplyDelete
  16. Nice find - what I'm not clear on is how the digital signal ended up on the recording. Was this signal picked up via induction on one of the microphone inputs? I can't image there is a speaker squawking the digital signal out loud... this is the real mystery to me.

    ReplyDelete
    Replies
    1. The Skymaster TX deliberately inserts position data to enable infographics etc.

      Delete
  17. you can demodulate 1200 baud. it's like 1990 all over again!

    ReplyDelete
  18. If they did this in a movie, I'd be like 'yeah, suuure you can do that..'.

    Very cool indeed.

    ReplyDelete
  19. Once again, epic hacking by you! :-)

    ReplyDelete
  20. I think you might enjoy ham radio.

    ReplyDelete
  21. Wow, incredible hack! Thanks for sharing.

    ReplyDelete
  22. Nice job. I second the comment about amateur radio, if you find analyzing signals interesting you'd probably enjoy some of the digital modes. If I came across this video I'd have piped that audio channel through a software decoder too! And you can do all sorts of things general public (just can't) like ping the space station as it passes over or have a voice chat with an astronaut. Or just talk to someone driving around. It's neat.

    ReplyDelete
  23. Hypnoottisen viileää puuhaa!

    ReplyDelete
  24. great work BUT...now they will encrypt it and NO one will be able to track them anymore
    only through ADSB and DFing the beacon and maybe WIFI/Bluetooth and cells in their pockets.

    MRX

    ReplyDelete
    Replies
    1. No I don't think they'd bother encrypting the telemetry, since the location data can be gotten from a subsidiary high bandwidth unencrypted back channel - by looking up!

      Delete
    2. Yeah, because it's really cheap and useful to replace this kind of specialized equipment. So they need to develop a crypto device with multi-platform phone app - two days? Then the news stations have to switch to the new device and throw away the old ones - another two days?
      And all this just for hiding publicly available data (just look at the sky).
      Nice try troll.

      Delete
  25. Ha! Good stuff, after reading this I remembered I had been planning a bike-to-bike telemetry system using some citizens band or whatever (semi)legal frequency. It would be sooo good to see co-bikers live locations on your navigator when traffic or something else divides the group. However even if I could build such application there remains one big problem - there aren't (m)any people who could/would join my adventures. :/

    ReplyDelete
  26. Wow, I am impressed. Most notably, with noticing the interference stream in a YouTube video and bothering to investigate it in this detail. Massive respect.

    ReplyDelete
  27. This is really impressive. Massive respect, and keep up the great work!

    ReplyDelete
  28. Why are you Fins all such geniuses?

    ReplyDelete
  29. Bell 202 FTW. I always loved that protocol - silence when no data was sent if I recall correctly. AppleCat was my first 1200 baud modem - more than 10x the speed of my prior Bell 101 device! Always a walk down memory lane to catch 202 bursts on bleed-overs from leaky phone line with CallerID delivery. But kudos to your demod on the weak data. Next up: if you can get 30fps of flickering LEDs in the background of some random scene, can you decode what they're transmitting? Tricky timing questions!

    ReplyDelete
  30. Did you come to any conclusions as to the use of bytes 11-13, and 20? After inverting, scaling, and offsetting packets 500-1800, they look like the other two groups of values. However before 500 and after 1800, they deviate somewhat.

    ReplyDelete
  31. I really can't believe people can do what you did, Oona. Go inspire some young girls. :-)

    ReplyDelete
  32. ..And Silver Medal goes to Chuck Norris.

    When Gods want to do something cool, they call her.

    ReplyDelete
  33. Do you think Kansas City's involvement with Google Fiber has anything to do with this?

    ReplyDelete
  34. As a media researcher, this is unbelievably impressive to me. Thank you for sharing your discovery and knowledge.

    ReplyDelete
  35. Hello... thanks for the interesting work with detailed description. Looks like magic... especially helicopter...
    Want to ask: what software was used for visualisating data here https://i2.sndcdn.com/artworks-000046118533-mfgxdq-t200x200.jpg?e30f094 or here http://3.bp.blogspot.com/-Fr-g6fxLWII/Uuy_45qhaYI/AAAAAAAADIg/j8RJAjyt9Qg/s460/bitteja.png . Looks pretty, want the same. )))

    ReplyDelete
    Replies
    1. The first one is Baudline and Inkscape. Second is oscillo.pl and Inkscape.

      Delete
  36. Great post, and thanks for sharing. I will explain it to my kids so to see if they get hooked into this kind of things and develop the curiosity that is needed to become like you. My compliments.

    I wonder if the interference noises that i can hear in my car speakers when my cellphone talks to the celltowers can be decoded too. Did you ever think about it?

    Marco
    @mgua

    ReplyDelete
    Replies
    1. Thank you again.

      Reading your blog a page of a book came to my mind: Peter Høeg's Smilla's Sense of Snow. Smilla gets an old audio cassette tape and brings it to an old blind audio and language expert. He manages to decode the language and to identify the speaker age, education, and place of origin, basing on dialect, accent and tone. A nice page about language reverse engineering.

      Marco
      @mgua

      Delete
    2. In fiction, identifying a criminal based on sound pattern analysis, without any tools at all goes back at least to Wilbur Daniel Steele's 1921 short story, Footfalls ( http://www.web-books.com/Classics/ON/B1/B1138/Henry_1920C15.html )

      Delete
  37. How about this? Quote from slashdot http://tech.slashdot.org/comments.pl?sid=4741341&cid=46129789

    You can decode it with off the shelf software, throw away the top bit, and get back mostly ASCII:

    $ ./minimodem --rx 1200 -f ~/helicopter.wav | tr '\200-\377\r' '\000-\177\n'
    ### CARRIER 1200 @ 1200.0 Hz ###
    282 0002.3
    #L N390374 W09432938YJ
    #AL #NA 282 0002.3
    #L N390374 W09432938YJ
    ...

    ReplyDelete
  38. What about the
    L #NA 272 0003.0 is this 272ft of elevation ?

    ReplyDelete
    Replies
    1. Yes, by the looks of the curve it makes, it's probably elevation.

      Delete
  39. Saw this via Twitter - Brilliant! As a deaf person, I'll never think you could find digital signals in the audio channel. Goes to show that all grounds must be covered including those that seem silly!

    ReplyDelete
    Replies
    1. Having listened to lots of them, I've begun to have an ear for certain modulations.

      Delete
  40. This is really cool. Nice work.
    Could you post the data file? Should be possible to add a live map view to the video.

    ReplyDelete
    Replies
    1. http://oona.windytan.com/coords.txt, but it doesn't contain timing so can't be plotted live. Some packets are missing from in between.

      Delete
    2. Yeah, without timing and missing packets its going to be difficult. Would have been cool though, especially since you also created a track with the car's location.
      With not too many and somewhat evenly distributed missing packets the average timing error might be low enough. Maybe I'll try anyway if I have some time to kill...

      Delete
    3. http://www.fileswap.com/dl/5lJeEkIc3D/HelicopterChase.kml.html
      Each point's title is the seconds into the video (minus 0.185), and the description is the elevation value.

      Delete
    4. Thanks for the kml file, Timothy. With that it should be possible.
      How did you manage to obtain the timing info?

      Delete
    5. You're welcome. When I decoded it in a Matlab script I wrote, I took care to preserve the timestamp of the beginning of each timestamp.

      Delete
    6. Sorry, I meant "the timestamp of each data packet."

      Delete
  41. so we left our freedom to intercept signals for this compendium of garage science

    ReplyDelete
  42. Wou, real life Lisbeh Salader. Terveisä Thimaasta

    ReplyDelete
  43. Well done. A very good program to decode the AFSK directly is called minimodem http://www.whence.com/minimodem/. I am familiar with a version from a different manufacturer of this type of tracking system, NSI or N-systems. Originally gps rs232 data from the helicopter’s nav system was used, a Trimble protocol called RNAV r1, sent to a bell 202 v.23 modem running at 1200 baud. This audio signal was sent to one of the audio inputs of the microwave transmitter on board the ship, the same one transmitting the live video. The audio was pulled off at the receive site, decoded locally by the tracking system that drove the receive antenna to automatically follow the live coordinates from the moving helicopter. As gps has become more common the units no longer depend on flight computers, but due to backwards compatibility issues they still use the strange protocol. The earlier system was waypoint based, the copter transmitter antenna was pointed at a known waypoint (the receiver) this bearing-to-waypoint info was simply reversed 180 by the receive site to point at the helicopter, but this system doubled any pointing errors. The newer systems simply track the lat/long coordinates.

    Much of the reason it is done with audio is for legacy compatibility. The original versions of these systems used analog microwave links, extremely susceptible to breakup and mulitpath with any antenna pointing errors. Bell 202 worked great with the limited bandwidth audio subcarrier used, and usually didn't creep into the second voice/nat sound subcarrier. Every ENG helicopter microwave system is now digital and much more resilient, but we still use AFSK tracking data because that's how the infrastructure is set up.

    ReplyDelete
  44. Respect. With 0x20 years experience of programming and couple rounds with GPS and Fourier, I'm not sure if I could have done this. The math and programming, sure, but the quess work... I loved millennium trilogy and agree with the ref to Ms Salander :-)

    ReplyDelete
  45. 我是一个中国高中生,买了代理服务器才能看你的文章,表示因为你的杰出成就使自己对你产生崇敬之心

    ReplyDelete
  46. I have to say first that I love your kind of hackers because you can find something that ordinary people don't even think of ever... :)

    Because you are a hacker you are maybe interested in knowing that this specific finding is not unique in these devices (even if the manufacturer says so) because this technique has been used already years and years in these video links.

    I am myself an FPV hobbyist (I was one of the first in Finland that started doing this FPV with RC helicopters about 6 years ago) and at least a few years we have had a $100 transmitter that sends GPS signal (and all FPV RC airplane/copter logger data actually) in video audio channel. This function is in many devices, but for example it is available in this EagleEyes FPV Station:
    http://www.eagletreesystems.com/index.php?route=product/product&product_id=84

    You will get also more info about these video links and extra data from my 4 year old video in Assembly 2010:
    https://vimeo.com/18302745

    ReplyDelete
  47. Any chance this signal could be recovered from crosstalk in other channel or video?

    ReplyDelete
  48. I don't get it; why not just contact the station and ask the pilot? They love to talk about this sort of thing.

    ReplyDelete
  49. Have you tried using GNU Radio for signal analysis? It can take some time to learn, but it is a very powerful and useful tool for this sort of reverse engineering project.

    ReplyDelete
    Replies
    1. GNU Radio seems a bit too complicated for me. I want raw samples and Perl.

      Delete
    2. I find GNU Radio to be a great way to get from raw samples to bits. If you get stuck on a complicated modulation scheme someday, you might want to give it a try.

      Delete
  50. This is some amazing work, thank you very much for sharing. After having read a few of your posts and watched some videos, I'm wondering how you generate these nice plots like the one with the raw signal, the demodulated data and the black background.

    I'm also wondering what program you used to generate the real-time spectrogram in the video on the 15.6 kHz high-pitch noise of the CRT. I like to visualize my results for my audience, but my visualizations are far inferior to what you are showing. Would you mind sharing?

    ReplyDelete
    Replies
    1. It's often a mixture of Baudline, oscillo.pl, and Inskcape.

      Delete
  51. Thanks for the info Ooona. The company I work for, Red Hen Systems, pioneered the technology behind this technique: http://www.redhensystems.com/

    ReplyDelete
  52. Say, Oona- you sound a lot like me (with your curiosity on things you hear and figuring out the signals) for reference, could you post the youtube video you finagled the signal from if possible? If you were signed into youtube when you watched it, it would be in your watched list. It would be interesting to see the very video you suspected that sound from. Thanks in advance- Kristin

    ReplyDelete
    Replies
    1. See the first sentence of the post, there's a link.

      Delete
  53. Look up APRS for more information on the Ham radio system that is very similar to this. Sending your location speed heading and other infomation in about a .7 second burst every x number or minutes.

    Jim K5JSW

    ReplyDelete
  54. wow, this is so cool. I love when you post articles like this. it inspires the inner geek within me.

    ReplyDelete

To prevent spam, comments will only appear after moderation.