Barcode recovery using a priori constraints

[Image: A hand holding a Finnish driver's license card with blurred details and the text 'specimen' across it.]

Barcodes can be quite resilient to redaction. Not only is the pattern a strong visual signal, but so is the encoded string that often has a rigidly defined structure. Here I present a method for recovering the data from a blurred, pixelated, or even partially covered barcode using prior knowledge of this higher-layer structure. This goes beyond so-called "deblurring" or blind deconvolution in that it can be applied to non-convolutional distortions as well.

It has also been a fun exercise in OpenCV matrix operations.

As example data, specimen pictures of Finnish driver's licenses shall be used. The card contains a Code 39 barcode encoding the cardholder's national identification number. This is a fixed-length string with well-defined structure and rudimentary error-detection, so it fits our purpose well. High-resolution samples with fictional data are available at government websites. Redacted and low-quality pictures of real cards are also widely available online, from social media sites to illustrations for news stories.

In Finland, knowledge of a name and this code often suffices as proof of identity on the phone, yet nothing on the card indicates that the barcode contains sensitive information. Consequently, it's not hard to find pictures of cards with the barcode completely untouched either, even if all the other information has been carefully removed.

All cards and codes used in this post are simulated.

Image rectification

We'll start by aligning the barcode with the pixel grid and moving it into a known position. Its vertical position on the driver's license is pretty standard, so finding the card's corners and doing a reverse perspective projection should do the job.

Finding the blue EU flag seemed like a good starting point for automating the transform. However, JPEG is quite harsh on high-contrast edges and extrapolating the card boundary from the flag corners wasn't too reliable. A simpler solution is to use manual adjustments: an image viewer is opened and clicking on the image moves the corners of a quadrilateral on top of the image. cv::find­Homography() and cv::warp­Perspective() are then used to map this quadrilateral to a 857×400 rectangular image, giving us a rectified image of the card.

[Image: The card cut out from the previous image and warped to cancel the effects of persective.]

Reduction & filtering

The bottom 60 pixel rows, now containing our barcode of interest, are then reduced to a single 1D column sum signal using cv::reduce(). In this waveform, wide bars (black) will appear as valleys and wide spaces (white) as peaks.

In Code 39, all characters are of equal width and consist of 3 wide and 9 narrow elements (hence the name). Only the positions of the wide elements need to be determined to be able to decode the characters. A 15-pixel convolution kernel – cv::GaussianBlur() – is applied to smooth out any narrow lines.

[Image: A blurred barcode on top of a graph depicting its gray level fluctuations.]

A rectangular kernel matched to the bar width would possibly be a better choice, but the exact bar width is unknown at this point.


One of only a limited number of characters can appear at any position in the barcode. The format of the driver's license barcode will always be *DDMMYY-NNNC*, where

  • The asterisks * are start and stop characters in Code 39
  • DDMMYY is the cardholder's date of birth
  • NNN is a number from 001 to 899, its least significant bit denoting gender
  • C is a modulo-31 checksum character; Code 39 doesn't provide its own checksum

These constraints will be used to limit the character search space at each string position. For example, at positions 0 and 12, the asterisk is the only allowed character, whereas in position 1 we can have either the number 0, 1, 2, or 3 as part of a day of month.

If text on the card is readable then the corresponding barcode characters can be marked as already solved by narrowing the search space to a single character.

Decoding characters

It's a learning adventure so the decoder is implemented as a type of matched filter bank using matrix operations. Perhaps it could be GPU-friendly, too.

This filter matrix is our dictionary, or character set. Each row of the filter matrix represents an expected 1D convolution output of one character, addressed by its ASCII code. A row is generated by creating an all-zeroes vector with a single pixel set to plus/minus unity at the positions of the wide bars. These rows are then smoothed by convolving them with a horizontal Lanczos kernel.

[Image: A matrix about 40 lines high and 40 columns wide. Each cell has a gray level. The cells of each line form parts of blurred barcodes. Lines are marked with numeric codes and ASCII characters.]

The exact positions of the peaks depend on the barcode's wide-to-narrow ratio, as Code 39 allows anything from 2:1 to 3:1. Experiments have shown it to be 2.75:1 in most of these cards.

The turquoise 1D wave in the previous section is divided into character-length pieces which are then multiplied per-element by the above matrix using cv::Mat::mul(). The result is reduced to a row sum vector.

This vector now contains a "score", a kind of matched filter output, for each character in the search space. Where the peaks of the signal coincide with peaks in the filter, we get a large positive response, and vice versa. The best matching character is the one with the highest score; this maximum is found using cv::minMaxLoc(). Constraints are passed to the command as a binary mask matrix that zeroes out the non-allowed rows, so that they can't give a response.

Barcode alignment and length

To determine the left and right boundaries of the barcode, an exhaustive search ("brute-force") is run through the whole 1D signal (around 800 milliseconds). On each iteration the total score is calculated as a sum of character scores, and the alignment with the best total score is returned. This also readily gives us the best decoded string.

[Image: A graph depicting gray level variations. On top of that, decoded characters with associated floating point values. The code says *010185-710P*.]

We can also enable checksum calculation and look for the best string with a valid checksum. This allows for errors elsewhere in the code.


The barcodes in these images were fully recovered using the method presented above:

[Image: Three barcodes each distorted in a different way - pixelated, blurred, or smudged.]

It might be possible to further develop the method to recover even more blurred images. Possible improvements could include fine-tuning the Lanczos kernel used to generate the filter bank, or coming up with a better way to score the matches.


The best way to redact a barcode seems to be to draw a solid rectangle over it, preferably even slightly bigger than the barcode itself, and make sure it really gets rendered into the bitmap.

Printing an unlabeled barcode with sensitive data seems like a bad idea to begin with, but of course there could be a logical reason behind it.


  1. Fun!

    I did something similar years ago. My task was to decode bar-codes from low resolution black and white fax images. The resolution was very close to the Nyquist limit so I had to do reconstruction of the signal.

    Turned out that averaging the signal wasn't working great. I got best results by scanning each line of the bar-code image individually and jumping between the different samples based on context.

    An error that occurred during fax transmission than was the breakthrough. During one fax transmission the paper jammed, and the bar-code was scanned rotated. This improved the quality because it just didn't happen anymore that thin lines of the bar-code fell exactly in between the scanning sensors.

    Fortunately I had control of the rotation and position of the bar-code, so I just rotated the box for the bar-code sticker rotated on the form. Problem solved :-)

  2. How long does the whole decoding process take? You mentioned 800 msec for exhaustive search through the 1D signal. Does this time include all possible character combinations?

    1. Yes it does, 800 ms from the original image to returning the best decoded string.

  3. Interesting approach, but it can be faster. Commercial bar code scanners do this same operation on embedded platforms (400 MHz arm chips) in about 100 ms, without requiring the constraints.

  4. Driver's license format is ddmmyyczzzq where c defines century. For millennials that's A.

    1. And for people born in the 19th century, the 7th character would be '+'. I am not sure how many people over 116 years of age are still alive and having a valid driver's license at this point. Assuming the 7th character being '-' is still a good guess, for at least couple of years.

  5. Once you've done the geometric transform and signal model, why not try a contrained soft Viterbi decoder over the model? And as for finding a barcode -- any barcode -- within an image, why not play with the ideas in projective Fourier analysis a bit: in orthogonal projection, you can easily catch such repetitive structures in orientation and size by utilizing homomorphic signal processing, and nonlinear processing in the mid stages.

    I'm not well versed enough with computer vision algorithms to be sure, but I'd gather going to a projective representation on top of that stuff, would land you with a neat starting point for perspective corrected images at least.

  6. `NNN is a number from 001 to 899, its least significant bit denoting gender`

    Range used in Finland seems to be 002-899. Bit vague on why though:


The comments section is pre-moderated; it will take some time for the comment to show up.

You might want to check out the FAQ first.