In your face: hacking facial recognition systems
So big brother is watching, so what? Technological surveillance systems might seem all powerful, but they are prone to different kinds of hacking. Like with this picture editing program that is fooling facial recognition software. Read More
So big brother is watching, so what? Technological surveillance systems might seem all powerful, but they are prone to different kinds of hacking. Like with this picture editing program that is fooling facial recognition software.
You know big brother is really getting scary when your smartphone’s photo program picks out all the photos with a tagged friend, your nan or a neighbor from your photo gallery in just seconds. It may seem funny at first and can sometimes be very useful. But it is also mildly terrifying because it shows just how advanced facial recognition software has become. It also offers a little insight into what can happen when the government or security forces use these programs. Today’s software processes huge amounts of data very quickly and misses nothing. Or does it?
In 2017, when the German Bundestag decided to allow state security forces future access to the biometric photos of all citizens, artist and activist collective Peng! decided enough was enough. Up to that point, access had only been granted in exceptional circumstances and for compelling reasons. Now though, German police, secret service, and tax and customs investigators could automatically sift through the headshots in the passport and identity card databases without justification or legal challenge. This meant the authorities could also use facial recognition to put a name to any person captured in an image recorded by a surveillance camera in a public space.
The computer program takes photos of two different people and morphs them into a single one.
The Peng! team of artists developed MaskID as a response to the new law. The computer program takes photos of two different people and morphs them into a single photo that resembles both the originals. The remarkable thing is that a person looking at the new image with the naked eye feels like they recognize both people. Peng! immediately tried out the impact of their invention by melding a photo of an activist with a portrait of then EU High Representative Federica Mogherini.
The activist used this new photo to apply for a passport and while officials at the passport office hesitated and pointed out that the photo was not a particularly good likeness, they ultimately let it pass. After all, when does a passport photo, especially the biometric ones, actually look like the passport holder? The coup succeeded and a passport was issued that both Federica Mogherini and the Peng! activist could theoretically have used to travel. Neither did, probably, and Federica Mogherini would certainly have wanted nothing to do with it.
Fooling automatic recognition systems
While morphing a few different people into one composite image is not illegal, using the result to apply for a passport certainly is. So MaskID is not really an effective tool against state surveillance, unless of course you want to be arrested for document falsification. The idea is not entirely new either. There are a range of photo processing programs that allow users to meld two or more passport photos, and they are being enthusiastically used by people making forged IDs. Experts estimate that around 1,000 cases of morphed photos in passports have been uncovered in Europe so far. Many more are likely to have gone unnoticed.In 2018, the Fraunhofer Institute for Computer Graphics proved scientifically what Peng! had already practically shown: that it is impossible to identify such manipulations with the naked eye. And the automatic facial recognition systems in use in more and more airports for passport control – especially during the coronavirus pandemic when “contactless security checks” are more popular than ever – do little to weed out the forgeries.
In an experiment, US researchers showed that both human border agents and facial recognition systems fall for the morphed photos. In summer 2020, the German authorities responded by requiring citizens to have their photos taken on-site at the passport office or submitted by an accredited photographer.
So while the Peng! artists have certainly not dealt a death blow to the facial recognition software used in surveillance, they have cast a bright light on the issue in a spectacular way. Their software is available to download for free online, so you can mess about at home and morph a photo of yourself with as many others as you like. This is both fun and potentially useful. Even though it cannot be used to trick the passport office, the results might at least make Google and the other Internet giants look twice.
