Camouflage from Computer Vision
This project is under active development
CV Dazzle is a type of camouflage from computer vision. It uses bold patterning to break apart the expected features targeted by computer vision algorithms.
The following is an overview of the project concepts, background, and initial tests. More technical information about reverse engineering and visualizing the vulnerabilities in computer vision algorithms will be published on the new Research blog in May 2016.
How It Works
CV Dazzle works by altering the expected dark and light areas of a face (or object) according to the vulnerabilities of a specific computer vision algorithm. In the image above (Look #1), the design targets the Viola-Jones face detection algorithm, a popular and open source face detector that is included with the OpenCV computer vision framework. CV Dazzle designs can be created using only hair styling, makeup, and fashion accessories for any type of face.
Computer vision is always based on an accepted probabilistic threshold that can be exploited by altering the key visual features to appear one step below the threshold of detection.
Because face detection is the first step in any automated facial recognition system, blocking the detection stage also blocks any subsequent facial analysis including recognition and emotional analysis. Therefore, CV Dazzle blocks facial recognition by blocking face detection.
The first design (Look #1) was created in 2010 as a proof of concept for my masters thesis at ITP NYU. The following 3 looks were created in collaboration with DIS Magazine (Looks 2-4). The goal of all 4 looks was to investigate the potential for a style that was functional but still within the margins of conceptual fashion.
These looks can be tested against each of the OpenCV haarcascade profiles to demonstrate their effectiveness. The table belows shows detection results using a 1.1 and 1.05 scale factor for the sliding window with a minimum of 3 detection overlaps against each of the 4 haarcascade profiles that are included with OpenCV. You can run these demos yourself using any haarcascade face detection demo (i.e. OpenCV for Processing or OpenCV in Python).
|Look||Frontalface Default||Frontalface Alt||Frontalface Alt2||Profileface|
|Look 1||Blocked||Blocked||Blocked||Not Blocked|
The results illustrate that hair and makeup alone can be used to lower the wearer’s probability below the threshold of detection for most of the OpenCV Haarcascade detection profiles. However, there are important limitations to keep in mind.
First, these looks were designed to work against the Viola-Jones Haarcascade face detector in 2D still-images in the visible light spectrum with the pretrained Haarcascade detection profiles. Other face detection algorithms including Lineary Binary Pattern (LBP), Histogram of Oriented Gradients (HOG), Covolutional Neural Networks (CNN), multi-camera 3D-based systems, and multi-spectral imaging systems require a different strategy. These looks are solely based on the Viola-Jones Haarcacade classifiers.
Second, lighting conditions will cause the results to vary. The pose and illumination in these photos is similar to a biometric enrollment (passport style) photo. Overhead or more direct lighting will change the intensity and location of shadows which will change the detection outcome.
For the best performance, a CV Dazzle look is highly specific to the situation, is unique to the wearer, and is (hopefully) still socially/fashionably acceptable.
Makeup Avoid enhancers. They amplify key facial features. This makes your face easier to detect. Instead apply makeup that contrasts with your skin tone in unusual tones and directions: light colors on dark skin, dark colors on light skin.
Nose Bridge Partially obscure the nose-bridge area. The region where the nose, eyes, and forehead intersect is a key facial feature. This is especially effective against OpenCV’s face detection algorithm.
Eyes Partially obscure one or both of the ocular regions. The symmetrical position and darkness of eyes is a key facial feature.
Masks Avoid wearing masks as they are illegal in some cities. Instead of concealing your face, modify the contrast, tonal gradients, and spatial relationship of dark and light areas using hair, makeup, and/or unique fashion accessories.
Head Research from Ranran Feng and Balakrishnan Prabhakaran at University of Texas, shows that obscuring the elliptical shape of a head can also improve your ability to block face detection. Link: Facilitating fashion camouflage art. Use hair, turtlenecks, or fashion accessories to alter the expected elliptical shape.
Asymmetry Face detection algorithms expect symmetry between the left and right sides of the face. By developing an asymmetrical look, you can decrease your probability of being detected.
Before and After
In 2013 the New York Times commissioned a new look (Look #5) as an Op-Art feature.
This project began in 2010 as my masters thesis at NYU ITP as a challenge to the growing power asymmetries in computer vision and, in particular, widespread facial recognition technology. Since publishing these proof-of-concept images in 2010 CV Dazzle has received considerable attention including appearing on 60 Minutes (2013), in classified intelligence document (2013), in the New York Times (2014), and as a presentation at the European Commission’s Annual Security Symposium (2015).
The name of the project was inspired by WWI ship camouflage called Dazzle that used cubist-inspired designs to break apart the visual continuity of a battleship in order to conceal its orientation and size. Similarily, CV Dazzle, short for Computer Vision Dazzle, uses bold, graphic designs that break apart the visual continuity of a face. While the end result is still visible to human observers, CV Dazzle blocks detection by computer vision.
CV Dazzle is motivated by a need to reclaim privacy in a world of increased visual surveillance. Computer vision poses new challenges that otherwise do not exist in human observation; it is low-cost, scalable, passive, remote, networked, and superhuman in its capabilities to recognize and understand faces, emotions, and even intent. Non-verbal facial communication is captured and stored in images and videos that are known to feed into both commercial and government surveillance programs from IARPA(Github), CIA and NSA, and many others. According to a document leaked by Edward Snowden and reported by Laura Poitras and James Risen, the National Security Agency intercepts 55,000 “facial recognition quality images” per day. And according to their own website, Affectiva claims to analyze over 5,000 faces per day with a total of “3,988,860 faces analyzed to date” (as of April 24, 2016). Many of these images originate from social media sites like Instagram and Faceook.
Ideally, people could appear in images that are visible to human observers but not to computer vision surveillance systems. This is the goal of CV Dazzle. Since beginning this project in 2010, the concerns of a widespread facial recognition have only become more urgent and apparent.
- 60 Minutes
- Hair and Makeup Outsmarts Computers. 2014.
- Lasst euch das eine Tarnung sein.Süddeutsche Zeitung Magazine. 2013.
- Reverse-engineering artist busts face detection tech. The Register. 2010.
- How to hide from face-detection technology
- Gesichtserkennung: Tarnschminke für Facebook
- Camouflage Gets High-Tech Makeover
- How to Hide From Machines. 2010.
In Popular Culture
- Facial Recognition Defense Workshop: A Make-Up Tutorial
- Anti-Surveillance Feminist Poet Hair&Makeup Party
- Tutorial by Jillian Mayer
- Minority Report. FOX. 2015
- WGSN - Radical Neutrality