CV Dazzle, camouflage from face detection

CV Dazzle was developed as my masters’s thesis at the Interactive Telecommunications Program at NYU. The goal of this ongoing research project is to create a growing catalog of designs that can be employed as camouflage from face detection, both physically and digitally.

Intro

CV Dazzle^™ is form of expressive interference that takes the form of makeup and hair styling (or other modifications). The name is derived from CV, a common abbreviation for computer vision, and Dazzle a type of camouflage used during WWI. Dazzle camouflage was originally used to break apart the gestalt image of warships, making it hard to discern their directionality, size, and orientation. Likewise, the goal of CV Dazzle is to break apart the gestalt of a face, or object, and make it undetectable to computer vision algorithms, in particular face detection.

And because face detection is the first step in any automated facial recognition system, CV Dazzle can be used in any environment where such systems are in use, including social network platforms like Facebook. For an example of CV Dazzle working against Facebook’s Face.com see CV Dazzle vs PhotoTagger by Face.com on Vimeo.

Collaborations

The CV Dazzle project is a collaboration between hair stylists, makeup artists, and fashion designers. The first look was developed with hair stylist Pia Vivas. Using a custom software application that exposes the vulnerabilities of the OpenCV face detection algorithm, a hair style was made to target these areas. Then, makeup was added to the cheek regions to increase the asymmetry of the face. The first look took about 3 hours to style and put together and required a few revisions mid way. After each style-adjustment the face was rescanned and the hair and make was adjusted until it worked.

DIS Magazine

The second collaboration, with DIS magazine, explored three more styles. The concept designs were developed by Lauren Boyle and DIS magazine. The hair was again done by Pia Vivas. The software applications was used to guide the process and test the looks while they were being applied. See the full story How to hide from Machines for more photos.

Looks #2, #3, and #4

---

Concept Sketches

Before pursuing this project, I sketched out what it might look like if someone were to wear CV Dazzle. I used Photoshop and the Processing OpenCV library to generate 60 designs and outcomes.

• Images with a red square tested positive, a face was found
• Images without a red square tested negative, no face was found
• Images under the section ”TEST PATTERNS“ are made according to results of the Haar deconstruction
• Images under ”RANDOM PATTERNS&lDquo; are random doodles made without the anti-face detection patterns in mind
• Images underneath the ”NO PATTERN“ heading are left untouched to show that the face detection works well on simple line drawings
• Original line drawings are from Figure Drawing for Fashion Design

Haarcascade Visualizations

As a next step towards understanding how face detection works, I visualized the haarcascade files. These are XML files that define a cascading set of features (haar features). I found this tutorial from Cognotics incredibly helpful in understanding how they work.

Basically, the Viola Jones algorithm looks for the difference between the light and dark regions and determines whether the difference between the sum of the white pixels minus the sum of the black pixels is within the threshold of a what it wants. These thresholds are fine-tuned floating point numbers. If the sum of a stage does not meet the threshold criteria, the outcome is rejected and the sub scan is halted. The algorithm then moves the scanning sub window to a new location and calculates a new sum. If the new threshold is met, the algorithm progreses through the other stages of the cascade (usually 20 or 30). This does not yet constitute a face though; it’s only a single positive. At the end of scanning the image, the algorithm will resize the scan window by a factor of around 1.3. After all scans are completed, another algorithm computes the overlap of these rectangles and if is enough overlap (usually 3) the region is likely to be a face.

Download visualization source code

Visualizing the OpenCV Detection Process

The entire scan process can be seen as a subwindow scans left to right, top to bottom, across an image. After it completes a full scan, the sub window size is scaled up, and the process is repeated. In slow motion, it would look like this:

What works and what doesn’t

The next step in this project is to implement supervised genetic algorithms to compute more possibilites for designs.

The images below show how the process was acheived using a “human-learning” algorithm known as trial and error.

Future plans

CV Dazzle heads to Rome in May to discuss the implications of spoofing with leading biometrics experts from the Tabula Rasa project. Biometrics is a interesting field and I hope to find Rome inspiring.

Credit

This project came out of my thesis class at ITP. I enjoyed help from my thesis advisor Despina Papadapolous, all my classmates, Pia Vivas, and extremely talented hair stylist with a vision for what hair might be like in the future and Jen, the model. Working with DIS magazine wouldn’t have happened without Obinna Izeogu, another person with a an eye for the future. Thanks.

Credits for DIS Magazine collaboration

• Creative Direction Lauren Boyle, Solomon Chase, and David Toro
• Photography Marco Roso and Adam Harvey
• Web Production Nick Scholl
• Hair Pia Vivas
• Makeup Lauren Devine
• Models: Maria Verdú Bertomeu, Irina Cocimarov, Jude MC, Michael Vontsolos, and Jen