From 2008 to 2010, as Edward Snowden has revealed, the National Security Agency (NSA) collaborated with the British Government Communications Headquarters to intercept the webcam footage of over 1.8 million Yahoo users.

The agencies were analyzing images they downloaded from webcams and scanning them for known terrorists who might be using the service to communicate, matching faces from the footage to suspects with the help of a new technology called face recognition.

The outcome was pure Kafka, with innocent people being caught in the surveillance dragnet. In fact, in attempting to find faces, the Pentagon’s Optic Nerve program recorded webcam sex by its unknowing targets—up to 11 percent of the material the program collected was “undesirable nudity” that employees were warned not to access, according to documents. And that’s just the beginning of what face recognition technology might mean for us in the digital era.

Over the past decade, face recognition has become a fast-growing commercial industry, moving from its governmental origins—programs like Optic Nerve—into everyday life. The technology is being pitched as an effective tool for securely confirming identities, with the financial backing of a new Washington lobbying firm, the Secure Identity & Biometrics Association (SIBA).

To some, face recognition sounds benign, even convenient. Walk up to the international checkpoint in a German airport, gaze up at a camera, and walk into the country without ever needing to pull out a passport—your image is on file, the camera knows who you are. Wander into a retail store and be greeted with personalized product suggestions—the store’s network has a record of what you bought last time. Facebook already uses face recognition to recommend which friends to tag in your photos.

But the technology has a dark side. The U.S. government is in the process of building the world’s largest cache of face recognition data, with the goal of identifying every person in the country. The creation of such a database would mean that anyone could be tracked wherever his or her face appears, whether it’s on a city street or in a mall. Today’s laws don’t protect Americans from having their webcams scanned for facial data.

Security CCTV. Peter Marlow/Magnum Security CCTV. Peter Marlow/Magnum

Not That Perfect

Face recognition systems have two components: an algorithm and a database. The algorithm is a computer program that takes an image of a face and deconstructs it into a series of landmarks and proportional patterns—the distance between eye centers, for example. This process of turning unique biological characteristics into quantifiable data is known as biometrics.

Together, the facial data points create a “face-print” that, like a fingerprint, is unique to each individual. Some faces are described as open books; at a glance, a person can be “read.” Face recognition technology makes that metaphor literal. “We can extrapolate enough data from the eye and nose region, from ear to ear, to build a demographic profile,” including an individual’s age range, gender and ethnicity, says Kevin Haskins, a business development manager at the face recognition company Cognitec.

Face-prints are collected into databases, and a computer program compares a new image or piece of footage with the database for matches. Cognitec boasts a match accuracy rate of 98.75 percent, an increase of over 20 percent over the past decade. Facebook recently achieved 97.25 percent accuracy after acquiring biometrics company in 2012.

So far, the technology has its limits. “The naive layman thinks face recognition is out there and can catch you anytime, anywhere, and your identity is not anonymous anymore,” says Paul Schuepp, the co-founder of Animetrics, a decade-old face recognition company based in New Hampshire. “We’re not that perfect yet.”

The lighting and angle of faces images must be strictly controlled to create a usable face-print. Enrollment is the slightly Orwellian industry term for making a print and entering an individual into a face recognition database. “Good enrollment means getting a really good photograph of the frontal face, looking straight on, seeing both eyes and both ears,” Schuepp explains.

How face recognition is already being used hints at just how pervasive it could become. It’s being used on military bases to control who has access to restricted areas. In Iraq and Afghanistan, it was used to check images of detainees in the field against Al-Qaeda wanted lists. The Seattle police department is already applying the technology to identify suspects on video footage.

The technology’s presence is subtle, and as it gets integrated into devices we already use, it will be easy to overlook. The most dystopian example might be NameTag, a startup that launched in February promising to embed face recognition in wearable computers like Google Glass. The software would allow you to look across a crowded bar and identify the anonymous cutie you are scoping out. The controversial company also brags that its product can identify sex offenders on sight.

As the scale of face recognition grows, there’s a chance it could take its place in the technological landscape as seamlessly as the iPhone. But to allow that to happen would mean ignoring the increasing danger that it will be misused.