There is a particular kind of horror in being accused of a crime against a child, and a worse one in being accused of it by a computer that was never asked to explain itself. Robert Dillon lived both. In June 2026, the American Civil Liberties Union filed suit on his behalf against the Florida law enforcement officials who, the complaint alleges, “let an error-prone artificial intelligence system stand in for an investigation.” The phrase is doing a lot of work, and it deserves to. Because the story of how Dillon ended up spending a night in a Florida jail is not a story about one bad cop or one bad day. It is a story about an entire investigative shortcut that police departments across the country have quietly normalized.
The incident that wasn’t his
In November 2023, someone allegedly tried to lure a 12-year-old girl away from her parents at a McDonald’s in Jacksonville Beach, Florida. It is the kind of report that, understandably, mobilizes a department immediately. A month later, in December 2023, police made their first contact in the case. What they had to work with was thin: surveillance imagery, the kind of grainy, low-resolution capture that any honest analyst would describe as a lead, not an identification.
So they fed it to a machine. The system was FACESNXT — the Face Analysis Comparison and Examination System — and it returned what reads, on paper, like certainty: a 93% match on facial features. To anyone unfamiliar with how these systems actually work, “93%” sounds like a probability of guilt. It is nothing of the sort. It is a similarity score between two images, generated by software with no knowledge of context, no ability to weigh evidence, and no stake in being right. But it was enough. In August 2024 — eight months after that first phone call — Robert Dillon was arrested.
What a match costs a human being
Dillon spent the night in jail. To get out, he had to borrow money and pledge the title to his truck for bond. Sit with that for a moment: a man’s vehicle, his means of getting to work, handed over as collateral because an algorithm decided his face was 93% similar to a blurry frame of someone else.
The thing is, Dillon could see what the machine couldn’t. “The scars are nowhere near alike,” he said. “It absolutely blew my mind.” A facial recognition system does not see scars as a person does, does not reason about distinguishing features the way a detective interviewing a witness might. It compares geometries. And here, the geometry was wrong about a human being’s freedom.
The charges were dropped roughly two months after the arrest. But “dropped” does not mean undone. An arrest record for attempting to lure a child does not evaporate from a person’s life because a prosecutor later concludes the case was built on sand.
The pattern, not the glitch
The ACLU’s Nathan Freed Wessler put the stakes plainly: the arrest proves “this technology is fundamentally dangerous.” The temptation, whenever one of these cases surfaces, is to treat it as an aberration — a misconfiguration, a careless analyst, a department that didn’t follow protocol. But that framing protects the technology at the expense of the truth. The danger is not that facial recognition occasionally fails. The danger is the structure it creates around failure.
Consider what the system did to the investigation itself. A 93% match is psychologically authoritative. It tells officers, in the language of mathematics, that they already have their man. Everything downstream of that number — the warrant application, the witness statements, the decision to arrest rather than investigate further — is shaped by the assumption that the hard part is done. The machine doesn’t just produce a lead; it produces a confidence that corrodes the rest of the process. This is the real mechanism of wrongful arrest by algorithm: not that the software lies, but that humans stop double-checking once it speaks.
And the populations who bear the cost are not randomly distributed. Facial recognition systems have repeatedly been shown to perform worst on darker-skinned faces, on women, on the young and the old — precisely the people already most likely to encounter the sharp end of policing. A tool that fails unevenly, deployed in a system that already polices unevenly, does not correct injustice. It launders it through a veneer of objectivity.
Why “just a lead” is a fiction
Defenders of the technology insist that facial recognition is only ever used as an investigative lead, never as the sole basis for an arrest. Dillon’s case is the answer to that claim. According to the complaint, the case against him rested on the FACESNXT result and a thin layer of corroboration — not on the kind of independent investigation that would have caught the mismatch a man could see in his own mirror. The promise that the algorithm is “just one input” repeatedly fails to survive contact with the institutional reality, where a high match score becomes the spine of the entire case.
This is the lesson that should outlast the headline. The problem with facial recognition in policing is not solved by better cameras, higher-resolution training data, or a department policy memo reminding officers not to over-rely on it. The problem is that the technology manufactures unearned certainty, and human institutions are catastrophically bad at resisting it.
What accountability would actually look like
Dillon’s lawsuit asks the courts to recognize what regulators and legislators have been slow to: that deploying probabilistic identification software against people’s liberty, without meaningful safeguards, is itself a harm. Real accountability would mean more than damages for one man. It would mean banning or strictly limiting facial recognition as a basis for arrest warrants, mandating disclosure to defendants whenever the technology was used, and requiring that any match be treated as inadmissible unless independently corroborated by traditional evidence.
Until then, the FACESNXT result that upended Robert Dillon’s life remains a preview, not an outlier. Somewhere right now, a grainy frame is being fed into a system that will return a number with two digits and a percent sign, and someone will mistake that number for the truth. The only question is whose truck title gets pledged next.
