How open government can turn evil thanks to big data and machine learning

The New Yorker recently published an insightful article called „We Know How You Feel - Computers are learning to read emotion, and the business world can’t wait.“, which discusses the potential of machine learning and big data. The author tells the story of how advanced facial recognition has developed. It is now possible to analyze single facial expressions and emotions we are not even aware of ourselves. That has triggered a lot of interest by advertisement companies. The goal is to increase the attention advertisements receive. One such company developing such software is Affectiva and it seems to be doing well.

Voice experts have trained computers to identify deep patterns in vocal pitch, rhythm, and intensity; their software can scan a conversation between a woman and a child and determine whether the woman is a mother, whether she is looking the child in the eye, whether she is angry or frustrated or joyful. Other machines can measure sentiment by assessing the arrangement of our words or by reading our gestures. Still others can do so from facial expressions.

You wonder what this has to do with open government? In another example the author discusses the recently introduced Happiness Index in Dubai This sounds like an amazing approach for social innovation and open government. You can rate government services by simply clicking a button, giving officials instant feedback. I will have to wait probably decades before this would be implemented in German government offices. Public servants who listen and learn from constant feedback.

But the above approach could be implemented much easier by using emotion analysis software because Dubai has the tightest CCTV surveillance network and that’s what is also discussed in the article:

Kaliouby forwarded me an e-mail that a member of her team wrote about a state initiative in Dubai, the Happiness Index, meant to measure social contentment: “Dubai is known to have one of the world’s tightest CCTV networks, so the infrastructure to acquire video footage to be analyzed by Affdex already exists. I feel that’s a promising opportunity.”

And here things become awkward. Imagine you can analyze in real-time the mood of your beloved citizens. It is all happiness, right? The granny sitting in the shopping mall is feeling sick. Better send an ambulance. The boy on the playground has some grim face. Better get the police to check by. You never know. This bearded man is for the 7th day nervously standing in front of the embassy. Preemptive arrest warrant is recommended by our electronic assistant. Let's order "Robot Cop" to take care of that issue.

It is scary to see how the technological enthusiasm leaves often little room to elaborate its consequences. The Dubai government also launched recently a Drones for Good event. Using drones for good purposes such as Drones watch for disease and collect real-time data on crop health and yields. How can we really control that flying cameras are always intended for good purposes? And who defines what a good purpose is?

As much as I am enthusiastic about these new technologies’ potential, I am slowly starting to feel worried how these new approaches are implemented. There is hardly any discussion about consequences. The above company does not seem to worry about them. Open government is a complex attempt to re-invent the relationship between citizen and government. We need to be careful that this does not turn into mass surveillance.