TV shows are right to be scared about AI, but not for the right reasons
A couple of weeks ago, Fox announced their new “science fiction” TV show “neXt”. The shows description makes it sound like one along the lines of Black Mirror or Phillip K. Dick’s Electric Dreams, a cautionary tale on the encroaching control technology has over its users:
“neXt is a propulsive, fact-based thriller about the emergence of a deadly, rogue artificial intelligence that combines pulse-pounding action with a layered examination of how technology is invading our lives and transforming us in ways we don’t yet understand.”
However, as far as I can tell from its trailer, the show couldn’t be further from that goal, as we see an Alexa-lookalike gain a will of its own and start causing havok, rebelling against its human creators and users with the subtlety and verbosity of a Bond super-villain. This H.G. Wells-ish approach of representing societal fears in the form of an “invader” figure may have worked for War of the Worlds or Godzilla, but used in the context of AI it’s not only harmless fiction but actively shielding us away from what we should be concerned about regarding AI and decision-making algorithms in general, many of which impact human lives in the decidedly non-fictional reality we’re living in.
It’s not (just) data privacy
I mean, it is. You should be concerned about where your data, the backbone of AI, is going, but it’s becoming increasingly inevitable that by being an actor in society you’ll be forced to yield a very sizable portion of your tastes, behaviors and very self to data-gathering companies dedicated to squeeze as much as value as they can from it, and there’s very little we can do about it without becoming digital hermits.
Did you know not only can you be tracked and identified through clicks and online searches, but also your physical actions? People blocking their laptop webcams may be on the right track, but they may not have heard of behavioral biometrics, techniques through which details such as the cadence of your typing on a keyboard or the travel patterns of your mouse can tie you back to your identity (learn how to stop that here).
That’s not to say don’t try to minimize your digital data footprint, but data privacy efforts should also focus on public outcry for strict safety and privacy policies. On a recent episode of the Vox podcast Recode Decode, DuckDuckGo CEO Gabe Weinberg correctly argues opting-in to data tracking should be the default, rather than the obscure opt-out approaches many data companies implement, a view that many European countries have agreed with through policy but needs to be pushed worldwide.
Behavior-shifting tactics
Money-hungry data companies have realized that not only can they use their assets and tools for data collection and behavioral predictions, they have started to realize they can deploy their insight to shift behavior, effectively betting on our future actions and rigging those bets in whatever ways they can. In the book The Age of Surveillance Capitalism, author Shoshana Zuboff argues the endgame for capitalist behavioral analytics may not just represent a threat to our privacy, but our very free will; a deterministic landscape of preset decisions and self-fulfilling prophecies:
“Surveillance capitalism unilaterally claims human experience as free raw material for translation into behavioural data. Although some of these data are applied to service improvement, the rest are declared as a proprietary behavioural surplus, fed into advanced manufacturing processes known as ‘machine intelligence’, and fabricated into prediction products that anticipate what you will do now, soon, and later. Finally, these prediction products are traded in a new kind of marketplace that I call behavioural futures markets. Surveillance capitalists have grown immensely wealthy from these trading operations, for many companies are willing to lay bets on our future behaviour”
The implications of non-regulated data tracking and manipulation under the control of the capitalist mindset should be evident. From threats to our very behaviors to democracy itself, we don’t really need a computerized super-villain to imagine very real and present AI threats. A strong, ethics-based approach to data policies is fundamental if we want to make it out of this new digital golden age with our autonomy intact.
No such thing as bias-free
Biases are everywhere, and algorithms aren’t helping, they’re exacerbating the problem by obscuring it. Data and algorithms are not an objective reflection of our world; they carry with them the very same prejudices, biases and misunderstandings as the cultures and organizations that created them, except they’re hidden inside packages that help the lack of accountability for these mistakes and are easier to justify using determinism mistaken for objectivity. And when we implement systems that decide who to prioritize for medical care, who to suspect of crime or what content to recommend to children online, it is something we should be concerned about.
Rachel Thomas, a professor at the University of San Francisco Data Institute and co-founder of fast.ai has spoken and written at length about this:
“our algorithms and products impact the world and are part of feedback loops. Consider an algorithm to predict crime and determine where to send police officers: sending more police to a particular neighborhood is not just an effect, but also a cause. More police officers can lead to more arrests in a given neighborhood, which could cause the algorithm to send even more police to that neighborhood (a mechanism described in this paper on runaway feedback loops).”
Computer vision algorithms perform worse on people of color due to the overwhelming majority of facial data being from people of European descent, most collected behavioral data comes from rich parts of developed world countries, autofill algorithms are more likely to suggest STEM-related occupational terms for men and housekeeping terms for women. These errors don’t come from anyone’s maliciousness but societal and personal biases reflected in the data’s breadth and width; regardless of origin, they need to be addressed.
Far from arbiters of objective truth, algorithms and data carry our biases, of which there are plenty, and lack of understanding from those that implement, deploy, consume and regulate them can lead us to being carried off the cliff by prejudiced, manipulable machines of our own creation. Ethical considerations have to be at the core of AI from its inception, we cannot wait until the damage has been done or trust that the threat of large financial penalties will be more than a band-aid.
What to do about it?
Sadly, solving these problems isn’t just a matter of calling on the scruffy no-rules detective to beat Alexa’s evil cousin like Fox’s neXt would have us believe. There is no “other” here, no rogue AI from an experiment gone wrong, just the empowerment of our societal prejudices and misgivings, packaged in shiny deterministic lines of code. Better curtains to hide behind for unconscious biases.
We already know how to handle biases, though. Humanity has been clever enough to recognize them in itself, and has been able to correct them (to varying degrees) before. The challenge is in having to apply these solutions to systems that we might have once imagined fair or objective, and making the people that act on each step of these data processes - its collection, storage, exploration, modeling and deployment both cognizant of these limitations and accountable for their actions. We need to recognize that these are easy to miss, and be open to considering our algorithms may well make mistakes, more frequently and impactfully than expected, and prepare to course-correct if not flat out discard them proactively and not when the damage is impossible to deny.
At an InfoQ presentation, Rachel Thomas outlines some solutions to problems we’ve covered here, such as:
-
Don’t maximize metrics - It’s easy to just want numbers to go up, but the more we reduce data into numbers, the more easy it is to miss how we may be rewarding negative behaviors in the services we provide or enabling feedback loops that restrict said services unjustly.
-
Hire diverse teams - Research shows diverse teams perform better, and in analytics they can contribute to a better understanding of the subject matters your datasets deal with, leading to better, actionable insights and an assurance that bias is not coming from your team.
-
Push for policy - Rachel writes “Advanced Technology is Not a Substitute for Good Policy”. Bias is hard to catch; no matter how good your algorithms or pipeline, regulations and protocols need to be in place so users can appeal decisions enacted by algorithms, results validated and damages prevented or undone at a level beyond that of companies’ own doors.
-
It’s a neverending job - We won’t reach a point where we or our algorithms will be bias free. That’s how we got here in the first place! We need to train people to always be on the lookout for potential feedback loops, biases and prejudices implicit in our data. Leave the black-and-white scenarios for bad science fiction TV shows.
