AI's moral architects: neither demi-gods nor code monkeys

Here we run up against the problem of many hands, or the difficulty in placing blame in settings where multiple individuals contribute to an outcome. This is inevitably the case in AI development, where potentially hundreds of practitioners contribute modules to a product. That is not to absolve the individual: “It is always individual developers who execute the work. However, there is no real accountability for the choices they make, and since society doesn’t take them seriously as moral actors, they don’t have to take themselves seriously either.” 

Since the project of AI and automation is predicated on surveillance and massive data gathering, developers are in an inherently manipulative relationship with the public, she argues. And with the amount of money in play, the industry is full of perverse incentives that encourages manipulation in the service of maximizing profit. “It’s in companies’ interest to treat developers as ‘code monkeys’, replaceable cogs that do the job without thinking about the bigger picture or asking ethical questions that might eat into profit margins or the race to gather data.” 

Her research left Griffin not only full of admiration for developers’ abilities but also with sympathy for the difficult dilemmas they have to face, often on their own and ill-prepared. “Developers can either quit or resort to malicious compliance, essentially doing a bad job or claiming an unethical request isn’t technically feasible. Of course, they can also become whistleblowers but that often means forfeiting any future career and with that a lot of money and prestige.”

Griffin recalls the example of a developer who had to balance fairness and safety and tried to aim for a compromise without understanding the implications of the trade-off. “Many of the practitioners I interviewed were aware of the ethical dimensions of what they were being asked to do but didn’t feel they had the tools, or access to a strong community of support, to make good decisions. I think academia has to do a better job here; ethics courses are often tacked on to curricula and taught in too theoretical a fashion. At the same time, we need work environments that encourage ethics conversations rather than keeping developers in the dark.”

This raises the bigger societal question about why AI developers don’t fall into the same category as heavily regulated and codified professions such as lawyers and physicians, in that their clients – an increasingly defenceless public – are entirely dependent on their obscure craft. “Developers should be aware of how vulnerable the people whose data they are harvesting are. And how vulnerable the people are who their systems are deployed on. It’s a grave imbalance of power.”

Griffin cites the child benefit scandal that brought down Mark Rutte’s last government in the Netherlands. “You could have your kid taken away because a judge relied on what an algorithm said instead of the actual evidence. But someone programmed that algorithm and set the parameters such that single mothers and people with foreign sounding names were deemed more likely to be fraudsters. And we never even considered including the developers in our conversations about this scandal.”

While in high demand, AI developers as a group have little political power. Talking about unionising feels almost frivolous for a profession that already enjoys the traditional fruits of collective bargaining: excellent wages, benefits, and working conditions. The ethical implications of labour are a new frontier here. “In the absence of political organisation and collective bargaining, it becomes ethics shopping for companies – in the end, pretty much everyone has a price – and the plausible deniability of only having contributed a small piece of the system makes it even easier to just go along.”

Once AI developers understand themselves as moral agents in a sense that transcends their role at work, they still need a community of developers who can aid each other. “At conferences, when developers present their models, it’s all about technical scrutiny – the ethical aspect is not at all something prestigious in the profession. Developers are using their technical imagination to build these systems; they can also use their ethical imagination to consider potential nefarious consequences of their models and how they can be mitigated.”

Griffin also thinks we should get students of computing reflecting and growing together when it comes to practical ethics. “Developers need to understand that ethics isn’t just common sense. To become a virtuoso at teasing right and wrong apart, you need to put in the work, like with everything else. If you decide you want to specialise in fairness, you quickly learn that there are half a dozen definitions that can’t be mathematically satisfied at the same time. Learning how to be an expert in fairness requires a community who cares about getting this right.”

Overall, Griffin does see some positive changes happening. “With AI now ubiquitous, people are becoming more aware of issues around privacy, and they are starting to explore ways to push back. I think you can be very enthusiastic about this technology and still be against the gross, dehumanising, capitalistic extraction of our data. But it will take a movement, and AI developers have to be a big part of that.”

Text: Florian Raith