Ihe demand for a six-month moratorium on artificial intelligence (AI) systems like ChatGPT has the merit of drawing attention to the many ethical questions raised by chatbots. One of the most fascinating concerns the moral architecture of what should be called, following Wendell Wallach and Colin Allen (Morale Machines, OUP USA, 2008), “artificial moral agents”.
ChatGPT probably does not have the properties required to be a “moral patient”, i.e. an entity to whom we would have moral obligations (like don’t hurt her if she’s sentient). On the other hand, the system developed by OpenAI can be described as an “artificial moral agent”, since he seems able to respect – with more or less success – moral obligations.
A conversational robot must indeed be programmed to deal with morally delicate situations: what to answer to a sexist insult? Does he always have to tell the truth, including to a child who asks if Santa Claus exists? Certainly, ChatGPT is devoid of any communicative intent and he doesn’t understand anything. But words can deceive and they convey values: they can, in particular, carry objectionable clichés and stereotypes.
From a technical point of view, moral normativity appears at different levels. First, there are the values and norms that imbue the textual data collected for learning; it is also possible to intervene at this first level to correct certain undesirable biases. Reinforcement learning with human feedback is a second source of normativity, as the AI is trained to give “useful, honest and non-toxic” responses. Finally, at a third level, it seems possible to introduce, via a prompt [une instruction] hidden (like Bing’s Sydney), rules such as always presenting yourself as a machine or shunning invitations.
This raises a (meta-ethical) question as new as it is fundamental: how to choose and justify the moral obligations that AI should follow? In short, what moral theory to implement in ChatGPT?
consequentialism and ethics
If we stick to the philosophical tradition, at least three moral architectures are possible: consequentialist, deontologist and according to the ethics of virtue.
For the consequentialist approach (of which utilitarianism is a version), ChatGPT will be well programmed if it produces positive effects on its users and on the world. It is important to ask whether such a setting will increase or decrease the well-being of moral patients. Thus, consequentialist programming could very well justify lying to the child about Santa Claus in order to maximize his enjoyment. How technically feasible and morally desirable is such a robot? These are two complex questions raised by the consequentialist robot.
You have 44.04% of this article left to read. The following is for subscribers only.