AI Systems and Criminal Liability

2.1 If not the AI System, Then Who is to Be Held Liable?

ʻWhoever kills another person […]’ serves as a global expression of an act that can incur criminal liability. Neither the wishful thinking nor the mere intention to achieve the harmful result are enough for holding someone liable in the way that liability is conceived in the realm of criminal law. There may be no universal definition, but there are common denominators of the different approaches to the notion of acting (known as actus reus), whether codified⁹ or not:¹⁰ some sort of conduct that is either positive (act) or negative (failure to act despite having a legal duty to do so; omission) and that has not only some social relevance but is also the outcome of self-reflection and free will. Against this backdrop, the first struggle the legislator needs to overcome to ascribe criminal liability to an AI system is answering the question of whether the latter is capable of acting in the sense of criminal law.¹¹

To address this question, one needs to navigate the (blurry) waters of the definition of AI and, particularly, unveil the meaning ascribed to the second component of this term, namely ʻintelligence’, inasmuch as it is the latter that seems to allow a comparison between humans and AI systems as equal (?) addressees of criminal punishment. The EU legislator has already provided a working definition of AI in Art 3, point (1) AIA¹² that should assist scholars and practitioners in classifying and making better sense out of multiple computational models.¹³ Yet, this definition will not necessarily shed any light into the concept of personhood for the purpose of solving the criminal liability knot. At the same time, the existing scholarship suggests that, although further progress is expected in AI settings, AI remains at the moment (and for the foreseeable future) narrow (as opposed to general AI). This means that AI systems ʻcan perform one or few specific tasks’.¹⁴ Building the capabilities required to achieve general AI, including self-awareness and the ability to define its own purpose, is still a technological challenge.¹⁵ This latter note suggests that the discussion on attributing legal personhood or capability to AI systems for the purpose of holding them accountable directly, namely without having to identify the responsible human agent acting behind them, is – at least – premature. Even if one considers the bodily dimension of conduct obsolete, particularly in those legal orders where corporate criminal liability is considered a welcome extension of criminal liability beyond human agents, the lack of meaningful purpose and self-awareness contradict the ability for judgement and free will (concepts that continue to pose obstacles to the universal acceptance of criminal liability of legal persons) as a fundamental component of blame and punishment.¹⁶ Thus, in the best-case scenario, we are dealing (again) with a new agent who has no soul to be damned, and no body to be kicked¹⁷ – at least, not in a way that would fulfil the purposes of criminal punishment (despite the lack of unanimity as to the prevalent doctrine), ranging from retributivism to rehabilitation.¹⁸

2.2 Lost in Causation

The rejection of AI personhood does not negate the relevance of AI-related offences, but certainly impacts on the content ascribed to this term. For instance, Hayward and Maas distinguish between:¹⁹ crimes with AI, where AI serves as a tool (the same way one uses a weapon or a knife to commit homicide or a computer system to commit fraud); crimes on AI, where AI becomes the object of the criminal act (similar to other already established offences such as the attacks against information systems); and crimes by AI. In the case of the latter category, where one could subsume car accidents caused by autonomous vehicles, AI – should we decline AI personhood – is to be classified as an intermediary the contribution of which to the harmful result is relevant to the extent it can be attributed, whether fully or partially, to a human agent or it can otherwise assist us in evaluating the action (or the lack thereof) of a responsible person.²⁰ This case appears to be the most problematic considering that, first, the attribution of the AI systems’ contribution to a human agent depends on the degree of autonomy of this particular system,²¹ and that, second, the expectations we may have regarding the human interaction with AI systems are not, nor should be (based on the current level of development of such systems), unlimited.

This already implies that, besides addressing the question of whether a new agent in the form of an AI system that presents a certain level of autonomy is arising or will arise any time soon, one should evaluate the conduct of a multitude of agents that interact with the AI system and are possibly involved in the commission of AI-related offences.²² This includes the persons, whether natural or legal, that design, program, train, develop, produce and monitor the AI system, the users and operators thereof as well as those otherwise interacting therewith (eg, other drivers in the case of autonomous vehicles or other users of digital platforms in the case of chatbots). Holding these persons liable in the sense of criminal law presupposes that they have not only acted or failed to act in a way that at least absorbs the ʻpoor judgement’ of the AI system (otherwise, one should employ the ʻin dubio pro reo’ principle),²³ but also there is a causal link between this specific conduct (eg, faulty programming, training errors or lack of monitoring)²⁴ and the harmful result in question.

The analysis of the different outcomes resulting from the adoption of one or another model of causation or causality, including but not limited to the conditio-sine-qua-non formula and the doctrine of objective attribution, falls outside the scope of this article.²⁵ Yet, all different approaches encounter the so-called ʻproblem of many hands’,²⁶ which is exacerbated in AI settings.²⁷ Before reaching the market, AI systems are in principle developed, trained and produced at industrial scale; that is, there is a multitude of persons, groups and departments that shape the ʻdeveloper-manufacturer-user’ chain, which is further complemented by those responsible for the post-market monitoring of the product in question.²⁸ This means that it can be challenging to single out the critical mistake(s) and, in general, to deconstruct the series of interactions of these actors and distinguish the foreseeable from the unforeseeable chain of events as well to evaluate those in the light of the skills and capabilities of each involved party.

This challenge is not new to the criminal-law doctrine, nor unique to AI settings, considering that the complexity of the corporate environment and decision-making has served as one of the main arguments in favour of introducing schemes of corporate criminal liability.²⁹ Instead, the decisive difference is the lack of predictability³⁰ that is associated with the use of an AI system – often described as the ʻblack box’ that even computer engineers seek to peek inside.³¹ In particular, depending on their level of autonomy and learning capabilities, the outcome of using an AI system may differ (substantially) from what the designer or the producer could envision or the (lay) user would expect based on the information provided by the former.³² This may pose a great obstacle to the verification of the causal link or even negate its existence. In the latter case, a scheme of criminal liability based on over-simplified approaches³³ to causation would violate the principle of guilt³⁴ – with the human ʻin the loop’ being treated as a mere object of liability³⁵ or as a mere means to achieve the end of reducing impunity in AI settings. Thus, acknowledging that there may be a gap of responsibility³⁶ in such cases would be an optimal solution that also complies with the fragmentary character of criminal law.

2.3 Mens Rea without Boundaries?

Shifting the focus onto mens rea, many authors argue that the cases where one deploys an AI system as an instrument with the intention to cause harm (eg, violations of property, privacy and data protection or even human life)³⁷ are less problematic or may present difficulties already addressed in the criminal-law doctrine (eg, the crime committed with the use of the AI system goes beyond the original intent).³⁸ This is only partially true inasmuch as intention, whether in the form of dolus directus (first or second degree) or in the form of dolus eventualis, consists of a cognitive (knowledge) and a voluntative element (will). To ascribe liability for an intentional offence, the offender should at least be able to objectively foresee that his/her conduct is likely to cause the harmful effect or the danger prescribed in law, but proceeds to act or omits to do so regardless of the consequences of his/her conduct. In AI settings, however, one may not comprehensively understand the inner workings of the system (s)he employs or what the consequences of the system’s decision will be and, thus, there can be no certainty or even strong likelihood that the AI system will serve his/her criminal intent. As Beck points out, there may be an abstract perception of the possibility of causing a damage, but it remains doubtful whether this generic knowledge is enough to establish liability on the grounds of intent.³⁹

Negligent offences⁴⁰ are expected to absorb the majority of criminal incidents involving AI systems, but the establishment of negligence can be equally challenging. Compared to intent, negligence stands for a qualitatively different combination of knowledge and will. Again, the offender needs to be able to at least foresee that his/her conduct could cause a specific result, although (s)he does not accept its occurrence and may have wished to avoid it. Even, in the case of the so-called ʻunconscious negligence’, where one does not foresee the harmful consequences of his/her behaviour and, thus, there can be no kind of will attached to them, criminal liability will be ultimately linked to the capability of this specific person to foresee the violation of the legal interest in question under the concrete circumstances.⁴¹ This means that negligence cannot, nor should, serve as a gatekeeper of criminal liability in those cases where the autonomous learning and development (information analysis and response) of the AI system negates the predictability of the harmful event for those designing, developing, manufacturing, monitoring or using the system. There can be exceptions to this rule. As Kaiafa-Gbandi correctly notes, liability should not be waived when the individual takes over the role of the system’s supervisor in terms of monitoring AI learning with the aim of intervening and interrupting the use of the system in place in case of dangerous system discrepancies, provided such a duty is prescribed in law.⁴² Even this solution does not necessarily guarantee better results in terms of establishing liability, unless the supervisor ʻhas actual meaningful control over subsequent events’, a capability that may be impacted adversely due to passive monitoring (as the example of autopilot has proved) or lack of time.⁴³

Lastly, no matter if one places the notion of negligence exclusively in the area of mens rea or ascribes thereto additional objective dimensions (an analysis that is beyond the scope of this article),⁴⁴ the standard of due care (otherwise the duty to take appropriate and reasonable care and, in particular, the failure to exercise it) is of central importance for establishing criminal liability on the grounds of negligence. This suggests that there should be different expectations towards each person involved in the design, development, training, production, monitoring or use of an AI system depending on their exact role and tasks. Furthermore, in certain cases, national or supranational (hard) laws or even non-governmental standards (eg, ISOs) may stipulate positive obligations for this purpose. The future implementation of the AIA is expected to signal the proliferation of such rules and standards, the number of which remains for the moment rather limited.⁴⁵ Yet, the involvement of the private sector in this rule-making procedure raises concerns as to whether these requirements will be shaped with the aim of ensuring the maximum care for the legal interests at stake or with the aim of minimising the financial burden on the respective industry.⁴⁶ In any event, they will serve as a basis for substantiating the claim of lawfulness of the conduct of AI designers, developers, manufacturers, users etc⁴⁷ and thus for negating, inter alia, criminal liability on the grounds of negligence.

3.1 Permissible Risk

The risks that arise in AI environments serve as a reminder of the challenges that technological progress, such as the mass-scale production and use of cars, the function of industrial plants operating with nuclear energy, or the globalisation of the internet, has historically presented to legislators, as well as of the fruitless nature of general or absolute prohibitions on endangerment that would defeat the social benefit arising from this progress.⁴⁸ In this context, the doctrine of permissible risk (also referred to as socially acceptable risk)⁴⁹ is used to cap the aforementioned standards of due care and, thus, to limit the scope of the neminem laedere principle.⁵⁰ This suggests that the legal order and the victim of an (undesirable) criminal act should accept a minimal level of risk that ʻpersists’ even when all precautions are taken, namely when the duty to take appropriate and reasonable care is properly exercised. Such tolerance is expected in the light of the permission granted by the same legal order to certain activities due to the societal benefit associated therewith. This stance should be re-examined if it becomes evident that the trust in the effectiveness of the care obligations designed to mitigate the risk is not or no longer justified.⁵¹ This implies that this ʻgap of responsibility by design’ is accepted only to the extent that the current state-of-the-art is closely observed. In this context, ascribing the duty of after-sale monitoring to the manufacturer of the AI system – compared to his/her privileged position to predict and mitigate risks compared to the end users – could be considered, particularly for those cases where the end-product is subject to updates and patches.⁵² Besides this, generally foreseeable product discrepancies will not justify the establishment of criminal liability, unless they amount to design, development, training, production, monitoring or operation-related faults that could have been prevented in the first place.

The use of the ʻpermissible risk’ doctrine is not merely of theoretical interest. It can have practical implications inasmuch as it will mean that the victim will have to tolerate certain harms without access to redress. Besides this, it has substantial societal implications, considering that it encompasses an answer to the question of how much risk our societies can and should tolerate in the age of AI,⁵³ but at a moment when neither the use of AI systems nor the risks emerging therefrom have yet been normalised (as this may be the case once they ʻintrude’ into the global market and their use becomes regular). This makes it notably challenging for the lawmaker to establish accurately the initial threshold for impermissible risk, as (s)he has to strike a satisfying balance between fostering innovation and ensuring safety, a task that one should not underestimate, as the lengthy negotiations on the AIA have showcased. This balancing act may remain a crucial aspect of the ever-evolving process of law-making, but it seems that the rapid pace of AI advancement is leaving the legislator struggling to keep up.

3.2 New Endangerment Offences

The regulation of (im)permissible risk in AI settings can take place either outside or within the realm of criminal law. Considering the gap of responsibility that may arise due to the difficulties inherent in identifying the responsible human agent, the acts of whom one can evaluate in terms of criminal law, establishing the causal link between his/her conduct and the harmful event and, lastly, overcoming the mens rea limitations, one may choose to intervene by means of expanding criminalisation and, particularly, adopting new (abstract) endangerment offences that will compensate for the lack of predictability that, based on the previous analysis, seems to be a horizontal obstacle.⁵⁴ Such an intervention may mean the criminalisation of the establishment or maintenance of a functional source of danger in the form of releasing an AI system with unlimited learning possibilities and without sufficient safeguards.⁵⁵

This solution would release the judiciary from the burden of proving causation, but it remains doubtful whether it would foster innovation or, instead, stall the development of learning capabilities of AI systems, a development that is considered prima facie beneficial to the extent that it allows the system to improve its performance over time (despite the unpredictability it goes together with).⁵⁶ Besides this, this may pave the way to over-regulation in the form of over-criminalisation, an outcome that would be hardly compatible with the ancillary nature of criminal law and the protection of fundamental rights in this realm.⁵⁷

Lastly, it is equally important to address the question of who may adopt these new AI-related endangerment offences. In the EU ecosystem, one would expect the national legislator to take the lead, considering that this step may presuppose interventions in the area of the General Part of Criminal Law, the provisions of which are not harmonised and which, given the cultural differences of the European legal orders, should preserve its unique character to avoid conflicts with fundamental tenets of national criminal justice systems.⁵⁸ This approach underestimates, however, the fact that the majority of EU Member States are not producers of AI systems, nor are systematically involved in the development thereof. This means that the know-how required to proceed with criminalisation of this kind will most likely be missing at national level.⁵⁹

On the other hand, resorting to the mechanisms of harmonisation of substantive criminal law as set out in Article 83 TFEU exposes us to their inherent limitations, whether one seeks to meet the criterion of serious, cross-border crime (para 1) in order to justify the expansion of the list of the so-called euro-crimes or resorts to the solution of ʻfunctional’ substantive criminal law (para 2).⁶⁰ The latter solution presupposes an existing Union policy in an area that has already been subject to harmonisation measures.⁶¹ In this case, the harmonisation of criminal laws should assist in securing the effective implementation of this policy, while a more cautious approach (that complies with the principle of subsidiarity and proportionality) would require a serious enforcement deficit that can only be resolved through the threat of punishment.⁶² This conclusion would be – maybe with the exception of autonomous vehicles, the production of which has already been subject to harmonisation of a certain level⁶³ – rather premature for the majority of AI systems.