Security by Design: Aspirations and Realities in a Regulatory Context

2.1 Engineering Method

The centrality of design for achieving adequate security of computer systems has long been recognised. In 1970, an influential report by the US Defense Science Board Task Force on Computer Security stated that ‘[p]roviding satisfactory security controls in a computer system is in itself a system design problem’.¹² Somewhat ominously, it also stated that ‘[d]esigners of secure systems are still on the steep part of the learning curve and much insight and operational experience with such systems is needed’.¹³ Over half a century later, both observations still ring true, as Section 4.2 indirectly indicates. Nonetheless, significant progress has been made since the 1970s in the conception, development and refinement of security engineering methods, partly under the flag of ‘security by design’.

The SbD concept originates partly in work by computer scientists who, back in the 1970s, propounded generic design principles for ensuring the security of computer systems.¹⁴ These principles did not expressly reference SbD, but they reflected the aforementioned view of the US Defense Science Board Task Force on Computer Security that the provision of adequate computer security depends on proper design. In subsequent years, various sets of security design principles were formulated, although not always consistently, without using the SbD mantra.¹⁵

Work on design-focused engineering methods supplemented the successive elaboration of design principles. For present purposes, a particularly noteworthy line of engineering research has been directed at establishing patterns of design to eliminate the accidental insertion of security flaws into software or to minimise the adverse effects of such flaws. This work was first referred to as identifying ‘secure design patterns’.¹⁶ It was subsequently called a ‘secure by design’ approach to software development. Santos and others have summarised this approach as follows:

Secure by design is an approach to developing secure software systems from the ground up. In such approach [sic], the alternate security tactics are first thought; among them, the best are selected and enforced by the architecture design, and then used as guiding principles for developers. … [D]esign flaws in the architecture of a software system mean that successful attacks could result in enormous consequences. Therefore, secure by design shifts the main focus of software assurance from finding security bugs to identifying architectural flaws in the design.¹⁷

Aspects of this approach were later incorporated into technical–industrial codes of practice, particularly with respect to the IoT. An early example is ‘IoT Security Guideline v1.2’, published in November 2017 by the Internet of Things Alliance Australia (IoTAA).¹⁸ The guideline expressly ‘promotes a “security by design” approach to IoT’, and sets out a comprehensive ‘IoT Trust Framework’.

A similar example is the set of IoT-related ‘Baseline Security Recommendations’ published at around the same time by the then EU Agency for Network and Information Security (ENISA)—now EU Agency for Cybersecurity.¹⁹ The recommendations embraced ‘security by design’ as a ‘security good practice’—along with several other such practices, including ‘privacy by design’—and delineated it as seven commands:

GP-PS-01: Consider the security of the whole IoT system from a consistent and holistic approach during its whole lifecycle across all levels of device/application design and development, integrating security throughout the development, manufacture, and deployment.

GP-PS-02: Ensure the ability to integrate different security policies and techniques.

GP-PS-03: Security must consider the risk posed to human safety.

GP-PS-04: Designing for power conservation should not compromise security.

GP-PS-05: Design architecture by compartments to encapsulate elements in case of attacks.

GP-PS-06: For IoT hardware manufacturers and IoT software developers it is necessary to implement test plans to verify whether the product performs as it is expected. Penetration tests help to identify malformed input handling, authentication bypass attempts and overall security posture.

GP-PS-07: For IoT software developers it is important to conduct code review during implementation as it helps to reduce bugs in a final version of a product.²⁰

2.2 Public Policy Ideal in Europe

Prior to the publication of the IoT-related codes cited above, the European Commission elevated SbD to a public policy ideal in its Cyber Security Strategy of the European Union, published in 2013.²¹ As part of this strategy, the Commission invited stakeholders to ‘[s]timulate the development and adoption of industry-led security standards, technical norms and security-by-design and privacy-by-design principles by ICT [information and communication technology] product manufacturers and service providers, including cloud providers’.²²

In the next iteration of its strategy, published in September 2017, the Commission called for ‘[t]he use of “security by design” methods in low-cost, digital, interconnected mass consumer devices which make up the Internet of Things’.²³ This in turn was pitched as a precondition for achieving greater ‘cyber resilience’, also in respect of critical infrastructure and essential services. A few weeks later, the European Parliament issued a resolution along similar lines, urging the Commission and Member States ‘to promote the security by design approach’ and urging ‘industry to include security by design solutions in all [IoT] … devices’.²⁴

None of these policy pronouncements elaborated on what SbD precisely involves or how it is to be legally operationalised, apart from linking it to a general ‘duty of care’ principle²⁵ and to the implementation of the EU Network and Information Systems Security Directive 2016 (NISD).²⁶ Much the same can be said about the Commission’s latest Cybersecurity Strategy, which also flags SbD as a measure for enhancing ‘resilience’.²⁷ Nonetheless, the Commission has implicitly viewed its overhaul of the NISD as (at least) one step in strengthening the mantra’s legal traction. Elements of this reform process are presented in Section 3.5 of this article. The EU Council of Ministers has also treated SbD as an essential element of ‘Europe’s overall cyber resilience’ and implicitly supported efforts to anchor it in ongoing reform of EU legislation, particularly with regard to cybersecurity risks of connected devices.²⁸

Consumer organisations and national governments have played an important role in adding flesh to SbD as a public policy ideal. Six months after the European Commission issued its 2017 Cybersecurity Strategy, ANEC (the European Association for Co-Ordination of Consumer Representation in Standardisation) and BEUC (the European Consumer Organisation) published a position paper calling for comprehensive reform of EU regulatory policy on IoT-connected devices used by consumers, with increased emphasis on requiring ‘security by design and by default’, which they defined as follows:

Security by design means that all connected products and services should better incorporate state of the art cybersecurity functionalities at an early stage of their design process and before the products are put on the market. Security by default means that the settings of a connected device and service are secure as a basic setting (e.g. only high-security measures for authentication such as complex and long passwords should be allowed for ID authentication).²⁹

Shortly afterwards, the UK Government published a report focusing on the need for companies to ‘design products and services with security in mind, from product development through to the entire product lifecycle’ as a pertinent way of meeting the consumer protection challenges posed by IoT.³⁰ The report set forth a ‘soft law’ proposal in the form of a draft Code of Practice for Security in Consumer IoT Products and Associated Services, which the UK Government hoped would suffice to motivate industry to embrace SbD. While noting its preference for a ‘market’ solution, the Government warned that if industry did not conform ‘quickly’ to the code of practice, it would ‘look to make these guidelines compulsory through law’.³¹ The code of practice was formally adopted in October 2018. Despite this initiative, the UK Government decided in April 2021 to act on its threat to enact legislation aimed at ensuring better security for connected consumer products.³² At the time of finalising this article, a legislative bill had yet to be published.

The UK Government’s 2018 report proved influential in shaping cybersecurity policy initiatives in other European countries. For instance, Norway’s ICT Security Committee (‘IKT-sikkerhetsutvalget’), which was appointed to review, inter alia, Norwegian cybersecurity safeguards, recommended in December 2018 that responsibility for the security of connected devices and services should, to a greater extent, be transferred from consumers to the producers and distributors of such devices/services. With reference to the UK Government’s report, it flagged ‘security by design’ (or ‘innebygd sikkerhet’ (built-in security)) as a means of effecting this transfer.³³ However, it did not provide much detail as to what this actually entails.

2.3 The Work of the OECD

This presentation so far might give the impression that SbD, as a public policy ideal, is a recent European invention. However, it is not a recent European invention, nor were EU institutions the first to express SbD-related ideals as transnational public policy. Almost three decades ago, the Organisation for Economic Co-operation and Development (OECD) issued Guidelines for the Security of Information Systems in which these ideals were indirectly flagged, albeit under the banner of an ‘integration principle’ which stipulated that measures for IS security ‘should be co-ordinated and integrated with each other and with other measures … so as to create a coherent system of security’.³⁴ The Explanatory Memorandum to the Guidelines briefly elaborated on this principle by noting that ‘[s]ecurity of information systems is best considered when the system is being designed’.

In the revision of the Guidelines a decade later,³⁵ the ‘integration principle’ was reformulated as ‘security design and implementation’, and delineated as follows:

Systems, networks and policies need to be properly designed, implemented and co-ordinated to optimise security. A major, but not exclusive focus of this effort is the design and adoption of appropriate safeguards and solutions to avoid or limit potential harm from identified threats and vulnerabilities. Both technical and non-technical safeguards and solutions are required and should be proportionate to the value of the information on the organisation’s systems and networks. Security should be a fundamental element of all products, services, systems and networks, and an integral part of system design and architecture. For end users, security design and implementation consists largely of selecting and configuring products and services for their system.³⁶

In effect, this formulation elevated SbD to a soft law principle in its own right, even if the text omitted the ‘by’ of the SbD mantra.

Other OECD instruments have since replaced the 2002 Guidelines,³⁷ and while their thrust is broadly in line with the latter and pays some heed to the importance of design, they significantly reduce the salience of SbD as a principle in itself.³⁸ This reflects a shift in the OECD’s focus from the ‘security of information systems and networks’ to the security risks for economic and social activities relying on such systems and networks. It also marks the OECD’s greater orientation towards encouraging holistic risk management and ensuring that security measures do not constitute a disproportionate interference with those actors affected by them. Unsurprisingly, the OECD’s work is rarely referenced in the more recent European policy initiatives that expressly flag SbD.

2.4 Beyond Europe and the OECD

Besides the OECD, other actors beyond Europe have also pushed public policy ideals similar to SbD. For instance, the US Federal Trade Commission (FTC) adopted in 2015 a set of ten ‘Start with Security’ principles for businesses that parallel the broad thrust of SbD but without expressly mentioning the mantra.³⁹ Arguably, however, support for SbD ideals may be found in FTC policy documents dating from well over a decade ago.⁴⁰

Another example comes from Latin America. In 2019, the Ibero-American Data Protection Network (Red Iberoamericana de Protección de Datos; RIPD)⁴¹ recommended ‘security by design and default’ as part of efforts to institute both ‘privacy by design’ and ‘ethics by design’ when developing artificial intelligence (AI).⁴²

A further example is the United Nations (UN) Committee on the Rights of the Child. According to the Committee, respect for the human rights that children enjoy under the UN Convention on the Rights of the Child requires States to ensure, inter alia, ‘a high standard of cybersecurity, privacy-by-design and safety-by-design in the digital services and products that children use’.⁴³

3.1 EU Legal Norms on Security by Design: An Overview

Having described the emergence of SbD from engineering method to public policy ideal, I continue to its legal status. The numerous references to SbD in the strategy documents outlined above tend to skip over its legal dimensions. Can one say that SbD is more than a publicly lauded engineering method or set of technical standards and also a hard law requirement? In the following, I argue in the affirmative, at least in respect of EU law and certain other jurisdictions.

In EU law, the manifestation of SbD and SbD-related ideals is springing up across multiple legal instruments and at various rungs of the legal hierarchy, although not always in the form of express references to SbD. As shown below, SbD-related norms initially emerged in EU secondary legislation on data protection and then spread to other EU secondary legislation. They have also put down roots in the European Convention for the Protection of Human Rights and Fundamental Freedoms (ECHR) and the Charter of Fundamental Rights of the European Union (CFREU).

EU data protection legislation has long required applying, in effect, an SbD approach to the development and deployment of information systems that process personal data. The GDPR’s predecessor—the 1995 Data Protection Directive (DPD)⁴⁴—expressed such a stipulation both in its preamble and operative provisions.⁴⁵ Under current law, Article 32(1) GDPR requires controllers⁴⁶ and processors⁴⁷ of personal data⁴⁸ to ‘implement appropriate technical and organisational measures’ to ensure that the data are kept secure.⁴⁹ This requirement operationalises the core data protection principles of ‘integrity and confidentiality’ (see Article 5(1)(f) GDPR), set out below) and ‘accountability’ (see Article 5(2) GDPR; see also Article 24(1) GDPR) with which controllers (and, indirectly, processors) must conform. While the requirement eschews express mention of ‘design’, a design element lies implicit in it, as elaborated further in Section 3.3 of this article.

The GDPR’s provisions dealing with ‘data protection by design and by default’ in Article 25 reinforce the design element and make it explicit.⁵⁰ In summary, Article 25 imposes a qualified duty on controllers (and, indirectly, processors)⁵¹ of personal data to ‘implement appropriate technical and organisational measures … which are designed to implement data-protection principles … in an effective manner’ so that the processing of the data will meet the regulation’s requirements and otherwise ensure protection of the data subject’s rights (Article 25(1)).⁵² As made subsequently clear in Section 5.5, the reference to ‘designed’ in Article 25(1) should be construed as denoting not simply ‘intended’ but also engineered and conceptualised. Although diffuse, Article 25(1) undoubtedly embraces a security remit, particularly given that one of the regulation’s core principles is—as noted previously—that of ‘integrity and confidentiality’. The principle states that personal data shall be ‘processed in a manner that ensures appropriate security of the personal data, including protection against unauthorised or unlawful processing and against accidental loss, destruction or damage, using appropriate technical or organisational measures’ (‘integrity and confidentiality’) (Article 5(1)(f)). A security remit is also clearly present in the ‘data protection by default’ requirements of Article 25(2) GDPR.⁵³ To a large extent, those requirements are essentially concerned with keeping personal data ‘lean and locked up’, as it were.⁵⁴

In respect of information systems that do not necessarily process personal data, both the EU Network and Information Systems Security Directive 2016 (NISD) and the EU Cybersecurity Act 2019 (CA)⁵⁵ manifest SbD ideals. The NISD requires EU Member States to ensure that ‘operators of essential services’⁵⁶ and ‘digital service providers’⁵⁷ implement ‘appropriate and proportionate technical and organisational measures to manage the risks posed to the security of network and information systems which they use’ in their operations or services (Articles 14(1) and 16(1)). This is basically the same formulation as used in Article 32 GDPR but applies regardless of whether personal data are being processed. Its SbD dimensions are examined in greater detail in Section 3.5.

The Cybersecurity Act has two main objectives: (i) establishing a long-term organisational framework and mandate for the EU Agency for Cybersecurity (ENISA) and (ii) establishing a pan-European cybersecurity certification framework. Especially pertinent for present purposes is the Act’s explicit endorsement of SbD. In this respect, it initially notes (in recital 2): ‘While an increasing number of devices is connected to the internet, security and resilience are not sufficiently built in by design, leading to insufficient cybersecurity’. As a remedy to this problem, the Act goes on to state (in recital 12):

Organisations, manufacturers or providers involved in the design and development of ICT products, ICT services or ICT processes should be encouraged to implement measures at the earliest stages of design and development to protect the security of those products, services and processes to the highest possible degree, in such a way that the occurrence of cyberattacks is presumed and their impact is anticipated and minimised (‘security-by-design’).

It also describes (in recital 13) a need for ‘security by default’:

Undertakings, organisations and the public sector should configure the ICT products, ICT services or ICT processes designed by them in a way that ensures a higher level of security which should enable the first user to receive a default configuration with the most secure settings possible (‘security by default’).⁵⁸

The Act further charges ENISA with playing a ‘central role’ in ‘promot[ing] security-by-design and privacy-by-design at Union level’ (recital 41). Although these exhortations are only part of the Act’s preamble, they underline important elements of its operative provisions. For example, the European cybersecurity certification scheme established pursuant to the Act ‘shall be designed to achieve’, inter alia, that ‘ICT products, ICT services and ICT processes are secure by default and by design’ (Article 51(i)). The references here to ‘secure by default and by design’ are not defined elsewhere in the Act’s operative provisions, so they must ordinarily be construed in line with their explication in the recitals.

Additionally, multiple EU sectoral regulatory instruments expressly or implicitly flag the importance of SbD. Examples are the Medical Devices Regulation 2017,⁵⁹ the Digital Content Directive 2019 (DCD),⁶⁰ the Financial Markets Directive 2014⁶¹ and the Electronic Identification, Authentication and Trust Services (eIDAS) Regulation 2014.⁶² SbD-related norms are also found in recent legislative initiatives concerning AI,⁶³ revision of the NISD,⁶⁴ revision of the Machinery Directive 2006⁶⁵ and the proposed Data Act.⁶⁶

It is also striking to observe sporadic legislative references to SbD as a ‘principle’. One example is the Electronic Communications Code 2019 (ECC).⁶⁷ This lays down as a core obligation that ‘providers of public electronic communications networks or of publicly available electronic communications services take appropriate and proportionate technical and organisational measures to appropriately manage the risks posed to the security of networks and services’ (Article 40(1); see also recitals 94–95). To this end, recital 97 ECC singles out encryption as an example of such measures and expressly links it to SbD (along with PbD), stating, inter alia, that ‘where necessary, encryption should be mandatory in accordance with the principles of security and privacy by default and by design’ (emphasis added). Another example is Regulation 2021/887 establishing the European Cybersecurity Industrial, Technology and Research Competence Centre and the Network of National Coordination Centres.⁶⁸ This stipulates ‘promoting … the principle of security by design’ as one of the Competence Centre’s objectives (Article 4(2)(b)), while recital 37 refers to the importance ‘that security by design is used as a principle in the process of developing, maintaining, operating and updating infrastructures, products and services …’.

What are the implications of using the word ‘principle’ to denote SbD? Is this terminology merely cosmetic in significance or does it signal a heightened normative status for SbD? I consider these questions in the next section.

3.2 Security by Design as Hard Law Principle?

The exact meaning of a ‘principle’ under EU law varies from context to context and is somewhat diffuse.⁶⁹ In legal philosophy, a distinction is commonly made between ‘principles’ and ‘rules’: the former tend to depict norms with a higher level of generality, abstraction, open-endedness, persistence and explicit value-embodiment than rules, and they often help ground and justify the latter.⁷⁰ Applying Alexy’s oft-cited perspective, for example, a principle is an ‘optimisation command’ or ‘ideal ought’ demanding realisation of a particular goal to the greatest extent possible, whereas rules are more concrete, conclusive ‘definitive commands’ with which compliance can be assessed relatively easily.⁷¹ While this distinction is contested,⁷² it is highly influential, informing much of the general constitutional legal discourse of continental Europe—and, increasingly, equivalent Anglo-American discourse as well.⁷³

Viewing SbD as a principle in the sense of ‘optimisation command’ makes apparent logical sense in light of the way the mantra is formulated in particular legal instruments, such as the Cybersecurity Act and its aforementioned recitals. Yet, as Von Bogdandy observes, one must keep in mind the distinction between what a principle is for the purposes of philosophy (or, indeed, for the purposes of applying technical standards) and what a legal system actually recognises as a principle.⁷⁴ Mindful of this distinction, a legal formalist may rightly query whether the aforementioned references in the ECC and Regulation 2021/887 to SbD as a principle properly elevate SbD into the pantheon of regulatory principles recognised in EU law. The ECC mentions SbD only in its recitals, as opposed to its operative provisions, and recitals are not legally binding in themselves.⁷⁵ The operative provisions of Regulation 2021/887 refer to SbD as a principle, yet the phrasing of recital 37 in the regulation suggests that this reference may simply be intended to denote an engineering approach or method—akin to, say, the ‘end-to-end’ (e2e) principle (explained in Section 4.2) that has steered the design of core internet architecture—as opposed to an overarching (or undergirding) legal norm. While SbD is doubtless a principle in the former sense, this does not necessarily mean it is a principle in the latter sense. Further, there is an almost inflationary use of ‘principles’-based language linked to SbD and related ‘by design’ mantras, with little indication of what the notion of ‘principle’ or that to which it is attached precisely means.⁷⁶ A cautious legal formalist could easily be forgiven for instinctively viewing this development as an essentially rhetorical fad rather than the emergence of a stable norm properly embedded in law.

Even if neither the ECC nor Regulation 2021/887 properly establish SbD as a legally binding ‘optimisation command’, they portend at the very least a development of SbD in that direction. The Commission’s proposal to replace the NISD regime with a new Directive (NIS2D) also refers to SbD as a ‘principle’, and in doing so, uses much the same formulation as the ECC.⁷⁷ The Council’s recently agreed General Approach in regard to that proposal takes the same line.⁷⁸ This indicates that signs are emerging of a legislative consolidation of this development.

In my view, however, strong evidence exists that SbD is already a hard law principle for the purposes of EU law. This is most obvious in respect of security of personal data. Articles 32, 25 and 5 GDPR are the key constituents of this evidence, as the following section shows.

3.3 Security by Design as Hard Law Principle in Respect of Personal Data

Article 32 GDPR requires security measures on the basis of context-specific risk assessment: ‘In assessing the appropriate level of security account shall be taken in particular of the risks that are presented by processing …’ (Article 32(2); see also recital 83 GDPR). Admittedly, Article 32 does not expressly stipulate when risk assessment should take place; in theory, such assessment could simply occur after a data processing system is established. Nor does Article 32 expressly refer to ‘design’. However, Article 32(1) requires that account be taken of the ‘state of the art’. What does this criterion mean? Some language versions of the GDPR give the impression that ‘state of the art’ applies solely to the state of technology or technical measures. For example, the German formulation for ‘state of the art’ is ‘Stand der Technik’, the Danish formulation is ‘det aktuelle tekniske niveau’ and the Norwegian formulation is ‘den tekniske utviklingen’. However, the criterion extends also to organisational measures.⁷⁹ In very general terms, the criterion means that security measures should be in accordance not only with the current state of technological advancement, but also with contemporary practices, standards and other norms for technical and organisational management that are generally regarded as offering the optimal degree of security at the time.⁸⁰ Most importantly (for present purposes), current international technical standards and management system standards indicate that effective risk assessment must be conducted proactively. For instance, the risk management guidelines adopted by the International Organization for Standardization (ISO) state as a general principle that ‘[r]isk management anticipates, detects, acknowledges and responds to … [risks] in an appropriate and timely manner’.⁸¹ A proactive, design-focused approach to risk management is also endorsed by the ISO and International Electrotechnical Commission (IEC) in their standards dealing specifically with information security.⁸² Overall, these standards pitch risk management as concerned with anticipating and planning for reasonably foreseeable events. Thus, risk management ought to be initially conducted at the stage when a system for processing personal data is being conceived, not when a system is already created and operational.⁸³ An ex ante ‘design’ element is implicit in this vision of risk management. Article 32 embodies this element and gives it legally binding force.⁸⁴

At the same time, Article 32 may be regarded as an offshoot of the general DPbDD requirements laid out in Article 25 GDPR, along with the accountability requirements of Articles 5(2) and 24(1) GDPR. Hence, Article 32 should also be read in light of these requirements. Article 32 has basically the same formulation as Article 25, albeit with a narrower compass and a lack of wording expressly addressing when its requirements should apply. That omission is of little substantial consequence given the explicit risk management focus of Article 32 which—as just indicated—demands proactive consideration of security at the outset of establishing a system for processing personal data. Article 25 is more explicit on this point: its requirements apply ‘at the time of the determination of the means for processing’ (Article 25(1)), as well as at the time of the processing itself. Thus, they must first be implemented at the planning stage for processing. According to the European Data Protection Board (EDPB), ‘[t]he “means for processing” range from the general to the detailed design elements of the processing, including the architecture, procedures, protocols, layout and appearance’.⁸⁵ The Board goes on to spell out SbD as one constituent in the consideration of means: ‘Security by design – Consider security requirements as early as possible in the system design and development and continuously integrate and perform relevant tests’.⁸⁶

Turning more directly to SbD’s status as a principle for the processing of personal data, it is doubtful this status is established by Article 32 alone. However, Article 32 should be seen as an elaboration of the principle of ‘integrity and confidentiality’ in Article 5(1)(f) GDPR, which is definitely one of the core principles of the GDPR and other data protection instruments.⁸⁷ The use of the phrase ‘using appropriate technical or organisational measures’ in Article 5(1)(f) not only creates a link to Articles 32, 25 and 24, it also helps usher the design element of the SbD (and DPbDD) mantra into Article 5. This is not to intimate that Articles 5(1)(f) and 32 are fully commensurate. While the latter overlaps with and embodies Article 5(1)(f), only controllers (not controllers and processors) bear primary responsibility for complying with Article 5 principles (Article 5(2) GDPR). Additionally, Article 5(1)(f) lays down a general principle for data processing whereas Article 32 concerns the measures that must be implemented to ensure observance of that principle.

Further, with respect to personal data, SbD constitutes a facet of DPbDD, which must also be considered a principle of the EU legal regime. Admittedly, Article 25 GDPR is not expressly formulated as a principle (or set of principles), but rather refers to the ‘data-protection principles’ elsewhere in the Regulation (primarily those in Article 5). Apart from the implicit connection between Article 5(1)(f) and Articles 25, 32 and 24, Article 5 principles fail to reference DPbDD. However, provisions elsewhere in the GDPR list ‘data protection by design and by default’ as belonging to the ‘general data protection principles’ (Article 47(2)(d)).⁸⁸ Other EU instruments also refer to DPbDD as belonging to the ‘core principles’ of the GDPR.⁸⁹ Accordingly, solid grounds exist for treating SbD as a basic principle of EU data protection law.

3.4 The Role of Courts

Jurisprudence of the Court of Justice of the EU (CJEU) and the European Court of Human Rights (ECtHR) bolsters the legal status of SbD. Both courts have injected SbD-related ideals into Europe’s constitutional framework for fundamental rights protection. The ECtHR judgment in I v Finland (2008)⁹⁰ is seminal in this respect. In this case, the Court held that Finland breached its positive obligation to ensure respect for private life under Article 8 ECHR because of a failure to provide ‘practical and effective protection to exclude any possibility of unauthorised access’ to patient data at a public hospital.⁹¹ Although the judgment did not explicitly reference SbD or closely related ‘by design’ mantras such as DPbDD, its thrust necessitates an approach in line with their ideals. In effect, the judgment renders SbD (and, concomitantly, DPbDD) an essential requirement of a state’s positive obligations to secure respect for the right laid out in Article 8 ECHR and imposes a high threshold for meeting that requirement, at least in relation to ensuring confidentiality of data concerning a person’s health. Good grounds exist, in terms of both lex lata and lex ferenda, for holding that these obligations also extend to other types of personal data and to functionalities other than just maintaining data confidentiality.⁹² In subsequent case law, the ECtHR has specified, in fairly generic terms, ‘procedures for preserving the integrity and confidentiality of data and procedures for its destruction’ as amongst the safeguards required by Article 8(2) ECHR.⁹³

The CJEU is likely to take a similar approach with respect to the obligations flowing from Articles 7 and 8 CFREU, along with Article 16 of the Treaty on the Functioning of the EU. This is because of the so-called homogeneity clause in Article 52(3) CFREU,⁹⁴ combined with judicial confirmation that Article 7 CFREU (which provides for the right to respect for private and family life, home and communications) ‘must therefore be given the same meaning and the same scope as Article 8(1) of the ECHR, as interpreted by the case law of the ECtHR’.⁹⁵ Moreover, in its landmark decision in the Digital Rights Ireland case, the CJEU strongly implied that the ‘essence’ of the right to data protection laid down in Article 8 CFREU requires respect for ‘certain principles of data protection and data security’, meaning that ‘Member States are to ensure that appropriate technical and organisational measures are adopted against accidental or unlawful destruction, accidental loss or alteration of the data’.⁹⁶ Interestingly, the Court’s reference to ‘principles of … data security’ is derived from the language of the former Data Retention Directive (DRD),⁹⁷ which the Court nullified in the case. Article 7 DRD described the data security norms it set out as ‘principles’. Two examples of these principles were: ‘the data shall be subject to appropriate technical and organisational measures to protect the data against accidental or unlawful destruction, accidental loss or alteration, or unauthorised or unlawful storage, processing, access or disclosure’ and ‘the data shall be subject to appropriate technical and organisational measures to ensure that they can be accessed by specially authorised personnel only’ (Articles 7(b) and (c)). The Court found these ‘principles’ to be insufficiently detailed and stringent to satisfy the requirements of Article 8 CFREU in light of the nature and quantity of the data being processed:

[Article 7 DRD] does not lay down rules which are specific and adapted to (i) the vast quantity of data whose retention is required by that directive, (ii) the sensitive nature of that data and (iii) the risk of unlawful access to that data, rules which would serve, in particular, to govern the protection and security of the data in question in a clear and strict manner in order to ensure their full integrity and confidentiality.⁹⁸

Additionally significant is the Court’s stance that the data security norms of the DPD and Electronic Privacy Directive did not weigh up for this failing because they allowed electronic communications providers to take account of ‘economic considerations … as regards the costs of implementing security measures’.⁹⁹ Arnbak has rightly observed that the Court thereby ‘elevated’ security of communications data ‘to a central safeguard under Article 8 EU Charter—a matter of ex ante legislation to ensure security rather than relying on delegation, economic considerations of market operators, nor [sic] ex post incident recovery through other means of law’.¹⁰⁰

Jurisprudence of Germany’s Federal Constitutional Court (Bundesverfassungsgericht) has also helped to heighten the legal status of data security, both in German law and EU law. In 2008, the Court recognised a ‘fundamental right to the guarantee of the confidentiality and integrity of information-technology systems’ (‘Grundrecht auf Gewährleistung der Vertraulichkeit und Integrität informationstechnischer Systeme’) as part of the German Constitution.¹⁰¹ This right is grounded in and a manifestation of the more general right of personality (‘Persönlichkeitsrecht’) provided under Articles 1(1) and 2(1) of the German Constitution, yet it has a distinct security dimension demanding protection of both the confidentiality of an information-technology system and the system’s integrity. The Court defined the latter property in terms of preventing the system from being ‘accessed such that its performance, functions and storage contents can be used by third parties’, meaning, in turn, that ‘the crucial technical hurdle for spying, surveillance or manipulation of the system has then been overcome’.¹⁰²

There is strong evidence that the Court’s judgment also helped lift data security into the collection of core data protection principles set out in Article 5 GDPR.¹⁰³ This evidence inheres not only in the similarity between the Court’s nomenclature for the constitutional right it recognised and the terminology of Article 5(1)(f) GDPR,¹⁰⁴ but also in the regulation’s preparatory works.¹⁰⁵ It is further likely that the judgment informed Digital Rights Ireland, even if the latter made no express reference to it.¹⁰⁶ Note, though, that the Bundesverfassungsgericht did not embrace SbD as a constitutional norm; it simply elevated a form of cybersecurity to such a norm, passing over the status of the design or means for such security. Yet, by heightening the legal importance of cybersecurity, the judgment implicitly heightened the need for measures to protect the legal right at hand, which necessarily implicates design processes. Later jurisprudence of the Court has recognised these implications.¹⁰⁷

3.5 Security by Design as Hard Law Principle with Respect to Non-personal Data

What of SbD’s status under EU law in relation to the security of non-personal data? Can SbD also be properly regarded as a legally binding principle in this context? The answer is less clear cut and varies according to actor and sector, as indicated by the overview presented in Section 3.1. The NISD lays out the least sector-specific rules in point, yet, as noted in the foregoing, these are nonetheless actor-specific, applying only to ‘operators of essential services’ and ‘digital service providers’. The security requirements for the latter category of actor are, as a point of departure, lighter and more flexible than those for the former category.¹⁰⁸ Recital 53 NISD states that digital service providers that are micro- and small enterprises may escape the Directive’s security requirements altogether. Accordingly, Articles 14(1) and 16(1) NISD—which, it will be recalled, contain rules paralleling Article 32 GDPR—do not apply to such enterprises, nor to a large array of other actors, such as manufacturers of electronic equipment or software developers.¹⁰⁹

Furthermore, the Directive does not contain provisions that, in the vein of Article 25 GDPR, make the design element explicit. This stunts the Directive’s ability to promote and act as a proxy for SbD ideals. Recital 51 of the Directive also stipulates that ‘[t]echnical and organisational measures imposed on operators of essential services and digital service providers should not require a particular commercial information and communications technology product to be designed, developed or manufactured in a particular manner’. This could be read as playing down the role of ex ante design. On the basis of the recital, Arnbak went so far as to claim that ‘the concept of security by design is clearly excluded from the regulatory measures’ concerned,¹¹⁰ although he failed to define precisely what he meant by ‘security by design’. Alternatively, I view recital 51 as being intended merely to signal that the requirements of Articles 14(1) and 16(1) NISD are commercially agnostic and thus open-ended in terms of innovation and ‘research and development’ (R&D). Thus, in my opinion, recital 51 does not necessarily exclude SbD.

More generally, it bears emphasis that risk management is key to the Directive’s ethos: ‘A culture of risk management, involving risk assessment and the implementation of security measures appropriate to the risks faced, should be promoted and developed through appropriate regulatory requirements and voluntary industry practices’ (recital 44 NISD; see also recitals 46 and 49). In addition, Articles 14(1) and 16(1) both state that their requirements are to take account of the ‘state of the art’, thus implying that the ISO/IEC standards described in Section 3.3 are also relevant here. The ‘front-foot’ approach endorsed by those standards is reinforced in Articles 14(2) and 16(2) NISD, which focus not just on minimising the impact of security breaches, but on preventing such breaches in the first place.¹¹¹ Overall, while the Directive fails to mention SbD explicitly, it implicitly embraces and enforces its ideals.

These ideals are more salient in the Commission’s proposal to overhaul the NISD regime by replacing it with a new Directive (NIS2D).¹¹² Recital 54 of the NIS2D proposal expressly links the ‘principle’ of SbD to the requirements of Article 18 in the proposed directive. Article 18 (entitled ‘cybersecurity risk management measures’) basically replicates Articles 14(1) and 16(1) NISD, but with some additions. Like the latter, Article 18 of the NIS2D proposal does not expressly mention ‘design’. However, it adds security criteria that strengthen the need for early planning and, hence, design. Particularly pertinent in this regard are express requirements of ‘risk analysis’ (Article 18(2)(a)), ‘supply chain security’ (Article 18(2)(d)), and ‘security in network and information systems acquisition, development and maintenance, including vulnerability handling and disclosure’ (Article 18(2)(e)). As stated earlier, these criteria are arguably not new additions to extant law but simply spell out criteria already inherent in the current Directive. The Council’s General Approach to the NIS2D proposal¹¹³ basically endorses its thrust, albeit with some further additions, such as express recognition of the need for an ‘all-hazards’ approach to security—that is, ‘protection for network and information systems and their physical environment from any event that could compromise the availability, authenticity, integrity or confidentiality of stored, transmitted or processed data or of services offered by, or accessible via, network and information systems’ (Article 18(1a)).

The NIS2D proposal dispenses with the current Directive’s focus on, and distinction between, operators of essential services and digital service providers (see Article 2 and Annexes 1 and 2). Hence, if the proposal is adopted, SbD will gain wider purchase as a legally binding norm with respect to systems that process non-personal data. It will, for instance, apply to manufacturers of electronic equipment (see Annex 2). The traction of SbD will likely be amplified further when and if other related legislative initiatives are adopted as well. One significant initiative in this regard is the Commission’s recent use of its delegated powers under the Radio Equipment Directive 2014 (RED)¹¹⁴ to extend the reach of some of the Directive’s ‘essential requirements’ for lawful placement of radio equipment on the European market, so that these requirements address, in effect, the cybersecurity capabilities of a wide range of IoT devices, including connected toys and wearables.¹¹⁵ Also noteworthy are the proposals for an Artificial Intelligence Act,¹¹⁶ a Cyber Resilience Act covering connected devices,¹¹⁷ a Data Act,¹¹⁸ and a new directive on the resilience of critical entities (CED).¹¹⁹ In addition, as outlined in Section 3.1, a range of other sectoral rule sets already embrace SbD. These are supplemented by the Cybersecurity Act, which posits SbD as a key agenda item both for ENISA’s operations and for the certification scheme the Act establishes, and by Regulation 2021/887,¹²⁰ which places promotion of SbD firmly within the remit of the European Cybersecurity Industrial, Technology and Research Competence Centre.

Looked at holistically, the current regulatory framework for the security of systems that process non-personal data still resembles a patchwork quilt with uneven patterns and thicknesses. While SbD provides a strong visible thread for many patches of the quilt, other patches remain threadbare.¹²¹ This makes it challenging to conclude firmly that SbD may now properly be regarded as a fully fledged, hard law principle with a broad horizontal scope in respect of such systems. However, the appearance of SbD ideals across a variety of disparate sectoral instruments—from rules on medical devices to rules on financial services to rules on AI systems—indicates the emergence of SbD as a trans-sectoral regulatory principle. This development is likely to become more pronounced in the near future.

It is also a welcome development, not least because it will help ameliorate a shortcoming in the scope of SbD application pursuant to data protection law. As noted in Section 3.1, SbD requirements imposed by EU data protection law fall on the shoulders of data controllers and processors. Much the same applies to DPbDD requirements. This is a problem, inasmuch as many fundamental decisions in IS design and development are not made by these actors.¹²² Admittedly, recital 78 GDPR does refer to ‘producers’ of products, services and applications that involve processing of personal data, but these entities (to the extent they are neither controllers nor processors) are merely ‘encouraged’ to respect DPbDD and, implicitly, SbD. The degree to which they are formally obligated to integrate security considerations into their design decisions, also in respect of products, services and applications that involve processing of non-personal data, varies considerably, as this section indicates.¹²³

3.6 Security by Design As Hard Law Norm Beyond Europe

Legislative support for SbD ideals is not limited to the EU. For example, in 2017, Israel adopted landmark regulations pursuant to its data protection law that implicitly mandate an SbD-attuned mindset in respect of databases containing personal data.¹²⁴ The regulations address security measures for personal data in a more elaborate manner than the GDPR does. In line with current risk management standards, they require, inter alia, database controllers and processors to identify and document the kinds of personal data in their databases, assess risks to the security of the data, classify the data according to these risks, formulate written procedures for addressing the risks, put the procedures into practice, conduct follow-up risk assessments periodically and remedy any new vulnerabilities identified.¹²⁵

Another example is legislation enacted in 2018 by California mandating, in effect, SbD for ‘connected devices’.¹²⁶ These are defined as ‘any device, or other physical object that is capable of connecting to the Internet, directly or indirectly, and that is assigned an Internet Protocol address or Bluetooth address’ (§1798.91.05(b)). The legislation has been characterised as ‘aimed mostly at forbidding devices from using generic passwords’.¹²⁷ This is misleading: the legislation encompasses much more than authentication. Its core requirement (set out in §1798.91.04(a)) is as follows:

A manufacturer of a connected device shall equip the device with a reasonable security feature or features that are all of the following:

(1) Appropriate to the nature and function of the device.

(2) Appropriate to the information it may collect, contain, or transmit.

(3) Designed to protect the device and any information contained therein from unauthorized access, destruction, use, modification, or disclosure.

This is supplemented by an authentication requirement (set out in §1798.91.04(b)) that—as Veale and Brown indicate—is essentially aimed at dispensing with the use of generic passwords. Despite significant carve-outs,¹²⁸ the legislation is especially significant given the paramount role of California-based corporations in shaping and developing digital architecture around the world. In effect since 1 January 2020, the legislation has caused confusion and panic over its implications, particularly owing to lack of authoritative guidance on how it is to be construed.¹²⁹

4.1 Intuitive Appeal

The preceding sections show a distinct upswing of interest in pushing SbD specifically, not just as an engineering method and public policy ideal but also as a hard law norm. Why has this upswing of interest occurred? Multiple factors are at play and many of them point to a pressing need for SbD and for greater legal support of its ideals.

The primary reason for the popularity of SbD is that it seems intuitively sensible. Building security mechanisms into IS architecture from the ground up and thereby averting security breaches that could be extremely costly to fix afterwards is a strategy that most people would find difficult to fault. The appeal of this strategy increases in line with the growing societal importance of the systems to which it applies. Accordingly, SbD has greater intuitive appeal with respect to systems that constitute or help constitute ‘critical infrastructure’. At the same time, it is becoming increasingly difficult technically, logically and economically to draw clear-cut distinctions between infrastructure that is ‘critical’ and other infrastructure.¹³⁰ This widens the potential appeal of SbD. The deployment of the IoT is a case in point. Thus, it is not surprising that many technical standards bodies and regulatory authorities first flagged SbD as a remedy for IoT-related security challenges, as described in Section 2.

4.2 Scandal, Economics and Design Constraints

Another factor is the disturbingly large number of scandalous security breaches that have come to light in recent years. Hardly a week goes by without news of a major data ‘hack’, ‘breach’ or ‘leak’. These incidents often evidence lax attitudes towards cybersecurity, also in respect of large collections of sensitive data.¹³¹ Such complacency reflects, in turn, a range of other factors.

Especially important is the paucity of strong economic incentives for companies and other profit-driven bodies to invest up front in cybersecurity.¹³² The penalties—legal or otherwise—that organisations face for failing to implement adequate security measures are often affordable, especially for large wealthy corporations,¹³³ although the degree of affordability will differ from jurisdiction to jurisdiction. Manufacturers of hardware and software frequently view security investment as an unnecessarily expensive drag on product development and deployment; accordingly, they will initially tend not to make the security of their products a high priority in their struggle to gain market traction.¹³⁴ This pattern is exacerbated by the fact that manufacturers often escape having to bear the full economic costs of security breaches arising from flaws in their products.¹³⁵

Further, Asghari and others note that when marketing software, vendors:

will lure customers with bells and whistles that are visible features or provide convenience. Security is rather intangible and does not easily fit into these considerations; it might even reduce functionality.¹³⁶

Insofar as security is a selling feature, it tends to give rise to a classic ‘lemons market’ in Akerlof’s terms.¹³⁷ This market is characterised by informational asymmetry between vendors and consumers, with the latter typically unable to assess the true value of the security feature(s) being marketed and thus unwilling to pay a high price for the more secure product, which in turn discourages vendors from offering the product.¹³⁸ Exacerbating this informational asymmetry is reluctance by many companies to share information about cybersecurity breaches and vulnerabilities they experience or know about.¹³⁹ The interest of particular national security agencies in exploiting such vulnerabilities buttresses and feeds off this problem.¹⁴⁰

Yet another factor contributing to the extensive number of security breaches is that basic internet architecture has been constructed with simplicity, flexibility, openness and resilience predominantly in mind, not with physical or logical fencing. This is less a symptom of the economic factors outlined earlier and more a reflection of the design philosophy of the computer scientists responsible for constructing the original internet—that is, the publicly accessible network based on the transmission control protocol (TCP)/internet protocol (IP) suite. Central to this philosophy is the ‘end-to-end’ principle, which basically posits that the medium for data transmission should be kept simple and focus only on transferring data packets efficiently; other applications (‘intelligence’) should be provided at the network ‘endpoints’. Thus, ‘[t]he network’s job is to transmit datagrams as efficiently and flexibly as possible. Everything else should be done at the fringes’.¹⁴¹ In this context, robust security has been relegated to the category of costly intelligence and left for insertion at the network ‘endpoints’, not in the network ‘pipes’.¹⁴²

More generally, concern for IS security has traditionally struggled to gain strong, widespread traction in the engineering community. This seems to be partly because of the feeling amongst a considerable number of engineers that the integration of security mechanisms into IS architecture is neither their primary responsibility nor pleasurable.¹⁴³ Thus, engineering culture exhibits particular traits that weaken security mechanisms’ grip in IS design. These traits also have consequences for the future traction of SbD ideals. I return to these traits in Section 6.2.

4.3 Regulatory (and Rhetorical) Trends

Finally, the growing popularity of specific regulatory strategies has inspired and moulded the upswing of interest in promoting SbD. Especially significant in this respect is the increasing embracement of a ‘risk-based’ approach to regulation, which involves the integration of risk management measures in legislative frameworks. It champions a pre-emptive, ex ante facto regulatory stance that prompts regulatees to anticipate and mitigate the risk of unwanted events, as opposed to a reactive, ex post facto strategy geared to allocating responsibility and liability after such events occur. The approach creates fertile soil for sowing SbD and related ‘by design’ ideals as a part of risk management measures. As Section 3.3 indicates, the data protection field exemplifies this development well, with the GDPR showing a pronounced concern for both DPbDD and SbD and for ensuring that risks to fundamental rights be properly managed by data controllers and processors prior to, and throughout the lifecycle of, the processing of personal data.¹⁴⁴

Accompanying this development is a burgeoning set of ‘by design’ discourses that have helped spur or shape the increased deployment of risk-based and design-based approaches to regulation. The upswing of popularity of SbD in these emerging regulatory frameworks is linked most closely to the growing salience of the interrelated discourses on PbD and DPbDD. Both of these discourses are aimed at ensuring that proper care is taken of privacy-related interests during the various stages of IS development.¹⁴⁵ In turn, these discourses feed into, and off, broader interdisciplinary endeavours, such as ‘value-sensitive design’ and ‘responsible innovation’, which seek to embed important human values, particularly those central to virtue ethics, in the technology design process.¹⁴⁶ On the legal plane, these endeavours resonate with the vision of ‘Ambient Law’ promoted by Hildebrandt and Koops, whereby legal safeguards for privacy-related interests, human autonomy and rule of law are to be ‘inscribed’ into the ‘socio-technical infrastructure’ during its construction so that it articulates and preserves those safeguards.¹⁴⁷ Additionally, there are calls for ‘by design’ norms emerging in a growing number of other, relatively specific contexts, including enforcement of intellectual property rights,¹⁴⁸ government administration,¹⁴⁹ algorithmic regulation,¹⁵⁰ robotics regulation¹⁵¹ and AI regulation.¹⁵²

Central to the rationale for all of these endeavours and discourses is the recognition that technology plays a key role in setting the parameters for human conduct and that it can often shape our behaviour more effectively than legal norms. Thus, embedding legal norms or ideals in IS architecture and other technology is assumed to improve their traction considerably, in part by helping automate their application. Linked to this assumption is the promise of a reduction in the degree to which technological–organisational developments outpace legislative efforts.¹⁵³

With its relatively technocratic focus, SbD does not fit squarely with all variants of these discourses (especially those concerned with the technological embedment of virtue ethics). Nor, concomitantly, have all of these variants played an equally important role in catalysing the growth and popularity of the SbD mantra. As already indicated, the PbD and DPbDD variants have played the largest role in the latter regard. This is because SbD has frequently been paired with or conceptually baked into them, especially the PbD variant. Examples of pairing occur in some of the aforementioned EU regulatory instruments.¹⁵⁴ As for conceptual merger, this is best exemplified in Cavoukian’s work. One of the seven foundational principles making up her influential conception of PbD is ‘End-to-End Security’, which she describes as follows:

Privacy by Design, having been embedded into the system prior to the first element of information being collected, extends securely throughout the entire lifecycle of the data involved—strong security measures are essential to privacy, from start to finish. This ensures that all data are securely retained, and then securely destroyed at the end of the process, in a timely fashion. Thus, Privacy by Design ensures cradle to grave, secure lifecycle management of information, end-to-end.¹⁵⁵

The conceptual merger of SbD with PbD has continued into the organisational sphere, with, inter alia, Cavoukian’s establishment in 2016 of an International Council on Global Privacy and Security by Design.¹⁵⁶

It must be remembered, though, that privacy and data protection are not fully commensurate with security and cybersecurity. While the latter is a component of a privacy or data protection regime, such a regime also embraces other rules and measures (eg concerning the transparency and fairness of data processing practices). Moreover, cybersecurity on its own may serve a broader range of concerns than data protection and, in particular contexts, these concerns may even conflict with privacy-related interests.¹⁵⁷ It also bears emphasis that Cavoukian’s conception of PbD is not necessarily fully commensurate with EU conceptions of DPbDD. The former reflects North American views and realities that can differ, sometimes subtly, from those in Europe, and it is not tied to particular legislation.¹⁵⁸ In contrast, DPbDD, as manifest in Article 25 GDPR, Article 20 LED and Article 27 EUIDPR, is a legally autonomous construct of EU law, and the measures it requires are tethered to EU legal standards.¹⁵⁹

5.1 Introduction

On the surface, the SbD mantra seems fairly self-explanatory: it denotes a result (‘security’) that is achieved by action(s) (‘design’) undertaken to achieve that result. Further reflection, though, reveals a range of problems that can stymie its smooth operationalisation as a legal principle. Some of these are directly semantic; for example, what precisely is ‘security’? Some concern the assumptions that the mantra suggests, such as the assumption that security is actually possible to achieve by design. The latter set of problems is dealt with in Section 6, while this section deals with semantic problems. Both sets of problems are approached in an exploratory way with a view to parsing general conceptual and methodological issues raised by SbD and with a view to casting light on how SbD may be understood in law (primarily EU law), but without pretending to arrive at clear-cut definitive conclusions on its legal interpretation.

The existence of a sizeable and expanding body of design-focused regulatory discourse, as outlined in Section 4.3, provides an increasingly rich set of reference points for assessing the meaning, mechanics and normative dimensions of SbD. In the following, I draw mainly upon the PbD and DPbDD discourses for this assessment, as they lie closest to the ideals of SbD.

5.2 The Meaning of ‘Security’

The semantics of the term ‘security’ are contested and often contentious. The term has been labelled ‘overloaded’,¹⁶⁰ ‘slippery’¹⁶¹ and ‘perilously capable of meaning all things to all comers’.¹⁶² The chameleon-like character of its semantics largely reflects the fact that security is usually not an end in itself but a proxy for some other interest or value—sometimes termed ‘referent object’¹⁶³—such as personal privacy, human safety, business profitability or state sovereignty. The nature of such interests or referent objects will shape the connotations of the term in a given context and how broadly drawn the term is. The peculiarities of the legal–regulatory culture in which the term is used may also impact the degree of conceptual clarity offered. The evolution of EU regulatory policy on cybersecurity illustrates this dynamic well. As Arnbak notes, this policy has developed without a ‘coherent understanding at the EU level about how to define “security”, and how its underlying values operate, relate or should be interpreted’,¹⁶⁴ a state of affairs that ‘has allowed powerful actors to paint communications security any color [sic] they like’.¹⁶⁵

Pinning down and parsing the term ‘security’ is vital to operationalise the SbD mantra usefully.¹⁶⁶ At a very abstract level, security generally denotes an ‘absence or limitation of vulnerabilities or threats’.¹⁶⁷ However, there is an obvious need to define the term in considerably greater detail to make proper sense of it, particularly as part of the SbD mantra. As the foregoing sections show, SbD is almost exclusively flagged in connection with digital information systems. Accordingly, the legal manifestation of SbD ideals occurs predominantly in law dealing with the security of information systems and the data and information they process. The following consideration of security semantics focuses on that particular context, even though the SbD mantra is sufficiently generic in the abstract to be applied in other contexts.

In relation to data, information and information systems, security is, at the very least, to be understood as safeguarding the confidentiality, integrity and availability of the assets concerned—in other words, the classic ‘CIA’ triad of security properties. A well-nigh universal agreement exists on their central role in this context. At the risk of spelling out the obvious, confidentiality basically indicates that the assets are protected from unauthorised disclosure, integrity basically describes their protection from unauthorised modification and availability denotes that they are accessible and usable on demand by authorised actors, systems or programmes.¹⁶⁸ Closely related properties that may be, and frequently are, added to this mix include resilience (the ability of an IS to deliver intended outcomes despite security breaches),¹⁶⁹ reliability (the property of the IS behaving and producing results consistently and as intended),¹⁷⁰ authenticity (the property that an entity (data, information or actor) is what it claims to be)¹⁷¹ and non-repudiation (‘the ability to prove the occurrence of a claimed event or action and its originating entities’).¹⁷²

The degree to which all of the latter four properties ought to be regarded as core security dimensions is debatable; some are arguably concerned primarily with the dependability and robustness of IS or with holding actors to account rather than keeping assets secure. The same applies to the property of auditability (the ability to trace all actions concerning a given asset), which some influential explications of security criteria emphasise.¹⁷³ Moreover, at least one of the properties—resilience—may be in tension with security in certain contexts. The development of the original internet exemplifies this tension well. As noted in Section 4.2, that network was designed primarily with a view to ensuring its overall robustness and resilience, at the expense of ensuring the security of the individual network nodes. It is not far-fetched to envisage other situations where there is a need, say, to compromise or sacrifice the security of a device to maintain the resilience of a larger IS to which that device is connected. This suggests that resilience is not necessarily a faithfully subordinate constituent of security, especially when applied as a network goal.¹⁷⁴ Nonetheless, resilience, dependability, robustness and security tend to be closely interlinked both in theory and practice,¹⁷⁵ while accountability mechanisms play an important part in ensuring security ‘on the ground’. Thus, it clearly makes sense to treat them as elements in the operationalisation of SbD.

5.3 The Relationship between Security and Safety

Another issue is whether safety can or ought to be readily incorporated into the SbD mantra. The issue is complicated by the fact that many languages other than English fail to distinguish between security and safety. German, for example, uses ‘Sicherheit’ for both concepts, Spanish uses ‘seguridad’, Norwegian ‘sikkerhet’ and Italian ‘sicurezza’.¹⁷⁶ In English, safety commonly refers to protection from threats to personal health or physical integrity arising as a result of natural disasters or benign human error, whereas security is often reserved for protection from malicious human conduct.¹⁷⁷ However, these definitional distinctions vary from one discipline, industry or policy domain to another and are frequently blurred.¹⁷⁸ In some domains, the distinction between safety and security may be based primarily on whether the threat is intentional; in others, it may depend primarily on whether the threat is extraneous to a particular system¹⁷⁹ or on the degree of potential harm involved.¹⁸⁰ Regardless, a well-established line of division persists amongst engineers in the computer industry, between safety experts ‘who see their role as preventing losses due to unintentional actions by benevolent actors’ and security experts ‘who see their role as preventing losses due to intentional actions by malevolent actors’.¹⁸¹ That said, the methodologies employed by each category of expert share much common ground.¹⁸²

The relationship between security and safety in the EU legislative context has mainly become relevant in discussion over the ambit of product safety laws, such as the EU Toy Safety Directive 2009.¹⁸³ These are traditionally regarded as addressing threats to personal health and physical integrity that are accidental or not maliciously induced. Consumer protection advocates view such legislation in its current form as failing to tackle cybersecurity-related vulnerabilities directly.¹⁸⁴ Under this view, an internet-connected or ‘smart’ toy may be deemed safe for the purposes of the Toy Safety Directive even though it suffers from a security flaw with respect to its internet connectivity.¹⁸⁵ To remedy this shortcoming, scholars, consumer groups and other policy entrepreneurs have argued in favour of greater legislative recognition of the fact that cybersecurity is, in fact, a prerequisite for safety, particularly in an IoT context. This ‘security-for-safety’ approach to regulation makes sense when an increasing range of physical devices that fall within the purview of product safety legislation—from heart pacemakers to cars and refrigerators—are becoming ‘connected’ as cyberphysical systems.¹⁸⁶

Yet, the propriety of a ‘security-for-safety’ approach to regulation does not necessarily resolve the issue of whether safety ought to be a necessary element of the SbD mantra. That issue concerns, rather, the propriety of a ‘safety-for-security’ approach. Nonetheless, the latter approach has much to commend as a matter of lex ferenda. In principle, there is no reason to exclude concern for safety from the embrace of SbD. It also makes practical sense given the growing complexity of information systems and the vulnerabilities they face. As Young and Levenson have observed:

Today’s increasingly complex, software-intensive systems … are exhibiting new causes of losses, such as accidents caused by unsafe interactions among components (none of which may have failed), system requirements and design errors, and indirect interactions and systemic factors leading to unidentified common-cause failures of barriers and protection devices. Linear causality models and the tools built upon them, like fault trees, simply lack the power to include these new causes of losses.¹⁸⁷

These developments require the adoption of a relatively holistic, systemic perspective. Omitting safety from security considerations (and vice versa) may result in unfortunate blind spots that undercut the ability to identify and plan for a large, disparate range of vulnerabilities.¹⁸⁸

As highlighted in Section 2.1 of this article, ENISA certainly pitches SbD as embracing safety,¹⁸⁹ yet so far, there has been little recognition of an explicit safety component in legislative elaborations of SbD-related norms. The bulk of EU legislative instruments that flag SbD or SbD-related ideals use the CIA triad to describe the chief properties of ‘security’, albeit with some additions, but without express mention of ‘safety’.¹⁹⁰ Accidental incidents sometimes figure under the rubric of security,¹⁹¹ and some legal instruments fail to differentiate on their face between protection from intentional or malicious threats and other types of threats.¹⁹² Therefore, these instruments may already implicitly cater to safety considerations. Further, EU legislators show increasing awareness of the need for more holistic approaches to security regulation generally. This is exemplified by the Council’s promotion of an ‘all-hazards’ approach to security in the context of the NIS2D proposal,¹⁹³ and the Commission’s proposed extension of European Health and Safety Requirements for machinery products to embrace cybersecurity risks stemming from malicious third party actions.¹⁹⁴

5.4 Legislative Inconsistency in Framing Security

The EU’s legislative elaboration of security properties is far from consistent. For instance, the definition of ‘security of network and information systems’ in Article 4(2) NISD adds ‘authenticity’ to the CIA triad,¹⁹⁵ while Article 32(1)(b) GDPR adds ‘resilience’ to its security mix but omits ‘authenticity’. Neither the NISD nor the NIS2D proposal include ‘resilience’ in their respective elaborations of security, although they do frame security in terms of an ability ‘to resist’. However, ‘resistance’ is not necessarily the same as ‘resilience’: the former may connote the ability to defend against attack or disruption, whereas the latter connotes the ability to maintain or resume operations during or after an attack or disruption.¹⁹⁶ Nonetheless, ‘resilience’ can be defined so broadly as to encompass both resistance and security, which is the case with the Commission’s proposal for a new directive on the resilience of critical entities.¹⁹⁷ This proposal, along with the NIS2D proposal, also highlights inconsistency in how the relationship between resilience and security is pitched. While the CED proposal seemingly treats security as a property of resilience, the GDPR embraces the converse line. Parts of the NIS2D proposal pitch resilience as a means of achieving greater cybersecurity,¹⁹⁸ while other parts describe the proposal as ultimately concerned with enhancing resilience.¹⁹⁹

Even embracement of the CIA triad within a single legislative instrument is sometimes incoherent. This is exemplified in the GDPR, which at one point uses ‘security’ as a supplement to ‘confidentiality’, thereby misleadingly suggesting that the latter is not a component of the former (or vice versa).²⁰⁰ Further, Article 5(1)(f) GDPR—which, as noted in Section 3.3, effectively flags SbD as a core principle for processing personal data—is problematic in terms of both its short-hand nomenclature and substantive provisions. Addressing the latter first, the list of security properties laid out in Article 5(1)(f) ought to have at least included a more explicit reference to the protection of data confidentiality. Furthermore, the inclusion of ‘protection against … unlawful processing’ in that list is out of place for a principle concerned with data security and fits much more squarely within the principle of ‘lawfulness, fairness and transparency’ set out in Article 5(1)(a).²⁰¹ The short-hand nomenclature fails to flag ‘availability’, despite the list of referents in Article 5(1)(f) including protection against ‘accidental loss, destruction or damage’ of personal data.

While some of the terminological variations may be justifiable in light of the differing policy objectives of the instruments concerned—particularly in respect of ‘resilience’—finding a rational ground for all of the variations is challenging, to put it mildly. Overall, one is left with an impression of muddle that bespeaks a continuation of the aforementioned conceptual problems that Arnbak observed with EU regulatory policy on communications security prior to 2016.²⁰² It also muddies the message behind SbD ideals.

5.5 The Meaning of ‘by Design’

Somewhat akin to ‘security’, the concept of ‘design’ is afflicted by slippery semantics. Reflecting on the history of its usage in general, Bruno Latour has observed that ‘design’ has both an expansive and expanding meaning.²⁰³ The capacity for ‘design’ to be defined broadly is exemplified by parts of the PbD discourse. For instance, in his engaging book, Privacy’s Blueprint, Woody Hartzog first defines ‘design’ as ‘processes that create consumer technologies and the results of their creative processes instantiated in hardware and software’,²⁰⁴ then as ‘how a system is architected, how it functions, how it communicates, and how that architecture, function, and communication affects people’²⁰⁵ and further, as ‘the creation of tools to understand and act on current conditions’.²⁰⁶ Here, Hartzog uses the term to refer to processes, methods and the results of such processes and methods.

Many other contributions to ‘by design’ discourses have tended to focus more on the goals or objects of design than the meaning of ‘design’ as such, leaving the term relatively nebulous. This is the case, for example, with PbD and DPbDD discourses.²⁰⁷ Somewhat surprisingly, specialist scholarship on IS design has also struggled to reach agreement on what such design entails: ‘no single, all-encompassing definition of either IS or design in IS can be established’.²⁰⁸ This is despite a ‘rapid growth in interest in the notion of design—and hence in the building of a design science—in IS’.²⁰⁹

There is, then, a risk of ‘design’—and, concomitantly, ‘by design’—suffering a similar fate to the term ‘governance’, which is criticised for having become ‘a rather fuzzy term that can be applied to almost everything and therefore describes and explains nothing’.²¹⁰ This risk notwithstanding, it is possible to build out a reasonably cogent conceptualisation of ‘design’ such that the term is lifted well above being a semantic nullity and is able to facilitate operationalisation of the SbD mantra (together with similar mantras such as PbD and DPbDD).

To begin with, there is little doubt that ‘design’ generally connotes, at the very least, intentional, directed activity—that is, actions consciously set in motion to meet some articulated or semi-articulated goal. This is particularly so when the term is used in a ‘by design’ mantra. Thus, in the context of SbD, we are talking about intentional security, not incidental or accidental security. SbD may accordingly be contrasted with ‘security by chance’, ‘security by accident’ and ‘security by disaster’.

There is also little doubt that design is generally more than mere intention. This is highlighted by the aforementioned work of Hartzog, along with many other design-oriented scholars. In this respect, Ralph and Wand’s oft-cited explication of ‘design’ is particularly germane. They first provide a definition of the term in its noun form. According to this definition, design is:

a specification of an object,²¹¹ manifested by some agent, intended to accomplish goals, in a particular environment, using a set of primitive components,²¹² satisfying a set of requirements, subject to some constraints.²¹³

Second, as a transitive verb, ‘design’ is ‘to create a design, in an environment (where the designer operates)’.²¹⁴ This stripping of ‘design’ to its very basic elements is conceptually useful and makes clear that it goes further than simply intention. It also involves engineering in the generic sense of working to bring something about, and conceptualisation. The engineering and conceptualisation dimensions are especially evident in the phrase ‘by design’.

At the same time, there is little doubt that design is ordinarily not the same as manufacturing, building or constructing, even though these activities are both shaped by, and a manifestation of, design. Yet, delineating precisely where design begins and ends in relation to such activities (or, indeed, other processes) may be challenging. The same goes when attempting to delineate the way(s) in which design happens. Ralph and Wand are fully cognizant of these challenges.

Because design is an activity, rather than a phase of some process, it may not have a discernable [sic] end point. Rather, it begins when the design agent begins specifying the properties of the object, and stops when the agent stops. Design may begin again if an agent (perhaps a user) changes structural properties of the specification or design object at a later time.

Our definition does not specify the process by which design occurs. Thus, how one interprets this scope of activities in the design process depends on the situation. If a designer encounters a problem and immediately begins forming ideas about a design object to solve the problem, design has begun with problem identification. If requirements are gathered in reaction to the design activity, design includes requirements gathering. In contrast, if a designer is given a full set of requirements upfront, or gathers requirements before conceptualizing a design object, requirements gathering is not part of design.²¹⁵

They accordingly go on to note that ‘design practice may not map cleanly or reliably into [sic] the phases of a particular process’, and that this can create problems for software development because computer scientists differ in their views as to what stages of software engineering are ‘design’ stages.²¹⁶ Some computer scientists pin software ‘design’ to a discrete and narrowly defined step. For instance, in the traditional ‘waterfall’ model of software development, design typically denotes a process subsequent to requirements specification and prior to programming.²¹⁷ Others, however, adopt a radically more expansive view of software design. Freeman and Hart, for example, state:

Design encompasses all the activities involved in conceptualizing, framing, implementing, commissioning, and ultimately modifying complex systems—not just the activity following requirements specification and before programming, as it might be translated from a stylized software engineering process.²¹⁸

Design from this kind of perspective is a lengthy, far-ranging, iterative process.²¹⁹

Defining design becomes even more multiplex when one considers who is involved in design activity. Traditionally, a distinction is drawn between designers of technology and users of technology—a view typical for actor–network theory.²²⁰ In reality, users are inevitably involved in design activity, even if that involvement is not formally recognised in a particular design process. As Waldman observes, ‘design is not complete until users have defined the uses and social valence of the technology in their hands’.²²¹ Further, Hartzog notes that ‘[g]ood design cannot happen without the participation and respect of all stakeholders, including engineers, artists, executives, users, and lawyers’.²²² These insights constitute a key point of departure for scholarship and practice on ‘participatory design’, which is aimed at encouraging and formalising user involvement in design processes.²²³

In a similar vein—and moving closer to how design processes may operate with regard to cybersecurity—Brass and Sowell promote a vision of ‘adaptive governance’ for managing IoT security risks.²²⁴ This involves users of IoT providing iterative feedback to IoT producers, regulatory authorities and other relevant stakeholders based on their expectations and experiences with these products. More generally, Brass and Sowell state:

Changes in the framing and scope of new regulatory interventions need to increasingly recognize that the expertise necessary to characterize emerging sociotechnical risks is rooted not only in those who design or regulate these technologies, but in the day-to-day operational expertise developed among those who deploy, operate, and manage them on the ground.²²⁵

It is otherwise important to note that standards bodies and other experts in security engineering stress the importance of cybersecurity-related design processes being concerned not only with software programming and other technical measures but with a large variety of safeguards that ultimately reflect a particular organisational (and not just technical) framework. In this regard, the seminal report of the US Defense Science Board Task Force on Computer Security referenced near the start of this article, presciently noted over fifty years ago that ‘software safeguards alone are not sufficient’ to deliver ‘comprehensive security’; rather, a ‘combination of hardware, software, communications, physical, personnel and administrative-procedural safeguards is required’.²²⁶ Similarly, the ISO and IEC emphasise that ‘the information security that can be achieved through technical means is limited, and can be ineffective without being supported by appropriate management and procedures within the context of an ISMS [Information Security Management System]’).²²⁷

5.6 Design Semantics in Law

How do the aforementioned facets of design semantics play out in the legal dimension, particularly with respect to the legislative instruments that incorporate the SbD mantra? In the EU legal context, an important preliminary point is that care must be taken not to uncritically adopt an interpretation of the term ‘design’ that is rigidly locked to its connotations in the English language. Looking across the various language versions of EU legislative instruments that expressly flag SbD, a variety of terms being used as the equivalent of design can be found, and their meanings do not always map closely with the English term. For example, the term used to denote SbD in the Danish version of recital 12 CA connotes embedment or integration (‘indbygget sikkerhed’), the French version connotes conception (‘sécurité dès le stade de la conception’), while the German version connotes both integration and conception (‘konzeptionsintegrierte Sicherheit’). Furthermore, the Danish and German versions seem to highlight an overall result, whereas the French version seems to highlight a stage or step occurring at the start of a process. Nevertheless, all of these variations point more or less to a process of deliberate, up-front planning. The legislative context in which they appear provides enough common ground across the language versions to neutralise the sorts of linguistic discrepancies just highlighted. The legally operational meaning of the terms ‘design’ and ‘by design’ is provided not so much by the terms in themselves as by the surrounding legislative text. The latter provides, in effect, the parameters outlined above by Ralph and Wand—that is, the specification(s) of objects, agents, components, requirements and constraints for the processes concerned. This is especially the case with legal instruments that do not expressly reference SbD or otherwise utilise the term ‘design’, but nevertheless embody the thrust of the SbD mantra.

A second important preliminary point is that there is a paucity of case law on the meaning and reach of legislative provisions that expressly or implicitly manifest SbD ideals. While technical standards bodies and regulatory authorities have issued a considerable amount of guidance and administrative orders, these do not have the same legally definitive status as judicial decisions. However, a case pending before the CJEU is likely to cast authoritative light on how Article 32 GDPR operationalises SbD ideals.²²⁸ This case may also cast light on the reach of similar provisions in other EU legislation. In the meantime, the parsing of the legal dimensions of the ‘by design’ element of the SbD mantra should proceed with caution.

The analysis in Section 5.5 indicates that SbD ought not to be neatly boxed in, conceptually or chronologically. Schneier has famously remarked that ‘security is not a product, but a process’.²²⁹ Similarly, SbD ought to be regarded as a recurrent, evolutive and relatively open-ended process rather than a one-off step. EU legislation provides fairly clear manifestation of this view. For example, the Cybersecurity Act states that ‘[s]ecurity should be ensured throughout the lifetime of the ICT product, ICT service or ICT process by design and development processes that constantly evolve to reduce the risk of harm from malicious exploitation’ (recital 12; emphasis added). Similarly, the GDPR hints at the ongoing nature of SbD when it requires—as one of the ‘technical and organisational measures’ for security under Article 32—a ‘process for regularly testing, assessing and evaluating the effectiveness’ of these measures (Article 32(1)(d)). Further, the references to ‘state of the art’ in the incipits of Articles 32(1) and 25(1) GDPR necessitate regular revisiting of security practices in light of changing perceptions as to which technical and organisational standards provide optimal security.²³⁰ The same can be said of, inter alia, Articles 14(1) and 16(1) NISD, Article 40(1) ECC and Annex I of the Medical Devices Regulation, which also use ‘state of the art’ as a touchstone. In contrast, the Californian legislation on ‘security of connected devices’ presented in Section 3.6 (hereinafter ‘Californian legislation’) is devoid of this dynamic approach, at least on its face.

Another point that can be safely drawn from the analysis in Section 5.5 is that SbD ought to encompass a range of design measures that include, yet go beyond, the strictly technical. In EU legislation, SbD is, on the whole, clearly pitched as embracing both ‘technical and organisational measures’.²³¹ OECD regulatory policy takes the same approach.²³² Exactly what measures fall within this bipartite category is not entirely clear. This is hardly surprising; it would be unrealistic and undesirable to attempt an exhaustive, let alone precise specification of measures in legislation.²³³ This is especially true when legislation is given a broad field of coverage. Even if legislation were to be narrowly scoped, other challenges remain, such as ensuring that a precisely delineated list of measures is sufficiently ‘future-proofed’. Hence, the legislative specification of measures tends (and ought) to be relatively generic in form. That said, it is unfortunate that some such specifications are not formulated as measures in a strict sense—that is, as intentional actions—but rather capabilities. This is the case, for instance, with two of the exemplifications of ‘measures’ listed in Article 32(1) GDPR: ‘the ability to ensure the ongoing confidentiality, integrity, availability and resilience of processing systems and services’ (Article 32(1)(b)) and ‘the ability to restore the availability and access to personal data in a timely manner in the event of a physical or technical incident’ (Article 32(1)(c)).²³⁴

The phrase ‘technical and organisational measures’ in EU law undoubtedly signals an intention to cover a potentially large variety of measures and to be applied flexibly. In other words, ‘technical and organisational’ is to be construed liberally—a fortiori when construed in the context of furthering protection of a fundamental right, such as privacy or data protection.²³⁵ Generally speaking, technical measures directly concern, and are often executed in, the mechanics or workings of devices, objects, systems or processes. While frequently embodied in software, hardware and other artefacts, technical measures may go beyond them. They thereby encompass more than what is often commonly understood as ‘technology’,²³⁶ although the latter term can also be defined expansively as any processes and methods for enhancing humans’ capacity to execute tasks,²³⁷ thus making ‘technical’ and ‘technology’ roughly commensurate with each other. Given the broad, generic sense of ‘technical’ just laid out, organisational measures will inevitably have a technical dimension in their operationalisation. This suggests that ‘technical’ and ‘organisational’ should not be seen as two entirely different and separate criteria. The chief distinction between them is that organisational measures directly concern the environment for the development and deployment of technical measures. More specifically, organisational measures primarily involve the assignment and management of roles, duties or tasks in connection with such development or deployment, typically within the aegis of a collective entity (corporation, state agency, etc). Security incident response management, staff training programmes on security procedures, and managerial decisions over who has responsibility for various security-related initiatives are obvious examples.²³⁸

Any measure that appreciably helps promote security is likely to be regarded as capable of being embraced by the phrase ‘technical and organisational’, even if it fits awkwardly within the ordinary literal meaning of the phrase. This would be the case, for instance, with certain economic measures (eg budgetary allocations for security spending) or particular legal measures (eg drafting terms and conditions for the use of cloud computing services) inasmuch as these have a fairly direct bearing on the security of information systems and their contents.²³⁹ The requirements of DPbDD pursuant to Article 25 GDPR operate similarly, albeit across a wider range of data protection mechanisms.²⁴⁰ However, as measures’ relevance for ensuring security decreases, so too does the likelihood that they are required. Precisely which measures apply will depend largely on context-calibrated methodologies constituting the ‘state of the art’, particularly as elaborated in soft law standards drawn up by security experts.²⁴¹ This may also be the case under legislative frameworks that are not directly linked to the protection of fundamental rights.²⁴² In line with such standards, the measures should typically form, and be applied as, constituents of an overarching ISMS that governs the entirety of the IS lifecycle.²⁴³ And they may variously apply before, during and after a security breach.²⁴⁴

Unfortunately, not all SbD-related rules take such an expansive approach. The design requirements of the Californian IoT security legislation appear to be pitched predominantly, if not exclusively, at technical measures. The same seems to apply with the provisions of the EU Cybersecurity Act on ‘security by default’ (see recital 13, set out in Section 3.1). However, this shortcoming in respect to the latter provisions is not as problematic as it is for the Californian legislation, because the ‘by default’ requirements are coupled to, and a by-product of, the more expansive ‘by design’ requirements elaborated in recital 12 of the Cybersecurity Act.

5.7 Ambition

An important legal issue concerns the degree of ambition that law demands of the design process. This issue can be broken down into two main questions: What level of security is required and how much effort is necessitated? The answers will partly depend on the normative status of the referent object at stake: if that object is a fundamental right or closely linked to such a right, the requisite level and effort will be relatively high. Much the same will pertain if cybersecurity becomes a fundamental right in itself.²⁴⁵

Nonetheless, it is generally assumed that complete security for information systems is well-nigh unattainable.²⁴⁶ Indeed, this is part of what makes achieving security a ‘wicked problem’. Legal references to security must therefore typically denote something less than complete security. With respect to Article 32 GDPR, for example, there is broad agreement that the provision constitutes an obligation of means, rather than result, in the sense that a security breach as such will not necessarily amount to an infringement of the legislation; rather, an infringement will only arise if the controller or processor has failed to take the measures required by Article 32.²⁴⁷ A similar standard most likely applies in respect of other equivalent requirements, such as Articles 14 and 16 NISD.²⁴⁸

In the context of Article 25 GDPR, the European Data Protection Board duly opines that ‘[e]ffectiveness is at the heart of the concept of data protection by design’.²⁴⁹ Accordingly, the Board adds that the paramount objective of DPbDD is ‘the effective implementation of the principles and protection of the rights of data subjects into the appropriate measures of the processing’.²⁵⁰ The same may be said for Article 32 GDPR with respect to security measures. Article 32 is not about symbolic measures; rather, it is part of a concerted effort to ensure that ‘law in books’ becomes ‘law in practice’. In other words, Article 32 GDPR should be understood as going well beyond a soft paternalism that simply encourages thought to be given to IS security without requiring a practicably significant enhancement of security. This point is buttressed by the jurisprudence of the ECtHR in I v Finland and of the CJEU in Digital Rights Ireland presented in Section 3.4.

At the same time, the requisite security effort and levels are tempered by contextual factors. The incipit of Article 32(1) GDPR is a central example:

Taking into account the state of the art, the costs of implementation and the nature, scope, context and purposes of processing as well as the risk of varying likelihood and severity for the rights and freedoms of natural persons, the controller and the processor shall implement appropriate technical and organisational measures to ensure a level of security appropriate to the risk, …²⁵¹

As Burton notes, this phrasing imports a criterion of proportionality, which requires active consideration of whether the security measures are reasonably likely to achieve their aims and whether any detrimental effects they incur on competing legitimate interests are justified.²⁵² Hence, the security standard required by SbD as a legal norm must be, in effect, one that is the result of best reasonable effort.

This is not obvious from the face of all the legislative instruments canvassed herein. For example, the above-cited recitals in the preamble to the EU Cybersecurity Act are, prima facie, relatively stringent and seem to operate with a ‘best effort’ standard. Recital 12 CA refers to protecting the security of ICT products, ICT services or ICT processes ‘to the highest possible degree, in such a way that the occurrence of cyberattacks is presumed and their impact is anticipated and minimised’. The same seems to apply in respect of the ‘security by default’ dimension which refers to the ‘most secure settings possible’ (recital 13 CA). However, the references in the CA recitals to what is ‘possible’ should be read in the light of what is reasonably possible. This follows from the provisions of the GDPR and other central EU laws that operationalise SbD and point to a broad range of factors for assessing appropriateness/reasonableness. The Cybersecurity Act must be construed in light of these provisions. The Californian legislation also seems to operate with a reasonableness criterion inasmuch as it expressly refers to a ‘reasonable’ security feature (§1798.91.04(a)).

The landmark judgment of the ECtHR in I v Finland presented in Section 3.4 deserves mention in this context. It will be recalled that the Court formulated a state’s obligation to provide security such as to ‘exclude any possibility of unauthorised access’ to hospital patient data.²⁵³ On its face, this is rather absolutist phrasing. However, the Court’s phrasing should be read in light of, first, the relevant Finnish law at the time which required security measures to be both ‘appropriate’ and de facto,²⁵⁴ and, second, the Court’s finding ‘that the records system in place in the hospital was clearly not in accordance with’ those legal requirements²⁵⁵––a finding that the Court described as ‘decisive’ for its judgment.²⁵⁶ In light of these factors, the above-cited formulation regarding the exclusion of ‘any possibility of unauthorised access’ seems unnecessarily forceful and misleading, also given that the absent security measure that was centrally germane to the litigation would not have prevented unauthorised access but merely discouraged it.²⁵⁷ Thus, the judgment should not be read as laying down cybersecurity requirements more stringent than those aimed at ensuring a level of security that is the result of best reasonable effort.

5.8 Sanctions

It is not the intention of this article to delve deeply into the issue of sanctions but some brief remarks are pertinent. Notwithstanding a paucity of comprehensive mapping of the actual behavioural effects of legal sanctions for security breaches, it is reasonable to assume that the bite of SbD is partially dependent on the sanctions that may be incurred in the event of non-compliance with its legal requirements. A hypothetical yet realistic example from the work of Michels and Walden underscores this assumption: ‘a well-intentioned but underfunded IT department could use the risk of sanctions to convince a disinterested, cost-focussed board to give it the budget it needs for additional security measures’.²⁵⁸

Sanctions for breach of SbD requirements may flow from a variety of legal frameworks. These extend beyond public law to private law, which may provide remedies under tort or contract. Despite this variety, legal sanctions have often constituted a weak point in cybersecurity regimes, as indicated in Section 4.2. The overarching problem in this regard has been the apparent failure of legal sanctions to incentivise organisations to invest up-front in strong security measures, in a situation where market incentives to do so are generally weak. Regulators’ reluctance to impose appreciable financial penalties for security breaches is part of this problem. Lack of consistency and transparency in the imposition of sanctions is another part. The NISD regime is a case in point. It simply requires penalties to be ‘effective, proportionate and dissuasive’ (Article 21 NISD)—a general principle of EU constitutional law—but otherwise gives Member States considerable leeway to implement sanctions as they see fit. At the same time, the Commission recently observed that ‘Member States have been very reluctant to apply penalties to entities failing to put in place security requirements or report incidents’.²⁵⁹

The equivalent provisions in the GDPR are much more comprehensive than Article 21 NISD and arguably constitute the most ambitious sanctions regime in the cybersecurity domain, particularly with regard to the imposition of administrative fines. Such penalties are to be calibrated according to a long list of context-dependant criteria listed in Article 83(2) GDPR, which can be seen as constituents of the overarching requirement that sanctions should be ‘effective, proportionate and dissuasive’ (Article 83(1) GDPR). One criterion is ‘the degree of responsibility of the controller or processor taking into account technical and organisational measures implemented by them pursuant to Articles 25 and 32’ (Article 83(2)(d) GDPR). This helps to operationalise, in effect, SbD (and DPbDD) ideals since appreciable efforts to implement security measures may be potentially rewarded by reductions of the amounts payable as fines.

An intriguing, albeit confusing, feature of the GDPR sanctions regime is that it operates with a concurrently applicable two-tiered system for the maximum amount payable in the case of breaches of the regulation’s security requirements. The GDPR states that a breach of Article 32 will attract a fine of up to 10 million EUR or, in the case of an undertaking, up to two percent of its total worldwide annual turnover, whichever is higher (Article 83(4)). These amounts are doubled when there is a breach of ‘the basic principles for processing’ under Article 5 (Article 83(5) GDPR). As pointed out in previous sections, one such principle is ‘integrity and confidentiality’—ie ensuring ‘appropriate security of the personal data’ (Article 5(1)(f) GDPR). Some commentators misleadingly point only to Article 83(4) with respect to breaches of security requirements, seemingly forgetting the possibility of applying the higher tier.²⁶⁰ It has also been argued that Article 32 constitutes the lex specialis of Article 5(1)(f) and should therefore take precedence when setting the level of fines for security-related infringements; in other words, the tier for such infringements should be governed by Article 83(4), not Article 83(5).²⁶¹ The argument has dubious merit for several reasons.²⁶² First, while Article 32 overlaps with and embodies Article 5(1)(f), the two are not fully commensurate, as noted in Section 3.3. Second, the regulation clearly stipulates that breach of Article 5 principles attracts the higher tier for levelling fines. There is no indication that the principle of ‘integrity and confidentiality’ is intended to be exempt from this rule, particularly in light of the elevated normative status of security of personal data in the EU constitutional framework. Third, the potential application of both tiers to security infringements does not necessarily create practical problems: the higher level of fine may (and ought to) be applied in cases of egregious or especially negligent behaviour on the part of controllers; the lower tier being reserved for other cases.

Unfortunately, it is extremely difficult to discern a consistent or clear approach from DPAs to how this two-tier system should be operationalised. A large number of fines have been issued for infringements of the GDPR’s security requirements, and the fine levels naturally vary a great deal.²⁶³ In many cases, DPAs have held that both Articles 5(1)(f) and 32 have been breached—which makes legal sense—but the weighting of each type of infringement in the calculation of the level of fine has often been left unelaborated.²⁶⁴ In one case, the DPA even incorrectly described the two-tier system by stating that Article 83(4) applies to infringements of Article 5.²⁶⁵ In some cases, breach of Article 32 has been found, without express consideration of possible breach of Article 5(1)(f), even though personal data have likely been insecurely processed.²⁶⁶ In some other cases, relatively paltry fines have been issued for Article 5(1)(f) infringements, without any mention of possible breaches of Article 32.²⁶⁷ Viewed as a whole, the pan-European operationalisation of the administrative fines scheme with regard to security-related breaches of the GDPR is far from satisfactory, not least in a ‘rule of law’ perspective. Unfortunately, the same can be said of the scheme’s operationalisation with regard to GDPR breaches generally.²⁶⁸ A key problem is that the EDPB has not agreed on a detailed methodology for calculating fines, although this is on its current agenda.²⁶⁹ While the Board’s predecessor has fleshed out the chief criteria for determining fine levels, the result does not go much further than the list provided in Article 83(2).²⁷⁰ This allows for divergent and often arcane decisional processes.

5.9 Users

The analysis in Section 5.5 points to users of information systems or other technology inevitably playing a role, directly or indirectly, in design processes. Accordingly, a conception of SbD without account taken of this role underplays the multidimensionality of design. The analysis in Section 5.5 also points to user involvement in design processes as being desirable. Disappointingly, the legal instruments canvassed herein generally demonstrate scant recognition of the desirability of user involvement in SbD and even the very fact of such involvement. This is not to say that user involvement is necessarily absent from the legal framework. It may figure, for instance, in the reference to ‘nature, scope, context and purposes of [data] processing’ in Articles 24(1), 25(1) and 32(1) GDPR, along with the follow-up reference to the ‘risks of varying likelihood and severity for the rights and freedoms of natural persons’ in the same provisions. Yet, such flagging of user involvement is oblique, and the user seems to be cast primarily as a potential victim rather than as a valuable source of input into IS design.²⁷¹

6.1 Existential Difficulties

The SbD mantra assumes that a meaningful level of security can be achieved by design. Does this assumption promise more than can actually be delivered? In a fundamental existential sense, it arguably does, at least if one considers insecurity to lie at the root of the human condition. From such a perspective, SbD could be characterised as overly optimistic; the struggle for security is really a perpetual Sisyphean task of moving a rock between various degrees of vulnerability. A similar criticism could be directed at PbD and DPbDD given the extreme erosion in recent years of the basic societal bedrock supporting privacy.

This sort of perspective is an important point of departure for many proponents of ‘cyber resilience’ as an overarching goal for IS development. For these people, insecurity is a fundamental, inescapable element of life; it is the rule, not the exception.

[T]he concept of resilience essentially treats adverse cyber events as a part of normal operations. The difference to the concept of security can therefore be crucial—it allows organizations to incorporate counter measures and contingency plans as a part of what could be considered as this new ‘normal’ condition.²⁷²

In this view, cyber resilience is a more realistic approach to coping with threat than cybersecurity. A corollary of this view would be that ‘resilience by design’ should receive greater prominence than SbD in the legislative limelight. It is beyond the scope of this article to delve into the strengths and weaknesses of this view—that is a topic for another article.²⁷³ Drawing on the latter, it suffices to note here that cyber resilience is an elusive concept whose goals are not necessarily significantly simpler to realise than those of cybersecurity. Björck and others, for example, pitch cyber resilience as ‘the ability to continuously deliver the intended outcome despite adverse cyber events’.²⁷⁴ They also pitch it as being concerned with designing information systems ‘to fail in a controlled way’.²⁷⁵ These are extremely ambitious goals. Their realism in many contexts is, at the very least, questionable.

To people who are unfamiliar with cybersecurity-related challenges, SbD in the abstract may well promise more security than can actually be delivered. Experts in security engineering and IS design are less likely to be deceived. Many of them appreciate the ‘wicked’ nature of the basic problem that the SbD mantra is intended to address. They also understand that the promised security is neither absolute nor permanent but rather a hard-won, contingent, transitory and imperfect result.²⁷⁶ Several of the items of legislation canvassed in the previous sections suggest that law makers in some jurisdictions share this understanding.

6.2 Governance Challenges

Existential difficulties aside, other problematic assumptions inhere in the notion of SbD and much of the broader ‘by Design’ (bD) discourse. That discourse—at least in regulatory contexts—tends to operate within a positivist paradigm focusing on ‘artefacts’ that are of a predominantly technical nature and that are separate to, or distinct from, yet controllable by, the humans that create them. A similar characterisation has been made of the field of design science.²⁷⁷ Much of the bD discourse also assumes that information systems can be built and governed along predictable paths. However, such systems often end up being used in ways beyond what designers are able to predict or model.²⁷⁸ Amara’s law is apposite here: ‘We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run’.²⁷⁹

In reality, information systems are rarely complete, monolithic and static with clearly delineated borders; rather, they are frequently amorphous, incomplete and evolving organisational–technological structures with unclear lines of control. The internet is a prime example of this; hence, its governance has given rise to myriad challenges and considerable conflict.²⁸⁰ Moreover, perceptions of the condition, parameters or direction of a given IS may differ. How those formally tasked with the design of a system conceptualise its functionalities and aims can often be at odds with how users conceptualise them and with how the system ends up being used.²⁸¹ This underlines the need to ensure user involvement in IS design.

Account must also be taken of software programming culture. In recent years, ‘agile’ or ‘evolutionary’ software development has become popular. This represents a movement from the relatively regimented ‘waterfall’ style of developing software—with neat, distinct, chronologically ordered and unidirectional stages of development accompanied by comprehensive documentation—to a more fluid development style characterised by a ‘no design up front attitude’.²⁸² The ‘agile turn’ goes hand in hand with a ‘minimum viable product’ approach that focuses on delivering a product that is just good enough to work but nothing more. It also goes hand in hand with a turn to modularity, whereby one constructs ‘a complex product or process from smaller subsystems that can be designed independently yet function together as a whole’.²⁸³ All of these trends foster the atomisation of information systems and the design processes directed at their creation. This introduces further difficulties in realising ‘by design’ ideals that assume an ability to shape IS development along desired linear paths.

I am not suggesting that the turns to agility and modularity bring the SbD enterprise to its knees. That enterprise involves more than software programming, and security needs can be catered for to some degree within an evolutionary, modular framework. Moreover, care must be taken to ensure that software programmers understand that SbD mandates are not hemmed in or defined by the ‘waterfall’ model, which as noted in Section 5.5, tends to tether design to a particular stage of software development. Nonetheless, the waterfall model has clear benefits for the SbD enterprise:

The strengths of the waterfall model are that it compels early clarification of system goals, architecture and interfaces; it makes the project manager’s task easier by providing definite milestones to aim at; it may increase cost transparency by enabling separate charges to be made for each step, and for any late specification changes; and it’s compatible with a wide range of tools. Where it can be made to work, it’s often the best approach.²⁸⁴

Writing about the history of Microsoft’s security engineering methods, Anderson states: ‘It’s telling that the biggest firm pushing evolutionary development reverted to a waterfall approach for security’.²⁸⁵

Another challenge that is somewhat related to software programming culture but afflicts the engineering community more generally, concerns the standing of SbD ideals amongst computer engineers. There is considerable literature on the difficulty of getting engineers to embrace PbD ideals.²⁸⁶ It might be expected that ‘security’, with its relatively technocratic character, has greater resonance amongst engineers than ‘privacy’ has. However, evidence suggests that SbD faces a similar struggle to gain widespread traction in the engineering community. A recent survey by Spiekermann and others showed that while ‘the vast majority of engineers are aware that they should be pursuing privacy and security by design’,²⁸⁷ a large percentage of the respondents (40 percent) felt that it was not their responsibility to integrate privacy and security into their work and did not find pleasure in doing so.²⁸⁸ Other factors were also at play, particularly lack of time and the culture of the organisation in which an engineer works.²⁸⁹

6.3 Legitimacy Challenges

Currently, the SbD enterprise is generally regarded as valuable and desirable. Apart from doubt over the ability to realise its ideals, the enterprise attracts little criticism. At the same time, SbD is tied to the concept of security which has a range of referent objects that may be politically or economically charged. National security and national sovereignty are two prominent examples. Both are regularly invoked to justify the extraordinary exercise of power, often at the expense of civil liberties. Both are central elements of securitisation.²⁹⁰ While SbD as such has yet to be invoked in contexts in which national security or national sovereignty are the dominant referent objects, securitisation is definitely shaping the development of cybersecurity praxis and policy more generally.²⁹¹ In the long term, it will be difficult to keep SbD separated from policy processes over which national security and national sovereignty hold sway. The ongoing discussions around security of 5G networks illustrate this difficulty well.²⁹² A problem is that when national security becomes the predominant driver of the push for SbD there is an increased risk of SbD being used to strengthen state interests in an authoritarian or semi-authoritarian way, thus calling the legitimacy of SbD seriously into question.

Government actors are not the only potential problem for the legitimacy of SbD. The actions of private corporations also need to be kept under scrutiny. Corporations could use SbD as a pretext for protecting their commercial interests at the expense of the legitimate interests of others. An indication of this potential is the clash over how much power producers should be given to determine how their products are used after being sold to consumers and other end-users. When wielding and justifying this power, producers can easily fly the flag of ‘security’. This has happened, for example, with Sony. In 2010, the company implemented an update to the operating system of its Playstation 3 consoles which took away a particular feature (‘OtherOS’) allowing users to install the Linux operating system and thereby employ the consoles for a variety of purposes beyond gaming. Sony justified the update by pointing to security concerns and the need to combat digital piracy, but many console users saw the move as essentially motivated by Sony’s commercial proprietary needs and instigated litigation.²⁹³ Similarly, printer manufacturers have cited security fears as a reason for installing smart chips in their ink cartridges in order to ensure that only those cartridges can be accepted by their printers. The fears are allegedly based on the potential for hackers to exploit vulnerabilities in smart chips from other ink cartridge manufacturers.²⁹⁴ Again, deeper, commercially grounded proprietary concerns are probably at play, with the security fears being used to ‘whitewash’ (or, more accurately, ‘security-wash’) those concerns.