AI Governance

Pushing the Bounds: AI Risk Assessments as Accountability Moats

Nikki Muggli Blog, Paper , , , , ,

By Michael Moran

Introduction
Before the dust settled from Meta’s colossal but controversial child safety settlement, another trial kicked off in New Mexico bringing the Cambridge Analytica scandal of 2018 back into the news.[1] In that data breach and privacy scandal, Facebook violated an earlier Federal Trade Commission (FTC) order, and agreed to pay a $5 billion penalty and adopt organizational restructuring and safeguards including compliance with a mandatory privacy program.[2] Following the settlement, Meta made substantial investments into its privacy program, but recent actions suggests the company is taking advantage of “political tumult in the United States” [3] to outsource privacy compliance to AI, delegating accountability for future privacy harms to an algorithm.[4]

Meta’s Legal Obligation to Provide Data Protection Assessments
In 2019, Meta entered a 20-year settlement order with the FTC mandating the creation of a privacy program with particular staffing and independent reporting requirements. While the order attracted headlines as the largest financial penalty assessed by the FTC, Commissioner Chopra criticized the agreement for granting individual immunity to Mark Zuckerberg, Sheryl Sandberg, and other executive officers, citing the importance of individual liability to encourage corporate accountability and compliance.[5]  The 3-2 majority of the FTC believed staffing and reporting mandates could improve “the transparency of decision making and hold[] Facebook accountable via overlapping channels of compliance.”[6]

Staffing
To empower decision-makers and overlapping compliance channels, Meta was required to “[d]esignate a qualified employee or employees to coordinate and be responsible for the Privacy Program (‘Designated Compliance Officer(s)’)”.[7] The Designated Compliance Officer(s) and “Principal Executive Officer” (Mark Zuckerberg) have reporting and certification responsibilities to ensure the privacy program meets the structural safeguards mandated by the order.[8] An independent privacy committee within Facebook’s board of directors was required, in addition to contracting an external third-party assessor for additional oversight.[9] All members of the independent privacy committee must be able “to read and understand data protection and privacy policies and procedures” to exercise their oversight duties.[10]

Reporting
Related to increased transparency of decision-making, designated compliance officers are required to conduct a privacy review prior to implementing any new or modified product that presents a material risk to privacy.[11] This decision must be documented in a written “Privacy Review Statement”, which describes the product, information collected, affirmative express consent mechanisms, and a full description of privacy risks and safeguards implemented or rejected. [12] The order also obligated Meta to promptly document and report any covered incident data breaches involving 500 or more consumers.[13]The Designated Compliance Officer(s) must deliver quarterly reports to the Principal Executive Officer based upon (1) personal knowledge, (2) sub-certification by “knowledgeable personnel charged with implementing the Mandatory Privacy Program”, (3) material issues from any privacy review statements and decisions from the prior quarter, and (4) material issues raised in covered incident reports for the quarter. Based on these quarterly reports, their own personal knowledge, or sub-certification by knowledgeable charged personnel, the Principal Executive Officer certifies compliance with the order.[14]

In addition to these reporting requirements, the order specifically mandates collection of user’s affirmative express consent prior to the use of facial recognition templates, after clear and conspicuous disclosure, separate and apart from privacy policy notices.[15] Outside of the FTC order, numerous state and international laws also require Meta to conduct and document a “data protection assessment” (DPA) for any processing activity that presents a heightened risk of harm, such as collection of biometric data.[16]  

Meta’s Evolving Privacy Program
Meta marketed the FTC settlement as an opportunity to incorporate privacy by design across the organization, touting $8 billion invested in the privacy program and development of new privacy features across Meta platforms.[17] To meet the staffing requirements of the order, longtime Facebook executive, Michel Protti was promoted to Chief Compliance & Privacy Officer,  who was later joined by Erin Egan, Chief Privacy Officer, Policy, to serve as designated compliance officers. By 2024, Meta increased the number of employees focused on privacy from a few hundred to over 3000, evaluating an average of 1,4000 product features each month.[18] The “Privacy Legal team [became] embedded in the design and ongoing execution of [Meta] program[s] and advises on legal requirements during the course of our privacy review process.”[19]

To manage an increasing number of product developments, and expanding digital compliance requirements, Meta began incorporating AI automation into product design and privacy reviews in 2025; building privacy code scanning solutions to replace manual “point checking” controls by engineers. [20] Meta argues that these tools enable the compliance team to add rigor at scale; to detect and screen risks earlier, flagging issues that need mitigation or escalation, but pushing through risks deemed insignificant to avoid delays to product development.[21] Using tools such as the Diff Risk Score AI tool, Meta uses AI to assess risk, displaying a color-coded risk thresholds with potential corrective responses for engineers to incorporate.[22] Meta is aiming for 90% of all risk assessments to be automated, even in sensitive areas including AI safety, youth risk, and misinformation.[23] Assessments of novel and complex issue are reviewed by senior subject matter experts across technical, legal, and policy teams until these decisions can be mapped, codified, and phased into greater AI risk management system.[24] With the massive scale of ongoing product development at Meta, and lack of clearly articulated threshold for novel and complex issues, it is the possible the AI algorithm could dismiss risks the FTC might deem substantial, intentionally, or through pattern recognition on training data dismissing risks. Users must trust that Meta’s experts are conducting ongoing oversight, to double check for accuracy after AI’s first pass.[25]

Meta maintains that “[h]umans are still very much in control of the process”, but the strategic decisions around organizational structure and product development suggest otherwise.[26] While adjusting its risk review process to incorporate AI analysis, Meta has reduced privacy team influence and oversight, telling employees who questioned the changes to “push the bounds” of their FTC agreement.[27] Last October, Meta laid off an undisclosed number of employees in Product Risk Program Manager, Shared Services, and Global Security & Privacy teams, “moving from . . . manual [risk] reviews to a more consistent and automated process”.[28]

Pushing the Bounds
Meta is one of many companies adopting AI tools to address a complex regulatory environment.[29] In a compliance setting, AI poses familiar challenges due to the unexplainability of ‘black box’ processes, or the potential hazards of relying on AI outputs in decision-making with unknown bias.[30] AI audit trails may be improving, but organizational and legal questions remain around accountability and oversight, particularly for Meta with 13 years remaining on the FTC order mandates.

Related to the staffing and oversight requirements of the order, the use of agentic AI in risk assessment dilutes transparency of privacy decision-making. By concurrently reducing the headcount of employees tasked with oversight, and introducing AI tools to sort and triage risk assessment, Meta has at best delegated, and worst, neglected the FTC’s mandate to designate “knowledgeable personnel [to] implement[] the Privacy Program”.[31] The designated compliance officers rely on a skeleton crew that act as an accountability sink, lacking sufficient resources to provide the human oversight of privacy decisions mandated by the FTC.[32]

With agentic AI, risk research and documentation can be supercharged, but best practices recommend, and some regulators require, that humans retain decision-making authority.[33] 2019 Department of Justice guidance recommended that compliance programs design risk assessments to maintain a “finger on the pulse”, not “asleep at the switch”, almost forecasting compliance outsourcing first to vendors, and now AI.[34]

Even the most sophisticated tech and compliance professionals would be hard pressed to oversee the evolving privacy program at Meta, keeping track of the granular shift of risk decisions from novel to normal. There is no reason to assume that AI tools errors will be recognized, or self-reported, even with the order’s obligation to report covered incidents.[35] An AI analysis could underestimate a risk that that violates an uncoded concept of human dignity, or fail to recognize a risk that is novel in the aggregate, due to compounding internal or external variables. The opposite, that the tool catches risk humans miss is certainly true, but with less humans in the loop, it becomes far less likely that an engineer would internally flag borderline novel or ‘icky’ features for legal review, or whistle blow, alerting the public about pending or future privacy harms.[36] In delegating initial reviews to AI, with lower-level compliance employees serving as accountability moats, Meta’s designated compliance officers have created layers of insulation and detachment from each unique privacy decision, yet still are certifying the privacy programs, based on personal knowledge or knowledgeable personnel that is in fact an algorithm.

Move Fast and Break Things
Through AI-automated risk assessments and a model of continuous deployment of code, Meta has drastically shortened its product release cycle from nearly two months down to days.[37] The rapid development and deployment timeline is further testing the regulatory appetite of FTC, and unnerving civil libertarians with new products and features outpacing legislative privacy protections and surveillance restrictions. A recent example of Meta’s rushed and unrestrained product development challenging the plain language and spirit of the FTC order are its controversial smart glasses.[38]

Meta’s Stealth Introduction of Facial Recognition to Smart Glasses
In early June 2026, the entire computational and storage stack for on-device facial recognition was detected in the Android version of Stella, the companion app for Meta’s smart glasses.[39] This “NameTag” feature is capable of detecting, matching, and storing biometric identifiers, often called faceprints. This code had been shipped to user’s phones, and could be remotely activated by Meta, allowing users to “see [their] connections” and “[r]emember the people [they] met and make new connections”.[40] The feature had not been activated, nor was the code evident to most consumers, but WIRED and EFF researchers were able to activate the code in debug mode to demonstrate the face identification feature using the glasses and code in real time.

Following the reporting and public scrutiny, Meta removed the unactivated code from the app, dismissed the story as “misleading” and referred to the code as “exploratory”. A week after removing the code, WIRED discovered a license agreement between Meta and a vendor called Rank One, selling access to military-grade facial recognition and “liveness detection” software to differentiate between the actual live face of a person.[41] These active and substantial steps towards implementing biometric identification invited the scrutiny of state AGs and multiple class action lawsuits from parents in Illinois.[42] Even with this backlash, Meta’s new AI personal agent app Muse follows a similar playbook, probing privacy boundaries until civil society pushes back.[43] All the while, new footholds for data collection are established, speedrunning new product features while the mandated privacy program operates on AI-autopilot until the next major violation or breach is discovered.

Conclusion
Back in Albuquerque, Meta’s counsel argued that New Mexico’s case was outdated because the company has undertaken serious soul searching and implemented structural changes to protect user privacy.[44] It appears more likely that Meta has taken advantage of political tumult to delegate the responsibility of protecting human dignity and privacy further away from leadership. The New Mexico jury agreed, finding Meta leadership had made 26 misleading statements, promising consumers transparency, internal investigations and compliance initiatives that never materialized.[45] With the integration of AI assisted risk assessments, Meta has ignored the opportunity to provide greater transparency, in favor of increased productivity and blurred accountability.

[1] As part of Meta’s August 2026 teen social media addiction settlement, 48 states released Meta from liability related to the Cambridge Analytica scandal. New Mexico was not involved in that multistate agreement, having previously won its teen safety trial against Meta. The New Mexico lawsuit was filed in 2021. Mikaela Schuettler, Nearly A Decade After the Cambridge Analytica Scandal, New Mexico Alone is Taking Facebook to Trial, AP News (Sept. 9, 2026), https://apnews.com/article/facebook-meta-new-mexico-privacy-lawsuit-fe250dc752777abd4f95f49111a2008a [https://perma.cc/7VMB-N7AK].

[2] Fed. Trade Comm’n, FTC Imposes $5Billion Penalty and Sweeping New Privacy Restrictions on Facebook (July 24, 2019), https://www.ftc.gov/news-events/news/press-releases/2019/07/ftc-imposes-5-billion-penalty-sweeping-new-privacy-restrictions-facebook [https://perma.cc/JA6M-8EYX]. Meta changed its name from Facebook in 2018, hence the varying references.

[3] An internal memo at Meta recommended taking advantage of a “dynamic political environment where many civil society groups that we would expect to attack us would have their resources focused on other concerns” to release facial recognition on smart glasses. See Kashmir Hill, Kalley Huang & Mike Isaac, Meta Plans to Add Facial Recognition Technology to Its Smart Glasses,N.Y. Times (Updated Feb. 14, 2026), https://www.nytimes.com/2026/02/13/technology/meta-facial-recognition-smart-glasses.html [https://perma.cc/FM9H-GKNA].

[4] Michel Protti, How AI Is Ushering in the Next Era of Risk Review at Meta, Meta (Mar. 31, 2026),https://about.fb.com/news/2026/03/how-ai-is-ushering-in-the-next-era-of-risk-review-at-meta/  [https://perma.cc/UW4E-94RM].

[5] Dissenting Statement of Commissioner Rohit Chopra, Regarding the Matter of Facebook, Inc. Comm’n File No. 1823109, at 19 (F.T.C. July 24, 2019) (Referencing a Dept. of Justice Policy Statement on Individual Liability). Commissioner Chopra also asserts that Designated Compliance Officer certification and CEO signature to ascribe compliance without benchmarks is simply a checkbox activity at 13.

[6] Fed. Trade Comm’n supra note 2.

[7] Facebook, Inc., Docket No. C-4365, Decision and Order, § VII, at 8 (F.T.C. Apr. 27, 2020).

[8] Id. at 8-11, 16-17.

[9] Id. at 12.

[10] Id. at 4.

[11] Id. at 14.

[12] Id.

[13] Id. at 5, 13.

[14] Id. at 23.

[15] Id. at 11.

[16] Cal. Code Regs. tit. 11, § 7150 (Lexis Advance through Register 2026, No. 34, August 21, 2026); Colo. Rev. Stat. § 6-1-1309(1), (2); N.H. Rev. Stat. Ann. § 507-H:8, Article 6(1)(f); Regulation 2016/679 of the European Parliament and of the Council of Apr. 27, 2016 on the Protection of Natural Persons with Regard to the Processing of Personal Data and on the Free Movement of Such Data, and Repealing Directive 95/46/EC (General Data Protection Regulation), art. 35, 2016 O.J. (L 119) 33 (EU).

[17] Michel Protti, Reflecting on Meta’s $8 Billion Investment in Privacy, Meta (Jan. 28, 2025),https://about.fb.com/news/2025/01/meta-8-billion-investment-privacy/, [https://perma.cc/UW4E-94RM].

[18] Michel Protti, Investing in Privacy, Meta (Jan. 25, 2024),https://about.fb.com/news/2024/01/investing-in-privacy/ [https://perma.cc/A79S-46GJ].

[19] Meta Privacy Progress Updates; Governance, https://www.meta.com/actions/privacy-progress/ [https://perma.cc/X73T-KH25].

[20]  The Privacy Aware Infrastructure allows Meta to demark “policy zones” across systems with distinct embedded purpose limitation rules for data. Wenlong Dong et. al, How Meta Enforces Purpose Limitation Via Privacy Aware Infrastructure at Scale, Engineering at Meta (Aug. 27, 2024), https://engineering.fb.com/2024/08/27/security/privacy-aware-infrastructure-purpose-limitation-meta/ [https://perma.cc/638N-WHE8].

[21] Meta, Responsible AI in Practice, https://www.meta.com/actions/privacy-and-risk/ [https://perma.cc/ZGL3-G35B], Responsible AI in Practice video.

[22] Rui Abrea et al., Moving Faster and Reducing Risk: Using LLMs in Release Deployment, Meta (Oct. 8, 2024), https://arxiv.org/abs/2410.06351 [https://perma.cc/C44S-AYJA].

[23] All Things Considered, Meta Plans to Replace Humans with AI to Assess Risks, NPR (May 31, 2025),  https://www.npr.org/2025/05/31/nx-s1-5407870/meta-ai-facebook-instagram-risks.

[24] Meta, supra note 17.

[25] Protti, supra note 4.

[26] Joe Duball, Meta’s Risk Assessment Updates Look Toward ‘Holistic’ Approach, IAPP (June 11, 2025), https://iapp.org/news/a/meta-s-risk-assessment-updates-look-toward-holistic-approach [https://perma.cc/U43S-HSK7].

[27] Hill, supra note 3.

[28] Jyoti Mann, Meta Tells Some Employees Their Jobs Are Being Replaced by Tech — Read the Memo, Business Insider (Oct. 23, 2025), https://www.businessinsider.com/meta-job-cuts-risk-org-replaced-by-tech-layoffs-2025-10 [https://perma.cc/N789-RUBF].

[29] JPMorgan Chase, Goldman Sachs, and Salesforce have announced plans to bolster their compliance programs with AI, in lieu of hiring. Jonathan Vanian, Meta Replacing Humans with AI for FTC-mandated Privacy Reviews CNBC (Oct. 23, 2025), https://www.cnbc.com/2025/10/23/meta-replacing-humans-with-ai-for-ftc-mandated-privacy-reviews.html [https://perma.cc/XE3H-H4J4].

[30] Explainability and model validation are particularly necessary in compliance settings to justify appropriate incorporation of AI-outputs in risk-management decisions. Int’l Compliance Ass’n, AI in Compliance – What It Is, How It’s Used, and Why Governance Matters (May 2026), https://www.int-comp.org/insight/ai-in-compliance/ [https://perma.cc/7KK5-ZE8N].

[31] Facebook, Inc., Dkt. No. C-4365 (F.T.C. Apr. 27, 2020), p. 17.

[32] By tasking employees with oversight over ever-increasing AI outputs, inevitable errors can be tied to the employee, not the corporate leadership decision to adopt the AI tool. Cory Doctorow borrows the term“accountability sink” from Dan Davies, applicable here to insulate both the CEO and Chief Compliance Officers from accountability for the risk assessments. Berkman Klein Ctr. Internet & Soc’y, Cory Doctorow on Enshittification and Reverse Centaurs at 17:57(YouTube, Oct. 8, 2026), https://www.youtube.com/watch?v=fYtIt0NjX2s.

[33] The EU requires human in the loop for any high-risk AI systems, and California also requires human corroboration for any AI-powered employment decisions. The NIST AI Risk ManagementFrameworkfocuses on a human-centric approach. See Regulation (EU) 2024/1689 of the European Parliament and of the Council, art. 14, 2024 O.J. (L 2024/1689); see also Chris Marr, State AI Laws Up the Ante for Human Oversight of Job Decisions, Bloomberg Law (Oct. 8, 2026)

 https://news.bloomberglaw.com/daily-labor-report/state-ai-laws-up-the-ante-for-human-oversight-of-job-decisions [https://perma.cc/MKB8-N5KP]; see generally Natl’t Inst. of Standards & Tech. NIST AI 100-1, Artificial Risk Management Framework (AI RMF 1.0) (Jan. 2023), https://doi.org/10.6028/NIST.AI.100-1.

[34] William D. Semins & Lauren Norris Donahue DOJ Revises Corporate Compliance Guidance Calling Attention to Three Areas Where Most Companies Fall Short, K&L Gates (May 16, 2019), https://www.klgates.com/thought-leadership/DOJ-Revises-Corporate-Compliance-Guidance-Calling-Attention-to-Three-Areas-Where-Most-Companies-Fall-Short-Risk-Assessments-Compliance-Culture-and-Continuous-Compliance-Program-Improvement-05-16-2019 [https://perma.cc/F6PN-37RX].

[35] In 2023, based on continued violations, the FTC sought to prohibit Meta from releasing new or modified products, services, or features without written confirmation of compliance. A heavily redacted public version indicates the independent assessor, Protiviti, identified deficiencies in risk assessment, transparency, and reporting. In re Facebook, Inc., Docket No. C-4365, Order to Show Cause, at 5-9. (F.T.C. May 3, 2023).

[36] Former Facebook execs and leadership report that Meta CEO Mark Zuckerman doesn’t like being told no, nor being told how Facebook products were hurting people. See Sarah Wynn-Williams, Careless People: A Cautionary Tale of Power, Greed, and Lost Idealism (Flatiron Books, 2025); Video posted by CBC Docs, Facebook, Whistleblower Frances Haugen: “Mark [Zuckerberg] doesn’t want to be told that he’s hurting people.” (Aug 21, 2026),https://www.facebook.com/cbcdocs/videos/facebook-whistleblower-frances-haugen-mark-zuckerberg-doesnt-want-to-be-told-tha/1344912297713251/

[37] Chuck Rossi et al., Continuous Development of Mobile Software at Facebook, Int’l Symposium on Found. Software Eng’g at 12, (Nov. 13, 2016), https://research.facebook.com/file/956687151563855/fse-rossi.pdf [https://perma.cc/SZ4M-HJ2N].

[38] Stories of users easily disabling the glasses minimal privacy measures and filming strangers has led to multiple lawsuits involving the “pervert glasses”. Victor Tangermann, Zuckerberg’s Pervert Glasses Have Been So Publicly Shamed That Meta’s Reportedly Now Working on a Version With No Camera, Futurism (Sep. 16, 2026), https://futurism.com/future-society/zuckerberg-pervert-glasses-shamed-meta-no-camera [https://perma.cc/6Q9B-XMPZ].

[39] Buchodi, Meta’s Smart Glasses Companion App Ships a Complete, Dormant Face-recognition Pipeline on a Stock Account, (June 4, 2026), https://www.buchodi.com/meta-glasses-facial-recognition/ [https://perma.cc/AZ9X-G85Y].

[40] Dhruv Mehrotra & Dell Cameron, Meta Silently Added Face-Recognition Code for Its Smart Glasses to Millions of Phones, WIRED (June 4, 2026), https://www.wired.com/story/meta-smart-glasses-face-recognition-nametag-connections/ [https://perma.cc/5V44-TG6U].

[41] Dell Cameron & Dhruv Mehrotra, Meta Tapped a Pentagon Supplier to Prototype Face Recognition for Its Glasses, WIRED (June 15, 2026), https://www.wired.com/story/meta-rank-one-computing-face-recognition-smart-glasses/ [https://perma.cc/7ADH-3CZL].

[42] Alvarez v. Meta Platforms, Inc., No. 1:26-cv-10773, Compl. (N.D. Ill. Sept. 4, 2026). https://www.biometricupdate.com/wp-content/uploads/2026/09/Alvarez_et_al_v_Meta_Platforms_Inc__ilndce-26-10773__0001.0.pdf ; Mike Scarcella & Jody Godoy Meta to Pay $1.4 Billion to Settle Texas Facial Recognition Data Lawsuit, Reuters (July 31, 2024), https://www.reuters.com/technology/cybersecurity/meta-platforms-pay-14-bln-settle-texas-lawsuit-over-facial-recognition-data-2024-07-30/  [https://perma.cc/F978-MEB4].

[43] Default settings allow Meta to further train its AI models, and encouraging users to share personal details with the Labubu-esque personal agent. Jonathan Vanian, Meta Pushes into Personal AI Agents As Company Faces Public Reckoning Over Privacy and Safety, CNBC (Updated Sept. 8, 2026, 3:09PM)

 https://www.cnbc.com/2026/09/08/meta-personal-ai-agents-public-reckoning-privacy-safety.html [https://perma.cc/M2FG-CS7V].

[44] Schuettler supra note 1.

[45] Special Verdict Form at 21-29, State of New Mexico v. Meta Platforms, Inc, No. D-101-CV-2021-00132 (N.M. Dist. Ct. Santa Fe. Cty., Sept. 25, 2026).

AI, a Watchful Eye: The Less than Stellar Performance of AI Security and the Consequences Thereof

mdecrow@maine.edu Article, Blog , , ,

AI, a Watchful Eye: The Less than Stellar Performance of AI Security and the Consequences Thereof

James Hotham

 

The use and abuse of widespread camera surveillance is not a novel fear. For decades, media has explored this concept. However, a new threat has arisen in a new form. It has not taken the form of an oppressive government, a terrorist group, or a supreme artificial intelligence. Rather, it comes from private party security providers. Several security providers have begun to work AI into their  security cameras for the use of threat detection.[1] However, the success of these threat detection models is dubious. Just this year in late October, one of these systems placed in a Baltimore school, detected an individual carrying a firearm.[2] Police arrived and identified the suspect as sixteen-year-old Taki Allen.[3] However, after the police drew their weapons and handcuffed young Allen, they discovered the “firearm” was actually just an empty bag of Doritos.[4]

Despite the fact that AI technology of this level of sophistication is relatively new, it has sprouted into a multimillion-dollar industry in just a few years. But despite years of development, mishaps like these still occur. This article will explore how these systems work, why they malfunction, ways consumers can avoid these malfunctions, and potential liability for when they malfunction.

Continue reading →

“Don’t Reinvent the Wheel, Just Realign It.” How Lessons from the Belmont Report Can Help Govern the Use of AI in Research

mdecrow@maine.edu Article , , ,

“Don’t Reinvent the Wheel, Just Realign It.”Just Realign It.”[1] How Lessons from the Belmont Report Can Help Govern the Use of AI in Research How Lessons from the Belmont Report Can Help Govern the Use of AI in Research

Steven Hammerton

 

Background

Artificial intelligence (AI) is becoming increasingly integrated into many areas of life, including research. However, legislation and regulation lag. Years into the widespread adoption of AI and the United States is still without meaningful guardrails to address the ethical quandaries that stem from the use of AI. Until there is comprehensive legislation, the burden of ensuring ethical training, development, and usage of AI will be on the developers, deployers, and users of AI, such as researchers and research participants. This article will explore three different ethical issues associated with AI and how principles from the Belmont Report can guide researchers and other users of AI in their pursuit of ethical AI.

Continue reading →

LAWS Need Laws: Distinction and Proportionality in the Age of Autonomy

mdecrow@maine.edu Blog , , , , , , ,

LAWS Need Laws: Distinction and Proportionality in the Age of Autonomy

Steve Hammerton

 

I. Introduction

There’s a lethal autonomous elephant in the room, and it’s only minimally regulated by DoD Directive 3000.09 (“DODD 3000.09”).[1] Under that directive, lethal autonomous weapon systems (LAWS) are said to be “weapon system[s] that, once activated, can select and engage targets without further intervention by an operator.”[2] In contrast to other nations who have called for an outright ban on such systems, the United States has resisted.[3] Instead, the Department of Defense (“DoD”) has required that LAWS, like all other weapons systems, “be designed to allow commanders and operators to exercise appropriate levels of human judgment over the use of force.”[4] A quick read of this policy would suggest that it requires a human-in-the-loop. However, a more exacting analysis of the language reveals that it only requires “human judgment over the use of force” which only seems to refer to broad themes of lethality like when and where it will be deployed, but not against whom. The directive also refers to an inchoate review process that does not spell out a clear framework for assessing the efficacy and safety of LAWS.[5] Without a clearer statement on the “appropriate levels of human judgment,” the lack of distinction in targeting conflicts with the two core jus in bello principles, distinction and proportionality.[6]

At the same time, LAWS may offer a comparative advantage over human trigger pullers. Canadian think-tank Centre for International Governance Innovation suggests that LAWS “may be able to assess a target’s legitimacy and make decisions faster and with more accuracy and objectivity than fallible human actors could.”[7] Simply put, LAWS could reduce unintended errors or deliberate unlawful killings. Indeed, technology-assisted precision weapons have already reduced collateral damage in armed conflicts.[8] Recent conflicts have been marked by an increased use of autonomous and AI-assisted weaponry, though it is too early to say whether the use of these weapons has identifiably reduced unintended civilian casualties.[9] With the increasing shift to LAWS and other AI-assisted weapons, it seems unrealistic to expect an outright ban. Consequently, the United States and its international partners should seek to preserve distinction and proportionality through a meaningful and complex review, such as a risk-benefit analysis, that recognizes the inherent dangers of using LAWS but appreciates the potential for harm reduction.

Continue reading →

Artificial Intelligence Liability

mdecrow@maine.edu Blog, Uncategorized , , , , , , ,

Artificial Intelligence Liability

By Susan-Caitlyn Seavey

1. Who is Responsible for Harm flowing from AI?   

Most people can easily recognize the immense impact technological developments have had in the recent decade, affecting practically every sector. While the laws and regulations governing our society have somewhat lagged behind these technological advances, we have still managed to create a framework that seems to effectively govern these modern tools. With the implementation and widespread usage of AI, our current legal and regulatory parameters do not neatly fit anymore. We are left with questions about who is ultimately responsible for harms that stem from AI. The issue of liability does not likely have a one size fits all solution, and our government and courts are working to understand and produce the new standards and guidelines AI requires. Stanford Law Fellow, Thomas Weber, says it well: “Generative AI is developing at a stunning speed, creating new and thorny problems in well-established legal areas, disrupting long-standing regimes of civil liability—and outpacing the necessary frameworks, both legal and regulatory, that can ensure the risks are anticipated and accounted for.”[1] Until there is substantial court precedent and more promulgated AI laws, scholars and professionals are limited to discussing different theories of liability that may be suitable for AI, such as strict liability and negligence law.

            In 2023, a man in Belgium ended his life after apparently becoming emotionally dependent on an AI-powered chatbot, leaving behind his wife and two children.[2] Also in 2023, Stanford’s Director of Law, Science and Technology, Professor Lemley, asked chatbot GPT-4 to provide information about himself.[3]> The algorithm offered defamatory information, believing Professor Lemley’s research to actually be a misappropriation of trade secrets.[4] In both of these cases, it is unclear who would and/or could be held liable for the death of the father and for the defamatory information. Traditional liability is long-established with laws and regulations in place and ample case law to support the structure we have created for it. However, AI transcends many of the boxes we have fit other technology into, including the liability framework.

For Professor Lemley to establish the requisite elements of a defamation claim, he would have to prove the bad actor’s intent to defame; the standard requires that a reasonable person should have known that the information was false or exhibited a reckless disregard as to the truth or falsity of the published statement.[5] But how does one show that a robot possesses such requisite intent? It would follow that liability may fall to the developers if intent cannot be apportioned to the AI technology at issue. The apparent irrelevance of intent with AI requires an alternative option to account for liability. A guide of best practices may be helpful to direct AI. “Professor Lemley suggests [that by] implementing best practices, companies and developers could shoulder less liability for harms their programs may cause.”[6] While not specifically broken down, this concept is supported by the Cybersecurity and Infrastructure Security Agency’s (CISA) work to develop “best practices and guidance for secure and resilient AI software development and implementation.”[7]

Continue reading →

Addressing the Vectors for Attack on Artificial Intelligence Systems Used in Clinical Healthcare through a Robust Regulatory Framework: A Survey

mdecrow@maine.edu Paper , , , , , , , ,

PDF Link

Addressing the Vectors for Attack on Artificial Intelligence Systems Used in Clinical Healthcare through a Robust Regulatory Framework: A Survey

By Benjamin Clark

Introduction and Overview

Artificial intelligence has captivated the current interest of the general public and academics alike, bringing closer attention to previously unexplored aspects of these algorithms, such as how they have been implemented into critical infrastructure, ways they can be secured through technical defensive measures, and how they can best be regulated to reduce risk of harm. This paper will discuss vulnerabilities common to artificial intelligence systems used in clinical healthcare and how bad actors exploit them before weighing the merits of current regulatory frameworks proposed by the U.S. and other nations for how they address the cybersecurity threats of these systems.

Primarily, artificial intelligence systems used in clinical research and healthcare settings involve either machine learning or deep learning algorithms.[1] Machine learning algorithms automatically learn and improve themselves without needing to be specifically programmed for each intended function. [2] However, these algorithms require that input data be pre-labeled by programmers to train algorithms to associate input features and best predict the labels for output, which involves some degree of human intervention.[3] The presence of humans in this process is referred to as “supervised machine learning” and is most often observed in systems used for diagnostics and medical imaging, in which physicians set markers for specific diagnoses as the labels and algorithms are able to categorize an image as a diagnosis based off the image’s characteristics.[4] Similarly, deep learning is a subset of machine learning characterized by its “neural network” structure in which input data is transmitted through an algorithm through input, output, and “hidden” layers to identify patterns in data.[5] Deep learning algorithms differ from those that utilize machine learning in that they require no human intervention after being trained; instead, deep learning algorithms process unlabeled data by determining what input is most important to create its own labels.[6]

Continue reading →

Generative AI Algorithms: The Fine Line Between Speech and Section 230 Immunity

mdecrow@maine.edu Paper , , , , , , ,

Generative AI Algorithms: The Fine Line Between Speech and Section 230 Immunity

 By Hannah G. Babinski

ABSTRACT

Russian-American writer and philosopher Ayn Rand once observed, “No speech is ever considered, but only the speaker. It’s so much easier to pass judgment on a man than on an idea.”[1] But what if the speaker is not a man, woman, or a human at all? Concepts of speech and identities of speakers have been the focal points of various court cases and debates in recent years. The Supreme Court and various district courts have faced complex and first-of-their-kind questions concerning emerging technologies, namely algorithms and recommendations, and contemplated whether their outputs constitute speech on behalf of an Internet service provider (“Internet platform”) that would not be covered by Section 230 of the Communications Decency Act (“Section 230”).  In this piece, I will examine some of the issues arising from the questions posed by Justice Gorsuch in Gonzalez v. Google, LLC, namely whether generative AI algorithms and their relative outputs constitute speech that is not immunized under Section 230. I will provide an overview of the technology behind generative AI algorithms and then examine the statutory language and interpretation of Section 230, applying that language and interpretive case law to generative AI. Finally, I will provide demonstrative comparisons between generative AI technology and human content creation and foundational Copyright Law concepts to illustrate how generative AI technologies and algorithmic outputs are akin to unique, standalone products that extend beyond the protections of Section 230.

 

Continue Reading