Navigating the AI Frontier: Confronting AI-Enabled Crime Through Robust Incident Reporting

The rapid advancement of artificial intelligence presents a double-edged sword. While promising transformative benefits across various sectors, it also introduces novel challenges, particularly in the realm of online criminality. As AI systems become more sophisticated and widely adopted, evidence is mounting of a significant surge in AI-enabled crime, impacting everything from financial security to the safety of the most vulnerable. This evolving threat landscape underscores the urgent need for proactive measures, including a robust and comprehensive system for tracking and understanding AI-related incidents.

Our exploration into this critical intersection, drawing on recent research, reveals the transformative potential of AI in the hands of criminals. The ability of AI to automate and rapidly scale malicious activities is a key driver of this growth. We are already witnessing AI augmenting existing online crime types, making phishing attacks more convincing, generating more realistic deepfakes for financial fraud, and even contributing to the creation of child sexual abuse material at an alarming scale. The diffusion of AI tools to criminal groups, coupled with their unconstrained innovation, paints a concerning picture of the future of online security. Furthermore, the exploitation of vulnerabilities in AI systems themselves, such as the removal of guardrails from large language models, allows criminals to harness the power of AI for illicit purposes.

In response to this escalating threat, a crucial element in building a safer AI ecosystem is the establishment of a mandatory AI incident reporting framework. Current independent databases that document AI incidents, while valuable, primarily rely on publicly available information and lack a federated policy framework for consistent reporting. To address this gap, a standardized and comprehensive system for recording, analyzing, and responding to AI incidents is essential.

The proposed mandatory AI incident reporting structure, informed by lessons from high-impact sectors like transportation, healthcare, and cybersecurity, outlines key components for effective data collection. These include details about the type of event (incident or near miss), the nature and severity of harm across various dimensions (physical, economic, reputational, human rights, etc.), crucial technical data about the AI systems involved, the context and circumstances surrounding the incident, and post-incident data such as the response and ethical impact assessment. Documenting near misses, a practice more established in healthcare and transportation, is particularly important for early risk detection and strengthening safety measures.

Implementing such a mandatory reporting structure offers significant benefits:

• Facilitates consistent data collection of AI incidents.
• Promotes tracking, monitoring, research, and information sharing.
• Enhances knowledge around AI-related harms and risks, including those exploited by criminals.
• Ensures that essential incident data is collected to prevent reporting gaps.
• Builds a foundational framework for agile reporting that adapts to AI advancements.
• Aids in enhancing AI safety and security measures.
• Reveals vital trends and provides greater clarity on severity.
• Allows for more accurate assessment of the impact and effectiveness of safety and security policies.

To fully realize these benefits, several policy recommendations are proposed:

• Publish standardized AI Incident Reporting Formats based on the identified key components.
• Establish an Independent Investigation Agency, akin to the National Transportation Safety Board, to conduct in-depth examinations of significant AI incidents and uncover underlying causes.
• Governments, regulators, developers, and researchers are encouraged to widely and consistently adopt and use the standardized key components.

Several key factors are driving the growth of AI-enabled crime:

• AI's ability to automate and rapidly scale the volume of criminal activity. This allows criminals to conduct more attacks and generate more illicit content with greater efficiency. Law enforcement interviewees noted instances like Operation Cumberland, where a single individual possessed 400,000 AI-generated images of child sexual abuse, highlighting the challenge of keeping up with this volume.
• The augmentation of existing online crime types with AI. AI can enhance the effectiveness and efficiency of traditional crimes. For example, AI can create more realistic content and automate its delivery at scale. The quality of AI-generated text can improve phishing attacks by making them more persuasive. AI can also be used at various stages of ransomware attacks, from network reconnaissance to negotiating with targets and laundering gains.
• The diffusion of AI to criminal groups. Criminals are gaining access to AI technologies through various channels, including peers, state actors, and the private sector. There's a general trend where capabilities developed by state actors tend to appear in organised crime within a few years, as "anything that's useful is being done".
• Criminal innovation in the development and use of AI. Well-organized cybercrime groups are highly innovative in adopting new technologies like AI for criminal purposes. Criminal innovation is often less constrained by ethical principles or regulation compared to the private and public sectors. The emergence of tools like WormGPT and FraudGPT, specifically designed for malicious activities, exemplifies this criminal innovation.
• The symmetry between AI's deceptive capability and widespread human, psychological, and cognitive vulnerabilities. AI's ability to produce manipulative content aligns with human psychological frailty, making deceptive criminal practices more widespread and sustainable.
• The exploitation of open-weight AI systems with fewer guardrails. Innovation in frontier AI, particularly from China, is impacting the threat landscape. Criminals are increasingly exploiting open-weight systems with fewer restrictions to carry out more advanced tasks, as seen in the creation of CSAM.
• Criminal attacks targeting AI systems and models. Exploiting vulnerabilities in AI systems, such as jailbreaking, removing guardrails, abusing commercial models, and developing bespoke criminal LLMs, allows for more effective human-machine teaming to achieve illegal objectives.
• Altered criminal market dynamics. Increasing AI proliferation can make crime areas like fraud and CSAM more lucrative, encouraging new entrants into these markets. The low barriers to entry and availability of starter packs for AI-driven fraud contribute to this.

These factors collectively contribute to the observed and anticipated growth in the scope, severity, and impact of AI-enabled crime.

Addressing AI-enabled crime requires a multi-faceted approach. Just as law enforcement needs to rapidly adopt AI tools to counter criminal activities, a comprehensive understanding of how and why AI incidents occur is paramount. By implementing a mandatory AI incident reporting framework with standardized key components, we can build a crucial evidence base. This data will inform better safety and security measures, facilitate proactive disruption of criminal activities, and ultimately contribute to the development of more trustworthy and secure AI systems for the benefit of society. The time to act is now, to ensure that as we navigate the promising yet potentially perilous AI frontier, we are equipped with the knowledge and mechanisms to mitigate harm and foster a safer digital world.