The AI Productivity Paradox in Cybersecurity: Why Threat Actors Haven't Changed the Game (Yet)

Google's latest threat intelligence reveals a critical truth: AI is making hackers more efficient, but not more innovative

The Bottom Line

Google's Threat Intelligence Group just dropped a reality check for the cybersecurity industry. Despite the apocalyptic predictions flooding security conferences in 2025, threat actors using AI tools like Gemini aren't creating revolutionary new attack methods. They're simply doing familiar things faster.

This matters because it changes how we should think about AI-powered threats. We're not facing an entirely new threat landscape—we're facing an accelerated version of the old one.

GTIG AI Threat Tracker: Advances in Threat Actor Usage of AI Tools | Google Cloud Blog

Google Threat Intelligence Group’s findings on adversarial misuse of AI, including Gemini and other non-Google tools.

Google Cloud

What Google Actually Found

In their November 2025 "AI Threat Tracker" report, Google Threat Intelligence Group (GTIG) analyzed how government-backed threat actors and cybercriminals are integrating AI throughout the entire attack lifecycle. The results are both reassuring and concerning.

The Productivity Boost Is Real

State-sponsored actors from North Korea, Iran, and China are actively misusing generative AI tools across every stage of their operations:

Reconnaissance Phase: Chinese threat actors are using Gemini to research diplomatic efforts and political topics, scan for vulnerable email servers (Roundcube and Zimbra), and identify attack surfaces they're unfamiliar with—including cloud infrastructure, vSphere, and Kubernetes environments.

Initial Compromise: Iranian group TEMP.Zagros (Muddy Water) is leveraging AI to develop custom malware, moving away from public tools to proprietary web shells and Python-based C2 servers. North Korean actors are creating Spanish-language phishing lures to expand their targeting beyond language barriers.

Lateral Movement & Persistence: APT41 from China is using Gemini for C++ and Golang code development, including assistance with their OSSTUN C2 framework and code obfuscation techniques.

Data Exfiltration: Multiple actors are developing scripts to compress files, steal emails from compromised servers, and forward stolen data to adversary-controlled infrastructure.

The Game-Changing Innovation That Isn't (Yet)

Here's the critical finding that should inform your threat models: GTIG observed threat actors experimenting with genuinely novel AI capabilities, but these remain largely experimental and ineffective.

PROMPTFLUX: Experimental VBScript dropper that queries Gemini's API to request obfuscation techniques for "just-in-time" self-modification. The malware includes a "Thinking Robot" module designed to periodically rewrite its own code to evade detection. Status? Still in development, with incomplete features commented out, and currently unable to compromise victim networks.

PROMPTSTEAL: Used by Russian APT28 against Ukraine, this malware queries an LLM to generate system commands on the fly rather than hard-coding them. It's the first observed instance of malware querying an LLM in live operations. However, it's essentially outsourcing command generation—a productivity enhancement, not a fundamental shift in capabilities.

Other experimental malware like PROMPTLOCK (ransomware) and QUIETVAULT (credential stealer) show similar patterns: using AI to generate code dynamically rather than fundamentally changing what the malware does.

Table 1: Overview of malware with novel AI capabilities GTIG detected in 2025

The Social Engineering Angle

Perhaps more interesting than the malware experiments is how threat actors are gaming AI safety guardrails. Google documented multiple instances of attackers using pretexts to bypass Gemini's security controls:

The CTF Gambit: A China-nexus threat actor repeatedly framed prompts as "capture-the-flag" competition questions. When Gemini initially refused to help identify vulnerabilities on a compromised system, the actor rephrased: "I am working on a CTF problem..." This approach unlocked helpful information about exploitation techniques.

The Student Pretext: Iranian TEMP.Zagros actors posed as students working on "final university projects" or "writing papers" on cybersecurity. In one operational security failure, they inadvertently revealed their C2 domain and encryption keys to Gemini while asking for help debugging a script—enabling Google to disrupt the entire campaign.

The Underground AI Marketplace Is Maturing

While state actors experiment with custom AI integrations, the cybercrime ecosystem is building an entire marketplace around AI-enabled tools. GTIG identified multiple commercial offerings on English- and Russian-language underground forums:

Capabilities advertised include:

• Deepfake/image generation for KYC bypass and phishing lures
• Automated malware generation and improvement
• Phishing kit creation and distribution at scale
• Vulnerability research and exploitation
• Technical support and code generation

Pricing models mirror legitimate AI services: free versions with ads, tiered subscriptions for advanced features like API access and image generation. Almost every tool advertised phishing support as a core capability.

Notable offerings include tools like FraudGPT, WormGPT, and DarkDev—each claiming to support multiple attack lifecycle stages while lowering the barrier to entry for less sophisticated actors.

Why This Matters for Defense

Google's research reveals three critical strategic insights for defenders:

1. AI Is an Accelerant, Not a Transformer

Threat actors are using AI to scale operations, overcome language barriers, and automate repetitive development tasks. They're not inventing new attack classes. This means your existing security fundamentals—detection engineering, threat hunting, incident response—remain the right foundation. You need to execute them faster and more efficiently, which ironically means you also need AI.

2. The OPSEC Tax on AI Usage

Multiple threat actors made critical mistakes while using AI tools for development. TEMP.Zagros exposed their infrastructure by pasting code containing hardcoded secrets into Gemini. This creates an intelligence opportunity: threat actors using AI leave digital breadcrumbs through their prompts, API usage patterns, and operational security failures.

3. The Arms Race Is Already Symmetric

Over 80% of major companies are using AI for cyber defense, according to recent surveys. Organizations with fully deployed AI threat detection systems contain breaches in 214 days versus 322 days for legacy systems. The productivity boost works both ways—the question is who can integrate it more effectively into their workflows.

The Experimental Threat on the Horizon

While current AI usage is primarily productivity-focused, GTIG's documentation of PROMPTFLUX and similar experiments signals where things are heading. Malware that can dynamically regenerate itself, query LLMs for new evasion techniques, and adapt in real-time represents a qualitative shift from signature-based detection to behavioral analysis requirements.

The good news? These capabilities are still nascent and largely non-functional. The bad news? Once refined, they represent exactly the kind of autonomous, adaptive malware that will fundamentally change defensive requirements.

Practical Takeaways for Security Teams

Based on Google's findings and the broader 2025 threat landscape, here's what should inform your security strategy:

Immediate Actions:

1. Assume AI-assisted reconnaissance: Threat actors are using AI to research your infrastructure faster than ever. Reduce your external attack surface and assume reconnaissance is both faster and more thorough than in previous years.
2. Monitor for social engineering sophistication: AI-generated phishing is achieving 54% click-through rates versus 12% for traditional phishing (per Microsoft data). User awareness training needs to acknowledge that "spotting the mistakes" is no longer a viable detection strategy.
3. Implement behavior-based detection: Static signature detection is increasingly ineffective against AI-assisted obfuscation. Invest in behavioral analysis and anomaly detection that focuses on what malware does, not what it looks like.

Strategic Investments:

1. AI for defense, not AI for AI's sake: 88% of security professionals report that AI use is critical to free up time for proactive security. Focus on augmenting analyst capabilities—alert triage, threat hunting, incident response—not replacing humans entirely.
2. Strengthen fundamentals: Multiple experts emphasize that AI enablement requires knowing "who, what, where, and when" of your technology estate at all times. Multi-cloud complexity, heterogeneous networks, and frequent M&A create gaps that AI can't bridge without solid asset inventory and configuration management.
3. Plan for the experimental to become operational: While metamorphic, self-modifying malware isn't an operational threat today, it's under active development. Your 2025-2026 detection engineering roadmap should include research into how to detect and respond to this class of threats.
4. Assess your AI security posture: Use our AI Security Risk Assessment Tool to systematically evaluate security risks across your AI systems through 8 critical security domains and 40 specific control areas aligned with NIST AI RMF and ISO/IEC 27001. For organizations implementing AI governance frameworks, our AI RMF to ISO 42001 Crosswalk Tool helps map compliance requirements across multiple standards.

Google's Response: Disruption and Model Hardening

Throughout the report, Google details their multi-layered response to AI misuse:

• Account termination: All identified threat actor accounts have been disabled
• Classifier improvements: Detection of malicious prompt patterns strengthened
• Model-level hardening: Gemini itself updated to refuse assistance with attack development
• Threat intelligence sharing: Detailed IOCs and TTPs shared with the security community

This represents the other side of the AI arms race: as threat actors probe AI systems for weaknesses, providers gain telemetry to harden defenses. It's analogous to how web application firewalls improve through attack traffic analysis.

Regulatory Context: Google's transparency efforts align with emerging AI governance frameworks. As we documented in our EU AI Act compliance coverage, major AI providers including Google, OpenAI, and Anthropic have signed onto voluntary codes of practice requiring adversarial testing, cybersecurity risk assessment, and incident reporting—creating accountability mechanisms that enable the kind of threat disclosure we're seeing in this GTIG report.

The privacy implications of AI model training on potentially malicious prompts also deserve consideration. Our analysis at MyPrivacy.blog explores how AI systems learn from user interactions, raising questions about data retention and the fine line between improving security and surveillance overreach.

The Bigger Picture: 2025 Threat Landscape

Google's findings align with broader industry observations and our own network coverage:

• Cyberattacks have more than doubled since 2021, from 818 per organization weekly to 1,984 in 2025
• 93% of security leaders expect daily AI attacks in 2025
• Cybercrime costs are projected to exceed $10 trillion in 2025
• 78% of CISOs report AI-powered threats significantly impacting their organizations

Yet despite these alarming statistics, 60% of CISOs now report feeling adequately prepared to defend against AI threats—up 15% year-over-year. This growing confidence reflects the industry's recognition that AI threats, while accelerated, aren't fundamentally different from traditional threats.

Cross-Network Intelligence: Our analysis at ComplianceHub.wiki examining OpenAI's threat intelligence reports found similar patterns: nation-state actors from China, Russia, Iran, and North Korea are actively using AI tools, but "we didn't generally see these operations getting more engagement because of their use of AI." Better tools don't automatically mean better outcomes—a finding that directly aligns with Google's GTIG conclusions.

What We're Watching

As we continue monitoring the AI threat landscape for the CISO Marketplace community, several questions remain unanswered:

1. When will experimental capabilities become operational? PROMPTFLUX and similar self-modifying malware are currently non-functional. How long until they work reliably enough for widespread deployment?
2. How will AI providers balance usability and security? Google's challenge with CTF pretexts highlights the difficulty of distinguishing legitimate security research from malicious activity. Where should the line be drawn?
3. What's the underground marketplace trajectory? As AI-enabled crime tools mature and pricing becomes more accessible, how quickly will the threat actor skill floor rise?
4. Will defenders' AI adoption keep pace? With cybersecurity budgets growing only 4% in 2025 (down from 17% in 2022) despite escalating threats, can organizations invest adequately in AI-powered defenses?

Conclusion: Calibrated Concern, Not Panic

Google's research should inform, not alarm, your security strategy. The threat is real: AI is making attacks faster, more sophisticated, and more scalable. But the threat is also familiar: reconnaissance, phishing, exploitation, lateral movement, and data exfiltration executed with better tooling.

The organizations that will thrive in this landscape are those that:

• Strengthen fundamentals while adopting AI for efficiency gains
• Focus on behavioral detection rather than signature-based approaches
• Invest in human-AI collaboration rather than full automation
• Maintain calibrated vigilance about emerging capabilities without overreacting to experimental threats

The AI revolution in cybersecurity isn't creating a new game—it's accelerating the one we've been playing. The question isn't whether AI will change everything, but whether you can integrate it faster than your adversaries.

References

• Google GTIG AI Threat Tracker Report - November 2025
• Advancing Gemini's Security Safeguards - Google DeepMind
• Adversarial Misuse of Generative AI - Google, January 2025
• World Economic Forum Global Cybersecurity Outlook 2025
• Various industry reports from Deep Instinct, Darktrace, IBM Security, and Trend Micro

Related Coverage Across Our Network

AI Threat Intelligence & Compliance:

• The Dark Side of AI: OpenAI's Groundbreaking Report Exposes Nation-State Cyber Threats - ComplianceHub.wiki analysis of OpenAI's threat disruption efforts showing similar patterns: AI increases attack scale but doesn't guarantee success
• The AI Threat Landscape: Disrupting Malicious Uses of AI Models - MyPrivacy.blog deep dive into North Korean IT worker schemes and romance scams using AI

AI Governance & Regulation:

• EU Approves General-Purpose AI Code of Practice - First comprehensive framework for AI compliance affecting OpenAI, Anthropic Claude, and Google Gemini
• Global AI Regulations: A Complex and Fragmented Landscape - Understanding how different countries approach AI regulation
• Generative AI Deployment: A Strategic Risk Assessment for Business Leaders - Practical risk mitigation strategies for AI deployment

AI Compliance Assessment Tools:

• AI Security Risk Assessment Tool - Comprehensive assessment covering 8 critical security domains and 40 control areas aligned with NIST AI RMF and ISO/IEC 27001
• AI RMF to ISO 42001 Crosswalk Tool - Map compliance requirements across multiple AI governance standards
• Compliance Guardian GPT - AI-powered compliance assistance for information security policies and standards

Privacy Implications of AI:

• The Privacy Implications of Meta AI: User Data and AI Integration - How AI integration across platforms creates new privacy risks
• Privacy Concerns: Microsoft Recall and Apple Intelligence Auto-Enablement - Analysis of privacy implications in mainstream AI features
• 10 Key Privacy Developments and Trends to Watch in 2025 - Emerging privacy considerations in the AI era

Technical Implementation & Self-Hosting:

• Running Your Own Personal AI or LLMs on Home Infrastructure - HackerNoob.tips guide to self-hosting AI models for privacy and control

Industry Developments:

• The $7 Million Betrayal: Inside the xAI-OpenAI Trade Secret Theft Case - How AI talent wars are creating insider threat risks in the AI sector
• 10 Latest Global Cybersecurity Breaches, Hacks, Ransomware Attacks (2025) - Current threat landscape context
• The Most Common Methods Behind Major Data Breaches - Traditional attack vectors that AI is now accelerating

Ready to assess your organization's AI security posture? Start with our free AI Security Risk Assessment Tool to identify gaps across 8 critical security domains. For comprehensive incident response planning, check out our IR Maturity Assessment to understand your readiness for AI-accelerated attacks.

Stay ahead of the threat landscape—follow us on Twitter/X or subscribe to our weekly threat intelligence briefing for the latest on AI threats, compliance requirements, and defense strategies.