Unveiling the Hidden Threat: How AI Becomes the Vulnerable Attack Surface

Boards of directors are increasingly pushing for enhanced productivity driven by large-language models and AI assistants. However, the functionalities that provide such value—like browsing live websites and maintaining user context—also introduce significant cybersecurity risks. It’s essential for businesses to understand that with every new capability, the potential attack surface expands, leading to vulnerabilities that could be exploited.

Tenable researchers have recently unveiled critical insights under the banner of “HackedGPT,” showcasing vulnerabilities associated with indirect prompt injection. These vulnerabilities can result in data breaches and malware persistence. While some issues have been addressed, others remain a threat, as highlighted in a recent advisory.

To mitigate these risks, organizations must implement governance and controls that treat AI as they would any sensitive device or service, necessitating rigorous auditing and monitoring.

Understanding the Risks of AI Assistants

Tenable’s findings call attention to how AI assistants can inadvertently become security liabilities. Indirect prompt injections can embed malicious commands within the web content that these assistants access, which could lead to unauthorized data retrieval without user consent. Similarly, a front-end query may introduce harmful instructions that compromise security.

The repercussions of these vulnerabilities can be extensive, demanding incident responses, legal reviews, and reputational damage control.

Research has already demonstrated that AI assistants can leak sensitive information through injection techniques. This highlights the urgent need for AI vendors and cybersecurity experts to continuously address and fix emerging problems.

Recognizing the pattern is crucial: as innovative features expand, so do the potential failure modes. By treating AI assistants as active, internet-facing applications rather than mere productivity enhancers, organizations can bolster their defenses against these risks.

Governing AI Assistants Effectively

1) Establish an AI System Registry

Begin by cataloging every AI model, assistant, or agent in use—whether public cloud, on-premises, or SaaS. This inventory should detail the ownership, purpose, capabilities (such as browsing), and data domains accessed. Without this, shadow agents—those operating without oversight—can pose significant threats to security.

2) Distinct Identities for All Users

It’s vital to differentiate between human users, service accounts, and AI agents. Those that interact with external websites and process data must have unique identities governed by a zero-trust framework. Mapping interactions can provide a clear trail of accountability, ensuring clarity around who does what.

3) Opt-In for Risky Features

Enable browsing and autonomous actions of AI assistants only on a per-use basis. For client-facing assistants, consider short retention periods for any data collected unless legally justified. For internal use, restrict AI assistants to isolated projects backed by rigorous logging, applying data-loss-prevention measures as necessary.

4) Monitor Like Internet-Facing Applications

Apply robust monitoring strategies similar to those for external applications:

• Capture structured logs of actions and tool calls.
• Set alerts for unusual activities, such as unexpected browsing patterns or access attempts beyond policy limits.
• Include injection tests in pre-launch assessments.

5) Build Human Capability

It’s essential to train developers and engineers in recognizing symptoms of potential vulnerabilities. Encourage users to report any odd behavior, like an assistant summarizing unvisited content. Normalizing quarantine procedures for assistants after suspicious incidents can also strengthen defenses.

Decision-Making Framework for IT Leaders

Make your evaluations based on essential queries:

• Which assistants have browsing capabilities? Such features can easily become pathways for data breaches.
• Do agents possess distinct identities and audit trails? This helps clarify responsibilities when issues arise.
• Is there a comprehensive registry of AI systems? This supports better governance and budget management.
• How are third-party integrations managed? Given their history of security issues, apply strict access controls.
• Are new vulnerabilities addressed promptly by vendors? Given the pace of tech changes, responsiveness is crucial.

Navigating Risks and Costs

• Hidden Costs: AI assistants that browse and retain information can lead to unexpected compute and storage expenses. A comprehensive registry helps manage these costs.

• Governance Gaps: Ensure the governance frameworks account for the distinct nature of AI system interactions to avoid oversight.

• Security Risks: Indirect injections are often subtle and can manifest through various media formats. Continuous monitoring is necessary.

• Skills Gap: Emphasize training, focusing on threat modeling and injection testing for AI systems.

• Evolving Posture: Stay alert for new vulnerabilities, as recent fixes can rapidly become outdated.

The Bottom Line

For executives, the core lesson is clear: AI assistants should be treated as powerful, networked applications that are integral yet potentially vulnerable. Establishing a systematic registry, separating identities, constraining risky features, and logging relevant interactions can significantly enhance your organization’s resilience.

With the right protective measures in place, AI can not only drive efficiency but also safeguard your organization against emerging threats.

Embrace these practices, and inspire a culture where technology enhances security rather than undermines it.