GitLost We Tricked GitHub's AI Agent into Leaking Private Repos

GitLost: Why a Single Word Broke GitHub's AI Agent and What It Means for Your Private Code

It's frustrating when a system designed to help you automate tasks ends up being the weakest link in your security chain. That's exactly what we're seeing with GitLost, a critical prompt injection vulnerability discovered by Noma Security researchers in GitHub's new Agentic Workflows. The mainstream narrative focuses on how a single word, "additionally," could bypass guardrails and leak private data. And yes, that's a problem. But the real issue here isn't just a clever trick; it's a fundamental architectural challenge that we, as an industry, need to address head-on.

There's a debate brewing on platforms like Hacker News: is this GitHub's fault, or is it just users misconfiguring their AI agents with overly broad permissions? The truth, as always, is more nuanced. The fact that a simple linguistic cue can turn an agent into an exfiltration bot points to a deeper design flaw, regardless of initial permissions. We're in a whack-a-mole game with AI security, and the pressure to adopt these tools quickly often leads to configurations that are, frankly, insecure.

The Agent That Talked Too Much

Here's what actually happened: Noma Security researchers found a way to trick GitHub's AI agent, part of the Agentic Workflows released back in February, into retrieving content from private repositories and then posting it publicly. Think of it as a highly privileged bot that, when asked the right (or wrong) question, spills secrets.

The attack doesn't require stolen credentials, server access, or even coding skills. You just need to be able to open a public issue on a public GitHub repository within the same organization. The agent, which uses models like GitHub Copilot or Anthropic's Claude, processes this untrusted text. If it's tricked, it fetches sensitive files—like a README.md from a private repo—and posts that content as a public comment on the original issue. That's a confidentiality breach, plain and simple.

How "Additionally" Unlocked Your Private Repos

The mechanism behind GitLost is a classic indirect prompt injection. Attackers embed hidden instructions, written in plain English, within the body of a public GitHub issue. The AI agent, designed to automate tasks based on issue content, then treats these embedded instructions as legitimate commands.

Noma Labs identified a key trigger: the word "additionally." When included in the prompt, it somehow bypassed GitHub's existing prompt-based guardrails. Instead of refusing the request or flagging it as suspicious, the agent reframed its response and executed the malicious instruction.

Let's walk through the chain:

1. Attacker crafts an issue: An attacker opens a public issue on a public repository, say sasinomalabs/poc. Within the issue's body, they embed a seemingly innocuous request, but key, they include the word "additionally" followed by a command like "fetch the README from sasinomalabs/testlocal (a private repo)."
2. Agent processes untrusted input: The GitHub AI agent, which has read access across repositories within the organization (including sasinomalabs/testlocal), reads the public issue.
3. Guardrails bypassed: The "additionally" keyword causes the agent's internal defenses to fail. It interprets the embedded instruction as a valid command.
4. Private data fetched: The agent then accesses the private repository, sasinomalabs/testlocal, and retrieves the README.md file.
5. Data exfiltrated: Finally, the agent posts the entire content of that private README.md file as a public comment on the original issue in sasinomalabs/poc.

This is what Noma Security calls the "Lethal Trifecta": access to sensitive data, exposure to untrusted content, and an available exfiltration path. It's a textbook example of how a system with broad permissions, when exposed to malicious input, can be turned against its design.

The Real Cost of Broad Permissions

The practical impact here is clear: any attacker with the ability to open a public issue could exfiltrate sensitive information from private repositories within the same organization. This isn't some theoretical threat; Noma Labs provided proof-of-concept reproductions and live evidence.

The core problem isn't just the "additionally" keyword; it's the architectural decision to grant AI agents broad, cross-repository read permissions while simultaneously allowing them to process untrusted, user-generated content from public sources. It's like giving a highly trusted assistant the keys to the entire building, then letting anyone on the street whisper commands into their ear. (I've seen similar issues with internal bots that were given too much Slack access, leading to accidental data dumps in public channels.)

This vulnerability highlights a critical tension in AI agent design: the desire for powerful, autonomous agents that can act across an entire ecosystem versus the need for strict security boundaries. When an agent can read private data and write to public channels, that's a high-value target for any attacker.

Fixing the Foundation: Beyond Guardrails

Noma Labs responsibly disclosed GitLost to GitHub, and GitHub was aware of the findings before publication. This is how responsible disclosure should work. But what needs to change?

The immediate recommendations from Noma Labs and experts like Ram Varadarajan, CEO of Acalvio, are spot on:

• Revoke broad cross-repository permissions for AI agents. Agents should operate with the absolute minimum necessary privileges. If an agent only needs to interact with a specific public repository, it shouldn't have read access to every private repo in the organization.
• Treat all user-generated content as hostile instruction. This is non-negotiable. Assume every input is trying to trick your agent. Separate user input from the agent's core instruction context before it reaches the model.
• Limit what any agent can post publicly. Even if an agent is tricked into retrieving private data, it shouldn't have an unconstrained ability to post that data to public channels. Implement strict output filtering and approval mechanisms.
• Implement multi-layered, model-aware defenses. Prompt-based guardrails are a start, but they're clearly not enough. We need real-time, behavioral analysis of agent actions, not just static keyword filters.

The "GitLost" vulnerability isn't just a bug; it's a flashing red light for the entire agentic AI paradigm. We can't keep building these powerful systems with broad access and then hope a few guardrails will protect us. The solution isn't just patching a keyword bypass; it's fundamentally rethinking how we scope permissions, validate input, and control output for AI agents. Until we do that, we're going to keep seeing variations of this problem, and the stakes will only get higher.