Most of us have the "don't commit secrets" reflex drilled in. Pre-commit hooks, secret scanners in CI, .env in .gitignore. Good. That covers the repo.
It doesn't cover the context window.
The gap: the repo is not the boundary anymore
When you work with an AI coding assistant, what leaves your machine isn't just the file you're editing. It's whatever you paste, whatever the extension decides is relevant, and whatever the agent reads while it's "helping." The security boundary now includes the editor extension, the chat history, any telemetry, and every service that processes that context on the way to the model and back.
Secret scanners watch commits. Nothing in a typical setup watches what you paste into a chat panel at 6pm while a deploy is failing.
Here are the usual ways a secret ends up in context without ever touching git.
1. The failing test that prints its config
An integration test fails. The assertion message helpfully dumps the client object, which includes the connection string with the password inline. You copy the whole test output into the assistant and ask "why is this failing?"
The question was about the assertion. The payload included the credential.
2. The environment dump
$ env | sort
...
AWS_SECRET_ACCESS_KEY=...
DATABASE_URL=postgres://app:...@db.internal:5432/prod
STRIPE_SECRET_KEY=sk_live_...
"Here's my environment, why can't the app find the config?" is a completely normal debugging move. It's also the fastest way to hand over every key on the box in one paste. Agents that can run shell commands can do this on their own if you let them.
3. Logs that log too much
A verbose HTTP client logs request headers. One of them is Authorization: Bearer .... You paste 200 lines of log to get help with a timeout, and line 143 is a live token.
4. The copied snippet
Someone shares a "working example" in Slack with the key still in it. You paste it into the assistant to adapt it. Now it's in two systems you don't control instead of one.
(These are illustrative scenarios, not specific incidents, but if you've debugged anything under pressure you've probably seen at least one.)
What actually helps
- Minimize what you send. Paste the relevant ten lines, not the whole log. Ask about the error, not the environment.
- Redact before pasting. Mask tokens, connection strings and headers. A small shell function or editor snippet that scrubs obvious patterns is worth having.
- Scope your credentials. If a dev key does leak, it should only be able to touch dev. Short-lived, least-privilege keys shrink the blast radius.
- Check how your tool handles data. What's retained, for how long, whether it's used for training, what the extension sends automatically, where telemetry goes. These are often separate settings and separate policies. One "privacy" toggle rarely answers all of them.
-
Restrict what agents can read. If your assistant can open files and run commands, keep
.env, credential files and secret stores out of its reach.
If it already happened
Deleting the chat message is not remediation. You don't know what was logged, cached or forwarded before you hit delete. Treat it like any other exposure: rotate the credential, revoke the old one, check access logs for use.
The reframe
Secret management used to be a version control problem. With AI tools in the loop, it's a data-flow problem: anything that reaches the model's context has, to some degree, left your control.
The safest secret is the one that never enters the context in the first place.
How does your team handle this today? Do you have any tooling that checks what goes into an AI assistant, or is it all down to developer habit?
Top comments (109)
The CI/CD case is where this gets really interesting. Microsoft documented a Claude Code Action vulnerability where prompt injection through untrusted GitHub content could lead to reading
ANTHROPIC_API_KEYfrom/proc/self/environ. The Bash path had environment scrubbing, but the Read tool followed a different path and bypassed that isolation boundary.That’s the part I think teams need to pay attention to: protecting secrets in Git or filtering terminal output doesn’t help if another tool can reach the same credentials through a different path. I’d treat the agent’s entire tool surface as part of the security boundary, with explicit access controls and isolation between untrusted input and sensitive runtime state.
The CI/CD boundary is definitely where this gets critical. I love the canary secret idea for runtime data paths, but for source code, static analysis acts as the ultimate pre-execution canary.
For example, in Web3 smart contracts, I wrote a Slither detector that flags hardcoded Hardhat test addresses or plaintext API keys directly in the codebase. It catches the leak at the syntax level, long before the CI/CD pipeline even has a chance to expose it to an AI agent's Read tool or the blockchain. Combining runtime canaries with compile-time static checks is the real defense in depth.
Agreed, those checks complement each other well. Static analysis catches what ends up in the code, while runtime testing catches what gets exposed through the environment, tools or logs. The important thing is not to confuse passing one with being safe overall. In the scenario Sina described, the secret never needed to be hardcoded in the repo for the agent to reach it.
Exactly. GitHub Actions OIDC is a good example here: you can remove long-lived cloud credentials from GitHub Secrets, but a job with
id-token: writecan still request an identity token and exchange it for cloud credentials if the trust policy allows it. If an AI agent processing an untrusted PR can reach that path, the problem isn’t a leaked key in the repo anymore; it’s what the job is authorized to obtain at runtime. I’d keep the agent in a separate, unprivileged job and make the privileged deployment job consume only validated artifacts, with no untrusted PR content passed back into its execution context.This is a really important distinction. OIDC removes the need to store long-lived credentials, but it doesn't remove the risk of granting an untrusted job access to the identity flow in the first place. Separating the agent from deployment permissions and passing only validated artifacts across that boundary feels like a much stronger design than trying to sanitize everything the agent might see. Least privilege still matters even when there are no static keys to steal.
Slither is useful for catching source-level issues, but the harder cases are secrets that only exist at runtime. A contract can pass static checks while the CI job compiling or testing it still has access to cloud credentials. I’d also be careful with Hardhat test addresses: those aren’t necessarily secrets themselves, but private keys, deployer credentials or API tokens embedded alongside them can be. The key question is whether the agent can access anything privileged while processing untrusted code, not just whether the repository passes a scan.
That Read tool detail is a great example of why checking only one execution path isn't enough. You can harden shell access and still have another tool reach the same sensitive data. The agent's permissions and isolation need to cover the whole tool surface, not just the commands we expect it to run.
This one hit close to home.
We had all the "don't commit secrets" reflexes and still caught ourselves doing the
env | sortpaste you describe. So we changed how secrets reach the machine in the first place.Nothing real lives in
.envfiles on our laptops anymore.Secrets sit in a self-hosted vault, and anything that needs them runs through a tiny wrapper that hands them to that one process. The terminal, and anything reading it, only ever sees the command.
We also stopped printing secrets, even to check them.
Earlier today we had to confirm a key in the clipboard was the same one in the vault We compared the length and the first few characters of a SHA-256, and never looked at the key itself, It feels a bit paranoid until you remember how much ends up in scrollback.
Two smaller lessons that cost us some time:
ps.$VARinside double quotes gets expanded by the outer shell. We once sent the literal text instead of the value, and it took a while to notice, anything complex now goes into a small script.And honestly, the guardrail I like most is the boring one, our coding assistant can't write to a secret store without one of us saying yes, every single time.
So, to your question: still mostly habit, but habit with a few doors locked.
This is a great example of making the safer path the default instead of relying on people to remember every time. I especially like the wrapper approach and the approval gate for secret store writes. One small thing I'd add is making sure the wrapper doesn't accidentally expose secrets through debug output or child process environments. It's amazing how many different places a value can leak without anyone explicitly printing it.
Agreed, and the child environment is the one we haven't fully solved.
Every process the command starts inherits those variables. What we do is keep the blast radius small.
Our vault is a self-hosted Infisical. The wrapper logs in with a machine identity and runs
infisical run --path=<folder>, so a command only gets the secrets of its own folder, not the whole vault.Debug output bit us in a different way. Calling the wrapper with the shell as separate arguments once printed every injected value. Now complex commands always go into a script file, called with one single-quoted string.
"Nobody printed it" really isn't the same as "it didn't leak".
the child env is the part that's actually fixable, and folder scoping doesn't touch it. infisical run chooses which values get injected, not what the child hands on to its own children. exec the command directly with an explicit allowlist (env -i plus the three or four vars it truly needs) and there is nothing inherited left to leak. if a shell sits in the path, BASH_ENV comes along with it and a sourced file can flip tracing on without you typing set -x anywhere.
and check it from inside the child,
tr '\0' '\n' < /proc/self/environ, not from the wrapper's log. the wrapper is the one process holding plaintext, so its own error paths are leak surface too, same class as the separate-arguments accident.This is a good distinction. Scoping which secrets get injected is only half the job if the process can pass them all to anything it launches. An explicit environment allowlist sounds like a solid next step, especially for commands that don't need the full set. I'd also test the child process environment and the wrapper's failure paths, since those are easy places to assume isolation is working without actually verifying it.
Agreed, and Kaj's point about the child environment is the one that matters most.
Our wrapper handles it in two ways now:
psnor in the child's environment;PATH,HOMEand a few more) plus only the secrets that command needs.And we check it from inside the child, reading the variable names from
/proc, not from the wrapper's own log :)That's a solid improvement. Keeping the vault token out of the child entirely and starting commands with an empty environment closes two different exposure paths. I also like that you're verifying from inside the child rather than trusting what the wrapper reports. It sounds like the wrapper is evolving from just a secret injector into an actual security boundary. Have you found any edge cases with commands that need a larger environment, like build tools or nested scripts?
Yes, and that's exactly why the empty environment is opt-in rather than the default
CI is the main case, jobs rely on
GITHUB_*,RUNNER_*and their own variables, so there the command keeps the caller's environment while the vault side stays locked downFor local commands that need more than the basics, we pass an explicit allowlist of extra names, every variable the command sees is one we chose
the edge case isn't a bigger environment. it's that the build tool writes your command line into the environment for you.
i ran it in a clean shell instead of guessing:
env -i PATH=... make FOO=sentineland FOO shows up in the recipe's env. it also lands in MAKEFLAGS as-- FOO=sentinel, so a nested$(MAKE)still has it. the wrapper never saw the variable, because argv is not the thing it filters.so an allowlist that answers "which variables did we set" is enforcing on the wrong side. the only question worth asking is whether the child's own view, read from inside the child, contains anything we didn't put there. same run,
tr '\0' '\n' < /proc/self/environ, diff it against the manifest, fail on any extra line.the other edge is a tool that needs a capability instead of a value.
ssh-addunderenv -isays "could not open a connection to your authentication agent". export one path, SSH_AUTH_SOCK, and it connects. nothing in that variable looks like a credential, and it is the credential: a socket you can reach is a signing oracle. so "allowlist the bigger set" quietly gives away the boundary it was there to draw.which is why failing closed on a missing manifest entry matters more than the allowlist being complete. the test isn't a command that passes, it's deleting one entry and confirming the command refuses to run. if it still runs, the allowlist is documentation.
agreed that the wrapper is turning into a real boundary. boundaries are only boundaries if refusing is the default.
Redaction before pasting is the same class of control as "don't commit secrets" — it works until the deploy is on fire and you paste the whole log at 6pm. The more durable lever is on the agent side: if the assistant can run
env | sortor read.env, a human redaction habit is the only thing standing between you and a full dump, and that habit is exactly what pressure erases. Regex scrubbing is also whack-a-mole since providers rotate token prefixes and opaque keys slip through, so I'd rather bound the blast radius with short-lived scoped keys and log egress from whatever the extension sends — because delete never un-sends anything.Spot on. The "6pm failing-deploy" is the ultimate threat model because human habits inevitably break under pressure. Regex scrubbing is indeed whack-a-mole.
This is exactly why shifting left to automated, pattern-based static analysis in the CI/CD pipeline is non-negotiable. The linter doesn't get tired at 6pm. In Web3, I built a custom Slither detector specifically to catch hardcoded test keys and CEX credentials during the PR check. It ensures that even if a developer accidentally pastes the wrong config, the build fails before it ever reaches an AI agent's context or the mainnet. Automation is the only reliable safety net for human error.
The CI checks are definitely useful, especially for catching secrets before they make it into the repo or a deployment. The gap I'd still worry about is what happens at runtime, though. A secret can be injected into the environment or returned by a tool without ever touching the code being scanned. That's why I think CI scanning and restricting what the agent can access need to work together.
Yeah, that's the key distinction. Redaction is useful, but it shouldn't be the main thing standing between an agent and production credentials. I'd rather limit what it can read in the first place, then add egress scanning and short-lived scoped credentials as backup layers. Logging what actually leaves the extension would be especially useful, assuming the logging itself doesn't become another place where secrets accumulate.
Point 2 is the one that worries me most with agents rather than chat. You don't even have to paste env yourself; the agent can run it on its own while "debugging", and the output lands straight in context. Scrubbing the agent's shell environment helps, but as the /proc/self/environ case above shows, a file-read tool can walk right around it. The more durable fix is making sure there's nothing long-lived to find: short-lived, narrowly scoped credentials locally, so anything that does leak expires before it matters. A pre-commit hook for the context window would honestly be a great tool.
Exactly. Once an agent can read files directly, restricting its shell environment alone isn't enough. Short-lived, scoped credentials reduce the damage if something slips through, and I like the idea of a pre-commit hook for context. It would need to cover more than pasted text, though, since tool reads and command output can enter context without the user explicitly adding anything.
Agreed, the hook would have to sit between the tools and the model, not at the input box. Something like a redaction layer on every tool result (file reads, command output) that masks anything matching secret patterns before it ever reaches the context. That would make a fun open-source project.
The prevention path should include an egress test, not only a pre-commit scanner. Teams can seed a harmless canary secret in a nonproduction environment and verify that logging, prompts, and support exports all redact it consistently.
I like the canary idea because it tests the actual data path instead of assuming the scanner works. You could seed a fake credential and check whether it makes it into prompts, logs or exported traces. I'd also test different ways the agent can access data, since catching it in terminal output doesn't guarantee the Read tool or extension handles it the same way.
A small path-by-sink matrix would make those differences visible: terminal, file read and extension output crossed with prompt, logs and traces. Use a unique fake canary per path so the test can identify where a leak originated, and verify the test harness itself detects a deliberately unfiltered control.
I like the path-by-sink matrix, especially using a different canary for each path. That makes it much easier to trace where the leak happened instead of just getting a pass or fail. And testing the harness with a deliberately unfiltered control is important too, otherwise a green result might just mean the test never caught anything in the first place.
The per-path canaries are especially useful if the matrix includes the boundary, not just the client: terminal paste, editor extension, browser upload, CI log, and vendor API. Each path then has an accountable owner when a canary escapes.
That's a useful addition. Including the boundary makes the test much more actionable because it shows not only where the data leaked, but which component owns that part of the path. The vendor API is particularly worth including since the data has already left our environment by that point. Having an owner for each boundary also makes it easier to turn a failed canary test into a concrete fix rather than just another security finding.
I ran this on my own MCP server tonight, exactly as this thread assembled it: one key-shaped canary and one prose canary per client-supplied field (@alexshev 's path-by-sink idea, @kaj_ilands 's "a fake key tests your regex, prose tests the logger"), then read back from CloudWatch and both DynamoDB tables with the raw AWS API, not through the app (@kaj_ilands 's point about verifying below the redaction layer).
Result: the three free-text fields masked every key-shaped string and kept the prose, the session id came back as its hash, and the user-agent header came back verbatim with the fake key in it. The policy promised masking for the whole log line; the code masked three fields. One-line fix, one test, redeployed, resent, read back masked. The prose canary was the control that proved the read-back worked at all.
The full run, with the steps so an agent can repeat it on your service, is here: worklore.dev/s/2026-10-30-a-canary...
the user-agent line is the one worth keeping. nothing in the masking code was wrong there. the list was. the policy said the log line, the code said three fields, and that gap is invisible until a sentinel goes somewhere the list didn't know about.
so a canary per known field can't catch this class of bug, because the bug is the field you forgot to name. the version that does: one sentinel in every client-supplied channel at once, including the ones nobody files under data (referer, cookie, x-forwarded-for, the page's query string), then read the store cold and search for the sentinel instead of checking fields one by one. if it turns up anywhere, the field list was never the boundary. the sink is.
and the prose control is the part most runs skip. a key-shaped canary passes on a logger that masks by pattern; prose doesn't. keep it as the gate on every field you add, not a one-time check.
The worst one I've run into never went near a chat window. On a site I took over, the Supabase service role key sat in the client JS bundle, so anyone who opened devtools had a key that skips RLS.
Leaks into AI tools worry me too, but I'd grep the build output first. Does anyone here run a secret scan on what actually ships, or only on the repo?
That’s a great point. Scanning the repo is only part of the picture because secrets can end up in generated bundles, build artifacts, or source maps even if the repo scan passes. I’d definitely add a secret scan on the actual build output or release artifact before deployment. The Supabase service role key is a particularly serious example because it’s not meant to be exposed to client-side code at all.
Good shout on source maps, those get forgotten a lot. Scanning the build output catches them anyway, which is one more reason to scan what ships rather than what's committed.
Exactly. Source maps can expose more than people expect, especially if they contain original source or embedded configuration. I think the safest approach is to scan the final deployment artifact, including generated files and source maps, rather than relying only on what the build process was supposed to include.
Agreed, the deployed artifact is the only version that matters. Thanks for the back and forth, good post.
The tricky part is that developers may not even realize what the agent can access. Redacting pasted logs helps, but limiting the agent’s file and command permissions seems like the more reliable fix. How do you balance that with giving it enough access to debug effectively?
I’d start with read access limited to the project and only expand it when a debugging task actually needs more. For commands, I’d be especially careful with environment dumps, broad filesystem access and anything that can reach production systems. If the agent hits a permission boundary, that’s a good point to ask for approval rather than giving it blanket access upfront. It might take an extra step sometimes, but I’d rather have that friction than discover later what the agent could read all along.
The habits that hold up are the ones that cost nothing in the moment. Keep production credentials off laptops entirely, so the worst thing in a paste is a dev key. Have the app fail with a message that never prints its config or connection string. Load secrets through a wrapper that injects them at run time, so they are not sitting in env output to be pasted. And treat the 6pm failing-deploy case as the real threat model, since that is when people skip the redaction step.
This is probably the most practical approach because it removes opportunities for mistakes instead of relying on people to catch them under pressure. Keeping production credentials off developer machines makes a huge difference, and avoiding secrets in error messages should be standard practice regardless of AI tools. Then even if someone pastes a full log, there's much less damage to worry about.
Agreed. One cheap way to enforce the error message rule is a test that deliberately triggers a failure and asserts the output does not contain anything shaped like a connection string or key. It turns a habit into something CI checks.
Nice, and that's a pretty low-effort test to add to an existing pipeline. I'd probably include a known dummy secret in the test config and assert it never appears in the error output, alongside checks for common credential patterns. It won't catch everything, but it makes sure a future change doesn't quietly bring the problem back.
This really got me thinking. I use coding agents pretty extensively, and while I’m careful about keeping secrets out of my repositories, I haven’t given nearly as much thought to what happens when an agent reads a local .env file, runs a command that exposes environment variables, or encounters credentials buried in logs or tool output. Those values could end up being sent to an external model provider as part of the agent’s context, without ever being committed to Git or explicitly shared by the developer. I’m especially curious about how effectively current coding agents detect and redact sensitive information before it leaves the local environment, and whether that protection extends across all their tools and MCP integrations. Definitely something I want to investigate in my own setup. Thanks for highlighting a security risk that’s easy to overlook!
Thanks, glad it got you thinking about it! I think the MCP part is particularly worth investigating because security controls that work for the main agent don't necessarily cover every tool or integration it can call. I wouldn't assume sensitive-data redaction is consistent across file reads, shell output, extensions and MCP servers without testing each path. A harmless canary secret is a good way to check what actually leaves your environment. It's one thing for a tool to claim it filters sensitive data, and another to verify that the value never reaches the provider.
Good to be aware of this! Just wondering:
Suppose your creds/secrets did get sent to OpenAI's or Anthropic's (or other AI vendor's) server - are there examples where those creds/secrets have actually got abused - and in what way?
Could they become part of training data, hence become 'public', or abused in another way?
Are there known cases where 'abuse' in whatever form did happen?
P.S. okay well, so I asked AI about it ;-) ...
Yes, there could be various ways stuff could then be leaked, hacked or abused - maybe low probability, but not completely theoretical ...
In a scenario like this, rotating/changing those creds/secrets (as you suggested) is the only sensible thing to do :-)
That's a fair question, and I think it's important to separate a confirmed incident from a plausible risk. I wouldn't claim that every secret sent to an AI provider has been abused or turned into training data. The risk is broader than training, though: depending on the service and configuration, data may be retained in conversation history, logs, support systems or other operational systems, and could potentially be exposed through an account compromise or access-control failure. There have also been real-world cases of sensitive data exposure in AI services, even if that doesn't mean a particular leaked credential was subsequently exploited.
For credentials, I'd treat exposure as compromise regardless of whether there's evidence of misuse. Revoke or rotate the secret, check its usage logs, and investigate where copies may have been retained. Rotation doesn't erase the old copies, but it makes them much less useful.
Yeah it was more of a "what if" question - the (only) right thing to do is definitely to replace (and not even rotate) those secrets ...
Agreed. Once a credential has crossed a boundary you don't fully control, replacing it is the safest response. I'd also revoke the old one explicitly and check its usage history before and after revocation. Even if there's no evidence anyone used it, there's no real benefit in continuing to trust a credential whose exposure you can't rule out.
Yeah you're right - not just replace, also revoke!
Some comments may only be visible to logged-in visitors. Sign in to view all comments.