u/WhichCardiologist800

man, you should use https://github.com/node9-ai/node9-proxy to protect pii and dlp

I counted by quarter. Confirmed incidents are roughly flat, 3 to 6 per quarter. Research disclosures are the part climbing: 1, 1, 2, 3, then 5 in Jul-Sep 2026. More people are looking at agents, and what they find tends to show up as incidents later. 20 cases is a small sample though, so I wouldn’t call a trend on it yet.

On Railway: in the PocketOS case Railway itself wasn’t breached. The agent used a Railway token that had far more access than it needed.

Totally agree!!! unfortunately, users will understand it after the first time they will face with breach

Thanks, the second pass was honestly the most useful part. I’d have shipped a couple of wrong numbers without it.

On pinning: it would have saved anyone on postmark-mcp who stayed at 1.0.15. The bad line only showed up in 1.0.16. So the weak spot isn’t the pin, it’s the bump you approve without reading the diff. Do you actually diff MCP server updates before moving the pin, or is that where it gets skipped?

And yeah, the over-permissioned ones are the most avoidable. In the PocketOS case the agent found a token in an unrelated file that was created for one small job but could delete anything.

Good point on the clean result. Today it says the agent configuration was inspected and nothing needs fixing, but it doesn't say what it can't see, like what a script does at run time. Noted.

Yes, and the gate wasn't always turned off on purpose. In some repos the gate is a label, and an issue template adds that label by itself, so it's open to everyone without anyone flipping a toggle.

Quick check for anyone reading: grep -rn allowed_non_write_users .github/workflows/

A "*" next to a github_token means anyone can drive the agent.

Not yet, that is exactly what I was hoping someone here had done.

But your numbers make the case. Going from .68 to .98 with a few examples means the task is very learnable. A yes/no “does this text answer the question” check is the kind of decision a small local model can learn, using the API’s answers as training labels. Same accuracy, no round trip, and the data never leaves the device.

The ML itself is not the hard part. Detection plus tracking on a belt is a solved problem in a notebook.

The hard part is the box, and more precisely thousands of boxes. Fixed compute, often no GPU, a frame rate the belt dictates, products and lighting that keep changing per site, and models that have to improve and be updated without you being there and without shipping the images anywhere.

So yes, it runs in a box. The work is keeping it accurate in all the boxes a year later.

Agree on frequency, but the problem is the loop, not the single call. 200ms is fine once. Five chained decisions with retrieval in between is over a second, and that is on a good day for the API.

Privacy is also an economics problem. Streaming images from thousands of endpoints to a cloud API gets expensive long before compliance gets involved.

Hybrid, yes. But anything reacting to the physical world runs on the device. The cloud gets training and aggregation. The decision itself should not need a network.

lol at everyone accusing each other of being bots in this thread. But seriously if you run agents in production using a standard user's permissions you are asking for a breach. They absolutely need their own dedicated service accounts so your logs actually show 'agent_x' did something instead of a real employee. As for the kill switch, u/radim11 got it right. Killing the agent process is pointless if it already grabbed a session token. SOC needs a big red button to just nuke the token at the API gateway

Don't jump into cloud just yet if you're already feeling overwhelmed, it's a completely different beast. Your idea about doing CTI from your current IR investigations is spot on. That's exactly how people build their resumes without burning out. Start documenting the TTPs you see every day and write internal intel reports. Once you get comfortable with that, you can start actively hunting for those same threats in your EDR. Just like that you've got IR, CTI, and hunting experience all from the exact same job. Save the cloud certs for next year.

If it is a classifier bypass, Anthropic runs a disclosure program for exactly that. Worth getting it to them before the NDA round.

Makes sense. Would be interested to read it when it's out.

That lines up with what we saw from the other side. In CI the agent doesn't verify where an instruction came from either: text in an issue body gets the same weight as the prompt the maintainer wrote. Did you see any difference in the 90% when the "Done" came from a different agent versus a human-looking comment?

Right now, node9 checks the hostname the agent connects to. We’re planning to extend that to check target URLs embedded in relay requests too. In the example from my post, the relays were blocked because they weren’t approved destinations.

Love this architectural approach. An LLM shouldn't be the one making security or routing decisions. It should just propose actions, while the larger system holds the authority. That's exactly the mindset behind node9! Did you build a custom system to wrap your LLMs, or are you using existing frameworks?

100% agree. Treating every hop as a separate trust boundary is the way to go. Explicit allowlists are actually the default behavior I built into node9. How are you currently handling the logging for this?

Exactly! A sandbox protects the local machine, but it’s completely blind to where the agent is sending data. Glad to see I am not the only one thinking about this

Spot on. That is exactly why we need to validate the request path. Letting an agent quietly change the trust model is a security nightmare. Have you dealt with this in your own projects

Here’s the code for anyone who wants to inspect the policy engine or try it:

github.com/node9-ai/node9-proxy

Happy to discuss the setup or test other cases.

Here’s the code for anyone who wants to inspect the policy engine or try it:

github.com/node9-ai/node9-proxy

Happy to discuss the setup or test other cases.

The hardest part? Traditional monitoring is completely useless.

In standard software, a critical failure is a 500 error or a crash. With AI agents, a critical failure is often a perfectly executed HTTP 200 OK where the agent confidently hallucinates a payload or gets stuck in a loop politely apologizing to a broken API.

Your APM dashboards show a sea of green, but the agent is failing silently.

You basically have to shift to specialized observability and policy tools that actually monitor the agent's logic and outbound requests. We recently put together a comparison of the different platforms built specifically for this, if anyone is trying to figure out how to actually monitor these things: https://node9.ai/compare

Here’s the code for anyone who wants to inspect the policy engine or try it:

github.com/node9-ai/node9-proxy

Happy to discuss the setup or test other cases.

It’s the classic bait-and-switch. They release an amazing model (Sol 5.6 at launch), get everyone hooked, quietly lobotomize it to save compute, and then drop Sol 6, which is basically just Terra wearing a fake mustache. You end up burning twice the tokens just trying to fix its mistakes, completely negating the 'lower cost'. Opus 5.5 is looking better and better every day.

that ןs really useful, thanks. the consolidation part is what i didnt expect.

so you are doing it by hand, per project? i have got maybe four repos going and i've never once opened that folder. curious how often you actually find a contradiction, and whether you do want something that just listed them all in one place with what changed since last time. or is that overkill for what is basically a folder of markdown.