Part 2: agents need infrastructure, not just models
How we build
vLLM AFD Plugin Cuts DeepSeek-V3.2 Response Time by 47% (vLLM Project)
Serving a Mixture-of-Experts model forces a bad trade. Attention layers are bound by memory bandwidth and expert FFN layers are bound by compute, so running both on the same GPUs wastes capacity on whichever workload loses. vLLM’s AFD plugin splits the two into separate services that are scaled on their own. It reports the response time for DeepSeek-V3.2 falling by 47 per cent. ByteDance, StepFun and Huawei had already shipped versions of the design; the plugin makes it an open, pluggable add-on to the most widely used inference engine.
Architecture: the same workload can cost you half as much again purely through where it has been placed, and none of that is visible from the application above it. Splitting the serving layer this way is becoming the default for large MoE deployments, which means a serving topology chosen a year ago is now a standing cost decision nobody is revisiting.
Ask who owns your inference serving topology. Then ask when it was last reviewed against how the models you actually run are built.
How we assure
Hermes Agent Ships iron-proxy Credential Firewall (Nous Research)
Every capable agent needs credentials. Most frameworks leave that as the developer’s problem. Hermes Agent now ships iron-proxy, an egress firewall that keeps real API keys off Docker sandboxes entirely. The sandbox holds opaque proxy tokens, all outbound traffic runs through a daemon on the host, and the daemon swaps the token for the real credential before the call goes upstream. The real key never crosses the container wall.
Security: one prompt injection, one compromised dependency or one rogue tool call can dump the environment variables out of a sandbox in seconds. That is the whole attack. This closes the gap in the infrastructure rather than trusting the model to behave, which is the only version of the control that survives contact with an adversary. Neither this nor the vLLM work came from making a model smarter, and both are the sort of systems engineering that decides whether an agent can be trusted with real access.
Find out where your agents’ credentials actually live at runtime. If they are environment variables inside the sandbox, one injection reaches all of them.