👋 Everything about EKS & AI Infrastructure Newsletter "#91" ☁️❤👨‍💻

Dear EKS & AI Infrastructure enthusiasts,

Welcome to Everything about EKS & AI Infrastructure #91.

Picture an AI agent running in a pod on your cluster. It can open a shell, call an API and read a secret. The code it’s about to run didn’t exist a minute ago, and nobody has reviewed it. Nothing has gone wrong yet. The question is what it could touch if something did. That question runs through this edition. Most of these pieces aren’t about what agents can do. They’re about the boundaries we build around them, and the platform work that keeps everything underneath fast and cheap.

Some of it is about fencing agents in. A free KubeCon workshop builds the full stack on EKS: an AI gateway so agents never hold API keys, an MCP gateway for tools, and microVM sandboxes for code nobody has checked. nono goes one level deeper and locks each agent down at the Linux kernel. AWS shows a self-healing EKS setup where every fix needs a second, guardrail agent to say yes. And a new observability guide traces every tool call, token and dollar an agent spends.

Some of it is about not wasting hardware. i14 serves a model whose memory stays fixed at 29.5 MB however long the conversation gets, and found that an 11-second recompile was eating almost half its decode time. SOCI cut a 13.2 GB image pull from almost four minutes to about six seconds. ECS now lets several jobs share one GPU. And TorchNeuron turns Trainium into a normal PyTorch device, so many models only need .to('neuron').

Then there are the parts nobody posts about, until they break. Sri Saran Balaji Vellore Rajakumar, a founding engineer of Amazon EKS, walks through control plane outages where every component was working exactly as designed. Emmanuel Akuffo and Yashaswi Kakumanu share what broke when they put LLMs on EKS, and how they fixed it.

And to close, a personal note: three talks done in Atlanta, and a Code Talk at re:Invent coming up.

Performance Engineering in Modern AI Systems 🌩️

🌩️ Beyond the KV cache: Training and serving i14’s Laplace 3B linear attention model on AWS Trainium2 with native PyTorch

Kevin Shabahang, Massimo Bardetti (i14), Iman Abbasnejad, Xiaodong Yu, Daniel Wirjo (AWS)

Most language models keep a note of every word in the conversation so far. This is called the KV cache. The longer the conversation, the bigger the notes get, and the fewer users one GPU can serve at once. i14 built Laplace 3B to work differently. Instead of keeping every note, it keeps a running summary of fixed size, like a reader who remembers the gist of a book rather than every page. That summary is 29.5 MB per conversation, whatever its length. The authors say that’s about 82 times smaller than a comparable KV cache at 8,192 tokens. Making the prompt four times longer added only 0.5% to the cost of each output token. The trade-off is that a summary can forget details, so recall needs testing for each use case.

The team trained it on one Trainium2 machine (trn2.48xlarge) using plain PyTorch. One change stands out: the chips were sending each other far more data than they needed to. Reorganising that traffic cut it 24 times, from 10.87 GB to 0.45 GB per step, and made training 1.63 times faster, at about 120,000 tokens per second.

For serving, they used standard vLLM with no changes to vLLM itself. One machine reached 11,544 tokens per second across 1,280 conversations at once. The most useful lesson for anyone serving models: each time a new batch size showed up, the system paused about 11 seconds to recompile. Those pauses happened on only 1.1% of steps, yet they ate 47.5% of decode time. Rounding batches up to a few fixed sizes made the pauses disappear and gave the same results. If your serving pods have sudden slow spikes, check for this first.

🌩️ Accelerate agentic AI with NVIDIA Nemotron models on Amazon EKS and SageMaker HyperPod EKS

Alex Iankoulski, Abdullahi Olaoye, Anton Alexander, Curt Lockhart, Leah Tucker, Wenhan Tan (AWS and NVIDIA)

Serving a large model for agents on Kubernetes means getting many things right together: GPU nodes, fast networking between them, somewhere to keep very large model files, and a serving layer that stays quick under load. This post builds that full stack on EKS for NVIDIA’s Nemotron 3.5 Super VL, a 120B mixture-of-experts model that uses 12B parameters per token. The same Kubernetes files run on three flavours of EKS. On standard EKS, you manage the GPU nodes yourself. On EKS Auto Mode, AWS handles scaling, GPU drivers and EFA networking for you. On SageMaker HyperPod EKS, you get a cluster built for long-running AI work.

The cluster setup is the part to study. Container images live in Amazon ECR. The model weights sit on one shared FSx for Lustre volume, so every pod reads the same copy instead of downloading its own. GPU nodes talk to each other over EFA, AWS’s high-speed network. Before deploying the model, the authors run an NCCL all-reduce test, a quick check that every GPU and network link is working at full speed. On top of that sits NVIDIA Dynamo with vLLM. It can serve the model from a single node, or split the work: one set of pods reads the prompt (prefill) and another writes the answer (decode). For image and video input, encoding can run on its own node too. The authors validated it on p6-b200 (8x B200) and p5en (8x H200) nodes. On H200, the model needs 8 GPUs in BF16 or 4 in NVFP4.

To size it, set a latency target first. Their example aims for under 600 ms to the first word and under 100 ms between words. One replica met that while serving about 320 tokens per second at 32 concurrent users. If the time between words climbs, add decode pods. If the wait for the first word climbs, add prefill pods. Everything is in the open-source aws-do-eks repo, with more inference blueprints in AI on EKS.

Starred Content ⭐

⭐ The hidden failure paths that affect Kubernetes control plane availability

Sri Saran Balaji Vellore Rajakumar, AWS

Most big Kubernetes outages don’t start with something crashing. They start with a part doing exactly what it was built to do, at the wrong moment. The author walks through three real cases from running clusters at fleet scale.

In the first case, an add-on API server upgraded a library and missed one annotation. It kept running fine, but the API description it published (the OpenAPI schema) became invalid. Helm reads that combined description, so Helm deployments broke even for charts that never touched the add-on. The fix: after every upgrade, check the schema clients actually receive, not just whether the process is up. In the second case, a DaemonSet’s agents had crashed across a large fleet, and a rolling restart brought thousands of them back at once. Each one asked the API server for the full list of Pods. In some request paths the server ignored the page size and sent everything back, which used up its memory and CPU. The fixes: run Kubernetes 1.31 or later, use WatchList where your client supports it, and have agents watch only what they need, such as one namespace or one node. In the third case, the liveness probe of an encryption plugin checked whether an outside KMS was reachable. During a network blip, Kubernetes kept restarting the plugin, and each restart threw away the cached keys the API server needed.

The lesson across all three: a health check should ask “is this part broken?”, not “is everything it depends on reachable?” Alert on the dependency, but don’t restart over it. Keep serving from the last known good state, the way CoreDNS and kube-proxy do.

⭐ One pod, one node: The architecture behind serverless Kubernetes on AWS

Sri Saran Balaji Vellore Rajakumar, AWS

Serverless Kubernetes changes who manages the node, but Kubernetes still expects a node to exist. On EKS Fargate, every pod gets its own small node, built just for it and removed when the pod ends. This article follows one pod through that journey.

A Fargate profile matches pods by namespace and labels. An AWS webhook then switches the pod to a dedicated Fargate scheduler. That scheduler doesn’t look for room on existing nodes. It asks for brand-new capacity sized to the pod. A trimmed-down kubelet registers the node, with the logs, profiling and debug handlers turned off, and it only accepts that one pod. The pod gets its own network interface (ENI) in your VPC, so security groups and flow logs apply per pod. Inside, three separate network namespaces keep AWS’s platform traffic, the node and your workload apart. To patch, EKS evicts pods using the normal Eviction API and respects your PodDisruptionBudgets. If a pod still hasn’t moved by the deadline, EKS deletes it anyway.

The catch is the one-pod, one-node model itself. Each pod uses its own ENI, so plan your subnet IP addresses for that. Anything that needs the node itself doesn’t fit: DaemonSets, host networking, privileged pods, GPUs, and hostPath or EBS volumes. For shared storage, the article recommends EFS.

⭐ Observability patterns for AI agents on Amazon EKS

Hari Muthusamy, Elamaran Shanmugam (AWS)

Normal monitoring tells you an agent replied with HTTP 200. It doesn’t tell you whether the agent picked the right tool, got stuck in a loop or spent far too many tokens. This post traces all of that for agents running on EKS. Every model call, tool call and step becomes a span, a timed record of one piece of work, labelled with the model, input and output tokens, latency and cost. The labels follow OpenTelemetry’s standard names for AI work, such as gen_ai.usage.input_tokens.

The demo runs on EKS Auto Mode. Agents are built with Strands, which has OpenTelemetry built in. They call Amazon Bedrock through Bifrost, an LLM gateway that falls back from Sonnet to Haiku when requests are throttled. Traces go to Langfuse, an open-source tool for viewing them. The post compares two ways to send them there:

• Decentralized: each agent sends its traces straight to Langfuse. There’s less to run, but every agent has its own settings to keep in sync.
• Centralized: all agents send traces to one shared OpenTelemetry Collector, which forwards them. You change sampling or the backend in one place, but the Collector becomes something you must size and keep healthy.

Both give the same view, and switching between them needs no code changes. A trace ID passed in the request header joins the agent’s spans and the gateway’s spans into a single trace. The results show why this matters. One research agent took 21.79 seconds and cost $0.0279 for four model calls and six web searches, with about 92% of the time spent waiting on the model. A simpler data agent took 7.67 seconds and cost $0.0061.

1. Announcements 📢

📢 Building, Scaling and Running Secure Agents on Kubernetes

Hosted by AWS at KubeCon NA 2026. Put together by Sai Kumar Samala, Aaresh Sharma, Niall Thomson, Carlos Santana, Ashok Srirama and team. Monday 9 November, 9:00 AM to 12:00 PM MST (check-in from 8:30), Kimpton Monaco Salt Lake City, Paris Ballroom, Floor 2.

An AI agent isn’t a normal app. It calls models, uses tools, and can write and run its own code. The usual container patterns weren’t built for that. This free, three-hour workshop works through those problems hands-on on Amazon EKS.

You build agents with an open-source agent SDK and run them as standard Kubernetes workloads, including coding agents like Claude Code. Model traffic goes through an AI gateway, a single front door to the models, so agents get model access without ever holding API keys. Tools are reached through an MCP gateway instead of direct connections. Code that a model writes, and that nobody has checked, runs in air-gapped microVM sandboxes. These are tiny virtual machines with no network access, handed out in seconds from a pool kept warm in advance. The user’s identity is passed along with every tool call, and the gateway checks whether that user is allowed to use each tool. Observability tooling on EKS traces every prompt, tool call, token and dollar.

For EKS readers, this covers the full stack in one sitting: cluster, gateways, sandboxes, identity and cost tracking. You don’t need a KubeCon pass to attend, but seats are limited and you need to bring a laptop. If you can’t be in Salt Lake City, AWS also lists an online “Secure AI Agents on Amazon EKS” workshop that covers the AI gateway, MCP tools and microVM sandbox parts.

📢 Secure AI Agents on Amazon EKS: Sandboxing Sub-Agents with nono

AWS webinar with the team behind nono, the open-source agent sandbox from nolabs (Luke Hinds, Stephen Parkinson). 2 November 2026.

AI agents don’t just answer questions any more. They write code, run it, call tools and start sub-agents to share the work. Each hand-off is a new way for something to go wrong: a leaked key, a bad dependency, or a sub-agent that reads far more files than its task needs. By default, an agent can touch everything the user who started it can.

nono locks this down at the operating system kernel. It uses Landlock on Linux, a kernel feature that takes away a process’s rights. Once applied, the limits last until the process ends, and the agent has no way to undo them. Every file the agent opens, every tool it calls and every network endpoint it reaches is checked against a written policy, and the kernel has the final say. A newer feature gives each tool call its own narrow permissions, and those expire when the call finishes. Tools never see real secrets. They get a placeholder, and a proxy outside the sandbox swaps in the real key, from a source such as Kubernetes Secrets, only for the approved API.

For EKS readers, the webinar shows two layers working together. EKS isolation sits around the pod, and nono sits around each agent and tool inside it. It needs no root and no code rewrites. Before you try it, check the Linux kernel version on your node AMI. nono’s own docs say Landlock file rules need kernel 5.13 or later, and network rules need 6.7 or later.

📢 Amazon opens its AWS Trainium AI chips to every PyTorch developer, no rewrite needed

Amazon / Annapurna Labs (PyTorch blog, GitHub)

Most AI code is written in PyTorch and built with NVIDIA GPUs in mind. Running it on other chips has usually meant rewriting parts of it and learning a new toolchain. TorchNeuron removes most of that work for AWS Trainium. It registers Trainium inside PyTorch as a normal device called “neuron”, the same way GPUs show up as “cuda”. For many models, the only change is .to(‘neuron’). It’s open source under Apache 2.0 and in public beta since 20 October 2026, on Trainium2 and Trainium3.

The tools you already use keep working. Eager mode runs one step at a time, so prints and breakpoints behave as usual. torch.compile turns the whole model into one optimised program. To split a model across chips, the standard options work as they are, including DDP, FSDP2, and tensor and pipeline parallelism, with traffic between machines going over EFA. The PyTorch profiler works too. Hugging Face finds the device automatically. AWS says it tested against 100,000 Hugging Face models, which together cover 99.9% of downloads. Deepgram reports about 60% more audio processed per dollar than on its GPU setup.

It’s a first release, so check the gaps before you move a job. Dynamic shapes aren’t supported yet, and about 22,000 PyTorch tests are skipped because some operations don’t run natively on Trainium yet. FP8 is still in private beta.

📢 Announcing SOCI-indexed images for faster SageMaker AI jobs

Kanwaljit Khurmi, Deep Shah, Michael Oguike, Mona Mona (AWS)

ML container images are huge. Add PyTorch, CUDA and their dependencies, and an image can reach tens of GB. Normally the whole image has to download before the job starts, so an expensive GPU sits idle while it waits. SageMaker AI training and processing jobs can now use SOCI (Seekable OCI) images instead. SOCI adds a small index to the image, like a book’s table of contents, so the job starts right away and pulls each file only when the code first reads it.

AWS’s tests show a big difference. A 13.2 GB image took 3 min 42 s to download the normal way and about 6 seconds with lazy loading. A 35.4 GB image went from 7 min 17 s to 3 to 11 seconds. GPU idle time during image pulls dropped by up to 90%. The first training epoch ran 3 to 16 seconds slower, because files were still streaming in. Every epoch after that matched the normal image, and jobs finished 1 to 6 minutes sooner overall. To use it, you add a -soci suffix to an AWS Deep Learning Container tag (PyTorch, TensorFlow, vLLM and SGLang variants exist), or you index your own image with the open-source SOCI snapshotter.

For EKS readers, it’s the same technology. Recent EKS optimized AMIs for Amazon Linux 2023 and Bottlerocket ship the SOCI snapshotter built in. On EKS it’s mostly used in parallel pull mode, which downloads the full image faster, and EKS Auto Mode uses that mode for GPU instances. The catch: lazy loading helps little on images under about 4 GB, or when your code reads most of the image at startup. It only works with private ECR repositories, and code that expects the Docker socket may break.

📢 Amazon EKS Auto Mode now supports the Kubernetes Gateway API

To reach an app on Kubernetes from outside, you need a front door: a load balancer and some rules about which traffic goes to which app. The old way, called Ingress, puts all of that in one file. That makes it hard for several teams to share one front door, or to send, say, 10% of traffic to a new version. The Gateway API splits the job in two. The platform team owns the front door (the Gateway). Each app team writes its own rules (the routes).

Until now, using the Gateway API on EKS meant installing and running an extra controller yourself. With EKS Auto Mode, AWS now runs it for you. You write the Gateway and routes, and Auto Mode creates the AWS load balancer:

• Web and gRPC traffic (HTTPRoute, GRPCRoute) gets an Application Load Balancer.
• Plain TCP and UDP traffic (TCPRoute, UDPRoute) gets a Network Load Balancer.

When you delete the Gateway, the load balancer is deleted too. It works in every Region where EKS Auto Mode is available.

For EKS readers, you don’t have to switch all at once. Your existing Ingress and LoadBalancer Services keep working, and you can move one app at a time. To check first:

• You have to turn the feature on in the cluster settings (gatewayApi.crds.managed).
• If your cluster already has Gateway API CRDs older than v1, AWS won’t take them over until you upgrade them.
• Not every Gateway API option maps to an AWS load balancer feature. Check AWS’s ALB feature comparison before you rely on one

Community & Career 🤝

🤝Building a Secure LLM Platform on Amazon EKS

Emmanuel Akuffo, AWS Community Builder (repo)

Running a model on Kubernetes is the easy part. Making it safe to expose is the hard part. Emmanuel’s rule is simple: the model is never the API. Every request goes through Cloudflare, then an ALB, then a small AI gateway he wrote in FastAPI, and only then reaches the model. The gateway is where the rules live. It checks API keys and refuses all requests if the keys are missing, instead of quietly letting everyone in. It limits each user to about 60 requests a minute and records the numbers that matter for an LLM: tokens, latency, errors and whether the model is up. NetworkPolicies on EKS block all traffic by default, so the model only accepts calls from the gateway. Secrets live in AWS Secrets Manager and reach pods through EKS Pod Identity and External Secrets, never through Git.

He started on CPU on purpose. A small Qwen2.5-1.5B runs on llama.cpp on its own m7i.2xlarge node group, with weights kept on a gp3 volume so they aren’t downloaded again on every restart. It generates about 34.5 tokens per second, with a P95 of about 2.4 seconds end to end. A g5.xlarge GPU node group already sits in Terraform, scaled to zero. Because the gateway speaks the OpenAI API, switching to vLLM on GPUs later only changes where the gateway points.

The lessons are the best part. Argo CD broke when apps synced before their CRDs were ready. A .gitignore rule silently hid a manifest, so Argo CD never deployed it. An image built on his Mac failed on x86 nodes. A 5-second latency alert was useless for CPU inference, so he moved it to 30 seconds. He also lists what’s still missing, including WAF rules, image signing, prompt-injection controls, and locking the ALB so Cloudflare can’t be bypassed.

🤝 Deploying an Open-Source LLM on Amazon EKS Using vLLM and NVIDIA GPU Acceleration

Yashaswi Kakumanu

Your GPU node is Ready, Kubernetes can see the GPU, and the model still won’t start. That’s the most useful lesson in Yashaswi’s first vLLM deployment on EKS. The setup is simple: one g4dn.xlarge node with an NVIDIA T4 GPU, vLLM v0.11.0 serving Qwen2.5-0.5B, and the API kept inside the cluster, reached only through kubectl port-forward. Even so, the vLLM pod kept crashing on startup.

The previous container’s logs showed the cause. FlashInfer, a library vLLM uses to speed up inference, was crashing while it checked which GPU it was running on. Nothing was wrong with the node or the driver. One library in the serving stack simply didn’t handle the T4 properly. Two environment variables on the pod got it running: VLLM_USE_FLASHINFER_SAMPLER=0 and VLLM_ATTENTION_BACKEND=TORCH_SDPA. Together they skip FlashInfer and use PyTorch’s own attention code instead.

Once it ran, each 100-token reply took about 2.83 seconds, roughly 35 tokens per second, with requests sent one at a time. vLLM held about 10.6 GB of the T4’s 15 GB of memory. That’s mostly space set aside for the KV cache, not the model itself, so it says little about how busy the GPU really was. If your vLLM pods crash-loop on older T4 nodes, check FlashInfer before you start debugging the node.

1. Highlights ✨

✨ Amazon ECS now supports GPU sharing with NVIDIA MPS

Small jobs, like a small model or an embedding service, often use only part of a GPU. But each one usually gets a whole GPU to itself, so the rest of the card sits idle and you pay for it anyway. NVIDIA MPS (Multi-Process Service) lets several jobs run on one GPU at the same time. Think of flatmates sharing one kitchen, each with their own shelf.

Amazon ECS now supports this, with no code changes. In the task definition, each container gets two settings. The first is how much GPU memory it may use, which is a hard limit. The second, optional, is the most GPU compute it may use, as a percentage. AWS’s example puts two models on one GPU: one with 12 GB of memory and up to 50% of compute, the other with 8 GB and up to 25%. The compute number is a ceiling, not a reservation. When one container is idle, the other can use the spare compute, up to its own cap. ECS places tasks by memory only: it adds up the memory a task asks for and finds a GPU with that much free. It works on G4dn through G7e, P4d, P5 and P6-B200 instances. On ECS Managed Instances, ECS runs the MPS helper process for you and replaces the instance if it fails. On ECS on EC2, you need the AL2023 GPU AMI from 20261019 or later and ECS agent 1.107.0 or later.

There are three catches. First, MPS is sharing in software, not a hard split of the chip. A crash in one container can take down every container on that GPU. Second, the usual GPU usage and GPU memory metrics in CloudWatch stop working for shared GPUs, so plan how you’ll watch them. Third, it’s not meant for large training jobs that need the whole GPU.

On EKS, there’s no managed version of this yet. You can turn on MPS yourself through the NVIDIA device plugin, which added it as an experimental feature in v0.15.0, or through NVIDIA’s DRA driver. Either way, keeping the MPS helper process healthy is your job, not AWS’s.

✨ Train Qwen3 on AWS Trainium with torchtitan

Josh Longenecker, Alex Hoffman (Annapurna Labs, AWS)

Most AI training code is built for NVIDIA GPUs. Moving it to another chip has usually meant new tools, new habits and rewritten code. That’s the gap this post closes for AWS Trainium. A layer called TorchNeuron makes Trainium appear in PyTorch as a regular device named “neuron”, the same way NVIDIA GPUs appear as “cuda”. Your existing PyTorch code and tools carry over as they are, including FSDP, which splits a model across many chips, and torch.compile, which speeds it up.

To prove it, the authors run torchtitan, PyTorch’s own training framework, on a single trn2.48xlarge with 16 Trainium2 chips working as 64 workers. They fine-tune Qwen3-4B and use a simple test to confirm the real weights loaded: the first training step scores a loss of 6.54, against 12.43 when starting from random weights. Loss is the model’s error score, so lower is better. They then train a scaled-down Qwen3-30B-A3B, a mixture-of-experts model, using Trainium-specific code for the expert step.

The last section shows how to add a custom kernel, a small program that runs directly on the chip. Trainium kernels are written in NKI, AWS’s kernel language, and you can plug one into torchtitan without touching torchtitan’s own code.

✨ Distributed training with Ray Train DLCs and EFA on EKS

Sunita Koppar, Kevin Wang, Sukhmeet Singh (AWS)

When you train a model on several GPU machines at once, the machines constantly swap data with each other. If that link is slow, the GPUs sit waiting instead of working. AWS has a fast network for exactly this, called EFA. But getting EFA working on EKS has meant fitting many small pieces together by hand, and one wrong piece means it quietly falls back to the slow path.

This post makes it much simpler. AWS now ships a ready-made container image for Ray Train, a popular tool for spreading training across machines. Everything needed for EFA is already inside. You run it on EKS and only add a few settings.

The example is small enough to copy. Two GPU machines, with 4 GPUs each, train a small open model (Qwen2.5-1.5B) together. The model is split across all 8 GPUs, so each one holds only a slice. Turning on EFA made training about twice as fast, with no code changes. The whole setup costs about $9.50 an hour to run.

Two tips save hours of debugging. Put all the GPU machines in the same Availability Zone, because EFA doesn’t work across zones. And label the GPU machines correctly (nvidia.com/gpu.present=true), or Kubernetes won’t see the GPUs at all. The example trains on random data, so it shows that the setup works, not how good the model gets. The full code is on GitHub.

✨ Controlled self-healing Amazon EKS clusters with AWS DevOps Agent and A2A remote agents

Sudheer Sangunni, Vikram Venkataraman (AWS)

Letting an AI agent fix your cluster on its own sounds great until it changes the wrong thing at the wrong time. This post shows a safer setup: the agent can only act after a second agent, a guardrail, says yes. Think of a junior engineer who can make changes, but only after a team lead signs off on each one.

AWS DevOps Agent is in charge of the investigation. It talks to two helper agents over A2A, an open standard that lets agents find and call each other. Both helpers run as pods inside the EKS cluster and aren’t reachable from the internet. One is an Argo CD agent. It reads findings from K8sGPT, a tool that scans the cluster for problems, and it can change apps through Argo CD. The other is the guardrail agent, which can only read and answers allow or deny. Every change needs its approval. If the guardrail is unreachable, the change is blocked.

The demo uses a pod that keeps running out of memory. For one deployment, the guardrail said yes, the agent doubled the memory limit, and it checked again to confirm the fix. For a second deployment, the guardrail said no, and nothing changed. The authors are open that the sample rule, “does this deployment have a PodDisruptionBudget?”, is only a placeholder. The real value is the separate approval gate, and you can swap in your own rules, such as a minimum number of replicas or a change-freeze calendar. The code is in the aws-samples repo.

🎉 Sponsor Section

At the moment, we don’t have a sponsor for this edition, but we look forward to working with companies and organizations that support the EKS & AI Infrastructure community in future editions. If you or your company is interested in sponsoring, please contact us at 📧 thecloudtechforall@gmail.com

📝 Words from the Author

I’m writing this from Atlanta airport, with three IBM TechXchange talks behind me and a flight home ahead.

I spoke on securing an agentic AI system that handles clinical data, on GPU scheduling for media workloads at scale, and on how one malicious PyPI package reached 33,688 AI deployments and took every credential with it. Three very different rooms, and somehow the same conversation kept happening after each one. People weren’t asking about models. They were asking about the platform underneath: what the agent can reach, why the GPUs sit idle, which dependency they should have checked.

That stayed with me on the way to the airport. Many of us are putting AI into production faster than we’re building the boundaries around it. I don’t say that as criticism, because I see it in my own work too. But it’s the work in front of us now.

A big thank you to the IBM TechXchange and HashiCorp teams for having me, and to everyone who stayed back to talk. Those conversations are the reason I keep doing this.

Next is re:Invent in Las Vegas, and this one means a lot to me. I’m giving a Code Talk on Amazon EKS about running AI agents safely. We spent years learning how to run containers safely on Kubernetes. Now the code inside those containers is being written by agents, live, with access to shells, APIs and secrets. Some of what we learned still holds. Some of it doesn’t. In the room, we’ll write the code together: putting a boundary around an agent, limiting what it can reach, and seeing what breaks when the defaults are left open.

The first session is on 1 December. So many people wanted in that a repeat was added on 3 December. In between, on 2 December, we’re meeting just to talk it through: what you’re running, where you’re stuck, what’s keeping you up at night.

I’ve wanted to give this talk for a long time. If you’re in Vegas, please come and say hi. I’d love to meet some of you in person.

Happy building. 😎