👋 Everything about EKS & AI Infrastructure Newsletter "#55" ☁️❤👨‍💻

Dear EKS & AI Infrastructure enthusiasts,

Welcome to Everything about EKS & AI Infrastructure #55. This week sits at the intersection of two very different feelings: on one side, we have genuinely hard, exciting problems in front of us—optimizing LLM serving without burning $10K per experiment, squeezing more out of GPUs on EKS, making multi-arch and multi-cluster setups actually manageable. On the other side, the way we get to those solutions is changing so fast that it’s hard to tell where our hard-won experience ends and the agent’s autocomplete begins. This edition leans into that tension: AI-driven performance engineering, GPU- and EKS-heavy architectures, and the very human question of what it means to build platforms in a world where the tooling is sprinting ahead of us.

1. Performance Engineering in Modern AI Systems🌩️

🌩️ Why LLM serving optimization costs $10K per experiment—and how emulation fixes it

Testing a single LLM serving configuration takes 2-4 hours on a GPU cluster and costs thousands of dollars. Want to optimize your vLLM setup? You’ll burn through budget before learning anything useful. The team behind Vidur, just released Revati, which solves this by running actual vLLM and SGLang code on CPUs instead of reimplementing logic manually. CUDA calls get intercepted and stubbed out, so new features work automatically—no maintenance tax.

The clever part is “time jumps.” Instead of waiting for GPU kernels to finish (which would be just as slow), Revati fast-forwards the virtual clock by the predicted kernel duration. The hard problem is keeping distributed workers synchronized—if one GPU jumps time while another is still processing, the system hangs. They solved it with a barrier protocol that preserves causality, achieving 5-17× speedup with less than 5% prediction error on latency distributions.

If you’re running vLLM on EKS and trying to figure out the right tensor parallelism or batch size, this changes the workflow: test configurations locally, deploy only the winners to your GPU nodes. Iteration goes from expensive trial-and-error to cheap exploration before you spin up a P5 instance.​

1. Starred Content ⭐

⭐ Simplify Kubernetes Cluster Management Using ACK, kro, and Amazon EKS (by Islam Mahgoub, Kumudhan Cherarajan, Markos Kandylis, Ramesh Mathikumar, and Sébastien Allamand)

Most teams managing multiple EKS clusters juggle Terraform, Helm, and scattered scripts. This post shows how to manage an entire fleet using only the Kubernetes API: ACK controllers create AWS resources from CRs, kro handles dependencies, and Argo CD deploys via GitOps.​

The interesting part is kro’s ResourceGraphDefinition. Creating an EKS cluster requires a VPC, subnets, IAM roles, and strict ordering. kro wraps this into a single custom resource that manages dependencies automatically using CEL expressions. For multi-account setups, ACK controllers assume roles in workload accounts and create resources there. The workflow: raise a PR with a cluster manifest, Argo CD applies it, kro orchestrates everything, cluster spins up with add-ons installed.​

One caveat: kro isn’t production-ready yet. But if you’re managing 10+ clusters and tired of fragmented tooling, this pattern is worth watching.​

⭐ Observing and Scaling MLOps Infrastructure on Amazon EKS (by Elamaran Shanmugam, Isha Dua, and Sanjeev Ganjihal)

Most teams running ML workloads on EKS hit the same wall: traditional monitoring doesn’t show you what’s actually slow. Is your training job stuck because of GPU memory bandwidth, gradient sync delays, or just bad scheduler placement? You can’t tell from CPU and memory graphs. This post walks through building observability that actually works for ML—layering infrastructure metrics (Node Exporter), Kubernetes state (kube-state-metrics), and hardware-specific telemetry. NVIDIA GPUs need DCGM for CUDA utilization; AWS Neuron chips need NeuronCore metrics. Different hardware, different signals.

The more interesting bit is autoscaling. You can’t scale ML pods on CPU thresholds—it doesn’t match how the workload behaves. The post shows how to use Prometheus Adapter to feed ML-specific metrics (active inference requests, JupyterHub sessions) into the Horizontal Pod Autoscaler, so your Ray or MLflow deployments scale when demand actually shifts, not when some arbitrary CPU percentage gets crossed. If your GPU pods are still scaling like stateless web apps, this is worth reading—the failure modes are different, and the tooling needs to reflect that.

⭐ Automating EKS Cluster Upgrades with Lambda and EventBridge

Manually upgrading EKS clusters across environments is one of those chores everyone dreads—easy to mess up, hard to standardize, and painful to repeat. This post lays out a clean automation pattern: use EventBridge Scheduler to trigger AWS Lambda functions that orchestrate control plane upgrades, add-on updates, and managed node group rollouts via CloudFormation, all while respecting Pod Disruption Budgets so workloads stay available. The companion GitHub repo (automate-eks-upgrades) ships the implementation, so you can deploy and adapt it instead of starting from scratch.

What I like about this is that it treats upgrades as a recurring workflow, not a one-off project. You get logged, repeatable runs, fewer manual steps, and a clear separation between “when” upgrades happen (EventBridge) and “how” they’re executed (Lambda + CloudFormation). If you’re running multiple EKS clusters and still doing upgrades by hand or copy-pasting eksctl commands, this is a sharp baseline to adopt and then evolve.

⭐ Multi-Architecture Workloads on Amazon ECS Managed Instances

This AWS Builder Community article by Abhishek Nanda walks through something a lot of teams are quietly wrestling with: how to run x86, ARM, and GPU workloads in one place without building three separate infrastructures. Using Amazon ECS Managed Instances, you can mix Graviton (ARM) for cost-efficient services, x86 for legacy or specialized workloads, and GPU instances for ML inference—all in a single ECS cluster, while AWS handles provisioning, scaling, and patching of the underlying EC2 fleet.

What I like here is the practicality: the guide shows how to wire up capacity providers for different architectures, combine Spot and On-Demand for flexibility, and let ECS place tasks on the right hardware without you micromanaging individual node groups. If you’re already comfortable with EKS but haven’t revisited ECS in a while, this is a good reminder that “managed EC2 with multi-arch support” is now a first-class option, especially for teams mixing web, data, and GPU workloads.

⭐ NVIDIA GPU to AWS EC2 Instance Mapping (by Gary Stafford)

Gary Stafford published a super handy reference chart mapping NVIDIA GPUs (T4, L4, L40S, H100, B100, and friends) to the AWS EC2 instance families that actually ship them. If you’re trying to choose between G, P, and new accelerated instance types for LLM inference, fine-tuning, or real-time workloads, this kind of mapping saves a ton of console-diving and doc reading. It also surfaces practical questions like “is upgrading from L4 to L40S worth it for my workload?” and “what do I really get in memory bandwidth going from Ada to Blackwell?”

This is especially useful if you’re running mixed GPU fleets behind EKS or ECS and need to standardize on a small set of SKUs for cost and ops sanity. Instead of guessing, you can line up GPU memory, bandwidth, and instance pricing in one place and pick the right tier for batch vs real-time inference. Bookmark material.

1. Announcements 📢

📢 Claude Opus 4.6: Multi-agent teams and 1M token context

Anthropic just released Opus 4.6, built for navigating large codebases, deeper planning, and expert-level judgment across domains. The standout features: multiple agents working in parallel on the same task in Claude Code, adaptive thinking and context compaction for longer-running tasks on the API, and a 1M token context window (in research preview)—the first Opus-class model to hit that scale.

📢 Kiro adds Opus 4.6 support for spec-driven development

Kiro integrated Opus 4.6 immediately, combining it with their spec-driven development workflow. If you’re using Kiro for AI-SDLC, the jump to Opus 4.6 means better reasoning during architecture and deployment phases, and the ability to handle more complex multi-agent orchestration. Worth experimenting with if you’re already running agent-driven builds.

Community & Career 🤝

🤝 Amazon Nova AI Hackathon: $40K in prizes for building production AI agents

AWS is running a hackathon focused on Amazon Nova Act—their new model designed for automating production UI workflows. Build AI agents that interact with real interfaces, compete for $40,000 in cash prizes plus $55,000 in AWS credits. Deadline is March 9, 2026. If you’ve been experimenting with agentic workflows or UI automation (or just want an excuse to try Nova Act), this is a solid forcing function with real prizes and credits to offset experimentation costs.

🤝 From Workshop to Production in 14 Days: AI-SDLC in Practice (by Adam Divall)

Adam Divall attended a Kiro CLI workshop at AWS re:Invent 2025 and built a production-ready AWS MAP portal in 14 days using AI-driven Software Development Lifecycle (AI-SDLC). Instead of traditional development, he used 5 specialized agents—Product Owner, Domain Architect, Technical Architect, Full-Stack Engineer, and Deployment Engineer—each handling distinct phases from requirements to deployment. The result: 6-8× faster delivery, 14 content pages, interactive calculators, a Bedrock-powered chatbot, and full auth infrastructure running for under $30/month.

What stands out is the Deployment Engineer agent autonomously resolving 80% of deployment issues without human intervention—no context switching, no boilerplate work, just strategic decisions and validation. Adam walks through the full build process, tooling choices, and what actually worked versus what didn’t. If you’ve been wondering what agent-driven development looks like beyond demos and toy projects, this is a real example with costs, timelines, and lessons learned from putting it in production.

🤝 Terraform Weekly #260 (by Anton Babenko)

This issue focuses on how real teams run Terraform in production: Preply’s IaC setup with Atlantis, versioning Terraform binaries/providers cleanly, and governance patterns for Terraform at scale. There’s also a handy piece on automating Route 53 DNS updates when ALBs are reprovisioned—useful if you’re cycling load balancers in front of EKS or GPU-backed services. If Terraform is part of your AWS platform story, this is a solid issue to skim and the newsletter is worth subscribing to.

🤝 Learn Kubernetes Weekly #168

This issue is packed with the kind of debugging and systems stories platform teams actually care about: a great post on tracing a cluster-wide network outage back to OOM-killed daemons and serial console bottlenecks, a hands-on guide to deploying an OpenTelemetry-native LGTM stack (Loki, Grafana, Tempo, Mimir) on Kubernetes, and a wild case study on shrinking a container image from 800 GB to 2 GB through ruthless trimming and multi-stage builds. There’s also Lynq, a SQL-like tool for querying Kubernetes state directly from inside the cluster, which feels like kubectl meets SELECT * FROM pods in all the right ways.

1. Highlights ✨

✨ On-demand and scheduled scaling of Amazon MSK Express clusters

Kafka workloads rarely run at steady state—they spike during business hours, burst during events, and go quiet overnight. This AWS blog shows how to automate horizontal scaling for MSK Express clusters using two strategies: on-demand scaling based on CloudWatch metrics (CPU, throughput, custom business metrics), and scheduled scaling triggered by cron expressions for predictable patterns. The solution leverages MSK Express’s intelligent rebalancing, which redistributes partitions 180x faster than standard brokers, so adding or removing capacity becomes a single API call instead of manual partition management.

The blog walks through deploying the solution with AWS CDK, configuring thresholds, and tuning monitoring frequency to balance responsiveness against Lambda and CloudWatch costs. If you’re running Kafka on AWS and manually adjusting broker counts or overprovisioning for peak traffic, this is a practical answer to matching capacity to actual demand.

✨ GitOps-Friendly Secret Management with External Secrets Operator and Argo CD (by Artem Lajko)

There are plenty of ways to handle secrets in Kubernetes—plain Secrets, Sealed Secrets, Vault, and more—but at scale, most teams want two things: GitOps for configuration and no secrets in Git. This article shows how to get there with External Secrets Operator plus Argo CD, using a hub-and-spoke model with dedicated vaults and both agentless and agent-based setups. The nice part is that it doesn’t stay theoretical; Artem Lajko shares what actually worked (and what hurt) while building the Kubara framework in real enterprise environments. If you’re trying to standardize secret management across clusters without bolting on yet another custom tool, this is worth a read.

🎉 Sponsor Section

At the moment, we don’t have a sponsor for this edition, but we look forward to working with companies and organizations that support the EKS & AI Infrastructure community in future editions. If you or your company is interested in sponsoring, please contact us at 📧 thecloudtechforall@gmail.com

📝 Words from the Author

I read Aditya’s post this week about being good at something that suddenly became “free and abundant,” and it hit me in a very specific, uncomfortable way.

For most of my life, the story in my head has been simple: if I work hard, learn fast, and care more than average, things will roughly make sense. Effort turns into skill, skill turns into value, value turns into some kind of security. Not perfect, not fair, but at least vaguely linear.

This new wave of AI breaks that story.

It’s not just that tools got better. It’s that the gap between “I’ve practiced this for years” and “I just tried this for the first time with an agent” is collapsing in real time. And if you’ve built your identity around being “the person who knows how to do hard things,” that’s not a small adjustment. It’s an existential one.

I’ve noticed something strange in myself lately:
On the surface, I’m more productive than ever. I can write, experiment, ship, and learn faster than I thought possible. But underneath, there’s this quiet question: what part of this is actually me?

Not the prompt. Not the output. Me.

There’s also a different kind of loneliness that comes with this. So much of my growth came from struggling with things alongside other people—late-night messages, shared confusion, slow progress. Now a lot of that struggle is happening in private chats with a model, and the human part—the messy, shared part—shrinks if we’re not intentional about it.

I don’t have a neat lesson here. I just know this:

• I don’t want my value as a person to be “I can do things an AI can now do faster.”

• I don’t want my days to be a quiet competition with a tool that doesn’t even know I exist.

• I don’t want to wake up one day and realize I optimized everything except the parts of life that actually matter.

So I’m trying something different this year:
I still care about the work. I still love the craft. But I’m paying more attention to who I’m becoming while everything is speeding up—how I treat people, how honest I am about my fears, how generous I am with what I’ve learned, how often I say “thank you” while people are still here to hear it.

If this era is showing me anything, it’s that skills can be automated frighteningly fast. The things that can’t be automated—how we show up for each other, what we choose to care about, what we’re willing to say out loud—suddenly feel a lot more important.

If you’re feeling excited and scared and a bit lost at the same time, same here. Maybe the goal isn’t to resolve that tension quickly, but to live honestly inside it for a while—and see who we become on the other side.

Thank you for reading Everything about EKS & AI Infrastructure Newsletter #55

Happy Building! 😎