š Everything about EKS & AI Infrastructure Newsletter "#84" āļøā¤šØāš»
Fixing GPU cold starts, catching bad nodes early, and letting agents watch your cluster while you sleep
Dear EKS & AI Infrastructure enthusiasts,
Welcome to Everything about EKS & AI Infrastructure #84.
Picture a GPU node turning on for the first time. It takes eight minutes just to say hello, downloading drivers, pulling images, and loading model weights before it can answer a single request. Now picture that same node failing at 3am, its logs gone the moment the crashed pod gets replaced, with nobody awake to catch it. This issue is about fixing both problems.
Two different deep dives take that eight-minute wait down to under a minute. One fixes how model weights get downloaded and how compiled GPU work gets reused instead of redone every time. The other looks at the parts most teams never check: driver setup and pulling the container image itself. NVCRE catches a bad GPU node before a training job wastes time on it, and Snowflakeās Semi-Persistence keeps models ready in memory so waking one up takes milliseconds instead of a slow reload.
And for that 3am failure: a new AWS tool watches your cluster all the time, grabs the evidence the second something breaks, before Kubernetes deletes it, and hands it to an AI agent to investigate while you sleep. Alongside that, Grove gives Kubernetes a real way to run multi-pod AI systems as one unit instead of stitching it together by hand.
Outside of pure infra, thereās a good read on why something as simple as capturing a GPU trace turns into a real headache once youāre on Kubernetes, a look at DuckDB (whose team just joined AWS) and why running a database as part of your app instead of a separate service is catching on, and a builder who beat a 70B model with a fine-tuned 3B one on receipt parsing, for about $0. Letās get into it.
Performance Engineering in Modern AI Systems š©ļø
š©ļøNVIDIA Cluster Readiness Engine (NVCRE)
New GPU clusters often have bad nodes hiding in them, a flaky NIC or a weak GPU, and you usually only find out once a real training job hits that node and fails. NVCRE is a Kubernetes controller that runs real NCCL and training workloads across the cluster first, watches for failures, and tells you exactly which nodes are bad and why, before you point production traffic at them.
The key design decision: NVCRE only detects and reports, it never touches the node itself (no cordoning, no tainting). Thatās a deliberate split, fixing the node is your platformās job, not the toolās, which keeps NVCRE simple and lets it plug into whatever remediation flow you already have. It already knows how to detect AWS, GCP, Azure, and OCI, and recognizes GPU types from H100 up through the newest GB300.
One practical gotcha for GB200/GB300 clusters: NVCRE needs the NVIDIA DRA driver and a standalone DCGM service thatās off by default even if you already have the GPU Operator installed, worth checking with kubectl nvcre setup status before your first run, so youāre not debugging a missing prerequisite instead of an actual bad node.
š©ļøSemi-Persistence: Fast Model Swapping for vLLM & GPUs, by Snowflake AI Research
If youāre running many models on shared GPUs, the usual choice is bad either way: keep every model loaded and waste GPU capacity, or swap them in and out and eat vLLMās slow sleep/wake cycle every time. vLLMās two sleep modes each pay a cost: one copies weights back to CPU every time you sleep a model, the other discards them and reloads from disk every time you wake one up.
Snowflakeās fix keeps one long-lived copy of the weights in pinned CPU memory and treats the GPU copy as disposable, so waking a model up means streaming weights back from CPU instead of hitting disk. They also spread that transfer across every PCIe lane and NVLink connection on the node at once instead of a single path, which is most of where the speedup comes from.
The numbers: single-GPU models go from 1.2-13.5 seconds down to 214-801 milliseconds, and trillion-parameter models like DeepSeek-v4-pro drop from a 15-minute cold start to about 33 seconds. The whole thing, including an orchestrator that automatically migrates and consolidates models to avoid one big model blocking a small one waiting behind it, is open source on GitHub.
š©ļøBuild a Physical AI model factory with NVIDIA Cosmos 3 on SageMaker HyperPod
Teams building robots or self-driving systems usually run separate GPU clusters for each step: one to make synthetic training data, one to actually train, one to test the result, each spun up and torn down on its own. Cosmos 3 gets rid of that split because itās one model that can act three different ways just by changing what itās asked to predict, so all three steps can share the same pool of GPUs instead of each needing its own. What matters here isnāt how fast any single job runs, itās how much useful work you get out of every GPU-hour youāre paying for across the whole loop.
The sharpest lesson: a normal, generic GPU setup can look fine on one machine and then quietly fail to use the fast network once you spread training across multiple machines, because the exact software versions have to line up precisely, and when they donāt, everything falls back to a much slower path with no obvious error. The fix was standardizing on a pre-matched AWS image instead of a general one.
The other useful takeaway is a simple rule for how often to save your progress: save too rarely and one crash costs you hours of retraining; save too often and the saving itself eats into your compute time. Thereās a formula that finds the sweet spot based on how long a save takes and how often failures actually happen, and itās worth remembering that the more machines youāre running, the more often something breaks, so bigger clusters need to save more frequently, not the same amount.
š©ļøGPUs For Actors In Agent Substrate, by Michael Levan
Most agent workloads (tool calls, planning loops) are CPU-bound, so GPU access is the exception, not the default, but when an agent runs local inference or a GPU-backed tool, sharing a small GPU pool across many intermittently-active Actors matters for cost. Agent Substrate attaches GPUs the way Kubernetes normally does, nvidia.com/gpu on the WorkerPool, but scopes GPU sharing per Actor instead of per Pod, so multiple Actors can time-share one Workerās GPU rather than each needing its own slice.
The sharp constraint: GPU support only works with gVisor-sandboxed Actors right now, enforced by a CEL validation rule on the WorkerPool CRD, so a microVM-sandboxed Actor canāt request one. The mechanism is also worth noting for anyone building similar sandboxing infra: the default distroless worker image canāt exec nvidia-ctk to inject the GPU into the sandbox, so getting this working requires building a glibc-based worker image specifically so the NVIDIA Container Toolkit CLI is available to configure the runtime.
- Starred Content ā
āAutomate proxy injection for Amazon EKS on AWS Fargate using Kyverno
Companies that route all outgoing traffic through a corporate proxy hit a wall on Fargate: thereās no node to configure the proxy on, since AWS manages that layer. This post uses a policy engine called Kyverno to automatically add the proxy settings to every container the moment itās created, only in namespaces that opt in. No changes needed to any applicationās own configuration.
A few things worth knowing if you try this: sidecar containers added by a service mesh can sometimes get missed, since thereās no guarantee about which add-on runs first. Pulling container images doesnāt go through this proxy setup either, thatās handled separately by AWS, so your image registry still needs its own way to be reached. And one setting choice matters a lot: the policy is set to āfail open,ā meaning if the policy engine goes down, pods still get created, just without the proxy settings, silently. Worth putting a monitor on that instead of assuming it just works.
āFast model loading for AI inference on Amazon EKS (with additions from Cut GPU inference cold start from 8 minutes to less than a minute, by Sajjan Gundapuneedi)
Every time a new pod starts up to serve an AI model, it has to pull the modelās weights and get the GPU ready before it can answer a single request, and that wait was taking anywhere from 80 to 460 seconds. Digging into where that time actually goes shows it depends on model size: for smaller models, most of the wait is the GPU compiling and optimizing itself, a fixed cost no matter how big the model is. For bigger models, most of the wait is just downloading the weights. Two settings changes, no code changes, fixed both.
For the download side, the fix was surprising: splitting files into more, smaller pieces to download faster in parallel actually made things worse or did nothing. The real problem was that each download worker was doing its downloads one at a time in sequence, not at once, so more pieces just meant more waiting in line on the same connection. Matching the piece size to the actual file size fixed it.
For the compile side, the GPU was redoing the exact same optimization work every single time a pod started, even though the result is identical every time for the same model and hardware. Saving that work to the fast local storage already sitting on the machine, instead of throwing it away when the pod restarts, meant every pod after the first one starts almost instantly.
The New Stack piece widens the lens to the full picture: those two fixes only cover two of six steps a pod actually goes through before serving anything, node startup, GPU driver setup, pulling the container image itself, downloading weights, compiling, and finally getting the serving engine ready. Two of those other steps turned out to hide real waste too. GPU drivers normally get compiled fresh on every new node, adding 2-3 minutes, avoidable by baking a pre-compiled driver into the node image instead. And pulling the inference engineās own container image (8-12GB) was assumed to be a network problem, but testing showed itās actually a CPU problem: even a machine with a very fast network only managed to use a tiny fraction of it, because unpacking the image was the real bottleneck, not fetching it, so downloading pieces in parallel and unpacking them on more CPU cores at once fixed it instead.
Put together, a model that used to take over 7 minutes to load on an existing machine now loads in under a minute, and even bringing up a brand-new machine from nothing dropped from up to 15 minutes down to around 5, with nothing more than a few settings and image build changes.
āOptimize EKS operations with agents: Reduce MTTR with AWS DevOps Agent and a Kubernetes Operator
AWS DevOps Agent can investigate an incident well, but it canāt detect one on its own, something else has to call it, and by the time a human notices a failure and manually triggers an investigation, the pod may already be rescheduled or deleted and the evidence gone. Kubernetes only keeps events for about an hour, restarted containers overwrite their own logs, and a deleted pod takes its logs with it. This postās answer is a Kubernetes Operator that watches pod state changes directly (not polling, not waiting on an external monitoring toolās alert cycle) and preserves the failureās data to S3 and CloudWatch within milliseconds of the failure happening, before it can disappear.
The sharper design choice is what the Operator collects, and itās not the same for every failure. kubectl only sees container-level data, but an OOMKilled podās real story is often in node dmesg, and IP exhaustion lives in IPAMDās internal state, neither reachable from inside the pod. Because the Operator knows the live pod-to-node mapping at the moment of failure, it pulls exactly the node-level evidence each failure type needs (dmesg for OOM, IPAMD/ENI mappings for IP exhaustion, restart history for CrashLoopBackOff) via SSM Run Command, then hands DevOps Agent both the Kubernetes data and that node-level context to investigate against connected code repos and observability tools.
The walkthrough example is a good illustration of the payoff: a newly deployed pod OOMKilled twice, and DevOps Agent traced it not to a resource-limit problem but to an unbounded list growing in a background worker thread in the just-deployed code, correctly telling the engineer that raising the memory limit would only buy about two and a half more minutes before the same crash. Itās an open-source reference implementation, not a supported product, so itās meant to be adapted (detection conditions, collection strategy) to what your own cluster actually fails on, rather than deployed as-is.
**
ā**LLM Inference Benchmarks: vLLM vs SGLang vs TensorRT-LLM on a Single L40S, by MichaÅ Wojdylak
Most engine benchmarks run on an H100 and call it a day, which isnāt what most teams are actually deploying on. This one runs on a single L40S, a more realistic mid-range GPU, and tests the things that actually matter in production: how each engine handles more concurrent users, long documents, different quantization formats, and MoE models versus dense ones.
The pattern that comes out: TensorRT-LLM pulls ahead under heavy load and with long documents, vLLM is fastest to respond when traffic is light, and quantization format matters more than people assume, 4-bit (AWQ/GPTQ) beats FP8 on vLLM because it frees up memory for a bigger batch, not because of precision. The MoE model was the biggest surprise of the whole post, it beat the equivalent dense model by a wide margin on every metric.
āSet up OpenAI ChatGPT Codex with LiteLLM on Amazon ECS and Amazon Bedrock
Codex still runs its task loop and tool execution locally on the developerās machine, but this puts a gateway (LiteLLM, running on ECS) between Codex and the actual model on Bedrock. That gateway becomes the one place to control which models are approved, issue per-developer scoped keys instead of one shared key, and enforce budgets and rate limits, all without touching how Codex itself works.
The sharpest point in the whole piece: a gateway returning plain text back to Codex proves almost nothing. Coding agents lean on specific Responses API behavior, streaming, being able to continue a previous response, and forcing a tool call with a proper call ID, and a gateway can pass a basic āhello worldā test while silently failing one of those. So the real validation gate tracks whether a unique marker planted in one response can actually be recalled in the next one, which is what catches a gateway that accepts continuation syntax but doesnāt actually preserve the underlying state.
The other useful framing is that this isnāt the only path, and knowing when not to add the gateway matters just as much. Direct IAM Identity Center access is the lower-complexity default when AWSās own identity and audit logs are enough, add the gateway only once you need centralized policy that native controls canāt give you, and a managed option like Portkey is a third choice when running your own gateway infra isnāt worth it. Same compatibility tests apply no matter which path you pick.**
ā**Why a GPU Profiling Capture Turned into a Distributed Systems Problem, by Zhenyu Sha
On one machine, ācapture a GPU trace right nowā is a single profiler call. On Kubernetes, itās a distributed systems problem: training processes are spread across nodes, one Pod can hide eight torchrun workers behind a single identity, and a file appearing on disk doesnāt mean itās complete or even belongs to the capture you asked for. Starting from Metaās MAIProf design, Zhenyu Sha built a Kubernetes-native version and ran into a sequence of failures that only showed up under real cluster load.
The two sharpest lessons: a node agent asking dynolog for a trace can get āprocessesMatchedā without āactivityProfilersTriggered,ā meaning the process was found but never actually entered the profilerās queue, so treating āmatchedā as success silently produces zero files. And a two-concurrent-capture bug that looked like a validation-speed problem turned out to be a design flaw: extending how long the control plane waits can never fix how slowly the data plane can observe, since both the timeout and the retry were gated on an observation that was itself being starved by validating the previous large file.
The rule that comes out of the whole piece is worth keeping regardless of stack: capture converges when every target has either a verified, archived, identity-bound artifact or an explicit terminal failure, request-dispatched is never the same fact as trace-delivered, and a test that passes whether the mechanism works or not isnāt actually a test.
āGrove, by ai-dynamo (NVIDIA Dynamo)
Kubernetes natively scales one pod at a time, but a sharded model instance (prefill leader + workers, decode leader + workers) is one logical unit spread across many pods, and scaling or scheduling only part of that group leaves the rest idle or deadlocked. Grove gives you a single CR to describe the whole inference system (prefill, decode, routing, whatever components) and handles hierarchical gang scheduling, topology-aware placement, and explicit startup ordering from that one spec instead of stitching together custom controllers and scripts.
The concept worth understanding is the four-layer model: a PodClique is one role (leader, worker, frontend), a PodCliqueScalingGroup is cliques that must scale and schedule together as a gang (like a prefill leader and its workers), a PodCliqueSet is the whole colocated system with autoscaling and topology-aware spread across replicas, and PodGang is the scheduler-level primitive underneath all of it that guarantees a minimum replica count gets scheduled together, all-or-nothing. Startup ordering solves a real MPI-style problem too: workers need to be Ready before the leader launches, and getting that wrong causes silent init failures rather than a clean error.
- Announcements š¢
š¢DuckDB and the changing physics of analytics, by Werner Vogels (guest post: Andy Warfield)
The news is that DuckLabs is joining AWS as a subsidiary, DuckDB itself stays open source under the DuckDB Foundation and MIT license, and the team keeps working from Amsterdam. The infra argument behind it is the more interesting part: a single m8g.48xlarge instance today has roughly 50x the memory, cores, and network bandwidth of the 2007-era m1.xlarge that motivated MapReduce and Spark in the first place, so a huge share of data work that used to genuinely need a cluster now fits comfortably on one machine. DuckDB leans into that by running as an embedded library in the same address space as your app rather than a remote service, the same engine works whether youāre querying CSVs locally, adding an aggregation to an app, or compiling to WASM and running in a browser tab.
For AWS specifically, the DuckDB Iceberg extension (built to work with S3 Tables) already pulls 800K downloads a week, and AWS is already using DuckDB internally for dashboards and in-server accelerators. Worth watching for anyone whose stack touches S3 Tables or does agent-adjacent structured-data work, since the framing here is explicitly āthe most natural tool developers, and increasingly agents, reach for when they work with structured data.ā**
š¢**AWS AI Fellowship x NVIDIA ā Nemotron edition
AWS opened applications for a selective cohort program pairing startups building on NVIDIAās Nemotron models with joint AWS/NVIDIA Solutions Architect support, infra credits, and co-sell access. For infra teams, the interesting part isnāt the marketing package, itās the SA support: dedicated guidance on Nemotron architecture and fine-tuning from people who work both sides of the stack tends to produce the reference implementations that later become public EKS/GPU-scheduling patterns.
The program bundles build credits and SME support for every accepted startup, with workshop slots, customer case studies, and event access at re:Invent and GTC reserved for whoever performs on co-sell.
š¢Cosmos3-Edge, Cosmos3-Nano, and Cosmos3-Super on SageMaker JumpStart
NVIDIAās Cosmos 3 models landed on SageMaker JumpStart this week, and theyāre built for robots and vision AI, not chatbots. Three sizes: Edge (4B) runs directly on the robot, Nano (16B) reasons about physics and generates video/action data, Super (64B) handles high-fidelity simulation and synthetic data.
Edge is the one worth watching. It runs on an NVIDIA Jetson Thor device and outputs 32 robot actions per inference at 15 Hz, real-time control, not offline planning. Most āedge AIā launches are really cloud inference with an edge label slapped on, this one genuinely runs on the device. It also handles 256p-480p video at 12-30 FPS, so itās reading the world and acting on it in the same loop, not just reacting to a single frame.
The three sizes actually map to a real pipeline, not just āpick your budget.ā Super generates the synthetic training data and runs simulations at scale, Nano reasons over that data with physics and common sense baked in, and Edge is what youād actually ship on the robot once the first two have done their job. Thatās the more interesting story here: NVIDIA is selling a full workflow from simulation to deployment, not three unrelated model sizes.
Deploy any of the three straight from the SageMaker JumpStart console or Python SDK, no extra infra setup to try them.
Community & Career š¤
š¤lambda-microvm-sandbox-shim by Jimmy Cowan
Normally, agent-sandbox runs your agentās code inside a sandboxed pod on your cluster. Jimmy Cowan built a shim so a Lambda MicroVM can do that job instead, an agent talks to the MicroVM exactly like it would talk to a normal sandboxed pod, no code changes needed. Itās already working end-to-end on EKS 1.36, including a chat agent running inside the MicroVM and talking to Bedrock.
The smart part is how it handles exec (running commands interactively). Instead of bouncing that traffic through an extra proxy inside the VM, it goes straight to the sandbox daemonās gRPC endpoint directly, which is faster.
The catch: Lambda MicroVMs only live for 8 hours max, even with pause/resume, and they only run on arm64 (Graviton) chips. So this fits short-lived, per-session sandboxes well, not anything long-running. Itās still a prototype, not ready for production, but itās a solid pattern for pushing agent sandboxes off your own cluster and onto serverless infra instead.
š¤Part 4: V3, Inference, and Serving models live, by John Enevoldsen
This is the last part of a four-part series where John built LLMs from scratch, and this one covers the part most people skip: actually serving the model. He trains a bigger model (672M params) across two GPUs, builds a KV cache from scratch, and puts all nine of his checkpoints behind a real chat page people can try.
The KV cache part is the most useful for infra folks. He shows why you save the keys and values but not the query, a position bug that shows up only at generation time, and why you have to turn off the ādonāt look aheadā mask once youāre generating one token at a time instead of a whole sentence. These bugs donāt crash, they just quietly give you wrong answers, which is worse.
Heās upfront that this isnāt production-ready: no batching multiple users together, no smart memory management, never tested under real load. He even says heād use vLLM instead of his own code for anything real. Thatās what makes it worth reading, he hit every wall a production serving system solves, so you can see exactly why those systems exist.
š¤Agentic Developer Experience on Amazon EKS, workshop by Raghuram Gururajan
Raghuram Gururajan is running a free hands-on workshop on 8th September on using Kiro CLI with the EKS MCP Server, describing what you want in plain language and having it explore clusters, diagnose issues, ship microservices, and stand up observability for you. Worth a look if you want a practical, guided look at natural-language ops on EKS rather than reading about it secondhand.
š¤Fine-tuning Qwen2.5-3B to beat Llama-3.3-70B on receipt parsing, by keerthidurairaj97
The interesting part here isnāt the fine-tuning, itās the gate before it. Before training anything, they ran Llama-3.3-70B on the raw task first: if a 70B model could already parse receipts into JSON reliably, thereād be no case for training a smaller model at all. It scored only 10% all-correct, so the task earned its fine-tuning budget instead of getting one by default.
From there: teacher-label 875 CORD receipts with the 70B model, LoRA fine-tune Qwen2.5-3B on a free Kaggle T4, and the 3B model ends up beating the 70B teacher outright, 50% all-correct against 10%, with a much stronger line-item F1 too. All on public data, for roughly $0.
The plan from here is merging the LoRA adapter and quantizing to GGUF to get it running on CPU, which is the more interesting infra question if it lands: a 3B specialist model doing a structured-extraction task better than a 70B generalist, cheap enough to run without a GPU at all.
- Highlights āØ
āØAWS Lambda SnapStart now supports container image functions
Container image functions on Lambda let you package up to 10 GB of dependencies, which is great for ML inference workloads that need real model weights and libraries bundled in. The tradeoff was cold start: pulling image layers and initializing the runtime could take several seconds, which is a dealbreaker for anything latency-sensitive. SnapStart was already available for managed runtimes (Python, .NET, Java), just not containers.
Now it works for containers too, taking startup times down from several seconds to sub-second. The mechanism is the same as before: Lambda snapshots the fully initialized execution environment at deploy time, caches it, and resumes from that snapshot instead of booting from scratch on every cold invoke.
For teams running ML inference or interactive APIs on Lambda container images, this removes the main reason to fall back to always-warm provisioned concurrency just to avoid cold starts. Itās opt-in and rolled out to all commercial regions except Asia Pacific (New Zealand) and (Taipei).
āØBuild Your Own Inference Engine: From Scratch, by martinuke0
Builds a working LLM inference engine in about 200 lines of plain Python on GPT-2: read the prompt, run it through the model once to build up the cache, then generate one word at a time reusing that cache instead of re-reading everything from scratch each time. The useful framing is that reading the prompt is a burst of heavy compute, while generating each new word afterward is mostly just waiting on memory, which is why the two get optimized in totally different ways in real systems.
The part worth remembering is the KV cache problem: every user talking to the model gets their own cache, and that cache grows the longer the conversation gets. Multiply that across many people chatting at once and the cache alone can eat up more GPU memory than the model itself. So if a server ever runs out of memory in a way that seems to make no sense given how āsmallā the model is, this growing per-user cache is usually the real cause, not the model.
āØAmazon ECS introduces Early Success Criteria for service deployments
ECS rolling deployments normally wait for every single desired task to be healthy before calling the deployment done, which is a real problem for GPU-backed inference services, since GPU capacity can be scarce or slow to provision and the last few tasks can hold everything up. Early Success Criteria lets you set a healthy percent instead, say 90% of 100 desired tasks, and the deployment is marked successful once that threshold is hit, with the remaining tasks launching through normal scaling outside the deployment lifecycle.
It also gives you a choice on cleanup timing for the old version: BLOCKING waits for old tasks to fully drain before declaring success, DEFERRED declares success immediately and drains the old tasks in the background, useful for services with long-lived connections that shouldnāt be forced to cut over all at once. Together, this unblocks CI/CD pipelines and dependent deployments sooner without waiting on capacity-constrained stragglers.
š Sponsor Section
At the moment, we donāt have a sponsor for this edition, but we look forward to working with companies and organizations that support the EKS & AI Infrastructure community in future editions. If you or your company is interested in sponsoring, please contact us at š§ thecloudtechforall@gmail.com
š Words from the Author
Years ago I watched someone try to fit a king-size mattress up a narrow staircase built for a much smaller apartment. It didnāt fit sideways, didnāt fit flat, didnāt fit at any angle anyone tried. In the end they didnāt get a smaller mattress, they took the window out and hoisted it up with ropes from outside. It worked. But every time they moved again, they had to do it all over, because the staircase was never actually built for that mattress. It was just built for whatever used to live there.
Thatās the feeling I keep having reading this issue. Kubernetes was that staircase. It was designed years ago around a simple idea: one small, disposable, stateless process, running in one small box, that you can kill and restart without anyone noticing. GPU workloads are the mattress. Theyāre big, theyāre stateful, they need to move together in groups, and they hate being killed and restarted from scratch. And every clever thing weāre covering this week, Grove, NVCRE, the cache fixes, the new Operator that catches failures before evidence disappears, is basically someone taking the window out and hoisting the mattress up with ropes. It works. Itās genuinely impressive. But itās a workaround, not a redesign.
Hereās the part that stuck with me: none of this is really about GPUs being hard. Itās about a scheduler that assumed the wrong shape of problem, built for one small thing at a time, being asked to handle several big things that all have to move together or not at all. Grove exists because Kubernetes literally has no built-in word for āthese eight pods are actually one thing.ā NVCRE exists because Kubernetes has no word for āthis machine looks fine but is quietly bad at its one job.ā We keep bolting on new vocabulary, one project at a time, to say things Kubernetes was never built to say.
So hereās the real question worth sitting with: are we actually teaching Kubernetes a new language for AI workloads, or are we just quietly building an entirely different scheduler on top of it, piece by piece, and calling it Kubernetes because thatās the house we already own? I donāt think it matters which answer is true for adoption, the ecosystemās too big to walk away from either way. But itās worth knowing which one youāre actually building, before youāre five CRDs deep and still hoisting mattresses through windows.
Happy building. š
