Cloud cost optimization and platform engineering

Twelve pods each request 2 CPU cores. The 95th percentile of actual use, p95, is 0.22 cores. The Kubernetes scheduler packs pods by the request, not by usage. Reserved: 24 cores. Work: 2.6 cores. The Horizontal Pod Autoscaler (HPA) looks at the percentage of the request, sees idle capacity, and does not reduce the replica count. The Cluster Autoscaler adds another machine of the same type, because the empty space on the node is occupied by a request nobody has touched since the migration.

Cloud cost optimization does not start with a discount from the provider. It starts with who, in platform engineering, is allowed to change the request, the machine family, and the hour at which a non-production environment shuts down. Cut the limit too hard and you raise p99 latency (the 99th percentile) or the process is killed for lack of memory (out of memory, OOM). Cut the request correctly and you remove nodes nobody is using.

Three words that collapse into one line on a brief:

• Platform engineering is the team that builds the internal platform: the cluster, continuous integration and continuous delivery (CI/CD), a service catalogue, budgets. Product teams do not assemble Kubernetes from scratch.
• FinOps (Financial Operations) is the rhythm in which engineering and finance see the cost of a unit of work, for example a thousand requests, not only the invoice at the end of the month.
• Slack is the gap between the limit or the request and real usage. At Google, manually configured jobs ran at 46% slack. The automation brought that down to 23%.

1. Three loops that do not see the same number

Three controllers run on the cluster. Each one optimises a different variable. None of them sees the invoice on its own.

1. Request and limit on the pod. The request is a reservation for the scheduler. The limit is a ceiling: above it, CPU is throttled and memory ends in an OOM kill. A request 4× above p95 leaves a hole on the node that the next pod cannot enter.
2. Replica count (HPA). HPA adds pods when the percentage of the request crosses a target, often 50–80%. When the request is 9× too large, usage looks like 11%. HPA sleeps. Or it oscillates, when the limit throttles CPU and latency rises before the average “catches up”.
3. Node count (Cluster Autoscaler). The Cluster Autoscaler adds and removes machines from a predefined pool. The pool is homogeneous: the instance type someone wrote into infrastructure as code (IaC). A pod that does not fit the free space wakes a new machine of the same family, even when a cheaper family (more memory, less CPU) would have done the job with one node.

The fourth item is not a controller. It is a calendar. Dev and staging at production size, 24 hours a day, 7 days a week. Working hours 08:00–19:00 are 55 hours out of 168. The rest, about two thirds of the week, is a bill for emptiness.

resources:
  requests:
    cpu: "500m"       # reservation: the scheduler and HPA look here
    memory: "512Mi"
  limits:
    cpu: "1500m"      # ceiling; above this, CPU is throttled
    memory: "1Gi"     # above this, the process is killed (OOM)

The Vertical Pod Autoscaler (VPA) can rewrite the request. In automatic mode it restarts pods. On memory it can hide a leak: the application grows, VPA raises the limit, the bug leaves the chart and stays on the invoice. A VPA recommendation without a gate is not the same thing as a rollout.

Tools that pick a machine type for pending pods (on Amazon Web Services that is Karpenter; elsewhere, node auto-provisioning) get around the homogeneous-pool limit. They do not replace request right-sizing. They pack whatever the pods ask for. If the pods ask for 9× too much, Karpenter buys a tidier version of the same hole, not zero hole.

The graphics-processor (GPU) bill and model training are a different loop: drift detection and retraining, not node bin-packing. We wrote that up separately in the article on MLOps services. Folding both briefs into one “DevOps person who does everything” produces an on-call rota. It does not produce FinOps, and it does not produce retraining.

2. What the research actually measures

A slide that says “we will save 40%” does not say on which setup, or against which baseline. Four sources measure different things. Separate them before anyone writes a percentage into a contract.

• Manual slack 46%, automation 23%. Rzadca, Findeisen, Świderski et al., Autopilot: Workload Autoscaling at Google (EuroSys 2020). Autopilot sets both horizontal scale (task count) and vertical scale (CPU and memory) from history. Slack falls from 46% on manually configured jobs to 23%. The number of jobs severely hit by OOM falls by about 10×. At publication, Autopilot covered more than 48% of Google’s fleet-wide resource usage. This is Borg, Google’s internal system, not a product you install on EKS (Elastic Kubernetes Service). What transfers is the mechanism: vertical and horizontal together, on history, with the goal “less slack, without OOM and without CPU throttling”.
• Identical machines and upgrades break the pattern. Hua, Yang, Qian et al., Humas (arXiv:2406.15769, 2024). Experiment on 50 real microservices and more than 11,000 containers. Against methods treated as the state of the art, Humas improves resource efficiency by about 30.4% and stability by about 48.0%. In the comparison table, slack on the raw traces is 52.87%, on a reimplementation of Autopilot 16.63%, on Humas 11.58%. Average CPU allocation falls by about 46.8% versus the raw traces. Separately: after a new version of a service ships, the usage pattern changes (pattern drift). An autoscaler trained on the old version misses again. This is not the same 23% as in Rzadca. Different trace, different number.
• A per-service CPU threshold does not see end-to-end latency. Sachidananda and Sivaraman, Collective Autoscaling for Cloud Microservices (COLA, arXiv:2112.14845). Each microservice scaled on its own CPU threshold does not know that the user’s latency is the sum of several services. On Google Kubernetes Engine (GKE), both standard and Autopilot mode, COLA meets a median or tail latency target on 53 of 63 workloads and is then on average 19.3% cheaper than the next autoscaler that also holds that target. It is the cheapest on 48 of those 53. On small applications, where every configuration can be enumerated, it is optimal in 90% of cases. The authors write that the saving pays for the training cost in a few days. That is their experimental setup, not a promise on your invoice.
• The Cluster Autoscaler does not change the machine family. Boghani, Kirimlioglu, Moturi and Tso, Cloud Resource Allocation with Convex Optimization (arXiv:2503.21096, 2025). The Cluster Autoscaler scales existing pools of identical instances up and down. It does not choose a mix of types. The paper compares a convex optimisation with that autoscaler in a simulation (Python, median of five runs), not on an AWS bill. Scenario 1, a simple web app from scratch: no meaningful difference. A standard autoscaler is enough. Scenario 2, adding to existing infrastructure: 42.5% cheaper (USD 0.12/h versus USD 0.07/h). Scenarios 3–5: 80.5%, 87.2% and 71.1%. Average across five: 56.3%. The memory-heavy workload (scenario 4): from USD 1.08/h to USD 0.14/h. Over-provisioning in the thousands of percent shows up in the scenario that allows only small instances. That is not a typical invoice. It is evidence that the wrong machine family multiplies nodes.
• VPA without a leak gate buys memory for a bug. Karakaya, Şengül and Kaplan, Safety-Gated Autoscaling (arXiv:2607.26503, 29 July 2026). A memory-leak detector (linear regression, R²) blocks the recommendation instead of enlarging a broken container. On a live GKE cluster, the 20–40% saving is a what-if projection, not a measured bill. The detector hits 83%. Test suite: 1,118 tests, 80.3% coverage. There is a dry-run mode and a human approval. Quote 20–40% only with that caveat.

3. A worked example: 24 cores reserved, 2.6 in use

The numbers in the papers do not transfer 1:1 onto your invoice. The order does: request first, then machine type, then shutting down whatever is not serving traffic. Starting with Karpenter on 2000m requests while usage is 220m buys prettier packing of the same hole.

4. Decision table: what to move, and what not to automate

DevOps / SRE / cloud: specialist rate 140–200 PLN/h, client rate 200–325 PLN/h. Supplier margin 10–25%. This is a market map, not a quote. The write-up: what IT body leasing costs in 2026. No Fluff Jobs reports Q1 2026 B2B bands for a DevOps specialist of PLN 23.5–28.5k. That is the contractor’s invoice, not the rate the client pays.

5. Anti-patterns the HPA tutorial skips

1. HPA at 80% of a request that is 4× too large. The autoscaler sees idleness. The nodes stay. The “cluster utilisation” dashboard shows 20% and nobody connects that to the manifest.
2. VPA in auto mode on memory, with no leak detector. The recommendation grows with the bug. The OOM incident disappears. The invoice does not. Dry-run exists so a person can see the trend before the limit goes up.
3. Four pools “just in case”, and a Cluster Autoscaler that can only add the same type. Boghani: on a simple app from scratch there is no gain. The gain is where the pod shape does not match the family written into Terraform two years ago.
4. FinOps as a PDF, and a DevOps engineer at 20% of a full-time role. CI/CD, on-call, cost, Kubernetes, and GPU on top. One person on five loops closes none of them. A cost tool in read-only mode, with no right to change a request, is a more expensive PDF.

6. Playbook: who to rent, and in which order

Do not start with another cluster. Start with the question of which loop is holding the invoice: the request, the machine type, the non-production environments, or the GPU.

1. The cluster, identity (IAM, identity and access management) and Terraform are already there. What is missing is an owner of the requests, of the namespace budget, and of a weekly review. That is classic DevOps body leasing: one person, your stand-up, your definition of done. Body leasing here means renting a specialist into your team, usually billed for time (time and materials, T&M). We do not sell a named bench for Monday morning.
2. Nobody owns the cluster. Four accounts, no IaC, dev at production size, manifests with a request copied from a tutorial. One person will not stitch that in a sprint. A platform squad: someone for the cluster and IaC, someone for CI/CD, someone with the right to say “this namespace shuts down at 19:00”. That is closer to IT team leasing than to “we will buy one more DevOps engineer”. When a squad beats one role is written up in team leasing vs staff augmentation 2026.
3. Data and the kubeconfig do not leave for a laptop. The contractor works in your IAM, in your cloud, under a non-disclosure agreement (NDA) and a data-processing clause. Exporting a kubeconfig “to calculate requests faster” is production access. Clauses: contracts, margins, copyright.
4. Ramp-up. One person into an existing team: first profiles within days, start after your interviews and the contract. A squad from zero: weeks, because you are stitching permissions and a definition of done for changing a request in production. A promise of “three senior platform engineers from Monday” is either a CV or a bench we do not keep.

Commoditech has done T&M and permanent hiring from Warsaw since 2012. The network is 80+ specialists. We do not keep them idle on a bench. A brief for body leasing, or for a permanent hire (a success fee, no hire no fee), can also be filed from the editor, through MCP for AI agents (Model Context Protocol, the protocol an assistant in the IDE uses to call tools). A human prices the specific stack. The bands live in the rates article, not in an automated reply. Contact: the form.

FAQ

Sources

• Rzadca, K., Findeisen, P., Świderski, J. et al. (2020). Autopilot: Workload Autoscaling at Google. EuroSys 2020. doi.org/10.1145/3342195.3387524. research.google.
• Hua, Q., Yang, D., Qian, S., Cao, J., Xue, G., Li, M. (2024). Humas. arXiv:2406.15769. arxiv.org/abs/2406.15769.
• Sachidananda, V., Sivaraman, A. (2022). Collective Autoscaling for Cloud Microservices. arXiv:2112.14845. arxiv.org/abs/2112.14845.
• Boghani, S., Kirimlioglu, E., Moturi, A., Tso, H.-T. (2025). Cloud Resource Allocation with Convex Optimization. arXiv:2503.21096. arxiv.org/abs/2503.21096.
• Karakaya, A., Şengül, E., Kaplan, A. (2026). Safety-Gated Autoscaling. arXiv:2607.26503. arxiv.org/abs/2607.26503.
• Commoditech — 2026 body-leasing rates, hire DevOps engineers, team leasing vs staff augmentation.