Commoditech Let's talk
← Back to articles

MLOps services: data drift, CI/CD and staff augmentation

Published: 10 September 2026 • B2B guide / MLOps

The model has an F1 of 0.81 on the April dataset. In August, the same endpoint processes requests that no one would manually approve. No one calculated PSI. No one triggered a retrain. Or vice versa: cron burns GPU every Sunday because "it's in the runbook", and the feature distribution hasn't changed in six weeks.

This is not an algorithm problem. This is a loop problem. MLOps services are not "DevOps who knows Python". It's CI/CD plus continuous training: drift detection, retraining gate, model registry, serving with the same feature code as training. Without this loop, you're just buying a pickle with a platform ticket.

Below is a distinction missing from "seeking an MLOps engineer" briefs: when to rent one person (body leasing an MLOps engineer), and when drift and serving require a DE + DS + MLOps squad. We've detailed hourly rates separately: how much IT body leasing costs in 2026. Here we calculate why timer-based retraining is the most expensive way to achieve stable F1.

One sentence not found on the "ML platform" slide

Kubeflow without a drift owner is CI for images. Not MLOps services. PSI above 0.25 is a signal, not a dashboard. If no one has the authority to stop serving and trigger retraining from a commit, you have a GPU demo.

1. Anatomy: calendar retraining is not continuous training

A typical week for a model in production at a bank, telco, or retail company does not look like an MLflow tutorial. It looks like three disconnected loops:

  • Training. Data Scientist in a Studio / notebook. Artifact: model.pkl or a card in the Model Registry that no one promotes.
  • Deploy. Ticket to DevOps. Image, probe, Ingress. The platform team can handle rollback. No one can handle rollback of a feature definition.
  • “Monitoring”. Grafana for latency and 5xx. Not for Population Stability Index, not for KS, not for a drop in F1 on a delayed label.

Kreuzberger, Kühl, and Hirschl in their MLOps architecture review (IEEE Access, 2023) separate CI/CD from continuous training (CT): the fourth loop that connects data, model, and serving. Without CT, you have a software pipeline. Not a learning pipeline. Sculley et al. (NIPS 2015) named this earlier: learning code is usually a small fraction of the system; the rest is glue, configuration, and hidden dependencies. Changing one feature triggers an avalanche — CACE, changing anything changes everything.

Three symptoms that you have cron, not CT:

  1. Calendar retraining. Sunday 02:00, regardless of PSI. Or never, “because the model is good enough”. Both variants burn either quality or cloud resources.
  2. Feature calculated twice. Pandas on a laptop, SQL in Airflow. Fill-rate diverges by 0.3 pp and becomes apparent after a quarter, when drift is already impacting decisions.
  3. Delayed label, blind metric. F1 in production can only be calculated after weeks. By then, the only signal is input drift (PSI, KS, KL, MMD) — or nothing at all.

Therefore, a brief for “hiring a DevOps for models” without drift context is the most expensive way to get another pickle. The competencies that truly constitute MLOps services are found in three areas: MLOps engineers, Data Engineering, Data Science. GPU serving and LLM add AI / RAG and DevOps / SRE.

2. What Research Says, Not "ML Platform" Decks

You don't need another definition of MLOps maturity. You need a threshold at which retraining is allowed to start — and the right for it not to start when the distribution is stable.

  • PSI > 0.25 is not a dashboard's opinion. Katalay, Dimandja, and Masakuna, A Multi-Criteria Automated MLOps Pipeline for Cost-Effective Cloud-Based Classifier Retraining in Response to Data Distribution Shifts (arXiv:2512.11541, December 2025): combine KS, KL, PSI, MMD, and ΔAcc/ΔF1 into a single score and trigger retraining only after a threshold is exceeded. In scoring literature, PSI above 0.25 has for years indicated significant drift; below 0.10 — noise. The pipeline doesn't guess. It calculates.
  • Retraining on every alarm is more expensive than drift. In the same experiment (autoencoder, anomaly detection datasets) four policies: STATIC (zero retraining) maintains accuracy 0,69±0,2 at a cost of 54,8; FIXED (fixed interval) and NAIVE (retrain on every drift) reach 0,75, but cost 160,6 and 130,5 with 3,0 and 4,3 retrainings. Auto-MLOps: the same 0,75±0,03, cost 108,1, retrainings 1,4±1,2. The same quality as calendar-based, one-third cheaper than FIXED, three times fewer triggers than NAIVE.
  • Architecture without roles is a slide. Kreuzberger et al. (2023): MLOps is about practices and roles (ML Engineer, Data Engineer, DevOps), not a marketplace product. CT is a loop, not a button in Vertex. DORA 2024 adds: elite vs low is an order of magnitude difference in deployment frequency and lead time. Five timesheets (DS from vendor A, DE from B, 'someone for Kubernetes' from C) will not add up to a lead time. They will add up to three replacement SLAs. We detailed the anatomy of this error in team leasing vs body leasing 2026. The data pipeline that feeds CT is in the article on predictive analytics.

Key Architectural Conclusion

MLOps services start at the gateway, not the cluster. The drift detector (PSI/KS/KL) writes an event. A policy decides whether to shuffle the dataset and train. CI validates the feature schema and metric vs baseline. CD promotes the version in the registry. Serving reads the same feature definition. If any stage involves a human pasting CSV, you don't have CT. You have an on-call rotation.

3. Production Case Study: Sunday Retraining That Fixed Nothing

From Engineering Practice: From cron to PSI

Fraud scoring, retail / payments. Sources: Postgres, events in Kafka, features in dbt, serving on Kubernetes. The team worked as follows:

  • training: Data Scientist, SageMaker Studio, manual artifact export,
  • deploy: ticket, Docker, Helm, without input schema testing,
  • retraining: cron on Sunday, full dataset, GPU for four hours,
  • monitoring: p99 and 5xx. No one calculated PSI.

Problem: After a change in channel mix (new payment partner), F1 on the delayed label dropped in two weeks. The cron on Sunday trained on a mix where the new channel was noise. GPU cost increased. Quality did not.

Change: Calculating PSI and KS on a sample from VPC, threshold 0.25 as an alert in CI, not in Slack 'just in case'. Retraining only when drift score exceeds τ and ΔF1 on the holdout is negative. The model registry (MLflow) promotes the version. Serving with the same feature code (offline batch = online). DevOps remains with the image and IAM. MLOps remains with the gate.

Measurement: Number of retrainings from 4/month to ~1.5. The same order of accuracy as 'always train', GPU bill closer to Auto-MLOps from Katalay than FIXED. Time from PSI alert to new version in production: hours, not a sprint. This is not Vertex magic. This is the owner of the CT loop.

The numbers from the paper do not translate 1:1 to your scoring. What translates is the mechanics: calendar-based and naive alarms are two ways to burn budget. A gate with multiple criteria is the third — and the only one that can be audited by a risk committee.

4. Decision Table: CT and Team Composition

Approach Complexity Quality under drift Cloud / GPU Cost Team Overhead When to Apply
STATIC — model once, zero retraining Low Decreases with drift (in paper 0.69 vs 0.75) Low Low, until it explodes PoC, no SLA, dataset is static
FIXED — retraining on schedule Low–Medium Maintains quality when drift is regular High (in paper 160.6 vs 108.1) Medium (Sunday on-call) Regulated batch, confident labels, GPU budget is not an issue
NAIVE — retrain on every drift alert Medium Maintains quality, many false starts High (4.3 retrainings vs 1.4) High (alert noise) When the detector is poor and you're afraid to miss a shift
Multi-criteria Gateway (PSI/KS/KL + ΔF1 + CI/CD) High Same order as FIXED/NAIVE Medium (lowest among loops) High at start, decreases when the loop is stable Enterprise, NDA, GPU, risk committee wants "why now" audit
Managed Service (SageMaker / Vertex / Azure ML) + CT owner Medium (integration) Depends on whether you enable the detector, not just the UI Medium–High (service metrics) Lower operationally if IAM is already in place One cloud, one IAM, someone still needs to set the threshold

MLOps is not Python CRUD and is not "regular DevOps". DevOps/SRE rates: 140–200 / 200–325 PLN/h (specialist / client). AI/LLM in production: 180–250 / 240–350 PLN/h. An MLOps Engineer with GPU serving and drift falls within this range, closer to the higher end when retraining is under SLA. Margin 10–25%. A map, not a price list — details in 2026 rates.

5. Anti-patterns not found in Kubeflow tutorials

  1. Brief for "DevOps with Python knowledge" for CT. You will get Helm and a probe. You will not get PSI or feature schema testing. The market has fractured: classic DevOps has supply, but an ML loop engineer does not. A mislabeled job posting collects CVs in 48 hours but zero relevant skills for drift.
  2. Dashboard instead of a gate. Evidently in Grafana, an alert in Slack, manual retraining "when there's time". This is monitoring. Not MLOps services. A gate has the right to stop version promotion without human intervention in the pipeline.
  3. Retraining on the full dataset because it's simpler. Katalay et al. show why mix new drift with the old dataset instead of training from scratch every week. Full retraining without a detector is the FIXED approach from the paper: quality is there, and so is the invoice.
  4. False ML team. DS from vendor A, DE from B, MLOps "20% from the platform". Three onboardings, zero shared DAG, zero shared Definition of Done for model promotion. This is not staff augmentation. This is an integration tax described under IT team leasing.

6. Playbook: Whom to hire and in what order

Don't start with the cluster. Start by asking which loop is blocking the SLA: data, model, drift or serving.

  1. One gap in an existing loop. You have Airflow, a registry, and someone who reviews PRs. What's missing is a PSI owner and model CD. This is classic body leasing of MLOps engineers: one person, your stand-up, your DoD. First profiles in days, not in a quarter — we source on demand, we don't sell named bench resources for tomorrow morning.
  2. The notebook is the only artifact. There's no DAG, no schema test, no serving. One person won't stitch this together. A team of 3–5: DE (sources, dbt/Spark), DS (model, validation), MLOps (image, drift, promotion). This is closer to team leasing than "let's buy another DevOps".
  3. Data does not leave the VPC. Contractor in your IAM, your cloud, NDA and data processing agreement. Colab with a production dump "to calculate PSI" is a leak. Offline features in batch, online serving with the same code.
  4. Ramp-up. Person for an existing team: first CVs in 24–48 h, start after your interviews and contract. Squad from scratch: weeks, not a sprint, because you're stitching together permissions, data, and DoD for model promotion. The lie "three senior MLOps from Monday" is a CV or a bench we don't have — and we won't pretend otherwise.

Commoditech has been doing T&M and permanent recruitment from Warsaw since 2012. 80+ specialists in our network, not an idle bench. A T&M brief or success fee can be submitted from an IDE via MCP for AI agents, not just from a form. Prices for a specific stack are calculated by a human; ranges are in the article about rates, not in the agent's JSON.

FAQ

How do MLOps services differ from standard DevOps for ML models?

DevOps delivers the image, probe, and rollback. MLOps also delivers: model registry, feature schema test, drift detection (PSI, KS, KL), retraining gate, and serving with the same feature code as training. CI/CD without continuous training is deploying a pickle. Not a platform. Kreuzberger et al. (2023) explicitly distinguish CT as a separate loop — not 'just another job in Jenkins'.

When to rent one MLOps engineer, and when a team?

One person, when DAG, registry, and IAM are already in place, and a drift owner and model CD are missing. A team of 3–5 (DE + DS + MLOps), when the only artifact is a notebook, and retraining is a Jira ticket. MLOps alone without DE will not fix the source. A Data Scientist alone without MLOps will deliver AUC. They will not deliver p99 or an audit of 'why this version'. If you don't have anyone to review a contractor's PR, don't buy body leasing. Buy a lead plus one role or a squad.

How much does body leasing of an MLOps engineer cost in Poland in 2026?

MLOps sits between DevOps/SRE (client 200–325 PLN/h) and production AI/LLM (240–350 PLN/h). GPU serving and retraining for drift push the rate higher. Vendor margin 10–25% — if someone promises 8% with a replacement in 5 days, account for it elsewhere. This is a market map, not an offer. The quote for a brief is calculated by a human. Details in 2026 rates.

Can an MLOps contractor work on data under NDA and RODO?

Yes: Your environment, your IAM, your cloud or on-prem, T&M contract with NDA, data processing agreement, and IP on the client's side. A production dump on a contractor's laptop 'to calculate PSI faster' is a leak. You calculate drift on a sample in VPC, not in Colab. Access audit is yours. Clauses: contracts, margins, IP.

Sources