MLOps Pipeline in Practice: Automating from Model Training to Deployment

1. MLOps Pipeline Overview

MLOps brings DevOps principles to the machine learning lifecycle, automating model training, evaluation, deployment, and monitoring. A well-designed MLOps pipeline can significantly shorten the time from experimentation to production deployment.

2. Training Pipeline Implementation

Below is a training pipeline implementation in Python, covering the complete workflow of data loading, model training, and evaluation.

Python Training Pipeline

python
import mlflow
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.metrics import accuracy_score, precision_score

def train_pipeline(data_path: str) -> float:
    """End-to-end training pipeline"""
    mlflow.set_experiment("production-model")

    with mlflow.start_run():
        # Data loading
        df = pd.read_parquet(data_path)
        X = df.drop("target", axis=1)
        y = df["target"]

        # Train-test split
        X_train, X_test, y_train, y_test = train_test_split(
            X, y, test_size=0.2, random_state=42
        )

        # Model training
        model = GradientBoostingClassifier(
            n_estimators=300,
            max_depth=6,
            learning_rate=0.1,
        )
        model.fit(X_train, y_train)

        # Model evaluation
        y_pred = model.predict(X_test)
        acc = accuracy_score(y_test, y_pred)
        prec = precision_score(y_test, y_pred, average="weighted")

        mlflow.log_metrics({"accuracy": acc, "precision": prec})
        mlflow.sklearn.log_model(model, "model")

        return acc

if __name__ == "__main__":
    accuracy = train_pipeline("s3://data/features/train.parquet")
    print(f"Model accuracy: {accuracy:.4f}")

Kubernetes Deployment Configuration

yaml
apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  name: production-model
spec:
  predictor:
    sklearn:
      storageUri: s3://mlflow-artifacts/1/model
      resources:
        requests:
          cpu: "1"
          memory: 2Gi
        limits:
          cpu: "2"
          memory: 4Gi

3. Summary

The core of an MLOps pipeline lies in automation and reproducibility. By standardizing the training, evaluation, and deployment processes, teams can iterate on models more rapidly while maintaining production stability.