Deploy

Deploy#

Sources

Setup#

import subprocess
import pandas as pd
from pathlib import Path

from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier

import mlflow
from mlflow.models.signature import ModelSignature
from mlflow.types.schema import Schema, ColSpec

We need docker containers with mlflow.

%%bash
docker run -p 5000:5000 -dt --name mlflow_deploy --rm \
    ghcr.io/mlflow/mlflow \
    bash -c "mlflow server --host 0.0.0.0 --port 5000"

4ea5fdd3eeab8fa057ad9c1ad1575039129e8e4b75a187f0c0449d9986c41156

To be able to work with these started mlflow, we need some setup.

mlflow.set_tracking_uri("http://localhost:5000")
exp_name = "penguin_classification"
mlflow.create_experiment(exp_name)

'448934722381259260'

When you have finished playing with the notebook, do not forget to stop the container.

%%bash
docker stop mlflow_deploy

mlflow_deploy

Add model to registry#

We will consider simple example - model that callasify penguins based on parameters of their “culmen”.

Create run#

Create a run with the model in question. Everything is basic except the signature parameter of the sklearn.log_model method.

So by code:

input_schema = Schema([
  ColSpec("double", "Culmen Length (mm)"),
  ColSpec("double", "Culmen Depth (mm)"),
])

We have defined that the model takes two parameters: Culmen Length (mm) and Culmen Depth (mm). This is important because it’s exactly the number that should be passed to the model implementation.

input_schema = Schema([
  ColSpec("double", "Culmen Length (mm)"),
  ColSpec("double", "Culmen Depth (mm)"),
])
output_schema = Schema([ColSpec("string")])
signature = ModelSignature(inputs=input_schema, outputs=output_schema)

mlflow.set_experiment(exp_name)
with mlflow.start_run() as run:
    run_id = run.info.run_id
    print(f"Started run {run_id}")
    # Load dataset
    print("Load dataset...")
    culmen_columns = ["Culmen Length (mm)", "Culmen Depth (mm)"]
    target_column = "Species"

    data = pd.read_csv(
        Path("deploy_files")/"penguins_classification.csv"
    )

    print("Prepare a train-test-split...")
    data, target = data[culmen_columns], data[target_column]
    data_train, data_test, target_train, target_test = train_test_split(
        data, target, random_state=0)

    # Initialize and fit a classifier
    max_depth = 3
    max_leaf_nodes = 4
    print(f"Initialize and fit a DecisionTreeClassifier with max_depth={max_depth}, max_leaf_nodes{max_leaf_nodes}")
    
    mlflow.log_params(
        {"max_depth": max_depth, 
         "max_leaf_nodes": max_leaf_nodes}
    )
    tree = DecisionTreeClassifier(
        max_depth=max_depth,
        max_leaf_nodes=max_leaf_nodes
    )
    tree.fit(data_train, target_train)

    # Calculate test scores
    test_score = tree.score(data_test, target_test)
    mlflow.log_metric("test_accuracy", test_score)
    print(f"Result: Accuracy of the DecisionTreeClassifier: {test_score:.1%}")
    
    # Log the model
    mlflow.sklearn.log_model(tree, "model", signature=signature)

Started run 1d48d9284dd141018bba82d5cf84ef9f
Load dataset...
Prepare a train-test-split...
Initialize and fit a DecisionTreeClassifier with max_depth=3, max_leaf_nodes4
Result: Accuracy of the DecisionTreeClassifier: 96.5%

Add run to registry#

There is a special storage for models - the model registry, so the following cell adds the model from the previous run to the registry.

model_name = "penguins_clf"
result = mlflow.register_model(
    f"runs:/{run_id}/model", model_name
)

Successfully registered model 'penguins_clf'.
2024/06/07 17:27:48 INFO mlflow.store.model_registry.abstract_store: Waiting up to 300 seconds for model version to finish creation. Model name: penguins_clf, version 1
Created version '1' of model 'penguins_clf'.

Run MLFlow side API#

Now you can run a Flask-based API that will render the model by executing a command:

MLFLOW_TRACKING_URI=http://localhost:5000 mlflow models serve --no-conda -m "models:/penguins_clf/1" -p 4242

The following cell starts a new termilan window that will handle the api.

start_api_command = 'MLFLOW_TRACKING_URI=http://localhost:5000 mlflow models serve --no-conda -m "models:/penguins_clf/1" -p 4242'
ans = subprocess.run([
    'gnome-terminal', '--', 'bash', '-c', start_api_command
])

After running the api, you can access it. So here is curl asking it - as a result we got a list with classes of penguins:

%%bash
curl http://127.0.0.1:4242/invocations -s -H 'Content-Type: application/json' -d '{
    "inputs":[
    {"Culmen Length (mm)": 1,"Culmen Depth (mm)": 3},
    {"Culmen Length (mm)": 14,"Culmen Depth (mm)": 120},
    {"Culmen Length (mm)": 200,"Culmen Depth (mm)": 100}
]}'

{"predictions": ["Adelie", "Adelie", "Chinstrap"]}

Or identical request using python:

import requests

url = "http://127.0.0.1:4242/invocations"
headers = {
    'Content-Type': 'application/json',
}
data = {
    "inputs": [
        {"Culmen Length (mm)": 1, "Culmen Depth (mm)": 3},
        {"Culmen Length (mm)": 14, "Culmen Depth (mm)": 120},
        {"Culmen Length (mm)": 200, "Culmen Depth (mm)": 100}
    ]
}

response = requests.post(url, headers=headers, json=data)

print(response.text)

{"predictions": ["Adelie", "Adelie", "Chinstrap"]}

Build docker image#

You can create a docker image that represents an api by using command mlflow models build-docker. full command will take veiw:

MLFLOW_TRACKING_URI=http://localhost:5000 mlflow models build-docker -m "models:/penguins_clf/1" -n penguins_image --enable-mlserver

Note The --enable-mlserver option tells the container to use MLServer as a wrapper. Flask is the default option.

After running the previous command, you’ll have the image penguins_container in the local docker demon.

!docker images | grep penguins_image

penguins_image           latest    dddd7cdc5ca2   2 minutes ago    1.25GB

Now let’s try to run this container:

!docker run --rm -p 4243:8080 -d --name penguins_container penguins_image 

69c8c7939075e083c5e5e76bc1c669dbb5982332a6679925920685f23bb42500

Note We refer to port 8080 of the container docker because this is the default in mlflow.

%%bash
curl http://127.0.0.1:4243/invocations -s -H 'Content-Type: application/json' -d '{
    "inputs":[
    {"Culmen Length (mm)": 1,"Culmen Depth (mm)": 3},
    {"Culmen Length (mm)": 14,"Culmen Depth (mm)": 120},
    {"Culmen Length (mm)": 200,"Culmen Depth (mm)": 100}
]}'

{"predictions": ["Adelie", "Adelie", "Chinstrap"]}

Don’t forget to stop the container with model API:

!docker stop penguins_container

penguins_container
Contents