Fine-Tune the Classification Model

In this example, we again use our sample PostgreSQL database.

First, we create and train the model using a subset of the customer_churn data, considering only female customers.

CREATE MODEL mindsdb.adjust_customer_churn_model
FROM example_db
    (SELECT *
    FROM demo_data.customer_churn
    WHERE gender = 'Female')
PREDICT churn;

On execution, we get:

Query successfully completed

We can check its status using this command:

DESCRIBE MODEL adjust_customer_churn_model;

Once the status is complete, we can query for predictions.

SELECT churn, churn_explain
FROM mindsdb.adjust_customer_churn_model
WHERE seniorcitizen = 0
AND partner = 'Yes'
AND dependents = 'No'
AND tenure = 1
AND phoneservice = 'No'
AND multiplelines = 'No phone service'
AND internetservice = 'DSL';

On execution, we get:

+--------+------------------------------------------------------------------------------------------------------------------------------------------------------------+
| churn  | churn_explain                                                                                                                                              |
+--------+------------------------------------------------------------------------------------------------------------------------------------------------------------+
| No     | {"predicted_value": "No", "confidence": 0.9887640449438202, "anomaly": null, "truth": null, "probability_class_No": 0.934, "probability_class_Yes": 0.066} |
+--------+------------------------------------------------------------------------------------------------------------------------------------------------------------+

Let’s adjust this model with more training data. Now we also consider male customers.

FINETUNE mindsdb.adjust_customer_churn_model
FROM example_db
    (SELECT *
    FROM demo_data.customer_churn
    WHERE gender = 'Male');

While the model is being generated and trained, it is not active. The model becomes active only after it completes generating and training.

To check the status and versions of the model, run this command:

SELECT name, engine, project, active, version, status
FROM mindsdb.models
WHERE name = 'adjust_customer_churn_model';

On execution, we get:

+-----------------------------+-----------+---------+--------+---------+----------+
| name                        | engine    | project | active | version | status   |
+-----------------------------+-----------+---------+--------+---------+----------+
| adjust_customer_churn_model | lightwood | mindsdb | false  | 1       | complete |
| adjust_customer_churn_model | lightwood | mindsdb | true   | 2       | complete |
+-----------------------------+-----------+---------+--------+---------+----------+

Alternatively, use the DESCRIBE command as below:

DESCRIBE MODEL adjust_customer_churn_model;

Let’s query for a prediction again.

SELECT churn, churn_explain
FROM mindsdb.adjust_customer_churn_model
WHERE seniorcitizen = 0
AND partner = 'Yes'
AND dependents = 'No'
AND tenure = 1
AND phoneservice = 'No'
AND multiplelines = 'No phone service'
AND internetservice = 'DSL';

On execution, we get:

+--------+--------------------------------------------------------------------------------------------------------------------------------------------------------------+
| churn  | churn_explain                                                                                                                                                |
+--------+--------------------------------------------------------------------------------------------------------------------------------------------------------------+
| No     | {"predicted_value": "No", "confidence": 0.9887640449438202, "anomaly": null, "truth": null, "probability_class_No": 0.9294, "probability_class_Yes": 0.0706} |
+--------+--------------------------------------------------------------------------------------------------------------------------------------------------------------+

Here after adjusting the model, there are no significant changes to the predictions. However, the probability class for Yes and No values has been updated. The probability of a Yes value has increased slightly, while the probability of a No value has decreased.

If you have dynamic data that gets updated regularly, you can set up an automated fine-tuning as below.

Note that the data source must contain an incremental column, such as timestamp or integer, so MindsDB can pick up only the recently added data with the help of the LAST keyword.

Here is how to create and schedule a job to fine-tune the model periodically.

CREATE JOB automated_finetuning (

    FINETUNE adjust_customer_churn_model
    FROM mindsdb
        (SELECT *
        FROM example_db.customer_churn
        WHERE timestamp > LAST)
)
EVERY 1 day
IF (
    SELECT *
    FROM example_db.customer_churn
    WHERE timestamp > LAST
);

Now your model will be fine-tuned with newly added data every day or every time there is new data available.

Use Cases