This documentation describes the integration of MindsDB with MindsDB Inference Endpoints, a cloud service that simplifies the way developers interact with cutting-edge LLMs through a universal API.


Before proceeding, ensure the following prerequisites are met:

  1. Install MindsDB locally via Docker or Docker Desktop.
  2. To use MindsDB Inference Endpoints within MindsDB, install the required dependencies following this instruction.
  3. Obtain the MindsDB Inference API key required to deploy and use MindsDB Inference Endpoints models within MindsDB. Follow the instructions for obtaining the API key.


Create an AI engine from the MindsDB Inference Endpoints handler.

CREATE ML_ENGINE mindsdb_serve
FROM mindsdb_inference
      mindsdb_inference_api_key = 'api-key-value'

Create a model using mindsdb_serve as an engine.

CREATE MODEL mindsdb_inference_model
PREDICT answer
      engine = 'mindsdb_serve',   -- engine name as created via CREATE ML_ENGINE
      model_name = 'model-name',              -- choose one of available models
      prompt_template = 'prompt-to-the-model' -- prompt message to be completed by the model


The following usage examples utilize mindsdb_serve to create a model with the CREATE MODEL statement.

Classify text sentiment using the Mistral 7B model.

CREATE MODEL mindsdb_inference_model
PREDICT sentiment
   engine = 'mindsdb_serve',
   model_name = 'mistral-7b',
   prompt_template = 'Classify the sentiment of the following text as one of `positive`, `neutral` or `negative`: {{text}}';

Query the model to get predictions.

SELECT text, sentiment
FROM mindsdb_inference_model
WHERE text = 'I love machine learning!';

Here is the output:

| text                     | sentiment |
| I love machine learning! | positive  |

Supported Models

For an overview of the models supported, visit the following docs. This list will help you quickly identify the right models for your needs.

Troubleshooting Guide

Authentication Error

  • Symptoms: Failure to authenticate to MindsDB Inference Endpoints.
  • Checklist:
    1. Make sure that your MindsDB Inference Endpoints account is active.
    2. Confirm that your API key is correct.
    3. Ensure that your API key has not been revoked.

SQL statement cannot be parsed by mindsdb_sql

  • Symptoms: SQL queries failing or not recognizing table and model names containing spaces or special characters.
  • Checklist:
    1. Ensure table names with spaces or special characters are enclosed in backticks. Examples:
      • Incorrect:
        SELECT input.text, output.sentiment
        FROM data AS input
        JOIN mindsdb_inference_model AS output
      • Incorrect:
        SELECT input.text, output.sentiment
        FROM integration.'travel data' AS input
        JOIN mindsdb_inference_model AS output
      • Correct:
        SELECT input.text, output.sentiment
        FROM integration.`travel data` AS input
        JOIN mindsdb_inference_model AS output