This documentation describes the integration of MindsDB with MindsDB Inference Endpoints, a cloud service that simplifies the way developers interact with cutting-edge LLMs through a universal API.

​
Prerequisites

Before proceeding, ensure the following prerequisites are met:

  1. Install MindsDB locally via Docker or Docker Desktop.
  2. To use MindsDB Inference Endpoints within MindsDB, install the required dependencies following this instruction.
  3. Obtain the MindsDB Inference API key required to deploy and use MindsDB Inference Endpoints models within MindsDB. Follow the instructions for obtaining the API key.

​
Setup

Create an AI engine from the MindsDB Inference Endpoints handler.

CREATE ML_ENGINE mindsdb_serve
FROM mindsdb_inference
USING
      mindsdb_inference_api_key = 'api-key-value'

Create a model using mindsdb_serve as an engine.

CREATE MODEL mindsdb_inference_model
PREDICT answer
USING
      engine = 'mindsdb_serve',   -- engine name as created via CREATE ML_ENGINE
      model_name = 'model-name',              -- choose one of available models
      prompt_template = 'prompt-to-the-model' -- prompt message to be completed by the model

​
Usage

The following usage examples utilize mindsdb_serve to create a model with the CREATE MODEL statement.

Classify text sentiment using the Mistral 7B model.

CREATE MODEL mindsdb_inference_model
PREDICT sentiment
USING
   engine = 'mindsdb_serve',
   model_name = 'mistral-7b',
   prompt_template = 'Classify the sentiment of the following text as one of `positive`, `neutral` or `negative`: {{text}}';

Query the model to get predictions.

SELECT text, sentiment
FROM mindsdb_inference_model
WHERE text = 'I love machine learning!';

Here is the output:

+--------------------------+-----------+
| text                     | sentiment |
+--------------------------+-----------+
| I love machine learning! | positive  |
+--------------------------+-----------+

​
Supported Models

For an overview of the models supported, visit the following docs. This list will help you quickly identify the right models for your needs.

​
Troubleshooting Guide

Authentication Error

  • Symptoms: Failure to authenticate to MindsDB Inference Endpoints.
  • Checklist:
    1. Make sure that your MindsDB Inference Endpoints account is active.
    2. Confirm that your API key is correct.
    3. Ensure that your API key has not been revoked.

SQL statement cannot be parsed by mindsdb_sql

  • Symptoms: SQL queries failing or not recognizing table and model names containing spaces or special characters.
  • Checklist:
    1. Ensure table names with spaces or special characters are enclosed in backticks. Examples:
      • Incorrect: 
        SELECT input.text, output.sentiment
FROM integration.travel data AS input
JOIN mindsdb_inference_model AS output
      • Incorrect: 
        SELECT input.text, output.sentiment
FROM integration.'travel data' AS input
JOIN mindsdb_inference_model AS output
      • Correct: 
        SELECT input.text, output.sentiment
FROM integration.`travel data` AS input
JOIN mindsdb_inference_model AS output

Was this page helpful?

MonkeyLearnOllama