The LLM() Function - MindsDB

MindsDB provides the LLM() function that lets users incorporate the LLM-generated output directly into the data queries.

Prerequisites

To use the LLM() function with MindsDB, choosing one of the available model providers and define the following environment variables.

Here are the environment variables for the OpenAI provider:

LLM_FUNCTION_MODEL_NAME
LLM_FUNCTION_TEMPERATURE
LLM_FUNCTION_MAX_RETRIES
LLM_FUNCTION_MAX_TOKENS
LLM_FUNCTION_BASE_URL
OPENAI_API_KEY
LLM_FUNCTION_API_ORGANIZATION
LLM_FUNCTION_REQUEST_TIMEOUT

Note that the values stored in the environment variables are specific for each provider.

Here are the environment variables for the Anthropic provider:

LLM_FUNCTION_MODEL_NAME
LLM_FUNCTION_TEMPERATURE
LLM_FUNCTION_MAX_TOKENS
LLM_FUNCTION_TOP_P
LLM_FUNCTION_TOP_K
LLM_FUNCTION_DEFAULT_REQUEST_TIMEOUT
LLM_FUNCTION_API_KEY
LLM_FUNCTION_BASE_URL

Note that the values stored in the environment variables are specific for each provider.

Here are the environment variables for the Anyscale provider:

LLM_FUNCTION_MODEL_NAME
LLM_FUNCTION_TEMPERATURE
LLM_FUNCTION_MAX_RETRIES
LLM_FUNCTION_MAX_TOKENS
LLM_FUNCTION_BASE_URL
LLM_FUNCTION_API_KEY
LLM_FUNCTION_PROXY
LLM_FUNCTION_REQUEST_TIMEOUT

Note that the values stored in the environment variables are specific for each provider.

Here are the environment variables for the LiteLLM provider:

LLM_FUNCTION_MODEL_NAME
LLM_FUNCTION_TEMPERATURE
LLM_FUNCTION_API_BASE
LLM_FUNCTION_MAX_RETRIES
LLM_FUNCTION_MAX_TOKENS
LLM_FUNCTION_TOP_P
LLM_FUNCTION_TOP_K

Note that the values stored in the environment variables are specific for each provider.

Here are the environment variables for the Ollama provider:

LLM_FUNCTION_BASE_URL
LLM_FUNCTION_MODEL_NAME
LLM_FUNCTION_TEMPERATURE
LLM_FUNCTION_TOP_P
LLM_FUNCTION_TOP_K
LLM_FUNCTION_REQUEST_TIMEOUT
LLM_FUNCTION_FORMAT
LLM_FUNCTION_HEADERS
LLM_FUNCTION_NUM_PREDICT
LLM_FUNCTION_NUM_CTX
LLM_FUNCTION_NUM_GPU
LLM_FUNCTION_REPEAT_PENALTY
LLM_FUNCTION_STOP
LLM_FUNCTION_TEMPLATE

Note that the values stored in the environment variables are specific for each provider.

Here are the environment variables for the Nvidia NIMs provider:

LLM_FUNCTION_BASE_URL
LLM_FUNCTION_MODEL_NAME
LLM_FUNCTION_TEMPERATURE
LLM_FUNCTION_TOP_P
LLM_FUNCTION_REQUEST_TIMEOUT
LLM_FUNCTION_FORMAT
LLM_FUNCTION_HEADERS
LLM_FUNCTION_NUM_PREDICT
LLM_FUNCTION_NUM_CTX
LLM_FUNCTION_NUM_GPU
LLM_FUNCTION_REPEAT_PENALTY
LLM_FUNCTION_STOP
LLM_FUNCTION_TEMPLATE
LLM_FUNCTION_NVIDIA_API_KEY

Note that the values stored in the environment variables are specific for each provider.

Here are the environment variables for the MindsDB provider:

LLM_FUNCTION_MODEL_NAME
LLM_FUNCTION_PROJECT_NAME

To use MindsDB as a provider, create a model in a project within MindsDB and use its name in the LLM_FUNCTION_MODEL_NAME environment variable and the project name in the LLM_FUNCTION_PROJECT_NAME environment variable.

Usage

You can use the LLM() function to simply ask a question and get an answer.

SELECT LLM('How many planets are there in the solar system?');

Here is the output:

+------------------------------------------+
| llm                                      |
+------------------------------------------+
| There are 8 planets in the solar system. |
+------------------------------------------+

Moreover, you can the LLM() function with your data to swiftly complete tasks such as text generation or summarization.

SELECT
    comment,
    LLM('Describe the comment''s category in one word: ' || comment) AS category
FROM example_db.user_comments;

Here is the output:

+--------------------------+----------+
| comment                  | category |
+--------------------------+----------+
| I hate tacos             | Dislike  |
| I want to dance          | Desire   |
| Baking is not a big deal | Opinion  |
+--------------------------+----------+

Was this page helpful?

Suggest edits Raise issue

Custom Functions CTEs

On this page

Prerequisites
Usage