LiteLLM
This documentation describes the integration of MindsDB with LiteLLM, a framework that simplifies access to models of various providers.
Prerequisites
Before proceeding, ensure the following prerequisites are met:
- Install MindsDB locally via Docker or Docker Desktop.
- To use LiteLLM within MindsDB, install the required dependencies following this instruction.
- Obtain the API key of the model provider, if required.
Setup
Create an AI engine from the LiteLLM handler.
Create a model using litellm
as an engine.
The parameters include:
-
engine
is the LiteLLM engine created based on the LiteLLM handler with theCREATE ML_ENGINE
statement. -
model
is the one of the models supported by LiteLLM. See the complete list of the supported providers and models here. -
base_url
is an optional parameter that stores the base URL for accessing models. -
api_key
stores the API key of the provider whose model is used. -
prompt_template
stores the instructions to the model.
Usage
Here is how to create and use models through LiteLLM in MindsDB.