This handler was implemented using the replicate library that is provided by Replicate.The required arguments to establish a connection are,
model_name: Model name which you want to access in MindsDB. e.g ‘air-forever/kandinsky-2’
version: version hash/id which you want to use in MindsDB.
api_key: API key from Replicate Platform you can found here.
Before you can use Replicate, it’s essential to authenticate by setting your API token in an environment variable named REPLICATE_API_TOKEN.
This token acts as a key to enable access to Replicate’s features.
Using pip:
If you’re working in a standard Python environment (using pip for package management), set your token as an environment variable by running the following command
in your terminal:On Linux, Mac:
Copy
Ask AI
export REPLICATE_API_TOKEN='YOUR_TOKEN'
On Windows:
Copy
Ask AI
set REPLICATE_API_TOKEN=YOUR_TOKEN
Using Docker:
For Docker users, the process slightly differs. You need to pass the environment variable directly to the Docker container when running it. Use this command:
Copy
Ask AI
docker run -e REPLICATE_API_TOKEN='YOUR_TOKEN' -p 47334:47334 -p 47335:47335 mindsdb/mindsdb
Again, replace ‘YOUR_TOKEN’ with your actual Replicate API token.
To use this handler and connect to a Replicate cluster in MindsDB, you need an account on Replicate. Make sure to create an account by following this link.To establish the connection and create a model in MindsDB, use the following syntax:
Copy
Ask AI
CREATE MODEL blipPREDICT textUSING engine = 'replicate', model_name= 'salesforce/blip', version ='2e1dddc8621f72155f24cf2e0adbde548458d3cab9f00c0139eea840d0ac4746', api_key = 'r8_BpO.........................';
You can use the DESCRIBE PREDICTOR query to see the available parameters that you can specify to customize your predictions:
Now, you can use the established connection to query your ML Model as follows:
Copy
Ask AI
SELECT *FROM mindsdb.blipWHERE image="https://images.unsplash.com/photo-1686847266385-a32745169de4"AND question="Is there lion in image?"USINGtask="visual_question_answering";
+------------+--------------------------------------------------------------+-------------------------+| text | image | question |+------------+--------------------------------------------------------------+-------------------------+| Answer: no | https://images.unsplash.com/photo-1686847266385-a32745169de4 | Is there lion in image? |+------------+--------------------------------------------------------------+-------------------------+
+---------------------------------------------------+--------------------------------------------------------------+| text | image |+---------------------------------------------------+--------------------------------------------------------------+| Caption: a bird is sitting on the back of a horse | https://images.unsplash.com/photo-1686847266385-a32745169de4 |+---------------------------------------------------+--------------------------------------------------------------+
Image Text Matching
Copy
Ask AI
SELECT *FROM mindsdb.blipWHERE image="https://images.unsplash.com/photo-1686847266385-a32745169de4"AND caption="Bird having horse Riding"USINGtask="image_text_matching";
OUTPUT
Copy
Ask AI
+-----------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+--------------------------+| text | image | caption |+-----------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+--------------------------+| The image and text is matched with a probability of 0.7730.The image feature and text feature has a cosine similarity of 0.3615. | https://images.unsplash.com/photo-1686847266385-a32745169de4 | Bird having horse Riding |+-----------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+--------------------------+
This is just an one model used in this example there are more with vast variation and use cases.
Also there is no limit to imagination, how can you use this.Note: Replicate provides only a few free predictions, so choose your predictions wisely. Don’t let the machines have all the fun, save some for yourself! 😉