Sentiment Analysis with MindsDB and OpenAI using MQL
Introduction
In this blog post, we present how to create OpenAI models within MindsDB. This example is a sentiment analysis where we infer emotions behind a text. The input data is taken from our sample MongoDB database.
Prerequisites
To follow along, you can sign up for an account at cloud.mindsdb.com. Alternatively, head to MindsDB documentation and follow the instructions to manually set up a local instance of MindsDB via Docker or pip.
How to Connect MindsDB to a Database
We use a collection from our MongoDB public demo database, so let’s start by connecting MindsDB to it.
You can use Mongo Compass or Mongo Shell to connect our sample database like this:
test> use mindsdb
mindsdb> db.databases.insertOne({
'name': 'mongo_demo_db',
'engine': 'mongodb',
'connection_args': {
"host": "mongodb+srv://user:MindsDBUser123!@demo-data-mdb.trzfwvb.mongodb.net/",
"database": "public"
}
})
Tutorial
In this tutorial, we create a predictive model to infer emotions behind a text, a task also known as sentiment analysis.
Now that we’ve connected our database to MindsDB, let’s query the data to be used in the example:
mindsdb> use mongo_demo_db
mongo_demo_db> db.amazon_reviews.find({}).limit(3)
Here is the output:
{
_id: '63d013b5bbca62e9c7774b1d',
product_name: 'All-New Fire HD 8 Tablet, 8 HD Display, Wi-Fi, 16 GB - Includes Special Offers, Magenta',
review: 'Late gift for my grandson. He is very happy with it. Easy for him (9yo ).'
}
{
_id: '63d013b5bbca62e9c7774b1e',
product_name: 'All-New Fire HD 8 Tablet, 8 HD Display, Wi-Fi, 16 GB - Includes Special Offers, Magenta',
review: "I'm not super thrilled with the proprietary OS on this unit, but it does work okay and does what I need it to do. Appearance is very nice, price is very good and I can't complain too much - just wish it were easier (or at least more obvious) to port new apps onto it. For now, it helps me see things that are too small on my phone while I'm traveling. I'm a happy buyer."
}
{
_id: '63d013b5bbca62e9c7774b1f',
product_name: 'All-New Fire HD 8 Tablet, 8 HD Display, Wi-Fi, 16 GB - Includes Special Offers, Magenta',
review: 'I purchased this Kindle Fire HD 8 was purchased for use by 5 and 8 yer old grandchildren. They basically use it to play Amazon games that you download.'
}
Let’s create a model collection to identify sentiment for all reviews:
mongo_demo_db> use mindsdb
mindsdb> db.models.insertOne({
name: 'sentiment_classifier',
predict: 'sentiment',
training_options: {
engine: 'openai',
prompt_template: 'describe the sentiment of the reviews strictly as "positive", "neutral", or "negative". "I love the product":positive "It is a scam":negative "{{review}}.":'
}
})
Default Model
When you create an OpenAI model in MindsDB, it uses the gpt-3.5-turbo
model by default. But you can use the gpt-4
model as well by passing it to the model-name
parameter in the training_options
clause.
In practice, the insertOne
method triggers MindsDB to generate an AI collection called sentiment_classifier
that uses the OpenAI integration to predict a field named sentiment
. The model is created inside the default mindsdb
project. In MindsDB, projects are a natural way to keep artifacts, such as models or views, separate according to what predictive task they solve. You can learn more about MindsDB projects here.
The training_options
key specifies the parameters that this handler requires.
- The
engine
parameter defines that we use theopenai
engine. - The
prompt_template
parameter conveys the structure of a message that is to be completed with additional text generated by the model.
If you’re using a local deployment, in order to use the OpenAI integration, you need to install the openai
Python package. You can do this by running the following command:
pip install openai
We use the OpenAI engine to create a model in MindsDB. Please note that the api_key
parameter is optional on cloud.mindsdb.com but mandatory for local usage/on-premise. You can obtain an OpenAI API key by signing up for OpenAI’s API services on their website. Once you have signed up, you can find your API key in the API Key section of the OpenAI dashboard. You can then pass this API key to the MindsDB platform when creating models.
To create a sentiment_classifier
model in MindsDB, you can run the following code:
mongo_demo_db> use mindsdb
mindsdb> db.models.insertOne({
name: 'sentiment_classifier',
predict: 'sentiment',
training_options: {
engine: 'openai',
prompt_template: 'describe the sentiment of the reviews strictly as "positive", "neutral", or "negative". "I love the product":positive "It is a scam":negative "{{review}}.":',
api_key: 'YOUR_OPENAI_API_KEY' -- MANDATORY FOR LOCAL MODE
}
})
Alternatively, you can create a MindsDB ML engine that includes the API key, so you don’t have to enter it each time. Please note that this command should be executed from MindsDB.
CREATE ML_ENGINE openai2
FROM openai
USING
api_key = 'YOUR_OPENAI_API_KEY';
Please note that it is required to provide an OpenAI API key when using MindsDB Pro version.
If you want to use the OpenAI API key provided by MindsDB, please confirm your email. Alternatively, you have the option to utilize your own OpenAI API key by specifying it in the api_key
parameter.
Once the insertOne
method has started execution, we can check the status of the creation process with the following query:
mindsdb> db.getCollection('models').find({
'name': 'sentiment_classifier'
})
It may take a while to register as complete depending on the internet connection. Once the creation is complete, the behavior is the same as with any other AI collection – you can query it either by specifying synthetic data in the actual query:
mindsdb> db.sentiment_classifier.find({
review: 'It is ok.'
})
Here is the output data:
{
sentiment: 'neutral',
review: 'It is ok.'
}
Or by joining with a collection for batch predictions:
mindsdb> db.sentiment_classifier.find(
{
'collection': 'mongo_demo_db.amazon_reviews'
},
{
'sentiment_classifier.sentiment': 'sentiment',
'amazon_reviews.review': 'review'
}
).limit(3)
Here is the output data:
{
sentiment: 'positive',
review: 'Late gift for my grandson. He is very happy with it. Easy for him (9yo ).'
}
{
sentiment: 'positive',
review: "I'm not super thrilled with the proprietary OS on this unit, but it does work okay and does what I need it to do. Appearance is very nice, price is very good and I can't complain too much - just wish it were easier (or at least more obvious) to port new apps onto it. For now, it helps me see things that are too small on my phone while I'm traveling. I'm a happy buyer."
}
{
sentiment: 'positive',
review: 'I purchased this Kindle Fire HD 8 was purchased for use by 5 and 8 yer old grandchildren. They basically use it to play Amazon games that you download.'
}
The amazon_reviews
collection is used to make batch predictions. Upon joining the sentiment_classifier
model with the amazon_reviews
collection, the model uses all values from the review
field.
Leverage the NLP Capabilities with MindsDB
By integrating databases and OpenAI using MindsDB, developers can easily extract insights from text data with just a few SQL commands. These powerful natural language processing (NLP) models are capable of answering questions with or without context and completing general prompts.
Furthermore, these models are powered by large pre-trained language models from OpenAI, so there is no need for manual development work. Ultimately, this provides developers with an easy way to incorporate powerful NLP capabilities into their applications while saving time and resources compared to traditional ML development pipelines and methods. All in all, MindsDB makes it possible for developers to harness the power of OpenAI efficiently!
MindsDB is now the fastest-growing open-source applied machine-learning platform in the world. Its community continues to contribute to more than 70 data-source and ML-framework integrations. Stay tuned for the upcoming features - including more control over the interface parameters and fine-tuning models directly from MindsDB!
Experiment with OpenAI models within MindsDB and unlock the ML capability over your data in minutes. Remember to sign-up for a free demo account and follow the tutorials, perhaps this time using your data.
Finally, if MindsDB’s vision to democratize ML sounds exciting, head to our community Slack, where you can get help and find people to chat about using other available data sources, ML frameworks, or writing a handler to bring your own!
Follow our introduction to MindsDB’s OpenAI integration here. Also, we’ve got a variety of tutorials that use MySQL and MongoDB:
- Sentiment Analysis in MySQL
- Question Answering in MySQL
- Text Summarization in MySQL
- Question Answering in MongoDB
- Text Summarization in MongoDB
What’s Next?
Have fun while trying it out yourself!
- Bookmark MindsDB repository on GitHub.
- Sign up for a free MindsDB account.
- Engage with the MindsDB community on Slack or GitHub to ask questions and share your ideas and thoughts.
If this tutorial was helpful, please give us a GitHub star here.