DBT
To integrate your predictions into your DBT workflow, use the dbt-mindsdb adapter:
Adapter for | Documentation | Install from PyPi |
---|---|---|
MindsDB (dbt-mindsdb) | Profile Setup | pip install dbt-mindsdb |
Usage¶
Initialization¶
- Create dbt project:
dbt init [project_name]
- Configure your profiles.yml,
currently MindsDB only supports user/password authentication, as shown below on:
~/.dbt/profiles.yml
mindsdb:
outputs:
dev:
type: mindsdb
database: 'mindsdb'
host: '127.0.0.1'
port: 47335
schema: 'mindsdb'
password: ''
username: 'mindsdb'
target: dev
mindsdb:
outputs:
dev:
type: mindsdb
database: 'mindsdb'
host: 'cloud.mindsdb.com'
port: 47335
schema: '[dbt schema]'
username: '[mindsdb cloud username]'
password: '[mindsdb cloud password]'
target: dev
Key | Required | Description | Example |
---|---|---|---|
type |
✔️ | The specific adapter to use | mindsdb |
host |
✔️ | The MindsDB (hostname) to connect to | cloud.mindsdb.com |
port |
✔️ | The port to use | 3306 or 47335 |
schema |
✔️ | Specify the schema (database) to build models into | The MindsDB datasource |
username |
✔️ | The username to use to connect to the server | mindsdb or mindsdb cloud user |
password |
✔️ | The password to use for authenticating to the server | pass |
Create predictor¶
Create table_name.sql (table_name will be used as the name of the predictor):
Parameter | Required | Description | Example |
---|---|---|---|
materialized |
✔️ | Always predictor |
predictor |
integration |
✔️ | Name of integration to get data from and save result to. It must be created in MindsDB beforehand. | photorep |
predict |
✔️ | Field to be predicted | name |
predict_alias |
Alias for predicted field | predicted_name |
|
using |
Configuration options for trained model | ... |
{{
config(
materialized='predictor',
integration='photorep',
predict='name',
predict_alias='predicted_name',
using={
'encoders.location.module': 'CategoricalAutoEncoder',
'encoders.rental_price.module': 'NumericEncoder'
}
)
}}
SELECT * FROM stores
Create predictions table¶
Create table_name.sql (If you need to specify schema, you can do it with a dot separator: schema_name.table_name.sql):
Parameter | Required | Description | Example |
---|---|---|---|
materialized |
✔️ | Always table |
table |
predictor_name |
✔️ | Name of predictor model from Create predictor |
store_predictor |
integration |
✔️ | Name of integration to get data from and save result to. It must be created in MindsDB beforehand. | photorep |
{{
config(
materialized='table',
predictor_name='store_predictor',
integration='photorep'
)
}}
SELECT a, bc FROM ddd WHERE name > latest
Note that each time dbt is run, the results table will be rewritten.
Testing¶
- Install dev requirements
pip install -r dev_requirements.txt
- Run pytest
python -m pytest tests/