Apache Pinot

This is the implementation of the Pinot data handler for MindsDB.

Apache Pinot is a real-time distributed OLAP database designed for low-latency query execution even at extremely high throughput. Apache Pinot can ingest directly from streaming sources like Apache Kafka and make events available for querying immediately.

​

Prerequisites

Before proceeding, ensure the following prerequisites are met:

1. Install MindsDB locally via Docker or Docker Desktop.
2. To connect Apache Pinot to MindsDB, install the required dependencies following this instruction.
3. Install or ensure access to Apache Pinot.

​

Implementation

This handler was implemented using the pinotdb library, the Python DB-API and SQLAlchemy dialect for Pinot.

The required arguments to establish a connection are as follows:

• host is the host name or IP address of the Apache Pinot cluster.
• broker_port is the port that the Broker of the Apache Pinot cluster is running on.
• controller_port is the port that the Controller of the Apache Pinot cluster is running on.
• path is the query path.

​

Usage

In order to make use of this handler and connect to the Pinot cluster in MindsDB, the following syntax can be used:

CREATE DATABASE pinot_datasource
WITH
    engine = 'pinot',
    parameters = {
      "host":"localhost",
      "broker_port": 8000,
      "controller_port": 9000,
      "path": "/query/sql",
      "scheme": "http"
    };

You can use this established connection to query your table as follows:

SELECT *
FROM pinot_datasource.example_tbl;