Build an Application Handler

In this section, you’ll find how to add new application integrations to MindsDB.

Prerequisite

You should have the latest version of the MindsDB repository installed locally. Follow this guide to learn how to install MindsDB for development.

What are API Handlers?

Application handlers act as a bridge between MindsDB and any application that provides APIs. You use application handlers to create databases using the CREATE DATABASE statement. So you can reach data from any application that has its handler implemented within MindsDB.

Database Handlers

To learn more about handlers and how to implement a database handler, visit our doc page here.

ML Handlers

To learn more about handlers and how to implement a machine learning (ML) handler, visit our doc page here.

Creating an Application Handler

You can create your own application handler within MindsDB by inheriting from the APIHandler class.

By providing the implementation for some or all of the methods contained in the APIHandler class, you can interact with the application APIs.

Core Methods

Apart from the __init__() method, there are five core methods that must be implemented. We recommend checking actual examples in the codebase to get an idea of what goes into each of these methods, as they can change a bit depending on the nature of the system being integrated.

Let’s review the purpose of each method.

Method	Purpose
`_register_table()`	It registers the data resource in memory. For example, if you are using Twitter API it registers the `tweets` resource from `/api/v2/tweets`.
`connect()`	It performs the necessary steps to connect/authenticate to the underlying system.
`check_connection()`	It evaluates if the connection is alive and healthy.
`native_query()`	It parses any native statement string and acts upon it (for example, raw syntax commands).
`call_application_api()`	It calls the application API and maps the data to pandas DataFrame. This method handles the pagination and data mapping.

Authors can opt for adding private methods, new files and folders, or any combination of these to structure all the necessary work that will enable the core methods to work as intended.

Other Common Methods

Under the mindsdb.integrations.utilities library, contributors can find various methods that may be useful while implementing new handlers.

API Table

Once the data returned from the API call is registered using the _register_table() method, you can use it to map to the APITable class. The APITable class provides CRUD methods.

Method	Purpose
`select()`	It implements the mappings from the ast.Select and calls the actual API through the `call_application_api`.
`insert()`	It implements the mappings from the ast.Insert and calls the actual API through the `call_application_api`.
`update()`	It implements the mappings from the ast.Update and calls the actual API through the `call_application_api`.
`delete()`	It implements the mappings from the ast.Delete and calls the actual API through the `call_application_api`.
`add()`	Adds new rows to the data dictionary.
`list()`	List data based on certain conditions by providing FilterCondition, limits, sorting and target fields.
`get_columns()`	It maps the data columns returned by the API.

Implementation

Each application handler should inherit from the APIHandler class.

Here is a step-by-step guide:

Implementing the __init__() method:

This method initializes the handler.

def __init__(self, name: str):
    super().__init__(name)
    """ constructor
    Args:
        name (str): the handler name
    """

    self._tables = {}

Implementing the connect() method:

The connect() method sets up the connection.

def connect(self) -> HandlerStatusResponse:
    """ Set up any connections required by the handler
    Should return output of check_connection() method after attempting
    connection. Should switch self.is_connected.
    Returns:
        HandlerStatusResponse
    """

Implementing the check_connection() method:

The check_connection() method performs the health check for the connection.

def check_connection(self) -> HandlerStatusResponse:
    """ Check connection to the handler
    Returns:
        HandlerStatusResponse
    """

Implementing the native_query() method:

The native_query() method runs commands of the native API syntax.

def native_query(self, query: Any) -> HandlerResponse:
    """Receive raw query and act upon it somehow.
    Args:
        query (Any): query in native format (str for sql databases,
            dict for mongo, api's json etc)
    Returns:
        HandlerResponse
    """

Implementing the call_application_api() method:

This method makes the API calls. It is not mandatory to implement this method, but it can help make the code more reliable and readable.

def call_application_api(self, method_name:str = None, params:dict = None) -> DataFrame:
    """Receive query as AST (abstract syntax tree) and act upon it somehow.
    Args:
        query (ASTNode): sql query represented as AST. Can be any kind
            of query: SELECT, INSERT, DELETE, etc
    Returns:
        DataFrame
    """

Exporting the `connection_args` Dictionary

The connection_args dictionary contains all of the arguments used to establish the connection along with their descriptions, types, labels, and whether they are required or not.

The connection_args dictionary should be stored in the connection_args.py file inside the handler folder.

The connection_args dictionary is stored in a separate file in order to be able to hide sensitive information such as passwords or API keys.

By default, when querying for connection_data from the information_schema.databases table, all sensitive information is hidden. To unhide it, use this command:

set show_secrets=true;

Here is an example of the connection_args.py file from the GitHub handler where the API key value is set to hidden with "secret": True.

from collections import OrderedDict

from mindsdb.integrations.libs.const import HANDLER_CONNECTION_ARG_TYPE as ARG_TYPE


connection_args = OrderedDict(
    repository={
        "type": ARG_TYPE.STR,
        "description": " GitHub repository name.",
        "required": True,
        "label": "Repository",
    },
    api_key={
        "type": ARG_TYPE.PWD,
        "description": "Optional GitHub API key to use for authentication.",
        "required": False,
        "label": "Api key",
        "secret": True
    },
    github_url={
        "type": ARG_TYPE.STR,
        "description": "Optional GitHub URL to connect to a GitHub Enterprise instance.",
        "required": False,
        "label": "Github url",
    },
)

connection_args_example = OrderedDict(
    repository="mindsdb/mindsdb", 
    api_key="ghp_xxx", 
    github_url="https://github.com/mindsdb/mindsdb"
)

Exporting All Required Variables

The following should be exported in the __init__.py file of the handler:

The Handler class.
The version of the handler.
The name of the handler.
The type of the handler, either DATA handler or ML handler.
The icon_path to the file with the database icon.
The title of the handler or a short description.
The description of the handler.
The connection_args dictionary with the connection arguments.
The connection_args_example dictionary with an example of the connection arguments.
The import_error message that is used if the import of the Handler class fails.

A few of these variables are defined in another file called __about__.py. This file is imported into the __init__.py file.

Here is an example of the __init__.py file for the GitHub handler.

from mindsdb.integrations.libs.const import HANDLER_TYPE

from .__about__ import __version__ as version, __description__ as description
from .connection_args import connection_args, connection_args_example
try:
    from .github_handler import (
        GithubHandler as Handler,
        connection_args_example,
        connection_args,
    )

    import_error = None
except Exception as e:
    Handler = None
    import_error = e

title = "GitHub"
name = "github"
type = HANDLER_TYPE.DATA
icon_path = "icon.svg"

__all__ = [
    "Handler", "version", "name", "type", "title", "description",
    "import_error", "icon_path", "connection_args_example", "connection_args",
]

The __about__.py file for the same GitHub handler contains the following variables:

__title__ = "MindsDB GitHub handler"
__package_name__ = "mindsdb_github_handler"
__version__ = "0.0.1"
__description__ = "MindsDB handler for GitHub"
__author__ = "Artem Veremey"
__github__ = "https://github.com/mindsdb/mindsdb"
__pypi__ = "https://pypi.org/project/mindsdb/"
__license__ = "MIT"
__copyright__ = "Copyright 2023 - mindsdb"

Check out our Application Handlers!

To see some integration handlers that are currently in use, we encourage you to check out the following handlers inside the MindsDB repository:

And here are all the handlers available in the MindsDB repository.

Contribute

​What are API Handlers?

​Creating an Application Handler

​Core Methods

​API Table

​Implementation

​Exporting the connection_args Dictionary

​Exporting All Required Variables

​Check out our Application Handlers!