Quick Start

1. Get your API keys

To use our API please get in touch with our team at hyperdata.so/contact. We can set you up with a plan that best fits your needs.

2. Install the client library

The best way to interact with our API is to use our python client library. We will be releasing official libraries in other languages soon. The client library helps with breaking up large datasets into more manageable chunks that can be easily processed by the API

Python

pip install --upgrade hdata

3. Import the package

Import the hyperdata package at the top of your file

import hdata as hd

3. Load data sources

To load raw data into the API to be transformed, you can pass the path to the data in as an argument. We will provide you with the transformation keys for each data source you are using. Each key corresponds to an AI-generated procedure that maps source data to the destination format.

The "Source" class can handle JSON, CSV, and PARQUET file formats and will automatically detect and interpret the structure of the source data

first_source = hd.Source(transformation_key='*****', data='path/to/your/dataset')
second_source = hd.Source(transformation_key='*****', data='path/to/your/dataset')
third_source = hd.Source(transformation_key='*****', data='path/to/your/dataset')

4. Transform

Combine all of your source objects into a list that will be passed to the transform function. If you are only transforming one source, you can pass that as a list with a single item.

sources = [first_source, second_source, third_source]

The transform function will compress, chunk, and stream the data to the HyperData API. Under the hood, the transform function uses the Transform endpoint. The function returns three data frames adhering to our strict data format (more on the output format here).

entity, attribute, record = hd.transform(sources, auth_token='*****')

Thats it! Transforming data is super simple with HyperData