Unified Data Model
The Entity-Attribute-Value Model
All data transformed with the HyperData API maps to the same EAV model.
Universally Unique Identifiers
We use UUIDs to index each entity and attribute for a couple of key reasons
Since UUIDs are globally unique, you can easily append new data without risking an index conflict where two entities or attributes have the same ID
The record table can be easily linked to the entity and attribute table
The entity table
The entity table can store just about anything—people, products, places, customers, websites, and so much more. It’s like a universal container for things in the real world that you have information on.
The attribute table
The attribute table serves as a detailed ledger, cataloging every data point related to an entity. Think of it as the specifics or the "about" details of anything you're tracking. This could be a person’s age, a product's price, a post's click-through rate, or any other characteristic.
The record table
The record table ties everything together by linking an entity and attribute to a specific time with a value. The "datetime" column can represent specific points in time such as "2023-04-01 11:30:13" or a stretch of time such as a whole month "2022-03". The "record_value" column stores the actual value of the entity-attribute pair with either a float, integer, string, or bytes-like object.
Example
The following is an example of the entity, attribute, and record tables that would be returned as the output of a transformation. The data in this table is completely fictitious and for demonstration purposes only.
| entity_uuid | entity_name | entity_description |
|---|---|---|
7ba0f437-be4d-71c8-12e6-006ce5a65f2e | 81 Bay Street, Toronto, ON M5J 0E7 | CIBC square located in downtown Toronto |
eef438f7-e448-aa48-c1f8-db568d27b591 | twitter.com/cibc | The offical X page of CIBC |
The Value of Standardization
Standardizing data is a crucial process that brings uniformity to information, making it easier to collect, analyze, and share across various platforms and systems. This harmonization not only enhances the accuracy and reliability of data analysis but also facilitates interoperability between different technologies and sectors. By adhering to common standards, organizations can ensure that their data is compatible, comparable, and easily integrated, leading to more informed decision-making, streamlined operations, and the ability to leverage insights across different datasets.
By consistently transforming data to the same model, HyperData eliminates the complexity and variability often encountered in data management. This standardization ensures that developers and organizations can reliably query and build applications on a stable foundation, without the concern of pipelines breaking due to data format inconsistencies. HyperData reduces maintenance efforts and provides the assurance that data structures will remain consistent, enabling more predictable and efficient application performance and data analysis.
Last updated