Unified Data Model

The Entity-Attribute-Value Model

All data transformed with the HyperData API maps to the same EAV model.

Universally Unique Identifiers

We use UUIDs to index each entity and attribute for a couple of key reasons

• Since UUIDs are globally unique, you can easily append new data without risking an index conflict where two entities or attributes have the same ID

• The record table can be easily linked to the entity and attribute table

The entity table

The entity table can store just about anything—people, products, places, customers, websites, and so much more. It’s like a universal container for things in the real world that you have information on.

The attribute table

The attribute table serves as a detailed ledger, cataloging every data point related to an entity. Think of it as the specifics or the "about" details of anything you're tracking. This could be a person’s age, a product's price, a post's click-through rate, or any other characteristic.

The record table

The record table ties everything together by linking an entity and attribute to a specific time with a value. The "datetime" column can represent specific points in time such as "2023-04-01 11:30:13" or a stretch of time such as a whole month "2022-03". The "record_value" column stores the actual value of the entity-attribute pair with either a float, integer, string, or bytes-like object.

Example

The following is an example of the entity, attribute, and record tables that would be returned as the output of a transformation. The data in this table is completely fictitious and for demonstration purposes only.

Entity

entity_uuid	entity_name	entity_description
7ba0f437-be4d-71c8-12e6-006ce5a65f2e	81 Bay Street, Toronto, ON M5J 0E7	CIBC square located in downtown Toronto
eef438f7-e448-aa48-c1f8-db568d27b591	twitter.com/cibc	The offical X page of CIBC

Attribute

attribute_uuid	attribute_name	attribute_description
fefa0a20-ec87-03b4-288a-34cb4b7a68ce	average rent per square foot	The average rent per square foot for all of the units in a building in USD
8d966b22-53a9-1708-6c86-04959e152243	total follower count	The total number of followers on a social media page

Record

entity_uuid	attribute_uuid	datetime	record_value
7ba0f437-be4d-71c8-12e6-006ce5a65f2e	fefa0a20-ec87-03b4-288a-34cb4b7a68ce	2024-03	45
eef438f7-e448-aa48-c1f8-db568d27b591	8d966b22-53a9-1708-6c86-04959e152243	2023-03-01 12:13:34	109453

The Value of Standardization

Standardizing data is a crucial process that brings uniformity to information, making it easier to collect, analyze, and share across various platforms and systems. This harmonization not only enhances the accuracy and reliability of data analysis but also facilitates interoperability between different technologies and sectors. By adhering to common standards, organizations can ensure that their data is compatible, comparable, and easily integrated, leading to more informed decision-making, streamlined operations, and the ability to leverage insights across different datasets.

By consistently transforming data to the same model, HyperData eliminates the complexity and variability often encountered in data management. This standardization ensures that developers and organizations can reliably query and build applications on a stable foundation, without the concern of pipelines breaking due to data format inconsistencies. HyperData reduces maintenance efforts and provides the assurance that data structures will remain consistent, enabling more predictable and efficient application performance and data analysis.