Persistence
A partitioned columnar historical database engineered for trillions of rows of financial data, with seamless ingestion from real-time tables and the same SQL surface throughout.
Built for finance, sized for the world
Long-term storage that doesn't break the query model
Storing financial data over years, market data, trades, audit trails, regulatory records, demands more than capacity. It demands a storage layer that doesn't force trade-offs between scale, schema flexibility, and query performance. The 3forge Historical Database (HDB) is a columnar archive designed for that brief: trillions of rows on disk, partitioned storage, evolving schemas, and a query surface shared with the real-time tables it complements.
Petabyte historical database for archiving
While the highest query performance is achieved with real-time databases, 3forge also offers columnar historical tables capable of holding trillions of rows. Persisted to disk and supporting partitioning, these tables are designed for storing large volumes of data at high speed with fast retrieval, all while using the same SQL syntax as real-time tables.
Data from historical tables can be queried and loaded into a real-time table where the full breadth of querying optimizations can be accessed, including joins with other tables.
3forge uniquely supports the following features in its Historical database (HDB):
Large column counts and heavy data types
Support for large column counts and heavy data types including blob fields, without compromising query performance or storage efficiency.
Configurable storage strategies
Configurable storage strategies for each column, including four storage types for optimal performance. The system dynamically adapts storage types during optimization on a partition basis for disk efficiency and query speed based on actual data usage.
| Type | Description |
|---|---|
| FLAT | Fixed-length types like INT, FLOAT, DOUBLE |
| VARSIZE | Variable-length types like STRING and BINARY, up to 1 TB |
| BITMAP | Efficient for low-cardinality strings |
| PARTITION | Organizes rows into isolated partitions |
Schema management
Add, drop, or modify columns without impacting historical partitions. Partition columns are immutable, so careful planning is essential when designing the table schema. HDB ensures older partitions are mapped to new schemas seamlessly, preserving historical integrity.
Row-level operations
HDB supports UPDATE and DELETE clauses while preserving partition optimization. Significant changes within a partition are re-optimized automatically. Sort indexing further enhances query performance.
Archiving real-time data from streaming updates
Seamlessly move data from real-time tables into HDB using event, batch, or timer-driven approaches. This ensures historical records remain up to date without interrupting ongoing operations.
Next steps
Archive trillions of rows without compromising query performance.
Book a 30-minute demo with a 3forge solutions engineer.