3forge data platform
High-performance columnar database
The 3forge Center brings real-time data and decision-making to application developers with a shortest learning curve and industry-leading performance.
Our flagship offering
High-performance columnar database at a fraction of the cost
With its proprietary columnar database, 3forge delivers industry-leading analytical flexibility for streaming and time-sensitive workloads, including:
- Real-time Tables (RDB)
- Version control
- Historical Tables (HDB)
- Scale horizontally
- Stream-processing
- Real-time subscriptions
- Comprehensive language
- Nearest joining (asof)
- Integrated development environment
- Lambda support
3forge adds unique features that address difficult data problems that software engineers encounter, including:
| 3forge Features | What They Bring to Engineers |
|---|---|
| Real-time streaming analytics | Delta based triggers provide fast and efficient aggregations, joins, projections, etc. |
| Intuitive yet comprehensive language | The combination of a Java/.NET syntax with SQL and Python. |
| Replication and scalability | Enable replication, processing sharding, and web load-balancing without additional code. |
| Streaming to Historical Tables | Support for streaming inserts into Historical tables with immediate consistency. |
| Historical Table Sparse Updates | Ability to insert, update, and delete on an ad-hoc basis. |
| Advanced conflation | Consume data at exponential rates and down-sample for downstream consumers. |
| Sophisticated data virtualization | 100+ Adapters for connecting to remote data. |
| Self-discovery schemas | Auto-accept schema updates from dynamic remote data sources. |
| REST admin and query APIs | Automate monitoring and various support activities, and create custom endpoints. |
| Remote Procedure Calls | Provides the ability to invoke functionality in remote systems. |
| Debugger and Intuitive error codes | Add break statements and step through custom logic. |
| Testing and code coverage tools | Powerful analysis of code invoked, including runtimes. |
| Query planner | Pre-compiler and planner to achieve optimized execution. |
| Multi-threaded queries | High-performance data access in multi-user environments. |
| No-fuss historical table schema changes | Alter schemas instantly without the need to backfill data. |
| Web-based development studio | Open a browser and start developing sophisticated applications. |
Database Stats
Metrics that empower businesses
Engineered for real-time and hyperscale, the 3forge data platform has been running in production at the largest Wall Street firms since 2013:
Historical database capacity
> 10 Trillion rows
Historical database columns
> 1,000
Real-time Database Throughput
> 2 Million ops/sec
Real-time Database Latency
< 100 µs
< 1ms
Queries round trip
Are you hitting a hurdle in your data app?
Contact our Solutions team to review how 3forge can help.
Out of the box
Convenient and reliable data replication and scaling
When processing and archiving high velocity data sets, 3forge makes improving performance and resiliency straightforward and safe.
Database replication
3forge delivers simple and effective data replication between primary and warm standby Center databases with a simple configuration. The standby can check for the health of the primary and take over in case of failure with all the data already loaded in memory.
Database load balancing
Replication can also be used for load balancing between multiple Centers based on routing rules in the Relay. These multiple Centers increase redundancy and scalability. The flexibility of the platform allows for more advanced deployments, such as multi-region and even global replication.
How data replication is configured
Show table <some_table> FROM <connected_center>
Every aspect of the tiered 3forge architecture can implement data replication for redundancy and scalability:
- Different feed adapters ensure backup data pipes are available to the 3forge Relay component
- 3forge Relays can disseminate data updates to multiple centers
- 3forge Centers can separate data processing like map reductions from subscription management to scale, on the same physical host or different ones
- 3forge Web components further scale to service user requests from dashboard, alerts or reports
- Relevant data is replicated at every level with additional optimizations
- The 3forge Web Balancer further ensures the appropriate dispatching of UI requests across 3forge Web servers.
One normalized source for
Effective Data Consumption and Virtualization
With over 35 database adapters and 15 feed adapters, 3forge can consume data from virtually any source, and present it as a unified data mode to data consumers. By insulating downstream apps from idiosyncratic data formats, 3forge accelerates project deployments, implements explicit access control. and performance management. Once in 3forge, the data can be queried via the REST API, JDBC, as well as through bidirectional libraries in .Net, Python, Java, and RPC between the 3forge Relays and external systems.
Database adapters
Feed handlers
Programmatic access libraries
Database Adapters
AMI Database
Flat File
AMI Shell
Excel
Fred
Quandl
Oracle
Microsoft SQL
MySQL
IBM DB2
SAP
SybaseIQ
KX
Hadoop
MongdoDB
R
Impala
PostgreSQL
Phoenix
Chronicle Queue
Netezza
MemSQL
Couchbase
SQLite
REST API
Snowflake
Hazelcast
Greenplum
HP Vertica
Hive
Deephaven
Bloomberg
Apache Ignite
Feed handlers
AMI Database
Flat File
AMI Shell
Excel
Fred
Quandl
Oracle
Microsoft SQL
MySQL
IBM DB2
SAP
SybaseIQ
KX
Hadoop
MongdoDB
R
Impala
PostgreSQL
Phoenix
Chronicle Queue
Netezza
MemSQL
Couchbase
SQLite
REST API
Snowflake
Hazelcast
Greenplum
HP Vertica
Hive
Deephaven
Bloomberg
Apache Ignite
Programmatic access libraries
REST API
JDBC
.NET
Python
Java
Google RPC
Hyperscale
Real-time Data Aggregation
The 3forge platform is particularly well-suited for time-sensitive, large scale data transformations and comparisons. Its inherent modularity allows for a broad array of deployment configurations that can be tailored for each specific use case. In addition, the pricing model of 3forge proves particularly beneficial for companies needing substantial scalability and flexibility while keeping costs predictable and affordable.
Price feed aggregation example
A 3forge Center can receive market data from different feeds, reconcile the streams in real time, and propagate the fastest price to downstream systems. These exchange feeds may be split between data centers for additional redundancy.
Order aggregation example
Similarly, a 3forge Center can receive orders from different order management systems representing asset classes or market access pathways, including no-touch and low-touch order flow, and aggregate them to achieve a unified representation of market activity for transparency, risk management, exposure control, and even internal order crossing.
Fine-tuned for high performance
Optimized for the largest data loads
Delta Processing
Rather than reprocessing or retransmitting the entire dataset each time an update occurs, 3forge only handles changes (deltas) in the data. This approach dramatically improves performance and efficiency in real-time systems.
Conflation
When replicating data across its tiered architecture, 3forge can be configured to send every update or instead send the latest one on an agreed interval. This, effectively discards intermediate values, dramatically reducing the burden for downstream consumers.
Summarization
3forge can calculate and store regular and delta-based summary metrics, including averages, sums, counts, mins, or max values to compress data volumes, allow for trend analysis, and support time-based analytics like moving averages.
Decoration
3forge supports automated data decoration through event-driven triggers invoked during data operations, enabling dynamic enrichment, validation, and propagation of data in response to table-level changes.
Delta-based processing
Message conflation
Summarization
Decoration
Perform immediate actions with
Complex Event Processing
3forge offers real-time tables and database-native functionality to allow business logic to be executed on the fly based on data updates. In combination with conflation, these triggers can smartly and efficiently process only as much work as needed.
The following triggers are supported out of the box:
- Aggregation, to create and update aggregation tables
- Projection for filtered tables
- Join to create joined tables
- Decorate to enrich the data with additional data from another table
- Relay for sending messages through the 3forge relay
- AMIScript to run custom scripts upon any insert, update, or delete activity.
How a script trigger is configured
Create trigger <trigger_name> Oftype Amiscript ON <table_name> [PRIORITY priority] Use...
How an aggregation trigger is configured
Create trigger <trigger_name> Oftype Aggregate ON <source_Table>,<target_Table> Use...
How a projection trigger is configured
Create trigger <trigger_name> Oftype Projection ON <sourceTables>,<targetTable> Use...
A reliable solution for
Petabyte historical
database for Archiving
While the highest performance is achieved with real-time databases, 3forge also offers columnar historical tables capable of holding trillions of rows. Stored on drives and supporting partitioning, these tables are designed for storing large volumes of data at high speed with fast retrieval, all while using the same SQL syntax as real-time tables.
Data from historical tables can be queried and loaded into a real-time table where the full breadth of querying optimizations can be accessed, including joins with other tables.
3forge uniquely supports the following features in its Historical database:
- Ability to populate via streaming updates
- Ad-hoc inserts, updates, and deletes after the initial data is loaded
- Large column counts and heavy data types like blob fields
Security and compliance
Granular Entitlements
and permissioning
3forge provides a robust, security-first architecture designed to meet the stringent access control and auditability requirements of modern financial and enterprise environments. Authentication is extensible via industry-standard Single Sign-On (SSO) protocols, including full support for SAML 2.0 and OAuth 2.0, allowing seamless integration with identity providers such as Okta, Azure AD, and Ping Identity.
Authorization is enforced through a highly granular Role-Based Access Control (RBAC) framework, with policy definitions supporting nested roles, group hierarchies, and data-level entitlements. Permissions can be applied down to the field or cell level, ensuring strict segregation of data visibility across users and teams.
All data‐whether at rest or in motion‐is protected via end-to-end encryption (TLS for transmission, AES for storage), with secure replication protocols ensuring consistency and integrity across distributed deployments. These capabilities form the backbone of a compliant, audit-ready platform suitable for regulated industries such as finance, insurance, and government.
- Single Sign-on (SSO)
- Security Assertion Markup Language (SAML)
- Open Authorization (OAuth)
- Role-based Access (RBAC)
- End-to-end encryption covering data transmission and storage
Do you need a data platform to enhance scalability, resilience, and agility?