High Performance Scripting in 3forge without the Complexities of Threading

New Capabilities

3forge's AMI script now includes additional support for concurrency and pausability without the complexity of traditional thread-based development.

Why it matters

The performance advantages of multi-threading are critical for applications connecting to remote data sources and processing large data sets. New concurrency and pausability support in AMI Script aim to optimize code that does not return immediately, including remote system calls as well as code that processes large data sets. Concurrency and Pausability allow 3forge to implement behaviors that perform like multi-threading, but lack the complexity and support burden.

Threading Explained

In Computer Science, a thread represents the processing of a single instruction stream by a processor (CPU). A single-threaded application is a contained set of instructions executed in a single stream. While some applications process and quickly terminate, other applications run indefinitely like a user interface (UI).

The term "Multi-Threading" comes into play when the computer is responsible for executing multiple instruction streams concurrently within an application. Threading is analogous to having one person writing a document versus multiple people. With one person, there is no need to coordinate updates or merge changes. With multiple people, the document may come together quicker, but the composition process requires coordination to prevent changes from being overwritten. And, further coordination is required to ensure that all updates are present before the document is complete.

Example: Single Threaded Trade Reporting Application

Example: Single Threaded Trade Reporting Application

The Trade Reporter is a single-threaded process that builds a "new trades" report every 30 seconds, by:

1. Querying for trade data (blocking call)
2. Querying for related counterparty reference data (blocking call)
3. Rendering a report file (e.g. CSV) based on both data sets
4. Publishing the report via email or SFTP (blocking call)

The reporting process happens sequentially as part of a single instruction stream. The Trade Reporter makes a blocking call to a Reference Data Manager that is hosted on a separate server. The Reference Data Manager's threading model is not relevant. However, it is important to understand that it listens for reference data requests and then builds and transmits a response, which takes precious time that slows the Trade Reporter down.

Performance Limitations

The single-threaded version of the Trade Reporter runs well for a while, until trade volumes and reference data volumes increase significantly. The process has to make two remote system calls to (1) get trade data, and (2) get client reference data. If the firm is now trading with 3x the number of counterparties and producing 5x the volume of trades, these calls will take longer and put the Trade Reporter's ability to build the report in under 30 seconds at risk.

Performance Tuning: Cache and Query Reference Data with a Separate Thread

In part because reference data does not change as often as trade data, a decision is made to employ a separate thread to cache the reference data, making the results available for fast query from a local Map-based data cache. This approach minimizes blocking during the report generation process.

Performance Advantages of Multi-Threading

Since the blocking call to the reference data server has been removed from the main execution stream, the Trade Reporter speeds up significantly as the calls to the reference data cache return immediately. The report can now be produced easily within the 30 second interval.

Performance Tuning: Cache and Query Reference Data with a Separate Thread

So far, so good, but as with the previously mentioned example of multi person document creation, the reference data cache updates must be coordinated or "thread-safe" in tech speak. If there is no coordination, the reporter may end up querying against an empty or partially filled cache that is in the process of being rebuilt. Coding mechanisms such as semaphores and synchronized blocks can help with this by forcing one thread at a time to perform operations on the cache. As a result, a thread may need to wait until the resource is available. If written correctly, the waiting will be minimal, but the resulting code is significantly more complex versus the single-threaded variant.

Asynchronous 3forge Development: You're already using it

Asynchronous execution via pausable, coroutine statements has been a seamless part of AMI script for years now. Many developers may not have realized it, but certainly appreciate not having their user interface freeze up when sending an email or executing one or more large database queries.

Coroutines offer the ability to suspend and resume execution without the complexity of threading. 3forge's implementation adheres to the async/await pattern, which aims to allow code to resemble synchronous design, but ultimately allows for asynchronous execution without adding threads. Therefore, a statement in 3forge is declared like any other synchronous code block, but instead of immediate execution, it is instead prepared for execution and queued by the active thread.

The Scope of a Pausable statement

When a method contains a pausable statement, it is considered a "pausable formula". All statements before the deferred statement are executed inline. The deferred statements, including subsequent code, are queued for later execution. Without a "dispatch" block (more on this below), the deferred statement boundary is determined to be the end of the method. Thus, a method that has two deferred statements will still execute sequentially unless concurrent blocks are used.

Concurrency

Concurrency allows multiple blocks of pausable code to be executed simultaneously. For example, if a method is required to query data from three data sources, a concurrent approach would allow for all sources to be queried at the same time and avoid "one after the other" sequential execution.

Increased Performance by Mixing Concurrency and Pausability

In 3forge, multiple pausable statements can be executed in parallel in a concurrent code block.

New AMI Scripting elements for Concurrency and Pausability

Concurrent execution is durable to failure

A dispatched block of code can fail and be isolated with appropriate error-handling, allowing the remaining streams of execution to complete and return.

When to use Pausable Statements

While powerful in many circumstances, there are scenarios where pausable statements do not make sense, including scenarios where a block of code is repeatedly executed in a tight loop (e.g. a column filter). To prevent system performance bottlenecks, 3forge does not allow pausable statements to be executed in lambda scenarios, including SQL statements and Web UI formulas.

When to use Pausable Statements

			//Return a hash of order data with additional counterparty and instrument data
			public Map hashEnrichedOrderData(List orders, List cptyRefDataSources){
			    Map ret = new Map();
			    Map instruments;
			    Map parties = new Map();
			
			    //load Instruments and Counterparty data sets in parallel
			    try {
			        Table tIns = USE DS="AMI" EXECUTE SELECT * FROM Instruments;
			        instruments = hashMapList(tIns.getRows(), "Symbol");
			    }
			    catch(Object errIns){ 
			        session.log("Error loading instruments: " + errIns); 
			        instruments = new Map();
			    }
			
			    concurrent {
			        for(int i = 0; i < cptyRefDataSources.size(); i++) {
			            dispatch {
			                try {
			                    List curDS = cptyRefDataSources.get(i);
			                    String dsName = curDS.get(0);
			                    String tableName = curDS.get(1);
			                    Table tParties = USE DS="${dsName}" EXECUTE SELECT * FROM ${tableName};
			                    parties.putAll(hashMapList(tParties.getRows(), "Account"));
			                }
			                catch(Object errParties){
			                    session.log("Error looking up Counterparties: " + errParties);
			                }
			            }
			        }
			    }
			
			    //hash orders by OrderID add cpty and instrument entries
			    Iterator iter = orders.iterator();
			    while(iter.hasNext()){
			        Map curOrder = iter.next();
			        curOrder.put("cptyInfo", parties.get("Account"));
			        curOrder.put("instInfo", instruments.get("Symbol"));
			        ret.put(curOrder.get("OrderID"), curOrder);
			    }
			
			    return ret;
			};
			
			//hash a list of maps by a given key.
			public Map hashMapList(List data, String key){
			    Map ret = new Map();
			    Iterator i = data.iterator();
			    while(i.hasNext()){
			        Map curItem = i.next();
			        ret.put(curItem.get(key), curItem);
			    }
			    return ret;
			};
			

Written by Andy George

Solutions Architect