The upcoming release 2.1 of AlgoTrader will include an embedded-in-memory H2 database.
Benefits of the embedded-in-memory H2 database:
- Reduces the duration of back-test runs by 30% – 50%
- Allows multiple parallel simulations on the same machine without needing multiple database instances
- Eliminates the necessity to install a physical database (MySql, Oracle, MS SQL, etc.)
This alone makes AlgoTrader-based simulations considerably easier and faster.
Power of combining AlgoTrader with Amazon EMR
By combining AlgoTrader 2.1 with the Amazon Elastic Map Reduce (EMR) service, you will be able to test new quantitative trading strategies more quickly than with any other algorithmic trading platform available.
Using Amazon EMR computational-extensive tasks can be distributed across a resizable cluster of Amazon EC2 instances. An AlgoTrader-based trading strategy can be configured to run many simulations in parallel, distributed across an Amazon EMR cluster.
You will be able to test your algorithmic trading strategy in minutes rather than weeks. You will be able to profit from emerging trading opportunities before anyone else.
Example FX Trading Strategy Optimization
Our example trading strategy does about 20 FX trades per day. We would like to simulate the strategy over 15 years using 1-minute ticks. This will result in about 5.5 million market data events that need to be processed during each simulation run and about 60,000 trades that need to be processed by the database. Depending on the complexity of the trading strategy and the number of trading rules involved, a typical simulation run might take around 5 minutes.
Let’s say that we would like to optimize this strategy based on 3 parameters, and we would like to try 20 values for each parameter. We will, therefore, need to carry out 8,000 simulation runs to find the optimal parameter set for our strategy. If we do the simulations sequentially, we will need nearly 1 month to complete all simulations.
However, if we distribute these 8,000 simulation runs to the Amazon EMR infrastructure, we can reduce this time tremendously. With an Amazon EMR cluster containing 200 Amazon EC2 instances of type m1.small, each one doing 4 simulations in parallel – resulting in 48 simulation runs per server per hour – we would be able to finish the entire simulation in just about 1 hour of processing time.
This is 1 hour of processing time instead of 1 month doing it the traditional way. During this 1 hour, the AlgoTrader / Amazon EMR infrastructure will have processed 44 billon market data events and around 500 million trades. Based on the current pricing of Amazon EMR for m1.small instances of USD 0.075 per hour, the entire simulation would have cost only USD 15!
A Word of Caution on Optimizing Algorithmic Trading Strategies
Even though it is necessary to do back-testing to a certain extent, over-optimizing a quantitative strategy can be dangerous. It might be tempting to pick that one-single parameter set with an outstanding performance over all other parameter sets, but this will likely lead to a live trading performance that is much lower than the simulation. It is, therefore, an absolute necessity to carefully analyze all simulation runs and pick parameter sets that are as stable as possible (flat hills instead of steep peaks!).