Professional Backtesting Engine?

S

ScoobyStoo

Afternoon all,

I wonder if anyone could help me. I'm currently performing a complete evaluation of the technical elements of my algo operation. Currently, I'm looking at my backtesting setup (i.e. market data replay and order execution simulation).

When I started out a few years ago, I couldn't find any retail platforms which offered a decent backtesting engine. The closest thing was the market replay functionality in NinjaTrader, but it was too limited for my purposes (e.g. only offered one second resolution on replayed market data, no ability to define matching engine logic). So, I took a few months to roll my own backtesting engine that would replay the market data I purchased from CME DataMine and simulate order executions.

I've used this with good results for the last couple of years. On the plus side, since it's home grown, it's totally customisable. However, it has a very basic GUI and quite frankly is a pain in the arse to maintain and enhance. I have a big enough codebase on my hands already without wanting the additional hassle.

Since backtesting is an essential element in all algo strategy development, I'm thinking there must be some decent commercial products out there that fit the bill. I'll be fucked sideways though if I can find them. Have had a good trawl through the interweb to see what's available and I can't find anything.

The key requirements are fairly simple:

1) Be able to replay imported market data in industry standard formats (e.g. FIX)

2) Be able to simulate execution of orders (e.g. tracking queue position of limit orders in a FIFO book)

3) Be able to display executions on a simple price vs. time chart (for quick eyeballing purposes)

4) Be able to perform statistical analysis of execution data

Am quite happy to pay the money for a institutional grade platform if it does the job properly.

Would be grateful for any suggestions.

Thanks in advance...
 
Bloomberg has a decent platform for this - but if you are talking HFT or at least high intra-day volume you still need to code quite a bit.

There is no real standard because everyone's data is different, each strategy is different, etc. and it's hard to build a backtest engine that's all-encompassing.

Why not build your own?
 
Quote from WinstonTJ:

Bloomberg has a decent platform for this - but if you are talking HFT or at least high intra-day volume you still need to code quite a bit.

There is no real standard because everyone's data is different, each strategy is different, etc. and it's hard to build a backtest engine that's all-encompassing.
More...

Thanks for the reply, but I don't understand your reasoning.

Market data is market data, it can be bought in industry standard formats from several sources. A good backtesting platform (for exchange traded products) just needs to expose a simple API which mimics the operation of an exchange matching engine. It needs to publish market data messages and process order generation/modification/cancellation messages. It should be completely decoupled from the strategy logic itself. Hence, why backtesting lends itself to a commercial out-the-box solution.

Quote from WinstonTJ:Why not build your own? [/B]
More...
I already have done this. I'm at the stage now though where it's a pain to maintain and my time/money is better spent elsewhere.
 
Many thanks to those of you that have PMed me.

The following products have been suggested as worthy of further investigation:

Alphacet - Discovery (http://www.alphacet.com/alphacet-discovery.html)
Barra - Aegis System (http://www.barra.com/support/aegis/)
CapitalIQ - ClariFI (http://www.capitaliq.com/home/what-we-offer/how-you-can-get-it/clarifi.aspx)
FactSet - Alpha Testing (http://www.factset.com/products/im/alphatesting)
Logical Information Machines (http://www.lim.com)
Quantitative Analytics - MarketQA (http://www.qaisoftware.com/quantitative-analytics/marketQA/)
Silexx - TimeBend (http://www.silexx.com/timebend)

If anyone has any experience with these solutions that they'd care to share (positive or negative) then I'm sure there are quite a few of us here that would be interested.
 
You may want to look at tradelink. TradeLink is open source and has super fast tick-based backtesting :

*supports level1 and level2
*supports both single instrument and N instrument simulations
*simulation playback throughputs 250,000-1million ticks per second
* supports execution simulation options mentioned below


1) Be able to replay imported market data in industry standard formats (e.g. FIX)

There are 10 different formats supported out of the box. You can customize these to add additional formats.

2) Be able to simulate execution of orders (e.g. tracking queue position of limit orders in a FIFO book)

This is included in the execution engine, for raw stop/limit/market and stop market orders. TradeLink can also support OCO/TIF/Brackets/Trailing Stops and other types of orders.


3) Be able to display executions on a simple price vs. time chart (for quick eyeballing purposes)

This is included in the off-the-shelf kadina application. Because tradelink is open source, you can also take the same charting and simulation components and embed it in your application.

4) Be able to perform statistical analysis of execution data

There are around 40-50 summary statistics calculated by default (winners/losers/averagewinner/sharpe/maxdrawdown/etc).

Also the orders and trades by default can be exported to either excel or csv for further analysis in matlab/R/etc.

tradelink has been in existence since 2008, has tens of thousands of downloads and hundreds of developers in the community.

google tradelink project or tradelink.org for more info
 
Quote from braincell:

Why not just hire someone to improve your own code, and maintain it etc?
More...

Yeah, I've considered that but it'd only marginally reduce the hassle. I'd still have to:

(1) Spend time finding a reliable, skilled freelance developer who would be prepared to take on the ad-hoc work (anyone half decent is going to cost me at least £400 a day and I'll have to pay them a retainer to ensure they are available for a few days each month)

(2) Spend time familiarising them with the application functionality and code base (this was developed as a quick and dirty app with no documentation since I knew I would be the only user)

(3) Spend time specing out and testing any future development work

It's just ball ache that I'll happily pay several grand to avoid if possible. I want a fully supported COTS product that I can easily install on a few machines in my analysis rig.

My operation is focused on developing and running low latency algo strats. I really don't want to be in the business of developing backtesting software.
 
You must log in or register to reply here.
Back
Top