Strategy development building blocks

D

dakr

Hi,

This post covers trading platform, data sources and programming. Let me know if I need to move to another section.

I have been using a mixed bag of tools and data sources to develop trading strategies. Mostly manual data download of equities and forex data. These are store in a local mysql database. I then use Matlab and MQL5 to do generate signals, perform backtesting and various analyses. I use use spotfire for interactive visualisation.

It's been quite slow to test new strategies since I tend to write new code for each trading idea. I'm looking to put in place building blocks for much more efficient strategy development and hopefully migration to deployment.

1) Data sources
I'd like the option of exploring strategies in equities,options, futures and forex and narrow down later. I've been through the forum threads and looked into the recommended providers by other users. Here is the comparison:
https://imgur.com/a/D0fVj8J

I spent just half an hour investigating so let me know if any errors. At the moment it looks like IQ feed has the most data available. Any other views?

2) Data storage
I like my current set up where data is stored in a local mysql database which allows me to slice and dice using sql queries. Let me know if you have other thoughts.

3) Analysis/strategy development
Have been using Matlab so far. Looking at using NinjaTrader which looks to be a very capable platform and/or python.

Ninjatrader looks like it nicely wraps the whole strategy development lifecycle and can migrate to live trading easily. Also hooks up to iqfeed and IB. I will want to do lots of number crunching. Given NT scripts are in C#, I'm assuming it will require more programming effort compared to Python? Further I would have to learn C#.

On the other hand python has a huge, evolving maths library and is more general purpose so I assume provides more flexibility, but how is it more flexible vs NT? Intraday is the quickest trading timeframe I would consider so python (vs c#/c++) speed is not relevant?

I assume there are existing python code/library for backtesting? Where can I find these?

So the biggest question for me is NT vs/and python, or any other options?

Thanks.
 
Last edited:
The table of data sources specifies that you need tick-by-tick data. Why do you need this?
I guess this requirement is also why you left out IB, as they don't really provide that?

The rest of your post may generate a whole lot of discussions. Should you use a database or simply flat text files? If you use a database, is SQL the right choice, or are there more suitable choices? Nowadays seems Python the "language of choice" but some claim that it has speed limitations and that other languages might be a better choice in case of intensive number crunching applications. And so on.
 
dakr said:
I'd like the option of exploring strategies in equities,options, futures and forex and narrow down later. I've been through the forum threads and looked into the recommended providers by other users. Here is the comparison:
More...

Are you retail? If so, you don't need tick data because your execution will never be good enough. Quandl is fine.

dakr said:
I like my current set up where data is stored in a local mysql database which allows me to slice and dice using sql queries. Let me know if you have other thoughts.
More...

SQL is fine.

dakr said:
Have been using Matlab so far. Looking at using NinjaTrader which looks to be a very capable platform and/or python.

Ninjatrader looks like it nicely wraps the whole strategy development lifecycle and can migrate to live trading easily. Also hooks up to iqfeed and IB. I will want to do lots of number crunching. Given NT scripts are in C#, I'm assuming it will require more programming effort compared to Python? Further I would have to learn C#.
More...

MATLAB is completely fine. Don't waste your time with anything else. The only reason I'd suggest switching is if you wanted to run code remotely - but if that's the case and you have a full-fledged MATLAB license you can just use MATLAB compiler and not worry about it.

dakr said:
I assume there are existing python code/library for backtesting? Where can I find these?
More...

The most famous is zipline.
 
Python / zipline and quantopian are the best backtesters out these days (IMO obviously).

Unless you are doing HFT, where milliseconds matter (and lets be honest, your not) Python is plenty fast.

The learning curve can be steep, but once you get it then zipline is extremely powerful.
 
GlobalMacro90 said:
Python / zipline and quantopian are the best backtesters out these days (IMO obviously).

Unless you are doing HFT, where milliseconds matter (and lets be honest, your not) Python is plenty fast.

The learning curve can be steep, but once you get it then zipline is extremely powerful.
More...

Python is plenty fast for most definitions of hft too... you just have to do it right. Check it out "numba"
https://numba.pydata.org/ . Additionally, you can use one of the many c/c++ python wrappers out there for your execution time critical code.

Odds are, the latency between the exchange and wherever you're doing your processing is far greater than your "tick to trade" lag....so dont waste your time worrying about execution speed when you dont know if its an issue yet. DO use programming best practices though(i.e avoid dynamic allocation of arrays, etc...).
 
Thanks all. Surprised no support for ninjatrader, given many contributors here use it. What are the pros/cons of NT vs python? What drew me to NT was it can hook into datafeeds and IB straightaway so no messing about if it comes to deployment.

Is it straightforward to plug into live data from python and hook into broker API?
 
gaussian said:
Are you retail? If so, you don't need tick data because your execution will never be good enough. Quandl is fine.
More...

Was thinking tick data will enable me to generate whatever bar I need. Had a brief look at Quandle - they don't appear to have intraday data?

Any thoughts on the free quantopian dataset:
https://www.quantopian.com/data/quantopian/us_equity_pricing

gaussian said:
MATLAB is completely fine. Don't waste your time with anything else. The only reason I'd suggest switching is if you wanted to run code remotely - but if that's the case and you have a full-fledged MATLAB license you can just use MATLAB compiler and not worry about it.
More...

The nuances of each trading idea has required me to write a different code for each backtest. Matlab doesn't have a library/framework for backtesting so would be inefficient which is the main issue I'm trying to address. Deployment may also be clunky depending on the broker.

Bottom line is I want to minimise time/effort for strategy development and don't want to reinvent the wheel.
 
You must log in or register to reply here.
Back
Top