scanning and backtesting stock market on 1 min timeframe

D

delta_1

I have a stock trading strategy in mind which I would like to backtest and trade.
This proves to be rather difficult as I need to scan the entire stock market for a pattern on 1 min bars in real time.
Regarding the backtest it seems equally difficult to find the proper approach as a realistic backtest would cover the next smaller timeframe like seconds or ticks.

Has anyone tried something like this?
Is there any software capable enough so it wouldn't need to be coded entirely from scratch?

Here is what I found out so far:

real time trading:

  • Multicharts could be capable
  • polygon provides real time data but it doesn't seem like multicharts is working with them
  • IQFeed restricts the amount of stocks that can be streamed so I would have to choose a subset of stocks that are most likely to pop up in my scanner

backtesting:

  • polygon seems to have all the historic data - including one seconds bars - I need but it doesn't seem like multicharts is working with them
  • IQFeed has very little data < 1min but enough data going back years on the one minute level + IQFeed has a great reputation
  • it would be huge amount of data i.e. I would have to backtest a subset of the market e.g. 2000 stocks and probably test those stocks over a month and then backtest the next month and so on...

Has anyone here ever tried something remotely similar?
I do have quite restrictive filters so the approach would only trade about 5-10 stocks a day.

Can't tell whether that is still way too demanding for the backtesting software or the data provider. Would a backtest like that crash the software?


Feedback very much appreciated if you have ever done something remotely similar especially regarding the backtesting process.
 
Too much data. You have to reduce your set of tickers to <= 500, or even <= 100 or so.
I think you need a data source that can provide near-time 1-min whole-market snapshot data (or of tickers in a select watchlist).
Maybe ORATS offers such data. Check their web. They are sponsor here (see link at bottom) and use the search function on the main ET page...
At least for options they offer (also) such snapshot data. For stocks this should much easier be possible, since stock data is much less than options data (options data is IMHO at least 15x more than stock data).

And: most of the data vendors out there don't grasp the fact that most of their advanced clients need such snapshot data of the whole market, not streaming data for single stocks etc.
These advanced clients, like in your case, need it for creating TopN lists and scanning for trade signals, not for constant monitoring of a single or a few tickers...
They just don't get it! :)
And they also seem not to know that compressed data significantly saves bandwidth and saves on the required # of download servers --> CDN...
 
Last edited:
delta_1 said:
The US stock market but "only" Nasdaq stocks would be fine as well...
More...

Normally brokers offer a streaming service that you can connect to via a web API. They have limits that will vary around the 100 symbols mark.

https://interactivebrokers.github.io/tws-api/market_data.html

if you need over 2500 symbols to scan a full market in order to run your scanner you would need to use several brokers, but it looks like an impossible task if you don't get a special service for it.

Maybe there are better services for trading firms in order to get this data, but I am not aware of it.
 
delta_1 said:
The US stock market but "only" Nasdaq stocks would be fine as well...
More...
So you would want to scan include stonks like TSLA (pre-Split) @ $926 and $1 stonks equally. Not very sensical. Finetuning to a level that is both practical and tactical is how best to use scans.
 
SunTrader said:
So you would want to scan include stonks like TSLA (pre-Split) @ $926 and $1 stonks equally. Not very sensical. Finetuning to a level that is both practical and tactical is how best to use scans.
More...

yeah, it doesn't look like a very clever scanner to run in the first place.
 
You must log in or register to reply here.
Back
Top