Tick data backtesting shows improvement in automated strategy test results

E

endsongs

Atrade only has 1 minute data for backtesting. So, I'm paying a fee every month to a vendor for access to a system with backtesting software that uses tick data. There is a definite improvement in the test results. Both the tick and 1 min backtest looked back several months using several different stocks. Hopefully, these more accurate testing results will allow me to improve my automated trading strategy.
 
Are you assuming that you get the price you want or you assume some slippage?
And the improvement is in the theory. Suppose a simple alway in system that buy over the MA30 and sell under. With the close on the 1 minute bar you buy at a higher price than the MA30 and same thing when you reverse. With the tick you get IN faster on the right side so the results are better. On the other side, you may get kicked out of stops on little blips.
Ticks have to be better because they represent more the reality.
 
I quit backtesting with bar data entirely. I test on ticks and build bars in arrays [C#]. That way I can know exactly what is going on at every moment and I can know that my deployed strategy will run exactly as tested. There can be assumptions made in the testing software regarding how bars are built and evaluated that is not reality... also by testing on ticks I can be absolutely sure that my code when backtesting never accesses any tick that has not happened yet, believe it or not it's easy to access a bar parameter before it happens in some software...
 
Problem with bars is how to handle trades in the bar. A good system having tick data AND order book can simulate folls pretty perfectly (heck, Ninja even simulates your position in the waiting list for a limit order - no joke).

Bars simpply... have a problem here. WHere do they put in the fill?
 
Tradestation has "look inside the bar backtesting".. you have to have tick data and bar data on the instrument and their tester will look inside the bar and calculate everything tick by tick.. but that still leaves me with complexeties of trying to figure out exactly how they do things.. It's more clear and simpler for me to use Ninja or Openquant and build arrays of bars. But then I can't use any of the canned indicators and the headache can be that I have to write all my own indicators that run on the psudo bars I build... but at the end of the day I achieve clarity on the entire process, I don't have to translate anything out of "Tradestation speak" or "Ninja speak" because the entirety of what is in the code is my creation and under my control.

I got very good at coding up backtests but I quit backtesting and went to screen trading about three months ago and I advanced the state of my trading by huge leaps, backtesting is too sterile of an environment, it is bypassing the fantastic computer between our ears for some little feeble tool to take over... I heartily recommend going entirely to screen trading occasionally to get a fresh view on things...
 
Quote from Eight:

I quit backtesting with bar data entirely. I test on ticks and build bars in arrays [C#]. That way I can know exactly what is going on at every moment and I can know that my deployed strategy will run exactly as tested. There can be assumptions made in the testing software regarding how bars are built and evaluated that is not reality... also by testing on ticks I can be absolutely sure that my code when backtesting never accesses any tick that has not happened yet, believe it or not it's easy to access a bar parameter before it happens in some software...
More...

It's funny. I did the same thing. But ended up building in a lot of features to C# for the code to allow simple conversion of EasyLanguage, AmiBroker, and NinjaTrader strategies and indicators since I had some from both. Plus, now that it's public, tons of other features to make the C# more user-friendly that were requested by beta testers.

But the strange thing is that almost all the customers of tickzoom insist on using bar data. So they forced me, against my will, to add standard bar data features. Argh. I personally hate bar data. Switching to tick only data made my automated trading profitable. It truly makes a huge difference to both understanding the markets and more accurate testing.

There's 3 main reasons other people still insist on using bar data.

1. For some securities, getting good historical tick data can be prohibitively expensive.

2. Almost none of the platforms out there off-the-shelf can handle raw tick data of any significant volume like 5 or 10 years. (That has changed with tickzoom which really zooms through tick data.)

3. And even with fast software and tick data, many users still want to pre-process the tick data down to bars even if it means 10 second bars because it greatly speeds up processing and therefore reduces research time into new strategy ideas.

You can think of this approach as "drafting" a trading strategy on bar data and then "fine tuning" on tick data. They find that to be a much faster option.

Users who have tick data also suggest a hybrid approach as an enhancement to tickzoom and our initial investigation says that will work.

The hybrid idea is to process using 2 data streams simultaneously for a single symbol. One is condensed data (bar-like data) and the other is raw ticks.

The idea is to primarily process everything using the condensed data for lightening fast performance but only refer to the tick data stream upon attempts to fill orders like stops or limits.

Since the frequency of actual filled orders is a tiny fraction of the amount of tick data, that will greatly improve performance and allow...

.. combining the strategy drafting and fine tuning into a single step.

Of course, that doesn't exist yet but's not far way since we already using similar logic for data generated from bars. But in that case, rather than look at a real tick data stream to get the fill, it simply inserts "fake" ticks to fill the orders much like TradeStation similates order fills.

In fact, one institutional user has had a developer using several of their strategies to do side-by-side results comparisons between tradestation and tickzoom and now (after reporting and some defects fixed) confirms that it gives identical bar data results.

I still find the use of bar data revolting. But such are the demands of building customer satisfaction.

I do look forward to the hybrid approach since that will speed up my own research by orders of magnitude.

Sincerely,
Wayne
 
Quote from Eight:

I got very good at coding up backtests but I quit backtesting and went to screen trading about three months ago and I advanced the state of my trading by huge leaps, backtesting is too sterile of an environment, it is bypassing the fantastic computer between our ears for some little feeble tool to take over... I heartily recommend going entirely to screen trading occasionally to get a fresh view on things...
More...

The exact same thing happened to me. Discretionary day trading for a few weeks revolutionized my understanding of the data that I look at on charts.

One more thing happened in my progress...did this happen to you yet?

I took my understanding and built an automated trading system. It was far from perfect or user-friendly back then but it worked. I ran it on Forex with very small position size at first.

Now when you REALLY trade an automated strategy using real money, it also changes everything about how you understand the markets and automated trading. It was whole new "paradygm shift" in understanding like when I did screen trading like you.

I'll share the biggest realizations from that. 1. The strategy when live trading must build and maintain confidence that it's "working properly" during draw downs. 2. It must handle money management in such a way as to have some kind of regular returns. That can be daily, weekly, monthly, whatever you pick.

The reason for these is that when my automated system went into any kind of drawdown--losing "real" money--it always forced me to reevaluate the strategy to make sure there wasn't some kind of error in the code.

So I would run the strategy historically offline on the same data to compare with the live running system.

If it gave different results that scared me into pulling the live one and spending time to debug. Sometimes there really were bugs.

Using tick data made a huge difference there also because it means my historical tests started matching almost perfectly with the real time system.

Another problem was that sometimes during the draw down it showed a market situation or condition that I failed to notice during historical testing.

Even when it matched the historical comparison it forced a reexamination into the entire strategy.

So now, when developing any new strategy, I find it necessary to carefully examine "every single trade" to understand why it lost money and how it profited. Of course, I start with the biggest losses and work backwards for efficiency.

The other problem (2 above) is that you need to define your goal. When running in real life, real human time is passing. Calendar days, or weeks. And what seemed like good results on paper me be very boring or uninteresting in real life.

Like, for example, let's say your system has periods where it goes a few weeks without trading. That may like fine or be unnoticed on a 5 year chart.

But when you're going day by day watching the market make all kinds of (allbeit small) gyrations while your strategy sits on the sidelines--not a single trade--it gets very annoying.

My strategies now always make at least a few traded every single day.
And my platform gives me daily, weekly, and month stats that also highlight any days with zero trades or the biggest winning days and losing days, etc.

My strategies now return consistent profit every single month even it's very small and over 80% of every week. The maximum loss per day, per week, and month is set as well as the daily, weekly, and monthly profit target.

I personally feel there's no excuse for going an entire month without making money trading. There's so many opportunities during any given month--that would be leaving money a lot of money on the table.

Hey, if you made X in a month, week, or day which is more than your average months why not take it and run so as to avoid losing it back during that period. Hey, remember, in the future that might be your grocery money if you quit your day job or your monthly payroll if you run a trading firm.

In short, after really trading automated it forces you to begin to look at historical testing from a more "human" approach to how it will serve your goals in real life and real human time, day after day.

Wayne
 
Thanks for the replies. There is very little slippage in the trades, so the key for analyzing the strategy is knowing the exact entry and exit points. Eight, I'm not at the point yet where writing custom code is an option. The accurate equity curve using the tick data with the canned software is definitely all that is needed now. 1 minute data was probably ok a few years ago. But with the market constantly reversing so quickly during the day, I'm only using tick data from here on for important tests.
 
I had not thought of returning to automation, I got pretty excited with the knowledge gains I made recently and have not thought beyond that. I'll get with Tickzoom eventually.

I want an airtight security situation before I expose any intellectual property in the form of an automated system to the internet at all... I can do that but it's a real engineering project. I have to hit the screens and rebuild my capital base real soon or the wife is going to do a murder-suicide here and those are so messy...
 
You must log in or register to reply here.
Back
Top