Questioning backtest results

Quote from PocketChange:

Do not use last trade data... Your real world results will be skewed. Can you enter trades using market orders exclsuively?
More...

Thanks for the advice. The indicators I am using are mainly built on the typical of last bar, not on last trade.

The limit orders I am using are mainly calculated on the close price of last bar.

Yes I can technically do market orders on entry signals, but my success percentage decreases as some of my signals call for me to enter at a few ticks +/- the last close for highest percentage.

That last part concerns me as well, and is why I am looking forward to live testing as well w/paper account. But the way I understand Ninja, it only considers a market order filled if the price is penetrated, not just touched.

Thank you.

Mike
 
Quote from ctrlbrk:

IQFeed real-time is the source and the interval is 1 minute bars.

The entry orders are limit orders, the stop losses are market.

The average trade is 0.8min, but there are only enough 'signals' generated for approximately 2-3 trades a day.

Thanks for your input.

Mike
More...

If you are using 1 minute bars then your results are not valid. In writing a trading system you cannot buy and sell on the same bar (of any time frequency) because you do not know the order of the high and low for that bar. Rewrite the trading system logic to only allow a buy or a sell on a single bar.

If you were to trade your existing system against real time data the flaws would be quite visible.
 
Quote from jeb9999:

If you are using 1 minute bars then your results are not valid. In writing a trading system you cannot buy and sell on the same bar (of any time frequency) because you do not know the order of the high and low for that bar. Rewrite the trading system logic to only allow a buy or a sell on a single bar.

If you were to trade your existing system against real time data the flaws would be quite visible.
More...

I am not sure I am following, can you explain more please?

If you look at screen shot #2 it lists entry/exit times and the # of bars before the trade closed. The average is only 1 bar, but it is not the same bar it was entered on.

If I am still in trouble, please help me understand.

Thank you for the valuable info.

Mike
 
Ok I spoke to soon, I looked closer and I was wrong, it does show the same minute for enter/exit on some trades.

So, are you saying that say 12:01pm it decided to buy @ 901, and the tick data showed it hit 903 which was my profit target 30 seconds later, it is not valid if I sell @ 903?

Again ninja would only consider it a successful trade if the limit order penetrated the price, not just touched it.

I also am including 1 tick slippage on all calculations on the backtest, just in case.

If I am still missing this, I look forward to your advice to help me better understand why this scenario won't work.

Thank you.

Mike
 
Alright, so thinking about what you said... not knowing what order the low/high came in on that bar...

But wouldn't Ninja know, since I am providing it tick level data? Even though calculations are on 1 minute bars, it has the tick data for the backtests.

Thank you.

Mike
 
Quote from Random.Capital:

How are you ensuring that your entry/exit targets are actually hit? Being inside the OHLC is not enough.
More...

I guess that is the unknown, or at least I don't know how to find out except in real-time vs backtest.

I was basing it on the Ninja rules which say if you place a buy limit order @ 900, even if 900 is touched you will not get a fill until 899.75 comes along.

I do grasp what you guys are saying... OHLC ordering... but isn't Ninja supposed to use the tick data it has to figure that out?

Thank you.

Mike
 
Quote from ctrlbrk:

Hello,

I am new to ET, and would like some expert advice to help interpret my backtesting results.

Let's take one script in particular that I wrote. I like to be in front of my computer when it is running, so the time frame is 12:00pm to 3:59pm (market time). I allow multiple entries per direction, have defined 1 tick slippage, and the quantity is 1 contract per order.

When you run it against ES for Oct 1-Oct 20 (present day), I get the following results (see image attachments for complete details on this period, sorry for image quality, I had to compress a lot to get it under the forum file size limits):

39 trades, 92% profitable, max drawdown $(-217.50). Profit factor of 10.35, average time in market 0.8min.

When you run it against ES for Sep 1-Sep 30, I get the following results:
12 trades, 100% profitable, profit factor 99.00, average time in market 2.3min.

Now, this seems too good to be true. I will spend quite a bit of time in the coming days to continue running this against new market data. I only looked at about 3 days of charts to code my strategy script, and I know markets fluctuate dramatically over time. I already realize no strategy will work for extended periods of time, etc etc.

Finally, here is my question -- asking for advice from experts -- what should I make of these backtest results? What I am really asking is whether or not I can 'believe' the results as I see them, and assuming yes, would it not be realistic to expect similar results from the real market in the _very short term_ period (matter of days)?

Thanks for your input.

Mike
More...

Trust me, there isn't any better expert to ask than me. Your backtest is not long enough, and I suspect curve fitting just on that data. No, you absolutely should not expect similar results. Stretch it out past 8 months and I'm sure you'll think otherwise, guaranteed.
 
Quote from ctrlbrk:

Ok I spoke to soon, I looked closer and I was wrong, it does show the same minute for enter/exit on some trades.

So, are you saying that say 12:01pm it decided to buy @ 901, and the tick data showed it hit 903 which was my profit target 30 seconds later, it is not valid if I sell @ 903?

Again ninja would only consider it a successful trade if the limit order penetrated the price, not just touched it.

I also am including 1 tick slippage on all calculations on the backtest, just in case.

If I am still missing this, I look forward to your advice to help me better understand why this scenario won't work.

Thank you.

Mike
More...

If you are testing against tick data then you have prices in time order as they occurred and it is obvious that you don't buy and sell the same tick.

If you are testing against 1 minute (or any time period) bars you only have prices in time order as they occurred from one bar to the next bar. Thus you cannot buy and sell on the same bar and you must avoid doing so for trading system design.

You have to be certain that in real time the entry price occurred before the exit price and not vice versa. From what you have posted so far it is not clear to me exactly what you have done.
 
I'm not a Ninja expert but I do not believe that it looks inside the bar.

I test only on tick data and I build the bars in an array. That way there just never are ambiguities and bar interval can be a testable variable.
 
You must log in or register to reply here.
Back
Top