Manual backtesting

Martinsos · May 4, 2016

Hey guys,
I am hoping that some of the members might be able to help me with a bit of a research I am doing!
I am working on a tool for manual backtesting that enables you to observe your trade in the past. What I would like to know is what are the biggest problems and obstacles that you face regarding manual backtesting right now?

globalarbtrader · May 4, 2016

Martinsos said:
Hey guys,
I am hoping that some of the members might be able to help me with a bit of a research I am doing!
I am working on a tool for manual backtesting that enables you to observe your trade in the past. What I would like to know is what are the biggest problems and obstacles that you face regarding manual backtesting right now?

Dumb question no doubt, but what on earth is manual backtesting? Sounds like an oxymoron. You know like "successful day trader"*

GAT

* This is a joke. Please don't start on OT flame war.

Martinsos · May 4, 2016

Good question, I should have explained it better

! So compared to automated backtesting, where you define the trade/strategy and let algorithm run it for certain period of time and report the results, in manual backtesting you pick some specific moment in past yourself, create a trade there and observe what happens with it (thinkBack is a form of manual backtesting).
I am not 100% sure if "manual backtesting" is globally accepted term, but I did find it used on some other places, for example here and here.

globalarbtrader · May 4, 2016

Martinsos said:
Good question, I should have explained it better ! So compared to automated backtesting, where you define the trade/strategy and let algorithm run it for certain period of time and report the results, in manual backtesting you pick some specific moment in past yourself, create a trade there and observe what happens with it (thinkBack is a form of manual backtesting).
I am not 100! sure if "manual backtesting" is globally accepted term, but I did find it used on some other places, for example here.

Okay...Isn't the danger that you'd know what was going to happen in advance, so you'd be able to cheat?

So if I was doing this the most important thing would be to mask the information on the market by removing any identifying information; eg the name of the instrument, the dates (or shift to an arbitrary date) and the price (change to 100 plus the change in the real price over the chart window).

Forgive me if this is standard practice, I have never used one of these things and never will so I don't know why I'm contributing really.

GAT

1245 · May 4, 2016

globalarbtrader said:
Okay...Isn't the danger that you'd know what was going to happen in advance, so you'd be able to cheat?

I agree.

Martinsos · May 4, 2016

globalarbtrader said:
Okay...Isn't the danger that you'd know what was going to happen in advance, so you'd be able to cheat?

Yes you could, but it really depends on how you use it. It can be hidden in some way, as you said, but on the other hand, if you are using it to test strategy that you defined well (entry and exit rules), then it does not really matter if you know what will happen since you are just following the rules of your strategy.

globalarbtrader · May 4, 2016

Martinsos said:
Yes you could, but it really depends on how you use it. It can be hidden in some way, as you said, but on the other hand, if you are using it to test strategy that you defined well (entry and exit rules), then it does not really matter if you know what will happen since you are just following the rules of your strategy.

Then why not automate it? By running it manually it's possible to let bias creep in. If it's totally objective and that isn't possible, it can and should be automated. To manually test enough trades to get a significant result is going to require 1000's of hours of work.

I can understand the point of a tool like this if you don't have well defined rules (in which case you need to be really careful about foreknowledge and mask the charts as much as possible), but if your rules are well defined you should automate their testing.

I realise not everyone has the skills to code, but there's plenty of software out there (okay a lot of it is rubbish and prone to overfitting, but "a bad workman always blames his tools" - it's how you use them) and if you're going to bet real money learning how to use some backtesting software doesn't strike me as a bad investment.

GAT

Handle123 · May 4, 2016

Sometimes you just want to see how something did without spending couple hours programming on a dumb idea, when I have manually back tested a month, I try hard to make the thing lose and if it looks credible then spend the time back testing. I get many ideas from people who post here and not too often what they post works for me, that not to say it doesn't work for them. So easy in trading to try to show others how you trade a pattern and they get it 180 degrees.

But I would never trade on what I manually back tested over a month, but many dim wits would and then hear the moans and groans, Lions, Tigers and Bears ate their account, Oh My.

Maverickz · May 4, 2016

Martinsos said:
What I would like to know is what are the biggest problems and obstacles that you face regarding manual backtesting right now?

The biggest problem you are probably going to run into with your tool is data. Based on your question it sounds like you plan on releasing this tool to the public. In that case if you don't have a free data source then you will need to allow your tool to connect to multiple popular data sources.

Martinsos · May 9, 2016

@Maverickz good point - I actually do have a non-free data source available, so let's assume that is not a problem. I am much more interested in hearing about your problems regarding backtesting!