Making money is an iterative refinement process primarily.
One of the facets may be back testing. I do not know much about back testing because I have studied the practioners and their results.
With regard to making money, the potential changes. I rank the following as key factors and I suggest that they apply to all backtesting because they are market issue elements:
1. the seasons. Summer is the best example.
2. daily volume. Again summer is the best example. Average volume shifts during the first few weeks after rool over is another example of an index getting into the groove for applying stuff to it.
3. daily price range. You can do yourself a nice favor by keeping track of the relative rank of the last year of daily H?L ranges. You can only make so much money in a day. The day may not offer much.
4. the trading fractal. I view each fractal as a separate entity (Think of applying a given FA analysis to each fractal). If you get to seeing them as separate as individual stocks or as independent mutual funds, then you see that bact testing is only valid on a given FA scenario (Go back to 1 and do each item with back testing)
5. making money. You can see by his comments that abogdan is not in the ball park for making money as yet. His stuff is not sensitive to change as yet. See all of the above which precipitate change in the money velocity that can be potentially optimized.
General comment. Traders who are successful go through stages of efficiency of extracting money from the market. If and when that process is ensuing and it is subjected to people backtesting it, they usual crap out around the beginner level and at that point in the operation there is no consistency of results among those who do the back testing.
Back testing generally falls into the category of "if you can define it I can back test it". This fallacy is not addressed too often. A second general understanding and in the neighbor hood of being an accepted myth is: paper trading is always more successful than actual trading". Paper trading is a cousin of back testing. As it is, then, you can fool around with what the relationship of back testing is to real trading with that bias factored in.
The most important thing vis a vis back testing is that it is very incomplete. It seems to focus on IQ related things. In trading communication turns out to be important in the aspect that the market is communicating to the trader. This involves the trader's senses. All sensory input has two components verbal and non verbal. The majority of the input is nonverbal (as usual). For most traders, sight predominates as the sensory factor. People to people, sight also dominates over sound when they are in each others presence.
It is possible to view backtesting as a simulation of the minority of information communicated to the trader. If the majority of the information is left out, then what is being done is not too substantial.
What is being left out? What is missing is the emotional communication that the trader induces as the visual data is received. After that, backtesting takes another major hit. The trader actually processes stuff through memory stuff that is not equivalent to the backtesting software or it's nuances if any.
To back test one has to set up a software emotional process that is in parallel to the data stuff. The data is simple the emotional is complex. Getting them to synchronize is even more difficult.
Back testing is very rudimentary compared to paper trading. Paper trading is most valuable for it's ability to detect unwanted emotions that are not under the control of the paper trader. The continuation of paper trading is of little value unless there is extensive debriefing vis a vis emotions. The first question is always: Where the hell did that come from? It wrecked everything I was doing.
As people begin to get into trading they do stuff like copy cat and read books. It tends to parallelback testing efforts; that is, it is focused upon IQ stuff. The IQ stuff is the minority of what is going on.
An example of a guy who has the picture of all of this is Ed Seykota. What sets him apart? Well he focuses on "fixing the non IQ part of trading. It happens to be a big deal to focus upon.
Set up back testing to deal with the majority of the input a tradersis getting with a given set of "rules" and you have a starting place.
Mentoring has this feature when the guy doing it is aware of how people are "processing".