Imagine a purely objective system that has a win percentage of 90%. For every 10 trades, this system racks up 9 winners. But each of the losers is 5 times as large as each of the winners. In other words, the system has a 1:5 win/loss ratio. The system was developed on a backtest that only ran back 6 months, but because of the high frequency of trades, and the fact that it was properly tested on out of sample data there are 3 traders who have been given the system and the assignment to maximize their trading profits, using the trading system as they see fit. They must choose what their next course of action will be. The question becomes, which trader took the wisest course of action? The three traders are Archie, Bob, and Chris.
Archie looks at the test results and decides that he needs more information. So he acquires more data for testing, and proceeds to test the system an additional two years back. Archie finds that while the system performs well over the last 6 months, it does less well over the previous two years of data. In fact, the system is a net loser over the first two years, and a net loser overall when he includes this new data.
Archie sizes up the situation and decides to optimize the system so that it performs better over the entire period. So he does more in sample and out of sample testing to come up with a parameter set that is more *robust* over the entire test period. He finally settles on a parameter set and finds that it produces net profits over the entire period, but only performs at breakeven level for the last 6 months of data, and in fact the system happens to be in a drawdown for the last two months of that time. But Archie reasons that the system works well on a longer timeframe so that it should pay off in the longer term, so Archie decides to start trading his system.
Bob examines the test results and decides that he needs to now do forward testing to ensure that his system actually performs as well as it backtested. So Bob begins forward testing. He reasons that because the backtest period was only 6 months, he would make up for this small amount of data by doing a longer forward test. Bob chooses 6 months because it is equal to the length of the backtest, and feels that this should give him enough time to fully evaluate how well the system performs in all types of markets.
Chris examines the test results and notices that the results are extremely strong, particularly at the end of the test. Chris decides to forward test the system. But he chooses only one week because he reasons that this system can't stay this hot for long, and his job is to outperform Archie and Bob to keep his job. If the system forward tests favorably after one week compared to the results of the backtest, Chris decides he will jump right in and start trading the system.
Which trader maximized his trading profits?
Archie looks at the test results and decides that he needs more information. So he acquires more data for testing, and proceeds to test the system an additional two years back. Archie finds that while the system performs well over the last 6 months, it does less well over the previous two years of data. In fact, the system is a net loser over the first two years, and a net loser overall when he includes this new data.
Archie sizes up the situation and decides to optimize the system so that it performs better over the entire period. So he does more in sample and out of sample testing to come up with a parameter set that is more *robust* over the entire test period. He finally settles on a parameter set and finds that it produces net profits over the entire period, but only performs at breakeven level for the last 6 months of data, and in fact the system happens to be in a drawdown for the last two months of that time. But Archie reasons that the system works well on a longer timeframe so that it should pay off in the longer term, so Archie decides to start trading his system.
Bob examines the test results and decides that he needs to now do forward testing to ensure that his system actually performs as well as it backtested. So Bob begins forward testing. He reasons that because the backtest period was only 6 months, he would make up for this small amount of data by doing a longer forward test. Bob chooses 6 months because it is equal to the length of the backtest, and feels that this should give him enough time to fully evaluate how well the system performs in all types of markets.
Chris examines the test results and notices that the results are extremely strong, particularly at the end of the test. Chris decides to forward test the system. But he chooses only one week because he reasons that this system can't stay this hot for long, and his job is to outperform Archie and Bob to keep his job. If the system forward tests favorably after one week compared to the results of the backtest, Chris decides he will jump right in and start trading the system.
Which trader maximized his trading profits?