Been trading profitably for the last few months, after putting in more than 5 years. The main thing I have noticed is that its all about finding the edge, and the edge is found by applying sound reasoning and logic, in the same way as is done to solve a very interesting but a very tough puzzle.
Testing has no part to play in it. Once you find the edge, or find your path through the puzzle, its almost always a proof in itself.
Its only when you don't have a clear edge, that you start testing with loads and loads of data, and I guess not much comes out of it..!!
You could get lucky and think you are a trading genius when you have just been lucky in your trades. I found what I thought was a trading system that was very good and had an edge. On paper (with a limited data of about 205 trades) seemed that would be close to a sure thing. Now, comes the actual test with real monies on the line and it fails terribly. Did not count on the huge slippage involved since, I trade options as a buyer. I took the lumps on the losing trades as I had proper risk management. Anybody can get lucky trading the stockmarket. You probably, need atleast, 500 actual real life trades to validate you even have something with an edge.