Evaluation of Backtested System Results

T

trend456

I've been developing systems for a while and recognize different ways to evaluate systems. For example: Monte Carlo sims, Correlation, and etc... I still haven't applied some of these testing to figure out whether a system is reliable in real life situations.

I wanted to start out a thread that talks about ideas on how to evaluate systems that work in real life. I'm not interested in specific systems. I think evaluation of a profitable system is more important.

It would be great if some of you can share some thoughts on how to evaluate systems.

Good Trade!

Trend
 
Quote from spreadem:

Have you used RiskOptimizer?
More...

No, Have you used it?

I use a different sofware that's an add-on for XL. I think it's called Insight.

Good Trade.

Trend
 
Quote from vhehn:

with real money and small size.
More...

Agree...

When I backtest systems, I test systems using 1 contract without any MM. I apply MM's later.


Good trade.

Trend
 
One thing I recognize which is a common sense to the experienced is most developers forget to evaluate there own psychology.

Most traders I've met who developed systems that don't work in real life tend to optimize there own life. These fellows naturally optimize or develop systems based on there curve fit goal and come with a system which is curve fit. It's interesting how psychology plays an important role in developing systems.

Why? Maybe satisfaction...?!

I wanted to say this so that I don't get curve fit answer. Maybe I should move this thread to the journal section.

What do you guys think?

Good Trade!

Trend:)
 
I take all the optimization results into Excel and just look at it all, sort it different ways until you get a feel for the "good spot". Three D graphing is great too, it will keep you out of the area of optimization where the system is hugely successful but only based on a few winners, you don't want that, not likely to repeat.

Max
 
Quote from maxpi:

I take all the optimization results into Excel and just look at it all, sort it different ways until you get a feel for the "good spot". Three D graphing is great too, it will keep you out of the area of optimization where the system is hugely successful but only based on a few winners, you don't want that, not likely to repeat.

Max
More...

Max,

Interesting....

I've read that Ed Seykota uses 3D graphing to find robust systems.

I think optimizations are easier if applied on systems that uses EOD data. When the system is a daytrading system the market tends to change more often and choosing a robust system is harder. I don't optimize my system so I would be interested hearing your process of finding a robust system using optimization.

Good Trade!

Trend:)
 
Quote from israelross:

Please explain how to use excell in backtesting a system.

Thanx

Isreal
More...

I don't use XL to backtest but use it to evaluate a system in different ways. I use a backtesting software like TS or WLD to backtest a system.

Good Trade.

Trend
 
You must log in or register to reply here.
Back
Top