Method of checking if a trading system "is broken"

[albertly]

I've found very interesting method of checking if a trading system "is broken".
From the book of John Wolberg "Expert Trading System - Modeling Financial Markets with Kernel Regression" page 181–183. One of the most serious book on system design.
(See attachment with links to relevant pages)

I tried to implement his method, but faced with some uncertainties (Bearing in mind that my knowledge of statistics is not good as I would wish)

First of all the author gives us
daily ratio of σ/μ = 10 (Sigma/Mu = 10)

My first question is where he gets it and what is statistical meaning of dividing standard deviation by mean?

No matter, I can calculate stdev and mean from my equity curve.
1. I get difference of previous day equity and current day equity.
2. Calculate StDev and Average of it.


My second question if is it correct ?

The author still needs to find σ and μ, but I think it is not the same σ and μ as above because he calculates
it trade days per year basis i.e. μ' = annual profit ^ (-250) - 1 . (daily μ')

So I calculate μ' . After that I calculate daily σ' as 10 * μ'



From his formula we can conclude that the less exponent in Eq. (2.13) the better result.
Very logical - the less standard deviation and bigger average change in equity the better.

However if I earn money with the system my equity get bigger and and consequently profit and drawdown is bigger.
It increases StDev.
And look at the formula 2μ/σ^2 -- Stdev in power of 2.
It means the more money we make, the chance of drawdown grows aprox. in power of 2 and not linear.

 

[braincell]

That's true in simplistic mathematical models but in practice you will find quite a few ways to counter that and make the ^2 go away. Also very logical, since like in most of math - the basic principles that describe systems in physics have exceptions at extreme values. Such books are interesting reading but they don't contribute much to the practical development of systems unless you need to shape and form your own ideas and put them into similar formulas for easier understanding, which depends on the way you like approaching problems aka how your brain works.

 

[albertly]

The answer of John Wolberg

Shalom Albert, I'll try to answer your questions.

First of all the author gives us daily ratio of σ/μ = 10 (Sigma/Mu = 10)
My first question is where he gets it and what is statistical meaning of dividing standard deviation by mean?

We are discussing a theoretical trading system that has shown an annual rate of return of 30%. We compute μ as the average daily rate of return = 0.00105. We can also, look at all the daily returns and compute a value of σ. Since we are not talking about a real system there are no daily returns so I just chose a value in which σ is 10 times μ.


You next state:
No matter, I can calculate stdev and mean from my equity curve.
1. I get difference of previous day equity and current day equity.
2. Calculate StDev and Average of it.
My second question if is it correct ?


Its almost correct. The daily fractional equity changes are [X(i) - X(i-1)]/X[i-1] where is X's are the values of the indices that you are following (for example, closing values of the S & P). If you have 100 values of X you will have 99 values of daily fractional equity changes. From these 99 values you can compute μ and σ.


The following statement is wrong: So I calculate μ' . After that I calculate daily σ' as 10 * μ'.


The values of μ and σ are computed from the equity changes. The number 10 was just a number that I used to explain the process.


Your next statement is: However if I earn money with the system my equity get bigger and and consequently profit and drawdown is bigger.


Since we work with fractional changes in equity, if equity increases, the fractional changes dont necessarily increase.


You also state: It increases StDev. And look at the formula 2μ/σ^2 -- Stdev in power of 2. It means the more money we make, the chance of drawdown grows aprox. in power of 2 and not linear.


Our basic assumption is that μ and σ remain constant and that is what causes the ups and downs of the market. If μ is positive than we can expect a rise over the long term but due to σ some days will be down and thus we will expect drawdowns.


This is a very simple model of market behavior. You cant use it to predict market movement but you can make some interesting predictions regarding future drawdowns.


I hope this is helpful... All the best...

The key point is [X(i) - X(i-1)]/X[i-1],
and not what I did [X(i) - X(i-1)] .

Now everything is pretty clear.

 

[Term]

Thanks Albert - I appreciate you following up with that response.

In an attempt to hijack this thread a little bit, does anyone have any tests to check to see if a system is broken that they'd like to share?

I've been reading through a few books on the subject and the ones that I've found so far are:

- Runs Test
- T-Test
- Edge Test (from Acrary)

There are many more than that but does any have one that they've found particular useful?

 

[albertly]

Quote from Term:


There are many more than that but does any have one that they've found particular useful?

Monte Carlo, of course.

If it is good for nuke, it is also good for trade