>I'm not entirely disagreeing with you here, Harry.
So do I with you Biomech
>Ideally one would understand exactly why their system is >successful.
Yes ideally. Ideal can never be reached all the time, but it doesn't mean that you must not try to reach it, not at any price but by conducting a process of progress. Some people just don't even try. Things work on short term they smile, when it broke then they cry trying to understand. That's how catatrophies happened "unpredictably" but in fact not so because they were so unpredictable but because of the dictature of "positive thinking" : people don't want to just consider the risk. "Positive thinking" is a good tool to energise oneself but it must not become a systematic way of looking at things so that one even occult risk and that's how risk burst into reality. As famous Phantom of the pits puts it: "Most traders plan only for the probability side and that, to them, is always what they consider the winning side. This is the biggest mistake you can make in trading. Instead, you must plan for the losing side." (see his rule 1
http://www.futuresmag.com/futuresclassroom/phantom/phantom.html )
I would generalise and say that "Most traders only consider the winner side of their trading systems. Instead they should consider the losing side."
>I think a system can be robust though without understanding >exactly why and how it works.
I didn't mean "how it works" I mean at least trying to understand why it works. It is not the same level of requirement. What I mean is that it is best to also know HOW since when you know how you know why. If you prefer you can replace why by "IF it has any chance to work in the FUTURE". To answer the question you can look just at the outputs.
>I don't have the scientific background that you guys do, but I do have a strong Comp Sci background.
You don't have to have a scientific background but logic (you must have since you are in computing) and will to be a harsh critic of yourself and the latter depends on your personality. If your personality suffer from being honest with yourself your logic can't save you.
>It is impossible to understand every detail of a million line program, and it is impossible for that program to work exactly as you intended, but with some extremely thorough testing, you can be 99.9% sure that your program will perform as designed.
I am also in computer business. In fact I am either a software architect or a project manager depending on (big) client's mission (only them can afford to have software architects small companies just don't have software architect or even project manager
) which have also to specify plan tests for software teams. And being sure 99.9% is not so evident even with software
the number of time above all with inexperienced programmers which you can get a headache having to check any regression they made through code "optimisation" and not respecting the architecture.
And the big difference between software testing and trading system testing is that in the first case you can really know all the deterministic paths it's just a question of time and ressources, whereas in the second case you have to face unknown law. That IS the problem. If you don't know the law you have to face something that is more than just checking a software program.
>Business conditions change, just as market conditions change. The program must constantly be adapted just as a trading system must be tweaked to handle current market conditions. I'll bet these quotes you have of people's systems falling apart were not constantly monitored and tweaked with and re-backtested etc. to keep their market perspective fresh.
Not the same thing. Read Futures Truth methodology for example. The first requirements they ask is not constantly optimising the system. I explained why: robustness. If the system constantly needs optimisation it can mean just some sort of curve fitting.
>As to your second and third points, I would say there is no way that ANYONE can know that there system will continue. Show me a system that you think will be valid for the next 100 years.
Inferential statistical method on the principle has been invented for that do you suggest to throw the foundation of scientific method
. The problem is to be in condition to use these inferential method. For example if one use linear statistical method, one must check for heteroscedascicity of variance ("constance" of variance but the term constance is not appropriate since it varies from sample to sample so the statiscal term heteroscedascicity - HSDC for short
), not for market itself of course since everybody knows variance is not HSDC, but on your own datas from your trading. As an experienced engineer in SPC (I began both in software and Quality ingineering) I know that many datas involving Human beings are most of the time not in what is called state control. So you have first to put them in state control by checking the HSDC of variance for example (I won't detail here it will be too long perharps another day or at occasion of a new answer). If you have read my post on Benford's law a mathematician has even created a new statistical law for that (although it is in fact a rediscovery from an astronomer).
> All you can do is do as thorough testing as your time allows for as large a number of variables as you can muster, regardless of whether or not you think you understand exactly why your pattern works (which is an entirely different discussion). Then you must constantly monitor it to make sure it is still relevant to the current market.
When you have a large number of variables to take into account you add degree of liberties to your system and allow it to adjust to any market. That's one critic from people towards elliott wave theory and it amuses me because they don't see they use exactly the same trick to pretend that their system work. If you have a deterministic model then you can cope with many variables. An example is the genetic mapping process. It has thousands of variables so there is a huge data bank necessary to check the theorical pattern since with so many variables the combinatorial law necessitates to check millions of cases. It is the same with my deterministic model although it doesn't reach the huge number of genetic coding. But if you are not in a deterministic framework then you must not cope with many variables but on the contrary eliminate as many variables as possible (so the famous "let's be simple" which means rather the number of variables or more exactly degree of liberty than simplicity of procedure per se although it is correlated ... sometimes. Think of E=MC2 very few variables but from this formula to a nuclear power plant well it is not so simple
). That's why I said many things are true or false depending on the implicit context or framework. One must be careful with the details. Hell is in details. Science or methodology is about being rigorous with details. If not so people just use them as false pretext to justify a belief. I am not against belief, I am against false justification. I prefer someone saying: "I believe I would like a proof but I don't see how" than a false proof. It is not evident to find a procedure to prove something like finding the demonstration of theorem when you were at school , so it is not a shame to say I can't prove it.
>
