Sounds simple, so is good. Whatever works for you.
When you look at charts you become biased, assuming that chart will repeat itself. It will never repeat, so one need to generalize more.
Developing Python/R/Matlab strategies vs. trading platforms
- Thread starter 2weels
- Start date
Sounds simple, so is good. Whatever works for you.
When you look at charts you become biased, assuming that chart will repeat itself. It will never repeat, so one need to generalize more.
Hello Simples,
What do you mean by "When you look at charts you become biased, assuming that chart will repeat itself. It will never repeat, so one need to generalize more."
Simplify things a bit here. No need to make complicated.
When you backtest, you test the past, which won't repeat exactly in the future. So forward testing is important, and also not fitting curves/pnl to the past. Not sure how to explain it simpler than that
L
lovethetrade
Coding it so there is no GUI when the program is running in either of the following languages:
C++, C#, Python, Java, R, Matlab
GUI = increased slippage and due to more resources being used/managed, increases chances of system freezing.
Proprietary software usually calls home also so there is security issues there as well.
Last edited by a moderator:
Simples,
So why do you back test then? Does back testing not give you a sense of edge or higher probability of winning? Is it not about placing the odds of making money in the future in your favor?
Last edited:
Yes, backtesting is a fundamental part of mechanical trading (not to be confused with discretionary trading). Without sound backtest results, where's the basis to make any trades using a mechanical system? Computer-systems today are dumb, so need to be calibrated to catch what you want to catch - the winnings, while minimizing losers. It's simply not enough to believe/hope RSI(2) < 20 is good, although a good trader could filter out the bad signals in realtime, without really knowing how she does it.
Backtesting pose new problems and require more efforts though. Whatever happened in the past, won't happen again, but it may look similar, so you may choose to program in those similarities. If the markets change though, the very parameters that brought profits may even become detrimental, so you better choose characteristics that do tend to stay.
Charts may lie, because they make you believe they will repeat themselves exactly. So it is with good backtests. There are many methods to make backtesting more robust and resilient to various biases and errors. Too much to discuss here, but for example data errors may make your backtesting results great, but won't happen live the same way. So fixing data becomes the first issue to resolve. Also the logic need to be resilient and general enough to make allowances for differences in price action, etc.
If what you do really works for you though, don't worry about it until you need to perhaps. However, in my experience there's lots of pitfalls in both trading and backtesting, and overcoming these consistently and mechanically is a very very long road indeed. Maybe faster doing it discretionary, but I choose not to spend my time trading manually.
Have you tried forward testing or testing on huge diversity of data (other instruments, time periods, future data) yet?
Last edited:
L
lovethetrade
It does give you confirmation of an edge and positive expectancy if your logic is strong. That's a skill in itself, knowing if your logic is robust. If you don't know the difference between strong and weak logic and youre strategy is based on curve fitting, then yes, you will need to forward test the he'll out of it and hope it will continue to work.
Hello Simples,
Thanks for the input and detail response.
I have notice the backtest results will lie to me when testing strategies with small profit targets or stop loss (scalping strategies). So, to avoid this, I simply build intraday strategies where the profit targets and stop loss are big. In other words, the biggest problem I see is lack of intra-bar granularity when it comes to back testing the standard way. I solved this by coding orders on smaller time frame and using NT High Fill Resolution order, which gives more intra-bar granularity fills.
Yes, currently forward testing on several different instruments with same time frame as backtesting.
Right now, for me, what is important is not curve fitting, or forward testing or strategy building/logic. That's easy, in my opinion.
What is challenging is trusting my platform will perform quality back test or optimize each time I click Optimize or Back test? After all garbage in, is equal to garbage out.
So i am thinking of a plan to how I can check the back test. So I am comparing real time trades to back test trades for x amount trades and will monitor the accuracy. Then every two weeks, I will back test those x amount of trades, just to make sure the platform backtesting simulation (and historical data) is not shifting or reliable. I guess sort of like preventative maintenance checking. Make sense?
It makes tremendous sense. The more validation you put into it, the more confidence you can have in your system. I lack experience with NT backtesting/optimization, as I just created my own backtester as an indicator, and later as as-hoc code together with my main indicator code. By using OHLC and maybe even Volume-data you can make your own backtesting code and compare with NT backtester perhaps? Using two real forward-testing weeks to validate backtesting and logic seems too long time, as you'd like as rapid feedback-loop as possible for fast learning and improvement cycles.
Last edited:
Simples,
It's an understanding conversation. Ideally, I would have to trust the backtester simulation is performing correctly.
One idea is to use NT market replay strategy in 5x fast forward. If I find a strategy that pass the criteria checklist to move forward to forward testing, I run it for 30 days of market replay and compare to back tester to make sure back tester did its job accurate.