How do you prevent overfitting when you run your optimizations?
Have you looked at marketcetera before? It's also Java
I'm building an open-source robot trader framework.
- Thread starter jbax
- Start date
How do you prevent overfitting when you run your optimizations?
Have you looked at marketcetera before? It's also Java
I run simulations against multiple instruments (40+) at the same time using each set of parameters, collect statistics about the trades made, and from these I select the parameters that worked well enough for the majority.
Rinse & repeat for a while, run a few market tests where the simulation ticks an internal clock and loads all candles for all 40+ instruments used earlier, and emulates what would happen if trading live (i.e. buy a few things until no more funds available, sell to make funds, etc), then forward test, then trade live with small amounts.
Never heard of marketcetera.
Author of *backtrader*
1. backtesting requires looping over bars. If not, you are talking about something else which is research using an approximation) Even looping over bars doesn't guarantee maximum accuracy, which depends on the resolution of the bars, volume filling approach, slippage et al.
2. The term "Vectorized" has become a bingo-bullshit word which people don't know when to apply and to what it really applies.
3. The default backtesting mode of *backtrader* uses a vectorized approach (even if not using the highly praised vectorized pandas) to pre-calculate all indicators.
3.1 But the trading logic is done on a step-by-step basis (the trading logic can actually be changed on each step, which is something which cannot be vectorized)
4. *backtrader* can change the vectorized approach to step-by-step calculation of the indicators to support purely recursive declarative indicators (the indicator can signal the platform that it needs a full step-by-step execution or it can be manually changed).
5. *backtrader* will directly switch to a step-by-step basis when live trading or even in backtesting when a data feed reports itself as not pre-loadable (or *live*)
With regards to speed, see this which backtests 2M bars and compares the standard Python interpreter and pypi
https://www.backtrader.com/blog/201...on-backtesting-performance-and-out-of-memory/
Why another backtester?
What is the main differences between this and another python framework like quantopian/zipline
*backtrader* is not only a *backtester* because it does also live trading and was not started yesterday. You could have asked the question in 2015. Today, being used by companies and professionals, in academia, with job offers asking for it, over 3000 users registered in the *backtrader* community. Well ... you may ask them.
As for the differences
1. Pure Python. If you don't mind compiling ... but this is not intended to be a deciding factor.
2. The rest: easier, and with *MANY* more features. One example (from many)
- You can trade spot-vs-future scenarios, which is actually used by professionals in Energy Markets (real professionals)
Some LinkedIn profiles listing *backtrader* in the CVs:
- See here: https://www.backtrader.com/home/references/linkedin/
Anything particular you may want to discuss? Because the website and documentation are quite comprehensive and there is even a "hello algotrading" section with 3 different examples on how to quickly implement (with plotting) a moving average crossover.
- See here: https://www.backtrader.com/home/helloalgotrading/
As for the differences
1. Pure Python. If you don't mind compiling ... but this is not intended to be a deciding factor.
2. The rest: easier, and with *MANY* more features. One example (from many)
- You can trade spot-vs-future scenarios, which is actually used by professionals in Energy Markets (real professionals)
Some LinkedIn profiles listing *backtrader* in the CVs:
- See here: https://www.backtrader.com/home/references/linkedin/
Anything particular you may want to discuss? Because the website and documentation are quite comprehensive and there is even a "hello algotrading" section with 3 different examples on how to quickly implement (with plotting) a moving average crossover.
- See here: https://www.backtrader.com/home/helloalgotrading/
Not true in every case, it entirely depends on what kind of approach you take. I'm writing my own as of this moment and I can absolutely have a single position (for now) indicator based entry/exit strategy without any looping.
Same goes for calculating basic portfolio backtest equity/stats. For complex position sizing and such I'm still relying on looping but that is not necessary in this example.
I see Pandas and Numpy are now supported in PyPy, exciting times.
Yes and my answer already indicated that the logic in the strategy can be changed at any time, which is the reason to not vectorize. If you write a single-case backtester, fine, but that's not the general case as you yourself point out, as soon as the smallest of things deviates from the standard single-case.
When you have written something that supports all cases without using a loop, let me know.
The point stands: a fully *vectorized* approach is good for (basic) research, not for backtesting, which is what comes after the research.
You can add that the so called *vectorization* which most people refer as to using `pandas` and or `numpy` doesn't support `append`. Because copying the entire dataframe to add an extra row is not `appending`. You therefore need a fixed size from the start. Because the structures provided by both libraries are meant for research on known data and not meant, for example, for step-by-step (or event based) approaches with unknown data lengths or even ring buffers.
Both `numpy` and `pandas` are extremely useful in any case when performing research.
To be more specific, does backtrader handle multiple instruments backtesting well?
Does it allow backtesting on order book depth of market data?
> To be more specific, does backtrader handle multiple instruments backtesting well?
How many do you want? There is no limit (except for your RAM)
Surprisingly ... you can even have multiple instruments with different trading days, and they will be aligned.
Do you want to also mix timeframes? How many? Any.
Do you want to calculate indicators on one timeframe and apply it another? Yes you can (like a PivotPoint indicator)
Do you want to calculate an indicator based on multiple instruments from different timeframes? You also can.
> Does it allow backtesting on order book depth of market data?
No. Not a feature. Never even promised.
It is not a bid/ask and/or order book depth backtester. Please answer these simple questions for this single use case: daily bars
- Which order book do you take? The one at the end of the day? What if the order being matched is a limit order?
Even if you consider a smaller timeframe ... 60 minutes ... the same questions apply (replace "end of day" with "end of bar")
How many do you want? There is no limit (except for your RAM)
Surprisingly ... you can even have multiple instruments with different trading days, and they will be aligned.
Do you want to also mix timeframes? How many? Any.
Do you want to calculate indicators on one timeframe and apply it another? Yes you can (like a PivotPoint indicator)
Do you want to calculate an indicator based on multiple instruments from different timeframes? You also can.
> Does it allow backtesting on order book depth of market data?
No. Not a feature. Never even promised.
It is not a bid/ask and/or order book depth backtester. Please answer these simple questions for this single use case: daily bars
- Which order book do you take? The one at the end of the day? What if the order being matched is a limit order?
Even if you consider a smaller timeframe ... 60 minutes ... the same questions apply (replace "end of day" with "end of bar")