Developing "Spartan"

[d08]

Not sure what you mean about native vs third party for Qt. Qt isn't native! Though I'm with you on Pandas for backtesting.

I meant native to Qt in that using Qt's module for something instead of something else. For example I used Qt charts but setting the x-axis ticks with simple python, it was considerably slower than using Qt's "native" dynamic setting.

Can you give an example how you'd run a Pandas based backtest, like a very basic indicator based entry/exit? (Apologies if you already did post it earlier).
I find it a bit of a challenge because I'm doing by best to avoid looping, I'm really too used to loops in backtesting, hard to retrain my thought process.
I'd also keep as much as possible in memory so certain adjustments to strategy would be almost instant. This of course doesn't work for very large amount of data.

 

[nooby_mcnoob]

an you give an example how you'd run a Pandas based backtest, like a very basic indicator based entry/exit? (Apologies if you already did post it earlier).

There are different kinds of backtests IMO, exploratory and detailed. I really only do exploratory backtests these days. The process is:

1. Develop a hypothesis
2. Check validity of hypothesis via exploratory backtests
   
3. Trade random periods over the last 20 years
4. Paper trade live
5. Live trade

I guess you can really count (3) as detailed backtests, but even those are not really detailed. I just run the algorithm over those days and manually inspect the trades.

The point is to get me to step (4) as quickly as possible. I know, the systematic traders are shitting their pants reading this. I don't care. It works for me better than the scientific approach. The focus then shifts to risk management. I can tell you that there is a 2% probability I will lose the full risked amount on a trade. or a 50% chance I will lose 10% of the risked amount. That's worth more to me than knowing how much money I will make. Edit: I also use these probabilities along with volatility to size my bets. The biggest game changer. Leveling up like a mofo with this one thing.

All that being said, I did post the method I use here: https://www.elitetrader.com/et/threads/vectorized-backtesting-with-pandas.332094/

Some of the 1337 h4x0rz quants think it sucks and is dumb but whatever.

 

[Real Money]

the systematic traders are shitting their pants

Isn't this systemic trading anyway? I don't use systematic or scripted trading techniques, so to me, this seems like the same thing as those guys that use the back test and benchmark method.

I read a bunch of quant forums and they say that the whole reason that they need quants to backtest, benchmark, grade, and characterize strategies is because it is illegal to market a fund using anything else.

Backtesting is a joke as far as i'm concerned and I've studied mathematical statistics, probability, and theoretical mathematics including analysis, topology, stochastic process, et cetera.

They make completely baseless claims about the quantified performance of strategies, using point estimates and other complete nonsense.

 

[nooby_mcnoob]

Isn't this systemic trading anyway? I don't use systematic or scripted trading techniques, so to me, this seems like the same thing as those guys that use the back test and benchmark method.

I read a bunch of quant forums and they say that the whole reason that they need quants to backtest, benchmark, grade, and characterize strategies is because it is illegal to market a fund using anything else.

Backtesting is a joke as far as i'm concerned and I've studied mathematical statistics, probability, and theoretical mathematics including analysis, topology, stochastic process, et cetera.

They make completely baseless claims about the quantified performance of strategies, using point estimates and other complete nonsense.

I can only say that I haven't found extremely detailed backtests as useful others I have read. This is very similar to the idea that many developers find that writing unit tests is not as useful as is advertised.

If I am to draw that analogy, I only write tests for complex areas of code. For example, at the heart of everything I do is "automatic" state management and persistence.

Whenever there is a bug, or something weird going on here, I write a real test.

I think backtesting is similar. If I see some results that are different from what is theorized and statistically sensible, I would do a backtest. But even that backtest is not that dissimilar from step (3) above: I find candidate days that reflect the circumstances I am experiencing and go through them. Until now, I have only realized that those days are the 2% or 50% or whatever where I'm expecting things to go badly, i.e., the probability is relevant.

Edit: personally, I would like to hear @destriero's opinions on backtesting. He is mostly discretionary if I understand correctly, so that perspective would be informative IMO.

 

[Real Money]

Well, now that I think about it, if you spread enough correlated instruments then you probably could use standard statistics theory just because the relationships are so much more well behaved, in terms of parameters.

The big funds are using that stuff to do stat arb and so you would be digging through data that is being fed into their black box algo's. Basically, it would be like reverse engineering their strategies, or even deriving the implicit parameter estimates from their trading ops.

I am seeing a lot of big money players in the really obvious spreads like ultra bonds vs bonds spread against index spreads (hedge systemic vol in both rates complex and equities).

Seems like the guys building truly quant systems with real $$ behind it are doing obvious stuff in terms of manipulating vol distribution. Basically, they are using membership and execution advantage to trade baskets that give them advantageous risk metrics.

 

[nooby_mcnoob]

Cool thing I developed using pandas dataframe, shows current price (rows of nans) in relation to probabilities as to where price could go (red = price will be below this level in N time periods with X probability, green = the price will be be above this level in N periods with Y probability).

This is the fun stuff to me. I'm trying to automate this but the first step is to make it systematic for me to use and this was the second step. The first step was doing this incorrectly manually many, many, many, many times. Sizing wrong, picking the wrong level, etc. Trades were still profitable, but very stressful to execute. I sketched out what would look correct to my eyes during trading and then filled in the data.

def highlightCurrentPrice(x):
  if x.name == 'current':
  return ['background-color: lightgreen']*len(x)
  else:
  return ['']*len(x)
df2.sort_values(by=df2.columns[0],ascending=False).style\
  .apply(highlightCurrentPrice,axis=1)\
  .bar(subset=[c for c in df2.columns if c[0] == 'probability' and c[2] == 'below'],color='rgba(255,0,0,0.25)')\
  .bar(subset=[c for c in df2.columns if c[0] == 'probability' and c[2] == 'above'],color='rgba(0,255,0,0.25)')

 

[nooby_mcnoob]

Well, now that I think about it, if you spread enough correlated instruments then you probably could use standard statistics theory just because the relationships are so much more well behaved, in terms of parameters.

The big funds are using that stuff to do stat arb and so you would be digging through data that is being fed into their black box algo's. Basically, it would be like reverse engineering their strategies, or even deriving the implicit parameter estimates from their trading ops.

I am seeing a lot of big money players in the really obvious spreads like ultra bonds vs bonds spread against index spreads (hedge systemic vol in both rates complex and equities).

Seems like the guys building truly quant systems with real $$ behind it are doing obvious stuff in terms of manipulating vol distribution. Basically, they are using membership and execution advantage to trade baskets that give them advantageous risk metrics.

For what it's worth, I'm nowhere near that sophisticated even in my thinking, let alone trading

 

[d08]

I can only say that I haven't found extremely detailed backtests as useful others I have read. This is very similar to the idea that many developers find that writing unit tests is not as useful as is advertised.

If I am to draw that analogy, I only write tests for complex areas of code. For example, at the heart of everything I do is "automatic" state management and persistence.

Whenever there is a bug, or something weird going on here, I write a real test.

I think backtesting is similar. If I see some results that are different from what is theorized and statistically sensible, I would do a backtest. But even that backtest is not that dissimilar from step (3) above: I find candidate days that reflect the circumstances I am experiencing and go through them. Until now, I have only realized that those days are the 2% or 50% or whatever where I'm expecting things to go badly, i.e., the probability is relevant.

Edit: personally, I would like to hear @destriero's opinions on backtesting. He is mostly discretionary if I understand correctly, so that perspective would be informative IMO.

Thanks. My current prototype approach on daily bars, 3 years, 5000 symbols takes roughly 20 seconds and I'm fine with that efficiency. Obviously it will take a bit longer with added complexity.
Lately I've been stuck with minor problems that are GUI related and totally irrelevant from the trading perspective. Coding is definitely extremely time consuming and offering little productivity wise.

 

[nooby_mcnoob]

Thanks. My current prototype approach on daily bars, 3 years, 5000 symbols takes roughly 20 seconds and I'm fine with that efficiency. Obviously it will take a bit longer with added complexity.
Lately I've been stuck with minor problems that are GUI related and totally irrelevant from the trading perspective. Coding is definitely extremely time consuming and offering little productivity wise.

Most GUI problems in my experience are related to putting too much "business" logic in the GUI. Also, Qt charts are a nightmare, I have no idea how you persisted. WTF is with that insane API.

 

[d08]

Most GUI problems in my experience are related to putting too much "business" logic in the GUI. Also, Qt charts are a nightmare, I have no idea how you persisted. WTF is with that insane API.

It's definitely a bag of dicks. Most of the things I've encountered in Qt are not intuitive at all and you have a mixture of deprecated and non-deprecated calls which are not obvious. Qt people aren't really from a financial background either, a simple QDateTime axis seems to plot everything, including the weekends when there are no data points on the input (so you have gaps on the x axis). Why? I have no idea.