Developing "Spartan"

[nooby_mcnoob]

I've been experimenting with running some intraday models to complement me and one of the problems I consistently have is my tweaking often leaves me stuck with no idea what I changed from one model to the next. This is actually a form of robustness because I delete everything and start again quite regularly when my results go crazy.

Anyway, I currently have a "series" table which can hold date/observation pairs associated with a series. Considering (ab)using this table to store model history as well. Would allow me to revert, compare and test trained models. However, models are only one part of it. The other part of it is the training dataset (preprocessing, normalization, etc). So perhaps each instrument model would have one or more associated characteristic:

* Instrument id
* Model
* Training data
* Test data
* Metrics

Creating tables isn't expensive or annoying for me, so I suppose I should simply have a Model table. The problem was likely to occur when looking at storing training data. But I just checked that and it looks like it's about 5MB per model according to pd.DataFrame.memory_usage() so I could compress it as I did before with tick data and make it nearly cost-free.

Not ready to commit to this yet, but just collecting my thoughts.

 

[HobbyTrading]

Whether you store a logbook of your activities and studies in electronic form or on paper does not really matter much (to me at least). It is the discipline to log each activity which is important. After a while the discipline tends to become less and the log becomes incomplete.

 

[d08]

I used to have a terrible CPU issue with Matplotlib and almost abandoned it but ended up solving it pretty easily. When you have your chart drawn out on the axis, everything on the chart can be accessed with the get_children() method for said axis. It's some kind of list and the first three elements in the list are always the candle bodies and stems. Call the remove() method on those and they disappear. Then just use the candlestick2_ohlc() function to put the updated candles back on there. It does completely redraw all the candles (though there is a way to just get the last one though I'm too lazy to put it together) but on my machine I can update my charts every 100 milliseconds and it takes about 10% CPU so maybe give that a shot. I'll attach a little snippet with some of the relevant code to give you an idea if you aren't familiar with it.

Edit: If you Google candlestick2_ohlc you'll find plenty of stuff

Why not use pyqtgraph for this? It's much much faster and very versatile. I began trying to implement QtChart but it's very cumbersome to use and I ran into silly problems like plotting the datetime axis properly - QtChart will plot all periods, that means weekends and holidays included. I switched to pyqtchart and after a few hurdles, I could achieve exactly what I wanted or very close to it. On top of that it's very fast looks great. It should be the first choice in my opinion.
Matplotlib is great for static charts but not for anything dynamic or with a lot of data points. I still use it to plot equity and drawdown analysis but that's about it.

 

[nooby_mcnoob]

Whether you store a logbook of your activities and studies in electronic form or on paper does not really matter much (to me at least). It is the discipline to log each activity which is important. After a while the discipline tends to become less and the log becomes incomplete.

I don't like to rely on discipline, but process. So something like:

* develop
* develop
* develop
* Hmm I like this, save with comments
* continue

Sort of like with source control which is the model I'd want to follow.

 

[nooby_mcnoob]

Why not use pyqtgraph for this? It's much much faster and very versatile. I began trying to implement QtChart but it's very cumbersome to use and I ran into silly problems like plotting the datetime axis properly - QtChart will plot all periods, that means weekends and holidays included. I switched to pyqtchart and after a few hurdles, I could achieve exactly what I wanted or very close to it. On top of that it's very fast looks great. It should be the first choice in my opinion.
Matplotlib is great for static charts but not for anything dynamic or with a lot of data points. I still use it to plot equity and drawdown analysis but that's about it.

Hmm, I will give this a shot though I use jupyter as the interface. https://stackoverflow.com/questions...aph-plot-to-work-in-ipython-repl-or-jupyter-c

 

[SteveH]

In pyqtgraph examples, there's a sample candlestick plot:

https://github.com/pyqtgraph/pyqtgraph/blob/develop/examples/customGraphicsItem.py

 

[Axon]

Why not use pyqtgraph for this? It's much much faster and very versatile.

Short answer is I have not found this to be the case. Slightly longer answer is Matplotlib is a tricky library that has been described by some to be almost "charting via side effect" haha. After using it for many years I've figured out all the quirks to make it very fast (I use it for charts that update every 100 milliseconds using less than 10% of CPU though actual real time is certainly possible) and flexible, i.e., there's nothing I ever want to do that I can't accomplish with it. I've tried a few different platforms like d3 and pyqtgraph but never found them to be a compelling alternative beyond being more noob friendly. However, I'm always open to something better so if you could give me some scenarios where pyqtgraph is appreciably faster or more flexible I'm open to it. Also if there's something you've tried to do with matplotlib that had you stymied fill me in and I'll tell you what I think.

 

[nooby_mcnoob]

Short answer is I have not found this to be the case. Slightly longer answer is Matplotlib is a tricky library that has been described by some to be almost "charting via side effect" haha. After using it for many years I've figured out all the quirks to make it very fast (I use it for charts that update every 100 milliseconds using less than 10% of CPU though actual real time is certainly possible) and flexible, i.e., there's nothing I ever want to do that I can't accomplish with it. I've tried a few different platforms like d3 and pyqtgraph but never found them to be a compelling alternative beyond being more noob friendly. However, I'm always open to something better so if you could give me some scenarios where pyqtgraph is appreciably faster or more flexible I'm open to it. Also if there's something you've tried to do with matplotlib that had you stymied fill me in and I'll tell you what I think.

I would suggest you give plotly a try. It is way more limited than matplotlib but it is also interactive by default.

I use the Jupyter integration, like this:
Computationally intensive primarily because I'm too lazy to make it efficient, but results in interactive charts (translate/scale/hover) like this:

 

[globalarbtrader]

Short answer is I have not found this to be the case. Slightly longer answer is Matplotlib is a tricky library that has been described by some to be almost "charting via side effect" haha.

Love this comment.

"Why offer just one confusing API, when we can offer two and make it twice as confusing?"

GAT

 

[nooby_mcnoob]

Love this comment.

"Why offer just one confusing API, when we can offer two and make it twice as confusing?"

GAT

There's a method to the madness... Convenience vs power. Used to bother me too.