Developing "Spartan"

nooby_mcnoob said:
@HobbyTrading's comment on my post where he asked me if I may be shifting goalposts was a good one. I sat and thought about what is the absolute minimal thing I need to do to get the next goal and I realized that order fills have nothing to do with it, simply orders and position updates which is like 8 lines of code. I will be ripping out any order fill related stuff for now. Thanks bro.
More...
You're welcome. There are basically two ways of handling a project: either top-down or bottom-up.
In case of bottom-up you work from one intermediate goal to the next and at the end connect all bits and pieces together, to achieve the ultimate goal. I've met many people who like this way of working. However, at the time of stringing everything together you might find that you strayed along the way and are not able to connect the bits and pieces in a nice and clean way.
The other approach is top-down, where you first describe the ultimate goal, subdivide that in smaller intermediate goals, and subdivide each of those in smaller steps. Implementing these smaller steps make that you won't stray as easily. This is the way I prefer to work: while working on implementing the smaller steps I still have the visibility of the ultimate goal of the project.
 
HobbyTrading said:
You're welcome. There are basically two ways of handling a project: either top-down or bottom-up.
In case of bottom-up you work from one intermediate goal to the next and at the end connect all bits and pieces together, to achieve the ultimate goal. I've met many people who like this way of working. However, at the time of stringing everything together you might find that you strayed along the way and are not able to connect the bits and pieces in a nice and clean way.
The other approach is top-down, where you first describe the ultimate goal, subdivide that in smaller intermediate goals, and subdivide each of those in smaller steps. Implementing these smaller steps make that you won't stray as easily. This is the way I prefer to work: while working on implementing the smaller steps I still have the visibility of the ultimate goal of the project.
More...

Sounds like what you're saying is not top-down design, but top-down goal setting. This makes sense. It doesn't sound like we're too far apart.

Top-down design often has the problem that you won't learn about problems in your design until you implement things and I've designed myself into a corner on many occasions.

As a result, the way I like to work is to have a high-level goal, which in this case, is something like "The parts of ForexTester I used in my backtesting, but for live trading" and then just make small incremental steps towards it. A good part of this process is continuously thinking critically about what I'm working on and whether it is the most important thing I could be doing towards that goal. Sometimes I do it unprompted, sometimes other people have to question what I'm doing for me to think about it while on a walk or whatever.

In the end, stringing things together for the first time is always gross and never works cleanly, so I optimize for making _that_ part easy as opposed to having it work nicely the first go around.

What do you think?
 
nooby_mcnoob said:
What do you think?
More...
I have also cornered myself on several occasions, especially as I was not really experienced in writing software (I still consider myself not very experienced). But lately I seem to have found a way which works nicely for me. When a new software project comes to mind I don't immediately open an IDE and start coding. Instead I first take time (which can take several days or a week, depending on the size of the project) to make it clear to myself what I want to achieve. In this phase I use a text editor to write it down. Writing it down forces you to think more careful and precise otherwise you won't understand your own words when you reread your text several weeks/months later. So I write this document for what I called in my previous post the "ultimate goal", each of the "intermediate goals" and, where relevant, for the "smaller steps". Once I've done this can I insert in the text the way how to implement each of these steps, what data formats to use for the input data, what data formats to use for the results and so on.
Only when this has all been completed do I start an IDE and start coding. It then becomes a translation of sorts, translating from English sentences into your programming language of choice (java in my case).
Additional benefit: once the code is ready do I immediately have a journal annex design and implementation manual, which describes what the goal was, how it is implemented and why this way was chosen. If, at a later date, I decide to make changes, I go back to this original document and describe the change plus its motivation. The document has now turned into a change log as well.
All this sounds very tedious but it seems to work efficiently for me.
 
Hey look at that, we do the same thing :)

upload_2019-2-10_11-54-11.png


The main difference is you seem to use that as your journal, whereas I generally only write things down to start. I think I may start doing this, great suggestion.
 
Buy vs build

One of the things I've learned in my work is to refactor mercilessly if you are after high quality. It gives you a chance to revisit and harden or rethink concepts that you have had the benefit of working with for a while. I hit a point in my progress where I was looking for a "current date", but there was no singular source for this. For forward testing, it is less relevant, as you can just use the real current date, but for backtesting, it's problematic. Since I am currently working on backtesting, it's important to get right.

I was upset with myself when I hit this hurdle, though I knew it was coming since I hadn't designed it ahead of time. Rather than attack the code, I let the problem sit in my head for a couple of days, didn't write any code, and then in two hours of coding, I had solved the problem. My favourite part of it all: less code, more understandable, though admittedly not by much.

Before (2051 lines):
upload_2019-2-14_8-36-55.png


After (1982):
upload_2019-2-14_8-36-30.png


This is the guts of the model (i.e., the part of the app that actually does stuff). I know a lot of people like using an inversion of control container but I am old school - having written 3 or 4 myself in various languages - they make it too easy to "move fast", which is not necessarily what I'm after.

The charting code is still pretty ugly. I think by the end of this week, I should have the backtesting (and thus the forward testing) pretty well sorted.

Code: 
        from spartan import orm
        from configparser import ConfigParser
        import os
        import logging

        self._config = config = ConfigParser()
        config.read([
            'TradingApp.cfg',
            os.path.join(os.path.dirname(__file__),'TradingApp.cfg')
        ])
        logging.basicConfig(level=getattr(logging,config.get('general','loglevel',fallback='INFO')))
        orm.initializeFromConfig(config)

        self._session = Session()

        def ibConnectionFactory():
            import ib_insync as ib
            connection = ib.IB()
            host = config.get('InteractiveBrokers','host',fallback="127.0.0.1")
            port = int(config.get('InteractiveBrokers','port',fallback=4001))
            clientId = int(config.get('InteractiveBrokers','clientId',fallback=1))
            connection.connect(host,port,clientId)
            return connection

        self._broker            = SimBroker()
        self._barManager        = BarManager()
        self._dataManager       = DataManager(self._session,self._barManager)
        self._timeSeriesManager = TimeSeriesManager()
        self._dataSourceManager = DataSourceManager(ibConnectionFactory)
        self._instrumentManager = InstrumentManager()
        self._accountManager    = AccountManager(self._broker)
        self._backtestManager   = BacktestManager(self._session,self._dataManager,self._accountManager)

        self._dataManager.addStudies( ...  )
 
Volume

One problem in getting bar data from IB for Forex is that IB does not have volume information unless you are getting tick data through reqHistoricalTicks. At the moment, I only have bar data, but no volume data. One of the signals I use requires volume data and this is one of the problems with using the IB interface: it does not show volume for Forex. Not sure why they do things this way, but they do.

So once the backtesting work is roughed in, I will need to go back and revisit the volume. I should probably post somewhere else asking if someone has done this.
 
Butting in here, with retail forex you are trading against the brokers algo that creates/maintains the spread. With Forex Futures there is no spread and your broker is not running an artificial spread algo against you.
 
They said:
Butting in here, with retail forex you are trading against the brokers algo that creates/maintains the spread. With Forex Futures there is no spread and your broker is not running an artificial spread algo against you.
More...

You're not butting in, thanks. To be honest, I do not really understand futures that well. I will make it a point to learn though.
 
You must log in or register to reply here.
Back
Top