Trading Catechism

noddyboy said:
Ahh I see, that makes sense. You are trading at a higher frequency than I am.
More...
No not really. I trade on all frequencies - from microseconds to days and occasionally week(s). The only part of what I explained above that requires extreme low latency (ELL) is the C++ stuff. Everything else looks really slow by comparison.

The rest of the infrastructure is mostly for extreme productivity, agility, robustness and scalability - mostly as a programmer but also an end user of my own software. For example, here is an interesting article why people use these apache and other technologies

http://thenewstack.io/reactive-fram...ities-for-scalable-cloud-native-applications/

I don't need or use any of the apache stuff for the ELL stuff. One thing I do that is very important is I use RAID, and then ontop of that, I use GlusterFS to scale reliability horizontally across the cluster. This is because I would be devastated if I lost terabytes of saved data. It has nothing to do with speed, but reliability. It has already saved me once. One of my nodes recently died and GlusterFS didn't even blink.

Don't laugh, but the name of my system is, ISIS. :D

My ISIS stands for

Infinite
Systems
Infinitely
Scalable
 
Last edited:
@nitro I agree that off the shelf software just doesn't cut it. I get frustrated how some software forces you down a dead-end retail trading approach. That is why I have also chosen to roll-my own from scratch.

I trade a mixture timeframes,
- short term - trades last from seconds to hours
- medium term - trades last from days to months
- long term - trades last from months to years (my pension :)

Each of the above have very different technology requirements. The short term is the most challenging and requires more advanced programming techniques to handle more advanced requirements, such as optimal legging in techniques to get good executions, etc.

However, in all cases the technology stack is the same,

Mac OSx & Linux - no way am I running on Windows. My production trading system runs on Linux and I do my research on Mac.

Java - highly optimised, multi-threaded concurrent programming techniques allow me to monitor and trade multiple assets in real-time. I run concurrent back-tests which crunch data, i.e. co-integration tests, genetic-algorithm based optimisations, etc. All done using Java, broken into micro-services which can be deployed and run individually and thus are horizontally scalable should I need to add more capacity. I run locally but am thinking of moving to Docker so I can scale on the cloud.

For data storage I use hand rolled persistence using memory-mapped files, which are very fast and suit my requirements.

For the front end UI - I'm currently moving to a web based front end developed in JavaScript. Some may say it isn't fast enough, but I disagree, web based front ends are the way to go for custom visualisation of data. I use it for research and monitoring purposes only, all of the actual trading is essentially automated in the back-end.

Like @nitro mentioned, data is of utmost importance, if it's not correctly aligned, your tests are invalid. You have to check and double-check your data is correct.

I check data against multiple sources - free and paid. I filter real-time data and drop ticks that fail filtering. I have tests and validation built into the software at every stage to verify and check inputs and outputs, I've caught so many bugs like this. I use unit testing and integration testing to ensure quality. I'm a seasoned professional software developer with expertise in real-time Java programming for banks in the City of London. Its not all plain sailing - I've had a few mishaps - once a bug triggered multiple trades! Took me months to recover from that.

Edit -
IAH - LOL
The name of my system is SUCKS - Scalable Utility Completion Keying System
The next version will be called REALLY SUCKS V2.0
 
Last edited:
@nitro these are all interesting cutting edge technologies. I personally love this stuff (I'm a programmer and this stuff gets me out of bed in the morning!) However, it looks like we have two paths that this thread could explore, the technology side or the business side.

I'd like to explore the business side a little more and pick your brains, but if you want to explore the technology side then I'd be glad to continue down that path (it is your thread after all and I don't want to derail it)

Specifically, I'm interested in finding out which assets do you focus your direction-less analysis on and why, and which assets do you tend to avoid and why?
 
runtrader said:
@nitro these are all interesting cutting edge technologies. I personally love this stuff (I'm a programmer and this stuff gets me out of bed in the morning!) However, it looks like we have two paths that this thread could explore, the technology side or the business side.

I'd like to explore the business side a little more and pick your brains, but if you want to explore the technology side then I'd be glad to continue down that path (it is your thread after all and I don't want to derail it)

Specifically, I'm interested in finding out which assets do you focus your direction-less analysis on and why, and which assets do you tend to avoid and why?
More...
I think you are approaching it the wrong way. That the technology side and the business side is mixed in this thread is no accident.

Let me put it to you this way. Technology sometimes leads the way to progress and disruptions, and sometimes ideas and philosophy leads the way to progress and disruption. I think that 95%+ of all ideas that are obvious have been discovered in finance. Further, the people that are going to discover new groundbreaking paths aren't on ET, they are at Universities. So, it makes very little sense to talk about trading without also talking about how you are going to get there. After all, we have to pay bills.

So, although I don't particularly like equities and I love FOREX, IRs, and options on just about anything, I research everything these days - I have become completely universal. I collect tick data on every instrument I can, then use state of the art technologies to try to discover ideas, or refine my current thinking. That is why the technology is also featured here. The place that I am always seeking intellectually is positive feedback in my understanding. Said another way, a Unified Field Theory of Finance. That is my financial nirvana.

Let me give you an example. I am currently researching an intra-day mming equities idea. I have about 1 TB of saved tick data on equities. How do you go about analyzing tick data on 3000 symbols where you have to analyse it all synchronously in time? How do I tie that to the 5 TB of futures and FOREX data? What about the 60 TB options data? What about the 10 TB of intraday news events? You see, unless you can break that down (or you have a computer with 50 TB or RAM) you have to think about the technology side. Think about CERN. Where would it get with all their theories if they didn't have to think about colliders, about how to store and analyse the data from the collider? Physics would be analogously where finance was when people are putting up RSI and MACD.

In conclusion, technology is leading my creativity these days. I have the models, I have the data. Now it is time to crunch the numbers.
 
Last edited:
nitro said:
I think you are approaching it the wrong way. That the technology side and the business side is mixed in this thread is no accident.

Let me put it to you this way. Technology sometimes leads the way to progress and disruptions, and sometimes ideas and philosophy leads the way to progress and disruption. I think that 95%+ of all ideas that are obvious have been discovered in finance. Further, the people that are going to discover new groundbreaking paths aren't on ET, they are at Universities. So, it makes very little sense to talk about trading without also talking about how you are going to get there. After all, we have to pay bills.

So, although I don't particularly like equities and I love FOREX, IRs, and options on just about anything, I research everything these days - I have become completely universal. I collect tick data on every instrument I can, then use state of the art technologies to try to discover ideas, or refine my current thinking. That is why the technology is also featured here. The place that I am always seeking intellectually is positive feedback in my understanding. Said another way, a Unified Field Theory of Finance. That is my financial nirvana.

Let me give you an example. I am currently researching an intra-day mming equities idea. I have about 1 TB of saved tick data on equities. How do you go about analyzing tick data on 3000 symbols where you have to analyse it all synchronously in time? How do I tie that to the 5 TB of futures and FOREX data? What about the 60 TB options data? What about the 10 TB of intraday news events? You see, unless you can break that down (or you have a computer with 50 TB or RAM) you have to think about the technology side. Think about CERN. Where would it get with all their theories if they didn't have to think about colliders, about how to store and analyse the data from the collider? Physics would be analogously where finance was when people are putting up RSI and MACD.

In conclusion, technology is leading my creativity these days. I have the models, I have the data. Now it is time to crunch the numbers.
More...


Are you trading for a bank or for your own account? Can you recommend a broker if you are trading on your own? Thanks in advance!
 
@nitro I completely and utterly agree that Technology and business are bed fellows, they co-exist and one often leads the other causing disruption to existing ideas and methods.

However, personally, I think have the technology side sorted out. I too can analyse huge amounts of data and can scale the analysis horizontally on the cloud. For me, this is the straightforward part, I understand technology and how to use it effectively. I'm not knocking your choice of technology or methods.

Collecting, storing, cleaning and analysing that amount of data is a major feat and I'm in awe of the scale of your endeavours, and I understand why you mention CERN. I congratulate you on that. A quote springs to mind - "Impressive, Most Impressive" - D.Vader

Like most on ET I'm here to learn and exchange ideas and thoughts. I'm trying to accelerate my own analysis by learning from others that may have blazed the trail before. Hence my original question of what assets you analyse and why.

Some questions,
Why do you choose not to prefer equities?
Why do you LOVE currencies and IRs?
What about more esoteric equity markets?

Sorry if there are a bunch of questions here, I'm really interested in finding out why particular markets / products are more attractive for direction-less analysis and trading than another. I have my own thoughts on this but I'd be interested in your views.
 
You must log in or register to reply here.
Back
Top