Write your own?

Quote from Gamer:

I am seriously thinking about starting to write my own charting software, consisting of only what I need, not bloated by thousands of useless marketing-type features that drags the performance down.

I have a strong technical background, so the programming knowledge is not a problem. I also appreciate that it's not going to be a short-term commitment.

Has anybody here written their own software that hooks up to their chosen feed through an API?

I am debating whether to stay with the native Win32 API or go with .Net. Of course Win32 with C would give the best performance, but I worry about the future of Win32, especially if more and more new Windows features are exposed only through the .Net API and Win32 becomes more like an undocumented part of Windows.

What do people think?
More...

I have done it in Java using jfreechart charting library with some extensions to that library for better "real time" performance. Supported data feeds are TWS, DTN IQ and Opentick (now defunct). I run it on Linux, but it should run on MAC or Windows with very little work.

I cannot see any justification for doing it in C/C++. There is enough work to keep you busy anyway without the extra imposed by C/C++. Java is easily fast enough. My current Q 9550 quad barely raises a sweat with four screens. Even in a fast market around the open, it's still > 75% idle when dealing with several hundred high volume stocks. Just about any dual core CPU would be quite sufficient.

With the trend being to CPUs with more and more cores, one consideration in choosing a programming language/environment is the quality of support for concurrent programming. Here Java easily wins over something like POSIX threads and no doubt WIN32.

The added value is not in drawing some charts as there are plenty of inexpensive packages that draw nice charts. Nor is it in making it "lightweight" as opposed to "bloated". The question is what are you going to do with it once you have some working charts ?
 
DCraig
Thank you. Exactly the sort of info i was after.

You have made a strong case for Java and having looked at the site I can see why.

I'll also have a look around C#, purely because it looks and feels like Java and I'll be able to use the .Net framework. I didn't seriously consider Java, I will now.
 
Quote from Gamer:

DCraig
Thank you. Exactly the sort of info i was after.

You have made a strong case for Java and having looked at the site I can see why.

I'll also have a look around C#, purely because it looks and feels like Java and I'll be able to use the .Net framework. I didn't seriously consider Java, I will now.
More...

I'm sure C# would do the trick, but you may have to pay for a charting library. If you go the Java route some other libraries (all free) that may be useful:

jdom - for XML eg configuration etc

joone - Java neural network library

jgap - Java genetic algorithm

rome - Java RSS

I use mysql for storing time series data, instrument database and stock fundamentals. There are arguments for/against relational database for time series, but it is free, robust. client/server and multiuser with few hassles and one less thing to worry about initially.

If you are starting from scratch you might like to look at the CEP stuff eg esper which is also free and open source.
 
Gamer,

I developed my app all in C++ for several reasons which are mostly of personal nature. First is that I develop in C++ on the Windows platform for a very long time. Second is that I really love to have great performance (even if a lot of people will claim that C# and Java will give you highest performance as well nowadays) – but using C++ I have full control when to use more than one core in a function (OpenMP) and when a functions should be separated into an own thread (MultiThreading). The third is that I do it beside my work – so I have no additional cost, only the time I spend, but I treat it as a hobby. Since I did all the development myself I can have a version for 32 and 64-Bit platforms and can make use of the enhanced features of each OS. So I will support DirectX 2D for enhanced charting in one of the next releases – the new 2D is released with Windows 7 and will also be available on Vista. So, if you stay with native Win32 your app will also be available on native Win64.

That were my reasons why I developed my application as a native Win32/Win64-Bit Application – others will have other reasons to have it done differently. But one thing is sure: If you start to develop the project, you will not have any time left to trade (with success!).

Daniel
 
I did all the heavy load server stuff with C++/Postgres and the nice front end stuff using C#. The C++ back end cross complies to Windows or Unix with the only difference being the daemon/service abstraction, I currently run it on Unix. Unless your front end absolutely has to run on Mac or Unix I would use C# as it destroys Java in terms of tools and ease of use. C++ has plenty of high level tools (STL, Boost, QT) and properly done cannot be beaten for speed (unless you go straight C or Assembler). I'm actually thinking of porting my front end to QT as it's about the same amount of code for platform agnostic GUI, the only bummer with QT is that it doesn't seem to have a good free charting options. But it's a 'spare time' thing at the moment.
 
I am doing that right now - starting my own. Mostly around the fact that it seems I am unable to get my hands on software that seems not to be written by people that should serve burgers at mcdonalds instead of programming (ninja, this is you) or just does not do what I need. Among these things are reliable 100% data collection (for an exchange or many, not a single symbol, and reliable even through a server restart for the trading server - by buffering input until the trading process is back online), client/server architecture, multi user capability and some other things.

I am going the way of C# - plan to write my own charting front end in WPF for a change. I use .NET 4.0 beta (timeframe matches the RTM) and Visual Studio 2010 - there is a lot of great new stuff there that makes sense in this scenario, and my timeframe allows to use it). Helps that once I have the prototype running, I will allocate 2-3 developers do that, but first I need to get my head around a lot of the basic.

I will use Zen-Fire as datafeed, using the "real" API (i.e. not the Zen-Fire API but the Rithmic API) and am currently struggling handling the feeds - things are easy if you go the usual way (ok, give me that symbol) and a LITTLE harder when you subscribe to all 400.000 or so CME feed symbols at once ;) We still have a performance issue in the RIthmic API side here (blows on AMD, works on Intel) - either they start working on it soonish or I will see I get a cheap intel server just to run the API connectivity process.

Their API is very powerfull but partially a little confusing / delicate to deal with. Not their fault in many cases, they simply expose a lot of the lower level functionality that is totally missed by a lot of other API's. I got the feed stuff handled (API wise - now working on storing the data, which is another problem in itself, thanks to the amount of data going through) and work on the order management.

If you want to discuss that in a more suitable scenario than a public forum... feel free to PM me.
 
It can be frustrating getting answers to the question you did not ask but in this case they are doing you a service discouraging you from writing the sleek program that is going to put those bars up like lightning.

The most important decisions involve setting priorities. Bolimomo has expressed the relative priority best.

Always remember: "To a man with only a hammer everything looks like a nail".


Quote from Bolimomo:


I think if you are serious about trading for a living, or trading to make money, you should spend more time on trading instead of writing your own charting package.
More...
 
What i am doing is that i take what works for me from an existing product and write the things that work differently than what i want myself. If you are on your own, without an IT department on your hands, that is much more manageable.

I'm using neoticker (native win32, fast) for visualization, trading system logic and testing, but roll my own OMS, riskmgmt, broker & feed connection and storage. Obviously you need a product with open specs to do this. Charts are covered extremely well by neo, so i never considered rolling my own in this area.

Netecture: what are you going to use to store & distribute large amounts of timeseries data? KDB+ and the likes are abit above my league..


P.s.: gamer, if you want to do this in C#, this might be of interest:

http://weblogs.asp.net/scottgu/arch...l-lt-asp-chart-runat-quot-server-quot-gt.aspx
 
Ah, the old storage question ;) THAT is actually where I am half stuck. Storing the data per se is trivial - despite anything, we do not really talk of that much data, not if you dump it into a text file (which is a first step, allows me to consolidate it later if something goes wrong - I basically write out a data dump file immediately when getting the data).

I plan using either file or SQL databases, but with a "catch".

* It makes no sense to store tick data in a relational table. Noone ever queries for it enough to justify the overhead.
* What may make sense is storing it as 15 minute per symbol binary encoded streams. The timeframe is "disputable" - I will try out various avenues here. I have no data yet how many bytes I really need for something like ES. I also yet lack proper disposition on how many updates I really get on extremely out of money options (which make up a significant portion of the symbol space).

Basically, becasue I also need to stream the data to workstations and between various subsystems I already have defined a binary market data format. THis is a stream type format that can handle different channels and multiple symbols....
...and I will use that to store blobs of X minutes of one symbol in a SQL database, using the new "SQL FileStream" storage (where the data ends up in a file in the disc).

The concept so far is:
* A MarketServer will handle distribution of real time data and also act as front end cache for rarely changing data (open, session high etc.) - so if some subsystem requests a current image, no db hit.
* The MarketServer is queueing items for storage, which get binary encoded into different streams, which get written out to the storage system.
* Requests for historical data are handled not "in sql" but together with the database in code - find the relevant files based on time, decode the streams etc.

I have to add that my hardware platform is possibly a magnitude faster than what most people have - the planned storage for the data right now is an 8 disc velociraptor raid 10 on a SAS cabinet with a high end Raid controller, and should I run into issues there, I can easily add another or two other sets of discs ;) I AM pretty concerned about the IO side, though - tests will see how that goes.

The REALLY funny thing is that so far, I have not ever gotten problems with C# performance wise. Helps to use .NET 4.0 (yes, beta - but they have new garbage collectors for servers that are "no stop") and VS 2010 (good new profiler). While I have tons of performance problems, they always point to "third party" code/reality so far (I have some identified at Rithmic, I think, and then SQL / Disc IO is beyong my control). I get surprisingly efficient execution at the C# level so far ;) I currently (right now) do tests mass-getting metadata from Rithmic (to identify and test code for allowed prices / price bands on instruments) and.... 65% of the CPU is purely within the Rithmic API ;)
 
Quote from Swan Noir:

It can be frustrating getting answers to the question you did not ask but in this case they are doing you a service discouraging you from writing the sleek program that is going to put those bars up like lightning.

The most important decisions involve setting priorities. Bolimomo has expressed the relative priority best.

Always remember: "To a man with only a hammer everything looks like a nail".
More...

Has it occurred to anybody yet that perhaps not everyone needs to be glued to a screen 24x7 to be a profitable trader?

Shock horror, I also read books, and spend time with my family! Crazy, isn't it? But then I sometimes have a thing called spare time.

I know, I know, I should be sectioned under the mental health act.

To the rest of the guys, many thanks for your inputs. I have enough material to get started, so your efforts are much appreciated.
 
You must log in or register to reply here.
Back
Top