Open Source Black Box Trade Platform in C#?

[good one]

I know that there's a high demand for a pure .NET FIX engine in the industry because there's a lot more front end programmers than the system infrastructure people, and people want a front-end FIX capability... you'll be surprised... seriously)

I'm sure this is the case. Just like I would love to have a group that comes over and cuts my lawn and takes out my garbage for free. It would be extremely effecient for me and would be in very high demand all around.....
The problem though is your ultimately just doing someone elses grunt work for free. I don't see how you escape that trap if you try to turn this into an institutional product.
Quantlib I'm sure benefits from the fact that its interesting work to do and share...this is just grunt work that I don't see how you will get people involved if its not purely out of their own individual self interest for their own projects.
As far as retail APIs, if you really want to get crazy I'm sure there is opportunity in some kind of open source retail version of FIX, maybe even a small license fee. I mean lets face it, the retail API situation is a complete joke and far from effecient.

 

[greaterreturn]

Quote from jdeezero05:

I'm sure this is the case. Just like I would love to have a group that comes over and cuts my lawn and takes out my garbage for free. It would be extremely effecient for me and would be in very high demand all around.....
The problem though is your ultimately just doing someone elses grunt work for free. I don't see how you escape that trap if you try to turn this into an institutional product.
Quantlib I'm sure benefits from the fact that its interesting work to do and share...this is just grunt work that I don't see how you will get people involved if its not purely out of their own individual self interest for their own projects.
As far as retail APIs, if you really want to get crazy I'm sure there is opportunity in some kind of open source retail version of FIX, maybe even a small license fee. I mean lets face it, the retail API situation is a complete joke and far from effecient.

Frankly, on further thought, jdeezero seems to be right too.

In fact, I don't need FIX personally. So Jdeezero really has hit the nail on the head. There has to be motivation to get something like that done.

There's only 3 I know of.

1. Someone using TickZoom wants/needs FIX so they add it themselves like Jdeezero says. The LGPL license will require them to contribute it back to the community source.

2. Philanthropy. (Let's not expect much of that.)

3. Someone pays money for it.

That can be either:

A. A single company/user who needs FIX and pays one of the TickZoom team or another developer to do it.

B. A market for a commercial version that makes it viable to charge a small fee for the FIX interface to each user.

It seems only time will tell which one it takes to get FIX added.

My guess is that A or B or both will happen down the road. FIX could be one of the added features why institutions pay for TickZoom and therefore contribute to the community effort financially.

Most open source projects survive by individuals using it for free but companies or power users paying for support and "power" features that business needs more than individuals.

Thanks Jdeezero for helping me keep my head on straight.

Sincerely,
Wayne

 

[greaterreturn]

Quote from Big:

Well, I don't mind doing it in an open source project, but as a professional software developer I get fairly picky as to what I am working on, so I'd have to see it first before committing.
That little project I am still going to do anyway, in my spare time (bear in mind I develop software for a big investment bank for a living during daytime) as a "test project" type of thing. Just to see what's what, and, say, how big is a db4o db with 10 years of ES tick data (or if I need to split it into years, months etc) and what to expect if I were to, say, ask for that particular week of that year to replay it. In fact I'll probably benchmark a little several scenarios to see if db size is an issue on performance, etc.
After that I'll see what I do with it, but I was indeed going to try my luck at backtesting few strategies of mine that work well in manual mode. If TickZoom is any good then sure, why not, I'll do it in there.

You're right. Let's wait till your benchmarks. Also, I was just realizing that the real power of OODBMS is storing object graphs and traversing them extremely quick compared to joins on RDBMS. Obviously, nothing like that is needed with tick data. There's no relationships to track. So it might be overhead to avoid.

When you get your benchmark, let's have some numbers like how many ticks in the file / DB. What size it is, and how long it takes to load into memory.

We can then compare apples to apples.

As for TickZoom, I already planned to allow selecting date ranges simply by including a map of dates available at the beginning of each tick storage file with the offset to the first tick of that date. That way the Tick loader can jump to specific dates.

That will be easy to add and keep the performance screaming fast. So the ability to "select" is of minimal comparison. The real question is speed.

So we can compare apples to apples when you get some numbers.

Wayne

 

[greaterreturn]

Quote from SideShowBob:

This sounds interesting and if it works as described I'd definitely pitch in.

Great! It certainly works as described. It might be a little bit rough around the edges since noone else has looked at.

It might take a number of manual steps to install it, etc. That could be cleaned up. But it's built by programmers for programmers anyway. So you kind of expect that, right?

In fact, I would love to do an old-fashioned informal code review before releasing it for beta test. Maybe I can talk a coworker into to do that with me.

Sincerely,
Wayne

 

[greaterreturn]

Folks, below please find a first draft of the official "roadmap" or definition of the TickZoom project and where it's going.

I tried to incorporate everything discuss here in this thread. If I missed something of you want to recommend a change, let me know.

---------------------------------------------------
TickZoom is an open source trading platform developed by a community of programmers who use it for their own personal automated or black box trading systems. It currently offers only limited features for manual or discretionary trading.

The major features of TickZoom are:

1. Built from the ground up to "zoom" through millions or billions of ticks.
2. Use any number or combination of tick, volume, range or price bars simultaneously.
3. Run a portfolio of your custom strategies against the same instrument.
4. Run against "baskets" of instruments to measure interaction or correlation.
5. Easily incorporate built-in or customized money management and exit strategies.
6. TickZoom runs in multiple modes:
Historical test, market replay, optimization, demo account, and live trading.
7. It reports all major industry statistics on system performance.
8. Custom strategies benefit from EasyLanguage-like features built-in:
A. This makes it much easier to write strategies in C# and..
B. Relatively simple to convert existing EasyLanguage indicators.

The different components are:

1. The main application which runs all the modes above including watching strategies in replay mode or even live trading with real time charts.
2. The order server which run the strategies in live or demo mode 24/7, hands free.
3. The execution/quote server that runs 24/7 to collect historical tick data and execute trades ordered by the order server.

Typically you run the order and execution/quote servers run on a hosted or VPS servers at a data center to benefit from redundant power and network systems.

FAQ

1. Does it support non-programmers?
While TickZoom has features to make it easier to create custom strategies and indicators in C#, and makes it relitively easy to convert EasyLanguage code, it still will be a learning curve for non-programmers.

ROAD MAP For the Future

The project will continue to build on and refine the core funtionailty above and:
1. Add support for redundant, clustered servers with fail over ability.
2. Eventually seek to engage paying customers by offering services that they need.

Pricing: TickZoom is available under dual license.

Open Source LGPL:

For the TickZoom community of developers who download and run the source code, they have free use of the software under the LGPL license. That license only has requirements of submitting any changes or improvements back to the community. We welcome more committers!

Commercial:

For other individuals or organizations who prefer a packaged product, user-friendly features, technical support, documentation and other services, TickZoom.org will offer these. The software also will have an unlimited duration trial period with a license required for running in production. Either life time fee or monthly subscription will be available. The subscription fees include support. Lifetime licenses will have an additional reduced yearly support fee.

Status:

At this time, the project is in beta testing only to a limited number of community members.
-----------------------------------------------

 

[heech]

Hi there,

Just stumbled onto this thread. I love, love, love this idea. I don't know if I have substantial time to devote to the project, but I absolutely will:

a) use it, (I have a 2000 line C# strategy running on NinjaTrader, and have played with both Wealth Lab Pro + OpenQuant),

b) debug it, (there are *always* framework problems... and I can't wait for an open source solution that lets me investigate thoroughly)

3) probably help improve it. I for one am very interested in FIX/backend integration, and would help in the effort.

I'm primarily a C/C++ and Java programmer, but picking up C# has been a snap. I'm currently running an always-on automated trading strategy implemented in NinjaTrader in a hosted server. It works reasonably well, but has several key flaws:

- not "really" meant to be automated. (I have to setup numerous macros to handle all of the error messages that can pop up, and to setup a strategy with my parameters at the beginning of the day.)

- doesn't support enough broker options.

By the way, just another sample point for you... I couldn't care less about tick data. I trade primarily on 30 minute bars, with occasional drops into 5 minute bars. I believe NinjaTrader just accumulates 1m bars into the larger bars, and that kind of solution works fine for me.

Really look forward to seeing this work out for my own selfish interests.

 

[bwolinsky]

Quote from greaterreturn:

Folks,

Please join in a discussion of your thoughts and/or encouragement to release an open source black box trading platform in C#.

This is a platform that I developed and use myself in automated trading.

I'm considering releasing it as an open source project for others to use. I like the idea of a community collaborating on building a great system.

Unfortunately, there's no such open source system available today.

Let's call this platform "TickZoom" for the sake of clarity in the discussion.

Why am I reluctant to release it?

Well, very few will ever use it unless there's very good support. So I've been exploring ways to generate revenue to provide support while still making the system freely available and open source.

After researching more about successful business models for open source, it works best to always keep the source code free of charge and easily accessible so the community can contribute without charge. To pay for full time developers, an organization sells services like creating a packaged product that's easy to install and selling support contracts, training, documentation, etc.

In other words, some people can contribute with time to the project while others contribute with money. But the software license to use and copy the source code or software will remain free of charge.

In my next post, I'll attempt to explain some of the features of this platform and how it differs from the commercial close source ones on the market.

Sincerely,
Wayne



Bold = text edited on 20081209 17:30

Sorry, this is exactly what Ninjatrader is, and it's even in C#. I wouldn't waste anymore time trying to re-invent the wheel.

 

[heech]

PS. This is a bit of a tangent here.. but in terms of "wish lists" and what I would implement, if given the chance...

I personally would work on building a clustered/quorum approach. I can't stand the idea that my code is dependent on a single point of failure, not with hundreds of thousands (and hopefully one day millions) sitting on the line.

I want to build a solution that sits on 2n+1 different servers. All of them receive the same (synchronized) data stream, run the same strategy, and will place trades as long as n+1 of the servers agree it makes sense to do so. This way, I can have n-1 servers (across different data-centers) completely fail, and still have confidence my strategy is running.

If n==1, then that means I run 3 different machines, and can tolerate any one machine going down.

If n==2, I'll get 5 machines, and can tolerate two going down.

That's the kind of reliability I want, would write code for, and/or would pay for.

 

[bwolinsky]

Quote from heech:

Hi there,

Just stumbled onto this thread. I love, love, love this idea. I don't know if I have substantial time to devote to the project, but I absolutely will:

a) use it, (I have a 2000 line C# strategy running on NinjaTrader, and have played with both Wealth Lab Pro + OpenQuant),

b) debug it, (there are *always* framework problems... and I can't wait for an open source solution that lets me investigate thoroughly)

3) probably help improve it. I for one am very interested in FIX/backend integration, and would help in the effort.

I'm primarily a C/C++ and Java programmer, but picking up C# has been a snap. I'm currently running an always-on automated trading strategy implemented in NinjaTrader in a hosted server. It works reasonably well, but has several key flaws:

- not "really" meant to be automated. (I have to setup numerous macros to handle all of the error messages that can pop up, and to setup a strategy with my parameters at the beginning of the day.)

- doesn't support enough broker options.

By the way, just another sample point for you... I couldn't care less about tick data. I trade primarily on 30 minute bars, with occasional drops into 5 minute bars. I believe NinjaTrader just accumulates 1m bars into the larger bars, and that kind of solution works fine for me.

Really look forward to seeing this work out for my own selfish interests.

EXACTLY! You're already in ninjatrader, so no other platform is needed. You also mentioned WL, which I'll assume is the 5.0 .Net version. Trust me, you need inside info as to how to deal with Financial Information Exchange protocols. The more you get into it, the more confused you'll be. Trust me, it doesn't get any better than NT for this.

 

[heech]

Quote from bwolinsky:

Sorry, this is exactly what Ninjatrader is, and it's even in C#. I wouldn't waste anymore time trying to re-invent the wheel.

Speaking as a (mostly satisfied) life-time subscriber to NinjaTrader who's dependent on it every day... no, this is absolutely not what NinjaTrader is.