Open Source Black Box Trade Platform in C#?

[heech]

"In the end, I believe well informed programmers will see that commercial and open source is a symbiotic relationship that benefits everyone."

I don't think this is a hard sell. Anyone who's drawn to this project will, by definition, be extremely interested in the making of money. I don't think many hard-core, anarchist, anti-capitalist ideologues... who are a pretty sizable community in the open source community... will be joining in!

 

[promagma]

Quote from HiFreekTrader:

- an Open real time tick server. There used to be OpenTick but it closed down. Data vendors unjustifiably raise prices for tick data, while prices for hardware and bandwidth fall all the time, even in relation to exchange volumes. This can be cooperative effort - if 1000 people subscribe and pay $10/month, that would be enough to rent servers, pay to the exchanges and hire one or two Ukrainian programmers to do the work.

Tick data is a big consideration .... many users will have a NinjaTrader level of C# competency, and will want to go through an install process and start coding some strategies, but will have no tick data.

Many of us programmers have been collecting data in our own systems ... I have some tick data for stocks, but no futures or forex. While a consolidated database would be a big undertaking - for now we could get in touch on the Nabble forum to share/swap data in TickZoom format.

Mark

(I am following this project, looks like things are off to a great start)

 

[TSGannGalt]

NOW...

This thread is starting to sound like a bunch of losers trying to get a piece of grandpa's inheritance...

GR. Be greedy. There's plenty of people who's looking solely for a damn good tester. I'm one of them, if I can cut my model testing time for my trading, let's say 10%, I'm willing to provide some help in this project.

Dudes...

Do you guys actually make money trading? Seriously, my confidence as a system trader comes from the ability to find edges, not from all the little codes I have. I'm only here to speak up because GR may have something that can help me make my hectic testing routines a lot more efficient.

I'm never going to give out my "models/mind" but I don't mind sharing my tools if there is a benefit to do so...

Tool is only a tool. It's hard to imagine a newbie to develop the same quality models as I do even if they use the same tool....

Look at technical analysis, it's a self-fulfilling stuff. Look at prop. firms, everyone in the firm has access to the same tools but the performance is different.

 

[greaterreturn]

Quote from TSGannGalt:

GR,

1. The performance numbers you posted are impressive. It's faster than what I have.

2. Stick with your original plan. And keep all the ownership.

Look at this thread with all the people trying to have a say in YOUR project. You should now realize that there is a high demand for this type of stuff.

Everyone needs a fast testing engine. If people want to use it, they contribute or pay. Don't worry about finding people. If your core engine is good, you'll find plenty of people wanting to use and join the bandwagon.

TSGG, you sold me. In fact, it kind of went to my head for 5 minutes and I thought about doing this whole closed source and commercial only.

And let me express my thanks to EVERYONE on this thread who has been so encouraging and especially patient with my concerns.

You have all convinced me to move forward. It'll be fun!!

Sincerely,
Wayne

 

[greaterreturn]

Quote from jprad:

You're right, there is a lot of interest for good tools and it sounds like Wayne's got a good core from what he's described so far.

And, I'd definately like to sink my teeth into a project that isn't going to have the rug pulled out from under us later on, which often happens in this industry.

IMHO, there's no way someone's going to contribute an equally good component, say a genetic optimizer, neural net or similar item if they have to forfeit ownership of their work.

Well, hopefully my other post cleared the "forfeit" concern you have.

As to your other comments, TickZOOM already has a genetic optimizer. That was critical due to the processing time of ticks (even after optimzing) because looping through 1000 combinations that take 10 seconds (even on a quad core CPU -- TickZOOM uses that technology) each is a waste when most of the combinations are meaningless.

Yep built that early on.

Maybe the algorithm can be improved but it's pretty good.

As far as neural nets. That's a different animal because it's part of the trade decision or "model" in the MVC pattern. So that would work better as a plugin or addon.

I implemented a business rules component and "built it in" but I'm going to rip it out and treat it as an add-on.

I wont' want to make the mistake of the windows operating system compared to the UNIK kernel.

Windows crashes and has to be rebooted from time to time. But the unix/linux kernel can run for many years with out rebooting--apparently forever except of needs of upgrading hardware, etc.

The secret or difference in the reliability is that the unix kernel only does 1 thing.

Whereas the windows operating system is so complicated it's easy to have bugs.

So my philosophy is to keep the idea of the tick engine. And keep it clean of the other stuff. It just focuses on generating bars and calling the custom stratagies and indicators.

Those custom strategies and indicators or plugin strategies and indicators can contain any kind of neural net or other stuff you want.

I hope this all makes sense.

Sincerely,
Wayne

 

[heech]

"Do you guys actually make money trading? Seriously, my confidence as a system trader comes from the ability to find edges, not from all the little codes I have."

Sounds like me. I have a specific strategy that has been moderately profitable (and will be more so once I get the bugs pushed out)... and I hope will continue to be profitable.

But my big gripe right now is my inability to really auto-trade reliably. I lost 1% today because NinjaTrader's partly broken, and I didn't realize a stop wasn't set. (It's partly my fault; there *was* an error message from a routine I wrote to catch this condition... I just didn't see it for 3-4 hours.) I also want to be able to integrate to a more reliable DMA broker. All in all, I just don't believe NinjaTrader is going to let me scale my positions up by a factor of 10 from where I am now.

If there was a better shipping solution that did this... and I'm actually looking into QuantHouse... no offense, I probably wouldn't have much time to devote to this project either. But right now, that doesn't seem to exist. (If one does, pm me!)

 

[greaterreturn]

Quote from heech:

Wayne,

Great, sounds good. And the tick database issue... again, that to me sounds like something the community of developers can work on in parallel. I don't see why the project wouldn't be able to support multiple databases... I assume you have some kind of database working now, and that's good enough for me.

I'm biased... I'm only working on minute bars, so I really don't care about the tick bar issue... but I'm probably not the only one. And I'm also greedy because I "feel" like getting started on a new platform as soon as possible.

So, here's hoping you'll plant that flag in the ground soon.

You're totally right, heech.

In fact, I don't even want to do it in parrallel. Let's just use db40 since it's GPL and we are too.

That's one less thing to spend time on.

If we have to deal with license issues or someone else later, let's cross that bridge later.

TickZOOM needs a better storage system quickly. Right now I manually rename and move files around to try different date ranges. Kind of mess.

Must be cleaned up and put that issue to bed. Anyone can implement other tick storage.

I'll always tinker with how to make things faster.

There's still a number of things to make c# faster but I may try converting the engine part of it to c++ as an experiment test to compare the performance. That might be faster in unmanaged code. But we'll see about that later.

Several people in PM and here have pointed out that my personal value add to the project it making it always perform very fast in the tick process and data flow. At least it's nice to feel appreciated.

Sincerely,
Wayne

 

[greaterreturn]

Quote from HiFreekTrader:

Great idea. Additional modules needed:

- an Open real time tick server. There used to be OpenTick but it closed down. Data vendors unjustifiably raise prices for tick data, while prices for hardware and bandwidth fall all the time, even in relation to exchange volumes. This can be cooperative effort - if 1000 people subscribe and pay $10/month, that would be enough to rent servers, pay to the exchanges and hire one or two Ukrainian programmers to do the work.

- an Open Broker. Wouldn't it be nice to pay commisssions on bare cost basis?

- an Open Exchange. Well, that's the direction.

Well, TickZOOM includes a real time tick storage and server. At least for MB Trading broker data. You can run it and collect your own real time ticks.

I think the real issue you mean is historical tick data. That's what got my attention about tick data.

But isn't there a company called "tickdata"? are they too expensive? I remember buying tick data from them for a currency about 10 years ago. It seems reasonable the time.

I have considered that if I keep collecting data, I may eventually have enough for these currency pairs to offer them to TickZOOM users at a price somewhat higher than the bandwidth cost to download.

Or let me daughter (she likes stuff like this) have the job of burning DVD's with data for a fee. I'm not sure which would be cheaper. But an option.

Especially this will be valuable if someone wants some history with Time and Sales (volume) and DOM which I'm certain tick data doesn't collect.

Wayne

 

[greaterreturn]

Quote from Batman28:

very interesting.. need a site dedicated to this.. dont want to lose track of this project

keep it going!!

Thanks for the encouragement. Guess I better stop answering this thread though and get it setup already!

 

[greaterreturn]

Quote from Pippi436:

GreaterReturn: how does TickZoom handle those large amounts of data during simulation/optimization? One limitation i faced with Neoticker was, that all of the data had to be loaded in memory to perform tick-replays - which is not possible for a long tick-history on multiple instruments.

By the way, this will be greatly helped by having a quad core PC. That's because loading the tick data is CPU intensive as well as processing the historical test.

It's much more cool if you have those running in parrallel which is what TickZOOM does naturally if you have multiple CPUs.

They're coll. They only help a little on a signal back test pass.

But when you run an optimization, it's cool to see all 4 CPUs peg out and clock the test finishes 4 times faster!!!

One thing I want to do later is use a technology I found to pool CPUs across PC.

That way if you happen to have a laptop or another PC and run a backtest, it's remotely kick off a job to run some of the test on the other PCs.

But what, that would be TOO powerful, wouldn't it?

Before I do that I will say to double the performance of TickZOOM itself at least once or twice more.

It's getting harder and harder to squeeze performance out of it. But it still has some more to give.