Open Source Black Box Trade Platform in C#?

[greaterreturn]

Quote from heech:

"In the end, I believe well informed programmers will see that commercial and open source is a symbiotic relationship that benefits everyone."

I don't think this is a hard sell. Anyone who's drawn to this project will, by definition, be extremely interested in the making of money. I don't think many hard-core, anarchist, anti-capitalist ideologues... who are a pretty sizable community in the open source community... will be joining in!

Heech. You hit the nail right on the head. You may have noticed I was a little gunshy.

A year ago I got an idea for an open source dual license project in a totally different industry and was FLAME and bashed by that bunch you describe.

While I can see how they feel. It was refreshing to see that this thread and the project seem to attract a different ilk.

I welcome you all with open arms and will work hard to earn your trust.

Sincerely,
Wayne

 

[greaterreturn]

Quote from promagma:

Tick data is a big consideration .... many users will have a NinjaTrader level of C# competency, and will want to go through an install process and start coding some strategies, but will have no tick data.

Many of us programmers have been collecting data in our own systems ... I have some tick data for stocks, but no futures or forex. While a consolidated database would be a big undertaking - for now we could get in touch on the Nabble forum to share/swap data in TickZoom format.

Mark

(I am following this project, looks like things are off to a great start)

Wow. Now there's an idea. Sharing data. I'm curious. Are you the least bit concerned with the reliability of the data?

I'd be little more inclined to pool our resources and purchase the best quality data we can find. We'd have to make sure the vendor license allows sharing it afterwards.

Or find a legal way to do so. Like maybe we form an LLC that buys the data and we're all made members of it as a pool when we contribute so that legally we're a "corporate entity". Usually there's ways. But maybe not.

Otherwise, they would at least give us a big volume discount.

You have a great vision there. I LOVE IT!

Sincerely,
Wayne

 

[TSGannGalt]

Quote from greaterreturn:

Well, TickZOOM includes a real time tick storage and server. At least for MB Trading broker data. You can run it and collect your own real time ticks.

I think the real issue you mean is historical tick data. That's what got my attention about tick data.

But isn't there a company called "tickdata"? are they too expensive? I remember buying tick data from them for a currency about 10 years ago. It seems reasonable the time.

I have considered that if I keep collecting data, I may eventually have enough for these currency pairs to offer them to TickZOOM users at a price somewhat higher than the bandwidth cost to download.

Or let me daughter (she likes stuff like this) have the job of burning DVD's with data for a fee. I'm not sure which would be cheaper. But an option.

Especially this will be valuable if someone wants some history with Time and Sales (volume) and DOM which I'm certain tick data doesn't collect.

Wayne

Forget about collaborative data, for now.

1. A lot of us, pay(paid) and store enough historical data. It's hard to imagine any mediocre system trader requiring what's required of them in their business. Not being able pay for their minimum requirement makes me skeptical.

2. Stick to the infrastructure and the testing engine. I have no problem transferring my own data to TickZoom's and coding up a few things so that my quote servers are compatible with it.

3. Seriously, who needs collaborative servers? I've paid and got my own. Why don't you? Oh... you don't make money enough money trading systems...

Leeching and all that gimme gimme gimme shit...

 

[greaterreturn]

Quote from heech:

"Do you guys actually make money trading? Seriously, my confidence as a system trader comes from the ability to find edges, not from all the little codes I have."

Sounds like me. I have a specific strategy that has been moderately profitable (and will be more so once I get the bugs pushed out)... and I hope will continue to be profitable.

But my big gripe right now is my inability to really auto-trade reliably. I lost 1% today because NinjaTrader's partly broken, and I didn't realize a stop wasn't set. (It's partly my fault; there *was* an error message from a routine I wrote to catch this condition... I just didn't see it for 3-4 hours.) I also want to be able to integrate to a more reliable DMA broker. All in all, I just don't believe NinjaTrader is going to let me scale my positions up by a factor of 10 from where I am now.

If there was a better shipping solution that did this... and I'm actually looking into QuantHouse... no offense, I probably wouldn't have much time to devote to this project either. But right now, that doesn't seem to exist. (If one does, pm me!)

Heech. One more cool thing about TickZOOM. And I need to put a warning on the site. I'm glad you made me think of it.

That is, I had something exactly like that happen to me. It wasn't really a TickZOOM issue it was my own strategy code through an exception error and shut down the model.

Turned out it was because I used rolling bars and one bar had more than 5000 ticks in it (which was an outlier volume) so it had an access violation exception.

Well, what I did in TickZOOM was implement very complete and elegant exception handling everywhere.

And instead of failing when catching an exception, it will try calling the strategy again with the next tick and keep a count of how often the exception gets thrown.

It twice in a row, it stops the model. But if it's less than 1 in 1000 ticks, it let's it keep running and just logs it.

We need to put a WARNING because I can't be sure that's always a good feature. We need to make it optional.

But it just give you the idea of the philosophy. It need to be refined. I'm okay with it because I know it has other fail safes.

So we need to beat our brains to think of every possible fail safe.

And EVERY time someone has a failure in a live trading environment, EVEN if it's their own fault, they need to report it so we can see a pattern and try to devise a way to prevent or protect from those types of issues.

Agreed?

Sincerely,
Wayne

 

[greaterreturn]

Quote from TSGannGalt:

Forget about collaborative data, for now.

1. A lot of us, pay(paid) and store enough historical data. It's hard to imagine any mediocre system trader requiring what's required of them in their business. Not being able pay for their minimum requirement makes me skeptical.

2. Stick to the infrastructure and the testing engine. I have no problem transferring my own data to TickZoom's and coding up a few things so that my quote servers are compatible with it.

3. Seriously, who needs collaborative servers? I've paid and got my own. Why don't you? Oh... you don't make money enough money trading systems...

Leeching and all that gimme gimme gimme shit...

Okay. It's midnight. I'm off this thread till I get this system setup folks.

Besides in the nabble forum its TONS easier to just reply to email to participate in a thread.

That's critical so I don't get burned out.

Wayne

 

[TSGannGalt]

Quote from greaterreturn:

Heech. One more cool thing about TickZOOM. And I need to put a warning on the site. I'm glad you made me think of it.

That is, I had something exactly like that happen to me. It wasn't really a TickZOOM issue it was my own strategy code through an exception error and shut down the model.

Turned out it was because I used rolling bars and one bar had more than 5000 ticks in it (which was an outlier volume) so it had an access violation exception.

Well, what I did in TickZOOM was implement very complete and elegant exception handling everywhere.

And instead of failing when catching an exception, it will try calling the strategy again with the next tick and keep a count of how often the exception gets thrown.

It twice in a row, it stops the model. But if it's less than 1 in 1000 ticks, it let's it keep running and just logs it.

We need to put a WARNING because I can't be sure that's always a good feature. We need to make it optional.

But it just give you the idea of the philosophy. It need to be refined. I'm okay with it because I know it has other fail safes.

So we need to beat our brains to think of every possible fail safe.

And EVERY time someone has a failure in a live trading environment, EVEN if it's their own fault, they need to report it so we can see a pattern and try to devise a way to prevent or protect from those types of issues.

Agreed?

Sincerely,
Wayne

The exceptions you put in is very dangerous. This is coming from a guy who's traded "for" LEH (siginificantly before their fall out). You want the Exceptions as an Exception to do what's appropriate when deep shit happens, which happens quite frequently... This happens frequently than what the retail environment notices providing that there's major screening the brokers do, ALL THE TIME.

There's a reason to a lot of the late fills you get.... Not something to be discussed on a public forum anyways....

 

[greaterreturn]

Quote from TSGannGalt:

The exceptions you put in is very dangerous. This is coming from a guy who's traded "for" LEH (siginificantly before their fall out). You want the Exceptions as an Exception to do what's appropriate when deep shit happens, which happens quite frequently... This happens frequently than what the retail environment notices providing that there's major screening the brokers do, ALL THE TIME.

There's a reason to a lot of the late fills you get.... Not something to be discussed on a public forum anyways....

Your partially but not entirely correct.

Each exception is a different situation.

Many exceptions in coding are expected and a communication technique between methods.

I think you're refering to unexpected exceptions and I agree.

On further thought, it was a mistaken idea to generically handle any unexpected exception. It's better just to cleanly close all positions and stop processing than just fail and leave positions on, however.

I do catch and handle all exceptions around the method calls to custom strategies and custom indicators to protect myself and others from forgetting to handle them. Plus it's better not to have that code repeated and cluttering up strategies and indicators.

What I will do is switch the algorithm from what's above to a "close all positions--a.k.a. go flat and stop trading" for any unexpected exception.

Hey, I'm entitled to one bad idea.

NOTE: AT first I won't release the code for download, I'll just make it availabe for web browsing for a code review. This code has never benefited from a peer review.

The web browser is cool because it's color coded and has links to jump to class defintions, etc.

So it makes it easy to browse.

Plus based on questions. I'll probably need to draw up some Visio diagrams for the site as to the design of how it works.

All feedback will be appreciated and I'll endeavor to clean up most of that stuff before releasing it to SVN for you to checkout and start using.

We're getting close. I might have everything ready this weekend or soon thereafter.

Good night!

Sincerely,
Wayne

 

[soks86]

Why C#?

Microsoft is not someone I want anywhere near my trading platform.

Personally I have a few Java things in the works that use Open Source software but I don't ever believe I would release it, maybe if it's good enough (and complicated enough) that I know people will want support for it.

Hrm...

Either way though, C# is not a stable enough platform in my opinion nor is the community support strong enough in C# for open source projects.

Please feel free to object, I would love to discuss but if you want to talk about a serious project I would suggest looking into Java instead.

 

[greaterreturn]

Quote from soks86:

Why C#?

Microsoft is not someone I want anywhere near my trading platform.

Personally I have a few Java things in the works that use Open Source software but I don't ever believe I would release it, maybe if it's good enough (and complicated enough) that I know people will want support for it.

Hrm...

Either way though, C# is not a stable enough platform in my opinion nor is the community support strong enough in C# for open source projects.

Please feel free to object, I would love to discuss but if you want to talk about a serious project I would suggest looking into Java instead.

Agreed on most accounts. I'm not user about your point of "stability" that's scary and if true. a Certain motivation to convert.

I did it originially because MB Trading had only COM interface (of costly FIX).

But now that execution server is seperated by SSL 256 socket protocol from the main server. So it could potentially stay c# but convert the trading platform and engine to Java.

NOTE: If I do any converting, I will first experiment with C++ due to the potential for gain in speed.

Then Java .Net and Mono could all be supported if people want to use those languages for custom strategies and indicators.

Anyway, can you remember to add this into the issue database once I get that setup? Then people can vote on it prioritize it. Assign it for research, etc.

Wayne

 

[dcraig]

Quote from soks86:

Why C#?

Microsoft is not someone I want anywhere near my trading platform.

Personally I have a few Java things in the works that use Open Source software but I don't ever believe I would release it, maybe if it's good enough (and complicated enough) that I know people will want support for it.

Hrm...

Either way though, C# is not a stable enough platform in my opinion nor is the community support strong enough in C# for open source projects.

Please feel free to object, I would love to discuss but if you want to talk about a serious project I would suggest looking into Java instead.

Couldn't agree more. My own position is that I would not consider joining a project based on a MS environment. I might for Java. I have a LOT of my own Java code. And I wouln't consider Mono under any circumstances - it just doesn't have the same backing or support as Java and must in the end remain subservient to MS specs.

I'll bet there are more open source libs for Java than for C#. A couple that come to mind are Joone (neural nets) and Jgap (genetic algorithms).

If you were building yet another charting package, then there might be a good case for doing it on MS, but for the life of me I can't see why serious systems traders would want to go down that route except to use established packages which is not the case here.