Anyone using Python on a large scale trading application?

[6yaNYCjm5m]

Quote from rosy2:

every place I have been converts all external messages to an internal (in house) format (ie. xml, json, binary something) so a market data message from IB looks like a market data message from FIX4.3 or whatever.

No issue there whatsoever. While I might heavily dislike xml and such, it is everybody's right to do in house data processing any way they need or see it fit, just don't force me to process it using the tools I don't like and to introduce one more message transformation layer that maybe (sometimes but not always) I wouldn't need to do. And when C,D,E,F,G (or whatever is the next in line language) comes to fashion, I would have to use the new client and rewrite my own interface for no good reason, because (assuming it was properly designed from the beginning) the actual protocol is not really changed. Well thought protocol could extend the new messages long way before any need for redesign. FIX is far from perfect example, but even today you could communicate with the COBOL system on the other side, without knowing anything about it, and you probably are

 

[6yaNYCjm5m]

Quote from chromosome:

Wow, great post 6yaNYCjm5m. I'm very familiar with the modules you mention. Another one of note is CVXOPT which is an optmization package.

I'll be damned, chromosome... I thought you were looking for some help, while in fact you are one old python lurking around for some inexperienced chicken... take care

 

[chromosome]

Actually, I'm just starting to explore the GUI side of the fence and your comments really helped. You also hear such mixed reviews of useful languages I like to seek opinions about using Python on a large scale. Most of what I hear is that it is used for rapid prototyping because it is slow compared to C++ etc.

I think with Cython however, it makes a really compelling choice for some pretty heavy lifting. I know a guy how uses Python/Cython exclusively for exotics pricing at a *large* fund.

 

[trading spaces]

Quote from 6yaNYCjm5m:


What people are missing in real-time charting is, it doesn't matter that CME pumps out at 4ms, you have network latency to start with and your screen can not refresh at that speed and your eye blinks in 150ms and your reaction is another 100 ms and people still believe they can scalp something if they click fast enough...

So would be better off trading with stops on globex servers?

 

[rosy2]

Quote from trading spaces:

So would be better off trading with stops on globex servers?

thats one of the few things the small guy can use to compete against the big technical fast firms.

 

[Pippi436]

Anyone tried Microsoft's Sho ?

http://research.microsoft.com/en-us/projects/sho/

Apparently MSFT is using this internally for data analysis since years. Its using IronPython (so easy .NET interop) and Intel MKL (should be fast). I like the interactive console for VisualStudio.

I have never really used it, but it looks sort of interesting.

 

[chromosome]

Looks pretty. I'm def with 6yaNYCjm5m on this one though: Linux/UNIX all the way

 

[Alcoeus]

Quote from 6yaNYCjm5m:

That's only the prototype, not having really intention to develop a full blown app, although 75% is already there. It's utilizing ChartDirector lib, which is a C++ lib and wxPython is just a wrapper for wxWidgets, which is also C++, so the whole thing is pretty fast. The charting lib is not natively built for real-time charting, but it can generate an image and pass it as a binary string to the wxwidget and it works OK. I actually keep it at 1sec refreshing time interval, although it could easily go down to 0.2, maybe even more, depending on how much stuff you load into it. That's when you would be pumping the real-time feed, in offline mode, it's irrelevant, you cant tell any lag while scrolling, zooming, etc. And I don't have any novelty graphic card, still running passive cooling, cant' stand the noise...
What people are missing in real-time charting is, it doesn't matter that CME pumps out at 4ms, you have network latency to start with and your screen can not refresh at that speed and your eye blinks in 150ms and your reaction is another 100 ms and people still believe they can scalp something if they click fast enough...
If doing something serious (maybe one day) I would probably develop core stuff from scratch mixing Cairo and wxPython, and that would look nice, be super fast, beat the crap out of .NET and Java app, run on everything.... As you can see I am constantly mean to Microsoft, Sun/Oracle and above everything to the new mean FASHION empire of Apple

Thanks for the tips! I never knew Python could be so capable graphically. I've only used it for scripting mainly and very little GUI work - I come from Perl

 

[mcdull]

Quote from 6yaNYCjm5m:


- Although it works perfectly on Windowz, try to move yourself to Linux, you will feel much better (I just can't resist not recruiting) and every now and then there are some things that are less awkward and patchy, hence more natural in that environment. Best example is multi-processing (not talking about multi-threading) where Linux forks processes much more efficiently and Python code is almost laughably simple, unlike Java counterpart (I just can't resist).

Hi 6yaNYCjm5m:

I have some python codes running test on tick data. I found the windows version of python is ~10% faster than the linux version on the same machine.

I am using Windows XP and Scientific Linux 6.1 (RHEL 6 Clone).

I ran some profile test and the major bottleneck is CPU and only Disk I/O on linux is faster.

Do you find the linux version of python slower? Or I did something wrong?

 

[6yaNYCjm5m]

Well, it's always tricky to enter the conversation about the performance, because it is very difficult to make so many variables involved in system to be static, while you measure the single one you are interested in. Usually I go with credible sources of information where numerous people report results in the amounts that make them statistically relevant.

Quote from mcdull:

I have some python codes running test on tick data. I found the windows version of python is ~10% faster than the linux version on the same machine.
I am using Windows XP and Scientific Linux 6.1 (RHEL 6 Clone).
I ran some profile test and the major bottleneck is CPU and only Disk I/O on linux is faster.
Do you find the linux version of python slower? Or I did something wrong?

All things equal, I am skeptical about 10% difference regarding python itself. It has to be something specific to OS. As I cited, I came across the statement that Linux is much more efficient in forking the process when you use multiprocessing module. Also, it has been stated that LInux handles opening of the files more efficiently, which would explain your better result in Disk I/O.
I have ditched Winz entirely several years ago, so I don't have the reference anymore. I am firmly in the Linux camp, but nevertheless think that 10% diff in either OS favor, points to something being configured wrong in the OS.