C

[stevegee58]

Even the Black Scholes example I gave doesn't actually accelerate the calculation of a single option. What it gives you is the ability to simultaneously calculate a whole option chain all at once.
It just depends on what you're trying to do.

 

[hftvol]

that is wrong on so many ends...so where to begin? For beginners, most all compiled languages you listed can read I/O that hits the ceiling of the bus, thus C cannot be faster than C# or C++, or Java in that regards. I/O is pretty much always the bottleneck thus saying C reads in data faster or is less resource hungry is hogwash. You can specify down to the single byte how large your buffer is thus you yourself set how much memory you want to spend on buffering read data before they are processed and garbage collected or destroyed.

With all due respect but anyone writing C or even C++ code to perform file I/O in 2013 either is hopelessly behind and has no clue how simple it is to do the same in, let's say C#, at exactly the same read/write performance that you would get had you written it in C or C++, OR, has an agenda possibly C being the only language he/she knows well and thus it must by his/her definition be the best way.

As an example, my time series binary data store reads in data from disk at a throughput that is capped at the bus speed, period. It cannot get faster than that. The only way to speed things up beyond that is to cache/store the data in memory and read from there. Not sure why someone would write the ultimately clumsiest code in C that you could find in this universe to perform file I/O.

Quote from bln:

Well, because C is a fast and resource efficient language compared to interpreted languages like Ruby, Python, Java, C#, etc. Python and Ruby runs around 5-10 times slower than C, that do mean longer delays for the end result.

If the main code is written in C then you may also want the support code around core it to be written in it C too. Modern C compilers is very good at optimization and can auto-vectorize your code to some degree, and automatically use all these nice instructions you have access to, SSE, AVX, AVX2, XOP, FMA4, etc.

 

[hftvol]

dude, seriously you must be kidding. Clearly you have no clue what you are talking about. Many of today's code is in fact designed from the ground up to execute in parallel. Many of my trading architecture's components run totally in parallel. Then look at .Net 4.5, pretty much the whole library makes it a dream to write from the beginning parallel code, PLINQ, Parallel.ForEach, TPL Dataflow, async/await framework, task parallel library, just to name a few. Then pretty much all modern web applications can be fully parallelized. Net traffic just naturally lends itself to being parallelized.

I think I showed enough examples to completely prove you incorrect.

In regards to algorithms, most divide and conquer algorithms are begging to be parallelized. Please think before you write.

Quote from vicirek:

There are almost no real life algorithms that would be immediately 100% parallel ready. Most of the programming effort is made to convert such algorithm to take maximum possible advantage of parallelism either through hardware or software acceleration. Usually there are some chunks of code that cannot be processed in parallel but there are ways to synchronize those operations with some degree of performance penalty but the end result is significant speedup of entire application.

 

[hftvol]

of course, if you have code that must execute sequentially because all intermittent results are path dependent then its hard to not run this synchronously. Else I am not sure what you are trying to say.

Quote from stevegee58:

Even the Black Scholes example I gave doesn't actually accelerate the calculation of a single option. What it gives you is the ability to simultaneously calculate a whole option chain all at once.
It just depends on what you're trying to do.

 

[stevegee58]

Quote from hftvol:

of course, if you have code that must execute sequentially because all intermittent results are path dependent then its hard to not run this synchronously. Else I am not sure what you are trying to say.

I was just amplifying (unsuccessfully) what I said earlier. We're in agreement.

I was just pointing out a real example where parallel calculations are possible. i.e. a whole option chain

 

[bln]

Quote from hftvol:

that is wrong on so many ends...so where to begin? For beginners, most all compiled languages you listed can read I/O that hits the ceiling of the bus, thus C cannot be faster than C# or C++, or Java in that regards. I/O is pretty much always the bottleneck thus saying C reads in data faster or is less resource hungry is hogwash. You can specify down to the single byte how large your buffer is thus you yourself set how much memory you want to spend on buffering read data before they are processed and garbage collected or destroyed.

I see people complaining all the time that they have problems running their Java or Python stuff on a Raspberry PI with 256 MB ram. Whats the problem then? that they can not fit and run their programs inside 256 MB? for me as a C programmer 256 MB of RAM is a huge amount of resources.

How about L0/L1/L2 cache utilization? If you do not have the ability to control the data types you use down to the exact bit length, you are wasting resources. It is dumb to use a 64 bit size Int then an 8 bit Int would suffice.

Memory bus utilization? Same thing, there is not an infinite capacity between CPU and the memory.

In the same way OOP wastes resources, both on the bus and cache/memory. And not let get started on the effects of automatic GC, thats a real speed block.

With all due respect but anyone writing C or even C++ code to perform file I/O in 2013 either is hopelessly behind and has no clue how simple it is to do the same in, let's say C#, at exactly the same read/write performance that you would get had you written it in C or C++, OR, has an agenda possibly C being the only language he/she knows well and thus it must by his/her definition be the best way.

That is not true. From personal experience I do know that I/O is faster in C vs Java.
Then downloading a file from the internet using Java's builtin http/url class vs uring libcurl from C, libcurl is much faster than the Java counter part. And this is TCP socket I/O. Why is this?

And the parts that are not I/O bound? usually a small part of the overall program is waiting from external I/O operations.

 

[hft_boy]

Quote from bln:

I see people complaining all the time that they have problems running their Java or Python stuff on a Raspberry PI with 256 MB ram. Whats the problem then? that they can not fit and run their programs inside 256 MB? for me as a C programmer 256 MB of RAM is a huge amount of resources.

How about L0/L1/L2 cache utilization? If you do not have the ability to control the data types you use down to the exact bit length, you are wasting resources. It is dumb to use a 64 bit size Int then an 8 bit Int would suffice.

Memory bus utilization? Same thing, there is not an infinite capacity between CPU and the memory.

In the same way OOP wastes resources, both on the bus and cache/memory. And not let get started on the effects of automatic GC, thats a real speed block.



That is not true. From personal experience I do know that I/O is faster in C vs Java.
Then downloading a file from the internet using Java's builtin http/url class vs uring libcurl from C, libcurl is much faster than the Java counter part. And this is TCP socket I/O. Why is this?

And the parts that are not I/O bound? usually a small part of the overall program is waiting from external I/O operations.

Generally in agreement. You ask why is the Java code slower? IMO it's due to laziness of Java programmers . They are so used to the JVM JITting their code that they don't realize that there are ways to optimize the code they write. IMO C programmers are more cognisant of the resources they are using so they use them more efficiently.

Of course you can always write faster code in C, but it'll take much longer to write the code which will even be as fast as the JITted counterpart.

This whole argument is pretty silly in my opinion. The OP wanted to write a library in C. Nothing wrong with that, and I'm sure the OP is aware that C is not used much nowadays except in legacy or high performance code (I might add that these are very important applications like OS kernel code, or JVM / CLR implementations which allow people to use higher level languages in the first place). But it is an interesting project and something useful may even come of it.

EDIT: Btw you are right that Java is more I/O bound than native code. There is a pretty large overhead due to copy in/out of the JVM. You can do stuff with NIO which helps some but at that point you basically may as well be writing C anyways.

 

[vicirek]

Quote from hftvol:

Please think before you write.

This applies to you because what I said is exactly the same what you repeated in your rant that states the obvious.

 

[cdcaveman]

favorite thread for learning about programming languages from difficult, know it all programmers.. haha

 

[hftvol]

256mb? Hey, wake up, we live in 2013, even my Galaxy Note 2 has 10gb of internal memory. Also, as I stated any well written C# IO library can into buffer any n-bytes thus total memory consumption lies well below your stated 256mb.

My quotes are 16byte of size. 8 bytes for the time stamp (yes I take the luxury to represent full time stamp rather than deltas), 4 bytes for the bid and 4 bytes for the ask. (please note this is for my non hft-platform, where microseconds really do not matter at all).

I do all that in C#, so please do you care to enlighten me what edge C has over C#? I argue that it has ZERO advantage, none, nil except for very niche performance critical code..., to the contrary it takes you longer to whip up code snippets than it would take me to write equally efficient code in C#.

Your bus OOP blurb makea zero sense. You cannot bypass bus capacity limitation with C!!! Nor with any other OOP language.

If you do not parallelize your web request code in C then I can show you how my C# version will run circles around your code. Care to code up a simple webcrawler in C and compare results?

Please do not participate in this discussion if you do not really know what you are talking about.

Quote from bln:

I see people complaining all the time that they have problems running their Java or Python stuff on a Raspberry PI with 256 MB ram. Whats the problem then? that they can not fit and run their programs inside 256 MB? for me as a C programmer 256 MB of RAM is a huge amount of resources.

How about L0/L1/L2 cache utilization? If you do not have the ability to control the data types you use down to the exact bit length, you are wasting resources. It is dumb to use a 64 bit size Int then an 8 bit Int would suffice.

Memory bus utilization? Same thing, there is not an infinite capacity between CPU and the memory.

In the same way OOP wastes resources, both on the bus and cache/memory. And not let get started on the effects of automatic GC, thats a real speed block.



That is not true. From personal experience I do know that I/O is faster in C vs Java.
Then downloading a file from the internet using Java's builtin http/url class vs uring libcurl from C, libcurl is much faster than the Java counter part. And this is TCP socket I/O. Why is this?

And the parts that are not I/O bound? usually a small part of the overall program is waiting from external I/O operations.