You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.
You should upgrade or use an alternative browser.
Dedicated Line to IB Data Centers
- Thread starter jhukov
- Start date
You are making exactly my point, today's CPUs are so sophisticated for memory layout and access patterns as you put it, that "strict" typing in a language have no impact on the code execution performance, it will be all optimized by the CPU. The "strict" type will have no impact on such performance.
now are we confusing native types with "process" ? the native type allocation would go through a few process and it might take a bit more "RAM" to process but again it has no significant impact on the speed of the code. The type will not be "64 bytes" as you put it, a few objects might be created in between but they will be automatically destroyed. In terms of process, yes, it would take more time, but not significant enough to be meaningful outside of a lab test. Good point about the JIT for Python, maybe someone is already working on one.
You are speaking old school programming, you can't think like this anymore, execution at the CPU level will be very different from what you "think" it will do. It's all optimized at the core. See below my argument.
it doesn't matter in today's CPU because it's something we can't control and something we know will be optimized properly automatically by the CPU. Speaking of cycles in programming like we used to in the good old days is no longer possible, simply because the CPU has a life of its own and will optimize it all at the end, and even the compiler is no longer "writing" for the CPU, but simply "piping" instructions that the CPU will re-order itself. The CPU is in control, not us or the compiler or even the syntax of the programming language.
I don't really understand how this is happening. I just laid out how Python is actually implemented but you are claiming some magic runtime which has no overhead. Go read the source.
Anyways, I didn't realize that I have no control over what the CPU does. Better give up trying to make my code faster and more predictable right about now. \sarcasm
it will have overhead but not significant enough to make a difference in the execution of an app outside of a lab test. Basically, other factors will have more significant impact on the speed of execution, like poor programming structure and poor programming design. In that regard, "strict" typing as volpunter claimed is not a determining factor in the execution speed of Python code in a REAL programming situation.
That's all I am saying, and again I didn't disagree with your arguments, I am just saying the design is more import than the "compiler" or the "runtime". The best compiler will still produce shit if it's fed crappy code. Garbage In, Garbage Out.
So stop blaming Python as a poor choice for developing trading apps,
That's all I am saying, and again I didn't disagree with your arguments, I am just saying the design is more import than the "compiler" or the "runtime". The best compiler will still produce shit if it's fed crappy code. Garbage In, Garbage Out.
So stop blaming Python as a poor choice for developing trading apps,
it doesn't matter in today's CPU because it's something we can't control and something we know will be optimized properly automatically by the CPU.
I see what you are trying to say. Intel processor requires the executing software do certain things before its optimization algorithms can kick in and improve performance... take a look at this: http://www.intel.com/content/www/us...-ia-32-architectures-optimization-manual.html
Intel's open source group (ssg or something) and its compiler group ensure that the latest versions of gcc, icc and visual c++ (large intel compiler group works at Redmond) adhere to this optimization techniques. Tailoring the compiler backends for processor optimizations is a herculean effort. Intel has a team of 300 developers just to do this. Now, if CPU has to be utilized optimally, python inventor (who currently works for Google) should also be leading this… I have not looked at python’s source code and so I can’t comment on it, but if such an effort was to be made, it’s a huge effort as far as I can tell… I am not aware of any such effort other than what Intel has initiated.
Speaking of cycles in programming like we used to in the good old days is no longer possible, simply because the CPU has a life of its own and will optimize it all at the end, and even the compiler is no longer "writing" for the CPU, but simply "piping" instructions that the CPU will re-order itself.
Not true! Not sure how you concluded that cycles are “old school” when CPU optimization techniques are bread and butter of every OS/kernel/driver/performance engineer. Intel CPU is not as smart as you think. You can take control over CPUs piping instruction sets… that’s how you achieve spinlock concurrency. We do this all the time using memory barriers and a bunch of other techniques to prevent CPU from doing whatever it does and have the CPU execute our algorithms (instead of its)… that’s how you get performance in high performance computing applications.
Some of the points you are raising to defend python are not completely correct, but other than that sure, python is a great language and can be used for trading application as long as you are not in the latency game. The medium frequency swing style ATS (futures) that I have developed in C++ can be rewritten in Python as well. My strategies can even deal with a latency as high as 4 secs and I don’t mind my bracket limit orders sit in the exchange when my ATS is down... Its not a question of “can”, but it’s a matter of “do you want to” and how a person decides is very subjective depending on the requirements and specifications.So stop blaming Python as a poor choice for developing trading apps,
Last edited:
What is the point of compiler level discussions if Python (interpreted run-time; sorry if I did not make it clear that I am talking about the interpreted run-time such as CPython before and going forward) still exhibits serious performance lags when it comes to actually running an application? I suggested to loop over 5 million or so tick level data (good performance metric to read from a binary data store, create structs), and optimally fan out the data to several strategies in parallel rather than linearly. We have not even talked performance metrics in regards to mathematical algorithm performance. I would love to see a Python based solution that can run such simple performance test and be limited by bus throughput, only. I have a C# solution that does so and hardly breaks a sweat on my quad core CPU utilization. In trading space as quant trader I understand how people sometimes get overly excited by applying mathematical and statistical tools popular in Physics space but then utterly fail to even produce a simple non-overfitted strategy that generated positive pnl even before transaction costs. Equally here, without perhaps fully understanding all the smallest nuances of micro memory and CPU optimizations (as I am not a CS major nor describe it as my edge), still it makes full logical sense why Python lacks in most categories when it comes to raw performance vs compiled languages. Dynamic typing is without a doubt one of the reasons for performance differences. I think even someone taking a 101 course in CS would understand that. But here are plenty reasons that support the basic points:
http://stackoverflow.com/questions/761426/why-are-dynamically-typed-languages-slow
http://stackoverflow.com/questions/672857/is-python-slower-than-java-c
http://benchmarksgame.alioth.debian...hp?test=all&lang=python3&lang2=java&data=u64q
P.S.: And please, if we already hold such discussion can we please agree we talk about **general** language performance comparisons rather than picking a single task that might be optimized/implemented optimally in one language vs another. I get that multiplying large integers in Python might be faster than doing so in Java, for example. ;-)
http://stackoverflow.com/questions/761426/why-are-dynamically-typed-languages-slow
http://stackoverflow.com/questions/672857/is-python-slower-than-java-c
http://benchmarksgame.alioth.debian...hp?test=all&lang=python3&lang2=java&data=u64q
P.S.: And please, if we already hold such discussion can we please agree we talk about **general** language performance comparisons rather than picking a single task that might be optimized/implemented optimally in one language vs another. I get that multiplying large integers in Python might be faster than doing so in Java, for example. ;-)
I see what you are trying to say. Intel processor requires the executing software do certain things before its optimization algorithms can kick in and improve performance... take a look at this: http://www.intel.com/content/www/us...-ia-32-architectures-optimization-manual.html
Intel's open source group (ssg or something) and its compiler group ensure that the latest versions of gcc, icc and visual c++ (large intel compiler group works at Redmond) adhere to this optimization techniques. Tailoring the compiler backends for processor optimizations is a herculean effort. Intel has a team of 300 developers just to do this. Now, if CPU has to be utilized optimally, python inventor (who currently works for Google) should also be leading this… I have not looked at python’s source code and so I can’t comment on it, but if such an effort was to be made, it’s a huge effort as far as I can tell… I am not aware of any such effort other than what Intel has initiated.
Not true! Not sure how you concluded that cycles are “old school” when CPU optimization techniques are bread and butter of every OS/kernel/driver/performance engineer. Intel CPU is not as smart as you think. You can take control over CPUs piping instruction sets… that’s how you achieve spinlock concurrency. We do this all the time using memory barriers and a bunch of other techniques to prevent CPU from doing whatever it does and have the CPU execute our algorithms (instead of its)… that’s how you get performance in high performance computing applications.
Some of the points you are raising to defend python are not completely correct, but other than that sure, python is a great language and can be used for trading application as long as you are not in the latency game. The medium frequency swing style ATS (futures) that I have developed in C++ can be rewritten in Python as well. My strategies can even deal with a latency as high as 4 secs and I don’t mind my bracket limit orders sit in the exchange when my ATS is down... Its not a question of “can”, but it’s a matter of “do you want to” and how a person decides is very subjective depending on the requirements and specifications.
from you own links,
so again strict typing is not the reason for a slower execution of a code, even though IN THEORY it could be logical, at least at some point in history it might have been real. IN REALITY and in the modern world, and in a REAL programming environment, for those who know what it means, it's how the code is constructed and structured, and of course how good the compiler or the runtime is.
so a poor Microsoftie programmer will still be slower than anything in C# just because he has no fucking about design, and that no matter how optimized the compiler is. There is so much a compiler can do to save a poor code in its execution LOL
As mentioned in the other answers this depends on the run-time system as well as the task at hand. So the standard (C)Python is not necessarily slower than Java or C#. Some of its modules are implemented in C. Thus combining speed of a native implementation with Python's language.
We did a small experiment: We compared the execution time of a Factorial computation in different languages. The test was actually intended to evaluate the performance of arbitrary-precision integers implementations.
The bar chart shows the results. Python is the clear winner. As far as I know Python uses the Karatsuba-algorithm to multiply large integers, which explains the speed.
so again strict typing is not the reason for a slower execution of a code, even though IN THEORY it could be logical, at least at some point in history it might have been real. IN REALITY and in the modern world, and in a REAL programming environment, for those who know what it means, it's how the code is constructed and structured, and of course how good the compiler or the runtime is.
so a poor Microsoftie programmer will still be slower than anything in C# just because he has no fucking about design, and that no matter how optimized the compiler is. There is so much a compiler can do to save a poor code in its execution LOL
Last edited:
Repeat:
P.S.: And please, if we already hold such discussion can we please agree we talk about **general** language performance comparisons rather than picking a single task that might be optimized/implemented optimally in one language vs another. I get that multiplying large integers in Python might be faster than doing so in Java, for example. ;-)
P.S.: And please, if we already hold such discussion can we please agree we talk about **general** language performance comparisons rather than picking a single task that might be optimized/implemented optimally in one language vs another. I get that multiplying large integers in Python might be faster than doing so in Java, for example. ;-)