any IB java api experts? code question

good points in general.

my whole thing on this thread is that threading should never really be needed with IB...

it seems lolatency is also agreeing that IB does not care about going after high frequency, low latency customers... which is probably one reason why they rate limit their feed so it serves the lowest common denominator.

although this thread also proves it's not a bad decision for the mass market, because even most well-intentioned people don't understand threads and probably shouldn't need to.
 
Quote from tradelink:

good points in general.

my whole thing on this thread is that threading should never really be needed with IB...

it seems lolatency is also agreeing that IB does not care about going after high frequency, low latency customers... which is probably one reason why they rate limit their feed so it serves the lowest common denominator.

although this thread also proves it's not a bad decision for the mass market, because even most well-intentioned people don't understand threads and probably shouldn't need to.
More...

They rate limit their feed because many of their customers can't cope with the full speed of the market. Some of their customers are sitting there with 200ms+ latencies and can get upto 1-3s latencies because their connections are so bad. If they (the customers) go into a state where they have to do TCP retransmits, by the time they finish sending ticks to a slow customer, he will be hopelessly behind the market. Thus, IB samples the market and sends their sampled information down to everyone at the same rate. This way, the slow guys get the most up to date stuff at the cost of losing information between each time interval.

Brokers like genesis actually will just start dropping information and overwrite their queues and then transmit what hasn't been sent. They are not reliable for feed data either, as you won't get a tick-for-tick representation of the market.

So it defaults to the lowest common denominator -- some dude on dial-up sitting on a mountain top in Canada who wants to see 100+ symbols and put on trades for 10-20min+.

This is why it's not practical to implement HF strategies using their data feed. It's better to subscribe to a real feed, and then use the IB ticks as a low-speed confirmation of the market picture.

No one doing anything serious is getting market data over TCP. Multicast UDP feeds are the only way to trade the market quickly without penalty.
 
Quote from tradelink:

good points in general.

my whole thing on this thread is that threading should never really be needed with IB...

it seems lolatency is also agreeing that IB does not care about going after high frequency, low latency customers... which is probably one reason why they rate limit their feed so it serves the lowest common denominator.

although this thread also proves it's not a bad decision for the mass market, because even most well-intentioned people don't understand threads and probably shouldn't need to.
More...

LOL. You're making yourself sound like an arrogant sod, tradelink.

"lowest common denominator"

What they've chosen is to offer a no lag feed for those people who don't think they need either:
- every tick
- every tick now.

My guess is that most who fall into those categories aren't the highest uncommon anything. They just believe they can make money with a different trading strategy - and lets be honest, if you're going down that fast path you want the decision making computer as close to the exchange computer as possible. A good business choice on IB's part.
 
kiwi,

my statement about lowest common denominator meant firms/people who don't have a need for high frequency low latency systems.

It was not a judgement, I like IB. You can trade a billion dollars and have very simple technical requirements and thus be well-served by LCD. Google uses google apps internally, and google apps is very LCD as an example.

I just thought it was interesting that lolatency worked at IB and they made design choices along this path.

Didn't intend to upset anyone. apologize if I didn't acheive this goal for you.
 
Quote from tradelink:

kiwi,

my statement about lowest common denominator meant firms/people who don't have a need for high frequency low latency systems.

It was not a judgement, I like IB. You can trade a billion dollars and have very simple technical requirements and thus be well-served by LCD. Google uses google apps internally, and google apps is very LCD as an example.

I just thought it was interesting that lolatency worked at IB and they made design choices along this path.

Didn't intend to upset anyone. apologize if I didn't acheive this goal for you.
More...

I didn't work at IB, I meant I worked at an investment bank (IB) working on issues that were similar to those in this thread.

lol.

I know that they were hiring when I was looking a few years ago, and that they wanted people to take math tests at the time.
 
lol

I guess these are the sorts of misunderstands that happen when a thread goes from a 'java interactive brokers api' to philisophical musings on industry at large...

I am going to bow out of this thread so I don't contribute to more confusions.

remember... don't use threads unless you know you need em! :)

-josh
 
Tradelink.

Yes - there is a new thread created every tick - but the first thing it does is hitting a synchronized method where it will block if there is too much work to process.

The code handles the situation where you have tickPrice occuring at a regular interval - say 5 seconds - and you have your process method some times taking more than 5 seconds.

If your process method on average takes more time than 5 seconds then the code won't work but then you got another problem than the stated one and you will have to not process some of your ticks.

There is some overhead in creating threads but it is very small in Java and surely it is not a problem when we are talking 5 seconds intervals.
 
Final comment on Java and threads. If you have any ambition to write Java code, you should get a handle on threads.

Otherwise you will inevitably end up with a mess. Java is inherently and unavoidably multi-threaded.
 
Quote from lolatency:

This thread isn't about HF trading, but...

In high-frequency trading, it isn't enough to just have a threadpool. You have to understand realtime schedulers and work with inversion and the standard OS scheduling mechanisms, as well as understanding how the scheduler picks processes with similar page-table constructions to schedule next.

The part of this thread that made me cringe was the idea that someone actually considered creating a new thread just to handle an event asynchronously. The issue at hand is not just context switch overhead, but heap allocation and fragmentation. To some degree, slab allocators like the one the linux kernel has will save you some penalty, but you are looking at a baseline cost of 50microseconds on the brk() (on a modern quadcore system @ 2.7GHz) system call from user space + some O(n^2) or worse algorithm penalty. System call overhead is enough to make you lose the opportunity to send out that IOC order and remove liquidity when you have to escape.

I don't know whether the Java VM is intelligent enough to recycle the blocks of memory for the creation of runnable thread objects, ... but in general, you can assume that there is substantial penalty in allocating objects on the heap in response to some sort of tick.

For this guy's case, maybe it was fast enough; however, I would not let thread-per-tick stuff anywhere near my codebase.

When I worked at the IB, I had to fight with people who believed it was ok to use something like std::vector<> in managing packets off the wire. Guess what happens when the market moves fast? The default allocator will kick in and try to double the allocation of memory at the time when you least want to deal with memory allocation.

Wrestling with people over these issues is becoming a flat out nuisance in the industry because people come out of these MSFE programs with 1 or 2 C++ courses under their belt and no background in architecture or OS development whatsoever.
More...

No doubt there is a lot of truth in what you are saying if you are dealing with very high frequency event streams, but it's orders of magnitude more demanding than what most here are doing (or aim to do). Problem is nobody ever defines what they mean by HF trading, quantifies the frequency of incoming events or specifies what they consider an acceptable system response time is.

There is also the general and mistaken impression that "real time" means fast. But it is much more accurate and useful to define "real time" as a system with defined worst case response time.

As for allocating new objects on the heap, I read some Sun stuff somewhere that claimed Java new outperformed C++ new by quite a bit. Can't remember the details. Certainly Java JVM does not call brk () for every new object created.
 
You must log in or register to reply here.
Back
Top