Super fast trading - all automatic

[nitro]

Quote from prophet:

I believe the network reliability...

Yes, most of these colocation services are multihomed and are using BGP4 routers with three or four peering agreements with other backbone providers. This is not only for redundancy, but the layer three routers are aware of the routes your packets are taking and will dynamically route you the best way. However...

...and low latencies with collocation can match or outperform PTP T1s, at a fraction of the price. You can also achieve multi-site redundancy without adding extra offices and T1s.

As far as your second statement, that is not entirely correct. There is no faster way to get from one connection to another than to do a PTP T1. You are confusing bandwidth with latency. If the colocation service does not have a peering relation with your destination, you will be routed there through God knows what intermediaries, otherwise known as a "hop." A PTP T1 will blow that away. However, if your colocation provider does have a peering relation with whomever is hosting the server of your destination, then the PTP T1 vs colocating would be more equally matched.

On top of that, the distance of your provider from the destination server will affect ping times. If you are colocating in LA and are trying to get to the Chicago MERC servers, you may lose the trade to someone that was on the same peer as the MERC's servers but was located downtown Chicago.

Even if what you are saying is true, it would still not be at a fraction of the price, as it depends on how much bandwidth you are using. You get what you pay for, and Internet connectivity is a near commodity today...

When I’m connected to my remote servers using Microsoft remote desktop (formerly MS terminal sever), it’s every bit as fast as being there. If my local Comcast network dies, I can dial in, and within 30 seconds MS RD has rerouted it’s connection. MS RD is surprisingly fast over 56K dialup… about as fast as VNC is over broadband!

LOL. Believe me, no one is going to implement a system that has $10M margined position on over a cable connection. Terminal Server is a good product (it is really a repackaged version of Citrix Metaframe,) way better than VNC, but this is not what it is meant for.

These serverbeach servers are rock solid, in terms of uptime and quality of the received IB data. I have much more piece of mind knowing my systems are on that kind of network, with a password-protected, IP-limited access, than running something in an office.

OK. If you are happy with it, continue...When I know of a single application on the scale of what I am talking about running on remote servers over a cable connection, I will let you know.

The point was to walk before running, especially when it’s so expensive to experiment. Either prioritize and only trade the high expectancy scalps, or design a system that is able to pay the spread which can then be more mechanical because you don’t have to worry about the order queue. Yes, it is a different and easier problem, but one that might lead to a solution for the harder problem. Otherwise it’s a big and costly risk to plunge into a system that tries to do 1000 trades/day.

There is no middle ground. There are some things you simply cannot do except to do them and see what happens. Chance favors the courageous.

There is no problem with "experimenting" with other technologies first and see if we can save ourselves a few bucks. But read my previous post - It takes on the order of $25K+/Month to get these kinds of systems going. 99% of the people on ET would kill to make 1/2 that a month, let alone invest that amount on an "experiment" that could take a year to pan out.

The people that are doing this already have a shit load of money. They do it because they are going after obscene amounts of money.

nitro

 

[prophet]

Quote from Aaron:

Which version of Windows do you have on your servers? Does that version include MS RD?

Is it possible to get MS RD separately (without XP Pro)?

I have Windows 2003 Server Web Edition which includes MS RD. It came pre-installed on the serverbeach server. I believe XP and Win2K contain the server (formerly known as MS terminal server under Win2K and earlier), but it is not enabled by default.

The client works on any windows OS and is available from microsoft.com by searching for "remote desktop".

 

[Aaron]

Quote from prophet:

I have Windows 2003 Server Web Edition...

Thanks! And that's great to hear that MS RD is faster on the client end than VNC.

 

[prophet]

Quote from nitro:

There is no faster way to get from one connection to another than to do a PTP T1. You are confusing bandwidth with latency. If the colocation service does not have a peering relation with your destination, you will be routed there through God knows what intermediaries. A PTP T1 will blow that away. However, if your colocation provider does have a peering relation with your destination, then the PTP T1 vs colocating would be more equally matched.

What if you have multiple feeds from different locations? Would you not need multiple PTP T1s? By choosing collocation centers close to your data sources, you simultaneously reduce latencies and cover multiple feeds, at substantially lower cost.

My latencies between my server and IB are flat over time (albeit at a high level of 66ms), generally for hours at a time, until a route changes and the latency changes by 3 ms or so. Spikes in latencies are very rare. Dropped packets are rare. This is from San Antonio TX to IB’s servers. I’m looking for a dedicated server with lower pings to IB.

Bandwidth for dedicated servers is priced at $169 per 700 GB / month / server. What does your T1 bandwidth cost?

Even if what you are saying is true, it would still not be at a fraction of the price, as it depends on how much bandwidth you are using. You get what you pay for, and Internet connectivity is a near commodity today...

The cost of colocated servers will always be low due to centralization, economies of scale, and the low cost of PC hardware.

LOL. Believe me, no one is going to implement a system that has $10M position on over a cable connection. Terminal Server is a good product, way better than VNC, but this is not what it is meant for.

My systems are not implemented on, or dependent on an active cable or dialup connection. Virtual Desktop is just used to restart the systems once per day and periodically install IB TWS and MS updates. The data handling, trading and TWS restarts are all fully automatic. A custom server service is used to monitor trades and system status. Reliability has been rock solid. If I were trading $10M, I would have three of these, exchanging heartbeat signals, each server able to alert an operator and hedge if a server, exchange or data feed went down.

There is no problem with "experimenting" with other technologies first and see if we can save ourselves a few bucks. But read my original post - It takes on the order of $25K+ to get these kinds of systems going. 99% of the people on ET would kill to make 1/2 that a month, let alone invest that amount on an "experiment" that could take a year to pan out.

The people that are doing this already have a shit load of money. They do it because they are going after obscene amounts of money.

I agree it’s not easy to make money competing with exchanges and other well capitalized high frequency traders. There are easier system challenges out there, from a data infrastructure and execution point of view, systems that operate on longer time frames, not requiring tons of data, not having to fight the spread and commissions. Starting with an easier challenge is a great way to move into harder challenges.

Even for the hard challenges, there are always ways to simulate and develop stuff on a budget, if only far enough to get the attention of a third party that has the resources to make it work.

 

[nitro]

Quote from prophet:

What if you have multiple feeds from different locations? Would you not need multiple PTP T1s? By choosing collocation centers close to your data sources, you simultaneously reduce latencies and cover multiple feeds, at substantially lower cost.

That is what is typically done. But you are over-stressing getting data. One or two providers typically will do the trick for the data part, and then, more PTP T1 are needed depending on how many exchanges you need to route to. The equally important connection is to the executing server(s) for the intent of the original question posed.

My latencies between my server and IB are flat over time (albeit at a high level of 66ms), generally for hours at a time, until a route changes and the latency changes by 3 ms or so. Spikes in latencies are very rare. Dropped packets are rare. This is from San Antonio TX to IB’s servers. I’m looking for a dedicated server with lower pings to IB.

You keep mentioning IB. IB is a retail oriented broker. They do have institutional accounts, but it is not the choice broker for what people are talking about when they want to do massive volume at high speeds. Like I said, I am not debating that you can get away with doing what you are doing. I am talking about the original thread. If you want to continue talking about IB and cable connections, it would probably be better on another thread.

66 ms is terrible.

Bandwidth for dedicated servers is priced at $169 per 700 GB / month / server. What does your T1 bandwidth cost?

A PTP T1 would cost between $500 and $1500 a month and would have between 8-15 ms of latency to the servers one would routing to.

The cost of colocated servers will always be low due to centralization, economies of scale, and the low cost of PC hardware.

Like I said in my previous post. If this works for you with what you are doing, continue. But do not try what the thread starter is talking about in this manner or you might have an accident you may regret.

My systems are not implemented on, or dependent on an active cable or dialup connection. Virtual Desktop is just used to restart the systems once per day and periodically install IB TWS and MS updates. The data handling, trading and TWS restarts are all fully automatic. A custom server service is used to monitor trades and system status. Reliability has been rock solid. If I were trading $10M, I would have three of these, exchanging heartbeat signals, each server able to alert an operator and hedge if a server, exchange or data feed went down.

OK.

But if you were trading $10M on each position you took, trust me, you would not be talking the way you do.

I agree it’s not easy to make money competing with exchanges and other well capitalized high frequency traders. There are easier system challenges out there, from a data infrastructure and execution point of view, systems that operate on longer time frames, not requiring tons of data, not having to fight the spread and commissions. Starting with an easier challenge is a great way to move into harder challenges.

The question was not should we trade your easy way to make money or a hard way, but "superfast trading systems - automated."

Even for the hard challenges, there are always ways to simulate and develop stuff on a budget, if only far enough to get the attention of a third party that has the resources to make it work.

I have yet to see one of these projects on time and on budget. Attention is gotten one way and one way only - trading statements and testing _forwards_ in realtime using real money and real executions.

nitro

 

[schlepers]

Why is anybody even talking about Window in super fast trading? Windows is trash. These guys don't use Windows
. They are using high tech computers, not trash Windows. They are most likely using Suns or some type of Unix system(not linux either). Why is anybody bringing up Window bullshit? Is that a joke or what. These guys will just crush you and your pc.

 

[nitro]

Quote from schlepers:

Why is anybody even talking about Window in super fast trading? Windows is trash. These guys don't use Windows
. They are using high tech computers, not trash Windows. They are most likely using Suns or some type of Unix system(not linux either). Why is anybody bringing up Window bullshit? Is that a joke or what. These guys will just crush you and your pc.

That is false.

One of the people that I know are using windows on the client end and are doing massive volume at high speeds everyday.

Also you say "they are using high tech computers , not trash windows." Windows is software, and high tech computers is hardware.

However, FWIW, you are on target in that most of the _servers_ at the exchanges or brokers are not running windows, but some form of (LU)nix for many reasons...

nitro

 

[prophet]

Nitro,

Although difficult to trade, it IS possible to prototype, simulate and demonstrate a ultra high frequency system without all the costs you bring up. It should be possible to simulate the order queue (for futures, don’t know about stocks), and calibrate such simulation with a limited number of real executions. It should be possible to execute the high expectancy trades and ignore the rest until you find lower commissions. It should be possible to demonstrate the system to a third party able to trade it. A collocated server can at least provide a source of quality high frequency data. All of these points are relevant to the original thread.

Sure I’ve argued for lower frequency systems, and that’s not exactly on topic. But who cares? I am merely sharing my experience, mainly because my input data is very high frequency and I am a firm believer that executions can be simulated more accurately than is popularly believed.

Regarding cable connections, I already corrected this misconception. My cable connection is for monitoring and software updates only. Everything else is automatic. Interventions are rare. If I was running systems that needed intensive monitoring “every second of the day” I would have a high speed backup network and write safeguards into the code. Furthermore I don’t enter trades by hand. I don’t know where you got that idea. Seems like your first response is to assume I am incompetent. I know you are intelligent, can read my posts and ask questions. So please don’t blow things out of proportion, or distort what I say in order to be condescending

You are right, the (round trip) 66 ms is not great for fast systems. However for systems not sensitive to a 66ms lag, it is no problem. My latency jitter is less than 2 ms and there are hardly any dropped packets. My data integrity is excellent when compared to servers getting 30 ms latencies. Obviously for super fast systems, a PTP line or closer colocation are needed.

 

[SethArb]

are out there for arbitrage between

ECNS , and NYSE or between ECN's and M.M. 's ?

are they from this ATD firm or various firms?

do they look for a teeny minimum profit ?

 

[nitro]

prophet,

Quote from prophet:

Nitro,

Although difficult to trade, it IS possible to prototype, simulate and demonstrate a ultra high frequency system without all the costs you bring up. It should be possible to simulate the order queue (for futures, don’t know about stocks), and calibrate such simulation with a limited number of real executions. It should be possible to execute the high expectancy trades and ignore the rest until you find lower commissions. It should be possible to demonstrate the system to a third party able to trade it.

This is somewhat true, but aren't you assuming quite a bit about how these systems work? It _is_ possible to test many of these things, but don't underestimate the act of actually trying to get filled and the liquidity available at each level when executions are measured in fractions of a second, or even in one or two seconds. The only way to know is to _know_.

A collocated server can at least provide a source of quality high frequency data. All of these points are relevant to the original thread.

You were doing great until you said this. _No_one_, not one person that I know that would be willing to back a system, would allow such a thing to run anywhere but within the confines of their office walls with a human being sitting in front of the machine monitoring it carefully and all supporting systems. They have contigencies for and rules of when to disengage the automated software and take over.

Further, what is the love affair you have with colocation, no less with a colocation that has 60ms of latency? LOL!

Players that are in this game give about as much thought to spending $1500/M on a PTP T1 connection to their broker/exchange as to when to get their next haircut. It simply is not an issue. Further, I have tried repeatedly to make you understand that the colocation solution, for the kind of trading that is being discussed, will almost certainly _wont_cut_it_ for reasons now given about five times. But you keep trying to bring in about how it works with your ten roundtrips a day to IB and I keep saying that yes there it doesn't matter and then you bring it up here yet again

Sure I’ve argued for lower frequency systems, and that’s not exactly on topic. But who cares? I am merely sharing my experience, mainly because my input data is very high frequency and I am a firm believer that executions can be simulated more accurately than is popularly believed.

Well, I value your experience, I even congratulated you on it. But if you had said, "I know that this post does not relate to high frequency systems" instead of trying to "defend" the colocation idea and remotely monitoring a system on a cable connection that takes 1000+ rountrips a day, or even your ten roundtrips, I just do not know what to think.

Regarding cable connections, I already corrected this misconception. My cable connection is for monitoring and software updates only. Everything else is automatic. Interventions are rare.

I understood this the _first_ time. It is you who have not heeded my warning. _ONE_DAY_, you are going to regret this setup. The fault tolerance of any engineered systems is as strong as it's weakest link.

If I was running systems that needed intensive monitoring “every second of the day” I would have a high speed backup network

Now you are making sense, and thinking defensively...

and write safeguards into the code.

Now you are back to not understanding the issues again.

Furthermore I don’t enter trades by hand. I don’t know where you got that idea.

I never got that idea? Where do I say that you enter trades by hand? Read what I am saying carefully - THE FAULT TOLERANCE OF ANY ENGINEERED SYSTEM IS AS STRONG AS IT'S WEAKEST LINK, and in my experience, Murphy's Rule is always so annoyingly trying to rear it's ugly face. What is worse, it is impossible to anticipate what may be the weakest link because it is a dynamic situation.

You may go years without a hitch, and one day, a fat finger event sends the ES or NQ or whatever you are trading spiraling 40 handles in five minutes and God help you that your program has taken _ALL_POSSIBLE_CONTIGENCIES_ at boundary conditions into account, or that this kind of event does not expose a bug in your program, or perhaps expose a bug in your datafeeds programs that cause data to lag, or on and on. But it is not for the reasons that I can think of that your setup scares me, it scares me for the things that may come up that I _cannot_ think of.

Add to any of those unfortunate events a network glitch on your end at the same time, causing your "monitoring" connection to go down, and in 30 seconds, if you have any kind of a position on, could cause you to wipe out your entire account, or more.

Have _you_ ever tested yourself in a situation where you are losing $10,000 every ten seconds. How long will it take you to dialup your internet provider with your hands shaking knowing from your squawk or CNBC that Osama Bin Laden has been captured, the spoos are in a fast market and the 1 minute chart looks like the space shuttle on take off, you are short, it turns out your datafeed turns out to be worthless in this case, and the guy next door to you decided to come home in the middle of the day from work and look at some live porn and essentially killing your cable connection? QUICK there goes another $10,000, and counting...

Seems like your first response is to assume I am incompetent. I know you are intelligent, can read my posts and ask questions. So please don’t blow things out of proportion, or distort what I say in order to be condescending.

I am not trying to be condenscending. It seems that what I say goes "in one ear and out the other," so I try to say things in a different way each time hoping each time that a light will go on.

You are right, the (round trip) 66 ms is not great for fast systems. However for systems not sensitive to a 66ms lag, it is no problem. My latency jitter is less than 2 ms and there are hardly any dropped packets. My data integrity is excellent when compared to servers getting 30 ms latencies. Obviously for super fast systems, a PTP line or closer colocation are needed.

OK.

nitro