except you can identify the owners by the lamob's they are driving.
http://www.7xr.com/lambo/DSCN0912.html
how to protect trading strategies for a colocated server
- Thread starter trend2009
- Start date
except you can identify the owners by the lamob's they are driving.
http://www.7xr.com/lambo/DSCN0912.html
Quote from MarkBrown:
except you can identify the owners by the lamob's they are driving.
http://www.7xr.com/lambo/DSCN0912.html
Perhaps but their system will remain totally invisible.
no doubt because all they would show you here is the (results) car. there was a mythical time when the methods were revealed on mailing list of years gone by.
then sharks came out and took big profitable bits out of the game without contributing anything - they were takers and haters. it fucXed it all up.
m
Your discussion is quite interesting. I was an IT Data Base manager for a large bank (retired to trade full time) that did trading applications. I was on the security team for trading applications. Encrypted code for execution? Well maybe if your security is crap. Encryption had another role. Our team studied a number of hedge fund security processes. We did this because of the tens of millions of dollars being traded. Here is a general outline of what we chose:
The first level of security is you are not going to play on retail box. You do it on special IBM mainframes even if they are out sourced. Not too many players can afford these.
Then you layer security so fewer and fewer get to the trading application. Using a program called ACF2 we defined every personâs or programs role in the security scheme - especially on out sourced platforms.
The next security features is data versus source code. As a mainframe DBA manager my first level of security was âNo hard coded values in source code.â That means all of the core data values that made the trading source code execute were stored in DB2 relational table structures that were NOT stored on the out sourced computer site. This data had to be accessed remotely using encryption from the executing site during the application set up. No data - no trading.
Next security level was the main application. Which I never was never privy to except as it accessed my DB2 Databases. That was like an octopus. Right down to the dual-redundancy of data. It could be split up to run on multiple boxes if needed.
Next security level was communications. We used IBM CICS with many levels of communication security and transmission protocols. This is where trades from the main application where executed. The exchange transmissions fascinated me but I was shut out from learning too much about them. But those were not my bailiwick. Iâm a data storage expert.
The next security feature was along the same lines. All traded objects, executed trades and journals were stored back in the DB2 Relational Database using encrypted transmissions. Some were cleaned up seconds after execution from IBM CICS. Then these transactions went to internal DB2 formats locked down in encryption. The outsourced applications were cleaned up each day leaving little. There are many more security products and processes involved that I am not mentioning.
The idea is simple. The more an application is split up into secured parts then the less of a chance that some one has of getting their hands on the entire process. The main risk is a top designer will jump ship and sell it all for big bucks. That is why they get big bucks each and every year if it works. If you have $100 million you may be able to buy a GS app. Then you better have an three times that amount to get it running as Iâm describing
I am still asked the question to this day is âWhy canât a bottom level designer steal it?â The answer is they never get to see enough of the whole to steal it. Go ahead and reverse engineer the source of the main application. So, what? It was not worth a dam without the data that ran stored in the DB2 database. Well I got that what next. Whoops got to get the CICS apps that do the actual trades. Believe me it would have taken an army to get through all of those pieces of the puzzle. Reverse engineering does one app and is not that complicated, however something this size is way beyond reverse engineering.
Once I ask the head honcho where the weakest link in our trading was and he said ââ¦well not from the inside⦠The weakest link is that some one picks up the parts of our trading transmissions and equates them back to known quantitative methods. But that would be hard because we fragment the tradesâ¦â
The first level of security is you are not going to play on retail box. You do it on special IBM mainframes even if they are out sourced. Not too many players can afford these.
Then you layer security so fewer and fewer get to the trading application. Using a program called ACF2 we defined every personâs or programs role in the security scheme - especially on out sourced platforms.
The next security features is data versus source code. As a mainframe DBA manager my first level of security was âNo hard coded values in source code.â That means all of the core data values that made the trading source code execute were stored in DB2 relational table structures that were NOT stored on the out sourced computer site. This data had to be accessed remotely using encryption from the executing site during the application set up. No data - no trading.
Next security level was the main application. Which I never was never privy to except as it accessed my DB2 Databases. That was like an octopus. Right down to the dual-redundancy of data. It could be split up to run on multiple boxes if needed.
Next security level was communications. We used IBM CICS with many levels of communication security and transmission protocols. This is where trades from the main application where executed. The exchange transmissions fascinated me but I was shut out from learning too much about them. But those were not my bailiwick. Iâm a data storage expert.
The next security feature was along the same lines. All traded objects, executed trades and journals were stored back in the DB2 Relational Database using encrypted transmissions. Some were cleaned up seconds after execution from IBM CICS. Then these transactions went to internal DB2 formats locked down in encryption. The outsourced applications were cleaned up each day leaving little. There are many more security products and processes involved that I am not mentioning.
The idea is simple. The more an application is split up into secured parts then the less of a chance that some one has of getting their hands on the entire process. The main risk is a top designer will jump ship and sell it all for big bucks. That is why they get big bucks each and every year if it works. If you have $100 million you may be able to buy a GS app. Then you better have an three times that amount to get it running as Iâm describing
I am still asked the question to this day is âWhy canât a bottom level designer steal it?â The answer is they never get to see enough of the whole to steal it. Go ahead and reverse engineer the source of the main application. So, what? It was not worth a dam without the data that ran stored in the DB2 database. Well I got that what next. Whoops got to get the CICS apps that do the actual trades. Believe me it would have taken an army to get through all of those pieces of the puzzle. Reverse engineering does one app and is not that complicated, however something this size is way beyond reverse engineering.
Once I ask the head honcho where the weakest link in our trading was and he said ââ¦well not from the inside⦠The weakest link is that some one picks up the parts of our trading transmissions and equates them back to known quantitative methods. But that would be hard because we fragment the tradesâ¦â
ah main frames in a secure environment - until the cleaning company comes in and sticks one of the reals in the trash and takes it out to the car.
for real i know this has happened, i was in a data center one night when construction remodeling was going on and there were holes all in the security.
i know of a cray computer which was once taken and the people didn't realize it was gone for over a year. duh.
also note its not idiots who dont know what to do with these things that get them. the idiots take leather chairs and paper weights etc.
for real i know this has happened, i was in a data center one night when construction remodeling was going on and there were holes all in the security.
i know of a cray computer which was once taken and the people didn't realize it was gone for over a year. duh.
also note its not idiots who dont know what to do with these things that get them. the idiots take leather chairs and paper weights etc.
excellent posts. thankyou, rabbitone. but in the case of high frequency trading, the latency is around microseconds, separating trading and database on a computer outside the subnet of the colocated serverr does not seem plausible. When timing is not so critical, as Jerry mentioned, it is simple to protect the trading system by just hosting the server in the basement. Please share your opinion.
You asked a good question. Your analogy was correct of the main program server is in the basement, the data in the attic and the transaction server is on the doorstep. It does not seem plausible to us today. However, this method will tomorrow run 95% of the worlds processing under the âcloudâ format. It has been evolving for decades. It will soon run your power grids, traffic control, water supplies, along with most of the worldâs financial transactions. These are the subnet of the colocated servers you mention.
IBM has been testing these âcloudâ structures in big banks for more than 15 years. The cloud model for the trading system servers we used came from California. I think San Mateo.
Remember Iâm the data guy. But what I do remember is the transaction servers did the high speed stuff based on specification from the main application. They did their jobs as part of the whole with in some guideline that the application server defined using the DB2 data. They had the fastest comm channels the bank had available.
These links to transaction servers were either âonâ and âoffâ line depending on what the main application did. I donât know what triggered trading from the main application or how frequent trading took place or at what speed on the transaction servers. What I do know is it ran in spurts with extremely large transmissions volumes the norm 24 hours a day. Heavy communications and data access occurred from the main programs before and after these trading spurts. When transaction servers they were trading for the most part they were not communicating with the application or the database.
I just remembered. The CICS/IBM DB2 mainframe transaction servers we used were eventually split up moved off our z/OS to a number of boxes (UNIX I think). These were all plugged into the DB2 for z/OS and the application via a âcloudâ. Here it gets fuzzy. I believe the UNIX boxes were close to IBMs network hubs near exchanges.
The answer is âyesâ it is simpler to secure a z/OS than a server.
Iâd like to express my sincere thanks for your post, which is outstanding. It contributes well written, concrete, detailed information on real world trading in terms of the topic on this thread.
Hats off to you!
It also shows the exception to the just prior post, which notes that nothing of value is ever shared, on-line anymore.
A few questions if you care to respond:
1) What kind of profitability did this system generate, expressed in any metric you know or care to share? One assumes that with this kind of effort the trading was profitable enough to make this worthwhile long term.
2) You mentioned compensation based on performance, if the system kept working. I donât want to know the numbers but could you give me an idea of the ratio between base pay and incentive bonus that was used to keep everyone working happily?
My personal theory is that this has to have a relationship to the net profitability of the operation in order to keep folks from trying to steal it and or jump ship and try to duplicate it independently.
3) You mention extensive compartmentalization in hardware, software and people. What would have prevented a gathering of yourself and a half dozen peers in a bar on Friday after work from turning into a plot to put some of the pieces toghter either for fun or to get the system? You as DBA working with others who had access to the parts you didnât.
A number of the basic concepts you mention can be adapted to the kind of operations in the range of folks on ET. I suspect none can afford the mainframes you mention, however the concepts of breaking the system and its operation into parts running on multiple separate independent servers at different locations has been discussed along with
other some of the other elements you mentioned.
You also lend some support to my perspective that the best way to steal the system is to get access to the trades being executed and use advanced CI to clone a functional equivalent even if precise algos or parameters are unknown. I have done a portion of this process for a client. It is called a Failure Model. The client had a very complex proprietary system used to trade private money. He wanted to improve performance but of course wouldnât reveal anything about the structure of the system. So I used neural networks and other advanced CI to create a predictive model of the systems performance. When the model predicted that the system would generate a loosing trade then the system would not trade. When there was no prediction of a loss the system traded normally. This eliminated over 20% of the negative trades and boosted net profitability considerably. The next step in this logical process would be a Success Model: predicting winning trades. The two together would essentially duplicate most of the original system (functionally), without ever knowing anything about it except the market, time granularity, market data in, and trades out.
Thanks, again for a high quality post.
Jerry
Hats off to you!
It also shows the exception to the just prior post, which notes that nothing of value is ever shared, on-line anymore.
A few questions if you care to respond:
1) What kind of profitability did this system generate, expressed in any metric you know or care to share? One assumes that with this kind of effort the trading was profitable enough to make this worthwhile long term.
2) You mentioned compensation based on performance, if the system kept working. I donât want to know the numbers but could you give me an idea of the ratio between base pay and incentive bonus that was used to keep everyone working happily?
My personal theory is that this has to have a relationship to the net profitability of the operation in order to keep folks from trying to steal it and or jump ship and try to duplicate it independently.
3) You mention extensive compartmentalization in hardware, software and people. What would have prevented a gathering of yourself and a half dozen peers in a bar on Friday after work from turning into a plot to put some of the pieces toghter either for fun or to get the system? You as DBA working with others who had access to the parts you didnât.
A number of the basic concepts you mention can be adapted to the kind of operations in the range of folks on ET. I suspect none can afford the mainframes you mention, however the concepts of breaking the system and its operation into parts running on multiple separate independent servers at different locations has been discussed along with
other some of the other elements you mentioned.
You also lend some support to my perspective that the best way to steal the system is to get access to the trades being executed and use advanced CI to clone a functional equivalent even if precise algos or parameters are unknown. I have done a portion of this process for a client. It is called a Failure Model. The client had a very complex proprietary system used to trade private money. He wanted to improve performance but of course wouldnât reveal anything about the structure of the system. So I used neural networks and other advanced CI to create a predictive model of the systems performance. When the model predicted that the system would generate a loosing trade then the system would not trade. When there was no prediction of a loss the system traded normally. This eliminated over 20% of the negative trades and boosted net profitability considerably. The next step in this logical process would be a Success Model: predicting winning trades. The two together would essentially duplicate most of the original system (functionally), without ever knowing anything about it except the market, time granularity, market data in, and trades out.
Thanks, again for a high quality post.
Jerry