Direct from the servers

[smersh]

Hello All,

New to the site but already with a question to do...
My brokerage firm provides a platform built almost entirely in dotNET assemblies. Looking into registry I found all remote servers and respective ports their platform communicates with. Get used to Wireshark, filtering the ports I was interested in, I was able to dump all messages in the processes of asking and receiving market data. In other words, once logged through the initial handshake, data are not encrypted.
Evidently, initial 3-way handshake is encrypted.
I aim to get data directly from servers, avoiding that platform's dotNET stuff. Problem is to discover how to handshake initially with my credentials. Had a look inside platform code with the aid of JetBrain's dotPeek tool but I'm a bit of a blind in dotNET weaponry albeit well experienced with C++ (besides starting to learn some network programming in Erlang).
Could someone point me some way to follow ? Listen, I'm not asking for someone to do my 'homework'. Just some tips from where to start.

Best regards and thank you all in advance,

smersh

 

[globalarbtrader]

Hello All,

New to the site but already with a question to do...
My brokerage firm provides a platform built almost entirely in dotNET assemblies. Looking into registry I found all remote servers and respective ports their platform communicates with. Get used to Wireshark, filtering the ports I was interested in, I was able to dump all messages in the processes of asking and receiving market data. In other words, once logged through the initial handshake, data are not encrypted.
Evidently, initial 3-way handshake is encrypted.
I aim to get data directly from servers, avoiding that platform's dotNET stuff. Problem is to discover how to handshake initially with my credentials. Had a look inside platform code with the aid of JetBrain's dotPeek tool but I'm a bit of a blind in dotNET weaponry albeit well experienced with C++ (besides starting to learn some network programming in Erlang).
Could someone point me some way to follow ? Listen, I'm not asking for someone to do my 'homework'. Just some tips from where to start.

Best regards and thank you all in advance,

smersh

And the winner of the most technical first post on ET goes to...

Seriously, no idea. Hope someone else can help though. I wonder if stackexchange might help if not.

GAT

 

[smersh]

And the winner of the most technical first post on ET goes to...

Seriously, no idea. Hope someone else can help though. I wonder if stackexchange might help if not.

GAT

Had the same thought but posted here at first once I was very doubtful about the putative intersection between SE and realtime trading. There's a 'fork' of SE in quantitative trading as well. Let' see. Thank you.

 

[stepan7]

Hello All,

New to the site but already with a question to do...
My brokerage firm provides a platform built almost entirely in dotNET assemblies. Looking into registry I found all remote servers and respective ports their platform communicates with. Get used to Wireshark, filtering the ports I was interested in, I was able to dump all messages in the processes of asking and receiving market data. In other words, once logged through the initial handshake, data are not encrypted.
Evidently, initial 3-way handshake is encrypted.
I aim to get data directly from servers, avoiding that platform's dotNET stuff. Problem is to discover how to handshake initially with my credentials. Had a look inside platform code with the aid of JetBrain's dotPeek tool but I'm a bit of a blind in dotNET weaponry albeit well experienced with C++ (besides starting to learn some network programming in Erlang).
Could someone point me some way to follow ? Listen, I'm not asking for someone to do my 'homework'. Just some tips from where to start.

Best regards and thank you all in advance,

smersh

Keep your platform running and siphon whatever data you need.

 

[smersh]

Keep your platform running and siphon whatever data you need.

Yes, stepan7, you made a point here. Problem is that TCP protocol streaming numerates each message in order to keep a liason between asking and respective response. So, we enter in a vicious circularity here: it would be a dependence to the platform code to keep that numbering. Leave alone I would depend on perfomance of dotNET stuff which is very low compared to C++ and Erlang.
Infact, I only need to take control of the precise platform's piece of code which initiates the authentication (my credentials) handshake and write it into my own coding.
Thank you for your reply.

 

[stepan7]

Yes, stepan7, you made a point here. Problem is that TCP protocol streaming numerates each message in order to keep a liason between asking and respective response. So, we enter in a vicious circularity here: it would be a dependence to the platform code to keep that numbering. Leave alone I would depend on perfomance of dotNET stuff which is very low compared to C++ and Erlang.
Infact, I only need to take control of the precise platform's piece of code which initiates the authentication (my credentials) handshake and write it into my own coding.
Thank you for your reply.

Your opening post implies developing thing called "impersonation", which technically is illegal in US. Unless broker does provide API, of course.

Would you like to self impose travel restrictions outside Königsberg borders like Nietzsche did?

Just a joke.

 

[Lee-]

I'm not sure I understand where your issue is. Is it that you don't know how the authentication is performed or you don't know how to write the network code to actually do the authentication?

If it's the former, is it because you don't know how to get the plaintext of the encrypted connection? It's likely easier than you think. Wireshark itself isn't the answer. I spent some time with a few brokers that didn't provide APIs a couple years ago to develop my own private API. Was a lot easier than I expected (and their messaging formats were horribly inefficient).

In any case, if you can catch me on discord I can explain it more: https://discord.gg/XJyezR

It's not some super secret info. I'm sure you could find by googling, but the way I did it was pretty simple. Granted this only gives you the plaintext. You still need to reverse engineer it.

EDIT:
I decided I don't care if brokers find out how easy it is. Just quickly going to run through the general steps. This will only work if your broker's software is using HTTPS, uses host names rather than direct connections to IPs, and utilizes the operating systems' certificate store to validate the certificate.

1) Is your broker's software communicating via HTTP/HTTPS? It probably is.
2) Check if your broker's software is looking up IPs via DNS (it almost certainly is). Modify your machine's "hosts" file to point the host name of your broker's server to a VM that you run on your machine or otherwise control.
3) Set up an https proxy to log the data (mitm proxy is easy to use). You will need to create a SSL/TLS key pair for the proxy to use. This will be self signed using the host name(s) that your broker's servers use. Remember, this proxy is intended to pretend to be the broker's server. If your broker's software is sane, it will be using the operating system's underlying certificate chain of trust, which means you need to tell your machine to trust your self-signed certificates -- this isn't a web browser, so you can't just click some "add certificate exception" box to make it happen. You'll need to add it to your OS's certificate store.
4) Run your broker's software. It should do a DNS lookup and find your proxy's IP. It will connect to your proxy's IP, which should have its certificate validated and decrypt the connection. It will then make a new encrypted connection to your broker's real server. Now you have the proxy acting as a MITM that holds (and logs) the plaintext.


If the broker's software uses an alternative method (ie some internal fingerprinting) to validate the remote certificate, then you'll need to come up with a new way to deal with this (hint the key is in memory, you just need to find it), or you can modify the fingerprints in the binary (possibly easier said than done depending on how the binary is packaged, idk I've never reverse engineered dotnet stuff). If the software has hard coded IPs rather than doing a DNS lookup, then you'll need to modify the binary to point to your IP or modify your route table is directing traffic to those IPs to your proxy machine and that the proxy machine is appropriately set up to listen on those IPs.

If I think of it later I might clean this up a bit since I wrote it in a rush. Good luck!

 

[smersh]

I'm not sure I understand where your issue is. Is it that you don't know how the authentication is performed or you don't know how to write the network code to actually do the authentication?

If it's the former, is it because you don't know how to get the plaintext of the encrypted connection? It's likely easier than you think. Wireshark itself isn't the answer. I spent some time with a few brokers that didn't provide APIs a couple years ago to develop my own private API. Was a lot easier than I expected (and their messaging formats were horribly inefficient).

In any case, if you can catch me on discord I can explain it more: https://discord.gg/XJyezR

It's not some super secret info. I'm sure you could find by googling, but the way I did it was pretty simple. Granted this only gives you the plaintext. You still need to reverse engineer it.

Lee, it wouldn't come with no surprise to me if it's easier than I think provided that they (brokerage firm) still use DDE linkage glued with that dotNET stuff. I suspect DDE was used to get Rotschilds informed about Napoleon advances :-D
I'll try to get in touch with you through discord which I didn't know before. Thank you.

 

[smersh]

Keep your platform running and siphon whatever data you need.

Sorry but impersonation per se isn't illegal. It depends on the server policy. And this is not the present case.
And congrats for finding out the former name of my city. :-D
Just joking.

 

[smersh]

I'm not sure I understand where your issue is. Is it that you don't know how the authentication is performed or you don't know how to write the network code to actually do the authentication?

If it's the former, is it because you don't know how to get the plaintext of the encrypted connection? It's likely easier than you think. Wireshark itself isn't the answer. I spent some time with a few brokers that didn't provide APIs a couple years ago to develop my own private API. Was a lot easier than I expected (and their messaging formats were horribly inefficient).

In any case, if you can catch me on discord I can explain it more: https://discord.gg/XJyezR

It's not some super secret info. I'm sure you could find by googling, but the way I did it was pretty simple. Granted this only gives you the plaintext. You still need to reverse engineer it.

EDIT:
I decided I don't care if brokers find out how easy it is. Just quickly going to run through the general steps. This will only work if your broker's software is using HTTPS, uses host names rather than direct connections to IPs, and utilizes the operating systems' certificate store to validate the certificate.

1) Is your broker's software communicating via HTTP/HTTPS? It probably is.
2) Check if your broker's software is looking up IPs via DNS (it almost certainly is). Modify your machine's "hosts" file to point the host name of your broker's server to a VM that you run on your machine or otherwise control.
3) Set up an https proxy to log the data (mitm proxy is easy to use). You will need to create a SSL/TLS key pair for the proxy to use. This will be self signed using the host name(s) that your broker's servers use. Remember, this proxy is intended to pretend to be the broker's server. If your broker's software is sane, it will be using the operating system's underlying certificate chain of trust, which means you need to tell your machine to trust your self-signed certificates -- this isn't a web browser, so you can't just click some "add certificate exception" box to make it happen. You'll need to add it to your OS's certificate store.
4) Run your broker's software. It should do a DNS lookup and find your proxy's IP. It will connect to your proxy's IP, which should have its certificate validated and decrypt the connection. It will then make a new encrypted connection to your broker's real server. Now you have the proxy acting as a MITM that holds (and logs) the plaintext.


If the broker's software uses an alternative method (ie some internal fingerprinting) to validate the remote certificate, then you'll need to come up with a new way to deal with this (hint the key is in memory, you just need to find it), or you can modify the fingerprints in the binary (possibly easier said than done depending on how the binary is packaged, idk I've never reverse engineered dotnet stuff). If the software has hard coded IPs rather than doing a DNS lookup, then you'll need to modify the binary to point to your IP or modify your route table is directing traffic to those IPs to your proxy machine and that the proxy machine is appropriately set up to listen on those IPs.

If I think of it later I might clean this up a bit since I wrote it in a rush. Good luck!

I've just seen your EDIT now. Athough broker isn't communicating through HTTP(S), you gave me a couple of good ideas from where to look for inital authentication...
Those guys are so worried about it that they store all their servers and remote ports(there's a lot of them) in a xml file inside the program dir...