Some Things I Learned About Automated Trading Systems From Writing a Poker Bot

[pipscooper]

Quote from onelot:

How are you defining "simulation based programs"? Are you referring to adaptive programs?

In the context of poker, simulation refers to looking at all the possible outcomes and calculating the available odds based on assumptions about what cards the other players might be holding. In the simplest sense you could take your starting hand, the cards on the board (the flop) and calculate your odds of winning based on all the remaining hands, and the cards that are likely to fall on the turn and river. In poker all the possible outcomes are knowable as there are only 52 cards in the deck, which are all known in advance.

So the simulation can be similar to monte carlo analysis by running through a number of random outcomes, or can deal with the actual probabilities by running through all the remaining possible outcomes. This paper describes it much better than I am doing:

http://www.cs.ualberta.ca/~darse/Papers/AAAI99.pdf

 

[pipscooper]

Unfortunately, any rule-based system is inherently rigid,
and even simple changes were difficult to implement and
verify for correctness. A more flexible, computation-based
approach was needed.
In effect, a knowledge-based betting strategy is
equivalent to a static evaluation function. Given the current
state of the game, it attempts to determine the action that
yields the best result. If we use deterministic perfect
information games as a model, the obvious extension is to
add search to the evaluation function. While this is easy to
achieve in a perfect-information game such as chess
(consider all possible moves as deeply as resources
permit), the same approach is not feasible for real
imperfect information games because there are too many
possibilities to consider [Koller and Pfeffer, 1997].
Consider a 10-player game of Texas Hold’em. By the
time the flop cards are seen, some players may have folded.
Let’s assume one player bets, and it is Loki’s turn to act.
The program must choose between folding, calling or
raising. Which one is the best decision?1
After the program’s decision, every other active player
will be faced with a similar choice. In effect, there is a
branching factor of 3 possible actions for each player, and
there may be several such decisions in each betting round.
Further, there are still two betting rounds to come, each of
which may involve several players, and one of many (45 or
44) unknown cards. Computing the complete poker
decision tree in real time is in general, prohibitively
expensive. Since we cannot consider all possible
combinations of hands, future cards, and actions, we
examine only a representative sample from the
possibilities. A larger sample and more informed selection
process will increase the probability that we can draw
meaningful conclusions.

http://www.cs.ualberta.ca/~darse/Papers/AAAI99.pdf

 

[horribilicus]

Pipscooper, is a "trading bot" fundamentally different than a "rule based mechanical trading system"? These have been explored for quite some time and are in use today as fully automated "hands off" trading automata. The one most familiar to retail traders is called "Tradestation", followed closely by "Wealth-Lab with the Interactive Brokers API".

Generalizing a bit, we realize that if we use an automaton to implement our rule based mechanical trading system, that automaton need not be made of software. Instead we can use an automaton made of meat, i.e., a human being. We can give our mechanical trading system to another person and tell her to faithfully implement the rules without exception. Quite a number of brokers offer this service and are called by the generic name "Robo Brokers". The most famous ones are Attain Capital, Trade Center Inc, Angus Jackson, Striker Securities, and Wisdom Financial Inc

 

[gryphes]

Have always assumed it would be a matter of time before BOTs give the online gaming industry a thorn in its side. In our own business, look at the success stories of IB (previously Timber Hill), O'Connor, Susquehanna, CRT, and even Moccatta Metals). Moccatta was the first to make institutional markets in one minute expiry options with their automated market making/risk mngmt system. Bet Anurag Dikshit of PartyGaming already has a small army out there to complement Party's revenues. Good thread.

 

[pipscooper]

My initial premise was that all you need to make a profitable poker bot was to apply a consistent set of rules that had an advantage. I thought that there must be a sort of optimal form of play that if I could describe to my system, would lead to a bot that could beat poki. I soon learned that my original premise was flawed.

Is there an optimal solution to poker? I thought about this question and searched for answers on various boards and on the www in general. The consensus is that there is no optimal solution to poker. There may be an optimal poker solution vs. a certain type of play. For instance if you are playing against a table of loose aggressive players (who never alter their play style), you could apply the optimal solution to loose aggressive play.

But it isn't quite that simple. Rarely do all the players at a table fit into the same player type category. Some might be tight aggressive, some tight passive, some loose passive and some loose aggressive. There are other ways to classify players but you get the idea. After reading through more of the U of Alberta papers I found that a major component of their poker bots is player modeling. This accounts for the reason why poki (and friends) continued to get better against my bots as it played more hands with my poker bot. My bot was rule based and as such did not vary at all from the strict rules established. In other words there was no variability in my bot's play so it became predictable. A predictable opponent is a defeatable opponent.

So I added two components to my poker bot: I changed the rule based system into a simulation based system to take advantage of the most up to date odds on each play. And I added a bit more randomness to the actions taken by my bot. Rather than raising every time my bot had a certain hand (like two pair) my bot would raise x% and check/call 1-x%. Changing the rule based system to a simulation based system improved my results overall by quite a jump. And strangely enough mixing up my play with a bit of randomness also helped improve my results against poki.

 

[pipscooper]

Now as to what can be learned that applies to trading I don't think mixing up one's trading is particularly helpful because rather than trading against a table of 9 others, we are trading against an unknown counterparty. We could be trading against an institution, a black box, a large private speculator or joe daytrader. In other words, there is no way to know who is on the other side of our trade when trading from a screen. Likewise "they" cannot tell when we are making a trade, and are therefore unable to profile us specifically. Now there may be cases where market makers or brokerage firms do profile us specifically but that is beyond the scope of this discussion.

My own conclusion on the subject was that in trading it isn't possible to profile other traders, so I decided not to do it for my poker bot. I do think this would be a very important feature in a poker bot, however my purpose was to learn about automated trading systems, and not to maximize my poker bot's performance.

My opinion is that the simulation concept does hold merit for trading. I will admit that to date, all of my trading systems have been rule based. I have a couple of ideas on how to create a simulation based trading system, but so far have not explored that avenue. My biggest difficulty has been in actually conceptually defining the probabilities. This is one area where I think poker is easier than trading because there are only so many card combinations in poker, but the possible future price outcomes seems to be infinite. I guess getting help in this area was my selfish motive for starting this thread. I would love to hear how you folks have defined the probabilities in trading to create a structure that is suitable for running simulations.

 

[pipscooper]

Pipscooper, is a "trading bot" fundamentally different than a "rule based mechanical trading system"? These have been explored for quite some time and are in use today as fully automated "hands off" trading automata.

I guess a trading bot would be fully automated but a rule based mechanical system need not be. Other than that they could be the same. Good points with regards to generalizing the concept to meat based trading systems.

With regards to teaching your method to your wife to trade in your absence, you might want to read this thread first:

http://www2.oanda.com/cgi-bin/msgboard/ultimatebb.cgi?ubb=get_topic;f=15;t=003366

 

[Banjo]

The sim bot affected the poker game because the other bots reacted to it. That's not going to happen in the mkt, the participant pool is too large. The sim bot was in effect creating a new situation that the others had to react to hence changing their behavior. You will always be reacting to the mkt flow, never creating it. If enough participants react in the same manner then it will create a new mkt flow. That's a lot of money. Some will argue that a particular instrument at a specific time of day will be vulnerable. That's also a lot of money . Rule based bots with rules that morph with the mkt, something that is always true would be my choice.

 

[nzbryant]

Great posts Pips. That simulation systems are better than rules based makes sense - the latter are too simplistic and not outcome focused.

 

[pipscooper]

Quote from Banjo:

The sim bot affected the poker game because the other bots reacted to it. That's not going to happen in the mkt, the participant pool is too large. The sim bot was in effect creating a new situation that the others had to react to hence changing their behavior. You will always be reacting to the mkt flow, never creating it. If enough participants react in the same manner then it will create a new mkt flow. That's a lot of money. Some will argue that a particular instrument at a specific time of day will be vulnerable. That's also a lot of money . Rule based bots with rules that morph with the mkt, something that is always true would be my choice.

Banjo,

Thanks for your insightful observations. Yes in that respect poker is more about adapting to how the others are playing, and them adapting to how you are playing, and trading is more about adapting to the flow of the market. I agree that the market doesn't care nearly as much about what you are doing than the poker players at your table care.