Trading bot construction (IB)

The bot made 0.75 pt. today.
It sold at 12:53:41 and bought at 13:38:43. Initially, it looked like a bad time to sell.
 
A bot has to implement the 'nextValidId' method.
One of the first things that TWS does when something
connects via the API is send a next-valid-order-number,
and when that is received EReader will call 'nextValidId',
supplying the number as an argument. Without that number,
the bot couldn't reliably place an order. After that number
is used, the bot increments it for use with the next order.
It appears that TWS only sends a next-valid-order-number when
a bot initially connects. (I have not tried submitting orders
to TWS in any other way than by incrementing the order id number
as each order is submitted; however, with the 'Sample' app I did
try using a number less than the one supplied by TWS and got some
sort of invalid order message back.)

Early in RandomBotMgr, currently about line 126, the thread sleeps
for one second to allow time for the number to be received.
That should use a timeout instead of arbitrarily waiting for one second.
There is timeout functionality in RandomBot, which is used to wait for
order confirmation. It occurred to me that, in order to implement a timeout
while waiting for next-valid-order-number, I could add a 'timeout' method to
the 'TimeOut' class; the 'timeout' method could be called from either
RandomBot or RandomBotMgr. It wouldn't be a difficult thing to do if a
method could be called with a parameter that is a class -- it would look
something like:
Code:
        boolean timeout(Class c) {

                int tries = MAX_TRIES;
                int ms =  MS_TO_SLEEP;

                while ((tries-- > 0) && (c.eventHappened() == false))
                        Thread.sleep(ms);

                if (tries < 0)
                        return true; // timed-out
                else
                        return false;

        }
Another argument that could be supplied to 'timeout' might be a
maximum-length-of-time-to-wait value.

Ha, ha! The compiler doesn't like that.
Eventually, I stumbled upon the
"Reflection API Trail" -- a long and winding road -- where this appears,
fortunately near the beginning:

"First, a note of caution. Don't use the reflection API when other tools more
natural to the Java programming language would suffice. For example, if you are
in the habit of using function pointers in another language, you might be
tempted to use the Method objects of the reflection API in the same way.
Resist the temptation! Your program will be easier to debug and maintain if you
don't use Method objects. Instead, you should define an interface, and then
implement it in the classes that perform the needed action."

I am in the habit of using function pointers in another language, so I
hope that an "interface" will let me do what is needed. What I hope to do is
something like:
Code:
class TimeOut {

        ...

        boolean timeout(...) {
                ...
        }
}

class RandomBot ... {

        ...

        boolean eventHappened() {

                ...
        }

        boolean timedOut = TimeOut.timeout(this);
}

class RandomBotMgr ... {

        ...

        boolean eventHappened() {

                ...
        }

        boolean timedOut = TimeOut.timeout(this);
}
Is this possible?
I may try it, later. Right now I am a little frustrated with Java.
 
Isn't Java wonderful?

Code:
interface Y {

  public boolean eventHappened();
}

class TimeOut {

  boolean timeout(Y c) {

    if (c.eventHappened()) return true; else return false;
  }
}

class A implements Y {

  public boolean eventHappened() {

    return true;
  }

  A() {

    System.out.printf("A(): my eventHappened is %s\n", this.eventHappened());
  }
}

class B implements Y {

  public boolean eventHappened() {

    return false;
  }

  B() {

    System.out.printf("B(): my eventHappened is %s\n", this.eventHappened());
  }
}

public class X {

  public static void main(String[] args) {

    A a = new A();
    B b = new B();

    TimeOut timer = new TimeOut();

    System.out.printf("a did%s time out\n", timer.timeout(a) ? "" : " not");

    System.out.printf("b did%s time out\n", timer.timeout(b) ? "" : " not");
  }
}

Is this as good as function pointers?
 
A review of the code revealed a bug:
Code:
// Does the time satisy the constraints?
			if (
				(isLaterThan(t, t0) || coincidesWith(t, t1)) &&
				(isEarlierThan(t, t1) || coincidesWith(t, t1))
			   )
				break;
			else
				continue;
That's in RandomBotMgr. There is a 't1' where there should be a 't0'. As is, a trade couldn't occur at the opening second.
 
That bug is in RandomTime.java, not RandomBotMgr.java. It's about line 53.

Today, the bot bought at 13:27:32 and
sold at 14:35:09, Chicago time, for a loss
of 0.5 pt. If it had waited a little longer
before selling ...
When TWS was in the toolbar, order confirmation
took more than 7 seconds; when it was
out of the toolbar confirmation took about
one second. The machine it runs on is on a busy network, which may account for some of the delay.
 
A usability issue has surfaced: if the bot is run every day with

java RandomBot/Main >> log

so that each day's log is appended to what was logged on previous
days, then there is no indication of the day on which events in the log
occurred. A workaround could be to run the bot from a script that
would append some indication of the date to the log before starting the
bot or the script could

java RandomBot/Main > log.today

To fix the problem in the bot, which is what I'm inclined to do, before
line 99 of RandomBotMgr a 'logger.newday' method might be called, which
would put a line like

Tuesday April 1, 2007 at 03:04:05 (Japan Standard Time)

into the log. It needs some more consideration -- maybe every log entry's
timestamp could include the date information.

Why the bot has a 'Logger' when Java has a 'Logger' class is another
Java war story -- it has a lot to do with the use of 'printf' in the bot.
Java's 'Logger' might facilitate internationalization of the bot, which
is another usability issue.
 
Thinking about adding a GUI to make the bot more configurable,
thinking that the GUI might be a Java program that
would run the bot (a separate Java program) when the user
clicked a 'run' button, after doing whatever configuration
was necessary. Keeping the bot and GUI separate helps keep
the bot simple. (I'd like to make the bot even simpler, not
so random-trading specific. Ideally, I'd have 'main'
'BotMgr', 'SomethingBot' and 'BotAssistant' -- 'SomethingBot'
would make trading decisions and might even do risk management,
or risk management might be done by 'BotMgr' or even some other
thread run by 'BotMgr'. 'BotAssistant' might do things like place
orders, subscribe to market data, etc.)

Anyway, these are the things I'd like to be able to configure,
along with their priorities:

Soonest: TWS client ID, port and host.
(These are now configured via the command line.)

Sooner: maxTrades and session hours.
(These now require code modification and rebuild)

Soon: A contract-details "database," to facilitate selecting
a contract to trade; Trading over multiple sessions.

Eventually: Choose a bot to run: random, moving-average-crossover,
range-breakout, etc.

I guess that all of these things could be passed via the
command line when the bot is started. It might make for a lot
of command line arguments though, which could be a problem on some systems.


Today's log, gross 2.75 pts, which is interesting to look at on a 5-minute
chart, along with an ema(4)/ema(8) (ema cross-over isn't always a good idea --
when is it a bad idea?):

08:37:24 connecting to TWS on localhost at port 7496 with id 0
Server Version:31
TWS Time at connection:20070214 07:37:20 GMT-07:00
08:37:28 TWS: (id=-1, code=2104) Market data farm connection is OK:hkfarm
08:37:28 TWS: (id=-1, code=2104) Market data farm connection is OK:usfuture
08:37:28 TWS: (id=-1, code=2104) Market data farm connection is OK:usfarm
08:37:28 TWS: (id=-1, code=2106) HMDS data farm connection is OK:ushmds2a
08:37:29 next valid order id: 169
08:37:29 round-trip interval begins at 08:37:29, ends at 15:15:00
08:37:29 scheduling buy/sell for 11:07:40/13:22:40
09:05:04 TWS: (id=-1, code=2106) HMDS data farm connection is OK:ushmds2a
11:07:43 order 169: BUY 1 ESH7, Placed at TWS
11:07:49 order 169: qty 1, rmng 0, Filled at 1458.000000
11:07:49 order 169: qty 1, rmng 0, Filled at 1458.000000
11:07:49 order 169: slept for 5750 ms. waiting for entry confirmation
11:08:57 TWS: (id=-1, code=2106) HMDS data farm connection is OK:ushmds2a
13:07:26 TWS: (id=-1, code=1100) Connectivity between IB and TWS has been lost.
13:07:34 TWS: (id=-1, code=1102) Connectivity between IB and TWS has been restored - data maintained.
13:13:08 TWS: (id=-1, code=2106) HMDS data farm connection is OK:ushmds2a
13:22:50 order 170: SELL 1 ESH7, Placed at TWS
13:22:51 order 170: qty 1, rmng 0, Filled at 1460.750000
13:22:51 order 170: qty 1, rmng 0, Filled at 1460.750000
13:22:51 order 170: slept for 1450 ms. waiting for exit confirmation
13:22:51 bot ret'd OK, entry and exit succeeded
13:22:51 disconnecting
13:22:52 main joined mgr
13:22:52 exiting
13:22:52 DEBUG: java.net.SocketException: socket closed
13:22:52 connection closed
13:22:52 connection closed


The long order confirmation time occurred when TWS was not in the toolbar;
the shorter one occurred when TWS was in the toolbar -- another theory bites the
dust. The "HMDS data farm connection ..." messages occurred when I was
refreshing a chart on TWS. Also note the IB <-> TWS connectivity loss and connectivity restoration entries.
Also interesting is that the disconnect exception occurred after RandomBotMgr
exited.
 
This may be a way to implement risk management, if there is a range-breakout
bot available:

1. Run a bot
2. If the bot enters a position, run a range-breakout bot, which would place
an order in the opposite direction of the step-1 bot if either side of the
range were hit.

What sort of system would this be if the range-breakout bot were permitted
to trade in the same direction as the step-1 bot? If that were allowed,
could another range-breakout bot be run to manage the position of the
first range-breakout bot? If that were done, how could the position that
the step-1 bot entered be risk-managed?

Before going any further, it may be important to get exception handling on
firm ground. That may involve narrowing the focus of exception handlers.
Does an exception thrown in a thread propagate to the thread that started the exception-throwing
thread, or does the runtime terminate the exception-throwing thread,
or both?
 
This is an interesting log (today):

09:07:23 logger time zone is set to America/Chicago
09:07:23 connecting to TWS on localhost at port 7496 with id 0
Server Version:31
TWS Time at connection:20070215 08:07:15 GMT-07:00
09:07:23 TWS: (id=-1, code=2104) Market data farm connection is OK:usfuture
09:07:23 TWS: (id=-1, code=2104) Market data farm connection is OK:usfarm
09:07:23 TWS: (id=-1, code=2106) HMDS data farm connection is OK:ushmds2a
09:07:24 next valid order id: 171
09:07:24 round-trip interval begins at 09:07:24, ends at 15:15:00
09:07:24 scheduling buy/sell for 10:16:32/10:59:10
09:07:45 TWS: (id=-1, code=2104) Market data farm connection is OK:hkfarm
09:43:16 TWS: (id=-1, code=2103) Market data farm connection is broken:usfuture
09:43:16 TWS: (id=-1, code=2103) Market data farm connection is broken:usfarm
09:43:22 TWS: (id=-1, code=1100) Connectivity between IB and TWS has been lost.
09:43:56 TWS: (id=-1, code=2103) Market data farm connection is broken:hkfarm
09:48:59 TWS: (id=-1, code=2104) Market data farm connection is OK:hkfarm
09:49:02 TWS: (id=-1, code=2104) Market data farm connection is OK:usfuture
09:49:07 TWS: (id=-1, code=2104) Market data farm connection is OK:usfarm
09:49:10 TWS: (id=-1, code=1102) Connectivity between IB and TWS has been restored - data maintained.
09:51:00 TWS: (id=-1, code=2104) Market data farm connection is OK:hkfarm
09:51:03 TWS: (id=-1, code=2104) Market data farm connection is OK:usfuture
09:51:08 TWS: (id=-1, code=2104) Market data farm connection is OK:usfarm
10:16:36 order 171: BUY 1 ESH7, Placed at TWS
10:16:47 ERROR: order 171: no entry confirmation
10:16:48 FATAL: bot ret'd EntryConfirmationTimeout

Two things happened: 1) there was a TWS <-> IB disconnect/reconnect;
2) the entry order wasn't confirmed (ten seconds is allowed). Looking
at TWS' trades, the buy order was filled at 10:16:44.

What happened? A confirmation message may have been in the network layer's
buffers; if the bot had attempted a disconnect before calling logger.fatal(),
would any additional info have appeared in the log? 'main', 'RandomBot'
and 'RandomBotMgr' have exited, but is 'EReader' still running? Maybe, because
the "API" tab still shows on TWS.

Concerning the IB<->TWS disconnect/reconnect: RandomBot bot was asleep
waiting to place the entry order. If RandomBotMgr had done something when the
disconnect message arrived from TWS, it might have interrupted RandomBot.
RandomBotMgr then would have been faced with two possibilities: RandomBot woke
up early sleeping for either entry or exit order placement, unless RandomBot
wasn't sleeping at the time it was interrupted (assuming that RandomBot
returns to RandomBotMgr when it catches an 'interrupted' exception).
If the interrupt occurred while sleeping for entry, there isn't any recovery
work necessary; however, if the interrupt occurred while sleeping for exit,
then there is an open position.

It looks as though ES quotes aren't arriving at TWS from IB -- the ES
charts look like they flat-lined about 09:57, Chicago time. The same
happened with the SPX quotes from CBOE. Refreshing the charts showed
that there was a lot of volume on ES when they flat-lined. Did EReader
choke on all the messages from TWS about that time? Could that be
why the order confirmation didn't arrive?

Complications ...
 
re: Today's disaster

The bot wasn't subscribed to market data.
TWS may have choked on the high volume.
I now wish that after I quit TWS I had restarted
it to see if the ES data started flowing. QM was
updating while the ES data was missing.

A more "normal" bot would be subscribed to market
data, and if it's able to do that it will also be
able to get the info about TSW<->IB disconnects and
whether market data farms are up or down. Doing something
reasonable with that info will be challenging.

The next thing I'll do is make the few lines of code
that do a disconnect a method, so that it can be
called from the normal path and from the path that,
like the one executed today, gets an error return from the
'Callable' thread. Then, I hope to find a java debugger so
that when the bot sleeps I can wake it (by interrupting it)
to see that things work as expected. Without a debugger,
I'll have to plant some code in the source to "simulate" the
problem, rebuild, then run.
 
Back
Top