You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.
You should upgrade or use an alternative browser.
Trading bot construction (IB)
- Thread starter doli
- Start date
After some more testing, I found that
a trade at the very start of trading hours,
09:30:00 for example, can't be generated. The fix is:
That is in randTime.java
a trade at the very start of trading hours,
09:30:00 for example, can't be generated. The fix is:
Code:
RandomTime(RandomTime open, RandomTime close) {
calls++;
while (true) {
hours = nextInt(close.hours);
minutes = nextInt(close.minutes);
seconds = nextInt(close.seconds);
// Does the time fall within exchange hours?
if (
(isGreaterThan(this, open) || isEqualTo(this, open)) &&
isLessThan(this, close)
)
break;
else
continue;
}
}re: "After thinking some more about extending Random,
it occurred to me that an extension of Random,
in order to fit with what Random offers, should
offer a nextRandomTime method, but the bot
doesn't need that functionality." Yet it could be done
with such a method: the bot would need to constrain a
value generated by nextRandomTime, assuming that
nextRandomTime() generated a time from 00:00:00
to 23:59:59, inclusive. Constraining the time would
involve throwing away generated values until
one that fell within trading hours was obtained.
The current RandomTime(t0, t1), with the latest
fix, appears to generate a time such that
t0 <= time < t1. It might be easier to write trading hour
constraints if RandomTime(t0, t1) generated a time such that
t0 <= time <= t1. So that a constraint that the bot only
trades the first 15 minutes of the market could be written
09:30:00 to 09:45:00 instead of 09:30:00 to 10:46:01. That
strange use of t1 is due to the direct use of the
closing-time-constraint's hour, minute and second values as
arguments to Random.nextInt(). It looks as though an easier
way to write a closing time constraint would be possible if
RandomTime(t0, t1) simply passed to nextInt() t1's hour, minute
and second values incremented by 1: if it gets a constraint of
09:45:00, it could simply use 10, 46 and 1 as arguments to nextInt().
Is it that easy?
By the way, I have been using Java SE 6. (1.6?)
It's the latest and was recently released.
A lot of Java class/method documentation can be found at
http://java.sun.com/javase/6/docs/api/
it occurred to me that an extension of Random,
in order to fit with what Random offers, should
offer a nextRandomTime method, but the bot
doesn't need that functionality." Yet it could be done
with such a method: the bot would need to constrain a
value generated by nextRandomTime, assuming that
nextRandomTime() generated a time from 00:00:00
to 23:59:59, inclusive. Constraining the time would
involve throwing away generated values until
one that fell within trading hours was obtained.
The current RandomTime(t0, t1), with the latest
fix, appears to generate a time such that
t0 <= time < t1. It might be easier to write trading hour
constraints if RandomTime(t0, t1) generated a time such that
t0 <= time <= t1. So that a constraint that the bot only
trades the first 15 minutes of the market could be written
09:30:00 to 09:45:00 instead of 09:30:00 to 10:46:01. That
strange use of t1 is due to the direct use of the
closing-time-constraint's hour, minute and second values as
arguments to Random.nextInt(). It looks as though an easier
way to write a closing time constraint would be possible if
RandomTime(t0, t1) simply passed to nextInt() t1's hour, minute
and second values incremented by 1: if it gets a constraint of
09:45:00, it could simply use 10, 46 and 1 as arguments to nextInt().
Is it that easy?
By the way, I have been using Java SE 6. (1.6?)
It's the latest and was recently released.
A lot of Java class/method documentation can be found at
http://java.sun.com/javase/6/docs/api/
Did you check out JSystemtrader? The scope seems to be very similiar.
http://sourceforge.net/projects/jsystemtrader/
http://www.elitetrader.com/vb/showthread.php?s=&threadid=77554&perpage=6&pagenumber=3
http://sourceforge.net/projects/jsystemtrader/
http://www.elitetrader.com/vb/showthread.php?s=&threadid=77554&perpage=6&pagenumber=3
chameleontrader: thanks for the link.
I looked through the code and will download and
try it. Some interesting things were in
places like chart, strategy, virtual tws -- a great
idea -- Trader.java and TradingSchedule.java.
It may be that random trading could be plugged
into it as a strategy.
In thinking about main's loop in randTime.java,
particularly how to incorporate that into RandomBot,
I looked into a problem. I may have mentioned
that RandomBot has to start before the time
specified by 'open' in Main.java. The reason is
that if it runs at a later time, 'now' in the 'run'
method occurs after main's 'open', which is
run's 'start'. When that happens, the arguments
to Thread.sleep may be less than zero, which
will cause an exception. The reason is that 'run'
may get an 'entry' time before 'now'. The solution
may be to have 'main' determine 'now' and
provide it to 'RandomBot' via its 'setStart' method,
eliminating the computation of 'now' in 'RandomBot'.
With that done, it appears that 'RandomBot'
would trade at a time relative, positive, to the
time that 'main' finds for 'now'. Another thing
that main's loop might need to do is determine
whether there is enough time betwen 'now' and
'close' to do a trade.
Yet another problem with looping over the
'RandomBot' constructor in 'main' is that,
as is, the bot to tws connecion won't persist.
If that is a problem, then a possible solution
may be to have 'main' set-up the connection.
Another thing: Is it alright if the bot's entry and
exit times are the same? There is now a requirement
that they be different. Would the bot's 'run' method break
if entry and exit were the same time?
Added: It should be possible to get the bot
to trade at a random time in a random interval.
Would that be useful? The random interval
could be found by 'main', which would set
the bot's stop time as well as its start time.
I looked through the code and will download and
try it. Some interesting things were in
places like chart, strategy, virtual tws -- a great
idea -- Trader.java and TradingSchedule.java.
It may be that random trading could be plugged
into it as a strategy.
In thinking about main's loop in randTime.java,
particularly how to incorporate that into RandomBot,
I looked into a problem. I may have mentioned
that RandomBot has to start before the time
specified by 'open' in Main.java. The reason is
that if it runs at a later time, 'now' in the 'run'
method occurs after main's 'open', which is
run's 'start'. When that happens, the arguments
to Thread.sleep may be less than zero, which
will cause an exception. The reason is that 'run'
may get an 'entry' time before 'now'. The solution
may be to have 'main' determine 'now' and
provide it to 'RandomBot' via its 'setStart' method,
eliminating the computation of 'now' in 'RandomBot'.
With that done, it appears that 'RandomBot'
would trade at a time relative, positive, to the
time that 'main' finds for 'now'. Another thing
that main's loop might need to do is determine
whether there is enough time betwen 'now' and
'close' to do a trade.
Yet another problem with looping over the
'RandomBot' constructor in 'main' is that,
as is, the bot to tws connecion won't persist.
If that is a problem, then a possible solution
may be to have 'main' set-up the connection.
Another thing: Is it alright if the bot's entry and
exit times are the same? There is now a requirement
that they be different. Would the bot's 'run' method break
if entry and exit were the same time?
Added: It should be possible to get the bot
to trade at a random time in a random interval.
Would that be useful? The random interval
could be found by 'main', which would set
the bot's stop time as well as its start time.
I've attached a new randTime.java, in randtime.txt.
It appears that trade entry and exit times are computed correctly,
so I think that the bot's random-trading strategy is ready.
The random number generator hasn't been evaluated; however, it is
one that generates a long sequence and doesn't appear to begin at
the same point in its sequence (Is the Java runtime seeding it?).
What has changed:
1. A logger has been added.
2. The entry time is permitted to be the same as the exit time.
3. It is easier to write the 'close' portion of market hours.
4. Time comparisons have been simplified.
5. Some names have been changed, e.g., isLessThan became isEarlierThan.
6. main's loop now trades as much as possible between 'open' and 'close'.
7. Recognized bugs/oversights were corrected. Some may remain.
Notes:
1. It is set up to "trade" the first 15 minutes after it starts.
It can also trade some other session if the 'open' and 'close'
in main are set-up for that session.
2. I'll need to retrofit what is in the attachment into RandomTime.java,
RandomBot.java and Main.java. I'll also need to create a new file, Logger.java.
3. Once the bot is working with tws again, there may be an issue with
main's looping over the RandomBot constructor.
3. After no. 2 & 3, above, some code will be needed to submit an order.
4. The bot will need to know whether its orders get filled.
5. Once the bot can place an order, there will be additional considerations:
The bot is allowed to enter and exit at the same time -- enter,
then immediately exit. Should the bot wait for order status/confirmation?
How long does that take? Is there a worst-case status/confirmation time?
One second, two seconds, five minutes? What can go wrong here? Note:
the bot will use market orders, but these things will still need to be
considered. The point is that requiring some time between the entry and
exit orders won't eliminate the potential problems -- although it could
reduce the number of times they occur -- unless brokers/exchanges guarantee
worst-case trading times.
6. Once the bot can submit orders, it is important to have the machine
sync'd up with broker/exchange time.
7. Another thing that will have to be done is a consideration of what
exceptions can occur. Which can occur? For each: How does the exception
flow through the bot? Can it be prevented from occurring? What must the bot do
if the exception occurs? Is "catch (Exception e)" too broad? Also, the bot may
intentionally generate exceptions; if it does so, does it handle them correctly?
This will require considerable effort; reading through the JSystemtrader code
may provide some insights.
8. There may be a synchronization issue (jumbled output) with the logger,
if its methods can be invoked by more than one thread.
9. The strategy deals in one second increments. Does a trade always occur
at the scheduled second? Could it occur a second -- maybe more -- later?
Could this be a problem? Can we guarantee any sort of worst-case delay?
If not, and if a delay is a problem, how could the risk of this occurring be
reduced?
10. Note that if the 'seconds' part of 'close' (in main's preliminaries) is 'n',
then the 'seconds' part of every exit/entry time will be <= 'n'. A 'close' of
XX:XX:59 may be best. Is this a bug or a feature?
It appears that trade entry and exit times are computed correctly,
so I think that the bot's random-trading strategy is ready.
The random number generator hasn't been evaluated; however, it is
one that generates a long sequence and doesn't appear to begin at
the same point in its sequence (Is the Java runtime seeding it?).
What has changed:
1. A logger has been added.
2. The entry time is permitted to be the same as the exit time.
3. It is easier to write the 'close' portion of market hours.
4. Time comparisons have been simplified.
5. Some names have been changed, e.g., isLessThan became isEarlierThan.
6. main's loop now trades as much as possible between 'open' and 'close'.
7. Recognized bugs/oversights were corrected. Some may remain.
Notes:
1. It is set up to "trade" the first 15 minutes after it starts.
It can also trade some other session if the 'open' and 'close'
in main are set-up for that session.
2. I'll need to retrofit what is in the attachment into RandomTime.java,
RandomBot.java and Main.java. I'll also need to create a new file, Logger.java.
3. Once the bot is working with tws again, there may be an issue with
main's looping over the RandomBot constructor.
3. After no. 2 & 3, above, some code will be needed to submit an order.
4. The bot will need to know whether its orders get filled.
5. Once the bot can place an order, there will be additional considerations:
The bot is allowed to enter and exit at the same time -- enter,
then immediately exit. Should the bot wait for order status/confirmation?
How long does that take? Is there a worst-case status/confirmation time?
One second, two seconds, five minutes? What can go wrong here? Note:
the bot will use market orders, but these things will still need to be
considered. The point is that requiring some time between the entry and
exit orders won't eliminate the potential problems -- although it could
reduce the number of times they occur -- unless brokers/exchanges guarantee
worst-case trading times.
6. Once the bot can submit orders, it is important to have the machine
sync'd up with broker/exchange time.
7. Another thing that will have to be done is a consideration of what
exceptions can occur. Which can occur? For each: How does the exception
flow through the bot? Can it be prevented from occurring? What must the bot do
if the exception occurs? Is "catch (Exception e)" too broad? Also, the bot may
intentionally generate exceptions; if it does so, does it handle them correctly?
This will require considerable effort; reading through the JSystemtrader code
may provide some insights.
8. There may be a synchronization issue (jumbled output) with the logger,
if its methods can be invoked by more than one thread.
9. The strategy deals in one second increments. Does a trade always occur
at the scheduled second? Could it occur a second -- maybe more -- later?
Could this be a problem? Can we guarantee any sort of worst-case delay?
If not, and if a delay is a problem, how could the risk of this occurring be
reduced?
10. Note that if the 'seconds' part of 'close' (in main's preliminaries) is 'n',
then the 'seconds' part of every exit/entry time will be <= 'n'. A 'close' of
XX:XX:59 may be best. Is this a bug or a feature?
re: "Note that if the 'seconds' part of 'close' (in main's preliminaries) is 'n',
then the 'seconds' part of every exit/entry time will be <= 'n'. A 'close' of
XX:XX:59 may be best. Is this a bug or a feature?"
One implication of this may be that if XX:XX:00 is
specified, then the RandomTime(t0,t1) constructor
will only generate times on the minute, i.e., the
'seconds' part of every generated time will always
be zero. Wouldn't, given XX:00:00, the constructor
always generate times on the hour? I think that is
the case and will test it. If this is happening, it probably
isn't what someone would expect -- if someone
wanted to random-trade the 1st half hour, then
they might reasonably ask for a time from
09:30:00 to 10:00:00, which may only generate
10:00:00. I just tried and believe that this is
happening, and while trying found a bug in the commented-out
code, early in 'main', that trades the day session.
It should read:
(It contained 'RandomTime.open' and didn't ask
for a "new" one).
I do see it only trading at 04:00:00. This can be
fixed: in RandomTime(t0,t1),
ask for a 'nextRandomTime', which should
provide a time from 00:00:00 and 23:59:59,
inclusive, repeatedly until it does provide
a time from t0 to t1, inclusive. Shorter periods
may require more trips to 'nextRandomTIme', in
order to find a suitable value, than longer periods require.
As is, it can be
used to trade between [09:30:00 and 10:00:00) by
initializing 'close' with 09:59:59. Is there a better
fix than 'nextRandomTime'?
That's the second problem that using the
'close' values as arguments to 'nextInt' has
created. It was too easy: a no brainer. It may
have been used that way because I was thinking
about running the bot over the full day session,
until XX:XX:59, which would permit every random
second before the close at XX:XX:00 to be generated.
then the 'seconds' part of every exit/entry time will be <= 'n'. A 'close' of
XX:XX:59 may be best. Is this a bug or a feature?"
One implication of this may be that if XX:XX:00 is
specified, then the RandomTime(t0,t1) constructor
will only generate times on the minute, i.e., the
'seconds' part of every generated time will always
be zero. Wouldn't, given XX:00:00, the constructor
always generate times on the hour? I think that is
the case and will test it. If this is happening, it probably
isn't what someone would expect -- if someone
wanted to random-trade the 1st half hour, then
they might reasonably ask for a time from
09:30:00 to 10:00:00, which may only generate
10:00:00. I just tried and believe that this is
happening, and while trying found a bug in the commented-out
code, early in 'main', that trades the day session.
It should read:
Code:
// Trade the day session (MST)
RandomTime open = new RandomTime(3, 30, 0);
RandomTime close = new RandomTime(4, 0, 0);for a "new" one).
I do see it only trading at 04:00:00. This can be
fixed: in RandomTime(t0,t1),
ask for a 'nextRandomTime', which should
provide a time from 00:00:00 and 23:59:59,
inclusive, repeatedly until it does provide
a time from t0 to t1, inclusive. Shorter periods
may require more trips to 'nextRandomTIme', in
order to find a suitable value, than longer periods require.
As is, it can be
used to trade between [09:30:00 and 10:00:00) by
initializing 'close' with 09:59:59. Is there a better
fix than 'nextRandomTime'?
That's the second problem that using the
'close' values as arguments to 'nextInt' has
created. It was too easy: a no brainer. It may
have been used that way because I was thinking
about running the bot over the full day session,
until XX:XX:59, which would permit every random
second before the close at XX:XX:00 to be generated.
I've attached a new randTime.java, in randtime.txt,
which fixes the problem mentioned last.
Here is the log output for a session between 05:30:00 and 06:00:00, started about 05:35:
Notice that the no. of calls to nextRandomTime()
is increasing as the round-trip interval gets
shorter. In the last interval, 9 seconds, there were
25739 calls. Could nextRandomTime be optimized?
In this fix, RandomTime no longer extends Random,
so there is a "static rand = new Random()".
I think that means that there will just be one
Random -- is that so? I am a real Java newb,
and this change is a consequence of the compiler
refusing to let me call "non-static" methods from
a "static" context. This may have started occurring
when a lot of things began to be called from 'main', which is "static".
Would I be better off if 'main' started a thread
that did what 'main' now does? Would a new
thread be a "non-static" context?
which fixes the problem mentioned last.
Here is the log output for a session between 05:30:00 and 06:00:00, started about 05:35:
Code:
05:34:04 Bot.run: round-trip interval begins at 05:34:04, ends at 06:00:00
05:34:04 Bot.run: scheduling entry/exit for 05:34:40/05:42:12
05:34:40 Bot.run: entering market (buying)
05:42:12 Bot.run: exiting market (selling)
05:42:12 DEBUG: main: 108 calls of nextRandomTime()
05:42:12 Bot.run: round-trip interval begins at 05:42:12, ends at 06:00:00
05:42:12 Bot.run: scheduling entry/exit for 05:55:20/05:59:47
05:55:20 Bot.run: entering market (buying)
05:59:47 Bot.run: exiting market (selling)
05:59:47 DEBUG: main: 523 calls of nextRandomTime()
05:59:47 Bot.run: round-trip interval begins at 05:59:47, ends at 06:00:00
05:59:47 Bot.run: scheduling entry/exit for 05:59:48/05:59:57
05:59:48 Bot.run: entering market (selling)
05:59:57 Bot.run: exiting market (buying)
05:59:57 DEBUG: main: 25739 calls of nextRandomTime()Notice that the no. of calls to nextRandomTime()
is increasing as the round-trip interval gets
shorter. In the last interval, 9 seconds, there were
25739 calls. Could nextRandomTime be optimized?
In this fix, RandomTime no longer extends Random,
so there is a "static rand = new Random()".
I think that means that there will just be one
Random -- is that so? I am a real Java newb,
and this change is a consequence of the compiler
refusing to let me call "non-static" methods from
a "static" context. This may have started occurring
when a lot of things began to be called from 'main', which is "static".
Would I be better off if 'main' started a thread
that did what 'main' now does? Would a new
thread be a "non-static" context?
Attachments
I've retrofitted that last randTime.java into a bot and ended up
with additional files: X.java, Logger.java and RandomBotMgr.java.
It traded last night, several times, for 5 minute sessions and made
a little money. I lost track of how many times it traded, so did the bot.
Early on, I had to bail it out by manually trading after a session ended.
Its not ready for prime time. Initially, it had a severe problem: it didn't
wait for order status from TWS. I instrumented the code to get some kind of idea
about how long it might take to get order status: 675 ms. is the worst-case,
so far. A side-effect of instrumenting it was that it began to wait for
order confirmation; however, it doesn't handle the case where confirmation
doesn't arrive before the "timeout" occurs -- I think it looks for confirmation
for as long as one second. Considering what can go wrong might suggest that the
bot shouldn't trade too close to the end of a session, but what is "too close"
to the session's end? The bot trades at times that are relative to the
computed entry time -- how relative those times are depends on whatever
delay may occur, whether in the bot, on the bot <-> TWS link, on the
TWS <-> IB link or on the IB <-> exchange link.
Logger.java contains things like 'timestamp', 'log', 'error', 'fatal', etc.,
and it does "synchronized (Object) { }"; whether it needs to do so should be
considered. Code in X.java can place an order, but just a canned buy or sell
order for an ES contract that expires next month. RandomBot schedules the orders
computed by RandomBotMgr. RandomBotMgr.java is essentially the code that is in
randTime.java's 'main' method; it could be considered to be RandomBot's manager.
RandomBot and RandomBotMgr are able to keep the order IDs straight, but there
are additional considerations, which are concerned with how RandomBot and
RandomBotMgr exchange information.
How the bot flows: 'main' starts a RandomBotMgr thread, which decides whether
the exchange is open and, if so, computes an exit and entry time, which are
then handed off to a RandomBot thread, created by RandomBotMgr. RandomBotMgr
opens and closes the connection between the bot and TWS for each session.
RandomBot calls into EClientSocket to place an order, i.e., to send
a message to TWS; EReader calls into RandomBotMgr when a message from TWS
arrives, i.e., RandomBotMgr extends the "wrapper." I see that RandomBotMgr's
code that handles EReader's calls should be placed into yet another file.
These things aren't ready to be posted yet. It seems that as more gets done,
more things that must be done are discovered.
with additional files: X.java, Logger.java and RandomBotMgr.java.
It traded last night, several times, for 5 minute sessions and made
a little money. I lost track of how many times it traded, so did the bot.
Early on, I had to bail it out by manually trading after a session ended.
Its not ready for prime time. Initially, it had a severe problem: it didn't
wait for order status from TWS. I instrumented the code to get some kind of idea
about how long it might take to get order status: 675 ms. is the worst-case,
so far. A side-effect of instrumenting it was that it began to wait for
order confirmation; however, it doesn't handle the case where confirmation
doesn't arrive before the "timeout" occurs -- I think it looks for confirmation
for as long as one second. Considering what can go wrong might suggest that the
bot shouldn't trade too close to the end of a session, but what is "too close"
to the session's end? The bot trades at times that are relative to the
computed entry time -- how relative those times are depends on whatever
delay may occur, whether in the bot, on the bot <-> TWS link, on the
TWS <-> IB link or on the IB <-> exchange link.
Logger.java contains things like 'timestamp', 'log', 'error', 'fatal', etc.,
and it does "synchronized (Object) { }"; whether it needs to do so should be
considered. Code in X.java can place an order, but just a canned buy or sell
order for an ES contract that expires next month. RandomBot schedules the orders
computed by RandomBotMgr. RandomBotMgr.java is essentially the code that is in
randTime.java's 'main' method; it could be considered to be RandomBot's manager.
RandomBot and RandomBotMgr are able to keep the order IDs straight, but there
are additional considerations, which are concerned with how RandomBot and
RandomBotMgr exchange information.
How the bot flows: 'main' starts a RandomBotMgr thread, which decides whether
the exchange is open and, if so, computes an exit and entry time, which are
then handed off to a RandomBot thread, created by RandomBotMgr. RandomBotMgr
opens and closes the connection between the bot and TWS for each session.
RandomBot calls into EClientSocket to place an order, i.e., to send
a message to TWS; EReader calls into RandomBotMgr when a message from TWS
arrives, i.e., RandomBotMgr extends the "wrapper." I see that RandomBotMgr's
code that handles EReader's calls should be placed into yet another file.
These things aren't ready to be posted yet. It seems that as more gets done,
more things that must be done are discovered.