This system looks a bit like standing on moss...sooner or later you are going to take a fall
If it turns out profitable then who are we to judge?
If it turns out profitable then who are we to judge?
BioComp Profit 8 - Trading the ES
- Thread starter chanelops
- Start date
Maybe that is why we haven't heard from him...a pity
OK, we finally got a signal at today's close, which I just executed by covering the short at 904.75. That resulted in a loss of 38.25 points, or -$1,912.50.
So our status is as follows:
Open Position: S&P mini (ESH9) - LONG@ 904.75
Closed equity to date: +$1,497 per contract.
W/L=10/19 or 53%
19 trades down, 11 to go.
I've got some thoughts on where we are and what's happening, which I'll try to post a little later, when I'll hopefully have some time.
So our status is as follows:
Open Position: S&P mini (ESH9) - LONG@ 904.75
Closed equity to date: +$1,497 per contract.
W/L=10/19 or 53%
19 trades down, 11 to go.
I've got some thoughts on where we are and what's happening, which I'll try to post a little later, when I'll hopefully have some time.
Something is wrong with the system if it didn't give you a long signal sooner.
Also, since there was no signal for about 5 trading days, can't you force it so it would give you at least a comfirmation or something? For me letting the market move against me 70 pts in clearly rally more for days would be unacceptable for a system....
The per trade average is down to 1.6 ES from 5.5 at the height. From a swingtrading system I would expect 4-5 ES at least...
Also, since there was no signal for about 5 trading days, can't you force it so it would give you at least a comfirmation or something? For me letting the market move against me 70 pts in clearly rally more for days would be unacceptable for a system....
The per trade average is down to 1.6 ES from 5.5 at the height. From a swingtrading system I would expect 4-5 ES at least...
I've finally got a little time to spend on this subject, having just finished resolving a very annoying computer problem that was driving me crazy.
First, thanks to everyone for their comments and criticism. It is read and appreciated. Here's some feedback on that: to the poster who criticized my use of a 150-point stop, perhaps you didn't read (or understand) the explanation of the stops. They are volatility based, which means the stop constantly varies, trade to trade. I'm currently using 3 times the 20-day ATR, which works out to 85 points, not 150. Also, to that same poster, the current version of Profit no longer uses neural nets. It did in earlier versions, which is why I mistakenly said it did in my first post of the thread.
In response to a post from Pekelo, the system provides a signal every bar (which is a trading day, in this case), so I do get an output of what it's current thinking is every day. (It's a signal varying from 1 (long) to -1(short), with 0 being out of the market.) I haven't bothered to post those signals unless they indicate a change in position.
Now on to some analysis. First, here is a graph of the equity curve of the system, including all the in-sample (development) period.
The system was developed on August 28, 2008, so you can disregard anything before that date, it's just how the system would have performed in that period.
As you can see, it starts off from late August quite nicely. As volatility in the markets increases, the equity curve rises very nicely. This is not because the system is performing better in terms of its efficiency, but rather because the same level of efficiency now means more $$, due to higher volatility.
This can be seen by looking at this chart, which shows what a "perfect" end of day trader would gain:
As you can see, there is fairly good correspondence between the two curves. The interesting thing is that the leveling off we see in the Profit system is to be expected, as market volatility has decreased markedly lately.
Now let's look at how the system has performed in terms of efficiency, which I defined in an earlier post as percent of perfect. Here is a plot of all of the system's trades since it was created in late August (first trade closed in early September).
The pattern is not very clear. It looks like there may be a trend down, but the 100% perfect trades tend to obscure it. So let's look at a 6-trade sliding window instead:
That plot is easier to read, things are trending down. Let's smooth it further one last time, by going to a 10-trade sliding window.
At this point there's no doubt about it: system performance is steadily declining. As I've discussed earlier several times, this was a highly-tuned system that delivered excellent performance at the outset, but I expected its efficiency to decline over time, and it definitely has. What I didn't know was how long this process would take, and it looks like from this chart that lifetime for best acceptable performance is something on the order of 2 months or so.
Since I'm trading real money here, it doesn't make sense for me to continue onwards with a declining system, so I will probably stop trading this system very soon. That means we won't reach the 30-trade target, but I still think I learned a lot from the experience. (Of course, if someone want to see all 30 trades, just remit funds and I'll continue onward)
The last thing I want to say is that it is important that one not take this in any way as an adverse reflection on the capabilities of BioComp's Profit. In fact, I think just the opposite is the case. I know of no other system that can deliver these levels of performance for a period of 2+ months, before needing retuning. And keep in mind that on a closed-trade basis, we never were in the red during the entire period of time since I started this thread. Profit is just a building block, and what you make of it is up to you. I'm reasonably sure that I could have built a less highly-tuned system that would be delivering good performance for months or years to come, and others have certainly done so. I was kind of pushing the boundary of what was possible here, and it was very interesting to see the results. And, best of all, I made some money in the process!
I'll be happy to try to answer any questions anyone has, or to read further comments.
First, thanks to everyone for their comments and criticism. It is read and appreciated. Here's some feedback on that: to the poster who criticized my use of a 150-point stop, perhaps you didn't read (or understand) the explanation of the stops. They are volatility based, which means the stop constantly varies, trade to trade. I'm currently using 3 times the 20-day ATR, which works out to 85 points, not 150. Also, to that same poster, the current version of Profit no longer uses neural nets. It did in earlier versions, which is why I mistakenly said it did in my first post of the thread.
In response to a post from Pekelo, the system provides a signal every bar (which is a trading day, in this case), so I do get an output of what it's current thinking is every day. (It's a signal varying from 1 (long) to -1(short), with 0 being out of the market.) I haven't bothered to post those signals unless they indicate a change in position.
Now on to some analysis. First, here is a graph of the equity curve of the system, including all the in-sample (development) period.
The system was developed on August 28, 2008, so you can disregard anything before that date, it's just how the system would have performed in that period.
As you can see, it starts off from late August quite nicely. As volatility in the markets increases, the equity curve rises very nicely. This is not because the system is performing better in terms of its efficiency, but rather because the same level of efficiency now means more $$, due to higher volatility.
This can be seen by looking at this chart, which shows what a "perfect" end of day trader would gain:
As you can see, there is fairly good correspondence between the two curves. The interesting thing is that the leveling off we see in the Profit system is to be expected, as market volatility has decreased markedly lately.
Now let's look at how the system has performed in terms of efficiency, which I defined in an earlier post as percent of perfect. Here is a plot of all of the system's trades since it was created in late August (first trade closed in early September).
The pattern is not very clear. It looks like there may be a trend down, but the 100% perfect trades tend to obscure it. So let's look at a 6-trade sliding window instead:
That plot is easier to read, things are trending down. Let's smooth it further one last time, by going to a 10-trade sliding window.
At this point there's no doubt about it: system performance is steadily declining. As I've discussed earlier several times, this was a highly-tuned system that delivered excellent performance at the outset, but I expected its efficiency to decline over time, and it definitely has. What I didn't know was how long this process would take, and it looks like from this chart that lifetime for best acceptable performance is something on the order of 2 months or so.
Since I'm trading real money here, it doesn't make sense for me to continue onwards with a declining system, so I will probably stop trading this system very soon. That means we won't reach the 30-trade target, but I still think I learned a lot from the experience. (Of course, if someone want to see all 30 trades, just remit funds and I'll continue onward)
The last thing I want to say is that it is important that one not take this in any way as an adverse reflection on the capabilities of BioComp's Profit. In fact, I think just the opposite is the case. I know of no other system that can deliver these levels of performance for a period of 2+ months, before needing retuning. And keep in mind that on a closed-trade basis, we never were in the red during the entire period of time since I started this thread. Profit is just a building block, and what you make of it is up to you. I'm reasonably sure that I could have built a less highly-tuned system that would be delivering good performance for months or years to come, and others have certainly done so. I was kind of pushing the boundary of what was possible here, and it was very interesting to see the results. And, best of all, I made some money in the process!
I'll be happy to try to answer any questions anyone has, or to read further comments.
Thanks channelops.
Have you done a walk forward test where you retrain every 3 or 5 days. Scheduling the time when you retrain is under your control. Especially using sample data before lehman meltdown to trade a post lehman world does not seem realistic.
For your next system how often do you plan to retrain?
If you retrain often, how will you know when the basic approach is no longer working? Not just the trained model, but the underlying approach?
Good effort!
Have you done a walk forward test where you retrain every 3 or 5 days. Scheduling the time when you retrain is under your control. Especially using sample data before lehman meltdown to trade a post lehman world does not seem realistic.
For your next system how often do you plan to retrain?
If you retrain often, how will you know when the basic approach is no longer working? Not just the trained model, but the underlying approach?
Good effort!