BioComp Dakota with Swamp Technology

[Kevin Schmit]

Quote from Jerry030:

your final response was that you and you staff lacked the time to load a standard benchmark dataset and run it.

Jerry what is stopping you from just running your benchmark yourself? As an owner, you have the software. Why do you need Carl's cooperation for this project?

So far, with reasonable stops, James' Biocomp model has done very well for me, with a Sharpe ratio over 2.5 and over 65% winning trades.

Why don't you do what James has done and post the signals for your own models? Don't you claim that you have a EURJPY model with returns of 100% per month and PF > 6? Why don't you silence your doubters and post the trades from that model?

 

[jalsck]

Signal for Wed Oct 29, 2008 MOC:
Long

Trading Instructions for Wed Oct 29, 2008
None

Open Position:
Long 1 ESZ8 @ 835.25

Closed Net Profit / Loss: ($2,292.50)

Note: ($amount) indicates a loss.

Hypothetical trading history is attached.

Kind Regards,

James

 

[Jerry030]

Quote from Kevin Schmit:

Jerry what is stopping you from just running your benchmark yourself? As an owner, you have the software. Why do you need Carl's cooperation for this project?

So far, with reasonable stops, James' Biocomp model has done very well for me, with a Sharpe ratio over 2.5 and over 65% winning trades.

Why don't you do what James has done and post the signals for your own models? Don't you claim that you have a EURJPY model with returns of 100% per month and PF > 6? Why don't you silence your doubters and post the trades from that model?

Kevin,

My BioComp product is from some years ago. As I mentioned I stopped using it some time ago when I found that general purpose modeling applications work better on the market. So benchmarking an "ancient" copy of Carl’s product would hardly be fair. Perhaps someone with a current copy would be interested in running some objective benchmarks? I'd be happy to see if the folks with NeuroShell and other products are still willing to participate. We could test the common thing people do to create trading systems: is the market in a trend, % change in close over the next X bars, etc.

You said "Why don't you do what James has done and post the signals for your own models? Don't you claim that you have a EURJPY model with returns of 100% per month and PF > 6? Why don't you silence your doubters and post the trades from that model?"

For several reasons I'll decline:
1) My clients could get miffed as they pay for the trades. Giving away what they pay for is bad form in business.
2) The topic of the forum is BioComp. I don't use BioComp although I did many years ago.
3) Who are my "doubters here"? I missed their posts.

Jerry

 

[Jerry030]

Quote from C_Cook:

I think you misunderstood. That was not the issue, nor my response. My point, as I sent you in the email I posted above, is that there is no such thing as a standard benchmark and I doubt you could come up with one that would give any meaningful comparison. Since Dakota and Profit are software tools, not trading systems, I would like to say politely that it's a non-sequitur. As an example that will be clear to most everyone who trades, go ahead and run a "standard benchmark dataset" in Tradestation. You can't. You have to set up a particular strategy, trading system, stops, settings, equity objectives and metrics, etc as defined by the user.

Now, if the software was a canned trading system, then maybe it would make sense, but even in that case I think coming up with a test suite that runs in a work bench that you would have to create to have equal calculations and metrics, that gives a valid comparison of canned systems, would be very difficult, because you have to prove one system is better than another.

Our products are not canned trading systems, nor automatically create trading systems, so I humbly suggest what you are asking for, while great in intent, is not anymore doable than you could with Metastock, Tradestation or other tools.

Thanks,

Carl

Carl,

Your product sounds more like art than mathematics. Success (or beauty) is only in the eye of the beholder. Are you saying it has no objective metrics at all? I’ve worked with many modeling applications and they all create various performance metrics, perhaps the most common being R2. Are you saying that your products generate none of these? One way simple way vendors and potential uses can rate applications is through established metrics like R2 and training time on standardized data sets. I think you'd have a hard time selling anything to non-trading users without some kind of performance metrics.

In any case how about this:

Dependent variable = 1 if High of next bar > high of current bar, else 0.

Run model.

If you system works anything like other modeling applications the forecast for the dependant variable will be >= 0 and <=1.

Performance Metric: the percent of correct predictions at each value of a selection filter from 0 to 1 in .01 increments. i.e. we only look at the instances where your result for the forecasted DV is . say > .85 (how many are actually = 1 (a new high))

If current BioComp can't do this very simple thing could someone explain how it could possibly be used in a trading system? My old BioComp product could do it.

Jerry030

 

[fundjunkie]

Quote from Jerry030:



In any case how about this:

Dependent variable = 1 if High of next bar > high of current bar, else 0.

Run model.

If you system works anything like other modeling applications the forecast for the dependant variable will be >= 0 and <=1.

Hi Jerry,
I am another Biocomp user and consider myself quite knowledgeable with the Dakota product in particular. I don't know which biocomp product you owned previously but, going by your post above, you appear to be a victim of a misconception regarding the central functional premise of Dakota.

In short, Dakota does not predict or forecast anything, instead it adapts it's parameters based on feedback regarding bot performance across a predefined parameter space. And by that I do not mean that it adapt it's prediction - unless you deem the decision to go long or short a prediction, and I don't think you do.

Dakota is a simple product in terms of accessibility but as your expertize grows can become as complex as you need it to be in order to achieve your performance goals. I'm referring here to swarm behaviour and performance feedback in particular.

In fact, I spend little to no time developing systems in Dakota. I do spend most of my time looking at swarm behaviour and performance evaluation however. I use Dakota in a similar fashion to James, piping signals into Tradersstudio to apply stops, money management overlay etc.

Why do I do that? Because the product is unique (it trades a parameter space), is conceptually robust and yet underdeveloped, ie. at an early stage of development with headroom to grow and improve. Indeed, I fully expect future version of Dakota to be much more capable than even the current version. Carl has had enough useable feedback, added to his own ideas, to keep him busy for a looong time.


Thx
D

 

[JackR]

Jerry:

Neither Profit nor Dakota nor Neuroshell are "systems". They are tool sets. I own Dakota and Neuroshell. I've posted a review of Neuroshell here on ET. What you seem to be missing is that both tool sets need to be applied by the user. Both NS and Dakota come with samples. Some samples are, in the case of NS, true systems. In Dakota, they are individual tools that are meant to be used as part of a system. NS is much easier for me to use because it is essentially scripted and I can build a model by clicking on a formula and then modifying it. Add an SMA by clicking on it. To build a system in Dakota you generally code in Visual Basic. Obviously, how well either NS or Dakota works would be almost entirely dependent on my skill in creating the model. I can put together a simple system fairly rapidly in NS, I cannot in Dakota. My tool of choice for speed is NS because I am familiar with it and my typing skills are lousy. Dakota models , being language driven developments, require typing accuracy and the learning of a "language". How well I know the language determines how/if I can translate my concept into code. What you are looking for could be done but as Carl says, it certainly would be suspect as to being of any value.

This is James' thread. If you'll read back through it you'll see what he is reporting on is a complex system he wrote using the Dakota "adaptation" concept to make it go. If you want to continue I think you should create another thread as we are detracting from this one's purpose - reporting on James' system performance and development.

Jack

 

[fundjunkie]

Quote from JackR:



This is James' thread.

Jack,
If Jerry is guilty of hijacking a thread so is James as he didn't start this one (it is not "his"). Noone's suggesting James start his own thead so I think it's perfectly reasonable for jerry to raise his concerns here.


Thx
D

 

[JackR]

Junkie:

You're right and technically correct. I suggested that Jerry create a new thread if he wanted to discuss the tool set, as opposed to a specific application developed using it.

Jack

 

[Jerry030]

Quote from fundjunkie:

Hi Jerry,
I am another Biocomp user and consider myself quite knowledgeable with the Dakota product in particular. I don't know which biocomp product you owned previously but, going by your post above, you appear to be a victim of a misconception regarding the central functional premise of Dakota.

In short, Dakota does not predict or forecast anything, instead it adapts it's parameters based on feedback regarding bot performance across a predefined parameter space. And by that I do not mean that it adapt it's prediction - unless you deem the decision to go long or short a prediction, and I don't think you do.

Dakota is a simple product in terms of accessibility but as your expertize grows can become as complex as you need it to be in order to achieve your performance goals. I'm referring here to swarm behaviour and performance feedback in particular.

In fact, I spend little to no time developing systems in Dakota. I do spend most of my time looking at swarm behaviour and performance evaluation however. I use Dakota in a similar fashion to James, piping signals into Tradersstudio to apply stops, money management overlay etc.

Why do I do that? Because the product is unique (it trades a parameter space), is conceptually robust and yet underdeveloped, ie. at an early stage of development with headroom to grow and improve. Indeed, I fully expect future version of Dakota to be much more capable than even the current version. Carl has had enough useable feedback, added to his own ideas, to keep him busy for a looong time.


Thx
D

D,

The BioComp product I own is a very old Neural Network product, logically similar to dozens of others. It trains a NN to make predictions of the Dependant varaible. If BioComp no longer markets such products than as you point out a comparative benchmark is impossible.

Jerry

 

[Jerry030]

Quote from fundjunkie:

Jack,
If Jerry is guilty of hijacking a thread so is James as he didn't start this one (it is not "his"). Noone's suggesting James start his own thead so I think it's perfectly reasonable for jerry to raise his concerns here.


Thx
D

Well perhaps someone should post a notice every week or two that "Only the following opinions and topics are permitted here and if you don't like to go away;

1) Topic 1
2) Opinion 1
...
..."

Personally I find that when I try to close my mind to differing opinions, it also closes to a lot of other things, some of which limit my ability to discover anything new.

Jerry