Quote from dtrader98:
Thanks for the reply Jerry. Couple of points...
The software I was referring to is called stockneuromaster, and they seem to be hawking it all over a site that (If I recall correctly) is one of the ones you referenced earlier, or at least it is on a major site that advocates rapidminer,
http://www.neuralmarkettrends.com/2008/04/28/apple-inc-aapl-neural-net-model/#comment-1972
The neural market trends site has a few nice tutorials. But about the only commentary that refers to trading systems is the proprietary "stockneuromaster" program they are pitching. And as I mentioned, I don't plan to learn on that platform.
There is a forum that has a few discussions regarding the potential for trading systems. From what I gather, no one seems to have much luck developing reliable forecasting models with stocks there, nor to they even mention much about systems terminology(although as you said, it's possible they just don't know enough to do so, miner software or not). The forum I looked over is :
http://sourceforge.net/forum/forum.php?forum_id=390413
I also spoke to someone pretty knowledgeable on RM, who mentioned in a few ways, that predicting via RM is pretty complicated and subject to many errors (such as curve fitting, normalizing input improperly with wide change in scaling regimes during learning/training vs. validating, etc...).
I think you are jumping a bit over my capability on the use of the tool by jumping to designing a system with specific trading goals(breakout, etc.).
I'm just not certain of how to translate those goals via RM, yet, although an example would help.
From my personal limited knowledge with rapidminer, I was expecting to start with a set we could all run to begin with that is not too complex. This way everyone would be able to get a common say xml model to verify I/O conditions and response.
Like say a simple daily equity model (like Qs for instance) nn, say with maybe 1,000 days of data for training and the end result would be to validate the one day prediction over some period and quantify the success over an out of sample set.
This is not so much to validate a system, for me, but to gain a rudimentary knowledge of using rapidminer and ironing out the bugs with some others attempting and looking at potential pitfalls, etc...
For instance, many of the tutorials they show were performed in a former version of YALE and did not port over easily.
Don't want to bog you down here with the rudimentary stuff. Let me know if my idea seems reasonable, otherwise maybe I'll keep working on getting up to speed with the tool while I have time. I would love to see your wheat run, but want to make sure I'm contributing something equal.
Time is very limited today so sorry for the very short reply below
1) Neuromaster...thanks for the info. That makes sense. The folks who made Rapid Miner free (open Source) didn't do it to benefit humanity but to make money. Hence the general consulting service offering on their web site and also hence the neuromaster offering. That's fair and fine, IMO. We all need to make a living.
Keep in mind though that they have identified the market well. Most traders are in a hurry kind of person. So why learn a very complex process (predictive modeling), when you can buy a packaged solution?
2) Specific on RM for the markets. The real issue isn't the software app but the process. Can a NN, or Decision Tree or any of the several dozen data mining paradigms RM incorporates work in the market? If the paradigm is implemented accurately the same method will work with any package. Think paradigm not package. Will a NN work? Yes or no...Decision tree ..Yes or no.
3) "I also spoke to someone pretty knowledgeable on RM, who mentioned in a few ways, that predicting via RM is pretty complicated and subject to many errors (such as curve fitting, normalizing input improperly with wide change in scaling regimes during learning/training vs. validating, etc...)."
Yep...these are issues and skills needed with most packages. The corporate grade stuff does all this automatically but we are talking software that is in the multiple 10s of thousands of dollars retail. For a free package you've got to understand and do this yourself. Lack of understanding is why most people who try predictive modeling with NN fail. There is an IT term GIGO -Garbage In, Garbage Out.
4)"I think you are jumping a bit over my capability on the use of the tool by jumping to designing a system with specific trading goals(breakout, etc.).
I'm just not certain of how to translate those goals via RM, yet, although an example would help."
Sorry about that.....you'll find I'll do that...keep reminding me not to.
"From my personal limited knowledge with rapidminer, I was expecting to start with a set we could all run to begin with that is not too complex. This way everyone would be able to get a common say xml model to verify I/O conditions and response."
An idea here may be to start with stuff that is easy to model, learn the process then get into the markets. RM has some test datasets, also there are public data mining data set out there used to benchmark software. Or we can do a very simple market related test...let me know what people think.
"Like say a simple daily equity model (like Qs for instance) nn, say with maybe 1,000 days of data for training and the end result would be to validate the one day prediction over some period and quantify the success over an out of sample set.
This is not so much to validate a system, for me, but to gain a rudimentary knowledge of using rapidminer and ironing out the bugs with some others attempting and looking at potential pitfalls, etc..."
1000 bars is on the small side. Youâd be better off with 10,000 even having to use hourly. You may not trade hourly that but you can learn with it.
Also perhaps forget the NN part to start as it is the hardest to master. Start with say a Decision Tree or Rule problem. Use performance as a base then get the NN to improve performance.
Or conversely take a know structure of a simple trading system, there are many â¦. MACD crossover with RSI confirmation and ADX for exit, and model a prediction of its success or failure on each trade. There you free yourself from the complexity of data normalization as most TIs are by their nature normalized â¦0 to 100 for the RSI.
Jerry030
