Sjfan
You said when I asked what your background was:
"How very presumptious of you."
***
I was just trying to figure out your level of expertise as you didn't appear to have much understanding of how Data Mining is done in sophisticated implementations in the corporate or government arena. You Profile on Elite Trader is pretty empty so I couldn't get much from that.
***
Then you say:
"They were proof-of-concept forecasting projects, in finance, custom built apps designed to see whether any of this was practical. "
***
Well there is you answer.
The custom built app was apparently not designed to generate output so that the results of the modeling process could be used in a production mode by another application.
Do you really think that tool like IBM Intelligent Miner, PSS, S-Plus, SAS and a bunch more only function so statisticians can model and explore data sets and look at nice graphs and charts but have no further use? Like save the results so they can be used by the rest of the corporation in a daily or minute by minute basis to process new data.
Here is a bit form an S-Plus web site:
"Deployment and Scoring
Web-ready graphical reports
HTMLmodel summary exports
Non-interactive batch execution of all components (Server version)
Model ports support automatically-updating scoring components
Score custom predictive models created using S-PLUS on very large databases
Predictive Model Markup Language (PMML) model import and export
Generate C code for run-time model scoring (Server version)"
I draw your attention to the last line "Generate C Code". They seem to be able to do this even if the application you worked on could not. We can all learn something new every day, I guess.
***
Then you go on to say:
"It's one thing to use NN for fraud detection. Sure. Facial recognition, no problem. It's good for stationary patterns. Non-stationary time series? not a chance. "
Do you say that because you tried and failed or because you know
it to be true? Do yourself a favor before making presumptuous sounding statements, Google the terms "neural network" "time series" "stock market" and "pdf". With the pdf in term you'll get a lot of detailed academic studies on the subject. Some of which will mathematically disprove the Random Walk Theory of market behavior.
If you like I'd be more than happy to educate you on this: give me a way to send you files and I'd be happy to share part of my research library on the subject.
***
Then you say:
"And in anycase, it doesn't generate code, just a bunch of weights. "
***
Then you should call up the major credit card companies and tell that that the software provided to them for credit card fraud detection by companies like Fair Isaacs and other firms is a massive fraud since the NN fraud models they though they were using in their transaction applications don't actually exist, since it's impossible to convert a series of weights into code. You might get a reward for uncovering such a massive and long running fraud!
***
Then you say:
"You want the thing to be 1 Gig? Seriously? You specifically said execution code. "
***
Correct in terms of a trading system. That includes a whole parliament of models. I've given the details a few days ago on this thread. Please reread as I'm not going to retype it.
***
Even if the code is machine generated (via your GA, which I very much doubt), it can't be 1Gig. That's a LOT of op codes.
***
Yes, but Neural Network, I don't know where you got GA from
***
More to the point, if you in fact do have most of these details figured out to any degree, how to use a simple IB API is the last question you would have.
***
Why? What does a predictive model have to do with the best application to use for automated execution?
***
I don't believe you. I think you are a liar.
**
Considering you understanding of how advanced software is used in distributed applications, time series forecasting and how to covert weights into C++, I'm not surprised, but at the same time neither worried nor offended.
Somebody once said something about any sufficiently advanced technology will appear to be magic until the observer can learn enough about it to understand its basis of operation.
Why don't you download the demo version of a high end modeling application, run a NN model and then export it to C++ yourself, and then you can say WOW it does work!
Or in terms of time series predictability, send me a time series data set that you have tried to use to predict future market characteristics and found it impossible, I'll run it and return the model output.
As Bob Dylan once said "if you're not busy born born (learning) youâre busy dying"
Jerry
