Automated execution
- Thread starter RogerwOwens
- Start date
puzzled me as well. maybe they somehow calculate time series
they further process and blow up the size - yet that is not what
you call "executable" code. we are currently processing a system
that chooses among different systems and does that via their
equity curves. the executable code is 1MB, the equity curves, in
binary files, take 3GB ...
if you have that run somewhere else, yes, the 3GB are the relevant
factor for the machinery they need to provide ... so maybe it was
just some misunderstanding ...
To explain what I meant by the 1 GB of executable code:
My system is based on a NN/GA data mining application, with the option to export the final model into C++ or other executable format. The application is a general purpose commercial application in contrast to something designed for trading: that is the kind of thin that probably checked your last credit card transaction for fraud or selected you for target marketing by a national retailer.
My approach uses a strategy called by some democratic modeling or parliament of models. Many model are created with different DVs (Dependent Variables) for the same IV (Independent Variable) or goal functions, say percent of change over the next 5 bars. Determination of the results is accomplished by having say a dozen model "vote" on the prediction....much like the US Congress, but without the bribes or sex scandals.
I also take the approach of using a vast number of dependent variables, in contrast to the typical predictive approach of only selecting a limited number to keep things simple. This results in a lot of models and a lot of code for each market. However I'm usually able to meet my performance goal of an 80% Win Loss ratio.
Jerry
--------------------------------------------------------------------------------
Quote from Craig66:
"1 GB of executable code", I've worked on big distributed telecoms projects in C++ where the collective code size got nowhere near that, 1 GB is like operating system size.
--------------------------------------------------------------------------------
puzzled me as well. maybe they somehow calculate time series
they further process and blow up the size - yet that is not what
you call "executable" code. we are currently processing a system
that chooses among different systems and does that via their
equity curves. the executable code is 1MB, the equity curves, in
binary files, take 3GB ...
if you have that run somewhere else, yes, the 3GB are the relevant
factor for the machinery they need to provide ... so maybe it was
just some misunderstanding ...
My system is based on a NN/GA data mining application, with the option to export the final model into C++ or other executable format. The application is a general purpose commercial application in contrast to something designed for trading: that is the kind of thin that probably checked your last credit card transaction for fraud or selected you for target marketing by a national retailer.
My approach uses a strategy called by some democratic modeling or parliament of models. Many model are created with different DVs (Dependent Variables) for the same IV (Independent Variable) or goal functions, say percent of change over the next 5 bars. Determination of the results is accomplished by having say a dozen model "vote" on the prediction....much like the US Congress, but without the bribes or sex scandals.
I also take the approach of using a vast number of dependent variables, in contrast to the typical predictive approach of only selecting a limited number to keep things simple. This results in a lot of models and a lot of code for each market. However I'm usually able to meet my performance goal of an 80% Win Loss ratio.
Jerry
--------------------------------------------------------------------------------
Quote from Craig66:
"1 GB of executable code", I've worked on big distributed telecoms projects in C++ where the collective code size got nowhere near that, 1 GB is like operating system size.
--------------------------------------------------------------------------------
puzzled me as well. maybe they somehow calculate time series
they further process and blow up the size - yet that is not what
you call "executable" code. we are currently processing a system
that chooses among different systems and does that via their
equity curves. the executable code is 1MB, the equity curves, in
binary files, take 3GB ...
if you have that run somewhere else, yes, the 3GB are the relevant
factor for the machinery they need to provide ... so maybe it was
just some misunderstanding ...
You have the concepts of "independent variable" and "dependent
variable" backwards. Your "goal function" is your dependent
variable, not your independent variable.
At that rate you will soon be joining Stevie Cohen and Jim Simons
at the top of the heap! Somehow I doubt that "80% win ratio" is
actual, rather than paper, trading.
And as far as I can tell, you still haven't explained how or why
your system requires 1gb of executable code.
Kevin,
--------------------------------------------------------------------------------
Quote from Jerry030:
created with different DVs (Dependent Variables) for the same IV (Independent Variable) or goal functions, say percent of change over the next 5 bars.
--------------------------------------------------------------------------------
You have the concepts of "independent variable" and "dependent
variable" backwards. Your "goal function" is your dependent
variable, not your independent variable.
*****
Oops, yes correct. I should sync my typing, thinking, and proofing to the same baud rate to avoid errors.
*****
--------------------------------------------------------------------------------
Quote from Jerry030:
However I'm usually able to meet my performance goal of an 80% Win Loss ratio.
--------------------------------------------------------------------------------
At that rate you will soon be joining Stevie Cohen and Jim Simons
at the top of the heap! Somehow I doubt that "80% win ratio" is
actual, rather than paper, trading.
******
Sorry but I don't know who these guys are. Also I trade only futures, where trading characteristics may be different than stocks, I don't know.
No, actual. I discard any system in the development stage with a lower rate. The actual percent in real trading varies but ends up averaging around 80%.
What is your target Win/Loss percent for a trading system?
It sounds like a bit less.
High profitability isnât impossible just extremely difficult. Have you heard the story of The Prediction Company? In brief, a bunch of physics grad students from UC Santa Clara started playing the markets with chaos theory, complex systems theory, NNs and the like in a small living room in Santa Few, NM. In 2005, UBS bought them out for $500,000,000. Their Win/Loss ratio was never made public, but to get half a billion dollars from the likes of UBS, I suspect most of us would find it was rather good.
*******
And as far as I can tell, you still haven't explained how or why
your system requires 1gb of executable code.
******
My approach is different than typical rule or even algo systems. These often have close to a 60/40 Win/Loss ratio with folks hoping to make a profit by "let profits run, cut losses short" or some other strategy. Many of these may depend on a small number of rules agreeing to get a signal. They also may trade every few dozen bars or less.
Now consider an alternative where instead of say 20 rules there are say hundreds, except they are separate models, with both different designs but predicting the same outcome and the same design predicting different outcomes. Then roll that to the next layer where there are variations in say window length: 2 bars into the future, 3 bars, 4 bars .....x bars.
You now have a parliament of models, with a large number having to be in significant agreement instead of 6 rules out of 10 to trigger a signal. There is more to it but this simple explanation should give an idea how it can generate a lot of executable code.
Additionally my app wraps the model output in an execution architecture so at least half the code is I/O housekeeping.
Granted, for a decision tree or rule based system this amount of code would be excessive.
What approach do you use in terms of a methodology: rule based, decision tree, NN, RBM, etc.?
Jerry
--------------------------------------------------------------------------------
Quote from Jerry030:
created with different DVs (Dependent Variables) for the same IV (Independent Variable) or goal functions, say percent of change over the next 5 bars.
--------------------------------------------------------------------------------
You have the concepts of "independent variable" and "dependent
variable" backwards. Your "goal function" is your dependent
variable, not your independent variable.
*****
Oops, yes correct. I should sync my typing, thinking, and proofing to the same baud rate to avoid errors.
*****
--------------------------------------------------------------------------------
Quote from Jerry030:
However I'm usually able to meet my performance goal of an 80% Win Loss ratio.
--------------------------------------------------------------------------------
At that rate you will soon be joining Stevie Cohen and Jim Simons
at the top of the heap! Somehow I doubt that "80% win ratio" is
actual, rather than paper, trading.
******
Sorry but I don't know who these guys are. Also I trade only futures, where trading characteristics may be different than stocks, I don't know.
No, actual. I discard any system in the development stage with a lower rate. The actual percent in real trading varies but ends up averaging around 80%.
What is your target Win/Loss percent for a trading system?
It sounds like a bit less.
High profitability isnât impossible just extremely difficult. Have you heard the story of The Prediction Company? In brief, a bunch of physics grad students from UC Santa Clara started playing the markets with chaos theory, complex systems theory, NNs and the like in a small living room in Santa Few, NM. In 2005, UBS bought them out for $500,000,000. Their Win/Loss ratio was never made public, but to get half a billion dollars from the likes of UBS, I suspect most of us would find it was rather good.
*******
And as far as I can tell, you still haven't explained how or why
your system requires 1gb of executable code.
******
My approach is different than typical rule or even algo systems. These often have close to a 60/40 Win/Loss ratio with folks hoping to make a profit by "let profits run, cut losses short" or some other strategy. Many of these may depend on a small number of rules agreeing to get a signal. They also may trade every few dozen bars or less.
Now consider an alternative where instead of say 20 rules there are say hundreds, except they are separate models, with both different designs but predicting the same outcome and the same design predicting different outcomes. Then roll that to the next layer where there are variations in say window length: 2 bars into the future, 3 bars, 4 bars .....x bars.
You now have a parliament of models, with a large number having to be in significant agreement instead of 6 rules out of 10 to trigger a signal. There is more to it but this simple explanation should give an idea how it can generate a lot of executable code.
Additionally my app wraps the model output in an execution architecture so at least half the code is I/O housekeeping.
Granted, for a decision tree or rule based system this amount of code would be excessive.
What approach do you use in terms of a methodology: rule based, decision tree, NN, RBM, etc.?
Jerry
so basically most of the 1GB are just logs. I was going to say...unless there are tons of graphics there is no way the actual code base could reach 1GB, at least not anything written by 1 person.
1GB is approx 10,000,000 lines of code in java.
------------------
From what i understand you are feeding the same product data input into many modules which contains the business logic, then massage the output and do some kind of recursion. At the end if a large percentage of these modules output true/favorably, you will make the trade?
I guess it depends on what each of these module actually does, is it a calculation based on indictors, patterns, or more likes mips/neural. Because out of box tools such as those on ninjatrader can easilly do those except a neural network implementation.
I did a lot of research and finally settled on ninjatrader, it's an amazing tool for automation integrated with backtesting and real time testing, all out of box. Only thing is the code is in c# instead of java
1GB is approx 10,000,000 lines of code in java.
------------------
From what i understand you are feeding the same product data input into many modules which contains the business logic, then massage the output and do some kind of recursion. At the end if a large percentage of these modules output true/favorably, you will make the trade?
I guess it depends on what each of these module actually does, is it a calculation based on indictors, patterns, or more likes mips/neural. Because out of box tools such as those on ninjatrader can easilly do those except a neural network implementation.
I did a lot of research and finally settled on ninjatrader, it's an amazing tool for automation integrated with backtesting and real time testing, all out of box. Only thing is the code is in c# instead of java
-------------------------
so basically most of the 1GB are just logs. I was going to say...unless there are tons of graphics there is no way the actual code base could reach 1GB, at least not anything written by 1 person.
1GB is approx 10,000,000 lines of code in java.
----------------------------
Reply: No not written. The code is automatically generated by the modeling application to convert a predictive model into an executable form. A computer program generates the code. not a human being. The human designs the architecture of the modeling process, the software applications creates the models with the option to save them as code for future execution.
All top of the line predictive applications do this. You want to be able to develop a model and then distribute it to a dispersed group of users if you are a global corporation, for example. When something in the model domain changes you rerun the model which automatically re-generates modified code.
High speed business would grind to a halt if when something changed you had to call up a dozen programmers and wait six months to make human based changes to a complex predictive model system. And then wait more months to debug it, by which time the modeled domain might change again. These types of applications were developed to get the programmers out of the loop and also because they couldn't be written by humans anyway.
The beauty of a neural network is that it can learn at a level of complexity that people canât. Our effective decision making limit is about 7 to 10 variables. A NN can learn using hundreds of variables and find patterns in n-dimensional space.
This has led to new methods of charting, for example, creating âsyntheticâ chart patterns
in say 20 dimensions. Humans are limited to observing 2 dimensions on a computer screen, even when 30 indicators are plotted at once.
so basically most of the 1GB are just logs. I was going to say...unless there are tons of graphics there is no way the actual code base could reach 1GB, at least not anything written by 1 person.
1GB is approx 10,000,000 lines of code in java.
----------------------------
Reply: No not written. The code is automatically generated by the modeling application to convert a predictive model into an executable form. A computer program generates the code. not a human being. The human designs the architecture of the modeling process, the software applications creates the models with the option to save them as code for future execution.
All top of the line predictive applications do this. You want to be able to develop a model and then distribute it to a dispersed group of users if you are a global corporation, for example. When something in the model domain changes you rerun the model which automatically re-generates modified code.
High speed business would grind to a halt if when something changed you had to call up a dozen programmers and wait six months to make human based changes to a complex predictive model system. And then wait more months to debug it, by which time the modeled domain might change again. These types of applications were developed to get the programmers out of the loop and also because they couldn't be written by humans anyway.
The beauty of a neural network is that it can learn at a level of complexity that people canât. Our effective decision making limit is about 7 to 10 variables. A NN can learn using hundreds of variables and find patterns in n-dimensional space.
This has led to new methods of charting, for example, creating âsyntheticâ chart patterns
in say 20 dimensions. Humans are limited to observing 2 dimensions on a computer screen, even when 30 indicators are plotted at once.
Yes, I am familiar with the Predco saga. Your price is off by
nearly two orders of magnitude. Did you just make it up?
For their trading with symetical risk/reward they were never
much more than 50/50, a long way from your 80% wins.
If you don't know who Jim Simons is, it might behoove you
to learn. Renaissance would have been a better example
for you than Predco.
Kevin,
In term of the price it may have been one of the books on the subject, the New York Times or the Internet.
Does the exact price matter? The point being they made a ton of money using advanced computing methods and were worth a good deal of money to a very large bank. That's something few chartists can claim.
I must say, Kevin, you ask a lot of questions for someone who won't answer any....like what is your minimum acceptable Win/Loss ratio for a trading system?
Or what is your approach to trading system development?...mine is predictive modeling using neural networks.
If you want to have a real conversation, that's great, but I have little interest in an online version of a TV quiz show with you as the moderatorâ¦..unless of course you have a really great prize behind curtain number 3.
Jerry
In term of the price it may have been one of the books on the subject, the New York Times or the Internet.
Does the exact price matter? The point being they made a ton of money using advanced computing methods and were worth a good deal of money to a very large bank. That's something few chartists can claim.
I must say, Kevin, you ask a lot of questions for someone who won't answer any....like what is your minimum acceptable Win/Loss ratio for a trading system?
Or what is your approach to trading system development?...mine is predictive modeling using neural networks.
If you want to have a real conversation, that's great, but I have little interest in an online version of a TV quiz show with you as the moderatorâ¦..unless of course you have a really great prize behind curtain number 3.
Jerry