Genetic Programming, C project

bln · May 14, 2014

Starting a project to develop a platform to explore the idea to use Genetic Programming to breed trading systems. This is cutting edge shit that is showing very promising results by the research papers I have come by. There are commercial platforms out there and they cost $10,000 and up for a single license.

Looking for C programmers interested in participating in this hobby project. As a participiant you will get full ownership to the software, you may sell it, license it, use it for your own benefit, etc.

* Must love working in C
* Like to do efficient and resource effective programming
* Basic understanding of program design and architecture, monolithic, micro-services, SOA, etc.
* Knowledge of multicore programming toolkits is a pro but not a must, Pthreads, OpenMP, Cilk/Cilk+, UPC, etc.
* Work and do development in a UNIX environment (Linux, OSX, BSD, etc).

If you are interested drop me a message with your email and we discuss the ideas and thoughts more in detail.

goodgoing · May 16, 2014

@bln, before you possibly waste a lot of time and many man hours I hope you are aware that genetic programming was big in the late 1990s and lost its shine due to the unsolved data-mining bias problems. One cannot avoid data-mining bias with genetic programming because each new generation uses the same data as previous generations and this introduces data-snooping bias. Only if you have solved the data-mining/snooping bias issues before you start then the effort may be justified but without wanting to discourage you I think this is impossible to do. If you have a reference to a paper that claims significant returns from genetic programming please share. Also which platforms you referred to?

SunnyIsles · May 16, 2014

I code C

traderslair · May 16, 2014

C forever!

2rosy · May 16, 2014

good luck. these projects never get off the ground. I wouldn't be surprised if 100% of projects posted here never go beyond the post itself.

bln · May 17, 2014

2rosy said:
good luck. these projects never get off the ground. I wouldn't be surprised if 100% of projects posted here never go beyond the post itself.

That true to some degree, time is a constraint, especially if you have a full time job and kids. If you are running the project alone it will take years.

Getting a working prototype up and running is doable in a month or two.

bln · May 17, 2014

goodgoing said:
@bln, before you possibly waste a lot of time and many man hours I hope you are aware that genetic programming was big in the late 1990s and lost its shine due to the unsolved data-mining bias problems. One cannot avoid data-mining bias with genetic programming because each new generation uses the same data as previous generations and this introduces data-snooping bias. Only if you have solved the data-mining/snooping bias issues before you start then the effort may be justified but without wanting to discourage you I think this is impossible to do. If you have a reference to a paper that claims significant returns from genetic programming please share. Also which platforms you referred to?

Over fitness problem may be somewhat solved by testing the strategy on the real live market that the strategy continues to perform according to back tested performance metrics, "Forward testing". This will sort out the strategies that are high ranking due to over fitness/data bias. A lot of the strategies with e.x. 50% monthly return will sorted out and you are left with a ranking of performing strategies returning less but still very good returns.

A strategy's metrics needs to be constantly monitored that they conform to spec, this is true to both discretionary trading and mechanical/rulebased trading.

Sergio77 · May 17, 2014

bln said:
Over fitness problem may be somewhat solved by testing the strategy on the real live market that the strategy continues to perform according to back tested performance metrics, "Forward testing".

This is know as: "My system worked for a few weeks, made some money and then lost it all and more". I saw this interesting article a few days ago. I think it is a good read. You may want to think about White's Reality test. Here is an article.

Craig66 · May 17, 2014

bln said:
Over fitness problem may be somewhat solved by testing the strategy on the real live market that the strategy continues to perform according to back tested performance metrics, "Forward testing". This will sort out the strategies that are high ranking due to over fitness/data bias. A lot of the strategies with e.x. 50% monthly return will sorted out and you are left with a ranking of performing strategies returning less but still very good returns.

A strategy's metrics needs to be constantly monitored that they conform to spec, this is true to both discretionary trading and mechanical/rulebased trading.

Doing this only means that you make your forward testing subject to data-mining bias as well, you're effectively making your test data part of your training data. I've seen people use genetic style ideas to explore specific ideas, but they used a lot of domain knowledge to make it work. This idea that you're just going to randomly shit out trading systems which are going to work is a pipe dream.

bln · May 17, 2014

Sergio77 said:
This is know as: "My system worked for a few weeks, made some money and then lost it all and more". I saw this interesting article a few days ago. I think it is a good read. You may want to think about White's Reality test. Here is an article.

So what these is saying is that adding domain specific knowledge is key that makes a difference between randomly generated garbage and something with a true edge. The more domain specific knowledge you add, the better.