Search results

  1. I

    Data mining challenge

    Evolutionary algorithms are not suitable for this kind of problem. They rely on a fitness function. In order for an evolutionary algorithm (such as GA) to work productively, the fitness function must be monotonic. This is not the case with searching for exact patterns. A simple example...
  2. I

    Data mining challenge

    Yes, that's what I am talking about. In this example problem I know in advance that there are 5 attributes max. The problem is what to do when it is not known in advance. One approach could be to use this "max number of attributes" as a training parameter. Then run training gradually...
  3. I

    Data mining challenge

    Yes, the search space is huge. But it doesn't mean that there are that many patterns in the data. In fact, the search space is greatly limited simply because only a very small fraction of them all are "interesting". And by "interesting" pattern I mean those that have at least some support, are...
  4. I

    Data mining challenge

    Yes, that is the core of the problem. The NN comes up with similar results (i.e. generalization estimate) no matter if I train it with all 300 variables or with the (48) relevant only: sometimes a little better, sometimes a little worse, but no significant difference. And I tried as many...
  5. I

    Data mining challenge

    Sorry for misunderstanding. You are probably viewing the file with a simple Windows text editor. The new lines in the file use Unix format. I recoded it to be viewable for Windows users (attachment). There are 10 patterns, one pattern per line, five variables per pattern. 48 variables are...
  6. I

    Data mining challenge

    Jack, what you wrote makes absolutely no sense. The result file contains 10 patterns. One pattern per line. Pattern is made up of variable numbers and sign before a number denotes if that variable's value must be true or false (1 or -1 in this case). For example, the first pattern: -175...
  7. I

    Data mining challenge

    The stupidity of the trolls in this thread is beyond belief. It turns out that I did not generate the data myself and I'm looking for a job (or have a job) in Wall Street or whatever the guy with long rants tried to say. Unbelievable :D Well, I was wrong thinking I could find an intelligent...
  8. I

    Data mining challenge

    Hi, can you elaborate on this? Sounds like an interesting idea but I have no idea how this could be applied in practice to solve the problem. Also note that the relationships between the patterns and targets are probabilistic, not deterministic. I tried to find at least one paper on the...
  9. I

    Data mining challenge

    True. True. This is part of the data generation process. The reason is very simple: to simulate data that occurs in reality and to test data mining techniques that are powerful enough to deal with this issue. This is just plain stupidity. Nice conspiracy theory you've got there :D...
  10. I

    Data mining challenge

    Sorry you worked up so much over my response. :D Didn't mean to attack you. I just don't want you to waste time over a problem you don't understand. I will reiterate: the problem is simple. There are 300 input variables and 1 target. All variables are binary. There are patterns hidden in the...
  11. I

    Data mining challenge

    The data is in semicolon separated format. You can easily rename the file to .csv and open it with Excel if that's what you really want. But the data is for machine learning, there's no point in looking at it through Excel. What particular part of the explanation did you find hard to...
  12. I

    Data mining challenge

    The data.
  13. I

    Data mining challenge

    I believe there are a few people here who do this or similar kind of work in their process of model building. Here is the deal: The data is synthetic. Means I have generated it and I know the rules (real model) that generated it. The values are separated by semicolons. There are 3000...
  14. I

    Fighting the HFT algos... daytraders and floor traders vs. HFT traders

    http://www.youtube.com/watch?v=kU0WMf465Tc http://www.cnbc.com/id/15840232?play=1&video=1504494395
  15. I

    From CEO to pizza delivery driver

    I'm completely aware that there are risk hedgers and investors besides the speculators. But before you belittle me, you could provide some facts to support your opinion. For a start, you could provide data on what percentage of total trading volume is made by risk hedgers and by speculators. The...
  16. I

    From CEO to pizza delivery driver

    Let's simplify things a bit. Imagine there are only a few market participants: you and 9 others. All of you bring 10 dollars each and make a bet on something. Some of the participants lose money and some of them win. The total sum of money is still $100 and none of you created anything...
  17. I

    From CEO to pizza delivery driver

    The point is that the money would have been spent anyway, by someone else. Usually a product or service must be provided for money. In the case of trading, no one is receiving anything useful from your actions. You either win or lose money (a zero sum game) and that's about it.
  18. I

    From CEO to pizza delivery driver

    I think he means that nothing useful is produced by running a hedge fund. You may say "oh but what about the money?". Well, it was not earn (in a usual sense), just taken out of the market, money changed hands, that's it. Nothing was manufactured, produced, researched, invented or serviced in...
  19. I

    Acrary is a genius!

    Again, you are talking about the unnecessary at this point details. If you're interested in this stuff, you could start by reading thread "System Development with acrary". The discussion here is about approach to the markets, system building. What you want to know is very dependent on...
  20. I

    Acrary is a genius!

    Jerry, you are asking the wrong questions. W/L ratio is meaningful when you look at a single system, not the system of systems. Furthermore, it's not directly related to consistency. I believe acrary tries to keep yearly return to max drawdown ratio at least 10:1. The most interesting thing...
Back
Top