Data mining challenge

[psytrade]

The way to solve this problem is using the Luby Transform code. Raptor codes are linear time codes which would solve this problem.

The reason is because an Exclusive OR solves the bipartite graph problem that your data presents.

http://en.wikipedia.org/wiki/Luby_transform_code

http://en.wikipedia.org/wiki/Bipartite_graph

You don't need a neural net because you have binary data signals.

The simplest algorithm would determine the original distribution by scanning through the rows and return a value until the job is complete.

Think of each row as a separate channel, and the original signal generator would be determined once all 3000 columns are decoded.

 

[Indrionas]

Quote from psytrade:

The way to solve this problem is using the Luby Transform code. Raptor codes are linear time codes which would solve this problem.

The reason is because an Exclusive OR solves the bipartite graph problem that your data presents.

http://en.wikipedia.org/wiki/Luby_transform_code

http://en.wikipedia.org/wiki/Bipartite_graph

You don't need a neural net because you have binary data signals.

The simplest algorithm would determine the original distribution by scanning through the rows and return a value until the job is complete.

Think of each row as a separate channel, and the original signal generator would be determined once all 3000 columns are decoded.

Hi, can you elaborate on this? Sounds like an interesting idea but I have no idea how this could be applied in practice to solve the problem. Also note that the relationships between the patterns and targets are probabilistic, not deterministic.

I tried to find at least one paper on the subject of using Luby Transform code for classification tasks but couldn't find anything relevant. This brings up some doubt.

 

[goodgoing]

Quote from Indrionas:

Also note that the relationships between the patterns and targets are probabilistic, not deterministic.

I hate people like you who are either complete morons or malicious individuals with a hidden agenda. When confronted they accuse others for being stupid, for not understanding the problem.

Didn't you state that:

Quote from Indrionas:

The data is synthetic. Means I have generated it and I know the rules (real model) that generated it...

A pattern can be in the form of:
if (variable #35 is true) AND (variable #184 is false) then target is "true"...

You are probably a liar and you never generated the data yourself. The relationship you stated above if fully deterministic. You never stated before that the relationship is probabilistic.

Do you have anything to say troll?

P.S. You even managed to confuse Jack Hershey, the giant of ET.

 

[limitdown]

Quote from goodgoing:

If you want people to take you seriously you have to describe the problem in detail and provide an excel file they can work with. The way you described it is not intelligible. It makes no sense. You are talking about random entries without explaining why they are there in the first place, for example.

the problem with these corporations today, is they hire based on race, religion, looks and an expectation that using those qualifiers will produce a better employee that can actually meet the job description as well as live up to their proclaimed resume.

another problem with these corporations is they craft these job descriptions that when reasonably broken down, or understood and compared to how they and other corporations used to hire just say 10 years, 20 years ago, is these job descriptions cover what two and three people used to do.

another problem with these corporations is the relegate, not delegate, the hiring process and interview process to the lowest level employee, as if they are also capable of determining who should be hired next. In such an environment EEOC laws and practices are violated both in principle and substance,as well as time and time again, the only accepted person through the process end up being of the same sex, looks, religion or religious beliefs, same accent and same moderate education level (irrespective of whether its at the graduate level, ivy level, public or private collegiate level too).

another problem with these corporations is such trust and excessive authority given to such low level employees (such as the joker that started this thread).

GS had to sue the Russian programmer who after making it through their religious preference hiring practices which produced him as an employee, they had to sue him to recover stolen decision based trading code.

Other Wall Street brokerages have tried and used the same preferential hiring practices mostly by limiting how a candidate is introduced to them, namely through preferential agents and agencies as well as seeking if not creating paperwork on candidates from sources that they steer inquiries towards. What this has produced are a dirth of competent collegiate candidates that never find the first line to wait in let alone ever get their paper on someone's desk to review, let alone ever get prepared (prep'd) for such interviews, and most certainly never get interviewed or considered.

another foul stentch that results from this race based, preferential hiring practice are threads like these where these essentially incompetent persons who are way underprepared, act like they are well endowed with knowledge and privilidge and post such highly profitable (potentially so) questions like these on threads like these expecting to get answers for free, so they can collect their 200% bonuses at year end.

get a life you jerk.

 

[limitdown]

Quote from psytrade:

The way to solve this problem is using the Luby Transform code. Raptor codes are linear time codes which would solve this problem.

The reason is because an Exclusive OR solves the bipartite graph problem that your data presents.

http://en.wikipedia.org/wiki/Luby_transform_code

http://en.wikipedia.org/wiki/Bipartite_graph

You don't need a neural net because you have binary data signals.

The simplest algorithm would determine the original distribution by scanning through the rows and return a value until the job is complete.

Think of each row as a separate channel, and the original signal generator would be determined once all 3000 columns are decoded.

after saying all that,

do you suppose you can really trade a simple buy / sell pattern?

really, because my experience on the Equity trading desks of a few firms has been, those who took their MBA's in Financial Engineering couldn't read a simple Equity chart, nor hold their water long enough to handle being down over -$80,000 on a $200,000 trading position in, say IBM, before closing the day out positive at, say +$4,285.

 

[limitdown]

Quote from limitdown:

the problem with these corporations today, is they hire based on race, religion, looks and an expectation that using those qualifiers will produce a better employee that can actually meet the job description as well as live up to their proclaimed resume.

another problem with these corporations is they craft these job descriptions that when reasonably broken down, or understood and compared to how they and other corporations used to hire just say 10 years, 20 years ago, is these job descriptions cover what two and three people used to do.

another problem with these corporations is the relegate, not delegate, the hiring process and interview process to the lowest level employee, as if they are also capable of determining who should be hired next. In such an environment EEOC laws and practices are violated both in principle and substance,as well as time and time again, the only accepted person through the process end up being of the same sex, looks, religion or religious beliefs, same accent and same moderate education level (irrespective of whether its at the graduate level, ivy level, public or private collegiate level too).

another problem with these corporations is such trust and excessive authority given to such low level employees (such as the joker that started this thread).

GS had to sue the Russian programmer who after making it through their religious preference hiring practices which produced him as an employee, they had to sue him to recover stolen decision based trading code.

Other Wall Street brokerages have tried and used the same preferential hiring practices mostly by limiting how a candidate is introduced to them, namely through preferential agents and agencies as well as seeking if not creating paperwork on candidates from sources that they steer inquiries towards. What this has produced are a dirth of competent collegiate candidates that never find the first line to wait in let alone ever get their paper on someone's desk to review, let alone ever get prepared (prep'd) for such interviews, and most certainly never get interviewed or considered.

another foul stentch that results from this race based, preferential hiring practice are threads like these where these essentially incompetent persons who are way underprepared, act like they are well endowed with knowledge and privilidge and post such highly profitable (potentially so) questions like these on threads like these expecting to get answers for free, so they can collect their 200% bonuses at year end.

get a life you jerk.

my response was directed at the starter of this thread,

and especially in calling such a jerk,

I have seen countless idiots, whom thought they were privilidged, especially bright or connected in these positions at these Wall Street firms, and stacked up to some pretty bright New Yorkers with degrees, they look as stupid as they really are.

let's face it, with all the favorable winds that these marmalukes have in getting these jobs, even a high school drop out could succeed instead of having earned the job in a properly competitive fashion.

Steve Jobs at Apple has such a strick policy about not even discussing internal projects and new development projects within the very same firm on pending punishment (dismissal from Apple and banning in the Silicon Valley). Wall Street firms, where the core logic of Data Mining as it pertains to advanced internal neural network logic and other advanced applications have the same if not more restrictive covenants. Imagine the surprise to see on these threads, this topic exposed...

Hint: Get your resume ready, and its not pleasant out on the street...

 

[Indrionas]

The stupidity of the trolls in this thread is beyond belief. It turns out that I did not generate the data myself and I'm looking for a job (or have a job) in Wall Street or whatever the guy with long rants tried to say. Unbelievable

Well, I was wrong thinking I could find an intelligent discussion here. This forum is dead.


P.S. here is the list (attachment) of the patterns in the dataset (that is just one of the many datasets that I generated).


Really, is that so hard to believe that I am just a guy who's interested in this on his spare time and have no connection to Wall Street in any way. I don't even live in the USA. I don't do any secret research for any government/private company/academic entity. I don't even know why I even respond to these absurd attacks.

Don't bother responding. I don't care any more.

 

[rdg]

Don't give up just yet. I was actively working on getting you your promotion at the government agency that doesn't (publicly) exist. Please post another problem with similar complexity and let this thread sit idle.

 

[jack hershey]

Quote from Indrionas:

The stupidity of the trolls in this thread is beyond belief. It turns out that I did not generate the data myself and I'm looking for a job (or have a job) in Wall Street or whatever the guy with long rants tried to say. Unbelievable

Well, I was wrong thinking I could find an intelligent discussion here. This forum is dead.


P.S. here is the list (attachment) of the patterns in the dataset (that is just one of the many datasets that I generated).


Really, is that so hard to believe that I am just a guy who's interested in this on his spare time and have no connection to Wall Street in any way. I don't even live in the USA. I don't do any secret research for any government/private company/academic entity. I don't even know why I even respond to these absurd attacks.

Don't bother responding. I don't care any more.

This space offset list represents market price shifts.
To re-express this series as a two level pattern that exhibits market sentiment and the profit making potential on two interlocking profit fractal levels, here is what emerges.

The slow fractal is set off by semicolons (; ) and the faster fractal within is set off by spaces. Occasionally a slower pattern “fails to complete” and this sub segment is “folded” into the existing developing slower fractal pattern. To show the this in the series I just began the “fold” with a ( and ended it with a ).
A value was arbitrarily assigned a sign (vector status) which is inconsistent so I changed the inconsistent sign. It is colored green so a reader can find it easily.

-175 -7 +18 +35 -79 (+145 -171 -213) (+15 -78) ( +109; -288, -0 -107 -293); +208 +161 +157 +13 -238 -180 +195 +270; -179 -205 -186 -204 +109 +36 -69 -75 -179 (+285 -106 -216) ( +138 -127 -200 -271) ( +47 +37 +12 -147 ) (+17 -94 -191 -114 -125); +284 -16 …..incomplete as yet.

For those who can read the market’s progress, the first short dominant sentiment is enhanced by two VE’s before the pattern overlap begins with a reversal that is long (non-dominant) going to long dominant as the RTL of the prior short is broken. Later after the long, the next short pattern ensues. As you see, four times the end of this short pattern was succeeded by a failure of a long pattern to ensue. Finally the pattern ends and the RTL has been broken ending the overlap with the prior pattern.

In my first post I concluded that the data set was not a conclusive one and now the reasons for this conclusion may begin to be evident.

Most people Who are informed will recognize the parenthetical sections as “bookmarks”; you may also see how just using vertical summations leads to possible VE’s. M1’s and M2’s aren’t observable.

There is a general misconception out there that "noise" and "anomalies" exist in markets. The proofs of noise and the proofs of anomalies just represent lack of progress in understanding markets.

If you wish to read a team's explanation of NOT proving noise is present and then "eliminanting" it in an arbitrary manner, read Lo, et al on patterns. Also if you wish to hear his words and orientation see the comedy "Inside Job".

To make money slowly trade the vectors between the semicolons. To make money on the next faster fractal patterns trade the signed segments.

What is represented by NOT using probability in trading?

What could have resulted if this effort were not just a PA exercise?

How can a researcher excuse himself when he arbitrarily eleiminates market variables?

Does anyone know any scientific sector where an engaged scientist can take such liberties?

Any sequence of output data can be subjected to an analysis based upon the paradigm of the market. Here output data just represents a series. Its information content can range from nothing to optimal; in any case what has to be added to the scene is the actual paradigm of the market.

So why cannot the financial industry get on baord and use the market's paradigm?

 

[jack hershey]

Quote from goodgoing:

I hate people like you who are either complete morons or malicious individuals with a hidden agenda. When confronted they accuse others for being stupid, for not understanding the problem.

Didn't you state that:



You are probably a liar and you never generated the data yourself. The relationship you stated above if fully deterministic. You never stated before that the relationship is probabilistic.

Do you have anything to say troll?

P.S. You even managed to confuse Jack Hershey, the giant of ET.

Your mistaken humor about me was a nice chuckle.