Absolutely false. I'm a statistician who specializes in large data. I'm guessing you aren't dealing with large data, even though you think you are. That aside, you prove my point -- for R to work at all with large data, you'll need to add on all sorts of packages like SNOW or foreach or multicore and spend hours reading documentation (and pray it works). Fire up a database package like Oracle or SQL Server or a statistical processing program like SAS, and you'll need no extras. Better yet, you only need to know SQL to obtain basic queries, which it sounds like the OP is after. viktor_k67 has got the right idea.
You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.
You should upgrade or use an alternative browser.
Historical Market Data - Where can I get a better format?
- Thread starter Jason-S
- Start date
LOL. What? Of course it does. Fast access through indexing for starters. Ability to subset data for subprocessing? You can't beat a database for storage. There's a reason you see all those add on packages for R.Saving Time series data in a relational dB does not have any benefit..
Completely false... I deal with hundreds of millions of rows of tick data very quickly with no dB... there's packages to parrellize operations as well.... just a statement in ignorance
I think what he meant is some really huge intraday data e.g. 1996 to 2015 milli-seconds intraday data... I think R cannot handle this kind of gigantic data
Absolutely false. I'm a statistician who specializes in large data. I'm guessing you aren't dealing with large data, even though you think you are. That aside, you prove my point -- for R to work at all with large data, you'll need to add on all sorts of packages like SNOW or foreach or multicore and spend hours reading documentation (and pray it works). Fire up a database package like Oracle or SQL Server or a statistical processing program like SAS, and you'll need no extras. Better yet, you only need to know SQL to obtain basic queries, which it sounds like the OP is after. viktor_k67 has got the right idea.
Only when I said you were ignorant did you get proved right.... calling yourself an expert statistician doesn't help you either.. the guy is using excel ... the next step is to use a high abstract free platform and language to do studies...
I deal with tick day .. most do... I know alot of people who use R for just what he is talking about. Granted there are several ways to crack a nut.. but I'd rather rather go with R programming as it keeps you away from programming details and focused on the work at hand... it's very powerful... so much so that Microsoft has taking a very large interest.... but hey this is just my recommendation to the OP.... it's more then worth a look... quandl R package enables super ease of access to quandl data.. I know the founder of quandl personally and I think you just have alot of intellectual arrogance to slander something you obviously know nothing aboutI think what he meant is some really huge intraday data e.g. 1996 to 2015 milli-seconds intraday data... I think R cannot handle this kind of gigantic data
Saving Time series data in a relational dB does not have any benefit..
For 15 years I am using SQL starting from SQL 2000 to store and analyze market data and I do not regret it, and I would never go back to excell. There could be other DB solutions like mySQL or Python and etc., Yet, if you want to work with excell (why to pay for Ms Office if SQL is free) good luck.
It is not just about storring and ot loading data. As I understand Jason is asking for a nice solution which would allow him in quick and easy ways to crete diffrent data views, scan and filter data he needs for his analysis, maybe export data in different formats for some soft and charting solutions. On my opinion he needs to go away from excell to DB. What type of DB is another question and it could be nother discussion.
For 15 years I am using SQL starting from SQL 2000 to store and analyze market data and I do not regret it, and I would never go back to excell. There could be other DB solutions like mySQL or Python and etc., Yet, if you want to work with excell (why to pay for Ms Office if SQL is free) good luck.
It is not just about storring and ot loading data. As I understand Jason is asking for a nice solution which would allow him in quick and easy ways to crete diffrent data views, scan and filter data he needs for his analysis, maybe export data in different formats for some soft and charting solutions. On my opinion he needs to go away from excell to DB. What type of DB is another question and it could be nother discussion.
Right. The original question asked about using Excel.
Without knowing what analysis the OP planned on doing, one "expert in all things" here basically said, You're all wrong. Excel won't work. You must use 'R.'
Lol.
As I've stated, he would likely need to use VBA to do as he asked. If it doesn't work because the OP's analysis is too involved for Excel/his-computer, then I suspect the OP will so state.
Or, the OP, unable to format data in excel, will decide to learn "R" all of a sudden.
Gotta love internet experts.