What can you do as a proficient programmer?

[Laissez Faire]

Quote from CalVolibrator:

WTF!!!!! For this nut job I wasted my time, even a single line of advice to this guy was not worth it. And to keep it to a minimum I am out of here, and hopefully better learned to read nut cases.

Goodbye! You won't be missed.

Quote from blah12345678:

I wrote this previously. It'd be a big enough challenge for you at this point. Give it a try. I used Perl. But it's easily doable using Python or Ruby or even PHP.

http://www.elitetrader.com/vb/showthread.php?s=&postid=3806660#post3806660

Later on, if you continue to maintain your own data infrastructure, you can ease things greatly by getting a subscription to eoddata.com and download things in one swoop instead of retrieving 7200+ pages from Yahoo.

Also, don't forget about that website I posted earlier for Java and Python programming lessons.

And if you get stuck, you can find any answer via Google. The results will likely lead you to either the language docs or StackOverflow.com

Thank you!

Great advice and resources.

Mucho appreciato.

Sincerely.

LF.

 

[slickpick]

Quote from blah12345678:

I wrote this previously. It'd be a big enough challenge for you at this point. Give it a try. I used Perl. But it's easily doable using Python or Ruby or even PHP.

http://www.elitetrader.com/vb/showthread.php?s=&postid=3806660#post3806660

Later on, if you continue to maintain your own data infrastructure, you can ease things greatly by getting a subscription to eoddata.com and download things in one swoop instead of retrieving 7200+ pages from Yahoo.

Also, don't forget about that website I posted earlier for Java and Python programming lessons.

And if you get stuck, you can find any answer via Google. The results will likely lead you to either the language docs or StackOverflow.com

I somewhat disagree with your post about subscribing to eoddata, the reason being is the only real upside I see to going to a data vendor for EOD data are as follows: 1) They handle delistings and symbol changes effectively and 2) provide corporate actions.

You'll certainly be able to pull everything in one swoop from them, but the upside is nominal IMO. Ok so you need to pause your loop when pulling from yahoo, you'll likely be running this process overnight so it's not really a big deal I don't think. You'll just lack delisted securities from Yahoo, as well as symbol changes.

However you can deal with symbol changes on a forward moving basis either create your own unique symbology or just use bloomberg's open symbology.

Plus there's definitely value in this guy learning how to get data from random sources.

 

[blah12345678]

Quote from slickpick:

I somewhat disagree with your post about subscribing to eoddata, the reason being is the only real upside I see to going to a data vendor for EOD data are as follows: 1) They handle delistings and symbol changes effectively and 2) provide corporate actions.

You'll certainly be able to pull everything in one swoop from them, but the upside is nominal IMO. Ok so you need to pause your loop when pulling from yahoo, you'll likely be running this process overnight so it's not really a big deal I don't think. You'll just lack delisted securities from Yahoo, as well as symbol changes.

However you can deal with symbol changes on a forward moving basis either create your own unique symbology or just use bloomberg's open symbology.

Plus there's definitely value in this guy learning how to get data from random sources.

I know of only 2 places to get listed and delisted data - Premium Data, and CSI. Premium Data is affordable, and comes with the CUSIP number. CSI requires a commercial license costing $10K at a minimum.

Downloading from Yahoo Finance doesn't take hours. My Perl script takes about 15 min for the download portion. One user in another thread wrote a C downloader using libcurl which downloads the data in less than a minute.

Why choose a random site instead of one reliable site like Yahoo? Heck, Google can't even come up with a decent Finance site. If they can't, how can we expect any other site to be better - more reliable, more comprehensive, and FREE?

Since Yahoo is a highly trafficked site, 40 API calls or 7,000+ data downloads won't be noticed, let alone the TOS suddenly being enforced with an IP address ban.

I recommended eoddata.com because at some point, if you're maintaining your own infrastructure, you just want to get the job done easily and reliably. I've used CSI (hacking their Market scan file to get 10 years of data for every listed stock). I currently use Yahoo Finance for US stock data. I've used Quandl for US futures data, but it's often not available until 5 am the next morning. So I subscribed to EOD Data to get today's futures data today.