Free data

yabz

yabz

As this thread shows it is becoming more and more difficult to find free data. One cannot even count on commercial data providers. This is very worrying for all those using automated trading techniques especially. Even if one finds new sources of data it requires re-writing of software.

I would like to suggest a solution, that would provide a long term and reliable source of financial data. I propose creating an open database similar to this site for gardeners. Instead of seeds it would provide financial data. The information would be crowdsourced by members who would collect and aggregate financial data of every kind, stocks, commodities, forex, cryptocurrencies and all types alternative data. The format would be standard and accessible from an API. The source code would be open source and available on github.

If enough people were interested in this idea it has many advantages.

Your thoughts, please.
 
It is a good idea. The problem is there will be many free-riders who contribute nothing but want to benefit from the efforts of others. Software is write once and that's it. Data collection is an ongoing effort. I don't know how you can get consistency of effort if the person doing the work is not paid.
 
The theory is nice, but in the end everything costs time and money one way or the other which leads to a bipolar shift, you either commercialize or it's everyone for themselves. Unless this is an obfuscated plan to build the entity and sell it for capital, or to have insight in to peoples strategies and use it for your own income, the word futile comes to mind because you miss one key point, trust.

But you know, experience doesn't count for anything these days because obviously we don't have these datasets and apis available that we use purely internally because we trust ourselves implicitly!
 
just21 said:
Already exists at https://www.quandl.com/data/WIKI-Wiki-EOD-Stock-Prices and has an api. No etf's and only 3000 stocks but it is a wiki so add some more yourself.
More...

Have you actually added or edited any of the data? I've searched long and hard and there is no way to modify the data despite the "community curated" thing they mention.
Because this is data, just adding something doesn't mean anything, you need to provide a source and code for retrieval or you'll spend time doing it manually every day.

On the idea. I've uploaded and shared data before on ET with the intention to "trade" it for other data. Needless to say the other person didn't deliver their end and disappeared when he got what he needed. So no, I don't believe sharing works in the trading community.
 
I envisage a site modeled on growstuff.org. Members would collect and curate data on a particular instrument and then offer to swap, or sell, this data with other members. For example you might be keeping a record of the gold price, but require historical data on sugar, so you would swap your data on the gold price in exchange for the sugar price data. If the other person didn't want your gold price data you could buy the sugar price data. This is how growstuff works. If you have some celery seeds for example you can swap them with someone who has hemp seeds (or sell them).
 
helpme_please said:
It is a good idea. The problem is there will be many free-riders who contribute nothing but want to benefit from the efforts of others. Software is write once and that's it. Data collection is an ongoing effort. I don't know how you can get consistency of effort if the person doing the work is not paid.
More...

That's why you have to charge for it. I think you can still download 10 years worth of historical data through NinjaTrader in CSV ormat.
 
yabz said:
I envisage a site modeled on growstuff.org. Members would collect and curate data on a particular instrument and then offer to swap, or sell, this data with other members. For example you might be keeping a record of the gold price, but require historical data on sugar, so you would swap your data on the gold price in exchange for the sugar price data. If the other person didn't want your gold price data you could buy the sugar price data. This is how growstuff works. If you have some celery seeds for example you can swap them with someone who has hemp seeds (or sell them).
More...

Check out all the free data on www.quandl.com Multiple databases all accessible with the same API for free.
 
just21 said:
Check out all the free data on www.quandl.com Multiple databases all accessible with the same API for free.
More...

Quandl is very good and I use them. There are 2 problems however:
1. Like yahoo, they could decide to abandon the system overnight. A new manager might want to take the company in a different direction. In this case you are screwed. You can't assume they will be around for ever.
2. Their database is not comprehensive. Some metal prices only go back a few years; they don't have crytocurrency data etc.
 
You must log in or register to reply here.
Back
Top