Long time no see!
- Thread starter aphexcoil
- Start date
Robert Morse
Sponsor
I was not. Would you like to tell us a little about you?
Bob
Bob
Sure! I am primarily a data scientist with an emphasis on big data (ingest, archival and search). I currently run a search API for Reddit comments and submissions and have monthly dumps of all Reddit comments and submissions. I have extensive experience in cataloging and making data easily available via search.
I am just getting back into trading and would like to create some cool data services for traders down the road.
If you are interested in using the search API to find social media comments related to trading, let me know and I'll forward to you the API documentation. It is free for non-profit use and open to the public.
I am just getting back into trading and would like to create some cool data services for traders down the road.
If you are interested in using the search API to find social media comments related to trading, let me know and I'll forward to you the API documentation. It is free for non-profit use and open to the public.
I'd be interested to check it out (tough I am for profit) - looking at social sentiment is on my to-do list.
Sorry, I should clarify -- you can use the data however you want. I just meant that if someone wanted to design a service around the API, I would want to talk with them first.
The API documentation is here: https://github.com/pushshift/api/blob/master/README.md
Currently, the system can search all public comments and submissions on Reddit and it ingests data in near real-time (a second or two delay). For instance, if you wanted to search for comments with the terms AMD and stock, you would make a call like this:
https://api.pushshift.io/reddit/search/comment/?q=amd+stock
If you wanted to search for the exact phrase "amd stock":
https://api.pushshift.io/reddit/search/comment/?q="amd stock"
The API also can do aggregations. The Elasticsearch back-end is also available directly if you know how to make ES queries and want to run more complicated searches.
The API documentation is here: https://github.com/pushshift/api/blob/master/README.md
Currently, the system can search all public comments and submissions on Reddit and it ingests data in near real-time (a second or two delay). For instance, if you wanted to search for comments with the terms AMD and stock, you would make a call like this:
https://api.pushshift.io/reddit/search/comment/?q=amd+stock
If you wanted to search for the exact phrase "amd stock":
https://api.pushshift.io/reddit/search/comment/?q="amd stock"
The API also can do aggregations. The Elasticsearch back-end is also available directly if you know how to make ES queries and want to run more complicated searches.
Well look who is back.
Pretty swift Reddit search API. Seems to be pretty fast from your examples.
Pretty swift Reddit search API. Seems to be pretty fast from your examples.
Thanks! It is currently running on 3 nodes so it generally responds quickly.
Good to see you!
Welcome back! Interesting story.