Okay. So you meant is I use some kind of scheduler that runs at the end of time interval and it appends the new data to an already existing data?
secondly,
Where do I store this data?
Memory or some database?
Live Streaming Historical Data
- Thread starter wackytacky
- Start date
Okay. So you meant is I use some kind of scheduler that runs at the end of time interval and it appends the new data to an already existing data?
secondly,
Where do I store this data?
Memory or some database?
Depends if you want the bar to be closed on time or existing ticks.
On existing ticks you can just use sth like below. Whereas I prefer to have the current bar also already in the existing array/data structure and not separate it
Depends. Whether you want to request the data over and over from your provider or have it on your system.
I store data in plain files because I can make better use of resources.
1 Minute Example in Golang, pseudo, without consideration if tick data might cross with historical data:
type Tick struct {
Time int64
C int64
V int64
}
type OHLCVBar struct {
Tick
O int64
H int64
L int64
}
type Ctx struct {
Bars []OHLCVBar
BarCur OHLCVBar
LenBarTime int64
}
func (self *Ctx)OnRTData(tick *Tick) {
isNextBar := OHLCVTimeBasedAddTick(&self.BarCur, tick)
if isNextBar {
self.Bars = append(self.Bars, self.BarCur)
StoreBarToDB(self.BarCur)
barNext := OHLCVBar{}
CalculateNextBarTime(&barNext, LenBarTime)
barNext.O = tick.C
barNext.H = tick.C
barNext.L = tick.C
barNext.C = tick.C
barNext.V = tick.V
self.BarCur = barNext
}
}
// assumes that OHLCVBar Time is on end of period
func OHLCVTimeBasedAddTick(barCur *OHLCVBar, tick *Tick) (isNextBar bool) {
if tick.Time >= barCur.Time {
return true
}
barCur.C = tick.C
barCur.V += tick.V
barCur.H = MAX(barCur.H, tick.C)
barCur.L = MIN(barCur.L, tick.C)
return false
}
@MarcG
Now, I'm not trying to be a dick... but if @wackytacky doesn't now proper coding maybe don't give him code at all.
Because, if there's a mistake in it or it doesn't work with the rest of whatever he's got it could end up in a drama.
(PS. I don't know coding either, but I do know that things can easily be overlooked if you don't know what you're doing)
Now, I'm not trying to be a dick... but if @wackytacky doesn't now proper coding maybe don't give him code at all.
Because, if there's a mistake in it or it doesn't work with the rest of whatever he's got it could end up in a drama.
(PS. I don't know coding either, but I do know that things can easily be overlooked if you don't know what you're doing)
Just a little help for him and to be honest, I don't want to be impolite but I really don't care if his account gets killed with my code or not. One important part of software development / engineering is testing. If I hadn't tested my code which integrated code from others in the past ... man I would have lost my job and my account a 1000 times.
Sadly, when it comes to money or software people often act brainless.
So ultimately, it is his responsibility ... if he wants me to give some ideas or have a short look at his final code before going real, no problem but the rest is up to him.
Offcourse it is my responsibility to review source code before going live.
Code:
import csv
from kiteconnect import WebSocket
api=' '
public=' '
userid=' '
kws=WebSocket(api,public,userid)
def on_tick(tick,ws):
crudeoilm = open('crudeoilm.csv', 'a')
crudeoilWriter = csv.writer(crudeoilm)
crudeoilWriter.writerow([tick])
def on_connect(ws):
ws.subscribe([53315591]) # Instrument token for Crudeoil Contract Dec-2016
ws.set_mode(ws.MODE_LTP,[53315591])
kws.on_tick=on_tick
kws.on_connect=on_connect
kws.connect()The above is the code I used to fetch Tick data and send it to CSV file, Will it work?
The attached file is the output of CSV file.
The thing is the Json data is pasted to one particular cell in CSV, rather than having Keys and values of it in different cell. How do I manipulate that?
Also how do I use this streaming live data and make an OHLC 5min data out of it?
This is the link to docs-
https://kite.trade//docs/pykiteconnect/#kiteconnect.WebSocket
Attachments
Again: Better you start with basics of programming.
There are many very good python tutorials out there.
You get a JSON string representing a tick via your Websocket. JSON is not comma separated and it makes no sense to split it in some way here.
Neither it makes sense to open a file and a writer on it each time a tick comes in.
Any tutorials you would like to suggest that are particularly for 'building automated trading systems in python'