new software tool development for technical Analysis of Quot data
- Thread starter aqtrader
- Start date
Nice...
A native time series data type of Vector and an implementation that utilizes the vector processing capability of the hardware...
Have you seen the xts package for R???
I would be interested in taking this puppy for a ride when it becomes available 8^)
Cordially,
-DD-
"It's like they just gave you a new horse." -Dan Ingalls-
A native time series data type of Vector and an implementation that utilizes the vector processing capability of the hardware...
Have you seen the xts package for R???
I would be interested in taking this puppy for a ride when it becomes available 8^)
Cordially,
-DD-
"It's like they just gave you a new horse." -Dan Ingalls-
For simple algorithms, Stockfetcher is easy to use for scanning the equity markets.
www.stockfetcher.com
However it doesn't support higher level constructs such as looping and recursion.
www.stockfetcher.com
However it doesn't support higher level constructs such as looping and recursion.
The source is here:
http://freechartgeany.sourceforge.net/download.html
It compiles just fine in Ubuntu.
I did not check this thread for the past few weeks. Thanks for the above few feedbacks. Stock fetcher and free chart Geany seem interesting but not the same tool I am working on. Aqtools is designed more for customization of indicators and filters of interest on any user defined watch list. It builds for speed and scalability for both eod and intraday data. Charting is not the essential part of the toolset. I just use gnuplot and qiv to draw and to slide show charts of my flavor.
Unlike the R language which is for professional generic math statistics, aqlang is more specific for vector processing of timed stock data.
Still work on this software, slowly and steadily making progress.
Aqtrader
Unlike the R language which is for professional generic math statistics, aqlang is more specific for vector processing of timed stock data.
Still work on this software, slowly and steadily making progress.
Aqtrader
> Speed performance - use multi-core vector CPU
Backtests only, I Hope. Not optimizing. WAY TOO SLOW.
You need to go HPC with a cluster of machines for that. You also want something command line driven. with queueing - so you can schedule a dozen backtests or optimizations then go to bed.
OTOS In memory database is not needed - I have around 350gb in tick data on a cluster of 7 active nodes now - with 1 node down (so I have 8) and another 19 nodes on backorder coming in the next 5 weeks. With backtests there is a performance issue on the network (currently 1gbit for the file server), but with optimizations it is not even showing up - totally low traffic.
The database is more important - you want a good database for the results so you can analyze them later. That will be a lot more - my own database right now has about 200gb in executions and trades, but we also just come out of a weekend (many will be deleted once having been looked at).
And the whole "own langauge" smells like you need a NIH treatment from a psychologist. Why the heck go with your own langauge - just use something that is open enough. Saves on time. Time is money - time to "market" means you get results faster than writing your own parser and compiler.
Backtests only, I Hope. Not optimizing. WAY TOO SLOW.
You need to go HPC with a cluster of machines for that. You also want something command line driven. with queueing - so you can schedule a dozen backtests or optimizations then go to bed.
OTOS In memory database is not needed - I have around 350gb in tick data on a cluster of 7 active nodes now - with 1 node down (so I have 8) and another 19 nodes on backorder coming in the next 5 weeks. With backtests there is a performance issue on the network (currently 1gbit for the file server), but with optimizations it is not even showing up - totally low traffic.
The database is more important - you want a good database for the results so you can analyze them later. That will be a lot more - my own database right now has about 200gb in executions and trades, but we also just come out of a weekend (many will be deleted once having been looked at).
And the whole "own langauge" smells like you need a NIH treatment from a psychologist. Why the heck go with your own langauge - just use something that is open enough. Saves on time. Time is money - time to "market" means you get results faster than writing your own parser and compiler.
If you were a good computer professional, you would have understood what I am talking about when I mentioned about all those terms about performance optimizations, database, languages, etc. Google the Dragon Book and The Whale Book. You will know a bit. If you were a computer geek or earned a PhD in computer science. You probably knew how to handle a challenging interesting project. I do not think I am mad on doing this one-person hard work. I live on writing software. I am doing this as a hobby in my free time. Anyway you made some points but not the last one on "language". Aqlang is much simpler than most of other computer languages not like C or R. I have already completed a prototype including a compiler and a "virtual machine" to execute aqlang programs. Actually, working on aqdb database takes me more time.
Start from a simple.
Make a simple program that has basics, post it online as an open source project and invite other interested to make an input. It is going to be faster and easier. If nobody helps that would mean nobody needs it.
P.S. there is always will be a problem with data source and data-feed support. Nobody offers it for free and you cannot redistribute t for free as there high exchange fees for redistribution.
Because of that this project could die simply because it will be for selective number of people only:
1. those who has Linux
2. those who has data-feed and historical data,
3. those who has necessary skills and may adopt this project to his/her data format.
Make a simple program that has basics, post it online as an open source project and invite other interested to make an input. It is going to be faster and easier. If nobody helps that would mean nobody needs it.
P.S. there is always will be a problem with data source and data-feed support. Nobody offers it for free and you cannot redistribute t for free as there high exchange fees for redistribution.
Because of that this project could die simply because it will be for selective number of people only:
1. those who has Linux
2. those who has data-feed and historical data,
3. those who has necessary skills and may adopt this project to his/her data format.
Viktor_k67,
Thanks for your comments.
Exactly I did my best to start with an extremely simple languaguge aqlang but the implementation of it took quite a bit of effort and fun. It is not perfect. It is built with a database optimized to work with aqlang programs. Now it can handle multiple tables and scalar groups for thousands of symbols on a decent Linux machine. I will post more about the design of aqlang and aqdb here soon to share with interested parties.
You are right that data feed is more challenging considering fees, speed and quality. However this is another topic for potential applications of aqtools. Right now, aqtools is good for post trading analysis and 1-min near-real-time analysis.
About data format, aqtools feed on csv format like yahoo/google eod and min data. Theoretically aqdb accepts any format because it allows a user define his own format for his table/array/vector. Internally, aqdb stores data in binary format but provides tools to allow users to display or to edit binary data in text form conveniently.
Aqtrader
I appreciate your work and I have a few questions and one suggestion at this point
How frequent trading per instrument per day do you want your backtesting system to simulate ?
I know you only said down to one minute but how fine time stamps have you implemented ?
Have you put in fields for the high and low bid and ask in the one minute period ?
Have you thought about complex event processing in relation to your language? Here is an example http://esper.codehaus.org/
May I suggest that for the source data rather than just main memory you make it (sequential) streaming main memory from disk or memory mapping. Most useful analysis will pull data slower than the sequential speed of modern spinning disks not to mention SSDs and there is not a need to randomly access quote data. Thinking about the design to allow for this will allow your tool to be more general.
How frequent trading per instrument per day do you want your backtesting system to simulate ?
I know you only said down to one minute but how fine time stamps have you implemented ?
Have you put in fields for the high and low bid and ask in the one minute period ?
Have you thought about complex event processing in relation to your language? Here is an example http://esper.codehaus.org/
May I suggest that for the source data rather than just main memory you make it (sequential) streaming main memory from disk or memory mapping. Most useful analysis will pull data slower than the sequential speed of modern spinning disks not to mention SSDs and there is not a need to randomly access quote data. Thinking about the design to allow for this will allow your tool to be more general.