Search results

I just edited my original posting we crossed postings ;) Intermediate result actually were a possible factors. The goal of the research is to determine if these are important factors or not. At this time we don't know that, but that is common with research. My point is for research we don't...

I am referring to an aggregation. If you have several tables which totally have ~1,000 columns if you want to access just one column of each table you have to make a join on all of tables to extract just one column of each table. In practice extracting just one column from one table means you...

This is not one 1,000 columns monster table - MS SQL do not allows so much columns in one table - it allows up to ~500. I was referring for a total of 1,000 columns of data but not in one table, they are organized of course in many tables but I don't want to make a long descriptions here...

It is ~1,000 columns per stock. I created a benchmark to look how long it would take the generate these ~1,000 columns just for EOD DJIA because it has the longest history - 1928-2009. DJIA 1928-2009 (100 years) - OHLCV is 20,212 lines(EOD - End Of Day records), ~20KB input .CSV file. The...

Thanks - it is a lot effort to formulate the problem and solve it - it took more than three years to figure out how to make a good screener and back testing platform. Let me try to explain better. After spending a lot of time at learning technical analysis and strategies I found the way...

I don't know if this new feature will help now but 2 years ago obviously .NET was not ready for the task we are talking about.

Matching involves a search right? Faster search means faster matching. Faster matching - faster execution. Do you see where this is going? The system under question do not reach more than 10,000 orders/sec [B] This part would be slow with MS SQL. What they do in other exchanges?

So you process just 1 order at a time, serially?

It seems to me MS or the others wanted to use all their stack in this project, .NET, MS SQL, the OS. If this is just a pass through problem - just to pass the data as fast as possible they had to use the OS + some programming. But still the data have to be stored somewhere and they don't had to...

A good point. ;) But you still have to store these data somewhere right? They can be used. ;) I suppose the orders are first stored in the memory. The data can be packed and compressed with SIMD to move faster through the pipe. It is possible bot not always. I read about the...

I believe they wanted to store these data somewhere and for that used MS SQL server right?

I am working with .NET from the very beginning ~5 years ago ver 1.0. In the beginning was very excited. Speeds the development time a lot, UI programming is quick, Visual Studio is probably the best development environment ever, SSIS under MS SQL is very good, paralelizes when you have >1 CPU...

The LSE TradeElect abandoned. http://blogs.computerworld.com/london_stock_exchange_to_abandon_failed_windows_platform "TradElect runs on HP ProLiant servers running, in turn, Windows Server 2003. The TradElect software itself is a custom blend of C# and .NET programs, which was created by...

By warehouse oriented I mean a large TB size DB's. I don't believe a lot of interested parties have an access to TB size financial data, probably the exchanges, very large financial firms, clearing houses, etc. Being TB size oriented, their focus is mostly on storage and management. For...

1) Amazon EC2-hosted 2) 500GB/$500/month/1 node 1TB/~$4,000/month/3 nodes 1) Is not the best idea for high performance computing. 1.1) you have the context switching overhead. 1.2) 500GB-1-5TB back and forth to Amazon servers - I don't see how with current transfer speeds this would a...

We are developing a historical time series screener/backtest/replay software with innovations such as multiple synchronized windows capable to operate on different stocks, different time frames, and different time periods and fast parallel processing. Moving your cursor in one window...

A quick note - Vertica is more oriented toward large warehouses like inventory analysis, etc. We need more time series oriented DB.

There is a different access pattern with SSD and the optimal way to write/read is still not researched well. When they stop the deterioration problem we will wait for 1 year to look for any other hidden surprises. they will get cheaper too.

That is because C-Store developer moved to Vertica ;)

I believe for your needs SQLIte is probably the right solution. If you r needs grow then you will need something more powerful. The simplest solution depending from the task is the best one - SQLite is very capable and if your tasks are small and don't require a lot of data probably it can...