Search results

I guess we would need to get too far into the details of your design to answer this question... I do understand what your saying about the joins, but I'm still not sure why you'd need to store intermediate results for instance. My thought is that perhaps not all 1000 columns need to be in the...

I was just referring to what you said earlier, the reason you gave that row-oriented databases were too slow... You're saying two different things. I realize that Sql Server won't allow you to create a 1000 column table, but look at your response from 7/1 and explain to me why SQL won't do...

I think the punishment is too harsh in this case...

Wow...how do you unsubscribe from a thread?

That's what I suspected. It's no wonder SQL Server wasn't performing well if you're trying to store hundreds of columns worth of intermediate results...why would you need to do that? Back to my point about good database design. How can you expect the database to function correctly when it's...

I will agree that higher frequency systems have the potential to be more profitable than lower frequency systems, due to the longer coastline. I wouldn't say they will always be more profitable, but I trade intraday myself so I agree with your basic premise.

Neither can really be "predicted" due to chaotic factors, but both can be traded profitably.

Anyone with experience using hybrid drives? I haven't but it seems like a possible solution. Not sure if they can be used in a RAID array though...

Great, thanks for the input. I don't agree with your views, but it's nice to hear how other people are processing their data.

A flat file would be stored on a disk also, wouldn't it?

You're not required to read all columns when you join. Just use two subqueries that reference only the close price if that's all you need for the calculation. Assuming the BulbSales table has 200 columns, we could re-write the example to: SELECT x.Id, AVG(y.Sales) moving_average FROM...

I think you've got a pretty good structure right there. I'd probably have a T&S table with symbol, consolidated timestamp, received timestamp, price and volume...primary key (clustered index) on symbol + consolidated timestamp or symbol + received timestamp. Then the L2 table with...

Ok, I see what you mean by sliding window now. When I do a moving calculation like that, I usually have good results with something like the query below. This is a moving average example taken from chapter 8 of a book called "Transact SQL Cookbook": SELECT x.Id, AVG(y.Sales) moving_average...

Nothing too fancy. Most of it is building indexes correctly and avoiding unnecessary duplication by using a larger number of simple tables rather than fewer complicated tables. For instance, store tick data by itself...the only columns in my price table are ticker symbol, timestamp, and price...

True, regression isn't built in, but you can write custom functions to do that kind of math. Could you clarify what you mean by sliding windows?

I completely (but politely :)) disagree. It depends to a large part on table structures, indexes, etc. Not to mention the hardware setup and the way Sql Server - or other RDBMS - is configured on that hardware. I'd love to hear more of your opinions on the thread I linked if you're interested.

Here's the thread I mentioned above, if anyone cares to discuss their feelings about SQL in detail: www.elitetrader.com/vb/showthread.php?threadid=168685

I use Sql Server as part of my setup, but I see a lot of negative reactions on ET about using this or other SQL databases. My use is for stat-arb, and I analyze (intraday) several hundred thousand possible pairs, with thousands of datapoints per pair...but I don't trade on millisecond...

Porche's have nice shiny engines, but they will be disappointing to drive if you only use the first 3 gears. :D Again, it's all about how you use it. I get great results, but I devote a good bit of time to designing the database for efficiency, and keeping in mind which types of requests...

Fair enough. I haven't had the issues with SQL that I hear other people complaining about here. I'd like to be the first to tell you that it doesn't have to be a bad idea. I've worked with companies in the past who put far too much emphasis on Excel for example, asking it to do things that...