current quotes to excel spreadsheet

I've been experimenting lately with using eSignal's DDE connection to excel only on about 60 stocks and 13 columns of data. I find that during "fast" markets, such as the opening, that eSignal doesn't update data (quotes and charts) though the data is being streamed in. The excel sheet continues to update while eSignal does not until the market slows. A big problem with excel is that updating a lot of cells can be very CPU intensive. I think this is my bottleneck.

Has anyone had experience with solving DDE performance issues? An alternative is to use activeX. Has anyone benchmarked differences in processing speed between DDE and activeX?
 
Quote from Bernoulli:

I've been experimenting lately with using eSignal's DDE connection to excel only on about 60 stocks and 13 columns of data. I find that during "fast" markets, such as the opening, that eSignal doesn't update data (quotes and charts) though the data is being streamed in. The excel sheet continues to update while eSignal does not until the market slows. A big problem with excel is that updating a lot of cells can be very CPU intensive. I think this is my bottleneck.

Has anyone had experience with solving DDE performance issues? An alternative is to use activeX. Has anyone benchmarked differences in processing speed between DDE and activeX?
More...

Hi Bernoulli...

I have given up on eSig because their Charting Platform is not multi threaded... and they are so costly with their symbols for DDE... i am now using NeoTicker and DTN and find its much better but not perfect...

the DTN DDE SpreadSheet does not initially load as quick as eSig's but works the same once it does load and is receiving the DDE pushed data...

The kicker is that NeoTicker even has its own RTD excel format and you can set the update throttle on that in NT...

because you can set the throttle and since RTD is more efficient than DDE to begin with it is much less costly on CPU...

so i am trying to move that way now since DDE is so CPU intensive on either platform...

also with any RTD format you can have more symbols updating and DTN does not charge that much for an extra 500 symbols... and they give you 1300 as part of their base plan...

one caveat - NeoTicker demo is a little restrictive so let them know what you want to test if you do try it... if not let me know what you find out anywhere else with any other Excel tool solution

:)

cj...
 
EdgeHunter-

Thanks for the response -- I'll look into your recommendations. I agree wholeheartedly with your multithreaded assessment of eSignal. I consider that a major liability for their package.
 
Bernoulli,

Excel dde performance could be affected by a combination of factors - memory, processor, hard drive; but also the quote vendor. I think the issue here is the vendor’s bandwidth (I think that’s what I’m referring to), ie the capacity of their systems to deliver data. This varies between vendors.

One of my sheets with dde’s takes around 40 seconds to load with my current vendor (VisualCharts). The same sheet with the previous vendor took around 5 minutes (still using the same pc).

This is a continuos problem for Reuters, for example, but is two-sided – ever-expanding information and data from the exchanges and news services which causes bottlenecks in vendor’s systems. This trickles down to the end-users causing slow-downs, delays, freezes or crashes (my friend’s Reuters system was a nightmare). If Reuters are having difficulties, think how the lesser operators are faring.

What I can’t understand is why the data can’t be filtered or blocked, eg why is all data delivered – fx, bonds, equities, etc - when one is only interested in one market? So although you don’t subscribe to fx, it will still be sent (although not accessible if not subscribed). This seems to the case with most – if not all – vendors.

Is what you say correct – occasionally Excel updates before eSignal?

Can an Excel sheet be converted to Active-X or would one need to start from scratch?

One should also consider software/hardware developments. My needs have changed little from DOS, Lotus 1-2-3 on a single floppy but now I’m on XP Pro 64, dual CPU, 2 gig memory. And it isn’t perfect.

Grant.
 
DDE is an older technology, although it is nearly a standard for market data applications. DDE is mainly limited to the number of "conversations". The definition of a "conversation" varies by provider and how they wrote their DDE server, but in the case of DTN IQFeed a conversation is a field of data for each symbol.
DTN.IQ, DTN IQFeed, and DTN ProphetX Active Trader Edition all support DDE natively and include the functionality in the basic service at no extra charge.

There are also a few 3rd party apps who's sole purpose is to feed Excel more efficiently than using the older DDE technology. Some of the products that work with DTN IQFeed include QuoteIN or XLQ. They write directly to the feed, then send to Excel using RTD (the newer and more efficient technology).

While many of our customers use DDE without any issue (we have seen spreadsheets with 500+ symbols), others have moved to the faster and more flexible RTD via one of these 3rd parties. If you are just getting started, it would make sense to go out the door using RTD.

the reference document for using DDE with DTN.IQ/IQFeed is at http://www.dtniq.com/product/help/v2.4/dde/
 
Grant-

This is what I think happens: ESignal consists of two main apps -- the data manager and the GUI. The data manager contains the DDE server and also puts out the live data to the GUI via ports. The result is that the live data is streaming to the data manager, which in turn feeds the DDE server and the port that the GUI is listening to. My excel DDE client is able to process the live data on a timely basis (I confirm this via another feed), but the eSignal GUI app freezes as it is unable to update the screens at the rate the data is coming in to it. When the market slows the GUI will catch up. I don't have any evidence that there is a bottleneck in the data reaching me, unless my two feeds are equally slow.

I don't know if all data is sent to the data manager, which would do the filtering, or if the server does the filtering. I assumed the latter. I guess someone could listen to the incoming port and see what data comes through.

A spreadsheet cannot be converted to activeX without a rewrite. The data would be processed via VBA code rather than with worksheet formulas. The reason for doing so would be to limit the amount of updating to the excel worksheet, which is very CPU intensive. If I can reduce CPU usage then the eSignal GUI can get more. I am going to lower the excel priority to see if that frees up cycles for the GUI app.

I am pretty up to date on CPU, memory and video card, so any enhancements there may not have much impact. The biggest impact would be gained by limiting the cycles that excel uses.
 
Quote from iqfeed:

DDE is an older technology, although it is nearly a standard for market data applications. DDE is mainly limited to the number of "conversations". The definition of a "conversation" varies by provider and how they wrote their DDE server, but in the case of DTN IQFeed a conversation is a field of data for each symbol.
DTN.IQ, DTN IQFeed, and DTN ProphetX Active Trader Edition all support DDE natively and include the functionality in the basic service at no extra charge.

There are also a few 3rd party apps who's sole purpose is to feed Excel more efficiently than using the older DDE technology. Some of the products that work with DTN IQFeed include QuoteIN or XLQ. They write directly to the feed, then send to Excel using RTD (the newer and more efficient technology).

While many of our customers use DDE without any issue (we have seen spreadsheets with 500+ symbols), others have moved to the faster and more flexible RTD via one of these 3rd parties. If you are just getting started, it would make sense to go out the door using RTD.

the reference document for using DDE with DTN.IQ/IQFeed is at http://www.dtniq.com/product/help/v2.4/dde/
More...

Thanks for the info. I have always hated the limitations of DDE, but eSignal is still in the early '90s with their technology (well, not 16 bit anymore, at least) so I used it. I'll definitely look into RTD to see what that does.
 
Iqfeed,

When you say “500+ symbols” would this be, eg stock XYZ last price? If so, for comparisons, one of my sheets contains roughly 4,500, that is DAX, STX, SMI options – 50 strikes, call and put premiums, times of last trade, 8 expiry months.

But to broaden the discussion, it isn’t the number of dde’s on a sheet but whether efficient (instantaneous?) calculation/manipulation is possible. My limited knowledge produces less efficient modes while an expert in VBA (or other language) would improve this tremendously (I’ve seen it).

At the moment this isn’t a problem. My priority is best described as reducing clutter. For example, to calculate the implieds for DAX, STX and SMI, each would have a separate sheet. Now these have been merged into a single sheet. Slower calculation time but simpler and tidier than working with 3 independent sheets.

I have looked at QuoteIN and XLQ and they are certainly impressive. Unfortunately, but for presumably sound commercial reasons, they only support the popular feeds like DTN.

Bernoulli,

If you have a broadband connection, could this be factor? Mine is 1 mb but, as an extreme, a neighbour told me his son-in-law has 40 mb’s.

How much data/calculation is on one of your sheets that causes the problem?

Grant.
 
Quote from Grant:


Bernoulli,

If you have a broadband connection, could this be factor? Mine is 1 mb but, as an extreme, a neighbour told me his son-in-law has 40 mb’s.

How much data/calculation is on one of your sheets that causes the problem?

Grant.
More...

I get about 3.5 mb/s. The problem is not with the data getting to me. The problem is that excel is too cpu intensive and crowds out my esignal app and everything else during periods of high throughput. I need solutions to reduce the load on the CPU. One way to reduce the load is too internalize the computing I am doing in code rather than via worksheet functions. Also I suspect a lot of cycles are used in updating the data points, which probably generates a recalculation with every price change, so a way to limit recalculations and yet update the worksheet fairly often is another solution. There are a lot of different routes to take.
 
You must log in or register to reply here.
Back
Top