Custom Chart Software Thread

E

EliteInterest

Ok, I am starting a new thread for my Qt 4 based (http://www.trolltech.com/products/qt/index.html) chart project. The old thread was becoming a bit cluttered with extra-wide code fragments as well as unnecessarily redundant progress screenshots. Ultimately, the software will not just be for charting, but to integrate trading in a way specific to my personality as well. This actually is the main motivation behind the project.

I have added the ability to 'dock' or 'undock' a given chart from the Main Window. I understand that many users like being able to have their charts float freely from the main window - possibly for an odd multi-monitor configuration (i.e. 1024x768 + 1280x1024). eSignal does this, Ensign does not. However, I do not think there is a way to add a widget to the titlebar as eSignal does - at least using Qt methods. It would be nice to have a little button next to the 'minimize' button that docks or undocks the chart.

I added Global Chart Properties - for now there are (12) color settings only. I will try to add templates so users can have fancy custom color groups. Crosshairs have a transparency slider - right now it modifies the 'alpha' component of the crosshairs color from 0 to 255, I might limit the lower bound to, say, 63. (0 is totally blank and is therefore pretty much useless). The properties will eventually include (no specific order):
Splitter Width (right now 2 pixels)
Crosshairs toggle
Crosshairs type (full, small)
Right pixel spacing (between price area and edge of last bar/candle) (0-30?)
Bar/Candle/Line default
Volume toggle default
Default spacing (0-99?)
Default font
Grid type - dot, solid line, dashed line
Grid toggle
Date turn grid toggle
Session open/close grid toggle
Probably more.

I have explored the possibility of using a custom frameless window, and adding widgets for minimize/maximize(restore)/close. This would make it easy to add a fourth widget for the 'dock/undock from MainWindow' methods, but I was playing with it and it's a pain. What happens when the window is Maximized? The titlebar has to be shown again, unless the maximize routine is customized to fill the screen manually. Additionally, minimizing must have the titlebar so the user can see the minimized window either on the bottom left corner of the desktop (undocked) or on the bottom left corner of the main window (docked). This would be a really cool feature though, as it would allow custom utilization of space of the top area where the titlebar normally is located. Additionally, the width of the frame could be customized, in a variable style. This one is on the backburner, however, due to the complications involved, and the fact that other issues are far more important at this point (such as consideration of network interfacing, symbol management, proper combination of real-time data with historical data/backfill).

I have a nice collection of files from CME's ftp server. The format of the files makes it very easy to split each ASCII line up and create a trade-by-trade custom binary useful for creating literally any type of dataset. The chart types available will be: minute (0-720? Not sure why anyone needs more than 720 minute [1/2 of a day] charts), daily, weekly, monthly, yearly, tick, volume, range?(I don't use range charts but the formula seems simple). I wonder if 'second' charts are useful - I do not use them. But, it is feasible to use the CME files (and/or a real-time data feed) to create, for example, 15 second data. Currently, I use tick and volume charts without synchronizing to the session open - it would be nice to at least make this a user-option. By synchronizing to the session open, consistency of the dataset is ensured - otherwise the whole dataset will be offset variably dependent on the first available data, and whether the source dataset is actually complete/accurate. I don't have the full historical tick dataset for CME products, so having this feature at least available seems preferable. I have already worked out the formulae for calculating tick and volume charts - it's not that difficult - just have to incorporate the synchronization, and consider the best way to handle data across multiple files.. right now I import the CME data so that each session is stored in a separate file, splitting the contracts up. There are actually some fields that CME uses that I am a bit confused about - maybe I'll ask when the time is right. Most is self-explanatory.

Moving on, right now I am brainstorming the best way to convert a given time-based dataset to a higher dataset. I have updated the ASCII import wizard to compensate for timezone discrepancies as well as 'late bar' data (i.e. the dataset represents the data from 9:30.00 to 9:30.999 as 9:31 - I prefer 'early bar' and this is what the program will use). The timezone adjustment merely moves all data to 'local' time - 'local time' will be a global setting or possibly simply read from the local clock. I think the global setting is preferable so as to maintain consistency should the computer's clock be inadvertently changed temporarily. This does beg the question - what if the user moves from Florida to Arizona and their data is off by a few hours? Anyway, I digress - that is not too terribly important as of yet. The time adjustment was done as follows - convert the YYYY,MM,DD,HR,MIN variables to a QDateTime, then use that variable to make the adjustment, then convert back to YYYY,MM,DD,HR,MIN. The reason is that when the data is used in the chart, I prefer to use the individual variables for performance reasons. The only overhead is that which I have coded specific to the charting component - i.e. in the date and time algorithms. I would rather have a slight performance hit during ASCII conversion (while making my life easier using Qt's very convenient QDateTime type), but keep performance snappy while charts are in use.

Anyway, back to the issue at hand - let's say I want to convert 1-minute data to 3-minute data, and synchronize with the 9:30 open. I usually just scribble some numbers on a piece of paper to come up with a formula, but for some reason this one has me a bit wrinkled right now. Maybe it's still early in the day, but it should be easy. I suspect that using QDateTime will, again, make things a bit easier for this conversion, particularly when considering times that cross over midnight due to the (for example) 9:30 synchronization over odd timeframes. Taking the example numbers, 9:30 means 9*60+30 = 570 minutes since midnight. 570/3 = 190. 570%3 = 0. Maybe this has something to do with it. Something like: 12:00 + (570%3) to 12:00+3 + (570%3). I think that's enough to get me started, I'll write some loops using that idea and see if it works. It's probably wrong, but it will get the ball rolling on this issue.

That's enough for now.
 
If for some reason, someone out there was interested in creating constant volume bars, maybe this code fragment could help you.

I am about 90% sure that it is 99%-100% right. I've read that many software vendors do not create their volume bars right. I'll definitely come back to this to make sure it is 100%. It looks much nicer in an IDE with proper tabs and w/out the comments but I'd prefer it to look nice on the forum - if anyone actually wants to look into it, they can copy and paste it to make the aesthetic changes for themselves.

Code: 
for (int i = 0; i < list.size(); ++i) {
 QString temppath = dirpath;
//temppath is the path of the folder with the raw CME files for this contract
 temppath.append(list.at(i));
//add the name of the next file from the list<string> (QList<QString>)
//the list holds all files that will be processed into this dataset
 QFile filea = temppath;
 filea.open(QIODevice::ReadOnly);		
 QByteArray bytearray = filea.readAll();
//assign to a bytearray to dramatically increase speed
//QByteArray appears to have a limit of size_of(int)(/2?) or (on most computers) gigabytes.
 QDataStream in(bytearray);
 double decimlocat = 100.0;
// decimlocat will be 1 X 10^(decimallocator), decimallocator is from raw CME file
 newvolumebar = true;
 volumecount=0;
 while (!in.atEnd()) {
  in >> yy >> mm >> dd >> hr >> mn >> sec >> price >> volume 
     >> sessionid >> askbidtrade >> cancelcorrect >> opencloseboth >> decimallocator;
//load in raw trade-by-trade data one line at a time
  if (volume > 0) {
//if the raw data has a line with non-zero volume
   if (newvolumebar) {
//if we are starting a new volume bar
    datetime = QDateTime(QDate(yy,mm,dd), QTime(hr,mn,sec));
    datetime = datetime.addSecs(timezonedifference);
//convert to QDateTime and offset for timezone difference
    yyout = datetime.date().year();
    mmout = datetime.date().month();
    ddout = datetime.date().day();
    hrout = datetime.time().hour();
    mnout = datetime.time().minute();
//put the updated values into the date/time holder variables
    openout = (double)(price/decimlocat);
    highout = (double)(price/decimlocat);
    lowout = (double)(price/decimlocat);
    closeout = (double)(price/decimlocat);
//then assign the current trade price to the variable holders
   }
   else {
//otherwise if we are updating an existing bar, update high/low/close
    if ((double)(price/decimlocat) > highout)
     highout = (double)(price/decimlocat);
    else if ((double)(price/decimlocat) < lowout)
     lowout = (double)(price/decimlocat);
    closeout = (double)(price/decimlocat);
   }
    volumecount += volume;
//always update the volumecount for this bar by the # of new contracts
    if (volumecount < volumebarsize)
//if the updated count is less than the bar size, simply break and start over
     newvolumebar = false;
    else if (volumecount > volumebarsize) {
//otherwise if the updated count is greater, print a bar with all price/date/time info
     out << k << yyout << mmout << ddout << hrout << mnout 
         << openout << highout << lowout << closeout << volumebarsize;
//'out' is the output datastream not shown in this code fragment
     ++k;
    for (int j = 1; j < volumecount/volumebarsize; ++j) {
//and if someone placed big trades that exceeded the barsize, print those bars
     out << k << yyout << mmout << ddout << hrout << mnout 
         << (double)(price/decimlocat) << (double)(price/decimlocat)
         << (double)(price/decimlocat) << (double)(price/decimlocat) << volumebarsize;
     ++k;
    }
    volumecount = volumecount%volumebarsize;
//and if there is any leftover from all that mess, assign that to the volume count
    if (volumecount)
     openout = (double)(price/decimlocat);
     highout = (double)(price/decimlocat);
     lowout = (double)(price/decimlocat);
     closeout = (double)(price/decimlocat);
     newvolumebar = false;
//and if the result of that operation leads to any leftover volume,
//udpate the price to the last trade but keep the date on the next pass 
    else
//otherwise if no volume is left over, start a new bar next pass
     newvolumebar = true;
    }
    else {
//else if the volumecount happens to EQUAL the barsize, print a bar,
//reset the volume count, and start a new bar next pass
     out << k << yyout << mmout << ddout << hrout << mnout << openout 
         << highout << lowout << closeout << volumebarsize;
     ++k;
     volumecount = 0;
     newvolumebar = true;
    }
   }
  }
  if (!newvolumebar) {
//once this file is completed, print the last bar (if any)
//with the volume count that will be less than the bar size
   out << k << yyout << mmout << ddout << hrout << mnout << openout 
       << highout << lowout << closeout << volumecount;
   ++k;
 }
 filea.close();
}
Here's what a 77-Volume chart looks like - haven't update the grid algorithm yet, which will probably simply use fixed count (i.e. every 10, 20 or 30 bars offset from the session open) instead of fixed time spacing. I think it might be 'wise' to add provisions for 'seconds' at the least, which might come in handy when using low-count volume charts like 77.

<img src="http://i2.photobucket.com/albums/y12/e647g9i/chart-20060326-253pm.png"></img><br>
Here's what the color settings dialog currently looks like. I have to get rid of that Global stuff, so the user can have more control over the program.

<img src="http://i2.photobucket.com/albums/y12/e647g9i/chart-20060326-240pm.png"></img><br>

For now, the algorithm is designed to start a new volume bar at the open of the Globex session, and does not reset at the Day session open. Whether there should be options to have full control over this, I am not sure.
 
Quote from shortskirt:

Sounds good. Do you have plans to make this an open source project?
More...
There is really only one scenario under which I will consider making this Open Source (it is GPL, non-commercial, but private for now - if I create a good idea that I want to commercialize, I will have to scrap this work and start from scratch after buying a 'dear' license - look up Trolltech's pricing info and you will understand :)):

If there is considerable interest in the community for making significant contributions to the project, then I will strongly consider it. Obviously, those interested will need the skill, experience (preferably), and (perhaps most importantly) the time to contribute. They will need to have experience with Qt, preferably version 4. Furthermore, it will be ideal if they have strengths different than my own. It will be nice to have people on board that have experience with design patterns in large scale programs, to ensure for example that the program is robust. I have used exactly zero exceptions. I do add bounds checking but ideally the program will be completely fault-tolerant to rogue data and any imaginable error. Additionally, someone will need to be well-versed in network programming. I can learn the concepts, but if I can focus on managing the project, designing dialogs, charts, and layouts, it would be easier to move things forward expeditiously. I envision interfacing with the most common providers for data and/or trading: IB, IQFeed, maybe eSignal, etc.

Sadly, I fear that anyone that can contribute does not have the time or inclination (i.e. they already have their own agenda). Maybe I am wrong; let me know. PM me if you would like or just post here - I can easily create a SourceForge page, and design diagrams with huge amounts of comments detailing exactly how the program works so far, what needs to be done, etc. It is a cool idea - but can only happen if motivated talent emerges.
 
A comment about comments and coding style. It might seem a bit odd, but there's too many of them buried in the above code fragment and that makes it harder to read. It's preferable to put the bulk of comments in a comment block preceding the function/method.

For C or C++ consider using doxygen

http://www.stack.nl/~dimitri/doxygen/

It is a great tool for code documentation and produces nicely formatted HTML something like javadoc but with more features.
 
Quote from dcraig:

A comment about comments and coding style. It might seem a bit odd, but there's too many of them buried in the above code fragment and that makes it harder to read. It's preferable to put the bulk of comments in a comment block preceding the function/method.

For C or C++ consider using doxygen

http://www.stack.nl/~dimitri/doxygen/

It is a great tool for code documentation and produces nicely formatted HTML something like javadoc but with more features.
More...

Actually, I agree (about the readability of the comments in the above fragment) - I only added the comments in hopes that a reader that is interested in constant volume bars might find it easy to follow line-by-line how the small routine works - guess maybe it was a bad idea. :) So far, I am zero for two on proper code comment etiquette, hopefully I'll get it right next time.

I appreciate the link and will look into it.
 
Quote from EliteInterest:

Actually, I agree (about the readability of the comments in the above fragment) - I only added the comments in hopes that a reader that is interested in constant volume bars might find it easy to follow line-by-line how the small routine works - guess maybe it was a bad idea. :) So far, I am zero for two on proper code comment etiquette, hopefully I'll get it right next time.

I appreciate the link and will look into it.
More...

While on the topic of tools, the other type of tool that I think is invaluable for C and C++ code are what are known as 'automatic runtime debuggers'. Commercial products include Purify and Insure++ which are quite expensive, but worth every penny in commercial software development. Somewhat more limited, but still very useful is valgrind which is free open source

http://valgrind.org/
 
Thanks dcraig.

Changing gears, I normally don't post charts here but I thought this one was kind of interesting. This morning, the upper channel on NQ held, so we'll see how the other futures react - I think a reversal now of some kind is likely. ER2 obviously has been a monster lately so we'll see if it can be pulled down as well. Risk could be defined by placing a stop just above the upper line. Obviously the best r:r entry came and went, oh well. (one would be into a winner already with a stop at breakeven). That's always been curious to me - even if you were short from the top, technically you still want the stop above the channel line (as the trade is proven wrong above the line, not at it), but by having the stop at breakeven or better, your monetary risk is zero (ignoring unforeseen technical or systemic problems). Psychologically the break-even stop seems preferable, but I've been stopped out of many good trades by using that stop instead of keeping the original stop in - which properly coincided with the exact point where the trade is proven wrong, below/above support/resistance). I had the upper line drawn from last week and pulled up the chart this morning..

<img src="http://i2.photobucket.com/albums/y12/e647g9i/20060330-NQ-60min.png"</img><br>

I have been hacking away incessantly on the chart project, and will add more on that front soon.
 
You must log in or register to reply here.
Back
Top