quick C++ STL questions..

[nitro]

Quote from EliteInterest:

error C3861: 'strtok_r': identifier not found, even with argument-dependent lookup

..msft always has to spoil all the fun. hope i can get vc71 compiler to work with it.. and if not mistaken, it seems that vc71 is not c99 compliant (unless i am missing something).. argh, there are so many cool c99 functions i am reading about..

Download Microsoft Windows Services for UNIX 3.0.

Search for all references to strsep in the following link:

http://www.microsoft.com/technet/interopmigration/unix/sfu/sfuposix.mspx

nitro

 

[EliteInterest]

Quote from nitro:

Download Microsoft Windows Services for UNIX 3.0.

Search for all references to strsep in the following link:

http://www.microsoft.com/technet/interopmigration/unix/sfu/sfuposix.mspx

nitro

Cool, thanks. Seems that 3.5 is the latest.

http://www.microsoft.com/windowsserversystem/sfu/default.mspx

strsep seems to be just fine, except that you have to check for empty string; surely a negligible performance difference:

One difference between strsep and strtok_r is that if the input string contains more than one character from delimiter in a row strsep returns an empty string for each pair of characters from delimiter. This means that a program normally should test for strsep returning an empty string before processing it.

 

[nononsense]

I'm glad you solved your problem. In fact I was going to ask about the C++ compiler you are using.
It is sometimes worthwhile to have another compiler & library handy for comparison. You may sometimes be surprised. Often they even will integrate with M$ Visual, e.g. gnu or mingw, intel (all free):
http://www.thefreecountry.com/compilers/cpp.shtml

Further, what you set out to do can be done in one stroke with 'pickle' in Python. (Also look at 'shelve'.)
To get another user's view on the Python's flexibility as a go-between and including a section (5.4) on your serialization problem:
http://www.omg.org/docs/orbos/97-11-20.pdf
http://www.python.org/doc/lib/module-pickle.html
nono

Quote from EliteInterest:

... - I would like to store standard numerical types such as ints and floats into a binary file, and be able to later reload them in the fastest manner possible.

 

[EliteInterest]

Thanks, nono - I will certainly investigate Python when time is available. For now, I am going to try to keep pushing on with C/C++.


Another quick STL question if someone has a second.

Using the <i>min_element</i> algorithm is easy in the following case:

vector< double> test;
test.push_back(...); // fill vector
vector< double>::const_iterator testmin = min_element(test.begin(), test.end());

..but how do you use it in this case?

enum
{
   baropen = 1,
   barhigh,
   barlow,
   barclose,
}
typedef vector< float> LINEOUT;
vector< LINEOUT> bardata;

//push data onto LINEOUT vectors, and push them onto bardata as shown above

This is where I am stumped - I wish to, for example, find the min_element of, for example, bardata[15][barlow] to bardata[30][barlow]. The min_element algorithm expects iterators; I can use iterators for the outer vector in the following manner:

for(vector< LINEOUT>::iterator bariter = bardata.begin(); bariter != bardata.end(); ++bariter)
{
  int i = (*bariter)[barlow];  // access the inner elements using
  //array notation, but use iterator to traverse through the outer
  //vector and simply dereference the iterator
}

My question is, how can you access the inner vector using iterators? How would this apply to the min_element() function, for example? If I am not phrasing the question correctly, please let me know and I will explain my question in better detail.

Sorry if this seems kind of basic..

Thanks.

 

[EliteInterest]

Firstly, I am a bit surprised that no-one was willing to give a helping hand with my simple coding question, given the level of expertise that permeates from the vast amount of 'code philosophizing' that takes place in other threads. No big deal, I will probably have to rethink how to structure my data for efficiency and reusability as the ideas expand.

I thought this was a worthwhile note and just wanted to pass it along:

I installed VS2005 Express - the free IDE from M$, and compiled/ran my code, which reads data from text files, and performs numerous calculations on OHLC data, based on various input variables.. Guess what? Ran approx. THREE TIMES as slow as the version compiled using VS2003 (VC71). Very disappointing. Maybe I have configured the compiler or IDE wrong - but what is there to configure? I just installed it normally and loaded my code. Shocked, to say the least. Maybe it is a 'dumbed down' version, being free, but I don't see why they would do that...

Now, here is the real kicker. I am experimenting with different free/not-so-free Linux distros.. currently mucking about with RHAT EL WS v4.. (nice and stable). So I figured, what the hell, let's see how Linux performs. I loaded the code, made a few very minor adjustments that were M$ specific (not pertinent to the bulk of the calculations or loading of the STL containers, etc.), compiled, and ran. WOW! It has to be approx. 3-5 times faster than the VC71 compiled version, and therefore approximately 9-15 times faster than the VC80 compiled version.

Thought you Linux addicts might enjoy such news.

The only notable difference that I can think of: I am using the x86_64 kernel on a Xeon 64-bit chip, but I did try using VS2005Express under XP Pro x64 - still slow as hell in comparison. I am not totally sure what difference this even makes - I guess to have a clean comparison, I could try a x86 based Linux kernel.. but does that even matter? My code is pretty standard stuff, using built-in types: ints, floats, double, etc.. no 64-bit precision going on here.. I am trying to make a case for Linux over Windows, speed of execution being a critical factor.. I still prefer Windows in some ways - GUI is clean, fast, and simple to use, but Linux might have some compelling advantages.. btw: I used KDevelop to 'port' (whatever that means - just changed a few lines) the source from Windows.. I'll keep toying with RHAT and try some others (commerical Novell or free SUSE come to mind)..

 

[rufus_4000]

Quote from EliteInterest:

error C3861: 'strtok_r': identifier not found, even with argument-dependent lookup

..msft always has to spoil all the fun. hope i can get vc71 compiler to work with it.. and if not mistaken, it seems that vc71 is not c99 compliant (unless i am missing something).. argh, there are so many cool c99 functions i am reading about..

Yuck ... here is a simple strtok_r implementation (in C), that works pretty much in any platform that I have ported my system (*nix, OpenBSD, Windows, OS X, BeOS, etc). rather than downloading and installing more packages.

I code pretty much in C (not C++), since I can get pretty much any performance tuning out of my code. Yes, I wrote a portion of my own code in Assembler (very small portion, basically to handle my own job scheduler and thread sychronization). There are some features in C++ that I like, but I have a historical dislike for templates (I was adamant to keep it out of ANSI C++ V3). The key trade off is that my code base is getting rather large, so I digress.

As a point of comparison, I have a standalone version of my system (used exclusively for backtesting), I run through a 600M file (daily ticks for 4 products) in about 5 mins, it is just disk I/O, that's all.

 

[rufus_4000]

Here is the code piece ...

 

[cmaxb]

Quote from EliteInterest:

Firstly, I am a bit surprised that no-one was willing to give a helping hand with my simple coding question, given the level of expertise that permeates from the vast amount of 'code philosophizing' that takes place in other threads. No big deal, I will probably have to rethink how to structure my data for efficiency and reusability as the ideas expand.

I thought this was a worthwhile note and just wanted to pass it along:

I installed VS2005 Express - the free IDE from M$, and compiled/ran my code, which reads data from text files, and performs numerous calculations on OHLC data, based on various input variables.. Guess what? Ran approx. THREE TIMES as slow as the version compiled using VS2003 (VC71). Very disappointing. Maybe I have configured the compiler or IDE wrong - but what is there to configure? I just installed it normally and loaded my code. Shocked, to say the least. Maybe it is a 'dumbed down' version, being free, but I don't see why they would do that...

Now, here is the real kicker. I am experimenting with different free/not-so-free Linux distros.. currently mucking about with RHAT EL WS v4.. (nice and stable). So I figured, what the hell, let's see how Linux performs. I loaded the code, made a few very minor adjustments that were M$ specific (not pertinent to the bulk of the calculations or loading of the STL containers, etc.), compiled, and ran. WOW! It has to be approx. 3-5 times faster than the VC71 compiled version, and therefore approximately 9-15 times faster than the VC80 compiled version.

Thought you Linux addicts might enjoy such news.

The only notable difference that I can think of: I am using the x86_64 kernel on a Xeon 64-bit chip, but I did try using VS2005Express under XP Pro x64 - still slow as hell in comparison. I am not totally sure what difference this even makes - I guess to have a clean comparison, I could try a x86 based Linux kernel.. but does that even matter? My code is pretty standard stuff, using built-in types: ints, floats, double, etc.. no 64-bit precision going on here.. I am trying to make a case for Linux over Windows, speed of execution being a critical factor.. I still prefer Windows in some ways - GUI is clean, fast, and simple to use, but Linux might have some compelling advantages.. btw: I used KDevelop to 'port' (whatever that means - just changed a few lines) the source from Windows.. I'll keep toying with RHAT and try some others (commerical Novell or free SUSE come to mind)..

Also had good results from using Intel's compiler icc, but they say there shouldn't be much difference. Compiling sure feels faster.

 

[Spelunk]

There is no simple way of using iterators on a custom object like you are creating. A much more useful way to implement this is to organize your data into vectors of Open, High, Low, etc rather than by bars. This will be much faster, flexible and will allow you to use the standard vector functions.

Quote from EliteInterest:

Thanks, nono - I will certainly investigate Python when time is available. For now, I am going to try to keep pushing on with C/C++.


Another quick STL question if someone has a second.

Using the <i>min_element</i> algorithm is easy in the following case:

vector< double> test;
test.push_back(...); // fill vector
vector< double>::const_iterator testmin = min_element(test.begin(), test.end());

..but how do you use it in this case?

enum
{
   baropen = 1,
   barhigh,
   barlow,
   barclose,
}
typedef vector< float> LINEOUT;
vector< LINEOUT> bardata;

//push data onto LINEOUT vectors, and push them onto bardata as shown above

This is where I am stumped - I wish to, for example, find the min_element of, for example, bardata[15][barlow] to bardata[30][barlow]. The min_element algorithm expects iterators; I can use iterators for the outer vector in the following manner:

for(vector< LINEOUT>::iterator bariter = bardata.begin(); bariter != bardata.end(); ++bariter)
{
  int i = (*bariter)[barlow];  // access the inner elements using
  //array notation, but use iterator to traverse through the outer
  //vector and simply dereference the iterator
}

My question is, how can you access the inner vector using iterators? How would this apply to the min_element() function, for example? If I am not phrasing the question correctly, please let me know and I will explain my question in better detail.

Sorry if this seems kind of basic..

Thanks.

 

[EliteInterest]

Quote from rufus_4000:

Here is the code piece ...

Thanks Rufus. I will try it out on VS.