Reading fixed columns of float data from text file with C#

B

bungrider

Hi guys,

I am pulling my hair out trying to find a FAST method for reading text data from a text file with C#. In C, there was fscanf, but I cannot find a corresponding function built into C#.

My data is a tab-separated text file with 5 columns (see below for exactly what it looks like). The data is all float, and all in 5 columns.

10.3 0 67.2 29.7 12.8
3.7 3.0 56.4 23.3 34.2
(the 5 columns continue downwards for 20mb of data)
**Here the data also has \r and \n (carriage returns and newline characters), but I can easily remove those, or keep them in, whichever's best...


Here's what I have done so far:

StreamReader object to read text file into string.
String.Split to break text file's data columns apart (they are tab-delimited).
Foreach operation in order to interate thru each string and convert to Single float data type.

Now that I have the tab-delimited floats, I would like to assign each column's data into a different float array, but I cannot find a quick way to do this...?

Any advice?

Thanks in advance,
-b


Remember, SPEED is the issue here, since my text files are BIG (~20mb). I tried to use Regex.Matches to match the data to a pattern, but it is WAY too slow and eats tons of RAM (in my hands, anyway :D ).


PS - Anyone who posts "I don't see what reading text files has to do with trading" can enjoy a pre-emptive "piss off" from me. :D
 
Bungrider,

What did you end up choosing as a method for storing data?

I have been stuck on this for too long. I need to pick a direction and get started. At this rate it will only take me 200 years before I code anything useful. Using advice I only half understand from other threads, I am thinking about putting the data in an arraylist and serializing it.

What is your approach?
 
brokershopping,

sorry for my delay getting back to you, but i had to actually code it & make sure it worked...

like i said above, i did this first:

StreamReader object to read text file into string.
String.Split first by Newline character \n to break text file's data columns apart (they are tab-delimited) and load each line into string[] arrays....i.e. string [] split = myString.Split(\n)...myNewString = split.ToString()
i used a standard for loop to interate thru each line
Then i used a nested for loop inside the first one to split each of those lines into the 5 float objects with myNewString.Split(\t)
i assigned each value to a 2D array, where array[r,c] was basically [\n,\t]


voila!
 
Do you then save the array by serializing it? Is that the correct term?

If the array gets too big will you break it into chunks?

(I am woefully ignorant, oh yes)
 
IMHO, I think there is so much more hype than promise in the next generation languages (Java, C#). Personally I find Im writing most of my code these days in python (and to a lesser extent ruby). What do you see as the advantages for C# bungrider? :confused:
 
It has been designed by the architect of Delphi so that it should be promising: I said it should because I still never looked at C# so i can't tell really :D

Quote from StillTrading:

IMHO, I think there is so much more hype than promise in the next generation languages (Java, C#). Personally I find Im writing most of my code these days in python (and to a lesser extent ruby). What do you see as the advantages for C# bungrider? :confused:
More...
 
Quote from harrytrader:

It has been designed by the architect of Delphi so that it should be promising: I said it should because I still never looked at C# so i can't tell really :D
More...

http://windows.oreilly.com/news/hejlsberg_0800.html
Deep Inside C#: An Interview with Microsoft Chief Architect Anders Hejlsberg
by John Osborn
08/01/2000
In July, O'Reilly editor John Osborn attended the Microsoft Professional Developer's Conference where he conducted the following interview with Anders Hejlsberg, Distinguished Engineer and Chief C# Language Architect about Microsoft's .Net framework and the C# programming language. Anders Hejlsberg is also known for having designed Turbo Pascal, one of the first languages available for PCs. Anders licensed Turbo Pascal to Borland and later led the team that created Delphi, a highly successful visual design tool for building client server applications. Also in attendance at the interview were Tony Goodhew, Microsoft C# product manager, and O'Reilly Windows editor Ron Petrusha.
 
The only advantage is industry acceptance/support and the
synergy that comes from this.

C++ dominated for years and I cant think of a worse OO language.

Management doesn't care about an obscure language that
is AWESOME, if everyone else isn't using it. Lemmings all of them.


peace

axeman



Quote from StillTrading:

IMHO, I think there is so much more hype than promise in the next generation languages (Java, C#). Personally I find Im writing most of my code these days in python (and to a lesser extent ruby). What do you see as the advantages for C# bungrider? :confused:
More...
 
You must log in or register to reply here.
Back
Top