Importing CSV File in to Matlab... need to skip remove lines

A place to discuss the implementation and style of computer programs.

Moderators: phlip, Moderators General, Prelates

brain_ofj
Posts: 48
Joined: Thu Feb 09, 2012 3:32 am UTC

Importing CSV File in to Matlab... need to skip remove lines

Postby brain_ofj » Mon Feb 10, 2014 2:51 am UTC

I have read a couple of entries about skipping header information when importing CSV files into Matlab. While I don't fully understand them yet, I know that I'll also need to skip lines with text interspersed in with my data as well. How would I import a CSV file that has a Header but also includes lines of text between "blocks" of data? For instance, in the attached file, Lines 1-45 can be considered the "Header" and are easily skipped over. Lines 46-74 contain the actual data... skipping Lines 75-76... and then Lines 77-105 contain the next "block" of data. This pattern repeats and, depending on the length of file to be handled, could repeat a couple of thousand times (meaning could have around 2K "blocks" of data). I would like to be able to import the data blocks only so that I can do math (summing, averaging, max and min values) for specific "blocks" of data... I could do this in Excel, but I don't know how to automate the process without using Matlab. Any suggestions would be appreciated. Thank you.

In another question... is this really the best way to do this?

https://www.dropbox.com/s/jc9xkar50qpz5w9/E04R_F-53-57.csv

User avatar
roflwaffle
Posts: 360
Joined: Wed Jul 01, 2009 6:25 am UTC

Re: Importing CSV File in to Matlab... need to skip remove l

Postby roflwaffle » Tue Feb 11, 2014 6:08 am UTC

Sed + Batch or Bash depending on where you are is what pops into my head first.

brain_ofj
Posts: 48
Joined: Thu Feb 09, 2012 3:32 am UTC

Re: Importing CSV File in to Matlab... need to skip remove l

Postby brain_ofj » Tue Feb 11, 2014 10:22 pm UTC

Ended up doing this

Code: Select all

fid = fopen(filename);  %opens file of name

r = 1;                  %starts at 1
tline = fgets(fid);     %reads first line of file of name

while(ischar(tline))    %while the first line of the file contains a character
   
    if(isstrprop(tline(1), 'digit') || (tline(1) == 'B' && tline(2) == ',')) %if the first line of the file is a number or a 'B' or a ','... do...
        tline = strrep(tline, 'B', '0');    %replace all "B's" with zeros for later mathmatical manipulation of data
        %disp(r)
        eval(['x = {' tline '}']);          %this line takes the string (each line) of file and places in a cell 'x'
        M(r,:) = x;                         %makes a row of the matrix M with x's
        r = r + 1;                          %keeping track of what row in matrix M we are in
       
    end
    tline = fgets(fid);                     %error case
end

fclose(fid);                                %closing file

end


But I admit, I got some help with the "eval" line and the while loop.


Return to “Coding”

Who is online

Users browsing this forum: No registered users and 10 guests