Read Tab-delimited File Into Object [C++]

[BuckGup]

I'm trying to ingest data from a file that is tab-separated or delimited and store the data accordingly into a new object per line. So for example

Pid	Burst	Arrival	Pri
1	1	1179	72	
2	38	2659	49	
3	78	543	2	
4	91	5026	42	
5	16	6118	59

and I I have a Job Object 

class Job {
public:
    int PID;
    int burst;
    int arrival;
    int priority;
};

So I want to read per line and fill the object with the data. 

myJob.PID = 1;
myJob.burst = 1;
myJob.arrival = 1179;
myJob.Priority = 72;

Most examples I find online introduce many complex ideas that I'm not sure I would need for something like this. I think I could just read per line and use the special character "$" to determine when a line ends. Then use the special character "^I" to find the tabs and count which tab you are on to know which data in-between corresponds to the information in the object.

[Eigenvektor]

As long as the format of your file is well defined, this should be fine. E.g the order of columns is fixed and the first line is always ignored because it contains headers and data can never contain a tab character.

 

Personally, I'd rather use a format like JSON and a library that does the object mapping for me.

[wanderingfool2]

6 hours ago, BuckGup said:

Most examples I find online introduce many complex ideas that I'm not sure I would need for something like this. I think I could just read per line and use the special character "$" to determine when a line ends. Then use the special character "^I" to find the tabs and count which tab you are on to know which data in-between corresponds to the information in the object.

As with everything, it depends what you are intending...you could do what Eigenvektor said (it's actually a really good solution if you are familiar with JSON).

 

If it's a more rigid standard though (like easily human readable, or ones where a human would be filling things out) then maybe something like this

 

std::ifstream ifRead ("[file]");
std::string line;

while (std::getline(ifRead, line)) {
	std::istringstream issLine(line);
    issLine >> myJob.PID >> myJob.burst >> myJob.arrival >> myJob.priority;
    //I think treats tabs as spaces...been a bit long since doing C++ (normally I stick to C)
    //This also fails when there is a "header"...but that can be easily corrected by initial processing of "line"
    //or just by checking if it wrote the data correctly :p
}

 

[Eigenvektor]

One point I forgot to make: This solution is fine, as long as you have one file type with a fixed number of columns that maps to exactly one class. The more classes you have, the more file types you have, the more tiresome your solution gets. Because you'll have to write the same code over and over again, unless you add some form of automatic mapping of columns to fields. At that point you could just use a library that does all this for you, unless you really want the experience of writing something like this yourself.

[BuckGup]

15 hours ago, wanderingfool2 said:

As with everything, it depends what you are intending...you could do what Eigenvektor said (it's actually a really good solution if you are familiar with JSON).

 

If it's a more rigid standard though (like easily human readable, or ones where a human would be filling things out) then maybe something like this

 

std::ifstream ifRead ("[file]");
std::string line;

while (std::getline(ifRead, line)) {
	std::istringstream issLine(line);
    issLine >> myJob.PID >> myJob.burst >> myJob.arrival >> myJob.priority;
    //I think treats tabs as spaces...been a bit long since doing C++ (normally I stick to C)
    //This also fails when there is a "header"...but that can be easily corrected by initial processing of "line"
    //or just by checking if it wrote the data correctly :p
}

 

Yeah as of now it returns -325564128 instead of the int value which makes me believe it's not setting it correctly but I also need to remove the first instance of it as that is the header

[Unimportant]

@BuckGup All the above examples don't incorporate any possible error checking such as failure to extract a line from the file other then eof and file formatting errors. That's why you're getting bogus values, it's probably hitting some error and marching on. Here's a proper example:

#include <iostream>
#include <fstream>
#include <string>
#include <sstream>
#include <vector>
#include <stdexcept>

struct Job
{
    int pid;
    int burst;
    int arrival;
    int priority;
};

void
ReadJobs(std::istream& is, std::vector<Job>& jobs)
{
    auto lineNumber = 0;

    while (1)
    {
        ++lineNumber;

        //Read a line.
        std::string line;
        std::getline(is, line);

        //Check if stream still valid.
        if (!is)
        {
            //If not, check for eof.
            if (is.eof())
            {
                //Invalid stream, but eof => reached end of file.
                return;
            }
            else
            {
                //Invalid stream, but not eof => some technical error.
                throw std::runtime_error("Error retrieving line " + std::to_string(lineNumber));
            }
        }

        //Parse line.
        Job job;
        std::stringstream ss(line);
        ss >> job.pid >> job.burst >> job.arrival >> job.priority;

        //Check stringstream state still valid.
        if (!ss)
        {
            //Invalid stringstream => file format error.
            throw std::runtime_error("Formatting error on line " + std::to_string(lineNumber));
        }

        jobs.push_back(job);
    }
}

int
main()
{
    //Open file as ifstream.
    std::ifstream infile("input.txt");
    if (!infile.is_open())
    {
        std::cout << "Failed to open file!\n";
        return 1;
    }

    std::vector<Job> jobs;

    //Read jobs from file.
    try
    {
        ReadJobs(infile, jobs);
    }
    catch (std::runtime_error& e)
    {
        std::cout << e.what() << '\n';
        return 2;
    }

    //Print jobs.
    for (const auto& job : jobs)
    {
        std::cout << "Pid: " << job.pid << '\n';
        std::cout << "Burst: " << job.burst << '\n';
        std::cout << "Arrival: " << job.arrival << '\n';
        std::cout << "Priority: " << job.priority << "\n\n";
    }

    return 0;
}

It assumes there's no header in the file, so remove the:

Pid	Burst	Arrival	Pri

line from the input file.

 

If you really want that line in the input file, or perhaps even check if it's there to validate the file, then report back for modified code.

[wanderingfool2]

6 hours ago, Unimportant said:

All the above examples don't incorporate any possible error checking such as failure to extract a line from the file other then eof and file formatting errors.

Well the EOF doesn't need to really be handled if you are using std::getLine, as it will return false when it no longer is able to read new lines.

Actually, could be wrong (again been a while since working in C++ directly), by checking for EOF after you read a line and before processing the line you actually end up tossing out the last line of data.

 

Honestly, in @gabrielcarvfer's solution, you could check for failure to read by just adding in an if statement.  My quick code though was just to show the concept anyways (as I did mention the issue with headers, and checking the data).

 

#include <iostream>
#include <fstream>
#include<sstream>

class Job {
public:
    int PID;
    int burst;
    int arrival;
    int priority;
};

//Overloads operator ">>" to make C++ understand how to read the object contents from a line
std::istream& operator>>(std::istream& is, Job& job)
{
    is >> job.PID >> job.burst >> job.arrival >> job.priority;
    return is;
};

//Overloads operator "<<" to make C++ understand how to print the object contents
std::ostream& operator<<(std::ostream& os, const Job& job) 
{
    os << job.PID << "\t";
    os << job.burst << "\t";
    os << job.arrival << "\t";
    os << job.priority << "\t";
    return os;
};

int main()
{
    std::ifstream in ("../fil.txt");//make sure the file was opened.
    if (!in.is_open())
    {
        std::cout << "File was not opened" << std::endl;
        return -1;
    }
  
    std::string line;
    Job myJob;

    std::getline(in, line);// Discards header
    while (std::getline(in, line)) {
        std::istringstream is(line);
        if(!(is >> myJob))//equivalent to is >> myJob.PID >> myJob.burst >> myJob.arrival >> myJob.priority;
        	std::cerr << "Error on line: " << line; 
        std::cout << myJob << std::endl; // prints content of the object
    }
    return 0;
}

 

[wanderingfool2]

5 hours ago, gabrielcarvfer said:

That is why I've included a check to see if the file was open. In that regard, still prefer the good old C. 

haha, yea I know what you mean.  Your code was still a lot more thought out than mine (and quite frankly better)

[Unimportant]

8 hours ago, wanderingfool2 said:

Well the EOF doesn't need to really be handled if you are using std::getLine, as it will return false when it no longer is able to read new lines.

Actually, could be wrong (again been a while since working in C++ directly), by checking for EOF after you read a line and before processing the line you actually end up tossing out the last line of data.

std::getline simply returns the stream you give it as input, it does not return a bool. By testing for bool you're employing the returned stream's bool operator, which evaluates to false if the stream is in a invalid state. That does not tell you *why* it's in a invalid state. Could be eof, could be a failure due to a network drive being down, or a bad sector on your harddisk. You need to separate between eof and technical errors.

 

And no, no lines are being tossed away. This is a common error when working with c++ streams. When eof is set (or any other error) the read has failed and thus nothing was read. If you process the data first before checking eof you actually add a bogus line. The only correct way is: read, check stream state and take approriate action, process data if stream still valid.

[wanderingfool2]

10 hours ago, Unimportant said:

std::getline simply returns the stream you give it as input, it does not return a bool. By testing for bool you're employing the returned stream's bool operator, which evaluates to false if the stream is in a invalid state. That does not tell you *why* it's in a invalid state. Could be eof, could be a failure due to a network drive being down, or a bad sector on your harddisk. You need to separate between eof and technical errors.

 

And no, no lines are being tossed away. This is a common error when working with c++ streams. When eof is set (or any other error) the read has failed and thus nothing was read. If you process the data first before checking eof you actually add a bogus line. The only correct way is: read, check stream state and take approriate action, process data if stream still valid.

So admittedly, I missed that you had a if(!is) before the EOF...but the statement still stands (just doesn't occur every time)...under your exception handling the last line can be tossed away.  You can get eof with the failbit error, at which point your last result gets tossed. Really what would need to happen is checking for in.fail().  As a note as well, if you have an input file with 1 line on it "1 2 3 4", EOF does get set before the stream becomes invalid and it does get set on the first readline call.  Checking for EOF is not a reliable way of distinguishing whether "technical error" had occurred.

 

I think the point I'm trying to make though, is that the statement against

On 1/20/2021 at 9:07 AM, Unimportant said:

don't incorporate any possible error checking

...because it is decently simple to add in error checking into the loop that was provided.

    while (std::getline(in, line)) {
        std::istringstream is(line);
        if(!(is >> myJob))//equivalent to is >> myJob.PID >> myJob.burst >> myJob.arrival >> myJob.priority;
        	std::cerr << "Error on line: " << line; 
        std::cout << myJob << std::endl; // prints content of the object
    }
    if(in.fail()) {
    	//Failed stuff here
    }

The above way effectively does what your code does for "errors", so yes it's possible to incorporate error checking

 

I am well aware that getLine returns a stream, but when it's in an if statement it's effectively returning a boolean value.  The bigger thing I have is the wording you said effectively sounds like you are saying with the structure you cannot find out if errors happens, which isn't true.  As I showed, you can easily check for errors...and if you really cared about catching the exceptions, then you can easily just

in.exceptions ( std::ifstream::failbit | std::ifstream::badbit ); (or after the read loop just checking for in.fail())

 

A note with your solution as well, you shouldn't be checking for EOF then, you should be checking for if(in.fail()) throw some exception

[BuckGup]

Thank you all so much for the help. I ended up going with a solution that looks like this

    string line;
    ifstream myfile(fileDirectory);
 
    if (myfile.is_open()) {
        getline(myfile, line);
        
        while (getline(myfile, line) && !line.empty()) {
            strcpy(char_array, line.c_str());
            
             if (strchr(char_array, '-') != NULL) {
                continue;
            }

            myJob = new Job();

            tok = strtok(char_array, "\t");
            myJob->setPid(stoi(tok));
            ...
            }     
    }

 

The first getline under the if statement removes the header of the file and the strchr with the '-' is input validation for negatives, which can't exist for my case, and then I do a copy to an array so I can do a string to int for the token. 

 

 

 

[Unimportant]

3 hours ago, wanderingfool2 said:

So admittedly, I missed that you had a if(!is) before the EOF...but the statement still stands (just doesn't occur every time)...under your exception handling the last line can be tossed away.

No, it can't.

if (!is)
{
	/* We only get here if the stream state is invalid, which can be caused by:
	eofbit
     	badbit
       	failbit
       	(operations that try reading past the end file fail, and thus eofbit and failbit are set together.)
       	Whenever one or more of these error bits are set the read operation failed AND NOTHING WAS READ! (so there is nothing to toss)
        We don't know which if these bits are set yet, only that the stream is invalid, so now we need to distingish between
       	eof and genuine errors.
	*/ 
	if (is.eof())
	{
		/* if eofbit set, we reached the end of file and we're done. */
		return;
	}
	else
	{
		/* if eofbit NOT set, it means failbit and/or badbit are set, WITHOUT eofbit, which indicates a genuine error. */
		throw std::runtime_error("Error retrieving line " + std::to_string(lineNumber));
	}
}

 

3 hours ago, wanderingfool2 said:

You can get eof with the failbit error, at which point your last result gets tossed.

Operations that attempt to read past the end of file fail, and thus eofbit and failbit are always set together on eof. (but failbit can be set without eofbit on other errors)

No, the last result is not tossed. Whenever a error bit gets set the read operation failed and NOTHING WAS READ (so there is nothing to toss).

 

3 hours ago, wanderingfool2 said:

As a note as well, if you have an input file with 1 line on it "1 2 3 4", EOF does get set before the stream becomes invalid and it does get set on the first readline call. 

No it doesn't. if eofbit gets set then the stream is invalid. If any of the eofbit, failbit or badbit are set the stream is invalid. You can't have eofbit AND a valid stream. And no, it does not get set on the first readline call. The first readline call will read the single line, NOT set eof, and the stream state will be valid. The second readline call will fail with eofbit and failbit set and the stream state will be invalid.

 

3 hours ago, wanderingfool2 said:

Checking for EOF is not a reliable way of distinguishing whether "technical error" had occurred.

The check for eofbit in my code is made inside the block that only executes if the stream state is invalid in which case a eof check IS reliable, as explained in point 1.

 

 

3 hours ago, wanderingfool2 said:

..because it is decently simple to add in error checking into the loop that was provided.

    while (std::getline(in, line)) {
        std::istringstream is(line);
        if(!(is >> myJob))//equivalent to is >> myJob.PID >> myJob.burst >> myJob.arrival >> myJob.priority;
        	std::cerr << "Error on line: " << line; 
        std::cout << myJob << std::endl; // prints content of the object
    }
    if(in.fail()) {
    	//Failed stuff here
    }

The above way effectively does what your code does for "errors", so yes it's possible to incorporate error checking

No it doesn't. That code will always fail. When the loop is done because eof is reached, eofbit and failbit are set together and the "//failed stuff here" block will *always* execute, even when there were no actual errors.

[wanderingfool2]

44 minutes ago, Unimportant said:

No it doesn't. if eofbit gets set then the stream is invalid. If any of the eofbit, failbit or badbit are set the stream is invalid. You can't have eofbit AND a valid stream. And no, it does not get set on the first readline call. The first readline call will read the single line, NOT set eof, and the stream state will be valid. The second readline call will fail with eofbit and failbit set and the stream state will be invalid.

That is false.  eofbit doesn't make the stream automatically invalid

 

Even try jamming the following cout into your code...

        std::getline(is, line);

        std::cout << "EOF bit: " << is.eof() << " If statement: " << !(is) << "\n";

You will find that the eof bit is set before the stream becomes invalid.  What is happening is it's reading the first line and setting the eof bit.  The results of a single 1 2 3 4 text file would be as follows for the cout part

Quote

EOF 1 If statement: 0  //eof bit is set, while still valid and this was the first call of readline
EOF 1 If statement: 1 //doesn't become invalid until the end

 

The point being that under contrived situations you could potentially get and eof set and some other error flags set.

 

And yes, admittedly mine is wrong as well...didn't bother sticking it in for a sanity check (but again, I haven't done C++ in forever or at least nothing that needs data integrity).  A while(!in.eof()) would fix that up though.