macOS fgets redux

[farmerdoug]

We discussed how fgets works or not depending on the end of line used in a file. (right?)

So I wrote code using fgets and it worked and then it didn't. It worked when I ran the code from a terminal window in X11 but failed in Xcode. I will be running this on X11 so I'm not going to worry about it but I'm curious if anybody can explain this.

 

[KnightWRX]

Might want to post this code up. fgets() reads either until it's read n-1 characters (n being the 2nd parameter sent to the function) or until it encounters the newline character.

If it worked under an xterm window and not Xcode, sounds more like a pointer issue. Are you sure you properly alloc'ed the buffer that you gave to fgets() ?

 

[chown33]

In addition to posting code, be sure to post the actual file of input data, preferably zipped or in some other format that's sure to preserve the exact data.

BTW, testing how fgets() works should be a small and well-isolated test program. If the problem occurs with the simple test program and test data, then you have evidence of a problem. If the problem doesn't occur with the simple test program and test data, then figure out why not. Either way, posting an actual compilable test program and actual test data is likely to yield better (and more conclusive) results than any amount of description ever could.

 

[farmerdoug]

Again
If you care to go through the forum, you will find a long discussion about fgets. According to what I learn previously fgets responds to a specific end of line character. Depending on how the file was made, the end of line character could be one several character; Only one of which is compatible with fgets. I replaced fgets in old code with fscanf. Now, as I said, I find that fgets works from the command line but still doesn't work in XCode. I'm not interested in debugging this; I'm happy running this code in the terminal.

So your guess is its my code and not any difference between X11 and Xcode.

 

[chown33]

Now, as I said, I find that fgets works from the command line but still doesn't work in XCode.

You'd have to explain exactly what data you're giving it in Xcode.

fgets() looks for one thing as a line terminator: \n (hex value 0x0A). If you give it data containing something else, it won't see any \n's, so it obviously won't work as expected. The problem lies with the data and/or the expectation. The problem does not lie with fgets().

So your guess is its my code and not any difference between X11 and Xcode.

You haven't provided enough information about the data source in Xcode or in X11 to be able to guess effectively.

You haven't described what "didn't work" means. Do you mean not detecting known line-endings? Detecting too many line-endings? Spurious line-endings appearing at random? Something unrelated to line-endings?

You haven't provided any code, either, so guessing about unseen code is futile.

 

[KnightWRX]

So your guess is its my code and not any difference between X11 and Xcode.

All we can do is guess until you provide us with a snippet of code and a dataset to test it with.

And you didn't answer my one question, is the buffer you're passing fgets properly allocated ? Errors like these are usually an allocation problem. What size are you passing to malloc to allocate your buffer and what size are you passing to fgets() for your buffer ? Are you even using dynamic allocation or just using a statistically defined array ?

If you post the code, we'll be able to see all of this.

 

[farmerdoug]

Let me rephrase the questions. Have any of you come across an example of code that runs in X11 but not in XCode?

 

[chown33]

By "X11", do you mean "the X11 terminal", or do you mean "the X11 environment in general"?

Because if it's the latter, then yeah, there's all kinds of X11 graphics code that won't run in Xcode.

Standard C and Posix console I/O stuff is identical, though.


Do you realize that the commands in X11 terminal are the identical commands you have in the normal Terminal.app? It's the identical bash, ls, grep, and so on.

 

[farmerdoug]

The X11 terminal.

 

[KnightWRX]

Like chown said, you're linked to the same libraries. There is no fundamental difference.

Post the code and let us help you.

 

[balamw]

It worked when I ran the code from a terminal window in X11 but failed in Xcode.

In addition to posting code, be sure to post the actual file of input data, preferably zipped or in some other format that's sure to preserve the exact data.

This seems awfully familiar to me. The working directory will be different depending on how you launch it.

Where is it looking for the input file? is it an absolute path (starts with "/") or not?

As chown33 and KnightWRX have said, we are just guessing because you haven't given us anything to work with. No code, no indication of HOW it failed (was there an error message? Did it grab stale data? ...)

Please read these two links on ways to get better results from forums such as this:

http://www.mikeash.com/getting_answers.html
http://whathaveyoutried.com

B

 

[farmerdoug]

First of all, it fails in the middle of the code in an so it's not working directory. However, that is what I am looking for: reasonable hypotheses as to
why the same code might fail in XCode and not X11. Forget the code. The code is propriety information; I am not allow to send it out. I don't want you to debug it; if I really want it to run in XCode I will debug it I just want some thoughts on the matter.

 

[balamw]

Make toy code that isn't proprietary. Post that. Debug that. Understand that.

We're done here without more input.

B

 

[chown33]

First of all, it fails in the middle of the code in an so it's not working directory. However, that is what I am looking for: reasonable hypotheses as to
why the same code might fail in XCode and not X11. Forget the code. The code is propriety information; I am not allow to send it out. I don't want you to debug it; if I really want it to run in XCode I will debug it I just want some thoughts on the matter.

Things that might differ:
1. working directory.
2. environment variables.
3. standard input stream (stdin).
4. standard output streams (stdout or stderr).
5. process group (i.e. the "controlling terminal" paradigm)
6. signal disposition (disabled signal inheritance)
7. 64-bit vs. 32-bit runtime arch
8. real & effective user/group id (unlikely, IMHO)
9. file-creation mask (umask)

Most of these things are accessible to C programs for self-inspection. That is, you can write C code that prints them out, so you can see what differs.

I'm not sure how to get a path from a FILE* or a filedes, so I don't know about self-inspection of stdio streams. However, see 'man lsof' for how to see another cmd's open filedes's.

1. man getcwd
2. man getenv
3. man lsof (use 'lsof' to get info on other cmd)
4. man lsof
5. man getppid, man getpgid
6. man sigaction
7. sizeof(void*)
8. man geteuid, man getegid
9, man umask

Also see man execve for other things that are inherited by the new process executable. You might see something there that you can also self-inspect and verify. My 1-9 list is basically a summary of what execve tells you.


And you still haven't said what fails. For all anyone knows, you're simply overrunning a buffer somewhere, and it's failing in a different way in the different environments. It could be a fatal malfunction in Xcode for some reason (e.g. any of reasons 1-9 above), yet a non-fatal malfunction in Terminal or X11 terminal. Or perhaps not really a non-fatal malfunction, but a not-yet-fatal malfunction.


Oh, and you could run your program under gdb in the terminal, just so you can mess around with some of the environmental settings. Who knows, maybe it's the fact that you're running under the debugger in Xcode (assuming you are, which is no worse an assumption than all the others made so far).

 

[jiminaus]

Are you reading from a file or from stdin? If from stdin, is it attached to the keyboard or redirected to a file?

I could imagine that reading from the keyboard could potentially be different between X11's xterm, the native Terminal.app and the XCode debug console. They would all have very different paths from the keyboard to your code.

But just using fgets should be the same. Although I do remember XCode 3 having something funky about newlines from the keyboard. I tried to find the old thread I recalled, but failed. I think it was something to do with hitting Return verses hitting Enter.

 

[farmerdoug]

Chown33 and jiminaus: Thank you. Exactly what I wanted.
Balamw: Glad you don't speak for everyone but I guess you are out of here.
So sad to see you go.

 

[KnightWRX]

Can't you just write a simple example routine that displays your problem and post that ? We don't want your proprietary app code, we want to see the code that fails.

Again, have you checked your buffer allocation code ? You never replied to me on this part. What type of buffer are you using ?

Just give us something like this :

size_t bufsize = 128;
char * buf = malloc(sizeof(char) * bufsize);
FILE * input = fopen("/path/to/datafile", 'r');

if(buf && input)
{
     while(fgets(buf, sizeof(char) * bufsize, input))
     {
           printf("--- I read this information --- \n%s\n", buf);
     }
}

(now I wrote this off the top of my head, might not even compile, I might not be remembering everything correctly nor would it compile, missing a function header etc...).

This example gives ample information so we can work to help you. Until you do, there's nothing much we can do but speculate. Undefined behavior like you describe though (failing in Xcode vs not failing in a terminal window) definately sounds to me like a buffer allocation error. Check your malloc, especially what you are allocating. Make sure it's reflected by the size parameter of fgets.

Is it failing on SIGSEGV in Xcode ? That would be a good indication of an allocation error.

BTW Balamw speaks for everyone here, we've all been asking for the same thing. He just as less patience than us it seems. Why make it hard for us to help you ? In the end, you'll only get frustrated that we aren't being much help.

 

[farmerdoug]

First of all, I am no longer sure that the problem is with fgets.
Second of all, since the code is going to run in X11, I really don't care exactly what the problem; The increase in my knowledge of X11/Xcode that I got from Chown et al is sufficient at the moment.
Third, writing more code so that you can debug it for something I don't need is not high on my list of priorities.

 

[KnightWRX]

First of all, I am no longer sure that the problem is with fgets.

Again, I doubt it ever was. fgets() is old, tried and true.

Second of all, since the code is going to run in X11, I really don't care exactly what the problem;

You should. If something isn't running right, the fact that it is not triggering an error under one environnement doesn't mean it's working right. It just means it's not triggering an error.

If you do have a buffer allocation problem, the problem can pop up later in your code. The sooner you sort it out, the better. Buffer allocation bugs can sometimes cause very weird behavior, and the more code you wrap around your input routine, the harder it will be to trace down.

I don't understand what issue you have exactly with refusing to provide us with help or even answer my allocation questions. In the end though, you're the only one that is losing out here. You obviously have broken code to fix and now your solution seems to be to leave it broken.

Good luck. I'll return once we have some more info and I can actually provide extra help.

 

[farmerdoug]

In the end though, you're the only one that is losing out here.

You may be right. But I have to choose what I think is the quickest path to what I need. Sometimes I win; sometimes I lose.

But please don't ever think that I am not grateful to the people on this forum.

 

[firewood]

The X11 Terminal, the XCode Terminal and the Mac OS Terminal app are 3 completely different terminals that can all each be configured with different line ending codes to be sent when you hit the "return" key. A Windows terminal might even send two codes. Look in the Mac Terminal app Keyboard Preferences to see some of this type of configuration. Note sure where they are for the Terminal in X11 or XCode, but probably hidden in some unix .rc file or Mac plist.

fgets() probably only looks for the number ten code, before doing something different.

The X11 term is probably sending 10, the Mac Terminal 13, and a Windows term both 13 and 10. No idea what XCode sends. But you can change them all to confuse fgets() or not.

If you want some code to respond properly no matter how the user has (mis)configured their terminal, you don't want to use fgets() to get line input.

 

[subsonix]

The X11 term is probably sending 10, the Mac Terminal 13, and a Windows term both 13 and 10. No idea what XCode sends. But you can change them all to confuse fgets() or not.

X11 sends the same line ending as Terminal does, it's easy enough to confirm. I have not been able to get fgets to behave differently in X11.

If you want some code to respond properly no matter how the user has (mis)configured their terminal, you don't want to use fgets() to get line input.

But, scanf needs magic numbers to be used safely. You still get the input, you might need to clean up the string even in Terminal as newline could be included in the string.