"C" CGI program, response headers, web sharing, home network, weird bug.

Discussion in 'Mac Programming' started by toddburch, Mar 15, 2008.

  1. macrumors 6502a

    Joined:
    Dec 4, 2006
    Location:
    Katy, Texas
    #1
    This is my first CGI app and I'm writing it in C. The intent of the C app is basically to echo all the environment variables passed the C CGI app, and any file attached as well.

    It works on the Mac as expected, for the most part. I do have a second question about the post method boundary marker that is posed below.

    First question: :confused:
    This CGI setup does not "always" work when I access it from my Windows XP machine on my home network. I have lots of info to illustrate the issue, so please bare with me as I present it.

    I'm running Tiger (OS X 1.4.11) on a Mac Pro. I've enabled web sharing and that seems to work fine from my Mac and from my Windows machine. I'm running the default Apache server that comes with Tiger. On Windows, I'm running IE 6 under XP.

    I've created a simple html form with various controls to see how data and values are passed to the CGI program. Here is the form.

    Code:
    <html>
    <head>
    <title>Form Test</title> 
    </head>
    <body>
    <form id='f1' enctype="multipart/form-data" method="post" 
    action="http://192.168.1.100/cgi-bin/getmethod">
    <input name="lowercase" type="text" maxlength="50" size="50" id="t1"><br />  
    <input name="password" type="password" maxlength="50" size="50" id="p1"><br />  
    <input name="uppercase" type="text" maxlength="50" size="50" id="t2" readonly><br /> 
    <input type="hidden" name="email" value="me@example.com"> 
    <input type="textarea" size="100,5" maxlength="500" value="enter your story here..." id="t3"><br />
    
    <input type="file" name="myfile"><br /> 
    <input type="submit" value="test me"><br /> 
    
    <textarea rows="5" cols="100">Wow.  Data.
    </textarea><br /> 
    </body>
    <html>
    My CGI program is called "getmethod" (I know, I'm using "post" - but it's evolved and I didn't change the name...) It's written in C. Here's is that code:

    Code:
    #include <stdio.h>
    #include <string.h> 
    
    //int main (int argc, const char * argv[]) {
    int main (int argc, char * const argv[], char * envp[] ) {
    
    	int i = 0 , j ; //, rc = 0 ; 
    	char * position  ; 
    	char buffer[256] ; 
    	char * ctype ; 
    	int clength = 0 ; 
    	
    	printf("Content-Type: text/html\n\n") ; 
    
    	printf("<html>") ; 
    	printf("<head><title>From A C Program</title>") ; 
    	printf("<style type='text/css'>\n") ; 
    	printf(".yellow { background-color: yellow ; border: 1px solid black ;}\n") ; 
    	printf("</style>\n") ; 
    	printf("</head>\n") ; 
    	printf("<body><table class='yellow'>\n") ; 
    	
    	while (envp[i]) { 
    		strcpy(buffer, envp[i]) ; 
    		position = strchr(buffer, '=') ;   
    		if (!position) { 
    			printf("\tcould not find equal sign in %s<br />\n", envp[i]) ; 
    			continue ; 
    		} 
    		buffer[position-buffer] = 0 ;   // null term the environment variable name 
    		printf("<tr><td>%-30s</td><td>%s</td></tr>\n", buffer, position+1) ; 
    		
    		for ( j = 0 ; j < strlen(buffer) ; ++j) toupper(buffer[j]) ; 
    		
    		if (strcmp("CONTENT_TYPE",buffer)==0) { 
    			ctype = position+1 ; 
    		} 
    		
    		if (strcmp("CONTENT_LENGTH",buffer)==0) { 
    			clength = atoi(position+1) ; 
    		} 
    
    		i++ ; 
    	} 
    	printf("</table>\n");
    	printf("<table frame='box' rules='all' cellpadding='5' >\n<tr><td>data... bytes = %d</td>\n<td>", clength );
    
    	for ( i = 0 ; i < clength ; ++i ) { 
    		char c = fgetc(stdin) ; 
    		if (c=='\n') printf("<br />") ; 
    		else printf( "%c", c ) ; 
    	}
    
    	printf("</td>\n</tr>\n") ; 
    	printf("</table>\n</body>\n");
    	printf("</html>\n");
    	fflush(stdout) ; 
        return 0;
    }
    Running getmethod from the command line in Terminal and under XCode produces the expected output (although the environment variables are different). The piece I focus on is the content-type header. Works great on the command line, works great on the Mac (Safari) and doesn't work the first time for Internet Explorer. Here is what happens. On Windows, I direct the url to the server on my Mac with the formtest.html to bring up the form. Works great. I click "test me" (aka "submit") and I get the very vague "Internal Server Error" screen. See attached screen grab.

    Now, interestingly enough, while on this "Internal Server Error" screen, if I click the IE's Refresh button, I get the prompt for "do I want to resend this form", I click RETRY, and VIOLA! The output of the CGI program appears. This is eating my lunch. Why does it fail the first request, but work on the refresh?

    The initial request error status code is 500, which seems to be a catch-all error code. If I tail the error_log and access_log, I can see the requests and errors, but I don't know what to do to fix them. Here's the access log:

    Code:
    todd-burchs-computer:/var/log/httpd toddburch$ tail access_log
    192.168.1.102 - - [15/Mar/2008:02:23:20 -0500] "POST /cgi-bin/getmethod HTTP/1.1" 500 608
    192.168.1.102 - - [15/Mar/2008:02:30:42 -0500] "POST /cgi-bin/getmethod HTTP/1.1" 200 3431
    192.168.1.102 - - [15/Mar/2008:02:41:46 -0500] "POST /cgi-bin/getmethod HTTP/1.1" 500 608
    192.168.1.102 - - [15/Mar/2008:02:41:49 -0500] "POST /cgi-bin/getmethod HTTP/1.1" 200 3423
    192.168.1.102 - - [15/Mar/2008:02:45:50 -0500] "GET /~toddburch/formtest.html HTTP/1.1" 200 755
    192.168.1.102 - - [15/Mar/2008:02:45:52 -0500] "POST /cgi-bin/getmethod HTTP/1.1" 500 608
    192.168.1.102 - - [15/Mar/2008:11:26:03 -0500] "POST /cgi-bin/getmethod HTTP/1.1" 200 3416
    [color=green]192.168.1.102 - - [15/Mar/2008:11:31:55 -0500] "GET /~toddburch/formtest.html HTTP/1.1" 304 -[/color]
    [color=red]192.168.1.102 - - [15/Mar/2008:11:32:11 -0500] "POST /cgi-bin/getmethod HTTP/1.1" 500 608[/color]
    [color=blue]192.168.1.102 - - [15/Mar/2008:11:36:02 -0500] "POST /cgi-bin/getmethod HTTP/1.1" 200 3416[/color]
    
    The line in RED above is matched with the line in RED below in the error log. The line above in BLUE, showing success, is what I get when I click REFRESH in IE.

    The GREEN line is interesting too. When I click REFRESH in IE when positioned on the FORM itself, this is what gets logged. In my IE session, I see no changes.

    Here's the error log:
    Code:
    todd-burchs-computer:/var/log/httpd toddburch$ tail error_log
    [Sat Mar 15 02:14:40 2008] [error] [client 192.168.1.102] Premature end of script headers: /Library/WebServer/CGI-Executables/getmethod
    [Sat Mar 15 02:15:36 2008] [error] [client 192.168.1.102] Premature end of script headers: /Library/WebServer/CGI-Executables/getmethod
    [Sat Mar 15 02:16:02 2008] [error] [client 192.168.1.102] Premature end of script headers: /Library/WebServer/CGI-Executables/getmethod
    [Sat Mar 15 02:20:39 2008] [error] [client 192.168.1.102] Premature end of script headers: /Library/WebServer/CGI-Executables/getmethod
    [Sat Mar 15 02:21:11 2008] [error] [client 192.168.1.100] File does not exist: /Library/WebServer/Documents/favicon.ico
    [Sat Mar 15 02:21:15 2008] [error] [client 192.168.1.102] Premature end of script headers: /Library/WebServer/CGI-Executables/getmethod
    [Sat Mar 15 02:23:20 2008] [error] [client 192.168.1.102] Premature end of script headers: /Library/WebServer/CGI-Executables/getmethod
    [Sat Mar 15 02:41:46 2008] [error] [client 192.168.1.102] Premature end of script headers: /Library/WebServer/CGI-Executables/getmethod
    [Sat Mar 15 02:45:52 2008] [error] [client 192.168.1.102] Premature end of script headers: /Library/WebServer/CGI-Executables/getmethod
    [color=red][Sat Mar 15 11:32:11 2008] [error] [client 192.168.1.102] Premature end of script headers: /Library/WebServer/CGI-Executables/getmethod[/color]
    
    (As you can see, I was up late last night working on this!)

    Why do you think I am getting the premature end of script headers? It obviously works under Safari, the command line output proves this, and is working under IE after the refresh, just not the first time. I've chmod'ed 755 so many times I'm blue in the face. I've tried renaming the getmethod execute to getmethod.cgi and there is no change in behavior. I've tried dozens of combinations of "\n\n" to add a blank line under the content-type header line. I googled and saw one tip for making sure I added fflush(stdout), and I didn't have that, so I added it to my C program.

    So, what's the deal? Is this an Apache thing?

    ---------------------------------------------------------------------------------------------------

    Now, for my second question.... :eek:

    When the output page is served up, for method="post", one of the environment variables is the boundary separator for the content-type data in stdin. See the attached image with yellow. Notice the 4 hyphens that lead off the start of the boundary separator.

    Now, look down at data (white back ground). The boundary marker starts off the 6 hyphens, and the last marker even has two trailing hyphens. Is how this is supposed to work is to simply look at the "line" and if the boundary marker is on the line, then treat that line as a boundary marker? It seems reasonable that this would be the approach to take.

    Sorry for the long post. Now that I've spilled my guts, any other advice for best practices would be appreciated too.

    Thanks, Todd
     

    Attached Files:

  2. macrumors G4

    Eraserhead

    Joined:
    Nov 3, 2005
    Location:
    UK
    #2
    Whats a CGI app?
     
  3. thread starter macrumors 6502a

    Joined:
    Dec 4, 2006
    Location:
    Katy, Texas
    #3
    Seriously? It's an app that runs on a server and processes form data from html pages.

    You've probably used one before!! ;)

    Todd
     
  4. macrumors G4

    Eraserhead

    Joined:
    Nov 3, 2005
    Location:
    UK
    #4
    So its like ASP or php then, I suppose I could have guessed from the post :p.
     
  5. thread starter macrumors 6502a

    Joined:
    Dec 4, 2006
    Location:
    Katy, Texas
    #5
    Yes, in concept, just like that. ;) Those are very elaborate programs.

    Most cgi "programs" are scripts, and most scripts are probably written in Perl. But I'm new to all this too, thus, my problems!
     
  6. thread starter macrumors 6502a

    Joined:
    Dec 4, 2006
    Location:
    Katy, Texas
    #6
    I think I'm getting close to finding the error. While staring at the /var/log folder, I noticed a system.log file. I did a tail system.log and it says there was a crash log (was not aware of this) for my getmethod program.

    Code:
    Mar 15 16:56:22 todd-burchs-computer cp: error processing extended attributes: Operation not permitted
    Mar 15 16:57:15 todd-burchs-computer crashdump[3177]: getmethod crashed
    Mar 15 16:57:15 todd-burchs-computer crashdump[3177]: crash report written to: /Library/Logs/CrashReporter/getmethod.crash.log
    Mar 15 16:59:18 todd-burchs-computer crashdump[3206]: getmethod crashed
    Mar 15 16:59:18 todd-burchs-computer crashdump[3206]: crash report written to: /Library/Logs/CrashReporter/getmethod.crash.log
    Mar 15 17:00:23 todd-burchs-computer crashdump[3225]: getmethod crashed
    Mar 15 17:00:23 todd-burchs-computer crashdump[3225]: crash report written to: /Library/Logs/CrashReporter/getmethod.crash.log
    Mar 15 17:00:33 todd-burchs-computer crashdump[3228]: getmethod crashed
    Mar 15 17:00:33 todd-burchs-computer crashdump[3228]: crash report written to: /Library/Logs/CrashReporter/getmethod.crash.log
    
    So, I then did a cat of the crash log and got this:
    Code:
    
    Host Name:      todd-burchs-computer
    Date/Time:      2008-03-15 17:00:33.311 -0500
    OS Version:     10.4.11 (Build 8S2167)
    Report Version: 4
    
    Command: getmethod
    Path:    /Library/WebServer/CGI-Executables/getmethod
    Parent:  httpd [457]
    
    Version: ??? (???)
    
    PID:    3227
    Thread: 0
    
    Exception:  EXC_BAD_ACCESS (0x0001)
    Codes:      KERN_INVALID_ADDRESS (0x0001) at 0x7da5c554
    
    Thread 0 Crashed:
    0   getmethod   0x00008b7e main + 160 (main.c:24)
    1   getmethod   0x00001e9a start + 258
    2   getmethod   0x00001dc1 start + 41
    
    Thread 0 crashed with X86 Thread State (32-bit):
      eax: 0x7da5c554  ebx: 0x00008af5  ecx: 0x00000048  edx: 0xbffff228
      edi: 0xbffff229  esi: 0x00008f4d  ebp: 0xbffff358  esp: 0xbffff1f0
       ss: 0x0000001f  efl: 0x00010a13  eip: 0x00008b7e   cs: 0x00000017
       ds: 0x0000001f   es: 0x0000001f   fs: 0x00000000   gs: 0x00000037
    
    Binary Images Description:
        0x1000 -     0x4fff getmethod       /Library/WebServer/CGI-Executables/getmethod
        0x8000 -     0x8fff getmethod       /Library/WebServer/CGI-Executables/getmethod
    0x8fe00000 - 0x8fe4afff dyld 46.16      /usr/lib/dyld
    0x90000000 - 0x90171fff libSystem.B.dylib       /usr/lib/libSystem.B.dylib
    0x901c1000 - 0x901c3fff libmathCommon.A.dylib   /usr/lib/system/libmathCommon.A.dylib
    0x90bd2000 - 0x90bd9fff libgcc_s.1.dylib        /usr/lib/libgcc_s.1.dylib
    0x963fa000 - 0x963fbfff com.apple.zerolink 1.2 (3)      /System/Library/PrivateFrameworks/ZeroLink.framework/Versions/A/ZeroLink
    So.... my cgi is crashing! Line 24 points me to the line where I test if envp is "true". Therefore, the environment variable paradigm must be different when the request is coming in from IE. I took that code out and merely severed up a simple "hi there", and it works EVERY STINKING TIME.

    I'll do more research and post back with whatever I figure out. Man, this is an ugly one.

    Todd
     
  7. thread starter macrumors 6502a

    Joined:
    Dec 4, 2006
    Location:
    Katy, Texas
    #7
    1 step forward, and 1 step backwards.

    I thought that perhaps the envp[] array was not getting set up the same for IE as it was for other browsers, so I converted the program over to use getenv() instead of using the global variable.

    However, I get the same exact results. The program crashes, as seen above in the crash log, but works after REFRESH is clicked. I think I may head over to the Apache forums. I'm feeling a lot of "deer in headlights" stares here... ;)

    Todd
     
  8. macrumors newbie

    Joined:
    Sep 17, 2007
    #8
    Hi!

    I have no definite explanation why you program crashes at first and then works upon reload in IE.

    What I m seeing is however a badly possible buffer-overflow, which can easily kill your program.

    char buffer[256];
    strcpy(buffer, envp);

    Since there is no guarantee that environment variables stay all the same all the times it is possible that upon some executions some env variable contains a longer string - this will overwrite buffer and kill more local stack variables.

    You can defer copying of the envp after the = check, or even better get rid of copying, and use the envp contents directly without modification.

    Regards
     
  9. thread starter macrumors 6502a

    Joined:
    Dec 4, 2006
    Location:
    Katy, Texas
    #9
    Ah. Good idea. I will try that now.
     
  10. thread starter macrumors 6502a

    Joined:
    Dec 4, 2006
    Location:
    Katy, Texas
    #10


    [size=+3]YOU ARE THE MAN!!![/size].

    That was exactly it. The HTTP_ACCEPT variable was longer than 256 bytes. Upon refresh, it was 3 bytes. I recoded my table to show the length too, and this confirms it!

    WOO-HOO!!!

    Thank you so much. I owe you a cool one!

    Todd
     

    Attached Files:

Share This Page