Register FAQ / Rules Forum Spy Search Today's Posts Mark Forums Read
Go Back   MacRumors Forums > Apple Systems and Services > Programming > Mac Programming

Reply
 
Thread Tools Search this Thread Display Modes
Old Mar 4, 2013, 06:23 AM   #1
farmerdoug
macrumors 6502a
 
Join Date: Sep 2008
strtok problem?

The routine below is called twice for two separate lists. The first time I call it, it reads and loads data from 100 lines in 5 files.
The second time I call it, it only reads 6 lines from a few files before messing up on reading the date and crashing. When I print out the results of fgets, every line is properly printed.
ideas?

Code:
void  load_data(char *dir_path, char* list, float **array1, float **array2, char **date_list)
	{
		FILE *FILELIST, *SEC;
		char *name, *full_path, *line_of_data, *close_s, *open_s, **symbol_list;
		int  day, i, symbol_number = 0; 
		float close = 0, open = 0;
		
		close_s = (char *) calloc (30, sizeof(char));
		open_s = (char *) calloc (30, sizeof(char));
		name = (char *) calloc (100, sizeof(char));
		full_path = (char *) calloc (150, sizeof(char));
		line_of_data = (char *) calloc (150, sizeof(char));

		symbol_list = (char**) calloc(number_of_indices, sizeof(char*));
		for (i = 0; i <number_of_indices; i++)
			symbol_list[i] = (char *)calloc(10, sizeof(char));
		if( (FILELIST = fopen(list,"r")) == NULL)
		{
		printf( "couldn't open  file list\n");
		}


	while (fscanf(FILELIST, "%s\n", name) != EOF  && symbol_number < number_of_indices)
	{
		
		
		{
		strcpy(full_path, dir_path); strcat(full_path, name);
		if ( (SEC = fopen(full_path, "r")) == NULL)
			printf( "couldn't open index file %s\n", full_path);
			else
			printf( "opened index file %s\n", full_path);
		day = 0;
		while ( (fgets(line_of_data, 150, SEC))  && day < back_test_period)
		{	
		//	printf("%s\n", line_of_data);
			close_s = strtok (line_of_data,","); //symbol
			printf("%s\n",close_s);
			if ( day == 0)
				strcpy(symbol_list[symbol_number],close_s);
			
			close_s = strtok (NULL, ",");
			printf("%s\n",close_s);
			close_s = strtok (NULL, ",");// date
			printf("%s\n",close_s);
		//	printf("%s %s\n",close_s,date_list[day]);
			if (strcmp(close_s,date_list[day]) == 0)
			{
			
			if (strcmp(close_s, "<TICKER>") != 0)
				{
				close_s = strtok (NULL, ",");  ///00000
				printf("%s\n",close_s);
				open_s = strtok (NULL, ","); 
				printf("%s\n",open_s);
				for (i = 0; i < 3; i++)
					{
					close_s = strtok (NULL, ",");
					printf("%s\n",close_s);
					}
				}
			close = (float)atof(close_s);
			open = (float)atof(open_s);
		//	printf("%s %f\n", close_s, close);
			array1[symbol_number][day] =open;
			array2[symbol_number][day] =close;
			
			
			}
	//		printf("%d\n", day);
			day++;
		}
		fclose(SEC);
		}
		symbol_number++;
		
	}

	fclose(FILELIST);
	return ;
	
	}
farmerdoug is offline   0 Reply With Quote
Old Mar 4, 2013, 07:05 AM   #2
gnasher729
macrumors G5
 
gnasher729's Avatar
 
Join Date: Nov 2005
strtok is not reentrant - if two functions call strtok at the same time, it will crash. Google for "man strtok_r" and use that; the change is quite trivial.
gnasher729 is offline   0 Reply With Quote
Old Mar 4, 2013, 07:29 AM   #3
farmerdoug
Thread Starter
macrumors 6502a
 
Join Date: Sep 2008
Unfortunately the C code I am using doesn't recognize strtok_r but the code is not reentrant anyway.
Code:
printf("loading stock data\n");	
	strcpy( dir, stock_file_directory);
	strcpy( list, stock_file_list);
	date_list = load_date (dir, list);
	load_data(dir, list, open_array,close_array, date_list);
	
	printf("loading index data\n");	
	strcpy( dir, index_file_directory);
	strcpy( list, index_file_list);
	load_data(dir , list,open_array, index_array, date_list);
farmerdoug is offline   0 Reply With Quote
Old Mar 4, 2013, 08:25 AM   #4
Senor Cuete
macrumors regular
 
Join Date: Nov 2011
calloc not freed

This won't solve your problem but I don't see where you release the memory you allocated with the calloc() calls. Don't you need to free() these pointers before returning from your function to avoid a memory leak?
Senor Cuete is offline   0 Reply With Quote
Old Mar 4, 2013, 08:42 AM   #5
farmerdoug
Thread Starter
macrumors 6502a
 
Join Date: Sep 2008
You're right. It didn't solve anything. But now the code is a little cleaner.
farmerdoug is offline   0 Reply With Quote
Old Mar 4, 2013, 08:56 AM   #6
ghellquist
macrumors regular
 
Join Date: Aug 2011
Location: Stockholm Sweden
Try swapping reading index file and stock file (in the calling program). If it breaks, there probably is an error in the content of the index file. If not, the error is either in the module or in how you use the arrays. I would be suspicious about the following line:

array1[symbol_number][day] =open;

because you pass the open_array twice, it just might update the open statement on two different places depending on the exact file ordering.


//gunnar

Last edited by ghellquist; Mar 4, 2013 at 09:06 AM.
ghellquist is offline   0 Reply With Quote
Old Mar 4, 2013, 11:28 AM   #7
gnasher729
macrumors G5
 
gnasher729's Avatar
 
Join Date: Nov 2005
Quote:
Originally Posted by farmerdoug View Post
Unfortunately the C code I am using doesn't recognize strtok_r but the code is not reentrant anyway.
The problem is not in your code, it is in strtok. If your code calls strtok, and some totally unrelated call in another thread calls strtok, things go wrong.
gnasher729 is offline   0 Reply With Quote
Old Mar 4, 2013, 01:32 PM   #8
farmerdoug
Thread Starter
macrumors 6502a
 
Join Date: Sep 2008
I copied the code into Xcode and changed to strtok_r. Code still fails after reading and parsing six lines. Code already successfully parse several lines of code when this and another routine was previously called.
farmerdoug is offline   0 Reply With Quote
Old Mar 4, 2013, 01:50 PM   #9
farmerdoug
Thread Starter
macrumors 6502a
 
Join Date: Sep 2008
The pilot found his error. Thanks guys.
farmerdoug is offline   0 Reply With Quote
Old Mar 6, 2013, 08:16 AM   #10
Senor Cuete
macrumors regular
 
Join Date: Nov 2011
Questions

Was the error strtok()?

My C books don't mention strtok_r. Where can I find out about it and what header has to be #included?

In a real world application if you read a file line by line from a disk it will go slowly particularly if it's a big file because each command to read or write is a transaction and will take 10 or 15 milliseconds. Your drive can send you a big burst of continuous data rapidly if it's only one transaction.

You might want to read the whole file into memory by doing something like this:

Code:
if((fseek(inputFile, 0, SEEK_END)) != 0)//set to file End
	bailout();
if(fgetpos(inputFile, &fileSize) != 0) //filePosition = file size
	bailout();
if((fseek(inputFile, 0, SEEK_SET)) != 0)//reset to file start for reading
	bailout();
if((fileBuffer = malloc(fileSize + 1)) == NULL) //+1 for  NULL termination
	bailout();
if((bytesRead = fread(fileBuffer, sizeof(char), fileSize, inputFile)) != fileSize)  //read whole file into memory
	bailout();
fclose(inputFile);
And then parse the data in memory.
Senor Cuete is offline   0 Reply With Quote
Old Mar 6, 2013, 08:40 AM   #11
subsonix
macrumors 68030
 
Join Date: Feb 2008
Quote:
Originally Posted by Senor Cuete View Post
Was the error strtok()?

My C books don't mention strtok_r. Where can I find out about it and what header has to be #included?
strtok_r is a posix function. Since posix added a threading api, it makes sense that they accompanied it with a thread safe strtok. Standard C does not do threads, (up until C11 that is).

A more convenient alternative to strtok is strsep imo, it's also thread safe.

If your program is single threaded then strtok is fine, the potential problem is if several threads use strtok at the same time since strtok use a static variable to keep track of it's progress.
subsonix is offline   0 Reply With Quote
Old Mar 6, 2013, 10:07 AM   #12
chown33
macrumors 603
 
Join Date: Aug 2009
Quote:
Originally Posted by Senor Cuete View Post
My C books don't mention strtok_r. Where can I find out about it and what header has to be #included?
Read the man page:
Code:
man strtok_r
Or look it up in Xcode's builtin docs.

Or google it.


Quote:
In a real world application if you read a file line by line from a disk it will go slowly particularly if it's a big file because each command to read or write is a transaction and will take 10 or 15 milliseconds. Your drive can send you a big burst of continuous data rapidly if it's only one transaction.
There's no speed difference for small vs. large files. If you think otherwise, please provide some evidence or explanation.

You should also look at how buffering works in stdio (Standard I/O), because it has a significant affect on how fast data is read or written. You can run tests with buffered vs. unbuffered to see and measure the difference. Hint: It may not be as slow as you think.

Reading a file entirely into memory isn't always a viable approach. Even when it's viable, it may be unnecessary. The OP wasn't having a problem with speed, but with correctness. There's no point in trying to solve speed problems until all the correctness problems have been solved first.
chown33 is offline   0 Reply With Quote

Reply
MacRumors Forums > Apple Systems and Services > Programming > Mac Programming

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Similar Threads
thread Thread Starter Forum Replies Last Post
GoodPlayer Problem - Never mind. Problem solved. BasicGreatGuy iPad Apps 0 Mar 14, 2014 01:18 PM
iTunes DRM problem? Or, HD Versions of Movie on ATV1 Problem? Aqueous Apple TV and Home Theater 5 Feb 9, 2014 07:54 AM
iPhone: Gmail App problem or Jailbreak Problem? ARSkemp Jailbreaks and iOS Hacks 6 Apr 9, 2013 11:43 PM
Help, unexplainable problem (for me) [Intermittent slowdown problem] LittleBig MacBook Pro 5 Nov 5, 2012 05:29 PM

Forum Jump

All times are GMT -5. The time now is 04:46 AM.

Mac Rumors | Mac | iPhone | iPhone Game Reviews | iPhone Apps

Mobile Version | Fixed | Fluid | Fluid HD
Copyright 2002-2013, MacRumors.com, LLC