macOS Did I do this right? Creating a 3D char array, saving it, then freeing...

[MatthewF]

I'm just wondering if I'm doing this right...

- (void)saveDataIntoFile: (NSString*)name
{
    //Create the 2000x2000x8 3D array
    int width = 2000;
    int height = 2000;
    unsigned char*** pBytes = (unsigned char***)malloc(height*sizeof(**pBytes));
	
    for (int i=0; i<height ; i++) {
		
        pBytes[i] = (unsigned char**)malloc(width*sizeof(*pBytes));
		
        for (int j=0; j<width; j++) {
			
            pBytes[i][j] = (unsigned char*)malloc(8*sizeof(pBytes));
			//Fill it with numbers
			pBytes[i][j][0] = 0;
			pBytes[i][j][1] = 0;
			pBytes[i][j][2] = 0;
			pBytes[i][j][3] = 0;
			pBytes[i][j][4] = 0;
			pBytes[i][j][5] = 0;
			pBytes[i][j][6] = 0;
			pBytes[i][j][7] = 0;
        }
		
    }
	
	
    //Get path to save to
    NSArray* splitPath = [[[NSBundle mainBundle] bundlePath] componentsSeparatedByString:@"/"];
    NSString* path = @"";
    for (int i = 0; i < [splitPath count]-1; i ++) {
        path = [NSString stringWithFormat:@"%@/%@",path,[splitPath objectAtIndex:i]];
    }
    path = [NSString stringWithFormat:@"%@/%@",path,name];

    //Save
     NSData * pSaveData = [NSData dataWithBytes:pBytes length:width*height*8];
    [pSaveData writeToFile:path atomically:YES];
    [pSaveData release];
	 
	
    //Set free
    for (int x = 0;x<width;x++){
        for (int y = 0;y<height;y++){
            free (pBytes[x][y]);
        }
        free (pBytes[x]);
    }
	
    free (pBytes);
}

Thanks,
Matt

 

[kpua]

The 3D array bits look right, although I usually prefer to just make one big buffer and do the math myself to map an (x,y,z) coordinate to an index in the array. (and, honestly, the last time I made a 3D array I think was in school...)

However, your path manipulation code is quite unusual. Are you familiar with the NSPathUtilities API? It has an NSString category that lets you do things like, for example, -stringByAppendingPathComponent: (which is what I think you are trying to accomplish).

 

[lee1210]

dataWithBytes does not know about your complex multidimensional, piecewise-allocated pBytes. You give it a pointer, it will read up 320,000 bytes from that address. That doesn't work the way you have done things.

You could allocate 320,000 bytes, assign this to a char*, then manually generate indexes based on your dimensions. There is a somewhat complex way to do one malloc, then calculate the offsets and assign the pointers to build a multidimensional array. It's probably not worth it.

On my phone, so examples are hard. Hopefully you've got some ideas. If you DO need to initialize 320,000 bytes, do so with memset, rather than assignments in a loop.

-Lee

 

[MatthewF]

However, your path manipulation code is quite unusual. Are you familiar with the NSPathUtilities API? It has an NSString category that lets you do things like, for example, -stringByAppendingPathComponent: (which is what I think you are trying to accomplish).

Yeah it is sort of unusual. I use it to get the path of the directory my application is in.

dataWithBytes does not know about your complex multidimensional, piecewise-allocated pBytes. You give it a pointer, it will read up 320,000 bytes from that address. That doesn't work the way you have done things.

You could allocate 320,000 bytes, assign this to a char*, then manually generate indexes based on your dimensions. There is a somewhat complex way to do one malloc, then calculate the offsets and assign the pointers to build a multidimensional array. It's probably not worth it.

On my phone, so examples are hard. Hopefully you've got some ideas. If you DO need to initialize 320,000 bytes, do so with memset, rather than assignments in a loop.

-Lee

That makes sense... Based on what you're saying I just changed the saving code to:

// Save to file
	NSMutableData * pSaveData = [[NSMutableData init] alloc];
	for (int x = 0; x < width; x++) {
		for (int y = 0; y < height; y++) {
			for (int v = 0; v < 8; v++) {
				[pSaveData appendBytes:&pBytes[x][y][v] length:sizeof(pBytes[0][0][0])];
			}
		}
	}
    [pSaveData writeToFile:path atomically:YES];
    [pSaveData release];

Appears to work, creates a 32,000,000 byte text file which... TextEdit can't read.

I'm a bit worried about the freeing code:

//Set free
    for (int x = 0;x<width;x++){
        for (int y = 0;y<height;y++){
            free (pBytes[x][y]);
        }
        free (pBytes[x]);
    }
	
    free (pBytes);

When I look in Activity Monitor it appears that the memory isn't actually released..
What do you guys think?


Thanks,
Matt

 

[chown33]

Why are you going through so many hoops simply to write 32 MB of zeros to a file?

If you made a single calloc() call, it allocates and returns zeroed memory. Then write the data to a file. Bytes are bytes, and writing them one at a time in a triply nested 'for' loop is silly when every byte is known to be zero.

And why do you need 32 MB of zeros to open in TextEdit? That seems silly, too. It's a huge amount of data, and TextEdit is hardly optimized for editing binary zeros.

Exactly what are you trying to do? I suspect there's a lot you're not telling us.

 

[MatthewF]

Why are you going through so many hoops simply to write 32 MB of zeros to a file?

If you made a single calloc() call, it allocates and returns zeroed memory. Then write the data to a file. Bytes are bytes, and writing them one at a time in a triply nested 'for' loop is silly when every byte is known to be zero.

And why do you need 32 MB of zeros to open in TextEdit? That seems silly, too. It's a huge amount of data, and TextEdit is hardly optimized for editing binary zeros.

Exactly what are you trying to do? I suspect there's a lot you're not telling us.

I'm building a World Editor for my online game. Previously I've used the archiving tools to save my MapData and TileData classes. Though when you save a 2000x2000x8 map which turns out to be 32,000,000 TileData NSObjects, that equals around 600mb. So I've turned to direct byte manipulation, which can take the same data and turn it into 32mb.
I use the zeros just as an example. I'm trying to achieve the correct code to make the 3D char array, save it, free it, load it, and resize it.
This is what I have for making, saving, and freeing:

- (void)saveDataIntoFile: (NSString*)name
{
    int width = 2000;
    int height = 2000;
    unsigned char*** pBytes = (unsigned char***)malloc(height*sizeof(**pBytes));
	
    for (int i=0; i<width ; i++) {
		
    pBytes[i] = (unsigned char**)malloc(width*sizeof(*pBytes));
		
    for (int j=0; j<height; j++) {
			
        pBytes[i][j] = (unsigned char*)malloc(8*sizeof(pBytes));
        pBytes[i][j][0] = 3;
        pBytes[i][j][1] = 3;
        pBytes[i][j][2] = 3;
        pBytes[i][j][3] = 3;
        pBytes[i][j][4] = 3;
        pBytes[i][j][5] = 3;
        pBytes[i][j][6] = 3;
        pBytes[i][j][7] = 3;
        }
		
    }
	NSLog(@"%d",pBytes[9][9][0]);
	
	
    //Get path
    NSArray* splitPath = [[[NSBundle mainBundle] bundlePath] componentsSeparatedByString:@"/"];
    NSString* path = @"";
    for (int i = 0; i < [splitPath count]-1; i ++) {
        path = [NSString stringWithFormat:@"%@/%@",path,[splitPath objectAtIndex:i]];
    }
    path = [NSString stringWithFormat:@"%@/%@",path,name];
	
    //Save
    // Save to file
    NSMutableData * pSaveData = [[NSMutableData init] alloc];
    for (int x = 0; x < width; x++) {
        for (int y = 0; y < height; y++) {
            for (int v = 0; v < 8; v++) {
                [pSaveData appendBytes:&pBytes[x][y][v] length:sizeof(pBytes[0][0][0])];
            }
        }
    }
    [pSaveData writeToFile:path atomically:YES];
    [pSaveData release];
	 
	
	
    for (int x = 0;x<width;x++){
        for (int y = 0;y<height;y++){
            free (pBytes[x][y]);
        }
        free (pBytes[x]);
    }
	
    free (pBytes);
}

This is what I have for loading and displaying one byte of the data:

- (void)loadDataFromFile: (NSString*)name {
    //Get path
    NSArray* splitPath = [[[NSBundle mainBundle] bundlePath] componentsSeparatedByString:@"/"];
    NSString* path = @"";
    for (int i = 0; i < [splitPath count]-1; i ++) {
        path = [NSString stringWithFormat:@"%@/%@",path,[splitPath objectAtIndex:i]];
    }
    path = [NSString stringWithFormat:@"%@/%@",path,name];
	
    // Load
    NSMutableData * pSaveData = [NSData dataWithContentsOfFile:path];
	
    unsigned char byte;
    [pSaveData getBytes:&byte length:1];
    int getal = *( (int *) &byte ) ;
    NSLog(@"%d",getal);
}

So far all this code appears to work though I'm wondering a bit about organization... At the moment when it loads all the bytes, it's just a long line of 1 byte numbers. I'm thinking of adding the width, height, and value ints to the beginning separated by "/r/n" which would allow me to put all the data in a 3D array of the correct dimensions.
Is that a good method?


Thanks,
Matt

 

[chown33]

I'm building a World Editor for my online game. Previously I've used the archiving tools to save my MapData and TileData classes. Though when you save a 2000x2000x8 map which turns out to be 32,000,000 TileData NSObjects, that equals around 600mb. So I've turned to direct byte manipulation, which can take the same data and turn it into 32mb.

You don't need to create an array of bytes first. All you need to do is take the smallest unit, a single TileData if I understand your description correctly, and translate that into is encoded form: 8 bytes. Then append those 8 bytes to an NSMutableData. Iterate for your entire 2000x2000 map. At the end, the NSMutableData will be the cumulative sequence of bytes, which you can then write to a file.

Reading the file later is the opposite. Get the file bytes into NSData. Read 8 bytes. Make one TileData. Add to map. Repeat until map is populated.

This strategy is called Factoring, Decomposition, or Breaking It Down. It is fundamental to all computer programming. It means taking a big problem and breaking it down into smaller and individually solvable sub-problems. You then string the smaller solutions together and it solves the big problem.
http://www.cocoadev.com/index.pl?BreakItDown


You also really need to take a look at the NSString methods:

stringByDeletingLastPathComponent
stringByAppendingPathComponent:

int getal = *( (int *) &byte ) ;

That's the most obtuse code I've seen outside of an Obfuscated C Contest.

You can use %d on the byte itself. Even if you couldn't, you should do this:

int getal = byte;

The code you wrote looks like you don't really understand C's fundamental types. This is a very shaky foundation to be writing code from.

 

[MatthewF]

You don't need to create an array of bytes first. All you need to do is take the smallest unit, a single TileData if I understand your description correctly, and translate that into is encoded form: 8 bytes. Then append those 8 bytes to an NSMutableData. Iterate for your entire 2000x2000 map. At the end, the NSMutableData will be the cumulative sequence of bytes, which you can then write to a file.

Reading the file later is the opposite. Get the file bytes into NSData. Read 8 bytes. Make one TileData. Add to map. Repeat until map is populated.

This strategy is called Factoring, Decomposition, or Breaking It Down. It is fundamental to all computer programming. It means taking a big problem and breaking it down into smaller and individually solvable sub-problems. You then string the smaller solutions together and it solves the big problem.

Figured it out, works. Took that 600mb file and squeezed it down to 12mb.

You also really need to take a look at the NSString methods:

True.

That's the most obtuse code I've seen outside of an Obfuscated C Contest.

Eheheh, yeah. I got that bit off the net.

The code you wrote looks like you don't really understand C's fundamental types. This is a very shaky foundation to be writing code from.

No worries, I pick up on things fast.


Thanks guys, solved.
Matt
P.S. Here's the working example code in case anyone's wondering: (For storing ints as single bytes in a char array, saving to file, and loading back into the char array. Allows for great storing of tons of data when NSObjects take too much space)

- (void)saveDataIntoFile: (NSString*)name
{
	int width = 2000;
	int height = 2000;
	int layer = 3;
	unsigned char*** pBytes = (unsigned char***)malloc(height*sizeof(**pBytes));
	
    for (int i=0; i<width ; i++) {
		
        pBytes[i] = (unsigned char**)malloc(width*sizeof(*pBytes));
		
        for (int j=0; j<height; j++) {
			
            pBytes[i][j] = (unsigned char*)malloc(layer*sizeof(pBytes));
			pBytes[i][j][0] = 0;
			pBytes[i][j][1] = 1;
			pBytes[i][j][2] = 2;
        }
		
    }
	
	
		//Get path
		NSArray* splitPath = [[[NSBundle mainBundle] bundlePath] componentsSeparatedByString:@"/"];
		NSString* path = @"";
		for (int i = 0; i < [splitPath count]-1; i ++) {
			path = [NSString stringWithFormat:@"%@/%@",path,[splitPath objectAtIndex:i]];
		}
		path = [NSString stringWithFormat:@"%@/%@",path,name];
	
	// Save to file
	NSMutableData * pSaveData = [[NSMutableData init] alloc];
	[pSaveData appendBytes:&width length:sizeof(width)];
	[pSaveData appendBytes:&height length:sizeof(height)];
	[pSaveData appendBytes:&layer length:sizeof(layer)];
	for (int x = 0; x < width; x++) {
		for (int y = 0; y < height; y++) {
			for (int v = 0; v < layer; v++) {
				[pSaveData appendBytes:&pBytes[x][y][v] length:sizeof(pBytes[0][0][0])];
			}
		}
	}
	[pSaveData writeToFile:path atomically:YES];
    [pSaveData release];
	 
	
	
	for (int x = 0;x<width;x++){
		for (int y = 0;y<height;y++){
			free (pBytes[x][y]);
		}
		free (pBytes[x]);
	}
	
	free (pBytes);
}

- (void)loadDataFromFile: (NSString*)name {
	//Get path
	NSArray* splitPath = [[[NSBundle mainBundle] bundlePath] componentsSeparatedByString:@"/"];
	NSString* path = @"";
	for (int i = 0; i < [splitPath count]-1; i ++) {
		path = [NSString stringWithFormat:@"%@/%@",path,[splitPath objectAtIndex:i]];
	}
	path = [NSString stringWithFormat:@"%@/%@",path,name];
	
	// Load
	NSMutableData * pSaveData = [NSData dataWithContentsOfFile:path];
	
	char * dimensionsBuffer;
	[pSaveData getBytes:&dimensionsBuffer range:NSMakeRange(0, sizeof(int))];
	int width = (int)dimensionsBuffer;
	[pSaveData getBytes:&dimensionsBuffer range:NSMakeRange(sizeof(int), sizeof(int))];
	int height = (int)dimensionsBuffer;
	[pSaveData getBytes:&dimensionsBuffer range:NSMakeRange(sizeof(int)*2, sizeof(int))];
	int layer = (int)dimensionsBuffer;
	NSLog(@"%d %d %d",width, height, layer);
	
	unsigned char*** pBytes = (unsigned char***)malloc(height*sizeof(**pBytes));
	unsigned char dataBuffer;
    for (int i=0; i<width ; i++) {
		
        pBytes[i] = (unsigned char**)malloc(width*sizeof(*pBytes));
		
        for (int j=0; j<height; j++) {
			
            pBytes[i][j] = (unsigned char*)malloc(layer*sizeof(pBytes));
			for (int k = 0; k < layer; k ++) {
				[pSaveData getBytes:&dataBuffer range:NSMakeRange(sizeof(int)*3 + (i*height+j)*layer+k, sizeof(char))];
				pBytes[i][j][k] = dataBuffer;
			}
        }
		
    }
	
	for (int x = 0;x<width;x++){
		for (int y = 0;y<height;y++){
			free (pBytes[x][y]);
		}
		free (pBytes[x]);
	}
	
	free (pBytes);
}