macOS File Copy without Pausing Program

[Jordan the Nerd]

Hi,

I'm new to Objective C (really only started yesterday), and I'm currently trying to make a simple program to copy a directory to a destination (or multiple destinations). I've done all the fun stuff (interface builder, love it) but now its time for the file copy stuff...

Looking around on the internet on blogs and things I have discovered that file copying isn't one of Cocoa's strong points (well, at least easily). I found out that I can use system() commands which I could call to get the job done, I also found the NSFileManager's copyItemAtPath toPath function. Both of these ways work, but they all seem to pause the entire program and even makes Mac OSX think the program is not responding. Due to the size of the directories I am wanting to use this program for, this isn't really what I want, especially as I want to have a spinning progress indicator running (because I am new to programming on the Mac and I think these are awesome! I've given up dreams of having a real progress indicator for now, these seem to be the next step).

I heard that this is because they are doing it synchronously, and that I need something asynchronously. I also found this here, http://www.cimgf.com/2008/05/03/cocoa-tutorial-file-copy-with-progress-indicator/, which is using FSCopyObjectAsync - apparently a Cocoa function which has memory leak issues, plus I'm not even sure if you can copy entire directories like this. I'm not sure how to get this one running, and if I should even use it for my program.

So is there any way to use native Cocoa to copy a file in the background or something, to keep the program from completely not responding and beach-balling, or do I have to try and figure out how to use something old and annoying like FSCopyObjectAsync, if I do, how do I use this for entire directories?

Thanks for your help and I hope you can help me out.

Jordan

 

[mfram]

I made a program that copies the contents of iTunes Libraries to other devices. Copying files and directories... essentially what you want to do.

My recommendation is to make the copy job a separate thread. Then it won't hold up your main program. Periodically I have my copy thread call an 'update' method on the main controller to give it status on the copy. Also, I have a method that the main UI thread can call to set a flag in the copy thread to tell it to stop. At the key points in the operation, the copy thread can check the flag to see if it should abort the operation.

I used the NSFileManager calls to do the directory operations and copies in the file-copy thread.

 

[holmesf]

Hi,

I'm new to Objective C (really only started yesterday), and I'm currently trying to make a simple program to copy a directory to a destination (or multiple destinations). I've done all the fun stuff (interface builder, love it) but now its time for the file copy stuff...

Looking around on the internet on blogs and things I have discovered that file copying isn't one of Cocoa's strong points (well, at least easily). I found out that I can use system() commands which I could call to get the job done, I also found the NSFileManager's copyItemAtPath toPath function. Both of these ways work, but they all seem to pause the entire program and even makes Mac OSX think the program is not responding. Due to the size of the directories I am wanting to use this program for, this isn't really what I want, especially as I want to have a spinning progress indicator running (because I am new to programming on the Mac and I think these are awesome! I've given up dreams of having a real progress indicator for now, these seem to be the next step).

I heard that this is because they are doing it synchronously, and that I need something asynchronously. I also found this here, http://www.cimgf.com/2008/05/03/cocoa-tutorial-file-copy-with-progress-indicator/, which is using FSCopyObjectAsync - apparently a Cocoa function which has memory leak issues, plus I'm not even sure if you can copy entire directories like this. I'm not sure how to get this one running, and if I should even use it for my program.

So is there any way to use native Cocoa to copy a file in the background or something, to keep the program from completely not responding and beach-balling, or do I have to try and figure out how to use something old and annoying like FSCopyObjectAsync, if I do, how do I use this for entire directories?

Thanks for your help and I hope you can help me out.

Jordan

File i/o, including copying, is normally what we call a "blocking" operation. This means that your program "blocks" until the operation is completed and then resumes execution. You can get around this by using non-blocking i/o, or as a more general approach, performing the copying in a separate thread. As a general guideline, any operation which takes a significant amount of time should be performed in a background thread so that the user interface remains responsive. Grand Central Dispatch (GCD) is a relatively new technology that makes this rather easy to add to a program.

Here's Apple's guide on threading:
http://developer.apple.com/library/mac/#documentation/Cocoa/Conceptual/Multithreading/AboutThreads/AboutThreads.html#//apple_ref/doc/uid/10000057i-CH6-SW2

 

[Jordan the Nerd]

Thanks for your replies.

So how do I make the copy process in a new thread?
Could threading make it easier to stop the copy process as well? Stopping it isn't a big deal, but it could be useful.

 

[holmesf]

Thanks for your replies.

So how do I make the copy process in a new thread?
Could threading make it easier to stop the copy process as well? Stopping it isn't a big deal, but it could be useful.

Threading won't let you stop in the middle of a Cocoa API call, but yes, if you broke the copying process down into a number of steps, then you could stop at some point between steps.

Here is an example of how you could make the copy process done in a separate thread using GCD:

/* up here we are executing in the main thread */
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^{
     /* any lines of code here get executed in a separate thread with low CPU priority */
     /* try putting your copying code here */
});
/*
    down here we are back to executing in the main thread
    dispatch_async returns immediately, in other words the background thread may not have started yet
    or it may still be running
*/

 

[Jordan the Nerd]

Thanks for that. That snippet of code will be useful.

Is there any way (this probably almost defeats the purpose of having it as a separate thread) of knowing when the copy has completed, so I can then start the next copy / update an onscreen status?

 

[holmesf]

Thanks for that. That snippet of code will be useful.

Is there any way (this probably almost defeats the purpose of having it as a separate thread) of knowing when the copy has completed, so I can then start the next copy / update an onscreen status?

Yes. Since UI changes normally need to happen on the main thread, in GCD you would do this by dispatching a block to the main thread. To expand the earlier example:

dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^{
     /* do your copying here */
     dispatch_async(dispatch_get_main_queue(), ^{
        /* anything in here will happen on the main thread after the background thread has finished copying.  For example, update your UI. */
     });
});

Anyway, I highly recommend reading the Apple guide I linked to you earlier. You can learn a lot about threading and GCD.

 

[Jordan the Nerd]

Wow, thanks.

So if I wanted to change a variable in the code executing in another thread, would it be able to change things in the main thread, or should I think about it as being in a completely different scope (like a function?).
Then would the code to run things as the main thread be able to access variables / objects that are in the main thread, or is it separate once again?

My terminology may not be correct for Objective-C, as the only other programming I've done is C++ in Windows.

 

[holmesf]

Wow, thanks.

So if I wanted to change a variable in the code executing in another thread, would it be able to change things in the main thread, or should I think about it as being in a completely different scope (like a function?).
Then would the code to run things as the main thread be able to access variables / objects that are in the main thread, or is it separate once again?

My terminology may not be correct for Objective-C, as the only other programming I've done is C++ in Windows.

If the variable belongs to the object (is an instance variable) then yes, you can change it in one thread and the changes will appear in the other thread. Threads all share the same memory space, although they have their own stacks (if that makes any sense). There are some scope rules to do with GCD, but for those you should read the guide.

You must be very careful changing a variable in one thread that is read from another thread, however. For simple statements it might work out, but generally you need to surround areas of code that read/write shared variables with a semaphore (or in objective-c a @syncronized { } code block, or NSLock) in order to ensure that the variables are in a coherent state whenever they are read.

 

[mfram]

The other alternative to GCD is using NSThread and explicit threading. Either solutions can work. Generally GCD is designed around the concept of queuing lots of small operations. In my program, I put the logic for copying all of the files into the separate thread and the let copy thread do all the work while the UI thread manages the UI. You can choose either design which fits better into your program.

In my program, I had a button that doubled as the 'Start' or 'Stop' button. The code looks like this in my controller:

- (IBAction) gotSyncButton:(id)sender
{
	if (isCopying)
	{
		[self stopCopying];
	}
	else 
	{
		[self startCopying];
		[SyncButton setTitle:@"Stop"];
		[SyncButton setNeedsDisplay:YES];
	}
}

- (void) startCopying
{
	NSThread *copyThread;
		copyThread = [[NSThread alloc] initWithTarget:self selector:@selector(copyThread:) object:nil];
	[copyThread start];
}

- (void) stopCopying
{
	shouldStopCopying = YES;
}

The method 'copyThread:' performs the copy operation. If you have a separate thread, you should have a separate AutoRelease pool (if you aren't using ADC) and should use a separate File Manger in each thread.

- (void) copyThread:(id)sender
{
	NSAutoreleasePool *pool;
	NSFileManager *thread_fm;
	
	if ([copyThreadLock tryLock] == NO)
	{
		return;
	}

	pool = [[NSAutoreleasePool alloc] init];
	isCopying = YES;
	
	thread_fm = [[NSFileManager alloc] init];

        // Do copy operations with [thread_fm copy....];

	isCopying = NO;
	shouldStopCopying = NO;
	[SyncButton setTitle:@"Sync"];
	[SyncButton setNeedsDisplay:YES];
	[thread_fm autorelease];
	[pool drain];
	[copyThreadLock unlock];
}

The lock was there to make sure I only had one copy operation going on at once. The copy thread checks the 'shouldStopCopying' flag periodically to see if the operation should abort.

Hmmm, looks like there a small leak with that NSThread allocation. Should probably have an autorelease in there... oops. It was one of my first Mac programs, I missed stuff like that.

 

[Jordan the Nerd]

Thanks. I'm sure this will also come in handy.

Another thing that is confusing me, is that now that I have a file copy process running (using holmesf's code) in a function (setup with int copyFile (id Source, id Destination). ), but I cannot seem to change attributes of InterfaceBuilder objects which are accessible through my other functions (I get "use of undeclared identifier" even though it is declared in my AppDelegate.h file).

Is there something different you have to do to make that function (I notice you're copy function is setup different from mine).

Probably because I'm still trying to get around the idea of object orientated programming.

 

[mfram]

Blocks as used in GCD are a little of an unusual beast. It is a C construct, not an Obj-C one. It's not really a function, and it's definitely not an Obj-C method so you won't have access to member variables in your objects. You'll have to read the GCD Documentation carefully to figure out some strategies for using it.

 

[Sydde]

One other possibility would be to copy the file content "manually" using NSFileHandle's -readInBackgroundAndNotify and its complementary write method. This would be slightly cumbersome, but it would, I think, give you possible process breaking points in the middle of very large files.

 

[Jordan the Nerd]

Thanks everyone. I ended up using holmesf's GCD code so that I can start a method in another thread once the button is pressed. Because it is a method (I think that is the correct term...) and not a function it seems to be able to change attributes of the objects created in interface builder. This lets me change interface attributes from the function I called above (function running in the new thread) without even having to return to the main thread, which is nice.
So thanks for your help.

Also, in the line provided by holmesf, I found that for proper GCD support in Snow Leopard I had to change dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0) to dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_LOW, 0). I don't know why, but it seems to work on Snow Leopard now.

 

[Catfish_Man]

Also, in the line provided by holmesf, I found that for proper GCD support in Snow Leopard I had to change dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0) to dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_LOW, 0). I don't know why, but it seems to work on Snow Leopard now.

Background priority queues are a new feature of GCD in Lion; they throttle all activity (including disk io!) down to a barely noticeable level, allowing things to run in the background without impacting the main user experience. Obviously it'll make your copy run much much slower as a tradeoff.

Something I haven't seen called out yet that you should know: you're in pretty dangerous territory when dealing with threads. Not all of Cocoa is thread-safe (for example, virtually all UI operations must be done on the main thread for safety, which it sounds like you're not doing; that will crash sometimes), and seemingly harmless code can lead to threading bugs that are incredibly difficult to understand. Simple things like "copy on a background thread, then get the result back to the main thread" should be ok, but tread carefully and be prepared to do some learning.

Not really trying to scare you off of doing this, just trying to set expectations.