Permission issue with external drive

[Levina]

So I updated to Sequoia and ran into a problem that I can't fix.

I have my apps on the internal drive and have my photo library on an external drive. When I'm in Canon's DPP (proprietary photo editing software), which I use to cull my photos, I can delete a photo from my internal drive just fine from within the app but I can't do it from the external drive. I first need to give permission, but after filling in my password (which in itself is annoying), it says I don't have permission to perform that action. But I'm the owner and admin on my Mac! How can I not have permission? And why do I need to give permission in the first place? Funny thing is that I can delete the same file from the external manually just fine.

I have re-installed the latest version of DPP and checked in System Preferences that it has full disk access.

Any way to fix this?

 

[FreakinEurekan]

Seems to be something in Canon DPP permissions. You said that it has "full disk access" so this may be redundant, but check:

* Check System Settings > Privacy & Security > Files & Folders, see if Canon DPP is listed there, and if so ensure the folder you're trying to use is listed.

* Could possibly be listed under "Full Disk Access" instead of "Files & Folders" - I wouldn't expect so, but possible.

If the folder isn't listed (or, frankly, if the App isn't listed) then Canon DPP isn't ASKING for that permission; Canon Support is your next step for an app update.

If the app IS listed there but you still can't delete - I'd look at how the disk is formatted. APFS is my recommendation; exFAT has its place if you also need Windoze compatibility but I don't personally use it. NTFS is right out - copy data to a different disk & re-format.

 

[Levina]

Seems to be something in Canon DPP permissions. You said that it has "full disk access" so this may be redundant, but check:

* Check System Settings > Privacy & Security > Files & Folders, see if Canon DPP is listed there, and if so ensure the folder you're trying to use is listed.

* Could possibly be listed under "Full Disk Access" instead of "Files & Folders" - I wouldn't expect so, but possible.

If the folder isn't listed (or, frankly, if the App isn't listed) then Canon DPP isn't ASKING for that permission; Canon Support is your next step for an app update.

If the app IS listed there but you still can't delete - I'd look at how the disk is formatted. APFS is my recommendation; exFAT has its place if you also need Windoze compatibility but I don't personally use it. NTFS is right out - copy data to a different disk & re-format.

DPP is listed under both Full Disk Access as well as under Files & Folders. With "Full Disk Access" enabled DPP is greyed out in "Files and Folders" saying "full disk access". I tried turning off Full Disk Access and then folders and externals are listed under "Files and Folders" and enabled but it makes no difference.

The external disk is APFS formatted.

 

[bogdanw]

Close DPP, remove it from Full Disk Access, then add it again.

Another thing you can try, reset all permissions for Full Disk Access from Terminal:

tccutil reset SystemPolicyAllFiles

 

[Levina]

Close DPP, remove it from Full Disk Access, then add it again.

Another thing you can try, reset all permissions for Full Disk Access from Terminal:

tccutil reset SystemPolicyAllFiles

Did both. Made no difference.

I'm going to try with a few other externals.

 

[Levina]

Well now, this is weird. I tried another external disk and I could delete a file from within DPP just fine. No permission needed.

So what is different between the two disks?

#1 WD Black SN850X 4TB in an Acasis TBU405M1 case (the problematic one)
#2 Samsung SSD 970 EVO Plus 2 TB in an Acasis TBU401E case

Both APFS formatted.

I'm getting a new warning now. I removed the disks the way I have since, oh I don't know, since I was born? But the system now tells me that the disk was not correctly removed (which one? I removed two). Then it says I should use "VM, Recovery" and "Preboot" before disconnecting or turning it off. What the hell?

 

[Fishrrman]

Try this trick that may help with permissions problems.

Connect the"problem drive".
Let the drive icon mount on the desktop.
Click on the drive icon ONE time to select it, then bring up the "get info" box (you can type "command-i").
At the bottom of get info, click the lock and enter your administrative password.
Put a checkmark into "ignore ownership on this volume".
Close get info.

Now try things again.
No promises.

 

[Levina]

Try this trick that may help with permissions problems.

Connect the"problem drive".
Let the drive icon mount on the desktop.
Click on the drive icon ONE time to select it, then bring up the "get info" box (you can type "command-i").
At the bottom of get info, click the lock and enter your administrative password.
Put a checkmark into "ignore ownership on this volume".
Close get info.

Now try things again.
No promises.

Unfortunately that didn't work either.

 

[M4th]

I don't know if this different issue with the same enclosure sheds any light on what the OP is experiencing?

 

[MarkC426]

This is freaky, I had this same issue at the weekend (couldn’t delete from external drive)…. 🤪

This solution fixed it for me (the automation setting)

Unable to delete rejected images in DPP on macos

I had been using DPP to cull images on my Windows laptop which was working as expected. However, I have a new Macbook Pro laptop and have loaded DPP. I review my images in Quick View and tag all rejected images. Then I highlight all rejected images and press CMD/Delete to delete all rejected...

community.usa.canon.com

 

[Levina]

I don't know if this different issue with the same enclosure sheds any light on what the OP is experiencing?

It's the same SSD and enclosure for sure. But I have never experienced problems like the person on the Apple community describes. It also doesn't get hot. That is so strange about this, it all worked perfectly fine - not even the slightest hiccup - until I updated to Sequoia. It could still be the SSD or the enclosure or the combo of course, something that doesn't agree with Sequoia.

This is freaky, I had this same issue at the weekend (couldn’t delete from external drive)…. 🤪

This solution fixed it for me (the automation setting)

Unable to delete rejected images in DPP on macos

I had been using DPP to cull images on my Windows laptop which was working as expected. However, I have a new Macbook Pro laptop and have loaded DPP. I review my images in Quick View and tag all rejected images. Then I highlight all rejected images and press CMD/Delete to delete all rejected...

community.usa.canon.com

I had that already enabled so it didn't fix it for me.

 

[bogdanw]

You could disable SIP, see if it works, then re-enable it.

Disabling and Enabling System Integrity Protection | Apple Developer Documentation

Disable system protections only temporarily during development to test drivers, kernel extensions, and other low-level code.

developer.apple.com

 

[MarkC426]

That is so strange about this, it all worked perfectly fine - not even the slightest hiccup - until I updated to Sequoia.

I am still on Sonoma, so it could be a Sequoia issue…. 🤔

 

[MayflyMaven]

My money was on "disabling ownership" to fix the issue. ¯\_(ツ)_/¯

Unfortunately that didn't work either.

It sounds like ownership was enabled initially though (which is not the default setting for an external volume). It sure seems like it is still a permissions issue, although possibly specific to the Canon software. Do you get the same prompt to authenticate when you delete the file manually from the external volume?

I'm getting a new warning now. I removed the disks the way I have since, oh I don't know, since I was born? But the system now tells me that the disk was not correctly removed (which one? I removed two). Then it says I should use "VM, Recovery" and "Preboot" before disconnecting or turning it off. What the hell?

Sounds like you have (or had) an APFS volume group in this APFS container, i.e. a volume had an installation of macOS. Those APFS helper volumes aren't usually mounted, but when they are, they're hidden so you wouldn't know that you need to unmount them first. I remember seeing these "disk not ejected properly" warnings way back in the day on Catalina, but assumed Apple had sorted it out since then. It's harmless, just annoying.

All of this begs the question – is the (external) volume that holds your Canon library a volume group member? If it is, it's possible that Canon's software just doesn't handle it well. Or perhaps if the volume is an "orphaned" group member, maybe that's the scenario that Canon doesn't handle well. Maybe this is a shot in the dark, but you can run a Terminal command to see if there are some old System remnants on your external volume:

ls -laO "/Volumes/Your Volume Name Here"

If you see items in the listing that you don't see in the Finder (like Volumes, etc, bin, cores), and especially if you see flags like "restricted", then the volume is probably "tainted" with some old OS installation. In that case I'd copy the data off of it to a new volume, then delete the original volume, then see if Canon has any trouble managing photos on that volume.

 

[Levina]

My money was on "disabling ownership" to fix the issue. ¯\_(ツ)_/¯


It sounds like ownership was enabled initially though (which is not the default setting for an external volume). It sure seems like it is still a permissions issue, although possibly specific to the Canon software. Do you get the same prompt to authenticate when you delete the file manually from the external volume?

No, that's so strange, I was expecting that but nope, it just deletes.

Sounds like you have (or had) an APFS volume group in this APFS container, i.e. a volume had an installation of macOS. Those APFS helper volumes aren't usually mounted, but when they are, they're hidden so you wouldn't know that you need to unmount them first. I remember seeing these "disk not ejected properly" warnings way back in the day on Catalina, but assumed Apple had sorted it out since then. It's harmless, just annoying.

👍

All of this begs the question – is the (external) volume that holds your Canon library a volume group member? If it is, it's possible that Canon's software just doesn't handle it well. Or perhaps if the volume is an "orphaned" group member, maybe that's the scenario that Canon doesn't handle well.

Here's a screenshot of Disk Utility with all the drives and their volumes. The selected one is the problematic external.

Both externals are APFS formatted so why the 4TB has the two volumes and the 2TB doesn't I'm sure I don't know. What I do know is that the 2TB is older, was formatted on the Mac Pro a couple of years ago whereas the 4TB I bought last year and was formatted on the Mac Mini M2pro. Maybe I should reformat the 4TB on the Mac Pro too...

By the way, that greyed out volume under Macintosh HD volumes says "systeemmomentopname geactiveerd" which in English would be something like a System Snapshot. The volume inside that says APFS startup snapshot (or whatever it is exactly in English).

I'm guessing this is a APFS feature and normal?

Going back on topic:

Maybe this is a shot in the dark, but you can run a Terminal command to see if there are some old System remnants on your external volume:

ls -laO "/Volumes/Your Volume Name Here"

If you see items in the listing that you don't see in the Finder (like Volumes, etc, bin, cores), and especially if you see flags like "restricted", then the volume is probably "tainted" with some old OS installation. In that case I'd copy the data off of it to a new volume, then delete the original volume, then see if Canon has any trouble managing photos on that volume.

I ran the command for both external drives and both show hidden and restricted items. But isn't that normal? I mean the Library is hidden and some features are restricted, no?

#1 = the problem disk:

#2 = the one without the problem:

I agree that probably the best solution would be to just erase the problematic disk and reinstall, see if the problem goes away.

You could disable SIP, see if it works, then re-enable it.

Disabling and Enabling System Integrity Protection | Apple Developer Documentation

Disable system protections only temporarily during development to test drivers, kernel extensions, and other low-level code.

developer.apple.com

That's something to try. I will do that later today. Thanks!

 

[Fishrrman]

Some questions:

How big is the photo library on the external drive?

Do you have ANOTHER external drive around that could hold it?

I was going to suggest something to try (moving library to another drive, that is formatted to HFS+ instead of APFS), but this only works if you have another drive that can hold the library...

 

[MayflyMaven]

I'm guessing this is a APFS feature and normal?

Yes, the startup disk is a "volume group" composed of a System volume and a Data volume. The System volume is read-only, and typically not mounted. Instead, a snapshot of the System volume is mounted on startup and the Mac boots from that. Apple introduced volume groups in Catalina, and then the "Signed System Volume" in Big Sur.

I ran the command for both external drives and both show hidden and restricted items. But isn't that normal? I mean the Library is hidden and some features are restricted, no?

That layout is normal for the startup disk, but not for a volume that you use just for storing photos. Granted, I don't know what you use these volumes for – they look like copies of the startup disk; are (or were) you using them as bootable external devices too?

The NVMe2TB volume is an "orphaned" Data volume – the associated System volume was deleted. That's perfectly fine, but there is a lot of system "noise" left behind on the Data volume that you may not need (unless that's a backup of your startup disk).

I agree that probably the best solution would be to just erase the problematic disk and reinstall, see if the problem goes away.

If you don't use the WDBlack4TB volume as bootable device, then it would probably be best to delete the System volume from that group. That would make it comparable to the NVMe2TB volume, and perhaps that would be enough to get Canon DPP to play nice with it.

Bigger-picture, though, if you use these two devices for making backups of your startup disk and for storing photos (i.e. photos that are not stored on the startup disk), I'd separate those two use cases on the external disks. Add volumes to each disk that you can use for photo storage, then use the NVMe2TB and WDBlack4TB volumes for the backup of your startup disk. You can add volumes in Disk Utility, e.g.:

- Select WDBlack4TB in the sidebar
- Click "+" in the toolbar to add a new volume

Then migrate your photos to the new volume, then delete the photos from the WDBlack4TB volume (to avoid any confusion about which copy is the one you're supposed to be working from).

APFS volumes share space on the underlying storage device, so it's more flexible than partitioning, but it gives you that same level of organizational "purity" if you're using the storage for multiple different purposes.

 

[Levina]

That layout is normal for the startup disk, but not for a volume that you use just for storing photos. Granted, I don't know what you use these volumes for – they look like copies of the startup disk; are (or were) you using them as bootable external devices too?

The NVMe2TB volume is an "orphaned" Data volume – the associated System volume was deleted. That's perfectly fine, but there is a lot of system "noise" left behind on the Data volume that you may not need (unless that's a backup of your startup disk).

I looked in System Preferences and both externals are start up disks with different OS's. Look:

I actually had the 2TB NVMe in my Mac Pro Classic 2009 as start-up disk. After switching to the Mac Mini M2pro two years ago, I took it out and used it as a backup for my images mostly. I never did anything with it so it's still on High Sierra. Strange though it's in Startup Disks whilst Disk Utility shows no system.

When I got the 4TB last year I decided to install the OS on it so I'd have a bootable back-up. SuperDuper at that time couldn't make a bootable clone.

So, this is bad? Do I need to upgrade the 4TB to Sequoia too maybe? Or maybe I should make it my #1 drive instead, be on Ventura. I do not like Sequoia, too many little annoyances.

Okay, here's my set up:

I have an internal SSD of 1TB. It holds the OS and apps. And some data. But the bulk of my data, including all the images are on the 4TB drive. I mostly work on the internal drive though. So culling my images, processing my images I do on the internal drive. When done, I move them to the 4TB.

If you don't use the WDBlack4TB volume as bootable device, then it would probably be best to delete the System volume from that group. That would make it comparable to the NVMe2TB volume, and perhaps that would be enough to get Canon DPP to play nice with it.

Bigger-picture, though, if you use these two devices for making backups of your startup disk and for storing photos (i.e. photos that are not stored on the startup disk), I'd separate those two use cases on the external disks. Add volumes to each disk that you can use for photo storage, then use the NVMe2TB and WDBlack4TB volumes for the backup of your startup disk. You can add volumes in Disk Utility, e.g.:

- Select WDBlack4TB in the sidebar
- Click "+" in the toolbar to add a new volume

Then migrate your photos to the new volume, then delete the photos from the WDBlack4TB volume (to avoid any confusion about which copy is the one you're supposed to be working from).

APFS volumes share space on the underlying storage device, so it's more flexible than partitioning, but it gives you that same level of organizational "purity" if you're using the storage for multiple different purposes.

I didn't know that was possible. Pretty cool. But yeah, the 4TB is not a backup but my main data drive (that in turn get backed up on two spinning drives). It's actually been a problem from the start how to handle this: 1TB internal, 4TB external. Life was a lot easier with the Mac Pro. It's one of the reasons I want to upgrade to a Mac Studio with a 4TB SSD.

 

[Fishrrman]

What I'd do:

Get another external drive.
Format it to HFS+.
DON'T install a copy of the OS onto it.
Move your photo library (nothing else) to the external drive.
Do a get info on the new drive, and choose to "ignore ownership on this volume".
NOW see how the drive (and files) behave when using your internal drive and DPP.

 

[MayflyMaven]

I actually had the 2TB NVMe in my Mac Pro Classic 2009 as start-up disk. After switching to the Mac Mini M2pro two years ago, I took it out and used it as a backup for my images mostly. I never did anything with it so it's still on High Sierra. Strange though it's in Startup Disks whilst Disk Utility shows no system.

Ah, gotcha. That system is obsolete on the new Mac (and it's funny that Startup Disk would even show it as an option – it won't boot your Apple Silicon Mac). It's harmless to leave the system files on that disk, but personally I'd remove them (e.g. System, Library, Applications; there are others, but that's the bulk of it) just to reduce the noise and any ambiguity about whether that device could function as a startup disk.

So, this is bad? Do I need to upgrade the 4TB to Sequoia too maybe? Or maybe I should make it my #1 drive instead, be on Ventura. I do not like Sequoia, too many little annoyances.

Having an OS on the external disk is fine, but if that's what is causing Canon DPP to have trouble managing the photos on that volume, then I'd add a new volume on that 4TB disk that you can use just for your photos. Having separate volumes for different purposes (external boot volume, data storage) goes a long way to avoid unintended consequences. It's certainly easy enough to try to see if it resolves the Canon DPP behavior.

I have an internal SSD of 1TB. It holds the OS and apps. And some data. But the bulk of my data, including all the images are on the 4TB drive. I mostly work on the internal drive though. So culling my images, processing my images I do on the internal drive. When done, I move them to the 4TB.

OK, then in this case I would definitely be more comfortable with one volume for photos and one volume for the bootable volume (I just wouldn't be comfortable having the bulk of my production data on a volume that is also a backup disk).

And maybe I should clarify – do you regularly boot your Mac from the WDBlack4TB volume, or is that more of a "just in case" deal? Does your backup software make regular backups of your startup disk to WDBlack4TB, or was that just a clean install of the OS on a volume that otherwise only had the photo data? If you only have a clean install of the system on this 4TB disk that otherwise only contained your photos (not the rest of your files, mail, applications, etc), I'd be a little more comfortable with that – in this case the disk has just two purposes: rescue/alt boot and photos, and those don't really conflict with each other. If the 4TB disk is or was a backup of your entire startup disk, and it has your production photos data, and you're regularly updating the backup on that volume, then I start to worry about potential conflicts between those two use cases.

Either way, having separate volumes on the 4TB disk would be the "Best Practice".

Get another external drive.
Format it to HFS+.

The other volume that is working correctly with Canon DPP is also APFS formatted, so I don't think that APFS is the problem. APFS offers native encryption support and better storage efficiency than HFS+, so I'd lean towards APFS for any new hardware if there isn't a specific reason to not use APFS.

 

[MarkC426]

The other volume that is working correctly with Canon DPP is also APFS formatted, so I don't think that APFS is the problem. APFS offers native encryption support and better storage efficiency than HFS+, so I'd lean towards APFS for any new hardware if there isn't a specific reason to not use APFS.

I agree with your statement…..
My external drive is APFS and works fine with Canon DPP.

 

[Levina]

Ah, gotcha. That system is obsolete on the new Mac (and it's funny that Startup Disk would even show it as an option – it won't boot your Apple Silicon Mac). It's harmless to leave the system files on that disk, but personally I'd remove them (e.g. System, Library, Applications; there are others, but that's the bulk of it) just to reduce the noise and any ambiguity about whether that device could function as a startup disk.

Okay. Will do.

Having an OS on the external disk is fine, but if that's what is causing Canon DPP to have trouble managing the photos on that volume, then I'd add a new volume on that 4TB disk that you can use just for your photos. Having separate volumes for different purposes (external boot volume, data storage) goes a long way to avoid unintended consequences. It's certainly easy enough to try to see if it resolves the Canon DPP behavior.

Because if the system fails on that volume the data would be unaffected and recoverable?

And maybe I should clarify – do you regularly boot your Mac from the WDBlack4TB volume, or is that more of a "just in case" deal? Does your backup software make regular backups of your startup disk to WDBlack4TB, or was that just a clean install of the OS on a volume that otherwise only had the photo data? If you only have a clean install of the system on this 4TB disk that otherwise only contained your photos (not the rest of your files, mail, applications, etc), I'd be a little more comfortable with that – in this case the disk has just two purposes: rescue/alt boot and photos, and those don't really conflict with each other. If the 4TB disk is or was a backup of your entire startup disk, and it has your production photos data, and you're regularly updating the backup on that volume, then I start to worry about potential conflicts between those two use cases.

Either way, having separate volumes on the 4TB disk would be the "Best Practice".

No, I don't boot my Mac from the WDBlack4TB volume. It's really a "just in case" deal as you call it. I bought it new 16 months ago, formatted it, installed Ventura on it and manually moved my data onto it.

I make regular back-ups of my startup disk but not to WDBlack4TB but to two other 1TB external drives mostly because I'm terrified of losing email, which happened a decade ago when I lost a few years of emails. Even now it hurts to think of it. It was Time Machine related and it was the reason I started using SuperDuper (in addition still to Time Machine).

The WDBlack4TB is really separate. But it has for the most part the same data as the start up drive (downloads and desktop excluded). I regularly sync certain folders from the start-up drive with the same folders on the WDBlack4TB. The WDBlack4TB itself with all the data and images is regularly backed up to two spinning 4TB drives. I was planning to get a third, in case one of these two fails. They're WD Black drives from my Mac Pro but they're not that old. I also use iCloud for my documents and jpegs.

What I'd do:

Get another external drive.
Format it to HFS+.
DON'T install a copy of the OS onto it.
Move your photo library (nothing else) to the external drive.
Do a get info on the new drive, and choose to "ignore ownership on this volume".
NOW see how the drive (and files) behave when using your internal drive and DPP.

I think installing the OS on the drive was not the way to go. But without SuperDuper I didn't know how else to get a bootable external drive.

The other volume that is working correctly with Canon DPP is also APFS formatted, so I don't think that APFS is the problem. APFS offers native encryption support and better storage efficiency than HFS+, so I'd lean towards APFS for any new hardware if there isn't a specific reason to not use APFS.

Maybe APFS is not the culprit but it's a good suggestion to get another drive. I could format it first as a APFS, move my images onto it, follow Fishrrman's instructions. If it then works as it should with DPP, it works. If not I could try again with HFS+.

 

[bogdanw]

The cause might be the presence of a bootable versions of macOS on the disk. When I go into System Settings – General – Startup Disk and select the clone I made on an external drive, macOS says something about ownership and asks for the admin password.

You could try that and then re-select the internal drive as the startup disk.

 

[Levina]

The cause might be the presence of a bootable versions of macOS on the disk. When I go into System Settings – General – Startup Disk and select the clone I made on an external drive, macOS says something about ownership and asks for the admin password.

You could try that and then re-select the internal drive as the startup disk.

I don't see anything about ownership, just a warning that Startup Disk is trying to unlock System Settings which I can allow by entering my admin password.

 

[MayflyMaven]

Because if the system fails on that volume the data would be unaffected and recoverable?

That's correct, you could delete the System member of that volume group and it would have no effect on the content of the Data volume. Likewise, you could delete the "System" folder from the Data volume (as well as a bunch of hidden system-related folders) without affecting your data. The bulk, or perhaps all of your data (based on your described use of the volume) is going to be in the Users folder on that volume.

But again, I think the safest path forward to test out whether Canon DPP is having trouble with the volume group business on that volume would be to:

1. Select WDBlack4TB in Disk Utility's sidebar
2. Click "+" in the toolbar to add a new volume (name it something like "Photos" or "My Data", I suspect you could be more creative than me
3. Copy all of the relevant data from WDBlack4TB to the new Photos volume (i.e. "same data as the start up drive (downloads and desktop excluded). I regularly sync certain folders from the start-up drive with the same folders on the WDBlack4TB" – copy all of that content to "Photos")
4. Open your Canon DPP project from the new "Photos" volume and see if it has any trouble managing the files on that new volume

If Canon shows the same problem on the new Photos volume, then you can simply delete that new "Photos" volume and you're right back where you started.

If Canon is happy with the new volume, then you can do some cleanup to keep things tidy. If you no longer have any original-source data on WDBlack4TB (because you moved all of it to "Photos"), then you could delete the WDBlack4TB volume group altogether. Or, you could at least remove your data from that volume to avoid any ambiguity about what the original/production source for that data is, then retain the WDBlack4TB as only a utility startup volume.

Lastly, don't forget to update any backup tasks that back up WDBlack4TB – set the Photos volume as the source to those tasks.