Auto-mount encrypted drive for multiple user accounts

[nello]

@Shamino

If you move the folder, the ACLs/EAs should move with it.

So, what I hear you saying is that for each of the directories that I’m going put on the external drive, there are two steps:

1. Move the directory from the internal drive to the external drive; and
2. Create a symlink in exactly the same place on the internal drive with exactly the same name and permissions and point it to the moved directory on the external drive.

If this is correct, can I do step one with a Finder move or must it be done in Terminal?

Or perhaps I should make the fist step a copy instead of a move and then compare the two directories with a ls -l@e to make sure that they are exactly the same before deleting the directory from the internal drive.

And I suppose that I should upgrade my User to have Administrative privileges before making any of these changes. (I noticed earlier tonight that I wasn’t able delete the Documents directory when my User was just a regular account.)

Thank you.

UPDATE Sunday, January 12, 2025 5:37 PM

This 2019 source says there is no way to to copy a directory that includes ACLs and Extended Attributes in Terminal and that you can see what’s different between the original and the copy with ls -leah@O; a complete copy can be done only with a Finder copy followed by a Finder paste while holding SHIFT+Option+Command, Paste Item Exactly:

How do I copy user home directories from one system to another while preserving all attributes?

Coincidentally, I’ve seen YouTube videos, i.e., those by Mac Sound Solutions, that show how to move an entire Home directory just a normal Finder copy/paste, including this one starting at 8:15:

3 Easy Ways to Move Your Home Folder to an external SSD Sequoia 15 2 made it harder

At 4:03 he demonstrates using Terminal to faithfully duplicate a directory and its content with the command:

sudo sync -avh --progress ~/

By the way, during early 2022 Howard Oakley recommends Tinker Tool for showing and setting ACLs with a demonstration of using it in a Home directory. Unfortunately, toward the end of 2022 he concludes that repairing home folder permissions is a mystery.

[Shamino (David C.)]

@nello

So, what I hear you saying is that for each of the directories that I’m going put on the external drive, there are two steps:

1. Move the directory from the internal drive to the external drive; and
2. Create a symlink in exactly the same place on the internal drive with exactly the same name and permissions and point it to the moved directory on the external drive.

Correct. Symbolic links don’t have any permissions (or more to the point, you shouldn’t care about the permissions of the link). The permissions of the file/directory to which the link refers will be what controls access to the content.

@nello

If this is correct, can I do step one with a Finder move or must it be done in Terminal?

Step 1 can be in the Finder. I would actually recommend that.

Creating the symlink, however, must be done from a terminal, unless someone here knows of a GUI utility for doing it.

@nello

Or perhaps I should make the fist step a copy instead of a move and then compare the two directories with a ls -l@e to make sure that they are exactly the same before deleting the directory from the internal drive.

I would suggest experimenting with a temporary account that you can play with and discard when you’re done experimenting. I wouldn’t want to try it the first time using an account containing real data.

@nello

And I suppose that I should upgrade my User to have Administrative privileges before making any of these changes. (I noticed earlier tonight that I wasn’t able delete the Documents directory when my User was just a regular account.)

I can’t say about that. My system (which has directories originally created in 2002 and migrated through many different Macs and macOS releases over the years) doesn’t have any ACLs on these folders. But yours does.

I suspect one or more of the ACLs is what is preventing you from deleting the directory. An administrator login should work. Or you could consider reviewing the specific ACLs and see about removing them. Maybe re-creating them after the move, if you think they are important.

Again, these are things I’d want to experiment with using a throw-away account.

We’re talking about things I haven’t done before and I wouldn’t want to accidentally trash something important on your system.

@nello

This 2019 source says there is no way to to copy a directory that includes ACLs and Extended Attributes in Terminal

Note that it’s a 2019 source and was tagged “high-sierra”. Apple has improved the capabilities of the various command-line tools since then. So it might not work exactly the same today.

But yes, I’d agree that using the Finder to make the copy is probably going to be the most reliable method of preserving all attributes.

[nello]

@Shamino

As for specific folders you mentioned:…

I found this summary table of what can and cannot be moved to an external drive and whether a symbolic link (or other method is used to do so):

Specific instructions including Terminal commands and troubleshooting in this PDF:

Cheatsheet for YouTube video How to Safely Move Mac Folders to an External SSD.pdf (138.4 KB)

Source:
Dated November 11, 2024

How to Safely Move Mac Folders to an External SSD (Step-by-Step Guide)

[nello]

@Shamino

@nello

So, what I hear you saying is that for each of the directories that I’m going put on the external drive, there are two steps:

1. Move the directory from the internal drive to the external drive; and
2. Create a symlink in exactly the same place on the internal drive with exactly the same name and permissions and point it to the moved directory on the external drive.

Step 1 can be in the Finder. I would actually recommend that.

I couldn’t figure out a way to use the Finder to copy/move the directories with their ACLs and Extended Attributes intact. A regular Finder Drag-and-Drop as well as a regular copy/paste creates the directory with the same name, ownership, and permissions but the ACLs and Extended Attributes are NOT included on the destination. Even Finder paste while holding Shift+Option+Command, Paste Item Exactly results in this error:

Furthermore, I also wasn’t able to find a flag for cp or mv that would keep ACLs and Extended Attributes intact at the destination.

Exasperated, I went back to this video and studied the three methods that he says can be used to move a Home directory to an external drive.

1. Drag-and-Drop while logged into another Administrator’s account doesn’t work because it changes ownership;
2. Syncing the Home directory to the external drive might have worked but I didn’t try it because (a) I wasn’t comfortable with the flags, and (b) he mentioned glitches that he ran into; and
3. Draging-and-Dropng a Time Machine back up of the Home directory to the external drive worked perfectly:

Internal Drive

nello@miniMe ~ % pwd
/Users/nello
nello@miniMe ~ % ls -l@e
total 0
drwx------@  5 nello  staff   160 Jan 16 15:18 Desktop
	com.apple.macl	  72 
 0: group:everyone deny delete
drwx------@  3 nello  staff    96 Jan 16 14:15 Documents
	com.apple.macl	  72 
 0: group:everyone deny delete
drwx------+  3 nello  staff    96 Jan 16 14:15 Downloads
 0: group:everyone deny delete
drwx------+ 84 nello  staff  2688 Jan 16 14:35 Library
 0: group:everyone deny delete
drwx------   3 nello  staff    96 Jan 16 14:15 Movies
drwx------+  3 nello  staff    96 Jan 16 14:15 Music
 0: group:everyone deny delete
drwx------+  5 nello  staff   160 Jan 16 14:38 Pictures
 0: group:everyone deny delete
drwxr-xr-x+  4 nello  staff   128 Jan 16 14:15 Public
 0: group:everyone deny delete
nello@miniMe ~ %

External Drive

nello@miniMe nello % pwd
/Volumes/Sidecar/Users/nello
nello@miniMe nello % ls -l@e
total 0
drwx------@ 3 nello  staff   96 Jan 16 14:15 Documents
	com.apple.macl	 72 
 0: group:everyone deny delete
drwx------+ 3 nello  staff   96 Jan 16 14:15 Downloads
 0: group:everyone deny delete
drwx------  3 nello  staff   96 Jan 16 14:15 Movies
drwx------+ 3 nello  staff   96 Jan 16 14:15 Music
 0: group:everyone deny delete
drwx------+ 5 nello  staff  160 Jan 16 14:38 Pictures
 0: group:everyone deny delete
drwxr-xr-x+ 4 nello  staff  128 Jan 16 14:15 Public
 0: group:everyone deny delete
nello@miniMe nello %

The next step was to delete the following directories on the internal drive and replace each of them with a symlink that points to the corresponding directory on the external drive:

1. ~/Documents
2. ~/Downloads
3. ~/Movies
4. ~/Music
5. ~/Pictures
6. ~/Public

For each of these six directories, I:

1. Removed the ACLs with chmod -RN, and
2. Deleted the contents and the directory itself with rm -iR.

Unfortunately, macOS recreates these directories if they are missing so the only way that I could ultimately replace the directory with a symbolic link was with a stacked conditional command in the form:

rmdir directoryName && ln -s /Volumes/Sidecar/Users/nello/directoryName ~/directoryName

This worked for ~/Documents and ~/Downloads but resulted in the error, rmdir: directoryName: Operation not permitted for ~/Movies, ~/Music, and ~/Pictures, leaving me with:

nello@miniMe ~ % pwd 
/Users/nello
nello@miniMe ~ % ls -l@e
total 0
drwx------@  5 nello  staff   160 Jan 16 15:18 Desktop
	com.apple.macl	  72 
 0: group:everyone deny delete
lrwxr-xr-x   1 nello  staff    38 Jan 16 19:17 Documents -> /Volumes/Sidecar/Users/nello/Documents
lrwxr-xr-x   1 nello  staff    38 Jan 16 19:21 Downloads -> /Volumes/Sidecar/Users/nello/Downloads
drwx------+ 84 nello  staff  2688 Jan 16 14:35 Library
 0: group:everyone deny delete
drwx------   2 nello  staff    64 Jan 16 19:23 Movies
drwx------   2 nello  staff    64 Jan 16 19:24 Music
drwx------   2 nello  staff    64 Jan 16 19:32 Pictures
lrwxr-xr-x   1 nello  staff    35 Jan 16 19:26 Public -> /Volumes/Sidecar/Users/nello/Public

Interestingly, each of these directories corresponds to an application that controls placement of files.

1. Movie
   In the application TV > Settings > Files, change /Internal/Users/nello/Movies/TV/Media to /Sidecar/Users/nello/Movies/TV/Media
   
   
2. Music
   In the application Music > Settings > Files, change /Internal/Users/nello/Music/Music/Media to /Sidecar/Users/nello/Music/Music/Media
   
   
3. Pictures
   In the application Photos create new library in /Sidecar/Users/nello/Pictures

What else can I try to make ~/Movies, ~/Music, and ~/Pictures into symbolic links?

Thank you for your help.

[Shamino (David C.)]

@nello

Even Finder paste while holding Shift+Option+Command, Paste Item Exactly results in this error:

Interesting. I usually only see this if the media is formatted with something that doesn’t respect file ownership. If the destination is formatted as APFS and is not configured with the “Ignore ownership on this volume” option (see below), you shouldn’t see this.

Make sure this option isn’t checked. It may be the default for newly-formatted external media (since ownership can make it difficult to use the volume on other Macs, which may not have the same users/groups configured).

But if you’re using this volume to store per-user content (like home directory contents) then you absolutely do not want to be ignoring ownership, since it would allow every user to access everything on the volume, when you only want a user to be able to access his own content.

@nello

Operation not permitted for ~/Movies, ~/Music, and ~/Pictures,

Also interesting. I wonder why Apple cares.

But you can still ignore that and just move the content elsewhere. Your libraries for the Music and Photos apps (and I assume TV and Books as well) can all be moved. If the app doesn’t auto-find it after the move, you can hold Option when launching and then select the library. Or you can double-click the library to launch the app. On the next launch, the app should auto-load the last-used library.

[nello]

@Shamino

@nello

Operation not permitted for ~/Movies, ~/Music, and ~/Pictures,

… I assume [your libraries for] TV and Books … can all be moved.

Yes, the TV app > Settings > File puts a TV library in ~/Movies by default and allows you to change this location.

However, I can’t find way to change the location of the Books library, which apparently is (or used to be) in ~/Library/Containers/com.apple.BKAgentService/Data/Documents/iBook/Books

Perhaps @james.cutler knows the Books library default location and how to change it in macOS 15.2.

@james.cutler

drwx------+ 108 xz4gb8  staff     3456 Jan  1 15:57 Library
lrwxr-xr-x    1 root    staff       35 Mar 28  2023 Movies -> /Volumes/MaxiJimMedia/xz4gb8/Movies
lrwxr-xr-x    1 root    staff       34 Mar 28  2023 Music -> /Volumes/MaxiJimMedia/xz4gb8/Music
lrwxr-xr-x    1 root    staff       37 Mar 28  2023 Pictures -> /Volumes/MaxiJ

James, how did you manage to create symlinks for ~/Movies, ~/Music, and ~/Pictures?

Did you do it from macOS Recovery? Turn off System Integrity Protection (SIP)?

How to delete directory ~/Pictures?

---

UPDATE January 17, 2025

I restarted in macOS Recovery—no need to turn off SIP—and was able to use Terminal to delete ~/Movies, ~/Music, and ~/Pictures.

Then I restarted and logged into my user and created the symbolic links successfully:

nello@miniMe ~ % pwd
/Users/nello
nello@miniMe ~ % ls -l@e
total 0
drwx------@  4 nello  staff   128 Jan 16 20:40 Desktop
	com.apple.macl	  72 
 0: group:everyone deny delete
lrwxr-xr-x   1 nello  staff    38 Jan 16 19:17 Documents -> /Volumes/Sidecar/Users/nello/Documents
lrwxr-xr-x   1 nello  staff    38 Jan 16 19:21 Downloads -> /Volumes/Sidecar/Users/nello/Downloads
drwx------+ 85 nello  staff  2720 Jan 16 21:30 Library
 0: group:everyone deny delete
lrwxr-xr-x   1 nello  staff    35 Jan 17 13:13 Movies -> /Volumes/Sidecar/Users/nello/Movies
lrwxr-xr-x   1 nello  staff    34 Jan 17 13:13 Music -> /Volumes/Sidecar/Users/nello/Music
lrwxr-xr-x   1 nello  staff    37 Jan 17 13:14 Pictures -> /Volumes/Sidecar/Users/nello/Pictures
lrwxr-xr-x   1 nello  staff    35 Jan 16 19:26 Public -> /Volumes/Sidecar/Users/nello/Public

And, yes, I can do a cold boot, log into my user (on the internal drive), and the encrypted external drive is automatically mounted. (Yes, the first time a dialog popped up asking for the external drive’s encryption key; I was able to log into my password manager, copy the key from there, and paste it into the dialog box and check the box to store it on my Keychain. Since then, external drive mounts without any interaction on my part, presumably using the encryption key from my Keychain. More precisely, there is a brief pause in the login progress bar and then it seems to start all over, perhaps after the external drive is mounted.)

Yippee! It all seems to work!

UPDATE Wednesday, January 22, 2025 8:17 PM

AirDrop won’t work correctly if ~/Downloads is a symlink; it must be a directory.

[james.cutler (James R Cutler)]

@nello

Perhaps @james.cutler knows the Books library default location and how to change it in macOS 15.2.

James, how did you manage to create symlinks for ~/Movies, ~/Music, and ~/Pictures?

Books.app stores media in obfuscated files in ~/library. I can not depend on Apple to keep it in any particular place. I am migrating away from Books as fast as I can to limit the size of my Home directory, ~/.

As to how to create softlinks, you have done well. Yippee, indeed.

But, just for completeness. here is the latest version of my CreateMediaSoftLinksTemplate.command file. I did use Carbon Copy Cloner to copy the Source folders to the external drive. As usual, I was using the latest macOS release when I used this. I have not lost any links through macOS updates.

#!/bin/sh
# 2018-11-01	jrc CreateMediaSoftLinks.command
# 2019-10-17	jrc Updated with edits from Ric Ford
# 2019-10-17	jrc Updated to use variables for shell command customization
# 2019-10-17	jrc Updated to increase verbosity of comments

# CreateMediaSoftLinks for ThisUser on ThisMaC

#   "ThisMacMedia" is a placeholder for the external volume to hold the media files.
#   "ThisUser" is a placeholder for the short username of the file owner affected.
#    Replace these placeholders with the desired values when creating and copying in Finder.

#    Edit the next two shell commands to automatically replace these placeholders 
#		with the desired values for the shell script actions.

ThisMacMedia="Put your external media volume name here";
ThisUser="Put your short username here";

#   Knowledge of disk management is assumed, including erasing, partitioning, and setting ownership and access.

#==================================================================
# PREPARATION STEPS:
#
#   Make your backups -- failure to do so is on your head!
#
#   These steps use Finder so that you don’t have to even think about permissions. 
#.  Logged on as ThisUser, do the following Finder creates and copies:
#
#   Create folder /Volumes/ThisMacMedia/ThisUser
#   —  The use of ThisUser folder on ThisMacMedia isolates user permissions from volume permissions
#		and keeps the user account files separate from other volume contents. 
#   Create folder /Volumes/ThisMacMedia/ThisUser/Movies
#   Create folder /Volumes/ThisMacMedia/ThisUser/Music
#   Create folder /Volumes/ThisMacMedia/ThisUser/Pictures
#
#   Copy contents of /Users/ThisUser/Movies to /Volumes/ThisMacMedia/ThisUser/Movies
#   Copy contents of /Users/ThisUser/Music to /Volumes/$ThisMacMedia/ThisUser/Music
#   Copy contents of /Users/ThisUser/Pictures to /Volumes/$ThisMacMedia/ThisUser/Pictures
#
#   Make at least a cursory check of the copied files.
#
#==================================================================

# The following steps should be done from administrative account other than ThisUser
# and may require booting from another volume if Mojave security rules interfere with rm commands.

# Remove existing directories in  boot drive ThisUser account folder
sudo rm -rf /Users/$ThisUser/Movies;
sudo rm -rf /Users/$ThisUser/Music;
sudo rm -rf /Users/$ThisUser/Pictures;

# Create Soft Links
sudo ln -s  /Volumes/$ThisMacMedia/$ThisUser/Movies		/Users/$ThisUser/Movies;
sudo ln -s  /Volumes/$ThisMacMedia/$ThisUser/Music		/Users/$ThisUser/Music;
sudo ln -s  /Volumes/$ThisMacMedia/$ThisUser/Pictures	/Users/$ThisUser/Pictures;

exit;

[nello]

@Shamino

@nello

@Shamino

3. Library
   I wouldn’t want to move this one. …

I’m mostly concerned with files downloaded from cloud services eating up space on the internal drive. And yes, I want to force downloads so that I can back them up locally and restore them as needed.

Normally, I wouldn’t concern myself with it, because all cloud storage services have a system for offloading local copies of remote files. So there’s no danger of filling the volume. But you explicitly said that you want local copies of everything for a backup, so that’s not going to help.

After doing a fresh install of Dropbox for macOS on File Provider, I was able to put my files on an external drive by following these instructions:

External drive support for Dropbox on File Provider

Prior to doing this, my Dropbox files downloaded to ~/Library/CloudStorage and “Dropbox” appeared in the Finder Sidebar in the Locations category.

As you recall (and can see in the directory listing above), I successfully replaced the Documents directory in my Home directory with a symlink to my external drive at /Volumes/Sidecar/Users/nello/Documents. I configured Dropbox to put its files at ~/Documents/CloudStorage/Dropbox, which put these files on my external drive at /Volumes/Sidecar/Users/nello/Documents/CloudStorage/Dropbox.

After the processing finished, clicking on “Dropbox” in Locations brought up a message saying “This is an unlinked Dropbox” and clicking on the button to remove the unlinked files removed “Dropbox” from Locations.

So now I have a local copy of my Dropbox files and they do not take up any room on the internal dive:

nello@miniMe library % pwd
/Users/nello/library
nello@miniMe library % ls -la CloudStorage
total 24
drwxr-xr-x+  5 nello  staff   160 Jan 19 18:56 .
drwx------+ 93 nello  staff  2976 Jan 18 08:40 ..
-rw-r--r--@  1 nello  staff  8196 Jan 19 20:48 .DS_Store
drwx------@  9 nello  staff   288 Jan 17 15:47 Box-Box
drwx------@ 10 nello  staff   320 Jan 17 15:56 OneDrive-Personal

Now that I know that macOS File Provider isn’t inconsistent with storage on an external drive, hopefully I also can find a way to sync files from these other cloud services to my external drive instead of on the internal drive at ~/Library/CloudStorage:

1. Box
2. One Drive
3. Google Drive

UPDATE 2025-01-30T21:56:00Z

Google Drive can NOT be moved if using File Provider, according to Google Drive Help. (Scroll down and and click on “Learn how File Provider differs from the alternative streaming method,” which reveals a table that includes this row):

Feature	File Provider	Alternative streaming method
Find your files	You can find your files in the Finder sidebar, under “Locations.”For advanced users: You can find them in “/Library/CloudStorage” by default. You can’t change this location.	You can find your files in the Finder sidebar, under “Favorites.”For advanced users: You can find them in “/Volumes/GoogleDrive” by default. You can change this location.

UPDATE 2025-10-13T16:25:00Z

Now that I’m finally getting around to implementing @Shamino’s recommendation of using Carbon Copy Cloner (CCC) to back up cloud storage volumes syncronized via macOS’ FileProvider service, I read the CCC Knowledge Base article on this subject very carefully. It appears to me that I need move Dropbox back to ~/Library/CloudStorage for two reasons:

1. CCC’s stanard method of backing up cloud volumes applies to only volumes found in ~/Library/CloudStorage, which is on the internal SSD. I can’t see how to make a single Backup Task’s Source include a directory on an external SSD as well as ~/Library/CloudStorage. Yes, I could create a separate Task for the external SSD, but doing is a slippery slope towards a Rube Goldberg backup because ….
2. CCC recommends backing up the Cloud Volumes separately from the backup of all non-FileProvider files because doing so simplifies restoring these files.

Unfortunately, the Dropbox app does not permit moving the Dropbox folder location from the external SSD to ~/Library/CloudStorage.

I followed the instructions for an advanced reinstall.

And now all four cloud storage volumes are in ~/Library/CloudStorage, which is their default location:

nello@miniMe % cd ~/Library/CloudStorage
nello@miniMe CloudStorage % ls -la
total 32
drwxr-xr-x+   7 nello  staff    224 Oct 13 10:54 .
drwx------+ 104 nello  staff   3328 Jul 20 10:45 ..
-rw-r--r--@   1 nello  staff  14340 Oct 13 12:13 .DS_Store
drwx------@   9 nello  staff    288 Jan 17  2025 Box-Box
drwx------@  37 nello  staff   1184 Oct 13 11:56 Dropbox
dr-x------@   7 nello  staff    224 Jan 30  2025 GoogleDrive-←snip→ 
drwx------@  11 nello  staff    352 Oct 12 11:12 OneDrive-Personal
nello@miniMe CloudStorage %

[tidbits35 (tidbits@malcolmhall.com)]

Another way to solve the original problem is make an Admin user that has their home folder on the internal as normal. Store the external disk password in their keychain (by mounting it with disk utility and choosing to save the password). Set their account to auto log in. Use Shortcuts to make a dock application that either locks the screen or logs out. Add that application as a login item. Now after every reboot the external will be mounted and the users whose home-folders are located on the external can log in.

[jzw (Jolin)]

@tidbits35

Another way to solve the original problem is make an Admin user that has their home folder on the internal as normal. Store the external disk password in their keychain (by mounting it with disk utility and choosing to save the password). Set their account to auto log in.

This is a good idea but from a security perspective I would advise not making the auto-login account an admin. Even if you have something that immediately runs and locks/logs out, that is trivial to bypass (hold down shift on startup) and then anyone with access to the machine has access to every user and all their data. Being admin isn’t necessary for auto-login or storing the external disk password in the keychain, so this solution should work with a normal user.

[tidbits35 (tidbits@malcolmhall.com)]

@jzw

then anyone with access to the machine has access to every user and all their data.

I don’t think that is correct (they still need the password) but interested to hear why you think so?

admin@Mac-mini ~ % pwd
/Users/admin
admin@Mac-mini ~ % cd /Volumes/External/Users/user/Documents
cd: permission denied: /Volumes/External/user/mh/Documents

admin@Mac-mini ~ % ls -la /Volumes/External/Users/user/ 
...
drwx------+  3 user    staff     96 15 Jun 14:35 Documents

But I agree the auto-mounter user might as well not be an Admin.

[jzw (Jolin)]

If you put sudo before your cd (or any command) it will ask for your admin’s user password and then give you full access. If you try and make a copy of the user’s Documents folder in the Finder, it will give you an authentication dialogue where you can put in your admin username and password and it will then make a copy of the folder with permissions that allow you to browse it. Not to mention that you can do a sudo chmod -R o+rwx /Volumes/External/Users/user/ command in the Terminal and that will change all the permissions on all the folders and subfolders in the user’s directory so that anyone can browse/open/edit. Basically, if you are using an admin account you can override all permissions on (almost) all of a user’s data. It won’t give you access to their Keychain or password manager, but pretty much anything else stored on the Mac.

[tidbits35 (tidbits@malcolmhall.com)]

@jzw

If you put sudo before your cd (or any command) it will ask for your admin’s user password and then give you full access.

All good then because they are unable to use sudo if they aren’t able to enter the admin user’s password because someone who reboots and holds shift to by pass the lock screen login item doesn’t know the password.

[jzw (Jolin)]

Ah, good point, I got a bit confused in the chain of events! I’d probably still opt for a standard user to reduce the chance of a privilege escalation loophole I’m unaware of, but I agree it’s less worrisome than I envisaged.