I’ve been using Macs since forever and have migrated to each new incarnation over the years. To make my migrations easier I’ve tried to maintain the same file structure so I already knew where everything was. I’ve been thinking I ought to ‘clean up’ my 30 years of cruft to hopefully streamline the system and get rid of things that actually no longer work on Mojave (leaving Catalina out of the discussion!)
Now for my novice question. This discussion refers to the Home folder as a place Apple expects to find many things. Is the Home folder referred to the folder in the Users folder that is named the same as the user’s name (as created when each user is created)? [In my case: Macintosh HD > Users > Jack Clay > everything I use.]
That’s all fine and dandy…but there is data that for a lot of users needs to be available to all users. While it is true that in Catalina it can go into /users/shared…for many people putting it 3 directory levels deep is overly complicated. Apple has created a data volume and reserved certain folder names at the root for themselves. Fine. However…I have yet to see any good reason…and “Apple says so” is not a good reason…why a user should not be able to create. Root level folder on the DATA volume, name it whatever they want, and set whatever permissions they want on it…and then expect Apple to leave the darned thing alone when system updates happen.
I would say that this is a case of Apple playing fast and loose with the terminology. They are creating two APFS volumes in order to protect the low-level system files, but they are, for all intents and purposes, a single volume, just like it was back in the HFS+ days.
The only volumes where you can safely put anything in the root are ones that Apple did not create as a part of system installation. Like external USB volumes and any new partitions you might create on internal storage devices.
I realize that this is a major case of culture shock for people used to the way things were done back when personal computers were single user systems (e.g. classic MacOS and the DOS-based versions of Windows like 95/98/Me), but it is an unfortunate side effect of designing systems for multiple users with a robust security model.
(On the other hand, for those of us who used dozens of different Unix-like systems in college, none of this seems the least bit surprising. Except that there isn’t some super-genius system administrator performing all the system maintenance behind the scenes - we are forced to be our own system administrators, with all of the good and bad things that go along with it.)
Yes, Apple Magic produces a single Boot Volume which is really two volumes joined at the hip. Attempting to place items in the root of this (synthesized) Boot volume destroys the magic. Just treat it as a single volume with a heavily controlled root.
Other volumes, especially on additional hard drives or SSDs, are subject to the user’s whims regarding ownership, permissions, format, and the like.
The user can put pretty much any additional folders within the home folder, but may not delete the apple required folders.
Unix soft links (ln -s) can be substituted for some apple required folders and redirect normal file access to somewhere else. An example might be soft-linking a /Volumes/othervolumename/username/Pictures folder to be reachable as /Users/username/Pictures. This does require certain arcane knowledge to accomplish in Catalina.
Any other user created folders within the may be soft-linked to folders elsewhere without restraint by Apple Rules.
And, in any case, data folders in external volumes such as may be on external drives are always accessible, subject to ownership and ACL restrictions.
So, put any additional folders as desired in your folder and use external volumes for bulk storage and backup as needed.
Special Note: Not using the ~/Document folder can help prevent iCloud from stealing away all your files to iCloud even ignoring any lack of iCloud space which may exist. YMMV
Would somebody who actually understands the details be so kind to explain how this APFS magic works? There’s two containers volumes, but just one volume container. The file tree reveals in /Volumes that there’s only one volume, Macintosh HD. But then we get a glimpse of another “volume” when we see that ls -l in / returns home -> /System/Volumes/Data/home. Interestingly enough there now appears to be a “volumes” Data, at least according to the symlink, but Disk Utility actually shows it as Macintosh HD - Data. So where are the mount points for these and how do you find out which parts of the file tree get stored on which volume?
Another question is, “Why should you care.” All user-provided objects are automatically placed on the read/write volume of an APFS composite boot volume. For example, Apple applications, excepting Safari.app, are inside Macintosh HD while Keyboard Maestro.app is inside Macintosh HD - Data but all appear as contents of /Applications or Finder → Macintosh HD → Applications.
I don’t even try to explain APFS magic. An attempt to second-guess Apple is not worth the effort. I point out the most important result of the magic: For most important path specifications, “~/” is the most useful path prefix.
From both the Finder and the Terminal, the view of volumes, mount points, user home directories, and files can be recognized as functionally identical in Catalina to that in Mojave. An application which makes use of hard-coded paths derived from APFS boot volume hidden internal paths will most likely not be portable or even guaranteed to continue to function correctly in the face of Apple’s future changes.
I disagree, and the change was a major part of my original problem/misunderstanding. There was a major change between Mohave and Catalina. As a mere user I’m not up to speed on this talk of volumes, containers, mount point, etc. But I was baffled to find that my hard disk appeared in two different places, one of which was nested inside the other but held different content. How can this not be baffling to a non-geek?
To be clear, we wrote about this twice, once when Catalina was announced, and again when it shipped.
In theory, you should never see what’s going on behind the scenes—Apple does a good job of hiding it. The only time you do is if you mount the drive on a Mac running an older version of macOS, or in Disk Utility. The Disk Utility view is confusing, since APFS introduces the concept of a container, but most users shouldn’t be dealing with Disk Utility on a regular basis. (And the Macintosh HD and Macintosh HD - Data volumes, or whatever they’re called, should be at the same level since they’re both volumes in a container, not one nested inside the other.)
The linked article includes the following information.
When a single APFS container (partition) has multiple volumes, the container’s free space is shared and can be allocated to any of the individual volumes as needed.
I have a 500 GB external drive formatted as APFS with three volumes. (All numbers are approximate. If it’s important, I’ll look them up.) According to Get Info, Volume1 has 40 GB, Volume2 has 80 GB, Volume3 has 80 GB, and the total space available is 340 GB, with 40 GB purgeable. I am confident that all 40 GB, 80 GB, and 80 GB are important to me (in other words, they are valid numbers), so what is purgeable? If I added 340 GB to the disk, I would expect to lose information. Can someone calm my fears, and perhaps even explain this new math? Thanks.
It’s hard to know exactly what’s going on on your system, but I can’t see any reason why you’d lose information—you’d just get an error that the disk was full and the current copy would fail. Nothing new there.
The purgeable statistic is hard to pin down, but could be related to iCloud Drive cached data that could be deleted, or data in the Trash that could be deleted or the like. I don’t know if macOS would actually purge that data in the middle of a copy to make room.
And what about those of us who have small (256GB) SSD’s in their MacBookPro? I have an external 1TB SSD (formatted MacOS extended-non bootable) attached using USB-C to my MacBookPro and used for all my personal data, photos, and non-Mac apps. I do not use the Apple supplied Document, or Photo folders on the system (Mojave) disk for any of my data/apps, although they are there. My data would overwhelm the non-upgradeable 256GB built in drive. Some apps complain about being run from an external drive, so they get transferred to Apple’s assigned App folder, but not many do. Seems to work well, but does anyone see a problem here? I do make separate clones of the system and external disks as well as a Time Machine backup of both.
While I do have documents in iCloud, the external disk that prompted the discussion is not connected to the Macintosh when the Macintosh is connected to the internet, so I wouldn’t expect any cached data. And on that external drive, there is nothing in the Trash. (I Delete Immediately… any items that I no longer want on that disk.)
So, I’m still confused, but as long as nothing would get purged, I guess I don’t care if Apple wants to claim there is purgeable information. Thanks, Adam.
This is a common situation, and generally speaking, it’s not a problem. Just put your data on the external drive however you want. As you note, apps often don’t like to be outside /Applications, but they’re not usually the problem. For many years, I kept my Photos Library on an external drive, and Photos is totally happy to work with a library outside the ~/Pictures folder.
As long as you’re backing up both, you’re totally fine.
I researched this a bit more, and Jason Snell suggests that as of Sierra, Apple combines free and purgeable space. So in your situation, @will_m, it’s all really just free.
I have a 2013 iMac whose internal drive is slow, so I use an OWC Mercury Elite Pro Dual enclosure with two ssd drives as my system and documents residence, and it is working way faster now than off the iMac’s hd. I have the latest Catalina system. Taking the iMac apart for a new drive was too much of a hassle.
Thanks, Adam; I’ll assume the engineers at Apple are smarter than I am. But the numbers still don’t add up.
Perhaps I should have elaborated in my initial query. Two of the three volumes hold backups created by SuperDuper! The third volume contains stuff that I copied (in the Finder) because it’s important to me. Nothing was directly added from iCloud, iTunes, or other internet sources. Unless Apple is clever enough to note that there are some duplicates in the SuperDuper! volumes or that there are some duplicates in the smallest volume (the one that contains the Finder-copied information), I don’t see how anything would be considered purgeable. Get Info tells me that the volumes hold 80 GB, 80 GB, and 40 GB on a 500 GB disk with 340 GB available.
As I said, I’ll assume the engineers at Apple are smarter than I am. But I’m confused.
home folder, but may not delete the apple required folders.