Variability in APFS volume size calculation

I am using Big Sur, current version 11.6.2, on a 2014 27" Retina iMac. I find that the free space number on Macintosh HD is very variable. For example, yesterday afternoon after installing a macOS update it was around 350Gb then I rebooted and it jumped to 404Gb, now, 12 hours later, it’s 456 Gb.

Finder appears to recalculate all file sizes every time you look at a volume or folder.

Do I have a problem or is this ‘normal’ behavior?

Update: Just deleted about 8Gb of unneeded Xcode platforms and the free space dropped to 407Gb.

I’m afraid this calculation has become a lot more complicated because of some of the smart things APFS allows macOS to do. While I wouldn’t say the swings you report are ‘normal’, they are definitely not by themselves an indication that something’s amiss.

Howard Oakley recently had a wonderful description of how complicated what we think of as a simple calculation has actually become.

2 Likes

Thank you! I will try to stop worrying.

This sounds like macOS cleaning up Time Machine local snapshots.

When you install the update, all of the old system files are retained by the local snapshots (you probably have 24 hours of them). So when system files are replaced by new ones, your SSD is retaining data for both the new ones and the old ones.

As the snapshots age-out, the blocks used by the old versions lose their references and get freed up when the last reference goes away. Which is why you’re seeing more free space 12 hours later. There will probably be even more 24 hours after the update.

I’m not sure why deleting Xcode platforms caused you to lose a lot of free space, but keep in mind that whatever you delete is being retained by the local Time Machine snapshots. The storage used by the deleted files won’t be returned to the system until the Time Machine snapshots are deleted, over the next 24 hours (assuming you don’t have any other snapshots holding the files - like ones you can manually create using CCC or other tools).

And, of course, all of this can be aggravated by files in the Trash. Those blocks won’t be freed until they are emptied from the Trash and the snapshots holding the trashed files are also deleted. If you have your Mac configured to automatically remove files from Trash after 30 days, that means anything you delete from the Finder will be consuming space for at least 31 days.

2 Likes

Thanks, most helpful. I always empty the Trash as soon as I delete a file or folder so nothing there. As far as I can tell Time Machine snapshots are not on the hard drive. I ran
tmutil listlocalsnapshots /
and it listed a snapshot for every hour the machine was not in sleep mode for the last 24 hours. But a search using the app Find Any File, which checks hidden files and searches all attached disks, finds nothing.
Where are the snapshots stored in Big Sur?

Snapshots need to be explicitly mounted, at which point they will appear as read-only storage volumes.

If you are running Monterey, you should be able to do this with the GUI Disk Utility.

If you are running an older macOS, you must use either the command-line diskutil or a third-party utility (like CCC).

See also my reply in the Migrating a Time Machine backup to a larger drive topic

Good, thanks. On Big Sur because my 2014 27" iMac has been made obsolete by Apple!

Are any snapshots stored on a non-system volume? Do any snapshots have a longer “lifespan” than 24 hours?

I assume you’re talking about the snapshots that are created by Time Machine.

  • TM local snapshots are not created for the system volume, since that one only changes when macOS is updated.
  • TM local snapshots are created for any APFS volume Time Machine backs up. They are created whenever TM runs (normally hourly) and the last 24 hours worth are retained. But they may be purged prematurely if necessary to create free space when necessary.
  • If you use an APFS volume for Time Machine storage, it will also create snapshots there. One for every backup stored on the volume.

You can see Time Machine’s policy in its system preference page:

Snapshots created by other applications are not automatically deleted by Time Machine. The software that creates them is expected to manage them. You can manually delete them if you feel you need to.

On Monterey, the GUI Disk Utility includes tools to easily create/destroy snapshots. On earlier versions of macOS, the command-line diskutil is needed for this, or you must use a third-party software tool.

1 Like

I’m not sure. If I’m following the discussion, @fellwalker57 initially asked about his or her startup volume, and I did not get the sense that the snapshots that were created there were necessarily created by Time Machine. Let me back up.

If one does not check Back Up Automatically in System Preferences > Time Machine, would Time Machine create snapshots on the startup volume? If one does check Back Up Automatically, doesn’t Time Machine require a non-startup volume as a backup target? What program was perhaps creating snapshots on @fellwalker57’s Macintosh HD?

My question was based on the perception that Time Machine, or some program that used the same routines as Time Machine, was creating snapshots of the startup volume on the startup volume, either with or without Back Up Automatically being checked. The discussion led me to conclude that the snapshots saved information about the disk on which the snapshots were stored. This led me to wonder if snapshots of non-startup volumes were saved on the non-startup volumes, and that’s what I trying to ask. Is that clearer now or have I just muddled things even more?

To clarify, yes I was asking about Finder’s report of the free space on the startup volume. Also, my TM is of course an external drive. David explained that some of the variation in free space may be caused by creation / deletion of Time Machine snapshots. I am not aware of any other program on my system taking snapshots.

I do not know where the snapshots are stored though again David explained why they are not visible to Finder etc.

The free space calc. on my system has now settled down, 24 hours after I installed the macOS update, and is showing a number that makes sense.

Finally, I appreciate Will’s sensitivity about gender but you can stick to male from now on.

If Time Machine is disabled (not checked Back Up Automatically), then Time Machine won’t be creating any local snapshots.

Yes, Time Machine is supposed to use an external storage volume. But even then, it also creates 24 hours worth of hourly local snapshots on APFS volumes (plus one corresponding to the most recent external backup, plus one created just before any macOS update). This serves a few purposes:

  • If your TM volume is off-line at the time a backup is made (e.g. if it’s a laptop and you’re on the go), it won’t get lost - the snapshot will record the backup
  • If your TM volume is off-line at the time you try to recover a file, you have the last 24 hours worth of updates to recover from.
  • I think TM can better handle files that are in-use with snapshots. Instead of just copying files to the external device, it creates a snapshot and then copies filed from the snapshot, thus ensuring that the files being backed up don’t change while the backup is in progress.

See also About Time Machine local snapshots - Apple Support

Disabling TM should cause macOS to delete all of TM’s local snapshots. If TM is disabled and there are are still snapshots on the volume, then something else created them. That something else should have its own rules for when they are created and destroyed. For example:

  • On Big Sur and later, macOS boots from a cryptographically signed snapshot of the System volume. This is designed to protect against the volume getting corrupted. This snapshot can only be created/deleted by macOS installers.

    This snapshot is only created for the System volume, not for the Data volume bound to the system volume or any additional volumes you may have connected.

  • You can manually create snapshots with a variety of tools. If you do, they won’t go away until you manually delete them.

  • Some backup software, like CCC, can be configured to use snapshots as a part of the backup strategy. The software should document when it creates and destroys them.

    In the case of CCC, there is a configurable retention policy that determines how they are used on your source volume. They are used to preserve historic backups on destination volumes. See also Leveraging Snapshots on APFS Volumes | Carbon Copy Cloner | Bombich Software

Regarding the original question, I am assuming that @fellwalker57 is using Time Machine. If he isn’t, then there must be another explanation for his observation. If it is snapshots (I still think it is most likely) then, something else (maybe a different backup utility) is creating and destroying them.

1 Like

Yes, I am using TM and I think that is the most likely explanation for the fluctuations in the free space calculation.

Thank you for the education and the links for further education. Especially the CCC page helped me gain what I think is an understanding. A lot of that I had seen before, but didn’t understand. This time, I think it made more sense.

I had not understood that, and that was the source of my confusion. Thanks again.