Time Machine blacklist vs. whitelist

TidBITS Talk
1

I’ve been having a very time-consuming problem with Time Machine, which could be solved if it would just let me set it up a little differently. I think what I want is impossible but I just thought I’d mention it in case I’m wrong. (BTW I’m using Time Machine version 1.3, in High Sierra i.e. macOS 10.13.6, which I think is the newest I can run on this old Mac Pro unless I added a Metal-capable video card which I’m unlikely to do.)

The problem is that I often have many volumes mounted, some from internal drives (both SSD and HD) inside the Mac, others external. The way Time Machine works, you have to specify every volume you don’t want it to back up. Which means I have to keep track of all the volumes currently mounted, and remember to make sure all of them are listed in TM’s “don’t backup this” list. (My volumes for TM to back up to are one internal and one external, but I don’t think that’s relevant here.)

I was surprised yesterday when TM started throwing errors, refusing to back up because there “wasn’t enough space” on either my internal or external backup volumes… surprised because they each had almost 100 GB free, and moreover TM normally deletes older backups as needed to make room for new ones. But it was complaining about the need for a lot of space to create a temporary cache or something like that (I don’t remember the error exactly), no mention of deleting older backups.

When I started looking inside the Backups.backupd folders (where TM keeps its backups) I was startled to see that it has been backing up some of my other backup folders (used by other backup programs I run) – TM is only smart enough to recognize its own backup folders. So it was backing up my other backups! Talk about recursion! No wonder it was running out of room so quickly, and needed gigantic temporary cache spaces.

Turns out dealing with this – fixing the problem – is very complicated; simply adding the other non-TM backup volume name to TM’s blacklist (which I’ve now done) won’t automatically delete all those older pointless backup folders. I’m working on that now, and can explain the possible approaches (and gotchas) if anybody’s interested.

Hopefully I’ll be more careful in the future, and make sure to update TM’s blacklist more frequently… but it’s a very long list. There are only two volumes I do want TM to back up – my boot volume and my main documents drive – I don’t want it to back up anything else, ever. What a PITA.

What would solve this whole mess going forward is if TM would offer a “whitelist” instead of a “blacklist” – in other words, a list of volumes I do want to back up, and ignore everything else. I don’t think this is possible, but I just thought I’d ask.

Sorry for the very-long post, but I’ve been struggling with the aftermath for two days and I’m tearing my hair out fixing this mess!

2

Yes, why only a blacklist? Why can’t you ask Time Machine to just back up your User folder, if that’s the only thing you are really interested in?

1 Like
3

There has to be a default, and the default should be “everything”. That’s how TM is marketed: all you have to do is plug in a drive and all your files will be backed up. You could argue that somewhere deep in TM’s settings there could be a “whitelist/blacklist” (are we allowed to use those terms these days?) toggle, I suppose.

4

This webpage seems to have a solution for Time Machine whitelisting using a shell script which I don’t understand. Maybe a shell script person can make heads or tails or it :

:-)

Doug

5

I don’t think so. I read his article, and his problem statement is how to script the exclusion of groups of files (specifically cascading package dependencies for software development). If I understand the OP’s request, this would be a blacklist, not a whitelist.

I also read the manpage for tmutil. There is no addinclusion command, only addexclusion. So not only is there no concept of flipping the default behavior of TM (i.e., back up everything), you can’t even say “Don’t back up any volume” (not even sure how you’d say that), but follow that with some exceptions to the blanket rule. As far as I can tell, the OP is stuck with what he’s currently doing: Making a list of all possible volumes, and ensuring all he does not want backed up are in the exclusion list.

8

TM was always intended to be a backup product for those folks who don’t do backups. Anyone with needs beyond that basic level need to look elsewhere.

Having said that, I use TM (once per day rather than once per hour) and have a very long exclusion list.