Or, ‘when one backup isn’t enough’.
This week, I tried to update my MacBook Pro’s OS from Yosemite to High Sierra. This update failed: after the update, the Mac would hang during startup. No problem, I thought. I’m well prepared for this:
- the MacBook contains a primary SSD and a second bootable harddrive (also Yosemite)
- I maintain 2 sets of Time Machine backups, one on a network volume (hosted by another Mac), one on a harddisk in a USB enclosure. I even test these on occasion by restoring individual files.
Here’s my experiences over the next 2.5 days, as a ‘should be straightforward’ recovery procedure turned up a bunch of pitfalls.
The first step was to boot into the secondary HD. This worked, but the High Sierra upgrade had converted the SSD to the new APFS file system, and Yosemite can’t read that. This left my SSD entirely inaccessible.
How to handle this? Maybe download the High Sierra installer again? I tried that, but App Store downloads require an Apple ID. Oops, that password lives in my 1Password vault which is stored on the SSD, and I didn’t have a paper backup of that vault. I did have Time Machine, so maybe I can recover the 1Password vault and open it.
So, next step: access the Time Machine backup on my network drive. Oops, that backup is encrypted. Again, this password lives in 1Password and in the Keychain, no paper backups. TM backup inaccessible. This was getting scary.
The following day, I remembered the second TM backup, which might still be unencrypted. That turned out to be correct, so I could start recovering. Booted into the Recovery partition, restore from backup. This meant erasing the drive, reformatting with HFS+. I rushed this step, not considering the consequences. You see: my backups weren’t complete. I had excluded some folders from the backup in an effort to reduce the size of the backup. Most of these were expendable (various cache folders), but I had also excluded ~/Pictures, which contains my Photos database.
Anyway, restore from second TM backup, which was 1 month old. This gave me access to Keychain and 1Password, so I could access the network TM backup. Wrote down the passwords for the next time.
Another restore attempt using the network TM backup (which was up-to-date). Oops, this does not show up in Recovery mode.
Booted into the secondary partition, accessed the TM backup by opening its disk image, and I copied everything to the SSD, overwriting the 1 month-old files with newer ones.
After several hours of copying, a restart showed that the OS on the SSD would hang during startup. So that doesn’t work, and it’s unworkable to go through 500,000 files and decide which ones can be safely overwritten and which ones can’t.
Luckily I have lots of external HDs lying around, so I copied the TM bundle to one of them, and was able to access that from Recovery. One more recovery step (4 hours of copying), and I had a usable system again. It gave me one more heart-stopping moment when Mail opened to a “I need to import your old mail” screen instead of opening my mailbox, but after that import everything was there.
In the end, it looks like I’ve lost ~4 years worth of photos, and a few files I’ve recently downloaded. Everything else is intact.
The cost to get to this point: 2.5 days, and stress levels that rose to the downright unhealthy.
Lessons learned:
- Passwords need a backup that is accessible when the system is down.
- Encrypted backups are both safe and dangerous.
- Always make a full backup before installing a new OS.
- The gain from reducing my Time Machine backup size was more than wiped out by the loss of data induced by it.
- Having a second, entirely independent computer on hand was a lifesaver, as I could look things up while the MBP was inoperable.
- Similarly, my habit of not throwing away external harddisks meant I had several of them to play with, copy files to, install recovery tools if necessary, etc.
In hindsight, I should have done these things differently:
- Instead of erasing the SSD, I should have installed High Sierra on an external HD, and booted from there to get access to the SSD.
- I didn’t take enough time to consider the consequences of erasing the SSD. One click and it’s all gone.
Harro de Jong