Recovery disasters

Or, ‘when one backup isn’t enough’.

This week, I tried to update my MacBook Pro’s OS from Yosemite to High Sierra. This update failed: after the update, the Mac would hang during startup. No problem, I thought. I’m well prepared for this:

  • the MacBook contains a primary SSD and a second bootable harddrive (also Yosemite)
  • I maintain 2 sets of Time Machine backups, one on a network volume (hosted by another Mac), one on a harddisk in a USB enclosure. I even test these on occasion by restoring individual files.

Here’s my experiences over the next 2.5 days, as a ‘should be straightforward’ recovery procedure turned up a bunch of pitfalls.

The first step was to boot into the secondary HD. This worked, but the High Sierra upgrade had converted the SSD to the new APFS file system, and Yosemite can’t read that. This left my SSD entirely inaccessible.

How to handle this? Maybe download the High Sierra installer again? I tried that, but App Store downloads require an Apple ID. Oops, that password lives in my 1Password vault which is stored on the SSD, and I didn’t have a paper backup of that vault. I did have Time Machine, so maybe I can recover the 1Password vault and open it.

So, next step: access the Time Machine backup on my network drive. Oops, that backup is encrypted. Again, this password lives in 1Password and in the Keychain, no paper backups. TM backup inaccessible. This was getting scary.

The following day, I remembered the second TM backup, which might still be unencrypted. That turned out to be correct, so I could start recovering. Booted into the Recovery partition, restore from backup. This meant erasing the drive, reformatting with HFS+. I rushed this step, not considering the consequences. You see: my backups weren’t complete. I had excluded some folders from the backup in an effort to reduce the size of the backup. Most of these were expendable (various cache folders), but I had also excluded ~/Pictures, which contains my Photos database.

Anyway, restore from second TM backup, which was 1 month old. This gave me access to Keychain and 1Password, so I could access the network TM backup. Wrote down the passwords for the next time.

Another restore attempt using the network TM backup (which was up-to-date). Oops, this does not show up in Recovery mode.

Booted into the secondary partition, accessed the TM backup by opening its disk image, and I copied everything to the SSD, overwriting the 1 month-old files with newer ones.

After several hours of copying, a restart showed that the OS on the SSD would hang during startup. So that doesn’t work, and it’s unworkable to go through 500,000 files and decide which ones can be safely overwritten and which ones can’t.

Luckily I have lots of external HDs lying around, so I copied the TM bundle to one of them, and was able to access that from Recovery. One more recovery step (4 hours of copying), and I had a usable system again. It gave me one more heart-stopping moment when Mail opened to a “I need to import your old mail” screen instead of opening my mailbox, but after that import everything was there.

In the end, it looks like I’ve lost ~4 years worth of photos, and a few files I’ve recently downloaded. Everything else is intact.
The cost to get to this point: 2.5 days, and stress levels that rose to the downright unhealthy.

Lessons learned:

  1. Passwords need a backup that is accessible when the system is down.
  2. Encrypted backups are both safe and dangerous.
  3. Always make a full backup before installing a new OS.
  4. The gain from reducing my Time Machine backup size was more than wiped out by the loss of data induced by it.
  5. Having a second, entirely independent computer on hand was a lifesaver, as I could look things up while the MBP was inoperable.
  6. Similarly, my habit of not throwing away external harddisks meant I had several of them to play with, copy files to, install recovery tools if necessary, etc.

In hindsight, I should have done these things differently:

  1. Instead of erasing the SSD, I should have installed High Sierra on an external HD, and booted from there to get access to the SSD.
  2. I didn’t take enough time to consider the consequences of erasing the SSD. One click and it’s all gone.

Harro de Jong

2 Likes

Thank you for your courage and humility to public share this story, Harro. This is a good reminder that even experienced Mac users can overlook one or two minor, yet crucial, details that can turn a minor computer hiccup into a very stressful event.

Your “lessons learned” section is spot-on, particularly the item about making a full backup before proceeding with an OS upgrade. Two things I’d like to add to that list, are:

  1. Create that full backup in the form of a bootable clone, and boot from that clone before proceeding with the update, so you know that it actually does work as intended. That gives you a truly up-to-date and complete snapshot of what’s on your computer. And it being bootable should reduce the reliance on a second computer to near-zero.

  2. Purchase your password manager’s companion mobile app, install it on your phone, and set up synching. Even if your Mac should become completely unusable, you’ll at least have access to all of your login credentials, as well as (hopefully!) your key personal data, such as bank accounts, etc. If you’re one of those people who drag their mobile phone everywhere they go, having that data along with you can also be a low-effort, yet integral part of disaster preparedness.

3 Likes

As well as Time Machine I use Carbon Copy Cloner once a macOS update appears to be working correctly. In theory I can then boot from the clone, restore more recent files from Time Machine and be on my way. Your experience suggests theory and practice don’t always align so thank you for the alert.

Incidentally, it was similar recovery issues that led me to abandon Windows and switch to Mac in 2003!
http://users.tpg.com.au/users/aoaug/ms_dig.html#nightmare

2 Likes

Oh, ouch. I’m so sorry. I was on the edge of my seat reading; and my heart ached when you confirmed the loss of photos, despite an otherwise seemingly thorough backup strategy.

[edit: sorry, I didn’t grok where it was that you were missing your iCloud password and couldn’t reinstall for lack thereof. Yikes. That’s one of four passwords that is always in my head, regardless of encrypted vaults. Bank, macOS login, iOS login, and AppleID. ]

I’m still left wondering why, if you did try, that you couldn’t try again to install (reinstall) High Sierra after the first boot into the new system failed. Did I misread that after the APFS conversion appeared to bork that the SSD was not even seen in Recovery? Or was it that you just immediately went to Yosemite on HDD?

Again, my sincere condolences, and thanks for sharing your pain to highlight to others the pitfalls and possible of even apparent due diligence.

And I agree with others, nothing short of a full bootable clone makes me feel safe prior to a major upgrade. But the encrypted password vault is one I’d not have thought of had you described your strategy in advance; I wouldn’t have realized from your description that you were facing a risk factor that could’ve just as easily come from a dead SSD. And since my Keychain and 1Password vaults also sync to iOS and other macOS, my own ingrained comfort would’ve made me overlook this critical assumption.

2 Likes

Sorry, just reread it again (three times), and what I derived was that your photos weren’t being backed up anywhere, at all? Since you eventually gained access to both Time Machine volumes, both external (unencrypted) and network (“full”, encrypted), isn’t the real lesson here to make at least one full backup of your data, regardless of upgrade issues versus dead or stolen Mac? It’s not that two backups aren’t enough; it’s that at least one of them has to be complete, is it not?

Again, sorry if I’m misreading (I’m not altogether sharp in my current situation). And, again, I’m beyond sorry for your loss of photos, whatever the reason.

1 Like

Purchase your password manager’s companion mobile app, install it on your phone, and set up synching.

My phone is actually what got things started: I wanted to update my iPhone to a newer iOS version. This failed: after the update the iPhone would hang during bootup. Apple’s support article told me to update my Mac. So in this case, the companion app wouldn’t have helped. I’m still going to convert my 1Password from standalone to the subscription/cloud version though.

I didn’t grok where it was that you were missing your iCloud password and couldn’t reinstall for lack thereof.

Well, my initial attempt failed. Then I created a new Apple ID so I could download the installer again, but when I ran the installer from the HD it didn’t see the SSD.

Did I misread that after the APFS conversion appeared to bork that the SSD was not even seen in Recovery?

I don’t remember if I tried that. I was getting frantic, and trying lots of things quickly (also a mistake, by the way).
When you select ‘Reinstall Mac OS’ from Recovery, I think it prompts you for your Apple ID because it has to download the installer.
In Recovery, I could restore a Time Machine backup to the SSD, but that process starts with reformatting the SSD.

your photos weren’t being backed up anywhere, at all?

Yes. I have some older backups (the most recent one is an iPhoto database from 2015). Around 2015 I started using Time Machine, and I noticed I was backing up 200 Mb/hour minimum, which I thought would fill up my backup disk quickly, which is why I excluded some folders. In retrospect, TM deletes most of the redundant files later on (it doesn’t keep every hourly backup).

That’s one of four passwords that is always in my head,

When I started using 1Password, I also switched to using its password generator for almost everything including my Apple ID and the Time Machine backup encryption, so all my passwords are autogenerated gibberish that’s impossible to remember. Now, I’ve written those down and they’re going into my physical vault.

Harro de Jong

1 Like

I was getting frantic, and trying lots of things quickly (also a mistake, by the way).

This is a good thing to remember in any “lost data” scenario: don’t panic.

Remain calm. Many times what you’re seeing is a fluke or temporary mistake. Sometimes the data is still around. Rushing to restore or fixes can actually make things worse (as you discovered).

At any rate, you’re not thinking clearly during a panic, so it’s the wrong thing to do. Walk away, take a breather, and try and think through all the scenarios.

I’ve done this a few times and often being calm will remind me of things I wouldn’t have remembered in a panic: that I had some data on Dropbox or last year’s version of the file on an old hard drive, or something else that helped.

Sorry this happened, but it’s a good reminder!

2 Likes

So sorry to hear this story, Harro, and thanks for sharing it and your hard-won lessons. A few points of agreement and perhaps expansion (and absolutely none of these are intended as criticism!):

  • One aspect of backups is having another Mac available. Having all your data won’t do you much good if there’s no Mac to run it on. And having another Mac available makes it easier to troubleshoot things, look up passwords, verify backup drives, and so on. The big honking problem here is that you have to be able to afford another Mac. I generally do this via the classic desktop/laptop approach—I wouldn’t want to use my MacBook Air for everything, but I can do a lot on it if my iMac were to fail.

  • It is NEVER worth excluding things from backups for space reasons unless you are 100% certain that they’re backed up in other ways or that you’re 100% OK with losing all that data. One of the reasons I recommend iCloud Photos is that while not a backup per se, it provides an independent copy of all your photos. I have even had to restore my photos by erasing a library and letting them all download again because the library had developed corruption that prevented it from being backed up.

  • The specific corollary to the above general statement is that photos are precious—back them up in multiple independent ways.

  • I’m not sure if it played a role in your experience, but I generally recommend staying more up to date in part because upgrading from a 5-year-old version of macOS to a 2-year-old version will come with many more gotchas than moving from the n-1 version to the n-current version. That’s the trouble with waiting—nothing forces you to upgrade, but the upgrades can be much more painful when you do decide to make the jump.

  • As @xdev said, remaining calm is so, so important. I’ve been where you were many a time, and my reaction to making a mistake it try to rectify it as quickly as possible, and I’ve often moved too quickly and made things worse. Now the only time I move quickly after realizing I’ve made a mistake is when I think it may be possible to prevent the spread of the problem by acting fast. Most of the time, however, it’s worth taking some time to think through what has happened, come up with an approach to address it, and if possible, run it by an expert friend to make sure you’re not missing anything.

Hopefully others can learn from your experience and the suggestions in this thread.

1 Like

My thought was if you could see your SSD using Disk Utility in Recovery Mode, to attempt First Aid, then another attempt to upgrade, before giving up and wiping it. But, frantic frustration, and assumption of a complete Time Machine might’ve led me down the same path. Ugh. Sorry.

F

I have used Stellar Software Photo Recovery software a few times to recover files accidentally deleted from SD cards (relatives who have just returned from a big holiday and “deleted all” instead of one photo!).

Stellar also have Mac data recovery software that (fortunately) I haven’t tried:

https://www.stellarinfo.com/data-recovery-mac.php

Not free ($79) but it may be worth considering for some situations. The trick is to not panic and try to leave the SD card/Mac unchanged until you are able to run the recovery software because your precious deleted/corrupt data could be overwritten.

I realize this thread is almost a year old. I had a similar situation. So I do have a few Tidbits to add.

I was in Catalina Beta and having trouble with the Photos App syncing to iCloud.

I though “Hey, why not reinstall the whole OS from the recovery partition so I can rule out any OS problems.”. Not a smart move.

I was running Catalina Beta Mac OS 10.15.5. Restoring the entire OS took up a long time, but it eventually finished.

I rebooted the Mac and instead of it restoring the latest beta it installed 10.15.3. So Time Machine and Photos were updated on the move to 10.15.5 and they would not work under 10.15.3.

I do I get 10.15.3 to go to 10.15.5? Apple thinks I am already at 10.15.5 so will not feed me the update. And there is no way to download a beta 10.15.5.

Apple provides no technical support to people in the public beta program. I tried calling Applecare but once they discover I am running beta software they will not talk to me further and politely hangup.

I finally thought to ask for a Level 3 technician. She must have taken pity on me, because after begging she let me in on a little secret I didn’t know.

I thought Time Machine only restored my User data and not any system files. I was wrong.

Apparently there is an option to totally erase the HD and restore both my data and the OS that had been backed up on Time Machine.

So I ran the full restore including OS and voila, I was restored to 10.15.5

Total time spent: 40 clock time hours.

Since this whole thing came about due to iCloud not syncing and only having low res thumbnails on my Mac, i decided to buy a subscription to SmugMug a cloud platform for photographers.

I had a huge amount of storage in my iPhone 10s Max, so chose to store the camera originals on the iPhone. Syncing between iPhone and iCloud has always worked much better than syncing from iCloud to the Mac. Two totally different teams at Apple.

SmugMug can be set to upload every new photo automatically. So I now had all my originals on SmugMug, safe in the cloud. SmugMug creates website photo portafolios and even has the ability to sell my photos online. But I am not that good yet. No storage limit.

With my photo library in SmugMug I started editing my photos in Luminar 4. I no longer cared about iCloud to Mac syncing. It became a nice to have instead of a need to have.

To Apple’s credit they did finally fix the syncing issues. It took over a year, but it works well now.

So I have my Photo Library with camera originals on iCloud and SmugMug. I have to pay Apple for the higher storage limit on iCloud.

Since I am not storing camera originals on my MacBook, Time Machine is only backing up low res thumbnails. But I feel very confident that my precious photos are super safe in two places in the Cloud

I no longer use the successor to iTunes. I don’t use Apple maps, music or news. I prefer Microsoft Office to Apple’s apps.

I do like the find my device features of iCloud and still use the App Store. I would use Apple wallet if they would approve me. I was declined in 2019. Maybe I should apply again?

Find My Device has some limitations that can prevent finding your device if someone gets access to your AppleID PW via iPhone PW reset.

Weird circumstance when I trusted someone I should have never trusted. Lesson learned: Never share your device unlock code with anyone. Period. Leave it in your will for your survivors to find after your death. Same thing with my 1Password Emergency Kit.

One lesson I learned is to set up every Apple device with its own unlock code. Numeric on iPhone and alphanumeric on the Mac. Plus FaceID on both.

1 Like

When someone drops a hint like that, I would appreciate it if that someone included instructions or a link to instructions.

This confuses me. As far as I know, Apple has not been involved in approving or disapproving any addition I have made to Wallet. (For a credit card, the bank must approve adding it.) What have I been missing?

I have alphanumeric passcodes on my iOS devices. Why restrict yourself to numeric on an iPhone?

This has got me thinking. On Macintosh computers, the admin accounts whose passwords I know have non-alphanumeric characters. Is this allowed on iOS devices? If so, I might need to change some passwords.

Recovery mode > Disk Utility > Format
Recovery mode > Restore from TM

Unless I’m missing something. :thinking:

1 Like

I’d assume you can use anything the iOS keyboard lets you get at.

1 Like

Yeah, thanks. Exactly what I had in mind. It’s been a while since I’ve done it myself so reassurance is always good. :slight_smile:

I never though to try letters or symbols on iOS.

Used to be a long time ago that iOS was restricted to numbers.

Anybody know for sure if the unlock code on iOS can be anything the user can type?

I’m using it myself that way on an iPhone SE1 with the latest iOS 13 right now. Have several ‘odd’ characters in there.

It’s been ages since iOS passcodes allowed for more than just numerical.

Yes. When configuring a passcode, you have a choice of:

  • 4-digit numeric code
  • 6-digit numeric code
  • Custom numeric code (Any number of digits. Tap an “OK” button when you’ve entered them all)
  • Custom alphanumeric code (Any text. Tap an “OK” button when you’ve finished typing)
1 Like

I get confused with which OS is installed via Recovery. My old MBP has been updated to High Sierra. I have a corruption in my files it cannot fix so I want to format and restore from Time Machine (or even my Carbon Copy Clone clone) but I’ve heard the OS installed will be the original to the MBP from when purchased which would be a pain in the neck for me. Any guidance? Thanks for the links.