This evening my iMac kernel panicked, and I couldn’t get it to boot again. After trying a few things I started it in verbose mode and saw I/O errors on disk2, which is likely one of the components of the Fusion Drive.
I booted it to my external SSD clone (last updated Wednesday). Disk Utility First Aid said that there were errors in the Fusion data structure that it couldn’t repair. I don’t have the exact error because it froze up. As did many things, because Disk Utility had mounted the Fusion Drive, and I think anything that tries to read it stalls because of I/O errors. I finally was able to force it to unmount again.
But here’s where it gets weird. I was getting repeated pop-ups that applications were damaged. At first I thought that they were unknown developers; now I think it was because they failed the code signature check because they are “damaged” in the sense that they are failing the code signature check. Most I could fix by reinstalling. (More on this shortly)
Then I found that I can’t launch iMazing even after reinstalling from its .DMG file; the crash says Library/Frameworks/iTunesLibrary.framework/iTunesLibrary is missing. But it does exist in System/Library/Frameworks, one of the locations where the crash says it is looking.
At this point I noticed that Software Update shows Ventura 13.7, even though I had installed that 3 weeks ago. And I realize: because updating macOS on a clone is pain, I don’t. The clone is running 13.6.6. But the Fusion Drive had been updated past that. So I’m thinking this is causing the code signature checks to fail, leading to all my issues.
My options are to run the Ventura 13.7 updater, or download the full Ventura 13.7 (if available) and run it. What I’m worried about is that the last time I was running off of the external SSD, I couldn’t get it to apply macOS updates. I don’t want to make things worse than they already are.
… I think I’ll clone from this SSD to another drive, just in case. And then try applying Ventura 13.7.
I was able to apply the Ventura 13.7 update to the external SSD, which is a relief. I wasn’t able to apply macOS updates – either point release or full installer – to this drive in Big Sur or Monterey. I guess the problem was fixed in Ventura.
After bumping up to 13.7 I didn’t get any new Gatekeeper / Notarization “this application is damaged” errors, but still got them on any application that had errored while on 13.6.6. That included applications that ran at startup like Lingon X, PowerPanel Personal, iMazing, SuperDuper, but also applications that happened to try to run their update check like Chrome, BBEdit, and Microsoft Auto Update. I don’t know how to force macOS to re-do the signature check so I reinstalled them.
Another weird thing is Homebrew (yes, I’ve gone to the dark side) was broken, because its symlinks were missing from /usr/local/bin. I don’t know yet if macOS removed them or if SuperDuper isn’t backing them up.
And I couldn’t figure out why Spotlight wasn’t indexing at all, until I remember that I had explicitly turned it off on the backup drive, which now of course needs to be on as its the boot drive.
I don’t think I’ve lost anything except for some minor updates to TextEdit notes I had made shortly before the crash. Arq didn’t help because it decided to purge all the hourly backups, leaving me with no backup for 10/11 before the crash at all!
I guess the next step is need to see if can diagnose the Fusion Drive without it hanging the computer. The SSD was component was replaced last year; I’m betting the HDD component crashed. It may not be worth replacing, since the external Thunderbolt SSD is faster than the internal Fusion Drive, and I really need to replace the entire computer, once I deal with the VMs and games that require Intel.
One more note: There have been, and continue to be forum posts that say that in the age of Apple Silicon Macs cloned drives are not supported or of little value. I BEG TO DIFFER.
Take this very case. Sure, I had a Time Machine backup (of most files), and a full clone with all of the data. But what could I do with that, without a clone? The internal drive in the computer is apparently toast. So if the recommended remedy is “install macOS on a drive and then migrate the data”, either I’d have a week of downtime, or I’d have to sacrifice the external SSD (to install macOS on it), and then migrate the Time Machine. But I said the Time Machine has most of the files; it runs out of space if I include the virtual machines.
But my way means that as soon as I identified that I couldn’t get the internal drive to boot, I was up and running (albeit with some unexpected issues) in minutes, on a drive that has all of my data as of a few days ago. And the rest of the data was easily restored from Time Machine’s backup.
I will admit that this clone is Intel. But I do have a clone of my M2 MacBook Pro, and I can boot from it. So I say, if you want to be backup and running in minutes, you should clone.
I used Macs from 1984 until earlier this year, and I endeavored to always have bootable clones, dating back to when that meant doing four disk swaps to clone a 400 KB floppy on an original 128K Mac with a single drive. Much of the time I was using those Macs, I was in fairly time-critical situations (but, then, aren’t we all?).
Fortunately, I could probably count on both hands the number of times that I had to resort to a bootable clone to get a crashed system up and running, but each of those times would have been expensive (beyond the cost of a complete new system) had my only option been to “reinstall the system and then restore from backup.”
In one sense this is literally true. Apple Tech Support will not help you with a problem on a booted external drive. Earlier this year, they made me move Sequoia onto my internal drive before they would help me. Three solid afternoons of running tests for them later, tech support found it was just as broken on the internal drive as the external.
Previously I said that upgrading the now-startup drive to Ventura 13.7 resulted in no new “App is damaged and can’t be opened” errors. That’s not true. I tested every application and found 14 more, for a total of about 2 dozen.
I don’t know what happened here. Either the applications truly were damaged, or something was messed up in whatever database macOS uses to track the Gatekeeper (or whatever it is) status. I got them all working again by reinstalling, since as I’ve said before, I keep copies of every download including copies of Mac App Store apps. Anyway, I’ve never seen anything like this before.
There was another problem. Sunday morning the iMac was shut down instead of asleep, and on restart it said that it shut down due to a problem. It also displayed the “The disk you attached was not readable by this computer” error. It had spontaneously de-Fused the Fusion Drive. The HDD component is now showing as unitialized; macOS can’t read the partition.
Same thing happened again Sunday evening. I’m thinking it means that the Mac can’t go to sleep; it times out trying to spin down the Fusion Drive. Maybe it will work if I turn off “Put hard disks to sleep when possible”.
On a positive note, Time Machine completed the first backup of the new startup disk without doing a completely new backup! It kind of automatically inherited the previous backup, despite different volume names. This is of course what you’d expect it to do, but until Ventura, I’ve never seen it happen.