Still no documentation on APFS

It’s now 6 mths since APFS has been introduced and as Apple has not released any documentation, companies that produce DiskWarrior, Drive Genius, TechTool Pro and Data Rescue have been unable to update their apps to work with APFS. Not only are these companies suffering but thousands of Apple Support Consultants worldwide are hugely inconvenienced. Any news yet as to when this might be resolved?

I have no updates, but here’s where I’d go to look for information about changes in APFS, since the SoftRAID folks at OWC are paying attention.

https://blog.macsales.com/?s=APFS

That’s a good resource. I’m trying to collect as much APFS information as I can on my blog:

https://mjtsai.com/blog/tag/apple-file-system-apfs/

However, this is all higher level stuff. The developers need to know the structure of the actual volume format, and I don’t think there’s been an announcement about when that will be available. There’s been some really impressive reverse-engineering, but it’s one thing to use that to try to recover files or read them from Windows and quite another to operate and make changes to a live disk.

My understanding from private discussions at MacTech Conference is that the Apple team working on APFS simply isn’t done, which makes it a moving target and explains why there’s no documentation. I think it’s fine to use APFS in completely supported ways, but I wouldn’t push on it in unexpected situations.

Besides looking at diskutil list in Terminal.app on macOS 10.13.3+, I read Howard Oakley’s blog The Eclectic Light Company. Howard looks into the arcana of macOS and art. He also creates some quite useful tools.

Adam, did they explain why it was introduced even though it was incomplete? So that makes every Mac user a beta tester. Why was APFS FORCED on all users of SSD’s. There should have been a choice as I believe there was in beta versions of High Sierra. As a consultant what did I do if one of my clients runs into an inexplicable problem with APFS. Just blame Apple and re-install. Doesn’t make me look very professional!

I think it was one of those situations where everyone agreed that HFS+ needed to be replaced, and APFS was far enough along for iOS, where Apple controls everything. And then, for whatever reason, higher ups decreed that it would ship with High Sierra. The engineers got it working well enough with SSDs, but not with hard drives or Fusion Drives, which is why support for those was pulled after the betas.

The problem, as I understand it, is that the Mac is just a much more diverse ecosystem, and it proved a lot harder to deal with all the edge cases in hardware, software, and scenarios.

1 Like

It seems to me that they didn’t know what shape it was really in for much of the development and kept underestimating the work involved. For example, right from the first announcement they had no answer for Unicode normalization. It shipped for iOS anyway. Then they had to do two redesigns before (seemingly) getting it right with iOS 11 and macOS 10.13.

Seems like it was a pretty close call even for SSDs. Someone (Howard Oakley?) reported that the on-disk format was changing all the way up to the GM build.

Maybe a bit off topic, but what is this? I assumed that the whole point of Unicode is that it is ‘normalised’. But are there file system issues that means you can end up with different ways of representing characters?

Well it is possible to install on an SSD with HFS+. This was my method. I installed the OS on an external SSD formatted as HFS+ (this won’t convert to APFS). Then at the restart where you start inputting country etc, I shut down. Then I cloned using Carbon Copy Cloner to my internal SSD formatted as HFS+ then continued the installation, imported accounts etc from a backup. Works absolutely fine.

The ability to boot of external (bootable) drives and use migration assistant to ‘port’ users ‘out’ of a previous install to a new one (with, say, an SOE image) is kinda broken. Especially if source AND target are High Sierra/APFS - Not so bad if coming from Sierra HFS+ to HS APFS, but only one way. Even more problematic if source is, say, a 2016/2017 MBP with encryption enabled.

I use a bunch of Samsung Evo and Pro SSD’s to move/recover users around (instant fall back/bootable if things go wrong), and APFS does NOT like multiple volumes, or even partitions within the same volume.

Had to re-design my upgrade workflow on-the-fly, quite a few late nights if not all nighters. Might see my family again by May :slight_smile:

It’s a huge can of worms that, hopefully, no one but driver-level programmers ever has to deal with.

The short version is that APFS is doing this the ‘right’ way according to UTF 9.0 (and later) specs, but this is causing some issues for people, largely programmers, in the transition phase.

As for what normalization is, imagine you have a file named Alpha where the ‘A’ is not the regular ASCII A, but is the unicode character 𝝖 which while it looks like an ‘A’ is really “MATHEMATICAL SANS-SERIF BOLD CAPITAL ALPHA; Unicode: U+1D756, UTF-8: F0 9D 9D 96”. How should the filesystem treat that filename, and would it collide with an ASCII file named “Alpha”?

1 Like

Ah, thanks. I can see how that sort of thing would be a can of worms. Glad it’s not me that has to deal with it. :sweat:

Please see:

https://mjtsai.com/blog/2017/03/24/apfss-bag-of-bytes-filenames/
https://mjtsai.com/blog/2017/06/27/apfs-native-normalization/

2 Likes

Ohhh, thanks! Clearly I should subscribe to your blog. :smile: