I might agree with that thought. Any change is an opportunity to introduce a bug, while creating a new sparse image and copying everything to it every few years has worked for me and has been fairly painless.
Has anyone had issues with disk images becoming corrupt?
I used to do this sort of thing (storing different projects on disk images to keep them more organized), but it was a long time ago. One reason I stopped was that I ran into times where the OS would tell me a disk image was corrupt and wouldn’t open. This might have been due to OS upgrades and trying to open an image I hadn’t opened in a while, so it could have been an OS compatibility problem. But unlike regular files, if an encrypted disk image won’t open, the data is toast. (At least with regular files you might only lose one file or be able to extract partial data from a file.)
The second reason I stopped using disk images is I started using syncing services like Dropbox so that my projects were on multiple devices and every tiny change to a file in a disk image required the entire disk image to by synced, which wasted a lot of bandwidth and was slow.
I’m just curious if either of these are still problems today.
There is another variation called a sparse bundle. This article explains its pros and cons:
https://eclecticlight.co/2020/04/27/sparse-bundles-what-they-are-and-how-to-work-around-their-bugs/
Thank you all for your responses.
It seems to me that using encrypted volumes is like trying to fit a square plug into a round hole. I have about 200 encrypted files/folders. Their content is important to me. I cannot chance losing them to corrupt volumes. I would like Apple to step up and release a supported file and folder application. Until then, I am going to do two things. Firstly, I will store the data in the Tresorit secure cloud. Secondly, I will migrate each of my encrypted files and folders to .7Z archived, compressed, and encrypted files and folders. I will use the Keka app.
Just a technical question, does compression add another layer where a process can get corrupted or are certain types of compression more stable? Does some compression happen automatically with certain encryption?
This is the perfect argument for regularly scheduled multiple backups. Multiple backups can provide spatial diversity (multiple drives and off-site backups).
Every once in a while, I have had to restore a clients dmg file. Usually to correct a user error; rarely to replace a corrupted volume. Sometime restoration is from a Time Machine backup, sometimes from CCC backup. Both are satisfactory.
I encourage use of a Safe Deposit Box for off-site storage of backups. Access to and subscriptions for Safe Deposit Boxes are generally easier and more reliable than cloud backups.
1 Like
This can be done with .sparseimage and .sparsebundle files using the hdiutil command line tool and the compact verb.
compact image [options]
scans the bands of a sparse (SPARSE or SPARSEBUNDLE) disk
image containing an APFS or HFS+ filesystem, removing those
parts of the image which are no longer being used by the
filesystem.
Also explained on the old Mac OS X Hints site:
http://hints.macworld.com/article.php?story=20040625012304236
For a GUI to do this, see Howard Oakley’s utility Spundle:
That was one of the reasons the .sparsebundle format was created: It is actually a folder with lots of individual files called ‘bands’, all of which are only a few MB in size. A change to the disk image should only affect a handful of bands, so syncing should be quick. This is compared with the .sparseimage format where an image with 10GB of data on it will be a single ~10GB file.
Overall, this still seems like a lot of hassle to me compared with simply encrypting the whole disk (but I know you said you didn’t want to use File Vault).
1 Like
Thanks. I didn’t know about that. For my use-cases, the disk images are relatively short lived (typically getting deleted once I come home from the trip where I was using the disk image), but for images that are used for an extended period of time, this will be quite useful.
1 Like
There’s Gnu Privacy Guard (GnuPG), which is based on OpenPGP. There’s a front-end for the Mac called GPG Suite, which is available at gpgtools.org. I’ve never used these tools, but it sounds like they would do what you want.
I have used an encrypted disk image on Macs for many years for precisely this purpose (confidential/private document filing). I have had one instance of corruption, the file would not open. However, I have multiple backups, including Time Machine and cloud storage, so nothing was lost.
For extra security I password protect the most valuable files but this opens up a separate topic as I have been unable to find out whether password protection in Office for Mac encrypts the file or just requires a password to open. Anyone know?
I’m pretty sure the file is encrypted.
Microsoft Office XML documents (.docx, .xlsx and .pptx) are zip files. You can actually view the contents using standard zip tools. For example:
$ zipinfo foo.docx
Archive: foo.docx
Zip file size: 17020 bytes, number of entries: 11
-rw---- 4.5 fat 1312 b- defS 80-Jan-01 00:00 [Content_Types].xml
-rw---- 4.5 fat 590 b- defS 80-Jan-01 00:00 _rels/.rels
-rw---- 4.5 fat 817 b- defS 80-Jan-01 00:00 word/_rels/document.xml.rels
-rw---- 4.5 fat 99738 b- defS 80-Jan-01 00:00 word/document.xml
-rw---- 4.5 fat 8393 b- defS 80-Jan-01 00:00 word/theme/theme1.xml
-rw---- 4.5 fat 2865 b- defS 80-Jan-01 00:00 word/settings.xml
-rw---- 4.5 fat 753 b- defS 80-Jan-01 00:00 docProps/core.xml
-rw---- 4.5 fat 1683 b- defS 80-Jan-01 00:00 word/fontTable.xml
-rw---- 4.5 fat 655 b- defS 80-Jan-01 00:00 word/webSettings.xml
-rw---- 4.5 fat 31420 b- defS 80-Jan-01 00:00 word/styles.xml
-rw---- 4.5 fat 993 b- defS 80-Jan-01 00:00 docProps/app.xml
11 files, 149219 bytes uncompressed, 13932 bytes compressed: 90.7%
If you try the same with a password protected document, the zip tools fail:
$ zipinfo bar.docx
Archive: bar.docx
[bar.docx]
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
zipinfo: cannot find zipfile directory in one of bar.docx or
bar.docx.zip, and cannot find bar.docx.ZIP, period.
I don’t know what Microsoft is doing, but I think it is reasonable to assume that it is still a zip file, but one that they encrypted after creation.
It is not encrypted using the Zip’s own encryption, because the zipinfo tool would show the table of contents for such an archive, only requiring a password to extract the contained files.
1 Like
Thanks, reassuring. I chatted with Microsoft Support on this a couple of years ago but the person at the other end didn’t understand the question.
Other people have suggested sparse images/bundles and GNUPG.
I use both. I have encrypted sparse bundle disk images stored in Dropbox. They are accessible to all my Macs. I’ve been doing it for years and I have never had a single problem.
I am careful to not open the same disk image on two Macs at the same time. I don’t know that that would cause corruption. I just don’t want to take any chances. Also, I keep reminding myself that Dropbox is just a folder so TimeMachine is backing it up as well. Worst case would be rolling back a version.
Some time back I got involved with a GitHub project and needed to start signing commits with GNUPG. While a YubiKey wasn’t essential, it seemed like as good a way as any of going about it. I followed Dr Duh’s excellent YubiKey Guide to set it up.
Although the GitHub focus was on digital signatures, it’s still GNUPG so encryption and decryption come along for the ride. You don’t get folder-level encryption with GPG, you have to zip (or tar) the folder first and then encrypt the result.
Encryption doesn’t need anything more than the command plus the availability of your public key. No password,no YubiKey, no PIN, no nothing. That means it’s really easy to automate things. I haven’t done it myself but there’s no reason why you couldn’t set up a LaunchAgent to watch a particular folder, and automatically encrypt anything added to the folder that wasn’t already encrypted.
Decryption is a different kettle of fish. To decrypt something, the physical YubiKey has to be in the USB slot and you have to know the YubiKey’s PIN to unlock it and you have to physically touch the YubiKey. If anyone steals your YubiKey and doesn’t know the PIN, they get 3 guesses and then the YubiKey is bricked until you unblock or re-provision.
The only caveat I’ll place on all of this is that I’m still a High Sierra, Mojave and Catalina person so I can’t say whether any of this works on Big Sur. I hate being constantly nagged-forward on Cupertino’s annual “what can we do this year to punish our users for their loyalty” so, mostly, I Just Say No.
I decided to create a new encrypted sparsebundle disk image and found Disk Utility did not present the interface that I remembered. Among other things, now APFS is the default; in the past I have used Mac OS Extended (Journaled). Does it matter?
As long as I’m here, please confirm that I should select GUID Partition Map. Thanks.
If you plan on opening the image on computers running macOS versions prior to 10.13 (High Sierra) or on non-Mac platforms (e.g. Windows or Linux), then you will definitely not want to use APFS. For these platforms, HFS+ (“Mac OS Extended”) will more easily transport/preserve a file’s metadata), while FAT or exFAT will be usable by non-Mac operating systems without add-on software.
An APFS image will support APFS-specific features like snapshots. But, just like with an external drive, this comes at a cost. Older versions of macOS can’t access it and it will have bad performance on hard drives.
In my case, I use HFS+ images because I travel with a pretty old Mac (a 2011 Air running macOS 10.12, Sierra), so I clearly need to use a format that it can use.
1 Like
Thanks, @Shamino. At present, I do not see opening the image on non-Mac computers, so I’ll ignore FAT and ex-FAT. Also, I expect to open the file only on Mac computers running Mojave or Big Sur, so it sounds like APFS would be reasonable. Still, there is an outside chance I would use something older than High Sierra, so I’ll stick with Mac OS Extended for this iteration. Thanks again.
Any problem having HFS+ images in an APFS drive?
I can’t imagine why it would be. As far as the containing file system is concerned, a disk image is just a file (possibly a sparse file). And when the image is mounted, the OS should treat it like it would a separate storage device.
Not for me. I have encrypted sparse file images for my financial records for each year that are HFS+ (except for 2020 and 2021, which are APFS) going back to 2007 and they open fine on my High Sierra, Mojave, and Big Sur Macs.
The only thing that bugs me about these images is that Apple doesn’t natively support them in iPadOS. I’m not quite sure why Apple hasn’t supported opening/mounting images in iOS/iPadOS yet. (There is a great third-party app called Disk Decipher that can open these disk images. They may be read-only but that’s fine for now.)
That’s fantastic, it never even occurred to me that disk images would be accessible on i(Pad)OS!
1 Like