Hard links also only work within the same volume.
2 Likes
IIRC, Time Machine did/does various clever things which weren’t/aren’t available to mere mortals. Directory hard links may well have been/be one of them.
It used to be possible to put soft links in ~/Dropbox, pointing at other folders, and Dropbox would follow the links and backup nicely. A few years ago, that stoped working: Dropbox would just back up the symlink files instead. I worked round that problem by putting the folders I wanted to sync in ~/Dropbox and putting symlinks to them where the various applications expected them to be. That continues to work, but of course doesn’t solve the issue here.
Some of the suggestions in the link you posted, @ace, make no sense to me but I don’t pretend to be a high-end Unix hacker.
1 Like
Normal Unix file systems can not hard-link directories. Not because there’s a technical reason why it should be impossible, but because it’s possible to create infinite-loop directory trees, which could break all kinds of apps.
Apple added hard-link-to-directory capability into HFS+, specifically in order to support Time Machine. I don’t know if other software is able to use this API. But even if you could, APFS does not have this capability anymore (because it’s not needed - APFS Time Machine uses snapshots).
But as @jajvj1 wrote, it’s moot because hard-links can only exist on a single volume. Internally, a hard link is a second directory entry pointing to another file’s inode. Since inode numbers are all local to a single volume, a directory entry pointing to another volume’s inode is simply not possible.
Apple did create a concept of firmlink starting with APFS in macOS 10.15 (Catalina). This is an Apple-proprietary mechanism primarily used to bond a single macOS installation’s System and Data volumes so they are presented as a single user-visible directory tree.
I suppose it could be possible, in theory, for two volumes sharing a single APFS volume group (which I assume would force them to share a single container) to have user-created firmlinks between them, but I think that would be pretty risky, since they’re not well documented. I’d be worried about them breaking (or worse) as a part of a system update, even if I could get them working.
And unless you have a special need for them (e.g. what Apple does by merging a read-only volume with a writable volume), it’s probably pointless. If the two volumes need to share a single APFS container, then they share the same pool of free space, so you might as well just use a single volume.
Links in general are always worth a try, just for fun 
But I would never feel comfortable trusting something like this, even if it were possible. Both Dropbox and macOS have been very inconsistent and unpredictable about behavior when users mess with things under the hood. I’m already normally paranoid that one or the other of them are going to delete all my files because of some bug or mixup. So I would never be able to sleep at night with all my personal data (and backups, if you consider Dropbox and Time Machine) depended on all the players being okay with an undocumented change like this. 
Why do we have to move files under ~/Library to download files on demand? Dropbox has allowed Smart Sync as a feature for a long time when it was stored under ~/Dropbox. I’m not getting it.
1 Like
That’s where the API requires cloud sync data apps to store data. It’s a question for Apple.
By the way, did Dropbox actually support download on demand for files, or simply allow you to specify which folders were actually downloaded? I seem to recall it was the latter. I think online only prior to this was at the folder level, not the individual file level.
They offer both. They’re called Selective Sync and Smart Sync. And they’ve had them for years, long before Apple forced this change:
And it worked fine, thank you. And yes, it was at the file level, not just the folder. I use it all the time. And yes, what I’m saying appears to contradict what Dropbox states at the link I posted at the top, where they appear to suggest those features are newly released alongside this change of directory. I find their post confusing for that reason. What I’m wondering is if APPLE is now allowing a “smart sync” equivalent as a freebie to any Mac consumers of a software product that utilizes the new APIs. That’s a bit irksome to me, since this was an upgrade that I paid Dropbox for. Now everyone’s gonna get it for free?
Anyway, if it was insecure the old way, I’d like to understand how forcing the files to be under ~/Library makes them more secure. And sure, I’m asking Apple (hello, Apple!)
1 Like
Has this taken place yet? I have my Dropbox synced to an external drive and I don’t see that it has changed yet. Is there a date this is going to happen?
I have seen Apple blamed as the force behind moving Dropbox local storage to ~/Library…
I have searched Apple Developer Forums and can find no mention of such things.
I would love to read the apple developer documentation describing this purported requirement that Dropbox references. So far, all I have read claims which do not reference any Apple documentation that I can find.
Help me, please, to find a clear basis for this change.
I have several systems with relatively small boot drives – certain smaller that required for local storage of my Dropbox data. I have a valid business case for local storage and have been using Dropbox folders on external drives for about ten years. I have been delighted with what has been available from Dropbox but the FUD circulating will require me to spend time deciding and implementing a substitute.
2 Likes
Arq author Stefan described it as:
Apple’s new File Provider API
Maybe search for that? And report back, please! But this looks relevant:
I’m running Catalina (max os) on a late 2012 iMac w/32GB memory, a 1TB SSD internal for the OS and approx 60TB of external HD storage on 20 usb drives in 5 enclosures and an additional 4 1tb drives individually attached.
My Dropbox is a 2TB dedicated external drive. I intend to continue to keep my data local, backed up locally and with Backblaze. Dropbox is a convenience but I will not get in a situation where most of my Dropbox folder is exclusively in the cloud and not on my local drive.
So at this point I am not planning to upgrade my Dropbox software if it will force me into this stupid no external drive business.
There is another vendor (can’t remember who) of Dropbox like service that claims to be able to get around Apples requirement. If they can so can Dropbox if they want to.
Following up here.
Thanks for this tip. I guess there are two approaches here to prevent the infinite backup loop:
- Tell Arq not to backup the backup folder (this is what I said I tried to do but couldn’t because the folder could not be found)
- Tell Dropbox not to sync the Arq folder (this is what you and Arq are suggesting).
It turns out the latter was already done:
I don’t recall if I did it explicitly to prevent this problem when I first set things up, or if that was the default all along. But either way, it explains why I don’t appear to have the kind of instability I’d expect with a backup loop :-)
Thanks again for the tip!
1 Like
Update:
It looks like, as we suspected:
-
this change affects all cloud sync providers, not just Dropbox (making the idea of jumping ship less appealing)
-
Apple may, indeed, be providing an OS-based “smart sync” (aka “online only”) technology solution that saves developers from “rolling their own” solution to this problem.
The latter does make you wonder if this is undermining the revenue potential for the cloud sync providers. Features like smart sync generated upgrade revenue, and deep integration with the Finder distinguished Dropbox from competitors. Going forward, will they all be functionally equivalent software, and they’ll just compete on their storage price points?
On the other hand, the up side for consumers is a unified experience, where you can drag and drop files between providers, such as via the Files app on iOS. It’s also possible that this will solve problems Dropbox has long struggled with, such as the way they (silently and rudely) broke support for syncing Aliases a couple years back, and the way their online-only files misreported local drive consumption, so now such files all say Zero K, which means we can no longer tell how large they are. Pick your poison. And maybe Apple will prevent the lost data situation I have had a couple times with Dropbox, apparently after editing a file whose latest copy had not yet synced from another device.
But I swore I would never use iCloud Drive for any mass storage after it held my entire home directory hostage for days once, making the contents even unreadable while it struggled to resolve syncing confusion; Dropbox has never been unreliable that way, and that is why I swear by Dropbox. Is the syncing algorithm for Dropbox now going to be identical to that of iCloud Drive? Yikes.
I’m not sure yet what to think about this, but I’m tempted to quote Han Solo…
I’m not sure, and it doesn’t appear to have happened to me, either. If you look at the linking my OP, it says:
When you’re eligible, you’ll receive a notification from the Dropbox icon in your menu bar stating Dropbox for macOS is now ready.
What does “eligible” mean? Does it mean they are rolling this out in stages to different customers? Does it mean Dropbox has completed moving my files into the new ~/Library/CloudStorage/ location, and so now syncing will resume?
I updated both my MacBook and my iMac Pro to Ventura, then updated Dropbox. On the iMac, it moved its folder to CloudStorage. On the MacBook, it didn’t. I have no idea why. Just capricious, I suppose.
I don’t think Dropbox syncing itself or any of its features were insecure. The insecurity was at a system level – allowing kernel and other system extensions greatly expands the potential for buggy or nefarious software to be a vector for spyware/malware/instability. Since Dropbox must have used some now-deprecated kernel/system extension mechanism, it has been forced to migrate to the File Provider API. That’s a replacement for this specific use case of extensions. There are other APIs for other use cases.
I don’t think Apple tries to define a ‘sync service’. I think they’ve just removed the ability for developers to install kernel extensions. The File Provider API is provided as an alternative for developers who were using a kernel extension for syncing (and related actions like badging icons). But if they can sync without installing a kernel extension (as Maestral does), then there’s no requirement to use the File Provider API.
I’m not sure Finder integration is a differentiator anymore these days, it’s been standard across all the main services for years now.
2 Likes
But, as previously posted - we’ve talked about this issue before, OneDrive has a mechanism that allows you to decide exactly where you want files stored on your system using this mechanism. (See also this post and this post in the same thread.)
Obviously this is a solution that Dropbox does not want to support.
And, Maestral allows you to chose where you want your Dropbox files stored. They do this without a kext and without using Apple’s new API, as it lacks the deep integration that DropBox, OneDrive, etc., have in the Finder.
I use the service sync.com as my primary cloud service provider, and it also (at least so far) does not use the API, and allows you to store your files wherever you like.
1 Like
That’s not quite true. OneDrive has a mechanism that allows you to store the files on an external drive (and you can choose where on the external drive if doing so). But if you don’t store on an external drive, you have no ability to choose where on the internal drive the files are stored, they have to be in ~/Library/CloudStorage.
And even when storying on an external drive, you access all your files through ~/Library/CloudStorage, it’s just that it uses the external drive as the ‘cloud source’ to dynamically ‘download’ the files for you to use. I realise this might seem like splitting digital hairs, but the practical result is that you need some amount of space available on your internal drive – at least as much as the smallest file you want to access or the combined size of multiple files you want to access at the same time.
This is not to criticise OneDrive (at least not for this aspect – there are plenty of other reasons to do so, don’t get me started on file naming and path lengths 
). It seems the best that can be done with Apple’s limitations. But it’s worth people knowing that it will be slightly different from the experience to now (or using Maestral) when storing files on an external drive.
2 Likes
Consider having multiple Photo Libraries. For example I keep a separate library for screen snapshots.
Check in the archives over at Steve Gibson’s grc.com site for the Security Now show notes…as I recall a year or so back he did a comparison of the various cloud services but he didn’t want to use DB since he wanted something that used his ideas of Pre Internet Encryption (PIE) and Trust No One TNO so that whoever the cloud provider was only got encrypted data…and he wanted that data available to him both at his office computer and his laptop at home.
IIRC he decided on Sync.com as meeting all of his requirements including storage at whatever location one desired.
I looked at sync.com at the time and it appears to be a viable DropBox alternative.
An additional alternative might be the app Mistral…it’s a third party DB client (at least I think that’s the name)…but whether it allows storing the folder at some other location besides ~/…/CloudStorage I don’t recall.
The DropBox folder issue is one of many reasons I got 2TB drives in both my Studio and M1 MBP.