I don’t recall if I did it explicitly to prevent this problem when I first set things up, or if that was the default all along. But either way, it explains why I don’t appear to have the kind of instability I’d expect with a backup loop :-)
this change affects all cloud sync providers, not just Dropbox (making the idea of jumping ship less appealing)
Apple may, indeed, be providing an OS-based “smart sync” (aka “online only”) technology solution that saves developers from “rolling their own” solution to this problem.
The latter does make you wonder if this is undermining the revenue potential for the cloud sync providers. Features like smart sync generated upgrade revenue, and deep integration with the Finder distinguished Dropbox from competitors. Going forward, will they all be functionally equivalent software, and they’ll just compete on their storage price points?
On the other hand, the up side for consumers is a unified experience, where you can drag and drop files between providers, such as via the Files app on iOS. It’s also possible that this will solve problems Dropbox has long struggled with, such as the way they (silently and rudely) broke support for syncing Aliases a couple years back, and the way their online-only files misreported local drive consumption, so now such files all say Zero K, which means we can no longer tell how large they are. Pick your poison. And maybe Apple will prevent the lost data situation I have had a couple times with Dropbox, apparently after editing a file whose latest copy had not yet synced from another device.
But I swore I would never use iCloud Drive for any mass storage after it held my entire home directory hostage for days once, making the contents even unreadable while it struggled to resolve syncing confusion; Dropbox has never been unreliable that way, and that is why I swear by Dropbox. Is the syncing algorithm for Dropbox now going to be identical to that of iCloud Drive? Yikes.
I’m not sure yet what to think about this, but I’m tempted to quote Han Solo…
I’m not sure, and it doesn’t appear to have happened to me, either. If you look at the linking my OP, it says:
When you’re eligible, you’ll receive a notification from the Dropbox icon in your menu bar stating Dropbox for macOS is now ready.
What does “eligible” mean? Does it mean they are rolling this out in stages to different customers? Does it mean Dropbox has completed moving my files into the new ~/Library/CloudStorage/ location, and so now syncing will resume?
I updated both my MacBook and my iMac Pro to Ventura, then updated Dropbox. On the iMac, it moved its folder to CloudStorage. On the MacBook, it didn’t. I have no idea why. Just capricious, I suppose.
I don’t think Dropbox syncing itself or any of its features were insecure. The insecurity was at a system level – allowing kernel and other system extensions greatly expands the potential for buggy or nefarious software to be a vector for spyware/malware/instability. Since Dropbox must have used some now-deprecated kernel/system extension mechanism, it has been forced to migrate to the File Provider API. That’s a replacement for this specific use case of extensions. There are other APIs for other use cases.
I don’t think Apple tries to define a ‘sync service’. I think they’ve just removed the ability for developers to install kernel extensions. The File Provider API is provided as an alternative for developers who were using a kernel extension for syncing (and related actions like badging icons). But if they can sync without installing a kernel extension (as Maestral does), then there’s no requirement to use the File Provider API.
I’m not sure Finder integration is a differentiator anymore these days, it’s been standard across all the main services for years now.
Obviously this is a solution that Dropbox does not want to support.
And, Maestral allows you to chose where you want your Dropbox files stored. They do this without a kext and without using Apple’s new API, as it lacks the deep integration that DropBox, OneDrive, etc., have in the Finder.
I use the service sync.com as my primary cloud service provider, and it also (at least so far) does not use the API, and allows you to store your files wherever you like.
That’s not quite true. OneDrive has a mechanism that allows you to store the files on an external drive (and you can choose where on the external drive if doing so). But if you don’t store on an external drive, you have no ability to choose where on the internal drive the files are stored, they have to be in ~/Library/CloudStorage.
And even when storying on an external drive, you access all your files through ~/Library/CloudStorage, it’s just that it uses the external drive as the ‘cloud source’ to dynamically ‘download’ the files for you to use. I realise this might seem like splitting digital hairs, but the practical result is that you need some amount of space available on your internal drive – at least as much as the smallest file you want to access or the combined size of multiple files you want to access at the same time.
This is not to criticise OneDrive (at least not for this aspect – there are plenty of other reasons to do so, don’t get me started on file naming and path lengths ). It seems the best that can be done with Apple’s limitations. But it’s worth people knowing that it will be slightly different from the experience to now (or using Maestral) when storing files on an external drive.
Check in the archives over at Steve Gibson’s grc.com site for the Security Now show notes…as I recall a year or so back he did a comparison of the various cloud services but he didn’t want to use DB since he wanted something that used his ideas of Pre Internet Encryption (PIE) and Trust No One TNO so that whoever the cloud provider was only got encrypted data…and he wanted that data available to him both at his office computer and his laptop at home.
IIRC he decided on Sync.com as meeting all of his requirements including storage at whatever location one desired.
I looked at sync.com at the time and it appears to be a viable DropBox alternative.
An additional alternative might be the app Mistral…it’s a third party DB client (at least I think that’s the name)…but whether it allows storing the folder at some other location besides ~/…/CloudStorage I don’t recall.
The DropBox folder issue is one of many reasons I got 2TB drives in both my Studio and M1 MBP.
Apple some years ago announced that it was deprecating kernel extensions for security reasons. As of Monterey 12.1, Apple’s recommended way to implement implement remote file access and file syncing services was to use the Apple File Provider framework instead of kernel extensions.
Some companies, like Microsoft, started rolling out new Mac clients with Apple File Provider when 12.1 shipped. Others, like Dropbox, have taken a little more time.
As of now, it seems that most third party remote storage and sync tools have been updated to use the Apple File Provider framework during the Monterey release cycle. This includes the major platforms, such as Microsoft OneDrive, Dropbox, Google Drive, and Box, as well as less widely used tools like Egnyte and others.
The process of switching from third party extensions to the Apple File Provider framework has not been smooth. Microsoft’s release of a new OneDrive for Business client removed some features, significantly impacted enterprise workflows, and created considerable user backlash and confusion. It was a bit of a fiasco for my organization’s help desk. Apparently due to pressure from enterprise customers, Microsoft addressed some of the issues introduced by the the new framework.
While multiple clients now use the Apple File Provider framework and share some characteristics (like having their sync directories inside ~/Library/CloudStorage), each client still has its own distinct user-facing features, user interface, and strengths and weaknesses.
If you are using a freshly installed third party syncing client on a version of macOS earlier than Monterey 12.1, you are not using the new Apple File Provider system. It is important to remember that third party tools will not be supported on older versions of macOS indefinitely. Google Drive no longer updates its client on Mojave, for example, and any client might stop syncing at some point in the future.
If you are using current versions of Monterey or Ventura but are not yet using the Apple File Provider framework, it probably is because your sync client has not updated itself yet.
My guess is that most users won’t notice any differences after switching, though some people who leveraged their clients’ capabilities more thoroughly in the past will have some adjustments to make. Unfortunately, since all of the providers are going through this change, switching from one platform to another is unlikely to avoid the disruption caused by adoption of Apple File Provider technology.
Sorry for being so wordy, but I hope that I clarified more than I confused.
Correction added June 15, 2023: Added “freshly installed” to “If you are using a third party syncing client on a version of macOS earlier than Monterey 12.1, you are not using the new Apple File Provider system.” It appears that at least some third party storage client providers, including Dropbox, have been rolling out client updates in a phased fashion, so if you had installed a third party syncing client prior to Monterey 12.1, it’s possible that you might be using the older, kernel extension version of the client for a little while longer, even if you upgraded to 12.1 or newer. You will be upgraded eventually, however. If you do a clean install of the client on a system that is running 12.1 or newer, the Apple File Provider version of the client will be installed.
What I don’t understand is why the File Provider API makes it necessary to store synced files only in ~/Library. What is special about ~/Library compared to elsewhere in the user home folder? It seems like an unnecessary restriction.
Adam,
This is something I would love to know more about. We are heavily dependent upon Google Drive in our workflows.
We also use 4D as our main database and it is very picky about saving documents to Drive. A year ago, 4D Tech Support instructed us to move our users’ Google Drives to their user directories in order for 4D to “see” the folders. We did that thinking we were home free. Then along comes Google complying with Apple’s File Provider API! Now, we can no longer put the Drive where our database can see it, but it has to live inside the ~/Library/CloudStorage.
Drive operates as normal in this new location. But our database cannot automatically save files to the various shared drives and folders any longer.
A few months back I found a terminal command (touch ~/Library/Application\ Support/Google/DriveFS/fp_left_beta) that would still allow us to keep the Drive in the user root directory, and therefore, visible to our database. But I see now that that command no longer works and I cannot reset the location of Drive anymore.
We cannot control when this happens or why - it just does random Macs for random users at a random time!
I cannot express what a disaster this is for my users! I am crafting a ticket for 4D as we speak to describe the issue again and look for resolutions. So any light you can shed on this would be most appreciated!
Just a guess on my part, but this location is invisible to less-technical users, so the likelihood of an accidental deletion or move of a location to a new space (something a less technical user can easily do from the Finder if looking at their home folder) is my guess. This is what Apple did with iCloud Drive. I’m not sure that Apple has ever explicitly said why the storage location is in ~/Library/CloudStorage, but Apple doesn’t make it easy to accidentally find your way in the user’s library folder.
Also note that the Finder sidebar does have an explicit one-click method to show these files under “Locations” (though iCloud Drive has its own in “iCloud”)
I can accept that as reasonable for things that belong solely to a single app (such as Mail), where the user doesn’t need Finder access to the files. But cloud storage is used for so many different kinds of files, many of which are used by multiple apps, under a variety of circumstances, that making the files invisible to the user makes things harder in many cases.
I don’t want my data hidden in enclaves, partitioned off from related files solely because of a single characteristic. I want my data to be with my data, with related data stored together. This is one of the reasons I abandoned Photos.app years ago—I want my images where I can find them in an Open File dialog, not archived away behind Apple’s date-focused interface. (That and the fact that photoanalysisd loves to crash on me.)
It’s also one of the reasons I don’t consider using an iPad for any serious work that’s not app-specific. The arbitrary separation of Photos and Files storage drives me batty. Many image-processing apps can open from and save to only Photos, not Files, while many other apps can’t access Photos, but can access Files. Some apps can share Photos but not Files, and some vice versa. It’s maddening.
Files usable in multiple apps need to be openly accessible to those apps. Hiding them in enclaves unnecessarily restricts that accessibility.
But they are not - the Finder Sidebar has specific entries for Dropbox and OneDrive that are quite easy to find. I just opened BBEdit, did a File Open - right there in the Sidebar, an easy-to-click OneDrive location that shows my OneDrive files.
Yes, this is precisely my point. Many of the replies have centered on Apple beefing up security and not allowing kernel extensions, but none of that explains why the directory has to be under ~/Library.
Hiding it from the user is a possibility. But I have another theory. I think whatever we find under ~/Library/CloudStorage is NOT going to be our familiar directory of Dropbox files. More likely, it will be an unintelligible data format that optimizes indexing, compression, and synchronization, like how iCloud Drive stores stuff.
Unfortunately, I can’t prove this because none of my Macs’ Dropbox has moved to this new location yet. (Maybe one of you has and you can look and see if those look like normal files?)
Now, raw files converted to binary is certainly a good reason to hide them from users. But sticking things under ~/Library is not the only way to hide files from users. Short of blobs, we have Packages. Both are opaque to users. And either could be relocated to a drive of one’s choosing. Again, I call iCloud Photo Library to the stand as an expert witness
I don’t have Dropbox, but do have OneDrive, and all that is listed in ~/Library/CloudStorage is a single listing for OneDrive. Double-click it and you get my actual files and folders stored in my OneDrive account. I imagine that DropBox is no different.
Because there’s no technical reason. It’s just what Apple decided to do, and they also decided not to make it user configurable. I don’t think there’s any great reasoning except to make it consistent with where they located iCloud Drive (and be deterministic across all systems). Like you I don’t agree with this decision, but I see no technical or security reason why Apple couldn’t allow more flexibility… they just don’t.
Actually, iCloud Drive stores normal files in normal folders on your Mac, it’s just the location (and naming of some top-level folders) that’s obscure.
). It seems the best that can be done with Apple’s limitations. But it’s worth people knowing that it will be slightly different from the experience to now (or using Maestral) when storing files on an external drive.
