Apple some years ago announced that it was deprecating kernel extensions for security reasons. As of Monterey 12.1, Apple’s recommended way to implement implement remote file access and file syncing services was to use the Apple File Provider framework instead of kernel extensions.
Some companies, like Microsoft, started rolling out new Mac clients with Apple File Provider when 12.1 shipped. Others, like Dropbox, have taken a little more time.
As of now, it seems that most third party remote storage and sync tools have been updated to use the Apple File Provider framework during the Monterey release cycle. This includes the major platforms, such as Microsoft OneDrive, Dropbox, Google Drive, and Box, as well as less widely used tools like Egnyte and others.
The process of switching from third party extensions to the Apple File Provider framework has not been smooth. Microsoft’s release of a new OneDrive for Business client removed some features, significantly impacted enterprise workflows, and created considerable user backlash and confusion. It was a bit of a fiasco for my organization’s help desk. Apparently due to pressure from enterprise customers, Microsoft addressed some of the issues introduced by the the new framework.
While multiple clients now use the Apple File Provider framework and share some characteristics (like having their sync directories inside ~/Library/CloudStorage), each client still has its own distinct user-facing features, user interface, and strengths and weaknesses.
If you are using a freshly installed third party syncing client on a version of macOS earlier than Monterey 12.1, you are not using the new Apple File Provider system. It is important to remember that third party tools will not be supported on older versions of macOS indefinitely. Google Drive no longer updates its client on Mojave, for example, and any client might stop syncing at some point in the future.
If you are using current versions of Monterey or Ventura but are not yet using the Apple File Provider framework, it probably is because your sync client has not updated itself yet.
My guess is that most users won’t notice any differences after switching, though some people who leveraged their clients’ capabilities more thoroughly in the past will have some adjustments to make. Unfortunately, since all of the providers are going through this change, switching from one platform to another is unlikely to avoid the disruption caused by adoption of Apple File Provider technology.
Sorry for being so wordy, but I hope that I clarified more than I confused.
Correction added June 15, 2023: Added “freshly installed” to “If you are using a third party syncing client on a version of macOS earlier than Monterey 12.1, you are not using the new Apple File Provider system.” It appears that at least some third party storage client providers, including Dropbox, have been rolling out client updates in a phased fashion, so if you had installed a third party syncing client prior to Monterey 12.1, it’s possible that you might be using the older, kernel extension version of the client for a little while longer, even if you upgraded to 12.1 or newer. You will be upgraded eventually, however. If you do a clean install of the client on a system that is running 12.1 or newer, the Apple File Provider version of the client will be installed.
What I don’t understand is why the File Provider API makes it necessary to store synced files only in ~/Library. What is special about ~/Library compared to elsewhere in the user home folder? It seems like an unnecessary restriction.
Adam,
This is something I would love to know more about. We are heavily dependent upon Google Drive in our workflows.
We also use 4D as our main database and it is very picky about saving documents to Drive. A year ago, 4D Tech Support instructed us to move our users’ Google Drives to their user directories in order for 4D to “see” the folders. We did that thinking we were home free. Then along comes Google complying with Apple’s File Provider API! Now, we can no longer put the Drive where our database can see it, but it has to live inside the ~/Library/CloudStorage.
Drive operates as normal in this new location. But our database cannot automatically save files to the various shared drives and folders any longer.
A few months back I found a terminal command (touch ~/Library/Application\ Support/Google/DriveFS/fp_left_beta) that would still allow us to keep the Drive in the user root directory, and therefore, visible to our database. But I see now that that command no longer works and I cannot reset the location of Drive anymore.
We cannot control when this happens or why - it just does random Macs for random users at a random time!
I cannot express what a disaster this is for my users! I am crafting a ticket for 4D as we speak to describe the issue again and look for resolutions. So any light you can shed on this would be most appreciated!
Just a guess on my part, but this location is invisible to less-technical users, so the likelihood of an accidental deletion or move of a location to a new space (something a less technical user can easily do from the Finder if looking at their home folder) is my guess. This is what Apple did with iCloud Drive. I’m not sure that Apple has ever explicitly said why the storage location is in ~/Library/CloudStorage, but Apple doesn’t make it easy to accidentally find your way in the user’s library folder.
Also note that the Finder sidebar does have an explicit one-click method to show these files under “Locations” (though iCloud Drive has its own in “iCloud”)
I can accept that as reasonable for things that belong solely to a single app (such as Mail), where the user doesn’t need Finder access to the files. But cloud storage is used for so many different kinds of files, many of which are used by multiple apps, under a variety of circumstances, that making the files invisible to the user makes things harder in many cases.
I don’t want my data hidden in enclaves, partitioned off from related files solely because of a single characteristic. I want my data to be with my data, with related data stored together. This is one of the reasons I abandoned Photos.app years ago—I want my images where I can find them in an Open File dialog, not archived away behind Apple’s date-focused interface. (That and the fact that photoanalysisd loves to crash on me.)
It’s also one of the reasons I don’t consider using an iPad for any serious work that’s not app-specific. The arbitrary separation of Photos and Files storage drives me batty. Many image-processing apps can open from and save to only Photos, not Files, while many other apps can’t access Photos, but can access Files. Some apps can share Photos but not Files, and some vice versa. It’s maddening.
Files usable in multiple apps need to be openly accessible to those apps. Hiding them in enclaves unnecessarily restricts that accessibility.
But they are not - the Finder Sidebar has specific entries for Dropbox and OneDrive that are quite easy to find. I just opened BBEdit, did a File Open - right there in the Sidebar, an easy-to-click OneDrive location that shows my OneDrive files.
Yes, this is precisely my point. Many of the replies have centered on Apple beefing up security and not allowing kernel extensions, but none of that explains why the directory has to be under ~/Library.
Hiding it from the user is a possibility. But I have another theory. I think whatever we find under ~/Library/CloudStorage is NOT going to be our familiar directory of Dropbox files. More likely, it will be an unintelligible data format that optimizes indexing, compression, and synchronization, like how iCloud Drive stores stuff.
Unfortunately, I can’t prove this because none of my Macs’ Dropbox has moved to this new location yet. (Maybe one of you has and you can look and see if those look like normal files?)
Now, raw files converted to binary is certainly a good reason to hide them from users. But sticking things under ~/Library is not the only way to hide files from users. Short of blobs, we have Packages. Both are opaque to users. And either could be relocated to a drive of one’s choosing. Again, I call iCloud Photo Library to the stand as an expert witness
I don’t have Dropbox, but do have OneDrive, and all that is listed in ~/Library/CloudStorage is a single listing for OneDrive. Double-click it and you get my actual files and folders stored in my OneDrive account. I imagine that DropBox is no different.
Because there’s no technical reason. It’s just what Apple decided to do, and they also decided not to make it user configurable. I don’t think there’s any great reasoning except to make it consistent with where they located iCloud Drive (and be deterministic across all systems). Like you I don’t agree with this decision, but I see no technical or security reason why Apple couldn’t allow more flexibility… they just don’t.
Actually, iCloud Drive stores normal files in normal folders on your Mac, it’s just the location (and naming of some top-level folders) that’s obscure.
I had my 140GB music library set up to sync to a backup folder on Dropbox. When I was forced to move to the Ventura version of Dropbox, somehow it removed the Mac’s version and left me with the entire library on the new Dropbox folder in iCloud. In the course of trying to help me get my library back onto the computer, I wound up with a Media folder that was 95% duplicates. Took me weeks to clean it up, and I’m still discovering stuff I missed the first time through.
FYI, as part of the attempt to fix that, I tried to use a Doug’s Scripts program to find and delete duplicates. The program didn’t work and the developer was 100% non-responsive.
Your suspicions are absolutely correct in terms of the Sidebar access and the CloudStorage folder. I have OneDrive, DropBox and Google Drive. All three are found in the CloudStorage Library folder, but when double-clicked, open up from the Sidebar Locations listing.
Seems completely arbitrary to me…and not well thought through. One would think with the inability to expand initial storage they would have considered large cloud storage spaces. I’ve got a 2 TB DB account that I am only using enough at this point that it’s not an issue…but it could easily become one and I’m sure I ain’t alone in that b
Unfortunately, I can’t prove this because none of my Macs’ Dropbox has moved to this new location yet. (Maybe one of you has and you can look and see if those look like normal files?)
Still a normal folder structure…but once you’re in DropBox in CloudStorage clicking on the top icon does not allow you to go back up to the CloudStorage directory. And I have an M1 MBP and a Studio both running 13.2.1 and the same version of DB…but on the laptop the folder is still at ~ and not in CloudStorage which does not exist on the laptop and I don’t have the little Update me red flag on the laptop since it’s already up to date on DB version.
Agreed. Another example of Apple dropping the ball on cloud storage is with iCloud Drive. I’m the only person on the planet that I know of who has their “hidden” upgrade option for 4TB. It’s like they’re not totally sure they want you to actually be using and enjoying the hardware and software they worked so hard to do a great job with (and which you have paid for). And almost no one on their support staff even knows about the 4TB option. And I lost half of the 4TB by cancelling my News+ subscription which I no longer needed since I signed up for Apple One. They couldn’t figure out how to fix it, but I got it back working by adding iCloud+. In typical fashion, I knew more than most of the Apple Support people I call (but often not enough to fix my issue), which is highly annoying.
Anyway, I’ve veered off topic, except to agree with you that I’m not convinced Apple has thought through implementation and storage details except for people with moderate consumption levels.
I have to admit I’m puzzled by the complaints that the files are inaccessible except through the Finder sidebar. I recently installed Google Drive and Microsoft OneDrive on my computers. Though the files are stored in the Library folder, there’s an alias in the home directory that points to the relevant folder in ~/Library/CloudStorage. From my perspective, it makes no difference if the actual folder lives in ~/Library/CloudStorage or in ~/. I would expect apps that aren’t doing anything tricky and use Apple’s APIs to navigate the file system would be in a similar situation. Can’t 4D access the files through an alias?
I can tell you some history of this. Used to be cloud sync vendors like Dropbox and Microsoft OneDrive could write their own sync app, and it usually was in the form of a kernel extension. Apple decided that for security reasons, only Apple should have access to the kernel, and therefore beginning in Ventura kernel extensions are no longer allowed. Monterey used to bark at you about this, although you could still allow a kernel extension. KEXTs were to be replaced by either System Extensions or Network Extensions, which do not burrow so deep into the system. ALSO, Apple decided to make the Files Provider Framework available in the system. And they were like, “Hey, cloud sync providers, just use our framework, it’ll be way easier than writing a new system extension or network extension for your stuff.” Now, I’m quite sure that this is what Apple itself uses for iCloud Desktop and other sync access to iCloud storage. But remember that Apple is myopic in its assumption that all Mac users are just brand new computer users who have never had any other computer than an iPhone with the hidden file system of iOS. So they have made some pretty stupid decisions about what the Files Provider Framework does, and how it does it. And EVERYTHING they have been doing to the Mac recently is with a view to hiding the file system (eg. Default new Finder window goes to Recents instead of somewhere smart like ~/ ) Specifically, with Ventura (and possibly later versions of Monterey) that the ONLY place FPF would sync is ~/Library/CloudStorage, regardless of where the developer of 3rd party software WANTED to be able to sync. Thus, all of the sudden, cloud storage providers such as DropBox and Box.com said YOU CANNOT SYNC EXTERNAL DRIVES, even though they used to be able to do this, because external drives (Like, say, your RAID5 Thunderbolt drive) mount to /Volumes/, not ~/Library/CloudStorage. So the moral of this story is you would probably need to lean on your cloud storage vendor to figure out a way out of Apple’s limitations, because there is no way to get to someone at Apple to change the decisions of the Files Provider Framework product manager or whatever. Yeah, I’m quite bitter about these changes as it directly affects a significant workflow at my company, forcing us into buying an expensive NAS device for storage and sync to DropBox, rather than our quick and dirty Mac Mini synching large storage RAID device to DropBox.
