Apple’s File Provider Forces Mac Cloud Storage Changes

It’s a good question, but I doubt anyone outside the iCloud team knows. ;-) My best guess would be that Apple needs to do special things with iCloud given how deeply it’s integrated into macOS, and some of those preclude being located in ~Library/CloudStorage.

1 Like

Can someone do a sanity check for me? I’m trying to use the Finder window’s search field to find a file that I know exists in Dropbox. (The same problem applies with Google Drive.) When I select Dropbox in the sidebar and then limit the search to Dropbox, it never completes. However, if I change the scope to This Mac, the file appears instantly. So it is indexed!

Finder 2023-03-21 at 12.20.21
Finder 2023-03-21 at 12.20.25

I’m seeing the same problem on my M1 MacBook Air, and both it and my iMac are running Ventura.

Do others see this behavior too? If so, it could be a problem with the File Provider architecture.

I don’t see that problem with either of my Google Drive folders. When I did the search, the process FinderHelper did ask for access to the files managed by Google Drive. Revoking that access didn’t make any difference with the search results.

I do see the problem, but not necessarily limited to cloud storage. Seems to me like a problem with finding in a particular Finder folder.

1 Like

As shown below, the web interface to Google Drive has three categories of storage, namely:

  1. My Drive
  2. Shared drives, and
  3. Shared with me

But, Finder shows only:

How do I get “Shared with me” to be accessible from the Finder?

1 Like

Huh! Curiouser and curiouser. I’m using Ventura—what versions of macOS are you two using?

Anyone else?

I’m running Ventura.

Adam, I found exactly the same behavior (see Apple’s File Provider Forces Mac Cloud Storage Changes - #46 by jamesajacobs). from following posts over in dropbox’s own discussion site, i’m hoping macos 13.3 will fix this?

1 Like

iCloud is not involved with Dropbox. DropBox Storage is a cloud.

I’ve been acquainted with the structure of DropBox (DB) for many years. Let’s start with the high-level view.

There are four components of DB, the cloud storage database, a local work area in macOS, the macOS DB application, and the DB website.

The Dropbox.com website (the cloud site) does the usual things - sets up an account, prepares the cloud storage and Oops areas,
and downloads the DropBox application.

When the DropBox application is installed, it sets up the local work area storage (the Apple File Provider ~/Library/CloudStorage) and the Oops areas - a mirror of the DB cloud structure.

The work area storage contains a simple file structure of folders and files and cannot be found in Finder. The head of this structure is the DropBox icon inserted into the User folder which shows up in the Apple Finder. Clicking the icon opens DBFinder which looks exactly like Apple Finder. (Calling this another SSD drive would be incorrect.) Here the user copies/moves folders and files between the two Finder structures.

In to early DB days, the DB work area was in the System folder, and the mirror image was that folders and files were synced and identical. This meant that there were multiple copies of the same file - a minimum of 4 (cloud/work areas/two Oops areas). One more if the original file was from macOS, and 2 x the number of times a file was opened and closed during the thirty-day Oops area copies.

It was no wonder there were many unhappy users about how much macOS disk space was being gobbled up.

There’s more to write here but I wanted to get this out first.

E.g., The Oops is a 30-day retention area for Restoring previous versions of a file. i.e. an archival area. …

Before getting into syncing, I need to be clear about “online/offline”. It’s probably the worst pair of words one could choose. The DropBox the cloud storage database is always online except when selecting Sleep or Shut Down. The start-up file contains the file that starts up the DropBox app which immediately looks for work to do, i.e. connected and online. Always, unless the user suspends and restarts syncing manually.

The underlying method employed by DB is called “store and forward”. This method solves problems like this. You’ve finished your wellness exam and the doctor sets up authorizations for pharmacy, blood test, and an x-ray exam. The goal is the make sure that individual is registered in each department to avoid waiting to be registered at each department before they can arrive to get treatment in any order of arrival. And one of the department computers may be down.

The “store and forward” design consists of two forms - hub and spoke or husband wheel. “Hub and wheel” is where everyone is connected to everyone. This is a huge mess when you want to add another member of the wheel (and beyond this discussion).

The hub (the DropBox cloub) becomes the focus of hub and spoke design and is under very heavy activity. For example, if DB had 1,000,000 accounts, with 5 employees sharing all files, and its 5 PM Friday night in a single time zone, and 5 million users closed their application after submitting 1 update or new file, the download would be 5M syncs and 20M uploads to sync the other 4 devices on the account. A perhaps extreme example but mathematically correct. In this “hub and spoke” example, there is no “I” or “me”. It’s an environment of queuing requests. It is a fantasy to expect a half-second response time.

Syncing
Many conditions prevent syncing. As an example, The Fri sync request (the updated file is moved into a DBFinder folder) may be ignored if the hub cannot respond to start or finish a download request because the security protocol requires that a machine must be shut down before leaving the office. That sync will begin syncing Monday when the computer is started up. The file will be stale if looked at over the weekend.

Stay tuned. Mike
To be continued.

Mike

2 Likes

Didn’t find this option mentioned, but you can have Dropbox backup your Documents folder to their cloud. It won’t synch between other computers/phones, but the files are available offline and doesn’t screw up files as sometimes happens in the Dropbox folder. If you need to share between device or provide a link to someone, just transfer or copy the specific file(s) to the Dropbox folder. You may ask, why not put everything as downloaded/offline in Dropbox. My experience was the Dropbox was just not up to it for some reason. Just not consistent with what was happening there.

Hi Albert, Welcome to Tidbits-Talk.
It was unfortunate that you have an bad experience with DropBox. In my message above (though incomplete) I intentionally made up specific names of the four main areas to not be bogged down in Dropbox terminology. I mentioned this was the first version of DropBox as the first version.

  • Without considering any syncing traffic, do you understand the structure and purpose as described? My update will discuss syncing.
  • Regarding other parts of your message, there was no “Option”, these structures were baked-in.
  • Regarding “you can have Dropbox backup your Documents folder to their cloud…”. Yes, but the user has to move the file from Documents to the DropBox file structure. The folders do not have to match names.
  • Regarding Apps. The IOS App store has DropBox available primarily for read-only as the Documents folder is not available to move files into and out of DB.
    (I’ve corrected/rewritten some text in this section)

Hi Mike,

oops, I wasn’t replying directly to you!

I just was putting out the option of using the Documents folder as it is an always-available offline folder that is synched to their cloud when online. As I mentioned, this only suffers from not synching between other devices automatically, as far as I can tell. You can access the Documents folder via a browser from other devices, though.

I use the “hidden” Dropbox folder for cross-device synchronization and just move files in and out as occasionally needed.

Thanks for the welcome!

Scott

installed macos 13.3 and spotlight is working again, including within dropbox folders!

2 Likes

With today’s Ventura update to 13.3 I noticed, on my M1 MacBook Pro, that “fileproviderd” is using an unfluctuating 110-125% CPU. I didn’t notice this with macOS 13.2 but after updating to 13.3, all file activity slowed down drastically (saving a 20 character BBEdit file took over half a minute while Activity Monitor reporting BBEdit “unresponsive” several times) and I started to look for a reason.

I see the same greedy “fileproviderd” phenomenon reported multiple times in both Dropbox and Apple support forums. Since the new Dropbox still describes itself as a “beta” it’s time to click the “leave beta” button!

Replying to myself before I confuse anyone else … with Dropbox disabled, “fileproviderd” still uses over 100% CPU. Back to the drawing board …

@mjtsai just posted about Google Drive implementing a 5-million-item limit. This is the first I’m hearing about it, but it sounds like Google has rolled back the change for now and promises to alert people if it reinstates a limit.

https://mjtsai.com/blog/2023/04/05/google-drive-adds-file-limit/

I was struck by one person’s complaint about not being able to know how many files were in Google Drive, but I think you can do this in Terminal if you’re using Drive for Desktop. Both Bard and Bing (via GPT, call it Bingbot?) claim this command works and it seems to for me:

find . -type f | wc -l

You have to navigate into the Google Drive folder first, which is easiest done with a cd command followed by dragging the Google Drive folder into the Terminal window.

I apparently only have 9375 files in Google Drive, so I’m not stressing about any limits. :slight_smile:

I think the filesize calculation is cached and therefor shows the wrong size. You could check filesize with terminal or perhaps after a restart.

We have big problems when our Google Drive without notice changed all Macs to fileprovider a few months ago. About 50TB of data. The automigration on each computer was awful with old favorites still linking to data that wasn’t synced.

Our biggest problem now is however the new way Google Drive prioritizes sync jobs. It can only do one syncjob at the time and when an offline folder has added content by someone it will start download but the user can’t open files until Drive has downloaded all updated content of the less important folder. Opening a file should always be prioritized.

1 Like

2 posts were split to a new topic: Ransomware protection from cloud storage services

This is probably even lower than a beginner 101 iCloud and Dropbox question but…

I note in Adam’s screen shot of his Finder window that it shows Documents and Desktop.

I use Dropbox for online syncing of my Macs and iPhone and yes I guess it does provide another level of backup to my CCC backups to my external HDs.

So, in light of this change in Dropbox…

Is signing up for iCloud Drive and moving all of my Dropbox files, which is close to 100% of my files, from Dropbox to Documents a good option for online backup and syncing between devices?

Thank you.