I ask this question purely out of intellectual or academic interest, now, as I worked around the problem. But I am interested to know why files were invisible one on external NTFS-formatted drive but visible on another.
Time Machine reported that the backup on my NAS had failed and needed to be overwritten. Before I took this drastic step, I copied the backup folder from the NAS onto an external drive plugged into the NAS. The drive came formatted NTFS, and the copy worked flawlessly; all files were clearly visible via the NAS Web interface.
But when I plugged the drive into my M1 Mac (OS 12.6, Monterey), the Time Machine folder wasn’t visible, although the free space on the drive was reduced by 1.5 TB, the size of the TM folder. (However, the default “installation” files on the drive were visible.)
I plugged an older NTFS-formatted drive in and everything on it was visible, so the Mac isn’t at fault.
I ended up moving the TM folder from the iffy drive onto the other NTFS drive (again via the NAS Web interface) and all is well—I can read the backed-up files on the Mac if I need to.
But the question is: Why did the Mac not see the TM folder on the newer drive but it can see it on the older drive? An invisible volume?
My guess is that the folder was hidden in a way that macOS respects but the NAS Web interface does not.
macOS files are hidden in two ways:
File name starts with a period, or
HFS invisible bit is on
(fun fact: in OS X 10.3 and earlier, there was a third way: list the file in /.hidden)
The HFS invisible bit is normally set with the chflgs [no]hidden command. Another way is SetFile -a “V” or “v”. On non-HFS+ volumes this creates a ._filename file to hold the metadata.
So one possibility is that the NTFS volume contained the ._filename file for the Time Machine file or folder, which told macOS to hide the file.
That’s quite plausible, and I’m happy to accept it as the likely explanation. The solution in the future is, of course, to reformat new drives in APFS. (I’ve now reformatted the drive, so I can’t search for hidden folders now.)
Can you see the files from a command-line? If you open a terminal and type ls -a /Volumes/volname (replacing “volname” with the name of the drive’s volume), do you see all of its root-directory files/directories?
If the files are visible from the command-line and are not being presented by the Finder, then it could be hidden in one of the ways @mschmitt mentioned. NTFS has its own “hidden” and “system” attribute flags that macOS may also be respecting.
You can tell the Finder to show hidden items by typing CMD-SHIFT-.. You can hide them again by typing it again. (h/t TechRadar). I couldn’t find a button or menu item somewhere to do this (on macOS 11 Big Sur), so this appears to be a truly undocumented command.
If the files are not visible from the command-line at all, then there may be something else going on. We’d need to investigate further to determine what’s going in in that case.
As I mentioned, I’ve now copied the files off the drive (via the browser-based interface to the NAS) onto another (NTFS) drive and reformatted the first one. But I will keep this in mind if it crops up again; it hadn’t occurred to me at the time that there might be hidden folders or files, which I would have known how to make visible if it had ever crosed my mind.
I use CMD-SHIFT-. myself to make hidden files visible, and am glad I stumbled on that trick.
This thread has been puzzling me. Without third party software macs can read but not write to NTFS. I have not seen any third party software mentioned. Does using an NAS web interface change that?
Yes. My NAS (Synology) runs an OS based on I know not what, but it emulates Windows in style and function. As the NAS is a headless computer, it needs a way to control it; that’s provided by a Web interface reached via a 192.168.1.nnn address. This brings up the various options that control the NAS and its various apps. That includes a file viewer, similar in style to Windows, but I assume proprietary.
Through the Web browser, I can list all the files on the NAS and move them around (compared with MacOS, however, it’s very clunky). So I could see that my Time Machine files were all present and correct, even though MacOS couldn’t see them when I plugged the USB drive into my Mac. And this is likely due to settings for hidden files that I wasn’t aware of.
When the drive is mounted in a NAS, macOS doesn’t know (or care) how the volume is formatted. It sees it as a network volume (probably SMB-based, but maybe AFP, NFS or other protocols).
It is the NAS (and whatever operating system it is running) that needs to know how the disk is formatted. Presumably, Synology’s software supports NTFS, which makes the entire scenario possible.
The only thing that surprises me here is that a new external drive came formatted as NTFS. The few I’ve purchased have always come formatted as FAT32 (for drives under 2TB) or exFAT (for larger drives), in order to ensure maximum out-of-the-box OS compatibility.
But, that having been said, I always reformat new drives before I do anything with them. The file system chosen will depend on the size of the drive and how I plan on using it, but I typically use HFS+ (for Mac-only HDDs), APFS (for Mac-only SSDs), FAT/FAT32 (for cross-platform devices where no files will be bigger than 2GB) or exFAT (for cross-platform devices that need to hold huge files, like VM images).
Depends on how they were hidden in the first place.
If the file’s name begins with a dot (e.g. .login), then it will always be hidden if you don’t request to see hidden files.
This behavior goes back to the earliest days of Unix and exists because the Unix standard is to use these “dot-files” for per-user configuration/preference files, and a typical home directory may have dozens if not hundreds of these. To always show them would clutter up the UI when there is rarely a need to directly access those files.
If this is the case, your only option is to rename the file. But since dot-files are typically configuration files for applications or parts of the system, renaming them may break something. If you didn’t create the file, don’t rename it.
The “hidden” flag may be set. This flag hides files from the GUI, but does not hide it from Terminal sessions.
To check if this is the case, In a terminal window, use the -lO option on the ls command to let you see if there are any flags on the file. For example:
$ ls -lO
total 8
-rw-r--r-- 1 user staff hidden 12 Sep 13 16:05 foobie.txt
If the file is not hidden, then the hidden keyword will not be present.
You can remove the flag with the following command:
chflags nohidden <filename>
Where “” is the name of the file.
For more information about flags type man chflags or go here. Note that the chflags command is a BSD-standard Unix command, but not all systems support all of the flags. Checking your local manual page (type man chflags) is the best way to see what your computer supports.
I think there are (or were, in older releases of macOS) other ways that files could be hidden, but I don’t remember the specifics at this time.
Thanks, @Shamino. I came back to report what I did, and you had already responded.
I have no idea how they were hidden or by what process; I just went to one of the folders with hidden files and found two files where there should have been many. I looked at the same folder from another device, and there were many files. None started with a period.
Before posting the comment to which you responded, I tried to look at the folder in Terminal. After navigating to what I thought was the appropriate (parent) folder, I typed ls (with no options) and the operation failed. (I don’t recall the message.) In my befuddled state, I did not recall that I could drag-and-drop a file or folder into Terminal. So I asked the internet, and here’s what I did. (Apparently the hidden flag had been set.)
In too much detail, there were three iCloud Drive folders (that I have found so far) that contained (over 100) hidden files. (Interestingly, not all the files in two of the folders were hidden; just most. The third folder had only two files, and they were both hidden.) The three folders were all in the same containing folder. The final command I ran (after some initial testing and reading man chflags multiple times) was chflags -R nohidden [containing_folder], where I dragged and dropped an iCloud Drive folder that contained all three folders that had hidden files.