What 25 years of managing personal media taught me

TidBITS Talk
1

In the early 2000s, as digital media devices became more common and personal computer storage started evolving, I — like many other people — began storing photos and videos from many different sources: Sony Cybershot cameras, Nokia N95 phones, images downloaded from ICQ and MSN, and many other devices and applications that no longer even exist today.

At that point, organizing media files was still relatively simple because external storage technology was limited. Network-attached storage was expensive and not easily accessible, and most files still lived on internal computer drives.

With the arrival of the iPhone and similar smartphones, the number of media sources increased exponentially. At the same time, external storage devices became much more accessible, which expanded the number of places where files could be stored.

By then, I already had older media stored on internal drives and started moving newer files to external disks. Around the same period, I also started using something that didn’t exist at the beginning of that journey: media library applications. I moved to iPhoto, which introduced yet another place where my media files existed.

Whenever I replaced my MacBook, I created backups of important files, including my iPhoto libraries. The new machine would continue syncing newer media, but the older libraries remained archived exactly as they were at that point in time.

Eventually, I had media files spread across:

· active libraries,

· library backups,

· exported folders,

· external drives,

· NAS devices,

· and later, cloud storage as it became more popular.

At some point, I decided to consolidate and reorganize that growing collection into a single structure. To do that, I exported media from Photos libraries, created a new library from scratch, and started importing both the exported media and the files scattered across multiple directories and storage devices.

It was not a successful experience.

Separating duplicated media from files without geographic metadata quickly became difficult, especially as the archive kept growing over the years.

Last year, I finally decided to perform a complete reorganization of my entire archive, which had grown to 363,575 files distributed across 10 Photos libraries — one of them containing 182 GB and more than 40,000 files, another with 165 GB and nearly 39,000 files — in addition to 247,130 media files spread across 8,861 directories.

What became clear during that process is that storage itself was no longer the main problem.

The files still existed.

The problem was that the archive structure had gradually become fragmented across:

· libraries,

· exports,

· backups,

· duplicate copies,

· disconnected folders,

· and different generations of catalog applications accumulated over more than two decades.

At some point, I realized I was spending less time thinking about storage capacity and started thinking more about structural consistency.

That was the moment when I stopped looking at the problem as simple media organization and started treating it more like a normalization problem.

9 Likes
2

I’m getting increasingly concerned with it as a legacy. What a pile to leave behind. Am thinking of simply printing a set number of photographs per year, boxing them, and at some point holding my breath and wiping the files, maybe after a set period, so I only had the prior three years in digital form.

3 Likes
3

I’ve never used iPhoto or Photos for files produced outside the iPhone and iPad ecosystem. Photos from film and dSLR cameras are stored on hard drives and Dropbox. For a while, they were on MO disks and on CD and DVD.

One of the biggest issues for me was coming up with a naming structure that made sense. I have older film scans that need to be rescanned or renamed because I was not careful about how I named the files. At some point, I started using file names that indicated the film type and the frame number and labeling slide and negative sheets and envelopes of prints and negs to make the source images more findable. It’s an ongoing battle: I got my first 35mm SLR in 1971 and my first dSLR in 2006.

Digital images were easier to deal with than scanned film. dSLR images are organized by camera, within which they are organized in folders named by date and location. I use Name Mangler to add prefixes to these images, indicating which camera was used. The counter on my Pentax cameras goes to 10,000, after which it resets, so to avoid duplicate file names, the prefixes are K5-1_ for the first 10,000 and K5-2_ for the second, and so forth.

I’m still trying to refine what I do with naming film scans. It’s a bit insane, because no one will care when I’m no longer around.

3 Likes
4

I have never used iPhoto (or Photos) except to keep some on iphone/ipad . Everything now is on digital media and catalogued — originally using iView Media Pro, then and currently lightroom. 50k+ photos going back to late 1800s (scanned family). But photos on computers aren’t readily accessible. So …. I rely heavily on Blurb books, printing books covering all those years, using the best or most important photos. My dad fought in the Phillipines in WWII and kept a scrapbook of everything from the moment he enlisted - documents, letters, photos, meal tickets - all now scanned, printed and on DVD. For the last 15+ years, i now create an annual “yearbook” of photos. Overseas and cross-country driving adventures get their own books. My best shots are also in large format “portfolio” books. I am comfortable knowing that the digital files may degrade, but family has hardbound books. And, most important, when my memory starts to fail, someone can sit down with me and open a book and say “See… You did that! You went there!” :rofl:

4 Likes
5

Yes, I did that with my uncle’s negatives, scanned and printed a bunch to put upon his nursing home’s walls. Staff and visitors get to talk about them, he went all over the world. I’m making a book over the summer which will feature more. It helps jog memory and discussion, so hard to have conversations when everyone walking in the door is effectively a stranger.

4 Likes
6

That’s actually one of the things I started thinking about during this project.

The technical problem was organizing the archive, but the long-term question became how future generations will be able to access and understand it.

Storage keeps changing. The files often survive. The harder problem is preserving structure and context so that the archive still makes sense years from now.

1 Like
7

Your comment resonates with me because naming conventions ended up becoming a much bigger topic than I originally expected.

What starts as a simple way to identify files gradually turns into a long-term archive management problem, especially when decades of scans, cameras, formats, and storage media become part of the same collection.

One thing I learned is that decisions that seem minor today often become extremely important twenty years later.

5 Likes
8

Sigh, yes. This was not something I was thinking about enough when I started accumulating images. The date on commercially processed negs, for example: when I was shooting color film, and got it processed, the envelopes containing prints and negs nearly always had a date on them that almost never included the year. So I have way too many “sometime in the ‘80s” or “sometime in the 90s” negatives. Slides were a bit better in that they sometimes included date info on the mount, but that doesn’t help with any slides that I got processed as a roll and then mounted myself. I can’t even see those frame numbers without taking the slide mounts apart (insert expletives here).

When I got more systematic about naming film scans, I found Tom Philo’s guide to Kodak film numbers useful sometimes. Including 4 digits in the file name often can be used to indicate the film used. Tom Philo list

I also encountered naming pitfalls when I was (slowly) ripping my CDs into iTunes, because probably 80–90% of the CDs are classical. The CD database from which track info is pulled is horribly inadequate for classical music, nearly always having misspelled foreign language entries and lousy info on performers, among other problems. What should have taken a few minutes per CD always took much longer.

1 Like
9

ALL of the above.

I’ve settled on a folder structure based on the date of the event (not the date of the scan), yyyy/mm/dd, Filenames don’t matter - metadata rules. I use NeoFinder to set keywords for the who, what, and when. Retrieval is easy with a NeoFinder query to find images with (keyword) between (startdate) and (enddate). (Spotlight might work, but I’ve never tried.)

I still have too many loose ends, but I’m making progress.

1 Like
10

Film:

79-12-09 is what I came up with in the early 80s when I started storing negatives in sleeves.
Most of my sheets just say something like 1983-03 on the top
As I did more developing, I’d often write everything out on the back of the picture because they weren’t always in the envelopes, and when I was photographing events, it was easy to find the original negatives if someone wanted a copy.
I have a bunch of archival boxes with photos organized by category and would have to rely on memory to date them without that writing.

1979
12th roll of the year
9th frame

I think when I scan them in though, I am forced to 3 digits for the last one.

The problem recently came when I started scanning in old family slides/negatives
It was easy to do this initially as they were developed. But decades after the fact? And knowing that the date on the slide/photo is when it was developed, but maybe not taken.

74-Aug-023
74-June-#4-026 (#4 being the fourth roll in the month)

I do love my old Lightroom way of filtering by exif

1 Like
11

With the Photos app sorting photos into albums one option for long-term storage of important photos is to use its Export feature to create a folder (or folders) with files that have the album name and a number (for easier recall later). Then use Graphic Converter to reduce files sizes in that folder using its batch features. Finally copy the files to 50Gb Bluray disks and delete the local files.

There are earlier discussions about long-term storage and I realise that finding Bluray readers could be an issue in a decade or so but I expect there will be ways to read the disks.

1 Like
12

I store all my pictures in a folders with YYYY-MM-DD Description. That way I can do a simple search, lightroom is good with the structure, and it works with my scanned, digital camera, and cell phone photos.

1 Like
13

Márcio — you’re describing the tip of the iceberg. I’m wrestling with a similar volume, stored the same scattered way, and I’ve hit the same walls:

  • Subscription creep. My collection grows, so the price grows — then the vendor “improves” the product (editing, now AI) and the price climbs faster than the value I get from it. Switching suppliers just resets the same cycle.

  • Manual organization. Cheap (you only pay for storage) but brutally time-consuming. For me time = money, so the project never quite gets finished or maintained.

My current patchwork: iPhoto for recent photos (2020–present), manual folders for everything older (1900s–2019), Plex on a dedicated 16 TB server for video and audio, and Evernote for PDFs, scans, and reference docs — though Evernote’s price has gotten hard to justify.

The bigger realization: media is just one slice of a broader problem — digital data. Since going paperless around 2012, I’m managing 350,000+ items. The real challenge isn’t storage anymore; it’s handling volume, cost, redundancy, classification, and fast retrieval all at once — and doing it on a budget.

And the part that matters most to me: this has to outlive my own ability to curate it. It’s my digital legacy, but really it’s my family’s data. Whatever I build has to be something my wife and kids can inherit and keep running.

So my recommendations:

  • Keep going — you’re on the right track.

  • Reframe the goal from “media” to “digital data.”

  • Design for budget and succession from the start.

  • Watch AI/LLMs closely; that’s likely where the real solution comes from.

Good luck.

2 Likes
14

After decades of facing the same problem, I’m left with another conclusion: for whom are you doing all this filing? You kids? Your legacy? Does anyone else care what box of Kodachromes, or collection of photo files you leave behind?

I have boxes of old Super 8 film in my basement that none of my kids seem interesting in. Will this be any different?

2 Likes
15

People will care more if they can easily view the photos and film.

Are there recommendations for a dedicated slide scanner, or slide and negative scanner? Is there one device that could also scan Super 8?

1 Like
16

When I was a kid, I wouldn’t have been interested in boxes of old film from my parents’ younger days. Now in my fifties, I would give anything for them.

3 Likes
17

The Swedes have a practice called death cleaning, essentially getting rid of things before you go so your kids won’t have to. My mother, Irish not Swedish, took a shine to this idea a few years ago and has whittled her things down, gave her three offspring all a set of old photographs each and put names on the back each of the artworks my father painted.
I will do a digital version of this at some point, my eldest has my login details, my master password, and knows where I keep all my important docs in devonTHINK. My main creative output is my photography and I think if I had a set of prints the kids could divide it up between them. I’ll spare them the terabytes.

1 Like
18

As a briefly pro photographer and 57 years casual shooter of film and digits, I am also thinking about this topic, getting a better grip on it all, but I often come back to @kat634e ’s point:

Which for many of us could be the case. Pros who’ve captured historical moments maybe not as much. I don’t have much of anyone besides my wife who might actually look at the stuff (no kids, family abroad). I’ve thought of laying out my faves in digital books in various formats (and maybe printing), with captions/explanatory text etc so anyone in future looking at post-me would have and idea of the significance of the photos.

Years ago I spent lots of hours digitizing old films and tapes from my parents and self, adding titles and transitions etc. and sent off DVDs to family. Maybe one word of thanks.

For whatever reason I have this compulsion to document my life but also realize all the documentation will just sit around or go in the dump eventually (shelves full of negs-slides-prints-clippings). I have various times printed a document for family to explain how I file things on my computers so they can find what they need, if they need, and want to explore, but it might be wasted effort.

Still, while I’m here it would be satisfying to have a better handle on the digital assets. I do have a fairly consistent organizing/naming method over several years, which is a start.

Having been burned by Aperture’s demise and library confusion from iPhoto and frustration at needing to be online to use Lightroom, I am simply organizing in Finder and using apps that can see the file structure, display it nicely, and make changes that save back to that structure (ie Graphic Converter, Nitro).

I might go like @TGarf and print annual books-themed books from a service, and add them to the shelves where my photo albums are (many also with documents-souvenirs-captions).

would be interested to read more about how you’re doing this, either as a Topic or dm.

Similar here. YYMMDD for me, then brief caption like description then original file name. When I go back and scan stuff from the 1900s I’ll figure out a way to name them for proper sorting. The leading 20 I leave off as I will not live into 21xx and saves a lot of typing! organizing by date makes sense for me and I don’t like to depend on any search tool any more and can remember roughly when something I’m searching for happened. And my film stuff is also by date ie ’85-12 and so on on envelopes, slide pages, etc.

We’ve seen a number of deaths in our neighborhood and friends recently so this downsizing before you go idea is on our minds and I do a bit at a time, getting rid of anything, digital or otherwise, that I don’t use anymore.

3 Likes
19

Re: making the book - I’ll probably simply use a service like Blurb or equivalent, who can take a PDF. I’ll layout the PDF in Affinity using one of the service’s template or sizes. The audience is small, primarily my uncle and any of his occasional visitors, his sister and some of my cousins who will be able name folks and recall stories. I’ll only make three or four copies, one for him, my aunt, me and a backup. To be honest going through the mayhem of negs, prints, unidentified people and events in several boxes is the actual work. I’ve an excellent scanner once that’s done, layout and production will be easy in comparison.

1 Like
20

FYI - Lightroom CLASSIC does NOT requIre you to be online. There are two varieties of Lightoom and CLASSIC is non-cloud driven, desktop. Classic has some cloud access but in all these years I have never needed or used it. Tim Grey (timgrey.com) uses classic and explains its advantages. So don’t give up on Lightroom if the online issue is what has kept you away.