Backing Up VM Image Files to Internet Backup Services

(Adam Engst) #1

Originally published at: https://tidbits.com/2019/05/15/backing-up-vm-image-files-to-internet-backup-services/

Code42 has announced that it will be preventing CrashPlan users from backing up applications and VM image files—is this just a selfish move to reduce storage needs? After chatting with Code42 competitor Backblaze, Adam Engst discovers there’s more to it. And there are alternatives.

(Scribblings) #2

I walked away from CrashPlan about two years ago when they really started messing with things, backups became flaky and they couldn’t fix anything. I went to BackBlaze and Arq. This year I won’t be renewing my BB subscription. The service is good enough, but I never used it to restore anything and having go use a web browser instead of a native app to do it seemed clunky.

My best goto nowadays is Arq Cloud Backup which is really plug and play. With one fee you can backup as many devices as you want - you are paying a flat $5 per TB of storage.

I still run TM, but it’s my last port of call to recover anything.

(Anton Rang) #3

Retrospect can perform a bare metal restore from a cloud backup, at least in theory, but my internet connection is slow enough that I haven’t tested a full restore yet.

(Beatrix Willius) #4

The Crashplan persons told me a year ago that I shouldn’t backup applications. That was when their software just wouldn’t do any backup anymore. I had too many files I was told. Head on desk.

Backblaze also doesn’t want to back up applications. Arq is just too unreliable.

(peter@stairways.com.au) #5

Seriously, does no one read The Tao of Backup (Copyright 1997!) any more?

It is right there in step one, Coverage:

The novice asked the backup master which files he should backup.

The master said: “Even as a shepherd watches over all the sheep in his flock, and the lioness watches over all her cubs, so must you backup every file in your care, no matter how lowly. For even the smallest file can take days to recreate.”

The novice said: “I will save my working files, but not my system and application files, as they
can be always be reinstalled from their distribution disks.”

The master made no reply.

The next day, the novice’s disk crashed. Three days later, the novice was still reinstalling software.

Backup everything.

3 Likes
(Nicholas Orr) #6

What Crashplan don’t tell you in those “product updates” is that if you were backing up files that they started excluding or even genuine data but in one of their newly excluded folders, that they will just silently stop backing it up.

And then they’ll prevent you restoring anything from their backups either…

And there’s no way to change the behaviour or get back anything you’ve been backing up for years…

I’d not recommend Crashplan any more to anyone, they silently stop backing up files with no warning and no recourse.

(Fearghas McKay) #7

Hi

Can you clarify what you are finding as too unreliable with Arq ?

Thanks

(Beatrix Willius) #8

Arq stopped working a couple of times. When my computer didn’t boot I found out that the last Arq backup was 10 days old. Once a month Arq verifies all files. This takes 2 days or so and during that time no backup is done. I now get an email for each completed backup and can see immediately when the backup stops.

(Simon) #9

Exactly. Which is why I find these cloud “backups” with all their petty restrictions to be a complete and utter waste of time. Not to mention money.

You can always move a disk off site. Heck, you can fedex it to another continent and rotate if you want to be really paranoid. And then there’s iCloud, Dropbox, et al. for individual files.

1 Like
(Adam Engst) #10

Of course, but there was no such thing as Internet backup in 1997, so The Tao of Backup couldn’t imagine a situation where you’d have such a thing as a tertiary offsite backup made over a bandwidth-constrained pipe. And it was written in a time when you couldn’t download a new version of the operating system and every app on your drive at will.

(OK, maybe Internet backup was a thing in 1997 since it certainly was in 1998.)

(And I can’t resist pointing out that I suggested the entire idea of Internet backup in 1992, in an April Fools article.)

Anyway, as I said in the article, an Internet backup should be in addition to a full versioned backup and a full bootable duplicate. Both of those provide the “everything” backup.

Anyone who relies solely on an Internet backup service is simply foolish. But there’s nothing wrong with ensuring that you have a highly offsite backup of your data that’s always up to date.

Sure, you could put a lot of time and effort into moving a drive offsite on a regular schedule, but in the event of a local catastrophe that destroys your versioned backup and bootable duplicate, the extra few hours to install macOS and key applications is meaningless (not to mention the fact that you’re probably not storing another Mac offsite, so you’ll have to spend time acquiring the necessary new hardware in the event of theft or fire or flood). And it’s pretty unlikely that your offsite transfer schedule happens every 15 minutes, so you could lose a lot of work and irretrievable data in the space between your last offsite transfer and the catastrophe.

I’ll stick with Internet backup, thank you very much.

1 Like
#11

I totally agree. We found the services to be more trouble and expense then they were worth. Time Machine, iCloud, and via work, Dropbox, have served us better.

1 Like
(Doug Miller) #12

Just remember with iCloud that you won’t be able to restore an older version of a file unless you backup locally (Dropbox by default saves 30 days of incremental versions in the cloud I believe.)

This happened to me just yesterday - an encrypted file in iCloud would not decrypt properly. I backed up the local version of iCloud storage to Arq so I was able to get the last working version back (plus I have a cloned backup from about a week ago; that version would have been fine, too, but I just thought of that right now.)

#13

That’s why we use Time Machine with Time Capsule (and I am still unhappy Apple put the kibosh on this) and some other removable hard drives. We don’t use iCloud to archive everything, we store critical documents, and stuff we’re currently working on or will need to work on, and some photos.

(Simon) #14

Personally I only use iCloud for stuff like contacts, calendars, and bookmarks. Actual files go onto Dropbox, onto my own home brew backup running on a server I co-own in Europe (rsync over ssh), or onto disk through TM and SD. I do TM and SD both at home and at work and those disks get rotated. That way if my lab or my house burns down I should still be ok. If all of Berkeley is wiped out by The Big One (and I manage to survive [despite 4M dead expected within the first 24 hrs—yeah take that insurance! :wink: ]), I can still get all of my stuff from HDDs I’ve sent to storage overseas and the more recent stuff from my European server.

Now if both California and central Europe are wiped out and for some reason even the Lab in Japan where I store a HDD is lost, well I assume I won’t survive an event like that anyway. :wink: And if I do, I assume my data will be the last of my worries. :smiley:

1 Like
(Dana Schwartz) #15

“few extra hours”? I don’t think so, grasshopper. More like “few extra days”, or weeks, If you can remember all the apps, settings, add-ons, support apps, etc on your system to begin with.

I absolutely agree with the previous, “Backup everything”. Else recovery means you restore forever. Namaste.

(Adam Engst) #16

No, I’ve done this, more or less, on the occasions when I decide that I need a clean start. I install the latest version of macOS, restore my documents and settings and all that from backup (which could all be retrieved from an Internet backup, if slowly), and then reinstall applications one at a time as I need them. It’s only a few hours to be up and running at a basic level, and I download extra apps slowly as I need them. It’s not a big deal and doesn’t take that long.

But as I said, remember, we’re talking catastrophe here—it’s very unlikely that you’ll be up and running that quickly anyway because your Mac was probably stolen, burnt, or water-damaged along with your local backups. So speed of recovery is already not an issue because most people don’t have a secondary offsite Mac to which they can restore to. And you’re probably more interested in dealing with the catastrophe than getting your Mac up and running again.

Of course, if we’re talking about a business scenario where downtime is seriously problematic, it would be smart to have a full disaster recovery plan in place, which would include alternative working space, plans for acquiring backup computers, and so on. But that’s a whole 'nother level.

Local backups, particularly a bootable duplicate, are provide quick recovery times; Internet backups of documents and data protect you from most dire situations.

2 Likes
(Tommy Weir) #17

I’m on the verge of dropping Backblaze.

I’ve a versioned backup and a bootable clone backup, one in the same building the other in a different building.

All critical files are on Dropbox or iCloud, 3Tb between them.

My photography is the biggest concern I have. I’ve got all the JPEGs backed up on Flickr at least, 150k of them, but my RAW and TIFF files are my key files and those are backed up on a local RAID.

They were also all up on Backblaze. I’ve got 7Tb up on Backblaze all in all. My new iMac arrived, a thing of joy compared to my 6 year old MBPro, but after weeks of trying I cannot get to inherit my backup status. The Backblaze option is now… start again. Rural broadband meant that the original upload took six months or so and there’s been a whole bunch up after that. I don’t know if I can face it.

I’ve had to load up my MBPro, reattach all my drives and keep the backup and drives active.

It doesn’t inspire too much confidence, the support person said my backup was either too large or corrupt.

1 Like
(Scribblings) #18

I have way too many backup systems and the Arq verification is annoying, but important. Not noticed it run for more than a couple of hours though myself and you can always cancel it. I use Arq to backup to both a local drive and Wasabi. Then I also use Arq CloudBackup for myself and family. Crashplan was great until I had too many files (or they decided I had too many) and it spent 99% of its time verifying backups and not backing anything up for days.

Arq CloudBackup is my goto when people ask for advice, otherwise BackBlaze but I’m less happy with them than I was.

Oh yeah, I back up everything even though I have a clone drive of my system. Storage and bandwidth is cheap and I’ve restored entire apps in the past when an upgrade didn’t go the way I wanted (or Apple started removing features from Preview and I wan’t the old version back).

(yevp) #19

Yev from Backblaze here -> Tommy, sorry to hear that. The inherit backup feature is definitely intended to make it easier to get back up and running should you get a new machine, but unfortunately at such a large data set we’ve seen that it can fail in some cases. I’ll make sure the engineering hears about this issue and we take a look at it. The rural broadband is definitely a bummer, we say that your broadband should be able to push your backup to us in 30-days or so for best results, so if the data set exceeds what the bandwidth can push, it definitely can put you in a jacked state. It sounds like you have a decent system in place already, I’ll just add that while it may take a while to get your 7TB backed up to Backblaze again, it is nice to have it offsite (not just on the local RAID) - but I definitely understand the reluctance to go through that again.

1 Like
(Tommy Weir) #20

Thank you Yev, for reaching out, much appreciated.

I’ll give it another go, and see how I get on. If your engineering team have a thought on what I might try, I’d appreciate it.

Cheers