Upgrade Your File Transfer Security with Encryption Keys

glennf · July 18, 2021, 3:54pm

Originally published at: Upgrade Your File Transfer Security with Encryption Keys - TidBITS

Anyone using FTP or SFTP to connect to their Web site or other servers should consider a tweaky but significant security upgrade that swaps passwords for encryption keys.

david19 · July 18, 2021, 7:12pm

The ssh-keygen program prompts you for a location to save the file: Enter file in which to save the key (/Users/gif/.ssh/id_rsa):. Press Return to save the file in a hidden .ssh directory inside your home directory. (You can specify a different path and name but that’s not necessary unless you’re creating multiple keys.)

I was wondering why you saved your private key to a directory that contains icons, and then realized /Users/gif is your home directory.

However, if all your Macs are under your control, it’s secure enough to use the same key pair across all of them. Simply copy the generated id_rsa and id_rsa.pub keys to each Mac

I don’t recommend copying the private keys from machine to machine. However, there’s no issue in sharing the id_rsa.pub file. In fact, you can combine the public keys, so all Macs can use sftp to connect to all your other Macs.

The biggest advantage of setting up sftp like this is for scripting. You can setup files that copy files back and forth between you machines without logging in. You could do that with ftp with a .netrc file, but it was trickier.

One disadvantage of sftp, because it’s based upon ssh, is that you can’t setup a separate file system. In ftp, you can setup a special ftp file system. When someone does log into you computer, they’re limited to just the ftp file system. With sftp, they have access to your siren as the signed on user. That makes me nervous.

glennf · July 18, 2021, 7:27pm

I’m not sure if I’m looking at the same thing. There are no icons shown and the directory I show in Transmit is on a Linux system—do you mean the preview in Transmit in the top screen captures? Those are mapped by Transmit to extensions.

For people with few specific security concerns—like a risk of being personally, directly attacked—it’s not really an issue if the key is copied only locally or using an end-to-end secured method. If someone has a level of risk great enough that they need to worry about the local method by which they copy a private key, that’s a whole other category than 99.99% of readers.

david19 · July 18, 2021, 8:21pm

Yeah. I just saw the directory name gif and immediately associated with various Unix directories named gifs where icons for various programs are sometimes stored.

I then saw it was under /Users/ and not /usr. I thought that was a strange place to put a directory of icons because that’s the directory where the Mac stores users’ home directory.

I finally realized your name is Glenn Fleishman and that GIF are your initials. Suddenly, it all made sense.

glennf · July 18, 2021, 9:02pm

And I pronounce it jiff!

mschmitt · July 18, 2021, 9:26pm

Once you set up the SSH keys you can change sshd so that it only accepts key-based logins, so there’s no more risk of an attacker breaking in by guessing the password.

But the catch is that macOS updates will reset the sshd_config file back to the insecure version. You have to check it after every update and restore your settings. macOS point upgrades always wipe it out, security updates sometimes do.

tjluoma · July 19, 2021, 4:05am

True. I’ve actually thought about doing this with my colocated Mac mini, but I’m wary about doing it to a computer that I don’t have physical access to, in case something goes kablooey.

I have a shell script that re-enables/reconfigures:

sudo via Touch ID
my ssh_config and sshd_config files
speeding up Time Machine

because those 3 get reset by seemingly every update that Apple puts out.

What was really funny was when I ran it on my Mac mini and then realized “Oh, right, I don’t have Touch ID on my Mac mini!”

(¿yet? I keep waiting for Apple to sell the new Magic Keyboard with Touch ID separately from the new M1 iMacs. )

I pronounce it Geoff!

paalb · July 19, 2021, 1:52pm

Another use of ssh-keys is if you want to edit files on your server. BBEdit uses SFTP to browse and open your files, edit and save. I got hooked on BBEdit many years ago when I was shown how you could change many files in one sweep using search/replace on your remote server.

Ratsima · July 19, 2021, 11:50pm

All of this begs the question of why MacOS ships with a version of curl that doesn’t support sftp:

MrMuscle:~ mnewman$ /usr/bin/curl -V
curl 7.64.1 (x86_64-apple-darwin20.0) libcurl/7.64.1 (SecureTransport) LibreSSL/2.8.3 zlib/1.2.11 nghttp2/1.41.0
Release-Date: 2019-03-27
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp

Same with the version installed with MacPorts:

MrMuscle:~ mnewman$ /opt/local/bin/curl -V
curl 7.77.0 (x86_64-apple-darwin20.5.0) libcurl/7.77.0 OpenSSL/1.1.1k zlib/1.2.11 zstd/1.5.0 libidn2/2.3.1 libpsl/0.21.1 (+libidn2/2.3.0)
Release-Date: 2021-05-26
Protocols: dict file ftp ftps gopher gophers http https imap imaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp

To get a version of curl that supports sftp you can install via brew:

MrMuscle:bin mnewman$ /usr/local/opt/curl/bin/curl -V
curl 7.77.0 (x86_64-apple-darwin20.4.0) libcurl/7.77.0 (SecureTransport) OpenSSL/1.1.1k zlib/1.2.11 brotli/1.0.9 zstd/1.5.0 libidn2/2.3.1 libssh2/1.9.0 nghttp2/1.44.0 librtmp/2.3 OpenLDAP/2.5.5
Release-Date: 2021-05-26
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp

The version shipped with Raspberry Pi OS supports sftp:

pi@raspcondo:~ $ curl -V
curl 7.64.0 (arm-unknown-linux-gnueabihf) libcurl/7.64.0 OpenSSL/1.1.1d zlib/1.2.11 libidn2/2.0.5 libpsl/0.20.2 (+libidn2/2.0.5) libssh2/1.8.0 nghttp2/1.36.0 librtmp/2.3
Release-Date: 2019-02-06
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp

tjluoma · July 20, 2021, 3:42am

Hello @Ratsima and welcome to the TidBITS forum!

Whenever Apple ships an outdated Un*x utility with macOS, the reason “why” is almost always because that utility is either covered by a license that is incompatible with macOS or which is suspected of being incompatible, and Apple seems to prefer to avoid any controversy.

I fully expect this is why Apple is moving to zsh for its default login shell (because zsh’s license is compatible), and why tools like perl and python will no longer be bundled by default in the future.

(This is a pipe-dream, of course, but I have this dream of Apple doing something like hiring all of the brew and/or MacPorts developers and have them develop something that would prompt users when they first launch Terminal.app that would say “Hey, would like us to download and install a bunch of useful utilities for you?” But I don’t imagine that will ever actually happen. A guy can dream, tho…)

lcho · July 20, 2021, 10:08am

When I worked as a network engineer, I used this method for all the servers I was responsible for. It made my life so much easier, being able to securely connect to any server without having to go through a login process. I have always wished that I could use the same method for all of my online accounts, but sadly, no one seems interested in supporting anything other than the login/password type of security. I can only surmise a lack of knowledge and understanding of the advantages, or a belief that it is too complex for the average user to implement.

My ideal scenario is that all my accounts have my public key, new account signups offer an option to enter a public key, and I can designate accounts that require a local authentication on my machine, to access. i.e. financial accounts. I would be interested to hear of any known drawbacks to this scenario.

jzw · July 20, 2021, 1:03pm

Where are you storing your private key though? If it’s on your local Mac (as it should be, security-wise), that means you won’t ever be able to sign in on another device. If you have to start copying private keys to your phone, iPad, or elsewhere, I think it quickly gets into risky territory. And it means if you lose your device(s) you are locked out of all your accounts. Maybe permanently?? I’m just not sure this would scale well in terms. It makes sense in certain specific situations (eg ssh) but I don’t think it generalises well to the standard security method for all services for all people.

lcho · July 20, 2021, 2:06pm

Well, if I could store my private key in something like 1Password, then it propagates to my other sync’ed devices. This would not be any different than storing login/password in a password app. Anyone else would still need to authenticate to access your private key. If that addresses your objections, what else do you object to?

ace · July 20, 2021, 2:44pm

Ooo, ooo! The Secure Enclave! A lot of iOS apps allow you to log in via Face ID and Touch ID now, so if that API could be extended in a secure way, it really would be the Holy Grail of security combined with ease of use.

theconsultant · July 20, 2021, 3:24pm

I use ~/.ssh/config to control some of this. In the top I have it set to:

# This allows for a set of config files with a bunch of machines listed.. and not polluting your main config file. ie. work.conf, servers.conf etc.
Include ~/.ssh/config.d/*.conf

Host *
	KeepAlive yes
	# ForwardAgent yes
	PubkeyAuthentication yes
	AddKeysToAgent yes
	Compression yes
    IdentitiesOnly=yes

I’m sure there was a good reason to turn off ssh agent forwarding… but I don’t recall it at the moment. That will allow you to login from your mac to machine one and then from there to machine two, without doing a new session from your Mac, assuming you have public keys on both.

Further down you can setup nicknames and settings per machine, like:

# The Raspberry pi
Host pi
	User pi
	Hostname pi.local

So I can then do ssh pi and that would be the same as doing ssh pi@pi.local.

The best part is not needing to remember complicated commands. You can add all sorts of settings in there, so I can do a reverse ssh tunnel with port forwarding … and just do ssh client123' instead of ssh -f -L 5900:localhost:5900 admin@123.456.789.012`. In this case the config would be:

Host client123
    Compression yes
    User admin
    Hostname 123.456.789.012 # or you can use the DNS address: server.client.com
    LocalForward 5900 localhost:5900

Thanks go out to various coworkers for these tips, as well as all sorts of websites

Ratsima · July 25, 2021, 10:43pm

As a result of this article I decided to follow up on my long-standing project of dumping my .netrc files and using encryption keys instead. I have five different machines (four Raspberry Pies and one Mac) that periodically upload files to a web host. I had already generated keys and updated the host’s authorized_keys files, so all I had to do was edit a bunch of shell scripts on three local and two remote machines. Tedious and time consuming, but not particularly hard.

And then, suddenly and without warning, it all stopped working. sftp via curl started returning curl error 67:

CURLE_LOGIN_DENIED (67)
The remote server denied curl to login (Added in 7.13.1)

Logging in to the host via ssh also failed:

Connection closed by 192.nnn.nnn.nnn port 22

The web host’s support staff is clueless and has no idea what is going on.

So, I spent a frantic hour changing all the code back to using passwords instead of encryption keys. Fortunately, I had simply commented out the code using passwords and had yet to delete any of the .netrc files.

So, now I’m conflicted. Theoretically, using encryption keys instead of passwords seems great. Until it fails, that is, when it becomes a huge PITA. At this point I’m not sure I’d go back even if I can get the web host to figure out what’s wrong.

charles1 · July 27, 2021, 5:17pm

Thanks for a great explanation, Glenn. Does anyone else find that cheap hosting plans often don’t support SFTP, though? At the moment I use one excellent US provider with great support for SSH and SFTP, but also two UK-based companies that don’t provide SSH with their budget plans – and indeed their plain FTP is often flaky. They do offer FTPS, though, so I’ve been experimenting with that today. Is FTPS a good thing? Less good than SFTP?

Shamino · July 27, 2021, 5:38pm

Two different solutions to the same problem.

FTP, as you know, is not encrypted. Someone snooping your packets can see it all, including your password. Which is why, these days, it is only considered appropriate for anonymous access.

FTPS is a quick-and dirty solution that runs the normal FTP protocol over SSH sockets. So the data is encrypted, but it doesn’t solve other issues people have had with FTP (e.g. use of a second socket for data transfer, which can create problems with some firewalls).

SFTP is a new file transfer protocol designed to work over SSH sockets. It comes bundled with nearly all SSH software packages. Many consider it better, but this is one of those arguments that really has no clear winner.

Here’s some additional information, if you want more detail:

https://www.msp360.com/resources/blog/sftp-vs-ftps/

FWIW, I use SFTP on the servers I set up at home. Mostly because it “just works” after you get SSH working. FTPS requires separate setup and configuration, which is just more work than I care about.

glennf · July 27, 2021, 5:38pm

Superb question! I think SSH and SFTP are likely a dwindling option for hosting plans, though I expect many TidBITS readers are using more technically oriented services. (I’m either an outlier or “normal”: I have a VPS at Linode and use Squarespace for my blog and an associated podcast.)

FTPS is entirely secure in the same way that SFTP is. That is, in both cases security information has to be installed at the server, and the connection is end-to-end encrypted with some kind of “out-of-band” validation that there’s no man-in-the-middle attack going on. Arguably, if the host is using a certificate issued by a certificate authority (or an intermediary), then it’s safer than SFTP in that you don’t have to assure your connection is uncorrupted.

It’s quite difficult to set up FTPS for an individual. For a hosting company, trivial.

That said, I recently added FTPS to my Linode server, where I already have an httpd (Apache) server running using Let’s Encrypt’s free SSL/TLS certificates. I was able to use vsftpd, a safe, open-source FTP server that’s widely available, disable all insecure FTP methods, lock logins to a handful of accounts, and reference the same TLS certificate that I use for the same site’s Web server. It took a little mucking about, but got it to work.

glennf · July 27, 2021, 5:39pm

Exactly! Some people prefer the old FTP over SSL/TLS because of certain FTP feature and an FTP approach, rather than SFTP, which is not really FTP but has similar functionality so that it can presented in client software (like Transmit) as nearly identical!