Seeing as how I’ve helped on here before with OpenSMTPD setup, and my local mail setup was broken for me recently by a macOS update that I thought was down to an update to OpenSMTPD, this topic seems fitting. I hope it saves some poor soul the few days I lost to this before I figured it out.
Some recent update to macOS, for at least Ventura and Sonoma and probably Sequoia as well, broke gethostname(3), the UNIX routine that’s supposed to return your system hostname. In the past, macOS included a fully-qualified name in the response, which is atypical for contemporary UNIX systems, but it generally meant that server software trying to figure out your fully-qualified domain name (FQDN) could just ask for the system hostname and get something that looked fully-qualified even if it simply resolved to a Bonjour local hostname such as “computername.local”. Well, Apple changed it, so that now, gethostname(3) is only returning “computername”. The problem is, software that expects this on typical UNIX systems have a strategy for working out your “real” fully-qualified name, that is unlikely to work on macOS. Specifically, they will try to look up the addresses for the hostname, and then do a reverse lookup on the addresses that result, in an attempt to find at least one name that looks fully-qualified (has a dot in it). And because the return address on many typical home networks is a private address that is unlikely to have a real, fully-qualified name, but just return the single hostname synthesised by the system itself, the server software panics and dies because it insists on a valid FQDN.
So what’s the solution?
For OpenSMTPD, there’s a very specific workaround: put the hostname you want to be treated as the canonical hostname, that would otherwise be discovered, into a file called mailname in the configuration directory, so probably /opt/local/etc/opensmtpd, if you install as I do from MacPorts. So in Terminal:
echo 'server.local|sudo tee /opt/local/etc/opensmtpd/mailname
A more general solution, that relies on server software behaving more-or-less as it did in the past, is to just set the system’s hostname manually, overriding the synthesised name. So in Terminal:
sudo scutil --set HostName my-server.local
Of course, if your system actually has a real hostname, then you could also (and should, obviously) arrange it so your system has the forward and reverse DNS for that hostname correctly configured in the DNS server; then the system will figure out its correct name. (And if it still doesn’t, for some reason, you just override it as above, so that it has a correct view of its hostname.)
Notice that the system hostname cannot be set anywhere in the UI: the human-readable “Computer Name” that appears in the network browser (set in the “About” Settings page) or the Bonjour “local hostname” that is used for local Bonjour resolution and ends in “.local” (set in the Sharing Settings page) are not the system hostname, or “node name”, which is just a short name associated with the processor. The system hostname is synthesised based on the local hostname if it cannot otherwise be determined from the DNS itself. In this respect I’d argue Apple’s change actually now constitutes something closer to “correct” behaviour, and is rather more like other operating systems, except of course for all the cases (like mine, on my home network) where it doesn’t work because reverse DNS resolution isn’t working properly or at all (and yes, I really ought to do something about that). I think Apple could do worse than to give sysadmins the option of reversing the behaviour, so that forward and reverse lookups result in fully-qualified names when the DNS does not support resolution. As it stands, you still can’t change the synthesised reverse lookup—that’s still a short name—but at least you can rely on server software that typically bites when gethostname(3) returns something “FQDN-looking”.
I hope it helped …