M1 Mini as server with port channels to switch, and other iMac; but still really slow file/folder transfers?

TidBITS Talk
1

Hello All, especially to those with networking experience, particularly, with the Netgear GS108T … :nerd_face:

This past Friday, I installed a brand new Mac-mini M1 on my network, to act as a server. When selecting this computer, I opted for the 16 GB memory option, but retained the base 256 GB drive storage - all in order to keep costs down.

My iMac is a: Retina 5K, 27-inch, Late 2014, with 32 GB of memory, and a 3 TB Fusion drive.

Both computers are connected to the switch with 3 ethernet cable, in a port channel or LAG. After many hours of tinkering with it this past weekend, I discovered that the port channels connection on the switch require the LACP to be turned on, otherwise the entire LAG or network connection was not working.

Now that I’ve got this setup connected and working; I’m surprised and disappointed at how slow the connections are working to transfer large folders between these computers?

Setting up these port channels on the computers was easy enough, so I don’t expect my speed problem is with the computer configurations.

But, this is an instance wherein I admit to not being a computer network wizard, so am somewhat lost as to what else to do in order to improve performances of these connections?

The problem with the manual for this switch is that it doesn’t explain things, such as for example, why ‘flow control’ is best turned off in my situation, as with it on, only Apples’ 'Screen Sharing’ works for accessing these computer over my network?

… if anyone knows of a listserv that addresses computer networking topics for hobbyists, I’d love to learn of it!

Cheers!

Bill Taylor,
Ottawa, Canada

2

I’m not familiar with port channels or link aggregation groups at all—I’ve literally never heard the term before—so if anyone else is in my boat, check out this Wikipedia page.

My main question is if this question might best be aimed at Netgear support—could there be a hardware or configuration issue that only they would be qualified to address?

1 Like
3

Hi Adam,

Thanks for clarifying this, for the benefit of others.

After a phone call with Netgear tech support, it seems I’ve erred in my ways. While a trunk line, port channel or LAG [Link aggravation] provides a greater amount of bandwidth, it doesn’t mean that file and folder transfers between my iMac and Mac-mini will move any faster. I was assured the files/folders being transferred will only move at 1 Gb connection speeds.

I don’t know if this is a simple limitation of computer networking, or perhaps more particularly, a limitation of how Netgear implements the networking protocols, in their hardware?

I’m sure I’m not the only one to be frustrated by the imitation of Gigabit networking, and so it seems the only way to make it faster is to opt for 10 Gb networking hardware/connections.

I thought that by simply combining three gigabit connections on my adaptable switch would provide faster file transfers of about 3 Gb/each; when in fact, these port channels or LAGs, only provide redundancy, incase one line in the bundled connection goes down for some reason, there’s another connection in the LAG to take over the service of that connection.

In effect, I thought that by setting up my hardware with three ethernet connections, I could achieve a 3 Gb network connection, and thereby save some money by avoiding a mini with a 10 gb connection.

Thought I should mention this, incase anyone is in a similar state as I am, facing limitations of gigabit networking. Don’t make the mistake I’ve made, here with my setup.

Hope this helps,

Bill Taylor
Ottawa, Canada

4

Link aggregation is not solely for network connection resiliency but it depends on the type of link aggregation and what the end points and the switches between support. A server with multiple 1Gb Ethernet ports aggregated can act as one so while no one connection to a client can exceed 1Gb, multiple connections with multiple clients can use all the Ethernet ports simultaneously on a single IP address. At my last job, a file server running FreeBSD had two 10Gb Ethernet ports aggregated using LACP that were both active when Mac clients with 10Gb Ethernet were accessing it (the switch between was a big Cisco one).

You probably can’t expect the Mac sending data, files dragged from a mounted file share for example, to load balance the packets, sending each packet over a different port in a round-robin fashion. However, it might use more than one port simultaneously for separate network connections, say files dragged from a mounted file share plus files downloaded from a web server running on the same Mac.

Instead of link aggregation, if your iMac and Mac mini are close enough (~2m), you can directly connect them using Thunderbolt (Thunderbolt 2 cable with a TB3-to-TB2 adapter on the mini) and use IP over Thunderbolt; back the day, the same could be done with FireWire. You’d probably want to confirm you got it working using a shorter TB cable you already have before buying a 2m cable.

There are longer length Thunderbolt cables that are optical instead of copper but they’re way too expensive. Instead, use 2.5Gb or 5Gb Ethernet adapters, they’re cheaper than 10Gb. Since there’s only two computers, you can connect them directly and not buy a higher-end switch.

The 2014 iMac’s Fusion drive is going to be a bottleneck for higher speeds anyway; the SSD portion might be faster than 2.5Gb but not faster than 5Gb and the magnetic drive may not even saturate a 1Gb connection. Of course, you can also attach a faster SSD to the iMac.

1 Like
5

They should provide increased bandwidth, but not for a single connection.

Any link aggregation mechanism will have some algorithm for determining which packets go on which link. While the algorithms may differ among implementations, a critical design feature of them all is that a single TCP connection will always be carried over only a single link.

The reason for this is because splitting a single connection across multiple links will lead to the packets arriving out of sequence. This isn’t necessarily a problem, but the TCP protocol will detect this as network congestion and will deliberately slow itself down to try and compensate - making throughput worse than using a single link.

If, on the other hand, you have multiple simultaneous connections between the two computers, you may find each connection carried over a different link (e.g. using the port numbers as part of the algorithm to determine which link to use), which will multiply your overall throughput by 3x, even though no single connection is running faster than a single link.

6

Hello All, and especially to you, Curits!

I can’t thank you enough for your suggestions and explanations here, Curtis.

I am this afternoon, off to the local Apple store to buy a Thunderbolt cable and a corresponding adapter to bridge my two computers via their Thunderbolt 2 and 3 connections.

It’s not what I’d envisaged when setting my sights on this mac-mini to be used as a server, as I’d favoured an ethernet connection, but your suggestions here are my most cost-effective, before considering a new switch.

I’ve confirmed with Netgear support, that none of their switches implement port channeling in any way beyond simple redundancy. In time I expect to be replacing switch with one that better matches my needs

Still, I can’t believe what a 'lunch-bag-let-down’ this new mac-mini has been. Hopefully this alternate cabling option you’ve suggested will be my savour.

Be well, and stay healthy one and all!

Bill Taylor

7

I’ve used TB2 optical cables, but the cost is likely too prohibitive for what you’re after:
https://www.bhphotovideo.com/c/search?q=optical%20thunderbolt%20cable&sort=PRICE_HIGH_TO_LOW

…also, even the TB3 optical ones (Corning’s ones only recently released circa early-2021!) are going to be out of date already. As TB4 has arrived, which basically means that USB4/3/2 are additionally supported on the same cable type – unlike full speed TB3 cables >0.8m length, that only offer TB3+USB2 speeds, all TB4 cables are designed to support all speeds of USB.

FWIW, the only future proofed TB4 cable I’ve seen on the market that’s 2m length and does 8K@60Hz, is currently from CalDigit (~$75):

8

Cable Matters to the rescue (not that I dislike CalDigit). Certified TB4, 2m, 100W, $59 shipped.

https://www.amazon.com/Cable-Matters-107022-Computer-Cables/dp/B084Z65YJQ

9

Nice, more options = good! :slight_smile:

As a lesser factor for most buyers, I’m not sure if any of these actually do 8K/60Hz or just 30Hz though…AFAICT they fail to specify?
(typically omission of advertising the Hz, would guess that likely actually means 30Hz. But it’s still unclear.)

Also there’s another brand I’ve never heard of before called Maxonar at $43 [+10%-off coupon].
Dunno if actually Certified though (it doesn’t say), however they do actually specify 8K/30Hz:
https://www.amazon.com/dp/B08ZXH6RJ3

Like their 2m TB3 cables, the CalDigit and Cable Matters 2m ones look exactly the same, apart from their differing logo’s on one side of the connector, so I wonder if they’re actually all mostly manufactured by the same company (in China or wherever) anyway. Lol!
The cheaper Maxonar looks completely different (a bit uglier TBH), so maybe not that one…?

10

As other mentioned I don’t think Link Aggregation is anything you want for your setup, but Gig-E should give you very fast transfers.

For those unfamiliar, link aggregation is used when a device (computer, network attached storage, etc) has more than one ethernet port. The network switch can treat two separate ethernet connections to the device as one, doubling the bandwidth. Larger drobo’s had that functionality.

If you configured the switch to try to aggregate two ports connected to different devices…I have no idea what would happen and could easily cause slowness.

Using a TB cable for networking is a great solution, but you should also have been getting very good network speeds with gigabit eithernet connections. I can’t remember the M1 mac mini stats off the top of my head, but I’d bet it has Gig-E. A 2014 iMac…might? I know the 2017 does. If it’s only standard Ethernet (“Fast” or 100-base-T), that could easily explain file transfers feeling slow.

It’s easy to check your network connection speed. In Network Utility, choose the ethernet connection, and it will be listed right there. If you’re on Big Sur and the Network Utility is gone (as I just discovered!), you can use the terminal to find the same info by typing:

ifconfig en0 | grep media

Note: If en0 isn’t your ethernet connection, simply run ifconfig and find the media line for your ethernet connection.

Another note about Gig-E, you need cables that support it. Cat-5 cables kinda supported Gig-E, but I’ve seen all sorts of weirdness, Cat-5e was more reliable. Anything you’d buy should be at least Cat-6 (good), but if you just used a cable that was lying around or came with a cable-modem or router, I’d be suspicious. Problem is they all look the same, though some have their type printed on them (upvote for Cablematters as well), but if it’s not printed on it I would assume isn’t not good.

I’d turn off LAG and see what you’re speeds are even if you’re happy with a TB connection, assuming you’re still using the switch. I used a previous model of that switch for over a decade, very dependable.

2 Likes