what equipment for a 10 gbit network?

[gregre]

Hi

I want to make a local 10 gbit network in the office. This will be used for video editing work.
Hardware list:
- 1x MacbookPro M1Max (now there is 2.5 gbe but I need to buy a card)
- 1x PC (now there is 2.5 gbe but I need to buy a card)
- Synology 923+ with 10gbe Rj45 card

I care about good performance and silence (that's why switch without additional fans).

I read that a network on SFP+ would be better than on RJ45 - just because of lower temperatures.

1. should I bet on networking via SFP+ or RJ45? I have to buy cards for MBP and PC and also Switch anyway - so I can choose.

2. which switch should I choose? Ideally it should be "plug it in, set it up and forget it" 😉 I have so with an internet bridge with Ubiquiti 60ghz.

3. what cards to choose for:
PC - here probably something on intel? X520?
MBP - here there is not much choice - although I wonder whether a card on TB3 (but there are reviews that they overheat) - or a network card on pcie and for that an external case)

Thanks for suggestions and hints on what works in practice.

 

[BeatCrazy]

How much are you looking to spend? 10GbE gets pricey quick. You might be looking at $300 just to get the MBP to 10GbE.

 

[gregre]

How much are you looking to spend? 10GbE gets pricey quick. You might be looking at $300 just to get the MBP to 10GbE.

As much as necessary, but as little as possible
I know that 10GbE for MBP is $300 - here it is possible that I will wait for MacStudio M3 with 10GbE, I do not know this yet
Switch is probably about $500 or more
PC card about $200-300?

These are my initial assumptions, but it is not rigid. I also do not want to buy a switch for 48 ports - because I will never have such needs here. I'm aiming for 4-8 ports. I don't know if I need to have a managed switch - I've never had one, but if it's needed in a network for 2 users + 1-2 NAS then I'm open too

Am I very much out of line in my cost assessment?

 

[ChrisA]

Hi

I want to make a local 10 gbit network in the office. This will be used for video editing work.
Hardware list:
- 1x MacbookPro M1Max (now there is 2.5 gbe but I need to buy a card)
- 1x PC (now there is 2.5 gbe but I need to buy a card)
- Synology 923+ with 10gbe Rj45 card

I care about good performance and silence (that's why switch without additional fans).

I read that a network on SFP+ would be better than on RJ45 - just because of lower temperatures.

1. should I bet on networking via SFP+ or RJ45? I have to buy cards for MBP and PC and also Switch anyway - so I can choose.

2. which switch should I choose? Ideally it should be "plug it in, set it up and forget it" 😉 I have so with an internet bridge with Ubiquiti 60ghz.

3. what cards to choose for:
PC - here probably something on intel? X520?
MBP - here there is not much choice - although I wonder whether a card on TB3 (but there are reviews that they overheat) - or a network card on pcie and for that an external case)

Thanks for suggestions and hints on what works in practice.

RG45 vs. SPF. You don't have many options, The 923+ is RJ45 You can buy a 10GBE card for the PC but the Mac does not have a way to install "cards" you would need some kind of Thunderbolt dongle. and I'd bet a lot it has RJ45.

Don't believe what bloggers say about "overheating". I doubt they looked up the maximum operating temperature specs for the parts inside the device. Maybe they just touched it and felt it was warm. Most stuff works just fine at 50C or even more. 50C would burn your finger.

All you really need is an 8-port 10GBE hub. Just add that. When it comes down to it, they are simplee devices and pretty much interchangeable. There is no way you could take advantage of a managed switch on such a small network.

Also don't forget about WiFi 6e. At very close ranges it is faster than 1 gig ethernet. Upgrading from WiFi6 is not worth it but if you still have 5, it might be.

Then on the Synology, add those SSD cache sticks. Synology comes with a utility that you run for a week or so and it measures how you use the NAS and recommends a minimum size cache. Adding more RAM helps a little bit also

 

[gregre]

I have seen a lot of unfavorable reviews (including about overheating, based on the fact that after 1-3 min the transfer drops and the link switches to 1gbe). - I have seen them here on the forum but also, for example, on Amazon or other stores. This is probably not written by bloggers 😉.
I realize that such problems can be due to various reasons that I do not know.

Card to MBP can be connected - there are more expensive and cheaper solutions, but it involves an eGPU case and network card.
That the corresponding kits work I found out.... From here 🙂

I use WiFi only to connect my phone, 5Ghz is enough.

Synology and cache does not work well in my application. I have terabytes of large files used at different times, and the cache works well when they are the same small files used frequently.
I tested, there was no benefit. I prefer to use nvme drives as storage (with an additional program from git hub supports drives from any manufacturer)

Thanks for your opinion, I appreciate it. You are right that it would be easiest to stay on Ethernet and rj45. However, I have it in my head that "spf+ works more efficiently and better" and I would like to verify that. Now I have the best moment because only in Synology I am forced to RJ45 😉 .

 

[svenmany]

If you’re trying to avoid RJ45, what transceivers were you planning on putting into the SPF+ sockets? Are you considering fiber?

 

[gregre]

What transceivers are probably the least of my problems (I think). I only planned two - one to connect the Synology and the other to the router.

 

[svenmany]

I recently considered buying a device that support 10Gbs, but only through SPF+ ports. I have a small LAN segment that I thought would benefit from the speed and it has CAT 8 cabling. Before I bought I learned that these particular SPF+ ports didn't support RJ45 transceivers. I have no experience with SPF and don't know how much of an issue it is to ensure ports of various devices on a segment have to support compatible transceivers.

I was puzzled by your phrasing the decision as a choice between RJ45 and SPF rather than a choice between RJ45 and fiber.

 

[etorix]

RJ45 is a dead end, and SFP+ is cheaper than you probably fear.

As switch you may go for a Mikrotik CRS-305-1G-4S+IN (or the CRS-309 sibling for a larger network) or something like a QNAP QSW-M408 or QSW-M2106 if you have many legacy RJ45 devices to integrate.
If using a pure SFP+ switch (which is cheaper than anything involving mixed RJ45/SFP+ ports), use a RJ45 transceiver for your single 10GBase-T device. (Switch to your local FS.com site.)
For short runs, use DAC cables. For longer runs (say, 3m or more), go for fibre (OM3 or OM4 patch cable, LC terminated) and transceivers which match the NIC or switch on appropriate ends.

For the PC, any supported NIC. Refurbished Solarflare or Chelsio T520 cards go for $50 on eBay.

Connecting the MBP is where you'll spend more, unfortunately, for a Thunderbolt dongle (no experience with those) or a Thunderbolt enclosure (with whatever card comes in there; if you build it yourself, Ventura and later now gives you the choice between Intel X520/540/710 and Mellanox ConnectX-4 cards and later).

 

[Kgeee]

I'm using the Sonnet Solo 10G SFP+ with my M1 MacBook Pro, connected to a UniFi USW Aggregation Switch. On the other end, I've got a Mac Mini connected to a Sonnet Twin 10G, which I was lucky to find used at a good price.

The Sonnet Solo is running fine; it never gets more than lukewarm. As Etorix mentioned, I'd recommend an SFP+ setup and using a transceiver for the single RJ45. Remember to check if the transceiver is compatible with the device you're plugging it into. I've had good experiences with transceivers from 10Gtek, which are also reasonably priced.

 

[Ifti]

I have a QNAP 10GBe NAS and use the QNAP switch along with it : https://www.amazon.com/QNAP-QSW-M1208-8C-Managed-Switch-Gigabit/dp/B08JTZ79KT

With my MacBook I use a Thunderbolt to 10GBe adapter : https://www.amazon.com/Sabrent-Thunderbolt-Ethernet-Adapter-TH-S3EA/dp/B08J59QKGT

Always worked perfectly well for me…..

 

[svenmany]

Remember to check if the transceiver is compatible with the device you're plugging it into.

What I stumbled over was the Netgate pfSense appliances which do not support any RJ45 transceivers.

 

[gregre]

Thank you for such detailed hints!

I think I will choose
Mikrotik CRS-305-1G-4S+IN
For MBP a Thunderbolt enclosure and Mellanox ConnextX-4 RJ45 MCX4121A-ACAT card
For the PC the Mellanox ConnextX-4 RJ45 MCX4121A-ACAT card

I am guided to have the ability to upgrade to 25G (if there is any affordable switch)

The devices are at a distance of 10m from each other - so at your suggestion I will choose Fibre OM4 LC terminated cable.

Here's a question because I'm new to such connections - I need for a set of
- 4x SFP to LC transceiver (2x on the switch side and one for each Mellanox card) link
- 1x SPF to RJ45 transceiver for connecting synology 923+
Is that correct? Link

Is there anything more needed?

 

[etorix]

Correct, but mind that the Mellanox cards might be vendor-locked so you may need at least two of the modules to be Mellanox-coded.
As for an affordable 25 GbE switch, if anyone knows of an even better option than the Mikrotik CRS510-8XS-2XQ-IN, I'm all ears.

 

[etorix]

I am guided to have the ability to upgrade to 25G (if there is any affordable switch)

Reality check: A handful of hard drives in a NAS with a puny CPU are NOT going to saturate a 10 GbE link, much less 25 GbE.

To go 25 GbE, you'd need a 25 GbE switch and 25 GbE transceivers; fibre remains the same OM4—which is good if you have to pass it through the wall: This part is a one-off. (OM4 is even good up to 100 Gb/, but then LC modules get very expensive; above 25 Gb/s, MTP/MPO cables and modules are a better choice.)
SFP28 cages are supposed to be backwards compatible with SFP+, so older modules may still be used if a part of the network is still on 10 G.

 

[gregre]

Reality check: A handful of hard drives in a NAS with a puny CPU are NOT going to saturate a 10 GbE link, much less 25 GbE.

I've already been looking around for a DS1621+ 😄 although I'm saving that topic for a little later. For now I will be able to use for PC<>MBP connection.

To go 25 GbE, you'd need a 25 GbE switch and 25 GbE transceivers; fibre remains the same OM4—which is good if you have to pass it through the wall: This part is a one-off. (OM4 is even good up to 100 Gb/, but then LC modules get very expensive; above 25 Gb/s, MTP/MPO cables and modules are a better choice.)

Here you indicated 25 Gbe transceivers for Mellanox (and rightly so, such cards I bought).

I also need to buy 2x transceivers on the other side of the cable, for Mikrotik switch:
- that is, for this example, I need to select "Customized" and choose Mikrotik and specify the switch model - correct?
- for the Synology 10G RJ45 connection
- for the connection to the router with 1G internet

2x 25 Gbe transceivers for Mellanox – 96 euro
2x 25 Gbe transceivers for Mikrotik - 96 euro
1x 10G RJ45 - 85 euro
1x 1G RJ 45 - 35 euro
Total: 312 euro. 1/3 the price of a Mikrotik switch :/ I just don't know whether to be happy with cheap Mikrotik or expensive transceivers

OM4 cable - here I also see many different types: LC to LC Duplex, LC to SC UPC Duplex, SC to SC UPC Duplex.
Is this the right one for my application and equipment (I suspect so)?

 

[etorix]

I also need to buy 2x transceivers on the other side of the cable, for Mikrotik switch:
- that is, for this example, I need to select "Customized" and choose Mikrotik and specify the switch model - correct?

Vendor-locking normally stops at the vendor, not any specific model. And there may be no lock or coding at all.
Go for "Mikrotik" if you find it, otherwise "Generic"… or probably just about any coded-module. (NB I haven't tested that.)

Total: 312 euro. 1/3 the price of a Mikrotik switch :/ I just don't know whether to be happy with cheap Mikrotik or expensive transceivers

You'll probably be just as happy with a 10 Gb/s network. Unless you go for a full-NVMe NAS, with enough U.2 drives and an EPYC CPU to manage them.

OM4 cable - here I also see many different types: LC to LC Duplex, LC to SC UPC Duplex, SC to SC UPC Duplex.
Is this the right one for my application and equipment (I suspect so)?

Correct. Or the armoured variant, depending how exposed the cable may be.
There are many options, but the principle is simple: Match everything with the modules.

Single Mode / Multimode (MMF)
wavelength (850 nm for short-range)
connector (LC Duplex for SFP+/SFP28 modules, SC is just too big to fit)
polish (UPC is most common)

ServeTheHome.com has short articles on each and every of these aspects, as does the blog section on FS.com, if you want to dive deeper.

 

[gregre]

I read here about the fact that Mikrotik CRS-309 + transceivers to RJ45 heats up (overheats) a lot. A person combines with additional fans to make it work without interference.

At my place there will be two such transceivers and it scares me a little.

Is there any better solution for mixed SPF+ and RJ45 connections?

Maybe it would be better to have such a combined solution by the manufacturer, for example. Qnap QSW-M3212R-8S4T? Or QSW-M408-4C?

 

[etorix]

10 Gb/s is pushing copper wiring to the limits of its possibilities. To ensure signal integrity, a higher tension is used, which means more power to the controller, which mean more heat—more than for 1 Gb/s, and much more than optical, especially for long runs.
So, yes, RJ45 SFP+ transceivers get hot. RJ45 transceivers should not be placed next to each other—and no more than two in a CRS305 (it's somewhere in the manual…).

For a single device with RJ45, one RJ45 module in a CRS305/309 is effective (you may use the RJ45 management port as uplink to your existing 1 GbE switch/router). With two or more RJ45 devices, a switch with combo RJ45/SFP+ makes more sense.
I currently have a QNAP QSW-M408-2C on my desk. Only my home-built NAS uses 10 GbE on RJ45 (onboard); all 10 GbE clients are on SFP+. The previous configuration had a CRS305 staked with an old 1 GbE switch for all legacy devices (2012 Mac mini, printer, satellite box, etc.). But if I add more SFP+ devices, I may go back to two stacked switches again: A CRS305/309 for the 10G network, and the M408 for legacy 1G and the lone NAS on 10GBase-T.
end up stacking a

 

[gregre]

I ordered a CRS309 today - tadam.wav 😄🤦‍

As you write - I can use the 1G port not only for configuration but to connect another 1GB switch (just for printer, second NAS) and also further to the router with internet?
I don't plan to manage the settings on the CRS309 in any special way.
Then I would only have 1 transceiver to 10GB RJ45.

 

[etorix]

I knew that for the CRS305, but checked for the CRS309:

The management port is also on the switch chip rather than the CPU, so using it for data flow should work fine, unofficially. Don't expose the switch to the Internet.

 

[gigapocket1]

You could just direct connect the synology to your comp. No switch required.