Utilizing Thunderbolt bridging for 20 Gb/s NAS

[treestar]

I'd like to stimulate a discussion on connecting multiple workstations to a high-speed Thunderbolt storage device. While an external Thunderbolt storage device might only allow for a single host, the combination of daisy-chaining and Thunderbolt networking might allow for the device to be shared amongst many users at up to 20 Gb/s.

In a simple setup conceive of two Thunderbolt 2 capable users connected via a Thunderbolt bridge, with a Thunderbolt 2 storage device being hosted by one of them. Should the other user be able to access the storage device through the host as a shared network volume? At what speed could it connect? How many more users could join the Thunderbolt bridge?

I haven't experimented with a Thunderbolt bridge yet, and I only have two workstations I can try with, but I suspect the amount of workstations you can connect is not limited to the number of Thunderbolt devices one controller can daisy-chain (up to 6). If more than two workstations can bridge over Thunderbolt, and if each workstation acts as a host, the chain or web could connect many.

Let's hear what people have tried and thought up. Can you make two Thunderbolt bridges from one host? Does it depend on how many controllers the workstation has? Maybe someone out there with multiple Thunderbolt ports can open Network Utility and tell us how many Thunderbolt network interfaces are available. What are the options for setting up a multi-user Thunderbolt network, such as a daisy-chain with a master host, a star configuration from a master host, or multiple hosts bridged together linking the networks outside of the protocol? Can a single Thunderbolt port be used for a bridge and to connect to a Thunderbolt peripheral simultaneously?

Building a Thunderbolt network could be very interesting and very fast. I at least want to investigate it so I can share a Thunderbolt storage device amongst all my workstations. I could do it over ethernet but that's a huge sacrifice of speed.

Thunderbolt has been great to me so far. I think it can do even more. I hope it survives against USB 3.1. I'd be impressed to see it used for high-speed networking.

 

[chrisn123]

I would be interested in exhaustive testing myself. I've connected my '13 rMPB up to my '13 nMP via TB2 and gotten about 600MB/s--- probably limited by my rMBP's SSD.

I have NOT tried "daisy chaining" off my TB storage box. I guess it would work, but why share the bandwidth when the array already saturates the link (and the nMP has two other TB buses?).

I have found the TB link to be kind of clunky/unstable. If I don't disable WiFi on the laptop, it seems to connect over that link (slow), but then using just TB, it doesn't always get a DHCP address from the nMP. It still finds the shared volumes and xfers fast, but using a self-assigned IP address.

For a rock solid production environment, I would stick with GigE at the moment. You could use a TB-GigE dongle (for laptops) to keep your storage network on a sep sub-net, and the main GigE for all other traffic. 100MB/sec is still pretty fast.....

TB "networking" feels like "SCSI voodoo" (at least to me). I can get it to work, but it requires fiddling.

 

[treestar]

Yes, the new Mac Pro has the great advantage of several TB ports, and I think it would be an awesome center hub for a TB star network. Unfortunately I have first gen TB devices only, so I only have one port per to work with. I have a TB external disk array coming next week so I'll do a test and report what I find. Thanks for the tip about disabling the other network interfaces.

Regarding your TB storage saturating the link, unless it's a new device with TB2, you still have a free 10 Gbps bidirectional channel to use! I don't know what you have but TB2 devices are only just trickling out.

The SCSI analogy is terrifyingly humorous. Hopefully it isn't that bad. It's been about two decades since I've had to fiddle with SCSI devices. If it turns out to be similar, I'm going to pack my bags and bail out on TB bridging. I really hope it works though. If there's an available networking option that is twenty times faster than GigE, I'd like it to catch on. Although, a TB network only makes sense for a small network in a tight space. I saw the prices on 30 meter TB cables and it's just not happening. This doesn't even begin to mention a lack of TB network devices, like switches and hubs, etc. But for me, with a few TB equipped workstations in a one room studio, trying to share one high-speed SSD RAID, and keep costs low, this might be a smart solution.

 

[chrisn123]

More tips for the sake of science:

By default, Mavericks bridges all TB ports together as a virtual interface. This is done at the ethernet layer. You can add other ports to this bridge (e.g., AirPort) and they will appear as one interface to the OS (hint: helpful because then your TB network is connected (via Airport) to a network that will hand out DHCP info). I'm not a network engineer, but this is like a "hub" or datalink style bridging.

You can also use Internet Sharing to do software routing at the IP level. This *seems* to be more reliable in establishing a link, but ultimately slower. This puts your TB devices on a separate subnet, sharing the IP address of your "hub" machine.

Again, I have NOT done careful testing-- just fiddled around until I got it to work (I wanted to speed up my initial time machine backup (to TB2 array hooked to nMP) ASAP).

Please report your findings.