Network Service Order... using Thunderbolt Bridge, Ethernet and Wi-Fi

Discussion in 'OS X Yosemite (10.10)' started by bxs, Jan 4, 2015.

  1. bxs macrumors 6502

    Joined:
    Oct 20, 2007
    #1
    I've raised this subject before and now I raise it again as I'm needing some clarity and better understanding on how the Service order is treated by Mac OS X.

    Yesterday I actually started my testing for this Network Service ordering configuration using two MP6,1s; named Titan and Earth. Titan is a 12-core and Earth is a 6-core model. Both have 64GB RAM and 1TB internal SSD and Dual D700s. Both are running Yosemite 10.10.1.

    Titan has all the office storage and project files and is considered the File Server for the office as well as being able to do useful Project work alongside this service. Earth is basically a Client system with but 4TB of local USB3 storage used for its Time Machine backups, a clone of its internal SSD boot volume and some 2TB of scratch space.

    Titan and Earth are connected with a direct Thunderbolt cable, 1GbE bonded ethernet thru a Cisco managed switch (SG200-18) and WiFi via Airport Extreme.

    On both Titan and Earth the Network Service order... is set to

    1) Thunderbolt Bridge
    2) Ethernet (bonded using both ethernet ports)
    3) Wi-Fi

    File Sharing is enabled on both Titan and Earth for obvious reasons. Both SMB and AFP options are checked. One or both (SMB and/or AFP) must be checked otherwise File Sharing gets disabled.

    On the face of it, one would assume that when Earth wants to grab some data from Titan, then Earth has to mount the appropriate Titan file system using say a Finder's Shared device. Once done, Earth can start using data being transferred from Titan using the Thunderbolt Bridge as it's first in the Network Service order. However, I'm finding this is hit-and-miss as sometimes the Ethernet will be used. Thunderbolt Bridge can transfer data as high as an average of 550 MBytes/sec and will at times will be well over 1 GByte/sec if the data size is small and is already in Titan's kernel File Cache. Ethernet of course is limited to a max of 125 MBytes/sec.

    I'm puzzled as to why this is hit-and-miss situation.

    So far, to get around this hit-and-miss situation I've configured Titan's and Earth's Network services as follows...

    On Titan
    1) Enabled Thunderbolt Bridge with self-assigned IP
    2) Enabled the Bonded ethernet
    3) Wi-Fi is OFF

    On Earth
    1) Enable Thunderbolt Bridge with self-assigned IP
    2) Made the Bonded ethernet Service inactive
    3) Wi-Fi is ON

    This allows Titan to access the Internet using Ethernet; allows Earth to access the Internet via Wi-Fi, and forces the Earth system to use Thunderbolt Bridge to obtain data from Titan.

    This configuration does work and I tested it using my home iMac13,2 and MBP8,3. The iMac acted as the Earth system and the MBP acted as the Titan system.

    Here's a quick test using the dd command in Terminal.app. No attempt was made to avoid using the kernel File Cache as I wanted to see what the Thunderbolt wire speed could support.

    Duncans-Saturn:~ bxs$ dd bs=1000000 count=10000 if=/dev/zero of=/Volumes/bxs/Desktop/AAA_test_file
    10000+0 records in
    10000+0 records out
    10000000000 bytes transferred in 37.103940 secs (269513157 bytes/sec)
    Duncans-Saturn:~ bxs$ dd bs=1000000 count=10000 if=/Volumes/bxs/Desktop/AAA_test_file of=/dev/null
    10000+0 records in
    10000+0 records out
    10000000000 bytes transferred in 21.908456 secs (456444761 bytes/sec)

    The write test gave some 270 MBytes/sec and the read test gave some 456 MBytes/sec. This is for Thunderbolt 1 as the iMac and MBP only support Thunderbolt 1. The Titan and Earth systems support Thunderbolt 2 so the data transfer rates will/should be higher.

    So with all the above, I'm wondering if Apple has some sort of bug managing the Network Service order when all three (Thunderbolt Bridge, Ethernet and Wi-Fi) services are active. That is, why would the system use Thunderbolt one time and ethernet another time ? In order to get the result I wanted I've had to cunningly configure the Network services on Titan and Earth, and I think from a user's standpoint I should not have to do this.

    I setup a test on my iMac to write and read a 10GB file to and from my MBP8,3. I created a shell 'while true' loop that had a '/bin/rm test_file ; sleep 3' after each write/read sequence in the loop.

    Here's my recording of the network traffic on the Thunderbolt Bridge service.

    Note the peak rate over 1655 MBytes/sec for either the write or read operation in top left corner of the graph. The two numbers below the 1655 are the amounts that have been written and read by the test. The Blue graph is for writes and it's max is at around 1050 MBytes/sec. The orange graph is for reads and its max is at around 900 MBytes/sec. The actual rates as reported by the dd writes and dd reads are considerably less than these max values as I believe there's some startup time before the writes and reads actual get going as well as periodic 'i/o stalling'.

    Comments are most welcomed. Thanks.... :)
     

    Attached Files:

  2. bxs, Jan 4, 2015
    Last edited: Jan 4, 2015

    bxs thread starter macrumors 6502

    Joined:
    Oct 20, 2007
    #2
    Issue resolved

    I had a lengthy chat with Apple Tech senior person and my issue is that having Ethernet AND Wi-Fi enabled and connected to the same office LAN, then the system will default to using Wi-Fi regardless of the Network Service order.

    The solution is therefore to turn Wi-Fi OFF and leave Thunderbolt Bridge and Ethernet active on all Macs.

    I tested this on my home iMac13,2 and MPP8,3 rig and it works as Apple said it should. The Thunderbolt Bridge is now always being used. Thanks Apple. :)

    My home rig is now showing a steady 300 MBytes/sec writing and a steady 440 MBytes/sec reading across the Thunderbolt 1 Bridge consistently with Wi-Fi OFF on both Macs.

    [EDIT]
    I should also have said that for this to work I found that before mounting the Shared device on Earth I had to make Ethernet inactive on Earth. Subsequently I could then make Ethernet active again and The Thunderbolt Bridge would be consistently used. This to me is still an issue that Apple needs to be aware of. So I called Apple Tech back and have been waiting to chat with them about this aspect. They have already said that is not right and will get back to me to hopefully get things resolved.
     

Share This Page