torrent creation and distribution

[DennisBlah]

Hi all,

I need some help with some brainfarts of mine.
I have 2 machines that are downloading iOS updates every 5 minutes.(if available)

I need to distribute this to ~200 machines asap when there is an update.

Current situation:
Operators walk to one of the 2 machines, cooy the files to usb
and walks around thr 200 machines and copy it manually.

How can I speed this up, and automize it?

I do have a ‘autoUpdater’ that runs each login on a machine. So tried to use that to transfer the files over the lan. However the LAN of these 2 machines can’t handle it and are really slow (d’uh)

So I was thinking, how about a local torrent file?
I can create a torrent file when the updates are downloaded and some how have the 200 stations fetch the torrent and start downloading together from eachother as well.

This would be a great situation?
But as the LAN part goes, I should limit connections for 2 downloaders simutaniously from the 2 machines and from eachother. (i guess)
And I need to automize the creation and distribution. I could have the 200 stations to boot every night (This region receives updates mainly in nighttime) and fetch a new torrent file (if there is any)

I did not look into torrent creation etc yet.
Does anyone here have any experience in this and knows what I’m talking about and strugling with?

Any advice would be appriciated!

 

[Mikael H]

Hi all,

I need some help with some brainfarts of mine.
I have 2 machines that are downloading iOS updates every 5 minutes.(if available)

I need to distribute this to ~200 machines asap when there is an update.

Current situation:
Operators walk to one of the 2 machines, cooy the files to usb
and walks around thr 200 machines and copy it manually.

How can I speed this up, and automize it?

I do have a ‘autoUpdater’ that runs each login on a machine. So tried to use that to transfer the files over the lan. However the LAN of these 2 machines can’t handle it and are really slow (d’uh)

So I was thinking, how about a local torrent file?
I can create a torrent file when the updates are downloaded and some how have the 200 stations fetch the torrent and start downloading together from eachother as well.

This would be a great situation?
But as the LAN part goes, I should limit connections for 2 downloaders simutaniously from the 2 machines and from eachother. (i guess)
And I need to automize the creation and distribution. I could have the 200 stations to boot every night (This region receives updates mainly in nighttime) and fetch a new torrent file (if there is any)

I did not look into torrent creation etc yet.
Does anyone here have any experience in this and knows what I’m talking about and strugling with?

Any advice would be appriciated!

1) What's the point in attempting to ship OS updates every 5 minutes?
2) If you don't actually need updates every 5 minutes, may it be that you'd be fine with the existing functionality in macOS Content Caching? (https://support.apple.com/guide/mac-help/about-content-caching-mchleaf1e61d)

 

[DennisBlah]

Hi Mikael,

Thanks for your reply! However, I think you missed the point.
1) It's not about OS updates, it's about ipsw files. These can be released any moment and any time, so I do a check every 5 minutes for new packages and download where they are available or not.
2) We have used the Caching server, but this is useless as the content must be downloaded first, then it will be cached.

let me explain a bit more about the exact situation.
- we are restoring thousands of iDevices, and they always have to be the latest version
- manually downloading the packages, and putting them on the required path (i.e. for iTunes or Configurator) is the fastest way at the moment caching is too late.

To get back on caching, it's almost same as: "I do have a ‘autoUpdater’ that runs each login on a machine. So tried to use that to transfer the files over the lan. However the LAN of these 2 machines can’t handle it and are really slow (d’uh)"

I think distributing all these gb's over the lan would be best using a peer2peer.
The more peers already downloaded files, the faster it can be distributed.

 

[Mikael H]

Hi Mikael,

Thanks for your reply! However, I think you missed the point.
1) It's not about OS updates, it's about ipsw files. These can be released any moment and any time, so I do a check every 5 minutes for new packages and download where they are available or not.
2) We have used the Caching server, but this is useless as the content must be downloaded first, then it will be cached.

let me explain a bit more about the exact situation.
- we are restoring thousands of iDevices, and they always have to be the latest version
- manually downloading the packages, and putting them on the required path (i.e. for iTunes or Configurator) is the fastest way at the moment caching is too late.

To get back on caching, it's almost same as: "I do have a ‘autoUpdater’ that runs each login on a machine. So tried to use that to transfer the files over the lan. However the LAN of these 2 machines can’t handle it and are really slow (d’uh)"

I think distributing all these gb's over the lan would be best using a peer2peer.
The more peers already downloaded files, the faster it can be distributed.

OK, then I understand your issue a bit better, and then I have nothing of use to add; sorry.

 

[belvdr]

There are plenty of instructions on creating a torrent file, but I'm not sure how you go about distributing it. Keep in mind that you could very well take your network to its knees, depending on how this is deployed.

I think a mobile device management system would be better so you can keep one central repository and have the devices download from there, assuming everyone has a fast connection to the repository. This is very similar to how Linux update repositories are done.

 

[DennisBlah]

OK, then I understand your issue a bit better, and then I have nothing of use to add; sorry.

No worries!
I hope someone around this forum has any experience with this.

There are plenty of instructions on creating a torrent file, but I'm not sure how you go about distributing it. Keep in mind that you could very well take your network to its knees, depending on how this is deployed.

I think a mobile device management system would be better so you can keep one central repository and have the devices download from there, assuming everyone has a fast connection to the repository. This is very similar to how Linux update repositories are done.

Thank you for your reply Belvdr!

We already took the network on its knees (well at least for that server) by enabling an folder sync between the server and client. multiplied by 200
This took ages ofcourse..

I was hoping we can limit the amount of downloading clients from peers.
i.e. allow 2 simultanious downloads from each.
Everytime one peer downloaded one of the files, other peers can connect there as well.

In the beginning it’ll be slow and pending up to the moment that like.. 25% already finished then it shud only speed up I guess?

Switch ports are limited to 100mbit of a 10gbit.

The central repository will not work as the machines that install the devices are installing them from a local ‘cached’ folder.

Did I note that these 200 machines install 10-20 devices simultaneously?
Which can differ from model and thus required package