Dear Apple GPU designers... Please make training my neural network cost less and go faster.

[MareLuce]

Dear Apple GPU designers...

Please make training my neural network cost less $ and go faster.

For my Convolutional Neural Network (CNN) model with 5 convolutional blocks and 3 dense layers,
training the model once (60 epochs) takes:

45 minutes - via Google Colab connected to a NVidia A100 GPU
2 days - via Google Colab connected to a Google Cloud 'free' CPU.

I've spent $110 in Google Colab Plus fees so far. My monthly maximum was used up in what seemed like a day.

Current MBP: a 64 GB, 8 TB, 2021 M1 Max. aka "Maxine"

Just now, saw 58 of 64 GB in use via Activity Monitor.
This is is the most most I've noticed ever using. I only checked because overall, Maxine 'felt' slower.

Even though I am training a model with a ton of parameters, am using Google Colab with Tensorflow, so I wouldn't expect so much local memory to be used. (right?)

 

[drewsof07]

I know absolutely nothing about NN's, but I do have some experience with databases. It is common for databases to load and run as many tables in memory as possible, up to 100%

 

[tahoeroscoe]

Looks to me like you should direct your complaints about memory management at google and snagit

 

[TGM85]

Maybe training a neural network is not a task to which a laptop is best suited?

Sounds like something that’s best done on a powerful multiple dedicated GPU equipped workstation?

 

[MareLuce]

@tahoeroscoe Agreed.
@TGM85 Of course, agree, but
- Maxine maxed out my budget.
- I'm at 94% accuracy on new test data on 'just' $110 of GPU time training and validating the model
( $110 is a lot to me for just compute. But big picture I realize it's nothing compared to what the big companies spend training and validating ML models)

About 10 minutes after I posted, what I never thought would happen did occur: