This is a great idea . have a jack that act like a secondary jack adaptor so the original one don't get damaged
Don't worry about changing headphones, life is too short. The wear shouldn't be too significant. They pleasure of having the right headphone for the right task is immense.
Mooch