Yes, it should work, but Apple doesn't give instructions on how to do it.
To pair it, make sure both ATVs (the one with the new Remote and the bedroom ATV) are unplugged. Turn on the new ATV by plugging it back in, when the ATV gets to the home screen, hold the old Remote a few inches from the new ATV box then press and hold both the menu (back) button and volume up button. Hold them both for about ten seconds and it should pair.
Repeat the same for the bedroom ATV, but use the new Remote.
There might be some easier way, but that is how I do it.