That depends on the design your receiver.
Receivers with an A/B speaker selector switch can be designed two ways: with the speaker switch internally wired in series or internally wired in parallel.
Usually you can tell which design your amplifier uses by the minimum allowable speaker impedance for your amplifier. An amplifier which supports a 4 ohm load will usually have the speaker selector switch wired in parallel. But if it only supports a minimum 8 ohm load, chances are the switch is wired in series. The reason for this is because most home speakers are rated at a nominal 8 ohms. Two 8 ohm speakers wired together (which would be the case if you're running two pairs of speakers at the same time) in parallel will present a 4 ohm load to the amplifier, while two 8 ohm speakers wired in series will present a 16 ohm load to the amplifier. So in the interests of protecting its customers' equipment, manufacturers will try to dummy-proof their design as best they can. You don't want to run a 4 ohm load on an amplifier designed for a minimum of 8 ohms, which will likely fry your output stage. But a 16 ohm load on a rated 8 ohm amplifier is okay.
To answer your question -- if your speaker switch is wired in series, it will likely affect the sound quality and will reduce the sound output from the speakers by half. But if it's wired in parallel, you shouldn't notice much of a difference, if at all.