The reason you were tood not to do this is because not using pairs of identical RAM means giving up on dual channel memory mode, which is a whole bit faster than single channel mode. Your Mac will work fine, but it's going to be a bit slower.
Mac Pro's have triple channel, meaning three banks of memory are expected for best results.