For RAM access speed, since there are four memory channels to take advantage of, its best to fill four slots (the first two slots of each riser cards) with RAM, even if there is a mismatch. The third and 4th slots are buffered by the RAM in slots one and two. So, in theory, access to those sticks is slower than the first slots. IAW, spreading 8GB of RAM over the 8 available slots is slower than spreading it over 4 slots.
More RAM usually provides better real world computer performance than higher speed RAM.