Apple has two webpages detailing the type of RAM needed for the 2017 27" iMac, but unfortunately they both have mistakes:
The support page mentions 204 pin SODIMMs (the correct is 260 pin):
https://support.apple.com/en-us/HT201191
The online manual has the correct 260 pin but refers to the SODIMMs as PC4-2400 (the correct designation is PC-19200):
https://help.apple.com/imac/mid-2017/#/apde7d36b188
So, in summary, the correct RAM type would be 260 pin DDR4-2400 SODIMM PC4-19200 memory.
There are 4 slots and you may use 4GB, 8GB and 16GB modules, for a maximum of 64 GB (4x16).
Regarding which brands and models specifically will work better with the 2017 27" iMac, nobody knows yet... Things like latency and other details could affect performance and stability. Past Macs have been known to be quite picky regarding 3rd party RAM, so I guess the safest bet is to wait for early adopters to get their hands on the computer and try out various RAM modules. That's what I'll do!