Your assumption is correct indeed. However, that's base on you have plenty of RAM and the system never ever need to use page files.
If you believe that $100 is not wroth, and you are OK for painfully slow boot (may be only once every few months), and wait for 15s to open an app (once every few days, or even less in your case?), why not?
However, there is something that can't be speed up by having enough RAM and keep the app open. e.g. when you have to open / save a file (your photos, word doc, PDF files, etc). Those operation will be much slower. If you seldom mange / use your files on your Mac, and only use it for something like Safari. Yes, with enough RAM, you won't notice any difference when the app is already launched with the 5400RPM HDD.
If you have limited budget, and can only choose max RAM or fusion drive (or SSD). In your specific case, max out the RAM sure is a better choice.
However, if you only plan to buy the min RAM config, then most likely your HDD plan won't work. It's better to get the fusion drive, that's much faster in general HDD operation, also, when you are running out of RAM, there is a huge difference between using the high speed SSD or the 5400RPM HDD for page files.
With the high speed SSD fusion drive, most likely you won't notice the difference if you are running out of RAM. But with a HDD only system, the computer may respond in slow motion.