Your starting point of course needs to be to back up that Fusion drive; if there is anything on there that is important to you, it needs to be copied someplace else... especially if you're going to follow my advice. And don't trust exclusively to Time Machine for the task; personally, my most critical files are copied periodically to a non-Time Machine backup drive which is stored offline, (as in, not connected to the computer) just in case. Also, if you can afford it, I highly recommend maintaining your backup on a new SSD drive, rather than one of your existing spinning rust drives.
Next... well, my gut tells me that more peripherals is likely to lead to an even slower computer; I realize that a 1TB drive can seem cramped after several years of data has piled up... but have you considered simply disconnecting wires from the machine, instead of adding more? Additionally, never underestimate the impact that software can have on performance, especially after multiple years of use. (Yes... even on a Mac.)
That is to say: your first diagnostics test should probably be to disconnect all of your externals and perform a proper clean install of macOS on the Fusion drive, preferably via Internet boot, if you can be patient enough for that. Reformat the Fusion drive and reinstall from scratch. If the performance immediately after the clean install is still not up to par, your fears about the Fusion drive could well be justifiable. If the performance seems to return to normal... well, you still might not be entirely out of the woods, as in theory, the most frequently accessed system files should have ended up on the high performance SSD portion of the Fusion drive anyway. Which leads to the next test...
Assuming a positive result from the first test, your next diagnostic will be to stress test the rest of the Fusion drive. In other words: write a crap-ton of data to the drive to fill it all the way up, and make sure that the data can successfully be read back. I haven't done this in awhile myself, but there are no doubt still tools out there which can help to automate this process to some degree. Needless to say, this test is going to take awhile.
If the Fusion drive turns out to be fine, add back in your external drives one at a time, to see where things start slowing down. If you're really fortunate, simply rebuilding the OS in-and-of-itself may solve your problem.
Good luck!
