so do you think there would be much speed difference from me replacing the internal hdd to an ssd compare to using an external ssd via a USB port?
Replacing any of the internal drives would give you about 600Mbps sequential read/write. Compared to a HDD, it will feel a lot faster due to the much higher random read/write speeds.
With your Late 2013 iMac, there is a lot of SSD options for you, both internal and external, but the cheapest, which is also probably the easiest, is using a 2.5" SATA drive externally with an USB3 enclosure or USB3/SATA adapter cable. Decent speeds, but the biggest problem is the lack of TRIM support over USB, which could be an issue for the long term. Although, there are plenty of people using USB boot drives without issues.
SATA SSD over USB is cheap, as you can get a 2TB 2.5" SATA SSD for less than $130, or a 1TB for less than $60. 2.5" SATA USB3 enclosures cost about $10-$15.
I like using USB3/SATA Adapter cables, because it just makes things so much easier, although, not as clean of a look. I like this one because the cable is a longer than many other ones I have used:
Product Description:Take advantage of the lightening speed of USB 3.0 to quickly transfer files up to 5 Gbps! Connect any 2.5'' Serial ATA (SATA) / SSD or SATA II Hard Drive to your computer through an available USB 3.0 port on your desktop or laptop. Reverse compatible with USB 2.0 and 1.1, this...
www.amazon.com
Currently, $10.79.
If you are looking for a name brand SSD, here is a 2TB PNY from NewEgg for only $138:
Buy PNY CS900 2TB 3D NAND 2.5" SATA III Internal Solid State Drive SSD7CS900-2TB-RB with fast shipping and top-rated customer service. Once you know, you Newegg!
www.newegg.com
There are cheaper ones with good reviews, but with obscure brands that I never heard of.
would be much speed difference
Like I said, there are plenty of options for the Late 2013 iMac, all with pros and cons, here are the external options:
SD Card - IIRC, I think it is limited to 5000Mbps link speed, and you would only see those speeds with a high speed SD card, which is very expensive. Not worth it, imo, but still an option. No TRIM support as well.
USB3 - About 400MBps max. Cheap, and easy to get and set up. No TRIM support. You can get a SSD external drive like the Samsung T5, or get a cheaper SATA SSD with an enclosure or USB3 SATA cable adapter. Recommended for most people unless higher speeds is required.
TB1 - A SATA SSD would be about the same speed as a USB3 SSD, but you get TRIM support. Hard to find these drives, and eBay would probably be the best way to source them. Probably not worth it, as they are expensive.
TB2 - Basically the same as the TB1 drive on your iMac if you use a SATA SSD. You can also find some NVMe and AHCI TB2 SSD enclosures. They tend to be pricey, but you could see speeds close to 800MBps on the NVMe TB2 drives that I have researched before. Just like the TB1, probably not worth it do to price. Has TRIM support.
TB3 - A TB3 NVMe drive is the fastest single non-RAID drive you can get for you Mac. Speeds over 900MBps, but at a cost. Requires Apple bidirectional adapter. Depending on the set up, could require a TB3 dock to power the drive. NVMe drives are falling in price, but between the drive, the bidirectional adapter, and maybe something to power the drive, this could be an expensive option. Also has TRIM support. Probably not worth it due to complexity and cost, unless you need that super fast speed.
USB3.2 Gen2x1 - 10Gbps link speed, easy to find. No TRIM support. The biggest issue with this is that you need to have a TB3 dock with 10Gbps USB ports, and also a bidirectional adapter to connect it to your Mac. I have never tried this, but would think speeds of 700MBps would be realitistic. Probably not worth it due to complexity and cost.
Internal options for the Late 2013 iMac:
HDD to SATA SSD Swap - Cheapest internal option, speeds over 600Mbps, TRIM support. Nice to get that hot running HDD out of the Mac. Biggest downside is removing the glass/display, cleaning up, and using the adhesive strips to put it back together.
Blade SSD swap/adapter - More complex install due to removing the logic board. More expensive than replacing the HDD with a SATA SSD, but same speeds. Has TRIM Support. Not worth it on the Late 2013, imo.
Internal RAID0 - Replacing both the blade SSD and the HDD swap with a SATA SSD. Both drives made into a SW stripping RAID. I think this would be the fastest possible boot drive on the Late 2013 iMac, probably 1000MBps+ speeds. I could be wrong, but I think there is no TRIM support. Obviously more complex, but probably still cheaper than doing a single external TB3 NVMe drive, while still being faster. Probably not worth it, but I am thinking of doing this on one of my Late 2012 or Late 2013 iMacs just for fun.
Piece of advice, if you decide to swap the internal drive, either one, I would set it up externally first and make sure it boots up and it working correctly. After testing it, then do the internal swap.
Install the latest OS on the internal drive prior to pulling it to make sure all the firmware is updated. You said you are running Catalina, but I am unsure if that was a clone or if you installed Catalina on the current SSD. If you just cloned the OS to the internal SSD, then the firmware may still need to be updated.
Wait on putting your display back on your iMac, meaning do not put the adhesive strips on right away. You can put the display back on, and use painters tape to hold the display glass onto the iMac body. This way, you can test everything, makes sure it works without issues prior to glueing stuff back together.