...dragging with the mouse your best bet...
If you're dealing with everything on the same hard drive, dragging a file MOVES IT. That is, it's erased from the old location, just like you wanted.
But when dragging from one drive to another (like when dragging a file to a USB drive) the Mac will instead COPY IT and you'll have the file in both places.
This makes sense when dealing with an external drive, but if you have multiple hard drives inside your Mac it can get a little confusing as to why the behavior changes depending on what folder you're dragging the file to.
And to get super complicated: You can reverse all of that with the keyboard. You can make a COPY on an inter-drive drag by holding down the Option key. And you can MOVE a file to a 2nd drive by holding the Command key.