Looks like clearing voice memos from your iPhone is done one at a time.
I don't see any way to clear all at once.
I'm curious - The Voice memos app naturally saves everything that you record, and should be recordings that you have a use for.
So, why do you now need to try to clear everything at once? Appears that voice memos don't quite work that way.
If you have daily need of recording, remember that the recordings don't go away until you delete them
If you done with voice memos, then you could delete that app from your phone. (just sayin'...)
One more thing, be sure that you don't have iCloud syncing turned on for the voice memos app