fat32 cf card - can't delete folders/files

I have a fat32 formatted compact flash card I am using in my vehicle for mp3 playback.

When I delete folders off of this through Finder, Finder shows as if they were deleted, but when you read the card on a Windows machine, or in my vehicle, the folder was not really deleted and remains.

I have found that it seems like you can only WRITE to the compact flash. You can't erase.

I had to reformat the entire card to fat32 and then rewrite everything just to change the contents of it.

Is there a workaround to this?