Not like the counter guy is going to run off with it, they do have cameras. I've never been concerned. Not much more risky than giving over your credit card, and ApplePay can't be used once that 60 second period expires, while that card you hand over can be used repeatedly until you disable it (if stolen).
It's not just theft of the physical device but the cashier dropping the phone and breaking it. Plus the possibility of seeing private data from a SMS, etc. notification.
Also, people giving their phones to cashiers makes it less likely stores will actually set themselves up the proper way (by installing customer facing equipment).