This is how I feel about them. You're no more/less safe with or without a smart lock. If someone wants in, it would be even easier to just break a window. Someone casing the place would know to break in when you're not home anyways, in which case, it doesn't matter what type of lock you have. I don't find it'll happen soon where there'll be "thieves" roaming your block trying to see who has a smart lock that they can easily hack into with their smartphone. Someone who does already know you have a specific brand smart lock to come prepared... well he already knows your schedule so no one would be home during the break-in anyways.
Smart locks seem more about convenience really. Everyone would still carry a key with their car keys just in case. Similar to how there are always still light switches to control smart lights in case the phone isn't around or acts up.
As for the manual latch inside, like the hotel style ones... you can only use those when you're home anyways. That extra security you're wanting for when you and your family are home won't be any different with a smart lock. You'll still be latching it before you go to bed. Actually, hotels is a good question too. A lot of them now just use electronic locks. So when you and your family are out seeing the sights, the only thing protecting your stuff is the one electronic lock. That would seem an ideal place for a hacking thief as all the locks are the same type, but most people don't even really worry about it.