There's also the risk of getting cited. Sure, it's not the same as driving without a valid license but you can still get a ticket/fined.
You can't be fined. You can be cited. All that is required after citation is going to the courthouse and showing them your license. The citation is then trashed.
But a cop can easily pull your info from their onboard computer. Name, address and DOB is all they need to pull a digital copy of your license, complete with picture.
Yes, all this can be avoided by carrying your license. But if you choose not to, you accept those risks.