I'd be curious to know, from these responses, where people are located. For me (Pittsburgh), it's tough. I have a case that can hold a card or two and I almost never use cash, so theoretically I could ditch the wallet and just carry my phone, with my ID and a CC stored in it. But ApplePay is only starting to take off here. I'd say 50% of the places I frequent use it. Perfect example is there is a strip mall near me with a Giant Eagle (grocery), a Target, and a Lowes all next to each other. The Giant Eagle requires you to scan your physical Giant Eagle card before going through the self checkout, so I'd have to carry that. but it accepts ApplePay. Target does not accept ApplePay. Lowes will scan your Lowes Card app on the phone, but does not support ApplePay. So if I want to run errands at those three stores at minimum I need my CC, my GE card, and my phone. Plus my drivers license, since you are required to have it on you when you drive.
What do people do about restaurants? I have yet to see a traditional restaurant, with a "bring the bill to your table" setup, support ApplePay. I'm going to guess that's different in the Silicon Valley area.