This isn't something only a purist or a photography buff would see, the difference in quality is plain-as-day obvious. My cell phone camera is 2 megapixels, but the lens is tiny and there's no flash. My first digital camera was a Canon S100, which also had a 2 megapixel camera. The difference is night and day. Maybe in optimal conditions they might be comparable, but in real world usage, they're not.
It's basic physics. A phone camera lens is, what, 1/8" across? The entire glass is no more than 1/4" wide. My P&S camera is easily 2-3 times bigger, and that means 4-9 times more light (surface area to collect light increases exponentially with the width). And a DSLR lens area is even larger. You simply cannot get the same quality photo with a fraction of the light, even with all other factors being equal.