The issue isn't so much about someone accidentally stumbling upon this and gaining access to your phone. It is about criminals who steal phones all the time; if they have a way to bypass the lock screen they can then remove all restrictions and reset the phone at their convenience. It also gives them access to all your personal information. At least if it is completely locked they would could only do a full reset and start from there; meaning your personal information was never compromised.
BUT, personally there are those who steal phones to resell them, I don't think these are often the same people who steal personal information to abuse it. It could be, but the numbers are much more slim. Most thieves want a phone they can steal, wipe, and resell as fast as possible.