Are you talking on the internet? Or in general? If you are talking internet since some pages use small font there is a setting in Safari to allow for a "minimum font size" and it can't go below that size. Its found under:
Safari -> Preferences -> Advanced
And check "Never use font sizes smaller than: " and choose a number. I have 14.
Did you try this OP?