Don't forget that Safari can decide a minimum font size in Safari > Preferences > Advanced.
It's disabled by default, I normally put it to 12. So if as you say some websites have too small font, now they will get bigger font. Websites with already bigger font will not be affected.