It's independent from Favorites/Bookmarks. I have an actual list of Favorites in Safari, and it's different from Frequently Visited.
And yes, Frequently Visited is just as the name suggests. It's even ranked based on how frequently you use each site. And prior to disappearing, Wikipedia was my number two, so it's definitely not a problem with how often I use it.