What happens when you open a new window or tab? Safari > File > New Window or Safari > File > New Tab? Do those new pages and tabs open to Yahoo? If they do, then that setting works as designed.
Those settings do not determine what Safari does when you open it, unless you closed all the windows and tabs before you quit Safari. Otherwise, Safari opens in the same state it had been in when you'd previously quit. If there's a page open to Wikipedia when you quit, then it'll re-open to Wikipedia. That Wikipedia window is not a "new window," it's an old window.
If you would like Safari to open "clean," close all the windows and tabs before you Quit, or hold the Shift key while you open Safari. That tells Safari to forget the "Saved State." Most people appreciate the convenience of apps opening in the same condition they were in when they last closed the app; especially after a crash or a restart.
Oh, and when you say "Applecare support could not offer effective advice," the only way you can know that the advice was not "effective" is if you took the advice and it failed to give you the desired result. If you ignored their advice, then you cannot judge whether or not it was effective.
Now, in this particular case, the advice probably would be ineffective, since what you've encountered is not a bug, but a feature. The feature hasn't changed between 10.11.3 and 10.11.5.