For whatever it's worth, you should double-check that "Swipe between pages" is enabled, under Trackpad preferences | More Gestures, in System Preferences.
I've been using Safari more in Yosemite, but I found the Swipe feature annoying, so I disabled it.
But this "Swipe between pages" option also disables the ability to dismiss notifications by swiping.
Just thought I'd suggest it, in case you ran into the same problem.
-CPD
YOU ARE A GENIUS.
THANK YOU THANK YOU THANK YOU!
Problem solved by this guy. Why the heck does that setting break the ability to swipe notifications? Doesn't seem related.
Either way, I'm so glad you mentioned this because I would have never thought to do that because it seemed unrelated.