I installed the GM and while it feels faster sometimes, it's definitely slower overall. The camera take longer to get to the point where you are ready to take a picture, my lock screen is sluggish to respond, and the keyboard seems to be much less responsive than it was with 3.1.x
Remember that 3G won't get access to multitasking or even background wallpapers, so the big things you are really "getting" with the upgrade are:
1) Unified inbox
2) Universal spell check
3) Folders (sure, less pages of icons, but still takes nearly as long to find the folder you're looking for, I'm not sold on this feature. I just use spotlight search for apps I use less often).
4) Camera places/faces .. again, fun, but not something i'd be willing to trade for overall responsiveness
It's also worth noting that Navigon East has stopped working for me, I hope they fix it in the iOS4 compatible release, but stops receiving location data after about a minute. The "location" icon in the top right corner disappears, and the app starts reporting inaccurate GPS data (triangulation only)