iPod Touch 2g
It's better, definitely. I was never satisfied with 3.1.3, and especially not with 4.0-4.0.2. The 3.1.3 build always felt "heavy" to me, and never seemed to respond quite as quickly as I needed it to. 4.0 was a disaster, of course. But 4.1 is actually better than all of them, as far as I can tell (I have yet to try 3.1.3 again, and DO wonder if living with 4.0 for so long skewed my perception of 4.1 into thinking it's faster than it is).
Let me put it this way: if 4.0 had never come out and they had gone straight to 4.1, nobody would be talking about lags and crashes as a problem. If you upgrade, I'm sure you'll be satisfied with the results, and if you're not you can always go back. The folders ARE handy, and so is the unified inbox. I'd say give it a try.