I am still rocking a Note 2, for about a year and a half now. It was really starting to run slow, as it had a touchwiz-based custom rom that hadn't been updated since spring of last year. But two weeks ago I flashed it with LiquidSmooth rom (KitKat 4.4.4. and no Touchwiz). It is like having a brand-new phone. Everything runs faster and loads faster. Truly amazed at how different it is.
I was leaning towards going back to an iPhone with the 6 plus but I had hoped it would have some new enhancements beyond just the larger screen (the payments thing doesn't do it for me). So it looks like the Note 4 will likely be the one to replace my Note 2. I don't feel in a big rush now, though, as the Note 2 doesn't feel slow at all with that new rom. Regardless, the Note 4 will be a big upgrade for me, compared to those with a Note 3.
A year ago I could have gone back to an iPhone a lot easier. But now the list of things I would miss if I were to go back to an iPhone has grown.
Mike