I think it does depend on what you’re using it for.
I’m going from the original SE to the new SE and my reason for upgrading is primarily down to ensuring I get updates for the foreseeable future (I’m guessing the original SE isn’t going to be getting many more updates).
At the moment, I’m carrying both SEs with me - I have two belt pouches, one for each phone.
My usage is quite low - browsing web pages, updating MFP, shopping at Amazon and eBay, and using banking apps. So I’m not a power user.
For my usage, I don’t find myself feeling that the new SE is massively faster than the old SE, despite the chip being four generations removed (from A9 to A13) and the RAM increasing by 1GB. Yes, there have been a couple of occasions when I’ve thought “that felt smoother” and “that opened quickly”. But, generally speaking, if I’ve inadvertently left my new SE on the wireless charger at my desk, I don’t resent taking out the old SE and using that instead.
I will reach for the new SE if I’m going to be taking photos but, even there, the old SE doesn’t take bad photos at all.
I’ve never used a 6 and I get that it’s a generation further back than my old SE but, if it’s anything like my old SE, it’s not a bad phone at all, although I imagine it’s on limited life due to no more iOS updates.
Ultimately, it’ll come down to whether you’re ready to spend the price of the new SE or not. The one thing I think we can be pretty certain of is that the new SE is going to be around for a good few years. Whether you upgrade from your 6 to an SE this year, next year, or the year after, it’s probably going to be the same SE. Buying it now means you get extra years of a better phone. Buying it later may mean you’ll be saying “why didn’t I upgrade sooner?”.