The upgrade is definitely worth the minimal price increase.
Compare the % improvement to the % increase in price:
2.6GHz/2.3GHz = 113%
$899/$799 = 112.5%
That's breaking even.
In addition, keep in mind that as the system ages and new software comes out that requires more from your machine, the extra CPU speed will extend the life of your machine somewhat.
Finally, as several people have pointed out you can always upgrade RAM/HD later, but once you buy the CPU you're stuck with it.
If you're springing for the i7, I'd definitely go for the 2.6GHz model.
The next upgrade I'd make would be to bump the RAM to 16GB, since it's so cheap right now.
Then when you feel SSDs have dropped to a low enough price I'd make that jump.
A note on SSDs; they'll give you the biggest immediate bump to performance, most noticeably when booting or launching programs. However, if your system gets bogged down due to CPU overload or insufficient RAM, an SSD won't help you.
I recently bought a Macbook Pro 15" (non retina). I paid the extra $100 for the same CPU bump, but didn't pay the extra $300 to go from 2.6GHz to 2.7GHz - a case of diminishing returns. I'd do it again in a minute.
Best of luck and enjoy your new Mini!