I decided to try giving up caffeine a few months ago. Wasn't, originally, to be a permanent thing but I felt that bad coming off it I avoid the stuff completely now.
I was having iffy patches sleeping and thought it may be too much caffeine. I never drink coffee, it was only pop: Dr Pepper, Irn Bru, Pepsi etc. And I NEVER get headaches, I'm really lucky that way. Even when I used to drink and get a chronic hangover, my head would be clear. But for the two to three days after stopping I had a really bad migraine. I expected a reaction but nothing like that. It was completely gone after five days. Now I avoid caffeine altogether if that's the effect it has. I was genuinely surprised how crappy I felt.
Now I get less heartburn, more regular/lengthy sleep and feel more energetic oddly (probably because I'm sleeping better).
If it's having such a chronic effect, don't go cold turkey but wean yourself off it instead. Look at how many caffeinated drinks you have a day (and when in the day) and reduce it by one drink every two days or something.