That one concept may have affected you deeply at some point when you were younger, and stayed with you.
I was thinking the same, for the mustard phobia.
Could it be that someone, a sibling maybe, dumped some on his head, before his current memory, and caused him grief. Perhaps got it in his eyes? That would hurt.