I can't see where it says it adds anything automatically. The text below the setting sounds like it will index them and offer them (contact info in mail, like the phone numbers you see at the top of an email) in other places, such as incoming phone calls, etc. Which could be useful.
I've left the setting ON and am waiting to see how it helps me. 🙂