An Achilles’ heel is a deadly weakness in spite of overall strength, that can actually or potentially lead to downfall. While the mythological origin refers to a physical vulnerability, metaphorical references to other attributes or qualities that can lead to downfall are common.
Or just "the heel"