The older style chargers are not the cause of it. It's just age and normal wear. You can either leave it as it or chip the rest away. It won't hurt anything thing to leave it or remove it. I'd suggest leaving it in place until the next time it's cleaned.