There seems to some confusion about how active implantable medical devices handle magnetic fields. Most of the devices have a reed switch or GIMS sensor for detecting a magnetic field. In the MRI-compatible models, the GIMS sensor may also be responsible for detecting an active MRI machine. Many pacemakers have a "magnet mode" where the patient can hold a manufacturer-supplied magnet (any sufficiently strong magnet will work) over his or her implant to cause it to pace at a constant rate. In the case of an implantable defibrillator, placing the magnet suspends tachyarrhythmia detection--which in turn causes it to cease delivering therapy (it won't shock your heart back into a stable rhythm anymore).
Most modern pacemakers will attempt to sense your heart's natural rhythm and pace only when required and at the rate at which your heart is naturally attempting to beat. However, if something goes wrong, (external electrical interference, internal interference due to cardiac tissue abnormalities, a lead getting dislodged, etc.), the patient can place the magnet to force the pacemaker into a consent 60 PPM. Getting shocked by a defibrillator usually isn't a very pleasant experience. If it's constantly shocking you (due to some of the same issues I listed with pacemakers), you can use the magnet to disable it. With many implants, the physician (or manufacturer representative) can disable these magnet modes.
Another issue is the inductive programming heads that the physician or manufacturer representative uses for configuring the implant. Some implants will only accept inductive programming commands if there's a magnetic field present (the programming head has a magnet in it). Other implants use special inductive wakeup codes or pulses. I believe all modern implants use CRCs or some other error detecting or correcting codes in the inductive programming commands as well.
I've only ever heard one story (there may be many out there, I don't know) of a pacemaker being inadvertently reprogrammed. In that case, the patient was up on a ladder using an AC electric drill with a brushed motor. The sparking brushes caused EM fields which reprogrammed the pacemaker. In that case, the pacemaker was an early-80s model which didn't use error correcting codes in its programming commands.
I think what may be a little more concerning about electronic devices with magnets in them is that they act more like a programming head in that they can cause inductive interference
and they have a magnet in them which will put some implants into a mode in which they'll accept programming commands. However, hopefully the error-detecting codes would still protect you in that case.
Implantable devices are also susceptible to EMI messing with their sensing algorithms. Like I mentioned, many will attempt to detect your heart's natural activity. If it's detecting EMI instead of real cardiac tissue depolarization, it may pace at the wrong time or cease pacing if it thinks your heart is doing just fine on its own.
Finally, many of these devices have RF radios in them for communicating diagnostics to a bedside unit or accepting programming commands. You probably won't be holding your iPhone over your implant during your physician visit or in the operating room, but if you fall asleep with it on your chest in your bed, then the signal-to-noise ratio may be so low that your implant can no longer communicate with your bedside unit. The same is true for devices you actively control (like a neuromodulation device)--if your iPhone is interfering with your device's radio, your RF remote will no longer be able to control it.
My experience with this is limited. I'm not giving any medical advice here.

The potential problems depend on the type of device you have, the manufacturer, the model, how it's configured, and even your condition. Therefore, getting advice from your physician (rather than an Internet forum) is always advisable. The physician can contact the manufacturer and get answers. Either the manufacturer will have tested the given scenario or they'll tell you, "don't do it."