I know that the manuals for my old Galaxy Note 2 warned that strong magnets might affect compass and gps functions and to avoid strong magnetic fields. I don’t know if damage could possibly occur, and a magnet in a case or belt is likely far too weak for any potential of damage.
Also keep in mind that iPads, which I imagine are internally quite similar to iPhones in terms of circuitry and other components, are designed to work with SmartCovers that directly attach to them magnetically with enough strength that you can hold and shake them.
When people think about magnets harming electronics, they're mostly thinking about devices that depend on magnetic storage mediums like old floppy disks and computers with HDDs, not devices with solid state storage.
Navigation performance and device spacial awareness can be negatively effected by a magnet.
The iPhone uses an magnetometer for heading, outside the presence of a magnetic field it can detector north. It can zero out an external magnetic source if its not TOO powerful though.
The Magnetometer is also used for motion detection, namely the devices pitch (x axis).
That said I wouldn't worry about it. None of that will "damage" the phone and if you notice adverse performance for certain task you know how to handle it now. Like if your porn VR isn't working right...