They can also potentially build the modem directly into the A-series die, reducing overall power consumption, allowing more room for battery, etc.Yes, Qualcomm will still get some cash (from patent licensing, when Apple eventually starts making their own modem chips), but Apple will gain more control of the hardware they use. They have a long history of being pained by other's control of the hardware/tools they need - most famously when they switched to PowerPC, and then after a few years IBM stopped caring about making faster/better processors that Apple could use. Apple jumped to Intel CPUs because in the Intel pond at least they could get the same options everyone else was getting. Then they started making (yes, not fabbing, but designing) their own ARM CPUs, which does give them nearly complete control over the CPUs they need (for iOS), and now iPhones are shipping with CPUs that are perfect for them, and tend to beat the CPUs the Android manufacturers use quite handily. It's clear that eventually, the Mac will switch to Apple designed processors (I don't expect that any time soon).
Apple did a great job on their ARM chips, if they can do similarly well on modem chips, they can get exactly the modem chips they want, and pay Qualcomm only for patent licensing.