Based on Apple's documented complaints about this guy, and his own letter in defense of himself, Apple is absolutely in the right to request his removal. They're not objecting to having a compliance monitor as ordered by the court, they're objecting to having this one.
The monitor has no right to investigate the company on a wider scale than was ordered by the court. He has no legitimate reason to meet with people like Jony Ive. He has no right to meet with employees of the company without legal representation. To top all of that, he has a long-time personal relationship with the judge in the case. That alone is a clear conflict.
They're also appealing the ruling entirely. So not only is it not in their interest to have someone inside the company gathering information that will benefit their opponent in court--there is a question of constitutionality at play. Apple should not be required to incriminate itself or open itself to further intrusive investigations. The appeal should be based on the original case and evidence at hand.
As for Apple engaging in illegal activity before this...that's questionable at best. Having followed the case closely and seen the evidence presented by the prosecution, I saw nothing that indicated that was the case.
What I saw was a judge that was inclined from the start to find them guilty. Despite records of meetings, emails and phone calls that didn't at all indicate an ongoing or long-term conspiracy, that's exactly what happened.
Apple (or any company) has a right to engage in competitive business practices. Even if those practices are aggressive or damage the income of their competitors. Content creators have the right to charge whatever they please for their content--even if it had previously been offered for less.
The one piece of evidence that seems to have hurt them was a draft email from Steve Jobs indicating that his desire was to raise the price of ebooks so that Apple could compete against the likes of Amazon. The problem is that this was a draft email. It was never sent. If having a personal desire to beat your competitors in business is a crime, or to think up potentially bad ideas to do so is a crime, then we'd have to lock up every business owner in the country.
It is quite clear to me that Apple was targeted by the DOJ (and multiple states) as an easy target. They assumed that like the publishers, Apple would just cave in, pay their shakedown money and the DOJ would get to claim a "Pro-consumer" victory. They're now being punished for not being obedient. This follows a long history of Apple not playing the corporate lobbying (bribery) game and came on the heels of Apple embarrassing the congressional committee that attacked them over their completely legal tax practices.