Interesting, since no one else posted I thought it was a Yosemite/iTunes 12 problem and not a iOS/Remote problem. I even tuned in a BR about it. Since you are using Mav/11.4 then it must be an iOS/Remote problem. Hopefully the Yo team will shoot it over to the iOS team.The remote app is not showing album art for me either. It's a bit annoying since I use the app a lot.
I'm on a iPhone 5S running 8.0.2. Remote app is 4.2.1. iTunes 11.4 on 10.9.5.