"Up Next" queue problem

So i've noticed that when I try to add a few albums to my "up next" queue on my phone, I'll add the first album i want to listen to to the "up next" queue, but when I go to add the second album it ends up adding the second album to being the immediate album to listen to first after the 1st song off the first album I wanted to listen to plays and the remainder of the first album ends up going into resume. So basically what I've noticed is that if the mini player is not shown with a "now playing" song the "up next" queue won't work for me correctly but if I do have a "now playing" song, that using the "up next feature works fine. It's kind of annoying that I have to choose a song to play first to add as my "now playing" in order to not have problems with the "up next" queue. Does anyone else have this problem? Am I overlooking something/doing something wrong? I know in iTunes it works fine when I try adding a few albums in my queue. I've have tried looking on the forum to see if I could find anyone else having this problem but for the most part have not found a solution if there is any.