Background app refresh is like that. When you turn the main switch off, it kills it for everything, no matter their previous selection. In that other drop down menu, its for if you want it on, but only for certain apps. Like you're allowing, but only for certain ones. Where as if you tick it off up top, it kills it for all of them, regardless if they were given permission before. I would just turn the mainswitch off and not worry about it.