Darren,
I did a similar thing. I was an IT Technician until I decided upon a Games Software Engineering degree just up the road from you in Newcastle.
I was 24 when I started to study. Got a placement in my third year as a (non-games related) programmer. I'm in my final year right now but I'm still employed as a part-time programmer with my placement company.
My point is this; your degree is what you make of it. I get the impression that you feel your late entry will somewhat hinder you throughout your degree. However, you'll probably find that your mature status will drive you to work harder on what you're doing and said drive will be absent from a fair few of your younger peers on the course.
My advice would be don't shy away from the things you don't understand. Your maths may not be as strong as other members but that in no way means that you can't be as proficient after applying yourself and working hard. Maths is a part of games and getting over that hurdle and accepting it is half the battle.
Furthermore, you'll probably find that maths and programming go handy in hand; as you get better at one, the other becomes easier. That's my experience, at least.
All speaking from my own experience, of course. Yours may differ. In any case, good luck with your degree and, most importantly, make sure you enjoy it.