Yeah most issues I've been noticing on A5 devices is RAM related not processor related. Would removing widgets from the notification center actually increase the amount of RAM available or it wouldn't matter? Haven't heard anything about this. Considering the notification center seems to refresh often, I would assume it's one of the first processes to get purge so it wouldn't make a difference.
It surely is one of the first processes to get purged, but still any widget is loaded in memory. They usually take just a few Mb, but on an already constricted space it could makes things worse.