The solution with the Airport Express seems to be, "when it doubt, do a hard reset": plugging it in, then holding down the reset button for 10secs.
I've been a little frustrated at times with my Airport Express since I bought it 2 weeks ago. It works great until I move it (like to a friend's house) and it starts blinking yellow. I do a hard reset and reconfigure it through Airport Admin Utililty. Then I'm good to go until I have to move it again. Sometimes the profiles are saved, sometimes not.