To save money when the craze was first getting traction, I just purchased a $39.99 cd player around 2000 that would read and play mp3 discs which was nice as I could load about 8-10 albums on one burnt CDR and be good to go for the day.
I was late the the MiniDisc craze and around 2002 I upgraded to a Sony Sport MiniDisc player that played compressed (sony format, not mp3s) music and I could get about 2 or so albums on one MD.
Finally, to jump into the MP3 player pool I shunned the high price of Apple products and in 2004 asked for a Creative Zen Micro 5gb player for Christmas. I still have this player and occasionally pull it out for play in my car as it still has great sound! Even had better sound than my iPhone 4! I do believe my iPhone 5 finally has supperior sound to the Zen Micro.
Since mp3 player pricing started to fall around 2006/7, I've only owned Apple mp3 players through the years, upgrading whenever I want...including:
Shuffle 2nd gen
Nano 2nd gen
Nano 6th gen
iPod Touch 1st gen
iPod Touch 3rd gen
Currently I primarily just use my iPhone 5 which connects with bluetooth to my car stereo. It has been simply amazing to think of where the tech has gone over the years. It'd be funny to think about taking a current iPod Touch 5th gen back even 5-8 years and show it to your buddies...they probably couldn't comprehend what the device is capable of now!