I would say format everything and do a long format (don't know exactly how it's called in English, only did it a couple of times in Dutch). Every file will be deleted and you have a clean HD, no clutter from half deleted files! The improvements in speed of Panther will be huge, the system will probably feel twice as fast, as not faster, that's waht I had when I installed jaguar and I have the same machine with the same specs as yours!
Let me know how it feels. I just ordered a dual 1.8 and also gonna install panther on this ole' B&W!