I think 10.5.2 is the first version to be updated for the 3870.
Even with older versions of 10.5 if you have problems booting after the install. You should still be able to boot in safe mode. Then run the 10.5.6 update from there. Or go to /System/Library/Extensions and delete all ATIRadeonX2000 kexts and reboot normally. You will just will not get any Quartz Extreme or Core Image support. Once you install the 10.5.6 update the updated ATIRadeonX2000 kexts will be installed.