Assuming it works (same power draw as GeForce GTX Titan X, so I don't see why it wouldn't), the 980 Ti should be a really nice card for the old Mac Pro:
Full review at Anandtech: http://www.anandtech.com/show/9306/the-nvidia-geforce-gtx-980-ti-review
Full review at Anandtech: http://www.anandtech.com/show/9306/the-nvidia-geforce-gtx-980-ti-review