2nd ethernet port? Well...
1. NAT
2. Network firewall
3. Proxy server
4. Two independent internet connections
To give an example of the first two:
A smallish company needs to connect from inhouse to the outside world. They need to control access and traffic. While a company that is less concerned about security might use a router with NAT and Firewall built in, this limits your options. For full control, you want a full system. This full system is your network's gateway to the outside world. You have a fully capable firewall where you can modify the rules based on detected inbound traffic. You need to be able to log all transmissions to detect penetration of the network. You want to enforce network policies. Meanwhile, you also need a NAT running with a DHCP server. At the same time, you want VPN access from the outside world.
By hooking one Mac Pro ethernet connector into the outside world, and one into the LAN, and running a OSX Server on it, you can do all that.