To stop possible virus threats does the firewall need to be switched on at all?
This:
The firewall does nothing against modern malware.
Application firewalls, as found in OS X system preferences since Leopard, and packet filters, as found in all releases of OS X (or since 10.1?), don't provide any protection from modern malware that relies on memory corruption exploits to modify processes while already running in memory.
Even the malware that firewalls defend against, namely trojans that modify binaries on disk rather than in memory, can easily bypass the firewall by creating exceptions for themselves.
A benefit of packet filters is that access to a specific port can be filtered on a per IP basis, which greatly reduces the exposure of the service bound to the port.
Application firewalls prevent incoming messages from communicating with apps not bound to the port but don't provide any protection for the bound service and don't provide as fine grained filtering but are easier to use.
A packet filter running with stateful packet inspection or a router using NAT are the most effective of this type of firewall.
But, a secure password will provide as much protection from unauthorized access to the bound service as a firewall given that firewalls provide no protection from memory corruption and firewalls are easily bypassed by less sophisticated malware.
These types of firewalls have been supplanted by sandboxing because sandboxing is a more effective mitigation against memory corruption exploits.
Mac OS X has included incrementally more sandboxing with each new version since Leopard.
But, enabling the firewall doesn't have any major negative impact so you might as well add that extra layer of security.
FYI, application firewalls and to a lesser degree packet filters do have a negative impact on the speed of the system's network communication with external sources.