Help! Xserve mail server being used as spam relay 10.4.11

Firstly, go to the following URL and run the 'exhaustive relay' option. This will tell you for sure whether your server is an open relay:

http://www.toxicservers.com/

If it comes back negative (which it should - OS X server has relaying disabled by default, so if it's on, it's because it's been turned on), then your problem is probably just that someone is spamming using a spoofed header showing an address from your domain as the originator - happens to everybody.

If, however, toxicservers show that you ARE running an open relay, then open the server admin tools, go to the mail section and restrict relaying. You can either specify a specific IP range (i.e. the range you use on your internal LAN), or check one or more of the checkboxes for SMTP authentication - this will require that a users' mail client authenticates via password whenever sending an email.

For further info, take a look at:
http://macos-x-server.com/wiki/index.php?title=Open_Relays

Ben Kei said:

Well it seems that we've got it fixed now.

After following instructions to secure our servers we were then routing our mail through Messagelabs to scan.

They picked up more instances of Spam originating from our server along with full details of message contents etc...

Somehow an account not connected to any of our users (a test account used to check the setup when we first set up the mail server some years ago) had been accessed and compromised and the account itself was acting as a relay.
It also gave us the ip address of the originator of the mail we were unwittingly forwarding.

It was in Nigeria and was sending out those 'with your help we can open the bank account' type phishing mails.

Now the question is how did this account become compromised and how come OS X server mail does not have anything in place to warn you of any compromised accounts?

Click to expand...

If its years old, maybe a disgruntled X employee. Its happened before. If its a test account, poor password (user:test pass: test123). I had a friend ask for an account on my server, which I allowed shell access at the time, he was a competent admin of some mail servers (worked for Verizon as a mail admin) and wanted to test some mail back and forth. I gave him an account, set his password to something pretty cryptic, though he was going to change it. He did change it, to something like act1v3 or some such. It was hacked within a day. I now do not hand out accounts to even the most competent of people without requiring a cryptic password. They also never get a shell anymore.

Its good you got this fixed. I also find it refreshing that you found the problem and actually did something about it. At my old work place, I was the main Policy Enforcement person. It surprised me how many times we would get calls from businesses telling us that it was ok to relay spam, as that was "normal". ?!?!?!111 I responded with, it was also "normal" for us to block mailservers that were "normally" spamming our servers.