Spam filtering based on message path

[Doctor Q]

New Scientist article: Retracing spam steps could halt mass emails

A team from IBM and Cornell University developed the anti-spam technique, which they call "SMTP Path Analysis". The algorithm "learns" by examining the string of IP addresses included in both spam and legitimate email headers. When a new message arrives, it is then able to judge, with relative accuracy, whether it is legitimate or, in fact, unwanted spam.

The researchers behind SMTP Path Analysis had to deal with the fact that spammers can forge the address of the mail server used to send a message out. To counter this, they developed another algorithm that judges plausibility of the overall path that a message claims to have taken.

This looks promising to me. They describe it as only one technique to be used in combination with others, such as content filtering, but I think it is a step in the right direction, using information shared among e-mail servers to identify messages with forged headers.

Spammers can forge header information, but only up to the point where their messages reach a legitimate server, so for example a message claiming to be from Citibank will have a path that differs from real Citibank e-mail. A system like this could notice that.

 

[rainman::|:|]

More likely, spammers will quickly find a way to forge legitimate-looking return paths. It looks like they're simply trying to find return paths that wouldn't normally occur, assuming most of today's spammers merely put junk data in that field to make it look real. Until servers are secure, there simply isn't any way to sort legitimate from non, except content (which we've been doing for years).

Would be cool if this, combined with content filters, made for a more accurate filter system...

 

[Doctor Q]

Another limitation: This won't stop spam from zombie PCs (where implanted software causes a victim's computer to send spam), because the sender could be the victim instead of a forged source.

In the long term, e-mail servers will have to be able to identify and authenticate each other so that they (and therefore you) know which mail is from a proven source and which is not.

 

[Lacero]

The good method to end spam is to set up a registry.

 

[absolut_mac]

Spammers can forge header information, but only up to the point where their messages reach a legitimate server, so for example a message claiming to be from Citibank will have a path that differs from real Citibank e-mail. A system like this could notice that.

I'm no expert, but it does sound like they are on the right track. It definitely seems more promising (and probably will be more accurate and reliable too) than MS's pay-to-send/receive-email implementation.

This way they will not only be able to track which servers the message passed through on its way to your in box, but it should also help them to identify which servers have been hijacked into becoming spam zombies.

 

[savar]

Another interesting article...

http://www.sci-tech-today.com/story.xhtml?story_id=23355

I cited this guy's paper in my Senior Thesis in the spring, but all I could turn up on google real quickly was the above article. The two authors haven't commercialized the technology yet, but it sounds pretty interesting to me.

Strong points: zero type 1 error and zero type 2 error (in admittedly limited experimentation). It refrains from classifying a significant portion of the messages, however.

Not too far from what these guys are talking about, but if you read their original paper they state at the end that all these technologies are just stop gap measures until somebody builds a secure email network.

 

[Doctor Q]

I cited this guy's paper in my Senior Thesis in the spring...

I love this description in that article:

Spam was once a wartime meat. Now, it's the meat of a whole new war.

 

[pubwvj]

SpamAssassin, with lots of custom scoring, Exim filter rules and Mail.app rules got my spam load down from over 5,000 spams a day to a manageable few a day with virtually no false positives. A lot of bother getting it all tuned and it takes some ongoing maintenance. The path analysis is a good idea but won't help in a lot of cases, especially with zombies.