Share |

Making Mincemeat of Spam

Reprinted from The Triangle TechJournal

Are we truly the society of overweight, sexually challenged, gambling, class reunion addicts with a penchant for Nigerian philanthropy that our spam makes us out to be? Well, of course not. But it would be difficult to argue against upon reviewing the inboxes of the emailing population who receive anywhere from 11 to 100 spam emails each day and then spend another hour and a half each week to get rid of them. And it's only getting worse. According to Gartner Inc. analyst Joyce Graff, "by 2004, unless an enterprise takes defensive action, more than 50% of its message traffic will be spam."
And as the volume of spam increases, so does its financial drain. Ferris Research reported that spam-related costs amounted to $8.9 billion for businesses in the United States in 2002 and estimated more than $10 billion in 2003. These costs are driven primarily by lost productivity, consumption of information technology resources (including hardware, IT personnel, bandwidth and storage), and help desk support.
So what are we going to do? Two of the most promising solutions are to filter it and to outlaw it. And what we're finding right now is filtering works and laws don't. Typically, it takes about two to three months from the time companies install anti-spam systems until they can effectively pick up on patterns. But once they do, most systems should be able to weed out 90% of spam with a less than 1% false-positive rate. But spam filtering is not a perfect science, and any system that claims to be able to eliminate 100% should be overlooked because they obviously don't know what they're doing.
And how are we going to do it? There are two basic approaches that a company can take to combat the spam problem: 1) deploy an in-house system at the server/Internet gateway level and/or at individual user desktops or 2) use a managed service provider (MSP) that will block spam before it ever reaches the corporate email system. The two main advantages of using an in-house system are cost and control. The fundamental advantage of a MSP-based solution is the ease of deployment and management of the anti-spam infrastructure - using an MSP is often as simple as modifying the MX record of a domain and pointing it to the MSP's system, there is usually very little effort required for a company to deploy or manage such an anti-spam capability. Consequently, most large organizations are much less open to the notion of outsourcing their anti-spam filtering, while most smaller organizations view the outsourced model as ideal.
Need proof? Xerox has implemented a homegrown antispam system that blocks 75% to 80% at the gate with an additional 20% of the remaining spam being filtered out later by IT staff. Norfolk Southern installed IronMail from CipherTrust Inc. which sits on its gateway and uses an array of filtering strategies. Even with the filter, spam has managed to get into Norfolk Southern's system, so employees send its information security department a local deny list of addresses to be blocked. This system has reduced Norfolk Southern's spam rate from 25% to 1%. Macrovision Inc. has opted for a voluntary spam-fighting program — letting end users decide whether they want to use the PerlMx filters they purchased from ActiveState Corp. If they opt-in the program, they then customize their filter settings so that employees continue to receive email that is relevant to them — newsletters will continue to go to sales representatives while mailroom clerks can minimize solicitations. And closer to home, my company, hesketh.com, has recently undertaken an anti-spam campaign that has eliminated approximately 96% of our unsolicited email with a false positive rate of 0.01%!

Our secret? After trying Razor, SpamAssassin, SpamBouncer, and a variety of other services and DNSBLs, our CTO, Steven Champeon opted for a combination of DNSBLs and digitalanswers.de's check_local, plus the following implementations:

  • ipchains: Block certain spammers at the TCP level, using listings from spews.org and the ROKSO database.
  • sendmail: Bounce all mail addressed to known spamtraps, or non-existent addresses targeted by spammers. We have approximately 1500 spamtraps on our servers.
  • sendmail access.db: Block/tag mail from known spammer domains.
  • sendmail rulesets: Bounce all mail sent direct from a dial-up, cable modem, or DSL using 320+ different patterns.
  • procmail: Tag and quarantine all mail sent to our low priority MX. Quarantine everything sent from an obviously forged domain. Quarantine any mail containing a known blacklisted domain in the body. Quarantine all mail with the MIME header 'multipart/alternative' but only one HTML part.

As with any anti-spam strategy, we continue to tune our filters, adjust our policies, and take measures to enhance the security of our systems and mailboxes. Future directions include establishing our own in-house DNSBL, making previously unenforced policies more explicit, writing custom rulesets that allow for whitelisting, putting policy decisions on what to block in the hands of individual users, and so forth. But the efforts have spared us some 115,000 spam messages in the first three weeks of June alone.
The bottom line — spam is definitely here to stay and is moving into mobile and instant messaging. It's too easy, too cost-effective and too cheap to kill entirely, but we can take effective measures to block most of it.

To recap, when selecting an appropriate anti-spam solution, the first decision to be made is whether or not to implement an in-house solution or to use an MSP that provides anti-spam filtering services. If the former, the next decision is at what level should the solution be implemented — at the perimeter of the messaging infrastructure, at the desktop, or a combination of both approaches. If an organization opts for the MSP solution, there are a series of questions that should be asked of prospective MSPs in order to determine their financial viability, their performance capabilities and their ability to maintain the integrity of customers' email data. Choosing the wrong MSP can have significant negative consequences for employee productivity, an organization's reputation and its ability to conduct business. But in the end, either approach will hit spammers where it hurts.