Ads:

Greylisting: The worst thing to happen to email since spam

Greylisting (or graylisting) is a new anti-spam measure implemented on email servers. It's starting to become more widespread, and if you don't run your own email server, your service might be greylisting (or might add it soon) without your knowledge or permission.

It has the potential to destroy a fundamental function of email in a futile attempt for service providers to save money, yet nobody knows about it.

It's very, very annoying. Much more annoying than spam.

What is greylisting?

Email servers work by accepting messages, figuring out where they need to go, and sending them along their way (or to another intermediate server) until they reach the destination. It was designed to be robust and accommodate unreliable servers, so if a destination server isn't available or temporarily refuses to accept a message, the sending server will try again later.

It's like getting a package delivered when there's nobody home to accept it: the delivery guy tries to deliver it for a few days before assuming you're dead and returning it to the sender.

Greylisting tricks the sending servers into an intentional delay: for every incoming message, a greylisting mail server says "I'm too busy - try again soon!" It's not too busy - it just inserts this intentional delay to see if the sending server will try again. Then it accepts the message if the sending server retries after a minimum delay (usually a few minutes).

Theory

Greylisting is supposed to reduce spam. Supposedly, spam-sending email software won't properly accept the "I'm too busy" messages and try again later, as requested — instead, it will assume that the message's recipient is not a valid email address, and remove it from further spam attempts.

This isn't to reduce the amount of spam that gets to the users — it's to reduce the amount of spam that the server's spam-filtering software needs to analyze (which can be very resource-intensive for a busy server). This is why it's in your email provider's best interest to implement greylisting: they can save money on servers.

Reality

It doesn't work. Plenty of spam still gets through. What's stopping the spam-sending servers from properly obeying the delay and sending the message again? Spammers have found ways around almost everything we've come up with. Why does anyone think they won't be able to figure this out?

We don't need it. Modern spam filtering is highly sophisticated and extremely accurate. Greylisting does nothing to detect spam: it only slightly delays spam delivery.

But that's not the worst part. If it didn't affect my life as an email user, I wouldn't care enough to write this article.

The worst part is that email is no longer instant. In some cases, it's not even close.

The email-delivery protocol (SMTP) doesn't have a standard way for greylisting servers to tell sending servers when to try again. Some wait 15 minutes, some wait an hour, and some wait a day.

This means that if your email provider uses greylisting, there's no guarantee that a message will get to you in a reasonable amount of time.

So what happens?

You can't email someone a link or text blob for their immediate use (like if they're in the same office). You can't email something to someone as you're on the phone with them for collaboration. Websites can't use email activation to reduce spam or verify correct contact information.

I've frequently seen 24-hour delivery delays on greylisted emails. On the internet, that's an eternity. You might as well use paper.

Greylisting destroys email as an instant communication medium.

Spam reduces the efficiency of reading email, but greylisting removes much of its legitimate use completely. At least spam filters only prevent me from talking to my friends about taking viagra while refinancing my mortgage with the cash I made at home by investing in diet-pill stocks.

I'd rather get more spam.

More information

Spread the word to prevent greylisting. Digg this story.

Update: Response to comments

It tells me a lot that the huge influx of comment authors (trolls from Reddit) who argue for greylisting are predominantly server administrators who have enabled it for their users.

Their users have been suspiciously absent from this discussion, probably because they have no idea why it takes forever to get their email. They chalk it up to another random disappointment with computers, or "the network" being slow, or various other mythical computer problems.

In the case of account-activation emails from websites, they email the website operator to find out why their activation email hasn't yet arrived after 10 minutes. How do I know this? Because I administer some big sites that have this problem, and I get the "bug" reports when users complain that the emails aren't being sent. Every time, I find the section of the mail log for their email address, and 75% of the time, it's the recipient's greylisting that's delaying mail from a very legitimate, fully compliant Postfix server hosted in a major datacenter. (Most of the remaining 25% are email address typos.)

They don't know that the problem is on their end, and was probably imposed by a short-sighted (or simply cheap) server administrator. They, like me, expect email to arrive within a minute, and assume that if it hasn't arrived by then, there's something wrong. There's no technological limitation to prevent most messages from being delivered within 10 seconds.

Greylisting can be configured to be slightly less annoying by having a generous whitelist (a list of senders whose messages aren't delayed), or by only delaying suspected-spam IP ranges such as broadband home users or most of Russia. But this still imposes the delay on a lot of legitimate email (at least one message from every sender), and most importantly, it relies too much on the server administrators to get things exactly right.

There are a lot of good server administrators out there, but like any job, the majority of them are mediocre at best. The administrators at FastMail, for example, never whitelisted the Marco.org server, despite repeated conversations in which I informed them that Rackspace is not a dial-up ISP and should therefore be eligible for their IP whitelist. (Yes, Rackspace. I was dumbfounded. A sysadmin who's never heard of Rackspace is like a mechanic who's never heard of Ferrari.) Rackspace servers still never get whitelisted by FastMail's greylisting, so every message is delayed regardless of the sender's history.

Many readers criticized me for my high expectations of email, claiming that I should use instant messaging for communication intended for rapid delivery. This doesn't really work either:

  • Both parties must be members of the same IM networks.
  • Each party must know the other's screen name. This requirement renders IMs useless for websites trying to validate their users.
  • Both parties must be running their IM clients, and must be signed in at that moment. I work with a lot of people with laptops who travel through various networks every day, so this just isn't realistic. Any laptop user will understand.
  • Ever try sending a file through IM? By the time you figure out how to get through the NAT on both ends, a greylisted email may have arrived.
  • IMs only work gracefully when the recipient is one person.
We already have a near-instantaneous communication protocol with minimal requirements on either end that's very tolerant of NAT and irregular connectivity, accessible by everyone running any software on any platform.

It's called email.

Stop ruining it.

Comments

Anonymous on April 5, 2007:
email isn't instant messaging.
Marco on April 5, 2007:
I agree. But it's not parcel post either.

You expect IMs to arrive within a second or two. You can usually expect email to arrive within 10-60 seconds. When that delay is unexpectedly multiplied and ends up taking between 15 minutes and 24 hours, that's extremely significant.
Dan on April 5, 2007:
It's not instant messaging--but it is nice to know that once you send it (theoretically) the other person has it or at least has access to it within seconds.
Anonymous on April 5, 2007:
I am no fan of greylisting either , but you mustnt contradict yourself
On one hand you say "Spammers have found ways around almost everything we've come up" and then you you go on to say "Modern spam filtering is highly sophisticated and extremely accurate. "

So people are obviously going to blow holes into your blog. That doesnt help the message
Marco on April 6, 2007:
I said "almost".

Spam filters can detect the vast majority of spam, but some messages still get through - mostly those that use randomly-generated images to contain the spam text.
Anonymous on April 6, 2007:
You're over-simplifying greylisting. I evaluated a pair of solutions for use with Postfix, and both were capable of automatically whitelisting servers that properly retry sending messages.

Options included how to whitelist (ip address, subnet, best-guess based on reverse dns), how many messages must be successfully delivered before a server is whitelisted, and how long to keep whitelist records.

What this amounts to is a one-time (or two-time, or three-time, or n-time) delay for messages coming from a given server, after which there is no delivery delay, provided the server continues to send messages within the whitelisting period.

And while you are correct that spammers can circumvent greylisting relatively easily, this does not mean that they will. Most spammers, as far as I can tell, expend as little effort as possible. Consider the fact that they spam common administrative addresses without concern--this is like calling the cops and telling them you're breaking into someone's house. I think you also underestimate the value of reducing the volume of messages that need to be processed.
Anonymous on April 6, 2007:
greylisting DO work. it's almost ULTIMATE antispam solution
which I knew of, making other antispam solutions a history.
I see no way to think back after living for over two years with
self-crafted greylister: I have no spam and no problems. think different.
Anonymous on April 7, 2007:

I've found greylisting to be very effective, especially when used in conjunction with DNSBL's and filtering. And to your comment about it not being about reducing spam that gets to users but reducing load on servers, I would say that reducing load on servers actually has improved our filter performance which has in turn further reduced spam for our users. We've gone from over 40 per user a day to less than four a week per user. For a company of 200 people not collectively wasting 15 hours a week deleting junk mail (say one second per spam: 40 * 7 * 200 / 3600), greylisting has been a critical part of a great payoff.

I do agree with your main point that the price of delaying spambots until they go elsewhere does have the collateral cost of delaying some legitimate user mail. This can be tremendously annoying when waiting for mail like a confirmation for a craigslist posting or notification that a particular stock just dropped to an attractive price.

Currently the only mitigation strategy I've seen is auto-whitelisting: after x (usually 3) number of successful deliveries, sending server ip addresses are added to a pool of OK servers and the delay is no longer imposed. Depending on the implementation, the whitelist entry may expire if there's no more traffic from that sending ip for 30 days. So if you receive a reminder email only every 31 days, it will probably be delayed unless it's put into a static whitelist.

There's definitely room for improvement of the legitimate mail end user experience, but I'm hard pressed to say what else would help. But there's a baby in that grey bathwater and it'd be a shame to discard it all.
Anonymous on April 7, 2007:
The way greylisting works is that once a message has been successfully delivered you will find that subsequent messages are delivered without delay. The delay is only introduced for servers that your email server has never encountered before.

You are also incorrect when you state that it does not work. I run greylisting on my servers and find that it eliminates a lot of the hit-and-run spam sent from zombie networks.
Anonymous on April 7, 2007:
i accept email on all the email addresses i've ever had, going back to 1988.

before greylisting, i received hundreds of spam messages a day. after, i get 5.

so yeah, it does work. any reasonable test will show that.
Anonymous on April 7, 2007:
Unfortunately for your argument, graylisting actually works great. It works particularly well in combination with a dynamic host blacklist (you only delay mail sent from a server on DHCP).

If it stops working, of course, mail admins will stop using it and will use something else.

Remember when it could take days for e-mail to get to you, depending on how often your mail host dialed up to other hosts and exchanged mail with them? And you had to write your address using bang paths from well-known hosts instead of at signs? And there wasn't even any spam back then as an excuse. So, um, yeah, don't expect e-mail to be instant and you'll be thrilled at how fast it's delivered.
Anonymous on April 7, 2007:
I use greylisting on my server. I don't like it, either, but the reason I use it is that it gets rid of about 70% of my spam. By that, I mean that now I get about 150 messages per day in my spam folder. Without greylisting, it's over 500.

Greylisting is a form of proof of work. The work that it makes the sender prove is that they can hold a message for a short amount of time (my greylisting interval is 10 minutes) in a queue.

It's rather appalling that greylisting works at all. But it works because of the way that some spammers use their botnets. When they change them so that they can handle greylisting (by using a real MTA with queueing), then it will no longer work. At that point, people wil stop using it.
Anonymous on April 7, 2007:
Greylisting is extremely effective for being so simple. It took away most my spam with this simple rule. I don't use bayesian filtering or its likes because of the false positives; I can't afford to miss an important email just because it is about viagra (or whatever :) ).

But you are dishonest, you say that "it doesn't work" because "plenty of spam still gets through". Well, d'uh. No solution is perfect. This is just better than many other. And what does it matter if spammers re-queue the mail immediately? It is just delayed again until the pre-set delay is met.

And most greylisting software keep a whitelist of approved senders so the mail will be delayed only the first time. All subsequent times will be instanteneous, just like you want. So I think most of your rant is misinformed, on the verge of becoming dishonest.
Anonymous on April 7, 2007:
You have it wrong.

greylisting isn't per email - its per sender ip address. if its the first time you see a connect from an ip, then it will get greylisted for a period of time (usually 5 minutes in most implementations i've seen). Subsequent connections from that ip won't be greylisted. The ip lists get cleared if no more mail is seen from that server for a period of time (usually a month).

This stops zombies and botnets from spamming you - they aren't repeat senders, and if they are they'll get listed in an real time blocklist by that point.

There are more important things to complain about than greylisting, which is fairly benign...


Anonymous on April 7, 2007:
email can still be instant. some services seem to get greedy- ive been using fastmail for years because of their singleness-of-purpose & subsequent lack of lag-
Anonymous on April 7, 2007:
Greylisting reduces our spam by about 90% pre-filtering, then we have a spamassasin after that. Most(?) greylisting implementations also automatically do whitelisting of sending servers after a successul mail, so.. the first mail from a sending server could be delayed, but after that, all goes through as normal.

You can also whitelist sending or receiving e-mail addresses if you don't need to greylist everything.

If you want to lessen the impact, you can just monitor the e-mail for a while before activating greylisting, so that mailservers you often receive mail from are already whitelisted.

Greylisting isn't a end-all-spam solution, and it won't work "for ever", but at the moment, it's extremely effective, and if implemented correctly, no big problem for either side.
Anonymous on April 7, 2007:
I happen to work for one of the arger providers of anti spam solutions, and yes, we will also start to (optionally) include greylisting. The way we implement is, you'll get the delay *once* for each combination of sender and recipient address. Meaning, the first time A sends a mail to B, A's mail server will receive a 421 "server busy" message. RFC states that the mail server should try again. If A is a legit mail server, it will try again and pass through. Now, what we do is keeping the sender and recipient information in a hash, and if that combination sends again we don't give the 421, as the sender and recipient have already been verified.
There are actually many other caveats with the use of greylisting, but the one you mention here would not be the main concern.
Anonymous on April 7, 2007:
First of all greylisting is a method implemented at the receiver side. If the receiver decides he can live with the delays it is up to him.

I have implemented greylisting for a med size ISP over 2 years ago and I also use it for myself. It has reduced spam by about 80%. (Activating greylisting brought me down from about 600 to 8-10 spam mails a day). And the biggest advantage of all is that you don't transfer the message, as the authorization is done while the sender is sending the address data (sender/receiver) saving lots of bandwidth. Thus you have no false positives and the receiver does not take over responsibility of the message that is put to some spam folder and probably deleted without further notice.
Classifying spam at the SMTP level also has the big advantage that you can reject it at the SMTP level and you do not have to create bounces to (faked) senders and harass them.

Before starting I created a list of "good" and well known mailservers (easily done from logfiles and mailboxes). These were fed to a DNS whiltelist. Servers in this whitelist will not have to pass the greylister but go through immediately. Also the triple had a sliding window lifetime of 40 days. Once authorized the authorization will last for 40 days and each message within that time frame will expand it further.
The number of old and rotten mailservers is <1%. Modern mailservers have reasonable retry times and most users will never ever have contact with emails from those servers.
The number of non-spam messages delayed by the greylister after about 2 month in production is <2%.
The greylister caught nearly each and every virus and was most helpful during virus outbreaks when there were no patterns available yet for the AV scanners.

As for the spammers' ways around: Having some spam traps and disabling greylisting for them and putting IP addresses hitting this traps in (24h) DNS blacklists gives you the time ahead to block retransmits that would pass the greylister.

\Maex
Anonymous on April 7, 2007:
Here's a concept:

email != IM

I don't know where you got that idea, but you're dead wrong.

The receiving of email from random sources on the internet is an untrusted, unauthenticated, process without guarantees who's administration is, at best, a tricky, slippery greased pig of a moving target that nobody had really been able to get their arms around.

I suspect that you have never had to administer an MTA/MDA for a large number of users (where large = >1000). If you had, you would know to being the email administrator is a losing proposition. I've personally seen everything from spammers who discover ways to exploit your MTA bog down a server until it is useless, to net vigalanties who decide that you're a SPAM source and DoS you're uplink to the point that your MTA goes idle to users who are never happy and refuse to take responsibility for their own problems (yes, Virginia, posting to USENET with a non-obfuscated email address or clicking the "remove me" link or putting your email address on a web page or corresponding with someone who clicks on attachments will cause you to be spamed excessively) to marketers who want to know why they can't get to your users... the list really is endless and *ANY* tool that helps bring things under control is a boon (regardless of whether you're willing to acknowledge the benefit or not).
Anonymous on April 7, 2007:
Surely this would be used in conjunction with whitelisting.
Anonymous on April 7, 2007:
You don't explain thouroughly the reasoning behind the theory.
Many spammers relies on botnets (ie, network of infected computers allowing remote use) to send spam. Botnets have the advantage of being very difficult to keep track of: spammers have access to network of hundred thousand of machine to send spam, a lot of these from various Internet providers. If you cannot track these computers, you cannot ban them, so you must process what they send.

The downside of botnets is that spammers :
* cannot really know how much ressources are available on the computers he is using, and usually these resources are very limited compared to a real server,
* cannot install heavy software (that is to say, complicated, or using up too much resources on its host). since the software used must be installed without its real owner being able to suspect it,
* cannot keep track of all the traffic that has been delayed: A piece of mailing software that must keep track of delayed mail would have to retain a massive amount of data to be sent later on, and that would make it use up all the resource on its host, rendering it ineffective and much more easily detectable.
As you can see, greylisting can be a very effective solution against spammers using botnets to spread there emails.

Now to your complains: you don't like greylisting because some servers delay emails too much. It's a configuration issue, not a flaw in the system itself, since it can be corrected. Also, a server which is busy checking a mail to see if it is a spam is not processing real email, thus slowing down delivery. By implementing greylisting, you allow a server to dedicate more time to process real emails, and avoid clogging its memory with bogus ones.

I will wait for your rebuttal, or hope that you will post an update to your article to reflect on what has been said in this message.

Thank you.
Anonymous on April 7, 2007:
Sorry

1) You don't understand greylisting

2) You lead your readers to a false belief that they, and not the system administrators, are in position to declare network policies.

3) The rest of discussion is immaterial.
Anonymous on April 7, 2007:
SMTP is not IM, that is why IM came around in the first place. SMTP one took days, now due to proliferation and cheaper pipes it takes seconds. What is wrong with greylisting a misbehaving mailer if its trying to send hundreds of messages to me at a time ? I get to decide how many messages are allowed per source IP since I am responsible for the pipe so I can balance all the traffic, not just accommodate some spammers. Even with or w/o greylisting the avg mailer throws away at least 90% of their mail (Assuming they have recipient verification, 70% for just bad headers).

Quit whining like some child user can can't send their myspace friends emails fast enough to share youtube links and think more like an adult that has to balance your traffic along with everyone else's data. You want instant send/recieve ? Own your own server and dedicated pipe. Since your traffic would never hit the avg greylist limits your email should go through just fine.
Anonymous on April 7, 2007:
Greylisting helps with blacklisting. I often had the case where certain mailservers would be blacklistet when they tried to deliver their mail the second time. This little delay makes a big difference in the amount of spam delivered, and not only in the amount of mails to be analyzed.

"spam filtering is highly sophisticated and extremely accurate", that's nonsense. The only way is to deny access from IPs from where no mails should originate, or deny service to malfunctioning mail servers.

A computer will never be able to "read" the contents of a mail and understand it. Using images for the advertising text is now so common, and computers have little chance to recognize this.



Anonymous on April 7, 2007:
You missed the point that greylisting can be configured to only greylist the first message from a specific person/domain. That way the first message is delayed, but everything afterwards is not. This works really, really well and email communications are still instantanenous (after the initial delay).
Anonymous on April 7, 2007:
You're absolutely wrong. my excellent Bayesian filter and blacklist analysis still lets 12x (twelve times) more spam through than the same system *with* greylisting enabled.

While spammers do foil filters, they can't afford to foil graylisting - it cuts their throughput by over 75% as each message much be re-queued in a timely manner. Most spammers never try again.

Legitimate mail always makes it through. Always. And when it gets through the first time, the greylisting agent marks the sender's information and *doesn't greylist anything from that address for 30 days*. It's only a one-time test on a per-server basis.

While filters set to agressive limits *always* bounce some email (lame!) greylisting has never been a problem for me, and any correctly configured and legitimate email server can deal with the delay request. If anything, I'd say turn your filters off and your greylister on if you want less spam and no chance of a false-bounce.
Anonymous on April 7, 2007:
It's pretty obvious you neither understand e-mail nor the spam problem. Like others already noticed, e-mail is not instant messaging. A slight delay is no problem at all.

Then there is spam. Although getting spam in your in-box as an end-user is extremely annoying, it's not the biggest problem spam is causing. Spam is threatening the entire email infrastructure. Email providers are not trying to "save money" through greylisting, they are trying to f***ing survive the barrage of incoming spam without having to resort to serverfarm the size of Google's.

And when you say "modern spam filtering is highly sophisticated an extremely accurete", you are a) wrong, spammers still find ways around it, and b) manage to emphasize the problem and yet ignore it: these increasingly sophisticated filtering systems need ever increasing resources to deal with the increasing sophistication of the increasing amount of spam.

Greylisting doesn't solve the problem, but it gives email providers a little breather before the have supersize their serverpark *yet again* to keep your inbox relatively free of spam.
Anonymous on April 7, 2007:
I recently implemented greylisting on my mail server, and, in fact, it works great at reducing spam. Any decent greylisting implementation will also allow you to whitelist good servers, and some even use SPF, in which case *you* can essentially tell the greylisting server to whitelist you. Maybe spammers will adapt to it, maybe not. Spammers count on being able to deliver instantly. When they can't, it really screws up their operation. That's a win to me.
Anonymous on April 7, 2007:
> The worst part is that email is no longer instant. In some cases, it's not even close.

Whoever said that email had to be instantaneous? Find me the RFC that says anything other than email delivery is a "best effort".

Sure, I've received some emails almost immediately after the author sends it. But in the past 20 years of receiving email, I've regularly gotten emails that didn't show up for 4 days or more.

But more importantly, what the author ignores is that an important aspect of greylisting is that it delays spamming hosts for long enough that they get detected and added to blackhole lists. Greylisting doesn't just help the server that performs it, it actually helps all the other servers that might receive spam from that host.
Anonymous on April 7, 2007:
For me it's taken greylisting along with other combined methods to finally put a dent in the flood of spam. This is from "spey", an open-source greylisting (and more) application on a fairly busy server, over the course of a bit over a month:

# speyctl stats
Counter Value
----------------------------------------------------------
malformed-domain 284949
malformed-address 9885
illegal-relay 900155
timeout 28914
greylisted 314116
accepted 145025
whitelisted 0
spoke-too-soon 156109
blackholed 3577307
Anonymous on April 7, 2007:
1 - it does work to reduce spam
2- email isn't SUPPOSED to be immediate. It never was.
3- Not ALL email is delayed. Greylisting can delay JUST the new senders never seen before with a completed delivery. Later, remembered IPs don't get delayed. Yes -- spam from legitimate MTAs continues. That is a lower percent than all spam.
Anonymous on April 7, 2007:
If you would rather get more spam - I am sure we can help you by redirecting ours to you

there is a RFC for re-tries on a busy - if you read the RFC (yes like reading legal briefs) you would see that.

I have been grey listing on 3 servers since august 2006, my wait period is 8 minutes for 1st message after that no wait. I have had 3 false positives in that time period and have blocked on a annualized basis 14.5MILLION messages that were spam.. These messages represent x minutes to x employees which would cost our company x hours/year - that x hours figure was over $25,000, not to mention the bandwidth and storage costs associated.

A google search shows you like to talk about stuff.. Talk about stuff you actually understand beyond the end of your nose
Anonymous on April 7, 2007:
Greylisting works. I am not sure how the writer can say the opposite unless he simply did not try it and he is choosing is adopting some purist point of view. The sad thing is that it probably won't work for very much longer. I am starting to see spam get through in the past few months.
Dan on April 7, 2007:
I don't understand where all these people get off on the idea that email is a form of communication. Personally, I don't think greylisting goes far enough. I've managed to stop 100% of spam from ever reaching me by smashing my motherboard with a hatchet. Sure, there's a bit of a delay before information gets to me--but whatever, if it's important enough I'll read about it in the newspaper the next day.

Back when I was your age, we didn't have email--all our information traveled by pony--and it got to us fast enough. These days, if somebody dies, you know almost immediately. Back then, it would take a month for the information to get to you--so you'd think they were alive for a month longer. Email is really killing people faster. Greylisting might keep them alive for another twenty-four hours.
Anonymous on April 7, 2007:
Greylisting is a necessary tool. Couldn't disagree with this more.
Slartibartfast on April 7, 2007:
Did this get submitted to the Society for the Promotion of Greylisting website or what? If my "instant" email takes 24 hours, THAT IS A PROBLEM. Especially if I have no way of knowing it will do that. Yes, it's configurable, so don't throw the baby out with the bathwater - but there is a problem, and it can't be dismissed as childish whining.
Anonymous on April 7, 2007:
++ to all the other messages.

I recently switched to a hosting service, instead of running my own machine for my domain. I've never gotten spam before, and my old server used greylisting. Now I get a small amount of spam without the greylisting.
Anonymous on April 7, 2007:
People don't understand the OP's point, which is that it is not the user who is implementing greylisting but his ISP, and he has no control over it. The mail admin at the ISP has no incentive to get your messages to you quickly and can easily set a 12 or even 24 hour delay for greylisting.

Alll this talk about using IM if you want to actually communicate (as opposed to what? send spam) ignores the fact that plenty of corporate firewalls block IP in worktime. IT also ignores the fact that plenty of people don't want IM turned on because it distracts them from working.
Anonymous on April 7, 2007:
I work in a freight brokerage office where email IS an instant communication tool. I receive well over 100 emails DAILY during the work-week, from interoffice communications alone. Instant messaging is not a reasonable replacement. Greylisting, as described above, would destroy our company. I don't mean that as a possibility, I can state it as a fact. I have to communicate instantaneously with customers and freight carriers numerous times a day. I have to send documents, contracts, pictures, and text regularly to people who don't operate their own servers. I should think that we are not the only office like this.
Anonymous on April 8, 2007:
I work for an ISP and we use a 5 minute (default) greylist along with almost a dozen other means to cut out spam. It was the single most effective way to cut out spam from anything else we tried. We cut out about 99% of our excess load by implementing a 5 minute greylist.

Before our grey list we moved from two email servers to six and our spam problems did not seem to budge. Our spam seemed to scale with the number of boxes we added and the delay in time to process messages only seemed to get slightly improve. The first day we moved to a greylist our average queue size dropped from about 1800 incoming messages to about 5 messages.

Now our email is instant whereas before our servers still had to process about 1000 spam message to every 1 legit message. Now we get about a 2 to 1 ratio and the filters do a pretty good job of taking care of the rest. I think it's a disservice to our customers to not cut down the load and wait time by not using a greylist.

Marco on April 8, 2007:
I've replied to your comments at the bottom of the article.
Anonymous on April 16, 2007:
I think graylisting is stupid. I agree, its a band-aid to a gushing wound.
I work for a pretty large company (think top 10 in fortune 500). we have employees. we can pretty much assume that at any point in time, one of those employees is waiting for an important email from a customer or vendor.

especially with all of the other options out there. I"m sorry that ISP's have lost the war against spam, but greylisting isn't the answer.
Anonymous on May 3, 2007:
Marco, are you real? For a lot of people email has NEVER been instant - never heard of POP3 only connecting every 15minutes?
I adminster a small server serving around 200 domains (1500 mail boxes) and I implemented Greylisting (via the excellant Community version of Endian Firewall -(http://www.endian.com)) and saw spam emails fall by 80%. Along with blocklists (sbl-xbl.spamhaus.org and dul.dnsbl.sorbs.net), Spamassassin, some manual blacklists (domain used by spam bots), some HELLO filtering (ever looked at messages coming from servers with HELLO of friend?) we're now getting less than .25% spam ratio - it works for me (and my clients love it)! It might not work for everyone (no two networks are identicle in terms of mail characteristics) but I think you'll find it works for most
Anonymous on May 7, 2007:
Your entry is obviously that of a ranting board admin, and or user who got the short end of the stick when it comes to the one downside of greylisting.

Your blog says two things that I am going to plug in the crap detector for.

One, you say that this is only for the mail hosts benefit, not yours. It reduces spam, a lot! I turned it on, on my own personal email server, why? Too much spam. I did it cause I don't want spam. Furthermore, the hosting company I work for, we implement it as OPT IN! It is only turned on, when our users complain of spam, and we tell them the advantages of greylisting, and the caveats of it as well. Our system also lets each email account Opt out(if their domain has opted in) at their own choosing via their webmail options. We use Smarter Mail, and anyone else using it, is using software that has the design of letting the user enable greylisting for their benefit.

Now you also said "It doesn't work.". That right there, makes this whole blog entry crap. It says, you have never used it, you have never investigated the success/failure of greylisting. You just got ticked, and blogged as many other blog, with nothing but ignorance and opinion. Greylisting WORKS! One one of our servers(opt in only, remember, so the number of users using greylisting is relatively low on this server). In the last 5 minutes, of users/domains using greylisting, our server has let 0 emails through, and 30 were greylisted. Most of these 30, if not all, will never resend the email, as they are most likely spammers. Those that do resend will get through fine, and the server will then whitelist the IP, so that server will not be greylisted from this point forward.

The white listing methodology works great, as it takes a short amount of time for our servers to learn the servers our users talk to. And most of the mail ends up going on with no delays, as most email users repeatedly speak with the same users.

Simply put, every single customer I've talked to has found it to be worthwhile to have a one time delay, for the amount of spam reduction greylisting provides.

P.S.
Implement a spam trap(honeypot) with greylisting and RBL's, and there is almost nothing left for SpamAssasin to worry about.
Dan on May 7, 2007:
"The one downside of gray listing."

"Aside from all the death it caused, was the plague all that bad?"

"The only disadvantage of selling your car is that you don't have a car anymore."

Since reading this article I've paid attention to the number of times something I was doing relied on near instantaneous email delivery. It happens about twice a week, usually for confirmation emails for something.
Anonymous on May 13, 2007:
It has been interesting reading ALL of the comments.

I am not qualified to offer an opinion on this matter, however, I would like to ask how I can find a decent list of the best and worst SMTP servers regarding spam emails, if possible. Our business relies on actually making sure that companies in lots of foreign countries actually get their emails without being unjustifiable classed as spam.

For instance, as an example. smtp. aol. com emails sent from a company in England, UK to a company in England are classed as spam even when they just contain a 'test' subject line and no other text. However Freedom2surf smtp does not fall into spam when sent to the same company, I suppose it is irrelevant where the original company is based as it depends on the smtp server and its spam reputation that counts good or bad against it.
For instance, If i use authsmtp at streamline.net will that be treated as a spam source or will it be treated more fairly?
To summarise, I basically access to a list of the most 'professional' and/or most 'unprofessional' smtp servers.

Please advise if you can.
bye
Anonymous on June 5, 2007:
Wow... Complaining about greylisting and you've got SO MANY of the details wrong... Perhaps you should get a little more familiar with the topic at hand before you go railing against it.

Greylisting will only delay the first email sent to a server and subsequent emails from good servers are not delayed after that. Get the details right.
Anonymous on July 6, 2007:
Reading all the comments above makes me wonder. So many claims that grey listing works for them but I see an obvious problem with greylisting from technical point of view.

What would happen to greylisting server if spammer send 2+ spams to every address? Will greylisting prevent such spam from coming in?
Anonymous on July 10, 2007:
LOL... this discussion reminds me of bayesian filtering a few years ago and how "effective" it was.

In an escalating war, greylisting is just another temporary solution to a much bigger problem. Once the botnets have rolled over to new greylist-countering software, it's time to move on.

I think there are signs this is already happening....
Anonymous on August 3, 2007:
Greylisting is wrong... but absolutely needed. Even with quite agressive spamfiltering software, a lot of spam slipped through every day. Greylisting reduced that about by around 90%.

While I hate waiting up to an hour for an activataion e-mail, I would prefer that instead of deleting spam from my inbox several times an hour.

People I regulary e-mail with are whitelisted now, so that's no longer a problem.

I think the autor is way off here
Anonymous on August 15, 2007:
The authors follow up shows he is still a tard and still doesn't get it. If his big sites (which I had never heard of - guess they ain't that big) don't have RFC compliant mail servers then he should fix that. You send it, you wait 5 min, you send again. What part don't you understand? (Well, obviously, the RFC.)
Anonymous on August 16, 2007:
What a strange crusade.

What's missing is this: false positives in lesser anit-spam strategies have fueled more delay in email delivery than grey listing ever will. And false positives non-existant in my admittedly small universe of it's implementation.

In that spirit, to Slartibartfast ... Mostly Harmless

In the end, it's up to the mail administrator to set policy sending/receiving as they are the ones who have to fight the fight to continually communicate with various mail servers based on their user's demands.
Anonymous on August 27, 2007:
Greylisting in hands of an idiot-administrator can kill (first I started to laugh to death... then I started to cry...)

(email parts are cut out for w well-known reason)

2007-08-26 11:52:13 H=elf.retsat1.com.pl [195.13.38.2] sender verify defer for
<******@retsat1.com.pl>: response to "RCPT TO:<******@retsat1.com.pl>" from
elf.retsat1.com.pl [195.13.38.2] was: 450 <******@retsat1.com.pl>: Recipient
address rejected: Greylisted for 552383421 seconds (see
http://isg.ee.ethz.ch/tools/postgrey/help/retsat1.com.pl.html)

Now take a calc-tool and compute how long is this. And don't tell me that greylisting doesn't suck! it sucks - it has nothing in common with any filtering, it only causes mail queues grow - and only who suffers is fair sender and fair mail server.
Anonymous on October 5, 2007:
Greylisting has cut out 90 percent of my incoming spam. Leaving 8 percent for my Outlook junkmail filter to identify based on content analysis. Leaving only 2 percent for me to delete out of my inbox. Then I peruse the junkmail folder to check for false positives. And whalla, spam is no longer a problem for me.

I think the author is having a tantrum.

Additionally, I can whitelist any IP address, sender of email, or IP address blocks, which bypass greylisting and thus bypass delays. And if the sending mailservers would be RFC compliant in the first place, and not overloaded, there would be no significant delay. Once the sending mailserver has resent successfully, it is automatically whitelisted for more than a month. So during that 35 days or so, there is no delay at all. The database has a fixed capacity. And as long as I get another message from the same person and through the same IP address, the 35 day period automatically gets extended, resulting in no delays except for senders who send once every 40 days or more.

Again, if the sending mailservers were not overloaded, and if they were configured properly to RFC specifications, a 5 minute delay would be trivial.

As far as spammers figuring out how to bypass greylisting, they havn't done it yet and greylisting has been in use for at least a few years. So it must be too costly for them to configure their spammer servers to be RFC compliant instead of configured as fire and forget servers. High value spam makes it through. But for me that is only about 10 percent of the spam.
Anonymous on October 31, 2007:
I find it difficult to believe that anyone replying to this blog post actually works for a large company. We are a medium sized online services provider who trialled greylisting for a one month period - and although the resources it saved on the servers were valuable they were quickly offset by the cost of customers calling our helpdesk to ask why they were experiencing delays and non-delivery problems with free email providers.

Apparently Yahoo and Google don't think greylisting is too hot, because they never retry delivery. If greylisting breaks delivery from enormous sites, how would we ever find out about the problem with smaller sites? Only when our customers complain that emails to them aren't getting through.

I don't believe that greylisting represents a good enough value proposition for anyone who has to provide free support to their users - generally, the most ardent supporters of greylisting are also hopelessly naive when it comes to foisting your problems onto innocents, and usually exhibit the same poor attitude toward other antisocial email server behaviors like "backscatter" and "challenge response".

Having said that however, I agree with those saying that the author's expectation for emails to be delivered instantly is flawed. I'm a veteran from the days when most email traveled by UUCP, being polled once per day. In those times an email could take a week to traverse the world.
Dan on October 31, 2007:
I agree with the previous commenter who mentioned emails that could take a week to traverse the world. Back when I was a kid, if you wanted to carry your phone with you in your pocket, you got a really long phone cord. You didn't complain about reception cutting out. Phones aren't meant to be portable and email isn't meant to be instant. That's how it was and how it always should be--and anybody who tells you otherwise is a whiner, a lunatic, or a liberal.
Anonymous on December 28, 2007:
Technically, greylisting works. Philosophically, greylisting shifts the burden to the sender trying to deliver the message. If ALL mail servers used greylisting it would dramatically increase the spammer's cost of doing business becuase they would need to send each SPAM 5 or 10 times. Furthermore, your premise is wrong... once an e-mail gets through, it is typically whitelisted and will not be subjected to delay.
Marco on December 28, 2007:
"Philosophically, greylisting shifts the burden to the sender trying to deliver the message."
That only makes sense if the message's speedy delivery has no value to the receiver. That's true for spam, but not a lot of *wanted* email.
"If ALL mail servers used greylisting it would dramatically increase the spammer's cost of doing business becuase they would need to send each SPAM 5 or 10 times."
Do you honestly think spam is teetering on the brink of being practical because sending emails is so costly? Spamming could get 100 times more expensive and it would still be incredibly profitable.
"Furthermore, your premise is wrong... once an e-mail gets through, it is typically whitelisted and will not be subjected to delay."
You didn't read the article. (Did you come from Digg? That's usually a dead giveaway...)

What about receiving confirmation emails from websites you've just registered with? What about order confirmation emails from an online store that you haven't previously bought from?
Anonymous on January 20, 2008:
Who said email wasn't supposed to be immediate. When I first started using email, back in the early 80's - it BLOODY WELL WAS THEN!!!!
Anonymous on January 28, 2008:
It's obvious that most of the posters here are admins, I'm an admin, 20 yrs exp, been there got the t-shirt. The problem with greylisting is that I have seen emails across multiple customers come back undeliverable and not just from Exchange Servers but from ISP's. How many people does this affect if that can happen, we're talking about a lot of email floating about waiting to be re-tried and it seems a lot is not getting through. The complete randomness to user emails getting through is very frustrating.
Anonymous on January 29, 2008:
If the author is so smart. Why not proposing his best shot at a solution ? Greylisting work, it keep 95%+ of mail traffic OUT of my servers so they can process REAL mail rather than spam. If I was his sysadmin, i'd RCPT-whitelist all incoming to HIS mailbox so he'd come back to me repentent and askme to re-enable greylisting for his box. No one ever said that greylisting is perfect (really, smtp IS broken) nor that its the ultimate solution, but since we tend to keep what'S work the best well ther you have it. Lusers... let'em bitch an moan while we keep their mailbox safe.
Anonymous on February 9, 2008:
So to be honest, there's a lot of comments here, and I haven't read them all. I will tell you my perspective:

1. I have, over the past 15 years, been a System & Network Administrator for an ISP, an active Internet user, and a "hobbyist" internet user. I _willingly_ chose to implement a greylist solution on my personal mailserver because it was _simple_ and _effective_. One of the things that spam mail relies upon is the ability to dump thousands of e-mail a minute at almost no cost to the spammer. _queueing_ a message actually costs something (disk space, retry time, etc). Even if a spammer is using infected computers to generate the spam, it's going to slow down the process _greatly_, and reduce the number of successful spam messages that will get sent.

Good greylisting remembers e-mail addresses that you communicate with on a regular basis, and lets them through instantly.

Instant gratification is overrated. I can live without it. For me, 15 minutes is not too much time to wait to receive an e-mail, especially when weighed against the hundreds of spam messages I'd have to sift through manually.

We're all entitled to our opinions. You've shared yours, and I've shared mine. I don't think either one of us is "right" for all cases.

-PT

Anonymous on February 26, 2008:
I just heard about greylisting last week. You want to know why? Because I found out the e-mail I sent and e-mail sent to me was not received. The explanation given to me (by my IS)was "greylisting". I asked my ISP how I can insure that my e-mail is delivered to me and delivered and in a timely fashion and I was told the following. "Basically, you need to ensure that all email sent to you is sent through a correctly configured email server that adheres to the RFC regulations regarding email servers."

HAH!! I thought. Like that is in my Control!!

In principle it seems fine, but it only works with it is configured correctly and continually monitored.

Ultimately it seems pathetic to me that "I" have to pay the price in the war SPAM. SPAM is indeed a problem, by this "greylisting" solution is "illconceived" "shortsited" and as of late "wasteful" of 'my time' which ultimately cost 'me money'.

Add a comment

Anonymous. Would you like to register or log in?
Comments are approved before posting. No HTML.