We criticise the sender authentication mechanism and show why SPF is insecure However, solutions being implemented as a quick and dirty patch to the notoriously insecure and broken SMTP protocol are introducing more threats, instead of fixing the problem at hand.

With spam, phishing and joe-jobs becoming more and more of a threat for the whole Internet community, sender authentication has lately become an important issue for electronic mail. However, solutions being implemented as a quick and dirty patch to the notoriously insecure and broken SMTP protocol are introducing more threats, instead of fixing the problem at hand.

The fact that the SMTP protocol and the whole system of Internet mail is completely unprepared for fighting everyday threats is nothing new. Everyone working deep with electronic mail knows, that every system that attempts to make the protocol more secure is, in reality, a quick and dirty patch, and that the only way to successfully fight spam, phishing, joe-jobs and such, is to devise a completely new e-mail protocol.

However, the experiences with introducing major changes in the Internet infrastructure have been dire straits for all parties concerned. Not only is it hard to reach agreement on what solutions to undertake, but when a good solution is devised, making everyone switch is almost impossible. DNSSEC is a good example. Despite the fact that the DNS protocol is unsafe and requires a general overhaul, there has been very little success in introducing more secure alternatives.

No wonder, that various solutions are devised to patch the SMTP protocol in order to make it more secure, the key issue being sender authentication. However, organisations and companies undertaking such efforts resemble a cat with its tail on fire jumping into a well. They realise that this has to be done quickly in order to fight the abuse, they realise that it's necessary to use as many current mechanisms as possible in order to make transition smooth, but they're not thinking of the consequences it might bring.

SPF - the good, the bad and the ugly

A sender authentication solution that has gained not only huge popularity, but also a lot of implementations and is being used now by huge, popular mail servers, is SPF. The base term for this abbreviation was Sender Permitted From, but it has been changed with the popularisation of the project, to Sender Policy Framework.

The idea of SPF is quite simple. A method was devised to let mail servers know, whether a given server attempting to transmit mail with a given envelope sender address is in fact authorised to use the domain name in this address. Spammers have been using fake sender addresses for years, usually spoofing existent or inexistent accounts at large free mail services such as Yahoo! or Hotmail. The SMTP protocol makes such spoofing a piece of cake, since all a spammer needs to do is supply any sender address and mail will be accepted by the recipient.

SPF is a method based on the current DNS infrastructure. The DNS records for a given domain hold information on which servers are authorised to use this domain in the sender addresses. This approach seems logical. A Hotmail user will most probably be sending his mail via the Hotmail SMTP server and not a Yahoo! SMTP server (although there is no logical reason, why he should not use a Yahoo! server for this purpose). Using the DNS infrastructure makes implementation of SPF easy, as no additional mechanisms have to be devised.

The good

SPF addresses the issue of sender spoofing quite accurately. If a domain publishes its SPF data, and the recipient uses SPF to check sender authentication, no mail with spoofed envelope sender domain name will be received by the recipient. If for example eBay has an SPF record for ebay.com (which it does), and a phisher attempts to send a lure to a nowhere.com mail server which is SPF-protected (i.e. checks whether the sender is authorised to use a given address), no nowhere.com users will receive phishing lures with a fake ebay.com envelope sender address.

However, it's plainly visible, that this scheme will be successful only if all the domains used in Internet mail exchange will publish SPF records and only if all the receiving SMTP servers will check SPF authorisation. However, not all domain administrators are aware of the need to use SPF (and even of those aware, not all of them want to use it) and not every SMTP software is able to use SPF for mail protection. There are solutions extending the capabilities of currently used mail servers, but in many cases these are either patches, proxies or add-on tools, not built-in capabilities. This makes the administrator's job a bit more complicated when implementing SPF protection.

All this makes SPF quite ineffective in elimination of mail address spoofing and the underlying idea is based on a bad approach: in order to be protected everyone would have to be using SPF, and not any other mechanism. And SPF is not even an Internet standard.

The bad

In reality, SPF does not protect the end-users against forgery. It does not protect against joe-jobs or phishing at all. The reason is simple. End-user software (MUAs, Mail User Agents) displays the sender address using the contents of the From: header, not the envelope sender (which is kept in the Return-Path: header in received mail). A scammer may supply a valid envelope sender address and a fake (eg. support@citibank.com) From: address in the data part of the mail. An SPF-protected mail server will be forced to accept such mail. If not, it would automatically also reject all mail from mailing lists. And the end-user still sees the fake address in their Outlook Express for example.

The other problem with SPF is that despite the fact it has been originally thought of as anti-spam protection, it's highly ineffective when fighting spam and should not even be called an anti-spam measure. It's an anti-spoofing measure which is supposed to protect a domain owner against their domain being used for malicious purposes. It can protect the recipient against receiving mail from spoofed addresses (but only envelope sender addresses). However, it does not and will not stop spammers from filling our inboxes with crap.

In order to understand why, we must first see what the state of SPF implementation looks like. Let's, for the purpose of this article, take a very optimistic assumption and let's say that 50% of all domains used in Internet mail exchange publish SPF data (although it's more like 10% at the time, figures from the end of 2004 by CipherTrust have shown only 5% of sender domains publish an SPF record) and 50% of mail servers are SPF-protected.

The protected servers might choose two different approaches for domains which do not yet publish an SPF record. They might either accept the mail from such domains or deny it. Since the other half of domains (according to our optimistic figures) does not publish such records, it's logical that if the protected server denies mail from such domains, it risks that half of its mail will be lost. No mail server may afford this, and it is wrong to force someone to publish an SPF record in order to be able to communicate.

This breaks the whole idea of Internet mail and makes SPF monopolistic. Also remember, that all this is still based on our optimistic figures; in reality denying mail from domains with no SPF record would be suicide for a mail server administrator - he could just as well turn the server off completely. So, let's assume that the other approach is taken and mail from a domain with no SPF record is accepted.

Now it's extremely simple for a spammer to send spam to such a server. All a spammer needs to do is find a domain, any domain, with no SPF record and use that domain for their sender address. This mail will be accepted by the mail server. Another thing a spammer might do is use a special <> envelope sender address, which according to RFC must be accepted (and it contains no domain name at all). The final thing a spammer might do is buy a domain and publish an SPF record for it, allowing everyone to use it for the purpose of faking sender addresses. The aforementioned CipherTrust survey from 2004 has already shown that over half of all the mail received by SPF-protected servers from domains publishing SPF records was... exactly spam!

sum:

0 €