Improving deliverability best practice

Part 1. How ISPs spot spam in the suspect email line-up

As this is my first post on SmartInsights I’d like to start with a hello; I hope you find this first post helpful! I’ve long been a reader of the SmartInsights blogs and I’m delighted to be a new regular SmartInsights contributor. I will be writing about best practice to get better results from your email marketing through optimisation of creative and targeting and deliverability.

Let’s kick off with deliverability. In the line-up of suspect emails, ISPs aim to can the spam and let the innocent go free. To understand the forensics that ISPs use to spot the guilty, you need to understand the habits of the guilty. In this post I look at the tell-tell signs of a spammer and how you can avoid looking like one.

How do ISPs identify spam

Spam is defined in the eye of the beholder. In the case of the ISPs and webmail services, they define spam as emails that users don’t want. This is a wide definition and a long way from any legal definition of spam.Spam falls into categories of emails to drive traffic for sales conversion, malicious emails and phishing emails. Malicious emails use social engineering to persuade the reader to take an action that will result in a virus infection. Phishing emails, again using social engineering, to trick readers into giving out their account details for a service whose value can be misappropriated, such as an online bank logon.

Its well understood the spam model works on the basis of sending a gargantuan amount of email so that even conversion rates of fractions of a percent provides an ROI. The fact spam exists says that there is an acceptable ROI.

A very low conversion rate provides an ROI to spammers as they have very low costs. They pay very little for data or the computers used to send the spam and are far from being customer centric in approach. The data and resources are mostly stolen or high jacked, through use of botnets.

Botnets are huge networks of virus infected computers that are remotely controllable without the owner’s knowledge. The infected computers are commonly called zombie computers. The botnet controllers can remotely execute activity such as sending emails across thousands of zombies, giving a large scale capacity. The illegal activity has even tiered into business services, with some gangs managing botnets as a service and renting them to other gangs who want to send spam.

Spammers steal and harvest their email addresses. Harvesting occurs by scraping email addresses from websites, forums and other publically accessible internet resources. The process is not unlike the search engine spiders that crawl the web for content, just these spiders crawl for email addresses. Email addresses are also harvested from individual user PCs by use of specialised viruses. These viruses quietly sit on your PC and look through your data for email addresses which they then invisibly send back to the spammers.

Since spammers cast their net very wide, volumes are high and delivery rates and engagement with their emails are low.  So these are the factors that ISPs use to spot spammers:

• Poor list hygiene. Often with limited ability or interest to receive and process bounces from botnets, bouncing addresses stay on spam lists.

• High spam complaints, users clicking the junk and report as spam button in their email client.

• Very low engagement. Low open rates and a high number of deletes without reading.

• No user reply emails in response to a spam email.

• Campaigns sent across hundreds if not thousands of IP addresses. The IP addresses rapidly change and come and go.

• The email volume from each zombie IP address does not have smooth flow. It shows high peaks of activity.

• Incomplete setup of technical criteria including DNS, reverse DNS, SPF and DKIM. (See Deliverability.com) for discussion of these.

• Sending from a dynamic IP address, such as those allocated to home ADSL connections.

• The spammers have limited interest and ability to provide personalisation. Ability to personalise extends little beyond assuming the text before the @ symbol is your name.

To not be treated as spam means to not look like spam with the above characteristics.

In the early days ISPs used content to filter spam, words such as Viagra were obvious choices. These techniques were crude and easily circumvented. The ISPs are clever people with large resources and have been moving closer and closer to measuring the metrics that are fundamental characteristics of the spammers’ business model. This is shrewd as it is something that spammers can’t fake.

So, what can I do to avoid being identified as a spammer?

The factors today that are important to inbox placement are low bounce rates, low complaint rates, high engagement, correct technical setup, content validity and consistent email volume flow.
These metrics are continuously measured and reputation is assigned to a senders IP address and domain name. It takes two to six weeks to build a reputation and it can be damaged in hours. Once you have a good reputation you can’t consider your job done. Its not static, but continually being updated by the ISP. You are judged by not what you did two months ago but in the last days and weeks. Inbox placement can even change in the middle of a large campaign send.

Within limits no one metric puts you into junk. If you have very low spam complaints and very low bounce rates, then you will likely reach the inbox with a lower engagement rate than if your spam complaint and bounce rates are higher. This makes it hard to answer a singular question such as “what is a safe bounce level?” Certainly over 10% is pushing your luck and under 3% is very good, between 3% and 10% are shades of grey; it will depend how well you are doing on your other metrics.

Will we ever see an end to spam? Only if filtering is so good that even a miniscule conversion rate is not enough to provide an ROI. Fundamentally the spammers’ model relies on high volume, low conversion. As ISPs are measuring almost all these metrics its hard to see how spammers can fake it and get around ISP filtering. However, spammers have proved to be hugely resourceful, they have even found ways around CAPTCHAs.

Cisco IronPort spam volumes show that spam has in fact reduced in recent months

Where might the ISPs go for more filtering metrics? The next logical step to be added to the filtering process is information about your social graph. If you are connected to someone through a social network or maybe connected two or three steps removed, then it is a vote to say the sender is not sending spam. This is an automated extension of personal whitelisting. The level of connectivity might in the future be aggregated, like spam complaints, across multiple recipients to obtain a connectivity score. A high social connectivity level would imply a well-targeted email campaign and thus not spam.

In my next post I’m going to look at the tools and metrics you can use to monitor and manage your email deliverability.