Email Testing Equilibrium

July 7th, 2009

We talk a lot about the great benefits of implementing testing strategies with our email efforts. I’ve made multiple references in various posts, and so far have had two dedicated posts on the topic: A/B Subject Line Testing and Email Marketing Testing . Marketers (not just email) LOVE testing. Combine with some juicy analytics and we’re entertained for a long time.

Nothing has changed with the fact that testing should be implemented and will inevitably improve your email marketing efforts. I do feel, however, that there can be a risk of over-testing, or ‘testing burnout,’ if you will. This mostly comes down to available resources a company has and the expected return on marginal testing programs. If we put 10X more resources into testing and only realize an x% increase in sales, our testing program could be producing negative returns.

This has a more costly effect on smaller companies, as the scale of return is much smaller. [The marginal return from an x% increase in open or click rate is much larger for a big company with a list size of 5,000,000 vs. a small company with a list size of 5,000.]

For that small company, testing is indeed important. They need to make improvements to their email marketing efforts and increase sales just like every other company. But at some point a negative return is realized.

Example

  • Let’s say a small e-commerce site selling backpacks has a list size of 15,000. Their monthly promotional email brings in $1,350 on average. [25% open, 12% click, 5% conversion, $60 average order]
  • This company creates a testing plan that will require an additional 3 hours per month of company resources.
  • After the test, they increase their metrics to: [28% open, 15% click, 5% conversion, $60 average order]. In this case we see a $540 increase in revenue. Perhaps a decent result for the small e-tailer. Their gross testing return was $180/hour.
  • Now let’s say they create a testing plan that is more robust and requires an additional 30 hours of company resources each month. (compared to no testing)
  • After the new testing program, they increase their metrics to: [30% open, 19% click, 6% conversion, $62 average order]. In this case we see an $1,830 increase in revenue. Their gross testing return was $61/hour. Depending on their margins and several other unknown factors, this may or may not be a profitable scenario

These same metrics with a big competitor would have a much different result. e.g. If another company had a list size of 750,000, their marginal return on the last scenario would be $91,530 with a $3,051/hour testing return. Probably quite favorable.

So where do you find that testing equilibrium? It comes down to the unique situation of each business. Regardless of size, start small with your testing program and work up from there. Pay close attention not only to the increased results of your tests, but the amount of resources that go into your various testing programs. Time for different content, designers, approval, segmentation, deployment, review & analysis etc, can add up when you are introducing complex testing strategies.

In the end, you should test to see what testing portfolio is optimal for your email efforts.

Cheers,

Forest Bronzan 

Questions or thoughts? Leave a comment or feel free to shoot me an email

Twitter Badge - Forest Bronzan

The Email Shotgun, Rifle, and Blow Dart

June 1st, 2009

We often discuss the shotgun and rifle approach to marketing, and with email communications it plays an extremely important role. The premise is straight forward, but the approach you implement will have significant implications on the effectiveness of your email efforts.

Every email marketer you ask will likely balk at the shotgun approach and jump straight to a rifle strategy. While in theory this is ‘optimal’ – it’s not always that black and white. Additionally, many email marketers pick up the rifle, put on their sniper costume, and call it a day. While this of course is a great starting point, there are opportunities to have a greater impact with multiple approaches also utilizing a high powered blow-dart.

Let’s Examine the Shotgun, Rifle and High Powered Blow Dart

1. Shotgun Approach

Basics: Your email communications are broad and promotions are developed for a wide and diverse audience. In other email terms; you are sending one general email to your entire list with no segmentation and little or no personalization.

Why it’s Bad: For starters, you are not leveraging the email channel. With available technology, you have the opportunity to create targeted segments and dynamically insert personalized data. With a ‘batch and blast’ approach like the shotgun, everyone receives the same email and promotion.

However: What if you don’t have any data (outside of email address) to segment? Many companies starting off with email don’t implement all the best practices for list acquisition, not to mention proper tracking of email engagement for segmentation purposes. In this case, you may be limited to a broad email – and this creates an opportunity to quickly get important information about your subscribers for more refined communications in the future.

For starters:

A. Make sure you have navigation in your emails — which will help you segment based on click activity.

B. Conduct A/B subject line testing to better leverage that big send.

C. Create an email preference center to get more data from you subscribers.

D. Implement some email segmentation just from email engagement.

We of course want to strive for targeted, ‘rifle like’ communication with our subscribers – and when possible this should absolutely be implemented. But In the event you are unable to do this from the start, do begin collecting information that will allow you to make better use of the email channel in the future.

2. Rifle Approach

Basics: Your email communications are focused and campaigns are directed to a select target audience/segment. Promotions are extremely relevant and timely, and you strive to create the impression of a 1-1 communication.

Why it’s Good: Instead of sending one message to 100,000 subscribers, we may be sending 20 messages to 5,000 subscribers based on strategically defined criteria. Here we place our subscribers into meaningful segments and serve them relevant content. This may be done through individual messaging or utilizing dynamic content to execute our segmentation and content strategies.

This will have tremendous impact on the success of your email efforts, and should be implemented whenever possible. This is an extremely summarized description, but the basic premise is that we want to provide relevant and extremely targeted communications and promotions to our subscribers.

3. High Powered Blow Dart

Basics: The blow dart picks up where the rifle left off. With the rifle approach, we are creating targeted segments, developing relevant and timely communications, and creating a better experience for our subscribers. This may or may not include advanced personalization within the email, but we like to make sure it goes that extra step and utilize a blow dart, if you will, to truly strive for a 1-1 communication.

Here we make sure we utilize dynamic content when possible, and further personalize the communication by inserting data that is relevant.

Example of dynamic content from American Advantage

Example of simple personalization from Wells Fargo

Summary

1. Create targeted email segments and focus your content and promotions. Become a sniper and provide extremely relevant and timely communications.

2. If for whatever reason you don’t have data to segment, then start on a basic level with email engagement, reviewing your acquisition process, and upgrading to a better email platform if needed. There is no reason you can’t segment based on some criteria (historical opens, click activity, list origination, products purchased etc)

3. Create an even more personalized experience and utilize dynamic content, personalization, and even more specific targeting.

Cheers,

Forest Bronzan

Follow me on Twitter

Email Marketing Metrics – What to Watch

May 21st, 2009

Marketers are obsessed with metrics. They provide key information to help us improve our marketing efforts, and a lot of entertainment; a Friday night with a bag of popcorn, cold beverage, and some fresh analytics reports sounds like a great night indeed.

It is, however, important to focus on the right (or best for a given situation) metrics – and talking about email marketing, it’s important to not measure success on one metric alone.

Different goals call for different metrics. What is the focus of your email campaigns? Is it a product recall announcement, brand-building newsletter, order reminder, special promotion, announcement of a new store?  Each of these would prioritize a different set of metrics.

For the product recall example, you’re probably not concerned that your sale conversion rate is near non-existent, but do want to make sure the message was opened and read. For your big email promotion email, it does not matter much if everyone opened the email, but no one made a purchase.

Below is a starter list of email marketing metrics:

1. Open Rate: [ratio of unique opens to total delivered] Don’t get too caught up here. Some companies base their entire success on the open rate. Do monitor this metric for every send, but don’t make it the only metric you monitor.

2. CTR (Click Through Rate): [ratio of unique clicks to total delivered or unique opens] The CTR can be helpful, especially when you have different links and you monitor the CTR for each. Knowing that your ‘Basketball Shoes’ link had a CTR of 39% while your ‘Soccer Shoes’ was 10% can be helpful.

3. Conversion Rate: [ratio of a 'goal' conversion to the total click through, or opens] If you are an e-commerce site, the conversion rate from your email initiatives is pretty important. In this case, the percent of visitors from the email that made a purchase. If you find that Promotion/Campaign X converts at 10% while another at 1%, you have some valuable data to work with.

4. Campaign Revenue: In addition to the previous metric, how much revenue did a campaign (or group of campaigns) bring in? This of course can be monitored for different email sends, or based on time to see how much money the email channel brings by month, quarter etc.

5. Bounce Rate: [ratio of bounced emails to total sent] This metric is universal no matter what the goals of your campaign are. Increasing bounce rate = look into the issue quickly. While not really as sexy as the conversion rate or total revenue, it is very important and will affect other metrics.

6. Delivery Rate: [ratio of delivered emails to total sent] See comments from Bounce Rate — the same apply, but here: decreasing delivery rate = look into the issue quickly.

7. Opt-Out (or Unsubscribe) Rate:[ratio of opt-outs to total delivered] Clearly we want to keep this one low. We will inevitably have some occurrences of subscribers no longer wanting our emails, but we need to monitor this metric to make sure it doesn’t get out of hand.

8. Link Revenue: Going off of our campaign revenue, let’s dig a little deeper and see how much money each specific link in our email is generating.

9. Time-Delay Metrics: What percent of our opens, or clicks, conversion etc occur on the day the email was sent? Within 3 days? Within 7 days? 2-4 weeks? On a somewhat related note, when providing reports to clients, we like to do so in delay, to account for that delayed engagement.

As mentioned at the beginning of this post, some metrics won’t be a priority for all emails. It’s important to know what your goals are for a particular campaign and for your email channel at large. This will let you identify and prioritize your key performance indicators. This data will then allow you to make meaningful decisions and improvements to your email efforts.

In a future post, we will discuss the importance of creating your own historical benchmark guide.

Follow me on Twitter

Let's talk email!

Call 909-363-1455 or email us to start improving your email marketing efforts.

Name
Phone
Website
Email