Blog Box

One of our clients regularly sends out tens of thousands of emails to their subscription database. We recently conducted an a/b split test (multivariate testing) on one of their emails to see if we could see any variances in the behaviour of people reading the emails.

A/B Split Test or Multivariate Testing?

The the email system that Brown Box employs to send out html email campaigns for our clients, we are able to monitor a few things. Some of the important statistics include the read rate, unsubscribe rate and link rate (which links in an email are clicked on). When an email is sent out - we want your read rate to be high, and your unsubscribe rate to be low. Apart from making sure that the content is useful to your customer and relevant there are few things you can do to see how useful your email actually is.

One thing you can do is multivariate testing. That is, sending out slightly different versions of the email to your database. Each email generated is either version A or version B of your email. That way you can measure the possible differences between things like link text, email titles, images and copy. In this case we changed just one thing: the email title.

Are Email Titles Important?

Naturally an email that has great information in it will be captivating and hold the reader’s attention, but to get the reader to open the email is itself a challenge - especially when you are sending out emails about your product/service on a regular basis. With this recent email we had two titles.

1. Petbarn : Pet Poem Competition; or
2. Petbarn : Poetry in Motion

Version 1 was quite obviously a poem competition email whereas the title of version 2 didn’t immediately let on about the contents inside. Would this make a difference?
With the email send being in the tens of thousands, here are some percentage statistics to show you the difference (if any) between the two emails.

1. Pet Poem Competition
   • Read Rate: 14.69%
   • Unsubscribe Rate: .44%
2. Poetry in Motion
   • Read Rate: 15.2%
   • Unsubscribe Rate: 0%

Quite simply - there are more reads for Petbarn : Poetry in Motion and less unsubscribes than the version titled Petbarn : Pet Poem Competition. That is easy to see and quite a simple test to conduct with a good number of emails sent for testing. While the read rate for both may speak more about the company itself and the willingness for its customers to accept marketing material, we think that the lack of an unsubscribe rate on version 2 may be related to the title. The hypothesis at this stage is that keeping the customer interested from the start may be a factor in keeping their long term attention (and thus their long term value to the company sending emails). We will continue to develop this idea over a series of emails to gain a true impression of whether or not this is correct.

What do you think?

Interested in getting your message to your customers effectively? Brown Box can deliver your email marketing with extremely competitive rates and great personal service. We can even advise you on the best way to setup and manage your campaigns. contact me today to find out how

This entry was posted on Tuesday, February 19th, 2008 at 11:32 am and is filed under Email Success. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.