If your email marketing program consists primarily of an auto responder program, then breaking out segments or running pure A/B tests isn't going to be a viable option for you. However, there are other ways in which you can test and determine what works best for you. In this section, we'll discuss how to narrow down your best practices for auto emails with some basic tests
Why is Testing an Auto Responder List Difficult?
As we've discussed throughout this ebook, there are many advantages to running an auto responder campaign. You can do all of the major work at one point, and then the campaign runs on its own with your only having to check in and optimize it infrequently. However, it can also result in a headache if you're trying to test and perfect your list. Here are some reasons why:
New Users Daily: For most auto responder campaigns, you're adding new users almost daily. That means that your list is never "set" and you can't get a good handle on how a stable list would respond to you. You may have added twenty new users right before you send an email, which will impact the response rate. Also, some users on your list may have received an auto email the day before, and some may not have received one for over a week. It's impossible to garner a pure test result with a list that isn't stabilized, and auto responder lists never truly stabilize.
Lack of Information Collection: While there are certainly exceptions, most auto responder campaigns collect very little user information because the priority is to capture a lead. The less information that you collect, the less you can personalize or tailor an email to run a test.
Inability to Segment: Similar to the lack of information collected, auto responder lists are difficult to segment. Because users have joined at many different points and times and have often provided very little information, pulling out database segments based on anything other than interactions with your email program can be incredibly difficult. Even finding a way to randomize a list split can be hard.
Ongoing Emails: As noted above, many of your auto responder subscribers are receiving various ongoing emails. The position that they're in within your auto reply cycle can greatly impact what they will and won't respond to. Because auto emails are often so lengthy in duration, it may be difficult to group people by where they are in the process.
Higher Spam Issues: One of the downsides of auto responder campaigns is that they often have greater issues with spam problems. Every time you run an email campaign test, you run the risk of negatively impacting your inbox deliverability, and this is even more true with an auto email campaign.
How Can You Overcome the Issues with Testing Auto Responders
However, the flexibility and high data collection values of email make it possible for you to effectively test almost any of the standard email marketing tests we've talked about in this section. You may just have to work a little bit harder and take a little more time. Here are some ways that you can effectively test email marketing concepts within auto responder email campaigns.
Use Long Tail Tests: The best way to overcome the challenges of testing email components with an auto responder list is to use long tail tests of a month or more. If you run your test for a long time, you'll be able to equalize out the data variations for the ongoing new subscriptions, unsubscribe requests, and various user states and timing issues. It may mean that it takes you four months to test one component. For example, you run a test of a lifestyle graphic to your list for two months. Then you switch it to a test of a product graphic for two months. At the end of the four months, you compare responses. Yes, it took a long time, but the data you collected was as clean as it can be. Of course, if your program is incredibly seasonal in nature, then this may be an issue for you.
Use Expired Segments: You can also take expired segments of your list (users who have completed the entire cycle of your auto reply list) and test concepts on them by sending them individual marketing emails. It's true that these users won't behave exactly as your main list will. However, it's also safe to assume that they represent the general trends of your email audience. If you send to your expired segments in an A/B test of email subject line and a time sensitive as email subject line outperforms a vague email subject line, then it's safe to assume that time sensitive subject lines would work well in your auto responder campaign.
Take More Time Crunching Your Data: You can also overcome obstacles by simply taking more time crunching your data. It will be harder to figure out what's going on with your list, but if you really take the time to look at user state and the timing of a send, you can make some educated guesses.
With anything in life, there are pros and cons. An auto responder campaign can reduce email marketing effort on your end while still providing a robust ROI. However, it may also be harder to optimize because it's harder to test. That doesn't mean, though, that testing it is impossible.