What Should You Test?

Should you test a live stamp versus an indicia? When do you test entirely new creative? Can the copy on your order form dramatically affect results?

As with many questions in direct marketing, the answer is, “It depends.” You may have heard someone say, “Direct marketing testing is a science.” And while testing requires a scientific approach, it is also an art. It takes experience, some finesse, and common sense.

If you find yourself thinking, “There’s no need to test that because it ALWAYS…” then you may be proven wrong. In direct marketing, things are constantly changing. What worked last year, last month, or even last week, may not work now. That’s why testing is so important.

Whether you’re marketing to individual consumers, businesses, or donors, you are still dealing with people. And people are fickle. Attitudes, motivations, wishes, dreams, and goals can change overnight.

Many direct marketers have found that the direct mail packages that performed well a few years ago are not working as well today. That’s why it’s important to continually test your package elements to make sure they are in tune with the needs of customers and prospects.

However, you can get “test happy.” You should be careful to test only those things that have a reasonable chance at beating your control.

Here are two key factors to consider when thinking about a test:

1) How much money will the test result save you?
2) What can you do with the results?

For example, if you are sending out a package and want to test mailing it first class versus standard mail, you need to determine if the anticipated lift in response will be enough to overcome the additional postage costs.

The point of testing is to improve your return on investment. The ROI equation can be satisfied in three ways:

1) increasing response while keeping costs steady,
2) keeping response steady while lowering costs, or
3) spending more to improve response significantly.

Big tests have the biggest opportunity. If you can work on the strategy, message, creative, and offer, test those first. In acquisition, always test multiple lists and list segments. Formats can also have a big impact on results. A 9 x 12 package versus a 6 x 9 may make a dramatic difference in response.

Sometimes less can be more. Through aggressive list testing and segmenting, you may be able to mail less with greater response. You may be able to eliminate a package insert while maintaining the same response rate. At Allegro, we once tested eliminating a premium from a package and actually increased response from certain list segments. The only way to know if you can get by with less is to test it.

There are also times that call for significant package changes. Even highly successful controls will eventually wear out. The marketplace or attitudes can change and response rates may begin to lag. Also, when testing acquisition mailings, it can be highly profitable to try an entirely new creative approach or package format. In some instances, a new package can become an instant breakthrough success.

Here are some of the significant areas to test:

• product or service features
• mailing lists
• envelope appearance
• number/type of enclosures
• strategic approach/appeal
• price or donation amount
• offer statement
• copy tone and message
• premiums
• overall package format
• timing of mailing

Always test any changes against what is currently working. If you’re going to test entirely new creative, put it head-to-head against what has been working for you. If you’re testing lists, always test one or more proven lists at the same time so you can have comparative results. If you’re testing copy, price, or offer, it’s also better to test more than one option over several lists rather than to test one package on one list. The more you are able to compare results among different approaches and lists, the more confident you can be that your test results are reliable.

Testing is part art and part science. It takes creativity, discipline, and the willingness to take some risk. And, you may just be rewarded with a higher return on your investment.