Indisputable Laws of List Testing

The list is arguably the most important variable to test in direct mail and e-mail marketing, though other candidates would be price, offer, format, sales appeals, envelope teasers and subject lines.

Here are some guidelines that can make your list tests more accurate and useful and increase response when rolling out to the winners:

o Use the same quantity of names for every list you test. Often this means 5,000 names, since many list owners will not rent smaller quantities for a test.

o The quantity of sample names you need to test is independent of the size of the total list. Some mailers mistakenly think that the bigger the list, the larger the sample size needed to generate a statistically valid result. File size has nothing to do with sample size.

o If a list is so small that it has little potential for rollout, just mail the entire list. For instance, it doesn't make sense to test 5,000 names from a 7,500-name list. Just mail the whole thing. Don't pass over a list in your test just because it is small. A small but potent list of 7,500 names that produces a 5 percent response is better than a list of 75,000 names that produces a 1 percent response.

o If a list seems perfect for your offer, test it regardless of size. However, if your budget limits you to testing only one or two similar lists, select the larger lists because they have the most rollout potential.

o Always test the hotline names, the most recent segments of any list, first. If they don't work, no segment will.

o Test new lists early. New lists tend to deliver the highest response rates when they are first placed on the market.

o Re-mailing the same offer to the same audience repeatedly over time will result in a decrease in response. The smaller the target market, the faster the response will drop off. Varying the package or offer significantly with each new mailing is required to stimulate interest and response.

o Use your own customer file to profile against prospecting lists. Segments that have the greatest match are most likely to produce the best response.

o Test new products and offers against the most responsive segments of a list.

o Every response device should bring back the label or ink-jet address from the list including a source code and date stamp. The source code tells you which list generated the reply. The date stamp tells you how long the name has been on the file and its lifetime value since being added to the file.

o Customer lists, even when fatigued, tend to outperform prospect files. Therefore, even the oldest segments on your house file should be mailed as long as they produce more orders than the best prospect list.

o Never throw away your inactive customer files. Lists of inactive customers, even five to nine years old, often produce greater response than prospect lists.

o If any segment of your house file does not out-pull rented prospect lists, it likely reflects the way these customers originally were acquired, as well as how they were treated. Research this to find out and fix the cause.

o It's always better to mail a different segment of the same list rather than make a repeat mailing to a segment already mailed.

o Most list segments can be mailed more than once a year, but it's important to test. Group your list into segments, large enough to test, based on recency. If a list generates a good response, re-mailing the same promotion to it about eight weeks after the first drop will generate roughly 50 percent of the original response. Example: If the first mailing pulled 5 percent, the second will produce around 2.5 percent.

o No matter how often you mail to your house file, you likely are not mailing enough. If you mail to your house file four times a year, try six or eight times.

o In business-to-business direct mail, it is better to limit the number of executives or titles mailed per establishment to just a few. Do not mail large quantities of the same piece to the same site.

o Don't test too many variables. If you do, you'll wind up with results anyone can argue with.

Stevan Roberts is president of Edith Roman Associates, Pearl River, NY, and is co-author of "Internet Direct Mail: The Complete Guide to Successful E-Campaigns" (NTC Business Books).