What do these results mean? Picking a subject line winner

Touchstone virtual tests indicate how much better or worse your subject lines are likely to perform compared to average.

What do these results mean? Picking a subject line winner

In the example above, a result of "85% Opens" means that based on the performance of similar subject lines, what was tested is likely to perform 85% better. Therefore, if your list average open rate is 12%, a result of "85% Opens" indicates that you could expect to achieve an open rate of 22.2% by using the subject line tested.

This is an indication. The more you use Touchstone, the more you will be able to get a feel for how the results relate to your own lists and segments. For even more precise results, you can sync historical performance data from your campaigns to Touchstone.

Are the predictions accurate?

In general, Touchstone predictions fall within 1-2% of the true value.  So yes, the tool is very accurate.

Testing on our own newsletter

We've tested the accuracy of Touchstone on our own newsletter: testing two subject lines within Touchstone, and then deploying them as A/B tests and seeing whether Touchstone's predicted winner actually performed better.

Here are our results, showing a 93.3% accuracy:

How does the algorithm make predictions?

The basic premise behind the algorithm is relatively straightforward to explain.  When you perform a search, here is what happens:

1. The algorithm finds all similar subject lines

Each subject line in our database is assigned a score describing how similar it is to your search, and the most similar subject lines are kept for further analysis.

2. The performance of these subject lines is analyzed

For each subject line that is similar to your search, we compare its performance against its sender's average performance.  By always comparing the results of a subject line against a sender's average, we avoid the possibility of a few users with abnormally high or low open rates throwing off the results.

3. All results are combined

The performance results from all these subject lines is combined into a single score.  This takes into account send volumes and the similarity score from Step 1 to optimize prediction accuracy.

4. Recommendations are generated

Finally, the algorithm tries dozens or hundreds of modified versions of your subject line (such as replacing "Good deals" by "Great deals") to find recommendations which will most improve your results.

How do I save the results?

You can export each search to PDF by clicking on the save  icon.

No results were found. What happened?

Although it uses an extensive database with hundreds of thousands of subject lines, Touchstone may not always find subject lines which are similar to your search.  If this happens to you, you have a few options:


1. If you have not uploaded your data onto the tool, do so.  Touchstone will then be able to use your past subject lines in order to provide you with recommendations.

2. If you were searching only on your data, then perform a search on the entire Touchstone database by selecting a sector at the top of the screen.

3. If you have uploaded your data and were searching the entire database but still did not find any results, then your subject must simply be too unique for Touchstone to make an accurate analysis.  However, do not despair: research shows that unique subject lines tend to be the most effective.

Why are the recommendations slow to appear?

The short answer is that recommendations take at least a few seconds to compute.  Our recommendation engine is very work-intensive and involves testing every combination of synonym and alternate word choice which are similar to your subject line.


For example, if you search for "Great deals", the recommendations engine will look for multiple subject lines such as "Awesome deals", "Great sales", and so on.  You can easily imagine the huge number of alternatives our tool goes through before finding the best one.  While you wait, however, imagine how long it would have taken had you needed to perform A/B tests on these dozens of combinations.

Does Touchstone work in different languages?

Some aspects of Touchstone will work in any language, and others are currently available only in American English.


Open, Click and Delivery Rates

The Touchstone search results of open rates, click rates and delivery rates will work in any language.  When you test a subject line, Touchstone searches through its database (or through your user-uploaded database) for any subject lines which are similar to yours and analyse how well they performed.  


Word Recommendations

Because language is complex, the word recommendation engine is currently available only in English.  


Do I have to upload data?

Uploading data is entirely optional.  However, we strongly recommend it, as it lets Touchstone give you personalized recommendations based on your own subscribers and your own campaigns.

Why should I upload data?

Uploading subject line data is both very easy and immensely powerful.  It lets you run Touchstone searches on your own data in order to analyse how your subscribers responded to the subject lines they received.  

While searching the entire Touchstone database provides valuable insight regarding new subject line strategies you could be using, searching your own database provides insight on your current writing strategies, and helps you identify what works and what doesn't work.  You should therefore ideally be using both approaches.

How do I upload data?

Step 1: Download Data from ESP

First, you need to download data from your ESP.  At this stage, you want to create a spreadsheet with 1 row per mailing, with each row  containing the subject line, mailing date, send volume, number of opens and number of clicks.

The specific procedure to download this data will vary by ESP and we are in the process of developing ESP-specific guides to help you out.

Step 2: Upload Data

Once you've gathered your data, log into your Touchstone account and follow the Upload Data link at the top of the screen.

Step 3: Identify Columns and Date Format

Finally, follow the instructions on the Upload page to upload your file and identify your columns.  Play close attention to the "Date" field and ensure that it is in the correct format.