FaQtor Test methodology offers quick learnings, maximum value

When it comes to direct mail testing, marketers can learn a lot from baseball’s front-office executives.

If you sign a .150 hitter to your roster, you might get him on the cheap, but he’s not exactly the guy you want up at the plate with one out in the bottom of the ninth. You already know there’s an 85 percent chance this guy will not get on base with only one at bat. But if you can be patient with him, over the course of a few games and seven or so trips to the plate, chances are he’ll eventually send one to the gap. But in the meantime, how many missed opportunities were there before you saw him get that hit?

“In direct mail testing, all we need is one hit,” said SeQuel's Co-Founder and Chief Operating Officer, Jay Carroll. “However, our clients can’t afford to run seven or eight A/B tests over the course of two years to get that hit. They want results, and they want them now.”

A/B tests can be likened to a ‘trip to the plate’ in baseball. The likelihood of success is low, if you have only one shot, but if you get enough opportunities, chances are high you’ll win at least once. For the vast majority of clients and prospects that SeQuel has engaged with, if they’ve been testing at all, they’re testing in an A/B environment.

“You’re going to throw one piece out and, if you’re going to read the results correctly, you’ll change only one thing about the package on the next try,” said VP of Data & Analytics, James Fussy. “You’ll change one element, try again, change one element and try again. That’s going to take you forever, and many marketers get discouraged and sometimes abandon testing—or even direct mail—altogether.”

In DM there are innumerable features you can test at a given time. It’s not often that anyone comes to SeQuel and claims they’ve tried it all, but in order to gather those learnings in one multivariate test, it’s often prohibitively expensive.

SeQuel’s proprietary FaQtor Test methodology seeks to deliver the best of both worlds. It provides the success rate of a full-scale multivariate test for near the investment level of an A/B test—it's a hybrid approach.

“Through the process of indexing and handicapping, we can show how various combinations of list, offer and creative would perform based on the relative performance of a specific list, specific offer and specific creative,” said Chief Marketing Officer Erik Koenig. “Indexing what that combination would look like and handicapping it based on actual and projected volumes will many times lead us to the ultimate control combination of list/offer/creative, which may not have been tested initially.”

"Our clients can’t afford to run seven or eight A/B tests over the course of two years to get that hit. They want results, and they want them now.”

-Jay Carroll, Co-Founder/COO

Most new controls SeQuel has built have come via an index that’s been proven in a back test, rolled out, and then proven itself again. A/B and multivariate testing are literal; indexing is what everybody’s missing. Many marketers are missing numerous potential—and affordable—winning combinations.

“You increase your likelihood of success by four or five times using our FaQtor Test method,” Carroll said. “You increase costs incrementally vs. an A/B test, but those costs are an excellent investment, when the likelihood of success is that much greater.”

If a company is performing A/B testing on a quarterly basis, which is considered above average, it would take more than a year to eclipse the chances of finding a winner with a single FaQtor Test.

“Our approach will be a fraction of the cost in time, energy and lost opportunity,” Carroll said. “You also might have 12 additional months of an under-performing control, when you could have a new winner now.”


Tired of inefficient testing methodologies?

Build a FaQtor Test matrix