Find Your One Thing: Relative Importance Analysis

© Metro-Goldwyn-Mayer Studios Inc/Columbia Pictures All Rights Reserved

© Metro-Goldwyn-Mayer Studios Inc/Columbia Pictures All Rights Reserved

Fans of the 1991 movie City Slickers might remember the crusty old cowboy telling Billy Crystal’s character that if he wanted to find happiness, he should find his “one thing”. In a related tale, our clients often want us to help make their lives easier and frequently ask, "What is the 'one thing' we can do to increase customer loyalty to our brand?" 

Our clients are smart to want to know how to specifically direct their efforts to drive loyalty.  Rather than try to improve their service efforts across the board, they want to make improvements in one or a few areas which will have the most impact on engendering loyalty, thereby increasing sales or improving client retention.  While they shouldn’t ignore poorly performing service areas or let service lag where they are performing well, they do want to get the most bang for their service improvement dollar by concentrating on those service attributes that are most impactful to the bottom line.

Typical survey design sometimes lends itself to methodological concerns about which predictors are actually valid drivers of loyalty. The problem is that predictors are often highly correlated, making the “one thing,” or best predictor, hard to identify. This issue is known as multicollinearity, and generally is best addressed with experimental design using MaxDiff or conjoint where one chooses whether a component is present or absent. However, these design schemes are not always appropriate. As in our example of a survey for the restaurant industry below, service attributes are best measured by degree with rating scales, rather than whether they are present or not (e.g., the restaurant is clean or dirty by some degree as opposed to being clean or not clean).

In 2014, Pacific Market Research conducted a study in the hospitality industry to determine which touch points of service are most impactful or drive overall restaurant loyalty.  In our survey design, restaurant goers were asked to rate the relative importance of 20 attributes of the dining experience in addition to providing ratings on three measures of loyalty to a restaurant establishment. In the resulting analysis, loyalty was the combination of the three loyalty variables and became the dependent variable in our model.

Traditionally, market researchers would look to Stepwise Regression to determine the key drivers of loyalty. Below, one can see the resulting output of the regression model (note - if you are on a mobile device, flip to the landscape mode):

Key Drivers of Total Loyalty

We expect no one is surprised to discover that price and quality of food are the biggest drivers of customer loyalty. However, on their face a few things in this output just don’t make sense. First, it tells us that food portion size and cleanliness of restaurant facility are negative drivers of loyalty. Have you ever heard someone say, “I love going to Joe’s because it is dirty and they don’t give me enough food?” Unless you are a rat on a diet, this doesn’t pass the sniff test (or, in this case, the taste test).

Besides, we know from additional analysis that this is an untrue representation of the data in that portion size is positively correlated with loyalty. Another limitation of Stepwise is we know nothing about 11 of our 20 predictor variables. The model doesn’t tell us anything about location, or cleanliness, or quality of the greeting – which goes against what we know from qualitative feedback to be important aspects of hospitality.  So, while our Stepwise Model isn’t wrong, it is providing results which call the model into question.

These model shortcomings likely arise from multicollinearity between our attributes and some level of model overcorrection. For example, the model struggles to distinguish between attributes that are similar yet distinct such as cleanliness of restaurant and cleanliness of the facility.  In this case, the model chooses only one variable. The model doesn’t need 20 variables to provide an accurate model equation and it doesn’t care about the sniff test; thus variables are left out.

So what are we to do about finding our one thing?

At Pacific Market Research, we often use what is known as the Shapley Value Regression to address these types of issues. The Shapley Value Regression takes all the possible combinations of Linear Regression equations and determines which variables are the most important to the model. Below is an example output (note - if you are on a mobile device, flip to the landscape mode):

Key Drivers of Loyalty

From this chart, we can clearly see that food quality represents 26% of the total importance when determining loyalty – taste is indeed the one thing. Next, it makes sense to concentrate on price and the final farewell in order to drive restaurant loyalty. Greeting, restroom condition and manager’s level of concern are less important factors.

While Shapley Value Regression has provided more plausible and actionable insights, we acknowledge some continued presence of multicollinearity. However, for this example and others, it does the best job of identifying the key drivers of loyalty with rating scale measurements.

Please ask us how we can help you find your one thing to improve loyalty or satisfaction.  We promise not to make you drive cattle to find it. 

Author: Trevor Taylor, Project Specialist/Analyst