Benchmarks…Who Needs Them?

As more and more firms engage in the important endeavor of measuring client perceptions of their products and services, the natural next question is…are the scores we collected good, bad, or somewhere in the middle? Assuming best practices as it relates to research design and methodology are in place, it’s fairly easy and straightforward to compare client experience scores internally across a range of different dimensions. With sufficient sample size of course, you may want to rank order your internal regions, branches, or even individual relationship managers. However, given the competitive nature of most industries, the next question inevitably goes to…how do we stack up against the competition?

What is a benchmark, exactly?
The term benchmark is a very common business term and an important measure across a wide spectrum of applications. At McLagan, we have a wealth of compensation benchmarks that we can use to help firms better design their compensation plans. At Scorpio, we have a global AUM benchmark study that allows firms to compare their AUM growth with competitors in an objective way. And individual firms themselves establish their own benchmarks or seek out third party ones to compare their client experience scores against the competitors. However, unlike compensation and AUM benchmarks where the data is in most respects a census and actual firm by firm results, client experience offers up much more complexity given the challenges of sampling.

Calculating benchmarks…not as easy as it seems
Consider the following chart from our internal database:

The most obvious conclusion you can draw from the data depicted above is that North America leads the world in delivering high level of client satisfaction. However, the answer is likely much more complex than that. There has been a lot of research on research published over the years around how different segments respond to surveys and most have found that there can be meaningful differences. I recently read an interesting paper on response styles that suggested that Acquiescent Response Style (ARS) and Extreme Response Style (ERS) vary by country.

What they found was that the US tests more toward higher ARS and ERS, possibly leading to higher scores that are provided. Does this mean that the North American score in the chart above is not meaningfully different from Europe or Asia? Not necessarily. There appears to be enough of a difference to suggest that there is a difference between the US and the rest of the world. However, the more important trend from this chart is that the rest of the world is improving. I would add that this generally matches my observations in that most firms in the US have had client experience programs in place for many years while most European and Asian firms have just recently started their program, or don’t have one in place at all.

Panels – a good source for competitive benchmarks, if…

For many firms, collecting client experience scores among their client base is not enough. Critically, they would also like to better understand where they stand in the marketplace. However, sampling clients at other firms is not as easy, and many firms turn to online panels. As many of you are aware, online panels are large collections of consumers willing to answer questions about the products and services they consume. The most sophisticated panels do a very good job of representing target audiences, in many cases in the wealth management industry by AUM or wealth management style. The biggest drawback, however, is that these panels are in many respects a different sample from a firm’s client base. I have found through my years of experience, and also supported by a lot of other research, that these differences can be important. Most importantly, typically internal client surveys skew higher in terms of sentiment given that the respondent knows the survey sponsor. On the other hand, panel surveys are typically more anonymous, and therefore you are more likely to get more critical feedback. This is not to say that firms should not turn to panels for competitive intelligence. Panels are in fact excellent sources of competitive client insight provided that some biases are accounted for. Sampling the client brand along with the competitive brands is one way to understand the differences between sample sources that you can then extend and adjust for when analyzing your internal measures. Another way to find a best in class measure would be to calculate the relative distance from top scores against the average and apply that same distance to your internal average.

Bottom line is that benchmarks are important, whether you’re an Olympic athlete measuring yourself against the world record or a company on the journey to client experience excellence. The important thing is to ensure the comparisons you make are appropriate. As luck would have it, we at Scorpio have a veritable treasure trove of data we’ve collected over the past 15 years and carefully calculated global and region specific benchmarks across the most important client experience and brand measures.

NewsBusiness