Although concern surrounding certain list-makers’ methodologies has been brewing for some time, a frenzy erupted when BusinessWeek’s Steve Hamm wrote a very compelling article questioning the practices of Black Book of Outsourcing creators Brown & Wilson.  Per Hamm’s article:

“Claes G. Fornell, a professor at the University of Michigan Ross School of Business who specializes in customer satisfaction surveying, says Brown & Wilson’s methods aren’t sound. First, he says, the firm can’t be sure all the people who respond are qualified. Second, the results could be tilted in favor of companies that urge their customers to participate. ‘You’d be better off not doing anything than doing a survey like this,’ says Fornell.”

Picking up on these concerns, Deborah Kops (CMO for WNS Global Services) repurposed her May 2006 article (“The Book of Lists”) for Fersht’s audience.  Among many insightful statements, Kops questions the value of lists and what they don’t tell us:

“A key component of editorial calendars and other sponsorships in the outsourcing industry, rankings and lists can provide a service to communities by identifying players and trends. But let’s not delude ourselves; they are also a business imperative for publishers, associations and pundits to build membership and/or circulation and sell adverts, publications and reprints, playing on the sell side’s need for recognition. They generally make someone money! This is not necessarily a negative, but is rather the way the world goes around. What’s critical is that that which is editorial and that which is financial should be kept completely independent at all costs. Pay for play in any form must not be the modus operandi.”

I believe Deborah was very gracious by not explicitly referring to those entities whose lists, rankings and awards have tainted the perceived or real value of the industry’s providers.  Those of us who have been in the market for sometime now can easily identify the organizations who have handsomely profited from such point in time, pay-to-play assignments of value.

The Catch-22 of our current industry state is this: In lieu of an attractive alternative, provider marketing organizations will continue to pour precious funding into the pre-existing ranking and rating processes. Thus, the cycle perpetuates itself.  We don’t like the lists yet many are afraid to not be on them.  However, until there is a collective halting of such spend, we can’t expect the system to self-correct. In the interim, we can hold out hope that a better mechanism will emerge that is truly transparent, unbiased and effective. Let’s keep the conversation going.

_________________________________________________

It you want to voice your thoughts on these issues, place your votes by judging the judges.