Did the same advisers consistently achieve better returns for their clients year after year? Did some advisers consistently display more skill than others?
To find the answer, I computed the correlations between the rankings of advisers in different years, comparing Year 1 with Year 2, Year 1 with Year 3 and so on up through Year 7 with Year 8. That yielded 28 correlations, one for each pair of years. While I was prepared to find little year-to-year consistency, I was still surprised to find that the average of the 28 correlations was .01. In other words, zero. The stability that would indicate differences in skill was not to be found. The results resembled what you would expect from a dice-rolling contest, not a game of skill.
Don’t Blink! The Hazards of Confidence - NYTimes.com
I’m not sure about this method. Why use rankings? After all, the point of being or hiring an investor is not to be ranked best out of the people in the firm, but to consistently make money.
This measure just tells us that none of the guys in the firm were consistently better than each other on a year-by-year basis. More interesting would be to know who over the seven year period had made the most money - after all, the one who can win hugely in good years, and lose only moderately in bad years may make the most money, even if they never top out the rankings.
More fundamentally, it still makes sense to reward good staff, even if all your staff are as good as each other, simply because if they are better than the population of people in the job, they could walk away. A much more interesting comparator for the firm would be to see who makes more than the industry average consistently (or not), over the available time horizon.
