1. The dangers of over analysing your 360° feedback report

    Posted in Tips & Best Practice, Using Appraisal360

    Lots of graphs and charts in your 360 degree review report make it easier for people to interpret the feedback and understand what’s being said, right?

    Well, no. Not necessarily.

    Just because computers are good at turning numbers into graphs and charts doesn’t mean that those charts are useful, valid or meaningful. Worse than that they may even be hiding underlying messages or creating a misleading impression of the underlying data.

    Take one very obvious example: averages. For our example let’s consider an average of 4.0. This could have been achieved by combining eight feedback scores each of 4. So far so good.

    But now consider the following scores: 2, 1, 2, 5, 5, 4, 4, 5 – these also give an average of 4.0 but they tell a very different story. Whereas the first set of numbers showed a consistent high score the second shows some dramatic differences in scoring and whereas some raters think highly of this person with respect to this question, others seem to have a very different view. This is clearly something that requires a closer look and shows how some very important feedback could have been lost if the results had been reduced to averages.

    “Can you plot a chart of people’s self scores and feedback scores and that will tell us the development areas.” Is typical of some of the comments we might hear. Normally we’ll try and explain that approaches like this are fraught with danger. Quite apart from the fact that feedback is supposed to make people think and presenting them with computer generated conclusions like this rather defeats the object, the underlying data simply isn’t robust enough to withstand this sort of manipulation.

    360 degree feedback scores are subjective measures of people’s opinions, perceptions and attitudes. Although these figures are useful they are not in the same category as measurements of electrical current, speed, or mass for example. Mathematical comparisons of one perception against another, although tempting, simply don’t cut it and aren’t valid Worse still they can mislead and can distract people from the basics of what their respondents are trying to say. Be careful.