The five hidden dangers of feedback statistics
Although I think grades and test scores have a valid place, I sometimes wonder if we don’t get so used to that mentality in school that we never let go of it in adulthood. It seems like there are a huge variety of places to see our progress or self-perception reflected in the ‘grade’ of salary and raises (or not), reviews, conversations, web site statistics, etc. While these can be valuable indicators, relying on them too strongly over your intuition may keep you from where you want to be. Sometimes relying on your own judgment is required over the perceptions of others and it’s one of the advantages of growing up! Here are five things to be aware of when looking at feedback, whichever form you are either exposed to or seek out:
1. Despite the numerous analogies out there, life isn’t baseball. There’s no specific calendar to know when you can definitively judge the feedback as to when something was good, great, or a waste of time. You might do a favor for someone and not realize until ten years later what it meant to them but because they are still talking about it, you now see just how much it meant. Or this blog – when I post this, if nobody reads it the first day does that mean it’s not of interest? What about the following week? What if 100 people read it the first day but nobody after that? Watching statistics too minutely is unlikely to provide realistic data in the long term.
2. You don’t know what you don’t know about. If someone praises you to your boss while that boss is frantically looking for the presentation that was supposed to start five minutes ago, what are the odds that you’ll ever hear about it? Exactly. Bloggers that check Google stats daily (or more often) because they’re attaching self worth to those numbers only know about the readers that permit cookies or leave comments; the rest are invisible, but not unimportant.
3. The smaller and less varied the sample the less accurate the data is generally. It’s the whole ’six out of seven doctors surveyed’ ad campaign. Was that seven doctors down the hall from the marketing department or 70,000 personal interviews? If your boss says the majority of your customers are happy/unhappy with your performance, it does make a difference how many customers you have. If that number is very small it doesn’t make the existing ones less important but that feedback may not apply beyond that small group to the next set of customers you work with.
4. Feedback is more likely to come from those that feel invested, thereby skewing the statistics. I find blogging to be fascinating to watch because it is a constantly changing social experiment. What I’ve noticed lately is that the majority of comments on my other blog are coming from people, organizations or competing organizations that were mentioned in the post. They are all more invested in what I’m saying and how it’s perceived than general readers. If you aren’t a great cook, that’s much more likely to come up in conversation if you invite friends over for dinner (invested) than if you mention to them what you made for dinner last night (not invested.)
5. People don’t read directions. Whether it’s a ranking survey (1-10) or an assessment someone is going to provide different information than was requested because they misread that 10 was excellent or a valid email address is required. Going back up to point number 3 that can change the feedback you see significantly. If you base your feelings or your actions purely on that feedback you could inadvertently head down the wrong path.
