Was I too harsh?

A place to discuss the implementation and style of computer programs.

Moderators: phlip, Moderators General, Prelates

wumpus
Posts: 546
Joined: Thu Feb 21, 2008 12:16 am UTC

Re: Was I too harsh?

Postby wumpus » Fri Jun 22, 2012 1:40 pm UTC

sourmìlk wrote:But yeah, that process is always fun. The sad thing though is that awareness of the Dunning-Kruger effect doesn't actually seem to diminish it. I was reading on the scientifically rigorous website Cracked.com about a study in which people were asked to rate some characteristic of themselves. Predictably, they all rated themselves above average. When the scientists gathered all the subjects together, told them that this was a mathematical impossibility, and asked if anybody might want to revise their opinions, nobody did.


Without a strong reference to exactly what the subjects were supposed to rate, it is impossible to tell if they were likely wrong. Without explicitly giving a definition of the direction "above average", they are simply asking the individuals if they feel they have remotely optimized these aspects of themselves. To take the classic example of driving, some people's goal is to minimize driving time, others wish to minimize risk of crashing, while still others may consider allowing traffic to maintain flow above all. Each driver sees themselves as "driving correctly" and anybody attempting to maximize some other goal as being "below average". Thus all but a few people consider themselves "above average".

Thus leaving the likely explanation is that the Dunning-Kruger effect has instead infected the scientists, making them feel they have constructed a far better experiment than they have (or they have, but following the classic example of scientist -> university PR flac -> tech journalist "breaking the news" -> hack journalist who re-reported the news you actually saw -> word of mouth [repeat] -> here, have mangled the report beyond all accuracy). [I suspect that experimental design is similar to software design. You can't find the errors in the most clever experiments you can design so you must limit yourself to far more simple experiments. Our scientists may already be out of their depth in this simple thing.]

User avatar
Anubis
Posts: 222
Joined: Sun Mar 01, 2009 7:59 am UTC

Re: Was I too harsh?

Postby Anubis » Fri Jun 22, 2012 11:05 pm UTC

wumpus wrote:
sourmìlk wrote:But yeah, that process is always fun. The sad thing though is that awareness of the Dunning-Kruger effect doesn't actually seem to diminish it. I was reading on the scientifically rigorous website Cracked.com about a study in which people were asked to rate some characteristic of themselves. Predictably, they all rated themselves above average. When the scientists gathered all the subjects together, told them that this was a mathematical impossibility, and asked if anybody might want to revise their opinions, nobody did.


Without a strong reference to exactly what the subjects were supposed to rate, it is impossible to tell if they were likely wrong. Without explicitly giving a definition of the direction "above average", they are simply asking the individuals if they feel they have remotely optimized these aspects of themselves. To take the classic example of driving, some people's goal is to minimize driving time, others wish to minimize risk of crashing, while still others may consider allowing traffic to maintain flow above all. Each driver sees themselves as "driving correctly" and anybody attempting to maximize some other goal as being "below average". Thus all but a few people consider themselves "above average".


That's actually a really interesting argument. I've never considered that possibility before, but it does seem to make sense. Of course, as you said, it may be that the original experiment was not subject to this criticism.

arbyd
Posts: 24
Joined: Thu Feb 11, 2010 4:33 pm UTC

Re: Was I too harsh?

Postby arbyd » Sun Jun 24, 2012 2:48 pm UTC

Jplus wrote:Also, I'd like to add the 10k hours rule to the mix. The 80/20 distinction doesn't tell you much about somebody's level of skill. In fact, the only thing that seems to matter for almost all things universally is the number of hours you've made. Typically people aren't really good at something (anything) until they've made at least 10k hours doing it*. So somebody at the bottom of the 80% (with some appropriate training and enough motivation) can perform much better than somebody high in the 20%, if the former made significantly more hours than the latter. 20% people will probably make more hours per week on average though, so they might reach the 10k hours sooner.


Some people have 5 years of experience. Others have 1 year of experience repeated 5 times. I see the 80% as the people who don't care about the art or craft of what they do. They might be expert in their application domain, i.e. accounting, vacuum process engineering, or some in house system, but all they're interested in is getting the job done. After a minimal learning curve, their code never improves. This is especially true if they are working more or less alone. Not having to read and debug someone else's code means they don't learn the importance of simple and standard patterns.

These people aren't necessarily stupid. In fact, the worst offenders are so bright that they can come up with workable solutions to many problems without researching how the rest of the world does the task. Unfortunately this results in systems that are far from optimum and virtually unmaintainable by any one else.

Yes, I'm annoyed. I'm working with someone that insists on writing new applications in standard BASIC, thinly disguised as VB6. No database, just flat files. No version control. No code review. Fortunately I get paid by the hour.

Falling
Posts: 175
Joined: Mon Nov 24, 2008 4:30 pm UTC

Re: Was I too harsh?

Postby Falling » Mon Jul 16, 2012 11:04 pm UTC

wumpus wrote:
sourmìlk wrote:But yeah, that process is always fun. The sad thing though is that awareness of the Dunning-Kruger effect doesn't actually seem to diminish it. I was reading on the scientifically rigorous website Cracked.com about a study in which people were asked to rate some characteristic of themselves. Predictably, they all rated themselves above average. When the scientists gathered all the subjects together, told them that this was a mathematical impossibility, and asked if anybody might want to revise their opinions, nobody did.


Without a strong reference to exactly what the subjects were supposed to rate, it is impossible to tell if they were likely wrong. Without explicitly giving a definition of the direction "above average", they are simply asking the individuals if they feel they have remotely optimized these aspects of themselves. To take the classic example of driving, some people's goal is to minimize driving time, others wish to minimize risk of crashing, while still others may consider allowing traffic to maintain flow above all. Each driver sees themselves as "driving correctly" and anybody attempting to maximize some other goal as being "below average". Thus all but a few people consider themselves "above average".


I've heard this a few times over the years and it's always bothered. In addition to it a poorly asked question (since there definitely are at leas a few different objective metrics for what makes a good drive), it's not even mathematically impossible! If they're talking about the arithmetic mean (which most people think of as an average), it's very easy to see the majority of people rating themselves as above average.

User avatar
WarDaft
Posts: 1583
Joined: Thu Jul 30, 2009 3:16 pm UTC

Re: Was I too harsh?

Postby WarDaft » Tue Jul 17, 2012 12:13 am UTC

It is mathematically impossible if the population actually does follow a bell curve. Interestingly, it almost does, but not quite, it's usually slightly heavier on the high end tail than the low end tail. That does not however justify much more than 1 or 2% more people claiming to be above average. Of course, if there are as few as 33 truly distinct skill sets (truly distinct as in proficiency in one is fully independent of proficiency in all others), it is entirely possible for every single person on Earth to be notably above average at a unique set of skills. In other words, for every single other person on the planet, it would be possible to be better than that person at something. Also, for two people chosen at random, you would expect to be better than them in about half of your proficiencies, let alone one. This kind of thing is probably what gives rise to the phenomenon.
All Shadow priest spells that deal Fire damage now appear green.
Big freaky cereal boxes of death.

User avatar
Jplus
Posts: 1721
Joined: Wed Apr 21, 2010 12:29 pm UTC
Location: Netherlands

Re: Was I too harsh?

Postby Jplus » Tue Jul 17, 2012 9:17 am UTC

WarDaft wrote:It is mathematically impossible if the population actually does follow a bell curve. Interestingly, it almost does, but not quite, it's usually slightly heavier on the high end tail than the low end tail. That does not however justify much more than 1 or 2% more people claiming to be above average. Of course, if there are as few as 33 truly distinct skill sets [...]

For clarity, I think you meant "that does not however justify much more than 50.5% or 51% of people claiming to be above average". It's what you say, but it took me a couple of times re-reading it before I realised.

In general, I think what you say about skillsets and so on is very true, but I'm afraid it's got nothing to do with people rating themselves above average... That's probably just irrational overconfidence.
"There are only two hard problems in computer science: cache coherence, naming things, and off-by-one errors." (Phil Karlton and Leon Bambrick)

coding and xkcd combined

(Julian/Julian's)

User avatar
WarDaft
Posts: 1583
Joined: Thu Jul 30, 2009 3:16 pm UTC

Re: Was I too harsh?

Postby WarDaft » Tue Jul 17, 2012 1:27 pm UTC

Most likely, but confirmation bias combined with the ability to find skills that you excel at over others more often than not (particularly since you are not likely to notice that you need to find different skill sets to excel over different people) can certainly reinforce the trend.
All Shadow priest spells that deal Fire damage now appear green.
Big freaky cereal boxes of death.

User avatar
sourmìlk
If I can't complain, can I at least express my fear?
Posts: 6393
Joined: Mon Dec 22, 2008 10:53 pm UTC
Location: permanently in the wrong
Contact:

Re: Was I too harsh?

Postby sourmìlk » Tue Jul 17, 2012 6:19 pm UTC

It's not that people were asked if they were above average in general, I think they were asked if they were above average in specific traits.
Terry Pratchett wrote:The trouble with having an open mind, of course, is that people will insist on coming along and trying to put things in it.


Return to “Coding”

Who is online

Users browsing this forum: No registered users and 4 guests