0882: "Significant"

[RockoTDF]

For this reason I don't think this comic can argue that more than 5% of papers are published with chance results.

It was really this sentence that I was refuting. If everybody uses an alpha of .05 still and they don't do a multiple comparisons adjustment then you can make the argument that more than 5% of published papers have results due to chance alone.

Ugh, I had a post that got eaten yesterday, but here is the gist:

Anyway, I think you are confusing alpha and p. I set the alpha, the universe sets the p. If my results are p < .001, my alpha can be .01, .05, or .99 and it will have no bearing on whether or not my results are due to chance or not (the key being at p < .001). Sure, .05 is the worst I'll take, but the worst I'll take and what actually happens are not the same thing.

[dash]

What's with the obsession with acne?

Want to know the cause, Randall? Read a book called "Acne Can Be Cured" by Gustave H. Hoehn, MD. Published 1977.

Here's the secret: Saturated fats cause acne. They clog the pores, which then become infected. Avoid saturated fats, consume oils, like olive oil, vegetable oil, etc. instead.

Cheese = bad
Beef fat = bad
Pork fat = bad (sic) Yes pork fat is saturated too
Poultry = good
Fish = good
Oils from nuts, vegetable oils = good

Think of a ball point pen, if the ink is too thick it won't flow. Too thin and it runs all out.

One other point the book makes is your own fat is saturated, so if you're losing weight, that can cause acne as your own body's fat starts circulating in order to be consumed.

So now, can we please move on from this subject?

ETA - reiterated that pork fat is saturated, added fish + oils lines. + note on human body fat being saturated also

[boudu]

though simple significance tests, like significant digits, must be accounted ineluctably silly, the joke is on those of us who say so : in this example, that one of twenty results should have proved 'significant' is no surprise; that that one should have turned out to be the one involving jelly beans that are green (of all colours) – now that merits investigation.

[RockoTDF]

though simple significance tests, like significant digits, must be accounted ineluctably silly, the joke is on those of us who say so : in this example, that one of twenty results should have proved 'significant' is no surprise; that that one should have turned out to be the one involving jelly beans that are green (of all colours) – now that merits investigation.

Errr...why? Isn't the whole point that the significance of green is merely by chance?

[achan1058]

EDIT: You think I'm kidding, right? Last week, during a class in college, one of my classmates was wearing an xkcd t-shirt. Yeah, I can't stand the goddamn thing, and even then someone has to rub it on my face. Okay, I can try to live with that. Today, at work, what do I see? A colleague, that is, another person entirely, wearing another xkcd t-shirt. THE GODDAMN THING IS EVERYWHERE.

Install blocksite, block yourself from this forum and xkcd. That way you don't need to stare at it 1/2 as much. The fact that you are still here means you aren't truly hating all of it.

On the comic itself, I think it is brilliant.

[matthewhaworth]

I don't get it. It even says in one of the panels that there is no correlation between green jelly beans and acne..

[Feddlefew]

EDIT: You think I'm kidding, right? Last week, during a class in college, one of my classmates was wearing an xkcd t-shirt. Yeah, I can't stand the goddamn thing, and even then someone has to rub it on my face. Okay, I can try to live with that. Today, at work, what do I see? A colleague, that is, another person entirely, wearing another xkcd t-shirt. THE GODDAMN THING IS EVERYWHERE.

Erm.... Noticing and being bothered by a t-shirt worn by another person like that is a strong symptom of a certain subset of obsessive compulsive disorders and phobias. Unless you've had a traumatic experience involving XKCD, that kind of behavior isn't normal.

[jthetzel]

Cheese = bad
Beef fat = bad
Pork fat = bad
Poultry = good

I hate to point out simple typos, but you meant to type:

Pork fat = good

Anyway, fats and oils are good for you. They keep your arteries well lubricated. Without them, the pipes get clogged. Science.

[sellyme]

Anyway, fats and oils are good for you. They keep your arteries well lubricated. Without them, the pipes get clogged. Science.

The pipes get clogged? My internet connection obviously needs more fats and oils.

</lamepoliticalinternetreferencefromsolongagoitwasprobablybeforeiwasborn>

[philip1201]

Anyway, fats and oils are good for you. They keep your arteries well lubricated. Without them, the pipes get clogged. Science.

The pipes get clogged? My internet connection obviously needs more fats and oils.

</lamepoliticalinternetreferencefromsolongagoitwasprobablybeforeiwasborn>

Whoa, you're only 3 years old?

[SirMustapha]

Erm.... Noticing and being bothered by a t-shirt worn by another person like that is a strong symptom of a certain subset of obsessive compulsive disorders and phobias. Unless you've had a traumatic experience involving XKCD, that kind of behavior isn't normal.

Okay. If I complained that every time I step in the bus someone is listening to Justin Bieber on his cellphone without headphones, it would be okay. If I complained that someone in the same room as mine is stinking like rotting piece of meat, it would be okay. Complaining about seeing xkcd T-shirts on a constant basis is a sign of OCD. HELP ME, I DON'T GET IT.

[cphite]

Erm.... Noticing and being bothered by a t-shirt worn by another person like that is a strong symptom of a certain subset of obsessive compulsive disorders and phobias. Unless you've had a traumatic experience involving XKCD, that kind of behavior isn't normal.

Okay. If I complained that every time I step in the bus someone is listening to Justin Bieber on his cellphone without headphones, it would be okay. If I complained that someone in the same room as mine is stinking like rotting piece of meat, it would be okay. Complaining about seeing xkcd T-shirts on a constant basis is a sign of OCD. HELP ME, I DON'T GET IT.

Have you tried not looking at the t-shirts?

Hearing is multi-directional, and sort of an all-or-nothing thing when it comes to your surroundings; so yes, someone playing music you don't like on the bus can be annoying. But with seeing, you generally get to pick and choose what you look at and pay attention to. The next time your classmate wears his xkcd t-shirt, try looking at something else.

If you do that and the t-shirt still bothers you, consult a psychiatrist.

[SirMustapha]

I suggest, politely, that we continue this discussion starting from the supposition that nobody here is mentally ill. Please?

The problem is not with staring at the thing: it's at bumping into the goddamn comic when I don't want to. Once I see it, the damage is done, I've already seen it, and "not looking" will not undo it. The effect is similar to hearing someone quote xkcd: I can't "unhear" it, and the damage is done as soon as I hear it.

Usually my focus shifts to something entirely different just a few seconds later anyway, but that's beside the point.

[Feddlefew]

I suggest, politely, that we continue this discussion starting from the supposition that nobody here is mentally ill. Please?

Sorry about that.

The problem is not with staring at the thing: it's at bumping into the goddamn comic when I don't want to. Once I see it, the damage is done, I've already seen it, and "not looking" will not undo it. The effect is similar to hearing someone quote xkcd: I can't "unhear" it, and the damage is done as soon as I hear it.

Usually my focus shifts to something entirely different just a few seconds later anyway, but that's beside the point.

So... Your day was disturbed because an independent entity decided to bring something (nonharmful) into your environment that you dislike? Is this because a person wearing the t-shirt obviously likes XKCD? Have you tried constructing scenarios that they do not like/know about XKCD but is wearing their roommate's t-shirt because they're out of shirts?

Edit: Missed a pronoun.

[asdfzxc]

Damn it now I'm playing Minecraft and I can't put it down

Curse you Randall

[SirMustapha]

So... Your day was disturbed because an independent entity decided to bring something (nonharmful) into your environment that you dislike? Is this because a person wearing the t-shirt obviously likes XKCD? Have you tried constructing scenarios that they do not like/know about XKCD but is wearing their roommate's t-shirt because they're out of shirts?

Well, if you think of the T-shirt in itself as something non-harmful, I can understand why you'd think I have a serious issue. The thing is, those folks are not assorted people who only by chance decided to wear a T-shirt with stick figures -- they are folks who willingly want to insert themselves into the stereotype of fashionable computer geek, as if they have to fit a determined mold just because they work with computer science; you know, the folks that think that computers are better than anything else, and computer science and maths are obviously more important than other useless fields like sociology, philosophy and history. In a discussion from my college course, someone has explicitly stated that people from computer courses are more intelligent than the others and their opinions are worth more than those from the humanities. It's people like that who want to use xkcd as a symbol of "computers are cool", as if they have something to prove to the others. The result is that anything that Randall does that affirms the coolness and superiority of computers is deemed as awesome. I can't stand that attitude.

Am I exaggerating? Well, I've just conluding the main reason why someone would wear an xkcd T-shirt: maybe they just like it that much, but liking it is not enough, and they have to show it to others. They want to be identified. And the only person who would identify an xkcd fan is... another xkcd fan. A normal people would have to stop and stare at the T-shirt to read it all the way to the end, and most people wouldn't get the joke, and many would just be indifferent. But a fellow xkcd fan would find it "awesome" by default. It's a badge of "if you like this, you're as cool as I am", in short. I mean, if people just want to identify themselves as computer folks, xkcd is not the best option for that! There are much more direct jokes that about anyone can understand. But not xkcd: it is an "elite club", and I hate being part of a group that wants to be so exclusive. I'm not special for studying computer science, and I don't think anyone is, in short.

[Sofie]

Well, I've just conluding the main reason why someone would wear an xkcd T-shirt: maybe they just like it that much, but liking it is not enough, and they have to show it to others. They want to be identified. And the only person who would identify an xkcd fan is... another xkcd fan. A normal people would have to stop and stare at the T-shirt to read it all the way to the end, and most people wouldn't get the joke, and many would just be indifferent. But a fellow xkcd fan would find it "awesome" by default. It's a badge of "if you like this, you're as cool as I am", in short. I mean, if people just want to identify themselves as computer folks, xkcd is not the best option for that! There are much more direct jokes that about anyone can understand. But not xkcd: it is an "elite club", and I hate being part of a group that wants to be so exclusive. I'm not special for studying computer science, and I don't think anyone is, in short.

That's what all sub-cultures do. The "uniforms" people choose are a baseline indicator of interests. Sports fans wear jerseys, or hats and shirts emblazoned with their favorite sports teams. Goths wear black and odd fabrics. Businesspeople wear suits. Music nerds wear obscure band t-shirts. What people are wearing tells you something about that person, and that includes a first glance, "Do I think that person and I have anything in common?"

All groups think they are better than other groups, whether that's because of their taste, (lack of) wealth, intelligence or whatever.

[scarletmanuka]

I wonder if this takes the prize for most panels in an XKCD comic (I know there have been comics larger than this, but I am refering specifically to the number of panels)

The 2008 Christmas Special had more panels, though a lot of them were just black.
XKCD Loves the Discovery Channel had the same number of panels as this comic, and far less repetition.

If we're not letting the 2008 Christmas special win due to the blank panels, then The Man Who Fell Sideways is the winner, larger by a whole extra row over XKCD Loves the Discovery Channel!

Maybe. There are other contenders. For instance, I'm not sure how to count the nunber of panels in Godel, Escher, Kurt Halsey. If you don't count Blogofractal as one panel, it probably wins.

[achan1058]

So... Your day was disturbed because an independent entity decided to bring something (nonharmful) into your environment that you dislike? Is this because a person wearing the t-shirt obviously likes XKCD? Have you tried constructing scenarios that they do not like/know about XKCD but is wearing their roommate's t-shirt because they're out of shirts?

Well, if you think of the T-shirt in itself as something non-harmful, I can understand why you'd think I have a serious issue. The thing is, those folks are not assorted people who only by chance decided to wear a T-shirt with stick figures -- they are folks who willingly want to insert themselves into the stereotype of fashionable computer geek, as if they have to fit a determined mold just because they work with computer science; you know, the folks that think that computers are better than anything else, and computer science and maths are obviously more important than other useless fields like sociology, philosophy and history. In a discussion from my college course, someone has explicitly stated that people from computer courses are more intelligent than the others and their opinions are worth more than those from the humanities. It's people like that who want to use xkcd as a symbol of "computers are cool", as if they have something to prove to the others. The result is that anything that Randall does that affirms the coolness and superiority of computers is deemed as awesome. I can't stand that attitude.

Am I exaggerating? Well, I've just conluding the main reason why someone would wear an xkcd T-shirt: maybe they just like it that much, but liking it is not enough, and they have to show it to others. They want to be identified. And the only person who would identify an xkcd fan is... another xkcd fan. A normal people would have to stop and stare at the T-shirt to read it all the way to the end, and most people wouldn't get the joke, and many would just be indifferent. But a fellow xkcd fan would find it "awesome" by default. It's a badge of "if you like this, you're as cool as I am", in short. I mean, if people just want to identify themselves as computer folks, xkcd is not the best option for that! There are much more direct jokes that about anyone can understand. But not xkcd: it is an "elite club", and I hate being part of a group that wants to be so exclusive. I'm not special for studying computer science, and I don't think anyone is, in short.

Until people stop wearing anything religious related on the streets, I see nothing wrong about the XKCD T-shirts. After all, your argument can be applied to all religious ornaments as well. Seeing how the former is never going to happen, I don't find anything wrong with the latter.

Besides, if you don't want to be part of the "elite club" or whatever you want to call it, don't. Block yourself from this forum, and don't associate yourself with xkcd in anyway. There are plenty of CS majors who don't look at or like xkcd.

[philip1201]

Wait, xkcd is elitist?

[achan1058]

Wait, xkcd is elitist?

I don't know how he came to this conclusion, nor the conclusion that xkcd represents all, or even a significant amount of CS and/or math majors.

[Bobbert]

Wait, xkcd is elitist?

I don't know how he came to this conclusion, nor the conclusion that xkcd represents all, or even a significant amount of CS and/or math majors.

Well, computer science, math, and other hard science majors being smarter than fine/liberal arts majors is a pretty ongoing theme in xkcd, as is "someone with computer knowledge is a dick to person without as much computer knowledge".

[SirMustapha]

Until people stop wearing anything religious related on the streets, I see nothing wrong about the XKCD T-shirts. After all, your argument can be applied to all religious ornaments as well. Seeing how the former is never going to happen, I don't find anything wrong with the latter.

Finding both annoying is not an option, now?

Besides, religion and xkcd are a world apart. It's completely meaningless to compare them. Religion is ancient and ubiquitous, while xkcd is very strict and unknown. This little "elite club" people want to form by identifying themselves as such is based on the fact that nerdy things are fashionable now. It's a ridiculous reaction to the fact that everything nerdy was utterly humiliated by everyone until just a few decades ago. Now, those folks don't just want to be accepted: they want a "genius" like Randall to show that they are better than everyone else.

Besides, if you don't want to be part of the "elite club" or whatever you want to call it, don't.

That is not terribly hard. But if I'm going to bump into xkcd strips in college classes and lectures, classmates, e-mails, blogs and everything related, I'd rather read it on my own accord, then.

[Dason]

That is not terribly hard. But if I'm going to bump into xkcd strips in college classes and lectures, classmates, e-mails, blogs and everything related, I'd rather read it on my own accord, then.

"I guess if every couple of months somebody is going to stab me in the eye I'll just stab myself in the eye three times a week on top of that..."

[Ephemeron]

That is not terribly hard. But if I'm going to bump into xkcd strips in college classes and lectures, classmates, e-mails, blogs and everything related, I'd rather read it on my own accord, then.

"I guess if every couple of months somebody is going to stab me in the eye I'll just stab myself in the eye three times a week on top of that..."

I'm just gonna throw this out there:

"No One Is Forcing You To Read It"

[achan1058]

"No One Is Forcing You To Read It"

Still, nobody is forcing you. If you can't take a little propaganda at all, get rid of your friends, go to somewhere that is as far from anything xkcd related. (somewhere in Africa or Yukon if you must.) If we can tolerate people unsolicited advertising of religion (and we do have people stopping you and ask about Jesus on the road here), as well as many other things, you should be able to tolerate a little xkcd.

[SirMustapha]

If we can tolerate people unsolicited advertising of religion (and we do have people stopping you and ask about Jesus on the road here), as well as many other things, you should be able to tolerate a little xkcd.

Patronising much?

I mean, is it too hard to fathom the fact that people want to vent their frustrations a little? Randall makes comics about the things that annoy him and he gets praised, while you are still banging your head on the wall in denial of the xkcd detractors.

On Monday I'm going to start posting "If I dislike xkcd so much, then you all shouldn't like it that much either!" around these threads, and point to your posts to cover me up.

[Twigshusband]

I like SirMustapha, he makes the forums a lot more interesting. I get tired of reading 'GOOMH' or 'this comic is brilliant', these comments should be things you say to yourself, not published for everyone else to read because you are the only one ever to have had that thought. In short most of the forumites are making the same mistake as Randall, not being particularly original or insightful. Surely everyone is allowed an opinion on the comic and has the right to publish it on the forums.

[wedwardes]

On topic

See this blog from The University of Hertfordshire:

Wine drinkers unable to tell the difference between expensive and cheap wines

http://blogs.herts.ac.uk/research/2011/04/14/wine-drinkers-unable-to-tell-the-difference-between-expensive-and-cheap-wines/

More details here:

Britons can't tell the difference between a fine wine and plonk

http://www.dailymail.co.uk/sciencetech/article-1376686/Britons-tell-difference-fine-wine-plonk.html

[floris1831]

The comic is being plagiarized by the Dutch newspaper NRC: http://www.nrc.nl/, http://www.geenstijl.nl/archives/images/NRCwetenschapkaternmetgestolencartoon.pdf (mirror)
Did XKCD give permission to let it be copied and accredited to Rik van Schagen? ( http://nl.wikipedia.org/wiki/Rik_van_Schagen ; http://www.zone5300.nl/agentschap/rikvanschagen/ )
It seems to me it is in clear violation of the Creative Commons licence... http://xkcd.com/license.html ==> "I L L U S T R AT I E R I K V A N S C H A G E N", not mentioning xkcd.com in any way

[Felstaff]

The comic is being plagiarized by the Dutch newspaper NRC: http://www.nrc.nl/, http://www.geenstijl.nl/archives/images ... artoon.pdf (mirror)
Did XKCD give permission to let it be copied and accredited to Rik van Schagen? ( http://nl.wikipedia.org/wiki/Rik_van_Schagen ; http://www.zone5300.nl/agentschap/rikvanschagen/ )
It seems to me it is in clear violation of the Creative Commons licence... http://xkcd.com/license.html ==> "I L L U S T R AT I E R I K V A N S C H A G E N", not mentioning xkcd.com in any way

This is currently being handled. Thanks!

[BigMikeNZ]

hey
i dont know about anyone else, but i had never heard of minecraft before this comic, so, naturally, i looked it up.
because of me reading this comic, me, 2 of my friends, and very soon, one of my friends brothers, all horribly addicted to minecraft, and now fully understand "but miiiiine craft" as we say it constantly.
i guess xkcd runs my life....

[biodomino]

For this reason I don't think this comic can argue that more than 5% of papers are published with chance results.

It was really this sentence that I was refuting. If everybody uses an alpha of .05 still and they don't do a multiple comparisons adjustment then you can make the argument that more than 5% of published papers have results due to chance alone.

Ugh, I had a post that got eaten yesterday, but here is the gist:

Anyway, I think you are confusing alpha and p. I set the alpha, the universe sets the p. If my results are p < .001, my alpha can be .01, .05, or .99 and it will have no bearing on whether or not my results are due to chance or not (the key being at p < .001). Sure, .05 is the worst I'll take, but the worst I'll take and what actually happens are not the same thing.

To further clarify this, even though an alpha of .05 is often chosen, p is often a value less than .05. Only 1/20 of studies where p=.05 will be "false positives." Likewise, even with an alpha of .05, only 1/100 studies where p=.01 will be a false positive (also called a type 1 error). If alpha is .05, and p=.001, then the odds of a false positive are 1/1000.

A study with a p=.05 would very likely be replicated a number of times (and likely across a variety of contexts and populations) before the results were taken as fact.

[Dason]

If alpha is .05, and p=.001, then the odds of a false positive are 1/1000.

No. Under the null hypothesis the p-value is uniformly distributed. If you set alpha to be .05 then the probability of getting a false positive is .05 (assuming the null is true).

[biodomino]

If alpha is .05, and p=.001, then the odds of a false positive are 1/1000.

No. Under the null hypothesis the p-value is uniformly distributed. If you set alpha to be .05 then the probability of getting a false positive is .05 (assuming the null is true).

You're right, so let me try that again while I'm actually awake. Given an alpha of .05, the lower your p value, the more leeway you have to choose a lower alpha value and still satisfy the criteria for statistical significance (which will result in a larger p value). What I mean to point out is that though many studies use .05 as the alpha, they COULD use .01 (and often do when they can).

Now if I'm wrong about that, please let me know so I can enroll in a remedial statistics course

[Dason]

If alpha is .05, and p=.001, then the odds of a false positive are 1/1000.

No. Under the null hypothesis the p-value is uniformly distributed. If you set alpha to be .05 then the probability of getting a false positive is .05 (assuming the null is true).

You're right, so let me try that again while I'm actually awake. Given an alpha of .05, the lower your p value, the more leeway you have to choose a lower alpha value and still satisfy the criteria for statistical significance (which will result in a larger p value). What I mean to point out is that though many studies use .05 as the alpha, they COULD use .01 (and often do when they can).

Now if I'm wrong about that, please let me know so I can enroll in a remedial statistics course

Well of course they could. But if you're choosing your alpha level AFTER you get your p-value then you're doing it wrong and somebody should smack you upside the head. It's unethical.

[biodomino]

I can see why you would say that, but I don't see how it's unethical to choose a stricter standard for significance ex post facto. Less strict, sure.

Many scholarly publications will consider a .05 significant, but report results for an alpha of .01 as well. Should we necessarily treat those studies as though the alpha is .05?

[Dason]

I can see why you would say that, but I don't see how it's unethical to choose a stricter standard for significance ex post facto. Less strict, sure.

Because they would have used anything less than .05. Sure the p-value they got was a lot smaller than that but if you advertise that you used a smaller alpha then you're a liar. It would be like if a university claimed that they only accepted the top 1% of students in the nation but really accepts anybody in the top 50%. The claim they are making makes them seem a lot better than what is true.

Quite frankly it's lying, it's unethical, it shouldn't be found in any good research. Sadly that probably isn't the case but I'll stick with what I think is right.

[biodomino]

In practice the alpha isn't just used to determine the arbitrary level of significance, though. It's also used as a counterpoint to the results.

I see your point, but seldom is an alpha of .01 or better chosen because the study is that strict. It is usually reported to demonstrate the potential robustness of the findings, with the (arguably unethical) understanding that if there were no significance at the .01 level, then the .05 level would have been used.

Considering the selection of the alpha is pretty arbitrary either way, I don't think there's much significance to insisting an a priori determination of what it should be. Rather, the alpha should serve as a scale for how the results should be interpreted.

[Pfhorrest]

I can see why you would say that, but I don't see how it's unethical to choose a stricter standard for significance ex post facto. Less strict, sure.

Because they would have used anything less than .05. Sure the p-value they got was a lot smaller than that but if you advertise that you used a smaller alpha then you're a liar. It would be like if a university claimed that they only accepted the top 1% of students in the nation but really accepts anybody in the top 50%. The claim they are making makes them seem a lot better than what is true.

Isn't it more like saying "all the students at Impressive U are in the top 1% of students nationwide", even though they would have accepted anyone in the top 5% if they had to? But the fact that the school filled up with students from just the top 1% still says that the school is good enough to attract enough of the top 1% of students to get by, even if the school would have accepted students merely in the top 5% if they didn't turn out to be that good.

Accept students : publish results :: university : study, so by analogy, the study would have published results even if they only got results in the .05 range, but they got results in the .01 range, so what's wrong with saying that they got better results than they would have demanded?

EDIT: Broken quote nesting.