2001: "Clickbait-Corrected p-Value"

This forum is for the individual discussion thread that goes with each new comic.

Moderators: Moderators General, Prelates, Magistrates

Mikeski
Posts: 994
Joined: Sun Jan 13, 2008 7:24 am UTC
Location: Minnesota, USA

2001: "Clickbait-Corrected p-Value"

Postby Mikeski » Fri Jun 01, 2018 5:50 pm UTC

Image

Alt-text: When comparing hypotheses with Bayesian methods, the similar clickbayes factor can account for some harder-to-quantify priors.

In related news, all scientific advancements in 2019 are predicted to center around Donald Trump and/or Kim Kardashian.

(And the phrase "one weird trick" is proved the most effective placebo of all time.)

sotanaht
Posts: 203
Joined: Sat Nov 27, 2010 2:14 am UTC

Re: 2001: "Clickbait-Corrected p-Value"

Postby sotanaht » Fri Jun 01, 2018 6:53 pm UTC

(And the phrase "one weird trick" is proved the most effective placebo of all time.)
Most effective placebo? So you are saying that clickbait writers think it works, so in spite of not actually doing anything it still produces better clickbait?

User avatar
Old Bruce
Posts: 104
Joined: Tue Jun 28, 2016 2:27 pm UTC

Re: 2001: "Clickbait-Corrected p-Value"

Postby Old Bruce » Fri Jun 01, 2018 9:25 pm UTC

I know that I know just enough to know that I don't know what the hell this is about. I am guessing it is a sort of wry humour. [confuzzled-face emoticon]
What I know is that Clickbait is something.

User avatar
Eternal Density
Posts: 5544
Joined: Thu Oct 02, 2008 12:37 am UTC
Contact:

Re: 2001: "Clickbait-Corrected p-Value"

Postby Eternal Density » Fri Jun 01, 2018 11:34 pm UTC

Heheh, nice work. Clever, funny, good comic.
Play the game of Time! castle.chirpingmustard.com Hotdog Vending Supplier But what is this?
In the Marvel vs. DC film-making war, we're all winners.

User avatar
Eebster the Great
Posts: 2978
Joined: Mon Nov 10, 2008 12:58 am UTC

Re: 2001: "Clickbait-Corrected p-Value"

Postby Eebster the Great » Sat Jun 02, 2018 12:37 am UTC

This "corrected" p-value frequently exceeds unity.

Mikeski
Posts: 994
Joined: Sun Jan 13, 2008 7:24 am UTC
Location: Minnesota, USA

Re: 2001: "Clickbait-Corrected p-Value"

Postby Mikeski » Sat Jun 02, 2018 2:10 am UTC

sotanaht wrote:Most effective placebo? So you are saying that clickbait writers think it works, so in spite of not actually doing anything it still produces better clickbait?


Something like that. Real-world drug placebos are getting better at treating pain, so why not clickbait placebos?

@maniexx
Posts: 25
Joined: Mon Aug 29, 2011 3:02 pm UTC

Re: 2001: "Clickbait-Corrected p-Value"

Postby @maniexx » Sun Jun 03, 2018 12:52 pm UTC

This... Should be actually we think really deeply about. A systematic way to correct for publication and researcher bias would be *awesome*.

Like, imagine you see a (recent) study saying "Women better at X". Should you trust it? Or Should you correct for the fact that a result saying "Men better at X" would be unlikely to get published?

Randalls formula is defintely not it, but maybe something to that effect could be created.
‮2‮0‮0‮1‮/‮m‮o‮c‮.‮d‮c‮k‮x

rmsgrey
Posts: 3358
Joined: Wed Nov 16, 2011 6:35 pm UTC

Re: 2001: "Clickbait-Corrected p-Value"

Postby rmsgrey » Sun Jun 03, 2018 2:05 pm UTC

@maniexx wrote:a result saying "Men better at X" would be unlikely to get published?


With some exceptions based on X - "Men better at child-rearing" would be more likely to get published than the equivalent about women.

madaco
Posts: 161
Joined: Sat Feb 13, 2010 11:25 pm UTC

Re: 2001: "Clickbait-Corrected p-Value"

Postby madaco » Sun Jun 03, 2018 9:30 pm UTC

This is an intriguing idea.
Ok so, the p value represents P(strength of evidence is at least as much as the observed results | H_0) (right?), so then this would end up representing

P(strength at least that of observed results | H_0) * (click(H_1)/click(H_0))

So, the expected clicks on claim of H_1 if H_0 would be click(H_1) * P(strength >= observed results | H_0) , and the expected clicks on claim of H_1 if H_1 would be, uh, click(H_1) * ( ??? )

I guess I'll approximate ( ??? ) as 1. I don't know if there is an analogous name for P( evidence strength >= observed evidence | H_1) .

And, the expected clicks on claim of H_0 if H_0 is true would be click(H_0) * ( ...hmmm...) ,

And the expected clicks on claim of H_0 if H_1 is true would be (...),

So, combining these, could get expected number of clicks on correct claims vs expected number of clicks on incorrect claims conditioned on H_1 and on H_0,

So, given prior probabilities for H_0 and for H_1, this could give a value for "expected number of clicks on correct claims minus expected number of clicks on incorrect claims" ?

Or, something like that.

That might require a bunch of committing to a methodology before doing the experiments or something like that?

Then, I guess the p value and other similar values required for the expected value of (clicks on correct claims - clicks on incorrect claims) to be sufficiently high could be computed,
And then one could commit to publish it exactly in those cases where those criteria are met.

That sounds tricky to implement, but theoretically maybe worthwhile?

Edit: I should make it clear that I don't know what I'm talking about. That is probably clear from the many question marks, but,

Epistemic status : You shouldn't trust my conclusions on this unless you can verify for yourself that my reasoning made sense.
I found my old forum signature to be awkward, so I'm changing it to this until I pick a better one.

User avatar
Eebster the Great
Posts: 2978
Joined: Mon Nov 10, 2008 12:58 am UTC

Re: 2001: "Clickbait-Corrected p-Value"

Postby Eebster the Great » Sun Jun 03, 2018 11:09 pm UTC

There is no meaningful way to deal with this formula as long as it produces probabilities greater than one.

Mikeski
Posts: 994
Joined: Sun Jan 13, 2008 7:24 am UTC
Location: Minnesota, USA

Re: 2001: "Clickbait-Corrected p-Value"

Postby Mikeski » Mon Jun 04, 2018 1:50 am UTC

Eebster the Great wrote:There is no meaningful way to deal with this formula as long as it produces probabilities greater than one.

Sure there is. If an event that happens with P=1.0 always happens, then an event with P=2.0 always happens twice. Simple.

madaco
Posts: 161
Joined: Sat Feb 13, 2010 11:25 pm UTC

Re: 2001: "Clickbait-Corrected p-Value"

Postby madaco » Mon Jun 04, 2018 6:04 am UTC

Eebster the Great wrote:There is no meaningful way to deal with this formula as long as it produces probabilities greater than one.


Why not interpret it as something other than a probability?

Like, say, as something reflecting a component contributing to likely accidental misinformation, and therefore something to be reduced.

Alternatively, if you must interpret it as a probability, just put a cap on it at 1, setting it down to 1 in any case that it exceeds 1
I found my old forum signature to be awkward, so I'm changing it to this until I pick a better one.

User avatar
Eebster the Great
Posts: 2978
Joined: Mon Nov 10, 2008 12:58 am UTC

Re: 2001: "Clickbait-Corrected p-Value"

Postby Eebster the Great » Mon Jun 04, 2018 8:07 am UTC

Mikeski wrote:
Eebster the Great wrote:There is no meaningful way to deal with this formula as long as it produces probabilities greater than one.

Sure there is. If an event that happens with P=1.0 always happens, then an event with P=2.0 always happens twice. Simple.

The present definition allows p = infinity.

madaco wrote:Why not interpret it as something other than a probability?

Like, say, as something reflecting a component contributing to likely accidental misinformation, and therefore something to be reduced.

Alternatively, if you must interpret it as a probability, just put a cap on it at 1, setting it down to 1 in any case that it exceeds 1

But then for every random event, P almost certainly equals 1. That also seems wrong. Well, I don't know that it seems "wrong" exactly, but it seems useless anyway.

madaco
Posts: 161
Joined: Sat Feb 13, 2010 11:25 pm UTC

Re: 2001: "Clickbait-Corrected p-Value"

Postby madaco » Wed Jun 06, 2018 4:46 am UTC

Why would p be almost certainly 1?

Surely there is a positive probability that someone would click the null hypothesis headline, and also a positive probability that the regular p value would be less than 1, so, ?
I found my old forum signature to be awkward, so I'm changing it to this until I pick a better one.

User avatar
Eebster the Great
Posts: 2978
Joined: Mon Nov 10, 2008 12:58 am UTC

Re: 2001: "Clickbait-Corrected p-Value"

Postby Eebster the Great » Wed Jun 06, 2018 6:34 am UTC

Yeah, not "almost every" in the technical sense, but in the sense that there are so many clickbait headlines that a subjectively large majority of them will have results greater than 1. Consider the example in the comic. It is not necessary that any actual article be published about anything; the formula is right there. And the likely number of clicks on headlines like "chocolate does not improve athletic performance" is so low that you can easily be multiplying ptraditional by huge factors.

gormster
Posts: 229
Joined: Mon Jul 23, 2007 6:43 am UTC
Location: Sydney

Re: 2001: "Clickbait-Corrected p-Value"

Postby gormster » Mon Jun 11, 2018 7:07 am UTC

@maniexx wrote:Like, imagine you see a (recent) study saying "Women better at X". Should you trust it? Or Should you correct for the fact that a result saying "Men better at X" would be unlikely to get published?


Neither of those are likely to get published unless they are talking about chromosomal sex, which would be unlikely to use the terms "men" and "women". And they would also have to be talking about something pretty damn specific to physical development.

"(Women|Men) more likely to x" will be published in a sociology journal with equal likelihood.
Eddie Izzard wrote:And poetry! Poetry is a lot like music, only less notes and more words.


Return to “Individual XKCD Comic Threads”

Who is online

Users browsing this forum: No registered users and 25 guests