## Meta-Probability Problem

For the discussion of math. Duh.

Moderators: gmalivuk, Moderators General, Prelates

Posts: 35
Joined: Wed Apr 09, 2014 1:33 am UTC

### Meta-Probability Problem

You have a coin, which has an unknown probability distribution biased towards heads.
(or tails, but that doesn't change the theory behind this question)
What is the probability you will, over two tosses, in no particular order, get a head and tail?

This is not a homework question, it arose from a discussion my nerd friends had about unknown weighted probability of simple events.
GENERATION -705 - 992 i: The first time you see this, copy it into your sig on any forum. Square it, and then add i to the generation.

Cauchy
Posts: 602
Joined: Wed Mar 28, 2007 1:43 pm UTC

### Re: Meta-Probability Problem

If you know nothing about the coin other than that it's biased towards heads, then the probability distribution for the coin is 2 on [0.5, 1] and 0 everywhere else. The probability of a heads and a tails given a probability of heads p is 2p(1-p), so we get that the probability is int_{0.5}^1 2(2p(1-p))dp, which I don't care to calculate but my intuition tells me is 1/3.
(∫|p|2)(∫|q|2) ≥ (∫|pq|)2
Thanks, skeptical scientist, for knowing symbols and giving them to me.

Meteoric
Posts: 333
Joined: Wed Nov 23, 2011 4:43 am UTC

### Re: Meta-Probability Problem

This comes to the fuzzy edge of what we mean by "probability".

Ostensibly, coin flips are still independent and identically distributed, we just know that the probability of heads p is larger than 0.5. The chance of a head and a tail is 2*p*(1-p), so we can immediately conclude that the chance of getting one head and one tail is less than 50%, since this is true for any value of p != 0.5. Beyond that, you can't conclude much: it depends on your prior distribution over possible values of p. In other words, it depends on what you believe about unknown unfair coins. Are they usually almost fair, or usually radically unfair, or what?

Once you pick a distribution for p, it's a straightforward calculation of E[2*p*(1-p)]. The simplest assumption would be a uniform distribution on (.5,1) but there's no clear reason that this (or anything else) must be the correct distribution for an unknown unfair coin.
No, even in theory, you cannot build a rocket more massive than the visible universe.

Posts: 35
Joined: Wed Apr 09, 2014 1:33 am UTC

### Re: Meta-Probability Problem

Meteoric wrote:This comes to the fuzzy edge of what we mean by "probability".

Ostensibly, coin flips are still independent and identically distributed, we just know that the probability of heads p is larger than 0.5. The chance of a head and a tail is 2*p*(1-p), so we can immediately conclude that the chance of getting one head and one tail is less than 50%, since this is true for any value of p != 0.5. Beyond that, you can't conclude much: it depends on your prior distribution over possible values of p. In other words, it depends on what you believe about unknown unfair coins. Are they usually almost fair, or usually radically unfair, or what?

Once you pick a distribution for p, it's a straightforward calculation of E[2*p*(1-p)]. The simplest assumption would be a uniform distribution on (.5,1) but there's no clear reason that this (or anything else) must be the correct distribution for an unknown unfair coin.

Well one of the heads involved in the discussion claims that because every distribution has a "complementary distribution" (i.e. p <=> 1-p)
then the averages of the infinite probabilities gives a uniform distribution.

I'm skeptical because of the way infinities are handled.
GENERATION -705 - 992 i: The first time you see this, copy it into your sig on any forum. Square it, and then add i to the generation.

cyanyoshi
Posts: 412
Joined: Thu Sep 23, 2010 3:30 am UTC

### Re: Meta-Probability Problem

Paradoxica wrote:Well one of the heads involved in the discussion claims that because every distribution has a "complementary distribution" (i.e. p <=> 1-p)
then the averages of the infinite probabilities gives a uniform distribution.

I'm skeptical because of the way infinities are handled.

I pretty sure it doesn't work like that. If I tell you that I have a biased coin, would you think that 0.5 > p > 0.6 is equally likely as 0.9 > p > 1 ? I wouldn't think so. If you have completely no knowledge of the biases of coins, then assuming a uniform distribution of probabilities isn't the worst thing you can do, but it's not really "correct" either. The beauty of Bayes' theorem is that your choice of prior distribution doesn't really matter after performing lots of experiments, and process almost certainly converges to the true probability if you assume a uniform distribution over all the possible p's. However if you need to estimate the probability of rolling one heads and one tails without performing any experiments first, the best you can do is bound this probability using the range of the possible values of p.

doogly
Dr. The Juggernaut of Touching Himself
Posts: 5531
Joined: Mon Oct 23, 2006 2:31 am UTC
Location: Lexington, MA
Contact:

### Re: Meta-Probability Problem

A uniform distribution of .5 < p < 1 makes sense to assume, because it is the simplest distribution that fits what you know (that it's somewhere between those two endpoints.) But you should attach little faith to this, start flippin coins and updatin dat prior.
LE4dGOLEM: What's a Doug?
Noc: A larval Doogly. They grow the tail and stinger upon reaching adulthood.

Keep waggling your butt brows Brothers.
Or; Is that your eye butthairs?

Cauchy
Posts: 602
Joined: Wed Mar 28, 2007 1:43 pm UTC

### Re: Meta-Probability Problem

I read the question as "All you know about the distribution is that it's biased towards heads.". If you want to bring other "knowledge" into this about intuitions regarding coin manufacturing and whatnot, then that opens a can of worms that can't be sealed again. How do we know it's biased towards heads? What's our confidence in the observations that led us to that conclusion? The problem pretty clearly suggest a prior, so if you don't go with that one, then things get very messy very fast, and not in a way that leads to some greater insight.
(∫|p|2)(∫|q|2) ≥ (∫|pq|)2
Thanks, skeptical scientist, for knowing symbols and giving them to me.

Posts: 35
Joined: Wed Apr 09, 2014 1:33 am UTC

### Re: Meta-Probability Problem

Cauchy wrote:I read the question as "All you know about the distribution is that it's biased towards heads.". If you want to bring other "knowledge" into this about intuitions regarding coin manufacturing and whatnot, then that opens a can of worms that can't be sealed again. How do we know it's biased towards heads? What's our confidence in the observations that led us to that conclusion? The problem pretty clearly suggest a prior, so if you don't go with that one, then things get very messy very fast, and not in a way that leads to some greater insight.

Actually, that may be it.
Our discussion assumed an unknown bias towards both sides.
GENERATION -705 - 992 i: The first time you see this, copy it into your sig on any forum. Square it, and then add i to the generation.

Meteoric
Posts: 333
Joined: Wed Nov 23, 2011 4:43 am UTC

### Re: Meta-Probability Problem

Paradoxica wrote:Well one of the heads involved in the discussion claims that because every distribution has a "complementary distribution" (i.e. p <=> 1-p) then the averages of the infinite probabilities gives a uniform distribution.

I'm skeptical because of the way infinities are handled.

This seems like a non sequitur. All the 1-p values would be biased towards tails, so we don't care about them in this question, and in any case it suggests symmetry, not uniformity.

But we can use a different argument to support the same point. The relevant thing isn't the complementary values, it's the indifference principle which states that without reason to favor any possibility over another, you should treat them as equally likely (until more information comes along). So in this problem as stated, with no outside information, a uniform is the most "natural" prior. It's not hard to pose slight variants of the same question where other priors are more natural.

There's really two answers to this question. Under the Bayesian idea of probability as subjective confidence, the probability of getting a head and a tail (or whatever other combination of flips) is determined by your information about the coin; when you have no information it's determined entirely by your prior. Then the probability will change as you gain new information about the coin, for example by flipping it a bunch of times.

Under the frequentist idea of probability as proportion in the limit, the probability of getting a head and a tail is some specific exact quantity determined strictly by physical properties of the coin, and at the outset you just plain do not know what it is. You cannot answer the question at all. But as you gain new information about the coin, for example by flipping it a bunch of times, you can determine that the true probability is located in ever-smaller intervals with ever-higher confidence.

And clearly, in the limit these two probabilities should agree. But in the beginning, the Bayesian blurts out "one in three!" and the frequentist elbows him in the ribs and says "you can't know that!"
No, even in theory, you cannot build a rocket more massive than the visible universe.

Squarian
Posts: 7
Joined: Mon Feb 15, 2016 8:23 pm UTC

### Re: Meta-Probability Problem

Wait, why are you guys getting 1/3? I don't see it (Warning: I never had a stats class, so I might have outrageous misconceptions):

Let's call [;p;] the probability of getting heads in one toss, the probability for tails is [;1-p;]

The probability of getting two heads is then [;p^2;], and the probability for two tails is [;(1-p)^2;]

We can then find the probability of getting head-tails or tails-head by subtracting the found probabilities from [;1;]:

[;P_t_h=1-p^2-(1-p)^2=1-p^2-1+2p-p^2=2p-p^2;]

Then (and this is where it gets shaky), we can find the average value of [;P_t_h;] from [;0;] to [;1;] by doing a simple integral:

[;P_a_v_e=\int_0^1 \! 2p-p^2 \, \mathrm{d}p=2/3;]

Which doesn't seem right, where is the error?

EDIT: Found it, when you learn integrals you forget that 1+1=2

[;P_a_v_e=\int_0^1 \! 2p-2p^2 \, \mathrm{d}p=1/3;]

Squarian
Posts: 7
Joined: Mon Feb 15, 2016 8:23 pm UTC

### Re: Meta-Probability Problem

Sorry for the double post, but I was thinking, what would happen if the probability p of heads followed a normal distribution? Would the probability change?

cyanyoshi
Posts: 412
Joined: Thu Sep 23, 2010 3:30 am UTC

### Re: Meta-Probability Problem

Squarian wrote:Sorry for the double post, but I was thinking, what would happen if the probability p of heads followed a normal distribution? Would the probability change?

In general, yes. If you come up with a new probability distribution, you should expect that the integral of 2p(1-p) dF will change also. However be careful that the domain of your probability function matches your knowledge of values p can take. Normal distributions are defined for all real numbers, so that would be like saying it's possible that p<0 or p>1 (or indeed 0<p<0.5 which you assumed is not the case). Maybe you could truncate the distribution to match the range of p, but then it stops being a normal distribution.

Meteoric
Posts: 333
Joined: Wed Nov 23, 2011 4:43 am UTC

### Re: Meta-Probability Problem

You'd run into a bit of a technical problem, because normals are over the entire real line, while probabilities are restricted to [0,1]. You could use some kind of truncated re-scaled normal, though, or something that approximates that.

Unlike the uniform, there's more than one possible (approximated-)normal you could use here. In fact, your choice of mean and variance are pretty much unconstrained, other than the [0,1] thing. And that would definitely change the probability of "a head and a tail". You could make it anything you want, really: by picking a small enough variance, you can almost guarantee that p will be very close to whatever mean you pick.
No, even in theory, you cannot build a rocket more massive than the visible universe.

cyanyoshi
Posts: 412
Joined: Thu Sep 23, 2010 3:30 am UTC

### Re: Meta-Probability Problem

Meteoric wrote:You'd run into a bit of a technical problem, because normals are over the entire real line, while probabilities are restricted to [0,1]. You could use some kind of truncated re-scaled normal, though, or something that approximates that.

Unlike the uniform, there's more than one possible (approximated-)normal you could use here. In fact, your choice of mean and variance are pretty much unconstrained, other than the [0,1] thing. And that would definitely change the probability of "a head and a tail". You could make it anything you want, really: by picking a small enough variance, you can almost guarantee that p will be very close to whatever mean you pick.

I should mention that if you assume 0.5 ≤ p ≤ 1, then the mean μ must lie in that interval as well. The Bhatia-Davis inequality also puts a constraint on the variance by saying that σ2 ≤ (μ-0.5)(1-μ). Other than that, you are free to pick whatever mean and variance you like!

Posts: 35
Joined: Wed Apr 09, 2014 1:33 am UTC

### Re: Meta-Probability Problem

Regarding Normals, just apply the argument [imath]\tan{\frac{\pi}{2}x}[/imath]

It has both real infinities and is completely continuous.

Of course, there's always [imath]\tanh^{-1}{x}[/imath]

As well as all the other functions that can be artificially constructed to be absolutely less than unity in domain but cover the entire real line in output.
GENERATION -705 - 992 i: The first time you see this, copy it into your sig on any forum. Square it, and then add i to the generation.

gmalivuk
GNU Terry Pratchett
Posts: 26765
Joined: Wed Feb 28, 2007 6:02 pm UTC
Location: Here and There
Contact:

### Re: Meta-Probability Problem

But there's no motivation for any particular choice of adjustment, and you'd still have to adjust the result so that the total probablility sums/integrates to 1.
Unless stated otherwise, I do not care whether a statement, by itself, constitutes a persuasive political argument. I care whether it's true.
---
If this post has math that doesn't work for you, use TeX the World for Firefox or Chrome

(he/him/his)

pex
Posts: 10
Joined: Mon Mar 28, 2016 3:51 am UTC

### Re: Meta-Probability Problem

Paradoxica wrote:As well as all the other functions that can be artificially constructed to be absolutely less than unity in domain but cover the entire real line in output.

gmalivuk wrote:But there's no motivation for any particular choice of adjustment

Might I suggest we pick the inverse of the normal CDF?