Cosmologicon wrote:Well, I don't see why that's the case at all. In the example you gave with the two couples there was nothing forcing you to take their intentions into account. You could just as easily have ignored their intentions, in which case you'd analyze them both as you would couple A. And conversely there's nothing with Bayesian analysis that forces you to exclude intentions. You could just as easily do it like I did, incorrectly.

Let me address those in reverse order.

As you note, you applied Bayes' Theorem incorrectly. In Bayesian analysis you're supposed to apply it correctly. Meaning you apply it to exactly what you observed. If you do that, then there is no place for your notions about what you think is relevant to affect your analysis.

You got different numbers because for couple B you didn't include the data point of the 8th daughter. However that is a data point that you have, and therefore a correctly done Bayesian analysis must include it.

Now to hypothesis testing. In hypothesis testing you set up a universe of possible answers and you're comparing what you did observe with what you think you might have observed instead. When you try to ignore intentions, you're incorrectly analyzing what you might have observed instead. In short while you

can try to ignore intentions, by doing so you've violated the hypothesis testing methodology.

Cosmologicon wrote:It also seems circular the way you've decided that it's incorrect to include intentions in the first place. You say that's the case because Bayesian analysis proves they don't matter. But they only don't matter because Bayesian analysis doesn't include them. And it only doesn't include them because it's incorrect to!

No, it is not circular. It only looks that way.

If you have 2 hypotheses, and a priori beliefs about their relative likelyhood, there is no question about how you should modify your beliefs in the face of evidence. You should modify them in accord to Bayes' Theorem. You should also include all of the evidence that you can include because it is all pertinent.

But once you include all of the evidence that you can include and apply Bayes' Theorem, you get an answer. There is no room to get a different answer, you have the one answer. And that answer has absolutely no room to be affected by differences in experimental setup. This method of drawing inferences is provably correct.

What about hypothesis testing? Hypothesis testing answers a different question. It tells you, "Under the null hypothesis, how likely is it that I'd have seen something at least as unlikely as what I saw?" It explicitly is not telling us how likely the null hypothesis is. It is merely answering a question that we think relates.

No theorem of probability theory tells us that hypothesis testing is a correct way to draw inferences. No theorem can possibly do so because we apply it in situations where it is impossible to quantify the odds that the null hypothesis is correct. It is a methodology that we use and hope works well. However results like this one demonstrate that it pulls in factors that we don't really want to pull in when drawing inferences.

Which is at least worth thinking about.

Logodaedalus wrote:Furthermore, I'm inclined to say that the knowledge about the couples' behavior *should* influence our conclusion, and that it's perfectly possible to construct a Bayesian model that does so. In fact, any respectable Bayesian would likely tell you that it would be irresponsible not to use all available information.

Fine. Tell me how to construct a calculation using Bayes' Theorem that does so.

Logodaedalus wrote:EDIT: to be more specific, the fact that couple B made it to their 8th child *is valuable information*, and counts as evidence against the null hypothesis. If we knew a priori that they were going to have eight children, then the mere presence of 8 children would be without evidential value. So "intention" is critical, since it affects the generating mechanism based on which we're inferring.

The fact that they made it to their 8th child is useful information. However we have more information to include, namely that they

stopped at their 8th child. When you factor in all of the information we have, you get the same exact answer for couples A and B.