Integrating the Gaussian pdf over a half-space

For the discussion of math. Duh.

Moderators: gmalivuk, Moderators General, Prelates

nullgeppetto
Posts: 8
Joined: Fri Nov 08, 2013 12:48 pm UTC

Integrating the Gaussian pdf over a half-space

Postby nullgeppetto » Fri Nov 08, 2013 1:04 pm UTC

Hi all,

Let X be a random variable following the n-dimensional Gaussian distribution with mean vector μ and covariance matrix Σ. The probability density function is denoted by f(x). It is a well-known fact that when the above pdf is integrated over the whole Euclidean space Rn the result is 1. I would like to find out what's happening when the gaussian pdf is integrated over the half-space of Rn that is defined by the hyperplane H: xTw+b=0, i.e., the region Ω = {x \in Rn | xTw+b > 0}. Someone could observe that if the bias term, b, is equal to zero, i.e., the hyperplane passes through the origin, then - by symmetry - the integral should be equal to 1/2 (am I wrong?).

I would appreciate every single hint or helpful comment!
Thanks in advance guys!

Nitrodon
Posts: 497
Joined: Wed Dec 19, 2007 5:11 pm UTC

Re: Integrating the Gaussian pdf over a half-space

Postby Nitrodon » Sat Nov 09, 2013 9:27 am UTC

The integral is equal to P(XTw+b>0) by definition. What is the distribution of XTw = wTX?

nullgeppetto
Posts: 8
Joined: Fri Nov 08, 2013 12:48 pm UTC

Re: Integrating the Gaussian pdf over a half-space

Postby nullgeppetto » Sun Nov 10, 2013 5:41 pm UTC

Nitrodon wrote:The integral is equal to P(XTw+b>0) by definition. What is the distribution of XTw = wTX?



What is P? x's follow the n-dimensional gaussian distribution, by the way.
Could you explain why you say that the integral is equal to P(XTw+b>0)? Am I missing something?
Thanks for your response anyway!

User avatar
Lopsidation
Posts: 183
Joined: Tue Oct 27, 2009 11:29 pm UTC

Re: Integrating the Gaussian pdf over a half-space

Postby Lopsidation » Sun Nov 10, 2013 6:00 pm UTC

P is probability. The integral of the Gaussian distribution over the space {x: xTw+b>0}, is equal to the probability that when you pick a random x from the Gaussian distribution, it satisfies xTw+b>0.

Do you know how to do this problem if all n variables are independent? (Useful fact: the n-dimensional Gaussian distribution is rotationally symmetric around the origin.)

nullgeppetto
Posts: 8
Joined: Fri Nov 08, 2013 12:48 pm UTC

Re: Integrating the Gaussian pdf over a half-space

Postby nullgeppetto » Sun Nov 10, 2013 6:13 pm UTC

Lopsidation wrote:P is probability. The integral of the Gaussian distribution over the spaaace {x: xTw+b>0}, is equal to the probability that when you pick a random x from the Gaussian distribution, it satisfies xTw+b>0.

Do you know how to do this problem if all n variables are independent? (Useful fact: the n-dimensional Gaussian distribution is rotationally symmetric around the origin.)



Yeah, I know that. But what I ask is how could I express that fact in the form of the integral. Is there a clever way? If you take a look at my first post you can see that I observe that if the hyperplane passes through the origin, then - using the symmetry of the gaussian pdf - the integral should give 1/2, as the whole n-space gives 1. And this happens no matter what the covariance matrix is.

flownt
Posts: 70
Joined: Sat Feb 09, 2013 5:24 pm UTC

Re: Integrating the Gaussian pdf over a half-space

Postby flownt » Sun Nov 10, 2013 11:08 pm UTC

nullgeppetto wrote:Hi all,
Someone could observe that if the bias term, b, is equal to zero, i.e., the hyperplane passes through the origin, then - by symmetry - the integral should be equal to 1/2 (am I wrong?).

This is only true if \mu=0 ie. the chance is actually only 1/2 if \mu^t w=-b

I am pretty sure that the problem is not solvable if \mu doesn't lie in H though, because e^{-x^2} doesn't have a primitive.

User avatar
jestingrabbit
Factoids are just Datas that haven't grown up yet
Posts: 5967
Joined: Tue Nov 28, 2006 9:50 pm UTC
Location: Sydney

Re: Integrating the Gaussian pdf over a half-space

Postby jestingrabbit » Mon Nov 11, 2013 3:03 am UTC

flownt wrote:
nullgeppetto wrote:Hi all,
Someone could observe that if the bias term, b, is equal to zero, i.e., the hyperplane passes through the origin, then - by symmetry - the integral should be equal to 1/2 (am I wrong?).

This is only true if \mu=0 ie. the chance is actually only 1/2 if \mu^t w=-b

I am pretty sure that the problem is not solvable if \mu doesn't lie in H though, because e^{-x^2} doesn't have a primitive.


Big hint: Its solvable in terms of the Gaussian. Have a look at the pic on the top right here.

http://en.wikipedia.org/wiki/Multivaria ... stribution

The 1 dimensional projections are all normal distributions. Working out which Gaussian distribution it is ie mean and sd, are left as an exercise for the reader. I'd look under "marginal distributions" and "affine transformations".
ameretrifle wrote:Magic space feudalism is therefore a viable idea.

nullgeppetto
Posts: 8
Joined: Fri Nov 08, 2013 12:48 pm UTC

Re: Integrating the Gaussian pdf over a half-space

Postby nullgeppetto » Mon Nov 11, 2013 12:31 pm UTC

First, thank you all for your responses! Really appreciate that!
I may not be clear enough, though.

You can take a look at the following questions I asked at math exchange:

1. http://math.stackexchange.com/questions/556977/gaussian-integrals-over-a-half-space
2. http://math.stackexchange.com/questions/562332/rotating-an-n-dimensional-hyperplane

Thanks again!

nullgeppetto
Posts: 8
Joined: Fri Nov 08, 2013 12:48 pm UTC

Re: Integrating the Gaussian pdf over a half-space

Postby nullgeppetto » Mon Nov 11, 2013 3:37 pm UTC

I think that the only thing we need is to find out what the second question asks for!

http://math.stackexchange.com/questions/562332/rotating-an-n-dimensional-hyperplane

Is anyone familiarized with all this stuff? Special Orthogonal groups in n dimensions, SO(n)?
Searching in the internet didn't help very much...
Oh, do I look like desperate?

flownt
Posts: 70
Joined: Sat Feb 09, 2013 5:24 pm UTC

Re: Integrating the Gaussian pdf over a half-space

Postby flownt » Tue Nov 12, 2013 6:20 am UTC

jestingrabbit wrote:
flownt wrote:
I am pretty sure that the problem is not solvable if \mu doesn't lie in H though, because e^{-x^2} doesn't have a primitive.


Big hint: Its solvable in terms of the Gaussian. Have a look at the pic on the top right here.

http://en.wikipedia.org/wiki/Multivaria ... stribution

The 1 dimensional projections are all normal distributions. Working out which Gaussian distribution it is ie mean and sd, are left as an exercise for the reader. I'd look under "marginal distributions" and "affine transformations".


I think you mean that you can get an expression involving a gaussian integral, which i would agree with.

Anyway I guess it might be useful to try to write your integral in the eigenvector basis of \Sigma.

User avatar
Lawrencelot
Posts: 23
Joined: Sun Oct 13, 2013 8:10 am UTC

Re: Integrating the Gaussian pdf over a half-space

Postby Lawrencelot » Tue Nov 12, 2013 9:18 am UTC

Are you inventing a new kind of neural network or something? :o

nullgeppetto
Posts: 8
Joined: Fri Nov 08, 2013 12:48 pm UTC

Re: Integrating the Gaussian pdf over a half-space

Postby nullgeppetto » Wed Nov 13, 2013 10:25 am UTC

flownt wrote:Anyway I guess it might be useful to try to write your integral in the eigenvector basis of \Sigma.


This is correct, and this is going to happen easily. But it's not enough. Writing the quadratic form (that is present in the gaussian pdf) as a sum of squares is feasible using the eigenvalues of the covariance matrix \Sigma. But what's happening with the limits of the integral?

Something else should be done. And I think that question above (@ math exchange) should be answered first...

nullgeppetto
Posts: 8
Joined: Fri Nov 08, 2013 12:48 pm UTC

Re: Integrating the Gaussian pdf over a half-space

Postby nullgeppetto » Wed Nov 13, 2013 10:30 am UTC

Lawrencelot wrote:Are you inventing a new kind of neural network or something? :o


Yeah, maybe, or I am just curious! I cannot tell exactly, otherwise I'll be shot!
By the way, I couldn't tell I am really in favour of neural networks!
Just kidding. It really concerns something totally different.

flownt
Posts: 70
Joined: Sat Feb 09, 2013 5:24 pm UTC

Re: Integrating the Gaussian pdf over a half-space

Postby flownt » Wed Nov 13, 2013 8:21 pm UTC

Okay If you have a "generalized" volume integral in R^n (the subset over which we integrate contains a non-empty open subset of R^n) and we want to change basis from variables x to variables x' via x'=Ax with A a matrix. Then your "volume" element changes as dx'=\det A \cdot dx. Does that help?

nullgeppetto
Posts: 8
Joined: Fri Nov 08, 2013 12:48 pm UTC

Re: Integrating the Gaussian pdf over a half-space

Postby nullgeppetto » Thu Nov 14, 2013 7:15 am UTC

flownt wrote:Okay If you have a "generalized" volume integral in R^n (the subset over which we integrate contains a non-empty open subset of R^n) and we want to change basis from variables x to variables x' via x'=Ax with A a matrix. Then your "volume" element changes as dx'=\det A \cdot dx. Does that help?



That's correct, but it holds when we want to find the volume over the whole $\Re^n$, not over a half-space of it.
It's tricky, I though it would be easier when I firtst encountered this problem.


Return to “Mathematics”

Who is online

Users browsing this forum: No registered users and 19 guests