[SciPy-User] Easy Way To Find Confidence Intervals For 2D Data?

[Joseph Smidt]

I have a function on a 2d grid that looks like a skewed mound. I would
like to calculate everything in the 68% (one sigma or standard
deviation) confidence interval, and 95% (two sigma) as seen done in
this plot: Loading Image....
(The first line is 68% and second line is 95% confidence intervals.)

I would like to the set everything in the 68% confidence interval to
1, 95% to 2 and everything else to 0. Thanks.

Joseph Smidt

[Robert Kern]

Post by Joseph Smidt
I have a function on a 2d grid that looks like a skewed mound. I would
like to calculate everything in the 68% (one sigma or standard
deviation) confidence interval, and 95% (two sigma) as seen done in
this plot: http://lambda.gsfc.nasa.gov/product/map/current/pub_papers/threeyear/parameters/images/Med/ds_f07_PPT_M.png.
 (The first line is 68% and second line is 95% confidence intervals.)
I would like to the set everything in the 68% confidence interval to
1, 95% to 2 and everything else to 0.  Thanks.

It depends somewhat on what that function is. Is it a proper PDF?
Let's say it is.

I'll assume that the data has been normalized such that the sum of all
the entries is 1. If not you'll have to keep track of that
normalization constant. Let's call this array normed_pdf. What you are
going to do is make a function that takes a value x, computes the sum
normed_pdf[normed_pdf > x].sum(), and then subtracts the target value,
say 0.68. Then use a root finder from scipy.optimize to find out where
this function equals 0. Repeat for 0.95. Now just do a contour plot
using those two values as the contour lines; e.g. in matplotlib, use
the contour(Z, V) form.

[Joseph Smidt]

Thanks, that's what i needed!

Post by Robert Kern

Post by Joseph Smidt
I have a function on a 2d grid that looks like a skewed mound. I would
like to calculate everything in the 68% (one sigma or standard
deviation) confidence interval, and 95% (two sigma) as seen done in
this plot: http://lambda.gsfc.nasa.gov/product/map/current/pub_papers/threeyear/parameters/images/Med/ds_f07_PPT_M.png.
 (The first line is 68% and second line is 95% confidence intervals.)
I would like to the set everything in the 68% confidence interval to
1, 95% to 2 and everything else to 0.  Thanks.

It depends somewhat on what that function is. Is it a proper PDF?
Let's say it is.
I'll assume that the data has been normalized such that the sum of all
the entries is 1. If not you'll have to keep track of that
normalization constant. Let's call this array normed_pdf. What you are
going to do is make a function that takes a value x, computes the sum
normed_pdf[normed_pdf > x].sum(), and then subtracts the target value,
say 0.68. Then use a root finder from scipy.optimize to find out where
this function equals 0. Repeat for 0.95. Now just do a contour plot
using those two values as the contour lines; e.g. in matplotlib, use
the contour(Z, V) form.
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
 -- Umberto Eco
_______________________________________________
SciPy-User mailing list
http://mail.scipy.org/mailman/listinfo/scipy-user

[Johann Cohen-Tanugi]

sounds like a perfect call for a scipy cookbook page!
JCT

Post by Robert Kern

Post by Joseph Smidt
I have a function on a 2d grid that looks like a skewed mound. I would
like to calculate everything in the 68% (one sigma or standard
deviation) confidence interval, and 95% (two sigma) as seen done in
this plot: http://lambda.gsfc.nasa.gov/product/map/current/pub_papers/threeyear/parameters/images/Med/ds_f07_PPT_M.png.
(The first line is 68% and second line is 95% confidence intervals.)
I would like to the set everything in the 68% confidence interval to
1, 95% to 2 and everything else to 0. Thanks.

It depends somewhat on what that function is. Is it a proper PDF?
Let's say it is.
I'll assume that the data has been normalized such that the sum of all
the entries is 1. If not you'll have to keep track of that
normalization constant. Let's call this array normed_pdf. What you are
going to do is make a function that takes a value x, computes the sum
normed_pdf[normed_pdf > x].sum(), and then subtracts the target value,
say 0.68. Then use a root finder from scipy.optimize to find out where
this function equals 0. Repeat for 0.95. Now just do a contour plot
using those two values as the contour lines; e.g. in matplotlib, use
the contour(Z, V) form.