About Normality and Testing for Normality

It is often said that with small sample sizes, everything looks normal, as the normality tests are, indeed, very sensitive to what goes on in the extreme tails. In other words, if we have enough data to fail a normality test, we always will because our real-world data won’t be clean enough. If we don’t have enough data to reliably fail a normality test, then there’s no point in performing the test, and we have to rely on the fat pencil test or our own understanding of the underlying processes. Read the detailed reasoning at:

http://www.r-bloggers.com/normality-and-testing-for-normality/

Why one shouldn't use Bivariate Correlations for Variable Selection?

In applied statistics, what typically happens is a researcher sits down with their statistical software of choice and they compute a correlation between their response variable and their collection of possible predictors. From here, they toss out potential predictors that either have low correlation or for which the correlation is not significant. The concern here is that it is possible for the correlation between the marginal distributions of the response and a predictor to be almost zero or non-significant and for that predictor to be an important element in the data generating pathway. Read more about why we shouldn't be using bivariate correlations for variable selection..

http://www.r-bloggers.com/stop-using-bivariate-correlations-for-variable-selection/

The Improbability Principle

The video and slides from David Hand's lecture on the subject of his new book 'The Improbability Principle'.

http://www.statslife.org.uk/events/events-calendar/icalrepeat.detail/2014/03/13/135/-/180th-anniversary-event-david-hand-the-improbability-principle

It is about extraordinarily improbable events. It’s about events which are so unlikely that we wouldn’t expect to see them during our entire lifetimes - or even the lifetime of the human race or the universe itself. And it’s about why, despite all that, we do see such events; and more, it’s about why we them again and again.

Coloured Noise

Have you ever wondered that there could be other colors to our all time favorite White Noise... like Red, Pink or Green. Read more about these coloured noises at:

http://www.ee.columbia.edu/~dpwe/noise/

Musings on Random Walk

"A drunk man will find his way home, but a drunk bird may get lost forever."
- Shizuo Kakutani
Want to know why? Read at:

http://mahalanobis.twoday.net/stories/228354/
http://www.math.cornell.edu/~mec/Winter2009/Thompson/randomwalks.html

Animation of the Construction of a Confidence Interval

The confidence interval is one of the more tricky statistical concepts. A way of explaining confidence intervals is as the region of possible null hypotheses resulting in corresponding significance tests that are not rejected. Turns out it is not easy to make a corresponding nice explanatory animation either, but that’s what has been tried here:

http://www.r-bloggers.com/an-animation-of-the-construction-of-a-confidence-interval/

Understanding gamma function

Waiting in One Line or Multiple Lines

As a statistician, what is better, having a single queue or multiple queues, in regard with waiting time? Have a look at the following link:

http://www.r-bloggers.com/waiting-in-one-line-or-multiple-lines/

Understanding Simpson's Paradox

When you look for overall trends, you often poke around the data in aggregate, but when you zoom out too far, you could miss details or within-category variation. Sometimes when you zoom in, you see a completely opposite trend of what you saw overall. This is known as Simpson's Paradox.

http://flowingdata.com/2013/09/19/a-visual-explanation-of-simpsons-paradox/