Applied Statistics for Data Science, Machine Learning, and AI Engineers
A Practical, Open-Source Guide to Regression, Gradient Boosting, Bayesian Methods, and A/B Testing in Python
Preface
This book is a practical statistics handbook for data scientists and engineers who need to run, read, and defend an experiment, not just pass a stats exam. Every chapter pairs the underlying theory with a worked example built around one running dataset: simulated response times for a checkout API, followed through descriptive statistics, hypothesis testing, regression, tree-based models, and a Bayesian A/B test.
Every figure in the book is a Plotly chart with a slider. Drag it, and the chart updates in your browser, no server and no setup required. The source for every chapter is plain Markdown; the source for every figure is a plain Python script in chapters/, so nothing here is a black box, and every chart is regenerable from the code sitting right next to the prose that uses it.
This book is free and open source.
See it in action
Three of the charts that appear later in the book, live and interactive right now. Drag any slider below before reading a word of Chapter 1.
Frequently asked questions
Is this book free? Yes. The full text is free to read online, with no paywall, signup, or account.
Can I reuse the text or figures? Yes, under CC BY 4.0 (text) and MIT (code), with attribution to Rudrendu Paul.
Does reading this book require a Jupyter kernel or a running Python server? No. Every figure is pre-rendered to a self-contained interactive HTML page, so the book works from a static file server and needs nothing more than a browser.
Does this book cover statistical significance, p-values, and alpha? Yes. Chapter 2 covers the p-value, the significance threshold (alpha), and warns against the p-value-as-probability-of-truth misreading. Chapter 4 separately covers statistical vs. practical significance.
Does this book cover Type I and Type II errors and statistical power? Yes. Chapter 2 covers Type I error, Type II error, the trade-off between them, and statistical power as a function of sample size and effect size, each with an interactive figure.
Does this book cover precision, recall, F1, and ROC/AUC? Yes. Chapter 7 has a dedicated section on evaluating a classifier, with an interactive confusion-matrix figure where dragging the classification threshold live-updates precision, recall, F1, and the ROC curve.
What is the difference between a p-value and the probability the null hypothesis is true? A p-value is computed by assuming the null hypothesis is true, and it measures how surprising the observed data would be under that assumption. It does not measure the probability that the hypothesis itself is true. Chapter 2 covers this distinction directly.
What is the difference between a confidence interval and a Bayesian credible interval? A confidence interval is a statement about a repeated procedure: 95% of intervals generated across repeated experiments would contain the true value. A credible interval is a direct probability statement given the observed data: a 95% credible interval means a 95% posterior probability the parameter lies in that range. Chapter 9 covers both side by side.
How does Ridge regression relate to a Gaussian prior? Ridge regression’s L2 penalty is mathematically equivalent to placing a Gaussian prior on the regression coefficients and taking the posterior mode. Chapter 4’s preview and Chapter 9’s full treatment both derive this.
How does Bayesian A/B testing differ from a classical significance test? Chapter 14 covers this end to end: Bayesian A/B testing produces a posterior distribution over each variant’s conversion rate, a win probability, and an expected-loss stopping rule, instead of a single p-value.
Why does a random forest split on only a random subset of predictors at each split? Restricting each split to a random subset of predictors (Chapter 7) forces trees to occasionally split on weaker predictors instead of always defaulting to the strongest one, decorrelating the trees so averaging reduces variance more effectively.
Why are boosted trees kept shallow while random forest trees are grown deep? A random forest (Chapter 7) controls variance by averaging many deep, individually overfit trees. A boosted model (Chapter 8) keeps its trees shallow on purpose, often 2 to 6 levels, so no single tree overcorrects.
How does robust regression handle outliers differently than ordinary least squares? Ordinary least squares minimizes squared error, so one extreme point can pull the fitted line far from the rest of the data. Chapter 9 covers robust Bayesian regression with a Student-t likelihood, whose heavier tails keep an outlier from dominating the fit.
What is a hierarchical (multilevel) Bayesian model? A model that fits a per-group estimate for every group while letting each borrow statistical strength from a shared population distribution, so a sparsely observed group shrinks toward the population’s typical behavior instead of reporting a noisy estimate built from almost nothing. Chapter 9 and Chapter 14 cover this.
Can a Gaussian process be used for classification, not just regression? Yes. Chapter 11 covers Gaussian process classification, where a latent function passes through a sigmoid link into a Bernoulli likelihood, producing a flexible decision boundary a plain logistic regression cannot represent.
What is quantile regression, and how does it differ from predicting the mean? Quantile regression predicts a specific quantile of an outcome’s distribution, such as the 95th percentile, instead of its mean, catching tail behavior a mean-based model would miss. Chapter 13 covers this alongside conformal prediction, a distribution-free method for a prediction interval with a coverage guarantee.
What is a sample ratio mismatch (SRM) in A/B testing? A gap between an A/B test’s intended and observed traffic split, checked with a chi-square goodness-of-fit test; an undetected SRM invalidates the test’s conclusion regardless of how the outcome comparison came out. Chapter 14 covers this.
Can an A/B test look like a win on average while harming a subgroup of users? Yes. Chapter 14 covers a latent (unobserved) subgroup heterogeneity example, where a two-component mixture model surfaces a harmed subgroup that a single pooled analysis would miss.