Department
Survival analysis, Bayesian experimentation, cohort economics, and the statistical infrastructure that turns raw data into decisions worth acting on.
9 essays
12 defined terms
The thesis
The gap between a dashboard and a decision system is where most analytics investment goes to die.
The gap between a dashboard and a decision system is where most analytics investment goes to die.
Descriptive metrics tell you what happened; they cannot tell you what to do next or why it matters.
This series bridges that gap with the quantitative methods that underpin genuinely data-driven organizations — survival models for churn prediction, Bayesian frameworks for experimentation under uncertainty, cohort-level unit economics, anomaly detection systems that distinguish signal from noise, and the metric ontology design that ensures everyone in the organization is optimizing for the same definition of success.
Core concepts in this department
Bayesian inference updates prior beliefs about a parameter using observed data via Bayes' theorem to produce a posterior distribution. In A/B testing it directly answers 'what is the probability that B beats A?' — the question product teams actually ask — unlike the indirect counterfactual framing of frequentist p-values.
Survival analysis models time-to-event data — how long until a customer churns, a subscription renews, a machine fails — accounting for censored observations where the event has not yet occurred. Cox proportional hazards is the standard semi-parametric model; deep recurrent survival models handle non-proportional hazards.
Cohort analysis groups users by a shared origin event (acquisition month, first purchase, signup source) and tracks behavior over time for each group. It separates true retention from the compositional distortion caused by new-user dilution, and is the foundational unit of analysis for subscription economics.
Anomaly detection identifies observations that deviate meaningfully from expected behavior, accounting for trend, seasonality, and variance. In revenue data it separates true incidents (payment outages, pricing bugs) from normal fluctuation. Isolation forests and Prophet-based decomposition are the practical workhorses.
Product-market fit is the empirical condition where a cohort's retention curve flattens above zero — a group of users has found sufficient value to make the product a persistent part of their behavior. It is not a feeling; it is a quantifiable property of retention, NPS decomposition, and usage depth.
Analytics engineering is the discipline of building reliable, tested, version-controlled transformations on top of a cloud warehouse, bridging data engineering and analysis. Tools like dbt, Dagster, and Airbyte formalize a software-engineering workflow for SQL transformations with tests, documentation, and lineage.
A metric ontology is a versioned, centrally-governed definition of every metric an organization uses, specifying the grain, filters, time-window, and source tables so that the same metric produces identical values regardless of tool, dashboard, or analyst. It prevents the drift that silently corrupts data-driven decisions.
Unit economics is the financial performance of a single customer (or transaction) decomposed into acquisition cost, gross margin per period, retention, and payback period. Cohort-level unit economics — computed per acquisition cohort rather than rolled up — is the only form that survives growth-driven distortion.
The peeking problem is the inflation of false-positive rates that occurs when a frequentist A/B test is repeatedly evaluated before reaching its pre-registered sample size. A nominal 5% false-positive rate can become 20–30% under daily peeking. Bayesian testing and sequential-analysis methods eliminate the problem.
The Cox proportional hazards model is the semi-parametric workhorse of survival analysis: it estimates how covariates multiply the baseline hazard rate without requiring a parametric form for the baseline. It yields interpretable hazard ratios under the assumption that the ratio is constant over time.
Isolation Forest is a tree-based anomaly detection algorithm that scores observations by how easily they can be isolated via random recursive partitioning. Anomalies are isolated in few splits; normal points require many. The algorithm handles mixed feature types without density estimation or distance calculations.
The dashboards-to-decisions gap is the structural failure of analytics investment: teams produce more dashboards but decisions don't get better or faster. Closing the gap requires moving from descriptive reports to decision systems — pre-specified trigger thresholds, automated action routing, and outcome logging for calibration.
Essays in this department
Before dbt, analysts wrote SQL that nobody reviewed, nobody tested, and nobody documented. The tool was simple — SQL templating with version control. The impact was structural: it created an entirely new discipline.
Posted by
Murat OvaCorrelation tells you that feature usage and retention move together. It doesn't tell you which causes which — or whether a third factor drives both. Causal discovery algorithms can untangle this from observational data alone.
Posted by
Murat OvaYour company has 47 dashboards. How many of them changed a decision last week? Dashboards describe what happened. Decision systems prescribe what to do next — and the gap between these two is where most analytics ROI evaporates.
Posted by
Murat OvaYour company's monthly revenue is growing 20% year-over-year. Your unit economics are deteriorating. Both statements are true simultaneously — and you'll never see the second one in an aggregate P&L.
Posted by
Murat OvaAsk five people in your company what 'revenue' means and you'll get five different numbers. The problem isn't the data warehouse — it's that nobody agreed on the definitions before building dashboards on top of them.
Posted by
Murat Ova'You'll know product-market fit when you feel it' is advice that has burned through billions in venture capital. Here's a quantitative framework that replaces gut feeling with a composite score — and it starts with retention curves, not surveys.
Posted by
Murat OvaBinary churn models answer the wrong question. 'Will this user churn?' matters less than 'When will this user churn?' Survival analysis models the timing — and the when determines whether intervention is profitable.
Posted by
Murat OvaA 4% revenue drop on a Tuesday could be a payment processor outage, a pricing bug, or just normal variance. The difference between these explanations is millions of dollars — and your monitoring system can't tell them apart.
Posted by
Murat OvaFrequentist A/B testing answers a question nobody asked: 'If the null hypothesis were true, how surprising is this data?' Bayesian testing answers the question that matters: 'Given this data, what's the probability that B is actually better?'
Posted by
Murat Ova