Drift Drill Down

machine learning, data visualisation

The Drill Down tab shows how the data a deployed model scores in production drifts, feature by feature and over time, away from the data it was trained on. It reached DataRobot’s cloud users as a public preview in October 2022.

Drift is measured as the Population Stability Index and drawn as a heat map: one row per feature, one column per time bucket, each cell healthy, at risk or failing against a threshold. A bar chart beneath shows the prediction sample size behind each bucket, so a red cell backed by a handful of predictions reads differently from one backed by thousands. Dragging across the heat map sets the comparison period.

Animated recording of comparing data drift heat maps across deployment features over time

Selecting one or more features opens a Feature Drift Comparison chart for each, setting its distribution in a reference period against a comparison period. That is where the cause shows up: a data quality issue, a change in feature composition, or a change in the context of the target variable.

Feature Drift Comparison charts in DataRobot comparing reference and comparison period distributions for utilization, hour of day, and wind chill features

Hovering a bucket gives the exact share of records in each period, so a gap between two bars can be checked against a number before anyone retrains on it.

Feature Drift Comparison histogram for a utilization feature, with a tooltip showing 46.1% of reference records and 34.4% of comparison records in the 0 to 82.3 bucket

Documentation · October 2022 release notes