Airbnb Host Retention Optimization

Airbnb wanted to send $1,000 retention gifts to hosts at risk of leaving. The trap is that a high-accuracy churn model isn't the same thing as a profitable intervention. Gifting low-revenue hosts loses money even if you correctly predict they'll churn.

• Engineered features from 12-month reservation trends, revenue projections, and K-Means geographic clustering (20 clusters).
• Trained a CatBoost ensemble across 8 random seeds with 5-fold cross-validation.
• Applied isotonic calibration to convert model scores into honest probability estimates.
• Optimized a profit threshold (τ* ≈ $1,083). Gift only when attrition_probability × annual_profit clears the threshold.

• Gifting only 307 of 1,312 hosts (23.4%) outperformed both 'gift everyone' and 'gift no one' on the test set.
• The profit-at-risk framing prioritized high-revenue at-risk hosts over equally-likely-to-churn but low-revenue ones. Same model, very different decisions.
• Geographic clustering meaningfully improved feature signal beyond raw lat/lon.

Modeled lift: $5.24M to $5.80M projected profit on a 1,312-host validation slice (about +$557K, or 10.6%). Model AUC was 0.9025, but the real takeaway is that reframing the objective from accuracy to dollars changed which hosts the model recommended treating.