The churn that wasn't
Every month I drop a dataset with real traps in it — not exam traps, the kind I've actually run into building my own projects. This month I hid two of them in a customer-churn case. If you catch them, you already think like someone who's been burned by a data leak.
Solution: end of the monthThe dataset
60 customers of a subscription app, six columns: user_id, plan (basic, pro or team), days_active (account age in days), support_tickets (open support tickets), last_login_days (days since the last login) and churned (1 if they cancelled, 0 if still active). Download it, open it in pandas, and look before you touch anything.
Download the dataset (CSV)A small CSV, ready for pandas.read_csv(). No sign-up, nothing asked for.
The questions
- 01
What accuracy does a dumb model get by always predicting "this customer stays"? Work it out before training anything — it's your reference floor.
- 02
One column leaks information from the future, measured after the thing you're trying to predict already happened. Which one is it, and why is it cheating?
- 03
How many rows in the CSV are exact duplicates? If the train/test split happens before you catch them, what happens to your evaluation when one copy lands in train and its twin lands in test?
Hints, if you get stuck
Hint 1
Before training anything, run value_counts() on churned and duplicated() on the whole dataframe. The numbers will tip you off more than you'd expect.
Hint 2
For every column, ask: would this value actually be available at prediction time, or could it only have been known after the customer had already left?
Got it?
Publish your solution wherever you like: a repo, a gist, a Colab notebook. Then send it to me through the contact form, or just mention me on GitHub @delcenjo. At the end of the month I publish my own commented solution on the blog and link the best ones that reach me.