Katkat
Back

Lease Renewal Prediction and Non-Renewal Reason Categorization Take-Home

EliseAIData Scientist · Technical Screen
MediumData ScientistTechnical ScreenMachine Learning

Last updated

A machine learning take-home on predicting lease renewals from tenant conversations and SQLite tables of residents, maintenance work orders and renewal offers. It covers preprocessing, a metric compared with a random baseline, a test-set prediction file, categorizing non-renewal reasons with gte-base embeddings, and presenting the results.

This take-home gives you 24 hours of work, followed by a 30-minute presentation that serves as a research deep-dive on your approach. The objective is to build a model that predicts the likelihood that a resident renews their lease. One task is a supervised learning problem and one calls for unsupervised learning.

You have two sources of data, and both are split into a training portion and a test portion:

  1. JSON files containing 1,000 conversations with tenants who are considering renewing their leases.
  2. A SQLite database with three tables:
TableContents
residentsResident information
maintenance_historyMaintenance work orders submitted by each resident
renewal_offersRenewal offers made to each unit

Constraints and Clarifications

  • No schema beyond the one-line table summaries is provided. You must discover the columns, the join keys and where the renewal outcome lives by inspecting the data.
  • The prediction file has one row per resident in the test set, with three mandatory columns, resident_id, unit_number and renewal_decision, plus probability_of_renewal if your model produces probabilities. The graders run their own evaluation on the test set.
  • Task 3 must use the gte-base sentence-transformer model.
  • Deliverables: the prediction CSV; a Jupyter notebook with clean, well-documented code, which is submitted and used during the deep-dive; and a short presentation of the high-level approach and results.

Clarifying Questions

  • Is the renewal outcome recorded per resident or per unit, and can one unit hold several residents with different outcomes?
  • How do conversations link to residents or units, and does every resident have one?
  • Were the conversations recorded before the renewal decision, or can they contain the decision itself?
  • Should renewal_decision be a 0/1 flag or a text label, and do false alarms and missed non-renewals cost the same?

Part 1 — Build the renewal prediction model

Build a model that uses the available data to predict whether or not a unit or resident renews. Discuss the preprocessing steps you apply.

What This Part Should Cover

  • Joining the three tables and the conversations at a stated grain, with a located and verified label
  • Preprocessing and features fit on the training split only
  • A leakage review of every feature source, especially the conversations
  • A justified model choice for a small dataset that mixes tables and text

Part 2 — Evaluate against a random baseline and produce test predictions

Identify a metric that demonstrates your model's performance compared with a random baseline. Then output a CSV with a prediction for each resident in the test dataset, in the required format.

What This Part Should Cover

  • A metric whose random-baseline value is known, and why it fits this decision
  • An evaluation protocol on the training split that estimates test performance honestly, with uncertainty
  • How the decision threshold and the probability column are produced and checked

Part 3 — Categorize the reasons tenants do not renew

Using the gte-base sentence-transformer model, train a model to categorize the reasons tenants do not renew their leases. Analyze the reasons tenants give and come up with categories that cover their range. Discuss how you defined the categories and any patterns or insights you observed in the data.

Clarifying Questions for this Part

  • Should the categorizer cover only tenants who did not renew, or every conversation?
  • Can one tenant give more than one reason?

What This Part Should Cover

  • Which text is embedded, and at what granularity
  • A repeatable method for deriving categories, and the human judgment used to name and edit them
  • How the trained categorizer is validated when no labels exist at the start
  • Insights that connect the categories to the structured data

Part 4 — Prepare the research deep-dive

Prepare a short presentation of the high-level approach and results, and have the notebook ready, since it will be used during the 30-minute session.

What This Part Should Cover

  • A storyline from objective to results to recommendations that fits a short slot
  • Honest limitations and next steps
  • A notebook that reruns cleanly and mirrors the presentation

Hints

Hint 1

Fix the grain first: Decide what one training row represents before building features, given that work orders are recorded per resident, offers per unit, and conversations as free text.

Hint 2

What random scores: Work out what a random predictor is expected to score on your chosen metric, and whether that value depends on how many residents renew.

Hint 3

No labels yet: The data contains no reason labels. Think about what the embedding space lets you find before there is anything to train a classifier on.

Hint 4

Expect live questions: The reviewers will have the notebook open. Think about which decisions they are most likely to challenge, and where in the notebook the evidence for each one sits.

Follow-up questions

  • How would the model change if it had to predict renewal months before any offer or conversation exists?
  • How would you check that the reason categories are stable across clustering runs and choices of the number of clusters?
  • If the graders' test-set score came back well below your cross-validated score, what would you investigate first?
  • How could the reason categories feed back into the renewal model, and what leakage would that risk?