Katkat
Back

How Would You Test a System That Counts the Leaves on a Tree?

GleanSoftware Engineer · Technical Screen
MediumSoftware EngineerTechnical ScreenSoftware Engineering Fundamentals

Last updated

This problem evaluates your ability to design a comprehensive testing strategy for an ambiguous, real-world system where an exact 'ground truth' is difficult to obtain.

The Scenario

You are tasked with designing a test suite for a vision-based system that uses a high-resolution camera mounted on a robotic arm to count the total number of leaves on a specific tree. The system processes images, performs segmentation, and outputs an integer representing the leaf count.

Your Task

Describe how you would approach testing this system. You should structure your strategy to address the following areas:

  1. Verification of Accuracy (The Oracle Problem): Since counting thousands of leaves manually is error-prone, how do you define a source of truth for your test cases?
  2. Functional Test Cases: What specific scenarios (e.g., varying lighting, occlusion, seasonal changes) must the system handle?
  3. Edge Cases and Robustness: How do you handle non-leaf objects, partial leaves, or damaged foliage?
  4. Non-Functional Requirements: What metrics would you track regarding performance and reliability, and how would you test them?

Provide your response as a structured test plan, outlining the methodologies you would use to ensure the system is production-ready.

Hints

Hint 1

Consider using synthetic data or controlled environments where the ground truth is known, such as a 3D-modeled digital twin of a tree.

Hint 2

Think about the difference between testing the computer vision model's inference performance versus the integration of the camera hardware and robotics.

Hint 3

Define what failure looks like: is a 5% error margin acceptable, and how does that influence your test data selection?

Follow-up questions

  • How would you handle a scenario where the system is consistently undercounting leaves in high-wind conditions?
  • If the model is deployed to production and users report drift in accuracy, how would you implement a continuous monitoring and testing feedback loop?
  • How would you adjust your testing strategy if the system needed to work on different species of trees with drastically different leaf shapes and sizes?