Katkat

Repair a Validator Whose Error Lists Leak None and Nest

StripeSoftware Engineer · Technical Screen

Last updated

Given a small schema-validation library and a failing golden-file diff, trace stray None values, nested error lists, and an inconsistent null-versus-[] result back to their source and fix each with a regression test. Tests root-cause debugging, return-type contracts, and safe flattening.

A small Python validation library, formcheck, checks nested user input against a schema and returns a list of human-readable error messages for each record. A nightly job runs report(schema, records) over a fixture set and diffs the JSON output against a golden file. The diff is failing.

You are given the library source, which has bugs in it, and the job's output. Find each bug at its source, fix it there, and add a regression test for each. Do not edit the golden file and do not post-process the report.

The library

def min_len(n):
    def check(name, value):
        if len(value) < n:
            return f"{name}: shorter than {n}"
    return check

def max_len(n):
    def check(name, value):
        if len(value) > n:
            return f"{name}: longer than {n}"
    return check

def digits_only(name, value):
    if not value.isdigit():
        return f"{name}: digits only"

def no_spaces(name, value):
    bad = [f"{name}: space at {i}" for i, c in enumerate(value) if c == " "]
    return bad  # a check may return None, a string, or a list of strings


class Field:
    def __init__(self, name, checks=(), required=False):
        self.name, self.checks, self.required = name, list(checks), required

    def errors(self, value):
        if value is None:
            return [f"{self.name}: required"] if self.required else None
        out = []
        for check in self.checks:
            out.append(check(self.name, value))
        return out


class Group:
    def __init__(self, name, children, required=False):
        self.name, self.children, self.required = name, children, required

    def errors(self, data):
        if data is None:
            return [f"{self.name}: required"] if self.required else None
        if not isinstance(data, dict):
            return [f"{self.name}: expected object"]
        out = []
        for child in self.children:
            out.append(child.errors(data.get(child.name)))
        return out or None


class Each:
    def __init__(self, name, item):
        self.name, self.item = name, item

    def errors(self, items):
        if items is None:
            return []
        return [self.item.errors(v) for v in items]


def report(schema, records):
    return [schema.errors(r) for r in records]

The job

schema = Group("signup", [
    Field("username", [min_len(3), no_spaces], required=True),
    Group("address", [Field("zip", [digits_only]), Field("city", [min_len(2)])]),
    Each("tags", Field("tag", [max_len(8)])),
])
records = [
    {"username": "jo e", "address": {"zip": "9a1", "city": "Rome"}, "tags": ["ok", "verylongtag"]},
    {"username": "maria", "tags": []},
    {"username": "maria", "address": {"zip": "", "city": ""}},
    {"address": "Main St"},
]

Actual output (one JSON line per record):

[[null, ["username: space at 2"]], [["zip: digits only"], [null]], [[null], ["tag: longer than 8"]]]
[[null, []], null, []]
[[null, []], [["zip: digits only"], ["city: shorter than 2"]], []]
[["username: required"], ["address: expected object"], []]

Golden output:

["username: space at 2", "zip: digits only", "tag: longer than 8"]
[]
["zip: digits only", "city: shorter than 2"]
["username: required", "address: expected object"]

Part 1: null entries

Explain where each None comes from and remove it where it is created. A check that passes must contribute nothing, and an empty string value (like zip: "") is still a value that has to be checked.

Part 2: nesting

Every result must be a flat list in document order, however deeply the schema (or a check's own return value) is nested.

Part 3: one empty value

Missing optional sections currently give null in some places and [] in others. Choose one representation for "no errors", justify it, and make every node type return it.

Deliver: the patched library, a short note per bug, and unit tests that fail on the original code and pass on yours.

Hints

Hint 1

Call Field.errors on one value by itself. Which part of the output is already wrong at that level?

Hint 2

What does a check return when the value passes, and what does Field do with that return value?

Hint 3

If every child returns a flat list, which list method keeps the parent flat?

Hint 4

Pick one return type for 'no errors' and check every node class against it, including the ones that look fine.

Follow-up questions

  • How would you enforce the list[str] contract so a future node type cannot break it (type hints, a base class, a runtime assertion)?
  • Error messages now need a machine-readable path like address.zip. How do you thread the path through without nesting?
  • Should validation stop at the first error per field? How would you make that configurable?