Fix Three Planted Bugs in a Tiny Python Template Engine

You inherit tinytpl, a 100-line template engine used to render notification emails. A teammate reports three failures and suspects that bugs slipped in during a refactor. Find each root cause, fix it with the smallest correct change to the library (not the callers), and add a regression test for each fix.

How the engine works

1. tokenize splits the source into text and three kinds of tags: {{ expr }} (expression with optional |filter chain), {% ... %} (control tags for/endfor and if/endif), and {# ... #} (comments).
2. parse builds a tree of ("text", ...), ("expr", ...), ("for", ...) and ("if", ...) nodes.
3. render_nodes walks the tree with a context dict.
4. Loader(root).load(name) reads a template file that must live inside root.

import os
import re

TOKEN_RE = re.compile(r"(\{\{.*\}\}|\{%.*?%\}|\{#.*?#\})", re.S)
FILTERS = {"upper": str.upper, "lower": str.lower, "trim": str.strip}


class TemplateError(Exception):
    pass


def tokenize(src):
    return [t for t in TOKEN_RE.split(src) if t]


def parse(tokens, pos=0, until=None):
    """Return (nodes, next_pos). Stops when it meets a tag named in `until`."""
    nodes = []
    while pos < len(tokens):
        tok = tokens[pos]
        if tok.startswith("{#"):
            pos += 1
        elif tok.startswith("{{"):
            nodes.append(("expr", tok[2:-2].strip()))
            pos += 1
        elif tok.startswith("{%"):
            words = tok[2:-2].split()
            if until and words[0] in until:
                return nodes, pos
            if words[0] == "for":                     # {% for x in items %}
                end = tokens.index("{% endfor %}", pos)
                body, _ = parse(tokens[pos + 1:end])
                nodes.append(("for", words[1], words[3], body))
                pos = end + 1
            elif words[0] == "if":                    # {% if cond %}
                body, pos = parse(tokens, pos + 1, until=("endif",))
                nodes.append(("if", words[1], body))
                pos += 1
            else:
                raise TemplateError(f"unknown tag {words[0]!r}")
        else:
            nodes.append(("text", tok))
            pos += 1
    if until:
        raise TemplateError(f"missing {until[0]}")
    return nodes, pos


def lookup(ctx, dotted):
    val = ctx
    for part in dotted.split("."):
        val = val[part] if isinstance(val, dict) else getattr(val, part)
    return val


def eval_expr(expr, ctx):
    name, *filters = [p.strip() for p in expr.split("|")]
    val = str(lookup(ctx, name))
    for f in filters:
        val = FILTERS[f](val)
    return val


def render_nodes(nodes, ctx):
    out = []
    for node in nodes:
        kind = node[0]
        if kind == "text":
            out.append(node[1])
        elif kind == "expr":
            out.append(eval_expr(node[1], ctx))
        elif kind == "for":
            _, var, seq, body = node
            for item in lookup(ctx, seq):
                out.append(render_nodes(body, {**ctx, var: item}))
        elif kind == "if":
            _, cond, body = node
            if lookup(ctx, cond):
                out.append(render_nodes(body, ctx))
    return "".join(out)


def render(src, ctx):
    nodes, _ = parse(tokenize(src))
    return render_nodes(nodes, ctx)


class Loader:
    def __init__(self, root):
        self.root = root

    def load(self, name):
        path = os.path.join(self.root, name)
        if not os.path.isfile(path):
            raise TemplateError(f"template not found: {name}")
        with open(path, encoding="utf-8") as fh:
            return fh.read()

Reported symptoms

Symptom 1. Two expressions on one line crash:

>>> render("Hi {{ first }} {{ last|upper }}!", {"first": "Ada", "last": "Byron"})
KeyError: 'first }} {{ last'

Expected output: Hi Ada BYRON!

Symptom 2. Nested loops crash:

>>> render("{% for row in grid %}[{% for c in row %}{{ c }}{% endfor %}]{% endfor %}",
...        {"grid": [[1, 2], [3]]})
ValueError: list.index(x): x not in list

Expected output: [12][3]

Symptom 3. A security review found that Loader("/srv/app/templates").load("../config/secrets.env") happily returns the file's contents. So does passing an absolute path. Template names come from a database column that support staff can edit.

Constraints

• Fix one bug at a time, and explain how you traced each symptom to its cause before changing code.
• Keep the public API unchanged.
• Your fixes must not break existing behavior: comments, if blocks, filters and dotted lookups.
• Standard library only.

Deliverables

For each bug: the root cause in one or two sentences, the diff, and a regression test. Finally, name one more latent issue you noticed but did not fix.