Katkat
Back

Evaluate an Agentic Decision System End to End

MillenniumData Scientist · Technical Screen
MediumData ScientistTechnical ScreenSoftware Engineering Fundamentals

Last updated

This problem tests the ability to design an automated evaluation framework for an agentic system that transitions from rule-based to LLM-driven decision-making, focusing on decoupling metrics and ensuring safety.

The Scenario

You are tasked with building an automated evaluation system for a new 'Agentic Decision Engine' at a financial firm. Currently, the firm uses a legacy system of static IF-THEN rules to route customer support tickets to specific operational desks. The new agent uses LLMs to interpret intent, choose a tool/desk, and generate a recommended action.

The Task

Design a Python-based evaluation framework that validates the agent's performance before it replaces the legacy system. Your architecture should support:

  1. Routing Accuracy: Did the agent choose the correct desk/tool?
  2. Reasoning Integrity: Does the rationale provided by the LLM match the action taken?
  3. Safety Guardrails: How do we detect 'hallucinated' actions that violate internal business logic?
  4. Human Alignment: Compare agent decisions against historical 'Gold Standard' human decisions.

Provide an implementation structure for an AgentEvaluator class that performs an asynchronous evaluation on a batch of test inputs.

Requirements

  • Define a structure for EvaluationResult that captures metrics for routing, reasoning, and safety.
  • Explain how you would implement 'Shadow Mode' to compare the agent against the legacy system without impacting production outcomes.
  • Consider how to handle 'override' logs, where human operators rejected the agent's recommendation.
class AgentEvaluator:
    def __init__(self, golden_dataset, legacy_system):
        # Initialize with test cases and current rule-based system
        pass

    def run_evaluation(self, agent_endpoint):
        # Return a report containing accuracy, drift, and safety violations
        pass

Hints

Hint 1

Consider decoupling the evaluator into three separate modules: an Intent Classifier for routing, a Log-Likelihood scorer for reasoning, and a Deterministic Rule-Checker for safety.

Hint 2

Think of Shadow Mode as a 'log-and-compare' process where the agent runs in parallel with the legacy system, and outcomes are stored in a diff-analysis table.

Hint 3

Use human-in-the-loop override data as 'negative samples' to tune the agent's confidence thresholds and identify failure modes.

Follow-up questions

  • How would you detect 'drift' in the agent's performance after it has been deployed for several weeks?
  • If the agent is non-deterministic (i.e., temperature > 0), how do you ensure the evaluation results are statistically significant?
  • How do you handle edge cases where the legacy system is demonstrably incorrect, making it a poor baseline for comparison?