Human-in-the-Loop AI ยท 2025
Human-AI Feedback Loop
A full-stack customer-support platform implementing a practical Human-in-the-Loop (HITL) system. Combines Rasa NLU automation with real-time human agent escalation and structured feedback capture for continuous model retraining.
The Challenge
Customer support bots often fail abruptly when encountering queries outside their training distribution. Users needed an uninterrupted transition to live human agents without losing conversation context, while engineering teams needed structured resolution data to prevent repeated errors.
Architecture & Services
Intent and entity parsing with dialogue state execution on port 5005.
Custom actions handling confidence scoring, escalation triggers, and SQLite persistence.
REST endpoints orchestrating pending queues, agent messaging, and feedback analytics.
User-facing chatbot widget paired with an administrative agent escalation dashboard.
Escalation & Retraining Loop
When intent confidence falls below the predefined threshold or the user explicitly asks for human help, the conversation state is persisted to SQLite. The agent dashboard polls the Flask API, enabling human agents to intervene live, resolve the issue, and categorize the resolution with problem tags โ directly feeding future Rasa retraining pipelines.
Context preserved during escalation
Decoupled full-stack architecture
SQLite resolution history capture