AI-Powered Tech Support Agents

An enterprise solution powered by LLM-RAG to deploy tech support AI agent

Led the successful delivery of an enterprise-grade LLM-based AI Agent solution across two-phased project, acting as both Technical Lead and Project Manager. This experience further deepened my expertise in building enterprise AI Agent solutions, while also strengthening my leadership capabilities.

Contents Overview
📚 Product Introduction

  • Background

    Our client feels that the current manual handling of technical support tasks is inefficient. There is a need for integrated management of customer support responses and the provision of an optimized environment to improve user convenience. In addition, a platform that provides analysis tools for customer conversations would be highly desirable.

  • Goal

    Based on the background, we aim to build a solution that:
    • Delivers instant, high-quality services for various customer tech support operations.
    • Learns from internal technical documents and refer it as a guidance for query.
    • Increases customer convenience and transparancy by allowing the download of relevant documents.
    • Supports multiple languages to meet a wide range of customer needs.

    Ultimately, the solution will reduce the workload on existing support teams. This will lower holistic operational costs by allowing support staff to be redeployed to more strategic tasks. Moreover, the solution will ensure high-quality and transparent services, ultimately established stronger customer trust.

  • Solutions

    Our solution is divided into three modules: LLM-RAG Engine, Agent Personalization, and Monitoring Dashboard.

    LLM-RAG Engine:

    • Lightweight, powerful, and scalable generative engine
    • Enhances Agent knowledge with internal docs through RAG.
    • Supports 8 languages for wide accessibility.

    Agent Personalization:

    • Persona: Customize language, age, and interests for a relatable persona.
    • Flow: Set dialogue paths and default responses.
    • Safeguard: Content filtering, privacy settings, and escalation to human agents.
    • Guidance: Edit prompts and enhance user intent recognition.

    Monitoring Dashboard:

    • Manage knowledge database effectively.
    • Monitor and filter user conversation streams.
    • Track performance and report conversation logs.

    System Feature Overview to Showcase Project Scope.


🌟 My Role and Responsibilities:
  • Project Manager: Led planning, coordination, and delivery across phases, ensuring stakeholder requirements were met.
  • Tech Lead (AI): Designed and implemented AI agent, drove key technical decisions, and contributed core system code.
  • System Architect: Led the design and architecture of AI, platform, and cloud infrastructure, ensuring alignment with architectural requirements.


💡 Key Achievements:

  • ✅ Secured Project Approval

    Problems

    The client expressed interest in adopting Agent-based AI technology but remained uncertain about its feasibility, ROI, and technical integration within existing workflows.

    Solutions
    • Developed a targeted proof-of-concept demonstrating core agent capabilities.
    • Conducted iterative reviews and incorporated stakeholder feedback.
    • Performed benchmark analysis against comparable market solutions.

    My Contributions
    • Lead the effort on develop pretty convincing POC. In this case, we showcase agent potential on answering user query, using curated article as reference to generate answer.
      Chat Interface for Our POC.
    • Conducted iterative reviews and performed benchmark analysis against comparable market solutions.
      Snippets of how we do feature benchmark to leading similar solution provider.
      Conducting research on the market share of CRM solution providers.
      Identifying potential patentable ideas by benchmarking existing domestic patents.

  • 🕒 Consistent On-Schedule Delivery

    Problems

    The project involved emerging technologies (LLM-RAG, AI Agent) ach requiring specialized technical familiarity. It was split into sequential phases, where the previous phase needed to succeed to proceed to the next.

    Solutions
    • Design a modular system to reduce cross-dependency between components.
    • Define clear acceptance criteria to ensure each phase's delivery is acceptable.
    • Streamline onboarding through concise documentation and knowledge-sharing sessions.

    My Contributions
    • Coordinated project planning, tracked progress, and ensured timely delivery. I shared my approaches to working in this tasks. [Learn more]
    • Defined the high-level architecture and integration strategy.
      I used this kind of diagram as communication media to devise development strategy with the team.
    • Led the detailed design and development of the AI Agent.
      I used Flowcharts to convey AI agent logic, as they were effective for debugging, pinpointing potential improvements, and as onboarding material.

  • 📈 Improved Scalability Concurrent User

    Problems

    The initial infrastructure exhibited inconsistent performance under concurrent load. Due to limited hardware resources, the system could not reliably support a predictable number of users. Under stress, it frequently experienced latency spikes and even crashed.

    Solutions
    • Develop a fallback mechanism to maintain smooth user experience.
    • Re-architect the system to support a fallback mechanism and overall scalability.
    • Optimize system components to improve overall performance.

    My Contributions
    • Implemented Dockerization for the AI Agent Engine to ensure efficient deployment and scalability. Interestingly, this task became an inspiration for my open-source project. [Learn more]
    • Engineered a cloud-based infrastructure to support scalability and ensure fallback mechanism integration.
      Using the initial infrastructure diagram (left) as a reference, I devised a production-ready infrastructure (right) with a core focus on scalability and resilience.
    • Refactored a previously blocking function into an asynchronous implementation, resolving thread bottlenecks that had limited concurrency to just 2 users even with vertical scaling. After the fix, the system reliably handled 4+ concurrent users on baseline hardware and scaled efficiently with additional resources.
      The previous implementation (black line) exhibited imperfect concurrency, with requests partially blocked and processed sequentially. The improved version (colored line) achieves true concurrency, where multiple requests begin processing nearly simultaneously.

  • 📈 Improved Agent Issue Resolution Rate

    Problems

    The initial version of the AI agent, alongside the early collection of documents registered in the vector database, was evaluated by the client’s internal tech support team and showed a low issue resolution rate.

    Solutions
    • Utilizing past technical support conversation logs has strong potential as a valuable knowledge base.
    • Improve retrieval mechanisms by optimizing the vector database, ensuring faster and more accurate document matching.

    My Contributions
    • Refactored the Chain-of-thought (CoT) pipeline to enhance processing speed and improve retrieval accuracy.
    • Proposed a data sanitization pipeline to prevent information leakage and optimal knowledge content.
      This pipeline restructures content to improve retrieval quality and answer accuracy, while ensuring no sensitive data is leaked. Data sanitization is performed locally to prevent exposure, followed by content restructuring using a 3rd-party SoTA model. A multi-layer approach is used to mask the data efficiently, considering local machine limitations.

  • 📉 Optimized System Operational Cost

    Problems

    The client needed flexible infrastructure options tailored to different usage stages to minimize and optimize operational costs.

    Solutions
    • Multiple infrastructure options to accommodate various usage stages.
    • Conduct thorough system benchmarking to evaluate potential infrastructure setups.
    • Insights into industry standards for performance and cost for similar systems

    My Contributions
    • Researched infrastructure solutions to support the system and conducted benchmarking studies. We based this effort on a case study from IBM, which guided the direction of our work. [Click to learn more]
    • Designed cloud-based infrastructure options tailored to client needs.
      The AWS Pricing Calculator was one of our key tools for communicating infrastructure costs, as it provides comprehensive reports that can be tailored to client requests.
    • Evaluate system performance under various setups to identify the pros and cons of each configuration. We align client requests with the agreed-upon acceptance criteria to define appropriate system evaluation settings.
      This table presents the results of testing to determine how many users the system can reliably support under various infrastructure setups.


🔮 Vision and Future Impact:

Through this projects, I’ve discovered a strong interest in agentic AI. This newfound passion has driven me to dive deeper into MLOps. I’ve since focused my personal development and research on these areas, with the goal of honing my capabilities and contributing to innovations in the field.