AI-Powered Assistant Development – AI Hotel Concierge

SoftTeco built an AI-powered concierge for Cisco Webex Desk Pro that enables voice-driven customer service, secure payments, and real-time assistance in hotels, airports and similar settings.

Type

AI

Industry

Tourism & Hospitality

Country

USA

Highlights

  • Agentic AI pipeline setup
  • Voice interaction and AI avatar communication
  • Vector database storage for RAG
  • Integration with Heygen via native APIs
  • LLM integration for AI agent training
  • AI-assisted engineering with Claude

Client

The client is a US startup that approached SoftTeco with an idea to create an AI-powered virtual assistant for Cisco Webex Desk Pro devices, commonly used in hotels, airports, and similar environments. The client wanted the assistant to communicate with users via audio and video formats, thus enhancing user experience and personalization. Since the client needed strong expertise in AI development, they selected SoftTeco as a recognized AI vendor.

Challenge

The product needed to combine several independent technologies within a single conversational experience: speech recognition, an AI avatar, LLM-based reasoning, RAG, ERP connectivity, and payment processing. The main challenge was therefore not only to implement an AI assistant, but to coordinate multiple third-party services with different APIs, data formats, latency patterns, and failure scenarios while maintaining a natural experience for the user. Since HeyGen provided the avatar-facing part of the solution, SoftTeco was responsible primarily for the backend, AI orchestration, integrations and cloud architecture.

AI hotel conceirge woman front

Ready to bring your AI vision to life?

Let’s talk about how SoftTeco can help you transform it into a high-quality, market-ready solution.

Solution

SoftTeco developed the backend for an AI-powered concierge that transforms Cisco Webex Desk Pro into an intelligent customer-service terminal. When a user speaks to the device, Whisper converts the audio into text and passes the request to an agentic AI pipeline. The agent interprets the request, uses available tools and the RAG knowledge base when necessary, and generates an appropriate response. The text is then sent to HeyGen, which converts it into audio and video and presents it through the AI avatar.

The solution can also retrieve current services and pricing from an ERP system. When the interaction involves a purchasable service, payment can be initiated through an integrated Stripe Terminal. The backend was deployed on AWS and designed around independently scalable services. Because the system depended on several external APIs and AI services, engineering work involved continuous analysis of API contracts, response structures, integration edge cases, and failure conditions.

SoftTeco engineers used Claude throughout the project to analyze technical documentation, work through third-party integration requirements, refine implementation logic, generate test scenarios, and review code changes. Its use was integrated into the existing engineering workflow, with engineers validating the output and retaining responsibility for architecture, security, technical decisions, and final code ownership.

Tech Stack

Backend
& Frontend

React

Python

Stripe

AWS

AI & RAG

OpenAI

Whisper

Claude

How it works

We started working on the client’s idea by analyzing and estimating possible solutions and available providers for integration. As a result, we selected Heygen as a provider of AI avatars to be integrated with the Cisco Webex Desk Pro and our backend. The integration with Heygen was performed via native Heygen APIs.

Since Heygen delivered the frontend part of the solution, we were responsible for the backend development. We used Agentic AI as the base for the backend and selected Vector database as the RAG storage of choice. To connect frontend and backend, we set up an Agentic AI pipeline which interprets user queries and uses ReAct agents and RAG approach to return requests in the needed format.

AI Hotel Concierge

The pipeline operation starts with the user query in the audio format being received by Whisper, an OpenAI transcriber that converts speech into text. The transcribed text is then sent to our pipeline and is processed by the ReAct agent with Tools. The response to the query is generated in the text format and is sent to Heygen, which transforms the text into audio and video and presents it to a user via the AI avatar. To ensure the answers are relevant and accurate, we used LLMs for model training.

In addition to voice recognition and customer service, the solution also has a card reader integration with Stripe Terminal WisePOSE and an ERP integration to retrieve services, pricing, and record completed orders. Thus, users are able to instantly pay for specific services, like room booking or purchasing an additional service. The solution is deployed on AWS for better scalability and availability and is based on the microservices architecture to ensure independent and rapid scaling.

Looking for a trusted AI development partner?

Our engineers specialize in complex integrations, real-time analytics and scalable architectures for enterprise solutions.

Results

SoftTeco delivered a market-ready AI concierge combining conversational self-service, relevant information, and transactional functionality in a single terminal experience. AI also supported the engineering process: SoftTeco engineers used Claude for implementation, integration analysis, testing, and code review across a complex backend. The resulting architecture provides a scalable foundation for new services, integrations, and AI-powered scenarios.

Talent pool platform development

Talent pool platform development

EasyRegister

SoftTeco developed a PWA platform that runs on different platforms, hosted it on an Amazon AWS server, and integrated multiple functionalities to help companies easily and swiftly assemble a pool of needed talent.

  • Professional Services
  • Web
  • Mobile
AI-powered bee counting app development

AI-powered bee counting app development

Beeguard

SoftTeco delivered an AI-powered iOS app using ML and computer vision, developed a training dataset from scratch, and cut hive inspection time by 80%.

  • Agriculture
  • Mobile
  • ML
Softteco Logo Footer