AI-Powered English Language Coaching Platform with Real-Time Speech Analysis

The Problem

The Missing Layer in Language Learning: Real Conversation

Improving spoken language requires real practice — not reading, not watching videos, not multiple choice questions. It requires speaking, being evaluated, and understanding specifically what to improve.

Existing language apps focus on vocabulary and grammar. None offer structured speaking practice with real-time AI conversation, objective-based scenarios, and detailed speech quality metrics. The client needed a platform that could act as a patient, intelligent conversation partner — available anytime, across multiple practice formats, with measurable feedback every session.

What We Built

An AI-Powered Platform for Real-Time Speaking Practice

A real-time AI voice coaching platform supporting four distinct exercise types, built on LiveKit for real-time audio communication and OpenAI models for conversation intelligence and evaluation.

Scenario Based Admin creates pre-built scenarios — a coffee shop, a job interview, a hotel check-in. The AI plays the role defined by the admin. Within each scenario, the admin sets specific objectives the user must complete — for example, ask for the price, order a specific item, confirm the reservation. The AI tracks objective completion in real time throughout the conversation. At the end, the user receives an overall score, a written summary, and a full metrics breakdown.

Impromptu The user is given a random question and must speak on it for a defined time limit. No preparation. Tests fluency, coherence, and spontaneous speaking ability. Same full metrics report generated at the end.

Read Aloud A passage is displayed on screen. The user reads it aloud. The system evaluates pronunciation accuracy, pacing, and fluency against the written text.

Free Chat Same format as Scenario Based — but the user defines the scenario themselves. No admin setup required. No fixed objectives. The AI adapts to whatever context the user sets.

How the metrics work

Two sources of evaluation working together

AI-generated — response quality and completeness scores, conversation summary paragraph, objective completion tracking, overall session score.

Audio and transcript analysis — filler word detection and count, pause frequency and duration, speech pace, fluency patterns — all calculated from the raw audio and transcript without AI inference.

Together they produce a complete picture of speaking performance — not just what was said, but how it was said.

Tech stack

Built on a Real-Time AI and Audio Intelligence Stack

LiveKit for real-time audio streaming and session management. OpenAI models for conversation, evaluation, and summary generation. Custom audio processing pipeline for transcript-based metrics.

Scope

Scope of work