How I Built an AI Calling Agent That Qualified 356 Leads

The Problem: Missed Calls Are Missed Revenue

Every business that depends on phone leads has the same leak: calls that come in after hours, during a job, or while the team is already on another line. Each one of those is a lead qualifying itself for whoever answers first — and that is rarely you.

I built a production AI calling agent to close that gap for a client: a system that answers inbound calls, holds a real voice conversation, qualifies the caller, and follows up over SMS automatically. It has now processed 29,600+ records, handled 1,540 calls, sent 3,334+ SMS messages, and qualified 356 leads — without a human picking up the phone for the standard case.

Here is how it actually works, and what I learned building it.

What We Built

The system has three moving parts:

• Telephony — Twilio handles the actual phone infrastructure: routing inbound calls, placing outbound calls, and managing SMS delivery.
• Voice — ElevenLabs generates the agent's speech. This was a deliberate choice over standard text-to-speech; the difference in how natural it sounds is immediately obvious to callers, and it matters for a system representing a real business on the phone.
• Reasoning — OpenAI's function calling drives the actual conversation logic. The model does not just generate text — it decides when to look up a caller's record, when a lead qualifies, and when to trigger a follow-up SMS.

Underneath, a Django REST Framework API handles the business logic and data, and a Next.js dashboard gives the team visibility into every call, SMS, and qualified lead in real time.

How the Agent Actually Works

1. Inbound calls get answered immediately

No hold music, no voicemail. The agent picks up, identifies itself, and starts a real conversation — it can handle interruptions, follow-up questions, and callers who don't answer in the expected order.

2. Function calling turns conversation into action

This is the part that separates an AI agent from a scripted phone tree. When a caller gives their details, the agent does not just say "got it" — it calls a function that actually looks up or creates their record in the system. When it determines a lead is qualified, that is a function call too, not a guess.

3. Every call ends with a decision

The agent classifies the outcome of every call: qualified lead, needs follow-up, not a fit, or needs a human. That classification is what drives what happens next.

4. SMS follow-up happens automatically

Qualified leads get a follow-up text without anyone on the team lifting a finger — this is where most of the 3,334+ SMS messages came from. It is also where a lot of the value is: a lead who just spoke to someone (even an AI) is far more likely to respond to a follow-up text within the hour than a cold callback the next day.

5. Everything is logged for the team to see

Every call transcript, every function call the agent made, and every outcome lands in the Next.js dashboard in real time. If someone on the team wants to know what happened on a specific call, or check whether a lead was actually qualified correctly, they do not have to ask me — the answer is already there.

Why This Stack

A few specific choices worth explaining, since they were not the "obvious" or cheapest option each time:

• Twilio over cheaper telephony providers — reliability on the actual phone connection matters more than saving a few cents per minute when the agent is answering real customer calls.
• ElevenLabs over built-in text-to-speech — the natural-sounding voice is not a nice-to-have here. Callers disengage almost immediately from a robotic voice, and disengagement on the first ten seconds of a call kills the entire system's value.
• OpenAI function calling over a fine-tuned model — function calling let me keep the agent's allowed actions explicit and auditable, rather than trusting a model to "just know" what it's allowed to do. That auditability turned out to matter more than raw model flexibility.

The Results

Over the course of production use, the system has processed:

• 29,600+ records — every call and interaction logged and searchable
• 1,540 calls handled — inbound and outbound, without a human answering each one
• 3,334+ SMS sent — automated follow-up tied directly to call outcomes
• 356 leads qualified — filtered and ready for the sales team, not raw unqualified inquiries

That last number is the one that actually matters to the business. 1,540 calls is a volume metric. 356 qualified leads is revenue.

What I'd Do Differently

A few things I learned the hard way, worth sharing if you're considering something similar:

• Scope what the agent is allowed to decide. Early on, I gave the agent too much latitude in edge cases. Tightening what counts as "qualified" and adding a clear escalation path for anything ambiguous made the system far more reliable.
• Voice quality is not optional. A robotic-sounding agent gets hung up on. ElevenLabs was worth the integration effort.
• Log everything. Every function call, every decision, every outcome. When a client asks why a lead got marked unqualified, you need a real answer, not a guess.

Is This Right for Your Business?

If your business loses leads to missed calls — outside business hours, during busy periods, or just from call volume — this kind of system pays for itself quickly. It is not a replacement for your team on complex calls; it is what catches everything that would otherwise go to voicemail.

If you want to see whether this fits your business, get in touch or read more about AI calling agent development.