The Cloud Girl

Author: Guest User

Uncategorized

How DoorDash Built a Voice AI Contact Center That Actually Works

by Guest User

How DoorDash Built a Voice AI Contact Center That Actually WorksJan 17Written By Akhil MohanPicture this: You're a DoorDash driver—a Dasher, as they're called—navigating traffic while trying to find a customer's apartment. Something goes wrong with the app. You need help. Now.You're not going to pull over and type out a detailed support message. You're going to call.This is the reality DoorDash faced when they decided to overhaul their customer support. Hundreds of thousands of calls every single day from Dashers, merchants, and customers—many of them needing answers while literally on the move. And when you're driving for a living, every minute on hold is money lost.So how do you build an AI support system that doesn't feel like talking to a brick wall? That's exactly what DoorDash figured out, and there are some lessons here that anyone building AI applications should pay attention to.Voice AI Is Harder Than TextMost companies that implement AI chatbots start with text. Makes sense—it's easier to prototype, easier to debug, and users can wait a few seconds for a response without feeling like something's broken.Voice is a completely different beast.When you're on a phone call, even a two-second pause feels awkward. Three seconds? You start wondering if the call dropped. Five seconds? You've already hung up and called back.DoorDash already had a decent self-service system running on Amazon Connect with Amazon Lex—the kind of "press 1 for account issues, press 2 for payment questions" setup you've probably used a hundred times. It was working. They'd reduced agent transfers by 49% and saved about $3 million a year in operational costs.But "working" isn't the same as "great." Most calls were still getting routed to human agents. Dashers were spending too much time waiting for help when they needed to be back on the road making deliveries.The question wasn't whether to use generative AI. The question was: how do you make it fast enough for voice?Why Amazon Bedrock?DoorDash partnered with AWS through their Generative AI Innovation Center—a program where AWS pairs companies with their AI experts to build production-ready solutions. Not a proof of concept. Not a demo. Something that could handle hundreds of thousands of real calls.They chose Amazon Bedrock as their foundation, and the reasoning is worth understanding.For beginners: Bedrock is essentially Amazon's "one-stop shop" for accessing large language models. Instead of managing your own infrastructure or figuring out how to deploy models, you get a managed service with multiple AI models available through a single API. Think of it like what AWS did for servers, but for AI models.For practitioners: The real value here isn't just convenience. Bedrock gives you access to multiple foundation models—including Anthropic's Claude family—without vendor lock-in. You can experiment with different models, swap them out, and fine-tune your approach without rebuilding your entire architecture. DoorDash reported a 50% reduction in development time just from using Bedrock instead of building custom integrations.For architects: The security story matters here too. Bedrock provides built-in encryption and ensures that customer data stays within your application boundaries. DoorDash explicitly noted that no personally identifiable information gets passed to the generative AI components—the architecture enforces this separation.Getting the Speed Right with Claude 3 HaikuRemember that latency issue? This is where model selection becomes critical.DoorDash tested multiple models and landed on Anthropic's Claude 3 Haiku for their voice application. The result: response latency of 2.5 seconds or less.Now, 2.5 seconds might sound like a lot if you're used to text chatbots that respond almost instantly. But remember—this isn't just generating a response. The system needs to:Transcribe the caller's speech to textProcess the request and understand the intentSearch the knowledge base for relevant informationGenerate an appropriate responseConvert that response back to speechAll of that in under 2.5 seconds. That's actually impressive.But Haiku wasn't just chosen for speed. Claude models have specific capabilities around hallucination mitigation, prompt injection detection, and identifying abusive language. When you're building a customer-facing voice system that handles hundreds of thousands of calls daily, you really don't want your AI confidently giving wrong information or being manipulated by bad actors.The RAG ArchitectureLet's get into how this actually works.DoorDash implemented Retrieval-Augmented Generation (RAG) using Knowledge Bases for Amazon Bedrock. If you're not familiar with RAG, here's the gist:Instead of asking the AI model to answer questions purely from what it learned during training, you give it access to your company's actual documentation. When a Dasher asks "how do I update my payment information?", the system first searches DoorDash's help center for relevant articles, then feeds that context to Claude along with the question. The model generates an answer based on your actual, up-to-date documentation—not whatever was in its training data.Why this matters:Accuracy: The AI can only answer based on your approved content. If your payment process changed last week, you update the help center, and the AI immediately gives the right answer.Controllability: You're not hoping the model "knows" the right answer. You're ensuring it does.Auditability: You can trace exactly which documents informed any given response.Knowledge Bases for Amazon Bedrock handles the messy parts—ingesting documents, creating embeddings, managing the retrieval workflow, and augmenting prompts. DoorDash didn't have to build custom integrations or manage their own vector database. They pointed the system at their help center and let the managed service handle the plumbing.The Testing Framework That Made It PossibleThis part caught my attention and doesn't get talked about enough.Before this project, DoorDash's contact center team had to pull actual agents off the support queue to manually test new features. They'd have agents call in, go through scenarios, and report back. It was slow, expensive, and couldn't scale.Working with the AWS team, they built an automated testing framework using Amazon SageMaker. The system can now run thousands of tests per hour—a 50x increase from their previous capacity. More importantly, it doesn't just check if the AI responded; it semantically evaluates whether responses are actually correct compared to ground-truth answers.This is the kind of infrastructure investment that separates companies that successfully deploy AI from those that launch something broken and never recover user trust.For teams building similar systems: Don't underestimate testing. Your AI might work perfectly in demos but fail spectacularly on edge cases. Build evaluation frameworks early. Test at scale. Have a way to measure quality beyond "did it respond?"The ResultsLet's look at the numbers DoorDash shared:Hundreds of thousands of calls per day handled by the generative AI solutionThousands fewer escalations to live agents dailyFewer agent tasks required to resolve support inquiries8 weeks from kickoff to live A/B testingThose are real, material improvements. The solution rolled out to all Dashers in early 2024 and has been running in production since.What I find more interesting is what they're planning next.DoorDash mentioned they're working on expanding the knowledge bases (more topics the AI can handle) and integrating with their "event-driven logistics workflow service." Translation: the AI won't just answer questions; it'll take actions on behalf of users.Imagine calling support and instead of just hearing "you can update your payment method in settings," the AI says "I've updated your payment method to the new card. You should see it reflected now."That's a fundamentally different product. And it's where this technology is heading.TakeawaysWhether you're building AI applications, evaluating vendors, or just trying to understand where this technology is going, a few things stand out:Model selection is architecture. Choosing Claude 3 Haiku wasn't just about speed—it was about the full package of latency, accuracy, safety features, and cost. Different use cases need different models.RAG isn't optional for enterprise. If you're building customer-facing AI that needs to give accurate, current information, you need retrieval-augmented generation. Relying purely on model training data is asking for trouble.Voice AI is harder than text AI. The latency requirements alone change everything. If you're thinking about voice applications, budget more time and expect different trade-offs.Testing infrastructure pays dividends. DoorDash's investment in automated evaluation let them iterate faster and deploy with confidence. This is table stakes for production AI.Managed services accelerate deployment. A 50% reduction in development time is significant. Sometimes the right move is to pay for infrastructure that lets you focus on your actual product.Where This Is All HeadingThere's a pattern emerging in how successful companies deploy generative AI: they start with high-volume, repetitive tasks where the cost of errors is manageable and the improvement potential is obvious.Customer support checks all those boxes. Hundreds of thousands of daily interactions. Many questions are routine ("how do I...?"). The existing baseline (hold music) is so bad that even imperfect AI feels like an improvement. And humans remain in the loop for complex issues.DoorDash didn't try to replace their entire support operation with AI. They augmented it. The generative AI handles the straightforward stuff, which frees up human agents to focus on complex problems that actually need human judgment.That's not the flashiest vision of AI, but it might be the most realistic one. Incremental improvements that compound. Real cost savings that fund further development. Gradual expansion of capabilities as the technology matures.Not every AI story needs to be revolutionary. Sometimes, "it works, it saves money, and customers like it better" is exactly the story worth telling.

Read more »

Uncategorized

What are SLMs (Small Language Models)? Why are SLMs the future?

by Guest User

We’ve spent years chasing bigger and bigger AI models—trillions of parameters, endless GPUs, and even more cloud bills. But a quiet shift is happening. Small Language Models (SLMs) are proving that intelligence doesn’t always need to be massive to be useful.With models like Phi-3 Mini and Llama 3 8B running locally on just 4GB of VRAM, developers can now build real, privacy-preserving, low-latency AI experiences—right from their laptops. Thanks to breakthroughs in quantization and knowledge distillation, edge AI has entered the “good enough” era for most practical tasks.This is the start of the 4GB revolution—where we cut the cloud cord, reclaim our data, and build faster, leaner, and more ethical AI. The future isn’t giant centralized models; it’s billions of tiny, specialized intelligences running everywhere.

Read more »

Uncategorized

How to Red-Team Your AI for Prompt Injection and Data Leakage

by Guest User

Your customer-support bot looked secure — until a user typed a jailbreak. This guide shows how the modern attack surface for LLMs (large language models / LLM) is the final assembled prompt used by RAG (retrieval-augmented generation) systems:Final Prompt = System Prompt + Retrieved Data + User Input. Attackers weaponize user input or retrieved documents to perform prompt injection, indirect prompt injection, and data exfiltration / data leakage.The threats (what red-teams do) Direct prompt injection (jailbreak) — attacker tells the model to “ignore previous instructions” and the model follows because tokens are tokens. Example: user forces the bot to become a comedian and leak info. Indirect prompt injection (Trojan horse via RAG) — malicious instructions are hidden inside uploaded docs (resumes, PDFs). When retrieved as context, the model obeys those instructions, bypassing user-input filters. This is the biggest RAG attack vector. Data exfiltration / The Heist — attacker social-engineers the LLM into dumping sensitive context or the system prompt (confidential internal commands, heuristics, DB schemas).Layered defenses (blue-team playbook) Input & output filtering — regex / keyword scanning for phrases like “ignore” or for leakage of the system prompt (useful but brittle). Robust prompt engineering (instructional fences) — separate instructions from data with strong delimiters (e.g., <system_prompt>, <retrieved_documents>, <user_query>). This tells the model which text is data vs instructions.Model-level defenses — prefer instruction-tuned models (GPT-4, Claude 3, Llama 3) and classification pipelines (cheap LLM for intent classification, strong LLM for answers). Use separate LLMs for safety checks. Automation for red-teaming — use tools like garak (open-source LLM vulnerability scanner) to run prompt injection probes and find weaknesses automatically. Follow standards — adopt the OWASP Top 10 for LLMs (LLM01: Prompt Injection, LLM02: Sensitive Information Disclosure, LLM08: Vectors & Embeddings, etc.) as part of secure development lifecycle.Takeaway / CTALLM security is a cat-and-mouse game. To protect against prompt injection, data leakage, and RAG-based Trojans, design defense-in-depth: input/output filters, strong prompt engineering, model-level controls, automated red-teaming (garak), and adherence to the OWASP Top 10 for LLMs. Red-team your models now — find the holes before someone else does.

Read more »

Uncategorized

Cheatsheet for Production-Ready Advanced RAG

by Guest User

Cheatsheet for Production-Ready Advanced RAGOct 9Written By Akhil MohanYou built it. The demo was a hit. Your Retrieval-Augmented Generation (RAG) app flawlessly answered questions, pulling information from a mountain of documents. But then, it met real users.Now, it’s giving vague answers. It’s confidently making things up (hallucinating). Sometimes, it misses information that you know is in the source data. That impressive demo has turned into a brittle prototype, and you're left wondering: "Why is this so hard in production?"Sound familiar? You're not alone. While basic RAG is easy to prototype, building a robust, trustworthy system requires moving beyond the basics. This is the playbook for doing just that. We'll dissect the entire pipeline, using the architecture below as our map, to transform your fragile prototype into a production-grade asset.The Core Principle: Garbage In, Gospel OutAn LLM in a RAG system is a powerful reasoning engine, but it's fundamentally limited by the context it's given. If you feed it irrelevant, poorly structured, or confusing information, you'll get garbage answers. The secret to a great RAG system isn't just a better LLM—it's a superior retrieval process.Our journey to production-readiness focuses on systematically upgrading the three core stages you see in the diagram:The Ingestion Pipeline: How we prepare and index our knowledge base.The Retrieval Pipeline: How we find the most relevant information for a given query.The Generation Pipeline: How we synthesize that information into a trustworthy answer.Stage 1: Ingestion—Beyond Naive ChunkingThe foundation of any great RAG system is laid before a single query is ever asked. It starts with how you process your data.Technique 1: Stop Using Naive ChunkingSimply splitting your documents into fixed 1000-character chunks is fast, but it’s a primary source of retrieval errors. This method often splits sentences mid-thought, separating a cause from its effect or a question from its answer.The Solution: Smart ChunkingRecursive Character Splitting: A step up. This method splits text based on a hierarchical list of separators (like \n\n, \n, ) and tries to keep related paragraphs and sentences together. It's the go-to choice for a better baseline.Semantic Chunking: The advanced approach. Instead of using character counts, this technique uses an embedding model to split the text at points where the semantic meaning shifts. This ensures that each chunk is as contextually coherent as possible. While computationally heavier during ingestion, it pays massive dividends in retrieval quality.Technique 2: Enrich Chunks with MetadataIn a basic system, every chunk is just a piece of text. In an advanced system, every chunk is a rich object with context. During ingestion, extract and attach valuable metadata to each chunk.The Solution: Metadata Filtering Imagine your documents are technical manuals. For each chunk, store metadata like:source_file: "Q3_Security_Protocol.pdf"version: "2.1"chapter: "4"author: "Karl"Now, when a user asks, "What's the latest security protocol?", you can apply a pre-retrieval filter to only search within chunks where source_file contains "Security_Protocol" and version is "2.1". This drastically reduces the search space, leading to faster, more accurate, and less noisy retrieval.Stage 2: The Retrieval Overhaul—Finding the Right NeedleThis is where the magic happens. A user's query is often imprecise. Our job is to bridge the gap between what they ask and what our documents contain.Technique 1: Query TransformationInstead of taking the user's query at face value, we use an LLM to refine it first.Hypothetical Document Embeddings (HyDE): This is a clever technique where you first ask an LLM to generate a hypothetical answer to the user's query. This generated answer, while potentially factually incorrect, is rich in the language and keywords that a real answer would likely contain. You then embed this hypothetical answer and use it for the vector search. It’s like asking a detective to imagine a solution to a crime to figure out what clues to look for.Multi-Query Generation: You can ask an LLM to re-write the user's query from multiple perspectives. For a query like "How can I improve RAG performance?", it might generate variants like "What are the best techniques for RAG optimization?" and "Methods for reducing RAG latency." Searching for all these variants retrieves a more diverse and comprehensive set of documents.Technique 2: Hybrid Search (The Best of Both Worlds)Vector search is great at understanding semantic meaning, but it can struggle with specific keywords, product IDs, or acronyms. For instance, it might not distinguish well between GCP-Project-ID-12345 and GCP-Project-ID-54321.The Solution: Combine Semantic and Keyword Search By blending traditional keyword search algorithms (like BM25) with dense vector search, you get the best of both worlds. BM25 excels at finding documents with the exact keywords, while vector search finds documents with related meanings. Most modern vector databases offer hybrid search capabilities out of the box.Technique 3: The Final Check—RerankingYour hybrid search might return 20 potentially relevant documents. Are they all equally good? Probably not. Sending all 20 to the LLM creates noise and costs more.The Solution: Add a Reranker Model A reranker is a lightweight but highly accurate model that takes the initial list of retrieved documents and the user's query and re-scores them for relevance. Unlike the initial search (which uses a bi-encoder), a reranker often uses a cross-encoder, which looks at the query and each document simultaneously, leading to a much more accurate relevance score.Fact: Implementing a good reranker can boost retrieval accuracy (metrics like nDCG or Hit Rate) by 5-15% in many benchmarks, which is often the difference between a good and a great answer. You retrieve more candidates initially (K=20) and then use the reranker to distill them down to the absolute best (N=3-5) to send to the LLM.Stage 3: The Final Polish—Crafting Trustworthy AnswersYou’ve done the hard work of finding the perfect context. The final step is to ensure the LLM uses it correctly.Technique 1: Masterful Prompt EngineeringDon't leave the LLM's behavior to chance. Your system prompt is a contract that dictates its rules of engagement.The Solution: A Robust System Prompt Move from a simple "Answer the question" to a detailed set of instructions.You are an expert Q&A assistant. Your goal is to provide accurate and concise answers based ONLY on the provided context documents.Rules:1.  Analyze the provided context documents thoroughly before answering.2.  Answer the user's question based solely on the information found in the context.3.  If the context does not contain the answer, you MUST state: "I do not have enough information to answer this question." Do not make up information.4.  For each piece of information you use, you must cite the source document.Technique 2: Citing Your SourcesTrust is paramount. Always empower your users to verify the AI's answers. By passing the metadata you collected during ingestion all the way to this final stage, you can instruct the LLM to cite its sources. This not only builds user trust but is also an invaluable tool for you to debug where the information is coming from.Putting It All Together: From Prototype to ProductionBy evolving your architecture from a simple sequence to the multi-stage pipeline shown in the diagram, you address the core weaknesses of basic RAG. You're no longer just hoping for the best; you're systematically ensuring quality at every step.But how do you prove it's better? You measure it. Start exploring evaluation frameworks like RAGAs or TruLens. They provide metrics to quantify the performance of your system, such as:Faithfulness: Is the answer grounded in the retrieved context? (Measures hallucination)Answer Relevancy: Does the answer actually address the user's query?Context Precision & Recall: Did you retrieve the right context in the first place?ConclusionMoving a RAG system from a cool demo to a reliable tool isn’t about one magic fix. It’s about a deliberate, engineering-focused approach. It's about smart chunking, rich metadata, query transformation, hybrid search, reranking, and disciplined prompting.By adopting this playbook, you can turn your unpredictable prototype into a robust and trustworthy Q&A system that truly delivers value.

Read more »

Uncategorized

Vertical AI is the New SaaS

by Guest User

Vertical AI is the New SaaSSep 3 Written By Akhil MohanHow industry-specific AI is reshaping enterprise software just like SaaS transformed computing two decades agoTwenty years ago, I witnessed the SaaS revolution firsthand. Companies were struggling with expensive on-premise software deployments, complex maintenance cycles, and rigid licensing models. Then came Salesforce, Workday, and ServiceNow—not just offering better software, but fundamentally reimagining how businesses consume technology. They didn't just digitize existing processes; they created entirely new paradigms around subscription models, cloud-native architectures, and continuous delivery.Today, we're standing at a similar inflection point with Artificial Intelligence. But this time, the transformation isn't about moving from desktop to cloud—it's about moving from horizontal, general-purpose AI to Vertical AI: purpose-built intelligent systems designed for specific industries and workflows.The Horizontal AI Foundation: Necessary but Not SufficientThe current AI landscape is dominated by horizontal platforms—ChatGPT, Claude, Gemini, and their enterprise variants. These models are remarkable achievements, capable of reasoning across domains, generating content, and solving problems with unprecedented sophistication. They've democratized access to AI capabilities and proven the transformative potential of large language models.However, much like the early days of computing when we had powerful but generic mainframes, horizontal AI faces inherent limitations when applied to specialized enterprise use cases:Data Limitations: General models are trained on publicly available internet data, missing the proprietary datasets that drive real business value—medical records, legal precedents, financial transactions, manufacturing telemetry.Compliance Gaps: Industries like healthcare (HIPAA), finance (SOX, GDPR), and legal services operate under strict regulatory frameworks that generic AI models weren't designed to navigate.Workflow Friction: Horizontal AI requires users to adapt their processes to the tool, rather than embedding intelligence seamlessly into existing professional workflows.Limited Context: Without deep domain knowledge, even the most sophisticated general models struggle with industry-specific nuances, terminology, and decision-making frameworks.Vertical AI is the New SaaSWhat is Vertical AI RevolutionVertical AI represents the natural evolution beyond horizontal platforms—AI systems built from the ground up for specific industries, trained on domain-specific datasets, and designed to integrate natively into professional workflows. The market data strongly supports the rising importance of vertical AI solutions. Vertical AI is on the rise, with this year's vertical winners surpassing the other category winners to capture over $1B in combined funding in 2025 YTD AI 100: The most promising artificial intelligence startups of 2025 - CB Insights Research, according to CB Insights' AI 100 report. The healthcare sector exemplifies this growth trajectory, with the global AI in healthcare market size estimated at USD 26.57 billion in 2024 and projected to reach USD 187.69 billion by 2030, growing at a CAGR of 38.62% AI In Healthcare Market Size, Share | Industry Report, 2030.Key Differentiators of Vertical AI:1. Specialized Training Data While horizontal AI models train on broad internet corpora, Vertical AI systems ingest industry-specific datasets: clinical trial data for healthcare AI, case law databases for legal AI, financial market data for fintech AI. This specialized training creates models that understand industry context, terminology, and decision-making patterns.2. Regulatory Compliance by Design Rather than retrofitting compliance, Vertical AI systems are architected with regulatory requirements as first-class constraints. Healthcare AI models are HIPAA-compliant from inception; financial AI systems are built with SOX controls and audit trails embedded.3. Workflow-Native Integration Instead of requiring users to switch contexts, Vertical AI embeds directly into the tools professionals already use—EMR systems for doctors, case management platforms for lawyers, trading platforms for financial professionals.4. Domain-Specific Performance By focusing on narrow use cases within specific industries, Vertical AI systems can achieve superhuman performance in their domains while maintaining explainability and auditability.

Read more »

Uncategorized

How AI is Reshaping Consulting

by Guest User

The consulting industry has long been the intellectual backbone of business transformation — guiding organizations through complexity using human expertise, frameworks, and playbooks. But today, something unprecedented is underway: the rise of Artificial Intelligence as both a disruptor and an amplifier of consulting.The headlines are provocative: “Is consulting becoming irrelevant in the age of AI?”The reality is the opposite. AI isn’t making consulting obsolete — it’s fundamentally reshaping how consultants create value, deliver outcomes, and engage with clients.The bottom line: Consulting is shifting from a purely advisory service to a technology-enabled, implementation-focused discipline — one that delivers measurable impact faster than ever before.

Read more »

Cart

Your cart is empty.

Start Shopping