Building GenAI Chatbots for Government

The Problem We're Solving

Government agencies sit on mountains of domain knowledge — regulations, program guides, compliance procedures, FAQs. Citizens can't find what they need.

The Policy Landscape

California has built the framework for responsible GenAI adoption. The mandate is clear — the barrier is execution.

State Policy Framework

The Momentum Is Real

The Compliance Fast Lane

Not all GenAI projects carry the same compliance burden. A public-data RAG chatbot is the lowest-risk GenAI project a department can build.

Why Public-Data RAG Qualifies for the Fast Lane

:::callout success The paperwork is manageable — and it runs in parallel with your build. Start your SIMM paperwork on Day 1, brief your ISO early, and compliance resolves while development is underway. :::

Phase 1: Curate the Knowledge

This is where your business and program staff do the work — not IT. They know the domain. They know what citizens ask. They know which documents matter.

The Two-Team Model

The handoff is simple: SMEs deliver a folder of organized documents + a list of 20-30 test questions with expected answers. IT builds from there.

The Content Pipeline

Your SMEs identify the source material. AI agents help organize, cross-reference, and identify gaps. The output is structured, searchable knowledge.

From Scattered Documents to Structured Knowledge

Quality Gates (Non-Negotiable)

Real example: WaterBot's knowledge base — 128 documents across 8 categories (permits, funding, compliance, water quality, entities, water rights, climate, public resources), curated entirely from existing SWRCB public content.

From Documents to Vectors

This is the technical core — how human-readable documents become machine-searchable knowledge. IT handles this step using the structured content from Phase 1.

How It Works

Phase 2: Build the Bot

IT takes the structured knowledge from Phase 1 and builds the bot. AI-assisted development means agents write components, configure workflows, and test integrations — a 1-2 person team can build production-quality bots.

What Gets Built

First bot: 40-60 hours. Second and third bots: 30-40 hours each — shared infrastructure already exists.

The Three-Mode UI Pattern

Every bot supports three ways for citizens to get help — each optimized for a different type of question.

Why Three Modes?

Not every question fits a chatbot. A citizen who needs a specific permit shouldn't have to describe their project in conversation — a decision tree gets them there in 4-5 clicks. A citizen exploring funding options gets more value from a matcher that filters 58 programs by their criteria than from a chat response.

The pattern: Build the chat first (every bot needs it), then add decision trees and tools based on your domain's most common workflows.

The Workflow Engine

When a citizen asks a question, here's the 2-3 second journey from query to answer.

What Happens at Each Step

Key Design Decisions

• Source attribution on every answer — citizens can verify; auditors can trace
• Conversation history included — multi-turn dialogue without citizens repeating themselves
• Graceful fallbacks — if no relevant chunks found, the bot says so instead of hallucinating
• Rate limiting + token validation — security at the API gateway level

The Infrastructure Stack

This pattern runs on infrastructure most departments already have. Here's how each component maps to Azure Gov and AWS GovCloud — the two platforms California state agencies primarily use.

What It Costs

:::callout success This runs on commodity cloud infrastructure you already have. No special hardware. No GPU clusters. No massive budget line items. The marginal cost of adding a second or third bot is minimal — shared infrastructure means most costs are already covered. :::

Phase 3: Serve Citizens

The bot is live. Citizens ask questions and get accurate, sourced answers in seconds — 24 hours a day, 7 days a week.

What Citizens Experience

What Staff Gain

• Routine questions handled automatically — staff focus on complex cases
• Consistent answers — every citizen gets the same accurate information
• 24/7 availability — nights, weekends, holidays
• Scalable — handles 10 or 10,000 concurrent users with the same infrastructure
• Measurable — every interaction is trackable for service improvement

Quality and Maintenance

A bot is a garden, not a monument — it needs tending. Quality assurance happens at launch and on an ongoing cycle.

Launch Testing: SMEs Are the Quality Gate

Your subject matter experts write the test questions and evaluate the answers. They don't need to understand embeddings — they need to ask the bot questions and say "that's right" or "that's wrong."

Scoring: 5-point scale per response. Target: average ≥ 4.0, no response below 3.0.

Quarterly Refresh Cycle

The Real Timeline

The Work Is ~50-75 Hours. The Calendar Depends on Dependencies.

Decision Gates

The Real Bottleneck

After the first bot: Second and third bots take 30-40 hours IT + 8 hours SME — shared infrastructure already exists.

Getting Started

Your Next Steps

1. Pick your domain — Which program area gets the most routine citizen inquiries?
2. Identify your SMEs — Who answers those questions today? They're your knowledge team.
3. Check your infrastructure — Do you have Azure Gov or AWS GovCloud access? LLM API agreements?
4. Start compliance early — File your SIMM 150 EZ, brief your ISO, check CDT master agreements
5. Curate your knowledge — SMEs: gather your top 50-100 source documents
6. Build your bot — IT: structure, embed, build, test, launch

Resources

Live Examples

Three bots built with this pattern are live at vanderdev.net:

State Water Resources Control Board · Innovation Fellowship · March 2026