Stripe Docs Support Bot
A Telegram bot that answers Stripe integration questions by retrieving Stripe's own docs, and says so plainly when it doesn't know.
Fully built and manually tested across ambiguous, out-of-corpus, and multi-turn conversation scenarios. No automated evaluation suite yet, correctness was verified by hand rather than with a formal test harness. Next phase is exactly that, a small labeled test set to catch retrieval regressions automatically.
A separate ingestion workflow pulls 26 Stripe documentation pages, runs them through a cleaning step (including the table-flattening fix below), chunks them with n8n's native Recursive Character Text Splitter at 1800 characters with 200 character overlap, and embeds each chunk via the Hugging Face Inference API using mixedbread-ai/mxbai-embed-large-v1 at 1024 dimensions. Chunks land in a Supabase table with pgvector, queried through a match_stripe_docs RPC function.
On the runtime side, a Telegram Trigger feeds a Gemini-backed agent with a stripe_docs retrieval tool and Postgres-backed chat memory keyed by Telegram chat ID, so a follow-up question carries the context of what was already discussed. Every turn is logged to Google Sheets. Replies are sent in Telegram's HTML parse mode with explicit entity escaping, since Stripe's own documentation is full of angle-bracket placeholders and snake_case syntax that would otherwise break Telegram's message parser.
- Chose Supabase with pgvector over n8n's in-memory vector store, since an in-memory store resets with the workflow and can't support a real production ingestion-then-runtime split
- Table-flattening runs as a fenced-code-aware preprocessing step rather than requiring Stripe's tables to already be well-formed Markdown, since they aren't consistently formatted that way
- Standardized on HTML parse mode with manual entity escaping over Telegram's Markdown mode, which broke unpredictably on Stripe's own code syntax
- Stripe's error reference page lists API fields using pipe-delimited table syntax, which embeds poorly against natural language questions, so a fully ingested page still failed to answer relevant questions about it. Fixed with a preprocessing step that detects pipe-delimited lines outside fenced code blocks and converts them to plain sentences before chunking
- The agent would use loosely related retrieved content as a springboard to answer from general training knowledge, rather than recognizing that content didn't actually address the question asked. Fixed by rewriting the grounding instructions to separate 'did retrieval return something' from 'does what it returned actually answer this,' and treating a no on the second question the same as an empty retrieval
- Went through several embedding provider swaps mid-build (OpenAI, then Ollama, then Gemini, before settling on Hugging Face's mxbai-embed-large-v1) chasing reliable dimensionality and quality before landing on the final stack
- A retrieval that returns no relevant content routes the agent to explicitly tell the user rather than falling back to general knowledge
- Conversation memory is scoped per Telegram chat ID, so one user's context can never leak into another's
- No special retry logic beyond n8n's default node-level handling, since this is a read-heavy, low-write pipeline where a failed retrieval call is safe to simply surface as 'I don't know' rather than silently retried
- Conversation logging to Google Sheets stores what was asked and answered, not any credentials or account-specific data, since this bot answers documentation questions, not account questions
- Chat memory isolation by Telegram chat ID prevents cross-user context bleed
The table-flattening fix. It's a small preprocessing step, but it was the actual reason a fully ingested, fully embedded page was still failing to answer real questions, and it's a good example of retrieval quality depending on document structure, not just on having the right content in the store.
Getting the agent to reliably tell the difference between 'I retrieved something' and 'I retrieved something that answers this.' Those look identical to a naive grounding instruction, and conflating them is what causes confident wrong answers even with retrieval in place.
- Build a small labeled test set to catch retrieval regressions automatically instead of relying on manual spot checks
- Expand the corpus beyond the current 26 pages (Stripe Tax, Stripe Connect)
- Add a lightweight relevance score threshold so a weak retrieval match is treated as no match, rather than depending entirely on the agent's own judgment
The Global Error Handler
Every workflow on this site, including this one, reports into the same error handler instead of failing silently. When any node in any workflow throws, n8n's Error Trigger catches it, a small code step pulls out the workflow name, the node that failed, and the actual error message, and it lands in my inbox as a plain email alert. One shared piece of infrastructure instead of duplicating error-handling logic four separate times.