AI email-triage pipeline
Classifies hundreds of ticketing emails a day with a fine-tuned multimodal LLM and routes summaries to the right departments. OpenAI and Gemini sit behind one interchangeable JSON contract; a defensive C# parsing layer means one malformed response never breaks the batch.
- 01MailboxIMAP pulls new ticketing mail
- 02Classifyfine-tuned model, PDF + body, JSON out
- 03Defendbalanced-brace extraction · tolerant converters · invariants
- 04StoreSQL Server, idempotent writes
- 05Reportsummaries routed to each department
The problem
The purchasing team received hundreds of emails a day from Ticketmaster, venues and artists. Buried in them were the messages that mattered — presales, ticket releases, cancellations that needed refunds — and missing one meant lost revenue. Each email had to be sorted into categories such as PRESALES, TICKETS_RELEASED and CANCELLED, with performer, venue, event date, promo codes and refund details pulled out as structured data.
What I built
- IMAP ingestion that tracks processed messages by UID, so nothing is classified twice and the mailbox is never rescanned from scratch.
- Multimodal classification. Text, HTML and inline images go to the model together — marketing banners often carry the key information as a picture with empty alt text.
- One contract, two providers. OpenAI and Gemini implementations sit behind the same JSON contract, so they can be swapped and compared on cost and accuracy.
- A fine-tuned model. The team’s labelled history only existed as Gmail PDF exports; those were extracted, cleaned, rendered to images and converted to JSONL to fine-tune GPT-4o-mini.
- Department digests. Results are grouped into consolidated summaries for each department, and cancelled orders feed the finance refund sheet automatically.
What broke, and the fix
- Keyword drift. Vague category descriptions let “new T-shirts” trigger
TICKETS_RELEASED. The ~120-line prompt was rewritten with explicit gating rules, plus time heuristics that drop stale “on sale” messages. - Duplicates. The same event arrives from several senders; a canonical de-duplication key with a priority chain keeps one record per event.
- Malformed JSON. Balanced-brace extraction and tolerant converters recover what they can; business invariants are enforced in code, not trusted to the model.
- An encoding bug. JSONL training files written on Windows came out as UTF-16 with a BOM. The upload step now sniffs the encoding and re-encodes to UTF-8.
Outcome
Nobody reads the raw mailbox any more. Notifications arrive as digests, refunds are queued automatically, and the fine-tuned model recognises ticketing-specific patterns better than a generic one.
Lessons
Prompt engineering behaves like versioned bug-fixing. Never assume the model complied — enforce the rules that matter in code. And when uploading files to a third-party API from Windows, check the encoding first.