From Tickets to Training Data

By Syncfusion®, Inc. · Last updated October 28, 2026

Turn a real helpdesk export into training lines: check consent, convert HTML, cut quoted replies and signatures, redact personal data and split by customer.

From Tickets to Training Data

The team knows what a training line should look like. Now someone has to produce 600 of them from the helpdesk.

They export the closed tickets. The file is nothing like the neat examples in the last lesson: HTML, email signatures, old replies quoted underneath, phone numbers, card numbers, home addresses, and a few tickets in Spanish.

Follow one ticket through the clean-up.

Six small steps turn a messy email into one safe training input. Each step is a short function you can test.

Simple lesson: real data needs cleaning before it is even "data".

A Real-World Example

Think about photocopying customer letters to train new staff.

Before you hand them out, you black out names, phone numbers and card numbers. You tear off the old letters stapled behind. And you leave out the letters from customers who asked you not to share them.

Training data is the same. The model will remember what you show it, so show it only what it needs.

Simple lesson: if you would not show it to a new hire, do not train on it.

What You Will Learn

In this lesson, you will learn:

1. What a real helpdesk export looks like, and why it is not ready to train on.
2. How to turn HTML into text and cut quoted replies and signatures.
3. How to redact personal data while keeping what the model needs, like order numbers.
4. Which tickets to leave out: no consent, wrong language, the wrong part of a thread.
5. How the same steps run in prepare_data.py from the starter kit.

TL;DR

Export = the raw file your helpdesk gives you: tickets, messages as HTML, customer fields.

PII (personally identifiable information) = anything that identifies a person: names, emails, phone numbers, card numbers, addresses.

Redact = replace PII with a placeholder like [EMAIL].

Check consent first. Then: HTML to text, cut quoted replies and signatures, redact PII, keep only the message your task needs, and split by customer.

What May You Train On?

Before any code, answer one question: are you allowed to use these messages for training?

The export has 36 tickets. 2 without consent and 2 not in English are left out, so 32 usable remain.

Clean the Text

An email body comes as HTML. Inside it: a greeting, the request, a sign-off, and often the whole previous conversation quoted underneath.

// An email body as the helpdesk export stores it.
const html = "<div><p>Hi,</p><p>Please cancel order 2532, I ordered the toaster twice.</p><br>" +
  "<p>Thanks,</p><p>Lina Ortiz</p><p>On Tue, Sep 4, 2026 at 11:03 AM Support wrote:</p><p>&gt; Hi Lina,</p></div>";

const text = html
  .replace(/<br\s*\/?>/gi, "\n").replace(/<\/p>/gi, "\n").replace(/<[^>]+>/g, "")
  .replace(/&gt;/g, ">").replace(/&lt;/g, "<").replace(/&amp;/g, "&").trim();

const lines = text.split("\n");
const quoteAt = lines.findIndex((l) => /^On .+wrote:$/.test(l.trim()) || l.startsWith(">"));
const noQuote = lines.slice(0, quoteAt < 0 ? lines.length : quoteAt);
const sigAt = noQuote.findIndex((l) => /^(thanks|kind regards|best regards|regards),?$/i.test(l.trim()));
const body = noQuote.slice(0, sigAt < 0 ? noQuote.length : sigAt).filter((l) => l.trim() && !/^(hi|hello),?$/i.test(l.trim()));

console.log("plain text lines:", lines.length);
console.log("training input:", JSON.stringify(body.join(" ")));

using System.Text.Json;
using System.Text.RegularExpressions;

// An email body as the helpdesk export stores it.
var html = "<div><p>Hi,</p><p>Please cancel order 2532, I ordered the toaster twice.</p><br>" +
  "<p>Thanks,</p><p>Lina Ortiz</p><p>On Tue, Sep 4, 2026 at 11:03 AM Support wrote:</p><p>&gt; Hi Lina,</p></div>";

var text = Regex.Replace(html, @"<br\s*/?>", "\n", RegexOptions.IgnoreCase);
text = Regex.Replace(text, @"</p>", "\n", RegexOptions.IgnoreCase);
text = Regex.Replace(text, @"<[^>]+>", "");
text = text.Replace("&gt;", ">").Replace("&lt;", "<").Replace("&amp;", "&").Trim();

var lines = text.Split("\n").ToList();
var quoteAt = lines.FindIndex(l => Regex.IsMatch(l.Trim(), @"^On .+wrote:$") || l.StartsWith(">"));
var noQuote = lines.Take(quoteAt < 0 ? lines.Count : quoteAt).ToList();
var sigAt = noQuote.FindIndex(l => Regex.IsMatch(l.Trim(), @"^(thanks|kind regards|best regards|regards),?$", RegexOptions.IgnoreCase));
var body = noQuote.Take(sigAt < 0 ? noQuote.Count : sigAt)
    .Where(l => l.Trim() != "" && !Regex.IsMatch(l.Trim(), @"^(hi|hello),?$", RegexOptions.IgnoreCase));

var jsonOptions = new JsonSerializerOptions { Encoder = System.Text.Encodings.Web.JavaScriptEncoder.UnsafeRelaxedJsonEscaping };
Console.WriteLine($"plain text lines: {lines.Count}");
Console.WriteLine($"training input: {JsonSerializer.Serialize(string.Join(" ", body), jsonOptions)}");

Expected output:

plain text lines: 7
training input: "Please cancel order 2532, I ordered the toaster twice."

Only the request survives. The quoted reply ("On Tue ... wrote:") is an older message, and the sign-off has the customer's name in it.

Simple lesson: one small function per step, each tested on real examples.

Redact Personal Data

The model needs order numbers to fill in the ticket. It does not need card numbers, phone numbers or addresses. So the rules must be precise.

function redact(text: string): string {
  return text
    .replace(/[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}/g, "[EMAIL]")
    .replace(/\b(?:\d[ -]?){12,18}\d\b/g, "[CARD]")
    .replace(/(?:\+\d{1,3}[\s-]?)?(?:\(\d{3}\)|\d{2,4})[\s-]?\d{3,4}[\s-]?\d{4}\b/g, "[PHONE]")
    .replace(/\b\d{1,4} [A-Z][a-z]+ (?:Street|St|Road|Rd|Lane|Ln|Avenue|Ave)\b/g, "[ADDRESS]");
}

const messages = [
  "Please refund order 2310 to my card 4111 1111 1111 1111.",
  "Order 2418 has not arrived. Call me on +1 555 010 2233.",
  "I moved, send order 2504 to 42 Mill Road.",
  "Cancel 2266 and email me at tom.baker@example.com",
  "3 lamps at $45.99 on 2026-09-04, order 2519",
];
for (const m of messages) console.log(redact(m));

using System.Text.RegularExpressions;

string Redact(string text)
{
    text = Regex.Replace(text, @"[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}", "[EMAIL]");
    text = Regex.Replace(text, @"\b(?:\d[ -]?){12,18}\d\b", "[CARD]");
    text = Regex.Replace(text, @"(?:\+\d{1,3}[\s-]?)?(?:\(\d{3}\)|\d{2,4})[\s-]?\d{3,4}[\s-]?\d{4}\b", "[PHONE]");
    text = Regex.Replace(text, @"\b\d{1,4} [A-Z][a-z]+ (?:Street|St|Road|Rd|Lane|Ln|Avenue|Ave)\b", "[ADDRESS]");
    return text;
}

string[] messages = [
    "Please refund order 2310 to my card 4111 1111 1111 1111.",
    "Order 2418 has not arrived. Call me on +1 555 010 2233.",
    "I moved, send order 2504 to 42 Mill Road.",
    "Cancel 2266 and email me at tom.baker@example.com",
    "3 lamps at $45.99 on 2026-09-04, order 2519",
];
foreach (var m in messages) Console.WriteLine(Redact(m));

Expected output:

Please refund order 2310 to my card [CARD].
Order 2418 has not arrived. Call me on [PHONE].
I moved, send order 2504 to [ADDRESS].
Cancel 2266 and email me at [EMAIL]
3 lamps at $45.99 on 2026-09-04, order 2519

The card, phone, address and email are replaced. The order numbers, the price and the date stay.

Across the whole fixture there are 45 pieces of personal data planted in the first messages. The lab checks that none survive, and that no order number is lost.

Names are harder: a regular expression cannot know that "Lina" is a name. The kit uses the customer's name from the ticket's own fields. For free text, teams often add a named-entity tool, then check a sample by hand.

Pick the Right Part of a Thread

A ticket is a conversation. Your task decides which part becomes a training example:

• Turning an incoming message into a ticket (this course): use the customer's first message.
• Training a reply assistant: each customer turn is an input, and the agent's reply that followed is the answer. Only use replies your best agents would be happy to repeat.
• Never mix both tasks in one dataset.

The label comes from the helpdesk's own category field, mapped to the schema's intents. If agents often pick the wrong category, the labels are wrong: the Data Quality lesson finds them.

The Same Steps in Python

The starter kit's prepare_data.py does all of this and writes a report. Run it on the same fixture:

python prepare_data.py fixtures/tickets-export.json --out data

{
  "tickets": 36,
  "kept": 32,
  "skipped": { "no consent": 2, "not English": 2 },
  "split": { "train": 20, "val": 3, "eval": 9 },
  "per_intent": { "refund": 6, "where": 4, "address": 6, "cancel": 6, "damaged": 5, "account": 5 }
}

The split is by customer: a hash of the customer's email decides train, val or eval, so one customer never lands on two sides. The kit's tests check that, and that no planted PII survives.

Common mistakes

Check Your Understanding

Exercises

What to Remember

• A real export needs consent checks, HTML clean-up, quoted replies and signatures cut, and PII redaction before it is training data.
• Redact with placeholders, and keep order numbers, prices and dates.
• Use the part of the thread your task needs.
• Split by customer, and write down what you left out.
• prepare_data.py in the starter kit runs the same steps, with tests.

Next: Data Quality, how to find duplicates, leaks, wrong labels and one customer who sent too many tickets.