Text Classification for Business Applications: A Practical Guide
Open a shared inbox on a Monday morning and you’ll see the problem straight away. Invoices sit next to complaints. A cancellation request is buried under a delivery query. Someone has to read every message just to decide who should deal with it.
Text classification does that first read for you. It uses natural language processing (NLP) to give each piece of text a label, such as a category, an intent or a priority, so the right person sees it sooner.
None of this is new. Spam filters have worked this way for decades. What’s different now is that modern language models can sort text into categories they were never trained on, cope with messy writing and handle dozens of languages. Jobs that were too fiddly to automate five years ago are now realistic.
This guide covers how text classification works, how to choose between the main approaches, where it earns its keep, and how the Department for Work and Pensions (DWP) uses it on its post. It also covers how to build something your team will actually trust.
What Text Classification Does
A classifier reads text and attaches one or more labels. What those labels mean is up to you.
| Classification type | Example labels | Typical use |
|---|---|---|
| Topic | Billing, delivery, technical fault | Routing tickets and emails |
| Intent | Cancel, upgrade, complain, enquire | Triggering the right workflow |
| Sentiment | Positive, neutral, negative | Spotting unhappy customers |
| Urgency or risk | Urgent, vulnerable, safeguarding | Escalating sensitive cases |
| Document type | Invoice, contract, payslip, letter | Feeding document processing |
| Spam or abuse | Spam, phishing, abusive, genuine | Protecting inboxes and communities |
| Compliance | Contains personal data, regulated advice | Monitoring and data protection |
One early decision is whether each item gets a single label or several. An email that is both a complaint and a billing query needs a multi-label setup. Switching later usually means redoing the labelled data, so settle it before you start.
Four Ways to Classify Text
Rules and keywords
“If the subject says refund, tag it billing.” Rules like this take an afternoon to build and anyone can read them. They fall over when customers use words you didn’t predict, and once you have a few hundred of them, nobody wants to maintain them.
Supervised machine learning
Here you train a model on examples people have already labelled. Logistic regression and support vector machines are the classic choices, and transformer models such as BERT are the modern one. With a few thousand good examples per category, supervised models are fast, cheap to run and very accurate on problems that don’t change much. If you’re unsure whether you need a neural model at all, our guide to machine learning vs deep learning sets out the trade-offs.
Zero-shot and few-shot classification
A large language model, or a natural language inference model, can sort text into categories you describe in plain words. Zero-shot means no labelled examples. Few-shot means a handful. It’s the quickest way to get a prototype working, and it suits situations where labels are scarce or the categories keep changing.
Topic modelling
Topic modelling is unsupervised. It doesn’t apply labels you chose. It finds the themes already present in a pile of text. It’s most useful before you’ve decided what to track, when you simply want to know what customers keep writing about.
| Approach | Data needed | Strengths | Weaknesses |
|---|---|---|---|
| Rules | None | Transparent, instant | Brittle, high maintenance |
| Supervised ML | Labelled examples | Accurate, cheap at scale | Labelling effort, retraining when categories change |
| Zero-shot or few-shot | Category descriptions, a few examples | Quick to start, flexible | Higher running cost, needs careful testing |
| Topic modelling | Unlabelled text | Finds themes you didn’t know about | A person has to interpret the themes |
In practice, the approaches are often combined. Topic modelling shows you the landscape, a zero-shot model gets you running, and a supervised model replaces it once you’ve collected enough labels to justify the switch.
Where Text Classification Pays Off
- Email routing: Finance, HR, procurement and customer service inboxes can receive thousands of messages a week. A classifier sends each to the right queue and pushes urgent ones to the top.
- Support ticket tagging: Tags applied by hand are inconsistent, because nobody agrees on what to call things. An automated tagger applies the same fifty tags the same way every time, so your reports start to mean something.
- Customer feedback: Survey comments, reviews and complaints can be grouped into themes like price, quality, delivery and staff. You get a ranked list of problems without weeks of manual coding.
- Document sorting: Before you can extract data from a document, you need to know what it is. Classification splits a mixed batch of invoices, contracts, ID documents and letters so each goes down the right path.
- Spam and moderation: Classifiers catch spam, phishing, abuse and unsuitable user content before it reaches staff or customers.
- Compliance: They can also flag possible mis-selling, complaints that must be logged under regulatory rules, or personal data that shouldn’t be leaving the building.
Case Study: How DWP Sorts 25,000 Letters a Day
DWP receives a lot of paper post from benefit claimants, known as whitemail. About 25,000 scanned documents arrive each day. Most just need to reach the right benefit team. A few come from people in real difficulty, mentioning suicidal thoughts, self-harm or domestic abuse. Those can’t wait in a queue.
DWP built two tools. The Vulnerability Scanner identifies letters that may come from vulnerable customers and tags them by theme. Whitemail Insights labels other letters that have no automatic handling rule, so they can be routed properly.
The process is straightforward. Letters are scanned and converted to text, and personal data is redacted before classification begins. The tools use a transformer model (BART) for zero-shot classification across eight vulnerability categories and nine general themes. Because it’s zero-shot, DWP didn’t need a large set of labelled letters to start.
People stay in the loop. An analytics team reviews a daily vulnerability report before passing cases on, and staff assess each document individually. The tools don’t make or influence benefit decisions.
DWP’s published algorithmic transparency record lists the benefits as less time spent sorting post, faster attention to urgent cases and more targeted support. It doesn’t publish accuracy figures, so don’t treat it as a performance benchmark.
Three lessons carry over to any organisation. Zero-shot can get a useful system running without a big labelled dataset. A sensitive label should send a case to a person, not trigger an automatic decision. And publishing how the system works makes it easier to trust.
How to Build a Classifier That Works
Write a definition for every category. Add a few examples too. If two colleagues would put the same message in different boxes, the model will struggle as well.
Keep the list short. Ten clear categories beat eighty overlapping ones. If you need more detail, use categories and subcategories.
Use real data. That means typos, one-line messages, long email threads and every channel the text comes from. If your data is scattered or inconsistent, getting your business data ready for AI comes first.
Label with care. Write guidelines, have two people label the same sample and settle the disagreements. A smaller set of well-labelled examples beats a large sloppy one.
Pick the approach to match your data. Start with zero-shot or few-shot if you have few labels, and move to supervised once you have plenty.
Check each category separately. A model can score well overall and still miss most of your rare, important cases, such as fraud or vulnerability.
Set confidence thresholds. When the model isn’t sure, a person should look at the item.
Keep watching it. Language and topics drift. Review samples regularly and retrain when results slip. Treating this as part of the wider AI development lifecycle stops it being forgotten after launch.
Measuring Performance
| Metric | What it tells you |
|---|---|
| Precision | Of the items given a label, how many belong there |
| Recall | Of the items that belong, how many were found |
| F1 score | How well precision and recall balance |
| Per-class accuracy | Whether small but critical categories work |
| Automation rate | How many items the model handles with enough confidence to act |
| Business outcome | Routing time, handling time, missed urgent cases |
Decide which mistakes you can live with. In spam filtering, blocking a real customer is usually worse than letting a few junk emails through. In vulnerability detection it’s the reverse: a false alarm costs a few minutes, and a missed person in crisis costs far more. Set your thresholds accordingly.
Common Mistakes
- Training on messy old tags. Years of inconsistent manual tagging make poor training data. Relabel a sample first.
- Too many categories. Overlap confuses people and models alike.
- Ignoring rare classes. They need deliberate attention in training and testing.
- Automating sensitive decisions. If a label affects someone’s rights or welfare, it should inform a human decision, not replace it. Our guide to AI governance for UK businesses shows how to set that up.
- Overlooking data protection. Text is full of personal data. Process as little as you can, redact where possible and follow UK GDPR principles.
- Leaving it ownerless. Someone has to look after the taxonomy, watch performance and approve changes.
Build, Buy or Configure?
Many helpdesk, CRM and email platforms now have classification built in. That’s fine for simple, standard categories. Cloud NLP services offer classification APIs that a developer can set up quickly.
A custom model starts to make sense when your categories are particular to your business, when accuracy on certain classes really matters, when you handle sensitive data under strict controls, or when per-call pricing gets expensive at your volumes. A common route is to start with the built-in tools and build custom only for your most important or highest-volume processes.
IIH Global’s natural language processing services cover designing, training and deploying classifiers around your own categories, languages and systems. If you’re still comparing providers, our list of top NLP companies in the UK is a good place to start.
A classifier rarely works alone. It needs to feed a helpdesk, a CRM or a case management tool, so that a label actually triggers something. That’s what our AI integration services are for.
Beyond Routing: Using Labels to Spot Problems
Routing is the obvious benefit, but it’s not always the biggest. Once every email, ticket, review and survey comment has a consistent label, you can see things you couldn’t before: which product draws the most complaints, which policy confuses people, which region has delivery trouble, and what changed after a launch. That kind of text mining often ends up more valuable than the routing, because it points you to the source of the problem.
Where Classification Is Heading
Large language models are making classification easier to change. You can add or adjust a category by editing its description, with no retraining. Smaller models fine-tuned from larger ones keep costs down at high volumes. Multimodal classifiers, which look at text, images and documents together, are starting to appear.
Regulation matters too. Public sector bodies are expected to be open about their algorithms, which pushes teams to document how a classifier works and who checks its output. Private companies would do well to follow suit.
Conclusion
Text classification turns a pile of unstructured text into something you can act on. Work reaches the right person sooner, urgent cases stand out, and patterns appear that manual sorting would never reveal. DWP shows that a zero-shot model can be useful quickly, as long as people review what it flags.
If you’re starting out, keep the first version small. Pick a handful of clear categories, measure each one separately, keep a person involved wherever a label affects someone’s welfare, and name an owner who will watch the results after launch.
If you’re planning a project, whether you’re choosing between zero-shot and supervised approaches or connecting a classifier to the systems you already use, get in touch with IIH Global and we’ll talk through what you need.
Share On :