It’s a bit of a conundrum. In today’s digital age, data makes the business world go ’round—which means companies need data—lots and lots of data. But data in its raw state is virtually useless. To make full use of the data you’ve collected, you need a toolbox filled with natural language processing (NLP) techniques to help you leverage the power of machine learning and extract the multiple insights hiding within your data.
Among the more useful of these NLP tools is the named entity recognition (NER) technique. Unlike more simple NLP techniques, NER is a supervised learning model: Before you can use an NER model, you have to train it first with a dataset of predefined entity categories. This highly customizable training is what gives NER power, because you predefine the information that you want extracted.
What is named entity recognition (NER)?
NER, also known as entity chunking or entity extraction, is an NLP technique that scans text data to identify and categorize predefined named entities. The process extracts structured data—the entities—from raw data. This structured data can then be analyzed for insights and applied in numerous business use cases.
What are named entities?
Named entities are the fundamental pieces of information found within every statement or sentence, such as:
- Names
- People
- Organizations
- Places
- Products
- Temporal units
- Date
- Times
- Numbers
- Measurements
- Distances
- Money / Pricing
- Quantities
Named entities are the key to the flexibility of an NER model: Because they can be whatever you choose, you have the ability to train your model to extract the exact information you need for your particular use case.
How does NER work?
Consider what happens when you read: As your eyes scan the words, you automatically identify any named entities. For example, when you read the sentence “On October 27, 2022, Wendy left Neverland to begin her new position as CEO of Lost Boys Inc.” you likely recognized the following named entities:
- Date: June 27, 2022
- Person: Wendy
- Location: Neverland
- Occupation: CEO
- Organization: Lost Boys Inc.
Machines, on the other hand, use binary language (0, 1). And 0s and 1s are a far cry from the richness and depth of human language. Because machine language and human language are considerably different, machine learning must be used to first train the NER model. This is done using predefined datasets containing your chosen named entity categories. For example, in the scenario above the entity categories date, person, location, occupation, and organization have been predefined.
So how does NER work? Once they’ve been trained, NER models use a two-step process to mimic the way humans read. First, the model identifies a named entity, and then it classifies or categorizes that entity.
Some NER systems use word vectors for improved speed and accuracy. Word vectors represent words as numbers, but instead of simply assigning each word a number, word vectors generate numerical representations in decimal format across a number of dimensions, such as frequency of appearance in a variety of contexts. The result? Similar words have numbers closely related to each other, enabling the NER model to find similar words quickly and accurately.
Here’s an extremely simplified example. Let’s say you’re working with a dataset made up of reviews of your furniture store. Working across one dimension only, the word vector your pretrained model generates for “lamp” is 0.223458993. Word vectors can help you find words similar to “lamp”:
light: 0.212835892
sconce: 0.212444586
chandelier: 0.199875213
Solving the ambiguity challenge
Part of the complexity of human language lies in the number of words that have multiple meanings. Also known as homonyms, for humans these words typically aren’t ambiguous as long as there’s sufficient context to decipher the correct meaning.
For example, we know what’s meant in each of the following sentences because of the context:
The pitcher threw three strikes in a row.
They asked for another pitcher of iced water.
Machines, however, aren’t able to understand context, so the ambiguous language in this example presents a real challenge. But because NER models are supervised learning models, meaning they must be trained first before they can be applied, machine learning approaches have been developed that help them to meet this challenge.
Popular NER use cases
You could simply use NER to collect and store more structured data in a database. But NER’s ability to extract structured data from raw data makes it useful in a large number of use cases, including:
E-commerce search function. Accurate search function can be crucial for e-commerce sales. For example, a customer searching for a “white round cocktail table” isn’t looking for white products, round products, cocktail products, or just any kind of table. An NER-powered search function would serve up the right results by categorizing “white” as [product color], “round” as [product shape], and “cocktail table” as [product type].
Customer support. Multiple departments, products, and branch locations can create quite a challenge for your customer support team. Before complaints can be addressed, however, customer emails and tickets need to be sorted to determine which locations, products, and departments are involved. NER can make your team’s workflow more efficient by categorizing entities such as [location] and [product], and automatically sending the sorted complaints and queries to the right team member.
Track recurring issues. These days, customers are just as likely to turn to social media to lodge a complaint as they are to email or call. Businesses aware of this trend often create a separate social media handle specifically for handling such complaints. An NER model can then be used on this complaint-oriented social media feed to sort tweets or posts into data that can be used to detect products, locations, or even key times of the day that are drawing recurring complaints.
Support chatbots. Chatbots provide a way for businesses to offer speedy resolutions to common problems. You can use NER to train your support bot to efficiently address a number of typical support issues by using a training dataset containing entities relevant to these issues within the chat context. Based on the identification and classification of these entities—for example, product serial numbers or coupon codes—the bot can either serve up a relevant response or flag the chat for escalation.
Targeting/audience segmentation. ShareThis applies NER to filter its data down by extracting entities from webpages carrying its Javascript code, enabling you to reach your perfect target audience. So, for example, by distinguishing Apple (the company) from apple (the fruit), NER can identify the audience segment that has an interest in Apple products.
Conclusion
The raw data you’ve collected can’t be used as-is. Enrich your data further with data from a provider like ShareThis, then apply NER models to identify, extract, and classify important entities. Using NER, you can transform your enriched data into an invaluable source of insights that can be applied across a variety of use cases, and enable you to better support your teams’ workflows.
