Introduction to NLP
Hello, I’m Sunilkumar Prajapati and I’m going to publish more and more blogs on NLP. I know NLP is very tough and because of that, first, we will see or overview NLP.
What will be the topics will cover in this Blog:
1] What is NLP?
2] What is the Need for NLP?
3] Real World Applications
4] Common NLP Task
5] Challenges in NLP?
1] What is NLP?
NLP stands for Natural Language processing. NLP is a subfield of human Language, Computer science, and Artificial Intelligence. The Goal of the NLP is to teach a natural language to a Machine. The idea is not only machines should understand, but they should also communicate as well.
As per Wikipedia the definition of NLP [It is a subfield of linguistics, computer science, and artificial intelligence concerned with the interactions between computers and human language, in particular how to program computers to process and analyze large amounts of natural language data].
2] What is the Need for NLP.
As per Wikipedia, In neuropsychology, linguistics, and the philosophy of language, a natural language or ordinary language is any language that has evolved naturally in humans through use and repetition without conscious planning or premeditation. Natural languages can take different forms, such as speech or signing. They are distinguished from constructed and formal languages such as those used to program computers or to study logic.
In simple words, Any language that has evolved spontaneously in humans, as a result of use and repetition, is considered a natural language or ordinary language by linguistics, neuropsychology, and philosophy of language. Natural language is different from constructed language. like Python, Java, C++ programming language.
Let me tell you a story about why NLP is important. If we know about human history or human evolution, we will notice in millions of years before we are like animals, but fast forward million years, humans are evolved. Like humans are going in space, they working various of technology and the Animals are still leaving in the jungle and living the same life for millions of years. So the question is what we (Humans )did that we have overtaken an animal. So I think that there are two factors, the first factor is communication and language. Whatever we have achieved the major factor was we were able to communicate with each other, we were able to share our idea. We passed lots think from one generation to other generation like Books. Language and communication help to grow and help to evolve a human begins. The Second Factor is a machine. We have created a lot of machines for different domains. That machine helps humans to grow and evolve more and more.
One more factor, which is the third factor, is yet to come. Therefore, the third factor will be that we will communicate with machines as we do with our fellow humans. As an example, imagine a world where we communicate with any machine, such as an ATM. Imagine an old man who wants to withdraw money from the ATM machine, but he does not understand the interface. The ATM machine will help the man communicate with the machine and assist him in withdrawing money.
3] Real World Applications
Some applications we know very well as we use in our daily life. Alexa, Siri, Cortana, and chatbots are examples of NLP applications.
Let's discuss more NLP applications.
a ]Contextual Advertisements: We know that in the 90’s we used to watch Matches or movies or serials, the same type of advertisement used to be shown all over India. Companies had the assumption that somebody will buy the product. But on today’s date, we have NLP and we can process and observed how people are behaving, what kind of personality he is, So here we can use targeted Advertisements.
We all know that we always get different advertisements on Facebook, Youtube. So what does the company do they check profiles, posts, or analyze the comments, and on the basis they decide that this person is interested in sports or technology or cosmetics. On the basis of that, they have shown us targetted advertisements.
b] Email Clients (Spam Filtering, Smart Reply): We all know about spam mail. In this scenario is some company has sent us a mail and if Gmail seems, this mail is spam then it automatically move to the spam folder and normal email are comes in a normal folder.
New features also have been added where if we received a mail from someone then we can see we get a smart reply( It kind of suggestion). The suggestion gets shown based on email content.
c] Social Media: One of the challenging tasks that come in Social media is removing adult content. Like we have created some social media application and where millions of people upload lots of content and from that content how we can filter the adult content or some users are spreading a negative or hate speech, so will tackle. These kinds of problems we can solve using NLP.
d] Search Engines: We can take the example of the Google search engine. where we can directly search some general questions to Google, that what is the capital of India. So google will fetch the data and it will show answers in a single line.
e] Chatbots: In today's world, lots of companies are using a chatbot. So the chatbots are communicating like some person are seated in the other ends like customer executive. Suppose we take an example of Zomato, as Zomato has millions of clients so they can not seat more customer executives. So chatbots help to solve and provide information on the initial level.
4] Common NLP Tasks
a] Text/Document Classification: Text Classification is one of the most basic NLP tasks and consists of assigning categories (tags) to a text, based on its content.
b] Sentiment Analysis: It’s a type of text classification where the NLP algorithms determine the text’s positive, negative, or neutral connotation. Use cases include analyzing customers’ feedback, detecting trends, conducting market research, etc., via an analysis of tweets, posts, reviews, and other reactions. Sentiment analysis can encompass everything from the release of a new game on the App Store to political speeches and regulation changes.
c] Information Retrieval: Suppose we have some text, from that text we have to extract entities like Name, Location, date, product name. Anything like information if we have to extract this comes in a retrieval.
d] Parts of Speech Tagging: This is one of the important text preprocessing steps. Where what we do, the text which we have, from that text each of word we assign a part of speech like it is a noun or verb or adjective or adverb. This method is used in chatbots so that chatbots can understand each word by word.
e] Language Detection and Machine Translation: We all know that Google Translate work on the same method. We use Google translate and we know that it is a such power full application. Where lots of languages are present. Even if we know a single language we can get to know about other languages also. We can convert our text data from one language to another language.
f] Conversational Agents: Conversational Agents is like a chatbot, but in that two kinds of chatbots are present. One is text-based and the other is speech-based. We can take one example of Siri or Alexa, they both are speech-based. If we talk about the telegram, hike, or on Swiggy or Zomato they have a text-based chatbot.
g] Knowledge Graph and QA Systems: Suppose we have a large amount of database and from that database, we try to connect entities using some logic, and from that, we make a knowledge graph, and then we can translate it into a Question-answer application. Google uses this method.
h] Text Summarization: This basically works like, Suppose we have a full-fledge article, from that article we can do a summarize. We can give a small form. One of the examples we can take is the Inshorts news application. So basically what this application does is that, helps to shorten the news into 60 words news summary.
i] Text Generation: We all use text generation in our daily life. i..e In keyboard if we see, when we type something it automatically on the basis of your previous typing behavior it predicts the next word and we all know that how much it is use full while chatting with someone.
j] Spell Checking and Grammar Correction: We all know about Grammarly. What Grammarly does. If there is a typo in sentences formation or some grammatical error happens, it highlights that word, so that we can improve that word or sentence.
k] Speed To Text: Here we create a conversational agent like Siri and Alexa. Also, this NLP task is used in Google translate where we speak and it converts into text. This also helps where we can speak in one language and we can convert into some other language. For example, the different countries of people trying to communicate but they don’t understand each other language. So google translate help them to understand each other language and communicate.
5] Challenges in NLP: Question comes to mind, Why NLP is so challenging. NLP is challenging is because it is applied to natural language and natural language evolved from year to year. Sometimes some weird thing happened in natural language and those weird things humans only can understand through communication but to explain weird things to a machine is very difficult.
Human language is complex, ambiguous, disorganized, and diverse. There are more than 6,500 languages in the world, all of them with their own syntactic and semantic rules.
Even humans struggle to make sense of language.
So for machines to understand natural language, it first needs to be transformed into something that they can interpret.
Let’s discuss the challenges faced by NLP.
a] Ambiguity: We can say that ambiguity is the capability of being understood in more than one way. one word, one phrase, or one sentence can mean different things depending on the context.
We will try with an example, “I saw the boy on the beach with my binoculars .” In this sentence, there is possible two meaning one is “I saw a boy on a beach who have my binoculars”. Another sentence will be like “I saw a boy with my binoculars on a beach”. So which is the exact meaning we can understand through a paragraph context. It is difficult for machines to understand.
b] Contextual Words: We will try to understand contextual words with the example “ I ran to the store because we ran out of milk.” Here we can see that ‘ran’ is used two times and both times the meaning is different. As a human being, it is easy to understand but for a machine, this is quite difficult.
c] Colloquialisms and slang: Both colloquialism and slang are spoken forms of the language. Both use informal words and expressions. Slang is more informal than colloquial language. Slang is predominantly used by certain groups of people while colloquial language is used in everyday speech by ordinary people.
We will take some examples like “Pulling your leg” or “piece of cake”. In short ‘This task is piece of cake’ and It is possible that the machine probably understands like ‘This is a piece of cake.’ So it is very difficult for machines to understand words like these.
e] Synonyms: Synonyms mean those words that have the same meanings.
This is an introduction to Blogs and soon I’ll post more blogs on NLP. I Would thank Nitish sir from campusX. He had created a great playlist on NLP. The way he teach. It makes it easy to understand topics like NLP.
