“Information is the oil of 21st century, and analytics is the combustion engine.”
Gartner

Photo by **Andrea Piacquadio** from **Pexels**

Companies produce large amounts of data regardless of their volume or the sector they run on. The data became decisive for institutions, especially in terms of providing a competitive advantage with the growing potency of big data and analytics. To analyze the data efficiently is key for preparing growth strategies, discovering new business opportunities, determining the recruitment of productive employees, and finding valuable information about maintaining plans. Using customer data (opinions), companies can analyze their customer’s purchasing behavior. Thus, they can achieve great success in attaining loyalty by keeping new customer acquisition and existing customer satisfaction high. Considering this information, they may have the opportunity to create the right product and service strategy.

Chatbot and Data Dissemination

The Chatbot market worth in 2019 is estimated $ 17.17 billion US dollars and will reach $ 102.29 billion by 2025. Research by Andrew McAfee and Erik Brynjolfsson published in Harvard Business Review. According to this research, the companies which use big data and analytics will have a higher efficiency rate (5% – 6%) and higher profitability than their peers. Another study by McKinsey confirmed that companies achieved a 6% increase in profits using data-driven analytics that grew to 9% after five years.

Chatbots allow businesses to make data-driven decisions quickly and efficiently. The chatbot design focus on understanding human behavior by nature. As part of the self-learning process, it keeps systematic records of data, metrics, preferences, and trends, and adapts its relevance and responses according to user interactions. Employees do not need to examine multiple dashboards to obtain meaningful information or endure a cumbersome process to jump into various applications. Robots can process large data sets, shorten process time, and increase employee productivity. This feature allows Chatbots to play an essential role in the field of Data Analytics, and companies try to use the best strategies, tools, and chatbot technologies for their structure.

As the chatbot carries out direct interactions with the end-user and proactively asks relevant questions to add further to the knowledge base, and can then be used to provide valuable information. Chatbots can automate data searches and extract data from various business systems such as ERP, CRM, Data Warehouse to analyze and generate critical insights. Chatbots are useable to send alerts, updates, and real-time information to employees.

Chatbots are programs that use Artificial Intelligence AI and Natural Language Processing (NLP) technologies to leave the impression of chatting with a real person. Natural language processing allows bots to receive chat-based inputs from the speech / messaging interface, make quick and efficient decisions, and respond with relevant information. This means that a bot can answer questions at any time and any place.

We can list the NLP methods and processes used while developing chatbots as follows:

Pre-processing

In most of the projects that use natural language processing, due to the nature and daily use of languages, the text received as input must first go through a pre-processing step. Detecting and correcting spelling mistakes with pre-processing allows additional factors such as emojis and links in the content to be extracted. The first steps of this pre-processing are the processing of the words or word groups in the text, morphologically and semantically.

According to the area of use, the result of pre-processing can output as follows:

Example: “Can I see your products here? 🤔artiwise.com”

Output: [Can, I, see, you+possessive, product+plural, here, ?+punctuation ]

As can be seen in the example above, the original text needs to be converted into a reduced form to tackle the feature sparsity problem, which is a problem that occurs when a feature may be seen in very few examples in which the data is represented occurs.

Language Detection

Since Chatbot applications are generally available online, users are likely to communicate in different languages. In this case, the language of the input text should be determined, and the text analysis should be done accordingly. If the input is in a language that the application supports, it can continue in that language. Otherwise, it can provide feedback that the language used is not supported. Alternatively, some apps serve users using Machine Translation technologies to offer multilingual support.

Intent Detection

In the dialogues, a party expresses an intention to the counterpart with the sentence it establishes, and the counterpart understands this intention and improves the speech by giving a response in that direction. A real person can identify the primary purpose of another person’s text; they can provide an answer that matches this intention. For this natural dialogue to be imitated by bots and at the same time to serve people effectively, the boat must be able to understand what its counterparts mean (which topic she/he is talking about). Understanding the intention is only possible with prior knowledge analysis. There are several classes of intentions according to the chatbot’s domain, and perceiving the intent of the user’s expression is the first step to continue the conversation. For this reason, bots should be capable of distinguishing the subjects that are likely to be discussed in the domain they serve. At this point, classifiers using Machine Learning and Deep Learning methods are generally preferred.

Hello! -> intention: greeting

How many cars can I rent? -> intention: car rental

What is the highest exchange rate for the US dollar? -> intention: exchange rate info

Named Entity Recognition

Understanding the intent of the user is not sufficient alone. Because, in addition to detecting the intention, some other parameters may be needed to continue the conversation. It may be necessary to ask the users for the parameters. Moreover, it might be necessary to extract the parameters that are needed from the input in the natural language entered by the user.

If there are n parameters needed for generating the requested response and k of them are provided in the content that the user also stated his purpose, the chatbot should not ask those k parameters again to give a natural dialogue appearance.

Instead, it must request the missing (n-k) parameters from the user in a proper flow. Then it should direct the user to confirm the command statement filled with the settings of intent. If it gets the necessary approval, it should execute the command and share the output with the user.

Named Entity Recognition is a study that aims to automatically extract the discourses that fall into categories that are specifically defined according to the task within a text. This task is frequently used in the NLP field, and the most common form of this task is extracting the values of the standard classes such as a person, location, date/time, currency, and amount.

On behalf of Ahmet Yılmaz, I would like to make a two-person reservation to the organization in Ankara.

On behalf of [Ahmet Yılmaz](person), I would like to make a [two-person](amount) reservation to the organization in [Ankara](location).

In this example, the name, location, and amount of people needed for the reservation can be automatically extracted from the user’s statement. In this case, the bot will ask the user for the date information, which is the only additional parameter needed. It will make the reservation according to these parameters.

Domain Knowledge

Chatbots generally operate in a specific domain, and users use the application to obtain the information they do not have in that domain or to take action in that domain. Chatbots must be equipped with data related to that domain to respond to these requests. If field information is limited, it can be kept by removing the decision tree with rule-based systems. However, it is known that such rule systems are insufficient in natural language processing problems and even cause great turmoil in the future. For this reason, the decision mechanism is developed with supervised or unsupervised learning methods when a field knowledge is required (which is usually the case) where a relevant field of activity is pervasive.

Keyword Extraction

If the field information is learned from documents, it is essential to know which expressions carry more information to extract data from the relevant material. These statements are keywords that summarize the text and give us the information we seek in the text.

For example; in a banking app, keywords of the information sentence “EFT operations can only be done in weekdays, from 8 am to 5 pm.” can be specified as follows:
“EFT” -> operation type
“weekdays” -> time slot – day.
“from 8 am to 5 pm” -> time slot – hours

Keyword detection contributes to making the necessary queries more effective by applying both in the user’s expression and in documents containing area information.

Proceeding from the example above, “What hours can I make an eft?” The chatbot can also analyze the keywords in this question as follows:

“eft” -> operation type

“what hours” -> question – time period – hours and the chatbot application that makes these analyzes can give the following response in the light of the information it has obtained: “ from 8 am to 5 pm.”

Natural Language Generation

Although the answer of the chatbot in the above example is acceptable, it may be dimmed in a dialog, or the keywords extracted for each question may not be understood as in this example. For this reason, when the chatbot understands the intention of the user and finds the appropriate answer for him, it must give this answer again by using natural language. At this point, the chatbot is expected to form a sentence using the rules of the natural languages.

Natural language production techniques are used to meet this expectation. These techniques can create rules according to the structure of the language, or they can be in the form of imitation and imitation of the structure in existing texts, which is generally a more effective method. Natural language generation techniques enable the chatbot to transform the information it possesses into a statement that conforms to natural language rules.

For our example of processing these techniques, the previous example can be transformed into such a sentence: “You can perform your EFT operations between 8 am and 5 pm. Is there anything else I can help you with?”

Analyze 360 Degree Customer Insight on a Single Platform with Artiwise Analytics

Channels to be analyzed by Artiwise Analytics

Chatbots are a great option to help customers efficiently. When equipped with artificial intelligence, chatbots can use contextual information to deliver relevant messages and proactive service when they detect a potential problem. Artiwise Analytics allows us to analyze all the data (chatbot contents, call center voice recordings, social media data, written survey content, e-mail notifications services, etc.), customers’ preferences, needs and approaches on a single platform with machine learning. Artiwise analytics allows you to interpret end-to-end customer experience and take quick action with real-time categorization.

Flexible Customer Service:

You can quickly identify the prior user in your emotion analysis and categorized customer insight data. By letting your customer experience team be informed about prior users rapidly, you can provide solutions before the problems turn into a crisis.

Customer satisfaction tracking:

You can discover how users interact with you about your brand, product, or service, and how they feel about you.

Detect emotion-triggering issues instantly:

Artiwise Analytics collects emotion change trends with emotion analysis and helps you discover triggers. You can analyze emotion trigger issues in detail about category trend change and emotion trend change charts.

Generate upselling strategies and discover new areas in your market:

You can see your development areas with the root-cause analysis in the feedback you receive about your product, brand, or service, and you can see the orientation of your audience with emotion analysis.

Continuous forwarding:

The problem in the product or service channels facilitates the transmission to the appropriate department by tracking satisfaction. The instant categorized data can be accessed from the relevant customer experience team to report data graphs and can follow by assigning priority customer insights among the groups.

Chatbots and Chatbot Analytics