Have you ever heard of the 80/20 rule? It says that you can accomplish 80% of a goal with 20% of the effort.
After extensive research about freeform conversational topics between businesses and their customers, we found that this rule does not apply when helping our customers build conversational artificial intelligence (AI) engines. We believe this is because our data shows that over 80% of customer communication is more complex than originally anticipated.
This can be viewed as selection bias, as a majority of the conversations we analyzed in our research all originated from websites that were designed to be at least somewhat self-serve. This leads to the “easy” questions being answered directly by the website’s general information or FAQs, while the “complex” questions flow into a chat or system of engagement such as Glia.
These options lead us to think about two ideas that are popular in the SEO world- “Short Tail” and “Long Tail”. If you think about it, search engines have a lot of similarities to conversational AI engines. For one, the input is the same. Free text and the output are the results ranked in the order that the search engine believes is the most accurate. A short tail search could be “Restaurants in NYC”, while a long tail search may be “vegan Italian restaurants in NYC”. While you may think that most searches are short tail, data shows that 70% of search traffic is really long tail.
When building a conversational AI engine, companies typically want to start with the most popular or frequently asked questions, or conversational topics that their customers are inquiring about. The goal is to cover at least 80% of the conversations by providing answers to the top 20% of questions. However, much like with SEO, the issue is that long tail queries comprises more than 80% of the conversational topics. This means even if a company focuses on the most popular topics, they would only be covering about 20% of cases.
These are some of the reasons why we don’t recommend deploying a conversational AI engine without going through a proper optimization phase to our clients. The use of an optimization phase allows for a conversational engine to learn from failures with a company’s agents. This way it avoids delivering a negative experience for customers.
Even if the failure rate is 80%, customers would have no idea. This failure actually allows the agents to be directly assisted by the conversational AI engine ~20% of the time and for the agents to help train the engine the other 80% of the time.
