This post originally appeared on the Finn AI blog, which is now part of Glia.
Finn AI is constantly looking for ways to improve, update, and refine our chatbot model. Two key areas we always want to see improvement on is the bot’s Coverage and Recall in the retail banking space: Its breadth of knowledge, and its ability to understand user questions respectively. Both of these abilities are drawn on a chatbot’s ability to process normal human speech (or in this case, text) into utterances that it can understand. This is referred to as its Natural Language Processing (NLP) abilities.
Improving a chatbot’s NLP is the key to its success: and it’s something that’s not always straightforward in how to achieve. Much of the focus on chatbot development in recent years has been focused on the model itself: the actual architecture that makes the chatbot run. Finn, however, has taken a different approach. We describe ourselves as data-centric, meaning that we’re not as focused on the model, but the actual information that the bot uses to generate its answers. A chatbot cannot function without a wealth of data for it to use, and ensuring that only high-quality, well labelled data is utilized by the bot makes for a more consistent, overall more accurate chatbot. Read more about our Data-Centric approach to chatbot development here.
We’ve combined this data-centric approach with new developments in Natural Language Understanding efforts and a Multi-Task Learning Approach to create what we call our Joint Model. The Joint Model has a vastly improved NLP ability, leveraging pre-trained embeddings to give the model an understanding of English in a generic sense. These improvements have improved the Recall of our chatbot by 8%, and decreased false-positives by 33%. This means our chatbot is more accurately able to understand user inputs, and respond in more helpful ways than before. Here are some user inputs that previously tripped our bot up, how the bot used to respond, and how it responds now:
| User Input | Old Response | New Response |
| Good morning. I would like to know why my card isn’t working. I did try and use it last night for a 53. Dollar purchase and it didn’t work. | Locked out of your account? Are you using your correct username and password?Try resetting your password and then log in again. If that doesn’t help, I’ll connect you to a human who can help solve this. We’ll fix it! | A card can be declined for a number of reasons. Some of the most common reasons are:• You’ve reached your credit limit.• Your card has been flagged for potential fraud.• You entered the wrong information when making a transaction, including an incorrect card number, expiry date, CVV number or PIN.• You’re behind on your payments and have entered a delinquent status.• Your card is expired./get_reactionI can help you check your balance to understand if you’re close to or have exceed your credit limit. One of our human agents can also help you determine if another reason is involved. |
| how do you charge for debit card for currency exchange when I’m in another country. Going to Cambodia and Thailand | For the latest information about the most common currency exchange rates, you can check the Government Federal Reserve Department’s website. | Great news – you can use your card overseas for free! Currency exchange rates are subject to the daily rate at the time of transaction.You can withdraw cash at any ATM in the Cirrus network for a fee of $5 per transaction, and there’s a $3 transaction fee for each transaction that you make with another bank’s ATM. Sometimes other banks will also charge a service fee for using their ATM. |
| How do i get my fiancé and my cashback credit cards onto the same account? Or can i get a second card on mine for him to use so that we keep the amounts together? | Hmm 🤔, did you mean any of these options?[Product Recommender], [Replacement Card] If not, please ask your question in a different way. Write a shorter sentence and I may understand better. Or type ‘Show menu’ to do something else. | You can add a cardholder to your account by logging into online bankingYou can also visit our bank to open a joint account, or an account for someone else. 😊 |
These are some examples of ways the chatbot’s performance has improved, but overall the Joint Model has led to a chatbot that better satisfies customers with its quick, seamless ability to assist them without needing to rely upon a human agent to finish the job.
One of Finn AI’s current customers was happy to report their newly improved chatbot was able to detect an additional 40 different user goals with the new Joint Model, providing even further coverage and granularity in its answers. This chatbot has already provided the bank’s users with a 5x faster customer service experience on average, and handled nearly half of all user inquiries by itself with no human interaction needed. Their model was upgraded for free, with no additional costs needed to deliver an even better experience than before.
This Joint Model serves as one of many new updates that Finn AI has done to our chatbot model throughout the years, and certainly will not be the last. We’re constantly looking for ways to improve the architecture and dataset of our chatbot in order to provide our customers, and their end-users, with the most exceptional digital self-service experience possible.
