There are 43 million of Bahasa Indonesia speakers globally. Indonesia is an important market for Clare.AI, and we have also benchmarked Clare.AI’s NLU against other NLU engines. We have only compared against WIT.AI as 2 other systems – IBM Watson and API.AI do not support Bahasa. Once again the format of our testing remained the same, except this time we imported 100 questions, and tested each system with 300 variations of those initial questions.
For the variations, we have used abbreviated forms as Indonesian tend to speak with slangs and short phrases. Again, business team has created the variations with local Indonesian speakers, and also kept the dataset confidential from the technical team before testing.
The table above shows the results. Once again, data can be found on Github
As you can see, Clare.AI processed and produced answers to all 300 question variations 6x faster than WIT.AI with almost double the accuracy. Clare.AI managed to correctly match 70% of the 300 variations to the original questions imported into the system compared to WIT.AI’s 35.3%, showcasing the strengths of Clare.AI’s NLU.
As proverb says, “if you cannot measure it, you can’t improve it”. We will continually refine the Indonesian NLU algorithm by ingesting additional data. We will share our continuous improvement.