Entermind, a data and AI consultancy, today announced the launch of Confessions of an AI Chatbot, a whitepaper revealing the findings of Malaysia‘s first systematic study of customer service chatbot quality across travel, telecommunications, eCommerce & Wallet, and financial services.
Malaysia is investing billions in AI infrastructure. But when a real customer opens a chatbot to cancel a flight, dispute a bill, or check their balance, the experience tells a very different story.
Drawing on the Entermind Chatbot Quality Index (CQI), a proprietary framework for the systematic testing and quality assessment of customer service chatbots, the study evaluated 24 of the country’s top customer service chatbots against 26 standardised binary tests grouped across five weighted categories: Comprehension, Access,
Experience, Functional Capability, and Safeguards.
The findings reveal a market in which a small group of chatbots are genuinely delivering, and the majority are quietly failing the people they were built to serve.
What the Data Found
The average CQI score across all 24 chatbots is 49.5%. Eleven of the 24 score below 40%. Six do nothing more than greet users and route them to a human agent. Only three can complete a transaction within the chat itself. The lowest score in the entire index? 4.0%.
The comprehension gap is particularly stark. Only 23% of chatbots correctly understand negation. Only 30% can handle a change of topic mid-conversation. Two-thirds cannot understand Bahasa Melayu, Manglish, or both. In a market of 34 million people where multilingual communication is the norm, this is a glaring shortcoming.
“Most chatbots in Malaysia are a glorified FAQ search bar,” said one telco evaluator whose feedback is featured in the whitepaper.
The Investment Gap
The whitepaper identifies capability investment and design as the two factors that separate the top performers from the rest. Chatbots scoring above 70%, seven of the 24 tested, share these common traits: genuine language-understanding, end-to-end transaction architecture, and robust safety infrastructure. Those scoring below 40% typically fail across all these dimensions simultaneously.
Commenting on the whitepaper, Prashant Kumar, CEO of Entermind, said, “GPTs are rapidly resetting expectations that customers have from enterprise AI bots. It’s clear from the audit that the gap is yawning. As a whole new Generation AI shapes up, the upsides are highly rewarding.”
The whitepaper is available for free download at entermind.com/media/whitepaper/confessions-of-an-ai-chatbot
Share Post:
Haven’t subscribed to our Telegram channel yet? Don’t miss out on the hottest updates in marketing & advertising!
