CRA’s Costly Chatbot – $18M Spent on AI That Gave Wrong Answers

Imagine needing help filing your taxes, turning to a tool designed to guide you, and getting the wrong answer more than half the time.

That’s exactly what happened to millions of Canadians who used the Canada Revenue Agency’s AI chatbot, “Charlie.” Despite its purpose to make tax help more accessible, the project ended up being a high-priced letdown — and it cost taxpayers over $18 million.

Background

Charlie the Chatbot was announced back in 2020 by the CRA, with big promises and even bigger price tags. The chatbot was supposed to help Canadians file their taxes by answering questions instantly on the CRA’s website. But from the start, the CRA admitted the bot was still “learning.” The idea was that, over time, it would get smarter through use.

Sounds futuristic, right? Except, here’s the catch — according to Auditor General Karen Hogan, Charlie was accurate only 44 per cent of the time during tests. That means more than half the answers were wrong or too vague to be helpful.

Breakdown

So where did all the money go?

Let’s break it down:

Expense Category	Amount Spent
Salaries & Benefits	$13.67 million
IT Consultants	$3.21 million
Total Project Cost	$18+ million

The majority of spending went toward internal staffing: salaries, benefits, and even travel. The rest went to consultants brought in to develop the AI chatbot technology.

That’s a lot of money for a tool that can’t even score a pass in accuracy.

Problems

Charlie’s performance was underwhelming. When tested by the Auditor General’s team with six tax-related questions, the chatbot got only two answers right. In contrast, other public web-based AI tools answered five out of six correctly.

Worse, the answers it did provide were usually short, lacking context or helpful links. And even though Charlie is still live on 13 CRA pages, its usefulness is clearly questionable. It’s one thing to miss the mark — it’s another to charge taxpayers millions to do it.

Stats

Despite the issues, CRA claims that since launch, Charlie has had over seven million “conversations” and answered more than 18 million questions. But are those answers helping or hurting?

Even the CRA admitted they can’t confirm how many responses Charlie actually got right, unless they go through every interaction manually — something they haven’t done. So while they estimate a 70 per cent “accuracy threshold,” there’s no way to know how reliable that number really is.

Updates

A newer, unreleased version of the chatbot has reportedly reached “about 90 per cent” accuracy. That sounds better, but without an independent review, it’s just another claim. And since the original version is still active and getting used by the public, the damage may already be done.

In CRA’s recent 100-day service improvement plan, they proudly claimed that 70 per cent of Charlie users had achieved their “goal” in November. But here’s the catch — reaching a goal doesn’t always mean getting the right answer. Sometimes users give up, find what they need elsewhere, or accept a vague answer just to move on.

Trust

At the heart of this is public trust. Canadians expect the CRA to get things right. Whether it’s a tax refund, an audit, or a chatbot, accuracy matters. When a taxpayer gets incorrect information — especially during tax season — it can lead to costly mistakes, missed deadlines, or penalties.

You wouldn’t trust a GPS that gets you lost half the time. So why should Canadians trust a tax chatbot with the same record?

Priorities

It’s not that Canadians are against innovation. We want smart tools. We want easier ways to get help online. But if you’re going to spend tens of millions on a digital assistant, it should work. Otherwise, that money could’ve been better spent — maybe on training human agents or improving call centre access, which has been a major issue for years.

The CRA did take steps to address these issues by rehiring over 1,200 call centre staff this fall. But that’s only part of the puzzle. The chatbot fiasco shows a deeper problem — rushing to adopt AI without proper oversight or quality control.

Reflection

Charlie was supposed to be the future of tax help. Instead, it’s become a cautionary tale. Canadians aren’t angry because the CRA tried to innovate — we’re frustrated that it was done with so little accountability.

There’s hope the newer version will perform better. But until then, many Canadians will go back to waiting on hold for a real person — a person who, hopefully, knows what they’re talking about.

FAQs