RUFARO MAFINYANI | Large language models — what they actually do

This is the seventh instalment of AI Fluency Corner, a 16-part weekly series building a working vocabulary for the technology reshaping business. Edition 06 examined natural language processing — how machines were taught to handle language at all.

Finish this sentence: the rand weakened sharply against the ….

You supplied a word before you reached the end of the line. No database, no reasoning about monetary policy. Your brain has met that phrase thousands of times, so it produced the most probable next word.

That is a large language model — not an analogy for one, but the mechanism itself, scaled to trillions of examples. It is also why analysis can arrive measured, credible and exactly what a consultant would write, with one figure in it that does not exist.

A Google paper published the transformer architecture in 2017; ChatGPT reached the public in November 2022; today it sits inside Word, Excel, WhatsApp and the search bar. Microsoft reports 20-million paid Copilot seats, yet barely a third of licence holders use them regularly. The tools have been bought but they have not been understood fully.

It predicts tokens, not truth

There is no filing cabinet inside the machine, only patterns absorbed from an enormous body of text. And it doesn’t even read words. Text is broken into tokens — fragments averaging four characters — and it predicts one at a time. Roughly 750 English words make 1,000 tokens. Three things follow.

Capacity and cost are counted in tokens, not pages: a 200-page set of financial statements is about 150,000, and the context window is how many fit at once. Nor are languages equal — the same meaning in Zulu or Sotho takes several times more tokens than in English, so it runs dearer and fills that window faster.

Fluency and accuracy are separate properties. The danger is not that the machine sounds artificial; it is that it sounds authoritative

Tokenisation also explains the arithmetic. A figure such as 4,285,193 arrives as fragments rather than a quantity, and the machine cannot count letters in a word because it never sees letters. Never accept a calculation it has not shown you.

Fluency and accuracy are separate properties. The danger is not that the machine sounds artificial; it is that it sounds authoritative.

A stack, not a religion

Which is better, ChatGPT or Claude? That is rather like asking whether Excel is better than PowerPoint. Better at what? They are broadly one skill wired to different things, and the wiring is the choice.

Perplexity searches first and answers second, with sources you can open; Copilot sees the documents, spreadsheets and mail inside your Microsoft tenant; Gemini does the same in Google Workspace. The question is never which is cleverest, but what a model is connected to — and whether your answer lives there.

Underneath, the ground has moved. Moonshot AI’s Kimi K3, released this month, is a 2.8-trillion-parameter open-weight model out of Beijing. Open weight is not open source — you receive the finished parameters, not the training data — but the direction of travel reverses: with a commercial service you send your data to the model; with open weights you bring the model to your data.

Different engines, one document

Here it stops being abstract. Subscribing to Copilot no longer buys you one model. Excel and PowerPoint already offer a choice of engine behind the same button — OpenAI’s GPT-5 models or Anthropic’s Claude — with Word joining shortly. Whether any of it is switched on is an administrator’s decision, not yours.

So route the work. Take a 200-page set of annual financial statements in a PDF. Use Copilot’s OpenAI models to lift it into Excel as structured figures rather than pictures of numbers. Then change engines inside the same workbook and give Claude the modelling: ratio trends, working capital movements, debt-service coverage, a sensitivity table on rates, three scenarios with assumptions written out.

Move to Word for the report, and here Copilot’s reach earns its fee: it draws in the workbook from OneDrive, the client correspondence from Outlook, the attachments on those emails and the summaries of the meetings where the variances were argued — context an analyst would have spent two days assembling.

A fluent answer is evidence that the model is working. It is not evidence that the answer is true

Then PowerPoint, for a board pack built from the approved report: 12 slides, one argument each, the awkward numbers left in. The machine compressed the labour. It did not supply the judgement.

At the point that matters, run it twice: ask a second model to attack the first’s conclusions. Microsoft has productised this — its Researcher agent offers Critique and Council modes, in which OpenAI and Anthropic models check each other’s work. When two capable systems disagree, that is not an inconvenience. It is the cheapest audit you will ever run.

A fluent answer is evidence that the model is working. It is not evidence that the answer is true.

Why the same question gives different answers

Which raises the obvious objection: if three models disagree, which is right? Possibly none. A language model does not retrieve a finished answer from memory. It constructs one in real time from learnt patterns, the context you provide and whatever tools it is allowed to use.

The chain runs from input to available context and data, to model, to inference and to output — and a weak result can originate anywhere along it. The request was vague; the document was missing; the training data was stale. Or a plausible prediction was mistaken for a verified fact.

The output is therefore never independent of the system that produced it. Change the prompt, the evidence, the model or the tool access and the answer changes, though the question never did.

Our task this week

Take one document you know thoroughly — a contract, a set of management accounts, a policy — and ask a question to which you already know the answer. Ask it properly: use only the attached document; give the clause, page or cell reference for every fact; and write “not stated in the source” wherever the document is silent.

Then put the identical question to a second model and compare. Where they agree, you have a working tool. Where they differ, you have found the place your judgment is still required.

• Mafinyani is senior partner in financial engineering & artificial intelligence at specialised finance, risk and applied technology firm Intellica Analytics.