Mandela had recently appointed Manuel to his cabinet as minister of finance, the first black man ever to hold this crucial post. Within a few years, Manuel was to achieve god-like status among the business community and South Africa's middle classes in general.
In 1996, however, this same strata of SA society - which encompassed the rugby fraternity - regarded him with intense distrust. Over a drink at the Vineyard Hotel, a few kilometres from Newlands Stadium, where the drama began, he recalled it for me.
"There was an unbelievable hoo-haa about the fact that I said I would support the All Blacks. Actually, it was a particular set of circumstances. It was the beginning of rugby union as a professional sport. The turning point was the 1995 World Cup. This was the first full season of professionalism. In that particular Test, a guy called Tromp was on the Springbok bench. He was an incredibly strong chap - a hooker - and they decided to bring him into the team."
Tromp had been convicted of manslaughter after beating one of his black farm workers so badly that the man died of his injuries. The racial and power dynamics of such a death in the context of the country's history made this crime particularly revolting.
"There was a huge row in our box. I happened to be in the Old Mutual box, and the row was actually about Tromp. The argument put forward by some who were there - who happened to be white - was that winning was the most important thing.
"I had a different take on this issue. National teams must engender national pride, and it can't be about winning at all costs. Hopefully, my sons, who were very young at the time, would aspire to wear the jersey, and you can't demean everything for the sake of winning. That was the debate, and that was why I took this strong stand.
"It was clear that this didn't matter to people. It wasn't an issue. It didn't matter to the chattering classes that this guy had beaten one of his black workers to death. If he could help win the game for us, that was neither here nor there.
"All this happened within a year of winning the World Cup, which in many respects was a great turning point for nation-building. In respect of rugby itself, it was also a turning point because, at the time of the 1995 World Cup, the only black player (in the SA side) was Chester Williams. This was not regarded as an issue, and we were just going to continue in the same vein. There was great recalcitrance on the part of the rugby administration when it came to effecting change.
"Of course, there were howls of derision when I took my stand, because if it's only about winning, then you support the team that plays the most attractive rugby. It's not about anything else, regardless of what happened at the World Cup.
"I suppose people who hold cabinet posts need to be very measured and can't wear their hearts on their sleeve, but I actually think that there was a voice that needed to be heard and a missed opportunity for debate. I include myself in that. I considered whether SA would have the appetite for its minister of finance to be involved in debates around sport, and I stepped aside."
Of course, Manuel was right. The battle to rid rugby - and, by extension, its supporters - of racism was in its early days.
Rian Oberholzer, then CEO of the South African Rugby Football Union, said of its selection of Tromp: "Sarfu is fully aware of the history of Henry Tromp, but the player has been selected for the squad and deserves his chance."
That Oberholzer and the coach, André Markgraaff, should have even considered fielding such an individual to represent the newly minted rainbow nation was extraordinary.
The following year, Markgraaff was forced to resign after being secretly tape-recorded referring to black administrators as "f***ing k*****s". The year after that, Oberholzer's father-in-law, Louis Luyt, took Nelson Mandela's government to court in an attempt to halt a commission of inquiry into rugby to investigate allegations of racism and financial mismanagement.
Andy Colquhoun, now the Springboks' strategic communications manager, wrote: "The fact that Mandela was forced to take to the witness box so outraged black sports administrators that they threatened to return rugby to international isolation. Luyt won the court case - although the result was later overturned on appeal - but lost the war. He was forced to resign, and quit rugby completely two years later ... Luyt's departure heralded a new era."
Manuel says that during this "dark period" of rugby's history, he distanced himself. "I'd go to games, primarily at Newlands. It's always a bit difficult. You don't want to be thumped by one of these chaps who has had too much Klipdrift and Coke. At the same time, you want to avoid the strictures that come from sitting in the box of a huge corporate. I was fortunate in that I had friends from small corporates and, for a while, a few of them shared a box and I could always slip in there. You were among mates and it didn't matter who said what."
A waiter comes up. He has recognised Manuel and wants to greet him. Manuel responds with warmth, smiling and shaking the man's hand.
The waiter leaves and Manuel continues. His three sons, he says, were growing up rugby-mad. Govan, his eldest, persuaded his dad to take him to Paris for the 2007 World Cup. They watched the semi-finals, but Govan had to return to university and Manuel went on to a meeting in the US, so they missed SA's triumph in the final. "We had lunch with the team, however, and a good rapport was built up."
In 2009, for the first Tri-Nations match against the All Blacks in Bloemfontein, the Springboks asked Manuel to hand out the jerseys.
"The message that I gave the players was: if you go back to that day in June of 1995, there is that one image of Francois and Madiba with the Webb Ellis Trophy, but the more important image was probably of black people dancing in the streets and claiming it as their victory. The ability of rugby to be more than just what happens on the field is phenomenal. I said the challenge in 1995 had been to bring into the rugby fold the black people who had been excluded. The challenge that we face now before this Test against the All Blacks is to bring in the white people who feel they no longer own rugby. As you walk onto the field, bear that in mind."
Touch, Pause, Engage! A Journey into the Heart of South African Rugby by Liz McGregor is published by Jonathan Ball (R175)