Three minutes after half-time, Twickenham was the quietest it had been since they opened the gates earlier in the day. The All Blacks were coming on strong after the restart, making wave after wave of breaks down the left side, then back towards the middle of the pitch.

England were reeling, leaning into the ropes under the weight of blows, the right corner of their own try-line closing in fast behind them. Scott Barrett banged the ball on, through Jamie George, and the New Zealand scrum-half Cortez Ratima stooped to gather it up. He took a quick look, there were three attackers lining up outside him, and only two defenders ahead. The try was on.

Across the gainline, Marcus Smith was watching. In the hustle-bustle of England’s scramble defence, he hadn’t taken his eyes off the ball, and his mind was working all the while on what was going to happen next. He watched as Ratima stepped, once, twice, and he strode up into the line as Ratima shaped to let go of the ball. Smith met the pass midway through its flight, grabbed it and set off downfield.

Time can behave in strange ways in Test rugby, Smith is one of those rare athletes who can fit more into his minutes than the rest of us, and that split-second hesitation by Ratima was enough for him to break the game open. He seemed to be 10 metres away before anyone else had clocked him, and then 20, 30 metres by the time they caught him up. When the cover closed in he waited and waited and waited until the very last moment, then he finally released the pass to George Furbank, who had caught up to his inside shoulder. Furbank sped the ball on to Immanuel Feyi-Waboso, who ran on to score near the posts.

It ought to have been the match-winner and the crowning moment of one of Smith’s best games yet. After 35 Tests and five years of international rugby, he is maturing into a masterful fly-half. He kicked five penalties and a conversion, made England’s only try, produced a superb, snap, 50-22 kick and even turned in a try-saving tackle on Wallace Sititi which allowed Ben Curry to win a turnover.

It ought to have been the match-winner. But it wasn’t. Because when Smith came off the field, and the scrum-half Ben Spencer went with him, England lost the run of the game. They stopped trying to throw punches and started soaking them up instead.

But if you’re not attacking the All Blacks, then you’re losing to them, and their comeback started to feel grimly predictable. You can beat New Zealand, but you can never really defeat them. They just don’t stop coming. In 10 games across the past decade, England have lost three games by a single point and four more by a single score. And this was one of the most painful of the lot.

There were three ways England could have won this game in the last five minutes. One was when they were still five points up and holding out during a defensive set, but failed to stop Mark Tele’a scoring in the corner. Another was when they were two points behind, but a man up, and George Ford had a penalty shot at goal which bounced back off the upright. And the third, and last, was when an attacking scrum gave them that last chance to set up a drop goal, which Ford kicked wide of the posts. For all England’s endeavour, for all England’s effort, the hard truth is that they had all the chances they needed, and they didn’t have it in them to take them.

It’s cheap to speculate on the what-ifs, but you can’t but wonder what Smith might have done with five long minutes to work in. Barrett, the All Blacks captain, said afterwards that his side were pretty surprised by Steve Borthwick’s decision to take off both Smith and Spencer given that the two of them seemed to have such a tight grip on the game: “Must have been something tactical.”

What little attacking threat England had when Smith was on the pitch vanished altogether once he was off it. And while Spencer made mistakes in his first start for England, especially when he was caught on the ball a couple of times when he was too slow to react to the referee’s instructions to play it already, he knows how to manage a game.

Harry Randall is a fine, sniping, scrum-half, but he got himself into a terrible tangle as he tried to orchestrate England’s attack in those final moments. Borthwick’s decision to make the replacements smelled like a decision that had been made before play, rather than in response to it.

They are left with their wounded pride and more talk about what might have been.