Fourth-tier Newport, one of several Welsh clubs who play in English domestic football, put themselves in sight of one of the all-time great upsets against eight-time FA Cup winners Tottenham.
It was all thanks to Padraig Amond’s first-half goal.
But eight minutes from time, in-form England striker Kane equalised.
This was when, after Spurs had won a corner kick, he tapped in from close range following an excellent flick-on from South Korean Son Heung-Min.
A 1-1 scoreline was how it finished as League Two Newport survived four minutes of injury-time to hold out for a lucrative replay at Wembley, Spurs’ temporary home.
Newport, a massive 72 places behind a Spurs side who are fifth in the Premier League, sparked delirious scenes at Rodney Parade when Amond gave them a 38th-minute lead.
Ben Tozer’s long throw into the box caused consternation in Spurs’ defence and Robbie Willmott’s clip across to the back post was nodded in by Amond.
It seemed Rodney Parade, where Newport Rugby Club beat giants New Zealand and South Africa in the 1960s, was about to play host to another great sporting upset.
But with County in sight of the last 16, boyhood Spurs fan Kane spared his side from a damaging defeat.
Earlier, West Ham manager David Moyes labelled Arthur Masuaku “despicable” after the midfielder was sent off for spitting in a 2-0 defeat by Wigan.
Premier League West Ham were reduced to 10 men four minutes into the second half when Masuaku saw red for spitting at third-tier Wigan’s Nick Powell.
They were then already 1-0 down to the first of two goals from Will Grigg,
“It was despicable,” said Moyes. “If you do that, then you’re going to get the punishment in any walk of life.
“I just asked him, did he spit and he said, ‘Yes’.”
Wigan have slumped down English football’s ladder since winning the FA Cup in 2013, but the League One leaders rekindled memories of that run to Wembley.
West Ham were without several injured first-choice players and an upset always looked on the cards.
It took Northern Ireland striker Grigg just seven minutes to score his fifth goal in as many Cup ties this season with a header beyond England goalkeeper Joe Hart.
Grigg made it 2-0 from the penalty kick spot in the 62nd minute after Reece Burke’s handball in the box.
“From start to finish we were the best side,” Grigg told reporters.
Elsewhere, struggling Premier League sides Swansea and Huddersfield played out 1-1 draws with lower-division Notts County and Birmingham City respectively.
The deadlock led to unwanted replay matches for teams whose priority is remaining in the top flight.
Premier League Brighton were also on the verge of a replay until Glenn Murray’s 90th-minute winner sunk Championship side Middlesbrough 1-0.
Saturday’s early kick-off saw Fousseni Diabate score twice on his Leicester debut as the Premier League club won 5-1 at Peterborough.
Nigeria’s Kelechi Iheanacho also scored twice for the Foxes, with Wilfred Ndidi making it 5-1 in stoppage-time.
Nottingham Forest’s FA Cup run ended just one round after they knocked out holders Arsenal courtesy of a 2-1 defeat by Championship rivals Hull.
Southampton beat Watford 1-0 as Jack Stephens marked his 24th birthday with a goal to ensure Javi Gracia’s reign as Hornets manager started with a defeat.
Chelsea face Newcastle in another all top-flight clash on Sunday, while Premier League leaders Manchester City continue their bid for an unprecedented ‘quadruple’ away to second-tier Cardiff.
Manchester United had made it through to the fourth round on Friday, with new signing Alexis Sanchez impressing on debut in a 4-0 win away to lowly Yeovil Town.