Labour leader Keir Starmer has been forced to withdraw comments in which he called Boris Johnson a “coward not a leader” following a rebuke from the Speaker.

During a fiery and ill-tempered session of prime minister’s questions, Starmer called on Johnson to apologise for his handling of the Owen Paterson scandal, in which the government sought to overturn punishment for the former Cabinet minister and paid lobbyist.

The Labour leader pointed out that while the business secretary, Kwasi Kwarteng, and the leader of the House, Jacob Rees-Mogg, had both apologised for their role in the affair, Johnson was yet to do the same.

“Across the country and belatedly across this house, there is now agreement that Owen Paterson broke the rules and that the government should not have tried to let him off the hook,” Starmer said.

“Many members opposite have apologised. The business secretary has apologised for his part. The leader of the House has apologised for his part, but they were following the prime minister’s lead.

“So will he do the decent thing and just say sorry for trying to give the green light to corruption?”

Johnson replied that it was “certainly a mistake” to conflate reform of the standards system with Paterson’s case before moving on to challenge Starmer’s former paid work at the law firm Mishcon de Reya.

Starmer hit back: “That’s not an apology. Everybody else has apologised for him, but he won’t apologise for himself.”

He continued: “A coward not a leader. Weeks defending corruption. Yesterday a screeching last-minute U-turn to avoid defeat on Labour’s plan to ban MPs from dodgy second contracts.

“But waving one white flag won’t be enough to restore trust.”

Starmer’s uncharacteristically strong language was seized on by Tory MP Michael Fabricant, who used a point of order to urge the Labour leader to withdraw his comment.

Speaker Lindsay Hoyle noted: “Coward is not what is used in this House.”

Starmer replied: “I withdraw it, but he’s no leader.”

During the session the Speaker repeatedly made clear his anger at the mood and rough language used in the House, especially in the aftermath of the death of Tory MP David Amess and the conversation it sparked about language in politics.

Hoyle also clashed with Johnson over his attempts to question Starmer’s links to Mishcon de Reya. Starmer turned down a second job with the law firm in the summer of 2017 following accusations it conflicted with his role as shadow Brexit secretary.

Hoyle told the PM: “I don’t want to fall out about it, I’ve made it very clear – it is prime minister’s questions, it’s not for the Opposition to answer your questions.

“Whether we like it or not those are the rules of the game that we’re all into and we play by the rules, don’t we? And we respect this House, so let’s respect the House.”

Despite the reprimand, Johnson attempted to ask again about the issue in a later exchange, to which the Speaker said: “Prime minister, sit down!

“I’m not going to be challenged, you may be the prime minister of this country but in this House I’m in charge.”

Johnson later accused Starmer of “Mish-conduct”, a play on words that prompted outcry from the Labour benches.

At the end of the session, Hoyle lamented the conduct of MPs and said the House had not done “any good” today.

“I’ll be quite honest, I think it’s been ill-tempered, I think it shows the public that this House has not learnt from the other week, I need this House to gain respect but it starts by individuals showing respect for each other,” he said.