Olympic medallists, Japanese sports officials and local volunteers have lined up to lambast octogenarian Tokyo Games chief Yoshiro Mori over his sexist remarks, describing anger and disappointment at his comments.

Mori sparked outrage on Wednesday for saying women speak too much in meetings, then dug himself into a deeper hole a day later when he admitted “I don’t speak to women much” during a hastily organised press conference called to make his apology.

The 83-year-old former prime minister’s comments and refusal to resign as president of the coronavirus-postponed Tokyo 2020 Olympics drew widespread scorn, including from top athletes.

“In a nutshell, it was very disappointing,” said swimmer Satomi Suzuki, who won two silver medals and one bronze at the 2012 London Olympics, after a race on Thursday.

“Honestly, it made me angry,” she added, according to the Asahi Shimbun newspaper.

Mori’s comments also upset athletes outside Japan, with former Canadian ice hockey player Hayley Wickenheiser – a six-time Olympian and a member of the International Olympic Committee Athletes’ Commission – among those taking aim.

“Definitely going to corner this guy at the breakfast buffet,” she tweeted. “See ya in Tokyo!! #oldboysclub”

Volunteers hoping to play a part this summer are also up in arms and have complained to the Tokyo city government over sinking morale, according to local media reports.

The Mainichi Shimbun newspaper quoted one 54-year-old volunteer as saying he was now considering quitting, explaining “it just feels like he’s passing the buck and not taking responsibility.”

Mori began his press conference on Thursday by apologising and saying he wished to retract his remarks, but then became defensive when questioned, insisting he had heard complaints that women speak at length.

His comments are the latest headache for organisers already battling public disquiet about the Games, with polls showing that over 80 percent of Japanese oppose holding the event this summer.

Mori, who has a history of controversial remarks, made waves earlier this week by insisting the Games would go ahead this summer “however the coronavirus (pandemic) evolves.”

He is also a former president of the Japan Rugby Football Union, and JRFU board member Yuko Inazawa told the Asahi Shimbun on Friday that she believed his comments were aimed at her.

“When Mr. Mori was president of JRFU, I was the only female board member,” she was quoted as saying.

“I wasn’t aware he felt my opinions were a nuisance. I’m surprised by his latest comments.”