Tom Daley’s legions of fans will for ever remember the Tokyo Olympics as the backdrop to his first gold medal, secured with Matty Lee with a flawless final dive in the men’s 10metre synchronised platform.

But the Tokyo Games, delayed by a year owing to the coronavirus pandemic, also marked Daley’s debut as an equally accomplished knitter and crocheter – a hitherto semi-hidden side to his life that continued through to his valedictory, silver medal-winning appearance in Paris this summer, and now into retirement.

No one was more surprised than the man himself by the reaction to the sight of Daley, seated on a blue plastic seat in the near-empty Tokyo Aquatics Centre, painstakingly creating what would become his personal tribute to the host city.

In a way, that white cardigan, embellished with the union flag, the Team GB logo and the Olympic rings, the kanji characters for “Tokyo” embroidered on the front, was symbolic of the extraordinary circumstances surrounding the Games.

Spectators were banned from all but a handful of venues, including the main stadium, while daily medal counts were accompanied by a running total of new Covid-19 infections in the Olympic village – the only neighbourhood in the city athletes were permitted to set foot in when they weren’t competing.

Knitting, Daley explained after the images of him mid-purl went viral, had been a distraction from the intense pressure of competition, and from the tedium of life in the pandemic-hit Games bubble.

Three years later, Daley is poised to revisit the city that cemented his association with yarn in the minds of the public, with his first knitted art exhibition, at Parco Museum Tokyo in the Japanese capital’s Shibuya district.

The event, that runs for 18 days from 8-25 November and with Daley in attendance on the opening day to give a live knitting workshop, will include personally selected pieces under the theme Made With Love – also the name given to his range of yarns and an Instagram account that has traced his second career in wool since it began a year before the Tokyo Olympics.

“I’d been knitting since March 2020, but no one really cared until they saw me do it at the Olympics,” Daley told the Guardian from his home in California, where he lives with his husband, the film-maker Dustin Lance Black, and their two young children.

“I think some people thought I didn’t care about the Olympics, that I was just knitting. Some people thought it was brave to be knitting in public, and some thought: ‘What the hell is he doing?’”

It’s reasonable to assume that the number of yarn threads in one of his medal pouches now vastly outnumbers the sceptics. While his famous cardigan was taking shape, Daley won a gold medal – only Britain’s second Olympic gold in the sport – in Tokyo, and three years later added to his haul of one gold and three bronzes with silver in Paris, again in the men’s 10m synchronised platform.

“I’m so excited about the exhibition,” Daley said, seated in front of shelves of colourful yarn. “It’s been a long time in the works now – over a year. I’ve been working on designs and ideas I wanted to portray. It’s the first time I’ve done anything like this and I feel like it’s really fitting to do it in Japan and Tokyo, which is where the fascination with my knitting began.”

The exhibition’s organisers have promised an immersive experience – a journey through a range of emotions, heightened by the use of light and music, and drawing on the now inseparable aesthetic and athletic components that make up Daley’s public persona. It will include his knitted works, photographs, interactive art installations, as well as personal videos featuring knitting tutorials.

“In our first meeting, what came through is that he’s doing this for all the right reasons,” Shai Ohayon, a Tokyo-based British curator who organised the exhibition, said. “He’s humble and thoughtful about what he’s doing.” Ohayon added that the exhibition would revolve around community and the calming effect imparted by a pair of knitting needles and a ball of wool.

Daley used knitting “as my way of being able to switch off from everything that was going on, being able to just focus on something that was right in front of me at that moment. That was quite powerful, so when I was photographed [making his cardigan] I didn’t really think about it because there was no one in the stands in Tokyo.

“The next day I woke up and had another million followers on Instagram, and I thought: ‘What on earth has happened?’ It was really nice to be able to break the stereotypes of who knitting is for and who should be able to knit … and also knitting in public wasn’t really a thing. It’s a great way to pass the time. You see people reading a book, so why not get out your needles and have something on the go?”

Visitors will be invited to knit their own simple pieces, guided by Daley in on-screen tutorials, and either take them home or stick them on a community board at the venue. “I want people to feel really immersed in my knitting brain, so that’s the journey I hope to take people on when they come to the exhibition.”

While he ponders a post-diving future that could include media work – as well as expanding his knitting and crocheting business – Daley says preparing for the exhibition has helped ease the transition from the pool to his new life as a knitter, a passion he shares with Kamala Harris’s stepdaughter, the fashion designer Ella Emhoff.

Since announcing his retirement at 30, days after the Paris Games, Daley revealed he hasn’t had time to climb on to a diving board. “I’m busy, but in a different way. It’s been great to be able to spend more time with the kids, but then I’ve also been travelling a lot.

“It feels very surreal to be done with diving, but I’ll stay involved. I’ll always watch it and hopefully I’ll be able to travel to watch competitions. It’s difficult to get access to a diving pool here, so I haven’t been in one since the Olympics. But I’d like to be able to get to the point where I dive for fun again.”

The Tokyo event will also be an opportunity for Daley to continue his campaign for the rights of LGBTQ+ people. Having once said the Olympics should never be awarded to countries that criminalise homosexuality, he had a rethink while filming the 2022 BBC programme Illegal to Be Me.

“The most important thing is that LGBTQ+ people can be visible in countries where the laws are against them,” he said. “The more powerful thing is to be able to go to a country that’s hosting a sporting event and just being you and succeeding and doing well.

“It’s such a powerful and strong message and sends a message of hope. I think all we can do as LGBTQ+ people is to be visible, to be out there and be ourselves. That in itself is a form of activism. And everyone can get involved in that.”

While Japan is home to a thriving LGBTQ+ community, its conservative government has resisted pressure to legalise same-sex marriage, despite polls showing a majority of Japanese support the change. The equality theme will run throughout the Tokyo exhibition, with the proceeds from sales of merchandise and a silent auction of Daley designs going to Marriage for All Japan, an organisation campaigning for same-sex marriage.

Daley, who is now taking sewing classes – he had just started on a pair of trousers when he spoke to the Guardian – hopes visitors will leave the Tokyo exhibition “inspired to pick up knitting needles, to try a craft like crocheting, and to just think about the process of slowing down and being mindful about situations. And maybe they’ll be inspired to try diving or another sport … something new and out of your comfort zone.

“You never know what it might turn into.”