May 7, 2024
HONG KONG – Nine-year-old Morris Kwok Yat-chun couldn’t hold back his tears when he unexpectedly found himself engaging with a lifelike AI replica of his late beloved brother, Yat-lai.
“I missed him a lot,” Kwok said on Sunday during a press conference which introduced a project utilizing AI and virtual reality technologies to reconnect people with lost relatives.
In 2020, when Kwok was five, his seven-month-old brother, Yat-lai, was in hospital suffering from a rare autoimmune disease and struck with pneumonia, Kwok’s mother said during the conference.
After exhausting various medical interventions, including injections and medications, doctors informed Kwok’s family that the only hope of saving Yat-lai was through a cord blood transplant from his brother. Cord blood is the blood that remains in the placenta and umbilical cord after the birth of a baby.
Years earlier, when Yat-chun was born, under medical advice, the family preserved his cord blood in the hospital as it could potentially be crucial for other people’s treatment in the future.
Incredibly, years later, the cord blood was a potential cure for his brother.
However, on the very day the cord blood treatment plan was proposed, Yat-lai passed away, which became an enduring regret for his family.
Last year, Kwok’s family enrolled Yat-chun in a voluntary program operated by the non-profit organization Love Our Kids Foundation, which aims to grant children’s wishes.
When volunteers asked Kwok what he wanted, he said he missed his brother a lot and wanted to see him again.
Yat-chun’s mother, a 41-year-old former piano teacher, said that she wished to witness Yat-lai experiencing the joyous milestone of learning to walk, the next natural progression in his life had he survived.
The volunteer team thought AI technology may be a good way to realize Kwok’s wish, and reached out to local start-ups and technology firms.
After months of efforts by technicians and volunteers, Yat-lai reappeared via the virtual world, taking shaky steps and playing with toys.
“Witnessing his presence in the virtual world felt like having him by my side once again”, Yat-chun said.
“I want to play hide and seek, basketball, and swim with him within the virtue world.”
His mother said she never expected the reunion could be made possible by technology.
She was deeply touched at the moment Yat-chun shed tears upon seeing Yat-lai. She had not anticipated he still had such a profound emotional response after his life had restored normalization for a long time.
The recreation allowed Yat-chun to release his love and longing for his late brother, his mother said.
In addition to the efforts in creating the portrayal of Yat-lai, the voluntary team also used AI to reconstruct a deceased mother from another family.
The volunteers initiated the process by asking the three daughters to write a letter to their late mother.
Using the letter as a basis, the volunteers created a video in which the reconstructed AI version of the mother replied to her children. This involved reconstructing her appearance and replicating her voice, with the volunteers crafting a script that captured the essence of her personality, expressing her feelings to her daughters.
Lee Tan, an associate dean of education at the Chinese University of Hong Kong’s engineering faculty, was in charge of reconstructing the mother’s voice, and Tom Tong Kwun-wah, co-founder of a technology startup Softmind, created the mother’s facial expressions.
Tong said that he took on this project because it was meaningful. He said he came across some challenges in accurately synchronizing mouth movements with Cantonese pronunciation, as most AI software and materials are predominantly designed for English speakers.
Tong had to record additional videos featuring people speaking Cantonese, capturing their unique mouth movements to achieve satisfactory results.
Lawrence Cheung Cheong-ming, the head of customer and product support for the virtual reality division at smartphone brand HTC, constructed the virtual reality living room where the mother was located for the filming.
Cheung said that his main task was compiling the materials he had gathered from Lee and Tong and integrating them into the virtual space station, which was relatively straightforward and uncomplicated.
Lee said that he had to produce a fluent minutes-long speech based on only a few seconds of voice recordings of the deceased mother.
From a technical standpoint, Lee said that the process itself was not difficult to execute. The true complexity of the task lay in the emotional significance it held for the family, Lee said.
He understood that while technological improvements could be made to the audio recordings, it did not guarantee an enhanced emotional experience for the family. In his opinion, in this case, improvements in technology should be in the service of the family’s feelings.
Lee said that he joined this project as he was deeply moved by the daughters’ grieving for their mother. Also, the family had given their consent for a limited reproduction of the mother’s voice.
Lee’s start-up, Vocofy AI, has developed a text-to-voice model that preserves the original voices of people who have lost the ability to speak.
Lee has been approached by various individuals seeking to use AI to generate content. However, he usually declines private requests to recreate the voices of deceased individuals.
He believes that any AI-generated content must adhere to ethical guidelines, and technicians must understand the purpose and implications of the resulting AI-generated content.
Compared to technical competency, whether the technology is well used is more important, Lee said.