How AI is helping to “reboot memories” during the time of war

[Aug. 14, 2020: Joseph Shavit]

The power of a colorized photo to reignite lost memories was eye-opening for Anju Niwata. Anju Niwata and Hidenori Watanave call the process “rebooting memories,” whereby colorized photographs revives memories that may otherwise be lost.

Now 18 years old, Niwata hopes the 75th anniversary of the end of World War II will bring her project to the attention of a Tokyo University professor to painstakingly colorize photos using artificial intelligence and thereby spark lost memories for the rapidly aging generation who experienced the war.

Three years ago, Niwata, then a student gave Tokuso Hamai a colorized photo as a present. When Tokuso Hamai, now 86, was first shown the colorized version of an old black-and-white photo of a picnic held under cherry tree blossoms sometime before World War II, long forgotten memories of family members, most of whom died in the atomic bombing of Hiroshima in 1945, came pouring out.

“Seeing Niwata share the colorized pictures with Hamai, and then watching him recall his old memories one after another, made it feel like the ice around his frozen memories was melting away,” said Hidenori Watanave, the professor who taught Niwata how to colorize monochrome pictures using AI.

Niwata and Watanave call their photo colorization project “Rebooting Memories,” and they published a book last month of the colorized versions of about 350 monochrome pictures taken before, during and after the war.

Niwata said that publishing the colorization book during the coronavirus outbreak has made her think about the pandemic's link to the war.

“Our everyday lives have been stolen away by the coronavirus in a flash, which I think resembles what happened in the war. That’s why I feel like now is an opportunity for people to imagine (wartime life) as their own experience,” she said.

Time, however, is running out; the average age of the atomic bomb survivors is about 83.

Watanave and Niwata use three different types of AI photo coloring software. The AI is useful in identifying the accurate colors of natural things, such as the sea, the sky and human skin, but it cannot accurately colorize human-made objects like roofs and clothes, Watanave said.

So Niwata and Watanave painstakingly finish the AI-colorized photos by hand to get more accurate colors based on historical documents, archives, advice from experts and most importantly the photo owners’ memories. Photos can take a few months each to finish.

The above 4-panel photo above demonstrates the digital colorization process developed by Anju Niwata and Hidenori Watanave, left, and original black and white image that Hiroshima resident Hisashi Takahashi and his parents, grandmother and younger brother pose for a photograph in a flower bed of dandelions.

The photo was taken around 1935. The original black and white photo is at top right. The second photo, bottom right, shows the colors after AI automatic colorization. The authors initially determined that the flowers in the foreground were white clovers, and adjusted colors of the flower bed accordingly in the third photo, bottom left. After interviewing the photo owner Takahashi, the flowers were found to be dandelions instead of white clovers, and was recolored yellow at top left.

Like these kind of stories? Get The Brighter Side of News' newsletter.

George Santayana wrote:

The promise of this technology is to reestablish links to the vivid memories the pictures hold. History is remembered and passed on to future generations with the accuracy needed to teach us not to repeat them.

The time it takes to painstakingly rebuild these photographs makes both AI and machine learning perfect tools to increase speed and efficiency.

Other use cases for this technology could be: