China re-rendered
The past as seen through the eyes of AI
![](jpg/dgspitzer-2.jpg)
This article was originally published on Neocha and is republished with permission.
How did the socialites of Shanghai dress a century ago? What did the Great Wall look like before a centuryโs worth of human and natural damage? What would one of the terracotta soldiers of Xiโan look like in real life?
New York-based artist Hu Wengu, better known by the online handle ofย DGSpitzer, has pondered long and hard on these questions. In his search for answers, heโs using AI and machine-learning software to restore archival footage of China, offering audiences a high-definition glimpse into the countryโs past.
Hu works full-time as a game developer, and his original impetus for learning AI was to learn how the technology can be incorporated into his game-design workflow. He quickly realized that it could be applied in nearly every area of game development, whether it be training in-game NPCs, generating vast maps, or even composing original music. โIt actually has interesting applications in a wide range of fields, even social and human sciences,โ Hu says. โSo this year I made a series of videos introducing people to how AI can compose music, paint, generate faces, and more.โ
As Huโs understanding of AI deepened, he began exploring how the technology could carry over into other domains. This interest eventually led him to AI-enhanced video. Since then, heโs leveraged the power of AI to create a stunning collection of color-restored videos showcasing Chinaโs past.
Three of Huโs AI-enhanced works include footage sourced from the University of South Carolina Moving Image Research Collections, which showcase a Beijing streetside barber, a Shanghai fashion show, and the construction of the Broadway Mansion Hotel. Hu discovered their footage archives by chance, and as he dug through the video library, he became more and more fascinated. โMIRC has recordings of China from the 1930s, which is extraordinary,โ he says. โIn the old Beijing barber video, there was a bald, charismatic guy who left a profound impression on me. So much so that, after I watched it, I got in touch with the university and asked if I could use their black-and-white videos in my AI project. They responded quickly and were happy to oblige. They even helped look into the exact dates of when the films were shot.โ
Huโs video restorations happen in three stages. The first involves adding in extra frames, then the footage is colorized, and finally, the resolution is upscaled. His arsenal of AI software, which includes DAIN, DeOldify, DeepRemaster, and PaddleGAN, all tackle these tasks independently. โFor example, the AI that adds missing frames, it analyzes the first frame and last frame to logically work out what should fit between them to smooth out the framerate,โ he explains. โAs for the colorizing AI, it almost like a human toddler. It examines large amounts of black-and-white photos, eventually learning that when a plant-shaped object appears, it should be green; when the sky and clouds appear, they should be blue and white; and when a human face appears, it should be flesh colored. The resolution upscaling AI works in a similar way, and by combining the output of these AI software, the footage is restored.โ
Despite the amount of work and time Hu has invested in the project, he confesses that heโs not exactly a history buff. Instead, he considers himself a sci-fi nerd. โWhen I watched black-and-white videos in the past, they felt like history lessons,โ he shrugs. โI would fast forward it to get through it quicker. They just didnโt catch my interest.โ
However, to restore a piece of footage, sped-up viewing was no longer an option. Watching at regular speeds, he was able to discover a ton of details that he overlooked on the initial watch, such as a young man staring at the cameraman from behind a tree, a group of carefree kids playing in the street, or a curious worker studying the camera as he passes by.
These little moments of connection fascinate Hu. โIt reminds me of the fact that even though weโre living centuries apart, weโre still under the same sky and walking the same earth,โ he says. โItโs like watching a century-old vlog.โ
Hu has also begun looking into restoring historical footage from other countries. Recently, heโs working with a German artist to restore a documentary shot in Dresden in 1936, even taking it a step further with the implementation of AR technology that will allow viewers to overlay the AI-enhanced scenes atop locations in real-life to see how the area has changed over the years.
Aside from video-related endeavors, heโs also begun using AI to colorize and upscale old portrait photography from China. Through his use of neural networks, Hu hopes for viewers to experience the simplicity of the past and immerse themselves in the lives of these people long goneโif only for a few short minutes. โIt can be a moment of respite when you allow people to look into the past,โ he says. โStepping into the shoes of these people on camera andย looking at history through the lens of the present can perhaps help us gain a greater understanding of culture and what it means to be human.โ
History can often seem dull, appearing as unrelatable lines of text tucked inside of course books, but Hu believes it doesnโt have to be. Through his AI-enhanced videos, Hu hopes that people who may have previously dismissed how interesting history can be are able to find renewed excitement for the subject, and in turn, develop a healthy appetite for knowledge in all forms and shapes.
Like this story? Follow Neocha on Facebookย andย Instagram.
Youtube:ย ~/DGSpitzer
Weibo:ย ~/ๅคง่ฐทSpitzer
Bilibili:ย ~/ๅคง่ฐท็ๆธธๆๅไฝๅฐๅฑ
Contributor:ย Chen Yuan