How rewriting history with a deepfake moon landing video could help spot misinformation

The 50^th anniversary of the Apollo 11 mission to the moon was coming up. During a brainstorming session at Lippmann House, journalists and artists were throwing around ideas for potential collaborations that brought together the arts, journalism, and tech to innovate in storytelling.

Bill Safire’s “In Event of Moon Disaster” came up. It was a contingency speech that was written for Richard Nixon, the sitting president at the time, in case of “mishaps” on the Apollo 11 mission. The speech is a beautiful eulogy acknowledging the astronauts’ bravery and accomplishments in pursuit of knowledge.

We had also been discussing developments in artificial intelligence that day, specifically deepfakes. We thought we’d bring the two together to create an alternative history where the moon landing had gone terribly wrong. The project would be a film and a physical art installation taking you back to 1969 to watch the moon landing broadcast on TV interrupted by a news broadcast with Nixon delivering the “Moon Disaster” speech.

The piece was commissioned by Mozilla as part of their Creative Media Awards and fellow artist Halsey Burgund and I set out to make this Nixon deepfake. Usually we painstakingly construct our projects ourselves, but this time we farmed out work to Canny AI, an Israeli company that does Video Dialogue Replacement, and Respeecher, a Ukrainian startup specializing in synthetic voice production. We used video of a Nixon speech about Vietnam with the mouth motion changed and some of Nixon’s audio replaced.

For the Nixon voice, we needed to produce a large set of training data for the AI to use to generate the contingency speech. Respeecher told us we’d need to find an actor willing to listen to thousands of tiny clips of Nixon and repeat each one with the same intonation and timing. The actor did this until we had two to three hours of recordings.

We also recorded 20 full takes of the contingency speech in audio and video to be used both for Canny to map the mouth movements and to act as the basis for the delivery of the synthetic version of the speech to be produced by Respeecher.

Making the deepfake was a roller coaster and it took three months in total. (The video won the International Documentary Film Festival Amsterdam’s 2019 special jury award for creative technology in digital storytelling.)

Lots of people have asked us why our project isn’t misinformation itself. Aren’t we just adding to the many conspiracy theories about the moon landing? Our answer is emphatically no. Information presented as not true in an educational context is not misinformation. Experiencing a powerful use of new technologies in a transparent way has the potential to be memorable and make people more wary about what they see.

If someone who experiences our deepfake later recalls the believability of our piece and as a result uses more caution interpreting a video in their Facebook feed, we will have been successful.