Meta's New Make-A-Video AI Can Generate Quick Movie Clips Based on Text Prompts

In July, Meta debuted its Make-a-Scene text-to-image creation AI, which, like Dall-E and Midjourney, uses machine learning algorithms (as well as large libraries of scraped online artwork) to produce imaginative renderings of textual prompts. Meta CEO Mark Zuckerberg unveiled Make-a-Video, a more animated version of Make-a-Scene.

Make-a-Video, as the name suggests, is “a new AI system that lets people turn text prompts into brief, high-quality video clips,” Zuckerberg said in a Meta blog on Thursday. Video functions similarly to Scene, relying on a combination of natural language processing and generative neural networks to translate non-visual instructions into visuals — it’s simply extracting material in a different format.

“Our intuition is simple: learn what the world looks like and how it is described from paired text-image data, and learn how the world moves from unsupervised video footage,” a Meta team stated in a research article published Thursday morning. As a result, the team was able to cut the time required to train the Video model and eliminate the need for paired text-video data, all while keeping “the vastness (diversity in aesthetic, fantastical depictions, etc.) of today’s image generation models.”

Make-a-Video, like the majority of Meta’s AI research, is being distributed as an open-source project. “We want to be thoughtful about how we build new generative AI systems like this,” Zuckerberg said. “We are openly sharing this generative AI research and results with the community for their feedback, and will continue to use our responsible AI framework to refine and evolve our approach to this emerging technology.”

Make-a-Video, like apparently every other generative AI that has been developed, has the potential to be abused. To avoid any potentially malicious pranks, the study team cleaned the Make-a-Video training dataset of any NSFW pictures as well as poisonous wording.