comscore
19 Feb, 2024 | Monday
Trending : LaptopsAppsAutomobileHow ToOPPO India

Google's new AI tool Lumiere can create videos from still photos

The Google Lumiere technology figures out where things are in a video (space) and how they simultaneously move and change (time).

Edited By: Shubham Verma

Published: Jan 28, 2024, 01:07 PM IST

Google Lumiere is the company's new text-to-video generator.
Google Lumiere is the company's new text-to-video generator.

Google has introduced a new video generation AI model called Lumiere that uses a new diffusion model called Space-Time-U-Net, or STUNet. Lumiere creates 5-second videos in one process instead of putting smaller still frames together. This technology figures out where things are in a video (space) and how they simultaneously move and change (time). This method, according to a report by Ars Technica, allows the tool to create a video in a single process instead of using the method of stitching smaller still frames together.

“We introduce Lumiere — a text-to-video diffusion model designed for synthesising videos that portray realistic, diverse and coherent motion — a pivotal challenge in video synthesis,” said Google researchers in a paper. “We introduce a Space-Time U-Net architecture that generates the entire temporal duration of the video at once, through a single pass in the model,” they wrote.

The design facilitates a wide range of content creation tasks and video editing applications, including image-to-video, video inpainting, and stylised generation. Lumiere can perform text-to-video generation, convert still images into videos, generate videos in specific styles using a reference image, apply consistent video editing using text-based prompts and create cinemagraphs by animating specific regions of an image.

The Google researchers said that the AI model outputs five-second-long 1024×1024 pixel videos, which they describe as “low-resolution.” Lumiere also generates 80 frames compared to 25 frames from Stable Video Diffusion. The most popular text-to-video generator Runway began offering more videos that looked more real, but even it struggled to smooth movements in a video.

“There is a risk of misuse for creating fake or harmful content with our technology, and we believe that it is crucial to develop and apply tools for detecting biases and malicious use cases to ensure a safe and fair use,” said the paper authors.

— Written with inputs from IANS

Get latest Tech and Auto news from Techlusive on our WhatsApp Channel, Facebook, X (Twitter), Instagram and YouTube.

Author Name | Shubham Verma

STAY UPDATED WITH OUR NEWSLETTER

Select Language