With Firefly, its new AI video generator, Adobe is stepping into the AI video editing world. Competing with OpenAI’s SORA, Meta’s Movie Gen, and Google’s Veo, Firefly aims to offer new tools for video editing using AI technology. Adobe has been teasing Firefly for months, and it’s finally ready for users to try in Premiere Pro and After Effects.
Let’s take a closer look at what Adobe’s Firefly can do.
Introducing Firefly Video Model
Firefly includes various features that allow users to generate and enhance video content in different ways. Here’s what you can do with Firefly right now:
Generative Extend
The Generative Extend tool is in beta for Premiere Pro. It allows users to extend video clips by up to two seconds at 720p or 1080p at 24 frames per second. This can help fix issues like a scene being too short or an unexpected movement without needing a reshoot. It also works on audio by extending ambient sounds, though it has limitations with spoken dialogue and music.
Text-to-Video
Text-to-Video lets users create short video clips using text prompts, with a maximum length of five seconds at 720p quality. It can generate different styles, such as realistic film, 3D animation, or stop-motion. Imagine typing, “cinematic drone shot flying over a red Martian landscape at sunrise,” and seeing that scene brought to life.
Adobe is not alone in the generative video race; OpenAI’s Sora and Meta’s Movie Gen have similar capabilities. However, Firefly puts control directly into users’ hands with customizable camera settings, angles, and motion effects — allowing you to create shots that are better tailored to your needs.
Here are a couple of examples:
Prompt: Cinematic closeup and detailed portrait of an elderly man in the middle of a street at night. the lighting is moody and dramatic. The color grade is blue shadows and orange highlights. the man has extremely realistic detailed skin texture and visible pores. movement is subtle and soft. the camera doesn't move. film grain. vintage anamorphic lens.
Or you can generate lights and texture like this,
Prompt: Filmic light leaks on black background, organic texture, realistic.
And you can add it to your shots with the screen blend mode.
Image-to-Video and Video-to-Video
Firefly can also generate Image-to-Video and Video-to-Video on the web. Image-to-Video allows users to generate video footage from a reference image. It can be used to create extra shots from photos or visualize a reshoot from a still frame. Currently, it is limited to clips up to five seconds long at 720p.
Video-to-Video lets users upload a video and change what’s happening in the video with the help of a text prompt. Similar to Text-to-Video and Image-to-Video, Video-to-Video is also limited to five seconds long at 720p.
This is before:
This is after using this prompt:
Prompt: A gloved astronaut hand enters the shot and unplugs one of the yellow cables. Cinematic.
How Adobe Compares to the Competition
Adobe’s tools are in beta with some limitations in the current versions. Generative Extend is capped at two-second extensions, while Text-to-Video and Image-to-Video clips are limited to five seconds with a maximum resolution of 720p. By comparison, OpenAI claims Sora can create videos up to a minute in length. This is a significant advantage over Adobe Firefly. However, unlike Firefly, SORA is not yet publicly available.
Meta’s Movie Gen can also produce longer HD videos with audio but is limited to 16 frames per second and is currently accessible only to Meta’s research teams.
Google’s Veo stands out for generating realistic videos of more than one minute in 1080p quality, though the exact maximum duration is not mentioned. However, it lacks the focus on content safety that Adobe offers. This means, Firefly ensures that the videos generated are legally safe to use by being trained on licensed or public domain content and uses Content Credentials to clearly show how the content was created.
What’s Next for Adobe?
Adobe is continuing to improve Firefly and has plans to add a “turbo mode” for faster video generation. Currently, it takes about 90 seconds to generate a video, but Adobe hopes to improve speed further. The company also introduced other AI features at Adobe MAX, showing its commitment to responsible AI development through the Content Authenticity Initiative.
Firefly’s tools are available in a limited public beta, and Adobe is inviting users to join the waitlist. Note that you need an Adobe Premiere Pro subscription to try it. The generative AI space is evolving rapidly, and Adobe’s addition of Firefly to its suite of creative tools provides new options for video editors.