Adobe, Firefly major update... ushering in the era of AI video editing

Source: TokenPost Original Title: Adobe, Firefly Major Update… Opens Era of AI Video Editing Original Link: Adobe(ADBE) has rolled out a significant update to its generative AI(AI) platform ‘Firefly(Firefly)’, greatly enhancing its video production capabilities. This overhaul shifts the focus from purely generation to directing, offering both professional video creators and general users increased freedom and precision in the creative process.

The core of this update is the introduction of the ‘prompt-to-edit(prompt-to-edit)’ tool, which allows individual scenes to be edited separately during video production. Users can input commands like “remove the left character” or “zoom in the center of the screen” to make precise modifications to specific elements without regenerating the entire scene. Adobe explained that this feature is useful for removing unnecessary objects or fixing scene inconsistencies within videos. It is based on Runway AI(Runway AI)'s ‘Aleph(Aleph)’ video model, enabling users to fine-tune visual expressions as if directing a scene.

The accuracy of filming techniques has also been improved. The ‘camera motion reference workflow(camera motion reference workflow)’ feature allows users to upload clips with desired camera movements, and Firefly will generate scene composition and camera motion simultaneously. This reduces the need to repeatedly adjust camera movements, helping to shorten production time.

Upscaling features for video quality enhancement are also noteworthy. The new tool ‘Topaz Astra(Topaz Astra)’ within Firefly Boards(Firefly Boards) converts low-resolution clips to 1080p or 4K quality and can be used for restoring older videos. Adobe emphasized that this feature can provide expanded expressive capabilities for creators focused on content distribution.

Additionally, a new model ‘FLUX.2(FLUX.2)’, specialized for image generation, is introduced in Firefly Boards. Developed by Black Forest Labs(Black Forest Labs), this model supports highly detailed, photo-realistic images with advanced text rendering and can utilize up to four reference images simultaneously.

A browser-based video editor that allows direct editing on the web has also been launched in beta. The Firefly Video Editor is designed to handle full video creation, editing, and the addition of music and visual elements within a lightweight multi-track environment. Users can continue working anytime without installing software, as long as they have a device.

Furthermore, Adobe is running a limited-time promotion offering unlimited video and image generation features to Firefly users. This applies to Firefly Pro, Premium, 7,000 credits plan, and 50,000 credits plan subscribers, and includes not only the main models but also various third-party models such as ‘Nano BananaNano Banana’, ‘GPT-Image’, and FLUX.2FLUX.2.

This Firefly update is significant as it marks the first attempt to evolve the role of generative AI in video production from simple image creation to a more director-like process. The technological advancements aimed at increasing the precision of AI-produced videos are expected to bring another turning point in content creation methods within the industry.