Google has introduced agentic video understanding, a new approach that enables Gemini to dynamically search, scan, and inspect video content for analysis.
Agentic video understanding
Unlike static video processing, where a model ingests video at a fixed frames-per-second rate, agentic video understanding combines Gemini’s core reasoning with its native video tools to dynamically search, scan, and inspect target video segments.
Static processing uses 1 FPS by default, although the frame rate can be adjusted through the API. With agentic video understanding, Gemini can analyze visual frames, audio, and transcripts, retrieving the moments and signals needed for a task instead of processing the entire video uniformly.
Across standard video analysis benchmarks, Gemini models with agentic video understanding deliver:
- Up to 66% lower analysis costs
- Up to 88% lower token consumption
- Up to 7% higher accuracy
The efficiency gains are particularly relevant for long-form video, including 10-minute how-to guides, 90-minute lectures, and multi-hour recordings. Static processing can require developers to choose between higher token costs and techniques that may drop critical details.
The gains span all three supported models. Google says Gemini 3.7 Flash with agentic understanding offers the best overall quality and the best combination of quality and cost efficiency among the tested models, placing it at the accuracy-to-cost Pareto frontier for video understanding.
The approach is similar to agentic vision, which combines code execution with Gemini models’ native image understanding. For video, the system uses Gemini’s native video tools for analysis.
How it works
Agentic video understanding allows Gemini to take an active, goal-directed role in determining what to watch, at what speed, and through which modality. It can determine:
- What to watch
- How quickly to examine it
- Which modality to use, including frames, audio, or transcripts
Gemini then fetches only the moments and signals needed for the task. Developers could previously build this process manually, while agentic video understanding allows Gemini to handle it through an agentic loop.
Through this loop, Gemini invokes an internal tool to load the relevant portion of the video file, reducing the development overhead involved in building the workflow manually.
Capabilities and use cases
The feature supports several video analysis tasks, including:
- Sub-second moment retrieval: Pinpoints split-second state changes and tight cut boundaries that can be missed at 1 FPS, making precise automated video editing possible.
- Long-form needle-in-a-haystack search: Answers complex queries across multi-hour videos without consuming millions of tokens.
- Anomaly detection: Resamples interesting time windows at a higher FPS to inspect rapid motion and subtle visual artifacts.
- Counting actions and objects: Tracks repeated physical movements and distinct objects over time.
Availability and getting started
Agentic video understanding is now available through the Gemini API for video uploads and YouTube videos in Google AI Studio and the Gemini Enterprise Agent Platform.
It is available across:
- Gemini 3.7 Flash
- Gemini 3.6 Flash
- Gemini 3.5 Flash-Lite
Other details include:
- Uses standard Gemini API token pricing
- No additional feature fee
- Developers can enable it by setting processing to “agentic” in the API configuration
- Google provides a developer guide with more information on using the feature
Google will roll out the feature to all users in the Gemini app across Flash and Flash-Lite models soon. In the coming months, agentic video understanding will also power YouTube’s Ask YouTube feature on the video watch page, using Gemini to provide answers grounded in the visuals of videos.