Google has rolled out a Feature Drop, introducing substantial improvements driven by its advanced AI model, Gemini. As Google’s most versatile and potent AI model, Gemini is crafted for seamless operation across a range of devices, spanning from data centers to smartphones. With its three different sizes—Ultra, Pro, and Nano—Gemini is specifically engineered to cater to a variety of computing requirements.
Gemini Nano Lands on Pixel 8 Pro
A standout feature of the Feature Drop is the introduction of Gemini Nano, the most efficient model crafted for on-device tasks. This optimized model is now operational on the Pixel 8 Pro, leveraging the prowess of Google Tensor G3 to deliver an unparalleled AI experience. Gemini Nano is engineered to run directly on mobile silicon, facilitating on-device tasks without compromising data security or experiencing latency issues.
Capabilities of Gemini Nano on Pixel 8 Pro:
Summarize in Recorder: Gemini Nano powers the Summarize feature in the Recorder app, delivering concise summaries of recorded conversations even without a network connection.
Smart Reply in Gboard: A developer preview of Smart Reply in Gboard, driven by Gemini Nano, is now available in the US for applications like WhatsApp, providing high-quality response suggestions with conversational awareness.
AI-Powered Multimedia Experiences:
Cutting-edge Video: Pixel 8 Pro’s Video Boost, powered by Google Tensor G3, improves uploaded videos by adjusting elements like color, lighting, stabilization, and graininess through computational photography models.
Additionally, Video Boost enables Night Sight video on Pixel 8 Pro, utilizing AI for noise reduction in videos captured during nighttime or low-light scenarios, resulting in enhanced detail and vibrant colors.
Timelapse Videos: Night Sight in Timelapse allows users to capture vibrant and detailed timelapse videos in low-light settings.
Enhanced Portrait Light: An improved Portrait light in Google Photos leverages new AI models, introducing Balance Light to eliminate harsh shadows and enhance portrait photos.
Upgraded Photo Unblur: Photo Unblur now excels at sharpening images of pets, ensuring clear and captivating photos.
Productivity Tools and Features:
Dual Screen Preview: Pixel Fold introduces Dual Screen Preview, facilitating photography with inner and outer screen collaboration.
High-Quality Video Calls: Pixel devices (Pixel 6 or later) can now be connected to laptops or desktops via USB for visually crisp and vibrant video calls.
Clean Feature for Document Scans: Pixel’s Clean feature enhances document scans by removing smudges, stains, and creases.
Passkey Support in Password Manager: Google Password Manager now identifies accounts supporting passkeys, simplifying the addition of passkeys with a few taps.
Repair Mode: Pixel introduces Repair Mode, ensuring the protection and preservation of personal data during device repairs.
Smart Replies in Call Screen: Google Assistant on Pixel Fold, Pixel 6, and newer phones suggest contextual replies in Call Screen, enhancing response options.
Pixel Watch Integration: Pixel Watch enables easy unlocking of Pixel phones when nearby, providing a seamless user experience.
Expanded Features and Customization:
Expanded Language Support for Recorder: The Recorder app now transcribes in 28 new languages.
Direct My Call and Hold for Me: Expanded to business numbers without a toll-free prefix and available in the UK.
More Customization Options:
Weather and Clock Integration: The World Clock tab in the Clock app now includes weather conditions and daily forecasts.
Pixel Watch Enhancements: First-gen Pixel Watch now supports sync of Do Not Disturb and Bedtime Modes with Pixel Phones, along with new watch face designs.
Android AICore
Google has introduced Android AICore alongside Gemini, a new on-device foundation model. This system service in Android 14 provides seamless access to Gemini Nano, managing models, runtimes, safety features, and more, simplifying AI integration into apps.
AICore, designed for privacy, aligns with Android’s Private Compute Core, ensuring network isolation through open-source APIs for transparency and auditability. It facilitates Low Rank Adaptation (LoRA) fine-tuning with Gemini Nano, utilizing ML hardware in various devices. Developers can join the Early Access Program to explore Gemini’s AI capabilities.