Imagine pointing your phone’s camera at the ingredients in your kitchen and asking, “What can I make with this?” Instantly, the AI identifies the ingredients and suggests recipes tailored to them. That’s exactly the kind of smart assistance Google’s Project Astra promises. This is powered by Google’s latest AI model, Gemini 2.0 Flash, and here’s everything you need to know about this.
Table of Contents
What is Project Astra?
At its core, Google’s new AI tool Project Astra is a multimodal AI assistant. Multimodal simply means it can handle different types of inputs such as text, images, videos, and audio simultaneously.
For example, you can point the camera at a shelf full of books and ask Astra, “Which one has the highest reviews?” Astra can identify the titles, check online ratings, and suggest the best option.
Not just the camera, it can also analyze what’s on your phone’s screen. For instance, if you’re browsing a webpage and come across a term or product you don’t recognize, you can ask Astra to explain it or find more information.
Astra can also perform actions on your phone, like ordering online, setting up an alarm, etc. More than that, it can also remember things. So if you point the camera at a drawing and ask, “Who drew this?” Astra will tap its memory and might reply – “It’s drawn by your daughter.”
Think of it like having a supercharged version of Google Assistant that listens and understands the world the way you do.
Real-World Use Cases
Here are some ways you can use Astra
- Suppose you’re trying to eat healthier. Astra can analyze your grocery list (via a photo or text) and recommend substitutes to meet your dietary goals.
- Suppose you’ve asked Astra to remember your apartment’s door code. Weeks later, when you’re at the door, you can simply ask, “What’s the door code again?” and it will remind you instantly.
- Snap a picture of a clothing tag, and Astra will decode the laundry symbols for you. It can even guide you on which washing machine settings to use.
- Point your camera at a plant, and Astra can tell you if it’s suitable for your garden. Astra can recollect info about your location from memory and can say, “This plant thrives in partial shade and moist soil, perfect for your New York garden.”
Google is also working on AI agents for games where it can suggest in-game strategies like where to deploy troops or which weapon to upgrade, etc. Also, they are experimenting with prototype glasses that can see, listen, and respond using Astra.
How to Access Astra Right Now
Project Astra is still in its testing phase and it’s not available to the public right now. However, you can get a preview of Project Astra’s capabilities through a feature called “Stream Realtime” available on Google AI Studio. Here’s how you can access it:
- Open aistudio.google.com/live in your mobile or desktop browser.
- Sign in with your Google account.
- Tap or click on Show Gemini and grant the necessary permissions to access your camera and microphone.
- Use your phone’s camera to show your surroundings and talk to Gemini 2.0 in real-time. On the laptop, along with the camera, you can also share your laptop screen.
Remember this is just the preview phase and the tool may not support features like multi-language support and memory as of now.
Also Read:
- 8 New AI Features in Google Chrome You Should Know About
- Top 9 AI Features in OxygenOS 15 You Should Be Using
- Microsoft Finally Launches Recall AI with Improved Privacy Controls
- OpenAI Canvas Unveils New Era of AI-Assisted Writing and Coding
Other Projects Powered by Gemini 2.0
In addition to Project Astra, Google has unveiled other AI initiatives powered by the Gemini 2.0 model. Project Mariner is designed to perform tasks in the Chrome browser, like navigating pages and gathering data, making it ideal for enhancing productivity. Jules, on the other hand, acts as a coding assistant, helping developers with GitHub workflows and programming tasks.
