Google's AI begins to make sense of images
Google continues to improve its AI-based search experience. The company's search-focused chatbot, which it calls AI Mode, can now interact not only with text but also with images. Thanks to this new feature, users can ask questions about images and receive detailed answers. Google has rolled out this update to millions of users on Android and iOS platforms via the Google app.
12punto
The update is based on a combination of a special version of the Gemini AI model and Google Lens technology. Users can ask the AI questions about content by taking a photo or uploading an image from their device. The system not only recognizes objects in the image but also evaluates the context of these objects. As a result, users not only get information about the objects in the image but also gain in-depth knowledge about the relationships surrounding them and the overall meaning of the image.
Google Search product manager Robby Stein describes this update as a more advanced stage of previous visual search efforts. The AI now reads elements such as shape, color, material, and layout to understand the entire scene. Each object in the image is treated not just as an independent entity, but as a whole in relation to other elements. This feature allows AI Mode to provide users with more natural and meaningful responses.
When analyzing images, the AI uses an approach called 'fan-out'. With this method, many parallel queries are generated for each object in the image. This allows for a layered and contextually consistent response regarding all the details of the image. For example, when examining a photo of a bookshelf, the system can recognize the books, suggest similar content, and provide reading recommendations.
This innovation is not limited to visual recognition; it also interacts with users to answer questions. When a user requests more information about an image, the system can generate additional queries and offer new suggestions. This feature transforms AI Mode from a static analysis tool into a dynamic assistant. Every image becomes a starting point for the system to access information.
Google first introduced AI Mode to Google One AI Premium subscribers with limited access last month. However, with this update, the feature is no longer exclusive to paid subscribers. The company announced that it is providing access to millions of users as part of the Labs program. This means that more people will be able to benefit from the advanced visual search experience.