Gemini for Android gets two new features that allow users to query the current display of the smartphone. These tools, called “Ask this page” and “Ask this video”, take a screenshot and process the content of the page. Then, when prompted by the user, you can respond with the relevant information that the user is looking for. The video feature, which seems to be compatible with YouTube, can also provide video summaries, if subtitles are added to the video.
Gemini lets users ask questions about their display
Gadgets 360 staff found these features in Google app version 15.33.38.28.arm64 and later on some phones smart. These appear at the top of Gemini’s flight window when the user invokes the AI assistant. Those who enter this mode will see a special rectangular line with the text “Ask this screen” and a mirror icon. If Gemini is enabled while playing a video on YouTube, the text will be replaced with “Ask this video” and the YouTube icon.
Clicking on the “Ask this page” icon will take a short screenshot in Gemini, and then the user can ask a question about the page. The answers are based on what the AI can see on the screen, but users can also ask Gemini to access the internet to find out more information. One use is to request a short summary of the text.
To ask a follow-up question, users can simply click the microphone icon or the keyboard icon at the bottom of the floating window. It should be mentioned that the screenshot taken in the device will not be saved. Additionally, long images are not supported with this feature.
“Ask this video” only works for YouTube video with subtitles added. In our testing, we found that no other video platform supports this feature, despite its subtitle capability. But in a YouTube video, using this feature allows users to ask questions about the video. Responses are generated based on captions and the video itself is not analyzed.
