Enhanced Capabilities of Apple's Visual Intelligence Unveiled in iOS 26 - Series of Upgrades Explained
In an exciting leap forward for iPhone users, Apple's latest iOS 26 update introduces a host of new features for Visual Intelligence, significantly enhancing the way users interact with content on their device screens. One of the key changes is the extension of Visual Intelligence technology to screenshots, a feature that was previously only available through the camera.
With the new on-screen awareness, users can capture a screenshot and then use Visual Intelligence directly within the screenshot interface. This allows for object identification, information extraction, and content interaction, all within the screenshot itself.
Highlighting specific parts of the screenshot image enables a focused search on that object or element, similar to Android's "Circle to Search" feature. This precision makes inquiries more targeted without the need for manual typing of search terms.
Integration with ChatGPT further enriches the user experience, allowing users to ask questions about the content on their screen, fostering a conversational AI experience to learn more or get explanations about the objects or text captured.
Visual Intelligence can also help users shop and discover new items by searching Google, Etsy, or other supported apps based on the highlighted or identified items in the screenshot. Additionally, event detection and calendar integration can automatically extract event details from screenshots, such as dates, times, and locations, and suggest adding them to the user’s calendar with prefilled key details.
On-device processing ensures user privacy and faster response times, as the analysis and interaction with screenshots take place on the device itself without needing to send data externally.
To use Visual Intelligence on screenshots, users take a screenshot, exit the default Markup interface, and then access Visual Intelligence options to begin interacting with the screenshot content. After taking a screenshot, a checkmark button and new Visual Intelligence commands (Ask, Add to Calendar, and Image Search) will appear on the screen.
These new features are available on iPhone models that support Apple Intelligence, and the iOS 26 developer beta is currently available, with the public beta expected to arrive this month. Users encountering issues can provide feedback to Apple using the thumbs up/thumbs down icons and a feedback screen.
The transformative capabilities of Visual Intelligence in iOS 26 promise to make screenshots more actionable and interactive, empowering iPhone users to shop, plan, learn, and navigate tasks more efficiently, all powered by deeper integration of Apple Intelligence and ChatGPT within iOS 26.
Users can leverage Visual Intelligence directly within the screenshot interface on their devices, enabling them to identify objects, extract information, and interact with content without the need for manual typing. This new feature is part of Apple's iOS 26 update and allows iPhone users to ask questions about the content on their screen, shop, plan events, and navigate tasks more efficiently, all powered by integration of Apple Intelligence and ChatGPT.
