How About a Google Lens Update that Combines Words and Images?

Google Lens, its visual search tool, is getting additional AI-powered language features. Users will be able to refine their searches using text when the update happens. So, if you take a photo of a paisley shirt using Google Lens to discover matching products online, you may add the command “socks with this pattern” to specify the garments you’re searching for.

In addition, Google is introducing a new “Lens mode” option in its iOS Google app, which will enable users to search using any picture that surfaces while browsing the web. This will be accessible “soon,” but only in the United States. Google is also releasing Google Lens for desktop within the Chrome browser, allowing users to search for visual search results by selecting any picture or video while exploring the web without leaving their tab. This will be accessible worldwide “soon.”

These updates are part of Google’s latest attempt to enhance its search tools through the use of AI language understanding. The Lens updates are driven by MUM, a machine learning model that Google introduced at I/O earlier this year. Google is also bringing new AI-powered tools to its web and mobile searches, in addition to these new capabilities.

The upgrades to Google Lens demonstrate that the firm is still interested in this tool, which has long shown potential but seemed to appeal more like a gimmick. Machine learning methods have made simple object and image recognition tools very straightforward to deploy, but as today’s upgrades illustrate, they require a little refinement on the part of users to be fully functioning.

Google hopes that these Lens upgrades will make its world-scanning AI a more helpful tool. It uses the example of someone attempting to repair their bike but not knowing the name of the mechanism on the rear wheel. They take a photo using Lens, enter the search term “how to fix this,” and Google returns results identifying the mechanism as a “derailleur.”

These demos as the examples provided by Google might seem easy and helpful. However, the new Lens has to go get tested well to determine if AI language understanding is truly making visual search more than just a parlor trick.