Google Lens and the future of Visual Search

The way forward for AI and retail is Visual Search

Machine learning is taking online search to new levels, and it’s not just voice search that’s on the rise. While many have been focused on the marketing potential of capturing conversational queries barked at Alexa over breakfast, big brands have been quietly developing a stronger competitor in interactive SEO: visual search.

For decades, we’ve searched for information and products online using a text search bar. The introduction of voice search has since made waves in local SEO, with users searching for opening times, directions, weather reports and other of-the-minute information – but it has left many online retailers feeling stumped. Not every trend fits with every business model, and for businesses who rely heavily on visuals to drive conversions, the opportunity brought by voice search has often felt limited.

Download our FREE Resource – 10 common website customer experience mistakes

The most common user experience mistakes, with strategy recommendations, examples and recommended resources.

Access the

Humans process visuals 60,000 times faster than text, and according to research by Kissmetrics 93% of consumers consider visuals to be the key deciding factor in a purchasing decision – which is why over the last few years, ecommerce sites have been bulking up photo galleries and adding 360° videos in an effort to increase conversions. Now, thanks to innovations like Google Lens and Pinterest’s Shop the Look, the payoff for that work seems set to increase.

What is visual search?

When you’ve seen an item or an image of an item that’s caught your eye but are unsure of the brand, the model, the name of that style, that’s where visual search comes in.

Unlike an image search, where an ordinary text search pulls possible relevant images using structured data, visual search is the process of fulfilling searches by using machine learning to analyze components within a submitted photo, and finding results that replicate or relate to those visual cues. Think of the way that Facebook now recognizes the faces of friends you’ve tagged in past images – it’s this genre of technology that is now being used to develop wider visual search.

Just as you can look across a room and see a variety of objects, read labels and observe features, so can visual search AI. Google Lens, for example, can see an image of a landmark and offer you details on its history or where to buy tickets to go inside. It can look at your photo of a book and offer recent reviews for that title, places to buy it online and alternative books by the same author.

In the same manner, Pinterest’s visual search functions allow you to select different parts of a photo – a pair of shoes, a lamp, a paint colour – and find similar products to purchase. It can also offer outfits or room décor suggestions that include other items to pair your selection with.

The current state of visual search

Right now, Google Lens is only available on Pixel phones, boosted by high-end camera technology but inaccessible to your average iPhone user. However, the less advanced Bing Visual Search is available to anyone, allowing you to select part of an image found in traditional image search and find images related to that one detail elsewhere on the web. Look up modern kitchen designs and you can select a stool or vase within the image, before browsing hundreds of similar options on sale.

eBay announced in 2017 that they were gearing up to launch Find It On eBay, adding image search functionality to their app and mobile website and enabling users to snap a photo and instantly find anything on eBay that looked like it. They’ve also cashed in outside of their own domains, recently announcing a collaboration with Mashable in the US where users can shop for eBay products that bear resemblance to clothes and items in Mashable images – all without leaving Mashable’s site.

The potential for ecommerce sites to capitalize on this style of search is huge. Keyword-generated image search can be frustrating when faced with vast inventories and products that may have been incorrectly or poorly tagged, or described using terms we haven’t thought of. Users who have found themselves typing in every variation of a colour name or style description in the hope of finding what they’re looking for are more than ready for a simple and effective visual-match search.

We already see shoppable images and videos on Snapchat and Instagram, tailored ads and sponsored posts popping up and begging us to click and purchase. But advanced visual search goes further in that not only can users search for items found in any image online and get relevant results in seconds, they can also search for things they’ve seen in the real world.

People can find cheaper alternatives to items they’ve seen in a shop window or print magazine, or identify a variety of plant they’d like to add to their garden. As well as offering increased convenience from a user perspective, retailers who optimize their sites effectively should find an increase in relevant traffic that’s ready to convert.

Optimizing for visual search

Though visual search will have uses in a range of industries, it seems fair to assume that this intuitive tactic will be retail-dominated. Sites should still ensure that their images are optimized using structural data and other traditional SEO tactics, but going forward there’s also a need for imagery to be clutter-free and easy for visual search tools to process while the technology is still developing. And of course, there needs to be plenty of imagery to digest in the first place.

How to optimize for visual search:

• Offer a range of clear images for each product
• Optimize image titles with target keywords
• Submit image sitemaps
• Set up image badges
• Optimize image sizes and file types
• Run structured data tests

Generally speaking, the more steps there are between the start of the purchase funnel and the checkout, the higher the cart abandonment rate. The Baymard Institute say  that on average, nearly 70% of online carts are abandoned before checkout, and a lengthy process to get to the payment screen can cause around a third of users to ditch a site and shop elsewhere.

Enabling users to find your product via one image search, thanks to clear product photography that downloads quickly on any device, cuts out the usual steps of searching by item type, size and colour, browsing through pages for the best fit and then, eventually, reaching the checkout stage.

In essence, traditional image optimisation is still half the battle – the other is ensuring that you’ve put enough time and thought into your product photography in the first place.

Visual search is set to dramatically improve the online shopping experience in the coming years, and with retail ecommerce sales in the UK predicted to reach a value of almost £94billion in 2018, there couldn’t be a better time to cash in.