Ecommerce Strategy: AI-Powered Search and Discovery

Michael Krigsman: We're discussing personalization for ecommerce and retail with Sean Mullaney, Chief Technology Officer of Algolia.

Sean Mullaney: We power search and browse and recommendation experiences across 17,000 websites all over the Internet. We're actually the second biggest search engine in the world behind Google. We power over 1.75 trillion requests every single year, so very exciting powering a lot of the foundation of the Internet.

Michael Krigsman: I have to say that, at CXOTalk, we're actually one of your customers. We use Algolia on CXOTalk.com. 

Sean Mullaney: It's great to talk to one of our customers. And, yeah, it's great to hear that we can help power your experience as well.

Ecommerce and the paradox of choice

Michael Krigsman: You're focused on ecommerce and retail, so why don't you give us just the high-level overview of that?

Sean Mullaney: We serve customers across a whole range of industries, but ecommerce is obviously one of the ones that we're best known for. The reason is because search is so important as part of the whole discovery journey in ecommerce.

People love ecommerce stores, and one of the reasons is because they get such an incredibly wide selection of products that they can shop for. But it's also hugely overwhelming for a shopper. 

It would be like walking into a physical store and it's the size of, like, a huge stadium. You would just have a million items to choose from, but it's very overwhelming. 

They call this the paradox of choice. The more choice you provide a customer, often the less satisfied they are or the more overwhelmed they are, and their experience can actually deteriorate with more choices.

E-commerce brings that huge opportunity, but it also means that we have to have new tools and technologies to be able to help customers and guide them through this experience.

Impact of AI and machine learning on ecommerce

Michael Krigsman: Sean, how do advancements in AI and machine learning (such as ChatGPT) affect this search landscape as you've been describing?

Sean Mullaney: E-commerce has been around 20, 30 years now, but the experience to a shopper still feels very much like it'd come out of the 1990s during the dot-com period. A lot of ecommerce sites are really just a product database or a product catalog that have a user experience on it. 

You still have to talk to the website in the way that you would talk to a computer. You have to type in specific words. You have to click on different filters to kind of narrow down the selection. This is going to change dramatically now. 

A lot of these language models that are powering ChatGPT have exploded in size and sophistication over the last few years. They're not only able to really understand humans in a way in which humans can use natural language, it understands the intent and the concepts behind what they're looking for. 

Improving user experience: How AI personalizes shopping

Michael Krigsman: Sean, can you elaborate on how AI helps make this shopping experience better for the user?

Sean Mullaney: You don't need to be able to talk to the computer anymore. The computer can actually understand you.

What we're seeing is people are using far more expressive language now. The number of words that they're typing into the search box is becoming significantly larger. It's doubled in size in the last couple of years.

But also, people are expecting experiences that are far more sophisticated and personalized. As I enter in searches or I click on filters or I view products, it expects the experience to adapt to me, to really understand what I'm looking for, to remember me when I turn up to the site.

Technology background: The role of large language models in ecommerce search

Michael Krigsman: Sean, you've described how natural language search just makes it easier for end users. You're Chief Technology Officer of Algolia. Can you give us a glimpse behind the scenes of how this is possible?

Sean Mullaney: There's been a real quantum leap in what we're able to do over the last year or two, as ChatGPT has really demonstrated to the world that you can interface with users using natural language both as the inputs to ask questions but also as an output to be able to answer their questions.

We're fundamentally changing the way in which we search and retrieve information with these large language models and these new breakthroughs. Instead of using the actual word, we're now actually taking words and we're turning them into something called a vector.

The vector captures the concept and the nuance behind either the word or the phrase or the sentence or your question. It means that we can then go and search through all of the products or all of the Web pages for other vectors. 

We turned everything in the index into a vector. Then it becomes this kind of exercise where we try to find similar vectors (just like you use words and try to match them with similar words in webpages). It means that you can do really, really interesting things. 

To give you a really powerful example, let's say you go to a website and you actually want to shop for a specific brand. Let's say I want a North Face jacket. 

If the site doesn't sell that brand, it's able to understand the concept behind the brand, so it understands North Face sells outdoor jackets, and it can surface other outdoor jackets that are like North Face. We can actually find products specifically designed to solve those problems even if the words that you're using when you describe the problem don't match the words in the product. 

What is vector search?

Michael Krigsman: How is this different from traditional search (whether on ecommerce sites or just broadly)?

Sean Mullaney: Traditional search is literally just taking the literal words that you type in [laughter] and trying to find products that use those literal words. A good example is if you wanted to search for chocolate milk or milk chocolate. These are two terms that have exactly the same words but very different categories to search for.

It can be very, very confusing sometimes for these search engines to be able to disambiguate the actual words. But when we can translate them into very specific concepts and really understand the intent behind them, then we're going to get far better, more accurate results. 

I have to say some of our early customers have just seen incredible increases in the amount of conversions and the amount of things that shoppers are able to buy because they're able to find it.

Vector vs. traditional keyword search

Michael Krigsman: Sean, you mentioned vector search. It's a topic that has been around for a long time, yet it's not widely discussed. Why is that, and what are you doing with this? 

Sean Mullaney: We've known that vectors are a better way to represent concepts than words. We've known vectors work (from a scientific perspective), and we can get great results in the lab. But it turns out that they're really hard to scale. 

I mentioned we do about 1.75 trillion search requests a year across 17,000 different websites. To be able to apply vectors on every single query is really, really challenging.

Vectors are very, very computationally heavy. They take up a lot of memory both in the server but also when you're training and storing them. They're actually pretty slow and expensive to roll out. 

The breakthrough that we've recently had is we've figured out how to compress vectors, so we can actually compress them into a ten times smaller format, keeping the same kind of relevancy, and we can do it at extremely high speeds. We can take every single query across those 1.75 trillion, and we can apply a vector search to them. 

This is the big breakthrough, this hashing technology that we've built for the real-world production environments when you need high speed, you need a reasonable cost, and you need really big scale. 

Michael Krigsman: This is what makes it practical for you to use vectors in this highly real-time shopping environment.

Sean Mullaney: Every single query for our customers, we're able to use both vector and keyword in a single kind of API call. By using both of these strategies, we actually get a lot more information back. We can use the extra information, like how many keywords did it match but also, how did the vectors match (to do a much better ranking). 

I think that's pretty unique. No one else in the industry is able to do this at the size and scale and speed that we're doing it at because we're using this kind of technique called hashing.

Michael Krigsman: The technology is enabling a simpler, easier user experience that's, at the same time, more effective.

Sean Mullaney: What we've seen is about 70% of all of the search queries are actually far more complicated, one-off type queries where people are asking for very specific things. They have a question. 

That kind of 70% of queries are going very unanswered at the moment because they're very difficult to answer with just matching words. This kind of new AI vector search is really monetizing and helping solve the customers' needs in that 70% of a longtail case, and it actually translates into a lot more sales. 

Using personalization to drive higher customer lifetime value

Michael Krigsman: Sean, where does personalization fit into this landscape?

Sean Mullaney: Creating a loyal customer is one of the most important parts about creating a great business because you pay to acquire the customer the first time. If they either don't convert that first time because they don't find something or they end up not coming back for a second or third shopping journey, then the economics of acquiring the customer become pretty unattractive.

How do I create an experience for a shopper where I recognize that they're a customer who has been to the store before? How do we make it feel so that when a customer comes to my store that they're going to get a superior experience the second and third time versus going to a competitor's store?

That's all about personalization. Once I've seen what a customer prefers (either through the searches that they make on the website or the products that they click on, the products that they buy, or the products that they click on and don't buy), you can observe the entire behavior of the customer as they're shopping, and you can learn from it.

You can do that in two ways. The first way is when they actually appear for the first time. You can really hang on every single click, every single word that they type in and category that they look at.

In real-time, even for a shopper who has never been to the store before, you can start to adapt. This is called real-time personalization and it's kind of like a cold start problem because you really don't know anything about the customer.

I can quickly find out if there are brands that they are clicking on more or if there are price points that they are gravitating towards. Then I can start to show and put more off these types of products around brands or prices or categories in front of them as they click and discover around the site.

The second really important thing is (once they've actually bought from you) the next time they come back. You acknowledge that they are a customer that you've seen before, and you're able to offer them, again, a similar personalized experience.

We've seen that these types of personalization algorithms and personalized experiences can drive substantial increases in conversions on the second and third visit, as well as this kind of real-time when they're shopping the first time. I think it's a very important part of the whole customer lifecycle from acquisition, first shopping, second, and third in creating a long-term, loyal customer.

Michael Krigsman: You're talking about two kinds of personalization. One is the cold start, as you described, when a visitor first shows up at your site. But then remembering that visitor when they come back the second time, the third time, and creating an even more tailored and personalized customer experience for them. What does this do to the overall customer lifetime value? 

Sean Mullaney: If you think of a lot of people when they first do their online experience and move online, they start to focus really on that very first experience of, "I put some advertising out. It costs me X. Shoppers turned up to the site, and they bought Y. Here's my return on investment."

But as you start to build a more loyal following and you start to become a little bit more sophisticated in the way you're thinking about your online business, you start to look at the return over a much longer lifetime with the customer. You take the whole lifetime value, and then you set that against the acquisition cost. What it means is if you create loyal customers, you can spend a higher dollar to acquire a customer.

This is really how to create a scalable customer acquisition business model online is trying to get your cost per customer acquisition and your customer lifetime value, and the ratio of these two numbers. You want to get that very high, so high customer lifetime value, low customer acquisition.

Michael Krigsman: How is this different from the traditional concept of customer journeys and looking at the total lifetime value of a customer?

Sean Mullaney: Typically, people will look in session, so they'll only look at the specific shopping session the customer is in. They'll attribute, basically, all of the value against the acquisition cost for the customer. But as you start to create much more AI-powered experiences and personalization, you can start to account for a much larger amount of value that's being created (when that customer was acquired) over more and more sessions. 

Using AI-powered search to improve ecommerce customer acquisition and raise customer lifetime value

Michael Krigsman: The technology enables this broader horizon and more accurate understanding of the customer behavior so that, in effect, you can amortize your initial customer acquisition cost over this longer, broader time horizon. 

Sean Mullaney: A great ecommerce store is one where you spend potentially even more money acquiring customers because those customers are going to spend over a series of months and years with you. They're going to become loyal and generate long-term profitability on an individual customer perspective.

Think of it like a P&L for a specific customer where your profits are the long-term value that they're going to generate through loyalty and the initial outlay. The costs are how much you could spend to acquire them.

Michael Krigsman: You've touched on this but what is the impact on the customer or the benefit that the customer receives to encourage them to come back and, therefore, provide that loyalty to the retailer?

Sean Mullaney: From a shopper's perspective, the benefit is really clear, which is that they're able to find better products, and they're able to do it in a much shorter time period. 

It's as simple as when I turn up to the site. Do I even feel like this is a shop that is kind of tailored towards my needs and to my tastes?

For shoppers, they're able to find the things they want, but they also have a sense, a feeling of being welcomed and a feeling that the place that they're shopping is the type of place that they want to continue doing business with because it reflects their taste, their style, their values.

Michael Krigsman: I think the benefit of this type of lifetime customer value analysis is obviously crucial for retailers. But it leads to the next question, which is, how do we implement this?

Sean Mullaney: You need to have a partner who can help guide you through the process. There are companies like Algolia and other people who have solved these problems and who have really focused on making the integration and the data collection pieces that are needed to power these AI algorithms really simple.

With a few lines of code, you're able to integrate with Algolia, unlock our search API. Working with vendors and working with folks who have dedicated large amounts of resources, time, and expertise to solving these problems tends to be the first starting point and often the best solution in the long run. 

Best practices for implementing personalization in ecommerce

Michael Krigsman: What are some best practices for implementing personalization across all touchpoints as customers interact with your site?

Sean Mullaney: The more touchpoints and the more data that you're able to capture about how your users are using your product the better. All of these algorithms really need to be able to understand clicks, views, conversions, but also the product data that the customers are looking at and having something that can indicate which customer is which between sessions. So, if they can log into your website, that's the best. But if you can add cookies to your website so you can track them between visits, that's good as well.

Personalization and the retail buying funnel in ecommerce

Michael Krigsman: What about the buying funnel where customers are on their journey, in their relationship to you as a retailer?

Sean Mullaney: Customers will turn up on the homepage, for example. You'll have very, very little intent about what they want in the shopping journey. 

At the high level of the funnel where you're not sure what they're looking for, you want to showcase a breadth to them of things that they can find in the store. And if they're a returning customer, you want to have a strong personalization bias on this homepage so that they can see things that are at their taste, style, or price points. As they go and search, and they start giving you more information about their intent, you really want to focus on the relevance and on the accuracy of the results you're showing them because they're giving you a very specific request. 

Then once they land on a product detail page, for example, you have an even more specific piece of information. They're interested in this product. Again, you want to become even more tightly focused on your recommendations.

You maybe might not want to use as much personalization when you're on a product landing page because, actually, the shopper has told you specifically something that they're looking for. And so, as a customer goes through the journey, you've got to think about how much intent you really have and either dial up personalization when you don't have as my intent or dial up relevance, accuracy when you have high intent.

Personalization and real-time data in ecommerce

Michael Krigsman: Sean, what is the role of real-time data in this process?

Sean Mullaney: You've got to think that a large number of customers coming to your website, it may be the first time that you've seen them. It's their first experience.

You want that first experience to be great. So, as soon as they click on a single item or they type in a search request, you want that data to flow back into your systems and adapt your algorithms in real time. 

You have to have systems that are extremely fast at both ingesting and getting the data in but also retraining and adapting the algorithm. I think real-time data is very important for that first experience. 

But also, what you sometimes find is customers come back and they shop a slightly different way to their taste, so you also want to be able to adapt in real-time, even for customers that you know a lot about.

Michael Krigsman: Then you use this data to make the decision around what to show next to that user and, of course, it's happening essentially instantaneously from the user perspective.

Sean Mullaney: Instantaneous. We're talking about milliseconds here that matter in the shopping experience. The faster the experience is, the more likely customers are going to continue. We've seen 100, 200 milliseconds worth of delay actually reduces the conversion rate and customers end up leaving the website, so speed is really, really important in the ecommerce world.

Michael Krigsman: Are there common implementation challenges that retailers face when they're trying to implement this kind of personalization?

Sean Mullaney: One of the hard things is making sure that you are capturing this type of one-to-one data around this is the user that's logged in, this is the cookie ID that they accepted, this is the session ID for their shopping session, and making that information available across every click, every product view, and every single time they come to the site so that the algorithms can do their work. 

The algorithms can only do so much. They need to be able to identify users. That's really one of the bigger areas where ecommerce companies need to make sure they've implemented Google Analytics really well (or whatever their analytics package is).

Michael Krigsman: What's the solution to this very common problem?

Sean Mullaney: Using off-the-shelf analytics products, but really focusing on doing a comprehensive job in implementing them. Also making sure that when you choose a vendor that you really take the extra time when you do the integration to send all the events, send all the clicks and conversions with the right fields and everything.

Ethical considerations in hyper-personalization

Michael Krigsman: How do you address the ethical or privacy considerations that people think of when we talk about this kind of hyper-personalization?

Sean Mullaney: One of the things that we have to understand first is that a lot of customers and a lot of shoppers are very happy to make the tradeoff of getting a better experience by allowing the merchant or the site that they're shopping on to be able to collect this type of data. 

When you think about it that way, you want to make it transparent to them what you're doing and you want to allow them to make that tradeoff. For customers who don't feel comfortable with that, you want to enable them to have a very clear tradeoff as well and provide a great default experience.

I think it's just important around transparency, but also telling customers what the value of it is so they know ahead of time when they click the accept cookies button or decide to log in and create an account, for example, that by doing so, they're really going to get a far better experience throughout the journey.

Measurement, KPIs, and A/B testing in ecommerce personalization

Michael Krigsman: Sean, let's talk about measurement and KPIs. What are the best mechanisms or approaches to measuring the results of these kinds of personalization efforts?

Sean Mullaney: We use A/B testing where we will make a change to the website, like we will update our algorithm, and we will send some of the customers down the A channel, which is the old experience, and some customers down the B channel, which is the new experience.

Using statistics, we can find out whether or not the customers who are getting the new product or new experience convert at a higher rate or spend more money per purchase or have a higher revenue in total. We can figure out the statistical significance of this so that when we actually decide to roll out the change, we're confident that it's actually going to improve the experience for customers.

It's exciting. It's a bit like scientific experimentation.

Michael Krigsman: In this experimentation, how much is managed by the retail shop owner versus how much is managed by Algolia behind the scenes?

Sean Mullaney: We have A/B testing built into everything that we do on the platform because we think it's just one of the highest velocity ways of improving your experience. It's definitely built into Algolia. We can run, manage all of the A/B experiments for you and show you the data so you feel confidence in making changes. 

Michael Krigsman: Are there specific metrics that reflect customer revenue but also customer loyalty and customer lifetime value?

Sean Mullaney: There are a lot of really important metrics that you need to keep track of when you're running an online business. It really depends on where in the buying funnel I think you are.

For example, when a customer first arrives on the site, it's important to look at the bounce rate. How many clicks did they get into the product before they're able to experience it?

Whereas if someone has made it all the way to a landing page, for example, you probably want to be more focused on the conversion rate and whether they're actually going to check out and buy something. Then after they've actually bought something, you really want to be looking at that lifetime value number. How often do they come back to the website and visit again? 

One of the things that I think is extremely important, particularly for search, is the position at which someone has clicked on a result when given a set of search results. If you have, let's say, ten rows worth of search results, the results that appear on the very first row will always get clicked on at a far higher rate than the ones that appear on the tenth row.

I really think that making sure that the products that you're getting into that first row or second row are extremely relevant. We see it makes a big difference.

Michael Krigsman: All of these very granular sets of reference data become the building blocks or the components of understanding the consumer, and then you can respond.

Sean Mullaney: You need to understand the products really well that you're selling and the options that you have to present. Secondly, you have to understand the shopper who turns up (through personalization, through profile building, through looking at all their past interactions). Then thirdly, you need to understand, in real-time, the things that they're telling you that they're looking for, the intent.

Understanding customer behavior and shopping modes in ecommerce

Michael Krigsman: How do you manage two different kinds of shoppers? You have folks who show up and they leisurely want to browse your catalog. Then you have other people who just, you know, "I don't have any time and I just need to buy this product. And if you don't have this product right now, I'm going away. And if you do have it, I'm going to buy."

Sean Mullaney: There are a couple of shopping modes. The first one, which obviously you need to be incredibly good at capturing a high-intent shopper who has come to buy a specific product. You need to do that with incredible search results and really great ranking of those results.

Then you have a second set of shoppers who really enjoy the experience of coming to a site and being inspired and discovering what this site has to offer. These are shoppers where you really want to create a browse experience which is far more interactive and far more inspirational.

I'll tell you. At the moment, most sites have a browse experience which is something along the lines of, "Okay, a customer clicks on the dress category. We have 30,000 dresses. Give them 100 pages worth of dresses and order that by the most popular." 

It's just entirely overwhelming. Often, the most popular products are some of the least inspirational. 

Future of search and discovery in ecommerce

Michael Krigsman: Sean, where is search going over the next few years?

Sean Mullaney: One of the big things that's going to happen to search, we talked about how vectors and large language models are really transforming the way that we search. We're not just matching keywords anymore. We're actually understanding customers, understanding the concepts and coming up with much better matching algorithms.

I think one other major change is going to be the way that we have a more conversational approach to shopping. When you think about the offline shopping experience, often people go into a physical retail store because they want to get some assistance, and they want to talk to someone who has some expert knowledge. 

That used to be an experience that wasn't very good online. It's very static. It's like, "Here. Just read the information. You figure it out for yourself." But with the power of ChatGPT and these large language models, we're going to be able to have much more expert personal assistant-like conversations at scale.

Advice for ecommerce retailers

Michael Krigsman: Sean, what should ecommerce retailers do now to take advantage of all of these capabilities you've been describing?

Sean Mullaney: I think there are three main things that an ecommerce operator should be thinking about. The first is the vendors they work with. They should be working with either their existing vendor today to take advantage of some of the AI capabilities that are coming or to select or add vendors who have these AI capabilities.

The second thing is that I think they really need to focus on their own analytics, so really capturing all of the data correctly about their customers (whether that's clicks and conversion data, but also making sure they capture the cookie IDs, who is logged in, et cetera) so that you can personalize the experience. 

Then thirdly, I think they should be thinking about the metrics that matter to them as a business. They can understand how the AI algorithms are actually creating a more sustainable, profitable, and long-term loyal customer base.

Michael Krigsman: We've been talking with Sean Mullaney, Chief Technology Officer of Algolia. For more information, check out www.algolia.com.