Finding our way with Sitecore Search

What is Sitecore Search, exactly?

Sitecore Search aims to deliver relevant and personalized content to the right audience at the right moment. It's a fully SaaS-based solution that includes the following components:

• Customer Engagement Console (CEC) - Your one-stop shop for search analytics, tests, configuration, and more.
• Search & recommendations service - An AI-based REST API for performing search queries and receiving results.
• Event service - A REST API for collecting visitor interactions which are key to optimizing the search experience.

Implementation strategy

With alignment between our use case and what Sitecore Search has to offer, we set out to revamp the search experience on Sitecore.com in September 2022. The project kicked-off with research and planning before we dove into implementation. Taking a phased approach helped us ramp up the team's knowledge gradually and learn the solution as we progressed.

Configuring the crawler

Our journey began with configuring the Sitecore Search crawler. Some of the key considerations around crawling included:

• Firewall
  Sitecore.com is protected by a Web Application Firewall (WAF) that includes bot protection measures. Some adjustments to the WAF were required in order for the crawler to access our site.
• Crawl rate/load
  We carefully considered the rate at which the crawler visits the website. Running a site-wide crawl against our pre-production environment at an aggressive speed gave us a baseline for how long a full crawl takes.
• Crawl tactic & frequency
  After considering many options, we configured the crawler to scan our XML sitemaps on a nightly basis as opposed to crawling an API or a real-time push-based approach.
• Multilingual
  Sitecore.com supports eight languages and it was important for us to be able to provide the same great search experience across locales.
• Multi-site/source
  It was important that we index not just Sitecore.com, but neighboring website properties as well. Additional crawling sources were configured for Symposium, Documentation, and Developer sites.
• Faceting
  While Sitecore Search is capable of faceting based on URL structure alone, we decided to take a more precise approach of exposing taxonomy to the crawler, by embedding it within metatags on every page of the website.

Planning for development

It was now time to start planning for Development. Sitecore's Search is unopinionated about how you architect things, how you develop, or what technologies you use. This made the planning phase a breeze.

We began with some investigation of the APIs using Postman, which provided examples of the responses and data structures we'd be working with. ReactJS with Axios was chosen as the front-end frameworks we'd develop with and that set the stage for defining requirements and tasks.

It's time to build!

Our development efforts were primarily handled by our Senior Front-End Developer over the course of one month. Some additional time was spent producing wireframes as well as some adjustments to our meta tags to ensure consistency of taxonomy across the website. All in all, initial development went very smoothly and was much easier to approach when compared to our previous solution. Working with a simple set of REST APIs made this possible.

Measuring success

The main KPIs we're keeping watch on are overall click through rates (CTR) and search conversions (CVR) for some of our key search terms. On Sitecore.com, we've defined a search conversion to be any form completions that were preceded by a search.

One month after going live, we are seeing some very encouraging results:

What's next on the roadmap?

Watch. This. Space. New features are in the works which we believe to be absolute game changers for content-rich websites. On our roadmap is a recommendation widget that we plan to embed throughout the website. These widgets are not only contextually aware of the page they're embedded on but they also understand visitors' affinities for content based on browsing history. They can be completely hands-off in terms of setup and maintenance, or precisely controlled with context-aware rules (a.k.a. recipes).

Producing personalized content recommendations with this level of flexibility is not a trivial task, even for highly-experienced development teams. Yet, when the recommendations are provided in an API-driven SaaS solution, development times are drastically reduced while Content Marketers can focus on producing content and tuning the recommendations to their liking.