For e-commerce platforms, a powerful search engine is one of the cornerstones of user experience and sales success. However, as your product catalog grows and user expectations rise, default search algorithms often fall short. In this article, we will learn how to fine-tune the relevance of your e-commerce search results using the world-class open-source search engine Elasticsearch, and how to deliver more accurate and satisfying results to your users with custom scoring techniques.

Basic Relevance and the BM25 Algorithm in Elasticsearch

Elasticsearch, by default, ranks search results using the BM25 (Okapi BM25) algorithm. This algorithm calculates a score (_score) by considering factors such as the term's frequency in a document (TF - Term Frequency) and its rarity across the entire index (IDF - Inverse Document Frequency). However, in e-commerce, textual matching alone may not be sufficient. For example, in a search for "phone", you might want to prioritize best-selling or newest phones.

Boosting Fields

We can use boosting to indicate that certain fields (e.g., product_name or brand) are more important than general fields like product_description. This is a simple but effective method for adjusting relevance.

GET /products/_search
{
  "query": {
    "multi_match": {
      "query": "samsung phone",
      "fields": [
        "product_name^3", 
        "brand^2", 
        "product_description"
      ]
    }
  }
}

In the example above, the product_name field is boosted by a factor of 3, and the brand field by a factor of 2, increasing their default score contribution.

Custom Scoring: The function_score Query

For more complex relevance logic, the function_score query comes into play. This query allows you to modify the document score by applying a custom function for each document. Some common functions you can use within function_score include:

• boost_factor: Adds a constant multiplier to the current score.
• field_value_factor: Scores based on the value of a numeric field.
• random_score: Adds a random score (for A/B testing or discovery purposes).
• decay_functions: Decreases the score based on the proximity of a numeric field to a certain value or the recency of a date field (e.g., gauss, exp, linear).
• script_score: Allows you to write entirely custom scoring logic with the Painless scripting language. This is the most flexible option.

Example 1: Scoring by Popularity and Stock Status (field_value_factor)

In e-commerce, prioritizing popular products or products with ample stock is a common request. We can easily do this with field_value_factor:

GET /products/_search
{
  "query": {
    "function_score": {
      "query": {
        "multi_match": {
          "query": "smart watch",
          "fields": ["product_name", "brand", "features"]
        }
      },
      "functions": [
        {
          "field_value_factor": {
            "field": "sales_count",
            "modifier": "log1p", 
            "factor": 1.2
          }
        },
        {
          "field_value_factor": {
            "field": "stock_quantity",
            "modifier": "sqrt",
            "factor": 0.5
          }
        }
      ],
      "score_mode": "multiply", 
      "boost_mode": "multiply"
    }
  }
}

This query includes both sales count (logarithmically increasing it with log1p) and stock quantity (taking its square root with sqrt) in the score for "smart watch" searches. score_mode and boost_mode determine how the scores from multiple functions or the main query are combined.

Example 2: Prioritizing New Products (decay_functions)

To make newly added products more visible, we can use decay_functions. The example below gives a higher score to newer products based on the added_date field:

GET /products/_search
{
  "query": {
    "function_score": {
      "query": {
        "match": {
          "product_name": "laptop"
        }
      },
      "functions": [
        {
          "gauss": {
            "added_date": {
              "origin": "now",
              "scale": "30d",
              "offset": "7d",
              "decay": 0.5
            }
          }
        }
      ],
      "boost_mode": "multiply"
    }
  }
}

• origin: "now": The center point is the current time.
• scale: "30d": The score decay starts within 30 days.
• offset: "7d": For the first 7 days, the score is full; decay does not start.
• decay: 0.5: Reduces the score to half of the original score when it is away by the scale value.

Example 3: Script Score for Complex Logic (Painless)

When you want to combine multiple fields or custom business logic, script_score comes into play. For instance, you might want to prioritize products with a "discounted" tag, popular products in a specific category, and products near a certain price point, all at once.

GET /products/_search
{
  "query": {
    "function_score": {
      "query": {
        "bool": {
          "must": { "match": { "product_name": "headphones" } },
          "should": [
            { "term": { "category.keyword": "gaming_accessories" } },
            { "term": { "tags.keyword": "discounted" } }
          ]
        }
      },
      "functions": [
        {
          "script_score": {
            "script": {
              "source": """
                double custom_score = _score;
                // Prioritize discounted products more
                if (doc['tags.keyword'].contains('discounted')) {
                  custom_score *= 1.5;
                }
                // Boost products in the gaming accessories category with an extra bonus
                if (doc['category.keyword'].contains('gaming_accessories')) {
                  custom_score *= 1.3;
                }
                // Slight bonus for products with high stock quantity
                if (doc['stock_quantity'].size() > 0 && doc['stock_quantity'].value > 100) {
                  custom_score *= 1.1;
                }
                return custom_score;
              """
            }
          }
        }
      ],
      "boost_mode": "multiply"
    }
  }
}

This script dynamically increases the score based on whether the product is "discounted", belongs to the "gaming accessories" category, and its stock quantity, in addition to the base BM25 score. Painless scripts are very powerful and allow you to implement almost any kind of business logic.

Best Practices and Considerations

• Iterative Approach: Relevance optimization is an ongoing process. Apply changes in small steps, test, and evaluate results using user feedback or search analytics.
• Performance: Especially when using script_score, pay attention to the complexity of the scripts. Very complex scripts can negatively impact search performance. Prefer simpler function_score types if possible.
• Monitor User Behavior: Track which searches lead to which products, which products users click on, and conversion rates to further refine your optimization strategies.
• Use Caching: Caching results for frequently performed searches can improve performance.

Conclusion

For e-commerce sites, enabling users to quickly find the right product is critical. Thanks to Elasticsearch's powerful scoring mechanisms and the function_score query, you can go beyond mere textual matches to deliver dynamic and intelligent search results tailored to your business logic. By implementing these techniques, you will enrich your users' search experience and increase your sales.