Skip to content

Deduplication

Deduplication is the concept of automatically removing duplicate products from panels in zones. This function enables eSales to maximise the number of different products shown on a page. Deduplication is supported by all panels that display products.

The only requirement for deduplication is that the whole page is retrieved by a single panel query as a zone. This enables deduplication of a whole page, even when the page is modelled as multiple zones.

Panels are ranked by popularity by eSales, and panels with high popularity choose content before less popular panels. Popularity is mainly based on visitor interactions with the panels and the products they display.

Panels that have had products removed is backfilled with products based on the sort order of the panel. Which panels returned in a query that will receive backfills are determined by their popularity as ranked by eSales followed by the panels' duplicates setting.

Settings

Deduplication is configured per panel and zone, and have three available settings: Keep, Allow Removal, and Force Removal. The panels supporting deduplication all have a recommended setting by default. A panel has a setting per zone it is used in, so the same panel can be used in different zones with different settings.

Keep duplicates

Using the Keep setting for a panel means that any duplicate products in this panel will be kept, i.e. the content of this panel is not affected by what is displayed in other panels.

Content in a panel with the Keep setting will cause removal of duplicates in all panels that are configured to force removal of duplicates, and in panels that are configured to allow removal of duplicates that are ranked as less popular by eSales.

Allow removal of duplicates

Using the Allow Removal setting for a panel means that the panel is allowed to remove duplicate products from its content, but only if those products appear on a panel that is ranked as more popular by eSales.

This mean that panels with the Allow Removal setting does not always remove duplicate products on a page.

Force removal of duplicates

Using the Force Removal setting for a panel means that the panel never display products that appear in any other panel returned by the same query. Panels with this setting choose content after panels with the Keep and Allow Removal settings regardless of panel popularity as ranked by eSales.

If more than one panel with the Force Removal setting are returned, these panels will choose content in descending order of panel popularity as ranked by eSales.

Panels supporting deduplication

Deduplication is supported by all panels that display products.

The panels supporting deduplication all have default settings which are recommended to use for most use cases. Complex pages, or pages with many zones and panel combinations, may require other panel settings than the default to show desired results.

Keep

The following panels use Keep as their recommended setting.

Allow removal

The following panels use Allow Removal as their recommended setting.

Force removal

No panels use Force Removal as their recommended setting.

Usage examples

Keep products in a product list panel with high popularity

The following example is a zone with two panels, a Product list panel with 8 products and a Recommend based on customer panel with 4 products. The Product list panel has the Keep setting and is ranked as more popular by eSales. The Recommend based on customer panel has the Allow Removal setting, so any product that is returned for both panels will be substituted in the Recommend based on customer panel.

The product P1 is returned for both panels in the request. As the Product list panel is more popular it selects the product P1 first. The product P1 is removed from the Recommend based on customer panel and the fifth best product to recommend, P5, is automatically backfilled and shown instead.

Graphic showing deduplication using recommended duplicates setting

Remove products from panels with high popularity

A panel may be prioritised by a retailer, but if that panel is ranked as less popular by eSales, unwanted duplication may occur as other panels select content before it.

The following example is a zone with two panels, a Product list panel with 8 products and a Recommend based on customer panel with 4 products. The Product list panel has the Keep setting and is ranked as less popular by eSales, but is prioritised by the retailer. The Recommend based on customer panel has the Allow Removal setting, so any product that is returned for both panels will be shown in both as the Recommend based on customer panel is ranked as more popular.

The product P1 is returned in both panels in the request. As the Recommend based on customer panel is more popular it selects the product P1 first, but as the Product list panel has the Keep setting it will also select the product P1 (fig. 1).

To avoid duplication, and have the Recommend based on customer panel to use the backfill as the Product list panel is prioritised, the duplicates setting for the Recommend based on customer panel must be changed to Force Removal. The product P1 is then removed from the Recommend based on customer panel and the fifth best product to recommend, P5, is automatically backfilled and shown instead. (fig. 2).

Graphic showing deduplication using the Force Removal setting for a more popular panel

Keep duplicates with ads panels

It is recommended to always show advertised products, even though it may cause duplicates on a page.

The following example is a zone with two panels, a Top sellers panel with 8 products and an Ads by customer panel with 4 products. The Top sellers panel has the Keep setting and is ranked as more popular by eSales. The Ads by customer panel has the Allow Removal setting, so any product that is returned for both panels will be substituted in the Ads by customer panel.

The product P1 is returned for both panels in the request. As the Top sellers panel is more popular it selects the product P1 first. The product P1 is removed from the Ads by customer panel and the fifth best product to recommend, P5, is automatically backfilled and shown instead (fig. 1).

To avoid deduplication, and have the Ads by customer panel show the product P1 too, the duplicates setting for the Ads by customer panel must be changed to Keep. The product P1 is now shown in the Ads by customer panel as well (fig. 2).

Graphic showing keeping products in ads panels by changing duplicates setting

Panel popularity and force removal

The Force Removal setting can be used to a certain extent to control in which panel a product will appear. This is useful when dealing with zones with more than 2 panels and a product that is preferred by all panels, but is only to be displayed in one panel.

The following example is a zone with three panels, a Recommend based on customer panel with 4 products, a Product list panel with 8 products, and a Recommend based on cart panel with 4 products.

The Recommend based on customer panel has the Allow Removal setting, is ranked as more popular by eSales, and is the prioritised recommendations panel by the retailer. The Product list panel has the Keep setting and is ranked as medium popular. The Recommend based on cart panel has the Allow Removal setting and is ranked as less popular by eSales.

The product P1 is returned for all panels in the request, but is only to be displayed in the Product list panel. As the Recommend based on customer panel is more popular it selects the product P1 first, but as the Product list panel has the Keep setting it will also select the product P1. The Recommend based on cart panel has the Allow Removal setting and will select the fifth best product to recommend, P5 (fig. 1).

To remove the product P1 from the Recommend based on customer panel, its duplication setting is changed to Force Removal. As panels with the setting Force Removal selects products last, the product P5 remains in the Recommend based on cart panel and the sixth most popular product P6 is shown in the Recommend based on customer panel (fig. 2).

As the Recommend based on customer panel is more popular than the Recommend based on cart panel as ranked by eSales, it should select products before the Recommend based on cart panel. To allow the Recommend based on customer panel to select products before the Recommend based on cart panel the duplication settings of the Recommend based on cart panel is changed to Force Removal. As both panels now use the Force Removal setting, the Recommend based on customer panel selects products first. The product P5 is now shown in the Recommend based on customer panel and the product P6 is shown in the Recommend based on cart panel (fig. 3).

Graphic showing deduplication using changing settings for multiple panels

Turn off deduplication

To turn off eSales automatic deduplication, change the duplication setting in all panels used in zones to Keep.

Recommendations and best practice

  • Always display all products and ads returned in a response from eSales. As eSales is not aware if products and panels are removed in the web layer, eSales may exclude a product from a visible panel for duplication avoidance.
  • Always show advertised products, even though it may cause duplicates on a page. All ads panels usee Allow Removal as their recommended setting.
  • When using paginated panels, always use the Keep setting as eSales is not aware of any deduplication that has occurred in previous requests.
×