2021-11-30 00:00:00 +0000

Coveo for Sitecore (Coveo Hive) comes with an out-of-the-box Coveo Tab component that can be useful for switching between different types of Sitecore content, but lacks a direct way within Sitecore to only show External Content without Sitecore content.

If you’re team is lucky enough to have the Enterprise version, then you may be able to get away with adding a rule within the Coveo Tab Datasource field Filter expression rules, but the rule “where specific field compares to specific value” doesn’t allow you to make a blanket statement rule to not inlude a document that contains a field, which is enough to keep Sitecore documents out of the mix.

A solution I recommend (for both Pro and Enterprise) is to use a Pipeline Advanced Filter. The Datasource for Coveo Tab contains the field Names of external content sources that will append external coveo sources from your organization to the Coveo API call. It also contains the field Pipeline for determining which Query Pipeline you would like the request to filter through within the Coveo Admin Portal. The following steps should be taken:

1. Add all of your external source names from the Coveo Portal to the field Names of external content sources:

2. Create a new Pipeline in Coveo Portal that your external sources will flow through and paste that Pipeline name into the Pipeline field:

3. Within the Coveo Portal, double-click on the created external source Pipeline and navigated to the Advanced tab and stay on the right-rail Filters section. Click on the Add rule button and add a new cq (constant query) parameter with the expression NOT @alltemplates. The condition section can be left empty since we never want to include Sitecore documents. The @alltemplates field is specific to Sitecore documents, but feel free to use any other field if your external source meta data has created this field:

2021-01-05 00:00:00 +0000

The Use Case

In a typical content migration you may carry over certain data to Sitecore that pertains to the UI behavior, such as a Publication Date for Articles and an End Date for Webinars. These values may be used hand in hand as sort criteria for Coveo as a single computed field, such as contentdate. When all of this content has been imported to Sitecore, the statistics field section Updated Date field may not be set to the true Date of when that piece of content was last modified due to the nature of content being created and the statistics being automatically updated on item::save events. The import of items can be customized to set the same value in the Updated field and then skip updating statistics, but that runs the risk of re-modifying a system field used to append the item to the publishing queue based on timestamps. I would recommend importing items in ascending order at this point.

With content imported, you’ll notice that an Article from 2012 has a similar last updated value as an Article that is less than a few weeks old. And this may be no different than what you see in the UI - especially when a Coveo Search Interface is loaded with relevancy and no query. This is due to Coveo’s ranking of documents.

Coveo Ranking in our Use Case

Relevancy heavily relies on two Coveo Ranking Phases called Term Weighting and Term Frequency & Adjaceny. These phases have important factors that are only used when a search query is applied. Without term ranking boosting relevancy, we are left with the Document Weighting phase, where one if it’s ranking factors, Date (Item last modification), plays a crucial role in ranking Items higher with the most recent modification date. This is not to say that a content item will be ranked higher, but a higher total score could be achieved. Due to the out-of-the-box Document Weighting, we find ourselves viewing too many irrelenvant articles and Past Webinars.

To further understand all of the ranking factors, please refer to Coveo’s Guidelines for Understanding Search Result Ranking.

Step 1: Fine Tuning Item last modification

Item last modification can be carefully fine tuned for a specific query pipeline. The emphasis on carefully is to say that if you don’t include a condition with this tuning, then any query that goes through the same pipeline will be affected. Let’s go ahead and navigate to platform.cloud.coveo.com and fine tune the Item last modification with a new condition below:

1. Select Query Pipelines > [Query Pipeline] > Ranking Weights Tab. In the screenshow below, I have identified a new rule for Item last modification to be set to a value of 2 (out of 5) when the result is either an Article or Webinar and the Query is empty.

1. Select button on the top right “Add Rule” to open a new Ranking Weight Rule that will allow us to adjust the Item last modification and create a new rule. In the screenshot below, I have tuned Item last modification on the left-hand side and have selected a pre-existing condition on the right-hand side:

1. Create a new Condition by going to Search > Conditions > Add Condition (conditions are re-usable across many rules for QPL and ML). In the screenshot below, I have identified a Coveo Custom Context Key, resulttype, that my team uses to keep track of the corresponding result’s Sitecore Template Name. I won’t get into the details of how to track and send custom context keys as this step is relatively lightweight and can be found by Coveo documentation here.

1. Select “Add Rule” to save this Rule with condition to the Ranking Weights of your Query Pipeline. With this tuning in place, Coveo states that a value of 0-4 will progressively reduce the weight of a ranking factor relative to its pre-tuned value. This did not drastically push a 2012 Article out of sight because of the quality of the document alone had one of the highest document weight scores.

Step 2: Query Ranking Function to the rescue

A Query Ranking function helps immensly with the ranking score of an item because the boost it provides is relative to the function and score limit provided. A range of scores become available based on the ranking function’s algorithm when a result item is passed through. You may notice within a Query Pipeline a tab for “Ranking Expression”, but do not get this feature confused with a Ranking Function. Ranking Expressions within the query pipeline can only apply statically modified ranking adjustments (reduction or boost). We could add a Ranking Expression for computed field @contentdate to reduce results that are greater than N years old, but this still doesn’t provide a percentage based boosting or sliding scale of ranking.

For our ranking function, we decided that we want a sliding scale of boosting to start at 8 years prior to now for results that have the @contentdate field and a max boosting limit of 500. I have found that an item from Jan 2012 will have 0 boosting, whereas an item from Jan 2013 will have around 1-3% of the 500 modifier total, and an item close to NOW will have around a 90-100% boost.

The ranking function should be added directly with JavaScript within the buildingQuery event listener of a Coveo Search Interface:

args.queryBuilder.advancedExpression
	.add("$qrf(expression: 'max(@contentdate, (NOW - (YEAR * 8)))', normalizeWeight: 'true', modifier: '500')");

The Result

Reducing the Item last modification weight for my Query Pipeline and adding a custom Query Ranking Function visibly shifted the results to a more date-eccentric ranking while keeping the Relevancy of other ranking factors. In the screenshot below, the first result has been boosted by an additional 466 points solely from the Ranking Function. We don’t see a true 500 point boost to an Upcoming Webinar due to Coveo’s ranking function algorithm which states that the boosting value used will not reach the modifier limit if the results around it don’t reach a point where a larger boost is necessary.

2020-11-04 00:00:00 +0000

The Scenario

In my scenario, we’re dealing with a complete Sitecore Implementation where Coveo has been tacked on. Let’s say that we’ve just finished implementing a Coveo Search Results page for a particular section. After the search results page usually comes an example implementation of a Page that has stand-alone search behavior that redirects the user to the search results page. Let’s also say that this has been configured on an example page for now by adding the Coveo Hiv Search Box component within a Search Section and an External Components Section. The External Components Section includes a Datasource necessary for connecting the Coveo Hive Search Interfance and the Search Box includes a Datasource necessary for pointing the Search box to the search results page.

The Process

This process of setting up the Coveo stand-alone search behavior has to be done within Experience Editor due to dynamic placeholders and can take roughly 15-20 minutes per Page, especially if there are a lot of Rendering Parameters that need to be set up. In many cases, this example page with stand-alone search behavior has to be switched out on every page that previously had a searchbox. Handling this manually could take over a day to configure during a deployment to CM. A first thought could be to configure this within Std Values, but then you would have to reset every item’s Rendering Definition and completely lose any overrides of components per page. By using Sitecore PowerShell we can maintain the positioning with dynamic placeholders for the searchbox and versioning of each item that is swapped.

SPE Script: Swap out Renderings

The SPE script below will handle swapping out the old search component whether from Shared or Final Renderings and maintain the Rendering Parameters and Datasource from the Coveo Hive components on the example set up page:

##########################################################
#
#		This script will switch out old search renderings with new Coveo (custom) Hive Renderings
#			- The new Renderings will maintain the Rendering Parameters and Dynamic Placeholders from the single Page Item that is set up correctly
#			- The old search renderings can be removed from either Shared or Final layout
#
###########################################################

Write-Host "BEGIN: Query for pages that have the old Search Keyword Rendering in either shared or final __renderings"

## Query all pages that contain the old search rendering either within the shared or final layout
$queriedItems = Get-Item -Path master: -Query "/sitecore/content/Shared/pages/energy//*[contains(@__Renderings, '{D7C47D4D-C50E-4AEF-B805-1ADE1B854605}') or contains(@__Final Renderings, '{D7C47D4D-C50E-4AEF-B805-1ADE1B854605}')]"
Write-Host $queriedItems.Count " items found" -ForegroundColor "Green";	Write-Host ""

$reportOnly = $false
$finalItems = @()

## get the single item that has already been converted to the Coveo Hive stand-alone search: /Pages/Coveo Energy
$coveoStandaloneSearchboxContentItem = Get-Item -Path master: -ID "{9921DC61-47D7-4269-9758-749DD785E113}"

## get rendering items
$ri__ExternalComponentsSection = Get-Item -Path master: -ID "{F569344F-3933-45F1-92FE-F6A44159D2AE}"
$ri__GlobalSearchSection = Get-Item -Path master: -ID "{FDCEBA24-58B4-4279-BB7C-E605F5A32307}"
$ri__CustomCoveoGlobalSearchBox = Get-Item -Path master: -ID "{E43BBC7B-2910-4B1E-8320-E3E747516826}"

## get rendering instances of renderings that we want to add that are pre-filled with Rendering Parameters (dynamic placeholder, datasource, etc.)
$renderingInstance_ExternalComponentsSection = Get-Rendering -Item $coveoStandaloneSearchboxContentItem -Rendering $ri__ExternalComponentsSection -Device (Get-LayoutDevice "Default") -FinalLayout
$renderingInstance_GlobalSearchSection = Get-Rendering -Item $coveoStandaloneSearchboxContentItem -Rendering $ri__GlobalSearchSection -Device (Get-LayoutDevice "Default") -FinalLayout
$renderingInstance_CustomCoveoGlobalSearchBox = Get-Rendering -Item $coveoStandaloneSearchboxContentItem -Rendering $ri__CustomCoveoGlobalSearchBox -Device (Get-LayoutDevice "Default") -FinalLayout

Write-Host "Swap out old search box rendering for Coveo Searchbox Rendering and Hive counterparts"

$queriedItems | foreach-object {
    ## item must have layout
    if (Get-Layout $_)
    {
        ## TESTING: only perform on single content item that is duplicate
        #if($_.DisplayName -ne "Copy of [Old Stand-alone Search Page]") { return }
        
        $isInShared = $false
        
        $renderingToRemove = Get-Rendering -Item $_ -Rendering (Get-Item -Path master: -ID "{D7C47D4D-C50E-4AEF-B805-1ADE1B854605}") -Device (Get-LayoutDevice "Default") -FinalLayout
        if($renderingToRemove -eq $null) {
            $renderingToRemove = Get-Rendering -Item $_ -Rendering (Get-Item -Path master: -ID "{D7C47D4D-C50E-4AEF-B805-1ADE1B854605}") -Device (Get-LayoutDevice "Default")
            $isInShared = $true
        }
        
        ## remove old search box
        if($isInShared){
            Remove-Rendering -Item $_ -Instance $renderingToRemove -Device (Get-LayoutDevice "Default")
        } else {
            Remove-Rendering -Item $_ -Instance $renderingToRemove -Device (Get-LayoutDevice "Default") -FinalLayout
        }
        
        ## add new renderings to *shared
        Add-Rendering -Item $_ -Instance $renderingInstance_ExternalComponentsSection -PlaceHolder $renderingInstance_ExternalComponentsSection.Placeholder -Device (Get-LayoutDevice "Default") -Index 0
        Add-Rendering -Item $_ -Instance $renderingInstance_GlobalSearchSection -PlaceHolder $renderingInstance_GlobalSearchSection.Placeholder -Device (Get-LayoutDevice "Default") -Index 1
        Add-Rendering -Item $_ -Instance $renderingInstance_CustomCoveoGlobalSearchBox -PlaceHolder $renderingInstance_CustomCoveoGlobalSearchBox.Placeholder -Device (Get-LayoutDevice "Default") -Index 2
        
        ## get new rendering instances and set placeholders of each based on their unique ids
        $r1_instance = Get-Rendering -Item $_ -Rendering $ri__ExternalComponentsSection
        $r2_instance = Get-Rendering -Item $_ -Rendering $ri__GlobalSearchSection
        $r3_instance = Get-Rendering -Item $_ -Rendering $ri__CustomCoveoGlobalSearchBox
        
		## the unique id of the rendering instance (minus the curly brackets) is used within Coveo Hive after "dynamic_coveo"
        $r1_ph = $r1_instance.UniqueId.Substring(1,8).ToLower()
        $r2_ph = $r2_instance.UniqueId.Substring(1,8).ToLower()
        $r3_ph = $r3_instance.UniqueId.Substring(1,8).ToLower()
        
        $r2_instance.PlaceHolder = "/pagebody/body/main/rail/coveo-ui-external-components_dynamic_coveo{0}" -f $r1_ph
        Set-Rendering -Item $_ -Instance $r2_instance -FinalLayout
        
        $r3_instance.PlaceHolder = "{0}/coveo-ui-global-searchbox_dynamic_coveo{1}" -f $r2_instance.Placeholder,$r2_ph
        Set-Rendering -Item $_ -Instance $r3_instance -FinalLayout
        
        $finalItems += $_
        Write-Host $_.name
    }
}

Write-Host "Final Items" $finalItems.Count -ForegroundColor "Green"

Testing

From the SPE script above, simply uncomment the line underneath “TESTING”, create a duplicate of any old stand-alone search page, and replace [Old Stand-alone Search Page] with said Item’s path. After running the script, the duplicate item will have swapped out the old search component with the Coveo Hive components.

2020-07-15 00:00:00 +0000

The Issue

After logging in to Sitecore as “admin” and navigating to [scheme]://[host]/sitecore/admin/pipelinedebug.html, I noticed that this module page still prompted me to log in. So I took a closer look at the front-end and noticed that it’s fairly straight forward to read the main JS file that the modules comes with, PipelineDebug.js. In this file you will find a service.update method that takes in a method parameter, such as login and listpipelines. Appropriate data is then passed via AJAX to a single Controller /pipelinedebug/[action], of which is registered part of the Sitecore.Mvc.Pipelines.Loader.InitializeRoutes processor. As seen below in dev console debug mode, the response.Status of any method was always returning “Unauthorized”:

Decompiling the solution shows us that the PipelineDebugController attribute [AdministratorOnly] checks for Sitecore.Context.IsAdministrator. Hint - we’re dealing with a change to the validation identity of unresolved site context possibly because the API calls above do not use a site switcher.

The Solution

We need to patch in the path of all of these custom routes to the list of siteNeutralPaths within the ValidateIdentity.ValidateSiteNeutralPaths processor like so:

<processor type="Sitecore.Owin.Authentication.Pipelines.CookieAuthentication.ValidateIdentity.ValidateSiteNeutralPaths, Sitecore.Owin.Authentication">
    <path>/pipelinedebug/pipelinedetails</path>
    <path>/pipelinedebug/addprocessor</path>
    <path>/pipelinedebug/moveprocessor</path>
    <path>/pipelinedebug/getdebugprocessors</path>
    <path>/pipelinedebug/removeprocessor</path>
    <path>/pipelinedebug/getdiscoveryroots</path>
    <path>/pipelinedebug/discover</path>
    <path>/pipelinedebug/getsettings</path>
    <path>/pipelinedebug/savesettings</path>
    <path>/pipelinedebug/getoutput</path>
    <path>/pipelinedebug/exportconfiguration</path>
    <path>/pipelinedebug/importconfiguration</path>
    <path>/pipelinedebug/saveprocessortaxonomies</path>
    <path>/pipelinedebug/listpipelines</path>
    <path>/pipelinedebug/login</path>
    <path>/pipelinedebug/logout</path>
    </siteNeutralPaths>
</processor>

Once this is patched in and saved, you will be able to now see the list of pipelines and not get any further authorization issues in the pipeline.debug admin page.

2019-10-06 00:00:00 +0000

The Scenario

You just completed a fairly substantial content migration from an archaic CMS into Sitecore. These new Product Detail Pages all contain associated content items labeled “Support Links” that holds a single Rich-Text Editor field that simply renders on the page as a Datasource. The client takes one quick look at the page and notices that all of the support links are missing super important icons that are used to indicate the file type of PDF or Zip. These icons were clearly in the new design, but this old data never had the markup required.

Let’s take a look at the old markup and the desired markup:

old markup

<some_html>
    <a href="~/media/[guid].ashx">Support Link 1 (Pdf)</a>
    <a href="~/media/[guid].ashx">Support Link 2 (Zip)</a>
</some_html>

new markup

<some_html>
    <a href="~/media/[guid].ashx">
        <em class='fa fa-file-pdf-o' aria-hidden='true' style='padding-right: 5px;'></em>
        Support Link 1 (Pdf)
    </a>
    <a href="~/media/[guid].ashx">
        <em class='fa fa-file-archive-o' aria-hidden='true' style='padding-right: 5px;'></em>
        Support Link 2 (Zip)
    </a>
</some_html>

Doing this manually would be exhausting. We would have to not only go through every content item of this type and add the em markup, but also parse the link’s Dynamic url to figure out if it’s a Pdf or a Zip.

Luckily for us..

We have PowerShell! And because we’re working in context of the ISE, we can already reference the Html Agility Pack included with Sitecore (I’ve yet to come across any issues with the version being from 2014 even in Sitecore v9.1.1, but you may find yourself using a newer method that can be resolved by loading a newer dll with [Reflection.Assembly]::LoadFile).

We can start the script with getting all the items named “Support Links” with a specific template id:

$supportItems = Get-Item -Path master: -Query "/sitecore/content/Tenant/Site A/Homepage//*[@@templateid='{A099DC2D-1E23-499F-B101-DBB0902148F4}' and @@name='Support Links']"

Loop through these items and load up the content from the Rich-Text Editor field into the Agility Pack magic:

$supportItems | ForEach-Object {
    $supportItem = $_
    $content = $_."Content"    ## accessing the field value this way lets us not worry about the Begin/End Edit requirements
    
    $htmlDocument = New-Object -TypeName HtmlAgilityPack.HtmlDocument
    $htmlDocument.LoadHtml($content)

Find all the anchors using HtmlAgilityPack XPath query:

foreach($x in $htmlDocument.DocumentNode.SelectNodes("//a")) {    ## foreach anchor in html
    $href = $x.Attributes["href"].Value

Check if the anchor is a Sitecore Dynamic Url, capture the Guid and retrieve the Media Item:

if ($href  -like "~/media/*") {
    $guid = New-Object -TypeName System.guid -ArgumentList $([System.IO.Path]::GetFileNameWithoutExtension($href))        ## parse out the guid with a C# Path Helper Method!
    $mediaItem = Get-Item -Path master: -ID "{$($guid)}"
}

If the Media Item is a Zip or Pdf AND the icon has yet to be added, adjust the anchor’s inner HTML to include the icon:

if ($mediaItem.TemplateName -eq "Pdf" -and (-not $x.InnerHtml.Contains("fa-file-pdf"))) {
    Write-Host "Found PDF Link for Link [$($href)], DisplayName [$($mediaItem.DisplayName)]"								## LOG some good info
    $x.InnerHtml = "<em class='fa fa-file-pdf-o' aria-hidden='true' style='padding-right: 5px;'></em>" + $x.InnerHtml		## append the icon
}
elseif ($mediaItem.TemplateName -eq "Zip" -and (-not $x.InnerHtml.Contains("fa-file-archive"))) {
	Write-Host "Found Zip Link for Link [$($href)], DisplayName [$($mediaItem.DisplayName)]"
	$x.InnerHtml = "<em class='fa fa-file-archive-o' aria-hidden='true' style='padding-right: 5px;'></em>" + $x.InnerHtml
}

Now that we’ve made our updates in the $htmlDocument object, we can take that HTML and put it back in the Sitecore field:

$newHTML = $htmlDocument.DocumentNode.OuterHtml
$_."Content" = $newHTML

Bonus Round

So you let the content authors in to the system a little too early and, lo and behold, over 100 content items were updated to include an inline-style attribute color:green on the icons. I mean, you gotta hand it to them for trying to fix the design, right? Fortunately you know that if you were to remove this inline-style, the anchor tag’s branded color would be inherited from the inner icon tag once again. Back to the script board we go.

Within the same inner foreach anchor tag, grab the font tag that can either be an <em> or an <i>:

$nodes_em = @($x.ChildNodes["em"])		## fun tip: default to an array, @(), since ChildNodes can condense down to returning a single item
$nodes_i = @($x.ChildNodes["i"])

If any nodes are found with HTML and contains the keyword “green”, remove this style using PowerShell’s IndexOf and Remove String methods:

if ($nodes_i.Count -and $nodes_i[0].OuterHtml -and -not [System.String]::IsNullOrWhiteSpace($nodes_i[0].OuterHtml) -and $nodes_i[0].OuterHtml.Contains("green")) {
	Write-Host "Found color:green to remove from <i>: [$($href)] [$($pcItem.Id)] [$($nodes_i[0].OuterHtml)]"		## LOG the icon tag html to be removed
	
	$attr = $nodes_i[0].Attributes["style"]			## grab the inline-style attribute value
	
	$idxStart = $attr.Value.IndexOf("color:")		## get the start and end indexes of the color attribute in order to remove it
	$idxEnd = $attr.Value.IndexOf(";", $idxStart)
	
	if ($idxEnd -gt $idxStart) {					## always check to make sure the end index was set, since some inline-styles don't end with a semi-colon
	   $attr.Value = $attr.Value.Remove($idxStart, $idxEnd - $idxStart + 1)
	   Write-Host "Removed inline-style color attribute, new inline-style value [$($attr.Value)]"
	}
}

In this walkthrough we learned how to manipulate existing HTML within the Sitecore CMS with the help of Html Agility Pack and some small C# methods. As always, make sure to set return in loops and exit before making changes that could make permanent incorrect changes to your environment! And if you’re looking to bulk update more than 260 items, look to switch to the Find-Items cmdlet that uses Sitecore Indexes.