As with any technology, search has really come a long way, especially cloud services and integrating those services with your web application. In light of that, we are going to explore adding a basic Semantic ranker to your Kentico 13 search index and how that can dramatically improve your users’ search experience. Let’s jump right in!
First, let’s break down what our tech stack looks like.
We should also cover some prerequisites to make sure you know where our starting point is, as this article will not be covering the below items:
In order to use Semantic ranker, you will need to install newer .NET Core packages to work with the correct objects.
We have a class library project, ProjectName.KenticoIntegration. In this project, we are going to install the Azure.Search.Documents package. Chances are you may already have it, but it may need to be updated. Make sure to update it in all of the projects using it.
Let’s dig into some code, shall we? There is going to be some refactoring needed, but you are mostly going to be adding to your existing code.
When using Azure as a search index, generally, you are going to customize the index process in some way so that you can add your own flavor of content to the index. To provide some context, in our case, we have the following:
[assembly: RegisterModule(typeof(SearchIndexCustomizationModule))]
namespace ProjectName.KenticoIntegration.Modules;
public class SearchIndexCustomizationModule : Module
{
public SearchIndexCustomizationModule()
: base("SearchIndexCustomization")
{
}
protected override void OnInit()
{
base.OnInit();
DocumentFieldCreator.Instance.CreatingField.After += CreatingField_After;
DocumentEvents.CreateSearchFieldsForPage.Execute += CreateSearchFieldsForPage_Execute;
SearchServiceManager.CreatingOrUpdatingIndex.Execute += EnhanceSearchIndex;
}
}
We are hooking into several different events with our own custom methods. We will dive into those one by one, as we will need to know what they are doing when creating the methods for configuring the index with a semantic configuration.
You’ll notice later that we have similar processes, but the objects used are from different packages. In the search index customization module, we are utilizing the Microsoft.Azure.Search packages, whereas in the helper class, we’re going to be utilizing the newer Azure.Search.Documents package to aid in creating the semantic ranker. The helper is required specifically to handle the newer Semantic ranker features we will be implementing.
When Kentico has to create and update the index, these search index objects have to match, meaning they both have to have the same configuration; otherwise you will run into errors such as the following:
“Creating or updating index ‘index-name’ failed with the following error: This index has suggesters defined in it that cannot be removed. This error is caused by attempting to update an index without including all of its existing suggesters or by attempting to update an index that was created/updated with a new version of the API.”
Now, let’s get into the actual inner workings of the index creation process.
One thing we can do to make our lives a little easier and avoid repetitive code is create a DefaultSuggester class that inherits Suggester (from the Microsoft.Azure.Search.Models library). We can reuse this across multiple methods, which you will see later. But, for context, here is the class now:
public class DefaultSuggester : Suggester
{
public DefaultSuggester()
: base(
Settings.GeneralSearchSuggester,
DefaultSuggesterFields())
{
}
public static List<string> DefaultSuggesterFields()
{
return new List<string>
{
DocumentFieldNames.Topic,
DocumentFieldNames.DocumentName,
DocumentFieldNames.Brand,
DocumentFieldNames.Company,
DocumentFieldNames.Categories,
DocumentFieldNames.NodeName,
DocumentFieldNames.SysContent,
};
}
}
In this method, we are setting up the fields that will have our desired attributes—in our case, that’s several filterable and facetable fields for our search page that is already implemented.
The first thing we do is validate the incoming CreateFieldEventArgs parameter, specifically the SearchIndex.IndexCodeName to verify it is indeed the correct index. We then configure our fields with their appropriate values based on our needs:
private void CreatingField_After(object sender, CreateFieldEventArgs e)
{
if (!CheckIndexName(e.SearchIndex.IndexCodeName))
{
return;
}
var facetableFieldList = new string[] {
SearchIndexFields.Topic,
SearchIndexFields.Brand,
SearchIndexFields.Company,
SearchIndexFields.Categories
};
if (facetableFieldList.Contains(e.Field.Name))
{
e.Field.IsFacetable = true;
}
if (e.Field.Name.Equals(nameof(TreeNode.DocumentPublishTo), StringComparison.OrdinalIgnoreCase) || e.Field.Name.Equals(nameof(TreeNode.DocumentPublishFrom), StringComparison.OrdinalIgnoreCase))
{
e.Field.IsFilterable = true;
}
}
In this method, we add our fields as necessary, formatting values that require formatting as well.
private void CreateSearchFieldsForPage_Execute(object sender, CreateSearchFieldsForPageEventArgs e)
{
if (!CheckIndexName(e.IndexInfo.IndexCodeName))
{
return;
}
if (e.Page is FormNode node)
{
AddField(e, SearchIndexFields.IsNew, CreateSearchFieldOption.SearchableAndRetrievable, node.NewTagExpirationDate >= DateTime.Today);
}
// additional fields and logic as needed
}
And for context, here is our AddField method:
private void AddField<T>(CreateSearchFieldsForPageEventArgs e, string title, CreateSearchFieldOption option, T value)
{
var field = SearchFieldFactory.Instance.Create(title, typeof(T), option);
field.Value = value;
e.Fields.Add(field);
}
This custom method ties everything together.
private void EnhanceSearchIndex(object sender, CreateOrUpdateIndexEventArgs e)
{
var index = e.Index;
if (!CheckIndexName(index.Name))
{
return;
}
AddSuggester(index);
AddSemanticConfiguration(e.SearchService.Name, e.SearchService.AdminApiKey, index);
}
The method calls two more private methods (AddSuggester and AddSemanticConfiguration), one to set up the suggesters and the other to set up the semantic configuration.
private void AddSuggester(AzureSearchModels.Index index)
{
if (index.Suggesters == null)
{
index.Suggesters = new List<AzureSearchModels.Suggester>();
}
if (!index.Suggesters.Any(s => s.Name == Settings.GeneralSearchSuggester))
{
index.Suggesters.Add(
new AzureSearchModels.Suggester
{
Name = Settings.GeneralSearchSuggester,
SourceFields = DefaultSuggester.DefaultSuggesterFields(),
}
);
}
}
For reference, one of the important things you have to do is make sure to reference the .NET Framework package here. Note the “AzureSearchModels” alias, which we have aliased from the Microsoft.Azure.Search.Models package. More on that momentarily. Let’s take a look at the AddSemanticConfiguration:
private void AddSemanticConfiguration(string serviceName, string adminKey, AzureSearchModels.Index index)
{
var contentFields = new List<string>
{
SearchIndexFields.Description,
SearchIndexFields.Content
};
var keywordFields = new List<string>();
var indexHelper = index.Fields.Aggregate(new SearchIndexHelper(index.Name, serviceName, adminKey),
(helper, field) => helper.WithField(field))
.WithSuggester(
Settings.GeneralSearchSuggester,
DefaultSuggester.DefaultSuggesterFields())
.WithSemanticSearch(
Settings.SemanticConfigurationName,
SearchIndexFields.Title,
contentFields,
keywordFields);
var response = indexHelper.CreateOrUpdate();
if (response != null)
{
var eventLogService = Service.Resolve<IEventLogService>();
var eventType = response.Status < 300 ? EventTypeEnum.Information : EventTypeEnum.Error;
eventLogService.LogEvent(new EventLogData(eventType, "Custom Azure Search Module", "AZURE_SEMANTIC_SEARCH_INDEX")
{
EventUrl = SearchIndexHelper.GenerateAzureSearchServiceUri(serviceName).ToString(),
EventDescription = response.ReasonPhrase
});
}
}
Notice we are using the fluent interface design pattern for chaining purposes. As noted above, we are doing all of this with the older package so that things are compatible with Kentico’s process, but we need to extend this with our new class to utilize the new packages and semantic options. We will explore that in a moment in the next section, but first let’s review some of what’s going on with the configuration method above.
In our configuration, the semantic title field is associated with the Title field of the node. The content fields for the semantic configuration were just the fields on the node that had a majority of the content that would be matched with a semantic search. We also didn’t have any fields in our case that made sense for the Keywords fields.
Another thing to keep in mind with semantic search is how it tokenizes the input and the limit on those tokens. Approximately 20,000 characters will max out the search, meaning if you have fields that are extremely long in your configuration, it could possibly eat up all of the available tokens. This is an important detail, so organizing the fields is crucial to ensure the most relevant fields are included in the search.
In our SearchIndexHelper class, located in our KenticoIntegration project, we instantiate a couple of private properties and our constructor. Note that the SearchIndex and the SearchIndexClient (and other objects) are both from the Azure.Search.Documents package. We also alias the older package so we can pass in the legacy field to the WithField method.
using LegacyAzureSearchModels = Microsoft.Azure.Search.Models;
...
private readonly SearchIndex index;
private readonly SearchIndexClient indexClient;
public SearchIndexHelper(string indexName, string serviceName, string adminKey)
{
index = new SearchIndex(indexName);
indexClient = new SearchIndexClient(GenerateAzureSearchServiceUri(serviceName), new AzureKeyCredential(adminKey));
}
We use our search admin key and service name that were retrieved from the key vault and passed in to create our client. After the constructor, there are our chaining methods.
public SearchIndexHelper WithField(LegacyAzureSearchModels.Field field)
{
if (field.IsSearchable is true)
{
index.Fields.Add(new SearchableField(field.Name,
field.Type.ToString().Contains("Collection"))
{
IsFacetable = field.IsFacetable is true,
IsFilterable = field.IsFilterable is true,
IsKey = field.IsKey is true,
IsSortable = field.IsSortable is true,
AnalyzerName = field.Analyzer.ToString() ?? string.Empty,
});
}
else
{
index.Fields.Add(new SimpleField(field.Name, field.Type.ToString())
{
IsFacetable = field.IsFacetable is true,
IsFilterable = field.IsFilterable is true,
IsKey = field.IsKey is true,
IsSortable = field.IsSortable is true,
});
}
return this;
}
public SearchIndexHelper WithSuggester(string name, List<string> fields)
{
var existingSuggester = index.Suggesters.FirstOrDefault(s => s.Name == name);
if (existingSuggester != null)
{
return this;
}
index.Suggesters.Add(new SearchSuggester(name, fields));
return this;
}
public SearchIndexHelper WithSemanticSearch(string name, string titleField, List<string> contentFields, List<string> keywordFields)
{
var semanticSearch = new SemanticSearch();
var fields = new SemanticPrioritizedFields()
{
TitleField = new SemanticField(titleField),
};
contentFields.ForEach(fieldName => fields.ContentFields.Add(new SemanticField(fieldName)));
keywordFields.ForEach(fieldName => fields.KeywordsFields.Add(new SemanticField(fieldName)));
semanticSearch.Configurations.Add(new SemanticConfiguration(name, fields));
index.SemanticSearch = semanticSearch;
return this;
}
public Response CreateOrUpdate()
{
var response = indexClient.CreateOrUpdateIndex(index);
return response.GetRawResponse();
}
This is the method that we will call from the customization module code to integrate the two classes, integrating the Kentico CMS with the newer .NET Core packages necessary to work with Semantic ranker.
And that is the overall concept of implementing Semantic ranker with your Kentico 13 search index within Azure. With this type of configuration, search results on your website will have a layer of semantic association. The biggest challenge I ran into was integrating the .NET Framework packages of Kentico with the new .NET Core packages. The indexes have to match definitions across both packages, including the suggesters.
Adding Semantic ranker to your Kentico 13 search index is a powerful enhancement, even with minimal configuration. If you are using a free search service, you will have to weigh the costs of creating a new Basic plan search service. However, if you are already paying for the Basic plan, it might be a no-brainer to enhance the user experience within your Kentico sites search.
Happy coding!
We love to make cool things with cool people. Have a project you’d like to collaborate on? Let’s chat!
Stay up to date on what BizStream is doing and keep in the loop on the latest in marketing & technology.
