Take Control: Programmatically verify SharePoint Managed Properties

This blog post is relevant to the following common search error:

Property doesn't exist or is used in a manner inconsistent with schema settings.

If you develop a custom SharePoint 2010 solution which consumes the FullTextSqlQuery class chances are good that after deployment to a new farm you will come across the following error:

“Property doesn't exist or is used in a manner inconsistent with schema settings”

This problem occurs when your source code tries to execute custom search queries against a Search Service implementation in which the crawled properties, managed properties or property mappings which your code is dependent on, are not configured correctly.

This blog post will show you how to programmatically take control of search dependant implementations and I provide a source code example which you can use to verify that the correct dependencies in place.

Imagine you developed a web part which allows a user to provide criteria and then search the site collection or specific webs (depending on search scope) to return a specific set of field values for each result item.

You have a document library:

The search code looks like this:

ResultType resultType = ResultType.RelevantResults;
string queryString = string.Empty;

try
{
    FullTextSqlQuery fullTextSqlQuery = new FullTextSqlQuery(site);
    fullTextSqlQuery.ResultTypes = resultType;
    queryString = "SELECT Title,Division, Region, Language FROM SCOPE() WHERE FREETEXT(*, '*test* ') AND  (\"SCOPE\" = 'Demo Site Scope') AND (\"Division\" = 'Technical')";
                   
    fullTextSqlQuery.QueryText = queryString;

    ResultTableCollection resultTableCollection = fullTextSqlQuery.Execute();
    ResultTable resultTable = resultTableCollection[resultType];

    if (resultTable != null && resultTable.RowCount > 0)
    {
        while (resultTable.Read())
        {
            StringBuilder output = new StringBuilder();
            output.Append("Title:" + resultTable["TITLE"].ToString());
            output.Append(", Division:" + resultTable["Division"].ToString());
            output.Append(", Region:" + resultTable["Region"].ToString());
            output.Append(", Language:" + resultTable["Language"].ToString());
            Console.WriteLine(output);
        }
    }
}
catch (Microsoft.Office.Server.Search.Query.QueryMalformedException querymalformedexception)
{
    Console.WriteLine("Query syntax error: " + querymalformedexception.Message);
}
catch (Microsoft.Office.Server.Search.Query.ScopeNotFoundException searchscopeerror)
{
    Console.WriteLine("Search scope error: " + searchscopeerror.Message);
}
catch (Microsoft.Office.Server.Search.Query.InvalidPropertyException invalidpropertyexception)
{
    Console.WriteLine("Property error: " + invalidpropertyexception.Message);
}

You test the solution on your development server and everything works well but after you deployed to a QA or production environment your custom search code throws an error:

Property doesn't exist or is used in a manner inconsistent with schema settings.

The error does not contain information specific enough to help us identify which properties are not in place.

If you open SharePoint Central Admin –> Manage Service Applications –> Search Service Application –> Metadata Properties, we discover that some of the crawled properties which our search code rely on are not mapped to managed properties. In other cases some of the crawled properties do not even exist.

This is quite a common problem. Custom search code is dependent on specific configuration to be in place and there is always a risk during new deployments that either the provisioning code did not work properly or the SharePoint farm administrator did not configure the custom components correctly.

Obliviously we want to be SharePoint Heroes and see that our custom solutions works well after every new installation, so instead of relying on people or process let’s rather build a simple ‘success verification’ application which we can run on each new farm implementation to tell us whether all the dependencies are in place.

The source code from this post will generate the following output which will tell us exactly what the problem is:

Validate Mapped Properties (command-line tool):

This example was developed as a C# Console Application but you can use the code almost anywhere (perhaps a custom SharePoint configuration page is a good idea).

Add references to: Microsoft.Office.Server.Search and Microsoft.SharePoint

Add the following using statements:

using System;
using Microsoft.SharePoint;
using Microsoft.Office.Server.Search.Administration;
using Microsoft.Office.Server.Search.Query;
using Microsoft.Office.Server;
using System.Collections.Generic;
using System.Collections;
using System.Linq;
using System.Text;

I developed a function called ValidateMappedProperties which takes two input parameters. The one input parameter is the SPSite and the other input parameter is a list of MetadataProperty objects.

The ValidateMappedProperties function will iterate through the list of MetadataProperty objects and for each item query the SP Search Service to check whether the crawled property, managed property and property mapping are in place.

MetadataProperty class

First let’s look at the MetadataProperty class. This class allows me to instantiate a new MetadataProperty object and set the expected properties. It is not necessary for you to create such a class but it does make it easier to manage the input- and result operations.

namespace ValidateMappedProperties
{
    class MetadataProperty
    {
         public MetadataProperty()
        {
            this.PropertySet = Guid.Empty;
            this.MappedPropertyName = String.Empty;
            this.MappedPropertyType = 0;
            this.CrawledPropertyName = String.Empty;
            this.Verified = false;
            this.VerifiedMessage = String.Empty;
        }

        public MetadataProperty(Guid propertyset, string mappedpropertyname,  Int32 mappedpropertytype, string crawledpropertyname)
        {
            this.PropertySet = propertyset;
            this.MappedPropertyName = mappedpropertyname;
            this.MappedPropertyType = mappedpropertytype;
            this.CrawledPropertyName = crawledpropertyname;
            this.Verified = false;
            this.VerifiedMessage = String.Empty;
        }

        public Guid PropertySet { get; set; }
        public string MappedPropertyName { get; set; }
        public Int32 MappedPropertyType { get; set; }
        public string CrawledPropertyName { get; set; }
        public bool Verified { get; set; }
        public string VerifiedMessage { get; set; }

    }
}

ValidateMappedProperties function

Now let’s consider the ValidateMappedProperties function.

This function will query the Search Service Application for a list of crawled properties and a list of managed properties. It will then loop through the list of supplied MetadataProperty items and verify whether all the dependencies are in place.

public static void ValidateMappedProperties(SPSite site, List<MetadataProperty> managedproperties)
{
    try
    {
        SPServiceContext serviceContext = SPServiceContext.GetContext(site);
        SearchServiceApplicationProxy searchApplicationProxy = serviceContext.GetDefaultProxy(typeof(SearchServiceApplicationProxy)) as SearchServiceApplicationProxy;
        SearchServiceApplicationInfo searchApplictionInfo = searchApplicationProxy.GetSearchServiceApplicationInfo();
        SearchServiceApplication searchApplication = Microsoft.Office.Server.Search.Administration.SearchService.Service.SearchApplications.GetValue<SearchServiceApplication>(searchApplictionInfo.SearchServiceApplicationId);

        Schema sspSchema = new Schema(searchApplication);

        IEnumerable<CrawledProperty> _crawledProperties;
        _crawledProperties = sspSchema.QueryCrawledProperties(string.Empty, 1000000, Guid.NewGuid(), string.Empty, true).Cast<CrawledProperty>();
        ManagedPropertyCollection allprops = sspSchema.AllManagedProperties;

        foreach (MetadataProperty property in managedproperties)
        {
            property.Verified = true;
            property.VerifiedMessage = "Success";

            var crawledProperty = _crawledProperties.FirstOrDefault(c => c.Name.Equals(property.CrawledPropertyName));
                   
            if (crawledProperty == null)
            {
                property.Verified = false;
                property.VerifiedMessage = "Crawled Property '" + property.CrawledPropertyName + "' does not exist.";
                continue;
            }

            if (!allprops.Contains(property.MappedPropertyName))
            {
                property.Verified = false;
                property.VerifiedMessage = "Managed Property '" + property.MappedPropertyName + "' does not exist.";
                continue;
            }

            try
            {
                bool hasmapping = false;
                ManagedProperty mp;
                mp = sspSchema.AllManagedProperties[property.MappedPropertyName];

                List<CrawledProperty> mappedcrawledproperties = mp.GetMappedCrawledProperties(1000);
                foreach (CrawledProperty item in mappedcrawledproperties)
                {
                    if (item.Name == property.CrawledPropertyName)
                    {
                        hasmapping = true;
                        continue;
                    }
                }

                if (!hasmapping)
                {
                    property.Verified = false;
                    property.VerifiedMessage = property.MappedPropertyName + " is not mapped to crawled property '" + property.CrawledPropertyName + "'.";
                    continue;
                }
            }
            catch
            {
            }
        }
    }
    catch (Exception ex)
    {
        Console.WriteLine("Error: " + ex.Message);
    }
}

Request Method:

And, lastly the following code will illustrate how to define the list of expected MetadataProperty settings then call the ValidateMappedProperties function and then write the results to a console window:

static void Main(string[] args)
{
    string siteURL = args[0];

    List<MetadataProperty> managedproperties = new List<MetadataProperty>();
    Guid guidPropset = new Guid("00130329-0000-0130-c000-000000131346"); //this is the SharePoint columns propertyset ID.
                       
    managedproperties.Add(new MetadataProperty(guidPropset, "Division", 31, "ows_Division"));
    managedproperties.Add(new MetadataProperty(guidPropset, "Region", 31, "ows_Region"));
    managedproperties.Add(new MetadataProperty(guidPropset, "Language", 31, "ows_ContentLanguage"));
           
    using(SPSite site = new SPSite(siteURL))
    {
        ValidateMappedProperties(site,managedproperties);
               
        foreach (MetadataProperty item in managedproperties)
        {
            if (item.Verified)
            {
                Console.WriteLine("Succcess : " + item.MappedPropertyName);
            }
            else
            {
                Console.WriteLine("Failed   : " + item.VerifiedMessage);
            }
        }
    }
    Console.WriteLine("");
    Console.WriteLine("Operation completed. Press any key to contine...");
    Console.ReadKey();
}

The result will tell us where the problem lies:

I can now use this information to make the necessary changes in Central Admin and run the tool again. I can repeat this process until I get success on all items:

Now that I know all the dependencies are in place I am certain that, my custom search solution will work.

The screenshot below shows the results produced by running the example search code:

Enjoy!!