SharePoint Code Check (SPCop) first impressions

SharePoint Code Check (SPCop) which is the free SharePoint code analysis component of SharePoint Code Analysis Framework (SPCAF) was released in the first week of October 2013.

I had a quick test-drive and this is what I found..

Historically I used a combination of FXCop , CAT.NET , StyleCop and SPDisposeCheck to ensure my custom code meets industry standards and best practices for SharePoint development.

As a SharePoint Architect I am constantly looking for ways to make it easier for our team to build reliable, robust and quality code. Code analysis tools are essential in our development lifecycle and I also find that a set of analysis tools is a very valuable training mechanism for ASP.Net developers to make the transition to SharePoint development.

SPCop is a great tool for ensuring the highest possible level of quality code is produced as it consolidates the functions of many of the individual code analysis tools into a single tool. It also allows the developer to analyse the XML code in SharePoint packages like Features, ContentTypes, ListTemplates and all the other files like controls (.ascx), pages (.aspx) and master pages (.master), stylsheets (.css), JavaScript (.js) etc.

The great thing is that SPCop can be used directly in Visual Studio 2010/2012/2013 which eliminates the need for the developer to first compile assemblies and run command-line tools to analyse the code. The results of the code analysis are displayed in the Visual Studio Error List which makes it quite convenient for the developer to navigate and repair issues.

As an example I used Visual Studio 2010 to create a C# SharePoint 2010 Empty SharePoint Project and I added a Feature with a Feature Event Receiver. I did not add any code to my project. Because I haven’t written any code yet I would trust that the (almost empty) project meets the appropriate standards but when I ran SPCop I was quite surprised as to how many valid warnings I received. (Imagine how much this would help a new SharePoint developer)

The results are shown below:
 
This means that, even before I start writing custom code I can analyse the configuration of my project to ensure everything is in place.

The normal behaviour in Visual Studio is that if a developer double-click on an error or warning in the Errors List, the code window will open at the location where the issue need to be addressed. What I like about SPCop is that because it is designed specifically for SharePoint it also has the ability to navigate the user into some of the SharePoint configuration windows. As example, in order to fix the deployment scope, the user is immediately presented with the Package editor window. This is something which I haven’t seen in other SharePoint tools and it is very helpful.

One of the Visual Studio solutions which I am currently working on contains 16 Visual Studio SharePoint projects. There are times when I want to run SPCop for only a specific project but there is also the need for me to run SPCop on the entire solution and consolidate the results of the code analysis of the 16 projects into a single MS Excel report.

To run SPCop against a particular project all you have to do is to right click on the project (in Solution Explorer) and select “Run SharePoint Code Check”

To run SPCop against all projects in a solution, simply right click on the solution instead of a project.
   
 
The output can then be copied from the Errors List into Excel (for analytics, reporting, etc.)

When I speak to development teams about code analysis tools and ALM, one of the first questions I always get asked is whether it is easy to customize the rule sets (the rules which the code analysis is based on). Face it, no development house is the same and no project is the same, so there will always be a need to make small modifications to the rule sets.

In Solution Explorer (in Visual Studio), right click on your SharePoint project or solution and select the SPCop Ruleset command. This will allow you to select a pre-defined rule set, modify an existing rule set or create a new custom rule set.



The SPCAF Ruleset Editor is used to modify existing or create new rule sets.
I appreciate the Schema and Help URLs which are available for the existing rules and actually think I want to encourage my fellow developers to use these as a study guide..whooohoo!!!

Overall I am immensely impressed with this (free) tool and I have no doubt that this tool will become the core of my code quality assurance.

Technological advancements, lack of experience,  moving into the SharePoint development realm (from traditional ASP.Net), etc. are only some of the challenges we face. This along with deadline pressures, demanding projects, complex work situations, etc. make it very difficult to excel and obtain a sense of achievement so encouragement, resources, learning materials and great tools are essential for us to be successful.

Having the right tools at hand makes an enormous difference in the life of a SharePoint developer and I am grateful for Matthias Einig and the RENCORE AB team for making SPCOP available. Thanks guys!!

Related - please see my blog post: The reason why Microsoft.Office.Server.Search.dll prevented me from using SPCOP to analyse code.

Why Microsoft.Office.Server.Search.dll prevented me from using SPCop to analyse code.

I recently evaluated SharePoint Code Check (SPCop) and I quickly came across a problem whereby I was unable to run SharePoint Code Checks on certain projects.

The problem was that SPCop would initiate the build but then, before the code analysis started, the following error would be shown in the Visual Studio Output window:

SGEN : error : An attempt was made to load an assembly with an incorrect format: C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\ISAPI\Microsoft.Office.Server.Search.dll.

 
The good news is after a bit of investigation I found that the issue has absolutely nothing to do with SPCop and it is actually quite an old SGEN and SharePoint 2010 custom development problem, so if you come across the same problem, don’t associate it to the new release of SPCop … I know it is easy for a developer to download software, install it and then consider immediate problems as something which was caused by misconfiguration or that there might be a known issue with the new release..

This problem is not because of SPCop!!

This post will explain the detail of the issue and how to work around it.

Scenario:
    I have a Visual Studio 2010 SharePoint project which contains:
    • a reference to Microsoft.Office.Server.Search.dll and
    • a web service reference
      (in this example a reference to SharePoint Authentication service at                       http://SiteCollectionURL/_vti_bin/Authentication.asmx)

I am busy with development and I can successfully build my project in debug mode.
 

I then try to run SPCop.. Right click on project, select SharePoint Code Check and then click on “Run SharePoint Code Check”
  
SPCop starts to create a new build but then fails and I am unable to analyse my code :-(

Why?
Before SPCop analyses your code it builds and packages the SharePoint project to create the .wsp file which it then inspects. During the build process SGEN actually fails. It is not SPCop which fails.
I was able to build and package my project in debug mode so why is SPCop unable to build the same code?
After a bit of investigation I realized that SPCop will build the project in RELASE mode even though my project configuration is set to DEBUG mode. This is actually great because before I finalize my code I need to build and test it in release mode anyway.

Why did SGEN fail?
If your project contains a reference to Microsoft.Office.Server.Search.dll and a reference to a web service SGEN will fail to generate the serialization classes.
This is due to Microsoft.Office.Server.Search.dll not being a MSIL dll but instead being specifically compiled to run on AMD64. I guess this means that SGEN is unable to reflect on it and build Serialization classes.
If you set your project to release mode and go to your project properties page, you will most likely notice in the Output section the setting for “Generate Serialization Assembly” is set to “Auto”.

The interesting thing is that when the setting is on Auto the following applies:
    • When build in debug mode: Generate Serialization Assembly = OFF
    • When build in release mode: Generate Serialization Assembly = ON

So, this explains why we can build in debug mode but not in release mode. And, because SPCOP builds in release mode it explains why you will receive the same SGEN error.

Proper Solution
I haven’t found a solution for building projects which contains both a reference to Microsoft.Office.Server.Search.dll and a web service reference in release mode. I assume that because of this issue one would build in debug mode and then use SGEN.exe to create the serialization assembly and then include the assembly in your package.

There is a Work Around

The work around is to set "Generate Serialization Assembly" to "Off" on the project properties for the release build configuration.

 

Once you changed this setting and saved the project you will be able to successfully analyse your code with SPCop

 

My concern is that if we set "Generate Serialization Assembly" to off, XmlSerializer will generate serialization code and a serialization assembly for each type when the SharePoint web app runs for the first time (example, after server reboot or IIS reset). Because this happens only once it is not a huge performance issue, but nonetheless, I would still like to find a proper solution.

Enjoy!!

Take Control: Programmatically verify SharePoint Managed Properties

This blog post is relevant to the following common search error:

Property doesn't exist or is used in a manner inconsistent with schema settings.

If you develop a custom SharePoint 2010 solution which consumes the FullTextSqlQuery class chances are good that after deployment to a new farm you will come across the following error:

“Property doesn't exist or is used in a manner inconsistent with schema settings”

This problem occurs when your source code tries to execute custom search queries against a Search Service implementation in which the crawled properties, managed properties or property mappings which your code is dependent on, are not configured correctly.

This blog post will show you how to programmatically take control of search dependant implementations and I provide a source code example which you can use to verify that the correct dependencies in place.

Imagine you developed a web part which allows a user to provide criteria and then search the site collection or specific webs (depending on search scope) to return a specific set of field values for each result item.

You have a document library:

The search code looks like this:

ResultType resultType = ResultType.RelevantResults;
string queryString = string.Empty;

try
{
    FullTextSqlQuery fullTextSqlQuery = new FullTextSqlQuery(site);
    fullTextSqlQuery.ResultTypes = resultType;
    queryString = "SELECT Title,Division, Region, Language FROM SCOPE() WHERE FREETEXT(*, '*test* ') AND  (\"SCOPE\" = 'Demo Site Scope') AND (\"Division\" = 'Technical')";
                   
    fullTextSqlQuery.QueryText = queryString;

    ResultTableCollection resultTableCollection = fullTextSqlQuery.Execute();
    ResultTable resultTable = resultTableCollection[resultType];

    if (resultTable != null && resultTable.RowCount > 0)
    {
        while (resultTable.Read())
        {
            StringBuilder output = new StringBuilder();
            output.Append("Title:" + resultTable["TITLE"].ToString());
            output.Append(", Division:" + resultTable["Division"].ToString());
            output.Append(", Region:" + resultTable["Region"].ToString());
            output.Append(", Language:" + resultTable["Language"].ToString());
            Console.WriteLine(output);
        }
    }
}
catch (Microsoft.Office.Server.Search.Query.QueryMalformedException querymalformedexception)
{
    Console.WriteLine("Query syntax error: " + querymalformedexception.Message);
}
catch (Microsoft.Office.Server.Search.Query.ScopeNotFoundException searchscopeerror)
{
    Console.WriteLine("Search scope error: " + searchscopeerror.Message);
}
catch (Microsoft.Office.Server.Search.Query.InvalidPropertyException invalidpropertyexception)
{
    Console.WriteLine("Property error: " + invalidpropertyexception.Message);
}

You test the solution on your development server and everything works well but after you deployed to a QA or production environment your custom search code throws an error:

Property doesn't exist or is used in a manner inconsistent with schema settings.

The error does not contain information specific enough to help us identify which properties are not in place.

If you open SharePoint Central Admin –> Manage Service Applications –> Search Service Application –> Metadata Properties, we discover that some of the crawled properties which our search code rely on are not mapped to managed properties. In other cases some of the crawled properties do not even exist.

This is quite a common problem. Custom search code is dependent on specific configuration to be in place and there is always a risk during new deployments that either the provisioning code did not work properly or the SharePoint farm administrator did not configure the custom components correctly.

Obliviously we want to be SharePoint Heroes and see that our custom solutions works well after every new installation, so instead of relying on people or process let’s rather build a simple ‘success verification’ application which we can run on each new farm implementation to tell us whether all the dependencies are in place.

The source code from this post will generate the following output which will tell us exactly what the problem is:

Validate Mapped Properties (command-line tool):

This example was developed as a C# Console Application but you can use the code almost anywhere (perhaps a custom SharePoint configuration page is a good idea).

Add references to: Microsoft.Office.Server.Search and Microsoft.SharePoint

Add the following using statements:

using System;
using Microsoft.SharePoint;
using Microsoft.Office.Server.Search.Administration;
using Microsoft.Office.Server.Search.Query;
using Microsoft.Office.Server;
using System.Collections.Generic;
using System.Collections;
using System.Linq;
using System.Text;

I developed a function called ValidateMappedProperties which takes two input parameters. The one input parameter is the SPSite and the other input parameter is a list of MetadataProperty objects.

The ValidateMappedProperties function will iterate through the list of MetadataProperty objects and for each item query the SP Search Service to check whether the crawled property, managed property and property mapping are in place.

MetadataProperty class

First let’s look at the MetadataProperty class. This class allows me to instantiate a new MetadataProperty object and set the expected properties. It is not necessary for you to create such a class but it does make it easier to manage the input- and result operations.

namespace ValidateMappedProperties
{
    class MetadataProperty
    {
         public MetadataProperty()
        {
            this.PropertySet = Guid.Empty;
            this.MappedPropertyName = String.Empty;
            this.MappedPropertyType = 0;
            this.CrawledPropertyName = String.Empty;
            this.Verified = false;
            this.VerifiedMessage = String.Empty;
        }

        public MetadataProperty(Guid propertyset, string mappedpropertyname,  Int32 mappedpropertytype, string crawledpropertyname)
        {
            this.PropertySet = propertyset;
            this.MappedPropertyName = mappedpropertyname;
            this.MappedPropertyType = mappedpropertytype;
            this.CrawledPropertyName = crawledpropertyname;
            this.Verified = false;
            this.VerifiedMessage = String.Empty;
        }

        public Guid PropertySet { get; set; }
        public string MappedPropertyName { get; set; }
        public Int32 MappedPropertyType { get; set; }
        public string CrawledPropertyName { get; set; }
        public bool Verified { get; set; }
        public string VerifiedMessage { get; set; }

    }
}

ValidateMappedProperties function

Now let’s consider the ValidateMappedProperties function.

This function will query the Search Service Application for a list of crawled properties and a list of managed properties. It will then loop through the list of supplied MetadataProperty items and verify whether all the dependencies are in place.

public static void ValidateMappedProperties(SPSite site, List<MetadataProperty> managedproperties)
{
    try
    {
        SPServiceContext serviceContext = SPServiceContext.GetContext(site);
        SearchServiceApplicationProxy searchApplicationProxy = serviceContext.GetDefaultProxy(typeof(SearchServiceApplicationProxy)) as SearchServiceApplicationProxy;
        SearchServiceApplicationInfo searchApplictionInfo = searchApplicationProxy.GetSearchServiceApplicationInfo();
        SearchServiceApplication searchApplication = Microsoft.Office.Server.Search.Administration.SearchService.Service.SearchApplications.GetValue<SearchServiceApplication>(searchApplictionInfo.SearchServiceApplicationId);

        Schema sspSchema = new Schema(searchApplication);

        IEnumerable<CrawledProperty> _crawledProperties;
        _crawledProperties = sspSchema.QueryCrawledProperties(string.Empty, 1000000, Guid.NewGuid(), string.Empty, true).Cast<CrawledProperty>();
        ManagedPropertyCollection allprops = sspSchema.AllManagedProperties;

        foreach (MetadataProperty property in managedproperties)
        {
            property.Verified = true;
            property.VerifiedMessage = "Success";

            var crawledProperty = _crawledProperties.FirstOrDefault(c => c.Name.Equals(property.CrawledPropertyName));
                   
            if (crawledProperty == null)
            {
                property.Verified = false;
                property.VerifiedMessage = "Crawled Property '" + property.CrawledPropertyName + "' does not exist.";
                continue;
            }

            if (!allprops.Contains(property.MappedPropertyName))
            {
                property.Verified = false;
                property.VerifiedMessage = "Managed Property '" + property.MappedPropertyName + "' does not exist.";
                continue;
            }

            try
            {
                bool hasmapping = false;
                ManagedProperty mp;
                mp = sspSchema.AllManagedProperties[property.MappedPropertyName];

                List<CrawledProperty> mappedcrawledproperties = mp.GetMappedCrawledProperties(1000);
                foreach (CrawledProperty item in mappedcrawledproperties)
                {
                    if (item.Name == property.CrawledPropertyName)
                    {
                        hasmapping = true;
                        continue;
                    }
                }

                if (!hasmapping)
                {
                    property.Verified = false;
                    property.VerifiedMessage = property.MappedPropertyName + " is not mapped to crawled property '" + property.CrawledPropertyName + "'.";
                    continue;
                }
            }
            catch
            {
            }
        }
    }
    catch (Exception ex)
    {
        Console.WriteLine("Error: " + ex.Message);
    }
}

Request Method:

And, lastly the following code will illustrate how to define the list of expected MetadataProperty settings then call the ValidateMappedProperties function and then write the results to a console window:

static void Main(string[] args)
{
    string siteURL = args[0];

    List<MetadataProperty> managedproperties = new List<MetadataProperty>();
    Guid guidPropset = new Guid("00130329-0000-0130-c000-000000131346"); //this is the SharePoint columns propertyset ID.
                       
    managedproperties.Add(new MetadataProperty(guidPropset, "Division", 31, "ows_Division"));
    managedproperties.Add(new MetadataProperty(guidPropset, "Region", 31, "ows_Region"));
    managedproperties.Add(new MetadataProperty(guidPropset, "Language", 31, "ows_ContentLanguage"));
           
    using(SPSite site = new SPSite(siteURL))
    {
        ValidateMappedProperties(site,managedproperties);
               
        foreach (MetadataProperty item in managedproperties)
        {
            if (item.Verified)
            {
                Console.WriteLine("Succcess : " + item.MappedPropertyName);
            }
            else
            {
                Console.WriteLine("Failed   : " + item.VerifiedMessage);
            }
        }
    }
    Console.WriteLine("");
    Console.WriteLine("Operation completed. Press any key to contine...");
    Console.ReadKey();
}

The result will tell us where the problem lies:

I can now use this information to make the necessary changes in Central Admin and run the tool again. I can repeat this process until I get success on all items:

Now that I know all the dependencies are in place I am certain that, my custom search solution will work.

The screenshot below shows the results produced by running the example search code:

Enjoy!!

Nice Tool to Troubleshoot SharePoint Exceptions

When developing custom solutions for SharePoint 2007 (or WSS3.0) it might at times be very challenging to troubleshoot exceptions.

The exceptions can be caused by anything from the underlying SQL database (eg. incorrect roles assigned or a full log file) to Native SharePoint problems.
(many cases by incorrect usage of the SP object model during custom assembly development).

Many times SharePoint will only provide an user with an error description as follows:

HRESULT: 0x80004004 (E_ABORT))
Cannot complete this action – please try again.

It is therefore very hard for a developer, administrator or tester to pinpoint the problem.

The UBER COOL news is that there is a nice utility to help to monitor SharePoint diagnostic tracing.

SPTraceview on CodePlex

SPTraceView moniors in real time all SharePoint diagnostic tracing (also called ULS tracing) and can notify you using a balloon-style messages in the tray bar when any information of particular interest to you is sent (traced) by any of the MOSS services and components.

For example if you haven’t disposed all SPSite/SPWeb objects properly from your code, which also will cause unmanaged memory leak, you will see a message similar to the one below as soon as the page that contains your web part is rendered:

Even configuration errors will be shown:

Because SPTraceView processes the tracing in real time you can identify errors and events as they happen. That is as soon as you interact with the SharePoint GUI when testing/debugging your custom SharePoint solutions including web parts, event receivers, workflows and all other SharePoint technology components.

As soon as you run it, SPTraceView will start receiving all messages from MOSS and will start checking if any of them match filters defined by you. The default filter matches all messages which severity level is: Critical Event, Unexpected, Exception, Warning Event, Assert or Unassigned.

If you click on the SPTraceView icon in the windows tray you will notice the following action menus:

You can configure your settings by choosing “Configure” from the context menu. The configuration form will show up and you will be able to choose the monitored levels and what you want SPTraceView to do when any of the messages match your filter.

SPTraceView allows more complex include/exclude filters to be configured by selecting the “More Settings” button.

If you want to intercept the trace messages from more than one server in your farm you will need to have SPTraceView running in each of the servers you want to monitor

The default option is a “Local Machine Viewer” (also called “Stand Alone Viewer”). In this mode SPTraceView will only capture and process the trace messages on the local machine. This is probably what most developers need when they develop in a single server farm or virtual machine.

To view the list of historic messages, double click on the SPTraceview icon in the system tray and you will see the following window:

For more information check out http://sptraceview.codeplex.com/

SharePoint Development Tools

Microsoft Tools

SharePoint 2010 Related

Microsoft SharePoint Designer 2010 Beta (32-bit)
Download from: http://www.microsoft.com/downloads/details.aspx?FamilyID=82df15bd-16a5-460e-a7c4-22599c669bb1&displaylang=en

Microsoft SharePoint Designer 2010 Beta (64it)

Download from: http://www.microsoft.com/downloads/details.aspx?familyid=EEDA9AB1-AC53-4870-9E1C-38940343D677&displaylang=en

SharePoint Foundation 2010 Beta
Download from: http://www.microsoft.com/downloads/details.aspx?familyid=906C9F5A-6505-4EBA-BF24-95E423AC1703&displaylang=en

SharePoint Server for Internet Sites 2010 Beta
This is a Beta version of SharePoint Server 2010 for Internet Sites, Enterprise that enables extranet and internet sites using the full power of SharePoint Server 2010, the business collaboration platform for the Enterprise and the Web.
Download from: Http://www.microsoft.com/downloads/details.aspx?familyid=77195785-0282-4ACC-B6F4-7569E1011581&displaylang=en


SharePoint 2007
(section under construction)


Other Tools

Reflector:
.Net Reflector allows you to easily view, navigate, and search through, the class hierarchies of .NET assemblies, even if you don't have the code for them. With it, you can decompile and analyze .NET assemblies in C#, Visual Basic, and IL.
Download from: http://www.red-gate.com/products/reflector/