Skip to main content

Wouter

Go Search
Home
Contact Me
  

 ‭(Hidden)‬ Admin Links

Advanced cache invalidation using HTTP

One common approach to enabling high performance web applications is to cache data that does not change frequently. SharePoint is no different in this and provides a host of caching options including blob caching, list item caching, object caching and of course the normal ASP.NET cache. This last caching infrastructure has the focus today. For that entire infrastructure I want to focus on only a small part of the equation; cache invalidation. Usually my preferred approach is to invalidate the cache when the data changes and reload the cache when the data is accessed. In essence this is not so difficult. Simply call Cache.Remove and later on do a Cache.Insert right? While this works for small deployments, as soon as you start to use some of the scaling features of SharePoint you bump into issues. What if you have extended your web application into multiple zones? What if you have multiple web servers? In both cases you will have to deal with multiple caches since each server / zone combination utilizes its own cache. The Cache.Remove code would only invalidate the cache on a single server, in a single zone! This article provides an elegant solution for invalidating a cache across the entire farm.

Code for this article.

It's about context!

At the core the issue is simple. You have multiple HTTP contexts, one per server/zone, each with a separate cache. Your code usually executes in only one of them. We want the code to execute in all contexts so that the cache gets invalidated everywhere. One approach that is almost entirely built-in to the caching infrastructure of ASP.NET is to use a dependency on a file. In ASP.NET you can make use of the native Windows ability to notify your application that files have changed. The ASP.NET cache makes use of this ability to allow cached data to be invalidated when this happens. Typically the scenario is that you load a file from disc into the cache and then monitor changes on that file to invalidate the data. However, you can also make use of this native ability to invalidate the cache in every zone on every server by placing the file in a file share and configuring the cache with a UNC path. You can have each server listen to the same file and hence update all caches at once. The upside to this approach is that it is already implemented for you. There is very little code required. There are also slight downsides to this approach. You will have to create a file share with your solution, something you cannot do natively with WSPs. You will have to configure the location of this share so you code call Cache.Insert with the right parameters. You will have to think about on what server to create the file share. This makes one server more special than another. This could potentially become an issue when changing the farm topology. Next, some organizations I work for have pretty strict rules around creating file shares, meaning, that it is not allowed to create any. One last note about this approach is that granular refreshes become more difficult. You'll want to differentiate between the various pieces of data you cache and only invalidate some of it. You could for instance use one file per type of cached data or similar constructs to provide the granularity. It's not so pretty though!

A new approach to an old problem

Recently I have developed another approach which alleviates many if not all of these issues. The approach involves notifying each HTTP context of a change in the cached data by simply sending a request to a known HTTP end point that runs within that context. The code running at that end point can simply perform the Cache.Remove to remove the changed data. There is one big concern with this approach and that is load balancing. If you have multiple front end servers and you send your HTTP request to a host such as http://intranet or http://www.contoso.com how do you ensure you hit all the servers? Through load balancing all your requests might go to a single machine! The simple solution to this issue is to craft a better HTTP request. You can send an HTTP request to the IP address of a single server to work around any load balancing. By applying the HOST header you can still target the correct IIS web site hosted on the server and hence call into the right context.

Let's go!

In this demo the first step is to create a well-known HTTP end point for your cache invalidation code to hit. A simple HTTP handler will do just nicely. Notice how we can simply access the HTTP context and the cache from the HTTP handler.

public class InvalidateCacheHandler
    : IHttpHandler
{
    public bool IsReusable
    {
        get { return true; }
    }

    public void ProcessRequest(HttpContext context)
    {
        context.Cache.Remove("TestKey");
        context.Response.StatusCode = (int)HttpStatusCode.OK;
    }
}
Now that we have our end point defined the next step is to ensure that the HTTP handler is actually called on each server and for each extended zone. For this we need to retrieve the host name or IP address of each server, all the zones and craft a HTTP request for each of combination of the two.

Given a web application, we can start to build out this code.

void InvalidateCacheForWebApplication(SPWebApplication webApp)
{
}

First, we use the web application to access all the servers in the farm. This gives us the host name of each server, which might be an IP address. Although not entirely necessary the code translates each host name into an IPv4 IP address using the power of a little .NET and the System.Net namespace.

IEnumerable<IPAddress> serverAddresses = webApp.Farm.Servers
    .Where(server => server.Status == SPObjectStatus.Online)
    .SelectMany(server => Dns.GetHostAddresses(server.Address))
    .Where(ip => ip.AddressFamily == AddressFamily.InterNetwork);

Next, enumerate over each zone and over each IP address. This is the matrix to where we must send HTTP cache invalidation calls. Note that we might hit a single server twice if it has more than one valid IP address. Oh well…

foreach (SPAlternateUrl zoneUrl in webApp.AlternateUrls)
{
    foreach (IPAddress ipAddress in serverAddresses)
    {
    }
}

Inside the double loop we craft the HTTP call. We want to send the HTTP request directly to a specific server. The server will then identify the right IIS web site using the HOST header. This means we must setup the host header for the HTTP request, which, I am afraid, is not supported in .NET 3.5. The common workaround is to use the HttpWebRequest.Create method to set up the HOST header and then use a WebProxy class to configure the destination for the request. This combination will send the request to the right server and to the right IIS web site on that server.

UriBuilder ipUri = new UriBuilder(Uri.UriSchemeHttp, 
    ipAddress.ToString(), zoneUrl.Uri.Port);
UriBuilder hostHeaderUri = new UriBuilder(zoneUrl.Uri);
hostHeaderUri.Path = "_layouts/CooleCacheCode/InvalidateCache.ashx";
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(hostHeaderUri.Uri);
request.UseDefaultCredentials = true;
request.Proxy = new WebProxy(ipUri.Uri, false);

After having crafted the HTTP request we need to, well, request the data. We can use the HTTP response code to see if the server answered and therefore has invalidated the cache.

try
{
    HttpWebResponse response = (HttpWebResponse)request.GetResponse();
    if (response.StatusCode != HttpStatusCode.OK)
    {
        // something wrong 
    }
}
catch (WebException)
{
    // something really wrong 
}

Kickass! We now have the code to invalidate caches across our entire server farm. A big benefit is that changing topologies means exactly no change to this code. There is no chance of someone accidentally deleting the file share and we now also have the power of adding granularity to the code by using HTTP query strings to identify the cached item that should be invalidated. All in all a pretty sweet solution!

Hope it helps!

Declarative Fun: Provisioning Web Part Connections in sandboxed solutions

One of my current projects involves the new sandboxed deployment model. It is pretty fun and challenging to translate requirements into testable code. With the promise of deploying into environments like Office 365 there is a strong will to work around the limitations of the sandbox. Of course without resorting to solutions based on a senseless full trust proxy. One of the limitations is that there is no SPLimitedWebPartManager class. This means it is not possible to modify the Web Parts on a page using code. This is a bit of an issue since I want to deploy two related Web Parts that are connected together using a Web Part Connection. Overcoming the issue was fun enough for me to write about. So here goes. Web Parts, Web Part Connections and all fit for sandboxed deployment.

Find the sample project here.

Provisioning Web Parts

First step is to provision a Web Part to the page. Since the imperative approach using the SharePoint object model is not available in the sandbox we can only use declarative XML. My Web Parts are deployed to a page using a <Module> element which provisions a bunch of <File> elements, each of which contain various <AllUsersWebPart> elements.

<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
  <Module Name="Module1">
    <File Path="Module1\default.aspx" Url="demo.aspx">
      <NavBarPage ID="1002" Name="Demo page" Position="End" />
      <AllUsersWebPart ID="testProvider" 
                       WebPartZoneID="Left" 
                       WebPartOrder="1">
        <![CDATA[...]]>
      </AllUsersWebPart>
    </File>
  </Module>
</Elements>

This approach allows you to deploy a custom page which has preconfigured Web Parts. One note is that you cannot use this approach to deploy new Web Parts to existing pages since you will blast the existing page out of the water (try changing demo.aspx to default.aspx and see).

Provisioning the connection

During initial development the project ran fine with just loose Web Parts. After a while the need arose for connected Web Parts as a means to show master/detail data. So the quest for a declarative approach began and quickly turned up the new WebPartConnection element on a <File>. It also turned up that no-one has actually bothered to document it yet.

Using the <WebPartConnection> you now seem to have a declarative way to define connections between Web Parts. I definitely prefer XML over code especially since it also allows me to define connections for Web Parts that I deploy as part of a sandboxed solution

Note that I do NOT mean that Web Parts who's codebase is defined within an assembly deployed as part of a sandboxed solution. But, I mean that you can deploy a Web Part to a page whose codebase is part of a normal full-trust assembly, such as all built-in Web Parts types (XsltListView, Image, etc..).

There are generally only four attributes you are interested in. The ID of the provider and the ID of the connection point within that provider. The same goes for the consumer. Here's a sample that I use for my custom testing class that you can find in the attached demo project.

<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
  <Module Name="Module1">
    <File Path="Module1\default.aspx" Url="demo.aspx">
      <AllUsersWebPart ID="testProvider">...</AllUsersWebPart>
      <AllUsersWebPart ID="testConsumer">...</AllUsersWebPart>
      <WebPartConnection ID="testConnection"
                         ProviderID="testProvider"
                         ProviderConnectionPointID="providerID"
                         ConsumerID="testConsumer"
                         ConsumerConnectionPointID="consumerID" />
     </File>
  </Module>
</Elements>

Getting sexy… Including a Web Part Transformer

In my later code I want to connect a row provider to a filter consumer. Connecting these Web Parts through the browser means that you get to fill in a dialog such as the following. You can put in the field mappings between the provider and the consumer. We need to do the same using the <WebPartConnection> and luckily it allows you to do it to!

The not so sexy part is the amount of current documentation on the mystic <WebPartTransformer> element. Other than it being a descendant of the <WebPartConnection> which allows you to point to an assembly / namespace there is nothing know to the human race outside of those in possession of the SharePoint kernel (Robin, send me a copy ok?). Not even the Microsoft.SharePoint DLL itself gives a hint of this hidden jewel. It took a bit of testing but I figured out how to it works.

The <WebPartTransformer> element points to an Assembly / Namespace because it is used like a <%@Register %> directive in ASP.NET. You point to a namespace and then use ASP.NET page markup to define the transformer. Since the ASP.NET markup uses < and > tags you need to throw it all in a CDATA section. It also appears that you must use the WebPartPages control prefix to make it work as expected. Here's a small sample taken from the POC.

<WebPartConnection ID="testConnection2"
                   ProviderID="testProvider2"
                   ProviderConnectionPointID="providerIDRow"
                   ConsumerID="testConsumer2"
                   ConsumerConnectionPointID="DFWP Filter Consumer ID">
  <WebPartTransformer Assembly="Microsoft.SharePoint, ..."
                      Namespace="Microsoft.SharePoint.WebPartPages">
    <![CDATA[<WebPartPages:SPRowToParametersTransformer 
                  ConsumerFieldNames="URL" 
                  ProviderFieldNames="Test" />]]>
  </WebPartTransformer>
</WebPartConnection>

Voila. Web Parts with connections and even with transformers, all declarative and available inside the sandbox deployment model.

Hope it helps!

 

 

 

 

Bulldozing your SharePoint install

About a year ago I wrote about a PowerShell script that uninstalls all your solutions. It's great for refreshing your development laptop after an intense week of training or learning. It's also good for inflicting serious pain on your production farm would you be silly enough to run in there. I have upgraded the script a bit to make better use of SharePoint infrastructure for managing solutions (using the SharePoint Admin Service).

Here's the script V2.

Add-PSSnapin "Microsoft.SharePoint.PowerShell" -ErrorAction SilentlyContinue

$service = Get-Service SPAdminV4
if($service.Status -ne "Running")
{
    Write-Host "SharePoint Admin Service not running. Trying to start"
    try
    {
        $service.Start()
        Write-Host "SharePoint Admin Service started"
    }
    catch
    {
        Write-Host "Error starting the SharePoint Admin Service. Exiting."
        return
    }
}

$deployedSolutions = Get-SPSolution | ?{$_.Name -ne "powerpivotfarm.wsp"}
if($deployedSolutions)
{
    foreach($solution in $deployedSolutions)
    {
        if($solution.Deployed -eq $true)
        {
            Write-Host Uninstalling ($solution.Name)
            if($solution.ContainsWebApplicationResource)
            {
                $solution | Uninstall-SPSolution -AllWebApplications -Confirm:$false
            }
            else
            {
                $solution | Uninstall-SPSolution -Confirm:$false
            }
            while($solution.Deployed -eq $true)
            {
                Write-Host "Waiting for solution to uninstall" -Verbose
                Sleep 2
            }
        }
        Write-Host Deleting ($solution.Name) 
        $solution.Delete()
    }
}

   

Hope it helps!


 
 

Creating a Silverlight application for SharePoint in 10 smalls steps

In this post I will show you how to create a Silverlight application that is hosted in SharePoint. Ease and grace are the keywords of course, no funky stuff.

1. Create a SharePoint project

First you need to create a SharePoint project. You can choose most of the templates. I typically use 'Empty SharePoint Project'. I could not help but name my project SilverPoint, which sounds better that ShareLight.

2. Configure the trust level

After creating the project you will hit the SharePoint Customization Wizard. The approach in this blog post allows you to use either the sandboxed or the farm solution. I will create a farm solution since I want to deploy my Silverlight XAP file into the 14\TEMPLATE\LAYOUTS\ClientBin folder. A sandboxed solution would not be allowed to deploy files into this directory.

3. Add a Silverlight project

Next add a Silverlight Application project to your solution. I will not add any custom code to the Silverlight project at this point since we'll focus on getting the Silverlight application into SharePoint.

4. Configure the deployment site

After creating the Silverlight Application you will hit the New Silverlight Application wizard. You can choose not to host the Silverlight application in a testing web site since you will use SharePoint to host it.

5. Add a Web Part to the SharePoint project

The Silverlight application needs to be loaded onto a page in SharePoint. An easy way to achieve this is to use a Web Part. Add a normal, non-visual, Web Part item to the SharePoint project. Since this Web Part will host the DemoApp Silverlight application I chose to name my Web Part DemoAppPart.

After adding this Web Part your project should be similar to the following.

6. Configure Project Output References

In the Solution Explorer, right-click the DemoAppPart folder and choose Properties in order to show the properties of this project item. In the Properties window that appears select the Project Output References property and press the ellipsis button (…) to edit the output references. Add a new output reference and configure it with the following settings.

Deployment Type:

TemplateFile

Project Name:

DemoApp

Deployment Location:

LAYOUTS\ClientBin\<projectName>

The <projectName> in the Deployment Location is only there to reduce the chance of collisions on the server with other people's projects.

7. Delete the Web Part code

There are a few approaches to loading up the XAP that we now deploy into the ClientBin folder. You could create your own Web Part that renders out an <object> tag. However, SharePoint already ships with a Silverlight Web Part that does just that. It's called the… SilverlightWebPart. So, that means we don't have to write any code but instead can deploy this Web Part. To not deploy any code is simple. Just delete the DemoAppPart.cs file

8. Reconfigure the Web Part definition

The .webpart file points to the class that SharePoint should load when adding the Web Part to a page. Currently the .webpart file is pointing to the class that was just deleted. To rebind the .webpart file to the built-in Silverlight Web Part you change one line in the .webpart file. Change the value for the <type> element to point to the Silverlight built-in Web Part.

From

<type name="SilverPoint.DemoAppPart.DemoAppPart, $SharePoint.Project.AssemblyFullName$" />

To

<type name="Microsoft.SharePoint.WebPartPages.SilverlightWebPart, Microsoft.SharePoint, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" />

9. Bind the XAP file

The last step is to make sure that the Web Part loads the correct XAP without the user needing to specify this. You can make use of the settings in the .webpart file to provide this configuration value.

<property name="Url" type="string">
~site/_layouts/clientbin/SilverPoint/DemoApp.xap

</property>

10. Deploy and test

Hit F5 to deploy the solution and test the new Web Part.

Yay!

Get ready for THE SharePoint Conference!

Only a few more weeks until the SharePoint Connections conference visits Europe again! I hope you have bought a ticket because there are few opportunities to interact directly with so many real world experts on the platform we all know and love! Get ready to be introduced to new things during the general breakout sessions, or go deep and register for a workshop in the field you are most interested in.

Personally I am looking forward to seeing Scot talk about BCS and I'll be sure to visit Ted's talk about Claims (which he claims to be aware of J). I'll also deliver a breakout session and I'll go deep in architecture and coding during my SharePoint Development Excellence workshop!

Hope to see you there!

About the CKSDEV and Microsoft PowerTools Sandboxed Web Part

Early last year I thought it was fun to rise to the challenge of creating a visual (designable) Web Part that could be deployed into the SharePoint 2010 Solution Sandbox. Mostly as a fun programming exercise and also because I wasn't sure how it could be achieved. In the end I got a pretty decent working solution that is now part of CKSDEV.

A while ago Microsoft also released a set of productivity extensions for the SharePoint sandbox. The PowerTools contain two new features; sandboxed compilation and… a Visual Web Part. Both features are also present in CKSDEV> A duplication which is best avoided. The CKSDEV team quickly decided to drop our versions in favor of Microsofts. Being the author of both features I want to go over the differences between the two and the reasoning behind dropping our features.

(Note that we only drop the template needed to create new sandboxed Web Parts, existing code will remain to work)

Sandboxed compilation

The motivation for this feature in CKSDEV was to provide additional compile-time information because certain methods are not available in the sandbox and normal compilation still allows you to call the unavailable code. My approach sucks a bit. It is only partly integrated into Visual Studio. When you hit 'Compile for Sandbox' the project references are updated to point to the sandboxed object model, a compile is forced and next the references are put back into place. This means you have to check out the project for this to work, and it modifies your project too. (It did only take a few lines of code thoughJ)

The Microsoft approach is integrated much further into the build process of Visual Studio. No fussing with swapping project references. All in all it is a much cleaner model. When you compile it either compiles against the sandbox or it compiles normally, depending on project settings. It also enables the red squiggly lines that appear as you type your code to indicate something will not compile when you would hit F5. This is something that we as the CKSDEV team cannot achieve with the current approach. So, Microsoft's version clearly wins in terms provided level of functionality. Luckily, we only have to remove our button and drop the code. There are no runtime dependencies within your actual projects.

Sandboxed Visual Web Part

This one is a bigger story. Both CKSDEV and the PowerTools provide their own template for creating a Sandboxed Visual Web Part. There is only one similarity in the approach. Both versions have no runtime requirement for including our DLLs. Everything you need is either generated or typed by you, but that is pretty much it. Let me begin with explaining how the CKSDEV version works (which was first to hit the market! They'll never take that away from me J).

Basically both approaches parse the ASCX file when you save it, and the generated code is added to your project. That way parsing does not take place at runtime outside of the sandboxed environment.

One of the biggest challenges for CKSDEV was to find a way to parse a User Control (ASCX) file and push the generated code into the project. The ASP.NET User Control parser is hidden from the public so calling it directly is not an option. There are a few ways to work around this issue. The workaround applied by CKSDEV was to call the aspnet_compiler.exe tool with the –debug flag. This forces the code generation to take place, obviously with too details like line pragma's and other unnecessary stuff. We then scoop up the generated code and push the code into the project. The overall code structure resulting from this approach is shown in the following graphic. Blue means generated, green is stuff you type.

Lots of blue stuff for little green stuff!

The Microsoft approach to parsing the ASCX is to use the ClientBuildManager class provided by ASP.NET. I also wanted to use this class but getting it to work exactly as I wanted involved lots of CodeDom and more work than I wanted to do. The ClientBuildManager is able to generate code into a CodeDom CodeCompileUnit which you then transform into code using an ICodeGenerator from .NET. This approach is more involved and requires the generated to be buffed and polished. We choose not to learn too much CodeDOM, but obviously that knowledge is already inhouse somewhere in Redmond. The structure is much different though. Since you get back a CodeCompileUnit (basically a parse tree) you can modify the tree to suite your needs. One cool effect the Microsoft approach then has is that your resulting code does not use the UserControl base class but directly creates a Web Part. The benefit is less classes to generate and maintain. The graphic is as follows.

The difference for you as a user of the template is primarily visible in the code-behind. For the CKSDEV approach the method that instantiates the generated controls is called automatically by the UserControl base class in the Init phase of the page lifecycle. For the PowerTools edition this method is called by you. Other notable differences is that the CKSDEV approach allows you to use <script runat="server"> blocks differently from the PowerTools. The reason for this is that the PowerTools use a single class and CKSDEV has a class hierarchy. Say that you use a script block to override the OnLoad method. If you do the same in the code-behind you'll have two methods with the same signature, going in the same class. For the CKSDEV version the OnLoad would be added in a different class and hence you can combine <script runat="server"> blocks with normal code-behind without thinking about it.

Supporting the CKSDEV Sandboxed Web Part

Given the fact that you may have already invested in the CKSDEV Sandboxed Web Part we are not happy to remove the feature. To ensure that we don't break your build we choose to keep the internal code in place, but we'll remove the Visual Studio template. This means you'll no longer have an easy time creating new CKSDEV Sandboxed Visual Web Parts, but the existing ones will keep on working as is. We do plan to provide a conversion tool to convert the CKSDEV version to the Microsoft version. But given the large differences in the approach there are a few technical difficulties that take a bit of time to overcome. (we'll for instance need to swap out your base class of UserControl in favor of WebPart). One thing that we'll not likely be able to achieve is a conversion tool that will not break existing deployed Web Parts, so there's still some discussing to be done how to handle that.

Hope this clarifies some of the differences between the PowerTools and the CKSDEV version. Of course CKSDEV will continue to provide you with much needed enhancements to the Visual Studio 2010 out of the box experience, these two will just not be part of that).

Did I say you should get CKSDEV RIGHT NOW! J

Naming your SharePoint accounts

One of my big focusses when writing software is not to get it to just work, but to get it to work correctly. Hence I spend a ton of time learning about different approaches to solving my SharePoint needs. One area that I focus on in a similar fashion is building my SharePoint VMs. Back in the 2007 days I started building my VMs according to the least privilege principle, e.g. not making everything run under the admin account. Now, if you do that, you must name your service accounts. It suffices to say I take the same level of passion I have on coding approaches to the world of IT Pro stuff. I come across a ton of funky account names that make me wonder how discoverable it all is. Stuff like SPSVCAPP. Right, I cannot even pronounce it, let alone remember.

So, without further ado, here's my list of account names I typically use when installing SharePoint. Usually I create a separate OU in active directory to group them, or I use the existing OU when running on 2008R2. Note that this is for SharePoint 2007.

Account

Goal

Configured when

SPSqlService

Run SQL Server

Installing SQL Server

SPAdminService

Run Central Admin

Installing SharePoint

SPSharedService

Run Shared Services

Creating the SSP

SPSearchService

Run WSS / MOSS Search

Configuring search

SPCrawlerService

Crawl SharePoint content

Configuring search

SPWebService

Run content application pools

Creating app pools

As you can see I have a naming convention with the following characteristics:

  • Readable, not just acronyms. Don't make me think rule at play here
  • Prefix with SP. Just to stuff it all together a bit more and to indicate that they are for SharePoint
  • Postfix with Service. Indicate that you should not use these accounts for actual logging in, except for the SPAdminService.

Personally I especially like the SPWebService account. You *must* be a SharePoint geek to enjoy that one right? It is the service account that spoons up webs out of the database, hence, SPWebService. Some name it SPWorkerProcess, also fine I'd say.

Hope it helps, and have fun naming your accounts.

Presenting at SharePoint Connections Autumn

Just got word that my proposal for the SharePoint Connections event was accepted. I'll be presenting on the new Access Services capabilities in SharePoint 2010 as well as presenting a full day post conference on SharePoint development techniques. Awesome! Personally I was quite amazed to learn how much cool stuff you can build on Access so I am very much looking forward to spreading the news. Many of you perhaps know me as a developer, and as it is with many developers, Access was a loooooong way down the Technologies-I-Would-Use stack, but with the pretty awesome server side presence in SharePoint….. Next up: post conference. This will be an ideal way for you to pick up on development techniques that are vital to any project, Foundation or Server, small or large. We'll go deep on a bunch of topics, all Foundation features so usable anywhere. Come visit and enjoy learning SharePoint even better!

Service Applications 104 – Across the wire

This is the fourth part of a series on Service Applications. The following has already been posted:

Find the code for this sample on my CodePlex demo portal under the Navigation Service with WCF Proxy release.

Introducing WCF as an offloading mechanism

In the first three parts of this walkthrough you were introduced to the concept of service applications, on the basic steps to create one and finally on how to add a proxy layer to provide an extra level of indirection between the application and the consumers. In this blog post will show how to move the proxy to the next level by introducing WCF as a mechanism to push incoming requests to backend application servers.

The earlier three blog posts laid down the foundation for this first full-blown service application. A few extra components need to be added. Another few existing classes need some extra attention. I also threw in a testing page that is accessible from the Ribbon on the application management page. Hopefully this enables you to have an easy time seeing whether your code actually works (mine does, yay!)

The following is a list of todo items for moving the last application with the simple proxy to one that is enabled by WCF

  • Start making use of new base classes for the service, service proxy, service application and service application proxy.
  • Create a new service instance class to represent the service on a particular backend server
  • Add a WCF contract interface
  • Add the WCF service and client configuration
  • Implement the WCF client and server code

That's it. A bit involved as you can see from the list of items that need to get done, but we can just start at the first and work your way down the list.

Introducing new base classes

Now that we are moving towards a WCF enabled system there are a lot of new base classes to use. Earlier we had the SPService, SPServiceProxy, SPServiceApplication and SPServiceApplicationProxy. For working WCF you can swap all these out with their counterparts. SPIisWebService, SPIisWebServiceProxy, SPIisWebServiceApplication and SPIisWebServiceApplicationProxy.

The SPIisWebService

Where earlier you derived your service class from SPService, you now need to move towards the SPIisWebService. Not that much difference in implementation. The only code that goes in this class is internal. It maintains a bunch of settings such as default endpoint names and network port numbers.

The SPIisWebServiceProxy

The replacement for the SPServiceProxy is the SPIisWebServiceProxy. Examine the class with .NET Reflector and notice that there is absolutely nothing it does extra. The reason it exists is likely to achieve some parity between all relevant classes (all SPIisWeb… classes).

The SPIisWebServiceApplication

The service application for WCF is a bit more complex in its internals. It maintains references to the IIS application pool used to serve incoming requests. There are many IIS related settings such as endpoint configuration. It iments a configurable backup strategy for the application, proxies and any databases you may have. You will also find a small model for configuring service security, allowing you to implement your own permission types. So, this class is definitely worth a further look after you have gotten your basic application going.

The SPIisWebServiceApplicationProxy

Closing the line of new base classes is the proxy for the service application. Again you will find support for easily integrating with the backup functionality. The proxy also serves as a place to store what address the proxy will connect to when used.

Updating the service and service proxy

When it comes to code changes, let's begin with the service and service proxy since these two components have the least going on. They are only there to support the larger infrastructure and do not contain any WCF specifics. The service itself only has a new base class. The rest of the changes are specific to the service proxy.

The first change you find in the service proxy is that you need to add the SupportedServiceApplication attribute. This attribute links the service proxy to a service application. You can add as many attributes as the number of service application types that your service supports. The attribute is used to fill the user interface presented as part of the Connect menu. After you enter the service URL you will be shown a list of service applications you can connect to. This list is created using the information from this attribute. Since our service will now truly support connecting proxies with applications you'll need to add this attribute. Earlier editions of the proxy didn't need this since we weren't using WCF yet.

How it works is that you will open the Connect menu and choose a service application type to connect to. Under the covers the value in this attribute is read. In the Connect to a Remote Service Application dialog that appears you will enter the URL of a farm, hit next. On the second page of the connection wizard a filtered list of available service applications is provided for you to choose from. This list is filtered based on the attribute's value, allowing filtering on the application type and the application version.

[SupportedServiceApplication(
    "f46c5a6a-0f73-4c1e-b11b-c2ffb70c19f9"
    "1.0.0.0"
    typeof(NavigationApplicationProxy))]
The next major change that you find in the service proxy is an implementation of the IServiceProxyAdministration interface. This interface is used by SharePoint during automated processes as well as to fill the Connect menu.


The implementation of this interface is a huge no-brainer. All the methods you need to add are one-liners.

Implementing the WCF Service
Moving to a client / server model obviously requires you to implement the server (hint, the client too!). In many of the service applications that you find inside SharePoint you find a common approach. Most of the service application objects implement the WCF service. For the Navigation Service we'll do the same. Although technically not necessary, we'll begin with defining a service contract. For the demo's sake it is a bit sparse in terms of the functionality that we'll provide to consumers.

[ServiceContract]
public interface INavigationApplication
{
    [OperationContract]
    string GetNavigationInformation();
}
The next step is to change the base class for the NavigationApplication. Instead of deriving from SPServiceApplication we now extend SPIisWebServiceApplication. This base class requires you to implement two properties that point to the .SVC file that is used as the endpoint address. You will deploy the .SVC file to the WebServices folder which is a new folder found directly under the SharePoint 2010 installation path.

public class NavigationApplication
    : SPIisWebServiceApplication
    INavigationApplication
{
    protected override string InstallPath
    {
        get 
        { 
            return SPUtility.GetGenericSetupPath(
                @"WebServices\CodeCounsel.SharePoint.NavigationService"); 
        }
    }

    protected override string VirtualPath
    {
        get { return "Navigation.svc"; }
    }
}
Under the covers the InstallPath and VirtualPath values are combined to locate the .SVC file. This file is deployed as part of your project. In the .SVC file there is a pointer to the NavigationApplication class as well as to a new class which task it is to instantiate the WCF service host.

<%@ServiceHost

language=C#

Service="..."

Factory="..."%>

The factory is specific to your scenario. There is a factory for SOAP based services, for REST services and for ADO.NET data services. In the factory class there is only one specific bit. It activates claims security for the service.

public class NavigationApplicationHostFactory
    : MultipleBaseAddressBasicHttpBindingServiceHostFactory 
{
    public override ServiceHostBase CreateServiceHost(
        string constructorString, Uri[] baseAddresses)
    {
        ServiceHost host = new ServiceHost(
            typeof(NavigationApplication), 
            baseAddresses);
        host.Configure(SPServiceAuthenticationMode.Claims);
        return host;
    }
}
The final part on the server side of the service application is the web.config that defines the service endpoints. You can specify details such as the communication protocol, TCP, HTTP, HTTPS etcetera. There are a huge amount of settings you can configure but luckily you don't need them all. The easiest is to copy an existing web.config, for instance from the Topology service and modify that to suite your needs. Just make sure you set the <service name=""> to the actual name of the service class (namespace + classname) and do the same for the contract.

Implementing the WCF client

Now that we have a server let's also create the client. The service application proxy will serve this task. When you examine existing implementations there is quite a lot of juggling with delegates to prevent duplication of code. In the Navigation Service demo however I chose to implement all functionality inline. Not smart, but better to read.

The first change in this class is that you can and most likely want to add a load balancer class. SharePoint provides a basic load balancer that chooses between available endpoints in a round robin fashion. You can also not use a load balancer, up to you. But it's only a few lines of code to add, so, add it already!

Since the WCF proxy class is mostly the same as the previous edition without WCF I'll not reiterate all the existing methods. Let's focus on the how to call the service application running on a backend server. The first thing you'll need to do is open the .NET configuration file that contains similar information as the server configuration. Endpoint and security data is what you'll find in the client.config file. You can make use of the .NET configuration APIs to open the correct client.config, which will be provisioned to the WebClients folder under the SharePoint 2010 installation path. Based on this configuration data you will instantiate a ChannelFactory for the service interface you added earlier. The ChannelFactory wants to know the specific endpoint that you want to use. In this sample it is hardcoded, but you could also determine the correct endpoint based on the URL that the proxy is connected to (URL starts with HTTPS:// it'll be the HTTPS endpoint right?) Next, use the load balancer to open a session and open the channel to the backend. The last step is to actually call the WCF service and return results.

public string GetNavigationInformation()
{
    string channelConfigPath = SPUtility.GetGenericSetupPath(
        @"WebClients\CodeCounsel.SharePoint.NavigationService");
    Configuration clientConfig = OpenClientConfiguration(channelConfigPath);
    ChannelFactory<INavigationApplication> factory =
        new ConfigurationChannelFactory<INavigationApplication>(
            "http", clientConfig, null);
    factory.ConfigureCredentials(SPServiceAuthenticationMode.Claims);

    SPServiceLoadBalancerContext context = _loadBalancer.BeginOperation();
    try
    {
        INavigationApplication channel = 
            factory.CreateChannelActingAsLoggedOnUser<INavigationApplication>(
                new EndpointAddress(context.EndpointAddress));
        return channel.GetNavigationInformation();
    }
    finally
    {
        context.EndOperation();
    }
}
In terms of doing things better you'll likely want to do some caching since instantiating channel factories and opening / parsing configuration files are expensive pieces of code.

Enabling the service on backend servers

Now that we have both a WCF server and a WCF client we also need a way to specify on what servers in our farm we want to run the service. This task falls in the hand of the SPServiceInstance, which is basically a class that couples your service with a specific server. What you need to do is instantiate an instance of this class for each server in the farm that you may want to run your service on. You can then go to the Services on Server page and start your service. This action will provision your service instance on that server and make it available for WCF requests.

Testing your work
The last step is to test the new WCF proxy based implementation. To make this easier to do I added a menu item to the Ribbon. It opens a testing page which calls the service application via the proxy.


Hope it helps!



















Service Applications 103 – Simple Proxy Big Results

This is the third part of a series on Service Applications. The following has already been posted:

Find the code for this sample on my CodePlex demo portal under the Navigation Service with Proxy release.

Working with proxies

The first sample Service Application that I showed only brings a certain level of usefulness. The first demo application did not use a proxy and without a proxy you are functionally stuck with having only a single Service Application instance to service all the web applications in the entire farm. This stems from the fact that you can't configure the Web Application binding to Service Applications when you do not create a proxy. So, quite quickly you'll want to move away from this simple approach and tack on a proxy layer too. For some added background, a service proxy is a component that relays calls to your application on behalf of a consumer. It allows you to implement an extra level of indirection between caller and callee. The most common implementation of this pattern is to use the proxy as a WCF client and expose the application as a WCF service. That way you can offload incoming requests to backend application servers. Great stuff! You can also create a Service Application / Proxy combination without using WCF. Simply call methods on the application directly from the proxy instead of instantiating a WCF client and going through that route. You still get to enjoy the benefit of binding web applications to specific instances of the service, but obviously you lose out on pushing requests out to the backend.

In this blog post I will show how to add this second layer of indirection to the Navigation Application introduced in part 2. No WCF proxies just yet though. I'll cover that in the next post.

New components in this version

In my previous post I showed the Navigation Application and the two primary components. You were introduced to the SPService, representing a farm-wide service and the SPServiceApplication which implements that farm wide service for specific web applications. When moving towards a proxy based implementation you get to add a few extra pieces to the puzzle, most notably a service proxy and a service application proxy.

The SPServiceProxy class allows SharePoint to know about what the proxies in your specific solution. This is similar to how the SPService provides information about the service applications that you have. Next is the service application proxy, which is a proxy to the service application itself, instantiated by the service proxy.

Service Proxies

The service proxy in the new demo application is doing two things. First of all it serves as a factory for new service application proxies. Next, it provides a known point for SharePoint to ask about what proxies you can create, for instance to populate the Connect menu on the Service Application Management page.

Service Application Proxies

These then, are the components that forward calls to the Service Application they are a proxy for. How they do this is to be decided however. In the simplest case you just look up the correct SPServiceApplication instance by ID and call methods directly on it. The harder case uses WCF as an intermediate layer. Doing so is not as hard as exposing a WCF service in SharePoint 2007. You now benefit from built-in support for WCF, and it is not as gnarly as creating web services in 2007.

The IServiceAdministration Interface

There are a few important things that SharePoint needs to know about the Service Applications that your code supports. First of all, during farm provisioning (psconfig) you can opt in for having your service application created. Also, SharePoint needs to populate the New and Connect menus on the Manage Service Applications page. You cannot really ask the item that is created on the New menu for the content of the New menu right? You need to use the parent, and in this case that parent is the SPService. Being a generic 'farm-wide' service it makes no sense to ask all SPService objects about their support for service applications. The IServiceAdministration interface was introduced to allow SharePoint to ask registered services for their needs and capabilities. If you do not implement this interface all is not lost, but you do lose out on specific integration points. You can still create new instances using your own means though, such as a feature receiver.

The IServiceProxyAdministration interface

Similar to how the IServiceAdministration interface allows SharePoint to find out about your service application capabilities, SharePoint also provides IServiceProxyAdministration. As you can expect from the name it serves a similar purpose but not for the application itself but for the proxy. The interface provides a connection point for SharePoint to create instances of the proxy during automated tasks (psconfig principally). It also provides a way to fill the menu items shown in the 'Connect' menu.

However!…

The design for this first proxy-enabled demo is to call methods directly on the service application, in-process. Meaning, the 'Connect' capability is a bit odd since you'll connect to an in-memory construct instead of a WCF backend that might be in another farm and is identified by a URI. In effect you are not connecting at all, or at least not in the way it was intended by the Connect functionality. So, we'll not implement this interface just yet and leave that one for a later version.

Building the application

In this section I'll show some of the code that make up the new application. The focus is on what is new in this version of the demo application. If you are interested in the basics check out an earlier blog post.

The NavigationServiceProxy

In the second version of the demo application that you can download from CodePlex you'll find the NavigationServiceProxy. This class is the service proxy. A not so interesting class right now, but it'll grow on you after I have had time to write my next blog post. It's basically an empty persisted object. In most persisted objects you find at least two constructors. The first is parameterized and used for initializing a new instance and adding it to the configuration store, the second is used for deserializing the object when it is retrieved later on. A persisted object is identified by both a name and an ID. The ID defaults to the value found in the Guid attribute. Since there is only going to be exactly one instance for this proxy class the name is defaulted to String.Empty.

[Guid("0bc4db91-dac2-4ad6-90cf-394997131d54")]
public class NavigationServiceProxy
    : SPServiceProxy
{
    public NavigationServiceProxy()
    {
    }

    public NavigationServiceProxy(SPFarm farm)
        : base(String.Empty, farm)
    {
    }
}

The NavigationApplicationProxy

Similar to the service proxy the application proxy is a persisted object stored within the configuration database. Since you can create more than one proxy the constructor takes in a name. The constructor also requires you to specify the NavigationApplication for which the proxy is created. The last parameter is a reference to the singleton NavigationServiceProxy that will serve as the parent object of the application proxy (it is passed to the base constructor).

[GuidAttribute("0c1a4b04-90d5-4cef-be3a-f4591b1ac873")]
public class NavigationApplicationProxy
    : SPServiceApplicationProxy
{
    [Persisted]
    Guid _serviceApplicationID;

    public NavigationApplicationProxy() { }

    public NavigationApplicationProxy(string name,
        NavigationApplication application,
        NavigationServiceProxy serviceProxy)
        : base(name, serviceProxy)
    {
        _serviceApplicationID = application.Id;
    }

    NavigationApplication GetNavigationApplication()
    {
        return (NavigationApplication)Farm.GetObject(
          _serviceApplicationID);
    }
}

The class is completed with two static methods. The first is a factory method used for creating a new application proxy instance. The second is used for retrieving existing proxies. One added treat is that you can specify whether the newly created application proxy should be added to the default proxy group. Only a few lines of code are needed to implement this behavior.

public static NavigationApplicationProxy Create(
    string name, 
    NavigationApplication application,
    NavigationServiceProxy applicationProxy, 
    bool addToDefaultGroup)
{
    NavigationApplicationProxy proxy = 
        new NavigationApplicationProxy(
            name, application, applicationProxy);
    proxy.Update();
    if (addToDefaultGroup)
    {
        SPServiceApplicationProxyGroup group = 
            SPServiceApplicationProxyGroup.Default;
        group.Add(proxy);
        group.Update();
    }
    return proxy;
}

public
 static NavigationApplicationProxy GetProxy(
    SPServiceContext context)
{
    return (NavigationApplicationProxy)context.GetDefaultProxy(
        typeof(NavigationApplicationProxy));
}

The class further contains a method which implement the business side of the system (it is specific to the navigation application demo). First, the application is retrieved using the persisted application ID. Next, a method is called directly on the application.

public string GetNavigationInformation()
{
    NavigationApplication application = GetNavigationApplication();
    return application.GetNavigationInformation();
}

This is the bulk of the functionality provided by the second demo application. Next there is also the IServiceProxyAdministration interface to allow some administration on the demo service.

The IServiceProxyAdministration implementation

This interface is used for two purposes. It is called by SharePoint to create new applications and proxies as part of the farm configuration. The interface is also queried for the items to show in the New menu. Since the implementation of these methods is so simple I'll omit the specifics. Download the demo application to see what tasks it performs (it just calls the existing Create method or returns some simple information).

public interface IServiceAdministration
{
    SPServiceApplication CreateApplication(string name, 
        Type serviceApplicationType, 
        SPServiceProvisioningContext provisioningContext);

    SPServiceApplicationProxy CreateProxy(string name, 
        SPServiceApplication serviceApplication, 
        SPServiceProvisioningContext provisioningContext);

    Type[] GetApplicationTypes();

    SPPersistedTypeDescription GetApplicationTypeDescription(
        Type serviceApplicationType);

    SPAdministrationLink GetCreateApplicationLink(
        Type serviceApplicationType);

    SPCreateApplicationOptions GetCreateApplicationOptions(
        Type serviceApplicationType);
}

The New Application Page

Version two of the navigation application does contain one added item of interest. The ASP.NET page used to create new service applications. You know. It's the page where you'll allow administrators to enter a name, perhaps choose an application pool or two and a database here or there. The following specifics are interesting when implementing your own New Application page.

First of all the page directive hardwires the page to use the dialog master.

<%@ Page Language="C#" Inherits="..." MasterPageFile="~/_layouts/dialog.master" %>

Next, in the OnInit for the page, the dialog master is retrieved and the OK button is hooked up to a delegate.

 

protected override void OnInit(EventArgs e)
{
    ((DialogMaster)this.Page.Master).OkButton.Click += OkButton_Click;
    base.OnInit(e);
}

In the method that is bound using the delegate, the following takes place. First you use the SPLongOperation (a little gem IMHO). To finish the long operation don't redirect but instead send some script to close the dialog and that's it!

void OkButton_Click(object sender, EventArgs e)
{
    using (SPLongOperation operation = new SPLongOperation(this))
    {
        operation.LeadingHTML = "Creating new Navigation Application";
        operation.Begin();
        try
        {
            ...
        }
        finally
        {
            operation.EndScript("window.frameElement.commitPopup();");
        }
    }
}

The logic contained within the try / finally block first creates and provisions a service application, and next it creates a proxy too, adding it to the default proxy group. The idea of also creating a proxy is that we are not offering any user interface that allows administrators to create new proxies. The reason for not providing this is that the demo proxy calls methods directly on the application without needing an extra level of interaction configured using a URI. So, if an administrator is not going to create a proxy, we'd better do it for him!

NavigationService service = SPFarm.Local.Services.GetValue<NavigationService>();
string title = nameField.Text;
NavigationApplication application = NavigationApplication.Create(title, service);
application.Provision();
SPFarm farm = application.Farm;
NavigationServiceProxy serviceProxy = (NavigationServiceProxy)farm.GetObject(
    String.Empty, farm.Id, typeof(NavigationServiceProxy));
NavigationApplicationProxy proxy = NavigationApplicationProxy.Create(title, application, serviceProxy, true);
proxy.Provision();

 

That's it for today. Hopefully I'll have the time to write up the next part. I'll cover how to move your service proxy to the next step by adding WCF. Hope it helps!




























 

1 - 10 Next

 Projects

Databinding toolkit for Word 2007Use SHIFT+ENTER to open the menu (new window).
Open XML Activities for Windows Workflow FoundationUse SHIFT+ENTER to open the menu (new window).
Package ExplorerUse SHIFT+ENTER to open the menu (new window).
Windows SharePoint Services 3 Workflow DesignersUse SHIFT+ENTER to open the menu (new window).
Word 2007 Source ViewUse SHIFT+ENTER to open the menu (new window).