Balan's Blog

SharePoint 2010 Search and FAST Search

2011-09-13T03:58:00.000-07:00

SharePoint 2010 Search and FAST Search

Like many of you, I was (and in some cases) was very confused about how SharePoint 2010 Search and the new FAST Search for SharePoint worked when the FAST product was installed in a SharePoint environment. My understanding was that when FAST was installed, SharePoint Search ceased to exist. Turns out – that was wrong. Let’s first take a look at a SharePoint Search only scenario:

So as you can see SharePoint Search indexed the content sources and the “people” directory to create a rich index and property database to be able to provide a rich Search experience. All of the goodness that has been added to Search in SharePoint 2010 is now available to the end user. While there has been a lot of enhancements to the new version of search, there still are some limitations that would drive someone to want to use FAST Search for SharePoint (more on that in another post).

So when FAST is added to the configuration, SharePoint Search is not removed from the configuration – they are actually sharing some of the roles. Take a look at this diagram:

Here you can see that SharePoint Search stays around to handle all search aspects of the People directory. FAST Search for SharePoint take point on indexing all all content. The secret sauce of providing a unified experience to the end user happens at the Query Object Model. It’s at this location where the 2 result sets are brought back together and ultimately delivered to the WFE and ultimately to the end user.

So there your go – when you use FAST Search for SharePoint – you are actually using 2 different search technologies.

2011-03-29T23:13:00.000-07:00

SharePoint 2010 Architectures Overview

SharePoint 2010

Published: February 2011

Summary: Learn about the architectures of Microsoft SharePoint Foundation 2010 and Microsoft SharePoint Server 2010, including the platform stack, the Microsoft ASP.NET-IIS integrated request pipeline, the server and client object models, and the execution process system for sandboxed solutions and farm solutions.

Applies to: Microsoft SharePoint Foundation 2010 Microsoft SharePoint Server 2010

Overview of SharePoint Foundation and SharePoint Server Architectures

This article provides an overview of the architectures of Microsoft SharePoint Foundation and SharePoint Server, including the platform stack, the Microsoft ASP.NET-IIS integrated request pipeline, the server and client object models, the execution process system for sandboxed solutions and farm solutions, and more.

What Is SharePoint?

The use of the plural term "architectures" in the title is not a mistake. SharePoint has many architectures, partly because "architecture" has many meanings in the context of software development, but also because SharePoint itself is many things—things that in the past would have been distinct applications or platforms.

The following is a selection of some of the most important things that SharePoint is:

A portal server with delegated administration. SharePoint enables information workers (IW) who have no knowledge of website design or website administration to create, almost instantly, attractive and functioning websites. This relieves information technology (IT) departments from the burden of creating and administering the sites, and it empowers the IWs to create their own sites for teams, blogs, wikis, and other purposes.

A groupware application kit. SharePoint provides a platform on which IWs can create collaboration solutions that include document libraries and workspaces, workflows, wikis, blogs, and team-oriented lists, such as Events, Announcements, and Tasks. Microsoft SharePoint Workspace (formerly Microsoft Office Groove 2007) provides an offline experience for these collaboration solutions.

A workflow host. Business processes can be systematized and modeled with workflows that are triggered by associated events; for example, the addition of a document to a document library.

A content management application. SharePoint Server Enterprise Content Management (ECM) features include document management, records management, and web content management.

A Business Intelligence (BI) application kit. The Microsoft Business Connectivity Services (BCS) features of SharePoint enable data from non-SharePoint sources, such as a SAP installation or Oracle database, to be accessed (read/write) just as if it were an ordinary SharePoint list.

The operating system of an intranet. SharePoint can provide for an intranet many of the functions that an operating system provides for a computer, including storing and copying files, hosting services, starting applications, and securing data. (This is not to imply that SharePoint can only be used on an intranet. SharePoint can also host extranet and Internet-facing solutions.)

A host for services. SharePoint deployments make data available through a client object model, the REST-based Windows Communication Foundation (WCF) Data Services (formerly ADO.NET Data Services), and many out-of-the-box ASMX web services. In addition, the SharePoint Service Application Framework provides a platform that enables developers to build scalable middle-tier services that can provide data or processing resources to other SharePoint features.

A data store. SharePoint stores data as multicolumn lists in a Microsoft SQL Server database. You can query the data by using LINQ and also using Collaborative Application Markup Language (CAML). The data can be mirrored, backed up, restored, and, depending on the edition of SQL Server being used, you may be able to take snapshots of the data store.

A data and processing layer for multiple user interfaces (UIs). Besides its native UI of webpages (including special versions for mobile devices), which can contain ECMAScript (JavaScript, JScript), SharePoint also supports access from Microsoft Silverlight applications and the Microsoft SharePoint Workspace client application. With the SharePoint client object model, you can access SharePoint using Windows Presentation Foundation (WPF), Windows Forms, or any other managed code application.

This article discusses the architectures of SharePoint 2010 (with many links to more detailed information) in several senses, including the platform stack on which SharePoint runs, the configuration layers, the client and server object models, the services framework, the HTTP request pipeline, and the worker process system.

SharePoint Development Platform Stack

Figure 1 shows how SharePoint Foundation is built on Microsoft .NET Framework 3.5, ASP.NET, and Internet Information Services (IIS). SharePoint is also built on SQL Server, but you can install SQL Server on a dedicated server that does not have SharePoint installed, which is typical in a production farm. All of these platforms must be running on a 64-bit installation of Windows Server 2008 on the server computers. SharePoint Server, in turn, is built on SharePoint Foundation. There are two editions of SharePoint Server: Standard and Enterprise.

Figure 1. The platform stack for SharePoint development

Perhaps the most noteworthy aspect of the figure is that IIS and ASP.NET are shown as a single platform. This is because SharePoint requires that IIS operate in integrated mode with ASP.NET. Hence, from a SharePoint point of view, they are effectively a single web-hosting application. For more information about this, see the next section, SharePoint as an ASP.NET-IIS Application.

In Figure 1, the smaller boxes that have no fill represent some selected subparts of the platform that contains them, or on which they depend. The two thin, downward-pointing arrows indicate some specific dependencies that are shown only as examples. Many other specific dependencies are not shown in the figure. The thick, left-pointing arrows indicate that the entity on the right side accesses the entity to which the arrow points. For example, BCS accesses external databases.

SharePoint as an ASP.NET-IIS Application

As the High Level Object Model section later in this article shows in more detail, the highest level of organization in a SharePoint deployment, other than the farm itself, is the web application. A web application in SharePoint terminology is closely related to what is called a website in IIS terminology. An IIS website monitors for incoming requests through a particular port, and it checks for a particular host header or IP address, or both. Every SharePoint Foundation web application is hosted in an IIS website that has the same name as the web application. It can be helpful, especially when you are trying to see the relation between SharePoint and IIS from a high and broad perspective, to think of the SharePoint web application and its corresponding IIS website as a single entity.

Nevertheless, the SharePoint web application and the IIS website are not quite the same thing (and they are represented by different classes in the SharePoint object model: the SPWebApplication class and the SPIisWebSite class). For one thing, although there is usually a one-to-one relation between SharePoint web applications and IIS websites, this is not always the case. It is possible to extend a SharePoint web application to multiple IIS websites, although that is not a common design.

Figure 2 shows the IIS websites and application pools on a SharePoint Foundation front-end web server.

Figure 2. IIS Manager on a SharePoint Foundation front-end web server

Just as in any application that is built on the ASP.NET-IIS integrated pipeline, when a front-end web server receives a request from a client for a page or other resource in a SharePoint site, the request is passed through a pipeline of units that process the request. This processing includes authenticating the user, verifying the user’s authorization, building the response, sending the response, and finally, logging the request. For more information about the integrated pipeline, see ASP.NET Integration with IIS 7.

The response to any request is produced by an HTTP handler object. Requests are assigned to one or another HTTP handler object (or to a handler factory class that creates HTTP handler objects) depending on the resource requested and the HTTP verb in the request. The assignment is determined by a stack of configuration files named applicationhost.config, machine.config, and web.config.

The request pipeline also contains HTTP modules. Modules are assemblies that typically contain one or more event handlers or define new events that other modules can handle. An HTTP module can register for one or more events in the life cycle of the request. They are often used to preprocess requests or postprocess the response. The result of a module’s preprocessing is stored in the HttpContext object. For example, the value of the User property is produced by an authentication module.

The processing of a request by HTTP modules and an HTTP handler is governed by an HttpApplication object or an object derived from that class. SharePoint installs a global.asax file in the root of each web application (IIS website) that identifies SPHttpApplication as the class from which an HTTP application object is created.

SharePoint also modifies the pipeline with a custom HTTP module (SPRequestModule), and with changes to the default applicationhost.config file. For detailed information about SharePoint and the integrated pipeline, see Microsoft SharePoint Foundation as an ASP.NET Application.

Figure 3 shows the relations among the classes that handle HTTP requests for a SharePoint front-end web server.

Figure 3. SharePoint customizations of the HTTP request pipeline

In addition to its use of the integrated pipeline, SharePoint uses some other features of IIS and technologies from ASP.NET. Among them are the following:

Active pages (aspx files): SharePoint pages are ASP.NET pages. SharePoint controls are referenced in the page markup with a registered "SharePoint" prefix. (For more information about the ASP.NET page parsing and compiling system, see Understanding ASP.NET Dynamic Compilation. Note that the standard page parser of ASP.NET compiles a page the first time it is requested unless the page is explicitly set not to be compiled. However, SharePoint pages that have been customized by a SharePoint website owner are stored in the SharePoint content database and are not compiled when they are accessed. SharePoint uses a custom virtual path provider, an internal class named SPVirtualPathProvider that is derived from VirtualPathProvider, to locate a requested page from either the file system or the content database and load it.

The master page/content page distinction: A SharePoint page is a merger of a master page and a content page. For more information, see ASP.NET Master Pages and Master Pages. (For some SharePoint Server pages, a third foundational page is involved in the merger, a page layout.)

Web Parts: Web Parts actually originated in an early version of SharePoint. The idea was adopted by ASP.NET. In most SharePoint Web Part development situations, we recommend that you use the ASP.NET version, but in some situations the original SharePoint incarnation is a better choice. The SharePoint UI for mobile devices also uses the ASP.NET concept of registered Web Part adapters. For more information about Web Parts in SharePoint, see Building Block: Web Parts and Building Block: Mobile Pages, Controls, and Adapters.

User controls (.ascx files): Many SharePoint web controls and Web Parts are built with ASP.NET user controls.

Web services (.asmx files): Some, but not all, SharePoint web services are implemented as ASP.NET ASMX services. For more information about ASP.NET web services in SharePoint, see ASP.NET Web Services.

Virtual directories: SharePoint maps several virtual directories in each IIS website to physical folders in the SharePoint root, which is the folder %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\. When you as a developer need to provision SharePoint web applications with images, resource files, and other important files, they are deployed to the appropriate physical folder on the front-end web servers. This process is handled automatically by the solution deployment mechanism of SharePoint. For example, user controls are deployed to %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\ControlTemplates (or a subfolder), which is mapped to the virtual directory _controltemplates.

ASP.NET development experience can be an advantage for a SharePoint developer, but the SharePoint development experience is different from the ASP.NET experience in many ways. Some examples of the differences are as follows:

SharePoint has its own installation and deployment model. For more information, see SharePoint Deployment Models later in this article.

SharePoint makes a distinction, unknown in ASP.NET, between application pages and site pages. For more information, see Pages, Parsing, and Safe Mode later in this article.

SharePoint has its own systems for modifying web.config files. For more information, see Working with Web.config Files.

The ASP.NET worker process model is modified in SharePoint. See Process and Execution Trust Model later in this article.

For more information about how SharePoint development differs from ASP.NET development, see ASP.NET vs. SharePoint: How Development Differs.

SharePoint Configuration Stack

SharePoint configuration settings exist at several levels. As in all ASP.NET applications that use the integrated pipeline, some fundamental settings are in the machine.config file, the global web.config file, and the applicationhost.config file, which is the IIS configuration store. SharePoint makes no changes in the default versions of the first two files. It does make some changes in the IIS configuration store. For more information about the changes, see Microsoft SharePoint Foundation as an ASP.NET Application.

Each IIS website and, therefore, each SharePoint web application, can have a web.config file in its root folder. This file is substantially customized by SharePoint whenever a SharePoint web application is created. For more information about the changes, see Microsoft SharePoint Foundation as an ASP.NET Application. SharePoint also takes advantage of the fact that web.config files can be applied to specific virtual or physical folders within a web application. For example, SharePoint puts a web.config file in the virtual directory _layouts\mobile (which is mapped to the physical directory %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\TEMPLATE\LAYOUTS\MOBILE\). Among other things, the web.config file registers a series of filters that control how a page is rendered, based on the capabilities of the mobile device that has requested the page.

All of the configuration files discussed so far must be identical across all front-end web servers. The SharePoint deployment model and its system for programmatically modifying web.config files helps ensure conformity to this rule. For more information, see Building Block: Solutions and Working with Web.config Files.

Farm-wide configuration settings are stored in the SharePoint configuration database on the computer that is running SQL Server in the farm. Configuration for specific types of websites is contained in various kinds of XML files, such as the Onet.xml file, and configuration for specific instances of websites is contained in the content database. In addition, several classes in the SharePoint object model have property bags that can be used to store custom configuration information for specific objects, such as objects that represent websites, Features, alerts, and files.

Finally, context information for specific HTTP requests is contained in the ASP.NET HttpContext object of the request. The SharePoint object model adds an SPContext class that does not inherit HttpContext; however, it gets many of its property values from the current HttpContext object and adds other properties to represent SharePoint-specific context information, such as the current website, list, and list item.

SharePoint Deployment Models

SharePoint solutions are generally not installed by using MSI or ClickOnce technology. (Some exceptions are noted later.) SharePoint has its own installation system. Instead of MSI files, the solutions are packaged in SharePoint solution package files. These are CAB files that have a special extension: .wsp. SharePoint solution package files can contain several kinds of elements, including assemblies, user controls, custom ASP.NET pages, XML configuration files, resource files, images, list definitions, Web Parts, Features, and others. ECMAScript files and Silverlight .xap files can also be deployed to the servers by using SharePoint Solution Packages.

There are two kinds of solutions in SharePoint: farm solutions and sandboxed solutions. The differences between them are discussed in this section and in Worker Processes, Farm Solutions, and Sandboxed Solutions later in this article.

Note:
The terms "solution" and "feature" are familiar in software development. The first thing to note about the SharePoint deployment model is that these terms are names of formal components of SharePoint that are defined in this section. By convention, "Feature" is capitalized when it refers to the SharePoint component of that name. Throughout this article, I will follow the additional convention of italicizing "farm solution," "sandboxed solution," and "solution" when the SharePoint component is being referenced. When either "solution" or "feature" is lowercase and not italic, it is being used in its ordinary sense.

Note:

The terms "solution" and "feature" are familiar in software development. The first thing to note about the SharePoint deployment model is that these terms are names of formal components of SharePoint that are defined in this section. By convention, "Feature" is capitalized when it refers to the SharePoint component of that name. Throughout this article, I will follow the additional convention of italicizing "farm solution," "sandboxed solution," and "solution" when the SharePoint component is being referenced. When either "solution" or "feature" is lowercase and not italic, it is being used in its ordinary sense.

Installation of a SharePoint solution package is a multi-step process:

Adding: A solution package file is added to one of two stores.
- Farm solutions are deployed by farm administrators to the farm's solution store, which is in the farm's configuration database. This is done by using either the SharePoint Management Shell or the object model.
- Sandboxed solutions are deployed to a specific site collection's Solution Gallery by a site collection administrator. The gallery is a special SharePoint list, so it is in the content database. Solution deployment to the gallery is done through the Site Actions UI of the site collection or by using SharePoint Management Shell. For information about the difference between site collections and websites, see Middle Level Object Model later in this article.
Thus, SharePoint Solution Package files are never installed to the file system of any server, although elements of a farm solution may be installed to the file system when the solution is deployed.

Deploying: The solution package is unpacked and its elements are deployed to their appropriate places.
- For a farm solution, this step also requires a farm administrator and can be done by using either Central Administration or SharePoint Management Shell, or the object model. Some examples of how elements are deployed: user control files (.ascx) are copied to %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\TEMPLATE\ControlTemplates and assemblies are deployed to the global assembly cache or to a web application's \bin folder. If there are SharePoint Features (defined later) in the farm solution, they are in this step copied to a subfolder of %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\TEMPLATE\FEATURES.
- For sandboxed solutions, the deployment step is taken by a site collection administrator. There are limits on what kinds of elements can be in sandboxed solutions and where they can be deployed. For example, nothing can be deployed to the file system of the servers from a sandboxed solution. Features in sandboxed solutions are deployed to the content database.
It is also in this second step that any Features in the solution are installed in the Feature Gallery of the farm, web application, site collection, or website, depending on the scope of the Feature. (For more information about the relations of farm, web applications, site collections, and websites to one another, see Content Hierarchy later in this article.)

Within a SharePoint solution package, there can be an additional level of encapsulation because a solution can have one or more SharePoint Features. A Feature can be installed at the scope of the farm, the web application, the site collection, or the website. After it is installed, a Feature must be activated by owners of any website within the scope; so Activating becomes a third step of installation for Features. Features can contain content types, controls, custom actions, custom fields, files, workflows, list instances, list templates, event receivers, and document converters; although some of these cannot be included in certain scopes. (Features that are deployed in sandboxed solutions can be scoped only to a site collection or website. For site collection-scoped features in sandboxed solutions, the second and third steps are combined. The Features are activated when the solution is deployed

Note:
SharePoint is not consistent in its terminology with regard to installation. The terms adding, deploying, and activating are the most frequently used to refer to the three steps of installation; but depending on what tool is used to complete a step and whether the solution is farm or sandboxed, you will see a variety of terminology. The first step may be called adding, installing, or uploading. The second step may be called deploying, activating or installing. There is a similar inconsistency in the terms for reversing these steps; but most commonly, reversing the second step is called retracting and reversing the first step is called removing. The third step, which applies only to Features, is always called activating, and its reversal is always called deactivating.

Note:

SharePoint is not consistent in its terminology with regard to installation. The terms adding, deploying, and activating are the most frequently used to refer to the three steps of installation; but depending on what tool is used to complete a step and whether the solution is farm or sandboxed, you will see a variety of terminology. The first step may be called adding, installing, or uploading. The second step may be called deploying, activating or installing. There is a similar inconsistency in the terms for reversing these steps; but most commonly, reversing the second step is called retracting and reversing the first step is called removing. The third step, which applies only to Features, is always called activating, and its reversal is always called deactivating.

Some SharePoint solutions target one of the client object models, either exclusively or in addition to targeting the server object model: the ECMAScript, Silverlight, and Microsoft .NET Framework client object models. (For more information about the client object models, see Client Object Models in SharePoint later in this article.) There is nothing unusual about the installation of the client portion of such solutions.

The script files that define the ECMAScript object model are downloaded to a client computer when a page that references them is opened.

Similarly, a Silverlight .xap executable is downloaded to the client computer when a page that is hosting it is accessed. The SharePoint assemblies that contain the Silverlight client object model are encased in the .xap file. Those assemblies are Microsoft.SharePoint.Client.Silverlight.dll and Microsoft.SharePoint.Client.Silverlight.Runtime.dll. (It is also possible to cache these assemblies on the front-end web server. For details, see RIA Technologies: Benefits, Tradeoffs, and Considerations.)

Stand-alone .NET Framework applications (such as a WPF application) that target the SharePoint client object model are installed just as any other client applications: using MSI files or ClickOnce. The assemblies that contain this client object model, Microsoft.SharePoint.Client.dll and Microsoft.SharePoint.Client.Runtime.dll, must be distributed as part of the solution. The redistributable is located at SharePoint Foundation 2010 Client Object Model Redistributable.

For more information about the processing of client-side logic in SharePoint, see Client Application Models in SharePoint 2010.

For more information about the deployment system, see Building Block: Features, Building Block: Solutions, and Packaging and Deployment.

Process and Execution Trust Model

In the simplest scenarios, the process model of SharePoint is the same as any other ASP.NET application; but in SharePoint, the distinction between farm solutions and sandboxed solutions entails some more complex scenarios.

Worker Processes, Farm Solutions, and Sandboxed Solutions

When an HTTP request is received by a front-end web server, a special driver, HTTP.SYS, detects the request and routes it to the application pool that is handling requests for the targeted IIS website, and thus, the targeted SharePoint web application. Every application pool has an IIS worker process (w3wp.exe) in which the request pipeline for each request is executed. (For more information about the IIS 7.0 worker processes and application pools, see Introduction to IIS 7 Architecture.) On a SharePoint server, the IIS worker process runs in the application pool account, which gives the process read and write permissions to SharePoint resources. On a multiserver farm, the application pool account is a domain user that is not a machine administrator on any server in the farm, but is a member of the WSS_WPG, WSS_ADMIN_WPG, and IIS_USERS groups on each server. (This should be a different account from the farm administration account. The latter is also not a local machine administrator on any farm server. The farm account is a member of all the same groups as the application pool account and also the WSS_RESTRICTED_WPG_V4 and Performance Monitor User groups. An exception is made for the application pool of the Central Administration web application: its application pool account is the farm account. Also, the SharePoint 2010 Timer Service executes in the context of the farm account.) For more information about the accounts needed in a SharePoint farm, see Plan for administrative and service accounts (Office SharePoint Server).

Figure 4 shows how an IIS worker process that is running on a front-end web server processes an HTTP request.

Figure 4. Request processing model for full-trust farm solution

However, unlike a standard ASP.NET application, SharePoint makes a distinction between sandboxed solutions and farm solutions. Farm solutions run in the IIS worker process just like any ASP.NET application. Sandboxed solutions run in a specially restricted execution environment. This is necessary because sandboxed solutions are installed on (and scoped to) SharePoint site collections without the intervention of the IT professionals that are managing the SharePoint farm. To prevent rogue or poorly performing code from slowing or crashing the application pool, SharePoint imposes restrictions on what the code in a sandboxed solution can do. As a crucial part of the implementation of this system, sandboxed solutions must run in a special sandboxed worker process (SPUCWorkerProcess.exe).

When a request attempts to access a sandboxed solution, a SharePoint execution manager that runs in the IIS worker process finds a sandbox worker process (or starts one, if none is running) in which the code of the sandboxed solution will run. In principal, this sandboxed worker process can be started on any server in the farm that is running the SharePoint 2010 User Code Host service (SPUCHostService.exe). (In the UI of the Central Administration application, this is known as the Microsoft SharePoint Foundation Sandboxed Code Service.)

The server that is running the SharePoint 2010 User Code Host service can be, but does not have to be, the front-end web server on which the IIS worker process is running. Which server is used is configurable in the Central Administration application: Administrators can choose to have each sandboxed process run in "local mode," which means that each request for a sandboxed solution is processed on the same front-end web server on which the IIS worker process is running; or they can have the execution manager start each sandboxed process in "remote mode," also known as "affinity mode." In affinity mode, the execution manager looks for a server that is running the SharePoint 2010 User Code Host service and which already has created an application domain inside its SPUCWorkerProcess.exe process for the very same sandboxed solution. (This would be the case if that same sandboxed solution was requested before, possibly by another user on another site collection.) If there is a matching application domain, the request is sent to that same application domain for handling. If none of the servers that are running the SharePoint 2010 User Code Host service already has an application domain for the sandboxed solution, the execution manager assigns the request to the least busy of those servers. The server then creates the needed application domain and processes the request for the sandboxed solution. The application domain stays alive after the request is processed and is reused if there is another request for the same sandboxed solution.

By default, all sandboxed solutions that are handled by a given server run in the same sandbox worker process, but this is configurable through the object model. Each sandboxed solution gets its own application domain within the common process, and this, too, is configurable through the object model. The SharePoint 2010 User Code Host service runs in an account that has the same rights as a typical application pool account. It should be a member of the WSS_WPG, WSS_ADMIN_WPG, and IIS_USERS groups on the server on which the service instance is running.

Code Execution and Access Constraints on Sandboxed Solutions

All code that runs in this sandbox worker process is subject to execution and access constraints. There are two systems of constraints: One applies to all and only calls to any assembly, except Microsoft.SharePoint.dll, whether it is a SharePoint assembly or not. The other applies to all and only calls made to the parts of the SharePoint Foundation object model that are in the assembly Microsoft.SharePoint.dll. The calls that this second system applies to are not just calls in custom SharePoint solutions. Calls to Microsoft.SharePoint.dll from other SharePoint assemblies (that have themselves been called by custom code) such as Microsoft.SharePoint.Linq.dll are also subject to this constraint.

The first system of is imposed by two mechanisms:

A highly restrictive code access security (CAS) policy significantly limits what code in the sandboxed worker process can do. This policy is defined in the wss_usercode.config file in %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\CONFIG, and it is referenced in the web.config file in %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\UserCode. Among the restrictions imposed by the CAS policy are the following:
- Code in the sandbox cannot call unmanaged code.
- Code in the sandbox cannot call the Microsoft .NET Framework 3.5 reflection APIs.
- Code in the sandbox can call only the .NET Framework 3.5 assemblies that have the AllowPartiallyTrustedCallersAttribute attribute. This blocks access to about two-thirds of all the .NET Framework 3.5 APIs, including System.Printing, for example.

Secondly, the sandboxed worker process has a low-privileged security token.
- The token denies the process the right to read from or write to the file system.
- The token denies the process the right to call to the network. Therefore, only resources available on the server that is running the sandboxed worker process may be accessed. An external database, for example, cannot be accessed.
- The token denies the process the right to write to the registry.
- The token denies the right to call to any assembly that is not in the general assembly cache, even if it has the AllowPartiallyTrustedCallersAttribute attribute and would otherwise be eligible to be called from the sandboxed worker process.

As noted, a second system of constraints imposes restrictions on what APIs in Microsoft.SharePoint.dll can be directly called by code in the sandboxed worker process, and a call to any forbidden API in the object model results in an exception (which is caught and reported to the user as an error). The implementation of these restrictions is accomplished by a pair of specially restricted versions of the Microsoft.SharePoint.dll assembly, sometimes called shim assemblies, that are located in %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\UserCode\assemblies. One of the two assemblies is loaded by the sandboxed worker process. The other assembly is loaded in a special proxy process (SPUCWorkerProcessProxy.exe) that runs in full trust and that is also managed by the SharePoint 2010 User Code Host. The standard Microsoft.SharePoint.dll assembly is also loaded in this proxy process.

The main job of the two shim assemblies is to filter out forbidden SharePoint classes and members. When the sandboxed solution calls an approved API, the first shim assembly passes it to the second in the proxy process, which in turn passes it to the standard Microsoft.SharePoint.dll. Any returned results are passed back to the original calling code. This cross-process interaction is possible through .NET Framework remoting. A sandboxed worker process and a full-trust proxy process are always started together and paired with one another. If either process crashes, the other is also stopped.

The shim assemblies also have a secondary job. Some SharePoint APIs are available to sandboxed solutions, but only with special restrictions on the parameters that are passed to them. It is the shim assemblies that enforce these input restrictions and ensure that an exception is thrown when there is a violation. The only case of this in SharePoint Foundation 2010 are the SPSite and SPSite constructors. These constructors can be called in sandboxed solutions, but only URLs or GUIDs that refer to the site collection in which the sandboxed solution is running can be passed to them.

The following are some of the restrictions on the SharePoint object model that can be accessed

The SPWebApplication class cannot be accessed. Among other things, this means that a sandboxed solution cannot access anything outside its hosting site collection.

Almost all classes in the Microsoft.SharePoint.WebControls namespace cannot be accessed, which means that you are mainly restricted to ASP.NET controls in sandboxed solutions.

For a complete list of APIs in Microsoft.SharePoint.dll that are available to sandboxed solutions, see Microsoft.SharePoint.dll APIs Available from Sandboxed Solutions

Important:
The deployment stage of a sandboxed solution itself runs in a sandboxed worker process and is subject to the same execution constraints. For example, you cannot deploy a file to the disk when you are deploying a sandboxed solution. This is the main reason why a user control (ASCX file) cannot be in a sandboxed solution. See SharePoint Deployment Models for information about the deployment stage.

Resource Usage Restrictions on Sandboxed Solutions

Sandboxed solutions are also subject to three kinds of resource usage restrictions that can be organized based on the kind of entity to which the restriction applies and the kind of entity on which the penalty for exceeding the restriction is imposed.

Per Request with the Request Penalized: There is a hard limit to how long a sandboxed solution can take to finish. By default, this is 30 seconds. If a sandboxed solution exceeds the limit, the request (but not the sandboxed worker process) is terminated. (This limit is configurable, but only through custom code against the object model. The relevant parts of the object model cannot be accessed by sandboxed solutions, so no sandboxed solution can change the limit.)

Per Request with the Process Penalized: A set of 15 resource limits apply to requests. If a request exceeds one of them, the process (and all the sandboxed solutions that are running in it) is terminated.

Per Day/Per Site Collection with the Site Collection Penalized: Each site collection is subject to a configurable maximum of daily resource points. These points accumulate based on an algorithm that takes into account the use of resources in the 15 resource categories by the sandboxed solutions that are installed in the site collection. When a site collection exceeds its maximum allowed points, all sandboxed solutions in the site collection are terminated and no more can run for the rest of the day.

SharePoint provides a solution validator framework that can be used to develop custom solution validators, such as a validator that verifies whether a solution is signed with a specific certificate. The validators in a site collection run when a sandboxed solution is activated (that is, deployed, in the terminology used earlier in this article). The activation of any invalid solution is blocked. If a validator is updated or a new validator is added, each activated solution is rechecked by the validators the next time it is executed. Invalid solutions are deactivated. For an introduction to custom validators, see Developing, Deploying, and Monitoring Sandboxed Solutions in SharePoint 2010.

For more information about the sandbox restrictions, see Sandboxed Solutions and its child topics and the Microsoft patterns and practices guidelines for Sandboxed Solutions.

Figure 5 shows how an HTTP request is handled when it accesses a sandboxed solution.

Figure 5. Request processing model for sandboxed solutions

The SPUCHostService.exe, SPUCWorkerProcess.exe, and SPUCWorkerProcessProxy.exe files are located at %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\UserCode.

Note:
A solution that is designed to run in the sandbox can be deployed by a farm administrator as a farm solution. It might perform better if it is, because it would run in the IIS worker process instead of the sandboxed worker process.

Not all SharePoint execution is in an IIS worker process, a sandboxed worker process, or the proxy process. The following are some examples:

The SharePoint Timer Service (owstimer.exe) runs on all servers and is used to execute prescheduled timer jobs. It runs under the farm account.

The SharePoint Tracing Service (wsstracing.exe) runs under the local service account.

The SharePoint Administration Service (wssadmin.exe) runs under the local system account.

Within the category of farm solutions, a further distinction can be made when the requested resource is an .aspx page. If the requested page is what SharePoint calls an application page, the returned page is passed to the regular ASP.NET page parser; but if the requested page is what SharePoint calls a site page, the returned page is routed through a special safe mode parser. The distinction between the two kinds of parsing is best understood in light of the distinction between the two kinds of pages. For more information about both subjects, see Pages, Parsing, and Safe Mode later in this article. (Application pages cannot be included in sandboxed solutions. All pages installed as part of a sandboxed solution use safe mode parsing.)

Assembly Deployment, Execution, and Persistence

Farm solutions divide into two types depending on where their assemblies are deployed and the trust level of the assemblies' execution:

GAC/Full Trust: The assemblies are deployed to the global assembly cache (GAC) of every front-end web server in the farm and run with full trust. They are callable from any SharePoint web application on the farm.

Bin/CAS: The assemblies are deployed to the \bin folder (on every front-end web server) of a specific SharePoint web application and their trust level is determined by a CAS policy referenced in the web application's web.config file. They have to be separately deployed to every SharePoint web application that needs to call them.

Tip:
We recommend that you use the Bin/CAS model only when neither full-trust farm solutions nor sandboxed solutions (nor the hybrid solutions described in Hybrid Solution Techniques) are possible for your solution.

Noncompiled files for both kinds of farm solutions—including, for example, images, user controls, and string resources—are deployed to subfolders of %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\.

Sandboxed solutions are deployed inside a SharePoint solution package (.wsp file) to the Solution Gallery of a specific site collection. Thus, they are deployed and persisted in the site collection's content database. As noted earlier, they do not run in full trust: Instead, they run within a highly restricted CAS policy and can only call a restricted subset of the SharePoint object model. A sandboxed solution can be accessed only in site collections to which it is deployed.

When a sandboxed solution is accessed for the first time, such as when a user navigates to a page that contains a Web Part from a sandboxed solution, any assemblies in the solution are unpacked from the solution package and copied to the file system of the server that is handling the sandbox request. The default location is C:\ProgramData\Microsoft\SharePoint\UCCache, but this is configurable on each server that is running the User Code Host Service. (Recall that the server that handles the sandbox request is not necessarily the front-end web server that is handling the initial HTTP request. The User Code Host Service can be run on back-end application servers in the farm instead.) Because the sandboxed worker process cannot copy anything to the file system, the copying is done by the User Code Host Service.

The assemblies do not stay in the file cache perpetually. When the user session that accessed the assemblies finishes, the assemblies stay in the cache for only a short time, and they may be reloaded from there if another user session accesses them. Eventually, if they are not accessed, they are removed according to a proprietary algorithm that takes into account how busy the server is and how much time has gone by since the assemblies were last accessed. If the sandboxed solution is used after that time, the assemblies are unpacked again and copied to the UCCache.

Caution:
Administrators, developers, and third-party code should not add, remove, or load anything from the UCCache. It should be accessed only by the SharePoint infrastructure.

Hybrid Solution Techniques

The SharePoint solutions architecture includes a technique by which a sandboxed solution can call custom operations that run in full trust. The technique requires that a farm solution be developed that includes one or more classes that derive from SPProxyOperation. Each of these defines an operation that will run in full trust and can be called from sandboxed solutions by using the ExecuteRegisteredProxyOperation method. Specifically, these full-trust proxy operations execute in the same proxy process (SPUCWorkerProcessProxy.exe) that was described earlier in Worker Processes, Farm Solutions, and Sandboxed Solutions. The proxy operations can return data to the sandboxed solution.

Like all farm solutions, the assembly with the proxy operations can be deployed only if it is from a trusted source.

Figure 6 shows how a request that accesses a sandboxed solution is processed when the sandboxed solution makes a call to a full-trust proxy.

Figure 6. Request processing model when a sandboxed solutions calls a full-trust proxy

The preceding description might give the impression that, with the hybrid technique, a farm solution and a sandboxed solution are always developed together by the same development team. In fact, the farm solution may be developed specifically to provide certain operations to any and all sandboxed solutions that need those services, including sandboxed solutions that are developed by other teams. For example, because a sandboxed solution cannot write to the SharePoint Unified Logging Service (ULS) logs, a farm solution that opened proxy logging operations to sandboxed solutions would be very useful.

Another hybrid technique uses client-side code to access the resources that cannot be accessed from a sandboxed solution. For example, a sandboxed solution could include a custom site page with JavaScript that makes calls to the SharePoint ECMAScript client object model. Also, a sandboxed solution could include a Web Part that hosts a Silverlight application. The latter application can make calls to the SharePoint Silverlight client object model. For more information about the client-side code in SharePoint, see Client Object Models in SharePoint later in this article.

For more information about hybrid solution techniques, see Hybrid Approaches.

Pages, Parsing, and Safe Mode

As noted earlier in SharePoint as an ASP.NET-IIS Application, SharePoint makes extensive use of the master page/content page distinction. But SharePoint Foundation also divides its ASPX pages along a different axis: It distinguishes between application pages and site pages. Both of these kinds of pages can be mergers of master and content pages and, indeed, every ASCX page that is built into SharePoint, whether it is an application page or a site page, is a combination of a master and content page. Hereafter, in this section, there is no further mention of the master/content page distinction.

Application pages differ from site pages in the following ways:

Typical purpose: Application pages tend to be function-oriented, especially functionality that is needed by many kinds of websites within a given web application; for example, the standard form for creating a new list item is an application page. Site pages tend to be content-oriented; for example, the list-of-lists page of a standard team site. However, exceptions to both tendencies are possible. Indeed, we currently recommend that third-party developers develop custom Web Parts, which can be added to site pages, to handle their solution's functionality whenever possible, rather than develop custom application pages.

Customizablility: Site owners (and other users who have appropriate permissions) can customize site pages, but not application pages. Users can also add an entirely new ASCX page to a website's Site Pages gallery, but only web application administrators can install a new application page.

Class inheritance: Application pages are objects of the LayoutsPageBase class or the UnsecuredLayoutsPageBase class. Site pages are objects of the WikiEditPage class or the WebPartPage class. (All of these classes derive from the ASP.NET Page class.)

Web Part support: Application pages cannot have Web Part zones or dynamic Web Parts. They can have static Web Parts, but there is little point to using a static Web Part instead of an ordinary control on an application page because end users cannot customize application pages anyway. Some kinds of site pages can have static Web Parts, and other kinds can have Web Part zones with dynamic Web Parts. See later in this section for more information.

Storage location: Application pages are stored on the file system of the front-end web servers in the _layouts virtual directory of a web application (which maps to the %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\TEMPLATE\LAYOUTS physical directory) or one of its subdirectories. A site page that has not been customized is stored in other subdirectories of %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\TEMPLATE, most typically in %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\TEMPLATE\SiteTemplates and %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\TEMPLATE\FEATURES. A site page that has been customized, such as with Microsoft SharePoint Designer, is stored in the content database of the website for which it was customized. New site pages that have been added to the Site Pages gallery are stored just as customized pages.

Note:
Even an uncustomized site page has an entry in the content database; but whereas the entry of a customized page contains the ASPX markup that constitutes the page, the entry for an uncustomized page contains the path of the .aspx file on the local front-end web server. An uncustomized page can be shared by many websites. If it is, it is referenced in each website's content database, with all the entries pointing to the same physical file. For example, every Team Site has a content database entry pointing to the default Team Site home page. If the home page is customized on any of the sites, it is copied into the page's entry in the content database in its customized form, and the pointer to the uncustomized version is removed. Sometimes the uncustomized site page is referred to as a page template and the pages in the content database as page instances. However, remember that if the page is not customized in a particular website, the "instance" of the page is simply the database entry pointing back to the file on the file system.

Note:

Even an uncustomized site page has an entry in the content database; but whereas the entry of a customized page contains the ASPX markup that constitutes the page, the entry for an uncustomized page contains the path of the .aspx file on the local front-end web server. An uncustomized page can be shared by many websites. If it is, it is referenced in each website's content database, with all the entries pointing to the same physical file. For example, every Team Site has a content database entry pointing to the default Team Site home page. If the home page is customized on any of the sites, it is copied into the page's entry in the content database in its customized form, and the pointer to the uncustomized version is removed.

Sometimes the uncustomized site page is referred to as a page template and the pages in the content database as page instances. However, remember that if the page is not customized in a particular website, the "instance" of the page is simply the database entry pointing back to the file on the file system.

Availability: An application page can be accessed from every website in the web application and is therefore shared among all the websites. But a site page is available only to a user of the website where it is deployed. (Uncustomized site pages are shared by multiple websites, as noted earlier, but only websites where they have been specifically provisioned either as part of a Feature or as part of a site definition.)

Parsing mode: Application pages are parsed in direct mode, which just means that they are parsed by the standard ASP.NET page parser. The first time an application page is requested, it is parsed and compiled and cached in the front-end web server's memory, where it remains until the application domain or the entire IIS host is recycled. With every subsequent request for the page, it is served from the cache, if it is there. Uncustomized site pages are also parsed in direct mode, but customized site pages, and new pages that are added to the Site Pages gallery, are parsed in safe mode. Safe mode processing differs from direct mode (that is, from standard ASP.NET page processing) in the following ways:
- Only controls (including Web Parts) that are registered as safe in the web application's web.config file are rendered.
- Inline server-side code is not allowed in safe mode, and an error page is returned to the user if the safe mode parser finds inline code on the requested site page. Inline code includes <script> elements, such as <script runat="server"> [code is here] </script>, and control event handlers, such as <asp:button OnClick="MyButtonHandler()" />. (Code behind that is compiled into a separate assembly is allowed, and so is embedded ECMAScript.)
- The page is not compiled. (And, thus, any compilation directives in the file are ignored.)
Note:
The terms safe mode parsing, safe mode processing, and safe mode rendering are used interchangeably in SharePoint documentation.

Note:
The terms safe mode parsing, safe mode processing, and safe mode rendering are used interchangeably in SharePoint documentation.

Security and performance are the motivations for the two kinds of pages, especially the difference in processing mode. One of the purposes of SharePoint is to delegate administrative control to ordinary business users instead of requiring the intervention of network administrators and IT professionals. For this reason, users are allowed to add new pages and customize existing pages. If these users were allowed to add any control or Web Part they wanted or add any code blocks they wanted to the pages, there is a great danger that poor performing or malicious code would be added to a page. The reason that pages that use safe mode are not compiled is that there may be thousands of pages on a SharePoint installation. If every one of them was compiled into an assembly and loaded in memory, the performance of the server would be degraded, and only recycling the application domain would remove them. Moreover, recycling the application domain would remove all assemblies, not just the seldom-used pages. There is also a limit on the number of assemblies that can be loaded in an application domain that is imposed by the .NET Framework.

The safe mode parser only creates a control tree for the page, which can be unloaded again from memory immediately after use. On the other hand, pages that are shared, such as application pages and uncustomized site pages should be compiled on first use, because these pages are used a lot and subsequent requests for them can be handled much faster if the pages are cached as assemblies in the server memory. That is why they run in direct mode.

Security Note:
It is possible to allow inline code, unsafe controls, or both, on selected customized site pages. You can do both things by adding one or more <PageParserPath> elements to the <SafeMode> element of the <SharePoint> section of the web application's web.config file. You can use the attributes of the <PageParserPath> element to specify a customized site page or set of customized site pages, and to enable inline code and unsafe controls for the designated site pages. However, you should use extreme caution when you make these kinds of changes, because they cancel the security benefits of safe mode. For example, if you allow inline code for all pages that have names on the pattern `Contoso*.aspx` in the directory `/sites/contoso/SitePages`, anyone with the right to add pages can create a page that has a name following that pattern and add it to that directory, including a page that contains malicious or poor-performing code. (The directory path is a "virtual path" that points to a set of files in the content database.) Notice that allowing unsafe controls only enables such controls that are added in an editing tool, such as SharePoint Designer. When an end user adds a Web Part to a page in the browser, the Web Part must still be registered as safe even if unsafe controls have been enabled in a <PageParserPath> element. You can turn on compilation for selected customized site pages by using an attribute of the <PageParserPath> element. This might be useful if you have reason to believe that the customized page will be visited often enough to justify compiling it into a DLL cached in memory.

Security Note:

It is possible to allow inline code, unsafe controls, or both, on selected customized site pages. You can do both things by adding one or more <PageParserPath> elements to the <SafeMode> element of the <SharePoint> section of the web application's web.config file. You can use the attributes of the <PageParserPath> element to specify a customized site page or set of customized site pages, and to enable inline code and unsafe controls for the designated site pages. However, you should use extreme caution when you make these kinds of changes, because they cancel the security benefits of safe mode. For example, if you allow inline code for all pages that have names on the pattern Contoso*.aspx in the directory /sites/contoso/SitePages, anyone with the right to add pages can create a page that has a name following that pattern and add it to that directory, including a page that contains malicious or poor-performing code. (The directory path is a "virtual path" that points to a set of files in the content database.) Notice that allowing unsafe controls only enables such controls that are added in an editing tool, such as SharePoint Designer. When an end user adds a Web Part to a page in the browser, the Web Part must still be registered as safe even if unsafe controls have been enabled in a <PageParserPath> element.

You can turn on compilation for selected customized site pages by using an attribute of the <PageParserPath> element. This might be useful if you have reason to believe that the customized page will be visited often enough to justify compiling it into a DLL cached in memory.

Within the category of site pages, there are additional distinctions. In SharePoint Foundation, there are standard site pages and Web Part site pages. Standard site pages are wiki-enabled pages that allow inline Web Parts. Standard pages are objects of a class that derives from WikiEditPage. Web Parts pages, on the other hand, derive from WebPartPage. They have one or more Web Part zones into which Web Parts can be added, and they have no wiki-editable area.

SharePoint Server 2010 adds a third kind of site page: PublishingPage. (There is also a PageLayout class in SharePoint Server 2010, but this is an extension of the master page/content page system of ASP.NET, not another kind of site page.)

SharePoint also has a set of built-in pages that are designed for mobile devices. They do not use the ASP.NET master page/content page technology, and they are not divided into application pages and site pages. SharePoint mobile pages are all application pages, and they are located in the \_layouts\Mobile folder. There is one respect in which a SharePoint mobile page is more like a customized site page: If the page contains a mobile Web Part adapter, the adapter must be registered as a safe control or it is not rendered.

For more information about page types in SharePoint, see Building Block: Pages and User Interface, ASP.NET vs. SharePoint: Page Development, SharePoint Page Types, and Publishing Programming Model. For more information about mobile pages, see Building Block: Mobile Pages, Controls, and Adapters and Overview of Mobile Pages and the Redirection System. For more information about safe controls and safe mode, see Microsoft SharePoint Foundation as an ASP.NET Application.

Data Model, Data Management, and Query System

The primary data structure in SharePoint Foundation is the list. Every list belongs to a list type. Similarly, every column in a list has a field type, and every list item has a content type. External data—that is, data from outside the SharePoint content databases—can also be shown and managed in SharePoint.

Lists

Data in SharePoint Foundation is primarily stored as tables much as it is in a relational database, except that the tables are called "lists" in SharePoint lingo. Indeed, the back-end storage of content data for a SharePoint Foundation web application is in one or more SQL Server databases. These are called content databases. (There is also a special configuration database on the computer that is running SQL Server that holds farm configuration data, and a special BDC database that supports Microsoft Business Connectivity Services (BCS). For more information about BCS, see External Lists and the Business Connectivity Service later in this article.) However, there are some differences between relational database tables and SharePoint Foundation lists:

Data is not queried by SQL. Instead, data is queried in server-side code either by LINQ or by queries formulated in CAML. For more information about server-side data querying, see Building Block: Queries and Views and the topic Querying from Server-side Code, along with its child topics. You can programmatically write data to the lists using either the server object model or the LINQ to SharePoint provider. For more information, see Managing Data with LINQ to SharePoint. From client-side code, data is queried by using either the client object model or WCF Data Services (formerly ADO.NET Data Services). For more information about client-side data querying, see Querying from Client-side Code.

Caution:
Directly accessing the back-end computer that is running SQL Server by using SQL queries, stored procedures, or any other method is not supported.

A list can have a column whose possible values are the values of a column on a different list. The lookup column relationship between the lists is somewhat like a foreign key relationship between two relational tables. However, the field on the target list that provides the values is not necessarily the foreign key. All SharePoint Foundation lists have an ID column. This column is, in effect, always the foreign key in any lookup relationship. For more information about lookup relationships in SharePoint Foundation, see Lookups and List Relationships and List Relationships in SharePoint 2010.

Lists can be joined just as tables can, but with some restrictions. There must be a lookup relation between the lists or they cannot be joined. For more information about list joins, see List Joins and Projections.

A SharePoint Foundation document library is a special kind of list in which each row includes an attached document, and the other columns are data about the document, such as its author, when it was last edited, and who has it checked out. Picture libraries are similar except that the attached file is an image file. For more information, see Building Block: Lists and Document Libraries.

Caution:
Directly accessing the back-end computer that is running SQL Server by using SQL queries, stored procedures, or any other method is not supported.

Every list has a list type, and SharePoint Foundation includes many built-in list types that enable end-users to create the most common kinds of business and team solutions. Among these are Announcements, Tasks, and Calendar. Developers can also create custom list types. For more information about list development, see SharePoint List Data Model.

Content Types and Field Types

A row in a list—that is, a list item—also has a type. These are called content types. Each is basically a set of columns and metadata. The simplest is the built-in Item content type. All other content types are derived from Item. SharePoint Foundation includes many built-in content types, such as Event and Announcement. Developers can create custom content types. For more information about content types, see Building Block: Content Types, Content Types, and SharePoint Columns, Lists, and Content Types.

A column in a list, also known as a "field," also has a type, and it is distinct from the data type of the values that can be stored in the field. A SharePoint Foundation field type includes not only information about the underlying data type, but also information about how the data is formatted and rendered on forms, such as the forms for creating, displaying, and editing specific list items. For example, it is the field type that determines whether a field value is entered as string or from a drop-down list of values. Many field types are built-in to SharePoint Foundation, such as the Modified By field on a document library and the Due Date field on a Task list. Developers can create custom field types. For more information, see Building Block: Columns and Field Types, Custom Field Types, and SharePoint Columns, Lists, and Content Types.

External Lists and the Business Connectivity Service

External data, such as data in an SAP installation or Oracle database, can also be represented as a list on a SharePoint Foundation page and within SharePoint Workspace and Microsoft Office client applications. The Microsoft Business Connectivity Services (BCS) of SharePoint Foundation enables read/write access to this data.

The critical components of BCS are Business Data Connectivity (BDC) service models. Each model is an XML file that describes a type of external data source, such as SAP services. One or more specific instances of the data source, such as a particular SAP database, are defined in the model, including connection and security information about the data source. The business entities in the data source, such as Customer or Order, are represented in the model by external content types, and the model also defines relationships between these entities. Finally, the model defines, for each entity, a set of standard operations, called stereotyped operations, that can be performed on the entity, including create, retrieve, update, and delete operations. For more information about the structure of BDC models, see BDC Model Infrastructure. The models are stored in a dedicated database on the SharePoint farm computer that is running SQL Server, which is distinct from the configuration database and the content databases. This database is called the metadata store.

After a model is added to the metadata store, users can access the external data in a variety of ways. For example, in SharePoint Foundation, users can create a SharePoint list out of an external content type, such as Customer, or add a column of data from the external source to an existing list. SharePoint Server 2010 has some BCS enhancements: It includes some built-in Web Parts for working with external data, and it enables users to search the external data. Also, SharePoint Workspace and Microsoft Office client applications can access and display the external data directly. Finally, with SharePoint Server 2010, users can index and search the external data sources.

A service named Business Data Connectivity service manages the interactions between SharePoint and the external data source. The service runs on an application server in the farm. There is also a BDC Runtime that runs on all front-end web servers. When a SharePoint client application requests external data, the BDC Runtime on the front-end web server requests the metadata that defines the BDC model from a locally cached copy of the BDC metadata store. The runtime then uses the metadata provided to perform data operations directly on the external system. Similarly, on client computers that are running SharePoint Workspace and Microsoft Office client applications, a BDC Client Runtime enables direct client access to the external data by using the BDC model, which is also cached on the client computer (although the SharePoint Server 2010 farm is still needed for long-term persistence of the BDC models). The external data itself can be cached on the client computer to enable an offline experience.

After the BDC service is running and the needed BDC models are registered, end users can create non-code solutions that involve external data. For more complex solutions, the BCS Runtime APIs enable developers to make custom Web Parts or other solutions to interact with the external data. Because a BDC model provides a kind of translation between operations in code and the stereotyped operations of the external data source, the same APIs can be used for all external data sources no matter how different their individual access systems are. BDC solutions are packaged and deployed as farm solutions.

Figure 7 shows some of the major components of BCS and their relationships.

Figure 7. BCS runtime and deployment components

The BDC service is built in conformance with the Service Application Framework of SharePoint Foundation. See also Services and the Service Application Framework and Services Hierarchy later in this article.

For more information about BCS architecture, see BDC Architecture, Mechanics of Using Business Connectivity Services, Business Data Connectivity (BDC) Service, and External Data in SharePoint 2010.

Services and the Service Application Framework

Services on a SharePoint farm can be usefully divided into four groups:

Windows Services

Web Services

IIS Web Services

Configuration Services

For details and examples of each of these four kinds of services, see Background: Service Entities in Microsoft SharePoint Foundation.

Services in the latter two categories can be created to conform to the Service Application Framework. Many of the services built-in to SharePoint Foundation and SharePoint Server implement the framework. Moreover, the framework lets developers build scalable middle-tier shared services that are hosted in SharePoint Foundation.

A service that conforms to the framework can be split into multiple configured farm-scoped instantiations (CFSIs). Each of these provides the functionality of the service, but each has its own individual permission and provisioning settings. A CFSI is not an actual running process on a particular server. The same CFSI can be running on multiple servers, but it is not the same as the entirely abstract service itself either. Each of the servers on which the CFSI runs has its own actual instance (a running process) of the CFSI. Moreover, more than one CFSI of a given service can be running on the same server or servers. Thus, the framework provides a means for different versions of the same basic service to be available simultaneously. A consuming application on a front-end web server can target a specific CFSI.

Figure 8 shows the services and service instances on a hypothetical 10-server farm. This figure is repeated, with a more detailed explanation of its contents, in Background: Service Entities in Microsoft SharePoint Foundation. For this article, note only that the light translucent rectangles represent services, the darker translucent rectangles represent CFSIs (also known as service applications), and the smaller solid rectangles represent instances of services. The CFSIs are present only for the services that implement the Service Application Framework, which in this example are the Usage Service, the Application Discovery and Load Balancer Service, the Security Token Service, and the BDC Service.

Figure 8. Services and service instances on a typical 10-server farm

Applications that need to consume a particular CFSI of a service do so through proxies. The front-end web server that hosts the application has a proxy to represent the service itself and a second proxy to represent the CFSI that is being targeted. The proxies are not depicted in Figure 8, but Figure 9 shows a single-server SharePoint Foundation farm immediately after installation. Note also the following:

Web services that implement the Service Application Framework are represented with a dot-bordered box. At initial installation, each has a single CFSI, sometimes called a "service application".

The service proxies belong to the farm, but each CFSI proxy (also known as a service application proxy) belongs to a web application. The content publishing web application and the Central Administration web application each have their own proxy for the Business Data Catalog CFSI, and they each have their own for the Usage and Health Data CFSI. Neither has a proxy for the Subscription or Application Discovery and Load Balancer CFSIs at initial installation.

Figure 9. Services, CFSIs, service instances, and web applications in a new single server deployment

For more information about services in SharePoint and the Service Application Framework architecture, see Background: Service Entities in Microsoft SharePoint Foundation, Service Application Framework Architecture, and SharePoint Service Application Topologies.

Site Definitions and Web Templates

What makes it possible for ordinary business users to create their SharePoint websites without the intervention of IT professionals is the fact that types of websites in SharePoint are given detailed, stored definitions. Using the SharePoint UI, users can then instantiate a particular website from any of the defined types. There are actually two kinds of definitions of site types: site definition configurations and web templates. A site definition is stored on the file system of the front-end web servers, in a subfolder of %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\TEMPLATE\SiteTemplates, as a set of XML configuration files and possibly also page files and other supporting files. A web template is stored in a SharePoint Solution Package (.wsp file) in the content database, specifically in the Solution Gallery of a site collection. Either kind of type definition specifies such things as a default home page, various aspects of the default look-and-feel and layout of the site's pages, the types of lists available in sites of the specified type, and other configuration details of the site type.

For more information, see Building Block: Web Sites and Site Collections and the Site Types: WebTemplates and Site Definitions node of the SharePoint Foundation SDK.

SharePoint Security

The SharePoint security system protects deployments from both errant users and errant code.

User Security

SharePoint Foundation supports security for user access at the website, list, folder, and item levels. Security management is role-based at all levels. The authorization process assumes that the user has already been authenticated, which refers to the process by which the current user is identified. SharePoint Foundation does not implement its own system for authentication or identity management, but instead relies solely on external systems, whether Windows authentication or non-Windows authentication.

Authentication

SharePoint supports several forms of authentication. The default is Windows claims-based authentication. The claims-based identity model for SharePoint is built upon Windows Identity Foundation (WIF). Under this model, the user presents an identity to your SharePoint farm as a set of claims. One claim could be the user's name, another might be an email address. An external identity system is configured to give SharePoint all the information that it needs about the user with each request, along with cryptographic assurance that the identity data comes from a trusted source. Other types of supported authentication include Windows classic authentication and ASP.NET forms-based authentication. For more information about authentication and SharePoint, see the Getting Started with Security and Claims-Based Identity Model and SharePoint Claims-Based Identity nodes of the SharePoint Foundation SDK.

Authorization

Access to websites, lists, folders, and list items is controlled through a role-based membership system by which users, and groups of users, are assigned to roles that authorize their access to SharePoint objects. By default, permissions are inherited in the sense that a user who has certain permissions for an object, such as a list, will have the same permissions for its child objects, such as folders and list items. However, it is possible to break inheritance and assign to users and groups a different set of permissions to a child object. A role definition is a set of rights, such as rights to read, create, or delete. A role assignment associates a user or group with a role definition.

SharePoint supports two kinds of groups: domain groups and SharePoint groups. Domain groups remain outside SharePoint control; users cannot use SharePoint to define, browse, or modify domain group membership. SharePoint groups are scoped to the site-collection level, and they can be used only within the site collection. Domain groups can be used anywhere within the scope of the Active Directory service.

For more information, see Authorization, Users, and Groups.

Code Security

Much of the code security story for SharePoint was already described in earlier sections. In this section, different parts of the story are very briefly pulled together, and some gaps in the story are filled.

Every web application runs in an IIS application pool that processes HTTP requests. The pool itself runs in a user account known as the application pool identity. In a multiserver farm, this is usually a domain user. The application pool identity is the user identity for code that runs in an IIS worker process. However, access to various SharePoint objects is determined by the permissions of the user who has made the request that is being processed. The isolation of web applications into separate application pools ensures that if one of them crashes, the others are not affected.

The IIS worker process calls assemblies that may operate under their own additional restrictions. If the assembly is loaded out of the global assembly cache, it operates in full trust. However, if the assembly is loaded out of the web application's \bin directory, it is subject to the trust limitations that are defined by a CAS policy.

If the request is for a sandboxed solution, the SharePoint execution manager that runs in the IIS worker process spawns a sandboxed worker process. The latter process runs within a highly restrictive CAS policy, is limited to a subset of the SharePoint server object model, and can only access resources within the site collection to which the solution was deployed.

If the request is for a customized site page, all Web Parts on the page must be registered as safe controls and, by default, the page is not returned at all if it contains inline code blocks.

Server Object Model in SharePoint

The server object model in SharePoint is large, and only some of the truly critical classes can be described in this article. These classes can be usefully divided into three hierarchies: the Physical Objects Hierarchy, the Content Hierarchy, and the Services Hierarchy. Each is described briefly in the following subsections with links to more extended discussions. For a synoptic overview of the major classes from all three hierarchies, see Server and Site Architecture: Object Model Overview.

Physical Objects Hierarchy

The Physical Objects Hierarchy includes classes that represent physical entities, such as servers and farms—the two most important.

A SharePoint Foundation farm, and its configuration database, is represented by the SPFarm class. A server farm consists of one or more physical servers. These may include one or more front-end web servers, zero or more application servers, and a computer running SQL Server that may be hosted on a dedicated database server or on one of the other servers in the farm. (If it is a dedicated server, SharePoint is not actually installed on it, although it is still seen as a member of the farm in the farm management UI of the Central Administration application.) A farm may consist of just a single server. If the farm has multiple front-end web servers, they are usually load-balanced. You can use any hardware or software load-balancing solution, including the built-in NLB (Network Load Balancing) in Windows Server 2008. SharePoint does not itself supply the load-balancing.

SPFarm inherits from SPPersistedObject, which means that the object (there is only one) that instantiates the class persists in the configuration database. The three most important child classes of SPFarm are SPServer, SPService, and SPSolution.

A physical server in a SharePoint Foundation farm is represented by the SPServer class. In addition to many inherited members, it has an Address property that holds the IP address of the server and a Role property that identifies the server's role in the farm. It also has a ServerInstances property that holds references to all the instances of Windows services and web services that are running on the server. SPServer also inherits from SPPersistedObject, so server objects are persisted in the configuration database.

For more information about the Physical Objects Hierarchy in the object model, see The Physical Objects Hierarchy of Microsoft SharePoint Foundation and Background: Physical Objects in Microsoft SharePoint Foundation.

Content Hierarchy

The Content Hierarchy includes classes that represent publishable items of data, such as list items. There are also classes that represent nested containers of data, such as lists, content databases, websites, site collections, and web applications.

High Level Object Model

Beneath the farm, the broadest content container is the web application, which is represented by the SPWebApplication class. An SPWebApplication object represents a content-publishing web application in SharePoint Foundation. It contains one or more content databases, which hold the data of one or more site collections. Each such web application is served by at least one (and usually only one) IIS website and typically has its own application pool in IIS. Also, each web application has its own security and authentication settings. The SharePoint Foundation object model provides some hooks into the web application's other life as an IIS object through the IisSettings and ApplicationPool properties. For more information about the relation between SharePoint web applications and IIS, see SharePoint as an ASP.NET-IIS Application earlier in this article.

The SPWebApplication class has a ContentDatabases property that holds all its child content database objects.

Every web application contains one or more content databases. Each of these is represented by the SPContentDatabase class. A content database is a SQL Server database that contains all the data (lists, list items, blog posts and comments, wiki pages, and documents in document libraries) and the customized page files that constitute the site collections that belong to the database. Objects that represent each child site collection are in the Sites property.

Both SPWebApplication and SPContentDatabase inherit from SPPersistedObject, so these objects persist in the farm's configuration database.

For more detailed discussions of web applications, content databases, and their classes, see The Content Hierarchy of Microsoft SharePoint Foundation and Background: Content Entities in Microsoft SharePoint Foundation.

Middle Level Object Model

At the middle level of the content hierarchy are site collections and their subsites. An SPWeb object represents a single website. An SPSite object represents a collection of websites within a SharePoint web application that are grouped together for mainly administrative reasons. (The SPSite class is not a collection in the sense of a class that implements ICollection.)

The SPSite class has a RootWeb property that holds its child top-level website. (Top-level websites were called "root webs" in the first version of SharePoint Foundation, Microsoft SharePoint Team Services.) In turn, the SPWeb object that represents the top-level website has a Webs property that holds all its immediate child subsites (but not the subsites of those subsites). (The AllWebs property returns all the descendent sites and the top-level website.)

Among the SharePoint elements that can be scoped to the site collection level are master pages, Web Parts, themes, lists, content types, and Features. A site collection can also be a unit of backup and restoration. It is also the level at which groups of users are created and assigned default permissions. Site collections are the largest possible scope for a search in SharePoint Foundation, although broader search scopes are possible in SharePoint Server 2010.

The content of a site collection is always included within a single content database.

Websites can be children of other websites, and all websites belong to a site collection. The SPWeb class has dozens of properties and methods for programmatic handling of every aspect of the website, including users, lists, fields, content types, Features, alerts, and much more.

For much more information about site collections and websites, see Building Block: Web Sites and Site Collections, The Content Hierarchy of Microsoft SharePoint Foundation, and Background: Content Entities in Microsoft SharePoint Foundation.

Tip:
Both SPWeb and SPSite evolved from early versions of what is now called "SharePoint" before there was a .NET Framework. Even today they wrap some COM objects. The .NET Framework garbage collector does not know how to release these COM resources. Accordingly, both classes implement the IDisposable interface. If they are not disposed of, website and site collection objects cause memory leaks on the farm servers. It is essential that SharePoint developers call the Dispose method of every SPWeb and SPSite that their code creates. There are some subtleties about disposing of such objects. For example, website and site collection objects that are obtained from the SPContext object should not be disposed of. Developers should carefully study Disposing Objects.

Tip:

Both SPWeb and SPSite evolved from early versions of what is now called "SharePoint" before there was a .NET Framework. Even today they wrap some COM objects. The .NET Framework garbage collector does not know how to release these COM resources. Accordingly, both classes implement the IDisposable interface. If they are not disposed of, website and site collection objects cause memory leaks on the farm servers. It is essential that SharePoint developers call the Dispose method of every SPWeb and SPSite that their code creates. There are some subtleties about disposing of such objects. For example, website and site collection objects that are obtained from the SPContext object should not be disposed of. Developers should carefully study Disposing Objects.

Low Level Object Model

At the heart of SharePoint are lists and list items, which are represented, respectively, by the SPList and SPListItem classes. The SPList class has members for programmatically adding, deleting, and retrieving list items, and also for managing metadata about the list such, as its content types and fields. The SPListItem class has members for managing the item's fields, the values of its fields, its content type, and any associated workflows. Lists can have folders that give them a hierarchical structure. Folders are represented by the SPFolder class.

Note that, although any given list item has a specific content type, the object model provides only the SPListItem class to represent all list items. Hence, working with list items and their fields is, in many ways, like working in a weakly typed environment. SharePoint includes a tool, SPMetal, that can generate code for an object relational mapping. The generated code defines classes for each content type in a website and, for each field in the content type, it declares a strongly typed property. The tool makes it possible for LINQ queries to be strongly typed. For example, if a developer mistakenly refers to the DueDate field of the Task content type as "DateDue", the compiler catches the mistake if SPMetal is used and the reference is to a property of a Task type object, myTask.DateDue. However, without SPMetal, the reference would have to be to a member of the fields collection of an SPListItem object, myItem["DateDue"]. The mistake would not be caught until run time.

Each field (column) on a list item is represented by an SPField object.

For more information about lists, list items, folders, and fields, see Building Block: Lists and Document Libraries, Building Block: Columns and Field Types, and SharePoint List Data Model.

Services Hierarchy

The Services Hierarchy includes classes that represent web services, Windows services, other types of services, instances of services, and CFSIs of services (see Services and the Service Application Framework earlier in this article).

A SharePoint Foundation service is represented by a class that inherits from SPService. The class provides members that get information about the jobs the service is performing. If a service implements the Service Application Framework, the SPService object has an Applications property that holds all the CFSIs of the service that are running on the farm. A CFSI of a service is represented by an object of a class that inherits from the SPServiceApplication class.

Every SPService object has an Instances property that holds all the instances of the service that are running on various servers in the farm. The instances that host a particular CFSI are held in the ServiceInstances property of the SPServiceApplication object that represents the CFSI. No more than one instance of each CFSI runs on any one server. However, a given service can have multiple CFSIs, and they can run on the same servers. Moreover, a given CFSI (and, thus, a given service) can run on multiple servers, in which case each server has its own actual instance (a running process) of the CFSI. If the service has no CFSIs, it still has instances on each server on which it runs, but there can be no more than one instance of such a service on a given server. Each instance is represented by an object of a class derived from SPServiceInstance.

Applications that consume services that conform to the framework are represented by proxies. Here, also, there is a proxy object for the service as a whole and a proxy for the particular CFSI that is being consumed. The consumer proxy for a service is represented by an SPServiceProxy object and the proxy for the CFSI is represented by an SPServiceApplicationProxy object.

Figure 10 illustrates the relationships among the main classes in the Service Application Framework.

Figure 10. Relation of major classes in the Service Application Framework

The following kinds of objects are persisted in the configuration database because these classes inherit from SPPersistedObject:

SPService

SPServiceApplication

SPServiceInstance

SPServiceProxy

SPServiceApplicationProxy

For more information about the services object model, see The Services Hierarchy of Microsoft SharePoint Foundation and Background: Service Entities in Microsoft SharePoint Foundation.

Client Object Models in SharePoint

There are three client object models in SharePoint Foundation: one each for Microsoft Silverlight applications, Microsoft .NET Framework applications, and ECMAScript. They are almost the same in the APIs that they expose. The *.js files that contain the ECMAScript client object model, like all *.js files, are downloaded automatically to the user's computer when a page that references the object model is accessed. (The standard built-in master page for SharePoint references these files.)

The assemblies that contain the Silverlight object model can be downloaded in the .xap file, but another alternative is to download at run time the file %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\TEMPLATE\LAYOUTS\ClientBin\Microsoft.SharePoint.Client.xap, which encases the Microsoft Silverlight assemblies.

The .NET Framework object model can be called only if the assemblies that contain the object model have been installed to the client computer. You must use the official redistribution package, SharePoint Foundation 2010 Client Object Model Redistributable.

These object models provide a subset of the classes in the server-side microsoft.sharepoint.dll assembly, although many of the class names have been slightly changed (usually by dropping the "SP" at the beginning of the class name). The client object models are implemented as a WCF service (.../_vti_bin/client.svc), but they use web bindings to implement efficient request batching. Commands are serialized into XML and sent to the server in a single HTTP request. For every command, a corresponding server object model call is made, and the server returns a response to the client in compacted JavaScript Object Notation (JSON) format, which the proxy parses. The client APIs provide a familiar, object-oriented interface to the WCF service, so developers are shielded from the details of the service. In particular, the client-side runtime handles all communication between the clients and server.

Microsoft.SharePoint.Client is the core namespace used for the Microsoft .NET Framework managed and Silverlight object models, and SP is the core namespace for the ECMAScript object model. Client objects inherit from the ClientObject class (ECMAScript: ClientObject).

SharePoint code on a client begins by retrieving a client context object that represents the current request context, and through this context, you can obtain access to client objects at the site-collection level or lower in the SharePoint Foundation hierarchy. Client objects inherit from the ClientObject class (ECMAScript: ClientObject), and you can use them to retrieve properties for a specific SharePoint object, to retrieve child objects and their properties, or to retrieve child items from a collection.

After a client object is obtained, a query is defined and loaded. This is followed by a call of the ExecuteQuery method or the ExecuteQueryAsync method (ECMAScript: executeQueryAsync) to send the query to the server. The query commands are packaged as XML and processed on the server by the WCF service, Client.svc, which runs the batched commands by calls to the server-side object model. Returned data is sent back to the client by the service as JSON data.

Figure 11 shows the major components and events of the client's interaction with the server when the client object models are used.

Figure 11. Client and server interaction with the client object models

Note:
The Silverlight client object model in SharePoint is not supported on Windows Phone 7.

For more information about the client object models, see the following topics:

Using the SharePoint Foundation 2010 Managed Client Object Model

Using the Client APIs

Managed Client Object Model

Object Model Hierarchy and Identity

Client Context as Central Object

Client Application Models in SharePoint 2010

Note:
The client object models are not the only way to query SharePoint list data from a client application. SharePoint also supports a WCF Data Services (formerly ADO.NET Data Services) interface to the list data. For more information, see Data Model, Data Management, and Query System earlier in this article. The section Low Level Object Model earlier in this article mentions that the server object model treats content types and their field types in a weakly typed manner, but that the SPMetal tool enables strongly typed programming against these entities. A parallel point applies to client-side computing: Content types are weakly typed in the client object model, but they are strongly typed when you are programming against the WCF Data Services interface.

Note:

The client object models are not the only way to query SharePoint list data from a client application. SharePoint also supports a WCF Data Services (formerly ADO.NET Data Services) interface to the list data. For more information, see Data Model, Data Management, and Query System earlier in this article. The section Low Level Object Model earlier in this article mentions that the server object model treats content types and their field types in a weakly typed manner, but that the SPMetal tool enables strongly typed programming against these entities. A parallel point applies to client-side computing: Content types are weakly typed in the client object model, but they are strongly typed when you are programming against the WCF Data Services interface.

Workflows in SharePoint

Like all workflows, SharePoint workflows model and systematize business processes. However, SharePoint workflows are generally oriented around the SharePoint list data model. In most cases, each instance of a SharePoint workflow embodies processes that surround an item in a list or document library. Indeed, the most common kinds of SharePoint workflow are workflows that are associated with a list or with one or more content types. Some SharePoint workflows are manually started, but they are most commonly designed to start automatically in response to some event connected with a list item or library item, such as adding, deleting, or updating an item.

There are also workflows associated with websites rather than lists or content types. These are always started manually (or programmatically), not automatically in response to some event. In addition, they are generally used to systematize processes that transcend particular list items, such as retrieving and setting values of multiple list items across different lists, or performing non-list operations, such as creating and configuring subsites.

Note:
SharePoint Designer envisages two Reusable and Globally Reusable workflows. These are really just list workflows that you can use on multiple lists without having to re-create them for each of those lists.

Workflows in SharePoint are built on the framework provided by Windows Workflow Foundation (WF). When a workflow is running in SharePoint, the WF runtime engine is hosted in the SharePoint process. The WF runtime engine loads and unloads workflow templates and provides sequencing and persistence for workflows. The persistence services are crucial because they enable workflows to remain active through discontinuous user sessions, through resets of the front-end web server, and even through reboots of the server.

For many of its services, WF enables customization by hosting applications. SharePoint provides custom implementations of the following services for the engine: transaction, persistence, notifications, roles, tracking, and messaging. For example, when a workflow instance reaches a point at which it is waiting for user input, SharePoint unloads that workflow instance from memory and persists its data in the content database. Then, when an appropriate event occurs that requires that the workflow instance start again, such as when a user enters input, SharePoint re-instantiates the workflow instance by using persisted data, so the workflow instance can receive and handle the event as necessary.

A SharePoint workflow type is represented by two entities that work in partnership with each other. First, there is a workflow definition. These can be defined in code and compiled into an assembly, or defined in XOML markup. When the workflow type is defined in XOML, it is called a declarative or "no code" workflow, and it persists uncompiled in the content database until it is called, at which point it is compiled just-in-time for use. The workflow definition, in either compiled or declarative form, specifies the parameters, events (such as activation), and sequence of activities in the workflow. It also defines the branching structures of the workflow and the conditions that determine the paths of execution.

The partner of the workflow definition is the workflow template definition. A workflow template definition is an XML file that contains the information SharePoint requires to instantiate and run the workflow, such as the following:

Name and description of the workflow

Class within the workflow assembly to call

Identity of the workflow assembly

Location of any custom forms used in this workflow

You can store these XML files in either of two locations. The first is %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\TEMPLATE\lcid\Workflow. For example, the template definition of the built-in moderation workflow is stored in the moderationworkflow.xml file. The second location is the Workflows list of the root website of a site collection. The built-in Three-state workflow is an example. Regardless of where it is stored, at run time, the workflow template definition is used to create an SPWorkflowTemplate object, which is cached to speed up the creation of workflow instances.

You can associate workflow templates with lists (including document libraries), content types, and websites. For any workflow template that is associated with a list or content type, an item of that list or content type can have an instance of the workflow in progress. Although there can be only one instance of a specific workflow type in progress at any one time for a given item, any item can have multiple workflows in progress, each of a different type.

You can create compiled workflows by using the Microsoft Visual Studio workflow designer. You can also create declarative workflows by using SharePoint Designer. Workflows are installed as Features at the site collection level. But you can also "publish" a workflow directly from SharePoint Designer to a website, in which case there is no Feature.

A workflow type is available for a list, content type, or website only if it is associated with the list, content type, or website. These associations are stored in the content database of each site collection. This association data typically includes whether the workflow is started automatically or by users. If a workflow is added to multiple content types, lists, or websites, it will have one entry for each such association. Likewise, if you add multiple workflows to a specific content type, list, or website, the database contains one entry for each workflow that is added to the content type, list, or website.

For more information about the SharePoint workflow infrastructure, see Introduction to Workflows in SharePoint Foundation.

Conclusion

Even an article as long as this one cannot cover all the SharePoint architectures or say everything that could be said about the architectures that it does cover. It does provide an overview of the most important SharePoint architectures and should enable you to do further research on each of them.

Difference between WSS and Sharepoint

2011-01-04T23:24:00.000-08:00

1. WSS is more affordable as it is shipped with Windows Server 2003 or Microsoft Small Business Server.

2. MOSS costs more than WSS because it requires its own licensing, and it requires purchases of other software and optional hardware.

3. WSS is suitable for small to medium projects, while MOSS is for highly budgeted, larger projects.

4. MOSS is actually based on the WSS platform, and only expanded with additional components.

5. WSS is limited, but much simpler to employ compared to the extensive but more complicated MOSS.

Sharepoint 2010 architecture diagram

2011-01-04T22:32:00.000-08:00

Dot Net 2.0 New Features

2009-08-05T23:14:00.000-07:00

A Look at ASP.NET 2.0's Provider Model

2009-06-26T06:45:00.000-07:00

Introduction
Back in 2001 I started work on an ASP.NET online messageboard application called WebForums.NET. The idea was to create an ASP.NET-based messageboard system that could easily be plugged into an existing website. One particular challenge building such an end-to-end application was providing a means for customers to be able to integrate it within their system. For example, clearly an online forum needs some sort of data store to store user information, forums, posts, and so on, but what's important is not to lock the customer into a particular data store. That is, you shouldn't say, "I'm going to have my application use Microsoft SQL Server 2000," because at that point, how are customers who use Oracle or Access going to be able to use your software?

Another issue is with integration into a customer's existing data. All online forum sites have user accounts and a means for creating new accounts; this is typically modeled in the forum's architecture as a Users table in the database. But customers might already have their own user-related database tables that already have thousands of user accounts. Or a customer might want to use the forums in an intranet setting, authenticating and storing user information against Active Directory, rather than some database table. When a forum software system creates a Users database table and says to the customer, "This is how you will store users," they alienate those customers who already have existing infrastructure and user data. Clearly this challenge faces any company or developer that's creating software to be used "in the wild."

This particular challenge arises when you build a system with a rigid API implementation. Rather than providing a means to customize the logic, a rigid API implementation hard codes the implementation details - you will use SQL Server as your backend data store... you will have a Users table in this database where all user information will be stored. This rigidity, however, can be broken quite easily using the provider design pattern. With the provider design pattern the system architect merely defines the API, the programmatic functionality offerred by the system. For an online forum application, this might include a Users class with methods like Authenticate(username, password) and GetUserDetails(username).

The beauty of the provider model is that the customer implementing the solution can specify a custom class that the system should use. This custom class must implement the system's well-defined API, but it allows for any custom implementation to be seamlessly plugged in. That is, once this API is defined, the system implementor can create a default concrete implementation - one that uses SQL Server and a Users table - that most customers can use without modification. Those customers that have a custom need - those that want to use Oracle or have user data stored in some other manner - can create classes that provide the necessary functionality and plug them into the system.

The provider design pattern is used throughout ASP.NET 2.0. There are also guidelines on how to provide this functionality in ASP.NET 1.x applications. In this article we'll examine the provider model and see how it's used in ASP.NET 2.0. Read on to learn more!
Breaking the Rigid API ImplementationFairly early on in my development of WebForums.NET I realized that the rigid API implementation was going to be a problem. Part of my design goal for the software was to make it as flexible and customizable as possible, and boxing users into using SQL Server and my implementation of the users data model seemed restrictive, at best. Unfortunately, I didn't have any insight into this problem, but luckily Andy Smith did. He suggested (and built!) a system that involved two pieces:
A set of abstract base classes that defined the system's core functionality, and
Code that would, at runtime, dynamically load a specified class that extended the abstract base class and use its functionality. Specifically, the code inspected the Web.config file, which contained a that gave the fully-qualified name for the class to use. With this architecture, I could define the system's functionality through a series of abstract base classes, provide concrete implementations of these classes using SQL Server 2000 and the Users table. Customers that were happy with this configuration could just use the application as-is and everything would work fine without them needing to write a line of code. Those developers who needed to customize the solution for their unique needs, however, could do so by creating their own classes that derived from the appropriate abstract base class(es). They could have this new class used by the system by simply dropping the assembly in the application's /bin directory and updating the Web.config file.
Specifically, WebForums.NET shipped with an abstract base class called DataProvider that spelled out all of the methods in the system, akin to:
public abstract class DataProvider
{
public abstract bool AuthenticateUser(string username, string password);
public abstract User GetUserInfo(string username);
...

public static DataProvider Instance()
{
...
}
}
The AuthenticateUser(username, password) and GetUserInfo(username) methods spelled out two of the many methods defined by the system. The static Instance() method was the workhorse of the DataProvider class. It contained the code that inspected the Web.config file for the WebForums.NET configuration information, which indicated the fully-qualified name of the class the system was to use. The method then used reflection (with caching) to create an instance of that class and return it to the system.
WebForums.NET shipped with a class called SqlDataProvider that extended the DataProvider base class, providing concrete implementations for the assorted methods. For example, all methods in SqlDataProvider worked with data stored in a SQL Server 2000 database; the user-related methods worked with a pre-defined Users database table. A customer who wanted to change the backend functionality could create her own class that derived from DataProvider, indicating in the Web.config file that their custom class should be used. For example, the Web.config in WebForums.NET would include the following content:
This setting information, by default, referenced the SqlDataProvider class that shipped with WebForums.NET. However, if a customer created their own implementation of the API, they could provide their class's details here, and the system would automatically start using their implementation in lieu of the default implementation.
With this architecture in place, the page developer working with the WebForums.NET could use code like the following to authenticate a user:
if (DataProvider.Instance().AuthenticateUser(username, password))
// User is authenticated
else
// Username/password invalid!
When the DataProvider.Instance() method is called, the configuration file is examined and an instance of the appropriate class is returned. This would be the default SqlDataProvider class if the customer has not created their own implementation; it would be their own class if they did. Once the DataProvider.Instance() method returns an instance of the appropriate provider, we can simply invoke the members of the API, AuthenticateUser() in this example.

The Benefits of the Provider ModelThe provider model offers a number of benefits. First, note that there's a clean separation between the code and the backend implementation. Regardless if whether or not the code to authenticate a user is done against a SQL Server 2000 database's Users table, or if it's done against an Active Directory store, the code from the page developer's perspective is the same: DataProvider.Instance().AuthenticateUser(username, password). The backend implementation changes are transparent.
Since system architects are strongly encouraged to create a default concrete implementation, the provider model offers the best of both worlds: to those who are content with the default implementation, the system just works as expected; for those that need to customize the system, they can do so without upsetting the existing code or programmatic logic. This design pattern also makes prototyping and agile development a lot easier. For example, in the early iterations of working with the system, it might be easier to just use the default implementation. However, later you suspect you'll need to cusomize certain aspects in order to integrate the work with your company's existing systems. When that time comes, you can achieve the needed customization through the provider model, meaning your earlier work need not be changed to reflect the backend implementation changes.
Like many good design patterns, the provider model also affords separation of duties among developers. One set of developers can be tasked with mastering the system's API, while others can be tasked with focusing on the backend implementation and customization. These two groups can work on the system without stepping on one another's toes. Furthermore, if the system being worked on is an industry standard - like ASP.NET 2.0 - skills from both tasks can be easily carried over into future jobs.
The Provider Model in ASP.NET 2.0ASP.NET 2.0 utilizes the provider model throughout its architecture. Many of its subsystems - Membership, Site Navigation, Personalization, and so on - utilize the provider model. Each of these subsystems provide a default implementation, but enable customers to tweak the functionality to their own needs. For example, the Site Navigation piece of ASP.NET 2.0 allows a page developer to define the navigational structure of their website. This data can then be used by a variety of Web controls, to display site maps, breadcrumbs, treeviews, or menus that highlight the site's navigation and/or show the user's location in the site. In addition to navigation-related Web controls, the site navigation API provides a bevy of methods for interacting with the website's navigation information
By default the site's navigational information must be encoded in a properly-formatted XML file. This is the data store that the default site navigation is hard-coded to use. However, ASP.NET 2.0's provider model makes it easy for you to use your own data store for site navigation. For example, in a project I'm currently working on a database contains information on what pages exist in the site and what permissions various users have for those pages. Rather than redefining this information in an XML file and having to try to keep two copies of this information up to date, in order to utilize the site navigation in ASP.NET 2.0 I will be able to simply create a provider class that works directly with the database information. Once this class is created and specified in the website's configuration, the navigation Web controls will be working against the application's custom navigation information stored in the database. (Note: as of the time of this writing, this project is still in ASP.NET 1.x. However, this example hopefully illustrates the benefit of the provider model.)
Personally I think that the provider model is one of ASP.NET 2.0's greatest features for migration. ASP.NET 2.0 offers a lot of new features that developers had to custom bake in 1.x. If these new features in 2.0 used rigid implementations, it would dissuade migration from 'living' 1.x applications that used custom solutions since many of the new ASP.NET 2.0 Web controls use these new subsystems. With the provider model in place, however, we can upgrade our 1.x apps to 2.0 and create a provider to have 2.0's new subsystems integrate with our custom baked solutions. That means when moving to 2.0 we can use the new Web controls and have them seamlessly use our existing systems thanks to the provider model.

Source: http://aspnet.4guysfromrolla.com/articles/101905-1.aspx

Introduction to ASP.NET 2.0 Roles and Membership

2009-06-25T23:45:00.000-07:00

Even in the days of ASP.NET 1.0/1.1, as a developer you might have spent a lot of time creating login / authentication modules and pages. You spent a lot of time designing and coding pages for user management. Till the last version you have to write and implement a login framework, authenticate users against a database, written code to react to the currently logged-in user. But with the Inception of ASP.NET 2.0, you will have some great tools available to help implementing such framework. Though you still have to write some code, but the initial pain of implementing has been eased with server-side controls provided with ASP.NET 2.0. With the new UI server side controls you can secure your web-site, and manage users. ASP.NET 2.0 has inbuilt site tool to configure and manage users and roles.In this article we will explore this great feature of ASP.NET 2.0. First we need to create a web app. Before that make sure SQL EXPRESS is up and runningFor this web app create one Master Page, one Default.aspx, CreateUser.aspx, ChangePassword.aspx, Login.aspx. The master page design should look like Figure 1. (You can change the same but for the demo using the format as shown in Figure 1)

Figure 1 - The Master Page Design

We will start now clicking the ASP.NET Configuration button. The button is placed at the upper right corner of the Solution Explorer (see Figure 2).

Figure 2 - Starting the ASP.NET Configuration

On clicking the button we are taken into the configuration page (see Figure 3), which is opened in the Internet Explorer

Figure 3 - The ASP.NET 2.0 Site Configuration Page

Now click the security link. The security configuration page will appear (see Figure 4). We need to do a couple of things as your site is enabled to support only windows authentication.

Figure 4 - Configuring Security for the ASP.NET Application

Click Select authentication type link.

Select From Internet option and click the done button (see Figure 5). Now you are back to the earlier page. Here click Enable Roles. As soon as you click the link, you will see that Create or Manage Role link gets activated. Click the link to create couple of roles. One as administrators and another as users. Now click the back button to come to the Home Page.

Figure 5 - Configuring Application Security Access to be from the Internet

Now let us create some users. Say one user as admin and password …anything of your choice, and put under administrators role. Remember Password length is minimum: 7 and one Non-alphanumeric characters required, i.e. characters like @, $, ! etc…..special characters.
At this point you can close the site and return back to the project. Now refresh the solution explorer. There you will find that one folder named App_Data has been automatically created and contains one SQL Database.
. That database contains the user name and password that you have created and will be creating in the future through ASP.NET Configuration tool or through your APP. The password stored in the database are encrypted by one-way Hash algorithm. The Site will be very much secured site and as a developer you don’t have to do much.Figure 6 - The Login Control

Now lets go back to our project and we will be using some special server controls shown in the LOGIN tab of the toolbox (see Figure 6). Open the default.aspx in the design. Drag and drop the LoginView control in the content place holder and the crate hyperlink of Login.aspx and CreateUser.aspx by simply drag and drop the two pages. The page should look like the Figure 7.Figure 7 - A Page Containing the Login Control

Now click the Smart tag in the control and select LoggedIn Template. Then write: "Welcome and drag and drop the LoginName control."
Now create a hyperlink to ChangePassword.aspx by simply dragging and dropping the control onto the form.
Now open the login.aspx and set up a login form. Drag and drop the login control from the toolbox. At this point you can run the app and see the outcome. Now lets create another page named ForgotPass.aspx. This page will help the user to retrieve the password. Now drag and drop the Password Recovery control from the toolbox. Now go back to the login page and open the property of the login control. Find the property named PasswordRecoveryText and type in “Forgot Password” and then select the PasswordRecoveryURL property and point to the ForgotPass.aspx. We will need to change some property and also need to go back to the ASP.NET Configuration page to make the Password Recovery works later.
Now open up the ChangePassword.aspx page and drag and drop the changePassword control form the toolbox (see Figure 8). That’s it. Your change password component is up and running.

Figure 8 - ChangePassword Control on a Page
Now let us work on the create user page. Open it and drag and drop CreateUserWizard Control (see Figure 9).

Figure 9 - The CreateUserWizard Control

Now run the app to see that without writing a single line of code how your role and membership of a site is ready.Before we conclude, there are some changes to made in the ASP.NET Configuration page to enable password recovery system to work properly.Open that page again and select Application Tab. There you will see configure e-mail settings. Click that and put in your SMTP server details.
Then close the configuration page and open the Password Recovery Page. In the property of the Password Recovery Control enter in the details as shown in Figure 10. (Change according to your Mail Definition). Now you are up and running.

Source: http://www.wwwcoder.com/tabid/68/type/art/parentid/258/site/6297/default.aspx

Easy way to create customized Menu Navigation using CSS and dot net

2009-06-18T02:26:00.000-07:00

There are several ways to develop Tab Menu navigation in web applications. One of the easier ways is to use Telerik Menu control, customize it and incorporate it with DNN which can show highlighted on mouse over, show highlighted on selection etc. Obviously, Telerik is a leading vendor of User Interface (UI) components for Microsoft .NET technologies through which we can create customized menu panel to the web application. But, for those who cannot afford to purchase Telerik controls could create a simple CSS based menu navigation through which they can highlight the menu on mouse over, show the selected menu on click using some JavaScript coding or by passing query strings in the URL of the menu which is clicked. And on page load, read the query string value and show the menu as selected.

This procedure can be slightly fine tuned to use dot net to perform the job using CSS with very less coding.

STEP 1: Create a dot net user control and place the part of the css code which would display the menu navigation panel. See Fig: 1.1.

Fig: 1.1

In this sample, a div tag is used for the menu panel followed by UL and LI tags. Let’s discuss with the first line item in the example:
UniqueID=="ctl00$ContentPlaceHolder1$pri_home"?"tabCurrent":"tabnormal"
Id: which holds the id name and definition of the list item specified in the css class which would show highlighted or not.

UniqueID: As we all know that the UniqueID property is the page-wide unique identifier of an ASP.NET server control. Its uniqueness is guaranteed by prefixing the ID of a server control with the ID of its NamingContainer. If the NamingContainer is the Page the UniqueID will remain the same as the ID.

ctl00$ContentPlaceHolder1: the default label created to the user control when it is rendered.

pri_home: id given to the user control while registering in the web page which has the user control. We need to create different id as specified in the user control and the same needs to be updated while registering the usercontrol in each page. See fig: 1.2
This web page would be displayed if ‘Home’ link is clicked in the user control. In fig: 1.2, the user control is included in the home page and the ID of the user control is given ‘pri_home’. Once the home link is clicked, the home page would be called and the user control with ‘pri_home’ will be rendered as ctl00$ContentPlaceHolder1$pri_home and the UniqueID will have this value during page load.

Fig: 1.2

STEP 2:
We have already creating a condition to set “tabCurrent” as the defnition for LI if the UniqueID is ctl00$ContentPlaceHolder1$pri_home” and “tabnormal” for other menus. “tabCurent” and “tabnormal” are defined in the external css to show the selected menu hightlighted and normal respectivley. See fig: 1.3

STEP 3:
Fig: 1.3

This code would not only show the selected menu highlighted, but would show the very first menu in the menu to be shown highlighted by default. The same can be achieved for sub menus in the same page.

Dot Net - What's New in the .NET Framework Version 3.5 SP1

2009-06-17T23:13:00.000-07:00

What's New in the .NET Framework Version 3.5 SP1

This topic contains information about new and enhanced features in the .NET Framework version 3.5 Service Pack 1.
ASP.NET
New ASP.NET features include ASP.NET Dynamic Data, which provides a rich scaffolding framework that allows rapid data driven development without writing code, and an addition to ASP.NET AJAX that provides support for managing browser history (back button support). For more information, see What’s New in ASP.NET and Web Development.
Common Language Runtime
Core improvements to the common language runtime include the following:
· Improved application startup and working set performance.
· Better layout of .NET Framework native images.
· Opting out of strong-name verification of fully trusted assemblies.
· Better generated code that improves end-to-end application execution time.
· Detecting approaching full garbage collections with Garbage Collection Notifications.
· Opting for managed code to run in ASLR (Address Space Layout Randomization) if supported by the operating system.
· Managed applications that are opened from network shares have the same behavior as native applications by running with full trust.
.NET Framework Client Profile
The .NET Framework Client Profile is a subset of the full .NET Framework that targets client applications. This improves the installation experience on computers that do not already have the .NET Framework installed.
Networking
Enhancements have been made that affect how integrated Windows authentication is handled by the HttpWebRequest, HttpListener, SmtpClient, SslStream, NegotiateStream, and related classes in the System.Net and related namespaces. Support was added for extended protection to enhance security. The changes to support extended protection are available only for applications on Windows 7. The extended protection features are not available on earlier versions of Windows. For more information, seeIntegrated Windows Authentication with Extended Protection.
Windows Presentation Foundation
Performance improvements have been made to Windows Presentation Foundation, including a faster startup time and improved performance for Bitmap effects. Additional functionality for Windows Presentation Foundation includes better support for line of business applications, native splash screen support, DirectX pixel shader support, and the new WebBrowser control.
ClickOnce
ClickOnce application publishers can decide to opt out of signing and hashing as appropriate for their scenarios, developers can programmatically install ClickOnce applications that display a customized branding, and ClickOnce error dialog boxes support links to application-specific support sites on the Web. For more information about ClickOnce, see ClickOnce Deployment for Windows Forms Applications.
Accessing Data with ADO.NET
The .NET Framework Data Provider for SQL Server (System.Data.SqlClient) provides full support for all the new features of the SQL Server 2008 Database Engine. For more information about .NET Framework support for SQL Server 2008, see New Features in SQL Server 2008 (ADO.NET).
The ADO.NET Data Platform is a multi-release strategy to decrease the amount of coding and maintenance required for developers by enabling them to program against conceptual entity data models. This platform includes the ADO.NET Entity Framework, the Entity Data Model (EDM), Object Services, LINQ to Entities, Entity SQL, EntityClient, ADO.NET Data Services and Entity Data Model Tools. For more information, see ADO.NET Technology Options and Guidelines.
Windows Communication Foundation
The Windows Communication Foundation (WCF) now makes the DataContract Serializer easier to use by providing improved interoperability support, enhancing the debugging experience in partial trust scenarios, and extending syndication protocol support for wider usage in Web 2.0 applications.
Windows Forms Controls
The Microsoft.VisualBasic.PowerPacks namespace introduces the new DataRepeater control that displays data in a customizable list format. This namespace also includes new vector shapes.
Source: http://msdn.microsoft.com/en-us/library/cc713697.aspx

infopathemailedform

2008-11-07T00:45:00.000-08:00

infopath

2008-11-03T23:32:00.000-08:00

corel to svg - problem while exporting to svg

2008-10-08T22:05:00.000-07:00

1) This is how the corel file looks. I've one bitmap background image to the left, one to the right which has a transparent part in the center to display the powerclip and a image inside the powerclip. Here the powerclip is set the order of "To back of the page" so that the border flowers in the background image comes to front.
2) once the corel is exported as an svg, these are the following images getting created.
img1:this is the image inside the powerclip, which is correct.
img2:this is the background image which has a transparent part in the center to display the powerclip. This should be created without the powerclip image. So, this is not correct. This image should be created without the powerclip image. I'm replacing the powerclip image from my dot net application using javascript. As the background image itself is created as a single image with hpowerclip image , if i replace the powerclip image, i cannot see the image getting changed as the same image is there in the background image also.
img3: this is the left background image. which is correct.