FAB40.....not so fabulous after SharePoint 2010 Upgrade

I don't mean to belittle the famous FAB40 templates, they were valuable addition to any MOSS2007 farm and users were writing poems, praises and a love song here and there in their honor. However, upgrading these farms to SharePoint 2010 come with their own set of problems.

When trying to do DB attach upgrade, I kept getting thousands of errors in the new farm, and lots of sites would simply not come up. After some googling, I run across Joel's post that suggests deploying the needed legacy FAB40 solution files, followed by hiding site templates and resolving duplicate content types.

BTW, if you are wondering how to hide site templates from your users (the easy way that is), go to C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\TEMPLATE\1033\XML and edit xml files deposited by these solution packages.

File names kind of match the solution, but the dead giveaway is DisplayCathegory="Application Templates" flag. Just change Hidden flag from FALSE to TRUE (on all WFEs that is).

Anyways, back to the reason for this post.....if you have good remedy for short-term memory problem, please do share :).

After the upgrade and the universe being at peace for a month or so, some user was trying to create site template and reuse it elsewhere. Creating a template would fail with:

Error exporting the site field named "Asset" 

Suspecting FAB40, I retracted all solutions from the farm, the error was gone, and site template creation started working OK.

As I was getting ready to pop the cork and call it a day, the most unfortunate news came in....now what!?

Yes, the site template is there but we are now getting this error when trying to create a site based on it:

The content type name ‘$Resources:er,ExpenseFormsTitle’ cannot contain: \ / : * ? “ # % < > { } | ~ & , two consecutive periods (..), or special characters such as a tab.

After playing with applicationtemplatecore.wsp solution package, I eventually realized that:

a) in order to create site template, the applicationtemplatecore.wsp needs to be RETRACTED
b) in order to use this site template, the applicationtemplatecore.wsp needs to be DEPLOYED

As much as I like looking at Central Administration, doing this every single time someone needs site templates just wouldn't cut it, so it was time to call the backup, aka Microsoft Support.
The guy I worked with was great and right on the spot.

So for all of you dear SharePoint admins, here's the solution (minus call to MS support):

1. deploy applicationtemplatecore.wsp

2. look for er.resx under C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\Resources

3. save it somewhere, and retract applicationtemplatecore.wsp (and the rest of FAB40 if you have them) - this will remove files from the folder

4. copy er.resx back under C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\Resources


As of right now, the fix looks promising and the universe is back at peace. But then, popping the cork on the champagne may anger the SharePoint gods, so for now I'll just keep it on ice.

User Profile Service Synchronization Timeouts

If there is one service application in SharePoint 2010 that's anything but boring, that's certainly the User profile Service App. I was recently building out a farm and after the UPS synchronization connection was successfully built, I tried importing user profiles form AD.

Looking at the  Forefront Identity Manager applet located at C:\Program Files\Microsoft Office Servers\14.0\Synchronization Service\UIShell\Miisclient.exe , I noticed timeouts.

DS_FULLIMPORT step would start and after about 5-10 min would fail with stopped-connectivity error, WCF Endpoint failure.

I was on the phone with MS for couple of hours going through the logs, and trying to sync over and over (and over) again, each time being a failure.
The proposed solution was to increase timeouts values for service endpoints. Apparently, each SharePoint 2010 service has two endpoints (bindings), one for HTTP and another one for HTTPS, and their configuration files are located under:

C:\Program Files\Microsoft Office Servers\14.0\WebService


Now I am sure if you are a developer you probably already knew this, but infrastructure guys like myself don't often have to deal with the inner workings of web services. Anyways, this is where these files are located, and you are looking for web.config

Look for

There are two bindings: HTTP/HTTPS, 4 timeout intervals each: receive, send, open and close, all 20 sec by default. We had them increased to something like 5 minutes each. 


There are couple of more locations the same process was repeated:

C:\Program Files\Microsoft Office Servers\14.0\WebClients\Profile\client.config

C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\WebClients\client.config 

Indeed, the UPS sync was more robust and did not time out as quickly as before. What I think ultimately was a fix (or part of the resolution at least) was very simple. I looked at another farm which was importing AD profiles OK, and compared the connections in Miisclient.exe side by side. 

Turns out, the farms were attaching to different domain controllers when trying to pull profiles, and my theory is that the problem child was looking at a DC that was either particularly slow, or at a distant physical location. 

I don't want to assume the latter, since I would think that any MS product would be site-aware and always use local domain resources based on SRV records and Domain sites and services subnet information. So I'll just blame it on a bad DC :)

All I had to do was to specify the same DC that the "good" farm was using, when building the synchronization connection in the "bad" farm. I never had to do that before, but hey, whatever it takes to get it fixed. After that, UPS Sync was running with no issues, and I was finally able to import my profiles.  

Change password for managed service accounts in SharePoint 2010

Sometimes you need to change service accounts password for a farm after it was built and configured for various reasons:

1. security team hounds you and you have to periodically change service accounts passwords

2. your SharePoint admin guy leaves the company and you don't trust him/her

3.consultant comes in and configures your farm (refer to number 2 above)

4. you are of the Chicago's school of thought "Vote early and vote often" and extend it to service accounts password security......

There are couple of blog posts out there that talk about how to change passwords for SharePoint 2010 manages service accounts. You can do it using the GUI (Central Admin) or powershell.

I actually had to go through this exercise recently, and what I found is that certain service applications will stop working after the passwords have been updated. Specifically, the user profile synhronization service stops (big surprise), and the search service application enters paused state.

After you change the passwords, you should probably check and make sure the underlying services are running. The two Forefront Identity Management services on which UPS depends will be stopped. Don't try to start them manually, instead do it through Central Administration > Manage Services on Servers. You will be prompted to re-enter the password for your farm account.

Additionally, you may need to update the password for SharePoint Server Search 14 service. In my case it was straightforward, I rebooted my index server and the service was stopped complaining of logon failure. I just updated the credentials in services.msc snap-in and started the service manually.

There is one more step to get search working again, and that is to resume the search service application from powershell.

Run get-spserviceapplication, this will give you list of all service app IDs.

Copy the ID for your search service app and then run:

resume-spenterprisesearchserviceapplication -identity .

This will resume the service app, and your scheduled crawls will start running again.

FAST for SharePoint Cluster Configuration

I spent considerable time last week trying to build 2-node FAST cluster for SharePoint 2010 implementation. it was my very first FAST cluster implementation, so cut me some slack people, will you. 

Creating the setup.xml file and building the admin FAST node was the easy part, but trying to connect the non-admin node is what took up most of my time.

(The default FAST installation creates 3 example files under C:\FASTSearch\etc directory, they are named deployment.sample.single.xml, deployment.sample.multi1(2).xml.  Most of the time you can tweak them to suit your needs instead of trying to create on from scratch.)

When running the configuration wizard, the error I kept getting was:

{drive}:\FASTSearch\bin\MonitoringServiceConfig.exe” Output – Error: The file ‘{drive}:\FASTSearch\etc\middleware.cfg’ was not found.

I searched through the blogs for quite some time, and I should here thank all the bloggers out there which provided useful information (including our very own David Sobiecki), links provided at the bottom here.This is the summary of things I did or checked for all in one place: 

1. Issue ncrtl status on the FAST admin node: we need to make sure all services show status running
2. Try running setfastsearchipsec -create on the non-admin FAST node: this will prompt connection to the FAST admin node and parsing the setup.xml file. Depending on the error reported it's usually either connectivity issue to the FAST admin node, or incorrect setup.xml formatting

3. Recreate the setup.xml file using notepad: apparently fancier XML editors add invisible characters that FAST configuration wizard doesn't like, so to be safe you should paste content on notepad and quickly proofread for any characters that are not supposed to be there

4. Disable firewall on the FAST Admin node: now I know this is not best practice, but in my case it worked. FAST configuration wizard will complain if the firewall is disabled, so I tried many combinations. Having the firewall enabled on the non-admin node and disabled on the admin node is what worked for me
5. Finally, unselect browser proxy settings (automatically detect settings, use proxy server etc.) on the non-admin node

Hope this summary will help other people who run into similar issue. 

Happy FASTing people!

If you feel inclined to read more, here is a list of blogs I used:

http://davemcmahon81.wordpress.com/2011/07/30/adding-a-non-admin-server-to-fast-search-for-sharepoint-2010-fails/

http://blogs.technet.com/b/speschka/archive/2010/05/25/solving-configuration-failures-with-non-admin-nodes-in-fast-search-for-sharepoint.aspx

http://blogs.msdn.com/b/scottos/archive/2010/07/16/we-need-to-talk-aka-it-s-not-me-it-s-you.aspx

http://social.technet.microsoft.com/Forums/en/sharepoint2010setup/thread/f80c1daf-50d0-4d33-84b5-c5533a6a129f

https://dotnetmafia.sys-con.com/node/1855150/mobile

http://blogs.pointbridge.com/Blogs/Sobiecki_David/Pages/Post.aspx?_ID=3

Fixing orphaned sitesFixing orphaned sitesTuesday, July 05, 2011

Before upgrade to SharePoint 2010 you may see orphaned sites reported in your upgrade logs. 

Now, stsadm.exe -o databaserepair is supposed to help you fix these references and get rid of the orphaned sites.  The keyword here being "supposed to"......If that fails, you can always find and delete these sites manually. 

First, enumerate all sites for a particular content database: 

C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\12\BIN>stsa

dm.exe -o enumallwebs -databasename SharePoint_Home_Content >> C:\AllSites.txt

What we are looking for is  InSiteMap  flag set to NULL, indicating that the particular site in question has no parent site, and is therefore orphaned. This is the TechNet article article that  about it. 

Record the GUIDs of the orphaned sites, and delete them one by one, by running: 

stsadm -o deletesite -force -siteid < Orphaned Site Collection ID > -databasename

Then, run the following command to map the remaining copy to the site map:

stsadm –o refreshsitemap

Now, just run the preupgrade checker one more time, and as you see no orphaned sites tap yourself on the back  for job well done :). 

SharePoint 2010 Search Method failed with unexpected error code 3

I run into very interesting problem when trying to modify the topology of my search service application. I was basically trying to move my administration component and index role to another server in the same farm, but the wizard kept crashing with TimerJob timeout error.

They say the definition of madness is trying the same thing over and over again and expect different result....what can I say, I tried at least 10 times. Draw your own conclusions here. 

When looking at my target server (where I was trying to place my admin and index role), the search service would be in status Error Staring and the application server log would show

Search Method failed with unexpected error code 3.

I then I decided to give a try to our old friend stsadm:

stsadm.exe -o osearch -action start but to no avail. Same error message. Off course, no amount of reboots and timer service restarts helped either.

Finally, I found this blog that talked about the underlying file structure and the share that's being configured. Sure enough, my folder structure under D:\Index_Files was missing, even though the share still existed.

The fix was simple, but then they all are once you nail down the problem:

1. Copy folders from working index server under D:\Index_files, or wherever you are pointing your index files on your particular search service application (namely Config and Office Server folders)

2. On command line, run psconfig -cmd secureresources (this will apply security on these folders automatically)

3. On command line, run stsadm -o osearch -action start (your search service should now be able to start)

4. Finally apply your topology changes

Moral of the story: when all else fails, look at the simple stuff first :)

Fab 40 templates in SP 2010 activation

If you downloaded the redesigned Fab 40 templates (there are actually substantially less than 40 templates for SharePoint 2010 as of yet) from here , you may run into difficulties when trying to create your sites. Apparently, there are multiple features that need to be installed on the site collection ahead of time, and the site creation wizard will keep complaining.

For example, you may see error message regarding feature 6c09612b-e1ef-4867-8396-531430c60f2 not being activated, and as soon you do that and retry, yet another missing feature error pops up.

Rather than trying to chase them down one by one , quick and dirty to bypass these errors is:

Step 1: install all features by issuing

Install-spfeature -allexistingfeatures

Step 2: Activate all features with the scope of "site" to the site collection where you want to create your sites based on Fab 40 templates. This will issue whole bunch of error messages since many of the site features are already activated, nevertheless it will take care of the ones that are missing.

get-spfeature | ForEach-Object{if($_.scope -eq "site" ){Enable-SPFeature $_.id -Url your_site_collection_url}}

Export and Import Content Types

Recently we needed to migrate custom content types from one web application to another.

I tried the most straight-forward approach first: use the new content type hub feature of the Managed Metadata Service App. The idea was to mark the site collection containing these content types as content type hub, and essentially subscribe the rest of the site collections to consume from it.
Due to the way they were originally created, we kept getting errors trying to use this.

 After digging around for a while, the solution was to break down the process in two steps: work with site columns first, and then work with content types based on these columns:

1. Export custom site columns from the original site collection: results will go into  site columns xml file. For this I used sitecolumn utility from Codeplex.
2. Export custom content data types from the original site collection: this will create xml file with definitions for content types from  the content type group you filter on

3. Import custom site columns from the exported site columns xml file : again Codeplex

4. Re-create custom content types in the target site

$sourceWeb = Get-SPWeb http://portal

$xmlFilePath = "C:\Install\Script-SiteContentTypes.xml"

#Create Export File

New-Item $xmlFilePath -type file -force

#Export Content Types to XML file

Add-Content $xmlFilePath ""

Add-Content $xmlFilePath "`n"

$sourceWeb.ContentTypes | ForEach-Object {

    if ($_.Group -eq "Custom Content Types") {

        Add-Content $xmlFilePath $_.SchemaXml

    }

}

Add-Content $xmlFilePath ""

$sourceWeb.Dispose()

$destWeb = Get-SPWeb http://portal/sites/migrationtest

$xmlFilePath = "C:\Install\Script-SiteContentTypes.xml"

#Create Site Content Types

$ctsXML = [xml](Get-Content($xmlFilePath))

$ctsXML.ContentTypes.ContentType | ForEach-Object {

    #Create Content Type object inheriting from parent

    $spContentType = New-Object Microsoft.SharePoint.SPContentType ($_.ID,$destWeb.ContentTypes,$_.Name)

   

    #Set Content Type description and group

    $spContentType.Description = $_.Description

    $spContentType.Group = $_.Group

   

    $_.Fields.Field  | ForEach-Object {

        if(!$spContentType.FieldLinks[$_.DisplayName])

        {

            #Create a field link for the Content Type by getting an existing column

            $spFieldLink = New-Object Microsoft.SharePoint.SPFieldLink ($destWeb.Fields[$_.DisplayName])

       

            #Check to see if column should be Optional, Required or Hidden

            if ($_.Required -eq "TRUE") {$spFieldLink.Required = $true}

            if ($_.Hidden -eq "TRUE") {$spFieldLink.Hidden = $true}

       

            #Add column to Content Type

            $spContentType.FieldLinks.Add($spFieldLink)

        }

    }

   

    #Create Content Type on the site and update Content Type object

    $ct = $destWeb.ContentTypes.Add($spContentType)

    $spContentType.Update()

    write-host "Content type" $ct.Name "has been created"

}

$destWeb.Dispose()

This script has been taken from Phil Childs' blog which can be found  here , so big thanks to him.

FAST for SharePoint- No Search Results Returned

FAST for SharePoint-no results returned

I recently encountered peculiar problem when configuring FAST for SharePoint server in my lab environment. I was using the MS technet document

http://technet.microsoft.com/en-us/library/ff381267.aspx

After configuring the two Search Service Apps (Content and Query) and starting full crawl, I was able to see items in the default search content collection "sp" by running

Get-FASTSearchContentCollection -name "sp"

The problem was, even though the items were supposedly in the index, I was unable to get any results while searching. This was true for both searching via SharePoint FAST search center site, and when trying to use the built-in QRServer on FAST (which can be accessed locally on the FAST server by browsing to http://localhost:13280 if you keep all defaults during configuration, or at http://localhost:base port+280 if you changed the base port)

To make things even more interesting, I was able to feed a document manually using:

Docpush -c sp

and this document was showing up on both QRServer and SharePoint FAST search center with no problems.

The problem turns out, is I had added the fast service account to the local administrators group on the FAST server, IN ADDITION to it being in the FastSearchAdministrators local group that's created by default after FAST configuration.

This I assume caused search results to be security-trimmed in very unexpected ways.

The fix was easy enough: remove the fast service account from local admins group on FAST server, reset the index on SharePoint, on the Content SSA, followed by cleaning up the sp content collection on FAST.

After re-indexing on the content SSA, documents started showing up in search results.