You heard that right... Fixing the ability to create site collections within SharePoint. Recently, I was tasked to get 3 environments in sync (DEV/TEST/PROD). Part of this task was to set up User Profiles and My Sites with Self-Service Site Creation enabled.
Setting up DEV and TEST went rather smoothly. At that point, I should have known there was a hidden whammie that would ruin a whole weekend of mine. I'm so naive sometimes...
Cannot Create Site Collections Error
This madness came to the forefront after setting up My Sites. One of the last steps is to create a site collection with a template of My Sites Host. While trying to create this site collection, I get a stack trace in my face:
Failed to load receiver assembly "Microsoft.Office.Server.Powerpoint.Web.MOSSHost, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" for feature "PowerPointServer" (ID: 5709298b-1876-4686-b257-f101a923f58d).: System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Office.Server.Powerpoint.Web.MOSSHost, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c' or one of its dependencies. The system cannot find the file specified. File name: 'Microsoft.Office.Server.Powerpoint.Web.MOSSHost, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c' at System.Reflection.Assembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, Assembly locationHint, StackCrawlMark& stackMark, Boolean throwOnFileNotFound, Boolean forIntrospection) at System.Reflection.Assembly.InternalLoad(AssemblyName assemblyRef, Evidence assemblySecurity, StackCrawlMark& stackMark, Boolean forIntrospection) at System.Reflection.Assembly.InternalLoad(String assemblyString, Evidence assemblySecurity, StackCrawlMark& stackMark, Boolean forIntrospection) at System.Reflection.Assembly.Load(String assemblyString) at Microsoft.SharePoint.Administration.SPFeatureDefinition.get_ReceiverObject() WRN: Assembly binding logging is turned OFF. To enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1. Note: There is some performance penalty associated with assembly bind failure logging. To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog]. &ErrorCorrelationId=c01e4e3a-1a39-401d-9b67-5a38961a7bd9
Clearly there is errant code running that isn't in the other two environments. Reading the error, it's easy to discern there's a problem with that assembly. Looking into the GAC, I noticed that this .dll didn't even exist.
I said this to myself...
It's early in the outage window, so I jumped into triage mode. Googling that error, there is not much out there at all. That was disheartening and a punch in the gut. There's nothing like fixing an error that didn't come up in DEV or TEST and nothing of value on the net to fix it.
Looking more into the error, it's easy to see there's a feature that is running this code:
Failed to load receiver assembly "Microsoft.Office.Server.Powerpoint.Web.MOSSHost, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" for feature "PowerPointServer"
Based on that feature being available to the farm, I was able to discern that Office Web Apps has been installed. But why isn't this working and why is the .dll missing from the GAC? Reading over the uninstallation steps for Office Web Apps, it became very apparent that this component was previously installed and then removed. For some reason, the features are still activated within the farm. GREAT!!! There goes my weekend...
Find and Remove the Office Web Apps Feature
Before last weekend, I hadn't much knowledge of installing or configuring Office Web Apps, so needless to say, I treaded lightly. First up was to find the feature that I thought was the culprit. Using this PowerShell command, I was able to peruse the features within the farm:
Get-SPFeature -Limit All | Out-File -FilePath "$pwd\spfeatures.log"
Armed with this, it's easy to find the OfficeWebApps feature. But that feature is scoped to a Site!
OfficeWebApps 0c504a5c-bcea-4376-b05e-cbca5ced7b4f Site
That cannot be the problem since I don't even have a site collection! So, I tried to create a site collection in a different web app, thinking maybe it was something with the My Site stuff. NO BUENO. Same stack trace in my face... Directly below this feature in the log is another feature that looked very promising:
OfficeWebAppsStapling fb67f269-fd1d-4f9a-af0b-50f5755e19d7 Farm
The name of this feature and since this is scoped to the farm, I got the warm and fuzzies. Since Office Web Apps wasn't in Programs and Features and the .dll was missing from the GAC, I decided that I could safely uninstall this feature from the farm altogether.
Uninstall-SPFeature -Identity "fb67f269-fd1d-4f9a-af0b-50f5755e19d7"
Once this was uninstalled, I hit paydirt. I was able to create site collections again!
As an aside, this farm hadn't created a new site collection in well over 2 years. Check the created dates on your own farm sometime.
Get-SPSite -Limit All | % { $_.RootWeb.Created }
Beware of using -Limit All, it may cause a degradation in performance.
Another Office Web Apps Error
It was only a small victory in the lengthy weekend battle. Now that the site collection is created, it's only natural to enjoy the fruits of you labor. Upon visiting the site, I'm thrown and error with a Correlation ID. TO THE ULS!!!
Error initializing Safe control - Assembly:Microsoft.Office.SharePoint.ClientExtensions, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c TypeName: Microsoft.Office.SharePoint.ClientExtensions.Publishing.TakeListOfflineRibbonControl Error: Could not load type 'Microsoft.Office.SharePoint.ClientExtensions.Publishing.TakeListOfflineRibbonControl' from assembly 'Microsoft.Office.SharePoint.ClientExtensions, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c'.
What we have here is ANOTHER .dll missing from the GAC. If you've been following along, you're probably yelling the answer out. Uninstall the OTHER Office Web Apps feature. I originally chose not too, since I was treading lightly. :)
Uninstall-SPFeature -Identity "0c504a5c-bcea-4376-b05e-cbca5ced7b4f"
I'm now able to visit the My Sites site collection! Now it's time to click on My Content and create my own personal site collection. No whammies, no whammies, no whammies!!!
UGH!!! I can't get that lucky... Another stack trace in my face.
My Sites Error
After clicking My Content, the site collection was being provisioned and started the best Mr. Burns impression I can. Not so fast...
There has been an error creating the personal site. Contact your site administrator for more information.
Unfortunately, this doesn't come with a Correlation ID, but the ULS will tell all. Cracking open ULS Viewer and watching the ULS log live, I found........ You guessed it ANOTHER missing .dll that's related to Office Web Apps. Are you starting to find a theme within this post?
The following exception occurred: Failed to load receiver assembly "Microsoft.Office.Excel.Server.HostExtensionWac, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" for feature "ExcelServerEdit" (ID: b3da33d0-5e51-4694-99ce-705a3ac80dc5).: System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Office.Excel.Server.HostExtensionWac, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c' or one of its dependencies. The system cannot find the file specified. File name: 'Microsoft.Office.Excel.Server.HostExtensionWac, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c'
at System.Reflection.Assembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, Assembly locationHint, StackCrawlMark& stackMark, Boolean throwOnFileNotFound, Boolean forIntrospection)
at System.Reflection.Assembly.InternalLoad(AssemblyName assemblyRef, Evidence assemblySecurity, StackCrawlMark& stackMark, Boolean forIntrospection)
at System.Reflection.Assembly.InternalLoad(String assemblyString, Evidence assemblySecurity, StackCrawlMark& stackMark, Boolean forIntrospection)
at System.Reflection.Assembly.Load(String assemblyString)
at Microsoft.SharePoint.Administration.SPFeatureDefinition.get_ReceiverObject() WRN: Assembly binding logging is turned OFF. To enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1. Note: There is some performance penalty associated with assembly bind failure logging. To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].
This one made me pause a little bit more, because it's not clear if this feature is a default feature of the farm. Treading lightly, I opted to simply disable this one and not uninstall completely, just in case.
Disable-SPFeature –identity "ExcelServerEdit" -URL http://webapp
Wrap up
I'm not sure how or why Office Web Apps was removed, but the features that come with this component were still activated and made for an interesting weekend, nonetheless. I was able to find some great resources out there for My Sites troubleshooting. If you ever run into issues with this, I'd definitely take a look at these two links:
As for Office Web Apps, I found an update that lists what I would assume to be most of the .dlls that can be found within. It helped me along the way:
I hope this helps someone save some precious weekend hours...
