Portable mail server on desktop for SharePoint

I used to run an SMTP server which comes along with IIS 6.0 whenever I need to test an outgoing emails from a custom SharePoint solution. After a long time I hate running a server process just to receive a bunch of emails relayed within my development machine and also due to compliance, network administrators may block the SMTP server in a desktop.

Here comes a nifty tool from Codeplex called Papercut for all our problems. Ok, how to resolve the SMTP puzzle ?

1. Download Papercut from CodePlex, unzip the zip package (Note: there is no msi package or a setup.exe, so you are not violating software compliance too) and keep it in a handy location
2. Find your machine IP first by using ipconfig and make a note.
3. Map your IP to SharePoint’s outgoing mail settings in SharePoint Central Administration console.

Open PaperCut.exe -> click on “Options” -> Map your machine IP here. Keep it open and running, do not close.

Now test all your email functionalities and you should be seeing all emails in the left pane with time stamp.

Watch all your mails are being forwarded to Papercut.exe.

SharePoint User Profile Service - method not found exception

While trying to access user profile service application in few machines, there came this exception

Method not found: 'Microsoft.SharePoint.Administration.SPIdentifierType Microsoft.SharePoint.Administration.SPAce`1.get_BinaryIdType()'

we were unable resolve by re provisioning the service application, restarting the timer jobs and restarting the servers. As this exception arises from a SharePoint assembly, looked at the version of the server and found it is SharePoint 2010 RTM version.

It requires following updates to resolve the issue

• Microsoft SharePoint 2010 Foundation SP1 - KB 2460058
• Microsoft SharePoint 2010 Server SP1 - KB 2460045
• Microsoft SharePoint 2010 Foundation Aug Cumulative updates 
• Microsoft SharePoint 2010 Server Aug Cumulative updates 

Don't forget to run the PSConfig wizard after installing Service pack and cumulative updates.

Re-provision the User profile service application and proceed.

No results in SharePoint Search but it exists in index

SharePoint Search crawler succeeds crawling the content and the same is available in crawl log. There is no crawl log error, content is a plain MS Word document, Web Application is properly associated with Search Service application but unable to see not even a single search result.

This was bit embarrassing and to nail down this issue, I was trying look in to Windows Event logs and SharePoint ULS logs, but there is no interesting message there for me to troubleshoot.

Removed the SSA association and deleted the SSA. Created a new SSA and did the association again. But the result is "none"

Verified that the Search service account has read permission in the web application and in the database as well. They exists perfectly but i don't not even a single search result.

Finally enabled the "Verbose" mode ULS logging and tried to filter using "Query Processor" category. There was new things started appearing in verbose mode

AuthzInitializeContextFromSid failed with 1355. The querying user's Active Directory object may be corrupted, invalid or inaccessible. Query results which require non-Claims Windows authorization will not be returned to this querying user.

Not sure why this exception occurs under "Unexpected" category but the recently I've configured a local admin account as service account through Power shell scripts. Probably this might be the culprit.

After hours of re-search found an MS KB article, which explained the same symptoms and cause.

Open up your SharePoint 2010 Management Shell console as an administrator and run the following PS script.

$searchApp = Get-SPEnterpriseSearchServiceApplication "SSA Name"
$searchApp.SetProperty("ForceClaimACLs",1)

Do a full crawl on all content sources to get the search results. Don't forget to switch off the verbose mode ULS Logger.

Render a spreadsheet as HTML with Excel REST services

                             An interesting question from one of my peer, whether we can show excel spreadsheet using REST services and they don't want all the features of Excel Web access web part. Just show the spreadsheet on a web page.

                            This question drove me to take a look at the capabilities of Excel services. IMHO Excel services were made for calculation and resource-heavy number crunching jobs along with a Excel web access web part which is used to render the sheets. But after looking at the REST API(yes it's beautiful), it totally revamped my thinking about this service.

First things first

1. You need an SharePoint 2010 Enterprise Sever and Excel services should be configured properly in that
2. Make sure you see ExcelRest.aspx under \14\ISAPI.

How to use this ExcelRest.aspx ?

       Say you have an Excel file stored in a document library path like below

http://Foo/Site/Shared Documents/Employee.xlsx

then the URL will be

http://Foo/Site/_vti_bin/ExcelRest.aspx/Shared Documents/Employee.xlsx

           If you are like me and worked a lot with asmx web services and hitting just the endpoint http://Foo/Site/_vti_bin/ExcelResta.aspx you will get HTTP 400 error because it's simple, REST is representational and nothing is represented here, its just an endpoint.

How to show as HTML and what are the other options ?

1. /_vti_bin/ExcelRest.aspx/Shared Documents/Employee.xlsx/Model - Find more subsects(Ranges,Charts,Tables, PivotTables) of what you are going to query.
2. /_vti_bin/ExcelRest.aspx/Shared Documents/Employee .xlsx/Model/Ranges('Sheet1!A1|H15')?$format=html - From sheet 1, Fetch the data specified in the range and render in HTML.
3. /_vti_bin/ExcelRest.aspx/Shared Documents/Employee.xlsx/Model/Charts(‘RevenueChart’)?$format=image - Fetch the chart and render it as a PNG image

What are the formats available for me to render ?

1. ?$format=html  - Renders as plain HTML fragment, for tables,sheets and pivot tables
2. ?$format=atom  - provides you a ATOM feed
3. ?$format=image - Renders an PNG image, only supported for Chart outputs
4. ?$format=workbook - Downloads the whole workbook.

Although there are some unsupported features, given the abilities of OOTB Excel services luxuries like load balancing the requests,caching etc it is worthwhile in a scalable solution.

Want to learn more, refer MSDN

1. http://msdn.microsoft.com/en-us/library/ee556413
2. http://blogs.office.com/b/microsoft-excel/archive/2009/11/05/excel-services-in-sharepoint-2010-rest-api-syntax.aspx
3. http://blogs.office.com/b/microsoft-excel/archive/2009/11/09/excel-services-in-sharepoint-2010-rest-api-examples.aspx

SharePoint performance tuning

There are various factors which comes in to scene when we enter the performance tuning for SharePoint, lets see the major players

HTTP Compression

• It provides on the fly compression on files when served from the web server to the browser. 
• It can be configured on file type extension(htm,js,css etc).
• This setting applies on web application level and in disabled state by default.
• Compression puts pressure on Server's processor but gives an instant boost to the application
• HTTP sniffing tools like Microsoft Fiddler or HTTP Watch can be used to verify the traffic.

Browser Caching

• Modern browsers come with private caching ability up to 1 GB.
• It can improve performance for sites which contain more static pages, can be tweaked in IIS HTTP response headers(cache-control: max-age=3600, post-check”)
• Easy to configure and implement 

BLOB Caching

• BLOB Caching stores all of its content on disk , it is also configured based on the file types
• Configurable in web.config the file types, max size(10 GB default) and enable/disable switch.
• It works at the web application level, this option is disabled by default.

 Object Caching

• Object caching caches site navigation structure, published pages and resources, draft pages etc.
• It is enabled by default at site collection level, the Farm admin might see it a bit daunting task to manage across all site collections :(.
• It also stores cached objects on disk and shares the space with BLOB cache.
• Need to be bit careful before increasing or decreasing the allocation of this Cache control, any unplanned configuration can deteriorate the existing performance

Browser Connections

• Internet Explorer 7 and earlier versions limit the number of concurrent file download as two at a time. IE 8 relaxed this limitation to six.
• When you work on faster connections, this browser setting might be a bottleneck for users in seamless performance.
• Microsoft provides a fix for this limitation  http://support.microsoft.com/kb/282402.
• This restriction was imposed by IETF RFC 2068 Page 45 and the explanation goes like this

 "Clients that use persistent connections SHOULD limit the number of simultaneous connections that they maintain to a given server. A single-user client SHOULD maintain AT MOST 2 connections with any server or proxy. A proxy SHOULD use up to 2*N connections to another server or proxy, where N is the number of simultaneously active users. These guidelines are intended to improve HTTP response times and avoid congestion of the Internet or other networks."

Other options

• Now a days Proxy servers & Load balancers are supporting static content caching, verify the feasibility on this.
• ISAPI filter from Aptimize can reduce significant performance bottlenecks.

SharePoint List items bulk delete

Recently while working on a production case, we needed to delete around 2000 list items in a single shot. My idea was to use the data sheet view ActiveX control to do the bulk delete. But in production case where the server runs on 64 bit and clients run on 32 bit, there was problem invoking the ActiveX control and shared  the following snippet to do a batch delete.

private static void deleteAllListItems(SPSite site, SPList list)
{
StringBuilder sbDelete = new StringBuilder();
sbDelete.Append("<?xml version=\"1.0\" encoding=\"UTF-8\"?><Batch>");
string command = "<Method><SetList Scope=\"Request\">" + list.ID + "</SetList><SetVar Name=\"ID\">{0}</SetVar><SetVar     Name=\"Cmd\">Delete</SetVar></Method>";
foreach (SPListItem item in list.Items)
{
Console.WriteLine(item.Title);
sbDelete.Append(string.Format(command, item.ID.ToString()));
}
sbDelete.Append("");
Console.WriteLine("now deleting");
site.RootWeb.ProcessBatchData(sbDelete.ToString());
}

This snippet deletes that huge list much faster than traditional object model way.

Resolving FAST Document conversion Failed error

After installing and configuring Microsoft FAST Search server, uploaded a bunch of word, pdf documents and images for crawling and indexing. Fingers crossed and waited till the Full crawl is completed in Crawl history page. Logs said that the documents were crawled successfully and ready to be queried from FAST index.

I was able to search a PDF document based on its metadata but not based on its inner content. Basic instinct goes for installing a PDF iFilters but the server behaviour is same even for Microsoft Office file formats and Jie Li's blog states that FAST comes with PDF iFilters in-built.

Tried using the DocPush from FAST toolbox as an exercise of isolating SharePoint from FAST,

docpush -c sp c:\test.doc

Docpush clearly provided me the error message "WARNING Document conversion Failed (Warning code 0)". Verified the FAST crawl logs and found the same error message there for Office file formats and PDF documents.

To resolve this you need to provide "Full Control" rights for FAST service account to folders C:\FastSearch\bin and C:\FastSearch\Tmp

Refer : KB2554579

How to open Microsoft FAST QR Server FQL console ?

         Microsoft FAST search server hosts a QR Server FQL Search console which allows us to make FQL queries from this page. This web application is hosted in port 13280. So, to access this application go to your FAST search server, open up the browser and hit http://localhost:13280

          As an interesting fact this web application is not hosted within IIS 7.0/7.5, so even stopping the IIS or World Wide Web Publishing services will not have any effect on this web application. This is a CGI application  hosted in service "FAST Search for SharePoint". Some pieces of  FAST Server is not yet fully transformed to Microsoft stack.So ensure that this service is running under services.msc console.

Host name validation failed - FAST search 2010 for SharePoint

While setting up a virtualized FAST search server in my local machine with the help of Hyper-V, post installation of FAST Search server ran the post setup configuration wizard. At one step it asked for the Server name where you want to configure, tried with many permutations and combinations but every thing ended up in a single error message box "Host name Failed" and there is not even a sample string or a help text there.

Bounced back to the documentation which spells that we should provide fully qualified domain name(FQDN) of the machine.Wizards are meant to simplify the process of configuring the servers, at-least a help text or an help button is required here.

Looks silly once the issue is resolved, still posting a screen scrap which will help another FAST explorer in dark to identify a FQDN in a server.