Shredded Storage in SharePoint 2013

How many times we thought through before enabling versions for a specific document library which will handle files of huge size during the capacity planning exercise in MOSS 2007 and SharePoint 2010.

After enabling versions in a document library, imagine there is a DSTD.docx of 20 MB word document of v1.0 in that document library and user is attempting to edit this document. SharePoint 2010 server will serve the full document to the MS Office Word Client. After making changes to couple of page, when the user hit the Save/Sync button, SP 2010 gets the full 20 MB modified document and stores it as a seperate entry in content database.

Say if we run across ten major versions of this specific document in SP 2010, the disk space occupied by this document in Sql Server content database would be 20MB * 10 versions= 200 MB

SP2013 changes this scene by using the Shredded storage technique, if you were a DB admin or an IT Pro the you should have heard about Differential backups or incremental backups. Shredded storage is similar to this.

In SP2013,   when the user hit the Save/Sync button after making changes in DSTD.docx v1.0 , SP 2013 gets the delta changes in that modified document and add an entry in content database.

So in SP 2013 enabling the versions is not gonna eatup disk storage, reduces disk I/O as only the delta is getting stored and efficient use of network bandwidth.

No results in SharePoint Search but it exists in index

SharePoint Search crawler succeeds crawling the content and the same is available in crawl log. There is no crawl log error, content is a plain MS Word document, Web Application is properly associated with Search Service application but unable to see not even a single search result.

This was bit embarrassing and to nail down this issue, I was trying look in to Windows Event logs and SharePoint ULS logs, but there is no interesting message there for me to troubleshoot.

Removed the SSA association and deleted the SSA. Created a new SSA and did the association again. But the result is "none"

Verified that the Search service account has read permission in the web application and in the database as well. They exists perfectly but i don't not even a single search result.

Finally enabled the "Verbose" mode ULS logging and tried to filter using "Query Processor" category. There was new things started appearing in verbose mode

AuthzInitializeContextFromSid failed with 1355. The querying user's Active Directory object may be corrupted, invalid or inaccessible. Query results which require non-Claims Windows authorization will not be returned to this querying user.

Not sure why this exception occurs under "Unexpected" category but the recently I've configured a local admin account as service account through Power shell scripts. Probably this might be the culprit.

After hours of re-search found an MS KB article, which explained the same symptoms and cause.

Open up your SharePoint 2010 Management Shell console as an administrator and run the following PS script.

$searchApp = Get-SPEnterpriseSearchServiceApplication "SSA Name"
$searchApp.SetProperty("ForceClaimACLs",1)

Do a full crawl on all content sources to get the search results. Don't forget to switch off the verbose mode ULS Logger.

Render a spreadsheet as HTML with Excel REST services

                             An interesting question from one of my peer, whether we can show excel spreadsheet using REST services and they don't want all the features of Excel Web access web part. Just show the spreadsheet on a web page.

                            This question drove me to take a look at the capabilities of Excel services. IMHO Excel services were made for calculation and resource-heavy number crunching jobs along with a Excel web access web part which is used to render the sheets. But after looking at the REST API(yes it's beautiful), it totally revamped my thinking about this service.

First things first

1. You need an SharePoint 2010 Enterprise Sever and Excel services should be configured properly in that
2. Make sure you see ExcelRest.aspx under \14\ISAPI.

How to use this ExcelRest.aspx ?

       Say you have an Excel file stored in a document library path like below

http://Foo/Site/Shared Documents/Employee.xlsx

then the URL will be

http://Foo/Site/_vti_bin/ExcelRest.aspx/Shared Documents/Employee.xlsx

           If you are like me and worked a lot with asmx web services and hitting just the endpoint http://Foo/Site/_vti_bin/ExcelResta.aspx you will get HTTP 400 error because it's simple, REST is representational and nothing is represented here, its just an endpoint.

How to show as HTML and what are the other options ?

1. /_vti_bin/ExcelRest.aspx/Shared Documents/Employee.xlsx/Model - Find more subsects(Ranges,Charts,Tables, PivotTables) of what you are going to query.
2. /_vti_bin/ExcelRest.aspx/Shared Documents/Employee .xlsx/Model/Ranges('Sheet1!A1|H15')?$format=html - From sheet 1, Fetch the data specified in the range and render in HTML.
3. /_vti_bin/ExcelRest.aspx/Shared Documents/Employee.xlsx/Model/Charts(‘RevenueChart’)?$format=image - Fetch the chart and render it as a PNG image

What are the formats available for me to render ?

1. ?$format=html  - Renders as plain HTML fragment, for tables,sheets and pivot tables
2. ?$format=atom  - provides you a ATOM feed
3. ?$format=image - Renders an PNG image, only supported for Chart outputs
4. ?$format=workbook - Downloads the whole workbook.

Although there are some unsupported features, given the abilities of OOTB Excel services luxuries like load balancing the requests,caching etc it is worthwhile in a scalable solution.

Want to learn more, refer MSDN

1. http://msdn.microsoft.com/en-us/library/ee556413
2. http://blogs.office.com/b/microsoft-excel/archive/2009/11/05/excel-services-in-sharepoint-2010-rest-api-syntax.aspx
3. http://blogs.office.com/b/microsoft-excel/archive/2009/11/09/excel-services-in-sharepoint-2010-rest-api-examples.aspx

Prevent your mails from being forwarded or replied

I always amazed at the rights management plugin with IRM services provided by Microsoft which restricts me from forwarding/replying a protected mail. But what is the case of small companies who dont have these IRM luxury on their infrastructure.

There is a niche solution provided by Microsoft but still that is a Micrsoft Research project. A Outlook plugin called "NoReplyAll", a light weight plugin installs right in to outlook 2007/2010. It adds up additional buttons on the ribbon like below

So whenever I send an email, I need to choose from the above option to make my mail secure. The end user who is receiving the mails don't need this plugin in their outlook. This plugin makes use of flag settings in Exchange and outlook, so if the recipient is not using any one of these then the flags are not honoured.

This cool plugin mandates the following 

• Outlook should be the mail client on sender and recipient's machine
• Both users should be in the same domain using Exchange

For more details about this Addin,

If you want robust and rigid security then this plugin is not your choice, you should head for IRM.

Now you gotta plugin which will avoid accidental and unnecessary "reply alls", Download/Deploy it in your Outlook and enjoy.

It is a research project not an RTM product, so beware.

SharePoint performance tuning

There are various factors which comes in to scene when we enter the performance tuning for SharePoint, lets see the major players

HTTP Compression

• It provides on the fly compression on files when served from the web server to the browser. 
• It can be configured on file type extension(htm,js,css etc).
• This setting applies on web application level and in disabled state by default.
• Compression puts pressure on Server's processor but gives an instant boost to the application
• HTTP sniffing tools like Microsoft Fiddler or HTTP Watch can be used to verify the traffic.

Browser Caching

• Modern browsers come with private caching ability up to 1 GB.
• It can improve performance for sites which contain more static pages, can be tweaked in IIS HTTP response headers(cache-control: max-age=3600, post-check”)
• Easy to configure and implement 

BLOB Caching

• BLOB Caching stores all of its content on disk , it is also configured based on the file types
• Configurable in web.config the file types, max size(10 GB default) and enable/disable switch.
• It works at the web application level, this option is disabled by default.

 Object Caching

• Object caching caches site navigation structure, published pages and resources, draft pages etc.
• It is enabled by default at site collection level, the Farm admin might see it a bit daunting task to manage across all site collections :(.
• It also stores cached objects on disk and shares the space with BLOB cache.
• Need to be bit careful before increasing or decreasing the allocation of this Cache control, any unplanned configuration can deteriorate the existing performance

Browser Connections

• Internet Explorer 7 and earlier versions limit the number of concurrent file download as two at a time. IE 8 relaxed this limitation to six.
• When you work on faster connections, this browser setting might be a bottleneck for users in seamless performance.
• Microsoft provides a fix for this limitation  http://support.microsoft.com/kb/282402.
• This restriction was imposed by IETF RFC 2068 Page 45 and the explanation goes like this

 "Clients that use persistent connections SHOULD limit the number of simultaneous connections that they maintain to a given server. A single-user client SHOULD maintain AT MOST 2 connections with any server or proxy. A proxy SHOULD use up to 2*N connections to another server or proxy, where N is the number of simultaneously active users. These guidelines are intended to improve HTTP response times and avoid congestion of the Internet or other networks."

Other options

• Now a days Proxy servers & Load balancers are supporting static content caching, verify the feasibility on this.
• ISAPI filter from Aptimize can reduce significant performance bottlenecks.