Aroh's Microsoft 365 Musings: September 2016

As a SharePoint consultant, I have seen quite a few end user complaints that SharePoint is slow in different organizations. The only thing which comes for those slow environments is poor planning and design flaw of overall SharePoint architecture.

In that context, getting proper requirement analysis from the stakeholders and scope for that project extremely crucial to have a successful SharePoint implementation and end users happy. In my 7 years of SharePoint administration and designing of SharePoint architecture, planning of SharePoint is the key and concrete scope with complete user requirements.

This blog post is devoted to this and what has to be watched out primary bottlenecks along with tools.
Let’s get started:

1. Planning for SharePoint Infrastructure - Hardware Specifications: Check out SharePoint 2013 hardware/software requirements.

Proper planning for hardware and supported infrastructure need to carefully done. SharePoint 2013 server RAM hungry. Therefore, many of SharePoint performance issue are attributed to lack of resources.

a. Capacity Planning:

Is SharePoint Infrastructure designed according to Microsoft Hardware and Software minimum requirements? This is one of the most important aspect where SharePoint architect needs to talk to the infrastructure team.

Key design for SharePoint architecture includes:

i) Latency: time between a user clicks on browser and reposed back. This is something that end user complain about SharePoint being slow.

Tools:
>> IE Developer tools to check the response. (It’s extremely useful )
>> Fiddler

ii) Throughput: Number of requests that as server farm can process in a unit of time. Some common examples of low throughput conditions,
processors on the farm are not over-utilized. For e.g. if CPU usage during peak hours or load spikes consistently exceeds 80 %, add more servers or redistribute services to the other farm servers, is sufficient memory on application and web servers to contain the complete cache, ensure that database servers are free of bottlenecks.

Tools:
>> Enable SharePoint Developer Dashboard.
>> SQL Server Logging Database

Below capacity planning is specifically for SharePoint 2013.

b. Choosing Processors:

SharePoint Server 2013 is available only for 64-bit processors. Therefore more processors will enable us to serve greater demand.

• WFEs: In individual web servers will scale up as we add more cores.

• Application server: Application server processor capacity requirements differ depending on the role of the server and services it is running.

c. Choosing Memory:

· Web Servers: Microsoft recommends allocate at least 8 GB RAM on each web server and 16 GB for servers that have greater traffic or deployments with multiple application pools set up for isolation

· Application server: Application server processor capacity requirements differ depending on the role of the server and services it is running. Ideal application server should have minimum 4 cores (quad-cores) and recommended 8 cores (octa-cores)

· The memory requirement are of database servers are tightly dependent on the database size.

d. Choosing Network:

There is significant traffic in a farm across the web server tier, the application server tier and the database tier. The network can easily become a bottleneck under certain conditions like dealing with very large files or very high loads.

Web Servers and application servers should be configured to use at least two network interface cards (NICs): one NIC to handle end-user traffic and other for to handle inter-server communication.

e. Choosing Disk and Storage:

We need to ensure that at least 30% additional capacity on each disk, above our highest data estimate, to leave room for future growth.

The web and application servers have storage requirements also. In most of the production environment, Microsoft recommends that we allocate at least 200 GB disk space for OS and temp and 150GB for disk space for logs.

2. Optimize SQL Server for SharePoint:

The database is a common area where performance hits are most prevalent. SharePoint is database intensive, about 95% of SharePoint is stored in a SQL database: files, images, videos, pages, content, user profiles, etc. It’s important to have a happy and health SQL database running.

a. Tasks Manager on SQL Server: Check the services which are consuming too much RAM and CPU. We also need to configure I/O counters as well.

b. Configure Model System Database Settings

i. Increase auto growth settings (Use MB and not %)

ii. Recovery model should Full.

c. Configure TempDB System Database settings

i. Increase auto growth settings (Use MB not %)

ii. Recovery model should be Simple

d. Max Degree of Parallelism
Set max degree of parallelism to 1 for instances of SQL Server that host SharePoint Databases, and it is to ensure that a single SQL Server process serves each request.

e. Minimum Server Memory
Default is 2 Billon way too much for SQL Server. Other applications may be slow because SQL Server is consuming maximum RAM available.

The following table gives RAM usage for SQL Server 2014

Server memory(GB)	SQL Server Max Memory
8	5012
12	8294
16	11571
24	19968
32	27136
64	55808

3. Optimize SharePoint:

Above points are primarily dictates (Hardware Requirements and Capacity Planning) are fundamental concepts and designed by SharePoint architect gather user requirements to provide a pilot SharePoint environment. Generally speaking, while setting SharePoint infrastructure & SQL server is quite easy to set up. The primary challenge is when more data is added to SharePoint lists/libraries and users keep complaining that SharePoint has become slow. The below points slow SharePoint:

a. Tasks Manager on servers: Check the services which are consuming too much RAM and CPU. Use PowerShell script the services which are consuming more RAM.

b. Offload the SharePoint logs from C drive to other some drive on server.
SharePoints logs are usually on the drive it was installed on, and in most cases that’s the C drive of the server. It’s recommended to move these logs to another partition, and preferably different drives altogether.

i. Go to Central Administration >> Monitoring >> Configure Diagnostic Logging.

ii. Note the top part, Event Throttling. This specifies what SharePoint will log. If any of these have been changed from default, they’ll be in bold. Your ideal settings for all of them are

iii. Least critical event to report to the event log: Information

iv. Least critical event to report to the trace log: Medium

v. In the next section, Event Log Flood Protection, make sure that’s checked.

vi. And in Trace Log section, feel free to move the logs to a different drive to either free up space on the current drive or to a unique drive. You may limit the amount of days or space the logs use per your preference.

vii. Click OK.

c. Web Parts:

i. Large number of closed Web Parts, often on the home pages, will slow down a page. Every closed Web Part causes a small amount of processing overhead, and it can add up. To check this, add ?contents=1 to the page’s URL and remove any Web Parts which aren’t displayed. It can make a huge difference.

ii. Enable Developer Dashboard - Any custom developed web part or third party web pat that is slowing the page. Developer dashboard contains information about Web Parts, events, DB calls and a whole lot of nifty information.

d. Large lists and libraries containing several thousand items can slow things down a lot too.
Optimize: Content Query Web Part, list view and metadata navigation and search

e. Consider Warming up your SharePoint Site
Warming up the pages across your SharePoint site is a good idea for two reasons. Firstly, ASP.NET’s Just-In-Time compiler will often cause the first access of a site to be extremely slow compared to usual. Secondly, the caching techniques discussed previously only kick in once a page has been visited for the first time. By running a script or job to warm up the site you can avoid the ‘first hit’ performance lag associated with these issues. There are a number of sources of information regarding this topic but one such article that seems to collate a lot of them is Wahid Saleemi’s Roundup: SharePoint Warm-Up Scripts.

f. Make Use of SharePoint’s Built-In Caching Functionality

i. Output Caching (Configurable)
The concept of Output Caching is something that natively comes with SharePoint 2010, as it builds on and relies on ASP.NET caching techniques. This means that you can configure your SharePoint 2010 site to cache the Pages it outputs. The reasoning behind caching a page is obviously that it takes time to generate the content on any page, and on a heavily accessed site it would be a performance impact to generate a new page on every request – that’s where Output Caching comes in handy.

ii. Object Caching (Configurable)
In SharePoint Server 2010 you’ve got the option to use Object Caching as well, which is a mechanism to cache specific page items. This is especially likable if you’re playing around with Cross-List data queries and need to cache the results of such a query.

                                                           iii.      BLOB Cache (Configurable)
In SharePoint 2010 you also have something called the BLOB Cache, which is a disk-based caching mechanism that caches resources on the disk. Normally these resources are files served by a web page and are named Binary Large OBjects (BLOB).

<BlobCache
    location="D:BLOB"
path=".(gif|jpg|jpeg|jpe|jfif|bmp|dib|tif|tiff|ico|png|wdp|hdp|css|js|asf|avi|flv|m4v|mov|mp3|mp4|mpeg|mpg|rm|rmvb|wma|wmv)$"
    maxSize="10"
    enabled="false" />Make Use of SharePoint’s Built-In Caching Functionality

g. Distributed Cache Service Application:

SharePoint Distributed cache service is an automatic service. However, if we configure to a dedicated distributed cache service you may face some challenge.

I had earlier written a detailed blog specifically the challenges with Distributed Cache service.

I had a chance to work an outsourced (inexperienced) vendor and they claimed that SharePoint server on pre-release were slow servers. As a matter of fact, vendors has absolutely no idea to solve the issue. It is a migration project from SharePoint 2010 to SharePoint 2013 and the SharePoint 2010 had a heavily custom site and branding.

This how I found the root cause.

i. Hardware specifications: I checked both the staging environment and pre-release environment hardware specs. There were absolutely identical. So it's ruled out.

ii. SQL Server: SQL server settings, parameters and assigned memory were identical. So it's ruled out.

iii. Firewall: The Firewall rules were verified for both the environments there were identical. So it's ruled out.

iv. Search settings: Content Sources were configured as incremental. Continuous crawl was configured. So it's ruled out.

v. Distributed cache settings: Service is started status on App server. It's not enabled on WEB server.

vi. Developer Dashboard: I turned on Developer dashboard both the staging and pre-release environments to check the page performance. I noticed staging environment Internet Explorer developer tools showed page load 1.5 seconds but whereas pre-release environment showed 6 seconds for the custom branding site.

I checked if any error ULS tab for developer dashboard. I couldn't find any error that showed whatsoever that jump of execution that 6 seconds.

ULS viewer didn't give error that gives any indication of 6 sec jump. However, I know it could be of Distributed Cache service. What I know is that there is some wrong in this pre-release environment.

vii. New Web Application: To narrow down this issue, I created a Web application using team template. I asked vendor if they have code related My Site or Distributed cache. They do have some custom code related to My Site. I removed their solution.

Turn off and turn on the Distributed cache sever. Restarted the timer service on App and web server. The new Web application's site loads within 1 second.
It tuned out that issue was custom code made web application non performant and vendor admitted that it was their code that had an issue.

h. Search Service Application:

One of the sluggish performance could from Search service application. Following the tips to improve:

i. Ensure that we have SharePoint 2013 March patch.

ii. Reduce the Search crawl time interval.

iii. Major change could be change the Search’s server’s noderunner. Make a copy of noderunner.exe.config (C:\Program Files\Microsoft Office Servers\15.0\Search\Runtime\1.0\noderunner.exe.config)

The default is Maximum. We can adjust the search performance by setting PartlyReduced or Reduced

Set-SPEnterpriseSearchService -PerformanceLevel Reduced

I hope this blog helps and please comment if this blog post is helpful.
Cheers,

Aroh

References:

· Planning Distributed cache in SharePoint 2013.

· Optimize Search in SharePoint 2013., Fix Search Performance

· Optimize SQL Server for SharePoint 2013.

· TechNet: SharePoint 2013 Hardware and software requirements

· TechNet: Capacity planning for SharePoint Server 2013

· TechNet: Capacity management and sizing overview for SharePoint Server 2013

· Warmup SharePoint 2013 sites

TROUBLESHOOTING SHAREPOINT SLUGGISHNESS

(SERVER SIDE ISSUES) http://blog.fpweb.net/troubleshooting-sharepoint-sluggishness-server-side-issues/#.V91gbJh96Ul

· 10 Reasons Why SharePoint Performance Can Slow- http://sharepointpromag.com/sharepoint/10-reasons-why-sharepoint-performance-can-slow

· SharePoint Running Slow? Here’s How We Solved the Mystery!
http://www.winwire.com/sharepoint-running-slow-heres-how-we-solved-the-mystery/

· My Users Don’t Like SharePoint because it is too slow! https://davidlozzi.com/2013/01/16/my-users-dont-like-sharepoint-because-it-is-too-slow/

· SharePoint site are slow intermittently: SharePoint 2010. - http://sharepointknowledgebase.blogspot.sg/2013/06/sharepoint-site-are-slow-intermittently.html#.V91h-5h96Ul

· Intermittent Database Server Connectivity and Microsoft SharePoint Products and Technologies
https://blogs.technet.microsoft.com/wbaer/2009/10/06/intermittent-database-server-connectivity-and-microsoft-sharepoint-products-and-technologies/

Hi freinds,

I came across a business requirement where end users can view the site collection quota (or site storage) on the home page.

Business requirement:

If the quota site exceeds 70%, then show the quota in amber colour and it exceeds 80%, then show the quota in red colour. It means end users have to clean up site and purge unnecessary Obviously this cannot be out of the box. While it can be achieved different ways such as:

1. Use Server Object model.

2. Use Client Object model.

3. Deploy custom master page and apply as deployment control.

I used the Server Object model as I have the most experience in writing the server side code.

Technical Design:

A couple of things have to consider and use appropriate approach for this solution. We will need:

1. Visual Web Part – place drag drop objects.

2. Custom Web Properties – fetch the custom web part used in the web parts for the web site URL.

3. Access the storage objects (site.Usage.Storage and site.Quota.StorageMaximumLevel) via server object model.

4. Storage, audit objects etc. cannot be used contributors or visitors have not enough permissions. In this case we require SPSecurity.RunWithElevatedPrivileges

Step by step:

Create a new Visual Studio 2015 project >> SharePoint 2013 - Visual Web Part
Specify the web application you want to deploy this web part to and choose deploy this web past as farm solution for debugging.
Coding for VisualWebPart1.cs.
The site collections administrator needs to place website URL for the site that we need to show the site quota for. We need a string variable to hold this variable and name it siteURL.

We also need to web part properties as well.

>> Visual Studio automatically create User Control.>> In CreateChildControl() method create a VisualWebPart1UserControl object SiteQuotaWebPart is defined on VisualWebPart1UserControl in next step.
Coding for VisualWebPart1UserControl.ascx.cs

>>Create a public constructor for SiteQuotaWebPart. Please note, we can't use static variable to pass siteURL values between passing values from VisualWebPart1.cs. to VisualWebPart1UserControl.ascx.cs as the static variable always keep the retain the same value.

In the page load for user control, imperson ate and wrap the code with SPSecurity.RunWithElevatedPrivileges.

>> Check if the site exists

>> Access the Site Quota Objects via Server Object Model.

>> First get an create a long variable named storageUsedMax and this object stores Site Quote template size that is defined by site collection administrator.

>> Secondly get an create a long variable named storageUsed and actual size used by the site collection.

>> Next, compute remaining and total size as the percentage.
>> After this computation, we can assign the colour code depending on the size of the site collection.
Used a static function to return the size of the site collection in GBs, MBs. KBs or Bytes.
Activate and deploy the site collection. But I still get an error as shown:

Error: A Web Part or Web Form Control on this page cannot be displayed or imported. You don’t have Add and Customize Pages permissions required to perform this action
Fix:
The solution to the problem is to navigate to the SharePointProjectItem.spdata file, associated with the web part and change the IsSafeAgainstScript to true in the <SafeControl.. IsSafeAgainstScript="true" />> element as shown:
Re-activate and deploy solution.
Edit the home page >> click on Insert tab >> Under Categories locate Custom >> Select "Display Size Quota" >> Click on "Add"
Edit Web Part
In the custom Web Part Properties, specific the site collection URL.
Save the page.
Show the web part in different site collection sizes as shown: