Collabshow.com

Have you heard about all the conferences going on in June?  SharePoint Saturday’s across the U.S. and the SPTechConn. I’m grounded for the month of June.  Staying in Seattle/Redmond, keeping myself busy with baby, family and NDA, doing a lot of writing, you should see the next paper in the Governance papers.

July I’m back on the road… I just got off a conference call with the Quest folks helping organizing my Australian SharePoint tour following the New Zealand SharePoint Conference.  I’m getting very excited about my next trip which will begin in Wellington at the New Zealand Community SharePoint Conference 2009.  The speaker lineup is looking very impressive.  If you’re in New Zealand and a SharePoint fan, I expect to see you there. 🙂

The following week July 6-10, I’ll be making my way across Australia in Sydney for a special SharePoint event, then Brisbane SharePoint User group, a special event in Melbourne, and round out the week at the SharePoint User Group in Perth.  Hoping to fit in some time touring in Western Australia, Spending a lot of well spent time on the fabulous SharePoint site: http://www.westernaustralia.com  Dates times and events all TBD with more detail to come. (Subject to change.) Contact me on twitter @joeloleson if you have any questions or want to discuss being on the panel at the events in SYD or MEL.

As well TechEd Africa 2009 (in Durban, South Africa) is now confirmed for August.  I’ve got 3 sessions and working out details for visiting a usergroup or 2.  Looks like a fun summer ahead.  How ironic is it I’ll be going to the Southern Hemisphere where it’s winter during the summer in the US?  Eric Harlan and I are teaming up on this one.  Zlatan and Michael Donovan, I’m really looking forward to it.  Will be great to see how the SharePoint MVPs and SharePoint community in South Africa have fun.

A number of significant things have happened in my life recently.  #1 being the birth of my son Dean Kyle Oleson.  My wife didn’t agree to SharePoint as his middle name, unlike Avi “MOSS is my middle name” from Israel.  He’s been consuming my thoughts rightly so.  Dean was born on May 22 at 3:20am.  I did some twittering around his birth.  The hundred or so that commented with congrats on both facebook and twitter, let me personally thank you 🙂  Was great to see the community come together like that. 

I can now talk about “My Three Sons” an old school TV show.  It’s great to be a dad again and you can bet he’ll be the best travelled baby as soon as he can get his passport.

SP2 issue

The other big announcement clogging the twitter pipes was an issue in SharePoint Server SP2 (Standard or Enterprise) were it doesn’t properly activate the license key and actually affects all SharePoint Server installations.  Not to worry, the product team is working on a hotfix, and it would take 180 days for your environment to expire.  Even less to worry there is a very easy workaround of simply activating the product again.  Jeff Teper gives an announcement on the SharePoint Team blog with a bit of a Q & A and the KB article: 971620 with the work around has been published.

Your Starting Point to the SharePoint Community

There is also a new SharePoint Marketing site http://sharepoint.microsoft.com based on SharePoint and Silverlight, congrats to the SharePoint team.  Nice to see information on the various upcoming SharePoint conferences, and the case studies are nice and easy to access, I see that as a big plus.  I almost thought the MTV guy and Jamba Juice guy must be twins.   I hope this means more of a consolidation of content or not just “another portal.”  The site explains this site is “your starting point for the worldwide SharePoint community.”   The goal is definitely portal and leveraging this platform as an agile publishing mechanism that the SharePoint team owns.

This week in SharePoint feature

In the name of consolidation, I’ve started a new feature called “This week in SharePoint by Joel Oleson” over on EndUserSharePoint.com.  Partnering up with Mark Miller, someone I’d call the key in bringing the business audience to the SharePoint community and visa versa.  His efforts on EndUserSharePoint and his panel of experts have brought a great and unique service which brings SharePoint community to the power user, and SharePoint business analyst, SharePoint project manager and SharePoint data guru.  I wonder what we’ll call this person in the future…  End user or client just doesn’t give them justice, and the amazing things they can do when combining Infopath, Excel, Designer etc… with SharePoint are simply unimaginable.  Back to the new feature… I’ve seen it become overwhelming not just for the developers and IT audience that are more dedicated, but all audiences in the SharePoint community trying to keep up with the announcements, conferences, and bits and pieces of PR around the new SharePoint Server 2010.  Everyone can’t stay plugged in all of the time.  My attempt is to give a quick 15 minute update on what happened in the past week.  I’m sure I’ll miss things, and that’s what the comments are for.  This 15 minutes is an audio MP3, WMV and accessible as a podcast on iTunes and soon to be Zune.  Love to hear your feedback on this new feature.  Right now I’m looking for positive feedback, don’t tell me you don’t like my voice.  If you want me to add a joke, let me know I could use that kind of feedback.

I wanted to share a little bit more from my deck I delivered at Teched with Paul Learning.  Had a great session, great attendance.  What we got as a result was the need for more crossover information on SQL and SharePoint.  I do recommend you start with this post on “Understanding the SharePoint SQL Databases” on SharePointForAll which I put together as post to better explain how the data is spread out and to help better understand the differences between the different SharePoint databases.

Performance and the various bottlenecks introduced by SharePoint leveraging SQL need to be better understood for us to optimize SQL for SharePoint.  Beyond RAM which is often the first bottleneck for SQL systems to hit when running with SharePoint databases is disk I/O.  It’s not the typical content database that has these challenges, but the temp and search db.  The SharePoint databases are most often disk bound, meaning the bottleneck is Disk IO.  You can easily optimize your databases for disk throughput requirements by understanding their read and write patterns and demands.

Above I’ve tried to break down the database disk demands into three tiers.  High, medium, and low.  Obviously there are operations which affect the performance of these databases at various times. 

The search database is extremely write intensive.  During a crawl of a collaboration environment you will find higher disk IO in a large SharePoint farm than even Exchange!  As well, the temp db is extremely volatile.  It is often the bottleneck to write performance for your databases.  I recently stated that after RAM, the temp db is most often the bottleckneck and can do more for your large scale performance and planning than anything else (Of course there are other performance considerations like the pipes between the servers and NIC).  Recently I explained in a post titled “SharePoint Performance and File Groups for Temp db, Search db, and Content Dbs” recommending for large environments looking for better peformance to split out the temp and search dbs with multiple NDF files (files and file groups) to optimize the write performance by allowing for better write throughput.    SharePoint admins may not realize, but everything is written to the temp db first, and then written into the transaction logs and then into the content database.

Tempdb optimization MSDN Article on Temp db optimization outlines this quite nicely (section quoted)

As well, in the hot/high importance category, the transaction logs for those who aren’t database admins, are often misunderstood.  The transaction logs need to be optimized despite whether your environment is in simple or full logging mode.  I highly highly encourage that your transaction logs be on optimized RAID configurations such as 0+1. Stripped and mirrored.  You need write performance for your transaction logs.

SharePoint Top Performance Killers!

1. Indexing/Crawling – watch out for the crawler threads and external as well as crawling within the farm.  Offloading this to Index as a FE will seriously help.  The better you can understand the ins and outs of search and crawling, the better off you’ll be in actually tuning the indexing to lessen the impact to your SharePoint farm.  SharePoint does pay attention to robots.txt if you need to block a crawler or disallow.
2. Backup (SQL & Tape) – serious write disk I/O performance hit and serious CPU hit.  This stuff hurts.  STSADM backup is one of the most inefficient commands you can run.  Using SQL backup is a good start, and DPM or other snapshots technology and even SQL Litespeed or SQL 2008 backup with compression all help to lessen the performance hit. (Tips for improving SharePoint backup performance)
3. Profile Import – watch your SQL CPU spike and it hold onto the cycles for hours.  You can attempt to be smarter about the Query in your LDAP query to your AD.  That’s a good start.
4. Misc Timer Jobs – User Sync for large #s of Users – the more users the longer the timer jobs will run.  Profile Synchronization – This job runs once every Hour and there is one per Web Application Quick Profile Synchronization – This job runs once every minute as performance permits and there is one per Web Application (MSDN.)  More information on miscellaneous timer jobs…
5. STSADM Backup/Restore – Note the updated information on 12GB as a size where they recommend putting site collections in their own database?  This is a recommendation around performance and scale. There is an article on monitoring for SQL blocking.
6. Large List Operations – I refer to the recent SharePoint MS IT Performance whitepaper where the IT Team monitors for lists over 3000 to watch and prevent SQL blocking. Also check out Revisiting SharePoint List Scalability
7. Heavy User Operation List Import/Write – Another scenario of users having power they don’t realize they have.  In SQL there’s little difference between this and bulk user operations.  Not a lot on the performance impact, but here’s an example of how to create a list based on a spreadsheet.  You can imagine an underground project to import a ton of data from file shares or from Access databases.  Totally under the radar.  Not much you can do other than plan for peak, and encourage people to plan with you for these type of long running operations.

Resources and related posts

“Understanding the SharePoint SQL Databases” on SharePointForAll

SharePoint Performance and File Groups for Temp db, Search db, and Content Dbs (blog post)

Performance recommendations for storage planning and monitoring (SharePoint CAT team “Storage” paper)

Technet article: Physical storage recommendations (Office SharePoint Server) TechNet Article

Using Microsoft Office SharePoint Server to implement a large-scale content storage scenario with rapid search availability Paul Learning & Knowledge Lake

Top SharePoint Storage (SQL) Resources (Links)

Key capacity planning information and links (Links)

I can’t complain about the opportunity to speak at Teched.  I always look forward to it and it was great to see a lot of the “regulars” including a very long list of SharePoint MVPs.  With my talk in the SQL track it was great to see a lot of new faces and find the SQL crossover track as a high value opportunity for the SharePoint community to embrace their SQL counterparts.  It was a great session with SQL and SharePoint admins sharing a high value session on Scale.

DAT307 Considerations for Large-Scale SharePoint Deployments on Microsoft SQL Server

Presenters: Paul Learning, Joel Oleson

Tue 5/12 | 1:00 PM-2:15 PM | Room 151

This session will identify architectural design considerations, provide prescriptive guidance and discuss best practices, based on real-world scenarios, around designing storage architecture subsystems for large-scale Microsoft Office SharePoint Server 2007 deployments using SQL Server. Also discussed are best practices for maximizing scalability and performance with SharePoint and SQL Server, as well as techniques for monitoring scalability metrics for existing implementations.

DAT307_Learning.pptx

Watch Recorded Session

If you aren’t a TechEd attendee you may have issues with the links above.  A copy of our deck is Teched09_DAT307_Oleson_Learning

The SharePoint booth this year again was THE place to hang out.  They came through with a few good sets of couches, but not as central as in the past.  At times those couches brush up memories of “friends” episodes of them hanging out with their coffee.  A number of conversations over the years have turned into businesses, serious networking, and valuable side projects.

While Teched outwardly was a success, and track owners were pleased with the results and feedback from attendees, it was clear that Tom Rizzo’s session was tough one toeing the line of how best to do a “Roadmap” presentation while having his hands tied.  Obviously the continued emphasis is on the SharePoint conference as the big coming out party for SharePoint Server 2010. 

Those running the SharePoint conference 2009 in Las Vegas are already very excited to see the SharePoint conference becoming such a swell of energy and excitement with customers, vendors and sponsors… it’s happening like there is no recession and as if SharePoint was THE future and THE answer.  I am personally expecting the SharePoint conference to compete with Teched as the biggest Microsoft conference of the year on multiple levels.  Seems everyone in the community is expecting a lot out of this one.