Collabshow.com

Recent Posts, Article, Whitepapers Drill Down into SharePoint 2010 Upgrade Insights

Can you tell I’ve been focused on SharePoint 2010 upgrade lately?  I wanted to recap some of those posts for easy access.  I’m doing some lectures/sessions at SPTechCon in a 2 part series on SharePoint 2010 Upgrade and needed to gather together some of the the top resources.  I’ve put a list of my recently authored blog posts below.  As well I’ve gathered some resources across TechNet and in the community and posted those on an updated post on my old blog in a post titled “SharePoint 2010 Upgrade Key Resources”

Blogs: SharePoint 2010 Upgrade Resources on SharePointJoel.com

Slides:

• Preparing for Upgrade to SharePoint 2010 Today – Slide deck on guidance to prepare today for upgrade
• Upgrading to SharePoint 2010 – Teched Middle East deck drilling into upgrade
• Upgrading SharePoint 2010 Methods Chalk Talk – Teched Middle East, Drill down on the upgrade methods

Article: Preparing for SharePoint 2010 Upgrade Today – Key thoughts and talking points around planning for Upgrade to SharePoint 2010

TechNet Download: Upgrade Approaches Poster in PDF, VSD, XPS

SharePoint 2010 Upgrade Resource Centers

• Upgrade and migration resource center for SharePoint Foundation 2010 – TechNet
• Upgrade and migration resource center for SharePoint Server 2010 – TechNet
• SharePoint 2010 Upgrade Resource Center – MSDN 
• SharePointUpgrade.com – Community driven wiki based SharePoint 2010 Upgrade resources

Just in case you were wondering… Yes, there’s more. 

While I agree consumer social media at work can be a conflict of interest for a business, but understanding the networking and valid business reasons of building the culture leveraging the new capabilities and transforming the business to better compete and stay on top of what the customer cares about… Do you want to block that too?  I do NOT agree that blind firewall rules to block both blogs and micro blogging should simply be created to put up an invisible wall.  Business departments across most companies will have valid BUSINESS reasons to use social media outlets and have legitimate reasons to participate in the networking or to data mine.  In response to Christian Buckley’s Top 10 Reasons Your CIO Blocks Social Media I put together 10 Reasons your CIO should NOT block Social Networking sites.  Should corporations be leary?  Of course they should, but at the same time, they should understand both the upside and the downside, and not only that, understand what might happen even more uncontrolled if they block it anyway.  The CIO is an important figure head, and the business is looking for serious guidance on technology.  Is your CIO a social media evangelist? Nigel Fenwick of Forrester thinks he should be.  So do I.

Social Networking is coming to your enterprise, is your business ready for it? 

Here’s 10 reasons your CIO should not block Social Networking.

1. Innovation and Idea Incubation – With the Rapid pace of today’s environment the future of your company may depend on it. It’s where both industry and new ideas are being born.

2. Real customer interaction happens – whether or not you are there, your customers are there and they are either praising or bashing your product and looking to engage.  Is someone there set to monitor it… at a minimum?

3. Social Networking – Think of the networking.  Getting like people together.  Your engineers and specialists need to be connecting with their industry and sharing ideas.

4. If you think you can block it with a proxy or firewall rule it will happen anyway – This is the 21st century and people have mobile devices and will likely be on it at home, which you can’t block anyway.  How much better would it be if you could use the positive energy and at least coach the people on how to use it properly including time management.

5. Training and Expertise – Business is transforming at light speed.  By leveraging what is happening your business can be taught to be more agile.  You’d be surprised how much the ideas of the consumer social web translate into the enterprise social web.

6. It’s not going away – while it will most likely be transformed, social networking is advantageous for those that take advantage of it.  As a platform for a CIO, it becomes a great way to become more approachable and scalable even more human.  Your ability to connect and network with your employees can be appreciated as you share your thoughts on a blog and make an accessible profile.

7. Data Mining – Stop looking at Twitter as noise and people just chatting about what they ate for lunch, and do some searching, some topic trending, and build some charts.  Look at Topsy search on your favorite terms and be blown away.

8. Community – When you start to look at what you could be building by simply catering to the needs of your product community.  There’s a lot of reasons you want a community.  Not just the obvious feedback, and research, but less understood changes in the heartbeat of your market.  Awareness of trends, market awareness, and news.

9. A major shift has happened… The Web has transformed – Do your IT, Marketing, and HR departments and product business units get it?  Do they understand how they can take advantage of it?  There’s a competitive edge, is it understood?

10. Your Departments have reasons that will not be well understood by IT – HR has recruiting needs, Marketing has research, R&D has research, PR has some tracking they need to do.  If it’s all blocked, they will find other ways of doing it, or live in ignorance, or pay someone else to gather and trust someone else’s data.  Oh, and as well, these are the obvious uses.  There are likely way more reasons that each department could find that either research or engaging their customers or clients ads real business value that would be extremely expensive and slow by other means.

All this social networking… With SharePoint 2010, will your company know how to leverage the technology?  The transformation in the consumer world will provide a shot in the arm to boost enterprise social platforms.  Enterprise corporate *Governance* and company culture are going to be KEY to that success.  While simply blocking it won’t solve the problem and in fact may exacerbate the problem, forcing employees to reach out in more untraceable manner.  Policies, practices are really the only things ultimately that you can do.  Blocking it won’t solve your concerns, it will make your employees feel like you are out of touch, and old fashioned… and despite whether you get it or not, they’ll think you don’t. As they update their status on their mobile phone… my CIO is out of touch!

There are more articles in this ongoing debate:

Facebook, Twitter becoming business tools, but CIOs remain wary

The CIO and Social Media: Social Police?

You hear about all the new features in SharePoint 2010, but what isn’t there?  I bet you haven’t had many of those discussions yet.  I think the news is better and I think the news is understandable.  When you look through the methods in a list like this, it can sometimes look overwhelming, but compared to older versions, this is much much smaller than previous.

There are a few notes to help you understand the deprecation from a developer perspective first, and then will discuss the features in the product.

Files listed on Code.MSDN. Useful for troubleshooting and being aware of how the API types and methods have changed.

SPS2010NewlyDeprecated.txt – A fairly short list of depreciated types and methods no longer supported in SharePoint 2010.

SPS2010Deprecated.txt documentation, 528K, uploaded Oct 28 2009 – 573 downloads

OfficeSharePointServer2007Deprecated.txt – to understand the API types and methods that were first deprecated in SharePoint 2007 including recommendations on many of them

 

Depreciated Features to be aware of in SharePoint 2010:

1. Performance Point as a separate product – Now baked in/Included in SharePoint 2010 Enterprise.

2. My Site Host Redesign – No visual upgrade for my sites.  The my sites really get a new look and feel… this is a good thing… really!  Also be aware the Project Web Access as applicable doesn’t support visual upgrade (related to Project 2007 on SharePoint 2007 farm).  I think most people didn’t customize this.

3. Side by Side Installation/Gradual Upgrade – No install of both binaries on the system, but you’ll be pleased that SharePoint 2010 ships with both 2007 and 2010 UI

4. SSP Admin UI – Any work you did on either the SSP Admin Site Collection or the Central Admin site has changed. 

5. Central Admin – Admin task list and other lists that you might have created… You should move this to another site collection prior to upgrade.

6. Reporting Server webparts – New Design.  Should work with an in place upgrade.

7. Deprecated Template – SSP Admin Site

8. Deprecated Template Great Plains (STSPKPL) – Template in 2003 plus pack.  Use the Great Plains Integration Manager for richer SharePoint Integration with GP.

9. Features – PortalLayouts (Legacy)

10. Missing Assemblies – Web parts (STSPKPL) – This was also the case for 2003 to 2007.  Shouldn’t be very common.

If you were at the SharePoint Conference, there was a session on Upgrading Code to SharePoint 2010 as well Sean Livingston mentioned some of this in his advanced upgrade

 

Let me know if I’m missing anything…

One of the greatest enhancements to upgrade to SharePoint 2010 is the work that was done in PreUpgradeCheck and in the powershell commandlets of Test-SPContentDatabase and Upgrade-SPContentDatabase.  Test-SPContent Database is very similar to the stsadm command PreUpgradeCheck, but it works with both 2007 and 2010.  This command can be pointed at a database that isn’t part of the farm!  That’s the coolest.  If you want to see how a content database would fare in a given farm you can run the Test-SPContentDatabase command and get information on issues that would impact the farm you’re importing it into.  Is the database compatible?  Does it have all the assemblies installed, all the relevant features and solutions all set?  Well run the commandlet to find out.

Many will be building new SharePoint 2010 farms and wanting to start clean from a hardware and software perspective.  I do expect this to be the most common scenario.  With just options of in-place upgrade or database attach, you’ll find this command to be the key.  If you only run preupgradecheck (more resources below) on the farm you’re moving from, you’re missing important steps.

Preparing for Upgrade with Powershell

Test-SPContentDatabase

Test-SPContentDatabase -Name -WebApplication [-AssignmentCollection ] [-DatabaseCredentials ] [-ServerInstance ]

Test-SPContentDatabase –name SPContentDatabase –WebApplication http://test

 

Here’s an example of a sample Test-SPContentDatabase

In the example above, you can see examples of missing files, with a note to remedy by installing the missing feature with a path to the missing .dwp in this example.  Also note that it shows if this will block the database attach upgrade.  Pretty cool.

If you were to compare PreUpgradeCheck and Test-SPContentDatabase you’d see the output and reporting is very different, but they do compliment each other.  For example with database attach at a minimum you should run them both.  On your source you should be running preupgradecheck and in the destination you should be running test-spcontentdatabase.

Source (Your 2007 farm): PreUpgradeCheck will tell you what is broken or missing in the source.

Destination (Your Clean 2010 Farm): Test-SPContentDatabase will tell you what you’ve missed in setting up your 2010 farm.

 

There’s more information on TechNet on actually running the Test-SPContent database command.  Here’s a couple of quotes on actual usage.  Note, this is beta content and it may change.  There’s some hidden gems of tips in these paragraphs below.

“Before you add the content databases to the Web applications, you can use a Windows PowerShell cmdlet to verify that you have all the custom components that you need for that database. At the command prompt, run the following cmdlet:

Test-SPContentDatabase –Name <database name> -WebApplication <URL>

[-ServerInstance <ServerInstanceName>] [-DatabaseCredentials <Domainusername>]

When you add the content databases, be sure that the root site for the Web application is included in the first content database that you add. In other words, before you continue, examine the root of the Web application in the original server farm to determine the first site collection. After you add the database that contains the root site, you can add the other content databases for the Web application in any order. You do not have to create any site collections to store the content before you add the database; this process creates the site collections for you. Be sure that you do not add any new site collections until you have restored all the content databases.

You must use the Stsadm command-line tool to add a content database to a Web application. Using the SharePoint Central Administration pages to attach a content database is not supported for upgrading.” emphasis added be me.

 

Database Attach with STSADM AddContentDb

“To add a content database to a Web application, you must use the addcontentdb operation.” (Emphasis added) 

Simple:

stsadm -o addcontentdb -url <URL> -databasename <database name>

With Options:

stsadm.exe -o addcontentdb -url <URL name> [-assignnewdatabaseid] [-clearchangelog] -databasename <database name> [-databaseserver <database server name>] [-databaseuser <database username>] [-databasepassword <database password>] [-sitewarning <site warning count>] [-sitemax <site max count>]

 

If you are actually running a database attach upgrade please note that some of these options on the upgrade are added for specific reasons related to troubleshooting issues, such as clearing the change log and assigning a new database id because it’s already taken.

The powershell command Upgrade-SPContentDatabase can be used to resume a database attach failure.  So you’d use the test-spcontentdatabase to check for missing, and use stsadm –o addcontentdb to add the content database, and use upgrade-spcontentdatabase to resume upgrades with issues.

 

After Infra Update each content database IDs are retained when you restore or reattach the database by using built-in tools. “Default change log retention behavior when using built-in tools is as follows:

• The change logs for all databases are retained when you restore a farm.

• The change log for a content database is retained when you reattach the database.

• The change log for a content database is NOT retained when you restore just the content database.

Clearing the change log forces Search to run a full crawl on that database so that the index no longer references items that do not exist.”  

 

Verify Status with Upgrade Logs and Upgrade Status Page

There are many places and ways you can check upgrade status.  You can use the Upgrade Status page in Central Administration to check the status of upgrade on your site collections and then check the upgrade log, and run STSADM.

• STSADM -o localupgradestatus  (To see if any sites were missed or skipped for upgrade, be sure to run it on all WFEs.)
• Upgrade Status page – From Central Administration, click Upgrade and Migration then Check upgrade status.
• To open the upgrade and err log files –  %COMMONPROGRAMFILES%Microsoft Sharedweb server extensions14LOGS.   Note there are logs for each session.

Recap with Steps for 2007 to 2010 database attach upgrade based on the information in this post:

Of Course.. Acquire hardware, run lots of backups along the way, and build out the 2010 farm with all the best practices, run it all on a test environment, etc…  Also note this is back of the napkin, obviously you need to add you own additional steps of communication, validation, and any other additional operational steps, etc…

1. Run PreUpgradeCheck on 2007 farm

2. Fix and address issues

3. Run PreUpgradeCheck to verify all issues addressed

4. Run Test-SpContentDatabase

5. Fix issues and make planning choices appropriately

6. Re-Rerun Test-SpContentDatabase

7. Upgrade the Service Apps and verify they are all good, then Run stsadm –o addcontentdatabase to add the relevant databases to the relevant web apps

8. Check the upgrade status both in central admin and in stsadm –o localupgradestatus on each SharePoint server and check the logs to verify the upgrade process itself was successful.

9. Re-Run Test-SPContentDatabase and verify database upgrade of sites was successful

10. Troubleshoot any issues then run Upgrade-SPContentDatabase if any databases need to resume upgrade

10. Visual verification 

— Ready to start visual upgrade

 

Powershell Commands useful around Upgrade

Test-SpContentDatabase – discussed in this post

Upgrade-SPContentDatabase – to resume failed upgrades

Upgrade-SPEnterpriseSearchServiceApplication – Upgrade Search

Upgrade-SPSingleSignOnDatabase – Upgrade SSO

 

STSADM Commands useful around preparing for Upgrade

stsadm -o ExportIPFSAdminObjects

stsadm -o MergeContentDB

stsadm -o EnumAllWebs

stsadm -o DatabaseRepair [-deletecorruption]

stsadm -o DeleteSite [-force] [-gradualdelete]

stsadm -o DeleteWeb [-force]

stsadm -o ForceDeleteList

stsadm -o VariationsFixupTool

stsadm –o Upgrade (Used for build to build upgrades, including service packs.)

 

More on TechNet

Use a trial upgrade to find potential issues (SharePoint Server 2010)

Run the pre-upgrade checker (SharePoint Server 2010)

Prepare the new SharePoint Server 2010 environment for a database attach upgrade

 

PreUpgradeCheck

Joel Oleson Preparing for Upgrade to 2010 Today – Preupgradecheck … 

Joel Oleson 5 Reasons SharePoint 2010 PreUpgradeCheck is better than Prescan …

Preparing for Upgrade to SharePoint 2010 with Joel Oleson Quest … (Slides on SlideShare)

 

Additional Resources from the Community:

Todd Klindt Using Powershell to Control Visual Upgrade

Gary LaPoint STSADM and Powershell in SharePoint 2010

Eric Kraus – Automating SharePoint 2010 Install with Powershell

One of the tough questions you will ask yourself when building out the SQL environment for SharePoint for your enterprise is what do you need?  Those determining the license may be asked this question far before the SharePoint consultants are engaged.  These questions apply to both SharePoint 2003, 2007, and 2010.

I see this as a set of questions rather than a magical formula that could be added into a spreadsheet.  Determining whether these servers are reallocated existing, virtual or physical is another conversation.

1. Organizations and Organizational Politics – Will more than 1 IT organization be managing their own SharePoint environment?  Frequently one departmental IT group will setup their own environment separate from another.  As an example, HR IT may setup their own WCM publishing portal environment and do their own deployment, while central IT does an enterprise portal, and finance sets up a third environment focused on calculation.  You’ve gone from 1 to 2 to 3 complete separate farms.  While you could consolidate into a common SQL environment, politics often dictate end to end farms be managed by separate groups.  While not a best practice, I’ve seen this often when combined with unique requirements that require custom development resources.

2. Pre-production – Development/Test/Stage – Will you have preproduction validation and/a development team or development resources and test resources working with this farm? Even if you don’t you should plan for a second for testing service packs and change control. Definitely when an organizational development team is setup, they’ll want a dev environment.  Sure you might have a dev box for each developer, there needs to be a common environment where the code comes together.  Each of these environments will require a SQL environment.  SQL express might be the choice for the dev, but for consistency you’ll find most IT departments will drive some level of consistency between dev, test and production.  Staging itself in best practices mimics the production environment. 

Add a second for sure for validation, but more based on development process and change control requirements.

3. High Availability – What are your availability requirements for uptime?  Common solutions to higher availability is clustering, log shipping, and mirroring.  Alone only clustering can provide higher uptime without requiring additional human intervention or scripting solution.

4. Disaster Recovery – What is your recovery point objective and your recovery time objective?  Don’t make the mistake that high availability requirements address your disaster recovery.  Often the simple backup requirements or a high availability solution does NOT address offsite recovery and especially those with strict recovery time requirements less than 24 hours, but realistically 72 hours.  How long does it take to acquire hardware if your datacenter is damaged one way or another?  Plan on a secondary farm if you require the entire farm be fault tolerant from a disaster recovery perspective.  It’s true you may not need 2 redundant clustered SQL servers in a fail over datacenter, but I’ve definitely seen that.

5. Dedicated Application or Isolation – Will you dedicate a separate farm to collaboration vs. your portal? While you can put your my sites, your collaboration, and your portal on the same farm, I find it common for enterprises to put the enterprise portal on it’s own dedicated hardware.  The collaboration and my sites may be separate web applications or fully separate farms.  Dedicating SQL servers vs. keeping these common is definitely another question.  They can be consolidated and the footprint of read vs. write may focus the dedication to drive vs. entirely different servers.  Indexing requirements may push out the number of SQL servers as the environment grows.  It is the ultimate isolation to put these on different servers which would help to provide top to bottom separate performance.  Coming up with a hosting model for SQL to support consolidation.

6. Performance – While performance is often addressed by bigger hardware, more CPU, more RAM, the strategy of scaling out vs. up can often times provide better memory and process isolation than continuing to add memory and CPU.  You have to figure out your economies of scale around big iron vs. additional images or additional blades for example.

7. Locations – Will you be deploying more farms in more than one location?  Plan on putting SQL servers in each location.  Three distributed datacenters equals at least 3 SQL deployments, 3 servers at a minimum.

8. Service Definition Branching – Application Hosting (Customization) vs. Out of box (Vanilla) – Do you have hosting plans for custom hosting vs. out of the box or vanilla farms? As companies determine how to divide up their SharePoint farms, some will choose for the SQL storage to be divided up when those driving the application with additional database requirements such as adding additional databases, or adding reporting

9.  TB Storage footprint – How many Terra bytes do you plan to support?  Plan for 5TB per server.  While that’s an easy answer, I’ve definitely recommended 2 TB and 1 TB per SQL server based on the operational levels and backup types.  I did a whole blog post “How many SQL servers for my X TBs” on this one question a while back.  It’s often the question people focus on.

10. Database Count – How many databases are you planning to support? While you may have heard 100 databases per SQL instance in the past, with 64 bit SQL server I haven’t heard of new numbers, but database mirroring and count when combined do push out the number of servers based on somewhere around 80 mirrored databases (or 10 or 30 depending on which doc you read), but ultimately it comes down to memory in the server as each mirrored database requires a dedicated.