SharePoint Custom Site Definition Battle – Common Ground

While some developers are exhausted, and brains are about full with all they can get with all the announcements and exiting news of a new Windows Cloud OS Azure, Office Web Applications/Office 14, and Windows 7 a few developers have had some spare cycles to respond to my "Just Say NO to Custom Site Definitions" (Updated 10/29).  Wherein I discouraged the use of custom site definitions, not intended as a scare tactic, but to discourage the use for two main reasons… 1) The difficulty of managing custom site definitions and my 2) concern of a difficult and challenging future upgrade for custom site definitions (How can the product team understand what people are doing in their own site definitions?). (More below on this one.)

Since that time and sifting through many very solid arguments, not by lazy devs, but from the best in our industry, I have to concede a few points soften my message and even have a few "a ha" moments.

First, I updated my post and took out some of the shrewd comments, but not in the spirit of the message more in ensuring it can be appreciated over time.  The new title "Do you really need custom site definitions?" where in I added the common ground, and provide some quotes from these extraordinary devs most of which have been awarded community SharePoint MVPs for good reason.

 

Common Ground – Summarized by Robert Bogue (See his post on SharePoint Guidance – Patterns and Practices).

1) Do site definitions when you need to.  (Mostly I think we agree this is when you need a unique ID/name — but there are other cases as well)

2) If you build a site definition is should be very minimal — everything should be built around features.  The features can be hidden.

Lots of attitude and perspective here.  Lots of great ideas and considerations.  Thanks for keeping the vulgarity to a minimum. 🙂

Say No to Heavy Site Definitions

"I think calling out bad practices (heavy site definitions) is absolutely appropriate.  However, I think that saying they’re all bad is taking it too far." – Robert Bogue

You don’t HAVE to create custom site defs

"For me, you post is too strong. My post that you reference [You don’t need to create site definitions] is more how I feel about it… simply tell people you don’t HAVE to create site def. There are reasons when you have to create each one. In fact, while I hate site def’s, EVERY WCM project I do starts with a custom site def. Mind you it’s slimmed WAY down as I only need something with a name (for the user to pick) and ID (to attach Features to), but it is a site def none the less. " — Andrew Connell

If SharePoint had the proper mechanism I wouldn’t use them

"In my opinion it all boils down to the last sentence AC said.  “as I only need something with a name (for the user to pick) and ID (to attach Features to), but it is a site def none the less."

If SharePoint had another mechanism for us to provide this functionality then we could probably forego custom site defs altogether.  I follow the same approach AC outlines on his blog, come to think of it, the article on my blog you linked to just provides the instructions to implement AC’s approach." — Todd Baginski

Site Definitions are a tool, you must consider the tools in your toolbox (don’t take away this tool)

Custom Site Definitions are a tool.  Like any tool, they have benefits and drawbacks.  Used properly, they provide much value.  Used improperly, they cause pain.
Even the body of the article contradicts the sensationalist-headline by saying that there are some things you can’t do without a custom site def.  The article of AC’s that this links to, and it’s comments, talk about a solution that is a totally blank CUSTOM SITE DEFINITION, that is then built up properly with Features/Solutions.  They also mention publishing scenarios that the recommended approach is a custom site def.
So, the best approach is to do your homework.  If a custom site def REALLY is the best approach, then feel free to use them.  Just make it a conscious decision, knowing the trade-offs, not your default reaction because it’s easier.  — David Mann

Everything else is a work around

Joel, my friend, you are way out in left field on this one.  Custom Site Definitions, when built properly, are EXACTLY THE SAME as what Microsoft provides out of the box.  They are nothing more than a "skeleton" that gives developers the ability to attach programmatic elements to in the form of Features.  Nothing more, nothing less.  There are many scenarios in which they are required:
1. Automated Provisioning – Self-contained solutions that have all necessary functionality baked in (think hosting and SAS models).
2. Repeatability – They migrate better from dev to staging to production better than any other method.
3. Maintainability – New Features can be added or removed as required and the solution upgraded.  Try doing that with an STP file.
4. One Click Deployment – The user simply selects the proper definition on the site creation page, to which you can add descriptive text and sample images (what do you think all those other options are?  They’re OOTB site defs, that’s what).
5. Control – Nothing beats a site def for restricting what features site owners can and cannot use.  Very important in many enterprise environments.
6. Ease of use – There are lots of workarounds for the power and flexibility that site defs provide but all require a great deal more code than a simple site def with stapled features.
Sorry to burst your bubble but you’re wrong on this one.  Next time, ask a developer with experience doing Site Definitions the proper way before you go off on an opinionated rant.  I’d be happy to help! — Eric Shupps

Speaking of Passion, Eric Shupps has written two blog posts recently that are very relevant in the context of this conversation… "Site Definitions – the Good, the Bad, and the Ugly" and "Site Definitions – the Good, the Bad, and the Ugly."

Site Definitions are best practice

Wow, way to create and document a development practice and then slam the use of it!
Site defs are a way to package up things into user-createable units, much as MS did with the "new team site" template. Ideally they’re implemented by preconfiguring a set of lists and features, and are very lightweight. But this post is way off base. —Daniel Larson

If you must use a Site Definition, understand the ramifications

None of this is to say that Site Definitions are obsolete, or should never be used. You just need to be aware that this is a very fundamental construct. Once you build a site from a site definition, you cannot change it to a different definition later. You can still tweak it with additional features, but you can’t change the "kind" of site it is. It will not be a "SharePoint Team Site", easily and transparently upgradable. It will be an "Acme Group Site", or whatever you have called it, and you need to plan for its entire life-cycle (Much like adopting a puppy). There may be extra steps in the upgrade process, for example (though, at this time, we don’t know what those steps might be). You may have trouble getting support if the person who created the Site Definition moves on to greener pastures. This may be worth it for your environment, but it is something you need to take into account. — Woody Windishmann

One area that I would be careful about is using custom site definitions.

These were available in WSS 2.0 and virtual discovered by the development community by reverse engineering soon after RTM. This approach to SharePoint customization lead to some documentation and even some sample site templates being released but then along came WSS 3.0 and only OOTB site definitions were upgraded automatically. WSS 3.0 also offers new feature and solution deployment options which can help you to add functionality to the out-of-the-box site definitions in a supported way. I suspect the OOTB site definition + Features may provide a cleaner upgrade path for you in the future (assuming your feature meets the evaluation criteria in the Code Acceptance Checklist 😉

If your SharePoint farm is used for multiple applications, both OOTB and custom, such as Intranet Publishing, collaboration and a 3rd part project management solution, I would use the Code Isolation design principal by installing the 3rd party solution in its own SharePoint Web Application with a dedicated IIS Web Application Pool. — Ian Shandling

Administrator Site Definitions vs. Features Upgrade Trade offs

Custom Site Definitions are not one time expenses, and just like other customizations requires continual support after implementation, especially at times like upgrade. However custom features also require maintenance and upgrade support, so maybe the cost is not that different.  Sure much of a site definition can be upgraded out of (convert schemas to features), but the core site definition IDs and template names last forever. That means being stuck trying to implement new UI changes for site definitions at each major upgrade when all you really initially wanted was a set of custom templates people could choose that had maybe a bit of UI changes, and sets of features. If you can live with just using features initially, then I would suggest skipping the additional burden of the site collection, because at some point someone will want agility that the custom site collection doesn’t easily or cheaply provide.

A couple of other potential options:

· SharePoint does have a mechanism to staple features to sites and it is global, but one could also use OM, via an external process or even activated through some custom master feature, to programmatically determine which features to staple at site creation. Or this could be a feature that has a first visit action to provision the customizations (not super friendly, but I know of some who do this).

· Another consideration is to create new custom site templates based on existing ones (like STS Blank) and then adding references to custom features or directly adding custom changes. Technically, I would say the burden of re-fiddling with the definition at each major upgrade vs. creating new site templates at each major version and ensuring they have the right customizations and features is fairly close in cost, but one can never become un-customized with the custom site definition, and the custom template can become un-customized. Either way though, the custom features still need to be confirmed for upgrade-ability and may need some adjustment in areas such as UI to work correctly or at least not stand out negatively in the new version.

For the area they say about creating a new definition just to be a UI placeholder and having all other aspect handled by features, sure it works. Based on historic experience, when they plan to upgrade they will have to revisit their definition or feature pages to update the UI. The operations person will have to keep the original and/or updated definitions perpetually, or at least until they deprecate the sites using them (whereas a feature only solution though would only require deprecating the feature, and then cleaning up whatever it creates)

I think the biggest caveat to new/custom site definitions is that there is no good way to migrate out of them. If you adopt them, you are stuck with them since [they] encode the sites onet.xml and setuppath values to the existing definitions copy of the files.  — Anonymous voice that matters.

So I’ve softened the language, but you’ve gotten the gist, and I think my post achieved the goal of helping us narrow down why they are some times needed. I do enjoy the points that custom site definitions are NOT one time expenses, and this should definitely be considered in any solutions that include them.

Let me reiterate the checklist from MSDN on evaluating third party solutions.

In a very constructive email thread with a number of Dev SharePoint MVPs and myself we discussed point 2 and my concerns around upgrade.  I had such an earful in V2 and I’ve been concerned about O12-O14 (V3 to V4) upgrades, but Todd’s comments really got me thinking that it could really be different this time around.

I’m not sure why there is such concern about upgrading a site definition that is nothing but a copy of an out of the box site definition with a bunch of Features stapled to it.  I’ve heard from several folks at MS since V3 was released that when V4 comes around the existing out of the box site definitions will be upgraded as part of the installation process.  As we know, an XML file is used to upgrade site defs from one version to another.  So, in the future, if MS writes the XML file to upgrade the Team Site during the upgrade process I’m not sure why I wouldn’t be able to use the exact same file to upgrade a site definition that is exactly the same as a Team Site with just a different identifier and a bunch of features stapled to it.  It doesn’t seem to me like an upgrade using this approach would take long at all; in fact I’ve heard MS people say the same thing.  Is there anything that leads you guys to believe what I am discussing in this paragraph doesn’t make sense?  More importantly, how can we work with the product team to ensure that we have a framework in place for the next version that allows us to add a line item to the list box on the create a site page and ensure that we have an approach that’s flexible and upgradeable?  Is it too late for that by now in the development cycle? — Todd Baginski

I think this is brilliant.  If the devs will simply let us know that they are simply copies of the out of the box site definitions with a unique ID then in theory we should be able to use the same config.xml that is provided for upgrade.  If there are no custom lists and customization that’s actually unique and bulky sitting there that needs custom feature mappings and junk, then an Admin in theory could have a fairly similar experience to an out of the box upgrade.  The important distinction would be that all these various "dev" best practices are producing in theory a COPY of the out of the box Site Defs, with no real difference other than the ID.  Knowing this, and understanding this as common the upgrade scanners should include features to check for such this case and the intelligence of both Dev and Admin should be to explain to each other the common ground of not FAT customized Site Definitions, but simple copies with a unique ID to both make the definition available for new features stapled to that new name, title, description, etc… 

I believe Todd, AC, and many of the others are saying the same thing, don’t throw away the tools, use them when necessary (sure us admins hate them, and would prefer for them to be isolated in their own app pools and own web apps and maybe even separate farms,) but upgrade need not be a blocker if we can minimize what is truly done with the custom site definition to avoid what was done in V2, and minimize the amount of work done to the on disk, and yes, please please package this as the SharePoint Cowboy, AC, Robert Bogue, and the rest of us demand.  Package it or deny.  No willy nilly installs of DLLs.

As best practices begin to be incorporated into MSDN you’ll see content like this section on Site Definitions that goes right along with our discussion.

"In earlier versions of SharePoint Products and Technologies, site definitions were a common method of implementing server-wide user interface changes, but in Office SharePoint Server 2007 and Windows SharePoint Services 3.0, the site definitions rely on individual features that are referenced from the site definitions for much of the user interface and functionality.

After you create a site by using a custom site definition, that site always relies on that site definition. Therefore, custom site definitions must be maintained for subsequent releases of SharePoint Products and Technologies. If an upgrade occurs, then the site definition also must be upgraded, requiring changes to the site definition and the new upgrade definitions to map the old site definition to the new one.

Site definitions are a powerful method of creating many sites with the same layout; however Features and feature stapling now allow you to give sites additional functionality without the development/support burden that can come with using a site definition."

Or this existing content that is as controversial as the conversations began… on Site Templates or Site Definitions, a conversation that I don’t think can have a winner.  This one is definitely a religious question… From the WSS SDK –  Deciding Between Custom Templates and Definition

  • "Are the changes you need to make simple or complex? If, for example, you need to make only minor changes in the look of certain pages and add a few fields in particular lists, you should create a custom site template. However, if you need to create new content types, add new Web Part definitions, and significantly restructure sites, you should create a custom site definition.

  • Can you deploy changes to the front-end Web server? If you do not have access to the file system of the computers running Windows SharePoint Services, you have no choice but to create a custom site template."

I hope this has been as constructive for you as it has been for me.  Sorry if I didn’t include your comments or posts.  Many of these developers are working with the patterns and practices group on dev best practices, and you can see just how disparate the conversations and voices truly are.  There is common ground as is suggested.  Using the right tool for the job, and understanding the implications…

I definitely can’t say these developers are lazy, they are obviously reading blogs, and keeping up with the latest… and are concerned about the future as are the admins 🙂

Joel Oleson

Office 14 and Office 14 Web Applications on SharePoint!

Thanks Takeshi, great Office 14 demos.

During PDC what we saw…

Three Platforms for Office 14: 1) Client 2) Mobile 3) Web

First demo:

OneNote 14 client to OneNote 14 web application auto updated across (Multi User editing and auto update)

A third device out in the crowd updated the others with OneNote 14 Mobile (Phone)

image

2nd Demo: Word 14 to Word 14 Multi User editing

Word web application with ribbon and rich editing

image

image

3rd Demo: Excel 14 web application opened with constent experience IE and Firefox including rich visualizations. (see screenshot below of client and web UI)

Then it showed a blog pulling in the data from the web based Excel 14 application in a chart.

Exciting STUFF!!! 

Channel 9 Video Office 14 First Look

Want more?  Check out Channel9 for ondemand viewing…

"Wouldn’t it be cool if all the Office applications worked online, cross-browser from anywhere with AJAX and/or Silverlight? Or maybe even from your cellphone? What would be even better is if they had real time collaboration, integrating the rich client with the web clients. Well, wish no more because all of the above is going to be a part of the next Office Suite.

Want to see how it works? Check out this visit from Antoine Leblong, Senior VP of Office Productivity Apps. "

Here’s the Office 14 First Look demos launched this morning at 2am.

A few features mentioned in the Channel9 Video… (including a mention of a limited technical by the end of calendar year)

1. Powerpoints embedded in blogs

2. "Perfect" fidelity viewing on the web (examples: Word Layout, Excel Calculations, Powerpoint viewing)

3. Multiple ways to get the web application versions – Consumers through Office Online, Business Customers through Hosted Services and as part of Volume Licensing ON SHAREPOINT IN THE ENTERPRISE (That’s the key word we were looking for!)

4. Word Web Application uses Silverlight to render the document (zoom is example in the video)

5. Multi editing in Word within the same document.  Partial locking.  Locking of a paragraph in the same doc.

6. Excel visualizations in client, web application client service, and cross browser fidelity.

Diagram: Client and Firefox version of Excel Web Application side by side.

image 

More as more becomes available:

Seattle Times: PDC: Microsoft to sell Office as a Web service, competing with Google Docs

Microsoft_office_comes_to_browser

Office Browser Based Web Applications Clear Screenshot Images

Interview of Takeshi and Quotes from Chris Capposella (VP) by E-week.Chris Capossela, senior vice president of the Microsoft Business Division, said:

“We’re announcing that as part of the next release of Office, Microsoft will provide Office Web applications—lightweight versions of Word, Excel, PowerPoint and OneNote—delivered through the browser. With these new applications people will now be able to create, edit and collaborate on Office documents through the browser. What’s great is this provides a consistent Office experience when and where our customers want it most regardless of whether they are accessing their Office documents through the PC, phone or browser.”

Microsoft is “on a path to deliver all our technology as software and services—and today is an important milestone in this journey,” Capossela said.

What’s Cool in Windows 7

PDC keynote this morning had a lot of pleasant surprises.  These 7 stood out to me.
 
1. Dual Monitor Remote desktop connection (You can see both screens on your screen!)
2. Jump Lists – from the task bar you can customize the list of what you can do.  (You have to see this to understand it)
3. Create a VHD natively within Windows 7.
4. Boot to and Mount a Windows 7 VHD
5. Decreased use of RAM, Disk IO, and Power (Steve Sinofsky was using a 1GB machine and OS fully loaded uses only 512M)
6. Multi Touch Support & Hardware Integration (lots of cool demos on this)
7. Improvements in core apps, MS Paint gets ribbon, new Calc and Wordpad support for ODF and Open XML.
 
It was a blast watching PDC this morning streaming live with an interesting twitter feed #pdc side by side.  All they were missing was a way to post from that screen, it was a pain to have to swap back and forth, or would be nice to blog right from that page.  — Next time 🙂
 
 
If you’re asking… but what does it look like?  This M3 build looks a lot like Vista with the black smokey glassy look. 
 
Check out the Windows 7 themes in this build.  Sure looks and smells like visa.  Not a bad thing.  I expect more changes to th UI come between beta 1 and beta 2.  The task bar, the ribbon in the windows core apps (mspaint and wordpad), and the music and photo management UI is going to be quite different with WPF as a platform (This is what will really make things different and smooth.
 
The big differences they showed were working with pictures, music, and all the cool multi touch interfaces…. with less memory and similar device support as Vista, (No new device drivers, like the big revamp for Vista.)
 

Great Move SharePoint and Windows Cloud OS “Windows Azure”

Today at PDC, Ray Ozzie announced Windows Azure (previously known as Red Dog), Microsoft’s big bet (HUGE bet) on cloud services.

Do you Really Need to Create Custom Site Definitions?

<Update> After a lengthy conversation with all our favorite SharePoint MVPs from Andrew Connell, Robert Bogue, Todd Baginski… I need to soften some of the language here, but emphasizing in clarity what I’m concerned about in the spirit of this post.  As well a number of IT pros backed up my enthusiasm to discourage the use of custom site definitions.
 
First, the spirit of this message.  My original intention was to ensure that people realize just what people were signing up for when they created custom site definitions.  As soon as you have sites that don’t use the out of the box definitions, the Upgrade process becomes much more complex and hard, it also pretty much signs you up for having developers there to help you upgrade those sites with custom site definitions.
 
The other concern which the devs above shared with me as well is, there are a number of developers who don’t think through the implications of a site definition.  It doesn’t mean you’re lazy if you use them.  Many of our best devs use them on WCM sites or in special LOB integration sites.  Here’s a snippet from our conversation where we are all on the same page… the real goal of this post.  Summarized by Robert Bogue.
 
1) Do site definitions when you need to.  (Mostly I think we agree this is when you need a unique ID/name — but there are other cases as well)
2) If you build a site definition is should be very minimal — everything should be built around features.  The features can be hidden.
 
"I think calling out bad practices (heavy site definitions) is absolutely appropriate.  However, I think that saying they’re all bad is taking it too far." – Robert Bogue

"For me, you post is too strong. My post that you reference is more how I feel about it… simply tell people you don’t HAVE to create site def. There are reasons when you have to create each one. In fact, while I hate site def’s, EVERY WCM project I do starts with a custom site def. Mind you it’s slimmed WAY down as I only need something with a name (for the user to pick) and ID (to attach Features to), but it is a site def none the less. " — Andrew Connell

"In my opinion it all boils down to the last sentence AC said.  “as I only need something with a name (for the user to pick) and ID (to attach Features to), but it is a site def none the less."

If SharePoint had another mechanism for us to provide this functionality then we could probably forego custom site defs altogether.  I follow the same approach AC outlines on his blog, come to think of it, the article on my blog you linked to just provides the instructions to implement AC’s approach." — Todd Baginski

So I’m softening the language, but you’ve gotten the gist, and I think my post achieved the goal of helping us narrow down why they are some times needed.

— Joel Oleson

</Update> 
 
 
I thought we were past mucking around with the site definitions except to make tiny little changes that could easily be backed out.
 
If you don’t need to modify them don’t do it.  Consider them product code!  If you need to build something, do it in a feature, staple the feature and deploy it in a solution.  Site Templates as tough to work with as they are, are better than custom site definitions.  Even the use of site templates is controversial in the community due to the customizations that it causes in the database.  From an upgrade perspective, it’s Much Much easier to upgrade a site based on a site template than it is a site based on a custom site definition.  Now a site template based on a custom site definition is just as bad if not worse.
 
Ok so it’s easier to modify an existing site definition.  WRONG Answer!  You just broke the out of the box product and you will have a hard time getting support.  Maybe the dev support people will help you, but poor customer.  Poor support.  Poor everyone who has to pay to try to undo what’s taken place.
 
Don’t modify the out of the box site definitions unless you are following some MSDN article…  Even then, make sure there is no other way and you know what you are doing so you can back it out.  Always back it up.  You may even consider backing it up to disk so you’ve got it for later.
 
I had to listen to customers crying about what consultants came in and did to their environments in the name of "Good SharePoint Development."  If you can, leave the site def alone, and package up your code so it can be added and later removed or replaced at least.
 
I need someone to give me a list of reasons WHY you need to mess around with the site definitions.  I’ve had a couple of devs take me up on it, but I still think it’s WAY better if you just leave it alone and pretend like you can’t.  It will keep you honest and it will make upgrade and support TONS easier.
 
You’ve recently seen me in favor of client side code running server side elsewhere.  That’s great.  See if you can take things to a higher level and go with a zero server footprint deployment.  Or go with Off the shelf code where you get support for upgrade from an outside company with assurance.
 
I’m not totally against custom code, but I do want to see it thought through.
 
I’m sure nearly all SharePoint Dev classes have info on creating custom site definitions.  You may even have something in the certification test on them.
 
Any SharePoint developer can create a custom site definition, but the challenge is to see if they can fulfill those same requirements without using a custom site definition (The albatross around an admins neck).
 
Worried about users turning off the features?  Make an STSADM extension for provisioning your special site that activates the hidden feature.  Or consider feature stapling.  Get creative and think outside the V2 Box.  If you’re building custom site definitions on a regular basis you haven’t learned how to do things in the new world.
 
Let’s continue to talk through the trade offs of site templates, feature stapling, and site definitions.  I think this is an important discussion.  In the future I’d love to see it to not be common at all when even hard things need to happen.
 
Ok, so developers don’t think much about upgrade, but let’s start preaching… Can we do this without custom Site Defs without a note from the teacher that agrees to says it’s a Requirement.  On the hierarchy of Scary Customizations.  The Custom Site Def is nearly the worst.  The only ones worse are customizing the out of the box site definitions and messing with the database and adding your own triggers and "fixing" the inefficient SharePoint stored procedures.
 
Todd Posts  "How to Create a Custom Site Definition" , but agrees we need to minimize what we do with them (see above).  Good job Todd, my friend, you show up as #1 using keywords… custom site definition)
 
I do think we need to see more about "How to NOT Create a Custom Site Definition" or
 
You don’t need to use Site Definitions (This is AC’s Love and the discussion) please point your devs to this one.  I continue to plan to point customers and developers to that one. 
 
If you’re a lost developer or even one that’s looking to go through a ten step program, I recommend the Hippocratic Oath of SharePoint – First Do No Harm  (Thanks Woody).  There’s also SharePoint Dev guidance at by the Patterns & Practices Group. AC talks about it.  I’m anxious to see them provide guidance on when site definitions are necessary (yes as an IT pro, I want to see them used when features and solutions don’t do the job).
 
 
<UPDATE>
I thought AC would agree with me and he does, in fact he’s given us the formula …. You don’t need to use Site Definitions.  Check out the discussion that he’s got going…
 
This post discusses both Microsoft support, but it’s about the long term view.  If you do have questions about Microsoft Supportability for Site Definitions KB 898631
 
Don’t do site definitions because they are easier or quicker, do them because it’s the right solution, but please convince me that you understand the upgrade implications and tell me AC’s solution above doesn’t help you.
</UPDATE>
Ok.  Back to your normally scheduled lives…  The result of this conversation will make us all better off as painful as it will be for both of us.