What are the most desirable skills in a recession or depression, or essentially any economic downturn?  Remember the Jack of All Trades, the guy who I say can be dangerous?  Don’t worry I’m not telling all developers to abandon their cutting edge focus, and I’m not telling admins to start coding.  Although I do think you should be looking more closely at Powershell, a skill that will strengthen your ability to scale and perform.  That’s most definitely one great example of a skillset to pick up.  After all your SharePoint courses, you should seriously consider a class on Powershell, and pick that as something to keep your mind on automation.

Guys like Jack, someone who can do a variety of things and looks like an architect, and be put on any project and be successful is definitely an asset that companies won’t want to cut off when they are looking to trim the fat.  IT often isn’t the first place where companies look, but contractors, vendors and consultants… many of whom are working very closely with IT in lines of business may be dumping their projects and handing them over to others.

Just don’t give jack the keys to develop an app, and then put it in production the next day.  You still need separation, staged deployment, security boundaries, but that’s for another discussion.  Let me tell you more about the skills you need to keep yourself recession proof.

The Skills

It’s important to understand how IT contributes to the business when businesses tighten up with various economic pressures and uncertainty.

Consolidation

The mantra of do more with less is happening at many levels throughout the company.  Let’s start with server consolidation a few years ago the move was to clusters and SANs.  More data ,less power, and big iron.  Then look today at Virtualization.  Various similar or like services which may do the same thing in different lines of business are both going to look to be centralized as IT attempts to achieve economies of scale. 

Migration

Are you a developer?  Can you write apps or leverage apps to migrate data from various platforms and push into a common platform.  Take SharePoint, what are all of the potential platforms that could be consolidated to SharePoint?

Fileshares, Notes, Documentum (migrate or coexist), Websphere, Legacy Frontpage server extensions, custom apps, various search platforms, and collaboration, portal and WCM platforms.  While SharePoint doesn’t currently completely take out records management platforms under more extreme circumstances it may be used to provide document management and asked to be used for more and more as companies try to do more with less.  It’s the guys who can touch all these things

Virtualization

IT folks should be looking at scaling strategies.  How do you provide one platform that can be used for many things.  In this how can you provide the availability and reliability of the previous?  This means server virtualization in a scale up and scale out to provide high availability and scalability.  Smaller hardware footprint in more or similar virtual nodes

Let’s take a file server, what can be done?  Archiving, life cycle management, consolidation, moving it to cheaper storage.  You have to get the unused data offline.  You’ve got to consolidate the on premise and move it to regional data centers, and those in regional data centers as the network infrastructure gets better.  Maybe you figure out a strategy to move it all the way in one move… now that’s someone you need.  Someone who can think ahead.

One of the biggest challenges that virtualization brings is manageability.  Even a one to one physical to virtual migration with the easiest click and go, now you’ve got tons and tons of images.  You’ve swapped out a bunch of hardware and have a smaller footprint in the data center, but now you’re watching disk I/O and network I/O.  Lots more concern for contention, and you’ve got to get much more serious about your monitoring strategies.  Patching isn’t any easier on those systems, at least today with out of the box solutions.  Same with monitoring.  If anything you have to watch things closer as apps get closer.  They have some security boundaries, but share physical assets.  That’s the challenge, and this requires new skills and  detail oriented focus to narrow down issues and resolve them quickly.  Many dashboards in off the shelf virtualization management suites have insight, but they will need more.

Let me caution you not to go hog wild with virtualization all at once.  If everyone on your team is brand new with it, you’ll find some wild and crazy stuff the first few times you do it.  I recommend a pilot program and starting with the apps that are not extremely I/O intensive.  As you dig into virtualization you’ll see there are many layers and management while similar to the old ways, does require some new skills and looking at a dedicated service in the datacenter with skilled virtualization admins will help everyone move a TON faster and help you achieve your migration goals.

Be Cautious

Mergers and Aquisitions

How do your skills relate to mergers and aquisitions?  How about your toolsets and your development skills?  Can you fully automate other people’s jobs? That may sound funny, but essentially during a merger those in IT with the ability to be seen as maleable and can do what they need you to do will be picked first.  We all want to be picked to be on the elite teams.  Those that are just looking to not be seen or be heard, will find during a merger they may be looking at severance.  That critical task that you were doing that you thought was your ticket, was easily looked over since they didn’t know you were doing it.  Even if your manager knows, how are they providing those details up stream? 

What happens to the Notes team when SharePoint is asked to replace it?  The most skilled developers may be asked how good their .NET skills are.  The admins may be asked to re-interview for their same job on the SharePoint team.  How are they going to do?  Have they continued to keep their skills relevant across the industry?  Have they kept up?

Outsourcing and Offshoring

With the demands for more with less often means less people.  Companies have found that offshoring and outsourcing works for some things, but other things not so well.  Do expect some companies to look at these strategies, but expect some things to come back, often it takes years for a company to realize a strategy isn’t working.  If I’ve seen one thing, it’s the lines of business apps and core apps that become so business critical to the company and are so critical they aren’t going to outsource.  If someone is just pushing buttons and it’s a repetitive task… those are the first to go.  You need to show mindshare, future thinking, architectural out of the box thinking.  You don’t want to outsource the core business functions and those people that can take you places.

Even if you are in India, you may find that your company is looking at Vietnam or China.  You’re in China? Then they are looking elsewhere in South East Asia.  Seems there is always somewhere cheaper.  If you’ve got the skills and mobility, then opportunities will open up.  From there the key is networking.  One thing I’ve found is social networks have become more and more open and strong.  It’s very easy for your reputation to precede you if you are top blogger, and involved in Linked In, twitter, friendfeed and various networks that push your skills, and in a pinch push your status and availability as it relates to your job.

BI and Data Mining

– Brush off your database management and design skills

Companies will look at doing things more out of the box, and use COTS (off the shelf solutions).  Developers will be moved to focus on data mining, and more core business solutions as a company tightens up.  Look to the CIO and see what they call out as the priorities of IT.  They will be asked to do more as other CXO’s are asked to focus more on the core business.  Everyone essentially is asked to focus on the bottom line.  The new scenarios BI and data mining.  So this means both IT Pros and Devs should freshen up their SQL skills.  Expect Microsoft to do well in this area as the cheaper solution as companies again try do discount shopping.  Instead of shopping at Macy’s they start shopping at Target.  More for less 🙂  Companies do the same thing.

Keep your eyes wide open

It sounds like a lot of projects going on right? Well, what you may find is projects that are currently in the hopper may simply be abandoned as other more core focus on consolidation focus happens.

I don’t mean to scare anyone, but I do want people to understand when there is writing on the wall.  Companies often do stupid things when it comes to money.  I’ve been in companies that have laid off all the contractors and started a hiring freeze, and went as far as telling people they would have to pay to bring their spouse to the company Christmas party. Yikes!

I saw the writing on the wall, my web admin team got cut in half and cubes were clearing out often without notice and although I was seen as a core asset, many of the recent hires weren’t.  Often each business unit is asked to trim X amount of jobs.  That promotion you were suppose to get, gets pushed out.

The best thing you can do is have confidence.  The way to do that is to have multiple places on your quick list that want you.  They are ready to take you the second you step out.  Talking to a head hunter and having your resume on monster.com, dice.com, or hotjobs.com ready to go.  Not saying you have to list it, but if you do want to have a few conversations that make you feel comfortable can bring peace.  If you haven’t seen the job market you’ll find the IT market isn’t as bad as it sounds, and this post may make it look bleak.  A lot of the changes are industry specific, and that’s where you need to touch base with the rumor mill on company news.  You don’t want to be caught off guard.  The tough part is when you’re in a really large company and mad layoffs push tons of competition for your position.  Where there is one job opening you now have tons of over qualified applicants.  Stuff like that happens.  Often it is dependent on what’s happening in cities and happens in regional pockets.

Be Conscious About Your Ability to Relocate

One thing that’s wierd this time around is this housing market.  Makes me upset.  You may find a house and a job in another city, but selling your current house is sloooow.  I’ve got some personal experience with that one.  I haven’t listed my home for sale yet, but was renting it out.   Your experience may vary as housing markets are different throughout the globe.  Even within a city the market can vary where it use to be hot, it could be very cold and slow moving.

I’ll follow up this post with some assistance for looking at the IT job market, with some real world numbers, tips on things like Linked In, how to leverage Monster.com, dice.com, amongst others, and managing head hunters (they can work for you, and you don’t have to pay them).

In the past couple of years since SharePoint has really solidified its position in the enterprise, and more and more assets look like they could be consolidated to the platform… E-mail archiving to SharePoint sounds attractive as a solid business proposition.  Before you throw caution into the wind and do something you may regret, let me share with you some answers to some of the Myths that are out there and provide some real best practices and truths… 

There are some features, such as email enabled lists, managed folders in Exchange which can point to SharePoint, but before you enable them, you need to know both sides of the story.

In a free online Webinar accessible from virtually anywhere with decent bandwidth, I will drill into these 5 Questions that will help you debunk these myths.

• Why Use SharePoint to Manage Email?

• Are Public Folders Dead?

• Will Managed Folders Solve Your Archiving Needs?

• What Should You Know About SharePoint List Scalability?

• How Should You Use & Control Email-Enabled Lists?

WHEN
Thursday, November 20, 2008 at 11:00 AM EST

SPEAKERS
Joel Oleson, SharePoint Sr. Architect & Consultant
Barry Jinks, President & CEO, Colligo Networks Inc.

HOSTS
Windows IT Pro and Office & SharePoint Pro

HOW
Register online HERE

My session will be followed by Barry Jinks, CEO of Colligo, who will drill further into metadata management and additional archiving techniques.

More details below from the flyer… 

Did you get this email?

Best Practices for Overcoming the 64K ACL, Security Principal Limits, and Future of Identity Management

One of the oldest, but more hidden and illusive capacity boundaries is the 64K ACL limit.  It’s gotten some attention lately, and is bound to scare a few people.  It’s one that takes some thought, and for enterprises it should be part of your security, identity and permissions strategy.  Eli Robilard SharePoint MVP, recently included some best practices down in a post titled "SharePoint Security: Hard Limits and Best Practices."  I may clarify a few of the items in his list in this post.  Todd Klindt, Keith Richie, Eli and I spent some time discussing this on the SharePoint MVP super valuable restricted DL (I couldn’t even respond to, they’d have to forward my responses).  An imortant customer ran into this issue due to lack of use of security groups and we want to make sure it’s clear for both planning and scale.  I also spent some time recently drilling in on a similar vein on the Indexing and Authentication aspects of this thread with the SharePoint CAT (Customer Advisory Team) team getting clarification, discussing Search/Indexing details and providing feedback.  I expect to see more clarity on the software boundaries doc and in various TechNet storage documents on the SharePoint site.

Here’s a quoted copy of the recently updated table on TechNet in the "Software Boundaries" doc.  I also refer to it as the capacity boundaries article if you follow my blog.  You’ll see more limits in this TechNet article than in any of the others.  Need more information than what you see in this post?  I put all of the scale and performance stuff in one resource post – Key Capacity SharePoint Info and with some minimal overlap, resources on Top SharePoint Storage Resources.

Let me break down this table section by section…

Users in groups – first off, the first "best practice" and logical example here for scale is keyword "ACTIVE DIRECTORY" Security Groups.  SharePoint Cross site groups and permissions levels will NOT allow you to scale.  They are simple containers for ease of management.  You WILL have serious issues if you do not use active directory and you try to add thousands of users using only the SharePoint related groups and permissions levels. 

If you have a site you want to use enterprise wide, but don’t want to use anonymous, use Authenticated Users (which is much better than "everyone") or even better for more security use "domaindomain users" for all of your corporate domains.  This will reduce the risk of trusts that may be setup outside your organization that would put someone in an "authenticated" state.  Many would say it’s a poor practice to use authenticated users.  Here’s a simple table that walk you through common examples with a rating to help you understand best practices.  If you don’t use groups at all you may find the built in groups very handy, but moving to domain groups will assist with better security and moves toward a better practice.

From a scalability perspective all of these groups will fit the bill in being able to secure content.  They each differ in what is better use.

To overcome the ACL issue using AD groups will reduce the number of security principals.  Each named account or when named accounts are in a SharePoint permission level or SharePoint group (or cross site groups), each of the users of those SharePoint groups or individuals count as a security principal.  Each AD group counts as one, and if you use built in security groups such as authenticated users, that only counts as one. Each security principal is 32 bytes, and you need to make sure you keep way under 64K.

While you can nest groups, it’s better not to go past a couple of nestings.  You’ll find SharePoint will stop working at somewhere around 4 or more.  I haven’t seen this written anywhere, but I have seen nested DLs in SGs that do fail, and they will.  That is not an acceptable solution.  DG in SG not viable.  SG in SG ok.  SG in SG in SG in SG… pushing it. More below on best practices on proper nesting from AD.

KEY: (DG = Distribution Group, DL = Distribution List (same thing), SG = Security Group, UG = Universal Group very powerful, but lots of replication (AD guys don’t like to create these))

User Info Doesn’t Scale — MYTH!

Some would say the userinfo list has a max of 2000 items, or that you’re limited to 2000 contributors to your site.  This is incorrect.  The Userinfo list is a very special list designed to dynamically scale to for millions of items.  For example, you want to do a company wide survey with a giveaway of a BMW.  You simply secure the survey list with contributor rights to "domaindomain users" with a minimal ACL footprint of a few Kb, and as people fill out the survey they will be added to the Userinfo list the item author of their list entries into the survey.  The index will not have a problem indexing that site collection, and performance while survey’s do not support folders can be exported to Excel or various reports and views can be generated off hours.

The Userinfo lookups are already optimized with indexed columns and optimized for query time.

Don’t confuse the Userinfo list with other SharePoint lists, it is special.

User Profile

The User Profile is a very special list with special views built in.  While the timer jobs to synchronize users will take some very special care and feeding.  There are some optimization techniques of putting like site collections in similar databases to optimize queries, but from a planning perspective the profile scales very well.

Security Principal

What is a security principal?  "A security principal is an entity that can be positively identified and verified via a technique known as authentication.  Security principal accounts are Active Directory objects that are assigned unique security identifiers (SIDs).  A security principal account can be defined as a user account, group account[…] and is also assigned permissions to access certain network resources or Active Directory objects. " (Tech FAQ)

This new detail in the table above has been a problem for companies that have flat user and group structures, no groups, or restricted access to group creation and management for active directory.  What I mean by Flat is a User OU with hundreds of thousands of users and little to no use of groups in active directory.  File servers in the past often had the problem of users being added individually with permissions on items.  Inheritance, broken inheritance in folder structures with massive complexity or rules upon rules as you work your way down the tree.  With SharePoint and granular security, it is easy to fall into the same trap of messy and confusing security principals that are difficult to understand. 

With the features of item level security with security trimmed UI, you’d think it would be simple to see what you have rights to.  Not so.  Many parts of the UI are not security trimmed.  Also when you setup granular security you are making real trade offs with caching, and performance.  All three of these things don’t fit well together.  If you want to cache your page and all your web parts and items, you trade off the use of granular security and security trimmed UI.  Also your performance takes a hit when you setup granular security.  It has a place, but you should limit the use of it.

Using Inheritance for easily managing permissions below is often a best practice for easing administration.  The challenges of broken inheritance, branching in folders, and on items, and differences across various site collections in a haphazard manner can be extremely confusing and difficult to manage.  Delegation is awesome, but often has trade offs with supportability and security.

The Real ACL issue

The ACL issue is a Windows Maximum size for reading the security principals.  It has been around for a long time.  You can see KB 885482: The content index is not updated successfully and error messages are logged to the gatherer log… site contains a large number of groups and users

"This behavior occurs if the size of the access control list (ACL) is larger than 64 kilobytes (KB). The maximum buffer size of the InitializeAcl function is 64 KB. Therefore, the maximum size of an ACL in Windows, including the access control entries (ACEs) that are contained in the ACL, is 64 KB."

You can see this limit is inherited from Windows.  See KB 166348 which applies to all Windows Operating systems. 

"This issue occurs when you reach the maximum size of the access control list (ACL). The size of an ACL varies with the number and size of its access control entries (ACEs). The maximum size of an ACL is 64K, or approximately 1,820 ACEs. However, for performance reasons, the maximum size is not practical."

The Most recent SharePoint specific KB article which again reiterates this spells it out even more clearly with a couple of recommendations… KB 953132

The maximum size of a SharePoint discretionary access control list (DACL) is 64 kilobytes (KB) or approximately 1,820 access control entries (ACEs). Exceeding the 64 KB limit can have a negative effect on the performance of features such as Search and Alerts. We recommend that you do not target the maximum size in your planning.

To add lots of users to a site, to a list, or to a document library, we recommend that you add the users by using domain groups instead of adding users directly or adding users to a SharePoint group. You can use domain groups to reduce the number of ACEs in the SharePoint DACL for the site, for the list, or for the document library to optimize performance.

Did you catch the best practices?  Use Domain Groups instead of adding users directly or adding directly to a SharePoint group.  The other is the explanation that it affects indexing which impacts the availability of search results.  You will see errors in the gatherer log if you hit this limit, but only if you’re looking.  If you’re trying to narrow down a site that is misbehaving or hitting this you can look through your gatherer log for these errors: The address could not be found (0x80040DB4 – The filtering process has been terminated). The address could not be found (0x80042617 – Error in the Site Data Web Service) External component has thrown an exception.  The first can be quite common, so you may need to do some sleuthing.

Need more detail on ACLs and Security Descriptors?  See this TechNet article on how ACLs work.  More on how indexing works in a post I did called Anatomy of Indexing.

FAQ

Q. Wouldn’t my site stop working, or will I have a security issue if I hit this limit? Is it only the crawler?

A. As mentioned in the KB above there is both performance and indexing issues.  The Index will fail to index a scope where there are more than 64K in security principals.  No the WSS API for doing site authorization does not use do it the way that Windows does, and does not have this limit.

Q. What about other types of authentication like Forms Based Authentication?

A. If you have more than 2000 users with forms based authentication you may have issues in the display of those users, but since the index continues to use Windows Authentication and would be indexing with one principal (unless you use security descriptors that map to 64K of ACLed Windows Users in one scope) you won’t experience this.  This shouldn’t be the reason to switch from Windows to something else.

Q. I want the members web part to show my users.  If I use AD groups the membership web part won’t work??

A. For small team and project sites or workspaces, it isn’t as critical.  Its those where thousands of users are in site groups or added individually.  It’s not the thousands that you need in your members web part.

Q. We are doing this in the extranet and using Basic over SSL and hence using Windows and AD.  How can we avoid doing this when we need to add thousands of partners?

A. There are many third party products to do user and group management for an OU.  You may consider doing user management with a tool whose UI sits right in SharePoint to manage AD security groups.  Both Bamboo Solutions and Idevfactory, think even bigger and look at Microsoft’s upcoming Identity Lifecycle Management 2 (currently in beta, RC coming soon, more below).

Q. Why are SharePoint Groups not good enough?

A. SharePoint’s site groups do provide ways of easily adding users across the various sites and managing permissions, but you should add AD security groups when you’re managing hundreds and thousands of users again for similar manageability reasons.  The important thing to note is the Index will expand the SharePoint site groups and each individual will count against the 64K max and 18400 or somewhere around there will cause it to bomb.  Here’s a reminder from the TechNet article "Also, because SharePoint Groups are expanded during the indexing process, having more than 2,000 users or Directory Groups in a SharePoint group and using that group for securing scopes may cause indexing of items secured with these groups, and all items below them, to fail."

Q. Can I use distribution groups?  This would solve my problems…

A. No, distribution groups can’t be used for securing content.  Even in WSS 2.0 when it would expand a DL it would dangerously throw tons of users individually on a site, and wouldn’t maintain the group going forward.  You can work with your AD team to convert the DL (distribution list) to a UG (Email enabled Universal Group).

Q. I’m confused by all these new permissions levels, cross site groups, site groups, etc… Where can I get some real info to help me understand this better.

A. I did a post with detailed info and resources called SharePoint Groups, Permissions, Site Security, and Depreciated Site Groups.  I think a lot of people learned how to do security in SharePoint from the old Windows 4.0 methods.  Here’s how things changed with AD and how it applies to SharePoint.  Here’s a clip from that post…

From Windows security on what to use when creating your AD groups:

"For each group, you need to know what objects it can contain, as well as the overall purpose of the group.

For the group scope, you are determining where the group should be used within the Active Directory enterprise. Your group selection here determines a lot about how you want to use the group within the overall assignment of permissions. Before we discuss each group specifically, the overall picture of group and user nesting is designed to be as follows:

Users go into Global Groups, Global Groups go into Domain Local Groups, and Domain Local Groups are listed on the Access Control List (ACL) of the resource.

Domain Local Groups: (DG)

• Members From Anywhere

• Used in Local Domain

Global Groups: (GG)

• Members From Local Domain

• Used Anywhere

Universal Groups: (UG)

• Members From Anywhere

• Used Anywhere

I highly, highly recommend avoiding using member or server local groups anywhere anymore minus a few such as the built in administrator group.  Otherwise it’s just confusing and painful.

If Universal Groups are used, then the following nesting rules apply:

Users go into Global Groups, Global Groups go into Universal Groups, Universal Groups go into Domain Local Groups, and Domain Local Groups are listed on the Access Control List (ACL) of the resource."

Mantra is AGULP.  Accounts go into Global Groups go into Universal Groups go into Domain Local Groups where they are applied as permissions.

More detail on "How to Nest Users and Groups for Permissions" and good visuals on Ask the MCT on Groups, Groups and more Groups

So what’s the rule for SharePoint?

Just follow your AD best practices above, and during the Permissions part you simply add those groups to permissions levels or roles.  Example, you want to add a couple of users individually as admins, then add your team which should be an existing security group as contributors, and for your broader readers as authenticated users or domain users (best practice above.)  For a small workspace where you want to get membership, you may add a few users individually, this is up to your discretion. 

Essentially you add your existing AD security groups (Global, Domain Local, or Universal (DG and UG preferred) to the SharePoint Group which have the appropriate Roles which have role definitions (permissions levels) which have role assignments (granularity).  Then manage those AD security groups.

So I wanted to give you a heads up on the Future of Identity Management, User and Group Lifecycle and AD.  The future of User Management/Identity Management is management on SharePoint. (http://www.crn.com/software/208403182)  I saw a demo at Tech Ed Australia… awesome demo, very cool!

"There’s a lot of challenges with identities," Muglia told the estimated 10,000 attendees in a keynote. "One of the key things we’re doing with [Identity Lifecycle Management 2] is automating the process of managing the life cycle of identity management. This is a product that’s focused on a problem that effectively all organizations have."

The public beta of Identity Lifecycle Management 2 follows two limited beta releases earlier this year. Microsoft is shooting to debut a release candidate beta of the software in the fourth quarter, with the final release to manufacturing version slated for early 2009, said Douglas Leland, general manager of Microsoft’s identity and access business group, in a press conference following Muglia’s keynote.

Identity management is a hot topic, not just for security but also for complying with company policies and government regulations, and for reducing IT management costs. "Today there is a significant burden on, primarily, IT organizations and help desks for managing identities and access privileges," Leland said. "The state of the art in the market today for identity and access management is, quite frankly, underdelivered."

Leland said Microsoft’s strategy is to provide identity management solutions for on-premise and on-demand applications running either in physical or virtual environments. Microsoft also will build an administration console into SharePoint for managing identities and add user self-service capabilities, such as profile management and password reset, to Microsoft Office.

Clearly spelled out on the Microsoft ILM 2 product page: (Currently in Open Beta!)

• User Management. One of the most important things Microsoft is delivering from a business standpoint is automated, codeless, user provisioning. ILM "2" delivers tools for integrated user management and self-service across enterprise applications without the costly coding of business rules or recoding of the target systems.

• Group Management. ILM ”2” provides powerful capabilities out of the box that help increase the productivity of end users, frees up IT from repetitive tasks and provide better security and compliance outcomes.