Complexity bites: When SharePoint = Risk

Send to Kindle

I think as you age, you become more and more like your parents. Not so long ago I went out paintballing with some friends and we all agreed that the 16-18 year olds who also happened to be there were all obnoxious little twerps who needed a good kick in the rear. At the same time, we also agreed that we were just as obnoxious back when we were that age. Your perspective changes as you learn and your experience grows, but you don’t forget where you came from.

I now find myself saying stuff to my kids that my parents said to me, I think today’s music is crap, I have taken a liking to drinking quality scotch. Essentially all I need now to complete the metamorphosis to being my father is for all my hair to fall out!

So when I write an article whining about an assertion that IT has a credibility issue and has gone backwards in its ability to cope with various challenges, I fear that I have now officially become my parents. I’ll sound like grandpa who always tells you that life was so much simpler back in the 1940’s.

Consequences of complexity…

Before I go and dump on IT as a discipline, how about we dump on finance as a discipline, just so you can be assured that my cynicism extends far beyond nerds.

I previously wrote about how Sarbanes Oxley legislation was designed to, yet ultimately failed to, provide assurance to investors and regulators that public companies had adequate controls over their financial risk. As I write this, we are in the midst of a once in a generation-or-two credit crisis where some seven hundred billion dollars ($700,000,000,000) of US taxpayers’ money will be used to take ownership of crap assets (foreclosed or unpaid mortgages).

Part of the problem with the credit crisis was through the use of "collateralized debt obligations". This is a fancy, yet complex, way of taking a bunch of mortgages, and turning them into an "asset" that someone else who has some spare cash invests in. If you are wondering why the hell someone would invest in such a thing, then consider people with home loans, supposedly happily paying interest on those mortgages. It is that interest that finds its way to the holder (investor) of the CDO. So a CDO is supposedly an income stream.

Now if that explanation makes your eyes glaze over then I have bad news for you: that’s supposed to be the easy part. The reality is that the CDO’s are actually extremely complex things. They can be backed by residential property, commercial property, something called mortgage backed securities, corporate loans – essentially anything that someone is paying interest on can find its way into a CDO that someone else buys into, to get the income stream from the interest paid.

To provide "assurance" that these CDO’s are "safe", ratings agencies give them a mark that investors rely upon when making their investment. So a "AAA" CDO is supposed to have been given the tick of approval by experts in debt instrument style finance.

Here’s the rub about rating agencies. Below is a news article from earlier in the year with some great quotes

Credit rating agencies, paid by banks to grade some of the new products, slapped high ratings on many of them, despite having only a loose familiarity with the quality of the assets behind these instruments.

Even the people running Wall Street firms didn’t really understand what they were buying and selling, says Byron Wien, a 40-year veteran of the stock market who is now the chief investment strategist of Pequot Capital, a hedge fund. “These are ordinary folks who know a spreadsheet, but they are not steeped in the sophistication of these kind of models,” Mr. Wien says. “You put a lot of equations in front of them with little Greek letters on their sides, and they won’t know what they’re looking at.”

Mr. Blinder, the former Fed vice chairman, holds a doctorate in economics from M.I.T. but says he has only a “modest understanding” of complex derivatives. “I know the basic understanding of how they work,” he said, “but if you presented me with one and asked me to put a market value on it, I’d be guessing.”

What do we see here? How many people really *understand* what’s going on underneath the complexity?

Of course, we now know that many of the mortgages backing these CDO’s were made to people with poor credit history, or with a high risk of not being able to pay the loans back. Jack up the interest rate or the cost of living and people foreclose or do not pay the mortgage. When that happens en masse, we have a glut of houses for sale, forcing down prices, lowering the value of the assets, eliminating the "income stream" that CDO investors relied upon, making them pretty much worthless.

My point is that the complexity of the CDO’s were such that even a guy with a doctorate in economics only had a ‘modest understanding’ of them. Holy crap! If he doesn’t understand it then who the hell does?

Thus, the current financial crisis is a great case study in the relationship between complexity and risk.

Consequences of complexity (IT version)…

One thing about doing what I do, is that you spent a lot of time on-site. You get to see the IT infrastructure  and development at many levels. But more importantly, you also spend a lot of time talking to IT staff and organisation stakeholders with a very wide range of skills and experience. Finally and most important of all, you get to see first hand organisational maturity at work.

My conclusion? IT is completely f$%#ed up across all disciplines and many will have their own mini equivalent of the US $700 billion dollar haemorrhage. Not only that, it is far worse today than it previously was – and getting worse! IT staff are struggling with ever accelerating complexity and the "disconnect" between IT and the business is getting worse as well. To many businesses, the IT department has a credibility problem, but to IT the feeling is completely mutual πŸ™‚

You can find a nice thread about this topic on slashdot. My personal favourite quote from that thread is this one

Let me just say, after 26 years in this business, of hearing this every year, the systems just keep getting more complex and harder to maintain, rather than less and easier.

Windows NT was supposed to make it so anyone who could use Windows could manage a server.

How many MILLION MSCEs do we have in the world now?

Storage systems with Petabytes of data are complex things. Cloud computing is a complex thing. Supercomputing clusters are complex things. World-spanning networks are complex things.

No offense intended, but the only people who think things are getting easier are people who don’t know how they work in the first place

Also there is this…

There are more software tools, programming languages, databases, report writers, operating systems, networking protocols, etc than ever before. And all these tools have a lot more features than they used to. It’s getting increasingly harder to know "some" of them well. Gone are the days when just knowing DOS, UNIX, MVS, VMS, and OS/400 would basically give you knowledge of 90% of the hardware running. Or knowing just Assembly/C/Cobol/C++ would allow you to read and maintain most of the source code being used. So I would argue that the need for IT staff is going to continue to increase.

I think the "disconnect" between IT and Business has a lot more to do with the fact that business "knows" they depend on IT, but they are frustrated that IT can’t seem to deliver what they want when they want it. On the other side, IT has to deal with more and more tools and IT staff has to learn more and more skills. And to increase frustration in IT, business users frequently don’t deliver clear requirements or they "change" their mind in the middle of projects….

So it seems that I am not alone πŸ™‚

I mentioned previously that more often than not, SQL Server is poorly maintained – I see it all the time. Yet today I was speaking to a colleague who is a storage (SAN) and VMware virtualisation god. I asked him what the average VMware setup was like and his answer was similar to my SQL Server and SharePoint experience. In his experience, most of them were sub-optimally configured, poorly maintained, poorly documented and he could not provide any assurance as to the stability of the platform.

These sorts of quality assurance issues are rampant in application development too. I see the same thing most definitely in the security realm too.

As the above quote sates, "it’s increasingly harder to know *some* of them well". These days I am working with specialists who live and breathe their particular discipline (such as storage, virtualisation, security or comms). Those disciplines over time grow more complex and sub-disciplines appear.

Pity then, the poor developer/sysadmin/IT manager who is trying to keep a handle on all of this and try to provide a decent service to their organisation!

Okay, so what? IT has always been complex – I sound like a Gartner cliche. What’s this got to do with SharePoint?

Consequences of SharePoint complexity…

SharePoint, for a number of reasons, is one of those products that has a way of really laying bare any gaps that the organisation has in terms of their overall maturity around technology and strategy.


Because it is so freakin’ complex! That complexity transcends IT disciplines and goes right to the heart organisational/people issues as well.

It’s bad enough getting nerds to agree on something, let alone organisation-wide stakeholders!

Put simply, if you do a half-arsed job of putting SharePoint in, you will be punished in so many ways! The simple fact is that the odds are against you before you even start because it only takes a mistake in one particular part of the complex layers of hardware, systems, training, methodology, information architecture and governance, to devalue the whole project.

When I first started out, I was helping organizations get SharePoint installed. However lately I am visiting a lot of sites where SharePoint has already been installed, but it has not been a success. There are various reasons;I have cited them in detail in the project failure series, so I won’t rehash all that here. (I’d suggest reading parts three, four and five in particular).

I am firmly of the conclusion that much of SharePoint is more art than science, and what’s more, the organisation has to be ready to come with you. Due to differing learning styles and poor communication of strategy, this is actually pretty rare. Unfortunately, IT are not the people who are well suited to "getting the organisation ready for SharePoint."

If that wasn’t enough, then there is this question. If IT already struggle to manage the underlying infrastructure and systems that underpin SharePoint, then how can you have any assurance that IT will have a "governance epiphany" and start doing things the right way?

This translates to risk, people! I will be writing all about risk in a similar style to the CFO Return on Investment series very soon. I am very interested in methods to quantify the risk brought about by the complexity of SharePoint and the IT services it relies on. For me, I see a massive parallel from the complexity factor in the current financial crisis and I think that a lot can be learned from it. SOX was supposed to provide assurance and yet did nothing to prevent the current crisis. Therefore, SOX represents a great example of mis-focused governance where a lot of effort can be put in for no tangible gain.

A quick test of "assurance"…

Governance is like learning to play the guitar. It takes practice, and it does not give up its secrets easily and despite good intent, you will be crap at it for a while. It is easy to talk about, but putting it into practice is another thing.

Just remember this. The whole point of the exercise is to provide *assurance* to stakeholders. When you set any rule, policy, procedure, standard (or similar), just ask yourself: Does this provide me the assurance I need that gives me confidence to vouch for the service I am providing? Just because you may be adopting ITIL principles, does *not* mean that you are necessarily providing the right sort assurance that is required.

I’ll leave you with a somewhat biased, yet relatively easy litmus test that you can use to test your current level of assurance.

It might be simplistic, but if you are currently scared to apply a service pack to SharePoint, then you might have an assurance issue. πŸ™‚


Thanks for reading


Paul Culmsee

Print Friendly, PDF & Email
 Digg  Facebook  StumbleUpon  Technorati  Slashdot  Twitter  Sphinn  Mixx  Google  DZone 

No Tags

Send to Kindle
Bookmark the permalink.

20 Responses to Complexity bites: When SharePoint = Risk

  1. PM Hut says:

    One of your arguments, categorizing the abundance of languages as a negative, is not really valid in my opinion. I am Project Manager in a company and we just use one language. People that we hire do not necessarily know it, but they know something else, and it takes them only 2 weeks to learn it.

    I have to say that this a +, it’s a diversification (which is good, and healthy, again, in my opinion), and I have to argue the point that it affects productivity.

  2. AJ says:

    PM Hut, I think you are referring to the mention of languages within the quote from the slashdot boards, which concludes with the author of the quote recognising that the need for IT staff is going to continue to increase. I don’t think that particular quote or this post take a negative view of the multitude of languages out there but both are recognise where they sit in the concept of complexity all round. I am also an advocate for diversifcation, it is definately interesting to see creative approaches and results from people who bring knowledge and experience in from different skills sets.

  3. admin says:

    Yeah Andrew you have nailed it. When things are complex, and no-one has a full understanding, you have wicked problem potential and risk. I think a lot of SharePoint sites, despie best intentions, have gotten out of hand and a degradation of assurance is the inevitable result.

    Interestingly learning theory comes into play here too. I have been to sites where local staff do not realise the seriousness of the risk (unconsciously incompetent and therefore high risk), compared to sites where staff realise its gotten out of hand and need help to right the ship (consciously incompetent and therefore lower risk)

  4. Pingback: SharePoint Daily for September 25, 2008 - SharePoint Daily

  5. Pingback: SharePoint Fridays: SharePoint’s Complexity Translates To Organizational Risk | Usability Counts, a blog about Usability and User Experience, and Social Media

  6. Dan says:

    Coming from the business end of the equation, much of this seems caused by a disconnect between IT and the Business leaders. As you say, the frustration from our side is the seeming inability of IT to deliver, but much of the problem comes from the way we treat IT. We act as though it’s impossible to understand, and we often shut ourselves out of the conversation because “we don’t understand all that tech stuff.” Business also rarely (in my experience) treats IT as a strategic partner; the IT side is just there to make the &^%$ machines work rather than planning & executing alongside.

    I guess you get to dump on finance & IT, I get to pile on by dumping on the strategic leadership on the business end. As long as they don’t understand & appreciate the real & sometimes necessary complexity of their IT needs, and they can’t explain what it is they want/expect IT to do to pull the company to success, they’re going to get crap. Overhyped, Overbudget, and overcomplicated. Sharepoint can’t save anyone from poor strategic thinking.

  7. admin says:

    Great comment Dan and i loved your last line – thats gold. “Sharepoint can’t save anyone from poor strategic thinking.” – I have to make a post using that title.

    As you say, poor organisational maturity always trumps technology. Business leaders who are tech savvy definitely can offer big advantages to their organisation, but what you would call a ‘global manager‘ needs to much more as well.

    Tell me what you think of this post where I looked at examples where business leaders did what we all want them to do and I think this does more to bridge the disconnect than to send all business leaders off to SharePoint school πŸ™‚

  8. Dan says:

    Feel free to use the phrase – it’s not copyrighted or anything. πŸ™‚

    I like the Globalisation post, but I’ll note that one underlying factor is people’s psychology. Vick’s Das touches on this briefly in one of the quotes where he mentions turf wars. Employees, often managers specifically, completely lose sight of the organization’s goals in the process of competing internally with their colleagues. The goal of any leader should be to set a tone that reminds people that their goal is to compete with other companies, not each other.

    The scale you mention makes getting everyone to pull together a monumental task; you’re asking a guy in India to partner (even indirectly) with a woman in Wisconsin to reach a common goal. It’s difficult enough when the boss is setting a tone of collaboration. It’s impossible if management is not clearly leading the way towards a goal.

    If I see one purpose to implementing Sharepoint, it’s for management to clearly and universally establish the tone and direction of the organization. Using the communication and collaboration to explain and encourage the work the company does. If they can’t express it clearly, it doesn’t matter what platform they put in , as you note.

  9. Pingback: It’s the little things « Intranettin’

  10. Pingback: Surviving Microsoft SharePoint - Addressing Risks - The Bamboo Team Blog

  11. Pingback: CleverWorkarounds » SBS 2008, Hewlett Packard, WSS3, Search Server 2008 Express and a UPS – Oh the pain!

  12. Kim35 says:

    Eventually, We get perfect knowledge just like this good topic? I propose to select the thesis or buy dissertation expert, just because that aid for getting high grade if you have dissertation writing.

  13. Eddy Williams says:

    Excellent article! It made some points I couldn’t understand before clear for me. Thanks for sharing

  14. mich says:

    Great read. I found your blog on google and i have your page bookmarked on my favorite read list!
    I’m a fan of your site. Keep up the good work

  15. This is great information! Certainly it makes sense to leave the mom in control of how much medicine she needs.

  16. CPA says:

    If you are wondering why the hell someone would invest in such a thing, then consider people with home loans, supposedly happily paying interest on those mortgages. It is that interest that finds its way to the holder (investor) of the CDO. So a CDO is supposedly an income stream.

  17. what a blog, I have been following your blog for so many months. Keep posting such informative posts on dissertation writing.

  18. In 1977, Marlowe Granola was born in the bustling kitchen of our fun, food-loving family. Through the years, our mom has perfected her delicious recipe, and though low in both sugar and fat, it never fails to satisfy our taste buds and hunger cravings. More Information About Natural Granola Visit

  19. essay writer says:

    They can be backed by residential property, commercial property, something called mortgage backed securities, corporate loans

  20. writers says:

    I like this article. It was very interesting and it could be useful for most of your readers, as I think.

Leave a Reply

Your email address will not be published. Required fields are marked *