This third post in the “Who do SharePoint Projects Fail” series has been hard to write, not because I am struggling with ideas, but because I have too many! It is hard to put all the reasons why SharePoint projects go wrong into a coherent chain of logic.

In the first two posts in this series, we did a basic examination of wicked problem theory.

Part 1 introduced you to tequila slammers, as well as the pioneering work by Horst Rittel and the concept of wicked problems.

Part 2 also delved into the murky depths of academic history to demonstrate that even back in the seventies when ABBA stole the hearts and minds of teenyboppers around the world, at least some people had time to look at wicked problems in relation to building IT systems.

If you take away anything from part 1 and 2, it is this.

• Too many tequila slammers hurt
• Before you blame the product, the project manager, the stakeholders, the nerds, the methodology or anything else in vicinity, go back to the problem you are solving and determine its ‘wickedness’

Now we will finally look at this large, complex, scary beast known as SharePoint. I have no means to quantify how much of a percentage of project problems arise from issues related to “the product”, but it definitely happens. Unsurprisingly enough, it is easy to argue that some of the areas that I highlight below are people issues, but we still get to indulge in Microsoft bashing – and who doesn’t enjoy a bit of that eh?

The Microsoft Effect – Good enough…

Microsoft in some ways remind me of Brittany Spears*. Both crash things, both can be abusive, both get themselves into trouble with the law, both have ignored legal judgements, both have loyal fans who love them no matter what and detractors that will never accept they can do anything right, both get caught with their pants down on occasion, and both have produced some pretty ordinary crap in their time. Despite this, they sell squillions of copies and make a ton of money.

*Just in case you read this article at some future point where Brittany is long forgotten, substitute whichever starlet is in rehab

But Microsoft is *big*, Microsoft are *safe*. We already use Microsoft in the organisation now and thus Microsoft products *integrate well*.

So what is it about SharePoint that makes it such a magnet for a wicked project? Put quite simply, with SharePoint they have taken integration to another level. Microsoft have done their homework and created a product/platform that has a combination of the right kind of capabilities that *together* can be extremely compelling. Each component, judged individually as a point solution would not necessarily compete against dedicated products in the space. But Microsoft long ago learned that a combination of tight integration and the old 80/20 rule usually wins.

There have been document management systems and web content management for years. I’ve previously been involved with a large Hummingbird deployment. As soon as I saw MOSS 2007 I know it was going to massive, not because it was as good as Hummingbird, because it wasn’t. But it was *good enough* and it was made by Microsoft. Market success therefore was assured.

Easy to Impress – A Mythical Example

To demonstrate how effective this combination of features can be as a sales tool, consider this mythical conversation between a somewhat green, yet enthusiastic Microsoft evangelist named Bill Gates, who is discussing MOSS with key stakeholders at the company GOOSUNACLE. Their names are Scott McNealy, Sergei Brin, Steve Jobs and Larry Ellison.

(1 point for each in-joke you spot)

Scott McNealy (Infrastructure Manager): Okay Bill, here’s our key problem. We can’t find anything and our knowledge sharing is abysmal. We waste so much time trying to work out where stuff should be, and nobody follows the standards. People reinvent the wheel constantly and we keep making the same mistakes over and over again. Our file system really sucks with so much duplication and complexity. It has so much history there, that nobody wants to own it or fix it up. This has cost us market share and our bottom line is suffering. What can you do to help?

Bill Gates: Well Scotty, (do you mind if I call you Scotty?), I am glad you asked. SharePoint represents a new way to manage your enterprise documents. It can help you with built in version control, check-in/out, metadata classification and a recycle bin. Additionally, it is web based, making it much more accessible to your staff. It is also tightly integrated with Microsoft Office, particularly the 2007 version. We also have document format conversion features and records management too.

Sergei Brin (Knowledge Manager) : “Sweet!. Kind of makes the whole notion of a separate intranet and file system start to look ‘old school’. What about search? I feel very strongly that a good search engine is worth its weight in gold.

Bill Gates: That’s a great point you make Sergei, but I think it is best you leave search to us experts at Microsoft. Now in relation to your question, search is vitally important and SharePoint has a very powerful, scalable, customisable search engine that allows you to index all sorts of data sources. 

Larry Ellison (Database Administrator). Okay, so you can classify and search your files and other content. But I’m a database kind of guy. What about our line of business data in our databases? Can we improve its visibility to a larger audience?

Bill Gates: You certainly can Larry. We have several features that facilitate this, but specifically, we have the business data catalogue feature, which allows you to connect to 3rd party data sources for search, business intelligence and reporting. That way, you do not need to train your users in your line of business applications. They can simply visit the SharePoint portal, and view their dashboards and key performance indicators, so gain visibility into the business. In addition we integrate with SQL Reporting Services and can easily surface enterprise reports into the portal. Excel services is another feature that falls into this area. Imagine being able to publish your complex Excel sheets to the portal and users can interact with then without requiring excel on their desktops.

Scott McNealy: That is all very interesting indeed, document management, business intelligence and reporting in the one product. The other big issue we have is ensuring that people do things consistently. Larry and Steve in particular, keep releasing marketing slogans without formal approval. The “unbreakable” and “sooper dooper secure”  campaigns kind of backfired on us, and it would have been much better we had a more rigorous review process.

Bill Gates: Ah yes Scotty my lad, that is an excellent example. SharePoint contains a powerful workflow engine under the hood, that allows us to automate business process. So for example, at Microsoft when staff members leave the company, as part of our exit process we automatically create a task for the ever popular “throw the chair sendoff”. Supporting the workflow, is the document management features that I described earlier, as well as business forms via infopath and forms server.

Steve Jobs (Communications and Branding Manager). What about the GUI? It has to look good for people to engage with it. Additionally, I’d like the content to be available via mobile devices too.

Bill Gates: SharePoint has a powerful web content management foundation as well. It is built upon ASP.NET, which uses a templating system known as master pages, page layouts and web parts. But rather than me get into the detail here, I’ll recommend you visit my all time favourite blog called CleverWorkArounds and read the branding series. It should answer all of your questions. In relation to mobile devices, content can be displayed on these devices with no additional conversion. Suffice to say that it takes advantage of some of the workflow and document management features to facilitate content editorial approval, scheduled publication, and the like.

Steve Jobs: We are sold, Sergei, pass me the chequebook.

The downsides of versatility

So, Bill has sold Steve, Larry, Sergei and Scott on the possibilities of SharePoint. All of their respective buttons have been pushed and that can see it solving particular problems in the business. At a high level, they can see the value of re-use and efficiencies gained from using the same platform.

But which problem are we solving? If you asked them individually at this point, they would give you different answers. Its clear that Steve likes the WCM features, while Larry digs Business Intelligence. Sergey loves search and Scott wants to sort out document management.

Already, there is a wicked problem in the making, because it is easy to fall into the trap of “a tool looking for a problem” and nobody yet has the same definition of the problem. (one of Rittel’s key wicked problem warning signs that we covered in part 1)

The panacea effect

With so many features in SharePoint, combined in just the right way, it is the panacea you have been looking for. So you have found your cure, lets use it for everything right? The panacea solution often costs more, or is overkill for an initial problem, so to justify it requires you to sell it’s potential to solve other, bigger problems.

If you have read the first two articles in this series, you will realise pretty quickly that this is very high risk approach and can get out of hand pretty quickly. Just because a tool is versatile and has the potential to solve multiple problems does not mean it will necessarily be the best option to solve your particular problem.

Add the Microsoft effect to the panacea effect and the combination is more lethal! 

IT departments are particularly guilty of going down the panacea road and the reason this happens is also unsurprising when you think about it. IT are conditioned to think in terms of ‘tools and services’. By that I mean that, they exist to support and manage the organisations IT services. All business divisions have a desire to consolidate, streamline and reduce wastage and the IT department is no exception. In terms of an IT Department perspective, consolidating the number of “IT services” is attractive as there is tangible cost in managing many disparate systems.

So on the surface of it, SharePoint brings the IT department the potential tangible advantage of a common platform to support and manage.

I won’t belabour this point any further, because I have written about this issue on a couple of occasions, the best example being my choose your own adventure post and more specifically, its follow up. To the left is the book cover I did for the choose your own adventure. Sometimes pictures speak a thousand words.

It is also not fair to pick on IT either. I’ve gone out to many sites where the ‘SharePoint champion’ is not an IT person at all. But there is a consistent theme there. They want to use it because they have seen the potential for it to solve problems. Seeing the potential to solve a problem is not a problem statement. Therefore, by deploying this tool into the organisation, what problem are you solving and how do you determine if it has been effectively solved?

As a result of the panacea effect, we end up at another great debate of IT projects.

Point solution vs platform solution

I believe that as far as web content management products go, there are better point solutions around than the features SharePoint supply. WordPress for me is a much better blog system than SharePoint, Drupal as a pure WCM offers similar features to SharePoint for a fraction of the cost. Documentum is a better records management system by far and Cognos is going to trump SharePoint in the business intelligence stakes. Closer to home, in my SharePoint for Cisco nerds series, I had a comment that suggested that RANCID was a much better choice than SharePoint for Cisco configuration backups. He is absolutely right too, it is much better. But he is equally wrong too, depending on what the problem is and the point of view of the individual. (to find out why you have to read the whole series and judge for yourself)

This debate is essentially a version of the “Off the Shelf” vs “Custom Designed Software” chestnut. A point solution will meet your core requirements, however it may require you to change your business processes to work around it. Custom designed solutions can integrate nicely with your business process as they are to your exact specification. But the time and cost to develop can be significantly more and requirements can be fluid or prone to creep. (and you know how project managers hate that!)

Ah, but of course, being a point solution, the advantage of those products are diluted as your requirements broaden. You need multiple point solutions, those point solutions need to play nice with each other. The integration and support costs start to increase.

Although I see SharePoint as a platform solution in general, it actually has the best and worst elements of both off the shelf and custom designed software. It can cost you a lot of dollars and it all depends on your problem being solved. For example, it can be used purely as an application development platform, where a custom solution can take advantage of SharePoint features to underpin their own application, saving development time as you do not have to write certain functionality yourself. In this scenario it was always going to be a custom development job anyway, and SharePoint actually saves you money.

However, some problems are best fixed with a point solution and some are not. Where you can get into trouble is investing a lot of time and effort into a platform, because of the adherence to the panacea effect. The goal behind the platform may well be legitimate and well intentioned, but at the end of day, you are potentially talking a high development cost to get it exactly the way you want when a point solution may be a fraction of the cost.

I previously blogged a 6 part series on SharePoint ROI considerations, and although I haven’t blogged about it yet, there are financial modelling techniques available to consider some of the esoteric considerations like ‘opportunity cost’ of point vs platform. I will blog on this topic in the future, but for now, if you take anything away from this section, it is not to underestimate the value of the point solution when looking at your problem being solved.

Conclusion

*pauses for a big sigh*

That was a tough article to write, and the next one is probably going to be tougher, as I will no doubt have an urge to go back and edit this article. But I’ll resist that temptation as otherwise I’d never get these articles out.

But I hope that what you take away from this post is a deeper understanding of some of the reasons why a SharePoint project in particular can get out of hand and become wicked in nature. The quicker you spot them, the quicker you can head off the potential pain and suffering that will follow.

In the next post we will keep digging around SharePoint’s dirty little secrets and then look into some techniques that we can use to try and mitigate some of these issues.

cheers

Paul

                      

No Tags

Okay, well maybe I didn’t quite do that. But what I did do was forward the KB article to a colleague whose company is the leading reseller for EMC Documentum in my town. We’d previously had one of those conversations over a few beers where we questioned the wisdom of SharePoint’s potentially unwise, yet unsurprising use of SQL Server as the storage engine.

…so what’s wrong with SQL?

I am going to briefly dump on SQL here for a minute. But first, let me tell you, I actually like SQL Server! Always have. I hated other Office Server products like Exchange until the 2003 version and SharePoint until the 2007 version. But on the whole, I found SQL to be pretty good. So hopefully that will stop the SQL fanboys from flaming me!

Those readers who appreciate capacity planning issues would appreciate the challenges SQL based storage brings to the table.Additionally, those who have used Enterprise Information Management products like Documentum or Hummingbird (now OpenText) would nod as if Microsoft have finally realised the error of their ways with this updated API.

All of the SharePoint goodies like version control, full text indexing and records management come at a price. Disk consumption and performance drain.  Microsoft say to plan for 1.5 times your previous growth in disk usage. In my own real-world results it is more like 2.5 times previous growth. Disk I/O is also increased markedly.

“So what? Disk is cheap”, you reply. Perhaps so, but the disk itself was never the major cost. Given that this is a SQL database we are talking about, a backup of a 100 gigabyte SQL database could take hours and a restore possibly longer. A differential backup of a SQL database would be the entire 100 gig as it is generally one giant database file! So the whole idea of differential backups during the week and full backups on weekends suddenly has to be re-examined. Imagine a disk partition gets corrupted rendering the data useless. In a file server, this might mean 20% of shared files are unavailable while a restore takes place. In a SQL world, you have likely toasted the whole thing and a full restore is required. Often organisations overlook the common issue of existing backup infrastructure not being scalable enough to deal with SQL databases of this size.

“100 gig”, you scoff, don’t be ridiculous”. Sorry but I have news for you. At an application level, there is a scalability issue in that the lowest logical SharePoint object that can have its own database is a Site Collection, not an individual site. For many reasons, it is usually better to use a single site collection where possible. But if one particular SharePoint site has a library with a lot of files, then the entire content database for the site collection is penalised.

Now the above reasons may be the big ticket items that vendors use to sell SAN’s and complex Backup/Storage solutions, but that’s not the real issue.

The real issue (drumroll…)

It may come as a complete shock to you, but documents are not all created equal. No! Really? If they were, those crazy cardigan wearing, file-plan obsessed document controllers and records managers wouldn’t exist. But as it happens, different content is handled and treated completely differently, based on its characteristics.

Case in point: Kentucky Fried Chicken would have some interesting governance around the recipe for its 11 herbs and spices, as would the object of Steve Balmer’s chair throwing with their search engine page ranking algorithms.

I picked out those two obvious examples to show an extreme in documents with high intrinsic value to an organisation. The reality is much more mundane. For example, you may be required by law to store all financial records for seven years. In this day and age, invoices can be received electronically, via fax or email. Once processed by accounts payable, invoices largely have little real value.

By using SQL Server, Microsoft is in effect allocating an identical cost of each document in terms of infrastructure cost. Since all documents of a site collection reside inside a SQL content database, you have limited flexibility to shift lower value documents to lower cost storage infrastructure.

How do the big boys do it then?

Documentum as an example stores the content itself in traditional file shares and then stores the name and location of that document (and any additional metadata) in the SQL database. Those of you who have only seen SharePoint may think this is a crazy idea and introduce much more complex disaster recovery issues. But the reality is the opposite.

Consider the sorts of things you can do with this set-up. You can have many file shares on many servers or SAN’s. Documentum for example, would happily allow an administrator to automatically move all documents not accessed in three months to older, slower file storage. It would move the files and then update the file location in SQL so the new location is hidden from the user and they don’t even know it has been moved. Conversely, documents on older, slower storage that have been accessed recently can be moved back to the faster storage automatically.

It also facilitates geographically dispersed organisations having a central SQL repository for the document metadata, but each remote site has a local file store, to make retrieval work at LAN speeds for most documents. This is a much simpler geographically dispersed scenario than SharePoint can ever do right now.

Restores from backup are quite simple. If a file server corrupts, it only affects the documents stored on that file server. Individual file restores are easy to perform and you don’t have to do a major 100gig database restore for a few files.

Furthermore, documents that have a compliance requirement, but do not need to be immediately available, can easily be archived off to read-only media, thus reducing disk space consumption. The metadata detail of the file can still be retrieved from the SQL database, but location information in the SQL database can now refer to a DVD or tape number.

For this reason, it is clear that SharePoint’s architecture has some cost and scalability limitations when it comes to disk usage and management, largely due to SQL Databases and the limitation of Site Collections for content databases.

So how can we move less valuable documents onto less expensive disk hardware? Multiple databases? Possibly, but that requires multiple site collections and this complicates your architecture significantly. (Doing that is the Active Directory equivalent of using separate forests for each of your departments).

Note to SharePoint fanboys: I am well aware that you can ‘sort of’ do some of this stuff via farm design, site design and 3rd party tools. But until you have seen an high end enterprise content management system, there is no contest.

So you might wonder why SharePoint is all the rage then – even for organisations that already have high end ECM systems? Well the short answer is other ECM vendors’ GUIs suck balls and users like SharePoint’s front end better. (And I am not going to provide a long answer )

Utopia then?

As I said at the start of this post, I was very happy to hear about Microsoft’s external storage API. In my mind’s eye, I envisaged a system where you create two types of document libraries: ‘standard’ document libraries that use SQL as the store and ‘enhanced’ document libraries that look and feel identical to a regular document library, but it stores the data outside of SQL. Each ‘enhanced document library’ would be able to point to various different file stores, configured from within the properties of the document library itself.

Utopia my butt!

Then a few weeks back some more detail emerged in SDK documentation and my dream was shattered. This really smells like a “just get version 1 out there and we will fix it properly in version 2″ release. I know all software companies partake in this sales technique, but it is Microsoft we are talking about here. Therefore it it my god given right…no…my god given privilege to whine about it as much as I see fit.

Essentially this new feature defines an external BLOB store (EBS). The EBS runs parallel to the SQL Server content database, which stores the site’s structured data. You will note that this is pretty much the Documentum method.

In SharePoint, you must implement a COM interface (called the EBS Provider) to keep these two stores in sync. The COM interface recognizes file Save and Open commands and invokes redirection calls to the EBS. The EBS Provider also ensures that the SQL Server content database contains metadata references to their associated BLOB streams in the external BLOB store.

You install and configure the EBS Provider on each Web front end server in your farm. In its current version, external BLOB storage is supported only at the scope of the farm (SPFarm).

Your point being?

If you haven’t realised why I marked the previous sentence in bold, it is this. Since this EBS provider can only be supported at farm scope, then every document library on every site on every site collection in your farm now saves its data via the EBS provider.

So there is utterly nil granularity with this approach. It’s an all or nothing deal. (There goes my utopian dream of two different types of document libraries). All of your documents in the farm are doing to be stored in this EBS provider!

But it gets worse!

The external BLOB storage feature in the present release will not remain syntactically consistent with external BLOB storage technology to be released with the next full-version release of Microsoft Office and Windows SharePoint Services. Such compatibility was not a design goal, so you cannot assume that your implementation using the present version will be compatible with future versions of Microsoft Office or Windows SharePoint Services.

So basically, if you invest time and resources into implementing an EBS provider, then you’re pretty much have to rewrite it all for the next version. (At least you find this out up front).

No utility is available for moving BLOB data from the content database into the external BLOB store. Therefore, when you install and enable the EBS Provider for the first time, you must manually move existing BLOBs that are currently stored in the content database to your external BLOB store.

Okay that makes sense. It is annoying but I can forgive that. Basically if you implement an EBS provider and enable it, your choices for migrating your existing content into it is a backup and restore/overwrite operation or simply wait it out and allow the natural process of file updates do the job for you.

When using an external BLOB store with the EBS Provider, you must re-engineer your backup and restore procedures, as well as your provisions for disaster recovery, because some backup and restore functions in Windows SharePoint Services operate on the content database but not on the external BLOB store. You must handle the external BLOB store separately.

I would have preferred Microsoft to flesh this statement out, as this will potentially cause much grief if people are not aware of this. It implies that STSADM isn’t going to give you the sort of full-fidelity backup that you expect. Yeouch! I feel I might get a few late night call-outs on that one!

Ah, but wait a minute there, sunshine, is that any different to now? STSADM backup and restore is not exactly rock solid now!

Any error conditions, resource drag, or system latency that is introduced by using the EBS Provider, or in the external BLOB store itself, are reflected in the performance of the SharePoint site generally.

Yeah whatever, this is code word for Microsoft’s tech support way of getting out of helping you. “I’m sorry sir, but call your EBS vendor. Thank you come again!”

Conclusion

I can’t say I am surprised at this version 1 implementation, but I am disappointed. If only the granularity extended to a site collection or better still an individual site, I could forgo the requirement to extend it to individual document libraries or content types.

So it will be interesting to see if this API gets any real uptake and if it does, who would actually use it!

later

Paul

                      

No Tags