As you may have noticed, this blog has been a bit of a dead zone lately. There are several very good reasons for this – one being that a lot of my creative energy has been going into co-writing a book – and I thought it was time to come clean on it.

So first up, just because I get asked this all the time, the book is definitely *not* “A humble tribute to the leave form – The Book”! In fact, it’s not about SharePoint per se, but rather the deeper dark arts of team collaboration in the face of really complex or novel problems.

It was late 2006 when my own career journey took an interesting trajectory, as I started getting into sensemaking and acquiring the skills necessary to help groups deal with really complex, wicked problems. My original intent was to reduce the chances of SharePoint project failure but in learning these skills, now find myself performing facilitation, goal alignment and sensemaking in areas miles away from IT. In the process I have been involved with projects of considerable complexity and uniqueness that make IT look pretty easy by comparison. The other fringe benefit is being able to sit in a room and listen to the wisdom of some top experts in their chosen disciplines as they work together.

Through this work and the professional and personal learning that came with it, I now have some really good case studies that use unique (and I mean, unique) approaches to tackling complex problems. I have a keen desire to showcase these and explain why our approaches worked.

My leanings towards sensemaking and strategic issues would be apparent to regular readers of CleverWorkarounds. It is therefore no secret that this blog is not really much of a technical SharePoint blog these days. The articles on branding, ROI, and capacity planning were written in 2007, just before the mega explosion of interest in SharePoint. This time around, there are legions of excellent bloggers who are doing a tremendous job on giving readers a leg-up onto this new beast known as SharePoint 2010.

So back to the book. Our tentative title is “Beyond Best Practices” and it’s an ambitious project, co-authored with Kailash Awati – the man behind the brilliant eight to late blog. I had been a fan of Kailash’s work for a long time now, and was always impressed at the depth of research and effort that he put into his writing. Kailash is a scarily smart guy with two PHD’s under his belt and to this day, I do not think I have ever mentioned a paper or author to him that he hasn’t read already. In fact, usually he has read it, checked out the citations and tells me to go and read three more books!

Kailash writes with the sort of rigour that I aspire to and will never achieve, thus when the opportunity of working with him on a book came up, I knew that I absolutely had to do it and that it would be a significant undertaking indeed.

To the left is a mock-up picture to try and convey where we are going with this book. See the guy on the right? Is he scratching his head in confusion, saluting or both? (note, this is our mockup and the real thing may look nothing like this)

This book dives into the seedy underbelly of organisational problem solving, and does so in a way that no other book has thus far attempted. We examine why the very notion of “best practices” often makes no sense and have such a high propensity to go wrong. We challenge some mainstream ideas by shining light on some obscure, but highly topical and interesting research that some may consider radical or heretical. To counter the somewhat dry nature of some of this research (the topics are really interesting but the style in which academics write can put insomniacs to sleep), we give it a bit of the cleverworkarounds style treatment and are writing in a conversational style that loses none of the rigour, but won’t have you nodding off on page 2. If you liked my posts where I use odd metaphors like boy bands to explain SharePoint site collections, the Simpsons to explain InfoPath or death metal to explain records versus collaborative document management, then you should enjoy our journey through the world of cognitive science, memetics, scientific management and Willy Wonka (yup – Willy Wonka!).

Rather than just bleat about what the problems with best-practices are, we will also tell you what you can do to address these issues. We back up this advice by presenting a series of practical case studies, each of which illustrates the techniques used to address the inadequacies of best practices in dealing with wicked problems. In the end, we hope to arm our readers with a bunch of tools and approaches that actually work when dealing with complex issues. Some of these case studies are world unique and I am very proud of them.

Now at this point in the writing, this is not just an idea with an outline and a catchy title. We have been at this for about six months, and the results thus far (some 60-70,000 words) have been very, very exciting. Initially, we really had no idea whether the combination of our writing styles would work – whether we could take the degree of depth and skill of Kailash with my low-brow humour and my quest for cheap laughs (I am just as likely to use a fart joke if it helps me get a key point across)…

… But signs so far are good so stay tuned 🙂

Thanks for reading

 

Paul Culmsee

www.sevensigma.com.au

Last year, Peter Serzo and I presented at the SharePoint Best Practices Conference in DC. We did an extremely serious talk called “SharePoint and SQL Reporting Services 2008 for the really really good looking” which rated rather well. As part of this, we recorded a bunch of screencasts that have never seen the light of day, so I thought that some would benefit from this being released to a wider audience.

Note: This post and content is really going make utterly no sense unless you have watched Zoolander. Even if you have seen the movie, before you launch into the webcasts, some scene setting is required.

The business need

Some time ago, Peter and I were contracted by the Derek Zoolander School for the Really, Really Good Looking after Derek saw Microsoft’s new SharePoint diagram when he accidentally picked up a “Computerworld” magazine. Apart from matching Derek’s suit colour rather nicely, the diagram captivated his imagination with the notion of “Insights”.

Zoolander thought that “Insight”, sounded like the perfect look to follow up from the highly successful “Magnum”, which he used to save the Malaysian prime ministers life. He took the diagram to his wife, and demanded that he must have “Insights” at all costs.

Zoolander’s wife saw the business problem that “Insights” would help to address. You see, the Derek Zoolander School for the Really, Really Good Looking, at great expense, custom developed an ERP system to manage everything you needed to know about male models. The system was called the “Computerised Records for Attractive People”…

The CRAP system stored all sorts of interesting information about male models, such as tracking their “hotness”, as well as important detail such as stated age versus actual age, and any cosmetic procedures that they have undertaken. After a long and expensive consultation, Peter and I concluded that SharePoint 2007, integrated with SQL Reporting Services, was the perfect solution to create the all important “Insights” that Zoolander so desperately needed.

As a result, we conducted a project kickoff meeting with Hansel and Peter tried to explain the architecture of reporting services using a nice diagram.

… but we worked out pretty quickly that this was not the way to explain how it all worked to poor old Hansel…

So instead, we went the live demo route. Being male models, custom development was totally out of the question. This solution had to be done using all out of the box methods in a quick and easy manner. Below are the four live demos that were recorded and now you can use them as inspiration for your own male modelling school.

• Our first webcast illustrates how we were able to create a meaningful report from the CRAP system within five minutes.
• The second webcast expanded on this idea, by illustrating how reports can be parameterised and linked together for drilldown reporting.
• The third demo modifies the user profile store to allow for recording of each users unique ID in the CRAP system
• The last webcast strings this all together for the final demonstration where we pimp the report to make it dynamic with no custom code.

 

The 5 minute report	Drilling down with Derek
User Profiles for the really really good looking	Pimp my report

 

We hope you find some value from these webcasts and we look forward to hearing about your hot new look as a result!

Thanks for reading

 

 

Paul Culmsee

www.sevensigma.com.au

Hi all

I am finding it increasingly difficult to find the time to post at the moment. Too many projects, too many initiatives and too many evil plans coming to fruition. It’s like every seed I planted last year suddenly sprouted this year and I can barely keep up. Whilst this is a good thing for a growing business, it is not a good thing when it comes to writing blog posts.

In April I’ll be jumping on a very long flight to London, to attend and speak at the SharePoint Evolutions conference, held at the Queen Elizabeth II Conference Centre.

This conference represents the evolution of the Best Practice conferences held over the last three years or so. It is one of the most unique and important SharePoint conferences of the year. SharePoint 2010 will be a key focus, yet unlike say a Tech-Ed, many of the topics have a heavy focus on the strategic side of the SharePoint challenge, in areas like Information Architecture, User Engagement and Planning and Deployment. There are five tracks in all, and over seventy speakers from all over the world.

• For the techie geeks who like to hang in datacenters and like to get paged late at night to fix things that have died, the IT Pro track (ITP) will push their buttons. ITP sessions will focus on topics such as Document Management, Database Sizing, SharePoint and SQL optimization or server farm deployments scenarios.
• For the developers and designers of the world, the DEV track is for you. DEV sessions will focus on topics in the areas of customization, development, and deployment best practices.
• For all of the cool people, we have the information worker track where I speak (IW). In fact the IW track is so damn cool that there are two IW tracks! Sessions here will focus around business strategy and adoption, information architecture, training your organization or developing a culture of collaboration.
• For the tech geeks who can code, who are therefore more elite than regular tech geeks and devs (looking at you Spence), there is a deep dive track to make you happy called level 400. In this track there will be IT Pro and Developer sessions that will be deep diving into the product and code. There will be very few slides, lots of source code and demonstrations.
• Finally, there is a community track. This track has sessions for all verticals and will include speakers from all types of companies who have implemented SharePoint and what they learnt by doing so. All speakers are actively engaged in the SharePoint community and user group and have a wealth of knowledge to share.

This conference is organised by Steve Smith of Combined Knowledge, who is renowned for putting together something special for all participants. The speaker list is pretty much a who’s-who of the SharePoint world, and I am very much looking forward to catching up with (Paul takes deep breath) Andrew Woodward, Ben Curry, Brett Lonsdale, Chandima Kulathilake, Dux Sy, Joel Oleson, Laura Rogers, Michel Noel, Mike Watson and Zlatan Dzinic to name a few.

Bob Fox will also be there, so we finally have that beer that apparently I am supposed to buy – according to Bob anyway :-).

So if you are going to attend a SharePoint conference this year, then my strong suggestion is to make it this one.

 

Thanks for reading

Paul Culmsee

www.sevensigma.com.au

Well, the event that I never thought would ever happen in Perth happened, and not only did it happen, it had more interest than expected and some people unfortunately missed out. Jeremy, as a result, had to take many upset phone calls. It seems that for Perth, once a few people got wind of SharePoint Saturday, everyone wanted in.

There were great sessions, great giveaways and I think overall, tremendous value for this free event. Seven Sigma sponsored the showbags, which we managed to fill with some awesome goodies, thanks to the generosity of Brett Lonsdale at Lightningtools, Michael Sampson, Bjoern Furuknap, Dux Sy, Combined Knowledge and the good folks at Colligo. If you attended the event, please show your support to these guys – they really went above and beyond. Mrs Cleverworkarounds, on the other hand, never wants to see or hear the word “showbag” ever again! 

For me personally, I enjoyed meeting Michael Noel. I think he and I were the only non devs at SharePint (okay well maybe Joshua Haebets too 🙂 ). Speaking of which, Joshua and Milan Goss were also great to meet too, and I’m sure that there might be projects in the future we will see each other on.

Seven Sigma also donated a seat on the first SharePoint 2010 week-long bootcamp to be held in Perth. As a background: I met Steve Smith in New Zealand last year and we got on very well. Recently, we asked him if he would consider Perth to run his 2010 bootcamps and he has agreed! This is a great outcome for Perth, having beat out Sydney and Melbourne for being the first to run them as this will be the first time the courses have been offered to the general public in Australia.

Steve Smith and Gary Yeoman will be flying in from the UK especially for this event, so it is not to be missed. Both Steve and Gary are internationally renowned for the quality of their training and the courseware itself is the very same material that Microsoft itself uses to train their own staff on SharePoint 2010. All you eastern states people reading this?  It’s about time you went west anyway, so come and check out Perth’s beer while you are here!

SharePoint 2010 Beta Developer Track     4 days

• Delivered by Gary Yeoman
• Date:  27th April  -  30th April 2010
• Cost: $3000 (+GST)

This course guides you through essential 2010 elements, from pre-requisites to system integration, giving you the skills to work confidently and leverage full value from new technology.

Please note: Due to our ANZAC public holiday this course is a 4 day course from 08:30 – 6:00pm. One additional session is added per day to make up for the Monday public holiday.

SharePoint 2010 Beta Administrator Track    5 days

• Delivered by Steve Smith – MVP
• Date: 10th May – 14th May 2010
• Cost: $3000 (+GST)

Step-by -step understanding is the key to successful implementation and deployment of SharePoint 2010. This 15-module course will guide you through each critical stage, giving you exactly the skills you need to leverage full value from the latest SharePoint technology.

Book now at Seven Sigma’s website:

http://www.sevensigma.com.au/2010/02/07/first-ever-sharepoint-2010-training-courses-2/

For more info visit: www.combined-knowledge.com.au

or contact: training@sevensigma.com.au

Vulgarity warning. Its the silly season, I am winding down and being more low-brow than usual with this post

There is this wonderful way to look at the world, through a lens of something called “Complex adaptive systems”. Unfortunately with a name like that, it is automatically doomed to be only spoken of and understood by, a small subset of those sort of dishevelled looking nerdy guys who others take the piss out of when they are not around.

The notion of complex adaptive systems explains many things, including why salesman can unintentionally really be damaging to an organisation. I thought that I needed to write about this, and given that I am going to talk about sales guys, I had to write in a manner commiserate with their level of understanding of how the world works. Since the chances of a sales guy reading my blog is probably low, I should be safe 🙂

So here we go.

Here is a sales guy. Although us geeks think they are assholes, for this metaphor we have to change our context of what an asshole actually is. I think of him more of a guy who gathers food and brings it to you.

Here is the world for a sales guy. He finds work, and feeds that work into the mouth of the organisation. For performing such a feat, he gets to nibble off a small morsel of the meal to keep for himself. If he feeds the organisation enough and makes it grow, he will get enough morsels to grow rather nicely himself. This is a pretty sweet deal if you are good at finding food, because your reward is a percentage of what you push into the organisations mouth. Therefore it is in the sales guys interest to find as much food as he can for the organisational “body”. In fact his performance is directly attributed to doing exactly that in the form of quotas or sales targets

 

At the other end of the chain, the implementers have to digest what has been fed them from mouth and produce output that makes clients happy. Therefore it is the people in the organisation that actually implement a project who are actually the assholes, not the sales guys. As a result, I can say with some confidence that most people reading this post, like myself, are all assholes.

As this cycle perpetuates over time, the body in between these two ends grows. To continue to feed this body and keep it growing, we need to seek out more food. To do this we try and incentivise our sales people to supply more food by offering them larger morsels if they make more ambitious targets.

Never forget the assholes

Now we all know that we have to eat a balanced diet with healthy foods. But some people find it a pain to do all of that preparation and effort and instead go and grab some Chinese takeout instead. To a sales guy who is being rewarded for the amount of food being delivered to the organisation, fast food is great! Remember that the sales guy only takes just enough of the food for no lasting effects and is the furthest away from the assholes to feel the negative effects on the organisational as a whole.

Now our sales guy starts to look like the image below.

Therefore, this process of incentivising sales guys by the amount of food that they pass into the mouth is not without its risks and often can damage the long term health of the organisational body. Fast food can be tolerated now and again of course. For example, we all get the occasional hankering for Kentucky Fried Chicken every 6 months or so, and delude ourselves that this time, unlike all of the other times, it will actually not be oily enough to power a small town and leave you with that queasy feeling that you get when your heart labours against your cholesterol.

This can be a self perpetuating cycle. For example, the sales guy feeds the organisation a blisteringly hot spicy lamb vindaloo. Naturally is a very unpleasant experience for the assholes and as a result, what is delivered to the customer is (literally) crap and costs much more than anticipated. This cost bleed puts pressure on the sales guys to feed the body to make up for the wasted time, effort and cost. But the sales guy is so far away from the assholes, it does not occur to him that it was the spicy lamb vindaloo was the wrong meal. Nor too, does he receive any feedback to let him know that the burning sensation still lingers.

So what does he do? He feeds an even spicier lamb vindaloo to the mouth. Why? because he now has learned how to find spicy lamb vindaloos and is reaping the rewards of many tasty morsels – a perfectly reasonable practice given that he is now put under pressure to deliver more food.

Despite good intentions…

This cyclical phenomenon is called the “ring of fire” and afflicts many organisations who just can’t seem to deliver projects on time and budget. The customers of these organisations, fed up with getting nothing but crap, start to look elsewhere, thereby increasing pressure and starting the cycle again. Management get all flustered and usually blame the assholes.

The essence of the notion of the complex adaptive system is that the assholes and sales guys need each-other. Attempting to optimise the sales guys performance in isolation, ultimately has a negative impact on the assholes, which in turn has a negative impact on the organization as a whole.  The organisation is a system that comprises of many parts that interact in different ways. The system is perfectly capable of self organising and self optimising. For example, if the sales guy feeds the organisation sushi and next time it is fed a burrito, the assholes have a certain amount of tolerance to deal with that. But when you optimise one end (reward for food) without considering the assholes at the other end, you actually reduce that tolerance to deal with change!

The lesson that should be learned here is that the command and control methods of problem solving or project management that operate by optimising one part of the system, will usually work in the short term, but to the long term detriment to the system as a whole. The result that I have seen first hand for many IT organisations in particular, is that they have developed a certain reputation in the market for being a bit on the nose because of their seeming inability to get a project completed. Once this happens, it is very very difficult for them to regain the lost trust.

Microsoft for example, has taken years to win back the hearts and minds of geeks for their actions more than a decade ago.

What sort of fast food is SharePoint?

 

If SharePoint were a fast food, it would either be one of those giant steaks that you get your name on the wall if you finish, or the Guatemalan chilli that sent the normally invincible Homer into the spirit world. It is so seductive to the sales guys because it is in demand, but their distance to the assholes means that they will think it should be just like any other IT infrastructure oriented project to install. Therefore, some integrators will be doomed to repeatedly bite off more than they can chew and by the time they realise it, the long term damage will be done.

So what do you do?

If you accept that the organisation is a system and that optimising one part of it will likely impact the rest of the organisation, often in unpredictable ways, then incentivising has to be more strategically focussed. In other words, the true performance indicator on a good sales guy is actually the success of the project, because it is a much more reliable indicator of the sort of food being passed to the mouth and results in customer goodwill – social capital. If sales guys received their morsels based on the success of the project as a whole, then it would force them to interact more with the assholes to achieve that end and think a little more carefully about what they feed the organisation.

But self interest is a very strong force and there are very few sales guys that would be enthusiastic about that idea. This is of course the other big problem. The longer you leave it, the harder it is for an organisation to make the changes necessary to produce the outcomes that they aspire to.

If you want to see that in practice, look no further than the Copenhagen climate change conference.

Final note about thinking in terms of systems

Of course, if we are taking a complex adaptive systems view, then one could argue that the affect of all of this would be that your sales guys will leave and find organisations who feed them bigger morsels with much less effort of (heaven forbid) being judged on real outcomes. As a result, opportunities for sales may be lost to competitors and the organisation still suffers as a whole

This is the dilemma of systems thinking and what frustrates the hell out of the “command and control” world. You can just end up with a giant talkfest and never actually make a decision on anything because systems adapt in ways that can’t be predicted.

Is it any therefore wonder that command and control usually wins out? 

 

Thanks for reading

 

Paul Culmsee

In the words of Doctor Smith from lost in space, while everyone else was in Vegas having a grand old time, I was at a client site, having to come to grips with the beast known as Windows Small Business Server 2008. I rarely work with SBS2003 and had never used SBS2008 until now.

This was one of those engagements that is somewhat similar to those awful dreams that you have when you are trying to get to some place, but you never quite get there and your subconscious puts all sorts of strange and surreal obstacles in your path. In my case, the surreal obstacles were very real, yet some of them were really really dumb. Whatsmore, it is a very sad indictment on IT at several levels and a testament to how complexity will never be tamed with yet more complexity.

As a result, I really fear the direction that IT in general is heading.

So where to begin? This project was easy enough in theory. A former colleague called me up because he knew of my dim, dark past in the world of Cisco, Active Directory and SharePoint. He asked me to help put in SBS2008 for him, configuring Exchange/AD/SharePoint and migrating his environment over to it.

“Sure”, I say, “it’ll be a snap” (famous last words)

I haven’t use the coffee or tequila ratings for a while, so I thought that this post was apt for dusting them off. If you check the Why do SharePoint Projects Fail series, you will see that I use tequila shots or coffee at times. In this case, I will use the tequila shots to demonstrate my stress levels.

Attempt 1

We start our sorry tale a few weeks ago, where my client had ordered Small Business Server 2008 and the media/key had not arrived by the time I was due to start. The supplier came to the rescue by sending them a copy of the media and promised to send the license key in a couple of days.

The server was a HP Proliant DL360 G6, a seemingly nice box with some good features at a reasonable price. HP/Compaq people will be familiar with the SmartStart software and process, where instead of using the windows media, you pop in the supplied SmartStart CD and it will perform some admin tasks, before asking for the windows media, auto-magically slipstreaming drivers and semi-automating the install.

On client machines, I never use the CD from the vendor because there is always too much bloatware crap. However on servers I generally do use the CD, because it tends to come with all the tools necessary to manage disk storage, firmware and the like. I dutifully popped in SmartStart CD, answered a few basic questions, and it asked for the windows media CD.

Cleverworkarounds stress rating: Good so far

Next it asked me for the Windows SBS2008 licence key. Of course, I was using media that had been lent to us from the supplier because my client’s media (and keys) had not arrived. Thus, since I did not have a license key I was unable to proceed with the install using this preferred manner. HP, in their infinite wisdom, have assumed that you always have the license key when you install via their SmartStart CD, despite Microsoft giving you 30 days to activate the product. To be fair on HP, they are hooking into Microsoft’s unattended installation framework, so perhaps the blame should be shared.

All was not lost however. The SmartStart CD can be run after windows has been installed. It then will install all the necessary “HP bits” like graphics and system board drivers. So I booted off the Windows CD and fortunately, windows installer detected the HP storage controller and the disk array,  and proceeded to let me partition it and install.

Cleverworkarounds stress rating: Minor annoyance, but good so far

Small Business Server 2008 did its thing and then loaded up a post install wizard that sets the timezone, active directory domain name and the like. At a certain step when running this wizard, SBS2008 informed me that there was no network card with a driver loaded, so it could not continue. As it turns out, Microsoft’s initial SBS2008 configuration wizard simply will not proceed unless it finds a valid network card. But since this is the pre-install wizard, we are not yet at the point in the installation where we have a proper windows desktop with start menu and windows explorer. All is not lost (apparently), because SBS allows you to start device manager from within the wizard and search for the driver.

Fair enough, I think to myself, so I pop in the HP SmartStart CD and tell device manager to search the media.

Cleverworkarounds stress rating: Spider senses tingle that today might not turn out well

Windows device manager comes back and tells me that it cannot find any drivers.

Why?

Well after some examination, the SmartStart drivers are all self extracting executables and therefore device manager could not find them when I told it to. Of course, the self extracting zip files have hugely meaningful names like C453453.EXE making it really obvious to work out what driver set is the one required… not!

Luckily, Ctrl+Alt+Del gave me task manager which allowed me to start a windows explorer session, and I was able to browse to the CD and run the autorun of the SmartStart CD manually. This loaded up HP’s fancy schamsy driver install software that produces a nice friendly report on what system software is missing and proceeds to install it all for you.

SmartStart did its thing, finding all of the driverless hardware and installed the various drivers. A few minutes and a reboot later and SBS2008 reruns its configuration wizard and this time finds the network and allows me to complete the wizard. This triggers another thirty minutes of configuration and another reboot and we have ourselves a small business server!

Cleverworkarounds stress rating: Spider senses subside – back on track?

Next I did something that is a habit that has served me well over the years (until now). I reran the Smartstart CD, now with a network and internet access. This time I told the driver management utility to connect to HP.COM. It scanned HP and reported to me that most drivers on the SmartStart CD were out of date. This is unsurprising because most of the time I do server builds for any vendor, I find that about half of the drivers, BIOS and various firmwares have been replaced by newer versions since the CD was pressed.

Since this is a brand new server build, it is a habit of mine to upgrade to the latest drivers, BIOS and firmware before going any further.

Among the things found to be out of date was the BIOS, the firmware on the RAID Storage controller as well as the network card. The SmartStart software downloaded all of these updates, and another reboot later, all are installed happily. Another hour of patching via Windows update, and we have a ready to go SBS2008 server with WSS3, Exchange, SQL Express and WSUS all configured automatically for you.

Cleverworkarounds stress rating: This SBS2008 stuff isn’t so bad right?

Okay, so things were good so far, but now here is where the fun really begins.

Windows 2008 SBS comes with a pre-installed WSS3 site called http://Companyweb. As we all know, search completely sucks in WSS3. It has a bunch of limitations and isn’t a patch on what you get with MOSS. But no problem – we have Microsoft Search Server Express now, a free upgrade which turns WSS search from complete horribleness to niceness fairly quickly.

For those of you reading this who run WSS3 and have not installed Search Server Express, I suggest you investigate it as it does offer a significant upgrade of functionality. Search server express pretty much makes WSS3 have the same search capabilities of MOSS 2007.

So, I proceeded to install Search Server 2008 express onto this Small Business Server 2008 box. I have installed Search Server Express quite a few times before and I have to admit, it is a tricky install at times. But given that this was a fresh Small Business Server 2008 install and not in production, as well as having successfully installed it on Small Business Server 2003 previously, I felt that I should be safe.

I commenced the Search Server 2008 express install, and the first warning sign that my day was about to turn bad showed itself. The install of search server express only allowed me to choose the “Basic” option. The option that I wanted to use, “Advanced” was greyed out and therefore unavailable.

Cleverworkarounds stress rating: Spider senses tingling again

Knowing this server was not in production, I went ahead and allowed Search Server Express to install as per the forced basic setting. The install itself appeared to work, but it died during the SharePoint configuration wizard. It specifically crapped out on step 9, with the error message

“Failed to create sample data. An exception of type Microsoft.SharePoint.SPException was thrown.  Additional exception information: User cannot be found”.

“Curses!” I say, “another trip to logs folder in the 12 hive”. For the nerds, the log is pasted below.

[SPManager] [INFO] [10/21/2009 3:38:47 PM]: Finished upgrading SPContentDatabase Name=ShareWebDb Parent=SPDatabaseServiceInstance Name=Microsoft##SSEE.
[SPManager] [DEBUG] [10/21/2009 3:38:47 PM]: Using cached [SPContentDatabase Name=ShareWebDb Parent=SPDatabaseServiceInstance Name=Microsoft##SSEE] NeedsUpgrade value: False.
[SharedResourceProviderSequence] [DEBUG] [10/21/2009 3:38:47 PM]: Unable to locate SearchDatabase. Exception thrown was: System.Data.SqlClient.SqlException: Cannot open database "SharedServices_DB_ed3872ca-06b1-44c5-8ede-5a81b52265f9" requested by the login. The login failed.
Login failed for user ‘NT AUTHORITY\NETWORK SERVICE’.
   at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj)
   at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
   at System.Data.SqlClient.SqlInternalConnectionTds.CompleteLogin(Boolean enlistOK)

The long and short of this error takes a little while to explain. First we need to explain the historical difference between SQL Server Express edition and SQL Server Embedded edition (also known as the Windows internal database). From wikipedia:

SQL Server 2005 Embedded Edition (SSEE): SQL Server 2005 Embedded Edition is a specially configured named instance of the SQL Server Express database engine which can be accessed only by certain Windows Services.

SQL Server Express Edition: SQL Server Express Edition is a scaled down, free edition of SQL Server, which includes the core database engine. While there are no limitations on the number of databases or users supported, it is limited to using one processor, 1 GB memory and 4 GB database files.

Why does this matter? Well Microsoft, being the wise chaps that they are, decided that when you perform a SharePoint installation using the “basic” option, different editions of SharePoint use different editions of SQL Server! Mark Walsh explains it here:

• When you use the "Basic" install option during MOSS 2007 installation it will install and use SQL Server 2005 Express Edition and you have a 4GB database size limit.
• When you use the "Basic" install option during WSS 3.0 installation it DOES NOT use SQL Express, it uses SQL Server 2005 Embedded edition and it DOES NOT have a 4GB size limit.

It happens that Small Business Server 2008 comes with WSS3 preinstalled. Annoyingly, but unsurprisingly, the Small Business Server team opted to use the BASIC installation mode. As described above, SQL Server Embedded Edition (known on Win2008 as the Windows Internal Database) is used. For reference, WSUS on Small Business Server 2008 also uses this database instance.

BUT BUT BUT…

Search Server 2008 Express, uses SQL Server Express edition when performing a basic install. As a result, an additional SQL Server Express instance (SERVERNAME\OFFICESERVERS) gets installed onto the Small Business 2008 server. Then, to make matters worse, the installer gets mixed up and installs some Search Server express databases into the new instance (a Shared Service Provider), but then uses the SQL Embedded Edition instance to install other databases (like the searchDB). Then later during the configuration wizard, it cannot find the databases that it needs because it searches the wrong instance!

The net result is the error shown in the log above. I tried all sorts of things like copying the Express databases into the embedded edition, but I couldn’t disentangle this dependency issue. Some parts of SharePoint (the search server express bits no doubt) looked in the SQL Express instance and the WSS bits looked in the Embedded SQL instance. Eventually, conscious of time, I proceeded to uninstall Search Server Express.

Cleverworkarounds stress rating: Some swear words now uttered

Uninstalling Search Server Express was attempted and tells me that it has successfully completed and wants a reboot. Unfortunately SharePoint is now even more hosed than it was before and I tried a few things to get things back on track (psconfig to create a new farm and the like). After more frustration, and conscious of time I decided to uninstall WSS3 altogether and then reinstall it according to the SBS Repair guide for WSS3.

This had the effect of stuffing up WSUS as well! (I assume because it shares the same Windows Internal Database instance), and after a couple of hours of trying all sorts of increasingly hacky ways of getting all of this working, I was forced to give up.

Note: Whatever you do, do not attempt this method. At one point I tried to trick WSS3 into temporarily thinking it was not a basic mode install so get Search Server Express to prompt for Advanced mode, but it made things worse because the configuration database got confused.

Cleverworkarounds stress rating: Installing SharePoint in basic mode is committing a crime against humanity.

Attempt 2

At this point I hear Doctor Smith abusing me like the poor old robot. “You bubble-headed booby, you ludicrous lump, adlepated amateur, dottering dunderhead”.

Since WSUS got completely screwed, as well as the Windows Internal Database through relentless uninstallation and repair attempts, I started to get nervous. Small Business Server 2008 is a very fussy beast. Essentially it can get very upset at seemingly benign changes. I felt that I had messed about so much that I could no longer guarantee the integrity of this server so I wiped clean and reinstalled.

I installed Windows as per the previous method, and once again the install wizard stopped, asking me for a network driver. Once again I popped in the HP SmartStart CD and proceeded to run the driver install program from the SBS configuration wizard.

This time, no network card was detected!

What the? During attempt 1, I happily installed the necessary HP drivers using the same %$^#% SmartStart CD! Why is it not detecting now??

Cleverworkarounds stress rating: Now thinking about how much fun everyone is having in Vegas while I am fighting this server

After some teeth gnashing, the cause of this problem hit me as I was driving home from the site. I had upgraded the firmware of the network card during attempt 1, as well as the storage controller and system BIOS. I realised that the stupid, brain-dead HP network card drivers likely could no longer recognise its own network card with a newer firmware.

The next day I came back refreshed, and found that indeed, there were newer network drivers at the HP site. I downloaded them and extracted them and sure enough, suddenly the network card was found and I was back in business. How dumb is that! Surely if you are going to write a driver, at the very least make it recognise the hardware irrespective of firmware!  

Once I got past this stupid annoyance, it did not take too long for SBS2008 to be installed and ready to go. Remember that WSS3 is installed for you in basic mode, so to change it requires an uninstall of WSS and then to reinstall it in a different mode. But the problem here is SBS2008 and its fussiness about messing with configuration. In going down this path, you risk future service packs and updates breaking because things are not as expected. Additionally you would have to create companyweb manually and it raises the risk of a misconfiguration or mistake along the way.

I logged a call with Microsoft and got a pretty good engineer (hey Ai Wa) and she was able to consistently reproduce all of my issues in the lab, but was unable to work out a supportable fix. In the meantime, I tried to force Search Server 2008 to install into advanced mode using a scripted install, using an article written by my old mate Ben Curry. Alas, I could not bend it to my will and at the time of writing this article, I had to give up on Search Server 2008 with SBS2008 for now.

For what its worth, I know that I can make this work by installing WSS3 differently, but I planned to properly nail this issue in my lab using the out of the box installs and then publish this article, but I didn’t have the sufficient hardware to run Small Business Server 2008. It requires a lot of grunt to run! So I will revisit this issue once my new VM server arrives and post an update.

But there is more…

If the whole Smartstart, network driver, SBS2008 with its dodgy scripted WSS3 install, with Search Server Express dumb installation assumptions were not enough, I hit more dumb things that resulted in showing how ill equipped HP’s support is able to deal with these sorts of issues.

The server, not surprisingly, supplied with a nice hardware RAID configuration and my client opted to buy some additional disk. When the disks arrived and were installed, we found that the RAID controller could see the disks, but we were unable to add the disks to the existing array using HP’s management tools. HP’s “friendly” support was unable to work out the issue and asked me to do things that were never going to work and insulted my intelligence. Eventually I worked out what was going on myself, via HP’s own forums. It turns out that the HP Server requires a write cache module to be able to grow an array. We had one of these installed. Upon further examination by opening up the chassis, we were missing a battery to go with the write cache module. HP were unable to determine the part number that we needed and we ended up working it out ourselves and telling our supplier.

Then HP stuffed up the order and after following up for two weeks, it turned out they accidentally forgot to put the order through to Singapore to get the part. It seemed that once we went outside of the normal supply chain system at HP, it all broke down. After two weeks and numerous calls, they suddenly realised that the part was available in Sydney all along and it was shipped over next day!. The irony was that next day the part from Singapore arrived!

So now we had two batteries.

Grrrr. It shouldn’t be this hard! Why supply a write cache module and not supply the battery! Dumb Dumb Dumb!

Cleverworkarounds stress rating: Really hating HP and Microsoft at this point.

… and piece de resistance!

Okay so we had a few frustrating struggles, but we more or less got there. But here is the absolute showstopper – the final issue that for me, really made me question this IT discipline that I have worked in for twenty years now.

My client rang me on Monday morning to tell me the server was powered off when we arrived on site. This was unusual because the rack was UPS protected and no other devices were off. We ran HP’s diagnostic tools and no faults were reported. The UPS was a fairly recent APC model, and we installed a serial cable to the server and loaded the APC UPS Management software. The software (which looks scarily like what 16 bit apps used to look like in the early 90’s), found the UPS and showed that was all hunky dory.

We decided to perform a battery test using the software. No sooner than I clicked OK, the server powered off with no warning or shutdown. Whoa!!

I put in a call to APC, and was told that the UPS was not compatible with HP’s new fancy shmansy green star rating power supplies. We had to buy a new UPS because of some “sine wave” mumbo jumbo (UPS engineer talk that I really wasn’t interested in). If the UPS switched to battery, this server would think power was dodgy and do a pre-emptive shutdown. The reason that the server was powered off on the Monday morning was that the UPS has a built in self test that runs every 7 days that cannot be disabled!

Cleverworkarounds stress rating: For %%$$ sake!.

Conclusion

Now I don’t know about you, but once a UPS cannot be “compatible” with a server, that’s it – things have gone too far. For crying out loud, a UPS supplies power. How $%#^ hard can that be? How is an organization supposed to get value out of their IT investments with this sort of crap to deal with.

Then add in the added complexity of Blade servers, Citrix, Virtualisation, shared storage, and I truly feel that some sites are sitting on time bombs. The supposed benefits in terms of efficiency, resiliency and scalability that these technologies bring, come at an often intangible and insidious cost – sheer risk from incredible complexity. If you look at this case study of a small organisation putting in a basic server, most of the issues I have encountered are the side effects of this complexity and the lack of ability for the vendors to be able to help with it.

As the global financial crisis has aptly demonstrated, when things are complex and no-one person can understand everything, when something bad does happen, it tends to do it in a spectacularly painful and costly way.

Finally, before you reply to this likely immature rant and tell me I am a whiner, remember this all you Vegas people. You got to have fun and marvel at all the new (complex) SP2010 toys, while I sat on the other side of the planet in a small computer room, all bitter and twisted, sprouting obscenities to HP, Microsoft and APC dealing with this crap. When you put that into perspective, I think this article is quite balanced! 🙂

Thanks for reading

Paul Culmsee

www.sevensigma.com.au

…and we’re back!

Well… that was a long commercial break wasn’t it 🙂

In case you missed part 1 of our version of the show “deal or no deal”, you missed the big cliff-hanger and you really should read part 1 first. For the rest of you, to quickly recap, I came out of the closet and admitted by secret teenybopper shame, told the world that my wife had a teenage thing for Jean Claude Van Damme, showed the effect of beer goggles and introduced the notion of cognitive bias and how it can affect judgement.

i also demonstrated how, by altering the frame of reference, to a problem something that at first seems completely unquantifiable “how the hell do I know how many SharePoint developers drive yellow cars?”, is actually not as “impossible” as you may first think.

At the end of the last post I left you with a $10000 dilemma. You had to make a “deal or no deal” decision about going with your estimate about SharePoint developers who own yellow cars, or to instead cast your lot with a bag of marbles with a 9 in 10 chance of winning the prize. Just to refresh your memory, here is the salient part of the pub conversation.

• Me: Okay, so you are 90% sure that here are between 300 and 2000 SharePoint developers in the world with a yellow car?
• Them: Yes
• Me: So, let’s make this like the game show “deal or no deal”. If you are right and the answer is within your range, you will win $10000. BUT you have an alternative…
• Them: Ok…
• Me: What if I were to present you with a bag containing 9 red marbles and 1 black marble and offer you $10000 if you pull out a red marble. Pull the one black marble, and you miss out on the money. Do you want to stick to your estimate or do you want to draw a marble?

So have you decided? Now be honest and see how you went against the 4 outcomes that I have experienced when trying this on people. Here are the possible answers in order of popularity…

1. The person chooses to pull from the bag of marbles rather than their ranged estimate. (This is the predominant answer for all people I have tried this with – perhaps 70-75% of all responses).
2. The person chooses to use their estimate over the bag of marbles. (perhaps 25% of people have answered with this option)
3. Upon hearing the bag option, the person wants to change their ranged estimate. (Happened to me once)
4. The person doesn’t care which method.. (never happened to me)

So which is the right answer to this question?

(drumroll) Lets tackle the possible answer in order of likelihood.

“Take the marble! take the maaaaaarble!”

For the 70 odd percent of you who opted to take your chances with the bag of marbles, GONG! you lose!

Better double check your estimates in future because you have demonstrated that you are over-confident in your estimates. In other words, you are suffering from optimism bias. To explain why, think about the original question carefully. I asked originally for a ranged estimate that you were 90% confident with.

I then presented an alternative that has a 9 out of 10 chance of success – also 90%. From a statistical point of view, you should be completely ambivalent as to which option to use. Therefore, despite being asked for a range that you were 90% confident with, the range you actually estimated is not really 90% at all. It has to be less than 90% for you to prefer a clear 9/10 probability.

So that is why you are so stressed and busy! You keep giving crap estimates that make life harder for you! 🙂 Either that or you are too nice and when your project manager looks at you with those big, sad project manager eyes, your heart melts and you relent.

Isn’t that cool in a nerdy way? It is very interesting to see people’s faces as the penny drops to this logic and they suddenly realise just how bad some of their past estimates have been as a result. The consolation prize is just about 4 out of 5 people do exactly the same as you and take the marbles.

“No deal, I will stick with my estimate”

For the smaller group who decide that their estimate is preferred, you also lose.

In this case, the reason why should be pretty obvious. You are so paranoid about getting it wrong, that you have made an estimate that is more like 95% or even 99% confident. Why? your range is too wide for 90% because when presented with a clear 9/10 chance of success, you chose your original estimate. While that may sound like you are confident, in reality you are a bit of a wuss, because in fact you are under confident with your estimate. So grow some balls you weenie 🙂

Honorary mention – “I want to change my estimate”

At the Best Practice Conference in DC, I attempted this pub trick on Yoav Lurie from Synteractive, who is much more of a business and strategic thinker than us IT nerds. His response I think, deserves an honorary mention for being the closest to winning the game. In this example, I asked him to estimate in feet, the wingspan of a Boeing 747. I knew instantly that he was a good estimator because of the logic he used to come to a range.

“Hmm, well an aircraft seat is maybe one and a half feet, and there will be 10 seats in the cabin, with two passages that are probably two feet in width…so that ads up to…”

What do you notice about what Yoav did? Straight away, he related the wingspan of an aircraft (a clear unknown), to something he could make a reasonable estimate of (the width of an aircraft seat). After all, we have all sat in an aircraft seat in sardine (economy) class and know how cramped it is. He knew there were three rows of seats and related this to the width of the cabin, which he then related to the size of the wing. Deducing that the wing might be 4 to 6 times the width of the cabin, he then was able to make a very good ranged estimate of the overall wingspan of the plane.

I was very impressed at his estimate and how he arrived at it, but I still got him 🙂

As soon as I presented him with the bag of marbles alternative, without missing a beat he said “I want to change my estimate”. It took only a split second of presenting a clear 90% probability made Yoav realise that his estimate was not 90% and he was still a little overconfident.

That being said, Yoav’s method of relating something known to help frame the reference to something unknown is the only time anyone has used any sort of rigour in forming an estimate and very impressive for the pub setting 🙂

The right answer

Okay, so as you may have guessed by now, the right answer is to shrug your shoulders and say “I don’t care” or wave your hand at me and say “pfft, whatever”. (This is one of the few times saying you couldn’t care less is the right answer). In doing so, you have placed equal weight upon the choices, based on the assumption that both are 90% probabilities.

Neat pub trick huh? It certainly gets people thinking.

How to calibrate yourself

Douglas Hubbard talks about “calibrated estimates” in his books and has an appendix of calibration questions, that are designed to help you perceive and account for cognitive bias in your estimating.

What you should take away from this exercise is that when asked to estimate on something you are uncertain about, make your initial estimate. Then, pretend you are in the game show and you have to pick between this estimate and the marble. If you feel that you would take the marble over your estimate, increase the width of your range until you feel that it doesn’t matter which option you pick.

Conversely, if you are one of the wimps who are under confident, then reduce the width of your range, until you feel that you have no particular preference of your estimate vs. the marbles.

In the same way that reframing a problem led from something being unquantifiable to something that indeed had a upper and lower range, by reframing the estimate against a unambiguous probability such as a bag of 10 marbles with 9 red, helps you to account for cognitive bias in your estimates.

Conclusion

So to reiterate my key points to this post

1. Many things that seem unquantifiable are easier to quantify than you think, once you think in terms of ranged estimates and probability.
2. Your bad taste in fashion and music when you were a teenager still manifests itself today and it is called cognitive bias.
3. There are easy methods that you can use to calibrate yourself better so that your estimating radar is more finely tuned.

Most importantly of all however, you learned that my wife liked Jean Claude Van Damme in the 80’s and you know that I am in big trouble when she reads this! 😛

Thanks for reading

Paul

Okay I’ll admit it, I used to really suck as a time and effort estimator. I happen to have a business partner who is much better at it than me (hey Peter), and every time I sought a second opinion from him on my estimates, he would almost always make a much less optimistic assessment then me. Of course, Peter was almost always right too, dammit.

So, why was Peter much more accurate with his estimates?

The answer to this question, all one has to do is think back to their teenage years, where they went through that awkward stage where you look back and cringe at the posters that were on your wall and your choice of fashion. For many, this period demonstrates some utterly appalling choices of taste. Mine are particularly cringe-worthy, given that these days I am a bit of a metalhead. My favourite song at the time was Respectable by Mel and Kim. I thought that Karate Kid II was the best film of all time (and that the girl in it was hot). Mind you, my wife has an even more shameful secret. She had a crush on Jean Claude Van Damme! Mwahahahah 🙂

These are examples of a phenomenon I like to call “Teenybopper bias” 🙂

Now, there is a point in telling you about my wife’s secret shame and it isn’t to see her reaction when she reads this (okay, well maybe a teenie bit). These examples of “what the hell was I thinking” are a form of cognitive bias that took place at the time the opinion was formed. In terms of teenybopper bias, the root of the bias is likely the same hormones that caused your face to break out with acne and hair to grow in funny places. Another very common cognitive bias that afflicts people whether young or old is good old “beer goggle” bias illustrated below.

There are many, many forms of cognitive bias documented, such as optimism bias, anchoring, hindsight bias and the recency effect to name a few. Now let’s take the final image above and pretend we asked someone at the pub for an estimate on a project at 8pm, 10pm and 1am. I’d be willing to bet that the estimate gets more optimistic on a par with how optimistic the perception of the people in the image above become.

Overcoming cognitive bias

Kailash writes about the risk that cognitive bias can play in project failure, particularly in the perception of risks.

overcoming biases requires an understanding of the thought processes through which humans make decisions in the face of uncertainty.  Of particular interest is  the role of  intuition and rational thought in forming judgements, and the common mechanisms that underlie judgement-related cognitive biases.   A knowledge and awareness of these mechanisms  might help project managers in consciously countering the operation of cognitive biases in their own decision making.

The essential difference between Peter and myself in our estimating, is that Peter happens to have a much more finely tuned radar to optimism bias in particular. Douglas Hubbard of Applied Information Economics fame, writes about the effect of cognitive bias extensively in his two books and offers a simple, yet highly useful method to quickly improve the quality of estimates which I will explain with an example below.

The great thing about learning about your cognitive biases and the methods for mitigating them, is that you can use it in the pub too. While I don’t recommend this method for picking up members of the opposite sex, it’s a pretty cool icebreaker.

Thus, I will demonstrate how to improve your estimating accuracy by a mythical pub conversation. Imagine you are onto your third beer…

• Me: “How many SharePoint developers worldwide own a yellow car?
• Them: “What the…I haven’t the faintest idea!”
• Me: “Well, I can understand that, so let’s do an estimate. Give me a range that the answer could fall in, that you are 90% confident with.”
• Them: “I still can’t give you an estimate, I can’t possibly know something like that.”
• Me: “Well, could there be a million SharePoint developers who like yellow cars?”
• “Them: “Don’t be ridiculous, there would be nowhere near a million SharePoint developers – period.”
• Me: “So you do have an upper bound then, less than a million. Remember this is not about the exact answer, I want a range that you would be 90% confident with.”
• Them: “Okay I get it. I think it is somewhere between three hundred and two thousand.

Note that at this point, we have already made the initial breakthrough. At first the person found it impossible to make an estimate, yet when I related it to something they did have a fair idea of (the thought of a million people), they made some mental associations and realised they did have some idea of limits after all. Thus, by presenting a better frame of reference that they could use to approach the problem, they were able to move from “I have no idea” to a wide range of possible values.

The width of the range reflects the uncertainty that someone has about the answer. The more the uncertainty, the wider the range. Some project mangers hate being given a ranged value because it really mucks up their task or project work breakdowns. As a result, they always want the “ball park” or something that is a single value. I completely understand why this happens, but what these people forget is that an estimation is uncertain by definition. The obvious way to express uncertainty is with a range of values! So asking someone for an estimation and then complaining that it is not accurate enough actually makes no sense. A manager might not like the “width” of the range, but you can’t force someone to reduce their uncertainty just because it doesn’t fit the plan. Unless you provide them with the means to reduce this uncertainty, you cannot and should not try and artificially reduce this range through pressure and coercion.

But despite my observation of the flawed logic of dealing with uncertainty in estimating, a ranged estimate alone is not enough yet. We still have not accounted for the sorts of cognitive bias that I described earlier in the article. So without further adieu, I present a simplified version of Hubbards ‘calibration’ techniques that account for bias. Let’s continue the bar conversation.

• Me: Okay, so you are 90% sure that here are between 300 and 2000 SharePoint developers in the world with a yellow car?
• Them: Yes
• Me: So, let’s make this like the game show “deal or no deal”. If you are right and the answer is within your range, you will win $10000. BUT you have an alternative…
• Them: Ok…
• Me: What if I were to present you with a bag containing 9 red marbles and 1 black marble and offer you $10000 if you pull out a red marble. Pull the one black marble, and you miss out on the money. Do you want to stick to your estimate or do you want to draw a marble?

 

I’d like readers to think about this before continuing with this article. Make a ranged estimate of the number of SharePoint developers worldwide who drive a yellow car, and then decide whether you want to stick to your estimate or take your chances with the marbles.

 

(Cue game show music where you have 10 seconds to decide with a little ping sound at the end.)

The suspense is now killing you I am sure. Want to know the correct answer?

Find out after this short commercial break (game show speak for wait till part 2 of this series 🙂 )

 

Thanks for reading

 

Paul Culmsee

www.sevensigma.com.au

Hi all

This post is somewhat related to the whole “Zen and the art of SharePoint governance” idea where the notion of a “best practice” is so culture and context dependent that the worst practice is to insist that your best practice applies universally :-).

Let me explain with this example. Consider two problem/solution scenarios, A and B.

• Scenario A: We have a technically inferior solution with a really deep commitment to that solution by the stakeholders and participants
• Scenario B: We have a technically superior solution with “uneven” commitment to that solution by the stakeholders and participants

Which scenario has a better likelihood of success?

Everyone I have asked this question to, has without fail, chosen scenario A.

Why is this? Because people instinctively know that strong commitment to a course of action will trump technical superiority most of the time?

Now, let me change the scenario to something a little more interesting and potentially divisive.

• Scenario A: We have a folder based document storage solution with a really deep commitment to that solution by the stakeholders and participants
• Scenario B: We have a metadata based document storage solution with “uneven” commitment to that solution by the stakeholders and participants

Which scenario has a better likelihood of success? I bet this time you probably feel like answering “Well, it is not that clear cut” or “Scenario A but…” or “Scenario A only because…”, etc. It seems that putting a specific, tangible example against the scenario creates the urge to question the validity of “either/or” in the scenario. This is because you implicitly recognise that there is a little more to it.

This is interesting to me because it gives away what I think is the more “correct” answer. The ideal outcome is actually a third scenario where we, as SharePoint consultants and practitioners, recognise the technical merits of a solution, and start the process of steering a group toward a commitment to that technically superior solution.

Well…most of the time anyway. 🙂

Considering the metadata vs. folders question above. Let me give you a common use-case where the metadata question actually gets murky. Let’s say you have a document library with four well defined metadata columns and have some views that are leveraging those columns. Some users are happily navigating to your site and uploading their files via the “Upload” options or the “New” option in the document library itself. In short, those users who are browser centric in their productivity habits find this to be a good solution. Let’s also say that some two hundred files have been uploaded to this library and are well classified and easy to find via the views. Below is a typical example that I doubt any SharePoint person would find unfamiliar.

But there is another chunk of the user population who has very different productivity habits, in that they are application centric. Let’s say a user lives their life in Microsoft Word (you might scoff but plenty of people who work with a lot of documents do work this way). They start MSWord up and then decide what document to open or work with. They click the open toolbar button (or the office home button) to open a document and navigate to the document library.

Uh, oh…

• Where is our metadata?
• Where are our views?

Well isn’t that interesting. We don’t get to leverage columns and views when accessing documents in this way. All we see is a large listing of files with no metadata. If there is one thing worse than too many folders, then it is surely none at all (with several hundred files in the root folder to choose from).

So now we have a tricky balancing act. Let’s consider what to do, based around the lens of user engagement and commitment.

If you chose scenario A earlier, the technically inferior yet deeply committed option, then you may argue against bringing out the big stick and trying to force users out of their productivity habits (i.e. switch from application centric usage to browser centric usage). You would be concerned that the big stick approach may result in an erosion of commitment, leading to a lack of adoption, leading to the project being deemed a failure.

But then, if we leave it as it is, users will continue to create and manage folders to compensate for the sheer number of files. We have failed to make use of some of SharePoint’s great document management features. We risk creating a chaotic installation, where the same poor information management habits that have likely created the interest in SharePoint in the first place, actually work against SharePoint and devalue it as a platform. This would also lead to user frustration, leading to lack of adoption and project failure.

Interesting dilemma…two options and both with significant risks of an undesirable outcome. What would you do here?

For me, it depends completely on the organisation, department and user tolerance for productive distress. (Don’t worry I will define productive distress in a minute).

Learning by opportunity

When sitting around the bar with Best Practice Conference attendees and presenters in DC, a rigorous debate occurred over aspects of SharePoint branding. One particular person made a fairly generalised sweeping statement and everyone else disagreed quite strongly. The person making that statement actually knew his stuff and everything said was technically and logically correct. But the instinct that I think the rest of us at the table had developed is that the notion of “best” and “right” is actually extremely fluid. No-one disagreed with his frustration that led to his big catch-all conclusion, but we all instinctively rallied against his single “best” solution that would apply to all scenarios.

So I will tell you openly that we have clients for whom we have developed relatively sophisticated information architectures around content types, metadata, search and logical architecture of site collections and sites (Particularly for BMS/QMS scenarios). Yet we also have clients where we have not used metadata at all at first, or else it is quite piecemeal across sites or site collections. What I have come to realise is that sometimes you have to allow people to work with the technically inferior (some would say wrong) option, before you can take them to the technically superior solution with the commitment required to see it become successful.

You have to remember that you, in some way, are pushing someone out of their comfort zone. Depending on the nature of the project, those people did not necessarily ask for you to do this. Thus, push too hard and you will have chaos and pullback. Don’t push at all and you remain with the status quo.

“Obtain management buy-in” is the standard catch-cry for most methodologies, right? It is often cited as if it is the be-all and end-all critical success factor for a project success. To me, it now sounds like a cliché that you say because it is the right thing to say. The same goes with vision and mission statements. Just because you have done those things in your project charter or plan does not mean that everyone is suddenly on board. It goes deeper than that.

I believe that real commitment from the users requires them to believe that a new system will indeed make things better. If they are not convinced that the change initiative will make a positive difference to them, then you are looking forward to a chaotic project that has the odds stacked against it.

Productive Distress

If, like me, you sometimes opt for doing things that go against what SharePoint blogs or books tell you, you simply have to provide the right sort of oversight to support your course of action. This oversight is part of the “how” that I spoke of in the secret to understanding governance post. I *know* that “client X” may only be scratching the surface of SharePoint’s capability, and it may frustrate a seasoned SharePoint professional that they are not making optimal use of what is available. But based on the nature of questions asked, we assess where they are in terms of SharePoint maturity and decide that they are not ready to jump to the deep end. Instead, we help them to work within their level of maturity and allow them to better understand their problem by learning about the solution. This small step approach allows the user base to be pushed just enough out of their comfort zone, where they are still productive and not pulling back. (Hence the term “productive distress”). Pretty soon, that productive distress is forgotten, becomes the new “business as usual” and we now move up a notch.

Repeat process as necessary. Fast forward a few day or weeks and suddenly the nature of the questions change, reflecting the improved understanding of SharePoint and its capabilities. There is a certain maturity around the questions asked, the sophistication behind the scenarios conceived and you can tell that they are now ready to move to the next level of productive distress.

So it is not really that “folders are bad” or the equally common “Keep SharePoint Designer out of the hands of mortals!” because, to repeat an oft abused cliché, they are just tools. If you can convince your users that SharePoint will improve the situation and allow them the time to adjust to the new paradigm, then they will work out for themselves whether folders are bad or not. As the scenario above demonstrates, it can sometimes depend very much on how you look at the problem. Also remember that most of the time, commitment trumps technical superiority.

Thanks for reading

Paul Culmsee

www.sevensigma.com.au

Hi there and welcome to part 3 of my series on the practice of Dialogue Mapping in the real-world. To recap, in part 1, I provided a brief overview of Dialogue Mapping and in part 2, I described a common real world usage scenario that we perform fairly often as SharePoint consultants.

The rest of this series will change tack a little. In this article I am going to describe a very different Dialogue Mapping scenario to you. This was a huge challenge and a large leap from what I described in part 2. There were some wonderful lessons learned from this work which I will cover off here.

The most “wicked” problems are not technical

My first Dialogue Mapping gig where I was not a subject matter expert also happened to be a real baptism of fire. Here was a problem that I had no understanding of, no discipline knowledge and no sense of background history of the project, the dynamics of the group, nor any idea of the positions of stakeholders on the problem.

By this time my IBIS fluency was pretty much down and I felt very confident with the usage of Compendium. I had not yet travelled to the USA yet to train under Jeff Conklin directly, but I carried his book around with me, and had read it many times over. I had performed Dialogue Mapping many times, however until this point all the projects or subjects that I was involved in I knew a lot about. This time I felt very vulnerable. Your domain knowledge is a form of armour, and it was unsettling to know that you’re going in stark naked. I was intimidated, yet excited at the same time.

So imagine this scenario. There was myself, a facilitator, and *fifteen* strangers sitting across from me. This was double the group numbers of the typical IT scenarios I had previously mapped. I remember emailing Jeff for advice when I heard that there would be so many and I recall him saying that 15 was a lot of people for a newbie. What was I in for! 🙂

Little did I know at the time that this group had been meeting for quite some time before that, but were really struggling on a complex urban planning issue. When I say complex, I mean complex in a social way, rather than technical. Interestingly, the issue was not a technically complicated issue at all. Anybody could have sat in the room and understood most of the dialogue (not necessarily the full context but you would not be completely blinded by science). What made this particular issue complex was the fact that the group members came from several different organisations, representing some quite diametrically opposing viewpoints. In part 2, I wrote about how it was hard enough just to get one IT department to come to the party and that was just one department of a single organisation – sheesh! If you have complained about organisational silos and think it is hard enough to get some degree of consensus within the realm of one organisation, imagine it when over a dozen representatives from different organisations are involved, organisations that straddle the full spectrum of public and private sectors, as well as the community.

There were simply so many stakeholders and interconnected issues that it was very hard to not get bogged down into tangents, repetition and frustration. Now, imagine eighteen months of this environment and the stressful social complexity impacts.

This is a long-term project that I am still involved with, so I will not be taking you through the specific details of this project just yet. Rest assured though, it has been such a great experience that I will write about it in detail in the future. For now, I will focus on lessons learnt so that others aspiring to perform this craft can learn from my own experiences.

People often tell you the best way to learn is to dive right in, and Dialogue Mapping is no exception. No sooner than I had put up a “what should we do…” type root question, it didn’t take long for debate to get … shall we say … rigorous! So I will go over some of the key lessons that I learnt from this experience thus far.

Lesson One: Confidence and assertiveness

The first thing that I had to contend with was not being in control of the conversation to the same extent as before. As previously stated, with SharePoint workshops I tend to direct the flow of the workshop as a mapper and as a subject matter expert. But this scenario was very different. I didn’t know the topic area at all and therefore some of the terms and acronyms made no sense to me. Also being new and unused to the decorum of the group, I erred on what I thought was politeness, rather than annoy the group by being direct and at times, interrupting them.

This, in my opinion, is a mistake and does a disservice to the other participants. It is also probably the most common thing a newbie mapper will experience when starting out, especially with a new group. As a result, the mapping in this first workshop ebbed and flowed. There were a few times where I was mapping the conversation really well, and the participants really engaged with the argumentation as it unfolded on the screen before them. Participants gestured to the map and asked me to add additional arguments and issues to what had already been built there. I could literally hear some of the initially sceptical, suddenly have that magic moment where they see it working. But at other times, during, say a particularly contentious issue, the conversation would fly at a rapid rate of knots. With so many people in the room, many wanted to have their say on these topics. This led to:

1. Side conversations started up
2. Some participants looked away from the map, and started debating directly to each other

As soon as one of these happened, and especially with the latter, I, as the Mapper, had no hope of following the conversation. As you might expect, I would lose focus and the quality of what was captured suffered (hence lots of idea nodes with no connections to anything). The focussing power of the map would be diminished and the wheels would start to fall off.

So remember this above all else. No matter what you do, you must be confident and assertive from the very start to keep the group focussed on the map. Don’t be afraid to interrupt someone to clarify a point, or pause them before they go too fast. If someone else starts to interject, interject back and make it clear that you will get to them as soon as you have finished capturing what someone else has said.

Here is the other critical thing: You must be consistent. It is the first ten to twenty minutes that will set the tone for the rest of the session. This is where people will implicitly learn the decorum of a Dialogue Mapping session and know what to expect. Your actions as a mapper, during this period, is critical to the overall quality of the session. Start it well and it will generally end well.

Jeff Conklin, of course, offers advice for this in his wonderful book on how to dealing with this. But of course, in the heat of dialogue, all of that advice goes straight out the window as you struggle to keep up with the rapid fire dialogue heading your way. Reading about how one should dialogue map is one thing, doing it is another. This is why I call Dialogue Mapping a craft and leads me to my second lesson learned.

Lesson Two: Remember that one guitar lesson you had? (Be realistic)

I think just about everybody at one point has had romantic notions of being a rock guitarist, banging out the blues like Clapton or blistering solos like Kirk Hammett or Brian May. A surprisingly large number of people have actually bought a guitar at some stage in their lives and have tried to live the dream. Most give it up once they find that the gap between their ability to play a G chord and their dream of playing the solo to Hotel California stretches to the moon and back. Inevitably, many guitars ends up collecting dust in the attic, along with the home gym set and many other items that were bought from late night infomercials.

You will hit this with Dialogue Mapping. Remember that wicked problems breed social complexity. Some problems may have some stakeholders with diametrically opposed views and discussion can be quite heated. The romantic notion of your group suddenly solving their wicked problem from your wonderful dialogue mapping has to be viewed with the reality that you still have to learn the G chord and audience can be fickle.

Thus, as far as audiences go, don’t go playing a stadium gig until you feel that you can handle sitting in the corner of a local bar or the school disco. In other words, start small and work your way up. A small, successful meeting will help you develop your style, confidence and then empower you to take on larger groups.

As Ali-G would say, keep it real.

Lesson Three: Stick to your domain of knowledge (at first)

This is a logical extension of lesson two, and also a tricky one because it can be just as much of a worst practice as a best one. This I suspect is probably a lesson on where Jeff Conklin or other dialogue mappers may disagree with me. One of the transitions that you have to make as a mapper is to move from what I would call Issue Mapping to real Dialogue Mapping. Just because you are doing mapping in front of a group, doesn’t actually mean you are necessarily “dialogue” mapping. Dialogue Mapping is often called “Issue Mapping with facilitation”, and when you work within your domain of expertise and you are a mapper as well as participant. Therefore you are not an impartial facilitator. This is the situation I described in part 2 and discovered very quickly, that pure dialogue mapping was much harder and mentally tougher.

But in terms of developing your skills, you will have good results when you are working in an area that you know well. You don’t have to worry about the meaning of acronyms and half of the questions you have likely heard before anyway. Remember though, that in a way it is kind of cheating because you are in effect, using the craft to get people to confront questions that you want answered. But it is an important stepping stone and will help you master lesson 4.

Lesson Four: IBIS grammar in the reptile brain

IBIS is the grammar that you use to map discourse. When mapping rapid-fire contributions from a group of people, they do not want to sit and wait for you to mull over whether their statement was an idea followed by a pro or an inferred question with an idea. If you find yourself having to do this, it is like trying to play a song on guitar and having to consciously think about how to play that G chord. Just think about how much you’d enjoy a Metallica concert if James Hetfield stopped every minute or so on a hard bit of a song and said “Wait, wait.. I’ve done this before… ok, hang on…oops, sorry”. This translation needs to burn itself into your reptile brain so that the process is as automatic as possible. To do this, you need to initially not worry about getting up in front of a group. As Conklin suggests in his book, listen to interviews on the radio or take an article, pull out its main points and create an IBIS map for it. There are a zillion ways to do this.

For all of you SharePoint people, a really excellent, highly relevant way of practice that I use, is to sit in the audience of a presentation at SharePoint Saturday or a conference and issue map the presentation. Below is a sample of some of the sessions where I have done this and the image after that demonstrates how much rationale I captured in the “NZ Web Standards and SharePoint” session.

  

Another great way to learn is to send one of your maps to an IBIS practitioner and let them pull it to bits. Even those of us who have done this for a while need that constant reinforcement and feedback. Like any grammar, different things can be written in different ways and one person’s IBIS will not always look like someone else’s. (There is another blog post in the works that will show this in a funny way). At various times I have sent maps to Conklin for constructive feedback (and then ducked for cover! – hehe)

Finally, if you are serious about this and like what you are reading, then do what I did – the 5 week Issue Mapping webinar based workshops that Jeff Conklin runs, or if you are in Australia and want something local, then contact me for a half or full-day in-house Issue Mapping intro workshop. Both will not teach you to dialogue map, but by the end of them you will dream in IBIS 🙂

Lesson Five: IBIS translation in the reptile brain

Once the language of IBIS is familiar to you, you can take any argumentation and form a consistent IBIS map. You then have to learn how to listen very carefully because that is half the art. Now you have to take prose and pontification by participants and somehow unpack the points made, articulate them into a summary, and form an IBIS based model in your head, and then commit it to the map.

This can be hard – very hard, and it is nigh on impossible to do without applying what I told you in lesson one. A dialogue mapper is not superhuman and does not have photographic memory. The skill you are developing here is one where you pick out the IBIS elements of some dialogue quickly, as well as knowing when to interject to make sure you do not miss anything.

A great example of this working is when someone has stated something, and although I cannot remember all points, I know that there was a question and three ideas offered. I might pause the conversation before it goes too far and say something like “Ah, that was important and I need to get this right. I heard you say three things there. You questioned the idea of X, and you offered an answer with 3 pros. One was Y, and what were the other two again?”

Conklin explains meeting discourse and question types in his book and the aforementioned workshop in a lot of detail. But once again, to apply it to a real-world situation can only be done by practice (and more practice). The absolute best way to do this is to watch an experienced dialogue mapper perform this and look at how they handle the situation, which brings me onto lesson six.

Lesson Six: Observe

One of the most enjoyable training experiences of my career was to travel to the picturesque town of Annapolis and learn dialogue mapping from Jeff Conklin himself. Up until then, I had been practicing the craft, but after those two days, I returned as a much better practitioner.

The single most important part of the time spent there was when we, the students, dialogue mapped each other as we discussed a real-world issue. We each sat in the hot seat for fifteen minutes or so, trying to map the discussion. We were all being completely evil, deliberately starting side conversations, interrupting and interjecting, playing the role of the dominant type A style participant, jumping all over the place and, overall, just being as difficult as we could.

Unsurprisingly, we all sucked big time trying to dialogue map this. However, when Conklin took the floor, we then saw how exactly Dialogue Mapping was done and what twenty years of practice does. He effortlessly brushed off our attempts to trip him up by observing all of the lessons that I have thus far described, but also with some subtle tricks that we didn’t even notice until he told us afterwards.

After Conklin had wiped the floor with us, so to speak, we all got to have another crack at it, with the benefit of observation and hindsight. This time the difference was significant in our performance and the way that we each handled the “mob”.

Moral? The very best thing you can do is to be involved in a Dialogue Mapping session where it has been done well. Watch carefully what the mapper is doing and how they are conducting themselves and the process.

Lesson Seven: It will make you tired

If you think of all the various things that you have to do simultaneously (for examples listening, understanding, mapping, and managing the group) during Dialogue Mapping, it is amazing that anyone with a Y chromosome can manage it, given that men usually cannot multitask at all. (Case in point: If sport is on the radio and my wife is speaking to me, one of them has to be switched off…Sorry honey 🙂 ).

When you are first learning this craft and you are going to be up in front of a group, plan for only an hour or so. While you have to think quite consciously about things, it will exhaust you even more. Once things become more automatic, you can go for longer. From my own experience, the limit for Dialogue Mapping a big group on a really wicked topic would be about four hours at the absolute maximum. (Usually by then the participants also need to take a break and sleep on it anyway).

Some sessions can be intense, and you, as a mapper, need to be switched on for that entire time. You are listening carefully to every person speak because you are trying to form it into IBIS. So unlike everybody else who can sit there and look interested, yet be mentally switched off, you have to be interested by definition.

One thing about this is that, while you are in the zone, you don’t need coffee. When mapping, you have enough endorphins racing around your system to keep quite alert, but as soon as there is a break, you can find that you will feel quite tired at times, and a well timed coffee can be very handy (real coffee, of course – none of that instant junk!)

The one thing that compensates for what Dialogue Mapping can take out of you mentally, is that exhilarating experience when the group is really getting into the process and the positive feedback that you receive when a really well formed map has been developed.

Lesson Eight: Learn to love “transclusions” and CTRL+R

I thought that I would drop in a left-field lesson learned at this point that is still very important.

Trans-what? Don’t worry – I don’t know why they called it that either. Ted Nelson, who also coined the term “hypertext”, came up with the name but I think the day “transclusion” sprung to mind, he was having an off-day. I asked Conklin what it meant and he said “It’s technically accurate … an "inclusion" of material *across* (‘trans’) several documents”. When I whined about the geekiness of the name he added “There was a time when the word "hypertext" was a wacko term for geeks, you know”. Damn! He’s got me there.

My explanation that will suffice for now? Transclusions is a fancy way of describing the process of breaking your big map up into smaller, linked up sub-maps. I have also heard it referred to as chunking, and when maps get too big, this is a necessity. (For the hypertext nerds that is somewhat incomplete but suffices for this point).

The compendium software that I choose to use for this work also has a great feature in it. Your map is re-drawn automatically when you hold down the control key and press R. I am now in the habit that after entering a node or three, I redraw the map via this method to keep it all looking orderly. That way, if there has been a lot of dialogue captured, you do not end up with a messy, cluttered map that participants find hard to follow. Like lesson number three and four, stopping conversation while you refactor a messy map will cost you group momentum so ideally if you have gotten into the Control+R habit, all you have to do is a quick transclusion at an opportune time.

The best time to perform a map transclusion is when a thread of discussion has been exhausted and the group has moved onto a new idea or area in the map. A trick I learned from Anapolis was to sit up and say something like “Okay, let’s just pause for a minute.” (Holding up my hand), congratulate the group on the quality of what they have captured and then say something like “Let’s just put this stuff into its own pigeonhole, so we can now focus on X idea”.

Lesson Nine: Nurture the holding environment

Okay, so if there is to be one big serious lesson learned, it is this one. To make the point, I am going to quote Heifetz and Linsky from their excellent book Leadership on the Line. You can read a PDF press release here, specifically the section entitled “Control the temperature.”

Changing the status quo generates tension and produces heat by surfacing hidden conflicts and challenging organizational culture. It’s a deep and natural human impulse to seek order and calm, and organizations and communities can tolerate only so much distress before recoiling.

If you try to stimulate deep change, you have to control the temperature. There are really two tasks involved. The first is to raise the heat enough that people sit up, pay attention, and deal with the real threats and challenges facing them. Without some distress, there is no incentive for them to change anything. The second is to lower the temperature when necessary to reduce a counterproductive level of tension. Any community can only take so much pressure before it becomes either immobilized or spins out of control. The heat must stay within a tolerable range—not so high that people demand it be turned off completely, and not so low that they are lulled into inactivity.

Heifetz and Linsky talk about maintaining the “the productive range of distress” but I have heard many metaphors like this. Another I like is “creative abrasion”, coined by Leonard and Swap in their excellent book When Sparks Fly . Both are essentially talking about making the whole environment conducive to getting the best out of the participants.

The key takeaway is this: Each group is different and each situation is different. In the normal discourse of the meeting, there will be times where the group works together in almost perfect unison and times where one wrong word will destroy that balance and require the group to stop, reset things and move forward. This is not about IBIS either. The fact is that over time, a particular pattern will emerge in the decorum of the sessions, where conducting the sessions or approaching the mapping in a particular way, will work consistently well for the group.

Let me give you a classic example that was absolute genius on the part of my client who did exactly this. Before Dialogue Mapping for a group of concerned residents who were facing the prospect of significant change to the amenity of their homes, a bus was hired and the residents were taken to an area where a similar urban transformation had been made ten years before. We all walked around the area for an hour, soaking in the vibe, learning about the history of the area, how the area was redeveloped and how certain planning challenges were overcome.

This allowed the participants to get a real sense of the issues they needed to confront, and they felt it with all senses, sight, sound and tactile, rather than some cold, rather detached room with a projected map on the wall. Later, when I dialogue mapped the session after the bus tour, the group did a fantastic job and the quality of the rationale that was captured was much richer and faster, through that sensory immersion that took place before the mapping process began.

So just remember, Dialogue Mapping is a great holding environment in the sense that Heifetz and Linsky talk about. It is a wonderful “rich container”, as Conklin puts it, for fostering and maintaining creative abrasion. But as the bus ride example shows, there is a lot of things that you can combine with it to enhance the experience further.

More examples like this will be covered in part 4 of this series.

Thanks for reading

Paul Culmsee

www.sevensigma.com.au