Hi all

Just a quick note to mention that I’m off travelling again, this time swapping 39 degree Celsius summer weather of Perth for somewhere between –6 to 5 degrees of Canada. I’ll be spending a week in Canada running two classes – one public and one private. The first class is a public SharePoint Governance and Information Architecture class running in Vancouver. MVP Michal Pisarek of SharePointAnalystHQ fame will be there and it should be a terrific two days of learning how to think a little differently to govern SharePoint strategy and deployment. You will learn a bunch of new skills, techniques and perspectives. Best of all, the skills learnt are applicable for many other types of complex projects.

The class flyer is here: http://www.sevensigma.com.au/wp-content/uploads/downloads/2011/02/SPIA.pdf

The registration site is here: http://spiavancouver.eventbrite.com/

In terms of course coverage and content it is worth noting the research performed by the Eventful group (who run the Share conferences). According to them, the hot topic areas for SharePoint are governance, user adoption, change management, information architecture and user empowerment. These sort of topics are the sort where plenty of people tell you what the issues are, but are typically lighter on what to do about them. This class covers why this is, as well as dealing with all of these areas and presents detailed strategies, tools and methods to address them. Furthermore, aside from the 500+ page manual of meaty governance goodness, as a take home, we supply a CD for attendees with a sample performance framework, governance plan, SharePoint ROI calculator and sample mind maps of Information Architecture.

At last count there were 5 places left for the Vancouver class, so if you have been pondering if it is a worthwhile class, check out some of the feedback from the class web site. Also, if you know anybody who might be interested in attending, please pass the course flyer and registration site details to them. We always end up with people who tell us “Ah – if only I knew about the class!!”

Thanks for reading

Paul Culmsee

www.sevensigma.com.au

www.hereticsguidebooks.com

                      

No Tags

About bloody time eh?

The Heretics Guide to Best Practices is now available through Amazon, Barnes and Noble and iUniverse.

]

In Paul and Kailash I have found kindred spirits who understand how messed up most organizations are, and how urgent it is that organizations discover what Buddhists call ‘expedient means’—not more ‘best practices’ or better change management for the enterprise, but transparent methods and theories that are simple to learn and apply, and that foster organizational intelligence as a natural expression of individual intelligence. This book is a bold step forward on that path, and it has the wonderful quality, like a walk at dawn through a beautiful park, of presenting profound insights with humor, precision, and clarity.”

—Jeff Conklin, Director, Cognexus Institute

“Hugely enjoyable, deeply reflective, and intensely practical. This book is about weaving human artistry and improvisation, with appropriate methods and technologies, in order to pool collective intelligence and wisdom under pressure.”

—Simon Buckingham Shum, Knowledge Media Institute, The Open University, UK.

“This is a terrific piece of work: important, insightful, and very entertaining. Culmsee and Awati have produced a refreshing take on the problems that plague organisations, the problems that plague attempts to fix organisations, and what can be done to make things better. If you’re trying to deal with wicked problems in your organisation, then drop everything and read this book.”

—Tim Van Gelder, Principal Consultant, Austhink Consulting

“This book has been a brilliantly fun read. Paul and Kailash interweave forty years of management theory using entertaining and engaging personal stories. These guys know their stuff and demonstrate how it can be used via real world examples. As a long time blogger, lecturer and consultant/practitioner I have always been served well by contrarian approaches, and have sought stories and case studies to understand the reasons why my methods have worked. This book has helped me understand why I have been effective in dealing with complex business problems. Moreover, it has encouraged me to delve into the foundations of various management practices and thus further extend my professional skills.”

—Craig Brown, Director, Evaluator

                      

No Tags

For one night only USA…

Ah, Erica Toelle – what a legend! Thanks to Erica and Fpweb, I’m thrilled to confirm that the Seattle SharePoint Governance and Information Architecture class is all systems go. Save the date as its very likely indeed to be the only SPIA class in the USA in 2011.  If it wasn’t enough that Erica will be joining me, but Ruven Gotz will be there too.

Thursday and Friday, May 05-06, 2011. (http://spiaseattle.eventbrite.com/)

The location is the Silvercloud Inn, 14632 SE Eastgate Way Bellevue, WA 98004

In this multimedia extravaganza of a blog post, lets take a closer look at this class and what you can expect. Below is a snippet of a talk I did in New Zealand called “SharePoint Governance  Home Truths”. This clip shows a little diagnostic test that I do on my audience, to see whether they have experienced the visible signs of wicked problems. If you want to know why you should go to SPGov+IA, then take my 2 minute test yourself.

If the two minute test has taken your fancy, then you might want to see what is in store on the course itself. Below is the first half-hour of module 1 (in the form of a conference session), as well as the accompanying slide deck.

Course Information:

Download Course Outline (PDF)

Download Class Flyer (PDF)

Most people understand that deploying SharePoint is much more than getting it installed.  Despite this, current SharePoint governance documentation abounds in service delivery aspects. However, just because your system is rock-solid, stable, well-documented and governed through good process, there is absolutely no guarantee of success.  Similarly, if Information Architecture for SharePoint was as easy as putting together lists, libraries and metadata the right way, then why doesn’t Microsoft publish the obvious best practices?

In fact, the secret to a successful SharePoint project is an area that the governance documentation barely touches.

This Master Class pinpoints the critical success factors for SharePoint Governance and Information Architecture and rectifies this blind spot.  Paul Culmsee’s style takes an ironic and subversive view on how SharePoint Governance really works within organizations while presenting a model and the tools necessary to get it right.

Drawing on inspiration from many diverse sources, disciplines and case studies, Paul Culmsee has distilled the "what" and "how" of governance down to a simple and accessible, yet rigorous and comprehensive set of tools and methods that organizations, large and small, can utilize to achieve the level of commitment required to see SharePoint become a successful part of your enterprise.

Some workshop sessions are hands on, we provide all of the tools and samples needed but please bring your own laptop.

Course Structure:

The course is split into 7 modules, run across two days.

Module 1: SharePoint Governance f-Laws 1-17:

Module 1 is all about setting context in the form of clearing some misconceptions about the often muddy topic of SharePoint governance. This module sheds some light onto these less visible SharePoint governance factors in the form of Governance f-Laws, which will also help to provide the context for the rest of this course

• Why users don’t know what they want
• The danger of platitudes
• Why IT doesn’t get it
• The adaptive challenge – how to govern SharePoint for the hidden organisation
• The true forces of organisational chaos
• Wicked problems and how to spot them
• The myth of best practices and how to determine when a “practice” is really best

Module 2: The Shared Understanding Toolkit – part 1:

Module 2 pinpoints the SharePoint governance blind spot and introduces the Seven Sigma Shared Understanding Toolkit to counter it. The toolkit is a suite of tools, patterns and practices that can be used to improve SharePoint outcomes. This module builds upon the f-laws of module 1 and specifically examines the “what” and “why” questions of SharePoint Governance. Areas covered include how to identify particular types of problems, how to align the diverse goals of stakeholders, leverage problem structuring methods and constructing a solid business case.

Module 3: The Shared Understanding Toolkit – part 2:

Module 3 continues the Seven Sigma Shared Understanding Toolkit, and focuses on the foundation of “what” and “why” by examining the “who” and “how”. Areas covered include aligning stakeholder expectations, priorities and focus areas and building this alignment into a governance structure and written governance plan that actually make sense and that people will read. We round off by examining user engagement/stakeholder communication and training strategy.

Module 4: Information Architecture trends, lessons learned and key SharePoint challenges

Module 4 examines the hidden costs of poor information management practices, as well as some of the trends that are impacting on Information Architecture and the strategic direction of Microsoft as it develops the SharePoint road map. We will also examine the results from what other organisations have attempted and their lessons learned. We then distil those lessons learned into some the fundamental tenants of modern information architecture and finish off by examining the key SharePoint challenges from a technical, strategic and organisational viewpoint.

Module 5: Information organisation and facets of collaboration

Module 5 dives deeper into the core Information Architecture topics of information structure and organisation. We explore the various facets of enterprise collaboration and identify common Information Architecture mistakes and the strategies to avoid making them.

Module 6: Information Seeking, Search and metadata

Module 6 examines the factors that affect how users seek information and how they manifest in terms of patterns of use. Building upon the facets of collaboration of module 5, we examine several strategies to improving SharePoint search and navigation. We then turn our attention to taxonomy and metadata, and what SharePoint 2010 has to offer in terms of managed metadata

Module 7: Shared understanding and visual representation – documenting your Information Architecture

Module 7 returns to the theme of governance in the sense of communicating your information architecture through visual or written form. To achieve shared understanding among participants, we need to document our designs in various forms for various audiences.

Putting it all together: From vision to execution

Attendees will be taking home a manual ~480 pages, containing the Seven Sigma Shared Understanding Toolkit CD with a sample performance framework, governance plan, SharePoint ROI calculator (Spreadsheet), sample mind maps of Information Architecture. These tools are the result of years of continual development and refinement "out in the field" by Paul Culmsee and have only been recently released to the public through this Master Class.

More Information:

• Praise for SharePoint Governance and IA Masterclass

Refund Policy:

No refunds will be issued for attendee cancellations once payment is recieved.  Class cancellation by the organizer will result in a refund less transaction fees.

http://spiaseattle.eventbrite.com/

                      

No Tags

Monte Carlo simulations are cool – very cool. in this post I am going to try and out-do Kailash Awati in trying to explain what they are. You see, I am one of these people who’s eyes glaze over the minute you show me any form of algebra. Kailash recent wrote a post to explain Monte Carlo to the masses, but he went and used a mathematical formula (he couldn’t help himself), and thereby lost me totally. Mind you, he used the example of a drunk person playing darts. This I did like a lot and gave me the inspiration for this post.

So here is my attempt to explain what Monte Carlo is all about and why it is so useful.

I have previously stated, that vaguely right is better than precisely wrong. If someone asks me to make an estimate on something, I offer them a ranged estimate, based on my level of certainty. Thus for example, if you asked me to guess how many beers per day Charlie Sheen has been knocking back lately, I might offer you an estimate of somewhere between 20 and 50 pints. I am not sure of the exact number (and besides, it would vary on a daily basis anyway) , so I would rather give you a range that I feel relatively confident with, than a single answer that is likely to be completely off base.

Similarly, if you asked me how much a SharePoint project to “improve collaboration” would cost, I would do a similar thing. The difference between SharePoint success and Charlie Sheen’s ability to keep a TV show afloat is that with SharePoint, there are more variables to consider. For example, I would have to make ranged estimates for the cost of:

• Hardware and licensing
• Solution envisioning and business analysis
• Application development
• Implementation
• Training and user engagement

Now here is the problem. A CFO or similar cheque signer wants certainty. Thus, if you give them a list of ranged estimates, they are not going to be overly happy about it. For a start, any return on investment analysis is by definition, going to have to pick a single value from each of your estimates to “run the numbers”. Therefore if we used the lower estimate (and therefore lower cost) for each variable, we would inevitably get a great return on investment. If we used the upper limit to each range, we are going to get a much costlier project.

So how to we reconcile this level of uncertainty?

Easy! Simply run the numbers lots and lots (and lots) of times – say, 100,000 times, picking random values from each variable that goes into the estimate. Count the number of times that your simulation is a positive ROI compared to a negative one. Blammo – that’s Monte Carlo in a nutshell. It is worth noting that in my example, we are assuming that all values between the upper and lower limits are equally likely. Technically this is called a uniform distribution – but we will get to the distribution thing in a minute.

As a very crappy, yet simple example, imagine that if SharePoint costs over $250,000 it will be considered a failure. Below are our ranged estimates for the main cost areas:

If you add up my lower estimates we get a total of $140,000 – well within our $250,000 limit. However if my upper estimates turn out to be true we blow out to $435,000 – ouch!

So why don’t we pick a random value from each item, add them up, and then repeat the exercise 100,000 times. Below I have shown 5 of 100,000 simulations.

So according to this basic simulation, only 2 out of 5 shown are below $250,000 and therefore a success according to my ROI criteria. Therefore we were successful only only 40% of the time (2/5 = .4). By that measure, this is a risky project (and we haven’t taken into account discounting for risk either).

“Thats it?”, I hear you say? Essentially yes. All we are doing is running the numbers over and over again and then looking at the patterns that emerge from this. But that is not the key bit to understand. Instead, the most important thing to understand with Monte Carlo properly is to understand probability distributions. This is the bit that people mess up on and the bit that people are far too quick to jump into mathematical formulae.

But random is not necessarily random

Let’s use Charlie Sheen again to understand probability distributions. If we were to consider the amount of crack he smokes on a daily basis, we could conclude it is between 0 grams  and 120 grams. The 120g upper limit is based on what Charlie Sheen could realistically tolerate (which is probably three times the amount of normal humans). If we plotted this over time, it might look like the example below (which is the last 31 days):

So to make a best guess at the amount he smokes tomorrow, should we pick random values between 0 and 120 grams?  I would say not. Based on observing the chart above, you would be likely to choose values from the upper end of the range scale (lately he has really been hitting things hard and we all know what happens when he hangs with starlets from the adult entertainment industry).

That’s the trick to understanding a probability distribution. If we simply chose a random value it would likely not be representative of the recent range of values. We still have to pick a value from a range of possibilities, but some values are more likely than others. We are not truly picking random values at all.

The most common probability distribution people use is the old bell curve – you probably saw it in high school. For many variables that go into a monte carlo, it is a perfectly fine distribution. For example, the average height of a human male may be 5 foot 6. Some people will be larger and some will be smaller, but you would find that there would be more people closer to this mid-point than far away from it, hence the bell shape.

Let’s see what Charlie Sheen’s distribution looks like. Since we have our range of values, for each days amount of crack usage, let’s divide up crack usage into ranges of grams and see how much Charlie has consumed. The figure is below:

As you can see, according to the 50% of the time Charlie was not hitting the white stuff particularly hard. There 16 occurrences where Charlie ingested less than 10 grams. What sort of curve does this make? The picture below illustrates it.

Interesting huh? If we chose random numbers according to this probability distribution, chances are that 50% of the time, we would get a value between 0 and 10 grams of crack being smoked or shovelled up his nasal passages. Yet when we look at the trend of the last 10 days, one could reasonably expect that its likely that tomorrows value would be significantly higher than zero. In fact there were no occurrences at all of less than 10 grams in a single day in the last 10 days.

Now let’s change the date range, and instead look at Charlie’s last 9 days of crack usage. This time the distribution looks a bit more realistic based on recent trends. Since he has not been well behaved lately, there were no days at all where his crack usage was less than 10 grams. In fact 4 of the 9 occurrences were over 60 grams.

This time, utilising a different set of reference points (9 days instead of 31), we get very different “randomness”. This gets to one of the big problems with probability distributions which Kailash tells me is called the Reference class problem. How can you pick a representative sample? In some situations completely random might actually be much better than a poorly chosen distribution.

Back to SharePoint…

So imagine that you have been asked to estimate SharePoint costs and you only have vague, ranged estimates. Lets also assume that for each of the variables that need to be assigned an estimate, you have some idea of their distribution. For example if you decide that SharePoint hardware and licensing really could be utterly random between $50000-$60000 then pick a truly random value (a uniform distribution) from the range with each iteration of the simulation. But if you decide that its much more likely to come in at $55000 than it is $50000, then your “random” choice will be closer to the middle of the range more often than not – a normal distribution.

So the moral of the story? Think about the sort of distribution that each variable uses. It’s not always a bell curve. its also not completely random either. In fact you should strive for a distribution that is the closest representation of reality. Kailash tells me that a distribution “should be determined empirically – from real data – not fitted to some mathematically convenient fiction (such as the Normal or Unform distributions). Further, one should be absolutely certain that the data is representative of the situation that is being estimated.”

Since SharePoint often relies on some estimations that offer significant uncertainty, a Monte Carlo simulation is a good way to run the numbers – especially if you want to see how many variables with different probability distributions combine to produce a result. Run the simulation enough times, you will produce a new probability distribution that represents all of these variables.

Just remember though – Charlie Sheen reliably demonstrates that things are not often predictable and that past values are no reliable indicator of future values. Thus a simulation is only as good as your probability distributions in the first place

Thanks for reading

Paul Culmsee

www.sevensigma.com.au

p.s A huge thanks to Kailash for checking this post, offering some suggestions and making sure I didn’t make an arse of myself.

                      

Technorati Tags: Charlie Sheen, estimating, monte Carlo, ROI

Hi All

Here is my simple, patent pending method to use to help users design good SharePoint sites. It combines two very effective IA methods into one and its amazing how it turns people from wanting 1990’s era sites complete with horizontal scrolling banners with animated GIF’s into usability and IA gurus within minutes.

The tools of the trade you need for this method is:

• Balsamiq, a terrific wireframing tool that I previously reviewed.
• http://www.websitesthatsuck.com/ – a hilarious site dedicated to teaching people how to design sites by looking at particularly crap ones

So now you know the ingredients, let’s run through the recipe

1. Put key stakeholders into a room (ensure the ones with poor taste are there together)
2. Visit websitesthatsuck.com and review the 2010 contenders for worst websites of the year. (For what its worth, my personal vote is Yale School of Art)
3. Have a good laugh and discuss all the crappy aspects to those sites – make particular note of the write-up on websitesthatsuck for each contender
4. With the group’s sucky website radar now primed, have them load up their existing intranet (if they are really big organisation, go around to various departmental sites around the intranet). This time they will not laugh, due to the effect of your “crapness calibration” ™ exercise, they will see many faults in the existing site straight away.
5. At this point, crank out Balsamiq and start to wireframe what the site should look like while you have the fleeting moment of clarity (crapness calibration fades with time and needs to be re-primed). The wisdom of the crowd should ensure that most of the common mistakes will be avoided there and then.
   • Statistically, one of every three times you do this, there is always one user who’s taste is so bad that calibration will take another round of deprogramming. So if you have someone that persists with crap taste or has ideas that 99% of the user base would balk at, move to the 2009 hall of shame for sucky sites. Faced with the reaction from their peers, as well as the parallels that can be drawn between their current site and the contenders, it usually does the trick.
   • Also be sure to draw attention to sites that have similar underlying concepts, but where one works well and the other has agonising lameness. For example, the New York Times compared to Havenworks. Discuss the layout, colours, fonts, images, navigation, search and the like and relate back to the site being envisioned.

In about 30-90 minutes, one of two things will happen.

1. You will have a pretty good wireframe or three
2. The group will realise that they have more soul searching to do.

Although your business development manager will whine at you if outcome 2 happens, consider it a good thing. You will be saving yourself and the participants a mountain of stress later and have them thinking more holistically about the outcomes they are trying to achieve.

(Final serious bit at the end alert)

What you will notice when performing this process, is that with a recent and clear frame of reference, some of the biases that people carry with them can be temporarily lifted. In some ways, this exercise is very similar to the “down the pub” calibration of estimates exercise that I wrote about previously. The trick is to find ways to change the lens people look through to see other aspects or facets to the problem at hand.

To that end, if you are in the UK or nearby, consider coming to my Governance and Information Architecture Master Class in London with Andrew Woodward and Ant Clay. Lots of other (more serious and rigorous) methods for developing shared understanding will be covered.

Thanks for reading

Paul Culmsee

www.sevensigma.com.au

                      

No Tags

As you may have noticed, this blog has been a bit of a dead zone lately. There are several very good reasons for this – one being that a lot of my creative energy has been going into co-writing a book – and I thought it was time to come clean on it.

So first up, just because I get asked this all the time, the book is definitely *not* “A humble tribute to the leave form – The Book”! In fact, it’s not about SharePoint per se, but rather the deeper dark arts of team collaboration in the face of really complex or novel problems.

It was late 2006 when my own career journey took an interesting trajectory, as I started getting into sensemaking and acquiring the skills necessary to help groups deal with really complex, wicked problems. My original intent was to reduce the chances of SharePoint project failure but in learning these skills, now find myself performing facilitation, goal alignment and sensemaking in areas miles away from IT. In the process I have been involved with projects of considerable complexity and uniqueness that make IT look pretty easy by comparison. The other fringe benefit is being able to sit in a room and listen to the wisdom of some top experts in their chosen disciplines as they work together.

Through this work and the professional and personal learning that came with it, I now have some really good case studies that use unique (and I mean, unique) approaches to tackling complex problems. I have a keen desire to showcase these and explain why our approaches worked.

My leanings towards sensemaking and strategic issues would be apparent to regular readers of CleverWorkarounds. It is therefore no secret that this blog is not really much of a technical SharePoint blog these days. The articles on branding, ROI, and capacity planning were written in 2007, just before the mega explosion of interest in SharePoint. This time around, there are legions of excellent bloggers who are doing a tremendous job on giving readers a leg-up onto this new beast known as SharePoint 2010.

So back to the book. Our tentative title is “Beyond Best Practices” and it’s an ambitious project, co-authored with Kailash Awati – the man behind the brilliant eight to late blog. I had been a fan of Kailash’s work for a long time now, and was always impressed at the depth of research and effort that he put into his writing. Kailash is a scarily smart guy with two PHD’s under his belt and to this day, I do not think I have ever mentioned a paper or author to him that he hasn’t read already. In fact, usually he has read it, checked out the citations and tells me to go and read three more books!

Kailash writes with the sort of rigour that I aspire to and will never achieve, thus when the opportunity of working with him on a book came up, I knew that I absolutely had to do it and that it would be a significant undertaking indeed.

To the left is a mock-up picture to try and convey where we are going with this book. See the guy on the right? Is he scratching his head in confusion, saluting or both? (note, this is our mockup and the real thing may look nothing like this)

This book dives into the seedy underbelly of organisational problem solving, and does so in a way that no other book has thus far attempted. We examine why the very notion of “best practices” often makes no sense and have such a high propensity to go wrong. We challenge some mainstream ideas by shining light on some obscure, but highly topical and interesting research that some may consider radical or heretical. To counter the somewhat dry nature of some of this research (the topics are really interesting but the style in which academics write can put insomniacs to sleep), we give it a bit of the cleverworkarounds style treatment and are writing in a conversational style that loses none of the rigour, but won’t have you nodding off on page 2. If you liked my posts where I use odd metaphors like boy bands to explain SharePoint site collections, the Simpsons to explain InfoPath or death metal to explain records versus collaborative document management, then you should enjoy our journey through the world of cognitive science, memetics, scientific management and Willy Wonka (yup – Willy Wonka!).

Rather than just bleat about what the problems with best-practices are, we will also tell you what you can do to address these issues. We back up this advice by presenting a series of practical case studies, each of which illustrates the techniques used to address the inadequacies of best practices in dealing with wicked problems. In the end, we hope to arm our readers with a bunch of tools and approaches that actually work when dealing with complex issues. Some of these case studies are world unique and I am very proud of them.

Now at this point in the writing, this is not just an idea with an outline and a catchy title. We have been at this for about six months, and the results thus far (some 60-70,000 words) have been very, very exciting. Initially, we really had no idea whether the combination of our writing styles would work – whether we could take the degree of depth and skill of Kailash with my low-brow humour and my quest for cheap laughs (I am just as likely to use a fart joke if it helps me get a key point across)…

… But signs so far are good so stay tuned

Thanks for reading

Paul Culmsee

www.sevensigma.com.au

                      

Technorati Tags: Book, Beyond Best Practices

…and we’re back!

Well… that was a long commercial break wasn’t it

In case you missed part 1 of our version of the show “deal or no deal”, you missed the big cliff-hanger and you really should read part 1 first. For the rest of you, to quickly recap, I came out of the closet and admitted by secret teenybopper shame, told the world that my wife had a teenage thing for Jean Claude Van Damme, showed the effect of beer goggles and introduced the notion of cognitive bias and how it can affect judgement.

i also demonstrated how, by altering the frame of reference, to a problem something that at first seems completely unquantifiable “how the hell do I know how many SharePoint developers drive yellow cars?”, is actually not as “impossible” as you may first think.

At the end of the last post I left you with a $10000 dilemma. You had to make a “deal or no deal” decision about going with your estimate about SharePoint developers who own yellow cars, or to instead cast your lot with a bag of marbles with a 9 in 10 chance of winning the prize. Just to refresh your memory, here is the salient part of the pub conversation.

• Me: Okay, so you are 90% sure that here are between 300 and 2000 SharePoint developers in the world with a yellow car?
• Them: Yes
• Me: So, let’s make this like the game show “deal or no deal”. If you are right and the answer is within your range, you will win $10000. BUT you have an alternative…
• Them: Ok…
• Me: What if I were to present you with a bag containing 9 red marbles and 1 black marble and offer you $10000 if you pull out a red marble. Pull the one black marble, and you miss out on the money. Do you want to stick to your estimate or do you want to draw a marble?

So have you decided? Now be honest and see how you went against the 4 outcomes that I have experienced when trying this on people. Here are the possible answers in order of popularity…

1. The person chooses to pull from the bag of marbles rather than their ranged estimate. (This is the predominant answer for all people I have tried this with – perhaps 70-75% of all responses).
2. The person chooses to use their estimate over the bag of marbles. (perhaps 25% of people have answered with this option)
3. Upon hearing the bag option, the person wants to change their ranged estimate. (Happened to me once)
4. The person doesn’t care which method.. (never happened to me)

So which is the right answer to this question?

(drumroll) Lets tackle the possible answer in order of likelihood.

“Take the marble! take the maaaaaarble!”

For the 70 odd percent of you who opted to take your chances with the bag of marbles, GONG! you lose!

Better double check your estimates in future because you have demonstrated that you are over-confident in your estimates. In other words, you are suffering from optimism bias. To explain why, think about the original question carefully. I asked originally for a ranged estimate that you were 90% confident with.

I then presented an alternative that has a 9 out of 10 chance of success – also 90%. From a statistical point of view, you should be completely ambivalent as to which option to use. Therefore, despite being asked for a range that you were 90% confident with, the range you actually estimated is not really 90% at all. It has to be less than 90% for you to prefer a clear 9/10 probability.

So that is why you are so stressed and busy! You keep giving crap estimates that make life harder for you! Either that or you are too nice and when your project manager looks at you with those big, sad project manager eyes, your heart melts and you relent.

Isn’t that cool in a nerdy way? It is very interesting to see people’s faces as the penny drops to this logic and they suddenly realise just how bad some of their past estimates have been as a result. The consolation prize is just about 4 out of 5 people do exactly the same as you and take the marbles.

“No deal, I will stick with my estimate”

For the smaller group who decide that their estimate is preferred, you also lose.

In this case, the reason why should be pretty obvious. You are so paranoid about getting it wrong, that you have made an estimate that is more like 95% or even 99% confident. Why? your range is too wide for 90% because when presented with a clear 9/10 chance of success, you chose your original estimate. While that may sound like you are confident, in reality you are a bit of a wuss, because in fact you are under confident with your estimate. So grow some balls you weenie

Honorary mention – “I want to change my estimate”

At the Best Practice Conference in DC, I attempted this pub trick on Yoav Lurie from Synteractive, who is much more of a business and strategic thinker than us IT nerds. His response I think, deserves an honorary mention for being the closest to winning the game. In this example, I asked him to estimate in feet, the wingspan of a Boeing 747. I knew instantly that he was a good estimator because of the logic he used to come to a range.

“Hmm, well an aircraft seat is maybe one and a half feet, and there will be 10 seats in the cabin, with two passages that are probably two feet in width…so that ads up to…”

What do you notice about what Yoav did? Straight away, he related the wingspan of an aircraft (a clear unknown), to something he could make a reasonable estimate of (the width of an aircraft seat). After all, we have all sat in an aircraft seat in sardine (economy) class and know how cramped it is. He knew there were three rows of seats and related this to the width of the cabin, which he then related to the size of the wing. Deducing that the wing might be 4 to 6 times the width of the cabin, he then was able to make a very good ranged estimate of the overall wingspan of the plane.

I was very impressed at his estimate and how he arrived at it, but I still got him

As soon as I presented him with the bag of marbles alternative, without missing a beat he said “I want to change my estimate”. It took only a split second of presenting a clear 90% probability made Yoav realise that his estimate was not 90% and he was still a little overconfident.

That being said, Yoav’s method of relating something known to help frame the reference to something unknown is the only time anyone has used any sort of rigour in forming an estimate and very impressive for the pub setting

The right answer

Okay, so as you may have guessed by now, the right answer is to shrug your shoulders and say “I don’t care” or wave your hand at me and say “pfft, whatever”. (This is one of the few times saying you couldn’t care less is the right answer). In doing so, you have placed equal weight upon the choices, based on the assumption that both are 90% probabilities.

Neat pub trick huh? It certainly gets people thinking.

How to calibrate yourself

Douglas Hubbard talks about “calibrated estimates” in his books and has an appendix of calibration questions, that are designed to help you perceive and account for cognitive bias in your estimating.

What you should take away from this exercise is that when asked to estimate on something you are uncertain about, make your initial estimate. Then, pretend you are in the game show and you have to pick between this estimate and the marble. If you feel that you would take the marble over your estimate, increase the width of your range until you feel that it doesn’t matter which option you pick.

Conversely, if you are one of the wimps who are under confident, then reduce the width of your range, until you feel that you have no particular preference of your estimate vs. the marbles.

In the same way that reframing a problem led from something being unquantifiable to something that indeed had a upper and lower range, by reframing the estimate against a unambiguous probability such as a bag of 10 marbles with 9 red, helps you to account for cognitive bias in your estimates.

Conclusion

So to reiterate my key points to this post

1. Many things that seem unquantifiable are easier to quantify than you think, once you think in terms of ranged estimates and probability.
2. Your bad taste in fashion and music when you were a teenager still manifests itself today and it is called cognitive bias.
3. There are easy methods that you can use to calibrate yourself better so that your estimating radar is more finely tuned.

Most importantly of all however, you learned that my wife liked Jean Claude Van Damme in the 80’s and you know that I am in big trouble when she reads this!

Thanks for reading

Paul

                      

No Tags