Back to Cleverworkarounds mainpage
 

A simple way to improve your estimating (and a cool pub trick) – Conclusion

…and we’re back!

Well… that was a long commercial break wasn’t it 🙂

In case you missed part 1 of our version of the show “deal or no deal”, you missed the big cliff-hanger and you really should read part 1 first. For the rest of you, to quickly recap, I came out of the closet and admitted by secret teenybopper shame, told the world that my wife had a teenage thing for Jean Claude Van Damme, showed the effect of beer goggles and introduced the notion of cognitive bias and how it can affect judgement.

i also demonstrated how, by altering the frame of reference, to a problem something that at first seems completely unquantifiable “how the hell do I know how many SharePoint developers drive yellow cars?”, is actually not as “impossible” as you may first think.

At the end of the last post I left you with a $10000 dilemma. You had to make a “deal or no deal” decision about going with your estimate about SharePoint developers who own yellow cars, or to instead cast your lot with a bag of marbles with a 9 in 10 chance of winning the prize. Just to refresh your memory, here is the salient part of the pub conversation.

  • Me: Okay, so you are 90% sure that here are between 300 and 2000 SharePoint developers in the world with a yellow car?
  • Them: Yes
  • Me: So, let’s make this like the game show “deal or no deal”. If you are right and the answer is within your range, you will win $10000. BUT you have an alternative…
  • Them: Ok…
  • Me: What if I were to present you with a bag containing 9 red marbles and 1 black marble and offer you $10000 if you pull out a red marble. Pull the one black marble, and you miss out on the money. Do you want to stick to your estimate or do you want to draw a marble?

So have you decided? Now be honest and see how you went against the 4 outcomes that I have experienced when trying this on people. Here are the possible answers in order of popularity…

  1. The person chooses to pull from the bag of marbles rather than their ranged estimate. (This is the predominant answer for all people I have tried this with – perhaps 70-75% of all responses).
  2. The person chooses to use their estimate over the bag of marbles. (perhaps 25% of people have answered with this option)
  3. Upon hearing the bag option, the person wants to change their ranged estimate. (Happened to me once)
  4. The person doesn’t care which method.. (never happened to me)

So which is the right answer to this question?

(drumroll) Lets tackle the possible answer in order of likelihood.

“Take the marble! take the maaaaaarble!”

For the 70 odd percent of you who opted to take your chances with the bag of marbles, GONG! you lose!

Better double check your estimates in future because you have demonstrated that you are over-confident in your estimates. In other words, you are suffering from optimism bias. To explain why, think about the original question carefully. I asked originally for a ranged estimate that you were 90% confident with.

I then presented an alternative that has a 9 out of 10 chance of success – also 90%. From a statistical point of view, you should be completely ambivalent as to which option to use. Therefore, despite being asked for a range that you were 90% confident with, the range you actually estimated is not really 90% at all. It has to be less than 90% for you to prefer a clear 9/10 probability.

So that is why you are so stressed and busy! You keep giving crap estimates that make life harder for you! 🙂 Either that or you are too nice and when your project manager looks at you with those big, sad project manager eyes, your heart melts and you relent.

Isn’t that cool in a nerdy way? It is very interesting to see people’s faces as the penny drops to this logic and they suddenly realise just how bad some of their past estimates have been as a result. The consolation prize is just about 4 out of 5 people do exactly the same as you and take the marbles.

“No deal, I will stick with my estimate”

For the smaller group who decide that their estimate is preferred, you also lose.

In this case, the reason why should be pretty obvious. You are so paranoid about getting it wrong, that you have made an estimate that is more like 95% or even 99% confident. Why? your range is too wide for 90% because when presented with a clear 9/10 chance of success, you chose your original estimate. While that may sound like you are confident, in reality you are a bit of a wuss, because in fact you are under confident with your estimate. So grow some balls you weenie 🙂

Honorary mention – “I want to change my estimate”

At the Best Practice Conference in DC, I attempted this pub trick on Yoav Lurie from Synteractive, who is much more of a business and strategic thinker than us IT nerds. His response I think, deserves an honorary mention for being the closest to winning the game. In this example, I asked him to estimate in feet, the wingspan of a Boeing 747. I knew instantly that he was a good estimator because of the logic he used to come to a range.

“Hmm, well an aircraft seat is maybe one and a half feet, and there will be 10 seats in the cabin, with two passages that are probably two feet in width…so that ads up to…”

What do you notice about what Yoav did? Straight away, he related the wingspan of an aircraft (a clear unknown), to something he could make a reasonable estimate of (the width of an aircraft seat). After all, we have all sat in an aircraft seat in sardine (economy) class and know how cramped it is. He knew there were three rows of seats and related this to the width of the cabin, which he then related to the size of the wing. Deducing that the wing might be 4 to 6 times the width of the cabin, he then was able to make a very good ranged estimate of the overall wingspan of the plane.

I was very impressed at his estimate and how he arrived at it, but I still got him 🙂

As soon as I presented him with the bag of marbles alternative, without missing a beat he said “I want to change my estimate”. It took only a split second of presenting a clear 90% probability made Yoav realise that his estimate was not 90% and he was still a little overconfident.

That being said, Yoav’s method of relating something known to help frame the reference to something unknown is the only time anyone has used any sort of rigour in forming an estimate and very impressive for the pub setting 🙂

The right answer

Okay, so as you may have guessed by now, the right answer is to shrug your shoulders and say “I don’t care” or wave your hand at me and say “pfft, whatever”. (This is one of the few times saying you couldn’t care less is the right answer). In doing so, you have placed equal weight upon the choices, based on the assumption that both are 90% probabilities.

Neat pub trick huh? It certainly gets people thinking.

How to calibrate yourself

Douglas Hubbard talks about “calibrated estimates” in his books and has an appendix of calibration questions, that are designed to help you perceive and account for cognitive bias in your estimating.

What you should take away from this exercise is that when asked to estimate on something you are uncertain about, make your initial estimate. Then, pretend you are in the game show and you have to pick between this estimate and the marble. If you feel that you would take the marble over your estimate, increase the width of your range until you feel that it doesn’t matter which option you pick.

Conversely, if you are one of the wimps who are under confident, then reduce the width of your range, until you feel that you have no particular preference of your estimate vs. the marbles.

In the same way that reframing a problem led from something being unquantifiable to something that indeed had a upper and lower range, by reframing the estimate against a unambiguous probability such as a bag of 10 marbles with 9 red, helps you to account for cognitive bias in your estimates.

Conclusion

So to reiterate my key points to this post

  1. Many things that seem unquantifiable are easier to quantify than you think, once you think in terms of ranged estimates and probability.
  2. Your bad taste in fashion and music when you were a teenager still manifests itself today and it is called cognitive bias.
  3. There are easy methods that you can use to calibrate yourself better so that your estimating radar is more finely tuned.

Most importantly of all however, you learned that my wife liked Jean Claude Van Damme in the 80’s and you know that I am in big trouble when she reads this! 😛

Thanks for reading

Paul



A simple way to improve your estimating (and a cool pub trick) – Part 1

Okay I’ll admit it, I used to really suck as a time and effort estimator. I happen to have a business partner who is much better at it than me (hey Peter), and every time I sought a second opinion from him on my estimates, he would almost always make a much less optimistic assessment then me. Of course, Peter was almost always right too, dammit.

So, why was Peter much more accurate with his estimates?

The answer to this question, all one has to do is think back to their teenage years, where they went through that awkward stage where you look back and cringe at the posters that were on your wall and your choice of fashion. For many, this period demonstrates some utterly appalling choices of taste. Mine are particularly cringe-worthy, given that these days I am a bit of a metalhead. My favourite song at the time was Respectable by Mel and Kim. I thought that Karate Kid II was the best film of all time (and that the girl in it was hot). Mind you, my wife has an even more shameful secret. She had a crush on Jean Claude Van Damme! Mwahahahah 🙂

These are examples of a phenomenon I like to call “Teenybopper bias” 🙂

Now, there is a point in telling you about my wife’s secret shame and it isn’t to see her reaction when she reads this (okay, well maybe a teenie bit). These examples of “what the hell was I thinking” are a form of cognitive bias that took place at the time the opinion was formed. In terms of teenybopper bias, the root of the bias is likely the same hormones that caused your face to break out with acne and hair to grow in funny places. Another very common cognitive bias that afflicts people whether young or old is good old “beer goggle” bias illustrated below.

There are many, many forms of cognitive bias documented, such as optimism bias, anchoring, hindsight bias and the recency effect to name a few. Now let’s take the final image above and pretend we asked someone at the pub for an estimate on a project at 8pm, 10pm and 1am. I’d be willing to bet that the estimate gets more optimistic on a par with how optimistic the perception of the people in the image above become.

Overcoming cognitive bias

Kailash writes about the risk that cognitive bias can play in project failure, particularly in the perception of risks.

overcoming biases requires an understanding of the thought processes through which humans make decisions in the face of uncertainty.  Of particular interest is  the role of  intuition and rational thought in forming judgements, and the common mechanisms that underlie judgement-related cognitive biases.   A knowledge and awareness of these mechanisms  might help project managers in consciously countering the operation of cognitive biases in their own decision making.

The essential difference between Peter and myself in our estimating, is that Peter happens to have a much more finely tuned radar to optimism bias in particular. Douglas Hubbard of Applied Information Economics fame, writes about the effect of cognitive bias extensively in his two books and offers a simple, yet highly useful method to quickly improve the quality of estimates which I will explain with an example below.

The great thing about learning about your cognitive biases and the methods for mitigating them, is that you can use it in the pub too. While I don’t recommend this method for picking up members of the opposite sex, it’s a pretty cool icebreaker.

Thus, I will demonstrate how to improve your estimating accuracy by a mythical pub conversation. Imagine you are onto your third beer…

  • Me: “How many SharePoint developers worldwide own a yellow car?
  • Them: “What the…I haven’t the faintest idea!”
  • Me: “Well, I can understand that, so let’s do an estimate. Give me a range that the answer could fall in, that you are 90% confident with.”
  • Them: “I still can’t give you an estimate, I can’t possibly know something like that.”
  • Me: “Well, could there be a million SharePoint developers who like yellow cars?”
  • “Them: “Don’t be ridiculous, there would be nowhere near a million SharePoint developers – period.”
  • Me: “So you do have an upper bound then, less than a million. Remember this is not about the exact answer, I want a range that you would be 90% confident with.”
  • Them: “Okay I get it. I think it is somewhere between three hundred and two thousand.

Note that at this point, we have already made the initial breakthrough. At first the person found it impossible to make an estimate, yet when I related it to something they did have a fair idea of (the thought of a million people), they made some mental associations and realised they did have some idea of limits after all. Thus, by presenting a better frame of reference that they could use to approach the problem, they were able to move from “I have no idea” to a wide range of possible values.

The width of the range reflects the uncertainty that someone has about the answer. The more the uncertainty, the wider the range. Some project mangers hate being given a ranged value because it really mucks up their task or project work breakdowns. As a result, they always want the “ball park” or something that is a single value. I completely understand why this happens, but what these people forget is that an estimation is uncertain by definition. The obvious way to express uncertainty is with a range of values! So asking someone for an estimation and then complaining that it is not accurate enough actually makes no sense. A manager might not like the “width” of the range, but you can’t force someone to reduce their uncertainty just because it doesn’t fit the plan. Unless you provide them with the means to reduce this uncertainty, you cannot and should not try and artificially reduce this range through pressure and coercion.

But despite my observation of the flawed logic of dealing with uncertainty in estimating, a ranged estimate alone is not enough yet. We still have not accounted for the sorts of cognitive bias that I described earlier in the article. So without further adieu, I present a simplified version of Hubbards ‘calibration’ techniques that account for bias. Let’s continue the bar conversation.

  • Me: Okay, so you are 90% sure that here are between 300 and 2000 SharePoint developers in the world with a yellow car?
  • Them: Yes
  • Me: So, let’s make this like the game show “deal or no deal”. If you are right and the answer is within your range, you will win $10000. BUT you have an alternative…
  • Them: Ok…
  • Me: What if I were to present you with a bag containing 9 red marbles and 1 black marble and offer you $10000 if you pull out a red marble. Pull the one black marble, and you miss out on the money. Do you want to stick to your estimate or do you want to draw a marble?

 

I’d like readers to think about this before continuing with this article. Make a ranged estimate of the number of SharePoint developers worldwide who drive a yellow car, and then decide whether you want to stick to your estimate or take your chances with the marbles.

 

(Cue game show music where you have 10 seconds to decide with a little ping sound at the end.)

The suspense is now killing you I am sure. Want to know the correct answer?

Find out after this short commercial break (game show speak for wait till part 2 of this series 🙂 )

 

Thanks for reading

 

Paul Culmsee

www.sevensigma.com.au



The practice of Dialogue Mapping – Part 4

Three weeks ago my plasma TV broke, freeing the family from the magic spell of hi-def television. My family took the loss in different ways. My four year old was devastated at the lack of Nintendo Wii, and constantly whined about being bored. My ten year old is a bookworm anyway, and continued to be one. I suddenly found mountains of time to write, churning out three Dialogue Mapping articles that I had been meaning to write for ages.

Today the repair man came and fixed the TV. I expect that the glow of the plasma screen will once again induce that zombie-like state, where my work-rate is dependant on what show happens to be on at the time (NCIS as I write this). Luckily, this is the last article of this particular series on Dialogue Mapping for now and I might have enough active brain cells to hang on long enough to squeeze this article out.

This article builds on the last section of part three that was entitled “Nurture the holding environment”. In that section, I introduced the concept of the “holding environment” and I offered a basic example (the bus trip that was conducted prior to the Dialogue Mapping session). This concept is so fundamental and important to the success of Dialogue Mapping and projects more broadly that I want to do it proper justice here in part four.

The paradox of individuality

Put a bunch of right-brained geeks in a room to solve a problem and you will probably find that they get on relatively well. Put a bunch of creative left-brained marketing people in a room to solve a problem and you might expect the same thing. The solutions offered, when compared to each other, are likely to be quite different and will also likely be sub-optimal. For a truly good solution, we need diversity in perspectives and, although this pains me to say, marketing people are therefore actually needed. This creates a bit of a problem though with the paradox of individuality because, as geeks, we all know the notion of marketing people being needed goes against everything we stand for.

I first read about the paradox of individuality in a book called Team Talk: The Power of Language in Team Dynamics and it was described as follows:

The only way for a group to become a group is for individuals to express their individuality, yet the only way for individuals is to become fully individuated is to accept and develop more fully, their connections to the group.

What? Geeks and marketing people accepting each other as equals? Unifying the laws of physics will come sooner and this is a classic example of what Conklin calls “social complexity”. Of course, social complexity goes much deeper than geeks vs. marketing people, but one of the effects of social complexity is a distinct lack of direct communication between parties. This is because conflict is not fun and avoidance is a natural reaction to situations that are not fun.

The idea of the “holding environment” is best summed up with the image below. Here, you can see that we have an area set aside for kids to play in a safe, controlled environment.

image

A holding environment for an organisation or a team is actually not that dissimilar to the example above. You are attempting to create a state where participants can step out of their comfort zones, but at the same time, are shielded from counter productive tensions that cause paralysis, chaos and pullback.

For this reason I maintain that beer is one of the best holding environments available and it forms a key part of my professional skill set 😉

As I stated in part 1 of this series, Dialogue Mapping is a very useful holding environment on its own, but it can be augmented with other things as well and you should always be on the lookout for complimentary tools and techniques. In the following sections, I will outline where Dialogue Mapping has augmented another method, or where we have augmented Dialogue Mapping itself with another method.

Information gathering for 40+ people

There are practical limits to how many people should be involved in a standard Dialogue Mapping session. Mind you, there are practical limits for how many people should attend a meeting too and that limit seems to be any more than one person :-).

By “standard”, I mean the sort of session illustrated below. The exact number that test the limits of Dialogue Mapping varies because it really depends on the wickedness of the problem being discussed and the past history of the group. For example, one of the teams I map for consists of around fifteen to eighteen members. They are working on a particularly wicked problem, yet I can work with this group alone quite easily. This is because over time, the group has worked out their decorum for the Dialogue Mapping sessions that work for all concerned. I also know everybody on a first name basis and some of the group have become personal friends of mine outside of work. In short, people are comfortable with each other and the process, and despite things getting heated people know that it is not personal. This is a simple, yet effective, example of a working holding environment.

image

A while back, my client invited representatives from academia, charities as well as various public sector government departments, to a half day workshop on the topic of social sustainability as part of a significant redevelopment project. There were approximately 40-45 attendees who were there for the first time. There was no way we would be able to cover off the required topics using a standard Dialogue Mapping set-up. With so many people, it would be hard for all attendees to have a say in the allotted time, let alone set up the room to handle that number of people for the process.

The way we got around this issue was to run a pre-workshop session among a much smaller group, to create a series of “seed maps” for each of the sub-areas of social sustainability. By the end of this process, we had around a dozen maps on various subtopics with a few questions, ideas, pros and cons. These maps were not complete at all, but that was not the aim. Instead they were well formed IBIS argumentations.

We then printed each of these maps out at large size. Initially each map was pinned to display boards, and when the attendees arrived they spent time wandering from map to map, examining the argumentation while mingling with other attendees. Below is a photo showing some of the seed maps prior to the attendees arriving.

image

Below is a diagram representing the table arrangement for the workshop. We started with a half hour overview and introduction as to the purpose of the workshop and why they had been invited. At this point, we removed four of the maps from the display boards above, and put a map on each table. We explained to the group that each table had a unique map on it and each map was on a particular topic. At this point attendees had the opportunity to move to a table where the topic was of most relevance or interest to them.

image

Each table contained copious amounts of marker pens. I then took the stage and explained the basics of IBIS grammar to the attendees and explained to them that we wanted them to start adding ideas to the existing maps. I did not belabour the grammar, nor did I expect them to suddenly know how to do IBIS properly, but what I made clear, was that I was going to walk from table to table and interrupt if I felt the additions to the map made no sense or were ambiguous in some way.

The group had just under an hour to work on each map and at the end of the hour, we removed the updated paper maps and replaced them with the next four from the display boards. The process was then repeated and I walked from table to table, asking for clarification or calling out implied questions on parts of the maps that made no sense to me. Interestingly, IBIS novices seemed to have little problems with the usage of ideas, pros and cons, but they would forget to make the underlying question explicit. I would ask them what was the question being answered by a particular idea and would write the question into the map and redraw the lines.

After the third iteration of this process we were done. The last half an hour was a “Where to from here?” session and an opportunity for the group to provide feedback to the organisers of the workshop.

After the workshop was completed, I took all of the updated paper maps and added the additional rationales into the seed maps in Compendium. The process was surprisingly quick because the majority of the additional argumentations that were added were actually pretty good IBIS form. I think that having existing argumentations on the seed maps made it easier for attendees to add rationales that looked similar to what was there already. It wasn’t perfect IBIS by any means, but it was not a difficult task for me to refactor the additional information without losing any of the intent behind the rationales.

For the record, additional workshops were conducted, but these reverted to standard Dialogue Mapping workshops with a subset of the attendees who had specialised skills and knowledge in the topic area. But what this particular process demonstrated was that with a little planning a single Dialogue Mapper could still manage to capture quality rationales from a very large group in a short space of time.

Dialogue Mapping with a facilitator

Dialogue mapping for a large group can be augmented with a facilitator and I have done this a few times. For a large group, this can be very helpful because the mapper can concentrate on capturing the dialogue and less on directing the meeting. Equally though, a facilitator can actually make the process more difficult. The key to a facilitator situation working is when the facilitator either knows IBIS or has been present in a number of Dialogue Mapping workshops and understands how the process works. This is because the facilitator is usually facing the group like the mapper, asking probing questions, directing the course of conversation and therefore is not looking at the map or listening in terms of IBIS translation of the dialogue. As a Dialogue Mapper, it is important for participants to verify what you have captured is correct, and if the facilitators are not following the map, they can easily get in the way of this verification process.

Facilitators can also get you into trouble at times because they can sometimes be conditioned to traditional meeting decorum where topics are allocated at particular times with an agenda that can preclude deeper exploration of a topic. Dialogue Mapping is a rich enough container to allow a group that deeper exploration, but this is not something that some facilitators are used to. One prime example that sticks out in my mind to this day was a workshop where we had a lot of options to explore. Conscious of the agenda, in an attempt to make the process more efficient, the facilitator asked the group whether any of the options had any “fatal flaws” that enabled that option to be quickly discounted. It soon became apparent (in a negative way) that one person’s “fatal flaw” was diametrically opposed to another person’s “fatal flaw”. This attempt to shortcut deliberations backfired badly and resulted in this line of question being completely abandoned.

This is a great example of the importance of nurturing the holding environment (lesson nine from part 3). After this “fatal flaws” episode, I deliberately stopped mapping while the group resolved the fatal flaw issue and resolved to try a different approach. This subsequent approach proved to be much more successful and we never deviated from it after that. “No fatal flaws” became a bit of a mantra among this group.

A key to working with a facilitator is to remember the lesson on confidence and assertiveness from part 3. Just because a facilitator is directing the meeting and influencing the direction of the conversation, it doesn’t mean that the mapper is purely a scribe. Work out a system with the facilitator where, if you raise your hand or signal in some way, you are not ready to move on straight away. Another technique that I have used in a large group situation was to assign someone else the traffic warden role, where if I am having trouble keeping up with the various conversations, and my eyes are on the map, they can call the group to order.

Dialogue Mapping, in tandem with another Dialogue Mapper ,can work very well and I have done many times with my colleagues at Seven Sigma. In this situation, you are both thinking in the IBIS grammar and both of you are mentally unpacking the conversation, although only one of you is actually performing the mapping. We have used this technique with particularly good results in SharePoint requirement gathering workshops, where one of us asks the questions and the other performs the mapping.

Dialogue Mapping and Debategraph

Compendium is one of several tools that can be used to create and render argument grammar like IBIS. For me, Compendium is the absolute best for Dialogue Mapping. Being a desktop application, I do not need internet access and once you are proficient with it, Compendium is very fast. This is of course, the biggest factor for Dialogue Mapping live. You do not want to be hindered by the limitations of the software tool that you are using.

I noticed that some CleverworkArounds readers created IBIS maps in Visio and were also using mind mapping tools after I published the “One best practice” series. But the problem is, although you can technically make an IBIS map, those tools would never work in a live session because of how slow it would be to add rationales to the map. Seriously guys, it might be technically possibly but do not attempt to use those tools live.

One size does not fit all and this is especially true of sense making tools. There are actually two main audiences for maps like this. Those who create the maps and those who consume the maps. The key point is that the ultimate audience for any map is quite often not the group creating the map in the first place. The whole point of capturing rationale is to make visible the process that a group went through when working on a problem, which ultimately shows why a particular decision was made or why a course of action was taken. Those who want to review the rationales are a very different audience to those who made the decision and wish to demonstrate justification. Just because the tool works well for the problem solving process, does not automatically assume that the tool is then best suited to the communication of that rationale to a wider audience.

Compendium maps work brilliantly well during the Dialogue Mapping process and from a broader communication point of view, work exceptionally well when detailed maps are printed onto large sized paper. But as a communication and distribution tool, Compendium is weaker than some of the alternatives. Compendium maps do not translate overly well to the web at this point, and asking all interested parties to install compendium is out of the question. For the sake of article length, I will not go into detail why this is, but to appease the Compendium fanbois, this is direct feedback from my clients and not just my opinionated rant.

For communicating the rationale that has come from Dialogue Mapping sessions to a wider audience, Debategraph is ideal. Unlike Compendium, Debategraph is a cloud based argument visualisation tool, designed to leverage the freeform updating capabilities of a wiki, along with the rigor of an argument grammar much like IBIS. Debategraph does not use a top down or left to right visualisation method. Instead each node is at the centre of the screen and surrounding issues, ideas, pros and cons surround the node, requiring the user to click nodes to explore further argumentation.

The beauty of Debategraph is the combination of its argument navigation, along with the streaming view of related content as shown below. My clients absolutely love the stream view because it is so simple for people to explore and work with. The ability to embed a map at any point in the debate on any web site is also pretty handy and I have pasted a sample map below to illustrate this. Click a node on the left pane (the “*” means there are sub arguments) and the content in the right window will change, based on which argument node is currently being examined.

Compare this to Compendium maps, where additional rich content like images, documents and the like are treated as additional nodes in the map. As you can see in the example below, it is possible to integrate rich content into the map very easily, but that rich content is linked in the same manner as the argumentation itself. Debategraph on the other hard, separates the argumentation from the supporting content and I think that this works much better and supports a richer form of argument based content delivery.

image

But once again, use the best tool that fits the purpose. From a dialogue and rationales collection point of view, Debategraph is an excellent way for a bunch of geographically dispersed people to debate a particular issue because the map will refactor on the fly as people self-contribute to it. But I personally would not use Debategraph for the Dialogue Mapping process, because it is not as fast as compendium and it is not as easy to view the map in full context as shown above. The over-arching point with all of this is that if the rationale has been captured in the first place, there are many ways to make creative use of it.

Note: To be fair on the Compendium makers there are many excellent examples of Compendium being used for some pretty impressive things. I am talking here specifically about online collaboration and communication to a wider audience.

Conclusion

This series of posts has examined the practical aspects of Dialogue Mapping, explored some of the techniques that I have used to augment it. Although I do not intend to write any more articles on this topic right now, the series is by no means complete. This is an ongoing learning process for all practitioners of this craft and I am sure that other Dialogue Mappers have tried different techniques than those that I have covered. (Some interesting things are happening on the SharePoint integration front too, which should enrich this experience even further, but that is a whole separate topic 🙂 )

But one final request. If you have used techniques such as these to enrich the experience for participants, then I’d love to hear from you. Even if it is not for Dialogue Mapping, any technique that is inclusive and augments the holding environment, please drop me a line or leave a comment.

Thanks for reading

Paul Culmsee

www.sevensigma.com.au



Am I a Business Analyst? What about those calling themselves BAs?

Hi

I attended and spoke at the Perth Business Analyst World Conference this week and really enjoyed it. This was a bit of a departure from the SharePoint events that I normally frequent, and I really didn’t know what to expect. Certainly, not having to fly 30+ hours just to speak is a big plus 🙂 The recommendation to the organisers to consider me, came about via Craig Brown, who has a very popular project management blog that I follow. Thanks so much Craig, I owe you a beer when I am in Melbourne next.

image

The conference report…

My talk was actually *not* about SharePoint and instead I was able to focus on more of my material on wicked problems, the shared understanding/shared commitment principle and then, the sense-making tools and techniques that I use to help bring this about. I was also able to demo the fruits of a very exciting, non IT project that I have been working on for a long time (more on that in a future post).

Despite my “This ain’t my normal crowd” trepidations, the feedback was great and the best thing to hear from participants, was that for many, it was stuff they have never heard before. That, for me, was really satisfying because I like the notion of presenting new ideas that actually have some decent practical examples to back them up. (This is something Andrew Woodward and I have in common. We love academic rigor for what we use, but it has to have been used in the real world with tangible success). Although I know that some people will disagree with the methods that myself and my colleagues use, I was able to demonstrate what I think is some pretty compelling case studies that support them.

What was interesting though, was that the examples and case studies were able to support what a lot of the other presenters had to say as well.

Ann Smith of Black Circle for example, had a great talk that was essentially about human cognition; essentially the wiring in our brains that serve to explain why big, fat documents are often not good ways to convey information. (Being a practicing dialogue mapper, no arguments from me there!) I am a nerd for this sort of stuff, having written previously on behavioural styles, learning styles and organisational culture, and Anne offered some new, interesting things that I have previously not considered or covered – more blog fodder for CleverWorkarounds, methinks.

Another highlight,the Western Power Business Transformation project, presented by Lorraine Pestell was also fascinating (I have a weakness for voice of the customer type sessions and this was no exception). Many of the strategic challenges that they are facing, such as sustainability and the changing business/regulatory environment, is very similar to the work I am doing elsewhere and it was great to see how Lorraine and her team were approaching the challenge and has given me some ideas and approaches to take back with me to my clients and projects.

The BA identity crisis

But back to the question suggested by the title of this post. There were some panel and round-table sessions about the topics of what actually *is* a BA, how you validate or recognise BA excellence, and the perennial BA versus PM turf-war debate.

Up until this time, I had actually never considered myself a BA because I had never actually given it any thought! As a self employed consultant, the only thing that matters is doing a good enough job to keep people wanting you to come back. So to that end, I didn’t worry so much about what I was called, provided that my clients were happy and the invoice was paid. But even if I wasn’t a consultant, I think that role titles often do not reflect reality and they also have a pigeonholing effect, depending on the attitudes and perceptions of what others think that role entails. Many position titles were discussed, “Solutions Architect”, “Business Architect”, “Change Manager” and some that were so pretentious that they bordered on wanky. More fancy words with no more clarity. No wonder many BA’s are struggling a bit for a sense of identity.

What I noticed when talking to the conference participants was that some attendees spoke from a lens where they seemed to feel that it was incumbent on them to provide a “translator” role between IT and “the business”. After all, nerds and CFO’s can’t communicate right? Enter the BA to ask questions and solve problems.

I have no major objection to that notion at some levels, but it is that *precise* mindset that makes me think “Well, I am definitely not a BA.”

Why? It was the notion that this “translation” was based on being the go-between from IT and the business. Thus, taking what one party says, transforming it and then passing it to the other party. As a result, BA’s are acting as a listener and interpreter, yet relaying second hand messages (messages that may be very different originally) between parties.

I personally balk at this. In fact, it really grates on me. By that definition, I don’t think I am a BA at all.

Interestingly, other topics of conversations were around “Well, how does a BA fit into Agile?”, “Is there a place for the BA in an Agile world”, and the like. What was interesting, and somewhat concerning, about these conversations was that those BAs who tended to think of themselves in terms of this “translation” role, really did not have a great grasp on the underlying principles of what we now call “Agile”.

Although Agile means a lot of different things and there are different sub-methods applied, these BAs got all focussed on the processes of Agile. They overlooked the fact that the process is actually the means to an end and it is the end-game that they have overlooked. Agile, (okay well Scrum anyway) attempts to use process and rigour (yes, rigour!) to make a project as conducive to shared understanding as possible. Probably the best thing that Agile does, above all else, is put diverse people in the same room. That alone will make bigger understanding breakthroughs than anything else!

Business Analyst KPI – shared understanding?

So, why am I not a BA?

My methods for translating are fundamentally inclusive. In other words, I do not “translate” anything, “take” it to another party and “relay” through my own words (and lens). I feel that despite all best intentions and whatever diagramming or modelling tool that you use, when you do this, you will always still find that you have your own cognitive biases that will not necessarily deliver the shared understanding that you think you are delivering. Instead, what I do is provide a rich container for a group to explore an issue together. In the same way that Agile tends to like all project members and stakeholders to be in the same room, Dialogue Mapping puts everyone in the same room and provides a suitable container for handling dialogue in a much better manner than traditional meetings and workshops.

If you agree with my previous assertions that a lot of the visible causes of project failure (scope creep, vague requirements, etc) comes from a lack of shared understanding among participants, and that BAs identify themselves as the bridge between IT and “the business” (which by the way is an insultingly gross simplification), then isn’t the ultimate KPI for the BA is to create and maintain that shared understanding? If not, yours is just another opinion that is counted no more or less than anybody else’s. Are you signal or noise?

So, in my humble opinion, the role of the BA is not to be the go-between from disparate stakeholders. Instead, it is your ability to create the sort of conducive holding environment that enables project participants to achieving shared understanding. How you do that is completely up to you of course, and if you have managed to progress a group from an agreed undesirable present state to a desirable future state, then your methods are totally validated.

Get over titles…

Now, if you call yourself a BA and think I am picking on you because you feel that you are the translator, don’t feel bad because plenty of PMs are guilty in their way too. In some ways, I feel that business analysts only exist as a career because enough people with the “Project Manager” title thought that time and budget alone were the only factors in project success. Some PMs who disagreed with this, felt that solving the problem was also critical, gravitated to the discipline of what we now label as “Business Analyst”. Some application developers that felt there was more to life than cutting code and made a similar gravitation. Put a bunch of like-minded people together and soon enough we have a “cool kids” club and lo’ and behold, we have a new discipline with a new set of titles.

(“Information architect” is a more recent example of this phenomenon than “Business Analyst”).

But, let me tell you something else about this title misconception. For a BA to label all PMs as interested only in time and budget is an insult to those PMs who actually understand that achieving and maintaining shared understanding is the end-game. The truly great project managers who I have had the pleasure of working with were actually leaders, not managers. They have all of the same characteristics of what makes a truly good business analyst: Critical thinking, soft-skills and most of all, a great radar for determining when stakeholders are not aligned and doing what is necessary to rectify the situation. They do not always dive into process and structure because their particular body of knowledge told them to. Instead, they have coffees, drink beer, conduct lunch-time workshops with free food and beverages, mediate, essentially whatever is needed to oil the cogs of dialogue that prevents something small becoming something nasty later.

By the way, I have met some angel application developers like this too, as well as infrastructure people.

If you want proof of a truly great project manager, then Kailash Awati’s wonderful site should be mandatory reading for both the BA and PM disciplines (and scrum masters too for that matter!). Kailash writes what essentially is a project management blog, but he has a deeper understanding of the sorts of soft factors that would put many BAs and some facilitators to shame.

Conclusion

In my talk at the conference, I emphasised that the ultimate success factor in any project is bringing about shared commitment through shared understanding among the participants. I believe that achieving these goals is the ultimate KPI for a BA, or anybody else who feels that they are there to help solve a problem, not deliver a crap solution that happens to be on time and on budget.

Thus, any method that helps a group achieve this is a good method because it has made a positive difference in advancing a group from understood present state to an understood desirable future state.

So, perhaps I am, after all, a BA?

Thanks for reading

Paul Culmsee

www.sevensigma.com.au



« Previous Page

Today is: Wednesday 3 June 2026 -