Modeling & The Current State (Modeling the “As Is” process is mostly a waste of time)

December 8, 2008

It seems a contentious point of view in Business Process Management – but when we come up to the “Understand Phase” (“As Is” or “Current State” model ), we recommend “time boxing” the work to ensure that the activity is kept at a suitably high level. The intention of this activity is really to create a baseline; a reference point for the BPM project.

Now those who continue with their “legacy thinking” perspective usually decide that it is important to create a detailed description of how work happens. They model everything in sight, trying to create an accurate representation of the work as it happens today. While this is good for the “billable hours” of consulting firms, it does little for the business managers engaged on a journey of change and discovery.

The point is that the amount of work expended here is usually wholly inappropriate to the benefit derived. If your intention is to change the way things happen, gathering a great deal of detail around current work practices is a waste of time. If you are going to improve things (with or without the use of automation), then you will be changing how the process is carried out … i.e. how things happen today will soon become a thing of the past.

Don’t get me wrong, it is absolutely essential to develop a baseline understanding of the ways things are done. It’s just a question of emphasis. The issue for those involved in the exercise is just what degree of detail is required. They should be asking “can we stop now?”

The real purpose of current state modeling is to establish a baseline – so that the team can establish a realistic business case (allowing them to track benefits and improvements during and after implementation), and to identify the areas that require attention.

This is more about a pragmatic assessment of reality and clarification of current performance metrics than it is about process modeling. The metrics in question are those that the customer of the process really cares about (not the detailed cycle times of some low-level sub-process). From a modeling point of view, the need is for enough structure to hang the metrics upon (and perhaps one level of detail below). Anything more than that is a waste of time and resources.

So how much detail do you really need? Well, I normally start with high level outline of the process – the major chunks and then draw a simple high level process model. I recommend a high level BPMN diagram, but I usually seek to contrast that model with a Role Activity Diagram (not the same as a flow diagram with swim lanes, RADs model how the Roles involve change state and synchronize their actions), and perhaps simple Object State Transition Network (how the things moving through the model change state).

With a high-level flow diagram or outline of the process, it is really very straight forward to develop these alternative views, but they really do help people see things differently. I often say that the problem with Flow Diagrams, is that “the more you look at them, the less they mean.” Flow diagrams always look correct – for example, in my recent book “BPMN Modeling and Reference Guide” (authored with Stephen White the main author of the BPMN specification itself), I have yet to receive a note from anyone telling us that we have a major flaw in one of the models (yes, there is at least one). It just looks correct (and this is a book where we tried our very hardest to make sure every model was “right”).

Incidentally, the best reference on RADs is Martyn Ould’s “Business Process Management – A Rigorous Approach.” And for OSTN, I prefer the IDEF3 perspective as it is relatively simple and easy to understand (UML also has similar modeling capabilities).

Coming back to the Understand Phase, in the workshops with the Subject Matter Experts (SMEs), I also seek to understand the volumes of work, any major exceptions and the percentages of items that follow the major paths from decision points. The other thing to understand is the numbers of employees involved in the work (FTEs and the amount of time spent on each area of the process).  From that information you can calculate the costs of undertaking the work and where the money goes. You also need to understand the roles involved, and the capabilities of the staff members who fulfill those roles. All of these become essential ingredients for the business case. Without it you are whistling in the wind (when it comes to asking for funding). Even if you already have the funding, you should do this anyway (it will certainly be needed later).

I could go on here at length, but the point I am trying to make here with this blog post is this … if your consulting provider is asking you to fund a detailed “As Is” phase of work, then you are throwing money away. They are more interested in lining their pockets than assisting the client. The only exception that I can think of is where the process is itself highly regulated (and a rigorous work definition is mandated by law). In such cases, I think you have to draw your own conclusions on how to avoid “analysis paralysis.”

Advertisements

BPM – Is it a Software Engineering Tool? A Technology? or a Management Discipline?

November 30, 2008

In his excellent posting, Keith Swenson makesmany good points. He points to the range of interpretations of BPM, and particularly highlights the issues associated with its interpretation by software engineers as just another piece of hype on the road to good programs. But I think there is another, perhaps more important strand that is buried in there. As Ketih points out, BPM is about the Management of Business Processes.

As we all know, everyone’s interpretation of the term Business Process is different. In my training (whether that be BPMN, or higher level training on BPM Methods), it is one of the first things I get people to write down (inside the first 5 mins), and not surprise, every definition is entirely different. And when those people are senior managers in a ciompany, their interpretation of the term is invariably what I call “Process as Purpose“. The point is that they see Processes as being more about the purpose than the constraint implied by sequencing of steps. They are there (at the training), because they see the importance of “Managing” their processes. Indeed that concept (Managing Business Processes) as central to the success of their companies.

[[ I am still down in Brazil, and I am really struck by the process sophistication of the people I am meeting. They all get it. I was more than surprised to find two “Business Owners” (people who own significant businesses), giving up 3 days of their time to come on a course around how to structure and run BPM programs. They were cherry picking from the broad range of techniques we covered, but ask yourself whether you could imagine the CEO or COO deciding that they should attend a public training course. That’s what I am getting at about the sophistication of the Brazilian business climate. ]]

Coming back to Keith’s post, he describes a spectrum of BPM interpretation – from pure Software Engineering (where the SW Eng tries to reflect the needs of the business person’s Business Process); through Business Processes being modeled by a business person, then tossed over the fence to a Software Engineer to finish them off; to the Business Process as modeled by the business person, then being directly executed (what he called “Pure BPM”). I am not quite sure I agree with the Pure BPM bit, but I do know what he is pointing to … where the processes of the firm are driven by models (without translation to some intermediate executable format (like say BPEL).

One of the comments on Keith’s post points to the challenges of getting business people to model their own processes and make the resulting collection of stuff useful. He described the usually resulting mess as an “expensive disaster”. And the reason for this is that business people dont usually have the sophistication to understand their business problem as a set of inter-releated processes that between them deliver on the “Process as Purpose” concept I referred to earlier.

Invariably, process modelers (whether IT or business) tend to see a process problem as a single process. They interpret the high level Purpose as a single implementation process (which invariably it is not). They make all sorts of mistakes such as mixing up the “Handle an Instance” with the “Manage the Flow of instances”; they switch from batch mode to handling a single instance; they dont think about the interfaces between processes (handing information from one to another), etc. What they do is try and connect up everything that sounds like an Activity into one convoluted process.

Now software engineers are usually more adept at the necessary abstract thinking, but that doesn’t mean to say that business people cannot wrap their pretty heads around the notions. It is merely a reflection of the fact that they have not had adequate training. What is missing (across this entire industry) is better learning around “Process Architecture” – what “chunks” do you need and why. Poor chunking leads to unnecessary complexity (and even “expensive disasters”).

We are still stuck with decomposition as the prevailing mind set – where sub-processes are always contained within the parent. SOA concepts seek to get around this, but there is also a higher level “Business Services Oriented Architecture”. Processes should not be regarded as some sort of static hierarchy, they are more accurately regarded as a network of interacting instances. Think more jigsaw puzzle than org chart.

When I gave a “Power Breakfast” at the last Gartner BPM Summit on BPM and Process Architecture I had a packed room (it was starting at 7:30 in the morning so these people were keen). I described a set of methods that you could use to go from “What Business Are You In” to what “Processes Do You Need” right down to the SOA components if that is what you wanted to do (I would recommend looking at a BPM Suite first rather than going straight to the SOA software engineers paradise). I only saw one person out of the 90 or so get up and leave, and nearly everyone else gave me their card at the end of it. The room really was comprised of mostly Enterprise Architecture folks from the IT community, all of whom struggle with this transition.

Switching tacks – the vendors BPM Suites are unconciously making this architecture problem worse. With only a few exceptions (Pega, Itensil, BizAgi … I am sure there are others in this category too, these are the ones that spring to mind), vendors interpret the business process problem as being entirely seperate from the data and artifacts associated with the process (business people see them as intertwined). They regard the process relevant data as a set of named value pairs … the information required by Process A is declared on Process A, and must be recreated on Process B and then mapped from one to the other (if the processes need to communicate with each other). That means that there is an extra (unnecessary) layer of complexity for business people trying to reflect their business problem. Moreover, if you change one process, then you need to refactor all the interfaces. This is “software engineering” oriented thinking.

The other approach is to define your data structures (perhaps as an “Entity” defined as an exstensible set of XML artifacts) and then describe the views on those artifacts at the level of the data structure. Then it is merely a matter of associating your processes with the Data Entity, and all the different views become available. Process interfaces become an order of magnitude more accessible (to the business user), you can use any number of processes to support a single case of work, and again it helps move away from the software engineering mindset we find in so many BPM tools (which were often created to solve the problem of Enterprise Applicatin Integration … hence their association with Software Engineering).


BPMN 2.0 – Marriage Made In Heaven or Trough of Disillusionment

October 31, 2008

Inside the OMG there has been a heated debate about whether BPMN 2.0 should become linked more explicitly to UML … so many heated exchanges to chew through. This blog posting was put together in that context.

It was originally Charles Box (and later Deming) who said: “All Models Are Wrong, Some Are Useful.” We should learn to live with that reality.

By modeling something, we are removing some aspect of the real world in order to represent it. And yet, the IT-oriented folks continue to flail about looking for one true modeling notation and set of semantics to rule them all (like string theory). As though how somehow everything must be translatable and interconnected. I think for most business folks – they don’t really care. They use models to communicate with each other … and yes, they use circles and arrows, and boxes and clouds, and … only a very few have the interest in making them all relate to each other.

It is only when we get down into the IT organization that all of this stuff has to be translatable and traceable … that all the classes and elements have to get along (be placed in some interconnected network of stuff).

We currently have a Business Process Modeling Notation (sans rigorous meta-model), we also have a Unified Modeling Language (avec rigorous meta-model)… both can be used to model processes (even businesses). But they are different and some folks feel the need to move stuff between these two approaches. We invented BPDM (another rigorous meta-model) as a mechanism for doing that sort of thing along with providing a competing BPMN serialization (to XPDL). But BPDM was deemed too hard by many (or too expensive to implement support for when you already have UML) … at least we have seen little appetite in the market by vendors for supporting it. Most of the BPM Suite/Workflow vendors out there are on XPDL.

The idea with BPDM was to create a semantic layer that would allow the translation between these modelling notations (and others). Or more precisely, that which can be translated should be able to be translated with “semantic integrity”. It would also allow for extension of the semantics for different needs. But for UML to work alongside this, would have meant a Profile for UML (or some other detailed integration at semantic level) – but the folks with the skills and expertise for this sort of thing chose not to invest their time and energy in developing such an interchange format (between UML and BPMN via BPDM).

But that’s all history now. What these well resourced players could sign up to was a future version of BPMN. So now we have BPMN 2.0 – with all the hope and promise of an effective marriage between orchestration (BPMN) and choreography (something that is needed for effective interchange of models but very few people understand fully).

The BPMN 2.0 RFP calls for: “A single specification, entitled Business Process Model and Notation (BPMN 2.0), that defines the notation, meta-model and interchange format … Extension of  BPMN notation to address BPDM concepts … [will need] changes that reconcile BPMN and BPDM to a single, consistent language. The ability to exchange business process models and their diagram layouts among process modeling tools preserving semantic integrity. Enhancements in BPMN’s ability to:

Model orchestrations and choreographies as stand-alone or integrated models. Support the display and interchange of different perspectives on a model that allow a user to focus on specific concerns.” Further … “Proposals shall specify conformance criteria that clearly state what features all implementations must support and which features (if any) may optionally be supported.”

At the same time, it now seems that BPMN 2.0 has to provide a high level modeling approach and traceability down through the stack (which means UML right). There are various other camps – all attempting to twist the specification in their own particular direction. I hear one group saying “let’s make BPMN reflect the needs of BPEL”; others saying well we should now make BPMN part of UML (I must get asked if that is going to happen at least once at every conference … always with a look of dread on the part of the person asking); others wanting stronger choreography support (personally I would like to see something emerge that could support a translation to Role Activity Diagrams which is a much more powerful approach to modeling how roles collaborate and inter-operate that what I have seen so far in BPMN 2.0).

So now that we are in the trough of disillusionment (the marriage vows have yet to be cemented, the RFP is but a hazy memory of a drunken engagement party). We have two different groups (power bases) lobbying for the ascendancy – well not really lobbying, lets just say they are struggling to work out what bits of each others proposals they like, what they can live with, and what they don’t like. And there is a lot more soul searching (work) to go on there.

Let’s recap on where we seem to be now:

  • There is a Notation Specification with some (IMNSHO) half baked choreography support (along with an abstract syntax). It fixes some things and has missed the point on others.
  • There is another Specification (that derives from the BPDM) which describes a more robust set of process semantics … let’s call that the Process Modeling Framework for the moment. This is still perceived as too difficult for some to wrap their heads around … but in the end it is where the UML piece will have to tie in (if someone is going to invest the effort).
  • Then there is a specification that is supposed to outline the mapping from one to the other.

As far as I can tell – all three of these documents require significant further work to marry and align – personally, I can’t see this being finished in the near future. It won’t be just a one cycle delay.  And that’s before we take on the UML interface challenge (although I am sure someone stepping up to the plate on that one would be welcome, they would be doing it against a moving target). We also need to think about how we will embrace the current XPDL community (an upgrade path). And now it looks like we are now about to invent a couple of new modeling approaches, which of course some are already saying should somehow be like BPMN (or UML).

In the end, this stuff only makes sense in context of enabling businesses to work more effectively. BPMN 2.0 needs to give true model portability (with semantic interoperability). We need conformance levels (but first we need to decide on where the lines in the sand are for those different levels). We need to … stop broadening the effort and focus more on getting to the result.

It can’t be that hard – especially when we have all the “Wise Wizards at OMGee” working feverishly on the problem.


Gartner BPM Summit Report

September 14, 2008

Well the book (BPMN Modeling and Reference Guide) is out, and we sold out at the Gartner BPM Sumit last week (we launched the book there). People would come up, thumb through it, think about it perhaps but then buy a copy. Several of them came back an hour or so later and bought more copies … with phrases like “this is actually designed for people to read …”, “the rest of my team need to read this …” and “I must get my boss a copy of this.” And then some of the vendors popped their heads above the parapet – one of the larger names in this space is now talking about buying 2000 copies (one for each member of staff I guess), and two others walked off with half a dozen copies for their colleagues. So all up, we were pretty pleased.

My “BPM and Modeling” track session was well attended (a couple of hundred delegates), and only 3 people fell asleep. Of course, it was directly after lunch, and modeling is not exactly the most rivetting of topics, although I did try to make it as interactive as possible (given the situation). I reiterated a central tenet of my approach to process modeling – in the early stages of a BPM initiative it is important to contrast modeling approaches to drive understanding (rather than slavishly following just one technique such as BPMN). The key point is that you need to be able to change your perspective … to see things differently if you want to really understand the process. Of course, when it comes to implementing a process using on a BPM Suite, then you need to resort to BPMN to get clarity into the execution model.

On Friday morning, I filled the (relatively small) room at the “BPM and Process Architecture” power breakfast, which was always going to be a difficult task given that the night before was the vendor parties (free booze). Anyway, several people complemented on an interesting session afterward. I had tried to communicate the set of methods we use to move from thinking about Strategy to through to implementation. This session included an overview of techniques to support the clarification and definition of:

  • What business are you really in, and what business services you need to support that vision.
  • From that, what is an effective Process Architecture -i.e. what processes do you need to support those business services and how do they communicate with each other, etc. (referencing the RIVA Process Architecture method).
  • And from there, how do you get to the SOA-based IT Services that are needed to support those business processes (i.e. how they are implemented). What service interfaces are required for each component, etc.

The problem is that, when it comes to process architecture there are very few reliable approaches … certainly functional decomposition falls down a hole here. Most rely on what I call the “black art” approach. They make the mistake of linking Process Architecture to the current mechanisms for apportioning blame (the organizational chat) – i.e. the Process Architecture should be independent of the organizational structure. If you change the way the departments and business units are structured, this shouldn’t force a refactoring of the processes.

I ended the session with a short overview of Case Handling approaches … an area that I feel is poorly understood (especially by vendors who have little incentive to change the status quo). Case Handling is really a design pattern enabling a balance between control and adaptation (efficiency and flexibility), where users are left in control, yet the organization can still provide support for the vagaries of customer interaction (see “Customers and Business Processes – Difficult Domains to Integrate” on page 2 of the papers available on the BPM Focus web site for more detail there).

I attended a few other sessions, but found new insights a bit few and far between (for me that is, too educated I guess). I only caught the last third of the opening keynote (I was told I didnt miss much earlier), and found myself taking notes (as much as anything on the language used and the phraseology for concepts). I sat through an entire session on Customer Interaction and BPM (which I felt missed the mark entirely … but still managed to salvage a few points here and there). I enjoyed Dan Roam’s session (keynote on day 2) on modeling on the back of a napkin (using simple rich pictures to communicate). I felt that the session on BPM and SaaS (Michelle Cantara) didn’t go far enough (stopping well short of talking about the implications of the cloud on the way work will happen in the future). But there were other sessions on that (competed with my modeling session), so no doubt they got more into that aspect then.

The Vendor Showroom floor was the usual zoo – with people wandering on and off stands or generally ignoring the vendors while they consumed their deserts or coffee. This format just doesnt work for either the vendors or the delegates who want information on what the products really do. Contrast that with the BPM Technology Showcase we are running on October 14-16 in DC. The last one of those in Nashville was a roaring success as people got up close and personal with what the technology can really do for you.

Otherwise, I seemed to be either signing books or getting button-holed by folks in the hallway.


Some amazing results already – Business Analyst Survey

August 12, 2008

Well I must say I am more than just a little surprised at some of the results that have popped out of our Business Analyst survey. With approaching 300 responses now, it is starting to become statistically significant (although with hindsight some of the questions could have been phrased more directly).

First, its the apparent popularity of Role Activity Diagrams. To my great surprise, the technique is more widely used than BPMN at this point. Something like 47% of respondents have indicated that they use RADs. Now I am sure that there is a certain element of confusion here (RADs are not the same as Flow Diagrams segmented by Swimlanes), but nevertheless, it indicates a heightened level of awareness around the technique. Of course, most Business Analysts still model processes without any formal techniques underpinnign their approaches.

Secondly, despite the attempted spin from some naysayers in our industry, around 65% of firms do use a BPM Suite or Workflow product somewhere in their organization (and a significant proportion have more than one BPMS). This is validated by the fact that a similar number of people say their own process models are used to support a BPM or Workflow implementation. And it is quite inciteful to see the distribution of the products used (it roughly follows what I would have guessed at the outset … which is somewhat at odds with the touted penetration of the vendors involved).

Thirdly, the percentage of respondents who feel they would benefit from further training. For the moment I will keep the details of this to myself, and the areas that are of most interest (let’s wait till the survey is completed in mid-October).

If you havent taken the survey then get your votes in … it should only take about 5-10 mins (something to do on those quiet summer days in the office ;-/). To encourage you to take part, we have put up some significant prizes (Nikon D200 or Bose Home Entertainment System being the first prize). The drawing (from those that complete the survey) will take place at the next BPM Technology Showcase in Washington DC (October 14-16).


Busy Busy Busy … Books On The Way

July 31, 2008

Another litany of excuses. Anyone who has looked at this blog over the last year or so will see the very poor track record I have established in terms of keeping it up to date. The reason – just not enough hours in the day. But that is probably true of all bloggers, but I wonder how many of them have been pushing the development of two books, along side a full schedule of training and consulting activities. Right now I am supposed to be outside on holiday with my family, but instead I am deep into the last push to finalize the content for the forthcoming BPMN Modeling and Reference Guide.

This has been a mamoth effort over the last few months involving detailed collaboration with Stephen White, the main author of the BPMN specification. We have done our best to make it as readable and accessible as possible, separating out the detailed reference section from a piece about modeling in general (everything from history of BPMN, why/how to model, etc).

There is also an extended scenario based introduction to BPMN functionality, bringing in new BPMN functionality in the context of an easily understood business problem. Throughout the section, the business scenario is elaborated upon and the corresponding models and BPMN functionality explained. Through our training courses, we have found that people learn far better this way.

Of course, there is a detailed explanation of all BPMN functionality. And for most, who are actively involved in modeling, this reference material is sorely needed. For the layman, the specification is somewhat hard to follow (and that is being kind). In the book, we explore each area of functionality and provide a detailed explanation for its use, and behavior.

The BPMN Modeling and Reference Guide will be launched at the Gartner BPM Summit in Washington DC.I am doing a session on BPM and Modeling and another on Developing Appropriate Process Architecures. Neither of these sessions are designed for beginners (although the modeling session should be pretty acessible.

Incidentally, I notice that I am the only non-Gartner BPM Analyst/Comentator presenting at the conference this time around – seems that the same old, same old’s have finally been found out ;-).

Book 2

My other book, that has been in devleopment for the last 5 years or so is in the later stages of finalization. Mastering BPM has been an evolving piece that will probably hit the presses in a similar time frame. I still have a chapter to write, but it is just about there.

Of course, all of this book development work takes cycles out of the day and impacts the ability to execute on other things. Anyway … I hope to get into a more regular pattern of blog postings and updates by the end of next month.


BPM and SaaS

December 18, 2007

I know I have been remiss in keeping this blog thing going. As I have alluded to before, the reality is that there are just not enough hours in the day. So as a next step, I thought I would start writing a bit about some of the work I have been involved in lately, drawing attention to the growing interest in BPM-SaaS oriented plays. As (if/when) I get permission to talk about these services more openly, I will post links and invitations, etc.

The first one I can start to talk about is Appian Anywhere (A2) – the Appian Enterprise (AE) product delivered on demand over the Web. It has been in private beta for a while now. I have been building the online training and video help for A2 which is due to launch sometime soon … probably early next year.

It has been very instructive for me – firstly, to have to get down into the innards of the BPMS and build actual applications … and secondly to appreciate just how much effort is required. That last point is at the level of developing effective applications and also building effective online training. Along the way, I have also discovered a few things about process modeling and what BPMN really enables (A2 is one of the stronger BPMN implementations), and also what is not in the process model but you need to be wary of.

I have had to grapple with application implementation issues that are really not part of the “process” model. I am really pointing at issues that the user has to deal with in getting their BPM system to present the related data and docs of the application (all deemed to be neatly outside of the scope of the BPMN specification). A2 (and AE) have one of the sexiest (Ajax) forms environments I have seen (I’ll see if I can put a clip up on YouTube). But even that forms environment relies on a robust set of process variables being defined for the process … which of course must be accurately mapped to all related processes and to the data sources themselves.

In A2 you declare the related variables at the outset (in the Process Properties), with a wide selection of types from the Boolean Yes/No through users, people, files, numbers, text, date-time, etc. Even discussion forums, folders, groups and messages can be defined as variables. Alternatively, you can just start process modeling and declare your variables as you need to, right in the middle of defining the process. Whenever, a new variable is needed, there is always an option to create one. The data itself is automatically persisted (although you do get the option of mapping the data to MySQL, Oracle and SQL Server).

Manipulating these variables is pretty straight forward – whenever you need to make a decision, place them on a form, or use the embedded Expression Editor – the context of the variable is automatically carried forward. For example, inserting a text data field into an external database, will automatically provide the text variables to choose from. If the field were of a type Date-Time then the declared Date-Time variables are presented to choose from.

The point that I am trying to make here is that defining useful processes is a lot more than merely the order of activities and the assignment of tasks to individuals. You need ways of dealing with the process relevant data … and that is where the complexity really comes in.

Which in turn points to one of the challenges in BPM modeling for SaaS – who really is the target market, and how much complexity can they handle? There is always the temptation to just keep adding functionality to the process model (as you think of yet another wrinkle in the usage scenario). But the same is also true of the BPM Suite vendors. The problem becomes more “what you want to do/support”, and inevitably the development environment for a BPMS becomes ever more “complex.”

If your target market are really business users without Business Analyst training, then this sort of development environment (in A2) is probably just a bit too complex. But dont get me wrong, if your target market are those people who would have otherwise defined a robust process model to support the build out of a workflow or Java app, then A2 is a doddle.

As a final thought, on this BPM-SaaS thang, I am seeing a lot of interest. I recently recorded a Webinar for Search-CIO which will probably air in early January (when I get links I will post here). Point is that all sorts of folks are now engaging in the promise of BPM at all sorts of levels. My perception is that if we can fix the usability angle (getting people to do it for themselves), then we will finally start to see this BPM market start to explode and be taken more mainstream. In my last posting, I mentioned Itensil who I believe have made some big strides in this area – but they have done that by deciding not to support the transactional process (which is the core of most BPM Suites). They are focused solely on the needs of knowledge workers in their collaborative processes (what I describe as “roughly repeatable practices” rather than “rigorous procedures”).