BPMN 2.0 – Marriage Made In Heaven or Trough of Disillusionment

Inside the OMG there has been a heated debate about whether BPMN 2.0 should become linked more explicitly to UML … so many heated exchanges to chew through. This blog posting was put together in that context.

It was originally Charles Box (and later Deming) who said: “All Models Are Wrong, Some Are Useful.” We should learn to live with that reality.

By modeling something, we are removing some aspect of the real world in order to represent it. And yet, the IT-oriented folks continue to flail about looking for one true modeling notation and set of semantics to rule them all (like string theory). As though how somehow everything must be translatable and interconnected. I think for most business folks – they don’t really care. They use models to communicate with each other … and yes, they use circles and arrows, and boxes and clouds, and … only a very few have the interest in making them all relate to each other.

It is only when we get down into the IT organization that all of this stuff has to be translatable and traceable … that all the classes and elements have to get along (be placed in some interconnected network of stuff).

We currently have a Business Process Modeling Notation (sans rigorous meta-model), we also have a Unified Modeling Language (avec rigorous meta-model)… both can be used to model processes (even businesses). But they are different and some folks feel the need to move stuff between these two approaches. We invented BPDM (another rigorous meta-model) as a mechanism for doing that sort of thing along with providing a competing BPMN serialization (to XPDL). But BPDM was deemed too hard by many (or too expensive to implement support for when you already have UML) … at least we have seen little appetite in the market by vendors for supporting it. Most of the BPM Suite/Workflow vendors out there are on XPDL.

The idea with BPDM was to create a semantic layer that would allow the translation between these modelling notations (and others). Or more precisely, that which can be translated should be able to be translated with “semantic integrity”. It would also allow for extension of the semantics for different needs. But for UML to work alongside this, would have meant a Profile for UML (or some other detailed integration at semantic level) – but the folks with the skills and expertise for this sort of thing chose not to invest their time and energy in developing such an interchange format (between UML and BPMN via BPDM).

But that’s all history now. What these well resourced players could sign up to was a future version of BPMN. So now we have BPMN 2.0 – with all the hope and promise of an effective marriage between orchestration (BPMN) and choreography (something that is needed for effective interchange of models but very few people understand fully).

The BPMN 2.0 RFP calls for: “A single specification, entitled Business Process Model and Notation (BPMN 2.0), that defines the notation, meta-model and interchange format … Extension of  BPMN notation to address BPDM concepts … [will need] changes that reconcile BPMN and BPDM to a single, consistent language. The ability to exchange business process models and their diagram layouts among process modeling tools preserving semantic integrity. Enhancements in BPMN’s ability to:

Model orchestrations and choreographies as stand-alone or integrated models. Support the display and interchange of different perspectives on a model that allow a user to focus on specific concerns.” Further … “Proposals shall specify conformance criteria that clearly state what features all implementations must support and which features (if any) may optionally be supported.”

At the same time, it now seems that BPMN 2.0 has to provide a high level modeling approach and traceability down through the stack (which means UML right). There are various other camps – all attempting to twist the specification in their own particular direction. I hear one group saying “let’s make BPMN reflect the needs of BPEL”; others saying well we should now make BPMN part of UML (I must get asked if that is going to happen at least once at every conference … always with a look of dread on the part of the person asking); others wanting stronger choreography support (personally I would like to see something emerge that could support a translation to Role Activity Diagrams which is a much more powerful approach to modeling how roles collaborate and inter-operate that what I have seen so far in BPMN 2.0).

So now that we are in the trough of disillusionment (the marriage vows have yet to be cemented, the RFP is but a hazy memory of a drunken engagement party). We have two different groups (power bases) lobbying for the ascendancy – well not really lobbying, lets just say they are struggling to work out what bits of each others proposals they like, what they can live with, and what they don’t like. And there is a lot more soul searching (work) to go on there.

Let’s recap on where we seem to be now:

• There is a Notation Specification with some (IMNSHO) half baked choreography support (along with an abstract syntax). It fixes some things and has missed the point on others.
• There is another Specification (that derives from the BPDM) which describes a more robust set of process semantics … let’s call that the Process Modeling Framework for the moment. This is still perceived as too difficult for some to wrap their heads around … but in the end it is where the UML piece will have to tie in (if someone is going to invest the effort).
• Then there is a specification that is supposed to outline the mapping from one to the other.

As far as I can tell – all three of these documents require significant further work to marry and align – personally, I can’t see this being finished in the near future. It won’t be just a one cycle delay.  And that’s before we take on the UML interface challenge (although I am sure someone stepping up to the plate on that one would be welcome, they would be doing it against a moving target). We also need to think about how we will embrace the current XPDL community (an upgrade path). And now it looks like we are now about to invent a couple of new modeling approaches, which of course some are already saying should somehow be like BPMN (or UML).

In the end, this stuff only makes sense in context of enabling businesses to work more effectively. BPMN 2.0 needs to give true model portability (with semantic interoperability). We need conformance levels (but first we need to decide on where the lines in the sand are for those different levels). We need to … stop broadening the effort and focus more on getting to the result.

It can’t be that hard – especially when we have all the “Wise Wizards at OMGee” working feverishly on the problem.

Case Handling Discussion

Mea Culpa – yes, like others in this space, the challenge is keeping the blog going. Usual story of not managing to keep the clones working properly while I am sleeping. Lots of things to start sharing here … and now that I am out from under the endless train of deliverables and trainign courses, I should be able to find the odd bit of time.

The reason for this long awaited posting … I felt I wanted to pick up on discussions emerging in the BPM space – driven by Henk de Man’s presentation at the OMG meeting last week. He was talking to the need for better modeling approaches to support Case Handling (or Case Management depending on your perspective).

Like James Taylor (name now corrected),  I thought Henk’s presentation was also interesting. And as I pointed out during the session, a great many processes should be viewed in the Case Handling context. Readers might also be interested in the papers I produced that discuss these sorts of issues. But really getting at it from the pov of the Customer and Processes – “Business Processes and Customers – Difficult Domains to Integrate” available in the White Papers section of the BPM Focus web site.

The core of Henk’s presentation was that BPMN style modeling is not much help when trying to capture the essence of Case Handling. His own product has a strong Case Handling orientation and uses “States” and “Events” to enable some of the flexibility that Case Handling apps demand. In my experience, the key differentiating factor (between a tranditional workflow/BPMS app and Case Handling) is that the emphasis is with Case – it may have many processes and documents associated with it.

I suggested to him that he investigate Role Activity Diagrams (a way of modeling at how the Roles involved change state as a result of the actions and interactions that occur). This is perhaps much more appropriate for the state based view he was hankering after. The best reference on this is Martyn Ould’s book “Business Process Management – A Rigorous Approach”

But all should understand that Case Handling approaches have been around for a very long time. They are everywhere you look once you get it in your head. Think of these:

• Government – State and local government, NGOs, Police, Justice (investigations), Land mgt …
• Financial Services
• Insurance – Every claim is an exception
• Banking – Trade exception handling, premium account management
• Healthcare – From clinical provision to administrative management and payment
• Oil & Gas Exploration- Knowledge workers spread thinlyaround the world
• Pharmaceuticals – Clinical trials, compound development, marketing campaign management
• Virtually all “professions
• Wide range of Small to Medium sized contexts
• All sort of Procurement situations
• Customer Contact Centers – across virtually all industries, where they validate, identify work items and then resolve … here 80% of all calls are WISMO (What Is the Status of My Order)
• Even the weekly Staff Meeting is a kind of case handling situation if you look at it from a process point of view.

All of them have continually unfolding, evolving scenarios. That is where BPM needs to concentrate its efforts. The transactional space that has characterised efforts to date is really pretty straight forward. Case Handling involves synchronous interaction with users, long running case resolution situations, multiple process fragments, knowledge work, …

Interesting vendors in this space are few and far between. At one level it is big systems implementations such as Cordys, Pega and Graham Technology. But there is a simpler more accessible level that is best characterised by folks like Itensil (in my mind one of the most itneresting I have come across). I am sure, that with care you could implement TIBCO, Appian and Lombardi to build effective Case Handling situations, but it is really a quesiton of adopting the right style of design thinking. And with more and more of these vendors offering SaaS delivery mechanisms, I think we are going to see an ever increasing level of innovation in this area.

Loosing Touch With Reality

In a recent posting Get Your BPMN Schema Today, Ismael Ghalimi shows how close he has come to loosing the plot. He has certainly moved well away from the reality of standards since he formed BPMI.org with Howard Smith.

Everything has become an element of spin – even down to misrepresenting the article of Bruce Silver. He suggests that Bruce thought “BPDM is essentially useless, and that the BPM industry badly needs an XML schema for BPMN.” While Bruce did talk to the need for robust interchange formats, I didn’t see him say it was essentially useless (although Bruce did demonstrate that he doesn’t really understand software, schemas or the semantics of process … but that is another subject).

Ismael went on …

BPDM, currently developed by the OMG, is presented by its promoters as a universal interchange format for business processes. In reality, it’s an utterly confusing metamodel that brings very little to the table, and is used by legacy workflow vendors as a way to fight against the upcoming dominance of the BPMN+BPEL standard combo. Workflow vendors like nothing more than to protect their respective little niches, and have done a phenomenal job over the past fifteen years of ensuring that no useful standard would ever be developed in the space.

Normally, I wouldn’t bother to comment on such clearly biased diatribe. But as many people never get to read the comments, I thought I should post the carefully worded rebuke that came from Cory Casenove (on of the main authors of BPDM):

We were active in the BPDM specification and feel that some of the views presented here are not well founded and others are out of perspective.

The comments from the author about “legacy workflow vendors as a way to fight against the upcoming dominance of the BPMN+BPEL” is, frankly nonsense both politically and technically. The dominant forces in producing BPDM were BPMN vendors, business and process modeling practitioners, business/process modeling vendors and government representatives – not “legacy workflow vendors” at all. There is no plot and no fight, BPDM fully embraces and supports BPMN and provides added value as we will discuss below. Technically BPDM is a superset of BPMN with added support for SOA – far from the centrally managed style of the old workflow systems. Perhaps we can keep our debate at a higher level.

The observation that BPDM is complex to understand has more merit – but lets look at the context and why;

Process semantics: Some other approaches to process “exchange formats” are little more than pictures in XML. While this provides interchange of the diagram it fails to exchange a process specification. One of the reasons that BPDM took so long is that nailing down what is really being said in process models is harder than it may appear, the team had the advantage of being able to work with the BPMN team directly, to iron out the real semantics – and there were a lot of arguments and disagreement about what they really meant. BPDM captures the result of this and provides a way to exchange the process semantics, not fuzzy pictures or partial thoughts. Semantics like this does take more time to grasp – but the result is REAL sharing of process, not lock-in to different vendor’s interpretations and extensions.

Integrating process and SOA: Process orchestration is a valuable tool in the enterprise toolbox, but not the only one. Process is part of the business model that includes services, organizations and resources. The SOA perspective in particular has been fully integrated so that the process for doing and the services offered and used are well integrated into a coherent enterprise perspective. BPMN can’t fight SOA, it needs to join it. Orchestration and SOA are just different ways to look at process. BPDM provides a bridge between those views. That bridge can be ignored by those who only want to understand orchestration, but we don’t think it will be ignored by the users who need both.

So this additional “complexity” of actually understanding what processes mean and how they relate to services and other processes has an impact, but that impact is primarily the responsibility of the builders of process tools, frameworks and infrastructures – perhaps having them take the time to understand the semantics behind their diagrams and runtimes is a good thing, I’m sure they are more than capable of grasping it.

There will be an open source Eclipse based implementation of BPDM on ModelDriven.org/bpdm as well as a schema for interchange – those who just want to know how to exchange BPMN models can use this, without the complexity. Perhaps we can also use this forum for an informed debate on the best way to represent process and provide the best approach to our communities and organizations.

Cory Casanave

Model Driven Solutions (www.modeldriven.com )

Business Process and Religion – Evangelism – Yes; Fundamentalism – No

I see I am going to have to dedicate more time to getting the blog out … but when I look at the broad range of tasks I have to get completed in the next 2 weeks I am back at the office – God knows where I will find the time.

Over the last 20 years I have found myself trying to help a broad range of people understand the various vagaries and wrinkes of business processes. But I have found a real difference in the receptivity between those people who know very little (and want to learn) those who think they know a bit (and want to be impressed). When introduced to a new modelling technique or approach, the common reaction is “why would I want to do it like that, I can always use <whatever technique I know already> to model that. What they seldom consider is what that new technique or approach might do for them, or how it might give them another subtle perspective.

So perhaps you will understand me when I say Business Process is a little bit like Religion – once people have been inured in one branch of the church, they tend to resist attempts by others to engage them (just think about Protestants and Catholics for a second – they have a lot more in common than they realize, yet they still seem to find each other repulsive). And the world of process is not that different. There are a lot of parallels between the differing factions of the business process movement, and those that one can observe in religion.

If you have been trained in UML, then that is what you want to use to model (everything must fit into that UML metamodel); if you grew up with IDEF, then all models appear as though they should be constructed around Inputs, Outputs, Controls and Mechanisms (or some other similar flavor). If Rummler Brache was your thang, then you favor the deployment flowcharts and swimlanes associated with the technique. Whether you have been brought up on LOVEM, BPMN or simple Fortran flowcharts, then the world is often colored by your original christening. It is only when you have been around for a while that you can see the benefits of the different approaches.

Putting all of that slightly different way, and maybe I am stating the obvious – a little knowledge can be dangerous. And in the world of business process, that is certainly the case.

As people search around for the meaning of life (or process) they discover different disciplines (new techniques and approaches). Sometimes, they become converted to a new religeon (say Pi Calculus, or conversational interaction loops of ActionWorflow) and feel it incumbent on themselves to act as missionaries, recruiting new sheep to the fold. Those that dont agree with them are clearly wrong, or misguided, or even worse, seditious. I suppose the point I am making is that while every branch of religion needs its evangelists, fundamentalism tends to alienate potential parishioners. And the problem with religion is that, for most people, once they have got some of it, they tend to shun all other approaches.

So by now, I hope you understand that I see the world of business processes as a pretty broad church. Personally, I am keenly interested in all process related innovation. But I don’t see that as a restrictive covenant that stops me from looking at, or even trying to explain, approaches that do not conform to some purists definition.

Indeed, I believe that it is only when you contrast different perspectives on a business process that you really understand it. You need to be able to step outside the box and see it for what it is. You need to be able to examine the interactions on one hand, and then flip it all around to look at the sequence flows; to look at what is required of the process and separate that from how it is achieved. With luck, the new BPDM metamodel from the OMG will enable the analyst to step around these different perspectives, sharing information between different modeling tools and techniques, without loosing the fidelity of the information.

A Process “A Ha” Moment – Two sides of the same coin – Orchestration and Choreography (and BPDM)

I was sitting in the Delphi BPIS Summit a couple of weeks back contemplating my closing keynote address and not completely listening to the speaker (Kevin Fickenscher) at the time who was doing a “call to arms” type of piece about how the world was changing (as if we didn’t know).

I shouldn’t sound disparaging, as the talk was actually very interesting. At one point he started to describe a concept he called “Simultaneity” (is there such a word I was thinking … having looked it up since, I can tell you there is).

He then went on to say that in Perot Systems they have literally thousands of associates working on a single client’s problem (system) at once. Really, what he was getting at was how things in the world often need to happen all at once, in parallel.

And it started a chain of thought – so how do you manage that (sort of problem). Or if you are say, Shell or Nokia with literally hundreds of different functions that need to collaborate on major projects; or indeed any large business attempting to deliver a consistent customer experience … just how do you manage that.

Ahhhh – Process. You need processes to help coordinate the various resources at your disposal. Of course!!

But the problem is that most people still think of process in terms of input output chains of activities borrowed from manufacturing. That notion of process doesn’t help you too much when you are talking about the coordination of multiple actors, operating in parallel.

And that chain of actors is increasingly up and down the value chain, no longer constrained to the familiar boundaries of the organization where command and control principles can apply.

What you need to do (as well) is to start thinking about processes as sets of “interactions.” This is the other side of the coin from the procedural notions of process that we are all so familiar with. Instead of solely focusing on the chains of activities with their (hopefully neat) sets of input-output documents, you also need to think about the protocols and agreements made between the actors. They represent the white space in between the functional silos, the hand-offs and commitments made from one party to another. Indeed, one could start to think about how a process is composed from fragments of interaction.

Having understood the interface requirements for your process, you can start to explore how your part of the business (value chain) wants to implement against that set of needs.

Really we are talking here about the difference between “Process Orchestration” and “Process Choreography”:

• Orchestration is a term used to describe the traditional notion of managed or directed execution, where activities are carried out, with branching and synchronization of different threads.
• Choreography is a more abstract notion of process (for most people). It is used to describe the “Interactions” of collaborating entities, each of which may have their own internal orchestration processes.

And this is where the emerging business process standards are starting to come to the rescue. The OMG “Business Process Definition Metamodel” (BPDM) provides the capability to represent the semantics of business process models, independently of the modeling notation. Think of it as a sort of “universal syntax” of process. The central idea is that BPDM is capable of supporting a mapping to the semantics of most common types of process model – whether that is BPMN on one side and Role Activity Diagrams or UML models on the other. As a result it enables the robust exchange of information between models of one type and other types of process models, while preserving the fidelity of the model content (as much as is possible).

At the heart of BPDM is support for both orchestration and choreography. This is going to become critical to modern organizations as the business boundaries change ever more quickly. It is self evident that businesses are constantly reorganizing, merging and splitting their operations, resulting in continuous changes in both the internal boundaries, and that which what owned or outsourced to both collaborators (in the supply chain) and the customer. Yet there is still a common process operating across the value chain. BPDM facilitates this evolving, permeable boundary by identifying and managing the common elements, enabling the modeler to easily transition between them.

Because of this sophistication, BPDM can support a very wide range of usage scenarios. For example, it enables high level, abstract “business capability models” used in the boardroom, to exchange information with lower level, procedural modeling notations used in BPM projects, and then on into process execution environments (BPM Suites).

BPDM will also facilitate the definition of complex inter-company interactions (and between the roles within the company). Such models could be created from scratch, or created through the composition of existing interaction fragments (from a library). The resulting model might then provide the terms of reference for each participants’ own internal orchestration model. In turn, this orchestration model could be teased apart to explore the business interactions that exist between the internal participants (internal roles) and/or used to generate a robust BPEL execution model that directly supported the firms agreed choreography with the other companies.

In this way, the process would not need to be completely ripped apart and redrawn every time there was a change of organizational responsibility. Roles (areas of responsibility) and their interactions could be modeled in an abstract way, then instantiated as needed to meet the requirements of the case in hand.

While most might think of BPMN collaboration diagrams to model interactions, a better approach is to use Role Activity Diagrams which focus on the interactions themselves and how a role changes state as a result of the actions and interactions that occur. So assuming we had tools that supported the BPDM package for BPMN and another tool that had mapped RADs to BPDM, it would be possible to translate between the two with clarity and fidelity. (BTW – the best reference on RADs is Martyn Ould’s book “Business Process Management – A Rigorous Approach”

So what does this mean in the real world of organizations like Perot Systems, Nokia or Shell – that business process standards, such as BPDM, will make it easier to model (and therefore understand) the simultaneity in a world where interaction has become as important as procedures (in some cases or more). It also makes it more straight forward to drive those processes using BPM Suites and report on the performance of the individuals and teams involved.