Breakthrough Process Design

August 3, 2009

We find ourselves presented with situations where our clients are looking for “Breakthrough Performance” rather than mere “Process Improvement”. In her guest post on Jim Sinur’s blog, Elise Olding points out that many rush headlong into implementation looking for an elusive Magic Bullet (there aren’t any, but you could describe BPM initiatives as a Golden Gun). She quickly alluded to the need for a number of techniques including “Process Walkthroughs” (following the work item), through Sticky Notes or PostIt Sessions, down to and including sitting with the users and observing what they do.

While all these techniques are useful and interesting, they don’t go far enough to deliver the breakthrough improvements that customers seek. In business today, generally what is needed is not “Better Sameness” but “Transformation”. But existing processes usually focus on the needs of the company – delivering stronger management control, and reinforcing functional priorities.  

However, we believe that the best practice to deliver breakthroughs is quite different from any of these approaches. We start from a different place – the “Customer Experience” – a stance that is all about building competitive advantage. 

Process breakthroughs come from thinking about everything we do in terms of how what we do can assist in delivering a Great Customer Experience. This dynamic lens is dramatically different from the traditional process improvement approach. It has the effect of inspiring completely different insights and generating new ways of doing things (rather than paving the cow paths). Once you are standing in the shoes of the customer, you no longer see the functional bias that reinforces existing behaviors.

In the early 90s the CEO of Sony pointed out that, every manufacturer had all the parts needed for a Sony Walkman sitting on their shelves – but only Sony asked the customer what they wanted. As a result, they transformed the way we listen to music (even if they did miss the disruptive innovation of hard disk based players).

In the end, we believe that you have to build a “Transformational Vision” around what the customer values – be it an internal customer, supplier customer, partner customer or end consumer. So we will continue to use the “Customer Experience” lens as the best route to achieving breakthroughs in process performance.


Cases Managed The World Over

June 21, 2009

A recent spate of interest in Case Management is good to see (I always called it Case Handling but the concepts are the same). The OMG is about to vote on an RFP (Request for Proposal for a new standard) on Case Management. As some of my regular readers will realize, I have a special interest in the subject of Case Management.

Some of us have been talking about the problems of dealing with Cases for a number of years (e.g. this post in 2007). My own experience started with the development of what would today be called Case Management systems starting in the mid-80s and culminating in an Oracle-based object oriented repository in the early 90s – way too clever by half for that era so I canned it in 1992 and became an Analyst looking at other peoples products, writing white papers, etc. Since then I put together a number of white papers specifically talking about the issues of Case Management:

  • In 1996 a paper called “The Business Case for Case Handling” and although the vendors I referenced at the time have since disappeared (bought out), the issues are just as relevant today.
  • Over the last few years, I published a couple of papers that start to explore some of the related issues (these papers are available on BPM Focus with registration). In particular, these two papers address many of core concepts of Case Management.
    • “Process Innovation and Corporate Agility – Balancing Efficiency and Adaptability in a Knowledge-Centric World”
    • “Business Processes and Customers – Difficult Domains to Integrate”

So with this post, I am having my own stab at defining the issue. I have been invited several times by those in the OMG to take part in this particular enquiry, but hesitate to get involved as these things can act as an enormous time sink. So first let me point you at some other perspectives. In recent months, we have seen several bloggers discussing some of these ideas (touching on the need for adaptability and agility):

  • Jim Sinur of Gartner talks of “… Agile processes that are tapped into emerging events and contexts driven by organizational and community goals … the need for creating and managing unstructured processes. This kind of environment requires organizations and vendors to master goal driven processes.” In another post he said “Today most processes are Flow directed, but the future will likely require goal direction for at least a portion of the process. This is what we call unstructured processes that are composed of process snippets that are flow directed and portions that are completely dynamic. A combo looks to be the way forward.” See here, here and here.
  • Neeli Basanth Kumar (of Cordys) talks of Process Patterns in Adopting Case Based Solutions (he even uses one of my diagrams from a 97 paper – The Workware Evaluation Framework … where I tried to highlight the role of Case Management).
  • A discussion paper put out by Dennis Byron at ebizQ provides a sort of summary of some of the thinking of the vendors that replied to his request for information (originally it contained references to my thoughts and some of my graphics but that content was pulled after I pointed out the provenance). 
  • Bruce Silver commented on the RFP discussion going on at the OMG and postulated what he sees as the difference between “traditional BPM” and “Case Management”.

For me, it all comes back to the continuum of Process. On the one hand, we have the image of the organization as machine, with mechanistic “Procedures” used to control the work of the resources available. Most BPM initiatives are still stuck at this level, seeking to automate things and remove (human) resources from the equation. If Productivity = Value / Resources, this reductionist approach is all about reducing the resources involved in the deliver of a given value.

At the other end of the spectrum, we could consider processes as more like evolving “Practices.”  Think of what you do personally and see if this concept makes sense – some parts of what you do are defined in Standard Operating Procedures, other parts you interpret and apply your judgment. The more leeway you have to make decisions (in your job), the more knowledge you exercise in carrying out your job. Most knowledge workers are goal oriented, regarding procedures as a means to an end, rather than an end in themselves. Managers tend to be goal oriented.

We could think of high level processes as being about a “Purpose” – and how that Purpose is interpreted will inevitably be somewhere on that spectrum Procedure and Practice. Indeed, one finds that most business problems need a combination of both – hence the approach that has become know as Case Management. Now we’ve got this concept established, from a process perspective you could think of Case Management as applying to the Practices end of the spectrum. Workers here are goal oriented, and typically apply processes to achieve those goals.

Case Management is a very important “Design Pattern” for supporting flexible work Practices instead of more rigorous Procedures (where no adaptability or run time flexibility is needed). Different products take vastly different approaches to Case Management, all with the aim of providing flexibility and adaptability to the user, yet still providing support for the organizational objectives (processing more work, more efficiently and providing traceability).

Case Management proffers a way of mixing overarching support for that Purpose – normally through a high-level, outline procedure, which is then supported by a library of process fragments that can be bound into the parent as determined by the user. In some cases, the user has complete control of what should happen next; in others, the ability to progress from one phase of that high-level outline to the next is constrained in some way. Some products leave it very loosey goosey, others are all about constraining the user. Depending on how strong the need for adaptability is (in the target domain) the user may even have the ability to develop new process fragments to support a given need (imagine a Software Engineering project … it’s not always possible to predict every possible permutation). In others (say a Bank for example), it might be good enough to have the user select from a library of available sub-processes to bind to the parent. Indeed, in a Bank or Insurance company, it is unlikely that the run time adaptability would be allowed (the last thing you need is a clerk getting creative with a bank draft).

With careful architectural design, it is possible to create a Case Management environment out of many different BPMS products. But that implies that the end-user organization already have a clear idea of how such environments are constructed. In a sense, they create an application layer above the BPM Suite. 

My concern with the OMG RFP is that it is trying to standardize something that is poorly understood (as evidenced by the varying perspectives given in the OMG BMI mailing list). 

While the rest of this document goes on to outline my own views on Case Management, I believe that developing a standard in this area at this point would only result in hampering innovation. Having said that – there is a definite need for much more discussion and exploration of the domain of Case Management.

I believe that the approach proffered by Cordys represents just one way of approaching Case Management. There are others, and I do not believe that tying everyone down to one interpretation of Case Management at this point will be a good thing in the industry. Other vendors with Case Management approaches include:

  • Singularity
  • Cordys
  • Global 360
  • Sword (was Graham Technology)
  • Itensil
  • TIBCO
  • EMC Documentum
  • IBM (FileNet)
  • BizAgi
  • Pallas Athena
  • Pega
  • Polymita
  • HandySoft

All these vendors have some sort of capability that could be described as Case Management (and I am sure there are plenty of others that would put themselves on the list).

Finally those in the Process technology world are starting to see that a pure “one sized fits all approach” to the standardization of process definitions is entirely inappropriate when it comes to the needs of humans and knowledge workers. Moreover, Case Management approaches provide all sorts of benefits to companies in that they enable a far more flexible response to the needs of customers.

As different vendors struggle to work out the best approach, the last thing they need is to be tied down to a “standard” approach. In the end, it will be the Darwinian process of selection that will see the best products win out; not some imagined need for standardization and interoperation between wholly different approaches to the problem. 

Notes

Most BPM efforts could be characterized by their incessant focus on process standardization. They are predicated on the assumption that overall business effectiveness improves through better control. And while this is true for procedural, back-office problems, the reality is that customer facing and knowledge work processes are extremely difficult to standardize (if not impossible).

This is a real problem for long term BPM adoption. Ask yourself how many organizations you know in the BPM arena that have more than 5 or 10 processes “under management” (i.e. using a BPM Suite to ensure that things don’t fall through the cracks). And then think about how many spreadsheet are used in those same organizations to coordinate work.

In the paper “Customers and Business Processes – Difficult Domains to Integrate” I suggested that there were several different types of Case Management (Case Handling). These range from the traditional BPM Suite (which struggles to support the necessary adaptability), through what I called “Design Time Case Handling” on to “Run Time Case Handling”.

Case Management and BPM

Case Management and BPM

The vast majority of BPM Suites and Workflow tools assume that all activities/tasks/steps (and the potential paths through them), are modeled a priori (beforehand). Putting that another way, they focus on driving work between employees based on a model that maintains the status of a work item. The process model must exist up front, which presents the first hurdle of process discovery—i.e. ensuring those models are “correct.”

Further, in most products, all work of a given type share a common process description (rather than copying the model to support each work item). In such situations, the engine will normally allow the end-user little or no choice but to follow the pre-defined procedure.

Of course, the challenge is then for process modelers to predict all possible permutations in advance—something that is virtually impossible to achieve in customer facing situations. To get around show stopping scenarios, a few products incorporate features that provide the ability to bypass, redo, and rollback steps, while most rely on re-assignment of work to the supervisor (who must then step outside the system to resolve the problem). It does not take long before the supervisor becomes the bottleneck as cases mount up (those that do not follow the “happy path”).

Change is only possible through re-development of the common process model. New items of work then follow the modified process description (most products incorporate some form of version control). Change to an individual work item normally requires the deletion of all threads of work and the work item is then recreated under the new model (compromising any future audit). Alternatively, mechanisms must are needed to move an existing case to the new model.

These adaptability issues are not constrained to customer facing scenarios. For example, as government regulations change, the firm needs to revamp its process models to handle that change. There might be thousands of cases in the system, the vast majority of which will complete before the new regulations come into force. But imagine that there are still 100 cases outstanding at the point the new regulation comes into effect. For most products, it would simply be impossible for them to handle this problem in any sort of constructive fashion. Each of those work items would have to be manually stopped, and then restarted (somehow) under the new process definition that met the new government regulations. The only viable way of approaching the problem is to incorporate mechanisms to migrate individual instances to the new model.

For Case Handling support, the key differentiating factor (of the BPM Suite) is the ability to link multiple processes to a given case of work—the primacy is with the case of work, rather than the processes that are used to support it. Each case is usually “managed” by a relatively loose (high-level) parent procedure, but the worker can then add new procedural fragments to handle each different requirement of the work in hand. Effectively, the user is binding new procedural fragments to the case at run time; either by selecting them from a library, or by developing new ones.

Of course, this sort of approach is reliant on a BPMS that can facilitate such modifications to work in flight. For most products, it will also require great care in the design of the process architecture itself, and may involve the development of an external application.

(Some of these thoughts have been culled from my past White Papers on the subject of Case Handling)


Process Portfolio Analysis Webinar – in 2 hours

February 3, 2009

I have been ignoring the blog while I have been trying to get the book finished. What a struggle – somewhat akin to passing yourself and all your works through the eye of a needle. But it is getting there.

Anyway, I shoudl have posted here the details of the upcoming webinar we are staging with the Value Chain Group.

Managing The Roadmap – Process Portfolio Analysis

Register here


BPM – Is it a Software Engineering Tool? A Technology? or a Management Discipline?

November 30, 2008

In his excellent posting, Keith Swenson makesmany good points. He points to the range of interpretations of BPM, and particularly highlights the issues associated with its interpretation by software engineers as just another piece of hype on the road to good programs. But I think there is another, perhaps more important strand that is buried in there. As Ketih points out, BPM is about the Management of Business Processes.

As we all know, everyone’s interpretation of the term Business Process is different. In my training (whether that be BPMN, or higher level training on BPM Methods), it is one of the first things I get people to write down (inside the first 5 mins), and not surprise, every definition is entirely different. And when those people are senior managers in a ciompany, their interpretation of the term is invariably what I call “Process as Purpose“. The point is that they see Processes as being more about the purpose than the constraint implied by sequencing of steps. They are there (at the training), because they see the importance of “Managing” their processes. Indeed that concept (Managing Business Processes) as central to the success of their companies.

[[ I am still down in Brazil, and I am really struck by the process sophistication of the people I am meeting. They all get it. I was more than surprised to find two “Business Owners” (people who own significant businesses), giving up 3 days of their time to come on a course around how to structure and run BPM programs. They were cherry picking from the broad range of techniques we covered, but ask yourself whether you could imagine the CEO or COO deciding that they should attend a public training course. That’s what I am getting at about the sophistication of the Brazilian business climate. ]]

Coming back to Keith’s post, he describes a spectrum of BPM interpretation – from pure Software Engineering (where the SW Eng tries to reflect the needs of the business person’s Business Process); through Business Processes being modeled by a business person, then tossed over the fence to a Software Engineer to finish them off; to the Business Process as modeled by the business person, then being directly executed (what he called “Pure BPM”). I am not quite sure I agree with the Pure BPM bit, but I do know what he is pointing to … where the processes of the firm are driven by models (without translation to some intermediate executable format (like say BPEL).

One of the comments on Keith’s post points to the challenges of getting business people to model their own processes and make the resulting collection of stuff useful. He described the usually resulting mess as an “expensive disaster”. And the reason for this is that business people dont usually have the sophistication to understand their business problem as a set of inter-releated processes that between them deliver on the “Process as Purpose” concept I referred to earlier.

Invariably, process modelers (whether IT or business) tend to see a process problem as a single process. They interpret the high level Purpose as a single implementation process (which invariably it is not). They make all sorts of mistakes such as mixing up the “Handle an Instance” with the “Manage the Flow of instances”; they switch from batch mode to handling a single instance; they dont think about the interfaces between processes (handing information from one to another), etc. What they do is try and connect up everything that sounds like an Activity into one convoluted process.

Now software engineers are usually more adept at the necessary abstract thinking, but that doesn’t mean to say that business people cannot wrap their pretty heads around the notions. It is merely a reflection of the fact that they have not had adequate training. What is missing (across this entire industry) is better learning around “Process Architecture” – what “chunks” do you need and why. Poor chunking leads to unnecessary complexity (and even “expensive disasters”).

We are still stuck with decomposition as the prevailing mind set – where sub-processes are always contained within the parent. SOA concepts seek to get around this, but there is also a higher level “Business Services Oriented Architecture”. Processes should not be regarded as some sort of static hierarchy, they are more accurately regarded as a network of interacting instances. Think more jigsaw puzzle than org chart.

When I gave a “Power Breakfast” at the last Gartner BPM Summit on BPM and Process Architecture I had a packed room (it was starting at 7:30 in the morning so these people were keen). I described a set of methods that you could use to go from “What Business Are You In” to what “Processes Do You Need” right down to the SOA components if that is what you wanted to do (I would recommend looking at a BPM Suite first rather than going straight to the SOA software engineers paradise). I only saw one person out of the 90 or so get up and leave, and nearly everyone else gave me their card at the end of it. The room really was comprised of mostly Enterprise Architecture folks from the IT community, all of whom struggle with this transition.

Switching tacks – the vendors BPM Suites are unconciously making this architecture problem worse. With only a few exceptions (Pega, Itensil, BizAgi … I am sure there are others in this category too, these are the ones that spring to mind), vendors interpret the business process problem as being entirely seperate from the data and artifacts associated with the process (business people see them as intertwined). They regard the process relevant data as a set of named value pairs … the information required by Process A is declared on Process A, and must be recreated on Process B and then mapped from one to the other (if the processes need to communicate with each other). That means that there is an extra (unnecessary) layer of complexity for business people trying to reflect their business problem. Moreover, if you change one process, then you need to refactor all the interfaces. This is “software engineering” oriented thinking.

The other approach is to define your data structures (perhaps as an “Entity” defined as an exstensible set of XML artifacts) and then describe the views on those artifacts at the level of the data structure. Then it is merely a matter of associating your processes with the Data Entity, and all the different views become available. Process interfaces become an order of magnitude more accessible (to the business user), you can use any number of processes to support a single case of work, and again it helps move away from the software engineering mindset we find in so many BPM tools (which were often created to solve the problem of Enterprise Applicatin Integration … hence their association with Software Engineering).


Gartner BPM Summit Report

September 14, 2008

Well the book (BPMN Modeling and Reference Guide) is out, and we sold out at the Gartner BPM Sumit last week (we launched the book there). People would come up, thumb through it, think about it perhaps but then buy a copy. Several of them came back an hour or so later and bought more copies … with phrases like “this is actually designed for people to read …”, “the rest of my team need to read this …” and “I must get my boss a copy of this.” And then some of the vendors popped their heads above the parapet – one of the larger names in this space is now talking about buying 2000 copies (one for each member of staff I guess), and two others walked off with half a dozen copies for their colleagues. So all up, we were pretty pleased.

My “BPM and Modeling” track session was well attended (a couple of hundred delegates), and only 3 people fell asleep. Of course, it was directly after lunch, and modeling is not exactly the most rivetting of topics, although I did try to make it as interactive as possible (given the situation). I reiterated a central tenet of my approach to process modeling – in the early stages of a BPM initiative it is important to contrast modeling approaches to drive understanding (rather than slavishly following just one technique such as BPMN). The key point is that you need to be able to change your perspective … to see things differently if you want to really understand the process. Of course, when it comes to implementing a process using on a BPM Suite, then you need to resort to BPMN to get clarity into the execution model.

On Friday morning, I filled the (relatively small) room at the “BPM and Process Architecture” power breakfast, which was always going to be a difficult task given that the night before was the vendor parties (free booze). Anyway, several people complemented on an interesting session afterward. I had tried to communicate the set of methods we use to move from thinking about Strategy to through to implementation. This session included an overview of techniques to support the clarification and definition of:

  • What business are you really in, and what business services you need to support that vision.
  • From that, what is an effective Process Architecture -i.e. what processes do you need to support those business services and how do they communicate with each other, etc. (referencing the RIVA Process Architecture method).
  • And from there, how do you get to the SOA-based IT Services that are needed to support those business processes (i.e. how they are implemented). What service interfaces are required for each component, etc.

The problem is that, when it comes to process architecture there are very few reliable approaches … certainly functional decomposition falls down a hole here. Most rely on what I call the “black art” approach. They make the mistake of linking Process Architecture to the current mechanisms for apportioning blame (the organizational chat) – i.e. the Process Architecture should be independent of the organizational structure. If you change the way the departments and business units are structured, this shouldn’t force a refactoring of the processes.

I ended the session with a short overview of Case Handling approaches … an area that I feel is poorly understood (especially by vendors who have little incentive to change the status quo). Case Handling is really a design pattern enabling a balance between control and adaptation (efficiency and flexibility), where users are left in control, yet the organization can still provide support for the vagaries of customer interaction (see “Customers and Business Processes – Difficult Domains to Integrate” on page 2 of the papers available on the BPM Focus web site for more detail there).

I attended a few other sessions, but found new insights a bit few and far between (for me that is, too educated I guess). I only caught the last third of the opening keynote (I was told I didnt miss much earlier), and found myself taking notes (as much as anything on the language used and the phraseology for concepts). I sat through an entire session on Customer Interaction and BPM (which I felt missed the mark entirely … but still managed to salvage a few points here and there). I enjoyed Dan Roam’s session (keynote on day 2) on modeling on the back of a napkin (using simple rich pictures to communicate). I felt that the session on BPM and SaaS (Michelle Cantara) didn’t go far enough (stopping well short of talking about the implications of the cloud on the way work will happen in the future). But there were other sessions on that (competed with my modeling session), so no doubt they got more into that aspect then.

The Vendor Showroom floor was the usual zoo – with people wandering on and off stands or generally ignoring the vendors while they consumed their deserts or coffee. This format just doesnt work for either the vendors or the delegates who want information on what the products really do. Contrast that with the BPM Technology Showcase we are running on October 14-16 in DC. The last one of those in Nashville was a roaring success as people got up close and personal with what the technology can really do for you.

Otherwise, I seemed to be either signing books or getting button-holed by folks in the hallway.


Busy Busy Busy … Books On The Way

July 31, 2008

Another litany of excuses. Anyone who has looked at this blog over the last year or so will see the very poor track record I have established in terms of keeping it up to date. The reason – just not enough hours in the day. But that is probably true of all bloggers, but I wonder how many of them have been pushing the development of two books, along side a full schedule of training and consulting activities. Right now I am supposed to be outside on holiday with my family, but instead I am deep into the last push to finalize the content for the forthcoming BPMN Modeling and Reference Guide.

This has been a mamoth effort over the last few months involving detailed collaboration with Stephen White, the main author of the BPMN specification. We have done our best to make it as readable and accessible as possible, separating out the detailed reference section from a piece about modeling in general (everything from history of BPMN, why/how to model, etc).

There is also an extended scenario based introduction to BPMN functionality, bringing in new BPMN functionality in the context of an easily understood business problem. Throughout the section, the business scenario is elaborated upon and the corresponding models and BPMN functionality explained. Through our training courses, we have found that people learn far better this way.

Of course, there is a detailed explanation of all BPMN functionality. And for most, who are actively involved in modeling, this reference material is sorely needed. For the layman, the specification is somewhat hard to follow (and that is being kind). In the book, we explore each area of functionality and provide a detailed explanation for its use, and behavior.

The BPMN Modeling and Reference Guide will be launched at the Gartner BPM Summit in Washington DC.I am doing a session on BPM and Modeling and another on Developing Appropriate Process Architecures. Neither of these sessions are designed for beginners (although the modeling session should be pretty acessible.

Incidentally, I notice that I am the only non-Gartner BPM Analyst/Comentator presenting at the conference this time around – seems that the same old, same old’s have finally been found out ;-).

Book 2

My other book, that has been in devleopment for the last 5 years or so is in the later stages of finalization. Mastering BPM has been an evolving piece that will probably hit the presses in a similar time frame. I still have a chapter to write, but it is just about there.

Of course, all of this book development work takes cycles out of the day and impacts the ability to execute on other things. Anyway … I hope to get into a more regular pattern of blog postings and updates by the end of next month.


BPM and Knowledge Work – Looking for Proof of Concept Partners

June 24, 2008

I am looking to prove that Business Process Management technology can really change team productivity, especially for teams that span diverse organizations. We will choose a half a dozen knowledge-centric organizations, and then build out a proof of concept or initial application for each of them that they can then take forward, deriving long term value and changing the rules of competition in their industry.

This will appeal to organizations in the following situations:

  • A consulting group looking to package and deliver their expertise via a robust, service-based model that creates a leave-behind environment and an extended client engagement.
  • Start-ups developing a knowledge-based product offering, seeking a comprehensive Proof of Concept to close the first round of customers.
  • Corporate teams focused on Project delivery or Product Launch, needing to capture their project/product artifacts and support evolving business practices.

A Short History

Business Process Management has been taking great strides—moving from the highly structured transactional procedures into the realms of ever more human-centric, evolving and adapting processes. Instead of limiting processes to the confines of IT (as the BPEL guys are trying to do), some vendors have been exploring providing better support for the roughly repeatable business practices—rather than always following a standard pattern, processes unfold and many fragments of process might come together to support an evolving business scenario.

But achieving a balance between the explicit world of processes with an effective collaboration and knowledge capture environment is no easy feat either. And all the while, ensuring the user interface is accessible to the average goal-focused knowledge worker.

Now, I believe I have found such an environment, one that can enable efficiency and effectiveness in knowledge work – all delivered as a service online (i.e., no hardware or software charges).

What You’ll Get

The organizations I select will receive a ready to deploy (internally or for resale), process-based web application. You will need to have your own IP (method) and some of the supporting documentation. I will personally lead the development, working with your organization to really capture and make explicit the processes and areas for evolution. I will build out the initial data structures and a set of processes that support a reasonable application. This should take just a few weeks (30 days at most) and it will include a full year of hosted service with a generous number of seats—and all of this for what you would pay for typical BPM product evaluation.

In the short term, please respond to me directly (miers at bpmfocus.org). Tell me a little bit about your company and what you are looking to achieve and we can start a dialogue as to whether you fit into my research initiative. Following up our work, I will expect to develop a set of short case studies, so please be prepared for a bit of blatant promotion of your concept going afterward.