On Semantic Technologies
A conversation with Eddy Vanderlinden
Semantic technologies have been some temptation for me for quite a long time. That was mainly due to my growing frustration about the utilisation of data resources both inside corporations and outside, on the Internet. Then all mainstream modelling methods used for analysis or for database design and application development, with all their charms and weaknesses, often leave me with the feeling that they put too much constraints for people to express things and too less for computers to understand them. That and the suspicion about the potential of Semantic Technologies is not new to me. What is new, is the experience of their pragmatic application and the opportunity to see some all too familiar areas through ST lenses. Both of these I owe to Eddy Vanderlinden. In this sort of interview, I asked him a few questions the answers to which might be of interest to the readership of this blog.
Ivo: When was you first meeting with ST?
Eddy: In 2007, when searching for possibilities to avoid the registered dysfunctions.
Ivo: What was the most fascinating thing for you in beginning and how did this change? I mean, which part or capability of the ST is the main driver for you now, after so many years of practice?
Eddy: The most fascinating aspect was the data modelling aspect, so that data became real information. The upgraded function for data towards information covered the common and precise understanding for all stakeholders of concepts through their relation with other concepts. Also the flexibility of the data model was contributing to the great benefit.
There are two main drivers added later on:
1. The knowledge discovery possibilities through the open-world assumption. Under the condition we adapt our state of mind from comfortably categorising "things" into a state whereby we discover through the ST technique new aspects. In philosophical terms, we should become a bit Tao: flexible, accept change as the only certainty and being attentive to capture change and utilise it for our benefit.
2. The possibility to convert knowledge models straight into running applications so that unambiguous goals are obtained through commonly understood and mastered methods.
Ivo: There are people over-enthusiastic about Semantic Web, calling it "Web 3.0", "the next big wave", "the gigantic …. graph" and so forth. At the same time there are many sceptics and even people that had been once enthusiastic about it, and now seem rather disappointed. What do you think about this? Is ST rather overrated or undervalued or somehow both? And why?
Eddy: ST is to my understanding by far undervalued. Lets see first why one could be disappointed. One reason might be because lots of people betted on the possibility that web publishers would start to use html tagging techniques massively so that slightly adapted search engines could simply exploit the semantics in the text. So, the search engines would not only have keywords (from the html header) and page titles but also tags from within the text as source for semantic searches. The most popular standard which was proposed to execute this is RDFa. Personally I never believed this would be a success. Not only because the user needs to put additional efforts but also because tagging does not provide relationship information between the tags. So, these people may be disappointed but can consider text analysis, spongers for email and spreadsheets as alternative approaches (see IBM’s DeepQA).
Another group of disappointed people could be the ones who thought that putting all information on a subject into an ontology would solve all their problems. This is by far not the case. Modelling should be done with a purpose in mind. As Dean Allemang writes, it is a craft. Don’t underestimate the power of the models but accept they provide answers to the questions you want answered. If more than that is needed, ST has to be combined with probabilistic methods (see earlier on IBM’s DeepQA project).
Another group of disappointed could be people who want massive amounts of data treated with today’s reasoners. Although reasoners became much more performing, industries with huge amounts of data (like the finance industry) should approach that data in specific ways, sometimes emulating the reasoners with alternative technologies (e.g. SPIN and rules engines of triple store suppliers). Honestly, if we don’t know what the reasoner should deliver, we cannot model effectively. It would probably mean we are not rightly involving ST.
The reason for my conviction of undervaluation is that for the above objections there are far-reaching alternatives which are opening new horizons in all fields of applications. Furthermore, there are new domains starting to adopt semantic technologies. Artificial intelligence is such a domain. To my knowledge, the most impressive result of this is IBM’s DeepQA project. I know the reluctance of ST people being associated with AI because they feel AI did not bring them much, on the contrary, they brought a lot to AI. In my opinion, implementing the probabilistic approaches, besides other AI techniques, with ST brings a lot of added value to ST. Lets not forget ST is strongly domain oriented, while probabilistic approaches may help generalise solutions from combined domains. I expect a lot of input also from the KM community . While this community used ontologies from the beginning (in different ways than web ontologies), the love between the ST an KM communities is not to be called ideal. When the KM community will embrace the possibilities offered by ontologies in Descriptive Logic (OWL-DL compliant), the benefits will contribute both communities. See http://semanticadvantage.wordpress.com/category/semantic-technology/
Ivo: It seems that you don’t much associate yourself with the Semantic Web community. Is that so? What do you think are the main mistakes, of fallacies of the Semantic Web movement that would actually jeopardise or postpone Semantic Technologies getting more tangible traction?
Eddy: First I would like to stress that the solution proposed for the operational dysfunctions owes everything to the SEMWEB community. I cannot thank them enough. If I am less associated today the reason might be laziness. I have to update my vision of their activities again. You see, when these technologies became of strategic importance for USA and for Europe, lots of financial means were directed to the development of standards. In the beginning, the SEMWEB community was mainly busy developing standards complying with Tim Berners-Lee’s architecture vision.
Later came the tools with pioneers as: Clarks & Parsia (Pellet reasoner), Stanford University (Protégé), Berlin University (software language adaptations for ST), Zurich university (Controlled English communication), HP with Jena Laboratories (triple stores and SPARQL). Very rapidly a vast community of tool producers followed. This was really exciting. I could only participate as tester, commenter in small fields since the main discussion point then was on tool development.
Later commercial players joined the community: Topquadrant with a software development platform including creation of services and web server, OpenLink Software for heavy duty triple and quad stores and a few more. The problem is the applications are mainly being adopted by the academic and scientific world, not yet by business users. I’ll check this for updates again.
Ivo: A recurring pattern in the lessons learnt you share on your site is related to losses when conceptual data models are transformed into physical. And another is related to the missing time dimension. Can you please tell us more about those two and how they are solved by applying ST?
Eddy: On the transformation conceptual to physical, Business analysts have different tools at their disposal to represent the real world operations in a conceptual manner. They are mainly grouped in what we know as the UML diagrams, a collections of diagramming methods. Starting from these conceptualisations, technical analysts, programmers and a collection of other specialised people (data engineers, interface designers, service engineers, …) start developing a functionality involving those concepts. I will not repeat the classic picture of the swing illustrating the difference for the user, anyone can make his own version here. This is solved in ST by software using the ontology as their data source. A perfect example is the Topbraid suite. Another popular tool is provided by Openlinks Software, not to forget the Fresnel lenses.
On the time dimension. There are only few universal laws. Knowledge about facts is merely related to and valid for a portion of time. Meaning, an assertion is true at a certain moment or time-period. E.g. an organisation, the price of an item, the specification of a product. Whoever has been involved in data mining will recognise 2 main challenges: denormalization of related data and time-dimensional information. The purpose is to reconstruct any state of the company at any moment in time. The reason is, we cannot find out cause and effect information if we can’t partition facts into the time dimension. This is done in ST, mainly in a similar way as with conventional data-management systems: adding a time-dimension property to any individual, member of a class.
Ivo: What are currently the best examples of using ST?
Eddy: Cancer research, DBPedia, NASA has database of metrics methods, linked Data analysis, US government enterprise models, search engines.
Ivo: Is there something that you see as a big potential of ST which has not come to fruition, which for some reason nobody was able to realise so far?
Eddy: To me, it is strange the possibilities for application development are not really applied.
Ivo: You have been and are currently involved in BPM activities. What do you think ST can bring to BPM? Do you see it as a flavour of BPM (there is something called Semantic BPM already) or as an alternative approach to what most of the companies use BPM for?
Eddy: I am not in favour of "semantic BPM". The reason is linked to my answer in the beginning: it requires tagging the models and their objects. I certainly see it as an alternative approach to what most companies use BPM for.
Ivo: You worked in banking for quite a long time. Let’s talk about Semantic Technologies as an investment. It seems that many types of innovation if judged using DCF or other popular methods, look riskier than do-nothing strategies. What type of evaluation would convincingly justify investment in Semantic Technologies today?
Eddy: The answer focuses on semantic technologies as an investment, not as a banker investing in companies with an expected DCF of X EUR/USD. The latter requires a much broader approach than just DCF, RONIC, or similar. Notably what we may understand by "expected" cash flows, introducing Beta factors, sector and currency valuation of future trends, market valuation in economic context,… So working in the banking sector will not help providing an answer here. Prior to the investment selection step, the proposal is to position ST in a strategic perspective analysis. That analysis would answer the questions:
A. On the portfolio management issues: A1. What new products can be offered in our portfolio with ST? A2. How will ST affect existing products in our portfolio? A3. Which new markets can be explored with the products discovered in A1 and A2? A4. How are these new markets evolving in the strategic time perspective? A5. How are the products discovered in A1 and A2 influence our competitive position in the new markets discovered in A3? A6. How is our competitive position evolving compared to A4? Suggested method: portfolio management matrix of the Boston Consulting Group.
B. On the strength of the organisation: B1. How does the application of ST reduce the risk of new competitors coming in the market? Where can the organisation start competing in new markets? B2. How does ST application affect our negotiating power with supplier? B3. How does ST application affect our negotiating power with customers? B4. How is the threat of substitute products in our markets mitigated through ST application? How can we form a defence against substitutes in existing markets? B5. How is our strength affected toward competitors considering the price of our products and services, the quality of our products and the service level? (Inspired by Porter’s 5-forces model.) After a thorough SWOT analysis, when the product/market matrix is finalised, the production simulations ran and the investments are figured out, cash flows can start to be projected on a strategic horizon (3-5 years). Eventually other analysing techniques may be needed. Choosing for semantic technologies gets a completely different perspective when compared to "do nothing" scenario if we make the exercise as mentioned above compared to a pure accounting approach.
Ivo: Now about reasoners. Why there are so many of them? How they differ from each other? How to choose? When to use which?
Eddy: I wished I had a clear-cut answer to that question. When building the finance ontology in 2007-2009, I tested 5 reasoners. On the correctness of the inferences: for an expected inference, I could get 4 different answers. The conclusion at that time was, the best reasoners depended on the inference needed. The reason might have been for a part the shift from OWL 1 to OWL 2 but it makes me test the inferences at each construction of a model with different reasoners. On the speed of the inference: in the beginning some very well performing reasoners went commercial only. Meanwhile some free reasoners upgraded their version and new players entered into the field. Further I would like to remind the relative importance of reasoners in heavy life applications: see higher under SPIN and rule engines.
Ivo: If I start asking about ST languages, the answer might be much bigger that all answers combined so far. May be we can post a separate conversation about that. Still, this one could not do completely without it. About the OWL then… OWL is (arguably) the leader in knowledge representation. Some think that the main advantage is that it is decidable, not probabilistic. Others state that the best thing about OWL is that it’s based on a solid mathematical foundation unlike all OMG standards, which are criticised for lacking it. What do you think are the main strengths of OWL, and the main weaknesses?
Eddy: to me it is the mathematical foundation when this enables advanced inferencing. On the other side, I very much appreciate the profiles with the different syntactic subsets. There is no clear-cut solution and OWL has to be tailored. This strength is also its weakness, together with the fact we cannot reason with classes (for the moment?).
Ivo: Thank you, Eddy! May be there will be some more questions put here from others. Now, instead of a conclusion, and for those wondering why Web Ontology Language is abbreviated as OWL and not WOL, one familiar passage:
He [Owl] could spell his own name WOL, and he could spell Tuesday so that you knew it wasn’t Wednesday, but his spelling goes all to pieces over delicate words like measles and buttered toast.
A. A. Milne, Winnie the Pooh
BPMN vs. EPC revisited, part 2
The previous part focused on areas such as expressive power, readability and enterprise architecture. This one, written jointly with Roland Woldt, dwells on a few more aspects such as semi-structured processes, exceptions, loops and data handling. Some of them could be sorted well under the ‘expressive power’ heading but as stated in the previous post, the intention is just to put some structure to the ideas and experience shared, not to give a list of criteria. If in spite of that the sections are regarded as comparison criteria, then each should get some weighting based on the application objectives. Additionally the ‘points’ should be reduced by the ‘cost’ of the advantage. For example, more expressive power brings increased complexity.
But again, as this was not the intention, the content is not suitable for such usage.
Semi-structured processes
A business process model is meant to show some structure and behaviour patterns that are known or at least expected with a good degree of certainty. For processes with high uncertainty, using a notation – no matter how good – would not be as useful as for well structured processes.
There are processes for which activities are known but not their sequence or number of performances. These types of processes are supported by BPMN via ad-hoc sub-process. No similar construct is offered by EPCs.
Advantage: BPMN
Business process notations are used in some way even for managing unpredictable processes. Interestingly, Adaptive Case Management systems such ISIS Papyrus, support both EPC and BPMN which is yet another example that each one has strengths in different application areas.
Exceptions, transactions and compensations
Both EPC and BPMN can support exceptions. However, there is no way to show that something happens during the activity execution in an EPC. Conversely, BPMN is rich in constructs that support exceptions. Those include the boundary events in combination with catch/throw semantics. Situations where an exception could happen everywhere within a process fragment are also well supported by putting the fragment in a sub-process with an exception boundary event. Additional benefits are provided by event sub-processes.
Exception handling is really important. Some claim that this is so only from IT perspective, but I doubt that. However, it is true that some modelling limitations tend to support wishful thinking and many processes are suffering of the ‘happy path syndrome‘. Now, the mere existence of so many exception handling constructs in BPMN might – well, at first frighten but then – prompt some more realistic business representations.
If we are completely fair with EPC, we should admit that there some activity interruption is supported. But this will mean to depart the pure notation discourse. There is an attribute ‘interruptible’ which, when set to “true”, will allow activity execution to be interrupted based on specific resource assignment. This capability is supported by ARIS Business Simulator. A token arriving at an activity will trigger it but when this activity is interrupted (attribute ‘interruptible’=true), it will wait until the activity performance is resumed. The reason for such interruption could be that the resource carrying out the activity has stopped working because of a break.
However, in addition to “interrupting” events BPMN supports “non-interrupting” events that trigger an exception while the process continues – you can show this in EPC with rules (“and” rule) and events, but the solution in BPMN is more elegant.
Transactions and compensations are supported by BPMN and not at all by EPC.
Advantage: BPMN.
Loops
Loops -either standard, parallel or sequential- are a powerful construct in the BPMN notation. They allow to show that a process can run multiple times without the need to draw confusing lines in a model or to repeat process fragments over and over again. In combination with using events or attributes that hold the number of repetitions this is a clean and easily understandable way to model.
Loops can also be shown in EPC notation, even though there are no special markers in the functions that shall be repeated. A modeller has to work with multiple events that are coordinated by rule objects (e.g. “and”) to describe the conditions when a loop or repeating activity has ended. This is not impossible but a less elegant solution when the process becomes more complex.
Additionally, BPMN provides 11 standard attributes for handling loop behaviour. When such formalization is needed, things like loop cardinality, data-based conditions and how multi-instances a produced by the activity can be specified.
Advantage: BPMN
Managing data
BPMN assumes that all data is freely available within a process and can be used in the tasks whenever needed. In addition to this data highlights can be shown by either using the data object or the data store for persistent data (even though persistence means “only for this process instance”). The notation is not meant to allow data modelling and the breakdown of data information in specific data models > see chapter 7 of the specification.
In opposite to this the EPC notation allows multiple ways to show data in a process: either as abstract clusters or technical terms, that describe an information, or depicted as information carriers, such as documents, CDs, etc. And even as fine as showing relations to classes and attributes. The biggest advantage of using an abstract data object lies in the capability of the ARIS overall method -not only the EPC notation- to decompose a cluster into entities and then into its attributes. These can then be mapped to tables or even synchronized with other dedicated data modelling tools like ERWIN (via a 3rd party interface). Strangely, EPC allows better integration between process models and UML Class Diagrams than BPMN do, while both UML and BPMN are maintained by OMG.
Advantage: EPC
Process notation for ERP Systems
As mentioned earlier the EPC notation became very popular when the ERP wave took off and companies had the need to figure out and model their processes. The most popular example here is SAP and until today all SAP ARIS customers use this notation to synchronize three levels of process models directly with SAP’s Solution Manager. In addition to this, EPCs can be used to create test cases that get synchronized with HP’s Quality Center for testing (either through Solution Manager or directly via export from ARIS). There is no BPMN synchronization in place as of today.
To further accelerate the modelling efforts SAP provides two sources of process information that can be included into the blueprint models: The “Business Process Repository” (the transactions) and the “Enterprise Service Repository”. Both sources will be enhanced by non-SAP and manual steps in the blueprinting phase of the ERP implementation.
But SAP is not the only ERP system on the market – its largest competitor Oracle OEM’ed ARIS in its “BPA Suite” tool and in that, the EPC notation is also the standard notation. Opposite to the SAP synchronization, the EPCs will be transformed into BPEL models that then will be exported as BPEL and WSDL files that can – after running an Oracle-specific script that transforms the standard BPEL export file into an Oracle-specific format – then be imported into the Oracle ERP system.
Similar approaches to the BPEL export have been implemented in process modelling for Microsoft BizTalk and IBM’s WebSphere products.
An interesting way was presented by Software AG in the “Enterprise BPM” approach at CEBIT this year. Here it is also recommended to model the business processes in EPC notation, since it is easier to understand for business users and fits into an Enterprise Architecture without workarounds. After the business model is approved it can be transformed into a logical BPMN 2.0 model and then enhanced with technical information, for example to show a split of a business process step like “check inventory” in multiple service steps. After the logical BPMN model is ready it can be pushed into webMethods where the physical BPMN model then gets implemented.
The last use case shows an implementation of the original purpose of the BPMN notation. I expect similar approaches for creating a true business process architecture, while being able to execute the lower level models from other vendors too.
Advantage: EPC
Tools support
An interesting observation nowadays is the marketing hype around BPMN. To a new user, who is investigating which notation/which tool should be implemented the options and sometimes conflicting marketing information are confusing. On the one side you hear that EPC is a proprietary notation (which is not true) that is only supported by ARIS, on the other side, the EPC templates ship with a standard MS Visio installation for years now, while BPMN (in the old version 1.2) was just introduced to the latest version Visio 2010. In addition to this a quick Google search will show dozens of tools for either notation.
I think the best approach to choose the right tool is to ask yourself about the goals of a BPM/EA implementation. Do you “just” want to model simple processes, shall they be executed in a BPMS or used in an ERP implementation, or is the procurement of a tool just the start of a larger, enterprise-wide implementation with the goal of creating and measuring a performance-driven organization, while using enhanced capabilities such as simulation, process costing or process performance management. Both cases depend heavily on the maturity of the organization and luckily the established vendors provide import capabilities for existing information.
By Ivo Velitchkov and Roland Woldt
BPMN vs. EPC revisited, part 1
There were several posts and discussions on the topic of “BPMN vs. EPC”. One of them is quite comprehensive and its discussion thread very interesting. But there are still many important points untouched and I’d like to share some of them for those facing a choice of business process notation.
That doesn’t mean that there aren’t other options. But they have quite lower utilisation potential and/or address different needs. Certainly if UML is used, a combination of activity, statechart and sequence diagrams would do, more or less. And then most of what EPC offers could be expressed with ArchiMate. Speaking of process modelling notations it’s worth mentioning also Petri-Net which has better execution semantics than BPMN (while that’s the main claim of BPMN) but is not at all adopted by the business and generally gets little attention outside academia.
So, having the long and successful history of EPC usage and the current growing popularity of BPMN, it seems a valid set of options. And there is a demand for more information to support decisions related to the choice of process notation, especially (naturally) among ARIS users.
The ideas here are based on the conviction that a single notation for business processes which can equally serve the requirements for analysis, process automation and enterprise architecture is achievable. There are some strong points by the advocates of the concept that the choice of the notation should be based on the modelling objective and it’s fine to have many languages. However, if one notation is able to serve the majority of applications, then it would eventually be a better means for communication and will require less learning and maintenance. Furthermore, when efforts for improvement are focused on one language, there is higher probability of getting it better sooner.
Having said that, I still hope that the information in this article would support not only XOR but also AND decisions. Another way to help such decisions would be to make a condition comparison, with a lot of “it depends”, if-clauses and suchlike. That is a valid approach but it will be avoided here leaving the “it depends” as conclusion for the readers, rather than using it as a way to structure the comparison. And speaking of structure, it would be quite loose – a dozen of short sections that could be read in any order.
One more thing before the actual comparison. I assume the readership of this article is familiar with BPMN and/or EPC. And, unless explicitly stated, by ‘BPMN’ I would refer here to its version 2.0 and ‘EPC’ would mean the extended Event-driven Process Chain. And when referring to EPCs I’ll use ‘gateway’ instead of ‘rule’ and ‘activity’ instead of function just for the sake of easier comparison. Hope you don’t mind.
Expressive power
BPMN features over a hundred modelling constructs. Most of them are sub-types of the three main flow elements. They are primarily focused on describing workflow and collaboration. As it can be expected, BPMN has much more expressive power than EPC in these two areas. The control flow elements of EPC are only five.
There are different ways (and viewpoints) to measure the expressive power of modelling languages. One of them is by workflow pattern analysis, another by ontology-based analysis and yet another by measuring the ability to integrate different aspects of Enterprise Architecture.
Workflow pattern analysis has been applied for comparison of BPMN 1.1 and EPC among other modelling languages by Nick Russell et al in 2006. According to this research, BPMN natively provides clear support for 24 of 43 patterns, while EPC supports only 10. Of course the real implications of that could be seen if there is a reliable statistic on the frequency of occurrence and the importance of unsupported patterns. Still, when you need to describe a workflow with more rigour and provide a model that is closer to reality, BPMN is the way to go. This is also confirmed by a more recent comparison between BPMN and EPC on the twenty main workflow control patterns.
Applying ontology-based analyses like the representational analysis of process modelling techniques, confirm expressive superiority of BPMN over EPC on the way it facilitates clear descriptions of real-world domains. (For those who don’t know and/or don’t care about this type of analysis, let’s just say that evaluation support of workflow control patters is not the only way to measure expressive power.)
However, if measured by the ability to integrate different aspects of Enterprise Architecture, EPC is much more expressive than BPMN. There will be some more information about this further in the post.
When evaluating the expressive power, it’s worth reminding that expressiveness comes with the price of increased apparent complexity. This reduces comprehension and communication effectiveness, especially among model readers that are not experts in process modelling.
Readability and efficiency
Which one is easier to read? And by whom?
That’s probably quite subjective. Some find BPMN event symbols more intuitive than the big hexagonal symbols for EPC events. Others defend EPC events for they somehow make you put a name unlike ‘none’ BPMN start and end events which are quite often left unnamed. Both are valid points and worth considering.
There is some research on the subject of process diagram readability but not much. This one applies cognitive effectiveness criteria based on the 5 principles introduced by Moody and Hillegersberg: representational clarity, perceptual discriminability, perceptual immediacy, visual expressiveness, and graphic parsimony. It seems EPC is doing a bit better there than BPMN.
When it comes to efficiency, BPMN diagrams are more compact than those of EPC expressing the same content. That’s not only because of the relatively smaller event symbols. EPCs require events after OR and XOR gateways, while in BPMN their semantics is conveyed by the attributes of the sequence flows following the splits. Moreover, the allowance of implicit events in BPMN as well as conditional sequence flows can make some diagrams really compact. Sometimes, however, this increased efficiency can lead to erroneous comprehension.
A smaller number of visual elements can indeed increase readability simply because there is smaller amount of things to comprehend. In other cases more compact diagrams such as the ones using implicit events and/or conditional sequence flows are not necessarily easier to read. They can take more time to understand and are prone to misinterpretation.
Resources assigned to activity
Quite often one single activity involves more than one participant at the same time. EPC not just supports this but offers more than ten different connection types between participant (organisation unit) and activity (function).
In BPMN on the other hand, assignment of multiple resource to one activity is natively not supported at all. There are some attempts to do it like those proposed by Michele Chinosi in one of the chapters of BPMN 2 Handbook. I’m not convinced by those workarounds.
Generally EPC is superior to BPMN when it comes to resource analysis, not just of human resources, but also in integrating data and system objects that can be further detailed in other Enterprise Architecture views. More on that in the next section.
One good solution offered in ARIS to solve this is to assign Function Allocation Diagrams (FAD) to BPMN activities. FADs can contain objects like organisational units, application systems, services, risks, data and more than one hundred other objects. But that’s a specific extension and it’s more or less barrowed from EPCs as FADs are introduced to reduce the complexity of rich process chains.
Enterprise architecture
EPC is a process-centric Enterprise Architecture method in itself. I personally don’t see the point of separating BPM and EA. And at the time there was only EPC (sorry, I don’t count flow-charts and suchlike as a way to describe processes), there wasn’t any separation between BPM and EA. And it’s strange when aiming IT/business coherence how first EA was separated from BPM and then how BPM became BPA to allow for business process execution to get the BPM label. Anyway, let’s not go into this here. The talk is about business process notations. Those who believe there should be a rigorous way to integrate process control flow with data, systems, products, services and infrastructure, will have that natively supported by EPC.
BPMN is not made for that. But we can’t say it as an objective not achieved. There wasn’t such objective in the first place. One way to partially support some EA aspects is by adding artifacts (but unfortunately not different types of associations). Another is to integrate with some EA notation such as ArchiMate. What would still be missing is the support of the motivational domain. And that’s another advantage of EPCs. There you can link activities to requirements, objectives and KPIs.
Well, that (+comments, hopefully) should be enough for Part 1. Next one will deal with handling of end-to-end processes, exceptions, sub-processes, messages, unstructured processes, loop, data and more.
Special thanks to Roland Woldt who reviewed the draft of this post and provided some valuable advice.
Where to start?
The Developer: "Let’s start coding. We don’t have time for solution design."
The Designer: "No, we should design first. It’s not a waste of time. It will reduce rework."
The Analyst: "Guys, please! How to start designing, if we don’t know what is the problem to be solved?"
The Tool-lover: "But how to diagnose the problem without a tool? Let’s choose the tool first."
The Architect: "Yes, but to choose a tool, we first need to make sense of the context and only when we know where we are, we can have criteria for choosing the right tool."
The Tool-lover: "Doesn’t that mean we need a sense-making tool? Again, as I told you, we need a tool first."
The Analyst: "No, this just means that we have to first analyse the context of the context. Let’s focus on that."
The Developer: "20 lines of code ready while you are discussing."
The Designer: "That’s a waste of time. But you developers don’t have sense of time anyway."
The Architect: "Time is an interesting thing. Good you mentioned it."
The Analyst: "Is ‘time’ the problem?"
The Tool-lover: "We probably need a stopwatch."
The Architect: "What we rather need is to stop and watch."
… and so on.
Integration of strategic, project and process KPIs
Some organisations, after straggling to implement Balanced Scorecard Systems (BSC), come to the conclusion that they need to at least map their processes. But then the challenge they face is how to link strategic with process KPIs. Others are practising some sort of BPM but have difficulties demonstrating its contribution to the bottom line. As Anatoly Belychook recently wrote:
business consultants know what should be done eventually but have vague understanding of how strategic goals can be achieved with the help of BPM
There are many ways to do that. One of them, usable in the ‘ordered’ domains, is to include process KPIs in the formula of certain KPI (or ‘KGI’, to conform CobiT’s terminology) of a strategic goal. It’s simple but not so straightforward so let’s first have the bigger picture.
There are basically three types of things in the motivational domain: goals, means and influencers. (If you need to dig deeper, Nick Malik’s EBMM is a good place for that. Interestingly, one of the things that had driven that extension to the BMM was the work of Alexander Osterwalder, one of the authors of the now popular Business Model Generation book) The means to achieve most ends, especially of those belonging to the classic ‘process’ perspective of the BSC, either require certain change to be managed or some parameter(s) to be kept within desirable range or both. Let’s illustrate this by combining it in a single case where a strategic goal is supported by certain process which is no yet implemented. Thus there will be another ‘means’ representing the project to implement that process.
KPIs are either primary or derived. In the example, the strategic goal is measured by a KPI 1 (derived) which is calculated by the value of KPI 1.1 (primary), KPI 1.2 (derived) and some specific attribute of their relations like weighting for example. KPI 1.2 is calculated from the values of the project KPI 1.2.1 and the process KPI 1.2.2. They are in reality more than one for both project and process management. If the project duration is shorter than the control frequency of the strategic goal, then for ‘period 1′ KPI 1.2 will take KPI 1.2.1 and KPI 1.2.2 value times their respective weighting. For ‘period 2′ KPI 1.2.1 will be equal to the value of KPI 1.2.2.
Such an approach needs the support of a repository based tool for modelling and analysis. No matter if its primary focus is EA or BPM, it should support enterprise motivational domain and ideally project and program management.
Patterns of Informal Architecture
The CEO was referred to as ‘the architect of the enterprise’ long before ‘Enterprise Architecture’ was coined. The ‘architect’ metaphor was used in 60s and 70s and became quite present in the strategy management literature of the early 80s. And really, big part of the job of C-level executives is to manage (different domains of) the architecture of the enterprise. That part is not labelled ‘enterprise architecture’ and probably shouldn’t be. But it must be recognised as such by the ‘formal’ Enterprise Architects. If formal enterprise architecture finds a way to play together with the informal one, this in turn would help the latter understand how to get more benefits from the former.
One way is to use less formal techniques to explain formal EA to the C-level and other practitioners of the informal EA. Storytelling looks like a good way to do that. Another way is to show how formal and informal can go hand by hand and best do it again in a story, the way Chris Potts did in ‘recrEAtion’. Yet a third way is to make it a habit for formal EA practitioners to recognise patterns of informal EA.
It’s not difficult to spot them in processes and cases such as creating/changing chart of accounts or profit/cost centre structure or product/service portfolio. (Or, is it?) Informal EA patterns are also easy to find in any standardization effort or planning a new business capability, opening a new branch or running an integration program after a merger. It’s easy because all those cases have some kind of formalisation, albeit not an EA one. That’s not the case with patterns of informal EA in people relations. But the effort is worth it. Relations between people are of primary importance both for the enterprise and for managing its architecture. This week it was the topic of some very good posts by Tom Graves and Nick Gall. So here, another touch in the context of informal EA.
Formal EA uses models a lot. All models represent certain aspects of the past or the future. (I put ‘as-is’ models in the ‘past’ category as well.) Models of people relations are no exception. Models of the past relations are basically two types: some are based on perceptions and decisions of the modeller about certain aspects of reality and others are visualisation of actual characteristics, registered using some technology:
All models, actual and perceived, belong to the formal EA tool-kit. If we superimpose the two pictures above, conceptual, logical and physical would go to the two ‘perceived’ quadrants, and ‘operational’ – to the ‘actual’ one. What is left is the ‘informal’ relations. They are relations, therefore fall in the EA domain. They work and could be very powerful, therefore are important. Moreover, they are adaptable and can easily cheat the formal one. Here’s an example:
Bill starts a new project and needs a team leader for one of the teams. He has to fill in a request form and send it to the HR Department. Then hope they could come up with the right person from the huge database of employees. Bill doesn’t want to leave things to chance. He calls his colleague and friend Kim and asks him if he knows somebody that has capabilities as that team leader, remember, from the project A which they were involved together last year, but at the same time can handle situations of the type they had three years ago and has that and that technical experience. And he/she should be available for the project period as far as Kim is informed. Kim doesn’t know such a person available but he knows Sarah. Sarah is in a different location. They never worked together but Kim trusts her and has reasons to believe she might be able to help. After a few hours Kim sends Bill a message with the CV of a person recommended by Sarah. Bill gets some specifics of the CV and sends them as requirements to the HR department.
Some may argue that this story is about Knowledge Management, not EA. I agree it’s KM and disagree that it’s not EA. It certainly is and belongs to the grey cloud in the picture above.
ArchiMate, BPMN and UML together
The question about “the remaining role of UML now that ArchiMate has arrived” generated an interesting discussion on ArchiMate LinkedIn group. Adrian Champbell‘s first comment was:
Archimate was deliberately designed to be mappable to BPMN and UML, but not to replace them. Not parallel universes but complementary ones.
Archimate is for modelling at an Enterprise Architecture level of detail and not at the Solution Architecture and Software development level of detail. BPMN and UML have much more detail in them than ArchiMate.
Conversely neither BPMN or UML can replace ArchiMate either.
I agree with Adrian. Actually I find the idea of EA notation set comprising ArchiMate, BPMN and UML quite appealing. They are complimentary, although having some representation-types that could be redundant in a unified method. I use here ‘representation-type’ to avoid ‘model’ which means different things in UML and ArchiMate. There are other fundamental differences but we don’t have to deal with all of them. The three notations can co-exist quite well as they are. Especially when supported by a repository based tool with separated content and presentation space. It seems that what is needed in a big variety of use cases is already there or - if not – an extensions could take care of it. The rationale behind such a combination of ArchiMate, BPMN and UML is to have a relatively small number of notations that could satisfy most stakeholders and comply to some common modelling rules. Finding a good way to integrate these three languages could bring a lot of benefits. Coming to one universal notation turned out to be a difficult task, maintaining a dozen is neither efficient nor effective and then having several kept separated, used per project, could bring little benefit for the enterprise in the long run. Then how about these tree only, and well integrated?
There are two basic questions. How to make them play together? How to add what is missing?
To make them play together
we need some mechanisms and some rules. We need mechanisms for transclusion, elaboration/navigation and transformation.We also need rules for avoiding redundancies and ambiguities.
A transclusion or some similar mechanism is needed to manage separated content and presentation space. This should enable some unit of behaviour, for example, maintained in the content space, to participate as a task in BPMN diagram, as a process in ArchiMate view and as an activity in UML Activity Diagram.
An elaboration/navigation mechanism is needed to link for example a process in ArchiMate with its BPMN elaboration and data object with it’s elaboration as UML Class Diagram.
Transformations could be handy in cases where we need to generate a UML Sequence Diagram from BPMN Collaboration Diagram. Such a demand may arise when at some point we need special message capabilities like create and destroy and/or we need to see when objects are active and when passive which would be a bit of a challenge for a pool to show.
Modelling rules along with method restrictions should be used to avoid redundancies and ambiguities. The diagram bellow shows some overlapping areas. It should be clear for example (area 1) when a process should be represented as ArchiMate process view, when as BPMN process diagram or UML Activity Diagram, if ever. It seems natural to expect ArchiMate process view to be used for more conceptual representation and then each process to be elaborated as BPMN process or collaboration Diagram (Area 2). Similar convention could be applied for data structures in area (3). In many cases UML Class Diagrams are used without specifying attributes and methods. Probably in those an ArchiMate data view should be more appropriate as then the same data objects could easily be reused in coherence views to show relations with a service for example.
The main idea behind BPMN is to bring processes easily to process engines for execution. As smooth as possible. How the recently released BPMN2 is another question. I’m not going to discuss it here. Just want to draw your attention to area (5). There are some cases where it would make sense to elaborate a Service Task from a BPMN diagram to a UML (component) diagram.
To add what is missing
we need some extensions. I mentioned a few things missing in a whole-of-enterprise EA in my previous post. Most of those should clearly extend ArchiMate, assuming we confine to this trilingual framework. What I see as the most ‘urgent’ need is for a motivation extension. So far there is one good attempt to satisfy it. Still a lot to improve but the first step is made.
Another extension could meet the demand for a loose sticky-notes type of modelling. This proved to have a lot of benefits, bringing people together and stimulating creativity and innovation being just a few of them. There are already some good methods for that as the Business Model Canvas. It would be even better to have the freedom for brainstorming and collaboration and then some way to transform ideas into a more rigorous model. I find the whiteboard in ARIS Align a very good example. There, in a collaboration (social media like) environment, sticky-notes from the whiteboard could be dragged and dropped as tasks in a method-restricted area for business process modelling. How about a similar thing with business models done using business model canvas? Just drag the sticky-note and place it in ArchiMate ‘area’ to create an object. A sticky-note from Value Proposition building block would probably be transformed to Business Service or Product in ArchiMate. Well, as it so often happens, a notations talk easily slips into a tools-and-features talk. Not good but now as we are here, let’s mention another feature. A toolset supporting integrated use of these three notations should always have a way to support an author-specified notation. Alternatively, some abstract diagrams like context-space maps or the one above, should be done outside.
In an environment not allowing a forth notation, an extension of ArchiMate might be needed to model Value Stream Maps and other lean and Six Sigma diagrams. Such an idea probably sounds heretical to most of the practitioners of these methods. I find it rather nice.
BPMN and UML (especially) have good extension mechanisms. Currently, I see more need for reduction than extension there.
Realization
We represent aspects of (or decisions about) reality to help us improve it. The bottom box in the diagram is labelled ‘realization’. I didn’t want to call it execution or automation. Our subsequent decisions could be realised with or without IT. That brings us to to the forth arrow on the right side. Some decisions represented in ArchiMate could be realised as changes in enterprise reality with no use of IT. Others can make use of IT or affect IT but without any need to go though BPMN or UML.
These were just a few thoughts on the idea of EA notation set comprising ArchiMate, BPMN and UML. Only user experience could show if and how that works. There are companies using ArchiMate and UML together, the latter to support Solution Architecture modelling. Others use BPMN and UML. May be there are some that have already experienced or probably standardised on the three.
All-inclusive Enterprise Architecture
That’s not the most appropriate title. My intention is not to dub another type of Enterprise Architecture (least of all one that sounds like a holiday package). That would defy the points I’m trying to make. Yet I couldn’t help it. The concept of Enterprise Architecture (EA) as I understand it – and luckily I’m not alone – is all-inclusive by definition. It doesn’t mean EA should include everything, just those things that are important for the enterprise and can be supported by EA management. Some of them are currently missing.
The reason we need EA, according to TOGAF, is to support “delivery of business strategy”. But after that line all listed benefits are solely around IT. The rest of TOGAF and most of the other frameworks are downright IT-centric as well, though admittedly business conscious. Of course they should be business conscious. Just remember what was the driver that brought EA into existence: the gap between business and IT. But it seems the current EA practice is too short a bridge which is now falling down that same gap instead of bridging it. Why? Because it’s not enough “IT” to deliver tangible value and it’s not enough “Business” to make business people really interested. No wonder BPM for example is doing much better on its own (How we came to have EA and BPM as separate disciplines will be the topic of another post). Not that the “B” in BPM is what it should be. Yet, the interest in BPM is growing while in EA is declining, or – to use the Gartner terminology – EA enters trough of disillusionment.
If on the other hand EA was really business driven, not just business conscious, then we should have seen it already in the MBA curricula together with Finance, Marketing, Management and probably right after
Strategy
There seems to be a broad consensus around EA being strategy enabler. Gartner defines EA as “the process of translating business vision and strategy into effective organizational change”. But then strategy is hardly anywhere in the frameworks and unfortunately not much in the EA practice. TOGAF does not provide any way of seeing how EA supports delivery of business strategy. Actually, motivation concepts were entirely missing until recently. What appeared in the version 9 was a motivation extension in the content framework humbly featuring driver, goal and objective. It’s difficult to understand why motivation should be an extension and not core concept. Shouldn’t it all be derived from there? Other frameworks seem to be doing a bit better in that respect. For DoDAF the concepts of capabilities, measures, and goals (desired effects) play central role in the meta-model and they are well integrated with the enablers – services, activities, people and systems. FEAF in its Performance Reference Model has clarity about value creation from inputs (people, technology, other fixed assets) through outputs (processes and activities) to outcomes (mission, business and customer results).
There were more efforts in this direction in the recent years. And some good results. Notably, one of them is the Enterprise Business Motivation Model of Nick Malik which extended the BMM of OMG, developed by Business Rules Group.
Some good news is coming also from the notations front. A very promising extension of ArchiMate was proposed to fill the motivation gap in the notation and achieve better traceability of requirements. Having motivation in, it’s mostly used to achieve traceability, to have the “why”-chain from the system we implement to what the ultimate business goal that it’s meant to support. Good, but is it good enough? And then, when we need to find that strategic goal, we usually search in the strategic plan. But are all strategies planned? How about “emergent” strategies as Henry Mintzberg called them back in the early 90s. Don’t we see more of those recently? Strategies that emerge from the user experience. Here, user experience. How about that in EA? Or brands? Or probably their place is not in EA. In 2009 Google market value was estimated around $108 billion and the value of its brand over $100 billion. The brand value of most companies in that same list is over 90% of their market value. Do we want to have EA which excludes 90% of the company?
EA needs strategy to direct its change and strategy needs EA to make things happen. And not only for that. Strategy needs architecture for its own design to improve its communication and execution control. After all it was the advent of strategy maps that turned Balanced Scorecard from an original idea to worldwide adoption.
Structures
Another general consensus probably first articulated by by Peter Drucker is
Structure must follow strategy
Which structures? Scope wise, here’s where contemporary EA is quite often organisation-centric. EA is about processes, systems and technology. About processes within organisational boundaries. That’s why the real architects of the enterprise, the CEOs, are sceptical about EA value proposition. Because for them
the customer has the process. We have to decide how we want to appear in that process – what we want to contribute to the customer experience – and make sure that’s what our day-to day performance actually achieves
From recrEAtion by Chris Potts
Content wise, we don’t see EA focusing much on structures for non-IT related initiatives. Like for example cost analysis. It’s not new that the actual product and service cost is hidden in the process and cost centre structures. That’s why when bigger and bigger part of the product costs went into the overheads, activity-based costing (ABC) came into the game . EA methods and tools could be an indispensable instrument not only for ABC analysis but much beyond that. And certainly much beyond technology cost optimisation. But OK, let’ have a look now at
Technology
I’ve seen many enterprise architecture descriptions. In all of them it seems that the enterprise has only one type of technology – IT. Yes, that’s of primary importance in the service industry. But that’s not the only type. And then, even in cases where investments of non-IT technologies are times bigger than those in IT, those other technologies are entirely missing in the EA blueprints. Why should you believe in enterprise-level description that is so different from what you read in the balance sheet?
Enterprise Architecture is indeed currently in the “trough of disillusionment”. Yet, it brings a lot of value even as it is. But it can do much better. And it should if it is to get out of the trough. One idea is not to leave out things such as strategy, brand, user experience, knowledge, trust, non-IT technologies. They need EA and EA needs them.