Here are the “slides” from a new session on “Essential Balances in Organisations” delivered last month in Brussels.
What’s Wrong with Best Practices?
That is a question I often get from people who know that I keep away from prescriptive approaches. I’ve been giving some quick responses, but it would be better if next time I can point to a more elaborated answer. And here it is.
While I have a lot of sympathy for those who object to ‘Best Practice’ as a name — even more — to those who object to it as a claim, my uneasiness is somewhat different. I’ll focus here on that.
Some best practices are useful. In fact, most mature and well-applied best practices for carrying out a technical task, from taking a blood test, painting a wall or repairing an engine, to building a factory or a ship, are indeed valuable (as long as they don’t suffocate innovation).
The real problem is when best practices are applied to people and social systems. I call this a ‘problem’, but it is in fact a huge opportunity for many. Most contemporary non-fiction books, especially management and self-help texts, seize this opportunity extremely well. It’s not easy to find a best seller in this category or a popular article that doesn’t provide some sort of prescription and advice, often numbered, on how to achieve or avoid something. Maybe it is a best practice for best sellers. Let’s give it a try then:
Four Reasons Why You Should Be Cautious When Applying Best Practices:
1. Correlations.
How do Best Practices come about? Some individuals or organisations, become known (or are later made known by the actions of the best practice discoverers and proponents) as successful according to some norms. Let’s call these individuals or organisations ‘best practice pioneer’. Then one or more observers, the ‘best practice discoverer’, studies the pioneers to find out what made them successful. The discoverer first takes certain effects and then selects, by identifying commonalities, what she or he believes were the causes. That is followed by a generalisation of the commonalities, from which point they begin their life as prescriptions, regardless if they are called ‘best practices’, ‘methodologies’, ‘techniques’, ‘recipes’,’templates’, or something else. Later, they are tested and, based on feedback, reappear in more mature forms and variations. In some cases, they are even supplied with scalability criteria and conditions for a successful application. The successes and failures of those who apply them give birth to Best Practices on applying Best Practices.
The problem is, that the discoverers select common patterns among the observed successful pioneers, and infer causal relation between these commonalities and those that were the criteria to select that set of pioneers in the first place. Then, such correlation leads the discoverer to select what to pay attention to, and every pattern that supports the hypothesis based on the selected commonalities would be preferred over those that don’t.
2. The risk of over-simplification.
Best practices help us deal with external stimuli, when they are too many to handle, by prompting which ones to pay attention to and how to react to them. The only way to deal with a situation is when the number of responses is higher than the number of the stimuli, within a given goal set (Ashby’s law). Or in other words, best practices are tools for reducing external variety. But not only that, they also provide means for amplifying internal variety in a special way – coupled with those stimuli that you are advised to pay attention to. So if that assumption was wrong, and it often is, neither the external variety is reduced, nor the internal is amplified.
3. Assumptions about the application context.
I used to be a practitioner of PRINCE2. What I still appreciate is a few smart techniques and in fact – the name itself. The name is the important disclaimer I’m missing in most methodologies and other types of best practices: they only work in controlled environments. They work when most of the conditions of the design-time if you allow me the IT jargon, are unchanged in run-time. This is rarely the case and is increasingly less so. This brings another interesting phenomenon: the same conditions that make the world less predictable also help quickly productise and spread best practices. They come with better marketing and with more authority in a world in which less of what has happened could prepare you for what will.
4. The habits created by Best Practices.
The worst is when people hide behind the authority of best practices or their proponents. If not that, best practices create habits of first looking for best practices, instead of thinking. And then, there is the alternative cost: the more time people spend on learning best practices, the less time they have for developing their senses for detection of weak signals and for developing their capabilities for new responses.
In summary, if you are sure that certain best practice is useful, and it’s not based on wrong inference and does not lead you to dismiss important factors, and the situation you are in is not complex, and it doesn’t weaken your resilience, then go ahead, use it.
The Pathologies of Silo-fighting
The division of labour has been the main principle for structuring organisations in the last two centuries. That is still the dominant approach for allocating resources, information and power in companies and public institutions. The new dynamics in a connected world have revealed a rich spectrum of problems related to these structures ranging from ineffective coordination to turf wars. This gave birth to the stigmatising label ‘silos’ and a whole industry of silo-fighters armed with silo-bridging or silo-breaking services, methods and technologies.
Here I will point out the two most prominent organisational pathologies brought by such silo-fighters. I’ll split the silo-fighters’ strategies into silo-bridging and silo-breaking – an oversimplification – to make the illustration of the two pathologies more clear.
‘Bridging the silos’ is not a strategy based much on appreciating their role and thus opting for bridging over breaking. It’s mainly due to silo-fighters’ insufficient resources and power. If they manage to sell successfully the story of the bad silos, coming with a rich repertoire of metaphors such as walls, chasms, stove-pipes, islands and such like, then they get permission to build – as such a narrative would logically suggest – bridges between the silos.
Now, the problem with bridges is that they are either brittle and quickly break, or they are strong enough to defend their reason to be. They break easily when they fail to channel resources for a longer time than the patience over their initial failures would allow. However, identity formation switches on viability mode. The bridges start to grow out of a network of decisions supporting their mission, now turned into an ongoing function. If the reason the bridges exist are silos, and the bridges want to keep on bridging, then the silos have to be kept healthy and strong as they are what the bridges hang on to.
The bridges reinforce and perpetuate themselves up to a point, in which they are recognised as silos, and the problem is solved very often by building new bridges between them. This is how a cancerous fractal of bridges starts to grow. As attractive as this hyperbole is, I have witnessed repeatedly only two levels of recursion, but isn’t that bad enough?
In contrast, the silo-breaking strategies want nothing less than the destruction of silos. There, the silos are seen only in their role of a problem. Nobody asks what kind of problems this problem was а solution to. Instead, these silo-fighters start waging exhausting wars. The wars can end up in several ways. A common one is resource depletion. Another is with the silos withstanding, or with the silo-fighter being chased away or transformed. And then of course it could be the case of victory for the silo-fighters. And this is when the disaster strikes. Having the silos down, the silos fighters are faced with all the problems being continuously solved by the silos during their lifespan. Usually, they have no preparation to deal with those problems, neither they have the time to come up with and build alternative structures.
When discussing these two pathologies, it is very attractive to search for their root cause and then, when found, fix it. But that would be exactly the fuel these two types of silo-fighters run on. It takes a deeper understanding of the circularity of and in organisations, to avoid this trap. By ‘understanding’, I mean the continuous process, not the stage, after which the new state of ‘understood’ is achieved. And it takes, among other things, the ability to be much more in, and conscious of it, and at the same time much more out, but only as a better point for observation, not as an attempt for excluding the observer.
The Role of Meaning and the Meaning of Roles
Let’s start with roles. ‘Role’ comes from ‘roll’, as it was on a paper roll where the actor part was written. It is about something prescribed and then performed. But it evolved from roles that were performed as prescribed, through those that were not, to performing roles that had not been prescribed at all.
Roles are about relations. In fact, in Description Logic roles play the same role as relation, association, property and predicate in other formal languages. If John has a son George and is 30 years old, then George is in the role of a son for John, and even 30, although not an actor, plays the role of an age for John. Roles are inherently relational. A relation to itself or to something other. There is never just a role, always a ‘role in’ or a ‘role for’.
Roles are not just relational but are often determined by the dynamics of interactions. Here’s a handy example. The role of this text in the situation of you being in the role of a reader, will be, as assigned by me in the role of a writer, to transfer my thoughts on the meaning of roles, but it would rather trigger the construction of both similar and complimenting thoughts of you, and by doing so will play a different role, which, while evoked by me, is determined by you.
As there is now something about the meaning of roles, it’s time to introduce the role of meaning. If the role is always a role-in, my interest here is in the role of meaning in living and social systems. These systems have some things in common. One such thing is that they have internally maintained autonomy. Such autonomous systems bring forth and co-evolve with their niches. And that happens by creating of meanings which motivate attitudes and actions. But how does meaning come about?
Before meaning, there is the primary cognitive act of making a distinction, bringing up something out of its background, distinguishing a thing of that which it is not. That act determines and is determined by the dynamics of the interaction between the system and its niche (by ‘system’ here I will only refer to systems with internally maintained autonomy). This dynamics is circular and recursive. Roughly speaking, it goes like this:
The act of distinction, the very making of difference changes the making of difference and brings forth “the difference that makes a difference”, in Bateson words, or the “surplus of significance” in Varela words, which is also co-dependant: the sense made changes the sense-making that changes the sense that is made and then brings forth the behaviour changing relation between the sense maker and the meaning of the distinguished element. This results in attitude of attraction, aversion or neutrality (or something more sophisticated like staying put, paralysed by the equal amount of temptation and fear). Or, in other words, the sense-making transcends into value-making. The value-making evolves itself and in downward causality influences the evolution of the distinguishing and the sense-making capability, which is in fact again a distinguishing capability but it is now distinguished as sort of a second-order one.
Before going further, I need to make clarifications on the use of ‘sense-making’ and ‘value-making’. ‘Sense-making’ is giving meaning to what is experienced. Here I use it with an emphasis of the action of making, of the creation, or more precisely, co-creation of sense. It is not that the sense is out there and all we need to do is to disclose it. No, we (or whatever the system in focus is) are the ones that actually make it, the origin is within us, or rather within the dynamics between us and what we interact with.
‘Value-making’ should not be confused with value adding. The system makes a distinction of a higher order in terms of directing its behaviour based on this distinction, hence the choice of ‘value’. It is not specified by the distinguished element, it is determined by the internal structure of the system and the dynamics between the system and the environment it’s structurally coupled with. This and the fact that value-making is a sense-making of a higher-order, is where the preference for ‘making’ comes from.
Only a small part of the environment, a niche, is constantly changing its content, is being interacted with, and actually matters. The niche is not one niche but a network of dynamically changing and influencing each other niches. A family is one niche, but that’s not the family described by somebody knowing all the facts, and not the family which will be invariant for each of the family members. The family as a niche is the subjective construction of interactions, memories, emotions, attention and imagination unique for every member of the family, as long as they consider themselves as such. This could easily exclude actual members and include those that have a lasting experience as such. And the same applies to work circle and friends circle, as to all occasional and recurring encounters, and virtual communities. But then, apart from interactions with other humans, we also have a niche out of the air we breathe, the grass we walk on, the stairs we climb and descend. And all that, changed by us, is changing us and changing the way we change it. But those changes, as long as a system is viable, serve to protect the identity from changing. We take from our niche things to make them into more of ourselves and become better in doing so.
Unlike the air, the grass and the stairs, a family or an organisation are autonomous systems which maintain their identity. They have their own niches. Moreover they are social systems. There are different ways to look at them. One way would be as autopoietic systems of communications, having humans as their niche (Luhmann), and another would be to have humans as both sub-systems and niche depending on weather the processes they participate in are part of the closed network of processes creating the identity of the system in focus. And that can be talked about in terms of the roles humans play.
Now we are ready to see what the role of meaning has to do with the meaning of roles. A living bacterium is at a stage of development where the sense-making has transcended into value-making. It does not only distinguish an environment with low from that with high sucrose concentration but prefers and moves towards the latter. As Evan Thompson put it, while sucrose is a “present condition of the environment, the status of the sucrose as a nutrient is not”, “it is a relational feature, linked to the bacterium metabolism”. And this is how the creation of meaning and roles are co-dependant. The dynamics between the bacterium and its environment, by making the sucrose relevant for the viability of the bacterium, realises the role of sucrose as nutrient.
But that’s not only relevant for cells and living organisms. It’s applicable for social systems as well. A company acts so that to turn some part of the environment into employees, other into clients, partners and so on. And for the same reason as the bacterium – to maintain it’s viability.
Once a formal role of an employee is assigned by a contract, a less formal roles are enabled by different mechanisms. It is now common to refer to such mechanisms as ‘Governance’ or an essential part of it. That part is a meta-role assigned to some people to determine the role of others. One and the same person often plays several roles.
Roles can be determined by formal assignment, by methodology, or by a Governance body but they can be also invented and self-assigned by the actors themselves, as typical for the organisations researched by Frederic Laloux. In that case, the evolutionary nature and the granularity of the roles are both dealing with the pathologies of assigned roles (being status currency, perpetuated even when obsolete, determined by politics and so on) and making organisations more responsive to change. Again the system, by what it does, takes from its environment what it needs to make more of itself, recursively: it turns non-employees into employees and vice versa, and employees take up and leave roles, as determined by the dynamics of their interactions.
And so it seems that the meaning of roles is continuously realised by the role of meaning as a way in which a system generates its niche by asserting itself and maintaining its viability.
Language and meta-language for Enterprise Architecture
That was the topic of a talk I gave in October 2014 at an Enterprise Architecture event in London.
Most of the slides are available as PDF slidedeck on Slideshare.
They probably don’t tell the story by themselves, and I’m not going to help them here unless this post provokes a discussion. What I’ll do instead is clarify the title. “Language” refers to the means of describing organisations. They could be different. Given the current state of maturity, I have found those based on description logic to be very useful. What I meant by the “current state of maturity” is that a method in its theoretical development, application, the technologies supporting it and the experience with their application justifies investments in utilising them and helping in their further development. Although I find such a language clearly superior to the alternatives in use, that doesn’t mean there are no issues and that no approaches are showing convincing solutions to those issues. However, the practice with the latter or with the available tools doesn’t give me enough reason to stand behind them. The situation with the “meta-language” is similar, but let’s first clarify why I call it that.
Metalanguage is commonly defined as language about language. If that was the meaning I intended, these notes here could have been referred to as a mixture of another meta- and a meta-meta-language. That’s not the case. But to clarify the intended meaning of “meta,” I need to first clarify “language.”
I have found that there is a need to describe properly the “objects” that people in organisations are concerned with and how they relate to each other. It could be some way of representing physical things such as buildings, documents and servers or abstract concepts such as services, processes and capabilities. And although it relates also to abstract things, I sometimes call it “language for the substance”.
Organisations are autonomous and adaptive systems, continuously maintained by their interaction with their niche, the latter being brought forth from the background, by that very interaction. While a language such as the one proposed can be useful to understand the components of an organisation, it doesn’t help much in understanding the dynamics and viability. The language for the substance cannot be used to talk about the form. That’s why there is a need, maybe temporarily until we find a better solution and probably a single language, to have another language and that other language I called meta-language in the presentation.
As this is a language for the form, I keep looking for ways to utilise some proposals. One nominee is George Spencer-Brown’s Laws of Form (this post includes a brief introduction). Papers like this one of Dirk Baecker give me hope that it is possible. Until then, for the purposes of Enterprise Architecture, I find the Viable System Model, with the whole body of knowledge and practice associated with it, as the most pragmatic meta-language.
Related posts
Essential Balances in Projects
These are part of the frames from the projects-flavour of the “Essential Balances” theme, delivered in a workshop format at a training event yesterday in Athens.
Update: currently the frame above shows a newer version of the presentation, the one from the PMI day in Hasselt.
Redrawing the Viable System Model diagram
I’ve been arguing repeatedly that trying to get the Viable System Model from overviews, introductions and writings based on or about it, can put the curious mind in a state of confusion or simply lead to wrong interpretations. The absolute minimum is reading at least once each of the three books explaining the model. But better twice. Why? There are at least two good reasons. The obvious one is to better understand some points and pay attention to others that have been probably missed during the first run. But there is also another reason. Books are of linear nature, and when tackling non-linear subjects, a second reading gives the chance to better interpret each part of the text when having in memory other parts which relate to it.
Still, one of the things that are expected to be most helpful is in fact, what brings about either confusion, aversion or misuse: the VSM diagrams. They clearly favour expected ease of understanding over rigour, and yet often they fail in both. Here is my short list of issues, followed by a description of each:
- Representation of the channels
- Confusion about operations and their direct management
- Notation and labelling of systems
- They show something between generic and example model
- Hierarchical implication
Representation of the channels
Stafford Beer admitted several times in his books the “diagrammatic limitations” of the VSM representations. Some of the choices had to do with the limitation of the 2D representation and others, I guess, aimed to avoid clutter. Figure 26 of “The Heart of Enterprise” is a good example of both. It shows eleven loops but implies twenty-one: 3 between environment, operations and management, multiplied by 3 for the choice of showing three operations, then another 9 = 3×3 for loops between same-type elements and finally 3 more between operation management and the meta-system.
Confusion about operations and their direct management
Depending on the context, System One refers either to operations or to their direct management. In some diagrams, S1 is the label of the circles, and in others – of the squares linked to them. Referring to one or the other in the text, depending on which channels are described, only adds to the confusion. That is related to the general problem of
Notation and labelling of systems
All diagrams representing the VSM in the original writings and all interpretations I’ve seen so far suggest that circles represent System One, and triangles pointing up and down, represent System Two and Three* respectively. Additionally, most VSM overviews state exactly that in the textual description. My assertion is almost the opposite:
What is labelled as S1 and what is shown as circles are both not representing S1.
That might come as a shock to many, and yet, now citing Beer, System One is not the “circles” but:
The collection of operating elements (that is, including their horizontal and vertical connexions)
The Heart of Enterprise, page 132
Strictly speaking, a system is a system because it shows emergent properties so it is more than the collection of its parts1that is by itself a popular but problematic statement. but even referring to it as a collection reveals the serious misinterpretation of taking only one of its parts to represent the whole system.
They show something between generic and example model
Communicating such matters to managers trained in business schools wasn’t an easy task. And it is even more challenging nowadays. There is a lot to learn and even more to unlearn. It is not surprising then that even in the generic models typically three operations are illustrated (same for System 2). Yet, I was always missing a true generic representation, or what would many prefer to call “meta-model”.
Hierarchical implication
It can’t be repeated enough that the VSM is not a hierarchical model, and yet it is often perceived and used as such or not used especially because of that perception. It seems that recursivity is a challenging concept, while anything slightly resembling hierarchy is quickly taken to represent one. And sadly, the VSM diagram only amplifies that perception, although the orthogonality of the channels serves an entirely different purpose. Stafford Beer rarely missed an opportunity to remind us about that. Nevertheless, whatever is positioned higher implies seniority and the examples of mapping to actual roles and functions only help in confirming this misinterpretation.
There are other issues as well but my point was to outline the motivation for trying alternative approaches for modelling the VSM, without alternating the essence of the governing principles. Here is one humble attempt to propose a different representation (there is a less humble one which I’m working on, but it’s still too early to talk about it). The following diagram favours circular, instead of orthogonal representation which I hope achieves at least destroying the hierarchical perception. Yet, from a network point of view, the higher positioning of S3 is chosen on purpose as the network clearly shows that this node is a hub.
System One is represented by red colouring, keeping the conventional notation for the operations (S1.o) and their direct management (S1.m). As mentioned above, apart from solving this, the intention is to have it as a generic model. If that poses a problem for those used to the hybrid representation, here’s how it would look if two S1s are shown:
I hope this proposal solves fully or partially the five issues explained earlier and brings a new perspective that can be insightful on its own. In any case, the aim is to be useful in some way. If not as it is now, then by triggering feedback that might bring it to a better state. Or, it can be useful by just provoking other, more successful attempts.
- 1that is by itself a popular but problematic statement.
More on Requisite Inefficiency
The “slides” supporting my talk on Requisite Inefficiency a couple of months ago have been on Slideshare since then, but I haven’t had the time to share them here. Which I do now.
The various manifestations of Requisite Inefficiency in both organisms and organisations can be understood by observing the maintenance of balances between homeostasis and heterostasis (as in the adaptive immune systems), exploration and exploitation (foraging of ants or curiosity-driven vs market-driver research) as well as various types of redundancy or shift of function. The latter can be elastic, as it is in degeneracy, or plastic, as it is in exaptation.
Having an underutilised structure/function that is capable of providing the deficit of variety to the utilised structures of a system in order to match the complexity of an external stimulus, or that can be adapted in a sufficiently short time to do so, is a prerequisite for survival.
Variety, Part 2
etyCan you deal with it?
Deal originates from divide. It initially meant only to distribute. Now it also means to cope, manage and control. We manage things by dividing them. We eat an elephant piece by piece, we start a journey of a thousand miles with a single step, and we divide to conquer.
(This is the second part of a series on the concept variety used as a measure of complexity. You may want to read the previous part before this one, but even doing it after or not at all is fine.)
That proved to be a good way to manage things, or at least some things, and in some situations. But often it’s not enough. To deal with things, and here I use deal to mean manage, understand, control, we need requisite variety. When we don’t have enough variety, we could get it in three ways: by attenuating the variety of what has to be dealt with, by amplifying our variety, or by doing a bit of both when the difference is too big1There yet another way: to change our goal..
And how do we do that? Let’s start by putting some common activities in each of these groups. We attenuate external variety by grouping, categorising, splitting, standardising, setting objectives, filtering, reporting, coordinating, and consolidating. We amplify our variety by learning, trial-and-error, practising, networking, advertising, buffering, doing contingency planning, and innovating. And we can add a lot more to both lists. We use such activities but when doing these activities we need requisite variety as well. That’s why we have to apply them at different scale2Some may prefer to put it more technically as “different level of recursion”.. We learn to split and we split to learn, for example.
What about the third group? What kind of activities can both amplify ours and attenuate the variety of what we need to deal with? It could be easy to put in that third group pairs from each list but aren’t there single types? There are. Here are two suggestions: planning and pretending.
With planning, we get higher variety by being prepared for at least one scenario, especially in the parts of what we can control, in contrast to those not prepared even for that. But then, we reduce different possibilities to one and try to absorb part of the deflected variety with risk management activities.
Planning is important in both operations and projects, and yet, in a business setting, we can get away with poor planning long enough to lose the opportunity to adapt. And that is the case in systems with delayed feedback. That’s also why I like the test of quick-feedback and skin-in-the-game situations, like sailing. In sailing, You are doomed if you sail off without a plan, or if you stick to the plan in front of unforseen events. And that’s valid at every planning level, week, day or an hour.
The second example of activity that both amplifies and attenuates variety is pretending. It can be so successful as to reinforce its application to the extreme. Pretending is so important for stick insects, for example, that they apply it 24/7. That proved to be really successful for their survival and they’ve been getting better at it for the last fifty million years. It turned out to be also so satisfactory that they can live without sex for one million years. Well, that’s for a different reason but nevertheless, their adaptability is impressive. The evolutionary pressure to better resemble sticks made them sacrifice their organ symmetry so that they can afford thinner bodies. Isn’t it amazing: you give up one of your kidneys just to be able to lie better? Now, why do I argue that deception in general, and pretending in particular, has a dual role in the variety game? Stick insects amplify their morphologic variety and through this, they attenuate the perception variety of their predators. A predator sees the stick as a stick and the stick insect as a stick, two states attenuated into one.
Obviously, snakes are more agile than stick insects but for some types that agility goes beyond the capabilities of their bodies. Those snakes don’t pretend 24/7 but just when attacked. They pretend to be dead. And one of those types, the hognose snake, goes so far in their act as to stick its tongue out, vomit blood and sometimes even defecate. That should be not just convincing but quite off-putting even for the hungriest of predators.
If pretending can be such a variety amplifier (and attenuator), pretending to pretend can achieve even more remarkable results. A way to imagine the variety proliferation of such a structure is to use an analogy with the example of three connected black boxes that Stafford Beer gave in “The Heart of Enterprise”. If the first box has three inputs and one output, each of them with two possible states, then the input variety is 8 and the output is 256. Going from 8 to 256 with only one output is impressive but when that is the input of a third black box, having only one output as well, then its variety reaches the cosmic number of 1.157×1077.
That seems to be one of the formulas of the writer Kazuo Ishiguro. As Margaret Atwood put it, “an Ishiguro novel is never about what it pretends to pretend to be about”. No wonder “Never Let Me Go” is so good. And the author, having much more variety than the stick insects, didn’t have to give his organs to be successful. He just made up characters that gave theirs.
- 1There yet another way: to change our goal.
- 2Some may prefer to put it more technically as “different level of recursion”.
Variety, Part 1
The cybernetic concept of variety is enjoying some increase in usage. And that’s both in frequency and in a number of different contexts. Even typing “Ross Ashby” in Google Trends confirms that impression.
In the last two years, the interest seems stable, while in the previous six – it was non-existent, save for the lonely peak in May 2010. Google Trends is not a source of data to draw serious conclusions from, yet it confirms the impression coming from tweets, blogs, articles, and books. On the one hand, that’s good news. I still find the usage insignificant compared to what I believe it should be. Nevertheless, little attention is better than none. On the other hand, it attracts some interpretations, leading to a misapprehension of the concept. That’s why I hope it’s worth exchanging more ideas about variety, and those having more variety themselves would either enjoy wider adoption or those using them – more benefits, or both.
The concept of variety as a measure of complexity had been preceded and inspired by the information entropy of Claude Shannon, also known as the “amount of surprise” in a message. That, although stimulated by the development of communication technologies in the first half of the twentieth century, had its roots in statistical mechanics and Boltzmann’s definition of entropy. Boltzmann, unlike classical mechanics and thermodynamics, defined entropy as the number of possible microstates corresponding to the macro-state of a system.
Variety is defined as the number of possible states in a system. It is also applied to a set of elements. The number of different members determines the variety of a set. It can be applied to the members themselves, which can be in different states, and then the set of possible transitions has a certain variety. This is the first important property of variety. It’s recursive. I’ll come back to this later. Now, to clarify what is meant by “state”:
By a state of a system is meant any well-defined condition or property that can be recognised if it occurs again.
Ross Ashby
Variety can sometimes be easy to count. For example, after opening the game in chess with a pawn on D4, the queen has a variety of three: not to move or move to one of the two possible squares. If only the temporary variety gain is counted, then choosing D2 as the next move would give a variety of 9, and D3 would give 16. That’s not enough to tell if the move is good or bad, especially keeping in mind that some of that gained variety is not effective. However, in case of uncertainty, in games and elsewhere, moving to a place that both increases our future options and decreases those of the opponent seems good advice.
Variety can be expressed as a number, as it was done in the chess example, but in many cases, it’s more convenient to use the logarithm of that number (in case that sounds like a distant memory from school years, nowadays there are easy ways to refresh it in minutes). The common practice, maybe because of the first areas of application, is to use binary logarithms. When that is the case, variety can be expressed in bits. It is indeed more convenient to say the variety of a four-letter code using the English alphabet is 18.8 bits instead of 456 976. There is an extra bonus. When the logarithmic expression is used, varieties of elements are combined by adding instead of multiplying.
Variety is sometimes referred to and counted as permutations. That might be fine in certain cases but as a rule it is not. To use the example with the 4-letter code, it has 358 800 permutations (26 factorial divided by 22 factorial), while the variety is 456 976 (26 to the power of 4).
Variety is relative. It depends on the observer. That’s obvious even from the word “recognised” in the definition of state. If, for example, there is a clock with two hands that are exactly the same or at least to the extent that an observer can’t make the difference, then, from the point of view of the observer, the clock will have a much lower variety than a regular one. The observer will not be able to distinguish, for example, 12:30 and 6:03 as they will be seen as the same state of the clock.
This can be seen as another dependency. That of the capacity of the channel or the variety of the transducer. For example, it is estimated that regular humans can distinguish up to 10 million colours, while tetrachromats – at least ten times more. The variety of the transducer and the capacity of the channel should always be taken into account.
When working with variety, it is useful to study the relevant constraints. If we throw a stone from the surface of Earth, certain constraints, including those we call “gravity” and the “resistance of the air”, would allow a much smaller range of possible states than if those constraints were not present. Ross Ashby made the following observation: “every law of nature is a constraint”, “science looks for laws; it is therefore much concerned with looking for constraints”.
There is this popular way of defining a system as something which is more than the sum of its parts. Let’s see this statement through the lens of varieties and constraints. If we have two elements, A and B, and each can be in two possible states on their own but when linked to each other A can bring B to another, third state, and B can bring A to another state as well. In this case, the system AB has certainly more variety than the sum of A and B unbound. But if, when linking A and B they inhibit each other, allowing one state instead of two, then it is clearly the opposite. That motivates rephrasing the popular statement to “a system might have different variety than the combined variety of its parts”.
If that example with A and B is too abstract, imagine a canoe sprint kayak with two paddlers working in sync and then compare it with a similar setting, with one of the paddlers rowing while the other holds her paddle in the water.
Yet, “is more than the sum of” can be retained but then another modification is needed. Here’s one suggested by Heinz von Foerster:
The measure of the sum of the parts is greater than the sum of the measures of the parts. One is the measure of the sum; the other is the sum of the measures. Take, for example, the measurement function “to square,” which makes this immediately apparent. I have two parts, one is a, the other b. Now I have the measure of the sum of the parts. What does that look like? a + b as the sum of the parts squared, (a + b)2 gives us a2 + 2ab + b2. Now I need the sum of the measures of the parts, and with this I have the measure of a (= a2) and the measure of b (= b2): a2 + b2. Now I claim that the measure of the sums of the parts is greater than the sum of the measures of the parts and state that: a2 + b2 + 2ab is greater than a2 + b2. So the measure of the sum is greater than the sum of the measures. Why? a and b squared already have a relation together
Heinz von Foerster. The Beginning of Heaven and Earth Has No Name (Meaning Systems) (p. 18)
And now about the law of requisite variety. It’s stated as “variety can destroy variety” by Ashby and as “only variety can absorb variety” by Beer, and has other formulations such as “The larger the variety of actions available to control system, the larger the variety of perturbations it is able to compensate”. Basically, when the variety of the regulator is lower than the variety of the disturbance, that gives high variety of the outcome. A regulator can only achieve the desired outcome variety if its own variety is the same or higher than that of the disturbance. The recursive nature mentioned earlier can now be easily seen if we look at the regulator as a channel between the disturbance and the outcome or if we account for the variety of the channels at the level of recursion with which we started.
To really understand the significance of this law, it should be seen how it exerts itself in various situations, which we wouldn’t normally describe with words such as “regulator”, “perturbations” and “variety”.
In the chess example, the power of each piece is a function of its variety, which is the one given by the rules and reduced by the constraints at every move. Was there a need to know about requisite variety to design this game? Or any other game for that matter? Or was it necessary to know how to wage war? Certainly not. And yet, it’s all there:
It is the rule in war, if our forces are ten to the enemy’s one, to surround him; if five to one, to attack him; if twice as numerous, to divide our army into two.
Sun Tzu, The Art of War
Let’s leave the games now and come back to the relative nature of variety. The light signals in ships should comply with the International Regulations for Preventing Collisions at Sea (IRPCS). The agreed signals have a reduced variety to communicate the states of the ships but enough to ensure the required control. For example, if an observer sees one green light, she knows that another ship is passing from left to right. If she sees one red light, it passes right to left. There are lots of states – different angles of the course of the other ship – that are reduced into these two, but that serves the purpose well enough. Now, if she sees both red and green, that means that the ship is coming exactly towards her. That’s a dangerous situation. The reduction of variety, in this case, has to be very low.
The relativity of variety is not only related to the observer’s “powers of discrimination”, or those of the purpose of regulation. It could be dependent also on the context. Easop’s fable “The Fox and the Stork”comes to mind.
Fables, and stories in general, influence people and survive centuries. But is it that do you need a story instead of getting directly the moral of the story? Yes, it’s more interesting, there is this uncertainty element and all that. But there is something else. Stories are ambiguous and interpretable. They leave many things to be completed by the readers and listeners. To put it in different words, they have a much higher variety than morals and values.
That’s it for this part.
And here is the next.
Reasoning with Taskless BPMN
Was it Lisbon that attracted me so much or the word Cybernetics in the sub-title or the promise of Alberto Manuel that it would be a different BPM conference? May be all three and more. As it happened, the conference was very well organised and indeed different. The charm of Lisbon was amplified by the nice weather, much appreciated after the long winter. As to Cybernetics, it remained mainly in the sub-title but that’s good enough if it would make more people go beyond the Wikipedia articles and other easily digestible summaries.
My presentation was about using task-free BPMN which I believe, and the results so far confirm, can have serious benefits for modelling of both pre-defined processes and those with some level of uncertainty. In addition, there is an elegant data-centric way to execute such processes using reasoners. Enterprise Architecture (EA) descriptions can be improved if done with explicit semantics. Currently, EA descriptions are isolated from the operational data and neither the former is linked with what happens, nor the latter get timely updates from the strategy. More on this in another post. Here’s the slidedeck1You can watch on YouTube the slides with animations (no voice-over) and a 7 min compilation of the talk.:
- 1You can watch on YouTube the slides with animations (no voice-over) and a 7 min compilation of the talk.
Requisite Inefficiency
In his latest article Ancient Wisdom teaches Business Processes, Keith Swenson reflects on an interesting story told by Jared Diamond. In short, the potato farmers in Peru used to scatter their strips of land. They kept them that way instead of amalgamating them which would seem like the most reasonable thing to do. This turned out to be a smart risk mitigating strategy. As these strips are scattered, the risk of various hazards is spread and the probability to get something from the owned land every year is higher.
I see that story as yet another manifestation of Ashby’s law of requisite variety. The environment is very complex and to deal with it somehow, we either find a way to reduce that variety in view of a particular objective, or try to increase ours. In a farming setting an example of variety reduction would be building a greenhouse. The story of the Peruvian farmers is a good example of the opposite strategy – increase of the variety of the farmers’ system. The story shows another interesting thing. It is an example of a way to deal with oscillation. The farmers controlled the damage of the lows by giving up the potential benefits of the highs.
Back to the post of Keith Swenson, after bringing this lesson to the area of business process, he concludes
Efficiency is not uniformity. Instead, don’t worry about enforcing a best practice, but instead attempt only to identify and eliminate “worst practices”
I fully agree about best practices. The enforcement of best practices is what one can find in three of every four books on management and in nearly every organisation today. This may indeed increase the success rate in predictable circumstances but it decreases resilience and it is just not working when the uncertainty of the environment is high.
I’m not quite sure about the other advice: “but instead attempt only to identify and eliminate “worst practices”. Here’s why I’m uncomfortable with this statement:
1. To identify and eliminate “worst practice” is a best practice itself.
2. To spot an anti-pattern, label it as “worst-practice” and eliminate it might seem the reasonable thing to do today. But what about tomorrow? Will this “worst-practice” be an anti-pattern in the new circumstances of tomorrow? Or something that we might need to deal with the change?
Is a certain amount of bad practice necessarily unhealthy?
It seems quite the opposite. Some bad practice is not just nice to have, it is essential for viability. I’ll not be able to put it better than Stafford Beer:
Error, controlled to a reasonable level, is not the absolute enemy we have been thought to think of. On the contrary, it is a precondition for survival. […] The flirtation with error keeps the algedonic feedbacks toned up and ready to recognise the need for change.
Stafford Beer, Brain of the firm (1972)
I prefer to call this “reasonable level” of error requisite inefficiency. Where can we see this? In most – if not all – complex adaptive systems. A handy example is the way immune system works in humans and other animals having the so called adaptive immune system (AIS).
The main agents of the AIS are T and B lymphocytes. They are produced by stem cells in the bone marrow. They account for 20-40% of the blood cells which makes about 2 trillion. The way the AIS works is fascinating but for the topic here of requisite inefficiency, what is interesting is the reproduction of the B-cells.
The B-cells recognise the pathogen molecules, the “antigens”, depending on how well the shape of their receptor molecules match that of the antigens. The better the match, the better the chance to be recognised as antigen. And when that is the case, the antigens are “marked” for destruction. Then follows a process in which the T-cells play an important role.
As we keep talking of the complexity and uncertainty of the environment, the pathogens seem a very good model of it.
The best material model of a cat is another, or preferably the same, cat.
N. Wiener, A. Rosenblueth, Philosophy of Science (1945)
What is the main problem of the immune system? It cannot predict what pathogens will invade the body and prepare accordingly. How does it solve it? By generating enormous diversity. Yes, Ashby’s law again. The way this variety is generated is interesting in itself for the capability of cells DNA to carry out random algorithms. But let’s not digress.
The big diversity may increase the chance to absorb that of pathogens but what is also needed is to have match in numbers to have requisite variety. (This is why I really find variety, in cybernetic terms, such a good measure. It is relative. And it can account for both number of types and quantities of the same type.) If the number of matches between B-cell receptors and antigens is enough to register “attack”, the B-cells get activated by the T-cells and start to release antibodies. Then these successful B-cells go to a lymph node where they start to reproduce rapidly . This is a reinforcing loop in which the mutations that are good match with the antigens go to kill invaders and then back to the lymph nodes to reproduce. Those mutations that don’t match antigens, die.
That is really efficient and effective. But at the same time, the random generation of new lymphocytes with diverse shapes continues. Which is quite inefficient when you think of it. Most of them are not used. Just wasted. Until some happen to have receptors that are good match of a new invader. And this is how such an “inefficiency” is a precondition for survival. It should not just exist but be sufficient. The body does not work with what’s probable. It’s ready for what’s possible.
(Note: This is the mainstream explanation of how the immune system work. There are other theories, and some of them – this one for example – I find way more convincing, especially when comes to the self/non-self problem. However, in all explanations the phenomenon of requisite inefficiency is equally prominent. )
The immune system is not the only complex system having requisite inefficiency. The brain, the swarms, the networks are just as good examples. Having the current level of study, the easiest systems to see it in are ant colonies.
When an ant finds food, it starts to leave a trail of pheromones. When another ant encounters the trail, it follows it. If it reaches the food, the second ant returns to the next leaving trail as well. The same reinforcing loop we saw with the B-cells, can be seen with ants. The more trails, the more likely the bigger number of ants will step on it, follow it, leave more pheromones, attract more ants and so on. And again, at the same time there always is a sufficient amount of ants moving randomly which can encounter new location with food.
The requisite inefficiency is equally important for social systems. Dave Snowden gave a nice example coincidently again with farmers but in that case experiencing high frequency of floods. Their strategy was to build their houses not in a way to prevent the water coming in but to allow the water to quickly come out. He calls that “architecting for resilience”:
You build your system on the assumption you prevent what can fail but you also build your system so you can recover very very quickly when failure happens. And that means you can’t afford an approach based on efficiency. Because efficiency takes away all superfluous capacity so you only have what you need to have for the circumstances you anticipate. […] You need a degree of inefficiency in order to be effective.
It seems we have a lot to learn from B-cells, ants and farmers about how to make our social systems work better and recover quicker. And contrary to our intuition, there is a need for some inefficiency. The interesting question is how to regulate it or how to create conditions for self regulation. For a given system, how much inefficiency is insufficient, how much is just enough and when it is too much? May be for the immune systems and ant colonies these regulatory mechanisms are already known. The challenge is to find them for organisations, societies and economies. How much can we use from what we already know for other complex adaptive systems? Well, we also have to be careful with analogies. Else, we might fall into the “best practice” trap.
(See also More on Requisite Inefficiency)
Frameworks and rigour
This is in response to the recent article of Richard Veryard “Arguing with Mendeleev”. There he comments on Zachman’s comparison of his framework with the periodic table of Mendeleev. And indeed there are cells in both tables with labelled columns (called “groups” in Mendeleev’s) and rows (“periods” respectively). Another similarity is that both deal with elements and not compounds. The same way the study of properties of oxygen and hydrogen will tell you nothing about the properties of water, the study of any two artefacts from Zachman framework will tell you nothing of how the real things they stand for work together. In fact you may not even get much about the separate properties of what the artefacts represent. Anyway, if there are any similarities, this is where they end.
I’ll not spend much time on differences. They are too many. But let me just mention two. The periodic table is a classification based on the properties of the elements. The order is determined by atom numbers and electron configuration. Both groups and periods have commonalities which make them an appropriate classification scheme. Once the rules are established, the place of each element can be confirmed by independent experiments and observations. That’s not the case with Zachman’s framework.
Richard comments on the statement that Zachman’s scheme is not negotiable:
What makes chemistry a science is precisely the fact that the periodic table is open to this kind of revision in the light of experimental discovery and improved theory. If the same isn’t true for the Zachman Framework, then it can hardly claim to be a proper science.
I haven’t heard the claim that Zachman’s framework is a “proper science” before. In my opinion, Zachman’s main contribution is not his framework as such but the fact that it created a new discipline and new profession. The scheme itself is arbitrary. The columns, as we know, are based on six of the interrogatives: what, how, where, who, when, and why. Whether is missing, also how much. In the old English there is also whither, which is similar to where but has an important specifics – it is related to direction (whereto). But I’m not questioning the number of columns. I have doubts about their usefulness in general.
Let’s just take three of the of the interrogatives: what, how and why and test some questions:
1. What do you do now? Answer: B
2. Why do you do B? Answer: because of A
3. How do you do B? Answer: by means of C
And now with a simple example:
B = buy food
A = I’m hungry
C = going to the supermarket
Now let’s focus on the answers and ask questions to learn more. First on C:
I’m going to the supermarket.
Why? Answer: to buy food
Why you need to buy food? Answer: because I’m hungry
Now let’s focus on A:
I’m hungry. Well, this is a problem. So we can ask:
How can I solve A? Answer: by doing B
How can I do B? Answer: by doing C
So if the relativity of the focus is ignored then what is equal to why is equal to how. (Speaking of focus, or perspective, this is where the rows in the framework come to play. This is a nice game itself which we’ll play another time).
In this example the answer of what is related to activities and not to their object (food) which by itself questions how appropriate is to label the data column “what”.
But of course rigour is not imperative. And neither is logic. After all it shifted its function from a tool to win arguments into a tool to seek truth. And then logic is quite materialistic, while EA seems a spiritual practice. Which reminds me also of the mind over matter one-liner:
If you don’t mind, it doesn’t matter.
drEAmtime
In the Australian Aboriginal culture, Dreamtime “is a complex network of knowledge, faith, and practices”. This definition can be reused for another human activity, Enterprise Architecture(EA), and Dreamtime fairly describes its current state1With this, I’d like to depart from further analogies as I respect the culture of Aboriginal people in general in the part related to Dreamtime. I’ll refer to drEAmtime in this article solely as to what I currently see as the EA’s state of play..
The drEAm of the common language
It is believed for a long time now that there is a widespread misalignment between ‘Business’ and ‘IT’. The IT in this context is used to refer to employees that carry out activities closely related to development, procurement and maintenance of information systems, and ‘Business’ – to those who don’t.
The division of labour, which is the predominant way of structuring organisations, naturally invites different specialists with different backgrounds. Naturally, they use different terms or the same but with different meanings. Even if this is the same between accounting and production, the miscommunication between IT and the rest gets more attention. Maybe it is due to the increasing dependency on IT combined with stories of spectacular failure. Or it might be because IT is an area with a high rate of innovation, and the sense of leading doesn’t go well with the need to follow orders from those belonging to somewhat lagging ones. In any case, there is general agreement about the problem of miscommunication and the associated high cost.
Here comes Enterprise Architecture to the rescue. By doing what? Introducing a third language.
What’s even worse, this language is ostensibly similar to both ‘Business’ and ‘IT’. Just check some EA terms to see how people from ‘Business’ and ‘IT interpret them. Take, for example, the most common ones like service, capability, function, model, viewpoint and artefact. The resulting situation looks something like this:
It looks absurd, but let’s imagine for a moment that it’s not. At best, we would see the following vicious circle. The strong IT smell of the EA language decreases the chance of the Business believing that they would benefit from learning it, hence the low motivation and buy-in. Even though the cognitive load for IT is much lower, seeing the willingness of the Business to ‘improve’ its literacy, IT people find better things to be busy with. And another funny point. When there is some buy-in by the Business, EA is put under IT, not between business and IT. Then EA people complain they can’t do much from there. But maybe they are put there just because they can’t do much.
Closely related to the dream of a common language is
The drEAm of bridging the silos
That it is a dream should be already obvious from the previous, about the common language. But here is another aspect. The EA people build bridges between silos that turn into new silos. At a certain point, they find themselves in a position where neither business nor IT regards them as a bridge anymore. Business people trust EA people even less than IT because they see them as cross-dressed IT. IT people lose trust in EA as well because they are not sure they understand the Business and if they still remember what IT was. Further, IT managers need support, which EA does not have the authority to ensure.
And there is an additional effect. EA is often in the position to attract some serious budgets for reasons we’ll see in another dream, and this way, the new island becomes a safe territory for people who have either failed or lost interest in pure IT. This further decreases the credibility of enterprise architecture and in some organisations, they get the image of people who are not good enough for IT and prefer to hide under EA labels where things are vague enough and difficult to measure. The lost credibility either undermines the work of the good EA practitioners or pushes them out of the organisation or both.
But what part of the work of EA is quantifiable? One tangible result of successful EA seems to be the rationalisation of the application landscape. And as this brings efficiency, it is easier to measure. This I call
The drEAm of IT cost reduction
Big organisations in all sectors, especially in the service industries, tend to amass a huge number of applications until there are far too many to manage. A good number of them are not used at all. Others are underutilised. Most of the critical applications have high maintenance costs, high replacement costs, or both. Inevitably, there are many which automate different parts of the same process, but they don’t talk to each other. And this justifies new spending on building interfaces or buying application integration packages first and then replacing them with BPMS and then probably with something better than BPMS. As a result, there is more spending and more applications to manage.
Application integration problems are accompanied by problems of duplication of IT investments. Existing functionalities are duplicated in one or more applications to some extent or completely.
Yet another common situation is patchwork applications. Those are applications that have certain utility but don’t quite well meet the needs, or just the needs change. In any case it is found better to add the missing functionality instead of replacing the whole application. And then, again and again, layers of patches of additions and fixes until we have a real monster and the roles of the master and servant are swapped.
One day all these silo applications, functional redundancies, patchwork systems, and suchlike create a serious mess and shocking numbers in the financial statements that are serious enough to convince the top management that something has to be done rather soon.
But just when the situation seems really critical, the door opens with a kick and EA cowboys enter. They pull out frameworks and architecture tools from their holsters and in slow motion (a very slow motion), they shoot inefficiency after inefficiency until all of them lie dead on the floor. Then they walk out and go to shoot inefficiencies in some other town and when new inefficiencies appear in this town they come back again to kill them out.
Here is what actually happens. Some attempts to achieve IT rationalisation fail spectacularly. I’m not going to list out the reasons for that. It is maybe sad that such failures discredit EA as a management discipline as a whole. But sometimes, enterprise architects are really able to find ways to discover what’s not needed and how to remove it, what is underutilised, and how to achieve better ROI for it. After all, most of them are smart people using good tools. And indeed, they shoot inefficiencies, and some get all the glory and the money to shoot more. However, as they rarely get to the cause of the inefficiencies or are in a position to influence the bigger system that produces these inefficiencies, the overall result is an increase in overall IT spending. Why? The success of such EA efforts justifies a bigger EA budget, which is almost without exception a part of the IT budget.
The drEAm of dealing with complexity
This dream has specifics of its own, but it can also be used to explain the whole dilemma.
If you are an Enterprise Architect, a popular way to deal with complexity is to arm yourself with a framework. It is believed that you can do two things with a good framework. First, reduce the complexity of the enterprise to just a few things that share the same properties, according to some classification theory and where things doesn’t fit, add more layers of abstraction. And second, reduce the things you can possibly do to just a few but well-defined and in a specific order, with well-prescribed inputs and outputs. That was common for so many organisations that did well that it became a best practice, and the chances are, if you follow this way, it will do you well as well. Now, because of the shared understanding of the beneficial role of the abstract layers, and the boundaryless imagination unconstrained by reality, there is a serious number of frameworks and, on top of them, other works on how to adapt and adopt them.
Then of course modelling itself is believed to help in dealing with complexity. But what kind of modelling? A very complicated architecture diagram does not show complexity. It just shows a lot of effort spent in denial of it.
The part related to complexity certainly deserves a separate post and maybe more than one. For now, let me just finish with the following: dealing with complexity is not reduced to finding ways to reduce it. It requires a different understanding of what happens when interactions are not linear when there is context, history, emergence, adaptation, politics and power. Complexity is a property of the enterprise, not a problem to be solved.
In summary, more often than not, when contemporary mainstream EA is trying to introduce a common language, it creates confusion and additional work to deal with it. When trying to bridge the silos, it creates new silos instead. When trying to reduce the IT spending, it, in fact, makes no change or increases them. When trying to deal with complexity, it’s just pathetic.
- 1With this, I’d like to depart from further analogies as I respect the culture of Aboriginal people in general in the part related to Dreamtime. I’ll refer to drEAmtime in this article solely as to what I currently see as the EA’s state of play.
Modernising Job Titles
HR Manager: What do you do?
DB Designer: I work with databases.
HR Manager: More specifically?
DB Designer: I design them.
HR Manager: Good, so you are an Architect!
DB Designer: Well, yes, you can put it that way, I guess.
HR Manager: Who uses the databases you design?
DB Designer: Many departments, in fact a big part of the enterprise.
HR Manager: Great, so you are an Enterprise Architect. That’s your new title, congratulations!
DB Designer: …!?
(related post: Where to start?)