Requisite Inefficiency

In his latest article Ancient Wisdom teaches Business Processes, Keith Swenson reflects on an interesting story told by Jared Diamond. In short, the potato farmers in Peru used to scatter their strips of land. They kept them that way instead of amalgamating them which would seem like the most reasonable thing to do. This turned out to be a smart risk mitigating strategy. As these strips are scattered, the risk of various hazards is spread and the probability to get something from the owned land every year is higher.

I see that story as yet another manifestation of Ashby’s law of requisite variety. The environment is very complex and to deal with it somehow, we either find a way to reduce that variety in view of a particular objective, or try to increase ours. In a farming setting an example of variety reduction would be building a greenhouse. The story of the Peruvian farmers is a good example of the opposite strategy – increase of the variety of the farmers’ system. The story shows another interesting thing. It is an example of a way to deal with oscillation. The farmers controlled the damage of the lows by giving up the potential benefits of the highs.

Back to the post of Keith Swenson, after bringing this lesson to the area of business process, he concludes

Efficiency is not uniformity.  Instead, don’t worry about enforcing a best practice, but instead attempt only to identify and eliminate “worst practices”

I fully agree about best practices. The enforcement of best practices is what one can find in three of every four books on management and in nearly every organisation today. This may indeed increase the success rate in predictable circumstances but it decreases resilience and it is just not working when the uncertainty of the environment is high.

I’m not quite sure about the other advice: “but instead attempt only to identify and eliminate “worst practices”. Here’s why I’m uncomfortable with this statement:

1. To identify and eliminate “worst practice” is a best practice itself.

2. To spot an anti-pattern, label it as “worst-practice” and eliminate it might seem the reasonable thing to do today. But what about tomorrow? Will this “worst-practice” be an anti-pattern in the new circumstances of tomorrow? Or something that we might need to deal with the change?

Is a certain amount of bad practice necessarily unhealthy?

It seems quite the opposite. Some bad practice is not just nice to have, it is essential for viability. I’ll not be able to put it better than Stafford Beer:

Error, controlled to a reasonable level, is not the absolute enemy we have been thought to think of. On the contrary, it is a precondition for survival. […] The flirtation with error keeps the algedonic feedbacks toned up and ready to recognise the need for change.

Stafford Beer, Brain of the firm (1972)

I prefer to call this “reasonable level” of error requisite inefficiency. Where can we see this? In most – if not all – complex adaptive systems. A handy example is the way immune system works in humans and other animals having the so called adaptive immune system (AIS).

The main agents of the AIS are T and B lymphocytes. They are produced by stem cells in the bone marrow. They account for 20-40% of the blood cells which makes about 2 trillion. The way the AIS works is fascinating but for the topic here of requisite inefficiency, what is interesting is the reproduction of the B-cells.

The B-cells recognise the pathogen molecules, the “antigens”, depending on how well the shape of their receptor molecules match that of the antigens. The better the match, the better the chance to be recognised as antigen. And when that is the case, the antigens are “marked” for destruction. Then follows a process in which the T-cells play an important role.

As we keep talking of the complexity and uncertainty of the environment, the pathogens seem a very good model of it.

The best material model of a cat is another, or preferably the same, cat.

N. Wiener, A. Rosenblueth, Philosophy of Science (1945)

What is the main problem of the immune system? It cannot predict what pathogens will invade the body and prepare accordingly. How does it solve it? By generating enormous diversity. Yes, Ashby’s law again. The way this variety is generated is interesting in itself for the capability of cells DNA to carry out random algorithms. But let’s not digress.

The big diversity may increase the chance to absorb that of pathogens but what is also needed is to have match in numbers to have requisite variety. (This is why I really find variety, in cybernetic terms, such a good measure. It is relative. And it can account for both number of types and quantities of the same type.) If the number of matches between B-cell receptors and antigens is enough to register “attack”, the B-cells get activated by the T-cells and start to release antibodies. Then these successful B-cells go to a lymph node where they start to reproduce rapidly . This is a reinforcing loop in which the mutations that are good match with the antigens go to kill invaders and then back to the lymph nodes to reproduce. Those mutations that don’t match antigens, die.

That is really efficient and effective. But at the same time, the random generation of new lymphocytes with diverse shapes continues. Which is quite inefficient when you think of it. Most of them are not used. Just wasted. Until some happen to have receptors that are good match of a new invader. And this is how such an “inefficiency” is a precondition for survival. It should not just exist but be sufficient. The body does not work with what’s probable. It’s ready for what’s possible.

(Note: This is the mainstream explanation of how the immune system work. There are other theories, and some of them  – this one for example – I find way more convincing, especially when  comes to the self/non-self problem. However, in all explanations the phenomenon of requisite inefficiency is equally prominent. )

The immune system is not the only complex system having requisite inefficiency. The brain, the swarms, the networks are just as good examples. Having the current level of study, the easiest systems to see it in are ant colonies.

When an ant finds food, it starts to leave a trail of pheromones. When another ant encounters the trail, it follows it.  If it reaches the food, the second ant returns to the next leaving trail as well. The same reinforcing loop we saw with the B-cells, can be seen with ants. The more trails, the more likely the bigger number of ants will step on it, follow it, leave more pheromones, attract more ants and so on. And again, at the same time there always is a sufficient amount of ants moving randomly which can encounter new location with food.

The requisite inefficiency is equally important for social systems. Dave Snowden gave a nice example coincidently again with farmers but in that case experiencing high frequency of floods. Their strategy was to build their houses not in a way to prevent the water coming in but to allow the water to quickly come out. He calls that “architecting for resilience”:

You build your system on the assumption you prevent what can fail but you also build your system so you can recover very very quickly when failure happens. And that means you can’t afford an approach based on efficiency. Because efficiency takes away all superfluous capacity so you only have what you need to have for the circumstances you anticipate. […] You need a degree of inefficiency in order to be effective.

It seems we have a lot to learn from B-cells, ants and farmers about how to make our social systems work better and recover quicker. And contrary to our intuition, there is a need for some inefficiency. The interesting question is how to regulate it or how to create conditions for self regulation. For a given system, how much inefficiency is insufficient, how much is just enough and when it is too much? May be for the immune systems and ant colonies these regulatory mechanisms are already known. The challenge is to find them for organisations, societies and economies. How much can we use from what we already know for other complex adaptive systems? Well, we also have to be careful with analogies. Else, we might fall into the “best practice” trap.

(See also More on Requisite Inefficiency)

Frameworks and rigour

This is in response to the recent article of Richard Veryard “Arguing with Mendeleev”. There he comments on Zachman’s comparison of his framework with the periodic table of Mendeleev. And indeed there are cells in both tables with labelled columns (called “groups” in Mendeleev’s) and rows (“periods” respectively). Another similarity is that both deal with elements and not compounds. The same way the study of properties of oxygen and hydrogen will tell you nothing about the properties of water, the study of any two artefacts from Zachman framework will tell you nothing of how the real things they stand for work together. In fact you may not even get much about the separate properties of what the artefacts represent. Anyway, if there are any similarities, this is where they end.

I’ll not spend much time on differences. They are too many. But let me just mention two. The periodic table is a classification based on the properties of the elements. The order is determined by atom numbers and electron configuration. Both groups and periods have commonalities which make them an appropriate classification scheme. Once the rules are established, the place of each element can be confirmed by independent experiments and observations. That’s not the case with Zachman’s framework.

Richard comments on the statement that Zachman’s scheme is not negotiable:

What makes chemistry a science is precisely the fact that the periodic table is open to this kind of revision in the light of experimental discovery and improved theory. If the same isn’t true for the Zachman Framework, then it can hardly claim to be a proper science.

I haven’t heard the claim that Zachman’s framework is a “proper science” before. In my opinion,  Zachman’s main contribution is not his framework as such but the fact that it created a new discipline and new profession. The scheme itself is arbitrary. The columns, as we know, are based on six of the interrogatives: what, how, where, who, when, and why. Whether is missing, also how much. In the old English there is also whither, which is similar to where but has an important specifics – it is related to direction (whereto). But I’m not questioning the number of columns. I have doubts about their usefulness in general.

Let’s just take three of the of the interrogatives: what, how and why and test some questions:
1. What do you do now? Answer: B
2. Why do you do B?  Answer: because of A
3. How do you do B? Answer: by means of C

And now with a simple example:

B = buy food

A = I’m hungry

C = going to the supermarket

Now let’s focus on the answers and ask questions to learn more. First on C:
I’m going to the supermarket.
Why? Answer: to buy food
Why you need to buy food? Answer: because I’m hungry

Now let’s focus on A:
I’m hungry. Well, this is a problem. So we can ask:
How can I solve A? Answer: by doing B
How can I do B? Answer: by doing C

So if the relativity of the focus is ignored then what is equal to why is equal to how. (Speaking of focus, or perspective, this is where the rows in the framework come to play. This is a nice game itself which we’ll play another time).

In this example the answer of what is related to activities and not to their object (food) which by itself questions how appropriate is to label the data column “what”.

But of course rigour is not imperative. And neither is logic. After all it shifted its function from a tool to win arguments into a tool to seek truth. And then logic is quite materialistic, while EA seems a spiritual practice. Which reminds me also of the mind over matter one-liner:

If you don’t mind, it doesn’t matter.