Same as Magic

When I started my journey in the world of Semantic Web Technologies and Linked Data, I couldn’t quite get what was all that fuss about the property owl:sameAs. Later I was able to better understand the idea and appreciate it when actively using Linked Data. But it wasn’t until I personally created graphs from heterogeneous data stores and then applied different strategies for merging them, when I realised the “magical” power of owl:sameAs.

The idea behind “same as” is simple. It works to say that although the two identifiers linked with it are distinct, what they represent is not.

Let’s say you what to bring together different things recorded for and by the same person Sam E. There is information in a personnel database, his profile and activities in Yammer, LinkedIn, Twitter, Facebook. Sam E. is also somebody doing research, so he has publications in different online libraries. He also makes highlights in Kindle and check-ins in Four-square.

Let’s imagine that at least one of the email addresses recorded as Sam E’s personal email is used in all these data sets. Sam E. is also somehow uniquely identified in these systems, and it doesn’t matter if the identifiers use his email or not. When creating RDF graphs from each of the sources,  URI for Sam E. should be generated in each graph if such doesn’t exist already. The only other thing needed is to declare that Sam E’s personal email is the object of foaf:mbox, where the subject is the respective URI for Sam E from in each of the data sets.

The interesting thing about foaf:mbox is that it is “inverse functional”. When a property is asserted as owl:inverseFunctionalProperty, then the object uniquely identifies the subject in that statement. To get what that means, let’s first see the meaning of a functional property in OWL. If Sam E. “has birth mother” Jane, and Sam E. “has birth mother” Marry, and “has birth mother” is declared as functional property, a DL reasoner will infer  that Jane and Marry are the same person. The “inverse functional” works the same way in the opposite direction. So if Sam.E.Yammer has foaf:mbox “sam@example.com”, and Sam.E.Twitter has foaf:mbox “sam@example.com”, then Sam.E.Yammer refers to the same person as Sam.E.Twitter. That is because a new triple Sam.E.Yammer–owl:sameAs–Sam.E.Twitter is inferred as a consequence of foaf:mbox being owl:inverseFunctionalProperty. But that single change brings a massive effect: all facts from Yammer about Sam E are inferred for Sam E from Twitter and vice versa. And the same applies for LinkedIn, Facebook, online libraries, Four-square and so on.

Now, imagine you don’t do that for Sam E, but for all your Twitter network. Then you’ll get a graph that will be able to answer questions such as “From those that tweeted about topic X within my network, give me the names and emails of all people that work within 300 km from here”, or “Am I in a same discussion group with somebody that liked book Y?”. But wait, you don’t need to imagine it, you can easily do it. Here is for example one way to turn Twitter data into an RDF graph.

Of course, apart for persons, similar approaches can be applied for any other thing represented on the web: organisations, locations, artefacts, chemical elements, species and so on. 

To better understand what’s going on, it’s worth reminding that there is no unique name assumption in OWL. The fact that two identifiers X and Y are different, does not mean that they represent different things. If we know or if it can be deduced that they represent different things, this can be asserted or respectively inferred as a new triple X–owl:differentFrom–Y. In a similar way a triple saying just the opposite X–owl:sameAs–Y can be asserted or inferred. Basically, as long as sameness is concerned, we can have three states: same, different, neither same nor different. Unfortunately, a fourth state, both same and different, is not allowed, and why would that be of value will be discussed in another post. Now, let’s get back to the merging of graphs. 

Bringing the RDF graphs about Sam E, created from the different systems, would link them in one graph just by using foaf:mbox. Most triple stores like Virtuoso, would do such kind of basic inferencing at run time. If you want to merge them in an ontology editor, you have to use a reasoner such as Pallet if you are using Protégé, or run inferencing with SPIN, if you are using TopBraid Composer. Linking knowledge representation from different systems in a way independent from their underlying schemas, can bring a lot of value, from utilising chains of relations to learning things not known before linking.

The power of “same as” has been used a lot for data integration both in controlled environments and in the wild. But let’s not forget that in the latter, the web,   “Anyone can say anything about anything”. This was in fact one of the leading design principles for RDF and OWL. And then even with the best intentions in mind, people can create a lot of almost “same as” relations that would be mixed with the reliable “same as” relations. And they did and they do.

The problems with “same as” have received a lot of attention. In one of the most cited papers on the subject, Harry Halpin et al. outline four categories of problems for owl:sameAs: “Same Thing As But Different Context”; “Same Thing As But Referentially Opaque”, “Represents”, and “Very Similar To”. Others worn about problems with provenance. Still, almost all agree that the benefits for the owl:sameAs for Linked Data by far outnumber the risks, and the latter can be mitigated by various means.

Whatever the risks with owl:sameAs in the web, they are insignificant, or non-existent in corporate environments. And yet, most of the data stays in silos and it gets integrated only partially and based on some concrete requirements. These requirements represent local historical needs and bring expensive solutions to local historical problems. Those solutions typically range from point-to-point interfaces with some ETL, to realisation of REST services. They can get quite impressive with the means for access and synchronisation, and yet they are all dependant on the local schemas in each application silo and helpless for enterprise-wide usage or any unforeseen need. What those solutions bring is more complicated application landscape, additional IT investments brought by any change of requirements and usually a lot of spendings for MDM software, data warehouses and suchlike. All that can be avoided if the data from heterogeneous corporate and open data sources is brought together into an enterprise knowledge graph, with distributed linked ontologies and vocabularies to give sense to it, and elegant querying technologies, that can bring answers, instead of just search results. The Semantic Web stack is full of capabilities such as owl:sameAs, that make this easy, and beautiful. Give it a try.

 

The Infatuation with "Building Blocks"

Why the majority of Enterprise Architects are in love with the concept of “building block”. Perhaps because blocks don’t answer back and spoil the design, Aiden Ward suggested.

In TOGAF 9.1 “building block” is used 349 times, not counting the separate use of ABB and SBB acronyms. There is a whole chapter devoted to building blocks. In there, we find the following:

A building block is a package of functionality defined to meet the business needs across an organization.

A building block has a type that corresponds to the TOGAF content metamodel (such as actor, business service, application, or data entity)

A building block has a defined boundary and is generally recognizable as ‘‘a thing’’ by domain experts.

Now, if a “building block” is a “package of functionality”, taking it together with the second statement, then “actor” and “data entity” are – according to TOGAF – types of “package of functionality”. That doesn’t bother me so much, as logic and consistency have never been among the strong sides of TOGAF. I’m more concerned with the last statement. There are indeed some entities from the model that are recognised by domain exerts as “things” with defined boundary, but I haven’t heard a domain expert using “building block” to discuss their business. Even if there are some that do, having surrendered to the charms of eloquent Enterprise Architects, I don’t think they  use that language among themselves or that they need to.

Enterprise Architecture is not unique in using technology metaphors for understanding its organisation. After some circulation of the brain as metaphor for computer, then the computer returned the favour and have been used for decades as a metaphor for the brain. Not even that, it tuned the thinking of some cognitive scientists in such a way,  that many of them managed to “prove” that the brain is doing information processing. That school of thought, the so called computationalists, has still a lot of followers today. The same school regards the mind “running” on the body just like software is running on hardware. That approach has been doing much more harm on an array of disciplines, than the usage of building blocks on Enterprise Architecture. But qualitatively, it is a similar phenomenon.

It’s now time to choose some structure for building the point about my puzzlement (and then hopefully solving it), and I select the following titles for my text blocks: name, history, and purpose.

Name

The primary source of irritation – I have to admit – is the name itself. But it is also a minor one, so we can quickly deal with it.

Where is the name coming from?

EA is often perceived by non-EA people as something coming from IT which is not even proper IT. The IT parlance in general has borrowed a lot of terms from other engineering practices. Software development in particular has shown preference for construction engineering over – what would have seemed a more natural choice – electronic engineering. And that’s why we are “building” software. Having in mind how iterative that is, if that “building” is brought back and applied to actual buildings, there won’t be many happy investors or inhabitants.

Anyway, “to build”applications is well established and doing well in software development. But what is the utility of “building block” in Enterprise Architecture? Indeed, representing some abstractions as boxes, and placing them one over another resembles playing with Lego. The metaphor is bringing a more than welcome simplicity to the architecture discussions. But, apart from smoothing communication between Enterprise Architects, is it useful for anything else? And how does the name reflect the actual use of  the concept?

My quick answer is: there is nothing that can be built using architecture building blocks. Something can be drawn and then reconfigured in another drawing. Isn’t it then more appropriate to talk about “description” blocks or “design” blocks?

Some Enterprise Architects are well aware of that. In fact it seems that the choice of  “building block” over “component” is exactly to remind the users that the former is not the latter but just an abstract representation of it. This has been debated for about a decade, and various solutions have been proposed, including “reusable” blocks, which would make the purpose explicit, at least.

But this awareness is neither widespread not persistent enough to prevent the stabilisation of some thinking habits leading to decisions that look good only within the protected virtual reality of architecture descriptions.

History

There are two lines of history we can follow. One is the evolution of “building block” in the Enterprise Architecture discipline; the other is how a building block comes about in a concrete EA practice. If that was about species, a biologist would call the first line phylogenesis, and the second one – ontogenesis.

Following the  first one would probably reveal interesting things but would require an effort that the concept does not justify for me. That’s why I’ll limit tracing that history to the hypothesis of the origin I discussed in the Name section.

A building block in a concrete enterprise architecture practice is typically coming as specialisation of a generic element. That generic element, called concept, stereotype or construct, is based on assumptions about what are, based on a history and knowledge of different concerns at the moment of the meta-design, the important patterns that need to be represented. These are, if we take Archimate as a popular example, things like Role, Event, Process, Service, Interface, and Device. Their application however, is limited by three restrictions:

  • first, when an entity is represented it has to be as one of these types;
  • second, the criteria for making the choice is coming from interpreting definitions of these concepts and not from what the represented object from the real world has or does (there are generalised relations, but they are not used for that);
  • third, something cannot be represented as more than one concept.

So, a building block, being a specialisation of a concept, asserted in this way, inherits both this set of restrictions and everything else that’s coming from the object-oriented paradigm, which the use of “specialise” and “inherits” implied already.

Purpose

The use of building blocks in Enterprise Architecture is either directly supporting, or historically linked with, reusability.

A building block represents a (potentially re-usable) component of business, IT, or
architectural capability that can be combined with other building blocks to deliver
architectures and solutions. (TOGAF 9.1, p.11)

In order to support reusability building blocks should have the right level of granularity. Basically, they need to balance between flexibility and efficiency. In order to be reused, which would mean recombined, they shouldn’t be too coarse-grained, otherwise there won’t be enough flexibility. On the other hand, if there are too fine-grained, it won’t be efficient to maintain reference architectures.

Now, this seems key for understanding the popularity of “building block”. Enterprise Architects have to justify their reason-to-be. As both reusability and flexibility make a good business case, stabilisation of that concept could be just due to survival pressures. And then Enterprise Architecture in order to exist as such, should not only be allowed and fed but should have ways to assert itself as a social system. And the “building blocks” provide one of the conventions that make the network of communications within the EA practice sustain its identity.

Fine then, problem solved: Enterprise Architecture – being predominantly practised  the way it is now – needs concepts such as “building blocks” in order to survive as economic entity, and as social system.

And yet, Enterprise Architecture can be practised and can bring more benefits without building blocks, layers, tiers and suchlike. It can be done by not reducing what is learned in the process to what was decided at the moment of knowing much less when the meta-model was fixed. The objects that need to be described and designed can get their type from how their descriptions satisfy the conditions for classification. And last but not least, it could be done entirely in the language that domain experts use and “building block” is not used by them, at least not in the service industry. When it is used by actual builders, I doubt they mean “package of functionality”.

What’s Wrong with Best Practices?

That is what several people asked me in the last few weeks. I have even made one quick attempt to answer but then I realised that the topic deserves more clarification.

While I have a lot of sympathy for those that object to ‘Best Practice’ as a name – even more – to those that object it as a claim, my uneasiness is somewhat different. I’ll try to shortly explain what I know about it.

Some best practices are very useful. In fact most mature and adequately applied best practices for achieving some technical task from painting a wall and repairing an engine, to building a factory or a ship, are indeed valuable, as long as they don’t suffocate innovation.

The real problem is when best practices are applied to people and social systems. I call this a ‘problem’, but it is in fact a huge opportunity for many. Most of the contemporary non-fiction books, especially management and self-help texts, have been seizing this opportunity extremely well. It’s not easy to find a best seller in this category, or a popular article, that doesn’t provide some sort of prescriptions and advices, often numbered, on how to achieve or avoid something. May be it is a best practice for best sellers. Let’s give it a try then:

 

Four Reasons Why You Should Be Cautious When Applying Best Practices:

 

1. Correlations.

How Best Practices come about? Some individuals or organisations, become known, or are later made known by the actions of the best practice discoverers and proponents, as successful according to some norms. Let’s call these individuals or organisations ‘best practice pioneer’. Then one or more observers, the ‘best practice discoverer’, studies the pioneers to find out what made them successful. The discoverer first takes certain effects, and then selects, by identifying commonalities, what she or he believes were the causes. That is followed by generalisation of the commonalities, from which point they begin their life as prescriptions, regardless if they are called ‘best practices’, ‘methodologies’, ‘techniques’, ‘recipes’,’templates’, or something else. Later they are tested, and based on feedback, reappear in more mature forms and variations, in some cases supplied with scalability criteria and conditions for successful application. The successes and failures of those that apply them give birth to Best Practices on applying Best Practices.

The problem is, that the discoverers select common patterns among the observed successful pioneers, and infer causal relation between these commonalities and those that were the criteria to select that set of pioneers in the first place. Then such correlation leads the discoverer to select what to pay attention to, and every pattern that support the hypothesis based on the selected commonalities, would be preferred over those that don’t.

2. The risk of over-simplification.

Best practices help us deal with external stimuli, when they are too many to handle, by prompting which ones to pay attention to and how to react to them. The only way to deal with a situation is when the number of responses is higher than the number of the stimuli, within given goal set (Ashby’s law). Or in other words, best practices are tools for reducing external variety. But not only that, they also provide means for amplifying internal variety in a special way – coupled with those stimuli that you are advised to pay attention to. So if that assumption was wrong, and it often is, neither the external variety is reduced, nor the internal is amplified.

3. Assumptions about the application context.

I used to be a practitioner of PRINCE2. What I still appreciate is a few smart techniques and in fact – the name itself. The name is the important disclaimer I’m missing in most methodologies and other types of best practices: they only work in controlled environments. They work when most of the conditions of the design-time, if you allow me the IT jargon, are unchanged in run-time. This is rarely the case and increasingly less so. Which brings another interesting phenomenon: the same conditions that make the world less predictable also help quickly productise and spread best practices. They come with better marketing and with more authority in a world in which less of what has happened could prepare you for what will.

4. The habits created by Best Practices.

The worst is when people hide behind the authority of best practices or their proponents. If not that, best practices create habits of first looking for best practices, instead of thinking. And then, there is the alternative cost: the more time people spend on learning best practices, the less time they have for developing their senses for detection of weak signals and for developing their capabilities for new responses.

In summary, if you are sure that certain best practice is useful, and it’s not based on wrong inference, and does not lead you to dismiss important factors, and the situation you are in is not complex, and it doesn’t weaken your resilience, then go ahead, use it.

 

The Pathologies of Silo-fighting

The division of labour has been the main principle for structuring organisations in the last two centuries. That is still the dominant approach for allocating resources, information and power in companies and public institutions. The new dynamics in a connected world have revealed a rich spectrum of problems related with these structures ranging from ineffective coordination to turf wars. Hence came the reaction, bringing the conventional stigmatising label ‘silos’ and the birth of a whole industry of silo-fighters armed with silo-bridging or silo-breaking services, methods and technologies.

There are various organisational pathologies brought by these silo-fighters. Here I will point out only the two of them which I see most often. The choice to classify the silo-fighters into silo-bridging and silo-breaking – an obvious oversimplification – is made to support the illustration of the these two pathologies.

‘Bridging the silos’ is not a strategy based much on appreciating their role and thus opting for bridging over breaking.  It’s mainly due to silo-fighters’ insufficient resources and power. If they manage to successfully sell the story of the bad silos, coming with a rich repertoire of metaphors such as walls, chasms, stove-pipes, islands and such like, then they get permission to build – as such a narrative would logically suggest – bridges between the silos.

Now, the problem with bridges is that they are either brittle and quickly break, or they are strong enough to defend their reason to be. They break easily when they fail to channel resources for longer time than the patience over their initial failures would allow. However, the identity formation switches on viability mode, and they grow out of network of decisions supporting their mission, now turned into an on-going function. If the reason the bridges exist are silos, and the bridges want to keep on bridging, then the silos have to be kept healthy and strong as they are what the bridges hang on to.

The bridges reinforce and perpetuate themselves up to a point, in which they are recognised as silos, and the problem is solved very often by building new bridges between them. This is how a cancerous fractal of bridges starts to grow. As attractive as this hyperbole is, I have witnessed repeatedly only two levels of recursion, but isn’t that bad enough?

In contrast, the silo-breaking strategies want nothing less than the destruction of silos. There, the silos are seen only in their role of a problem. Nobody asks what kind of problems this problem was а solution to. Instead, this type of silo-fighters start waging exhausting wars. The wars can end in several ways. A common one is resource depletion. Another is with the silos withstanding, or with the silo-fighter being chased away or transformed. And then of course it could be the case of victory for the silo-fighters. And this is when the disaster strikes. Having the silos down, the silos fighters are faced with all the problems being continuously solved by the silos during their life-span. Usually, they have no preparation to deal with those problems, neither they have the time to come up with and build alternative structures.

When discussing these two pathologies, it is very attractive to search for their root cause and then, when found, fix it. But that would be exactly the fuel these two types of silo-fighters run on. It takes a deeper understanding of the circularity of and in organisations, to avoid this trap. By ‘understanding’, I mean the continuous process, not the stage, after which the new state of ‘understood’ is achieved. And it takes, among other things, the ability to be much more in, and conscious of it, and at the same time much more out, but only as a better point for observation, not as an attempt for excluding the observer.

 

The Role of Meaning and the Meaning of Roles

Let’s start with roles. ‘Role’ comes from ‘roll’, as it was on a paper roll where the actor part was written. It is about something prescribed and then performed. But it evolved from roles that were performed as prescribed, through those that were not, to performing roles that had not been prescribed at all.

Roles are about relations. In fact, in Description Logic roles play the same role as relation, association, property and predicate in other formal languages. If John has a son George and is 30 years old, then George is in the role of a son for John, and even 30, although not an actor, plays the role of an age for John. Roles are inherently relational. A relation to itself or to something other. There is never just a role, always a ‘role in’ or a ‘role for’.

Roles are not just relational but are often determined by the dynamics of interactions. Here’s a handy example. The role of this text in the situation of you being in the role of a reader, will be, as assigned by me in the role of a writer, to transfer my thoughts on the meaning of roles, but it would rather trigger the construction of both similar and complimenting thoughts of you, and by doing so will play a different role, which, while evoked by me, is determined by you.

As there is now something about the meaning of roles, it’s time to introduce the role of meaning. If the role is always a role-in, my interest here is in the role of meaning in living and social systems. These systems have some things in common. One such thing is that they have internally maintained autonomy. Such autonomous systems bring forth and co-evolve with their niches. And that happens by creating of meanings which motivate attitudes and actions. But how does meaning come about?

Before meaning, there is the primary cognitive act of making a distinction,  bringing up something out of its background, distinguishing a thing of that which it is not. That act determines and is determined by the dynamics of the interaction between the system and its niche (by ‘system’ here I will only refer to systems with internally maintained autonomy). This dynamics is circular and recursive. Roughly speaking, it goes like this:

The act of distinction, the very making of difference changes the making of difference and brings forth “the difference that makes a difference”, in Bateson words, or the “surplus of significance” in Varela words, which is also co-dependant: the sense made changes the sense-making that changes the sense that is made and then brings forth the behaviour changing relation between the sense maker and the meaning of the distinguished element. This results in attitude of attraction, aversion or neutrality (or something more sophisticated like staying put, paralysed by the equal amount of temptation and fear). Or, in other words, the sense-making transcends into value-making. The value-making evolves itself and in downward causality influences the evolution of the distinguishing and the sense-making capability, which is in fact again a distinguishing capability but it is now distinguished as sort of a second-order one.

Before going further, I need to make clarifications on the use of ‘sense-making’ and ‘value-making’. ‘Sense-making’ is giving meaning to what is experienced. Here I use it with an emphasis of the action of making, of the creation, or more precisely, co-creation of sense. It is not that the sense is out there and all we need to do is to disclose it. No, we (or whatever the system in focus is) are the ones that actually make it, the origin is within us, or rather within the dynamics between us and what we interact with.

‘Value-making’ should not be confused with value adding. The system makes a distinction of a higher order in terms of directing its behaviour based on this distinction, hence the choice of ‘value’. It is not specified by the distinguished element, it is determined by the internal structure of the system and the dynamics between the system and the environment it’s structurally coupled with. This and the fact that value-making is a sense-making of a higher-order, is where the preference for ‘making’ comes from.

Only a small part of the environment, a niche, is constantly changing its content, is being interacted with, and actually matters. The niche is not one niche but a network of dynamically changing and influencing each other niches. A family is one niche, but that’s not the family described by somebody knowing all the facts, and not the family which will be invariant for each of the family members. The family as a niche is the subjective construction of interactions, memories, emotions, attention and imagination unique for every member of the family, as long as they consider themselves as such. This could easily exclude actual members and include those that have a lasting experience as such. And the same applies to work circle and friends circle, as to all occasional and recurring encounters, and virtual communities. But then, apart from interactions with other humans, we also have a niche out of the air we breathe, the grass we walk on, the stairs we climb and descend. And all that, changed by us, is changing us and changing the way we change it. But those changes, as long as a system is viable, serve to protect the identity from changing. We take from our niche things to make them into more of ourselves and become better in doing so.

Unlike the air, the grass and the stairs, a family or an organisation are autonomous systems which maintain their identity. They have their own niches. Moreover they are social systems. There are different ways to look at them. One way would be as autopoietic systems of communications, having humans as their niche (Luhmann), and another would be to have humans as both sub-systems and niche depending on weather the processes they participate in are part of the closed network of processes creating the identity of the system in focus. And that can be talked about in terms of the roles humans play.

Now we are ready to see what the role of meaning has to do with the meaning of roles. A living bacterium is at a stage of development where the sense-making has transcended into value-making. It does not only distinguish an environment with low from that with high sucrose concentration but prefers and moves towards the latter. As Evan Thompson put it, while sucrose is a “present condition of the environment, the status of the sucrose as a nutrient is not”, “it is a relational feature, linked to the bacterium metabolism”. And this is how the creation of meaning and roles are co-dependant. The dynamics between the bacterium and its environment, by making the sucrose relevant for the viability of the bacterium, realises the role of sucrose as nutrient.

But that’s not only relevant for cells and living organisms. It’s applicable for social systems as well. A company acts so that to turn some part of the environment into employees, other into clients, partners and so on. And for the same reason as the bacterium – to maintain it’s viability.

Once a formal role of an employee is assigned by a contract, a less formal roles are enabled by different mechanisms. It is now common to refer to such mechanisms as ‘Governance’ or an essential part of it. That part is a meta-role assigned to some people to determine the role of others. One and the same person often plays several roles.

Roles can be determined by formal assignment, by methodology, or by a Governance body but they can be also invented and self-assigned by the actors themselves, as typical for the organisations researched by Frederic Laloux. In that case, the evolutionary nature and the granularity of the roles are both dealing with the pathologies of assigned roles (being status currency, perpetuated even when obsolete, determined by politics and so on) and making organisations more responsive to change. Again the system, by what it does, takes from its environment what it needs to make more of itself, recursively: it turns non-employees into employees and vice versa,  and employees take up and leave roles, as determined by the dynamics of their interactions.

And so it seems that the meaning of roles is continuously realised by the role of meaning as a way in which a system generates its niche by asserting itself and maintaining its viability.

Language and meta-language for EA

That was the topic of a talk I gave in October last year at an Enterprise Architecture event in London. These are the slides, or most of them, anyway.

They probably don’t tell the story by themselves and I’m not going to help them here unless this post provokes a discussion. What I’ll do instead is a clarification about the title. “Language” refers to the means of describing organisations. They could be different. Given the current state of maturity, I have found those based on description logic to be very useful. What I meant by the “current state of maturity” is that a method in its theoretical development, application, the technologies supporting it and the experience with their application justifies investments in utilising them and helping in their further development. Although I find such a language clearly superior to the alternatives in use, that doesn’t mean there are no issues and that there are no approaches showing convincing solution to those issues. However, the practice with the latter or with the available tools doesn’t give me enough reason to stand behind them. The situation with the “meta-language” is similar but let’s first clarify why I call it that.

Metalanguage is commonly defined as language about language. And if that was the meaning I intended, it would have that type of relation to the language and then probably what I’m writing here could have been referred to as a mixture of another meta- and a meta-meta-language. It wasn’t the meaning I intended. I have found that there is a need to describe properly the “objects” that people in organisations are concerned about and how they relate to each other. It could be some way to represent physical things such buildings, documents and servers or abstract concepts such as services, processes and capabilities. And although it relates also to abstract things, I sometimes call it “language for the substance”.

Organisations are autonomous and adaptive systems, continuously maintained by their interaction with their niche, the latter being brought forth from the background, by that very interaction. While a language such as the one proposed can be useful to understand the components of an organisation, it doesn’t help much in understanding the dynamics and viability. The language for the substance cannot be used to talk about the form. That’s why there is a need, maybe temporarily until we find a better solution and probably a single language, to have another language and that other language I called meta-language in the presentation.

As this is a language for the form, I keep looking for ways to utilise some proposals, one of the most fascinating being George Spencer-Brown’s Laws of Form. Papers like this one of Dirk Baecker give me hope that it is possible. Until then, for the purposes of Enterprise Architecture, I find the Viable System Model, with the whole body of knowledge and practice associated with it, as the most pragmatic meta-language.

Essential Balances in Projects

These are part of the frames from the projects-flavour of the “Essential Balances” theme, delivered in a workshop format at a training event yesterday in Athens.

Note: new versions of the workshop slidedeck will appear in the frame above as soon as I update the file in SlideShare. That should explain why instead of Athens and July, the first slide refers to a different event.

Redrawing the Viable System Model diagram

I’ve been arguing repeatedly that trying to get the Viable System Model from overviews, introductions and writings based on or about it, can put the curious mind in a state of confusion or simply lead to wrong interpretations. The absolute minimum is reading at least once each of the three books explaining the model. But better twice. Why? There are at least two good reasons. The obvious one is to better understand some points and pay attention to others that have been probably missed during the first run. But there is also another reason. Books are of linear nature and when tackling non-linear subjects, a second reading gives the chance to better interpret each part of the text when having in the memory more less all other parts which relate to it.

Still, one of the things that are expected to be most helpful is in fact what brings about either confusion, aversion or misuse: the VSM diagrams. They clearly favour expected ease of understanding over rigour and yet in some important points they fail in both. Here is my short list of issues, followed by description of each:

  • Representation of the channels
  • Confusion about operations and their direct management
  • Notation and labelling of systems
  • They show something in between generic and example model
  • Hierarchical implication

Representation of the channels

Stafford Beer admitted several times in his books the “diagrammatic limitations” of the VSM representations. Some of the choices had to do with the limitation of the 2D representation and others, I guess, aimed to avoid clutter. Figure 26 of “The Heart of Enterprise” is a good example for both. It shows eleven loops but implies twenty one: 3 between environment, operations and management, multiplied by 3 for the choice of showing three operations, then another 9 = 3×3 for loops between same-type elements and finally 3 more between operation management and the meta-system.

Confusion about operations and their direct management

Depending on the context, System One refers either to operations or to their direct management. In some diagrams S1 is the label of the circles, and in others – of the squares linked to them. Referring to one or the other in the text, depending on which channels are described, only adds to the confusion. That is related to the general problem of

Notation and labelling of systems

All diagrams representing the VSM in the originals writings and all interpretations I’ve seen so far, suggest that circles represent System One, and triangles pointing up and down, represent System Two and Three* respectively. Additionally most VSM overviews state exactly that in the textual description. My assertion is almost the opposite:

What is labelled as S1 and what is shown as circles are both not representing S1.

That might come as a shock to many and yet, now citing Beer, System One is not the “circles” but:

The collection of operating elements (that is, including their horizontal and vertical connexions)

The Heart of Enterprise, page 132

Well, strictly speaking a system is a system because it shows emergent properties so it is more than the collection of its parts but even referring to it as collection reveals the serious misinterpretation when taking only one of its parts to represent the whole system.

They show something in between generic and example model

Communicating such matters to managers trained in business schools wasn’t an easy task. And it is even more challenging nowadays. There is a lot to learn and a even more to unlearn. It is not surprising then that even in the generic models typically three operations are illustrated (same for System 2).  Yet, I was always missing a true generic representation, or what would many prefer to call “meta-model”.

Hierarchical implication

It can’t be repeated enough that the VSM is not an hierarchical model, and yet it is often perceived and used as such or not used especially because of that perception. It seems that recursivity is a challenging concept, while anything slightly resembling hierarchy is quickly taken to represent one. And sadly, the VSM diagram only amplifies that perception, although the orthogonality of the channels serves an entirely different purpose. Stafford Beer rarely misses an opportunity to remind about that. Nevertheless, whatever is positioned higher implies seniority and the examples of mapping to actual roles and functions only help in confirming this misinterpretation.

 

There are other issues as well but my point was to outline the motivation for trying alternative approaches for modelling the VSM, without alternating the essence or the governing principles. Here is one humble attempt to propose a different representation (there is a less humble one which I’m working on, but it’s still too early to talk about it). The following diagram favours circular, instead of orthogonal representation which I hope achieves at least in destroying the hierarchical perception. Yet, from a network point of view the higher positioning of S3 is chosen on purpose as the network clearly shows that this node is a hub.

GenericCircularViewOfTheViableSystemModel

System One is represented by red colouring, keeping the conventional notation for the operations (S1.o) and their direct management (S1.m). As mentioned above, apart from solving this, the intention  as to have it as a generic model. If that poses a problem for those used to the hybrid representation, here’s how it would look if two S1s are shown:

 

Circular Netwrok View Of The Viable System Model-2operations

I hope this proposal solves fully or partially the five issues explained earlier and brings a new perspective that can be insightful on its own. In any case the aim is to be useful in some way. If not as it is now, then by triggering feedback that might bring it to a better state. Or, it can be useful by just provoking other, more successful attempts.

 

More on Requisite Inefficiency

The “slides” supporting my talk on Requisite Inefficiency a couple of months ago have been on Slideshare since then but I haven’t had the time to share them here. Which I do now.

The various manifestations of Requisite Inefficiency in both organisms and organisations can be understood by observing the maintenance of balances between homeostasis and heterostasis (as in the adaptive immune systems), exploration and exploitation (foraging of ants or curiosity-driven vs market-driver research) as well as various types of redundancy or shift of function. The latter can be elastic, as it is in degeneracy, or plastic, as it is in exaptation.
Having an underutilised structure/function that is capable of providing the deficit of variety to the utilised structures of  a system in order to match the complexity of an external stimulus, or that can be adapted in sufficiently short time to do so, is a prerequisite for survival.

Variety – part 2

Can you deal with it? It is amazing how language evolved to adapt to reductionist mindset.  “Deal”, which originated from divide and initially meant only to distribute and then to trade, is now used as synonym for cope, manage and control. We manage things by dividing them. We eat elephants peace by peace, we start journeys of thousand miles with a single step, and we divide to conquer.

(This is the second part of a sequence devoted to the concept “variety” used as a measure of complexity. It’s a good idea to read previous part before this one but even doing it after, or not at all is fine.)

And that indeed proved to be a good way of managing things, or at least some things, and in some situations. But often it’s not enough. To deal with things, and here I use “deal” to mean manage, understand, control, we need requisite variety and when that is not the case initially, we could get it in three ways: by attenuating the variety of what has to be dealt with, by amplifying our variety, or by doing a bit of both when the difference is too big.

And how do we do that? (I use we/them instead of regulator/regulated or systemA/systemB or organisation/environment, because I find it easier to imply the perspective and the purpose this way) Let’s start with an attempt to put some very common activities in each of these groups. We attenuate external variety by grouping, categorising, splitting, standardising, setting objectives, filtering, reporting, coordinating, and consolidating. We amplify our variety by learning, trial-and-error, practising, networking, advertising, buffering, doing contingency planning, and innovating. And we can add a lot more to both lists, of course. We use activities from both list but when doing these activities we need requisite variety as well. That’s why we have to apply them at the lower level of recursion. We learn to split and we split to learn, for example.

Attenuate and amplify variety

It could be easy to put in the third group pairs from each list but now the task is to classify single types. Here are two suggestions that seem to fit: planning and pretending.

With planning we get higher variety by being prepared for at least one scenario, especially in the parts of what we can control, in contrast to those not prepared even for that. But then, we reduce different possibilities to one and try to absorb part of the deflected variety with risk management activities. Planning is important in operations and projects but somehow in the business setting we can get away with poor planning, at least for long enough to lose the opportunity to adapt. And that is the case in many systems with delayed feedback. That’s why I like the test of quick-feedback and skin-in-the-game situations, like sailing. In sailing, not having a good plan could be equality disastrous as sticking to a plan for too long and not adapting or replacing it quickly based on evidence against the assumptions of the initial one. And that’s valid at every level, no matter if the plan is for a week, day or an hour.

Pretending is even more interesting in its dual role. It can be so successful as to reinforce its application to the extreme. Pretending is so important for stick insects, for example, that they apply it 24/7. That proved to be really successful for their survival and they’ve been getting better at it for the last fifty million years. It turned out to be also so satisfactory that they can live without sex for one million years. Well, that’s for a different reason but nevertheless their adaptability is impressive. The evolutionary pressure to better resemble sticks made them sacrifice their organ symmetry so that they can afford thinner bodies. Isn’t it amazing: you give up one of your kidneys just to be able to lie better? Now, why do I argue that deception in general, and pretending in particular, has a duel role in the variety game? Stick insects amplify their morphologic variety and through this they attenuate the perception variety of their predators. A predator sees stick as a stick and stick insect as a stick, two states attenuated into one.

Obviously snakes are more agile than stick insects but for some types that agility goes beyond the capabilities of their bodies. Those snakes don’t pretend 24/7 but just when attacked. They pretend to be dead. And one of those types, the hognose snake, goes so far in their act as to stick its tongue out, vomit blood and sometimes even defecate. Now, that should be not just convincing but quite off-putting even for the hungriest of predators.

If pretending can be such a variety amplifier (and attenuator), pretending to pretend can achieve even more remarkable results. A way to imagine the variety proliferation of such a structure is to use an analogy with the example of three connected black boxes that Stafford Beer gave in “The Heart of Enterprise”. If the first box has three inputs and one output, each of them with two possible states, then the input variety is 8 and the output is 256. Going from 8 to 256 with only one output is impressive but when that is the input of a third black box, having only one output as well, then its variety reaches the cosmic number of 1.157×1077.

That seems to be one of the formulas of the writer Kazuo Ishiguro. As Margaret Atwood put it, “an Ishiguro novel is never about what it pretends to pretend to be about”. No wonder “Never Let Me Go” is so good. And the author, having much more variety than the stick insects, didn’t have to give his organs to be successful. He just made up characters to give theirs.

Variety – part 1

The cybernetic concept of variety is enjoying some increase in usage. And that’s both in frequency and in number of different contexts. Even typing “Ross Ashby” in Google Trends supports that impression.RossyAshby_as_seen_by_GoogleTrends In the last two years the interest seems stable, while in the previous six – non-existing, save for the lonely peak in May 2010. That’s not a source of data to draw some conclusions from, but it supports the impression coming from tweets, blogs, articles, and books. On one side, that’s good news. I still find the usage insignificant, compared to what I believe it should be, given its potential and the tremendously increased supply of problems of such nature that if not in solving, at least it can help in understanding. Nevertheless, some stable attention is better that none at all. On the other side, it attracts a variety of interpretations and some of them might not be healthy for the application of the concept. That’s why I hope it’s worth exchanging more ideas about variety, and those having more variety themselves would either enjoy wider adoption or those using them – more benefits, or both.

The concept of “variety” as a measure of complexity had been preceded and inspired by the “information entropy” of Claude Shannon, also known as the “amount of surprise” in a message. That, although stimulated by the development of the communication technologies in the first half of the twentieth century, had its roots in statistical mechanics and Boltzmann’s definition of entropy. Boltzmann, unlike classical mechanics and thermodynamics, defined entropy as the number of possible microstates corresponding to the micro-state of a system. (These four words “possible”, “microstates”, “micro-states” and “system” deserve a lot of attention. Anyway, they’ll not get in this post.)

Variety is usually defined as the number of possible states in a system. It is also applied to a set of elements, as the number of different members determine the variety of a set, and to the members themselves which can be in different states and then the set of possible transitions has certain variety. This is the first important property of “variety”. It’s recursive. I’ll come back to this later. Now, to clarify what is meant by “state”:

By a state of a system is meant any well-defined condition or property that can be recognised if it occurs again.

Ross Ashby

Variety can sometimes be easy to count. For example, after opening the game in chess with pawn on D4, the queen has variety of three – not to move or move to one of the two possible squares. If only the temporary variety gain is counted, then choosing D2 as the next move would give variety of 9, and D3 would give 16. That’s not enough to tell if the move is good or bad, especially having in mind that some of that gained variety is not effective. However, in case of uncertainty, in games and elsewhere, moving to a place which both increases our future options and decreases those of the opponent, seems a good advice.

Variety can be expressed as a number, as it was done in the chess example, but in many cases it’s more convenient to use the logarithm of that number. The common practice, maybe because of the first areas of application, is to use binary logarithms. When that is the case, variety can be expressed in bits. It is indeed more convenient to say the variety of a four-letter code using the English alphabet is 18.8 bits instead of 456 976. And then, when the logarithmic expression is used, combining varieties of elements is done just by adding instead of multiplying. This has additional benefits when plotting etc.

Variety is sometimes referred to and counted as permutations. That might be fine in some cases but as a rule it is not. To use the example with the 4-letter code, it has 358 800 permutations (26 factorial divided by 22 factorial), while the variety is 456 976 (26 to the power of 4).

Variety is relative. It is dependant on the observer. That’s obvious even from the word “recognised” in the definition of “state”. If, for example, there is a clock with two hands that are exactly the same or at least to the extent that an observer can’t make the difference, then, from the point of view of the observer, the clock will have much lower variety than a regular one. The observer will not be able to distinguish for example 12:30 and 6:03 as they will be seen as the same state of the clock.

Clock with indistiguishable hands

This can be seen as another dependency. That of the capacity of the channel or the variety of the transducer. For example, it is estimated that regular humans can distinguish up to 10 million colours, while tetrachromats –  at least ten times more. The variety of the transducer and the capacity of the channel should always be taken into account.

When working with variety, it is important to study the relevant constraints. If we throw a stone from the surface of Earth, certain constraints, including those we call “gravity” and the “resistance of the air”, would allow a much smaller range of possible states than if those constraints were not present. Ross Ashby made the following inference out of this: “every law of nature is a constraint”, “science looks for laws; it is therefore much concerned with looking for constraints”. (Which is interesting in view of the recent claim of Stuart Kauffman that “the very concept of a natural law is inadequate for much of the reality” and that we live in a lawless universe and should fundamentally rethink the role of science…)

There is this popular way of defining a system as something which is more than the sum of its parts. Let’s see this statement through the lenses of varieties and constraints. If we have two elements, A and B, and each can be in two possible states on their own but when linked to each other A can bring B to another, third state, and B can bring A to another state as well. In this case, the system AB has certainly more variety than the sum of A and B unbound. But if, when linking A and B they inhibit each other, allowing one state instead of two, then it is clearly the opposite. That motivates rephrasing the popular statement to “a system might have different variety than the combined variety of its parts”.

If that example with A and B is too abstract, imagine a canoe sprint kayak with two paddlers working in sync and then compare it with the similar setting but one of the paddlers rowing while the other holds her paddle in the water.

And now about the law of requisite variety. It’s stated as “only variety can destroy variety” by Ashby and as “variety absorbs variety” by Beer, and has other formulations such as “The larger the variety of actions available to control system, the larger the variety of perturbations it is able to compensate”. Basically, when the variety of the regulator is lower than the variety of the disturbance, that gives high variety of the outcome. A regulator can only achieve desired outcome variety, if its own variety is the same or higher than that of the disturbance. The recursive nature, mentioned earlier, can now be easily seen, if we look at the regulator as channel between the disturbance and the outcome or if we account for the variety of the channels at the level of recursion with which we started.

To really understand the profound significance of this law, it should be seen how it exerts itself in various situations, which we wouldn’t normally describe with words such as “regulator”, “perturbations” and “variety”.

In the chess example, the power of each piece is a function of its variety which is the one given by the rules reduced by the constraints at every move. Was there a need to know about requisite variety to design this game? Or any other game for that matter? Or was that necessary to know how to wage war. Certainly not. And yet, it’s all there:

It is the rule in war, if our forces are ten to the enemy’s one, to surround him; if five to one, to attack him; if twice as numerous, to divide our army into two.

Sun Tzu, The Art of War

But let’s leave the games for a moment and remember the relative nature of variety. The light signals in ships should comply with the International Regulations for Preventing Collisions at Sea (IRPCS). Yes, here we even have the word “regulation”. Having the purpose in mind, to prevent collision, the signals have reduced variety to communicate the states of the ships but enough to ensure the required control. For example, if an observer sees one green light, she knows that another ship is passing from left to right, and  – from right to left – if she sees red light. There are lots of states – different angels of the course of the other ship –  that are reduced into these two but that is serving the purpose well enough. Now, if she sees both red and green that means that the ship is coming exactly towards her that’s an especially dangerous situation. That’s why the reduction of variety in this case has to be very low.

The relativity of variety is not only related to the observer’s “powers of discrimination”, or those of the purpose of regulation. It could be dependant also on the situation, on the context. An example that first comes to mind is the Easop’s fable “The Fox and the Stork”.

Fables, and stories in general, are interesting phenomenon. Their capability to influence people and survive many centuries is amazing. But why is that? Why do you need a story, instead of getting directly the moral of the story. Yes, it’s more interesting, there is this uncertainty element and all that. But there is something more. Stories are ambiguous, interpretable. They leave many things to be completed by the readers and listeners. And yes, to put in different words, they have much higher variety than morals and values.

That’s it for this part. Stay tuned.

The Change of the Change

Let’s have a variable V representing at this moment an aspect of interest I from the behaviour of a system S. This variable W is changed through transduction of a certain characteristic C of S using a transducer T.

Now, which variable are we talking about, V or W?  Probably Z? It would be V only if I, S, C and T and the meaning of ‘variable’ remained the same at the moment of writing the second sentence and even that would require ‘only if’ to work as assumed by logic.

Yes, it can be that fast.