Quickly changing software is becoming an increasingly important part of any business. Cloud Native is highly regarded as the central catalyst of this new software generation, and traditional software architectures are increasingly being turned upside down. The empirical Cloud Native Study 2022 “Much more than just technology”, which Cloudflight conducted together with IONOS by 1&1, shows that 43% of companies in DACH are planning to expand Cloud Native.
However, it is important that microservices, serverless, and the like are not used simply because they are in trend but rather because a company can successfully differentiate and individualize itself through them. Especially in companies with an entrenched organization, it is not always easy to introduce new technologies. Especially in architectural innovations, this can even be a structural disadvantage. But especially in today’s world, it is important that an architectural approach remain adaptable and able to respond to change. Systems are no longer built for eternity (built to last) but rather for changeability (built to replace). Or, as the American computer scientist Fred Brooks[1] put it: Because we never built the best solution in the beginning, we have to be able to change the concept and be flexible. And that applies to systems and companies alike.
The following article is intended to provide decision-makers with an insight into how “Conway’s Law projects” can influence companies. It also shows how the law can be used as a tool when introducing Cloud Native and the like. If you need experts in development and operation, please do not hesitate to contact us. Cloudflight can support you in the complete transition of your development processes to Cloud Native and ensures the smooth operation of the solutions we develop.
Page Break
Conway’s law
JavaScript expert Douglas Crockford[2] has no sympathy for the fact that we spend all our time struggling with the effects of mistakes made many years ago.
He knows that “The structure of software systems tend to reflect the structure of the organization that produce them.” But the structures in a company are not always entrenched and rigid but rather change (e.g., because of new products or in the course of new corporate objectives). The IT infrastructure must also be adapted accordingly. Otherwise, tensions due to incompatibilities between the company and the infrastructure will arise.
Of course, it’s not quite that simple. According to the computer scientist Melvin E. Conway[3], the structures of systems are predetermined by the communication structures of the organizations implementing them. He submitted this observation in 1968 with his paper How Do Committees Invent?
“Organizations which design systems […] are constrained to produce designs which are copies of the communication structures of these organizations.”
His observation that organizations design systems that copy their communication structures is now more than simply a hypothesis. Although his paper was rejected by the Harvard Business Review because of a lack of evidence, there are now numerous studies that point to the correctness of the theory.
From design to system
The concept of system used by Conway does not only refer to the IT systems used by a company but also considers the products and services offered by the company.
According to Conway, any company that develops a system will inevitably produce a design that mirrors the communication structure of the organization. However, the actual design cannot emerge until certain preliminary milestones are reached. This includes:
- An understanding of the limits set by the client as well as the real conditions for both the design activity and the system to be designed
- Achieving a preliminary idea of the system organization so that design tasks can be distributed in a meaningful way.
Decisions must continually be made in activities relating to system design. Many of these decisions are not simply design decisions but rather personal decisions that the team working on the software are making (or have already made) about their own future. Thus, one implication of Conway is that people always put their own individual stamp on their contributions to a software project and that they are inherently incapable of working together in a unified way to write source code.
Conway also describes that once an organization has selected a design team, it is possible to delegate activities to a subgroup. But even in doing so, the potential outcome of the design activities is diminished. Once the areas of activity are defined, a coordination problem can once again arise.
In practice, the subgroups mentioned include UI, back-end, database, and business processes, which must communicate with each other in order to be able to work together successfully. However, if the communication is one-sided or lacking, this causes ripples, and other subgroups suffer. Effective communication leads to better deliverables (e.g., delivery times, quality) that require less rework with each sprint.
During requirements analysis with customers, there are different business processes, specializations, and needs that are relevant to the software and which are passed on to a developer team. Here, too, communication is crucial because each department – whether sales or production – will convey its own view. Limitations can thus arise in the design of IT systems.
Christoph Holter, Project Lead at Cloudflight confirms the challenge: “I have already experienced it myself in projects – how it is when one hand does not know what the other is doing. The Engineering and Product teams did not talk to each other. The Engineering team accordingly made decisions that seemed completely useless to the Product team – and vice versa. I made a genuine effort to act as a mediator and almost slipped into the role of a product owner. Alas, in vain.”
It’s all about the team
The architecture of an IT system depends on place, time, and knowledge as well as technology and is thus not absolute. A system developed by a group, the sub-system developed by a team, and the interface developed by a team leader – or the whole thing can be split into an enterprise system, micro-services, and classes – creates a bounded framework at the organizational level. In other words, organizations have limited flexibility. Different team constellations arrive at different solutions. Depending on the level of knowledge and the form of organization, the result is influenced differently.
Particularly complex IT systems often seem to cause great architectural difficulties and are often at risk of disintegrating. There are numerous prominent examples of this, especially in connection with the introduction of SAP. In addition to Lidl, Haribo, and Otto, Deutsche Bank invested millions in an SAP project before it ultimately failed because the company’s processes were not adapted.
In addition to the lack of adaptation, time pressure, or budget difficulties, large IT projects can fail because they are overly complex. Once the limits of understanding the complexity of a system are reached, either systems are broken down until the complexity is understood, or control is lost.
Problems of this kind often result in statements like: “It’s really a big project. I’ve tried it with a team of 50 developers, and it simply doesn’t work.”
From a purely accounting perspective, four developers working 50 days is the same as eight developers working 25 days. However, the two teams will deliver different systems. The system of the four developers is expected to be better than that of the eight developers. According to Conway, that’s because the more people involved in a system, the more communication relationships exist and the more complex things become. Similarly, Fred Brooks describes it in his book The Mythical Man-Month: “Adding manpower to a late software project makes it later.”
From this, Parkinson[4] derived his law: work can be stretched like rubber to fill the time available for it. As the number of workers increases, the work becomes so bloated that the time of all workers available is used. As a result, relationships also become increasingly complex until communication breaks down. The same happens with the architecture.
Inverse Conway Maneuver
The Inverse Conway Maneuver takes a different perspective on organizational and systems design challenges. While Conway states that the structure of the company and the interpersonal relationships of the people in the company directly influence the design of the technical systems produced, the Inverse Conway Maneuver describes the principle of the other path direction. If a particular system is to be the result, this must first be mapped at the organizational level. Accordingly, it drives organizational change that targets communication structures and takes into consideration the interaction of the team.
The right set-up – a conclusion
“You should never be able to reverse engineer a company’s organizational chart from the design of its product. ”Eric Schmidt[5], How Google Works.
Software systems evolve from mere keystrokes into intricate functional networks that connect many cooperating modules, objects, classes, methods, and subroutines. At the same time, they reflect the communication structures of an organization.
Conway’s Law is a powerful concept that has yet to be internalized by most. It describes how deeply the existing communication structure of a company is linked to its activities or end product. In order to benefit from a flexible IT infrastructure and gradually change systems, companies and their communication structures must be set up to be just as lean and adaptable. At the start of the project, the stakeholders and interfaces involved should be clearly identified in order to come to decisions that ensure that the right system emerges.
If an ongoing project is at risk of failing, outside support often helps. In addition to experienced project or change managers, Agile Coaches can act as troubleshooters. To loosen rigid structural boundaries, Scrum Masters in particular have an important sphere of influence. But even on a small scale, a group of employees can be introduced to agility with creative methods (for example, as part of a group outing) that successively lead to exchange and communication throughout the organization.
In order to successfully accompany the change to a modern software architecture, it is primarily decision-makers who are unconditionally behind this project that can persuade a company to adapt its old processes and work flows accordingly. Certainly, the question of whether such complex new systems are even necessary is also legitimate. Those who initially thought that standard solutions were the general answer to Conway were wrong. Most off-the-shelf solutions have a lot of configuration options and allow one to develop their own extensions without changing the core of the software. Standards are part of the preferred strategy when internal processes or technological capabilities are not sufficiently understood and are not used to differentiate from the competition.
Those who focus on the use of microservices and the like (because this scales development and allows more people to work on the projects) should be careful not to artificially inflate the system and create architectural problems.
The following recommendations for action can be derived from this:
- Conscious lock-in: Standards for infrastructure, security, and monitoring that do not differentiate should be used. Conscious decisions for a lock-in should be made, the cost of change should be known, and necessary measures should be prepared (e.g., infrastructure change, new tools, and scaling)
- Decoupling and microservices: Splitting software and platform architectures into small microservices that can be developed, maintained, and integrated independently (however, there are exceptions where monoliths can still make sense)
- APIs and integration: A high degree of interconnection and good documentation with centralized and standardized API management solutions
- Trial and error: It’s better to start quickly than to work on the perfect plan until the end of the day
- If you can’t beat them, join them: If there is a better standard for a service or software, this should be used
| [1] Fred Brooks: Frederick Phillips Brooks, Jr. (born April 19,1931 in Durham, North Carolina, USA) is an American computer scientist. Brooks first became known as the person responsible for the development of the OS/360 at IBM and later for the honest description of the development process in his book The Mythical Man-Month (German Vom Mythos des Mann-Monats: Essays über Software-Engineering). Source: https://en.wikipedia.org/wiki/Fred_Brooks |
| [2] Douglas Crockford: Douglas Crockford is an American programmer involved in the development of the JavaScript language. He has specified the JSON data format and developed several JavaScript-related tools such as the static code analyzer JSLint and the minifier JSMin. His book “Javascript the Good Parts” was published in 2008 and “How Javascript Works” in 2018. He was a senior JavaScript architect at PayPal until 2019 and is also an author and speaker on JavaScript, JSON, and related web technologies. Source: https://en.wikipedia.org/wiki/Douglas_Crockford |
| [3] Melvin E. Conway: Melvin Edward Conway is an American computer scientist, computer programmer, and hacker who coined what is now known as Conway’s Law: “Organizations that design systems […] are forced to create designs that map the communication structures of those organizations.” He is also known for developing the concept of coroutines. Conway coined the term “coroutine” in 1958 and was the first to apply the concept to an assembly program. He later authored a seminal paper on coroutines entitled “Design of a Separable Transition-diagram Compiler.”
Source: https://en.wikipedia.org/wiki/Melvin_Conway |
| [4] Cyril Northcote Parkinson: Cyril Northcote Parkinson (born 30 July 1909 at Barnard Castle in County Durham; † 9 March 1993 at Canterbury) was a British historian and publicist. He is the discoverer of Parkinson’s laws, which are named after him. Parkinson’s first law: “Work expands to the exact extent that time is available for its completion.”; Parkinson’s Proportional Rule: “In budget debates, the time spent discussing an expenditure item is inversely proportional to its amount.” He became known worldwide through more than 60 books. Source: https://en.wikipedia.org/wiki/C._Northcote_Parkinson |
| [5] Eric Schmidt: Eric Emerson Schmidt is an American computer scientist and manager and served as Executive Chairman from April 2011 to August 10, 2015 (prior to that as Chief Executive Officer) of Google. In the course of the restructuring of Google, Schmidt subsequently moved as Executive Chairman to Alphabet Inc. until he left the company in 2020. From 1983 to 1997, he worked, among other positions, as CTO at Sun Microsystems. From 1997 to 2001, he was CEO at Novell, and from 2006 to 2009, member of the Board of Directors at Apple. From 2009, Schmidt was a member of the team of advisors to then U.S. President Barack Obama on technology issues and teaches at Stanford University. Source: https://en.wikipedia.org/wiki/Eric_Schmidt |