Drawings help people to work out intricate relationships between parts.
Christopher Alexander
As I was reading an interesting book by Christopher Alexander – A Pattern Language: Towns, Buildings, Construction (1977), it hit me. Software engineering should not be that much different. One, it should have some people who create a vision of the work to be completed, and then work hard to create a language by which most of the other people in the enterprise would communicate. Two, software products should have some kind of a technical pattern language.
Not much is being done in this area in today’s software world besides creating a technical approach to doing things, at the same time appreciating there is the communication factor and the complexity factor of touching the unknown. I still fail to find an approach that would be marrying both.
Software architecture as a craft has become for many just another chart you write for others to implement, and not for others to use as a discussion tool. Many more software architects suffer from the syndrome on preparing solutions for things they have never physically touched. They rarely code.
At the same time some work was put into making the pattern language. Yet, software patterns are not easily accessible for an average developer. They opt to high on completeness, being closer to an academic solution to a problem (Just like the difference between theoretical physicists and engineers, some of us might have seen in The big Bang Theory). They are right in terms of the craft, but how useful is a tool, where you need to spend significant amount of time making sure your builders understand it? And the person who is supposed to do it and offer help, draws something like:
Most of the explanations around patterns deal with how things should be done and only slightly touch with why they should be done that way. Academic disconnect is present when reading:
Composite should be used when clients ignore the difference between compositions of objects and individual objects. If programmers find that they are using multiple objects in the same way, and often have nearly identical code to handle each of them, then composite is a good choice; it is less complex in this situation to treat primitives and composites as homogeneous.
https://en.wikipedia.org/wiki/Composite_pattern
Will this tell a developer whether they should use this pattern, while building a process which is calling other processes (like for example a simple email sender)? Will they know to use composition over dependency injection and which one would be better? Or would they know when to use them both?
My answer is – not necessarily. These patterns tell you how to lay the bricks, but they miss something bigger. They are missing the context. And I have not found yet a good solution on how to drive the context within software engineering AND take good care of the software landscape at the same time.
I would define the challenge, I am embarking to tackle as: todays software engineering suffers from high-level, top-down architectures, where low-level academical tools are used to apply a design, by people, who do not actually touch the code themselves, while human and communication element is almost practically absent.
When you pepper this with the pressure by your product management to squeeze out the value of your product to get to the market as soon as possible, you can easily end up in an approach, which can be summarized as per picture below.
No sane organization wants that! But most very sane organizations understand very little about how engineering works and still see it as kind of something between a factory with many workers (large corporations) and a chess club for geeks. They then attempt to solve it with software architects and fail miserably, as these architects just fall into the same hole I have described above.
Would be great to have a different approach …
What I suggest in return, is principal engineering. If you are a software engineer who prefers analytical or mathematical solutions, you might not like what I am about to write. These things might sound as random ramblings, at the same time I have seen this work and the other to fail continuously.
What is then, the key difference between principal engineers and software architects? First of all – they still code, they know most of their work will be talking to others, and they are masters of setting the context.
Principal engineers must be the right people for your organization, as their responsibility is for technical-to-technical and technical-to-business communication. In a sense they own the technical culture and how your company’s technical ‘neighbourhoods’ co-exist.
They are also masters of their skillset. How could they not be? One of their roles is to participate in very deep, technical discussions. They need to be able to actively listen, rely suggestions and be able to convince the audience on why an approach A might be better than B,C or D. Like good building architects, who understand materials, cost and skillset of builders so need the principal engineers understand software engineers.
A typical day for a principal engineer might involve helping to re-write some nasty piece of code, popping in for coffee with team A, participating in some business discussion, drawing a diagram and quickly documenting ideas, popping up for coffee for team B, etc … But overall they are responsible to create and maintain an organization-wide pattern language, that is understandable for both tech and for the business. These patterns are then used to build your solutions.
(although at this point of the history I should probably stop splitting business and software – software if THE business for most companies today – whether you like it or not, your engineering department is the centrepiece of value generation).
To give the principal engineers some tools to work on creating the pattern language of the organization, I will be taking the approach suggested by Christopher.
His first pattern is INDEPENDENT REGIONS, which I would call INDEPENDENT PRODUCTS or TEAMS. The key here is that your company is a meta-region and principal engineers should work out first, what are the regional policies, that will help to protect the business value and mark the boundaries of products.
There is one timeless way of building. It is a thousand years old, and the same today as it has ever been. The great traditional buildings of the past, the villages and tents and temples in which man feels at home, have always been made by people who were very close to the center of this way. It is not possible to make great buildings, or great towns, beautiful places, places where you feel yourself, places where you feel alive, except by following this way. And, as you will see, this way will lead anyone who looks for it to buildings which are themselves as ancient in their form, as the trees and hills, and as our faces are.
Christopher Alexander, The Timeless Way of Building (1979)
For example, they might be thinking whether you choose AWS or Google Cloud. They should set some rules of engagement between the teams and ensure the word independent is understood within the organization. As in teams manage what is given to them, but there should be expectations set on the result.
And no – I am not a fan of stricte tribal model. As we all know tribes had not survived the test of time. There needs to be somewhat more thought put into – what binds the tribes, rather than just saying – do what you want!
The next pattern, which is called THE DISTRIBUTION OF TOWNS in the original, is more about resilience in IT world. Thus I call it THE DISTRIBUTION OF DATA CENTRES. This basically deals with how to ensure you are covering the needs of your population (software product users), without overloading or starving any area for resources. You might end up with just one location, still it should be a decision that at least some thought it put into.
Then there is the COUNTRY TOWNS. In your company, you will definitely have product or products, which you can call the big town/towns. It’s great to have these powerhorse products. The pride of the crowds! At the same time, it’s the beauty of your small products that enables the growth in future. Have COUNTRY PRODUCTS. The things, that people work on in spare time, because they are just passionate about it. Do not fall behind the myth of total efficiency, by killing everything not earning money right away. Your software engineers, product managers and most likely your clients will love you for it. You will remain innovative and experimental, while having that quiet country project, where your employees still do not feel the pressure of a big town.
The business we’re in is more sociological than technological, more dependent on workers’ abilities to communicate with each other than their abilities to communicate with machines.
Tom DeMarco
This is it for today. I’ll be coming back to my vision of principal engineering in the future. There is yet a lot of patterns to go through and start discussing the ones that actually help to build an accurate value vessel diagram of your company.
Apologies if some of my thinking sounds like a bit of rambling, I am attempting to organize it as I write :).
Just summarizing today:
- Diagrams help as long as they are not made into laws. They are a discussion tool and precision of a diagram is less important, than whether they are supporting a common understanding of a problem space
- If you’d really like to stick to software architects and architecture, ensure your architects still work on your code and delivery or at least are actively engaged with the teams building the solutions
- Start finding your pattern language
- Remember it’s your people, who build the product. They operate within your company culture. In this sense your products and software quality is just an image of your company culture (and more and more consumers begins to understand this!).
- Principal engineers and your own pattern language can help you to overcome a lot of these challenges
We’ll be back. Stay safe!
Gustaw Fit 
