Software Architecture For Developers with Neal Ford

Hello, and welcome to podcast dot in it, the podcast about Python and the people who make it great. I would like to thank everyone who supports the show on Patreon. Your contributions help to make the show sustainable. When you're ready to launch your next project, you'll need somewhere to deploy it, so you should check out linode@podcast

init.com/linode

and get a $20 credit to try out their fast and reliable Linux virtual servers for running your app. And now you can deliver your work to your users even faster with the newly upgraded 200 gigabit network in all of their data centers. If you're tired of cobbling together your deployment pipeline, then it's time to try out GoCD, the open source continuous delivery platform built by the people at Thoughtworks who wrote the book about it. With GoCD, you get complete visibility into the life cycle of your software from 1 location. To download it now, go to podcastinit.com/gocd.

Professional support and enterprise plug ins are available for added

peace of mind. You can visit the site at podcastinnit.com

to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions, I would love to hear them. You can reach me on Twitter at podcastinit

or email me at host@podcastinit.com.

To help other people find the show, please leave a review on Itunes or Google Play Music. Tell your friends and coworkers and share it on social media. We'll get a few announcements before we start the show.

First, there's still time to register for the O'Reilly Software Architecture Conference happening in New York, February 25th to 28th.

Use the link podcast the niche.com

/sa

khan dash new dash york to register and save 20%.

If you work with data or want to learn more about how the projects you heard

about on the show get used in the real world, then join me at the Open Data Science Conference happening in Boston from May 1st through 4th.

It has become 1 of the largest events for data scientists, data engineers, and data driven businesses to get together and learn how to be more effective.

To save 60% off your tickets, go to podcastinit.com

/odscdasheastdash2018

and register.

I'd like to thank O'Reilly Media for offering 2 items to give away.

To enter, you just need to subscribe to the mailing list at podcastinit.com,

and you'll have the chance to win either a copy of Neil's book, Building Evolutionary Architectures,

or a bronze pass to the O'Reilly Software Architecture Conference happening in New York.

I'll be picking the winners on February 21st. Your host as usual is Tobias Macy, and today I'm interviewing Neil Ford about the principles of software architecture for developers. So, Neil, could you start by introducing yourself?

Absolutely. Thanks for having me. My name is Neil Ford. I work at a, international consulting company called Thoughtworks. I've been there for almost 13 years now, and most of my professional work is in the intersection

of software architecture and things like agile engineering practices.

And do you remember how you first came across Python? I first saw Python many, many, many, many years ago,

because it was

a sort of a variation on lisp, but I have a deep, deep love for,

Lisp and variations of Lisp. And Python is a a, in that in that family of languages, so I've always had a fondness for it. And we keep bumping into it professionally.

1 of the things that Thoughtworks does is twice a year we release this white paper.

Basically, I mean, 1 of

the the superpowers of being a consultant in a large consulting company like this is we get to pattern match things across a large swath of projects. And,

we do this,

twice a year with this thing called the Thoughtworks Technology Radar. And on our last radar, we actually called out the fact that Python keeps growing

in our clients' usage, and it keeps popping up in all sorts of interesting places. And, we're seeing more and more of it showing up in the enterprise y world.

And as you mentioned, a major focus of your work has been on software architectures

and how those can be used to facilitate the delivery of working software systems. So I'm wondering if you can start by just giving a high level overview of what software architecture is in your experience and how that fits into the overall development process.

Well, I would love to be able to give a nice concise high level overview of what software architecture is, but I don't have a clue of how to do that.

In fact,

better minds than me have

cast themselves against that particular mountain and failed.

Martin Fowler famously wrote a white paper called Who Needs an Architect where he talks about the lack of definition of software architecture and why that exists. And 1 of the sort of side roles I have is the co chair of the O'Reilly Software Architecture Conference. It happens

in York in February and in London in October.

And 1 of the things when I moved into that role that they asked me sort of as a subject matter expert at O'Reilly about all things software architectures, can you give us a really nice, succinct definition of software architecture? And I said, no, I can't. And they kept pushing me on this, and I tried to explain to them that it's a it's a very nuanced,

very broad, very multifaceted

discipline

and really hard to define. And they kept asking me, so I finally produced

a fairly complex mind map that tried to touch on all the different different aspects of software architecture. And even that was willfully incomplete. But I think they started gathering up the scope of what you talk about when you talk about software architecture. Because it really encompasses

the things that most people think about when you talk about software architecture, which are the kind of structural scaffolding

of how software projects fit together. But it also encompasses a lot of things like,

soft skills, like, you know, being able to conduct a meeting and lead a team and

be able to convince people. You know, it doesn't matter how brilliant your architectural vision is. If you can't convince business people to spend money on that vision, you're never gonna get to implement it. So part of that is part of software architecture as well. So,

it's hard to pin it down because it's so nuanced, and it keeps growing over time.

For example,

many years ago, maybe 15 years ago,

architecture and operations were completely separate things and were often

housed in different buildings and sometimes run by different,

completely different organizations.

In other words, it was common to outsource operations.

But you look at modern architectural styles like microservices,

and part of the superpower

of microservices

is this ability to treat operations

as an architectural aspect

of your system.

And so that's an example of how the influence and the scope of architecture has grown even within the last

5 or 6 years.

And when I was looking through some of the,

beginnings of your recent book about evolutionary

system design, 1 of the things that it covers in the first chapter is the concept of software architecture. And as you said, there are so many different components to it beyond what you just think of with the software pieces of it, including the data

and the processes that go around it. So 1 of the things that people have traditionally said about software architectures is the stuff that's hard to change. That was 1 of the the kind of colloquial tongue in cheek definitions of software architectures. It's It's the stuff that's hard to change later. But then things like microservices came along, and they actually built change into the architecture. And we realized, hey. If you if you actually build it to facilitate that, then it's not nearly so difficult. 1 of the problems I think that we're trying to address in our book is that for a long time, software architects have been treating software architecture almost like a mathematical equation. It's something to be solved. And once you solved it, you can kind of drop the whiteboard marker and walk away.

But it's a living, breathing thing, and it's constantly changing, and it changes

in 2 ways.

And we normally think of change just as a singular thing, but it's really 2 way 2 things. 1 way it changes is driven by the business because, you know, the business changes or you merge with another company or you started a new line of business.

But the other kind of change is kind of insidious, and it's the 1 that really nabs us, and that's the ecosystem change.

So I've been talking to a lot of

enterprise architects over the last couple of years in the process of writing this book, and I'm preparing to write another book more focused on enterprise architecture. And I've been asking him this question sort of tongue in cheek, but to really drive home the point. I've been asking the question, can you predict with any certainty what JavaScript web framework you'll be using 2 years from now? And of course, they can't because it probably hasn't been written yet. But that's a big problem for the traditional role of software architects, which is all about long term strategic planning. You can't do long term strategic planning in an ecosystem that is as dynamic as ours. And so 1 of the things we're trying to do in this book is to get people to think about architectures as living things that change all the time. And in fact,

change may be thrust upon you by the ecosystem that you're in for you know, you didn't plan to change your JavaScript front end, but you had to because the 1 you're using suddenly

the dude who wrote it decided he wanted to become a surfer instead. And now no one's maintaining it, and you know, you gotta move to something else. So,

I think the world that we live in increasingly forces us to treat architecture as very much a dynamic thing and not a static thing anymore. And the counterpoint to that too

is the system that anyone who's been in the industry long enough has come across where it started off with the best of intentions, but over time, it has accrued

enough technical debt that forward progress becomes nearly impossible because of the level of fear that's involved with changing any aspect of the system because you don't know what's going to break and why. So I'm wondering if you have any sort of general principles that you've come up with that are useful for being able to guard against that eventuality

or to get somebody unstuck when they do end up at that situation?

Yeah. We've got a bunch of them. In fact, we wrote a whole book about them. But that's a really common problem that you run into. And part of that is this idea that architecture can't change. And and part of that is really around something you alluded to earlier is the engineering practices that really lead you to be able to do really good architectural stuff.

So the great case study that we,

talk about in our book from, the GitHub engineering guys,

they have an engineering blog, and there's a blog entry

called Move Fast and Fix Things on the GitHub engineering blog. So 1 of the things that people are terrified of

are

the architects are terrified of making

substantive structural changes to existing systems

because they're afraid of introducing

regression bugs and breaking things that were working before. And, you know, suddenly things are unstable where they used to be really stable. And that's exactly what this blog talks about. Now GitHub is a very, very,

aggressive organization

from engineering

standpoint. They deploy on average 60 times a day, so they practice continuous deployment, and they're very aggressive about that. And this blog talks about

them creating this tool called Scientist, and they actually open sourced it, which allows you to experimentally

replace bits of your infrastructure without breaking stuff. And so basically, what it does is allow you to set up experiments so that for 1% of the cases, it runs the experiment in addition to what it would normally run and allows you to compare times and outcomes

and make sure that the thing you're replacing is working exactly the same way as the old 1 before you actually make that structural change. And so that's a good example of so 1 of the things that people used to think was, well, you know, you wanna build architecture to be stable as possible so you never have to change them. But that's that's a fool's errand now because everything changes all the time, and you can't make it stop. The only way to really make that possible is to embrace the idea that everything's gonna change all the time and get really good at being able to incorporate change. And so that's all the stuff that we learned over the last deck, I don't not quite decade

of about continuous delivery and continuous deployment and all the engineering that come along with that. So that definitely helps,

build an architecture that can withstand change as having the actual moving parts to enable you to to,

withstand change. But there are other ways that you can do this as well. We have an entire chapter about migrating architectures from 1 kind to another because you very often

build an architecture for 1 purpose, and then your business changes, or the nature of why you're building that system changes. And

very often, you know so there are a bunch of common architectural patterns out in the world, like, you know, layered monoliths or microservices

or micro kernel or

event driven architectures. And each of those encapsulates some desirable set of architectural characteristics, but no architecture supports all of them maximally.

1 of the words that you hear most often associated with software architecture is the word trade off because there are no good clean binary choices in software architecture.

Everything is some sort of messy trade off. And the really the the art and craft and skill

of software architecture is given the constraints that you have

driven by the business requirements, plus all the external constraints

driven by the ecosystem you're building the code in and expected number of users.

And what sort of change you're anticipating

is finding the right sort of architectural pattern that tries to maximize as many of the desirable characteristics you want and minimize the ones that you don't want. And so you choose a particular pattern, but then

your business goals change. Maybe you started out by being, you know, a slow moving commodity kind of business, and you're trying to convert yourself into a fast moving kind of modern economy kind of business,

it's hard to change the fundamental characteristics of your architecture.

You can do that, but it's a very difficult thing to do. And you have to do it very slowly at a piecemeal.

1 of the best pieces of advice that I can give from an architectural standpoint

is improve the modularity of what's already there.

So most good architectures

have some degree of separation and

And generally, the better, firmer the degree of separation,

the better off you are as long as you haven't gone crazy and created artificial

degrees of separation.

And so a really common

goal that a lot of people are pursuing right now is taking monolithic applications

and restructuring them into

service based or microservice architectures.

And we do a lot of that work, actually professionally right now. And the first step in that process is improve the modularity of what's already there.

Figure out what is actually coupled to everything else and which of those things do we really need to share and preserve that coupling point in the new architecture, or do we need to duplicate that functionality

and break the coupling point? And, of course, there are pros and cons and trade offs of each of those things. But that analysis

helps you

figure out exactly where the seams are between the modules in your existing architecture. And that, in turn, helps you figure out how you can successfully restructure.

And for people who are working in smaller companies, when they think about a software architect, it's generally associated with larger companies or enterprise

and a b as being a dedicated role of somebody who says, this is how the system is going to work together, so go ahead and build it. So I'm wondering if you can talk a bit about

where the responsibility of software architecture lies,

and at what point does it make sense to break it out into a dedicated role within an organization versus making it just be, something that each of the developers needs to consider in the process of building a working system?

It's a great question. In fact, 1 of the things that we've identified

for the O'Reilly Software Architecture Conference is this role that we describe as the accidental architect.

Because if you make a decision on a software project, even if you're a 1 person project, if you make a decision that has a major structural impact about architectural characteristics,

then that is an architectural decision. Whether you're whether you have that title or, you know, you have the the,

the accolades and the, headaches of that the the officialness of that role or not, that is the role of architect. So it is not a something that has a hard distinct boundary. It's a very, very fuzzy spectrum

because any decision

that really impacts these things that we call architectural characteristics.

So there's a great list on Wikipedia

called a system a list of system quality attributes,

which are all the things that you could support in architecture.

Really being able to support all those things is is impossible in architecture, and so that's really what you're trying to do in architecture is maximize

or pick which 1 of those you think is

is the best ones that that you want for an architecture. And if you're making that decision as a developer, you're making an architecture

decision whether you call yourself an architect or not. Digging a bit deeper into that, I'm wondering if you can sort of characterize

the responsibilities

of somebody who does have that architect role as their job title versus somebody who is operating at the individual contributor level? Like I said, it's a really fuzzy spectrum. So we have some projects that have that distinct role. And very often what that role means is that's the decision maker or the person who is accepting responsibility for this whole slew

of things that need decisions about. So for example, this is something that came up on a project not too long ago.

So we had an existing architecture, and we needed to make a change

because we were changing the nature of the business. So we had a single

connection to the outside world, and we're adding 2 new kinds of connections to the outside world. And the question came up from an architectural standpoint,

do we wanna use a dedicated message queue for those new connections, or do we wanna share the 1 we already have? That's an architectural level decision whether you're an architect or not, and being able to make that decision requires some analysis.

You have to go and be able to ask some questions. What what's the expected throughput

of this? All we do we need to know from the destinations from where the message is coming? Because that might have an impact on that architectural decision.

And so

that thought process that you go through is really the role of architecture.

Whether you're formally doing that or informally doing that, you should capture that somehow,

because that's really what you're doing as an architect is working through

this list of, you know, here are the things I need to consider.

Here are the outcomes of those things. And the assumptions that I'm making as I build

this particular architecture

because, you know, someone going forward may need to know that assumption because, you know, fundamental assumptions may change over time,

And so that may change why you're using 1 particular architecture.

And bringing it to the language level, I'm wondering if there are any aspects

of different programming languages and their associated ecosystems

that play a major role in influencing

the style of architecture that is generally used within that community or the architectural patterns that you're actually able to implement? Yep. Absolutely. It has a huge impact because in many ways, the language is the building blocks for the kinds of things that you could build easily within architecture. You can put an enormous amount of effort and, you know, build almost anything using any kind of software sometimes with enormous ridiculous effort to do that. But there are sweet spots that each language and each platform offers.

A great example of the

your question

is when the functional programming stuff finally made it to Java in Java 8 because

a there's a common architectural pattern in this kind of pipes and filters architectural pattern that was not commonly used in Java because it was a little bit cumbersome to put together, and it was much more commonly used in ecosystems that

more commonly had that language pattern, like functional programming languages. This is where MapReduce kind of ecosystems and frameworks and tools like Hadoop come in.

So those are a very natural kind of pattern in those platforms, and that became available in Java. And so consequently, you're seeing more of that kind of pipes and filters architecture showing up in the Java world just because

those native capabilities exist in language. And I think that's been more common in Python for a while because those kind of map reduce

and functional programming

concepts have been available in Python for a while. And so it's more common to see those

kinds of capabilities show up in architecture just because it's a natural way that developers in that ecosystem think. And what are some of

the architectural anti patterns that you have found to be the most commonly occurring and the most difficult to address?

Potentially, because most of the, architectural patterns that we talk about are structural things like monoliths or microservices,

But almost all the anti patterns are behavioral things,

that people keep doing

stupid things about.

So I'll give you a few of them. Most of these get talked about at some point or another in

our Building Evolution Architecture's book. 1 of them is the witch's brew

anti pattern where you don't have anyone on a team who's making

architectural decisions. And so everybody's trying to contribute to the architecture, and you end up with a witch's brew architecture, which is no clear vision of any kind. It's just sort of a hodgepodge of a bunch of different stuff. Many of the people who are listening to this have have come across a project that has, you know, 4 or 5 different web frameworks in it and, you know, 6 or 7 different XML parsing libraries in it because there was no real clear vision. And so everybody was just kind of doing their own thing. That's a great example of the, the witch's brew antipattern.

Another really common anti pattern that we call out of the Evolution Architecture's book is 1,

that I refer to is the meta work is more interesting than work anti pattern.

Because every developer I know

would much rather write a new framework

than use an existing framework to create something useful, because metal work is more interesting than work because and I ask this question a lot when I speak at conferences. When you when you really boil it down to its essence. So for most developers, when they're talking to their grandparents or

someone who doesn't know a lot about software and you really boil down their job to its pure essence,

They take things from web pages and put them in databases, and they take things from databases and put them on web pages.

Yeah. There's a bunch of details in between there, but, you know, that's the essence of what they're doing. And once you've done that professionally for 5 or 10 or 15 times, it gets kind of boring.

Developers don't like to be bored. And so what developers who are bored do is they invent puzzles to solve at work so that they have cool puzzles to solve at work. And sometimes that's your own homegrown

homegrown web framework that you have to maintain all the time when there's a perfectly good open source 1 that does 90% of what yours does, but that 10%, you know, is somehow critical. And so you spend all your time

nurturing this, you know, this baby that you have. But that's a really common anti pattern that a lot of architecture teams fall into is the

we want to chase really interesting things, not things that are actually valuable to try and get stuff done.

And so, you know, a lot of a lot of projects end up being these kind

of sci fi research development skunkworks projects when that effort could be much better spent, you know, cleaning up old, terrible data and other sorts

of pragmatic, but not as not as exciting sort of things. Yeah. The, the wonderful not invented here syndrome

and, the the the concept of resume driven development where I just wanna use this new tool because it's new, not because it necessarily solves the problem that I'm trying to work on.

So I worked we did I did some work 1 time for a financial service company

that had, written their own web framework, that written their own Korma Orb, that written their own application server.

I asked them at some point, did you write your own operating system? And they said no. And I said, why not? You've written everything out from scratch. But, of course, the problem, anytime you find a company like that, is the very best developers in the company are in full time maintenance mode

of their craptaculous

internally built half assed frameworks

that, you know, don't do half of what you can download for free from the open source world because there are platoons of people in the open source world working on those things. And,

I've seen so many companies are wasting their very best developer talent, who you would hope is right on the front lines of innovation trying to build the coolest stuff, and instead, they're in the worst kind of maintenance mode of fighting rear guard battles for awful homegrown infrastructure.

And you bring up an interesting point there too with the rise of open source and its level of popularity

and the fact that you can just go out and take advantage of so many man hours worth of work. I'm wondering if you have any viewpoints on the effects that open source has had on

the available

architectural styles and the capabilities

of software teams to be able to implement successful architectural patterns.

Oh, there's no way that things like microservices could possibly exist without open source. Because here here's a thought experiment for you. So let's say it's 8 years ago,

and you go to the head of operations of your company, and you go, hey. I got this great idea, this new architectural style. It's really decoupled. And what I'm what what we're gonna do is run every service in their own operating system with their own database.

And so I'm gonna need 200 Windows licenses and 200 Oracle licenses

so I can run each of these services or own dedicated machines, they would have run you out of your operation center so fast your head would spin.

Because it made no sense to do an architectural like microservices if you had to pay per instance the operating system and per instance the database server. That was actually 1 of the external influences on architecture that we had to deal with for a long time,

which was this resources computing resources are really expensive and scarce. And so you have to build

architectures

that try to maximize shared resources as much as possible.

And so that's where you get things like service oriented architectures because 1 of the outcomes of service oriented architecture was you could run

enterprise level computing

with all these shared resources on,

you know, a fixed set of of operational machinery.

But then an interesting thing happened. This goes exactly back to what I was talking about before about the shifting ecosystem,

was that

gradually, Linux became good enough for enterprises to start trusting.

And, of course, Linux is commercially free, but that wasn't quite enough.

It was when the DevOps revolution happened,

and Puppet and Chef and related tools came along.

And suddenly,

Linux became operationally free. And then things like Docker came along. And now you can say from an operational standpoint, look. I'm gonna let each team own their own image of for the service they're building, and I'll just manage those. And suddenly, you have a level of control and responsibility

at the service level that was impossible

10 years ago. And so that's a great example of open source enabling

some capability that would just literally be impossible

in the, the commercial software world.

And we're seeing this accelerate. 1 of the observations we made on our last Thoughtworks Technology Radar

is that a lot of Chinese companies are starting to aggressively

open source software.

And there are a lot

of software developers in China.

And we're starting to see a flood of really high quality open source software coming from China, and they're doing it for the exact same reason that companies like Facebook

are open sourcing some of their software. So if you think about it, things like React,

that Facebook open sources.

If this were 10 years ago,

that source code would be in the deepest, darkest vault at Facebook with 24 hour armed guards because that's some of the coolest stuff they

have from a company. Why would they possibly open source that?

Well, 1 of the reasons is because if you're in a market like Silicon Valley, where everybody's paying the same amount of money for developer talent, how do you attract top talent?

You show them that you're building the coolest stuff,

and then people will flock to you because they wanna build cool stuff too. That's exactly what's happening in China. The job market is heating up for software developers in China, and so a lot of the big companies like Baidu and Alibaba

are starting to really aggressively open source things to show people that, you know, hey. We're building cool stuff. You should come work for us. So that's gonna have a big impact

on the open source ecosystem because suddenly,

there's this brand new source of really high quality open source

starting to appear that we can start taking advantage of. And bringing it to the delivery point,

no matter

how interesting

or

capable your software system is, it's ultimately useless if there's no way to put it in front of the end user.

And I'm wondering what are some of the challenges that are most often overlooked by engineering teams

as they're trying to be able to deliver that software, and what are some of the methods that you have seen to be able to solve for those challenges?

Fortunately, most of those are nicely encapsulated in the book, Continuous Delivery,

which is written by Jess Humble and Dave Farley,

published in 2010. It really,

so the name of that book is a really nice marketing move because it's a nice catchy term. If I had written that book, I would have given it some sort of highly accurate, but never, a title. It would never sell, like, you know, engineering practices for software projects or something like that,

which would never sell a single copy, but that's really what that is.

And that that really addresses DevOps and a lot of the

challenges

that that a lot of companies have. So, you know, I'm I'm a consultant, so I get to see a lot of interesting broken

software in companies. And 1 of the common refrains that we hear at every company, it's it's an inside joke within Thoughtworks, every single company that we go to. At some point, as we're talking to them, someone says, well,

you know, but our problem is much harder than everyone else's problem. And it's really not. You just made it interesting and difficult in stupid ways.

1 of the best pieces of advice that you could could do is so there's this phenomenon,

known as the out of town consultant effect. And what that says is

that the further you had to travel to get to a meeting, the more credibility you have. And if you rode it in the airplane to get to a meeting and everyone else rode in a car, you automatically have way more credibility than there are all the core people have.

That's the out of town consultant effect. But there is some truth to that.

Because what happens in companies so there's a story that's told in the Pragmatic Programmer book. And it turns out it's apocryphal, but it's a great story

about boiling frogs.

So the pragmatic program book alleges that if you want to boil a frog, that you can't just throw a frog in a pot of boiling water, that they'll hop right out. But if you put a frog in a pad of cold water, heat it gradually, they'll cook without realizing it. Well, it turns out that frogs are not that stupid, but large corporations

are that stupid.

This happens

all the time where

the out of town consultant comes in and points out that, hey. The thing that you're doing right there is really, really profoundly stupid, and it's almost like the people wake up out of a daze and they go, wow. You're right. That is stupid. Why are we doing that? And it's because, well, we did it this way, and, you know, this way led to that way. And this and so you'll you know, all this history, but the history makes no difference. What you're doing right now is dumb, and it doesn't make any sense from an engineering standpoint. So 1 of the benefits of this out of town consultant thing is looking at those things with fresh eyes. And so 1 of the piece of advice I give people, so when you go into work Monday morning,

look at everything with fresh eyes and ask, is this really the best way we should be doing this? I'll give you a great example of this. It's happening on many, many projects right now. Is people are using branches in version control instead of better techniques like trunk based development. This is a great website

run by 1 of my former colleagues, Paul Hammett, called trunkbaseddevelopment.com.

And a lot of projects end up bending themselves into pretzels around the branching strategies they build

around their version control systems.

This is a great example of meta work is more interesting than work because they built this cool puzzle around branching and version control and how releases work and all this stuff, where you can use things like feature toggles, trunk based development, and branch by abstraction, and a bunch of engineering practices,

and actually,

get better

turnaround for continuous integration, better and other engineering practices, and get away from all the complexity that you're building from

branching and other things just because you've done it that way forever. And going back to your point about being about fresh eyes being very useful for an organization

is that for companies who are hiring new people, 1 of the most useful things that you can have them do is for the first, you know, day or week or month that they're on the job, just have them document all of the painful things that they come across in their onboarding process

and ask them to fix them or help you fix them so that the next time somebody comes along

after the new person has already become jaded and,

submitted to Stockholm syndrome and think that everything is fine, then the,

the next person can go through the same process. And eventually, the onboarding will become less painful and a lot of their systems will be much improved by the virtue of having those fresh eyes coming in. And that's also where junior engineers can be most valuable as well is because they don't have all of the institutional

knowledge or all of the bad practices that have already been learned, then they can come in completely fresh and say, I don't understand what's happening here. And then if you're forced to explain it, then maybe some 1 of the senior members of the team will also realize maybe this isn't the right way to be going about it.

Absolutely. I think that's a great way.

We always encourage people to question anything that we're doing as to, you know, why are you doing that? And, you know, is there a good explanation for why you're doing that? Sometimes there is. And, you know, we're not trying to, you know, get everybody to question every single thing all the time. But, you know, it is good to look at things with fresh eyes and rethink them on a regular basis. I think people get really blind to the the the habit

part of their job. And, you know, things change and, you know, capabilities show up that you don't take advantage of. And

that's, that's deadly in the software world because everything changes all the time. 1 of the things that we're seeing out of the world is the gap between companies who are who really get this and, you know, are really

investing in engineering practices and, you know, can have really fast turnaround on new features.

They are pulling away from companies who still have a very 20th century attitude about IT, that IT is overhead,

and it's a cost center, and you try to minimize cost as much as possible.

What's gonna happen is, someone's gonna move into their market that is very aggressive

in their strategic use of software, and, they're gonna find themselves in a difficult situation.

And beyond the purely technical aspects

of software systems and the

technical patterns that go into building them, what are some of the other elements of production

software and delivery that are necessary for a successful architecture that we didn't already talk about.

I touched a little bit on, some of the soft skills and other things that you have to have as an architect. 1 of the things that you should really,

1 of the skills you'd have as an architect is be a really good presenter

and be able to present your eyes really, present your ideas really well in front of group. Because that's the only way you're ever gonna get to implement any of your architectures, no matter how brilliant you are. And so,

1 of the ways to to get your architecture,

implemented is to convince people that you have the the right vision to be able to do this. So 1 of the things I tell architects,

if you've got some new vision for,

what you're thinking you want the system to look like, you should practice drawing it on a whiteboard

so that you can do it without thinking about it. Have you ever seen people who can draw all 50 states, just kind of freehand and draw, you know, the outline of the United States? You wanna be able to do that with your architecture because if you find yourself at an impromptu meeting with someone and they say, hey. You know, Fred's got a good idea for how to solve that problem. Why don't you explain it to us, Fred? The ability to walk up and confidently be able to draw it and not run out of room off the edge and not back up and go, oh, no. No. Wait a minute. You know, just being able to draw, that that conveys this level of certainty that,

you know, you might not have, but, it gives you that that impression that you've thought through it because you have, because you've drawn it a few times. And

so all those soft skills about, you know, leadership,

I've been on the architect on projects where

there's a fundamental disagreement about things like how we should represent data. And, you know, at the end of the day, both both sides of the argument came to me and gave me their best case. And as the architect, I had to decide, and I had to choose 1 of them over the other. There was no way they were ever going to reconcile their difference because it was just a fundamental philosophical difference. And 1 of the things I had to deal with over time were the repercussions of, you know, picking a winner versus a loser in that particular, you know, head to head showdown. So all those little things are the things that, you know, you become an architect by all of your technical acumen,

but then the real skills of architecture have little to do with the technical stuff and way more to do with soft skills and presentations and writing and and documenting and meetings

and setting agendas

and

scheduling and prioritization and all those other things

that are the necessary

byproducts

of the job architect.

And 1 of the things that you mentioned in there that stuck out to me is that a lot of times when somebody thinks about architecture,

the concrete

artifact that they consider from that is a system level diagram, whether that's the

infrastructure that's necessary to run the application or a flow diagram to show the user interaction paths.

And those are very valuable things to be able to have for a software system, but due to the nature of change, they very quickly come to be out of date. So I'm wondering if you have any recommendations

for tools or practices for being able to generate and maintain those diagrams to be able to help with the onboarding process for new sis for, for new people so that they can understand the system that they're working with, but also ensure that the diagrams that they're looking at aren't going to mislead them

into a false sense of understanding.

Yeah. We've we've hit upon a really, really important aspect of a system, and it actually reinforces something I was saying earlier that all of these things are living, breathing things. They constantly change over time. So the static diagram you capture right now in 6 months, probably

it will have changed in legitimate ways away from that. So 1 of the pieces of advice I always give people is that as the architect, you want to have, you know, some sort of resource for that for important diagrams or software project. But I always have 2 directories there.

1 is current, and 1 is archaeology.

And as the architect, everything that's in current is important enough that I'm going to keep it updated the current state.

And as soon as

I come to the conclusion that it's more trouble than it's worth to do that, I'm gonna move that over to archaeology.

Everything in the archaeology directory

are things that were important accurate depictions at 1 time, but we moved away from them for 1 reason or another.

So if you're interested in the way that the data design was originally designed to work with the persistent system, then it's there, but that may not be the way it works now. And so

as the architect, that kind of holds my feet to the fire for, okay, these things have to be living, breathing documents, and they have to be valuable enough for me to actually put effort into keeping them updated

as, as part of the project. And I think that's the responsibility of an architect

because the worst thing, as you said, to have is

some sort of documentation, but then you start looking at it and you go, oh, created date

3 years ago. Oh, last modified date 7 months ago. Oh, I don't know how accurate this is. If last modified date was 3 days ago,

and there's only 3 documents in current folder, then you've got a pretty good confidence that what you're looking at is an accurate depiction

of what's going on.

The other thing that we identify about those kinds

of artifacts is that lo fi is best.

There's an anti pattern that we identify that I call, irrational artifact attachment.

There seems to be a direct correlation between the amount of time that you spent creating some sort of artifact and how irrationally attached you are that artifact.

So if you draw something on a whiteboard, it takes you 5 minutes and somebody wants to change it, you're not mad about it. But if you spit some tool like Omnigram or Visio and spent 3 days working on something and somebody starts,

starts complaining about it, you'll start defending it because you spent so much time creating it. That's irrational artifact attachment. So lo fi things are better. And in fact, a lot of times now, I don't do whiteboard diagrams. I draw it on an iPad connected to an overhead projector. So I've got 1 of the ridiculously expensive little dongles that allow you to attach your iPad to an overhead projector.

And I use a drawing piece of software,

and that way I can have as many pages as I want. I can annotate

them. Those get automatically get synced up to my, Evernote account, which does optical character recognition on them.

And so, that's a great way to keep those kind of ephemeral

diagrams is by creating them on the fly using something like an iPad, and then they're easy to update because they're already in electronic format. You're not trying to take pictures of glary

whiteboards and other sorts of, you

know, MacGyver like moves.

And for people who are interested in learning more about the principles of software architecture

and how they can become more involved and informed in that process, whether as an individual contributor or as a full time architect. I'm wondering if you have any specific

recommendations

for resources

or sort of practices that they can incorporate into their day to help them progress along that path?

It's actually really, really difficult. A few years ago, Mark Richards, who's a really well known soccer architect, and I started looking around. And I made the observation at the time that if you look at and this is not just in the US. This is pretty much worldwide.

The top 10 salary lists.

A software architect usually falls somewhere in the top 10 of, you know, most desirable salary jobs. But the

how you get to be a software architect is not at all obvious because there are no good resources

to take you from not a software architect to becoming a software architect. And so he and I actually created a series of videos for O'Reilly called Software Architecture Fundamentals

that starts talking about all those things. Mark actually does a lot of public training classes around this idea of software architecture fundamentals because there aren't any really great resources that are comprehensive that tries to touch on all the things, you know, that that you have to deal with from a software architecture standpoint.

So there are a lot of resources you can find out in the world around software architecture patterns. There are a lot of pattern books out there. Martin Fowler has a patterns of application architecture

book. There's a book by Gregor Hope and Bobby Wolf on enterprise integration patterns. So these are all the integration architecture patterns that exist out in the world. But 1 of the problems you run into there is that when you read about patterns until you're blue in the face, but, you know, until you actually get to implement some of these things, it's hard to get the kind of practical,

application.

And that's a real fundamental problem with architects is you don't get to create new architectures very often. You get to work in existing ones a lot.

Recruiting new ones is quite rare. And so 1 of the exercises that we go through when we do a software architecture fundamentals class, And I'll turn this on for all your your listeners here if they wanna do this. On my website, there's no obvious link on my website. But if you go to my website, which is nealford.com/katas,

k a t a s,

That leads you to these architectural katas exercises.

Each 1 of these is a little puzzle

that's a list of business requirements and a little bit of additional context. And when we do a training class, what we do is create little teams of 3 to 5 people as architects

and then assign each 1 of them 1 of these and

have them go through all these architectural processes of identifying what architectural characteristics are important, what are the component boundaries,

what sort of architectural pattern would be suitable,

etcetera.

And so people who are listening to this can go through this exercise themselves if they want. So get some people within your company who are interested in software architecture together for, like, a brown bag lunch.

Go through a Kata's exercise, and then get some existing architecture in your company to come make some comments about your design. What did you get right? What are the things that you missed? And work through this process because,

you know, the only way you get better at anything is practice it. And so this is a way that you can actually practice the practical mechanical parts of software architecture of assessing trade offs, and identifying component boundaries, and that sort of stuff in a sort of a classroom style setting.

Alright. And are there any other topics that we didn't cover yet that you think we should discuss before we start to close out the show?

No. I don't think so. We, we touched on, I think, 1 of everything.

Alright. There are definitely many areas that we could have gone much deeper on, but those are probably an entire episode unto themselves.

So

hopefully, this will serve as a useful overview to peoples who are interested in learning more about the ideas of software architecture and how they can begin applying that in their own roles. Yeah. It's it's definitely a big, a big topic.

Just evolution architecture by itself, of course, is a big enough topic to write a book about. So, yeah, there's a lot of stuff going on in that world, and it's ever changing. So it keeps me on my toes.

And for anybody who wants to follow the work that you're up to or get in touch, I'll have you add your preferred contact information to the show notes. And so with that, I'll move us on into the picks.

And this week, I'm actually going to choose a couple of different movies that I watched yesterday that both happened to be about the jungle.

Not it didn't do that consciously, but,

I went and saw the, newest Jumanji movie in the theaters yesterday, which was absolutely hilarious. So I definitely recommend everybody go and watch that. I had a good time with my whole family on that 1. And then, yesterday evening, I watched The Lost City of Zed

on Amazon, and that was based on a true story of a British army officer who,

spent a lot of time exploring the jungles

of South America in search for lost civilizations.

So, that was an interesting story and well presented. So,

for anybody who's curious about that, I definitely recommend that as well. And so with that, I'll pass it to you, Neil. Do you have any picks this week? Yeah. I'll give you 1 that just came out. You were asking me earlier about good resources for,

burgeoning

architects.

There's a great website that Mark Richards just created called developer

architect.com.

It's just for developer, to architect.com,

and it's a great resource that has videos and white papers and a bunch of other stuff

for brand new architects.

So, that's a good resource for people to check out. And and Mark only just launched this a few weeks ago. We have a website around our book, and we're actually starting a blog series shortly,

illustrating a bunch of the things in the book. It's at evolutionaryarchitecture

dotcom, and there's some other resources there,

related to the book. Something completely unrelated to software architecture

that,

is,

this science fiction book series that has really captivated me and a bunch of my friends. It's the broken earth series.

And the first 1 of them is The 5th Season.

Really terrific book. The third 1 just came out. I think they've all won awards and for really good reason.

Really, really inventive world building, really fantastic

science fiction book series. I'm just finished up the second 1, and I'm chomping at the bits to start the third 1. So, if you like science fiction world building, and this

is set so far into the future that the continents have burned back together into a single continent.

So very far future kind of science fiction. It's a great, great piece of work.

Yeah. It definitely sounds very interesting. I'm,

currently reading a sci fi series myself that I mentioned in,

1 of the episodes I recorded recently called the Berserker series by Fred Saberhagen. So I'll have to take a look at the Broken Earth series.

It's so good. But, boy, it's deadly. Once you start reading it, you can't put it down. So reserve some time.

Alright.

Well, I really appreciate you taking the time out of your day to join me and talk about the work that you've been doing in the software architecture space. It's definitely something that I have found interesting for a while and hope to continue to learn more about. So thank you for your time and I hope you enjoy the rest of your day. Yeah. It's my pleasure. Thanks for having me.

The Python Podcast.init

Summary

Preface

Interview

Keep In Touch

Picks

Links

The Python Podcast.__init__