The Python Podcast.__init__

Share on social media:

[00:00:14] Unknown:

Hello, and welcome to podcast dot in it, the podcast about Python and the people who make it great. When you're ready to launch your next app or you want to try a project you hear about on the show, you'll need somewhere to deploy it. So take a look at our friends over at Linode. With 200 gigabit private networking,   scalable shared block storage, node balancers, and a 40 gigabit public network, all controlled by a brand new API, you've got everything you need to scale up. And for your tasks that need fast computation, such as training machine learning models or running your CI pipelines, they just launched dedicated CPU instances.   In addition to that, they just launched a new data center in Toronto, and they've got 1 opening in Mumbai at the end of 2019.  

Go to python podcast.com/linode,   that's l I n o d e, today to get a $20 credit and launch a new server in under a minute. And don't forget to thank them for their continued support of the show. And to keep track of how your team is progressing on building new features and squashing bugs, you need a project management system that can keep up with you that's designed by software engineers for software engineers.   Clubhouse lets you craft a workflow that fits your style, including per team tasks, cross project epics, a large suite of prebuilt integrations, and a simple API for crafting your own.   With such an intuitive tool, it's easy to make sure that everyone in the business is on the same page.  

Podcast.init   listeners get 2 months free on any plan by going to python podcast.com/clubhouse   today and signing up for a free trial.   And you can visit the site at python podcast.com   to subscribe to the show, sign up for the mailing list, read the show notes, and get in touch. And if you have any questions, comments, or suggestions, I'd love to hear them.   And to help other people find the show, please leave a review on Itunes and tell your friends and coworkers. Your host as usual is Tobias Macy. And today, I'm interviewing Harry Percival about domain driven design and enterprise application architecture in Python. So, Harry, could you start by introducing yourself?  

[00:02:05] Unknown:

Hi, Tobias. Thank you very much for having me. So my name is Harry. I'm a Python programmer from the UK. I've been doing Python for maybe   10, 11 years. And, at the moment, I work at a sort of furniture retailer   or design brand called made.com.   We sell kind of   furniture and other homewares   throughout the UK and Europe.  

[00:02:25] Unknown:

And for anybody who hasn't encountered you before, they might know you from your,   Obey the Testing Goat book or from your fantastic jokes at various pythons.  

[00:02:37] Unknown:

Yeah. Sure. That's right. So,   my my my previous career or my previous job before this was at a startup   called Python Anywhere, which is a kind of Python hosting platform as a service type of thing.   And even before that, that was a company that morphed out of a spreadsheet company. It used to build   a kind of Python based spreadsheet like Excel, but with Python as the kind of scripting language, which is really cool. And I joined them as a kind of an intern during my my second degree, and that is now,   2,009, I think.  

And they were an extreme programming shop, so they did XP. They did test driven development for absolutely everything. They did pair programming on everything.   And, basically, after a couple of years of that,   I wrote a book about it, which which was called Test Driven Development with Python,   just saying, you know, for self taught programmers,   here's here's a kind of restructured rigorous way to approach coding. Test first, incremental,  

[00:03:32] Unknown:

you know, version control, all this good stuff. And I'm wondering if you can just share how you first got introduced to Python.  

[00:03:38] Unknown:

Yeah. So, actually,   when I was,   I mentioned spreadsheets. While I was kind of,   paying my way through my university degree,   I had a client. And I would basically help him build these these spreadsheets, which were like pricing models that he would use to go out to do competitive tenders for legal services. And, like, the, you know, the lawyers would fill in, like, these spreadsheets to say what kind of prices they would do for different things.   And and then, you know, having been doing this to view in computer programming for a couple of years, I eventually said to him, you know, like, we could we could stop using spreadsheets, and we could build this as a web application. And people could, like, submit their pricing, and then we can have a UI for, you know, clients to to go and find the best price and stuff.   And that was my first kind of real   consulting engagement, my first real   programming client. And he said, that sounds good. You know? I trust   you. And, he says, tell you what. Since you've never really built a website before, why don't you call up my hosting company and get them to just support you? Now at least provide the hosting and give you some advice.  

So I called them up and I said, hey. What's up? I thought I was gonna build this thing. And they're like, right. Well, we use Django, so that's we recommend you use. And so I said, okay. I'll give it a go. And so I think it was just before Christmas,   I went home for the Christmas break with,   a copy of Dive Into Python   and a copy of, I think the the original Django book.   I basically read those 2 things over the Christmas holiday and sort of came back having taught myself Python and Django as best I could in the space of kind of 2 weeks,   and and I was off to the races.  



[00:05:07] Unknown:

Yeah. That's definitely 1 of the great things about Python   and Django in particular is that it is approachable enough where that's something that's feasible as opposed to if you had gone home with a, teach yourself c plus plus book and whatever passes for a web framework in that community. I'm sure it probably would have taken a bit more than just the 1 Christmas break to be Right. Right. I mean, it it's so often in the in the case in the in the Python world of of you know, I I say to people who are just learning a car, you know,  

[00:05:34] Unknown:

oh, gosh, I have to learn this whole programming language. And I'm just I just say to them, you know, actually, especially if you already know a bit of programming,   the way Python does it is kind of the most straightforward and intuitive way you can think of. You know, like it like, what's the simplest and most obvious way I could do a for loop? I want iterate for all the items in array. How about for item in array?   And, like, literally those words,   colon. And   and so many things like that. You know, it's it's all about pipe and strength in saying   there should be 1,   preferably only 1 obvious way to do it. Yeah. And so in your work of building  

[00:06:08] Unknown:

applications   and supporting   other members of your team and working in these extreme programming environments,   you appear to have   gotten bitten by the bug of being interested   in application architecture   and the concepts of domain driven design.   And I know that,   architecture is a very broad term. It encapsulates a lot of different ideas and potential approaches and   design patterns.   And the concept of enterprise application architecture   is something that   grew largely from the Java community.   And so I'm wondering   if you can just start by describing   what you mean when you're talking about application architecture   and how it compares to some of the typical types of design patterns that Python developers rely on. Yeah. Yeah. Yeah. Absolutely.  



[00:06:57] Unknown:

So   the   I'm a bit scared of the word enterprise. It's not there there is a subject called enterprise architecture, and that's really about   building a strategy for a large company for how are they going to evolve their portfolio of applications from   their, you know, what's insourced, what's outsourced, what do they buy off the shelf, you know, and they're people call enterprise architecture to kind of build technology strategies,   for for big companies. And that's kind of you know, a smaller company, that would just be the role of the CTO. So I'm not here to talk about enterprise architecture, and I'm really talking about application architecture, insofar as everyone agrees on on that bit of terminology.   And to sort of explain what that is,   design patterns is a thing. So people might have heard of design patterns.  

I'm talking about architectural patterns. Are they the same thing? Not really.   Design patterns, which are the classic Gang of 4 book,   was about giving you suggestions for how to build particularly   code in the object oriented,   paradigm. And it's things like singleton pattern, abstract factory, blah, blah, blah. And and lots of people out there in the Python community have spoken a bit about design patterns,   and particularly saying the way that, you know, something like 40% of those patterns that are famous and from that   book aren't really applicable in the Python world because,   in a way, they're workarounds for for features of the Java language that are just not really necessary in in the lovely dynamic world of Python. So I'm not here to talk about design patterns.  

I'm talking about application architecture. And I suppose   application architecture is a level of section higher than design patents.   And it's about saying, how do I build my application as a whole? How do I side how to structure my code? How do I   side how to split it up? How do I side what code to put where?   And and to make that more concrete,   for people, 1 of the most common architectural patterns that people have heard of is MVC, Model View Controller.   And so the idea is I'm gonna split my application up into a kind of model,   which talks about the data and the way I model the data. And then I'm gonna have   some views, which is how do I present the data to the user. And then I'm going to have some controllers, which are going to say, okay, well, given that the user,   asks me to do these sorts of things, how do I   control, orchestrate, put together that request with things from the model, and and build a response back for the user? That's Model View Controller.  

And then in the Django world, that translates more or less to the way Django asks you to structure things into   models which represent database modics and templates which are the kind of views of data and what Django calls view functions which are maybe more equivalent to controllers and MVC. But they're both examples   of a layered architecture which says, you know, I'm gonna build my application in layers. There's gonna be the lower layer of the data stuff. There's gonna be a kinda middle layer of business logic and and controllers,   and there's gonna be a presentation layer at the top.  

That that's the most, like, well known and and familiar architectural pattern probably.  

[00:09:57] Unknown:

1 of the interesting things about the idea of application architecture   is that there has been a bit of pushback   on the idea of upfront design with the advent of things like agile programming and the idea that we're trying to move away from waterfall where you   put everybody into a room for days or weeks. You write out everything about how the software is going to operate, and then you just go and spend all of your time building it exactly to that specification, which, as we all know, never works out the way that we plan. And so we've kind of gone to the extreme other end where rather than putting in a decent amount of thought into how everything needs to be modeled, we just say, we're just gonna start writing code, and we're gonna figure it out as we go, and everything will be wonderful, which also doesn't usually work out. And so I think as an industry, we're starting to oscillate back to a middle ground of actually having some of this upfront design, trying to get an understanding of the domain that we're working with, but still being able to approach the actual implementation in a agile and iterative approach. And I'm wondering what, in your experience, are some of the influences   in general, whether in team dynamics or industry dynamics that tend to lead engineers into   suboptimal architectures that end up becoming brittle or coalescing into the dreaded big ball of mud and some of the ways that you can potentially guard against those influences.  



[00:11:17] Unknown:

Yeah. There's a there's a bit to unpack there. But I guess   if the if the idea is, okay, you know, let's not worry too much about design upfront. We're just gonna be agile. Let's build something as quickly as you can. The way we do that is we pick a framework like Django. And Django's, you know, tagline is this, the framework for perfectionists with deadlines. And so it's saying, you know, here's the thing, it's going to get you up and running really quickly and it's going to let you meet some deadlines. And so I'm not gonna think too much about architecture. I'm gonna pick this framework. But what you've actually done is picked an architecture without thinking about it. And so Django is pushing you towards this kind of, layered architecture with kind of models and views and templates.   And and that's fine, and it's very successful, and it gets you to a lot of places very quickly. But I think it does run into some problems as things scale. And I think in the in   the in the Python world, we're starting to see some of the problems that that other languages and other communities have been through,   a long time ago. And so once you know, if you're just building a startup and you're building a web application and you're you're you're up and running for the 1st couple of years, Django is great and Flask is great and whatever it is. But after a while, you can run into limitations of that simple 3 layered architecture. Firstly, because it's not always even that easy to stick to that 3 layered architecture if you're not thinking about it. But even if you do, you know, you can run into problems. And so any application that grows to a certain level of complexity,   if you don't have a sort of reasonably structured way of thinking about, okay, well, like, where am I gonna you know, here's a new bit of functionality. Where am I gonna keep the code for this? Where is the place that's gonna make it easy to maintain, that's gonna make it testable? Then you will end up with a slightly haphazard approach, and you're gonna end up with the classic problems of of the increasing complexity of applications.  

The the kind of the jargon is the big ball of mud where everything kind of depends on everything else. And that, you know, making sort of 1 part of the system over here is hard to disentangle from from places over there. Or you just find yourself, due to the architecture of your framework, finding that, you   you know, your tests are just getting slower and slower. So it might even be that your application is quite well structured. But, you know, your test suite that used to run-in   under a second is now taking an hour and a half to get through CI. Or, you know, in the case of Python anyway, which is admittedly a bit of a a unique case, but it was taking, you know, half a day. I'm not saying that's, you know, particularly because Python Anywhere made, bad architectural choices. It's a unique case. But what I'm saying is, you know, what works really well at a small scale, you can start to struggle with as you as you grow.  

[00:13:40] Unknown:

And   so 1 of the   core concepts in the problem space of application architecture that has been adopted by more mature organizations   and language communities is the idea of domain driven design,   which has proven fairly successful in terms of being able to support large   applications and large organizations   as well as some of these evolutionary and iterative cycles of development practice.   And so I'm wondering if you can just unpack the term there and some of the benefits that it provides to software architecture   in general and some of the core concepts that are necessary   to get introduced to that approach to application design? Sure thing.  



[00:14:22] Unknown:

So,   domain driven design is a a kind of, a philosophy for for how to build applications, but also how to approach   your conversations with your stakeholders, with your business, so the the, you know, kind of requirements gathering side as well as just the coding side. And the the central conceit of it is that the most important part of an application is the model of the core domain, the core business domain, which, you know, maybe you'd use the word business logic or business rules or the representation of the problem space. And that is the beast   that delivers the most value and the conceit is saying, look, if get this right, if we have a model that really does capture the essence of the business and the way it does things or the way it understands the world, then that model   will A, be really easy to understand for both the business and developers.  

The application will be in sync and in tune with the business so it's gonna help them do their jobs more. And it may also even enable possibilities that people didn't even realize   were there. So if I've done a really good part a a job of modeling the business, I may suddenly be able to say, hang on. This concept and this concept, we can now bring these things together.   And, you know, did you realize that we could do this and sell this product in this way or or or deliver this service in this manner? So that's the that's the central conceit is the most important thing is is is modeling the domain. And so domain driven design is a series of kind of techniques and methodologies and ways of coding that try and bring that to the front. Having conversations with the business and trying to build what he calls a ubiquitous language. So, business people have jargon. If developers sit there and try and really understand that jargon   and understand nuances of it, get rid of ambiguities, and then they can decide on single words to define concepts in a certain context,   then you can have the code and the way that you talk to your business be the same. So that when you say,   sales order, you know, everyone understands that that is is an order from a customer, not an order from the company to China to get a   piece of,   furniture or something, for example.  



[00:16:18] Unknown:

There are some other approaches to software design and application architecture   that have gained some level of popularity. And I'm wondering if you can just briefly   outline some of the other common approaches and   why you think that domain driven design in particular is worthy of additional attention and,  

[00:16:38] Unknown:

exploration for people who are working in the Python space. Well, I guess what you'd contrast with it is sort of letting things evolve naturally over time. And and I think what I found is in my experience, you know, I've worked at some places where there isn't really a domain.   Like your Python application   is a very thin layer over a database, really. We're just letting people create read update records.   So is there a complex domain with a lot of business rules there? No. Or in that case, maybe you don't need to go to the full extent of building a complex model. Right. And so, I think the ways that this gets more relevant in the Python world is as we as our applications   start   to be more than just kind of data pipelines   or or, glorified shell scripts or you know, if you look at, Dropbox,   that was basically just building a UI around a file system on a server that wasn't in your house. Like, it's a web UI over a file system. So, you know, to a certain extent,   there wasn't much domain there for a while, although I'm sure as that's grown in complexity,   they've grown their own way of thinking and talking about these things and there is 1 now. I guess that's what I'm saying is it's something that really pays off when you have   a business domain of sufficient complexity   that   the, you know, the the conceit of domain driven design which is that the modeling the domain correctly is what will give you the most value is true. Perhaps it's a bit of a,   a self referential argument but   I think you can see what I'm driving at. And so domain driven design  

[00:18:03] Unknown:

is something that brings with it this idea of as you said, there is a business domain that you're trying to be able to model effectively,   but there might be situations where the project that you're working on is small enough scope or with a small enough number of people where you feel, oh, I can just go ahead and code something out in an afternoon. It'll do exactly what I want. It's never going to evolve beyond this. Is that just a logical fallacy and everything ultimately converges towards a big ball of mud? Or do you think that there are actually cases where domain driven design is,  

[00:18:33] Unknown:

just more effort than is absolutely necessary for a particular type of project? No. I I think absolutely. It it it can be, you know, more effort than you need to go to.   And, like, if the very fact, you know, if it's all about having conversations with domain experts so that that you're finding experts in the domain and having a conversation with between you and your, you know, your programming team. And if you sit there and are asking the question like who is the domain expert? Then, you know, if there's no domain expert and it's just you, then there's probably nothing to it. Similarly, I guess 1 of the, you know, you mentioned other architectural patterns. I suppose 1 of the things that that,   is kind of related to DDD is is microservices,   which is very much the kind of buzzword architectural pattern of the moment. I just came back from a conference called, MuConf   which is kind of short for microservices.  

And that's a it was actually the merger of a DDD conference and a microservices   conference. And the 2 have meshed together very well.   And that's because   1   1 of the main concepts of DDD is trying to identify bounded contexts.   So, if you have a big enough company,   there's not 1 domain model. The company has different capabilities. It needs to be able to,   I don't know, it needs to be able to invoice customers. It needs to be able to purchase inventory. It needs to be able to sell it to customers. It needs to be able to do after sales service.   And your conception of a customer   can be different depending on what context of the company you're talking about. And DDD has always, you know, said 1 classic mistake is to build a single   class to represent your customer and try and make it serve all of the use cases in the whole company. And pretty soon, your customer object is unbelievably   complicated and hard to maintain. And, you know, it has fields called slightly different things for different cases. And it's, you know, because it's used everywhere, it's hard to modify because you impact every single area of the company when you try and modify the customer object.  

And that in the Django world, it's the sort of giant   user profile   kind of anti pattern. And what DDD says is, listen, what you need to do is identify bounded contexts where you've got a sort of reasonably coherent   piece of business functionality, a reasonably coherent domain.   And what you're gonna do is model that domain and not the other 1. So like what is the user as far as,   sales orders are concerned? That's 1 thing. And then over here, what is the user as far as, I don't know, monthly billing is concerned? Then it might be a different concept. You know, 1 might have an address, the other 1 might not.   And and they would tend to say, let's let's isolate those concepts. And then the 2 should be reasonably independent. And if they ever need a d to talk to each other, we'll have a translation layer. You know, we'll have a way of translating the user as far as sales orders are concerned to the user as far as invoicing is concerned. But we want to keep them separate.  

And that can work in the context of a of a monolithic application, but it also obviously ties in very nicely with the idea of saying, actually, maybe my sales order service and my invoicing service, you know, parts of my application should be different services. They should be microservices.   And then you can get into splitting hairs about how tiny a microservice should be, but it is something that fits very well We're trying to talk about decomposing monoliths. We're trying to talk about, like, what makes a single coherent application. So application architecture in this sense of deciding what is an application, what should be multiple applications.  

[00:21:47] Unknown:

Hope that made sense. Yeah. That made good sense. I I definitely think that 1 of the   valuable pieces of DDD in general is this focus on understanding   what are the boundaries   of the different problem spaces that I'm trying to I'll just bolt on this other piece that isn't really part of the core concern   because I'll just bolt on this other piece that isn't really part of the core concern because it's fast for me to just get it out the door. And that's what starts leading people towards this big ball of mud because they just say, oh, I'll just bolt something on here. Oh, this already isn't quite exactly what the semantics   are sort of implying it's supposed to do, but, you know, I can just spend 5 minutes getting this working instead of a day and a half redesigning how this is going to work and, you know, future proofing against   some of the, accretion of technical debt that would happen otherwise. Yeah. Absolutely. I mean, we've always got this tension between,  

[00:22:42] Unknown:

you know, trying to anticipate future problems and, build a design that is going   to manage complexity for the future and make our lives easier and maintainable.   And just saying, you know, you ain't gonna need it. Alright. Yeah. Fine. Maybe if I was building some huge piece of enterprise software, then it would be really sensible to put all these layers in and build all this indirection. But I'm just trying to to build, you know, a social network for pet owners. I don't know. I'm just trying to build my little thing.   Is it not really worth it? Or do how do I know that it's now is the right time to start being rigorous and disciplined about architecture?  

I don't have any, magic answers to that. But what I do have is is some suggestions for,   okay, people in the world of DDD which is   often the world of Java   and C   have been doing this kind of modeling of   ideas about how you   build a model and the kinds of problems that you're always gonna encounter and some classic solutions to it. So these concepts like,   you know,   in your classes that represent your model, distinguishing between,   and there's some jargon here, I don't know how much detail we want to go into, but distinguishing between value objects that have no identity over time and are replaceable,   and entities, which are objects that represent something in the real world that has an entity that varies over time even though its attributes may change over time. And finally, aggregates, where we're saying, when I've got lots of classes in my model, I'm gonna put some individual models to be in charge of a bunch of others, and they are gonna be the only, way of modifying the state so that I have a single,   object or class that's in charge of my   consistency   rules, my invariance, my integrity rules.  

So promoting some of your models to being kind of public models in the same way as you have private and public methods. And, you know, you can you can hear that there's kind of language and and ways of thinking that have come out of the of the o 0 world, which, you know, we're all a bit suspicious of in the Python world. You know, when someone says to me, you know, private methods,   like the whole point of Python is that we're grown ups. Nothing is private. You can put an underscore in it if you want, but I, you know, reserve the right to look at that. But these things that, you know, all all exist for a reason. And once you have a certain level of complexity, these kind of patterns and tools for managing complexity,   really do come into their own. And the nice thing is is that when you do them in Python, you know, you don't have to sit there   writing public static void main   and all the all the kind of Java verbiage.  

You can have this kind of quite lightweight   way of of building abstractions and building a structure into your application that doesn't feel, as heavyweight as as some of the things you get into with interfaces and and whatnot in Java. And I think that this is a good point too to start digging into  

[00:25:21] Unknown:

the actual implementation details of some of these patterns. Because as you're mentioning, there are these ideas of interfaces   where I have this, sort of abstract base class that says that any class that inherits from it is going to have these types of methods.   So as long as I have something that implements the actual business logic behind them, then I could plug this in anywhere, which then feeds into the idea of dependency injection or inversion of control.   And so in the Java world, often means that there is some sort of dependency injection framework. And I have seen   various attempts to port those ideas and even very Java esque implementations into the Python community or the the Python landscape, which usually doesn't translate very well.  

And so I'm wondering   if you can just talk through some of the ways that you approach these ideas in Python that leverage the native capabilities of the language while still   maintaining the core benefits of the ideas without necessarily  

[00:26:17] Unknown:

catapulting somebody into Java. Yeah. Right. Well, so I guess it's maybe it's time then to talk a little bit about,   dependency injection and the the closely related and more important topic of of dependency inversion. So so 1 of the the patterns or rules of thumb, I tell you what, Bob Martin has been popularizing this recently so people who've read Clean Architecture which I hasten to add that I haven't. And maybe this is a good time to add a caveat that says, like, I am not an expert in this stuff. I have been learning about all these sorts of architectural patterns while I've been at Made, which is about 18 months now. But the experts are the is the well, probably, I'll just say is Bob, the lead architect here who came from the c sharp world and has seen all this stuff in his previous   careers. So, I'm I'm a little bit talking about things I don't fully understand. But with that caveat in mind, dear listener, the idea of of, dependency inversion is saying it's this rule that says that   our high level model should not depend on details. They should depend on abstractions.  

And actually, some of our popular frameworks like Django   do this for us already. They say, Okay. Well, listen. If I was building a model class to represent   my domain, I'm gonna build a class to represent orders and a class to represent customers and whatever, I know that at some point I'm gonna wanna save that to the database.   So the kind of naive way of doing that is to say, okay, well, let's have my model kind of import and do a bunch of SQL code and it will know how to save itself to the database by writing some raw SQL. And then my model depends on a detail because the specific   dialect of SQL in a specific database that I've chosen or even the fact that I've chosen to persist to a SQL database rather than a NoSQL database or to the file system is a detail. It's an infrastructure detail. It's an implementation detail. And you can build code where your   high level modules, your domain should be the highest level module,   depends on,   infrastructure details. And the dependency inversion principle says, well, let's let's flip that on its head. What we want to say that, you know, our model shouldn't depend on the database,   it should depend on an abstraction,   and then the database can depend   on the abstraction as well. And that's what Django does. It says, okay, well, your model doesn't know about the database directly. It knows about an abstraction   which is the djangomodels.modelclass.  

So we already kind of are used to doing this and it's about taking it to the next level really. And it's saying, okay, well, even you know, your model maybe depends on the Django models dot model class, it still depends on some sort of infrastructure. Yes. Your   your, framework, you know, has made it easy to swap Postgres for MySQL, but you've still got a dependency there. And so when you want to run all your tests, you have to instantiate some sort of database. And some of the architectural patterns we're talking about is saying, well, let's really invert that dependency all the way and say rather than having the model depend on the database, even if it's an abstraction of the database, let's have the model depend on nothing at all. And instead, we'll have the database look in at the model. So instead of like, you know, my model class is importing   django.db.models,   I'm gonna have some sort of database module import my domain model and say, okay, let me if I want to build my database, let's go and have a look at the models and we'll figure out which, you know, which tables will map to which columns   the other way around. I'm not sure if I've explained that very well,   Tobias. Maybe you've got some questions.  



[00:29:26] Unknown:

No. I think you did a good job of talking that through. And for anybody who wants to dig a bit deeper, we can provide some links to, maybe some visuals that will make it a little easier to grok, because something like this is definitely hard to encapsulate fully just through a podcast. And also, you know, we we haven't touched on this yet, but you are working on a book that is intended to cover a lot of these different topics. And in the process of preparing for this conversation, I was reading through bits of it, and I think that you did a very good job of talking through that specifically in terms of the Django ORM versus SQLAlchemy  

[00:30:00] Unknown:

and some of the ways that you can approach it. And so I'll add links to those specific pages as well for somebody who wants to dig a bit deeper. Great. Well, I mean, it's maybe that's an interesting thing to talk about because my last book was all about testing, and these architectural patterns can really help with testing. And so I think near the end of the book, I said, listen, how do you decide   how   to, what kind of test to write? You know, you're gonna have you could have some unit tests.   If you're doing Django, you have these sort of kind of unit tests, but maybe they're gonna use a real database of some kind or even an in memory SQLite database, but they're gonna be a little slower. We were just talking about how that can be a bit of a break on development. You can have integration tests where you're gonna check the boundaries between your systems, like, can I really talk to a database? Can I talk to a third party API? Whatever it'll be. And you might have end to end tests where, you know, if you   if you follow my book, you're gonna drive a browser with Selenium and use a real database. But, you know, even if it's just an API, you know, you're actually gonna spin up some containers and a thing that looks like your production system and test end to end whether, you know, everything is really wired together. And we started taking talking in the book about, okay. Well, how do we decide what kind of test to write and how many of which ones? And and we know that the end to end tests are the ones that give us the most reassurance about our system working, but they're also the slowest. So they give us the slowest   feedback and the the least information about whether our design is correct. And we know that the unit tests are the fastest and they give us the most feedback about, like, individual low level bits of design, like, is this piece of code well written? Is it easy to use? Is it easy to test? But on the other hand, you never know if you've pulled all your units together. And so, you know, you read around this stuff and everyone says, well, the test pyramid. Okay. The answer is you want a pyramid   such that you have a few kind of end to end tests on top. You have a sort of middle layer of integration y tests,   where you have a few more of those but not too many. And then you try and have the bulk of your   application tested through unit tests.  

And this always seems like like great advice. And then you go away and you build your Django app and you find that, you know, actually doing that in real life is quite hard.   And what I've found from some of the architectural patterns that are in use here at Made and that Bob made everyone,   have a crack at is that they really enable this this test pyramid by using these patterns where you can actually divorce your domain model completely from from the infrastructure,   by even maybe building a service layer to capture   some of those kind of controller   use cases,   orchestration types of pieces,   I can actually write unit tests for a vast amount of my application totally independently   of the storage system or the specific presentation system   the the specific boundaries at the edge of my,   of my application. And it's it's very similar to what Gary Bernhardt called functional core imperative shell of trying to have the boundaries of your system be as as thin as possible,   and you write them, you know, using whatever style you need.  

And you can test them with integration tests or maybe not at all. And then having the core of the application being functional, having no dependencies where you can test it with just data in and data out. And so so beyond just kind of these buzzwords, functional core imperative shell, hexagonal architecture,   test pyramid, like, actually seeing them in practice. Right. Oh, okay. If I build my application like this, it really can work. I really have can have a ratio   of, you know,   10 to 2 to 1, unit test to integration test to to end to end tests,   has been really eye opening. And that's that's something that I've been excited to to share with the world through, you know, the best way I know how, which is writing another book about it, like you're saying. And so at this point, I'm sure that there are at least some people who have decided that it's worth exploring  

[00:33:38] Unknown:

domain driven design and trying to incorporate it into their projects.   And so I'm wondering   what your advice is as far as next steps for actually getting involved in DDD,   and particularly   when they're trying to implement it on top of an already existing code base where they might need to refactor or rearchitect?  

[00:33:58] Unknown:

Yeah. Yeah. I don't have any lovely,   answers to the, like, how do I refactor   to get there from here?   So let me just try and treat treat the question separately. The first 1 is about like, oh, listen. I think well done, Harry. You've convinced me that this DDD thing is at least worth looking into. Where can I find out more?   So, yeah, by all means, there's my book.   At some point, we should be publishing an early release on Safari. So that will be public.   And then you can have a look at the sources   on GitHub. We can share that after the show. Sure. And that's a nice lightweight and Python based introduction. Then there's classic books on DDD. I refer them on my blog. That's beta testing goat dot com. So the last post has a a couple of links to books you can read. Now the classic books is massive.   So called blue book, which is Eric Evans, the kind of original proponent of these ideas, is 600 pages long. I've read about 60% of it.  

Everyone says it's incredibly dense. It's not incredibly dense. It's not impossible but, like, there's a lot to it.   And so that's that's the way to go if you wanna go in with both feet.   And then the other thing to do is is, you know,   start looking.   There are DDD conferences,   and start maybe watching a couple of conference talks. It's been so refreshing for me having been going to Python conferences   and a few   open source conferences   to go to a new kind of conference where people are talking about software architecture and design.   And they have all kinds of solutions to problems that we run into all the time that somehow we never seem to talk about   in the in the Python world or in the in the kind of Linux y world sometimes.  



[00:35:31] Unknown:

And so   once people do start going down the road of paying more attention to   the ideas   of application architecture   and maybe using domain driven design   in their projects, I'm wondering   in terms of your experience,   both as far as the projects that you've been involved with and also just talking to other people who are using similar approaches.   Who's responsible for actually implementing   and adhering to these different   architectural principles and designs? Is it just that you have somebody who's got the architect's title, and they are the ones who have to be responsible for all of that? Is it just the individual developer?   Just   what are some sort of best practices from an organizational perspective to keep everybody on the same page? Yeah. Well and I wanna, like, this is obviously something that's gonna be really,  

[00:36:22] Unknown:

specific to everyone's case. You know, like a   a a 3 person startup is gonna be very   different to a company like Made with a 100 developers. It's gonna be different to a bigger 1. At Made, we're lucky that that we have an architect who knows how to do these things, who came along, was very credible,   was a really lovely person, and was able to sort of show people,   you know, these patterns, why they work, why they're a good idea, try them in real   life, and then, you know, from experience, people are able to say, okay, this really works. Let's do it everywhere. So he was able to to bring things along and be a kind of mentor and coach.   So that's great. And if your company has an architect that already knows these things, you know, go ahead and listen to them and and see if they're convincing in your use case. But I think it's it's   you you know, it'd be quite dangerous to have a team in which there was just 1 developer who's obsessed with this stuff and was just gonna do it on their own. Everybody else is just gonna be baffled because   there is jargon here. And to a newcomer, it's confusing. Like, what is NNT? What is a value object? What is a domain service? These are all things that have meaning. And once you know them,   it's it's a it's a very good language for talking about code. But if you don't know them, it's just confusing and you're gonna ascribe meanings to things that aren't exactly what they are and you're gonna get very confused. So I would suggest it's something that you have to try and adopt as a team and and and find your resources for it wherever you can to try and try and do that altogether.  

But the good thing is once you do, it's very powerful because you do have this shared language. In the same way as you have a ubiquitous language for talking to the business about their requirements and how the application should grow, you have a shared language   to use with your colleagues about, you know, the application's architecture, what we how we describe the moving parts and where we put code. So these ideas like domain service or entity or aggregate will have meaning,   for everyone. And then if you suddenly start hiring a bunch of Java developers or whatever it is, they'll show up at the codebase.   They'll, be confused by Python for 3 or 4 days and then they'll get it because Python is so wonderful. And when they see things like entity aggregate   repository pattern,   they'll go, okay, yeah, I know these things because we've used them before.  

So that's a team dynamic that's valuable as well.   I hope that answers the question. Yeah. So if you have experts, listen to them and obviously  

[00:38:33] Unknown:

you've got to build some consensus around it and then the advantage of having a sort of shared jargon. Yeah. I definitely think that that's useful because, as you said, particularly in the case where you maybe have 1 developer who is very enthusiastic and says, I'm just gonna do it all, and then everybody else is gonna have to follow in my footsteps. It's just it's it's not gonna end out well. It's something that requires buy in from everybody and   is a collaborative effort where   you might have 1 person who's maybe driving the vision. Everybody needs to be onboard and interested and engaged with actually implementing those details and holding themselves and their teammates accountable for ensuring that they are adhering to the sort of agreed upon designs.  

[00:39:12] Unknown:

Yeah. Right. And so you were saying, how do I how do I you know, I've got an existing application. How do I get there from here? 1 classic way is, you know, splitting out a small part of your ModernaF and deciding to build it into some kind of microservice.   And that might be a really nice testing ground for saying, okay, rather than building this just the way we've built everything else, you know, maybe this is a place to go and use some of these architectural patterns.   And if it is if it truly is a microservice,   some of those architectural patterns might be overkill. But   you have a nice controlled design environment for practicing them and you can at least start to get a feeling for the code, how it all hangs together, and have conversations together so that when you, you know, next time you pick off a bigger piece, you've got some knowledge. So, you know,   it's the same way as I would tell people to try out TDD   is   pick a project   of a sort of like contained   experiment where you have, you know, timebox where you can say, well, we're gonna try it. We're gonna try it for this many months, not 3 weeks, but we're gonna try it for a few months, really get to learn it, and really see how it works in this organization. I think it's probably the same kind of approach would work well for for taking on DDD and taking on some new architectural patterns.  



[00:40:16] Unknown:

And in terms of just overall   trends in system design and application architecture   or just overall influences   from the technology   I'm   I'm wondering what are some of the ones that you are keeping an eye on that you're excited for and some of the ones that you think are potentially going to have maybe a negative impact on the ways that people design and implement their projects? Alright. 2 things, I guess, come to mind there is talking about event driven architectures and talking about,  

[00:40:51] Unknown:

serverless versus Kubernetes.   So the the the thing we haven't really spoken about today is is this idea of event driven   architectures.   And   1 of the things that happens at MADE is that you have microservices   or at least several different services.   And 1 of the patterns   that, you know, so once you've built your microservices, you've gotta decide how they will talk to each other. And the classic way is they all expose APIs and they can talk to each other over those APIs.   And the other 1 is to have kind of some kind of message bus,   and I've seen that be very successful here.  

So that's definitely something to look into if you're if you're working in microservices and you haven't or or you haven't fully decided how they're all gonna talk to each other,   investigate the idea of of a message bus of of asynchronous   event driven communication between your services,   because that that can work really well.   And then 1 of the talks I saw,   a couple of weeks ago,   sort of convinced me that there's there's there really is maybe something to this idea of serverless   where,   you know, even sitting around   having Kubernetes, which gives us this great abstraction and and way of orchestrating and managing containers,   may be still worrying about a lower level of detail   than we need to. And,   and so that's just on a personal level. It's made me slightly more open to the idea that that serverless,   whether it's AWS Lambda or whoever,   is   perhaps more than just a buzzword and the and the fad of the month, but maybe   maybe something quite   interesting about the way we're gonna build applications for the future.  

So, yeah. You know, you can tell I haven't really looked into that, but that is certainly something I'm intrigued by.  

[00:42:36] Unknown:

Yeah. Serverless in general is something that I was skeptical about for a while,   but I actually had the privilege of speaking with a gentleman,   who has founded and built the company, Data Coral, on a couple of recent episodes. And he has actually built his entire platform on serverless technology. So it's definitely interesting   hearing his experiences about the   ways that it has enabled him to approach a different way of building and deploying a SaaS application. So, I'll add links to the show notes for anybody who's interested in following up on that as well. But, yeah, it it it's definitely something that comes with a lot of hype, and I think a lot of people are   probably getting a bit overenthusiastic   about it. But there are some real world use cases where it can actually be beneficial  

[00:43:22] Unknown:

and enable some different ways of designing and delivering applications. Right. What's the classic Gartner hype cycle thing wherein the the the there's the the something of over inflated ex but the peak of inflated expectations and the trough of disillusionment. So maybe we've both been through the trough of disillusionment and we're now into the plateau of whatever it's called productivity,   plateau of blah blah   blah plateau of   delivering shareholder value.  

[00:43:58] Unknown:

In   in general in that field that you are keeping an eye on that we didn't discuss yet that you'd like to cover before we close out the show?  

[00:44:05] Unknown:

Yeah. No. I don't know. Just I'm interested to see how how,   well it it starts to penetrate the the Python world. So I'm talking about it, but I'm not the only person. There's   An Italian called Leonardo Giordani has written a book in English,   in perfectly good English, called The Clean Architecture in Python, which is talking about some of these application architecture patterns very closely related to what I'm talking about   in my book. There's a   bunch of crazy Russians have built a framework   called dry Python,   d r y.  

And they have got their own view and a series of abstractions and toolkits   for building,   you know, for for application architecture. And that again is kind of closely related to DDD. So it's, you know, like I when I set out on this book, I did think a little bit, you know, is is no 1 gonna care about this? Is maybe the Python world not doing the kinds of applications that need this? And it's been really nice actually building having these conversations,   meeting Python people at DDD Conferences,   talking to people at PyCon   who are expressing an interest. And I think, yes,   as a community, we're starting to see the level of maturity   in the applications that we're working on, that these kind of problems really are gonna start paying off. So it's, it's it's a nice place to be in. I'm really excited about it. And  

[00:45:22] Unknown:

so as you mentioned,   the Python community in general hasn't really latched onto a lot of these ideas until somewhat recently because   I think partly because of the fact that it is so easy to get up and running and get something functional and usable.   And because of the features   of the language and the sort of ease of onboarding,   it's become possible to   maybe extend some of these applications   beyond where if you were trying to implement them in a similar language, you would have run into the point where starts to crumble and fall over because of just the difficulty of   contorting your logic into these various shapes. And so I think that now we're starting to run up against the limitations of the Python language for being able to evolve to larger and more complicated architectures. And, also, the fact that Python is starting to make its way into some of these bigger organizations and larger applications. So things like Instagram is 1 of the,   1 of the examples that get that gets, put forward as well as Dropbox. And, you know, some of these businesses that started small with Python have grown to the point where they actually need to be thinking about these more complex,  

[00:46:31] Unknown:

architectures and design patterns design patterns. Absolutely. And Made is in that case. This is the company that was, you know, just a little a a PHP PHP shopping cart,   with a bunch of furniture on it,   and probably about 12 employees. And over the course of 7 years has grown to like 500 people and,   hundreds of thousands of orders a month. So   and still Python   for the whole of their everything after the shopping cart is all Python here. And so that's grown from, like, let's hack it together and do our best to to something much more, I guess, enterprise y.   And yeah, I think you're right that that some of the the flexibilities of Python has helped. And I think just as as the language grows in popularity, we're just gonna see more and more big things. So what I'm really excited to do is see if we can actually make these patterns   Pythonic.  

Because if you ever read a book on architecture patterns, all the examples are in Java or c plus plus or c sharp. And   if you haven't done Java or c plus plus since university or indeed ever, I mean, they're just they're just,   eye bleeding, you know, like, you just read them and you kind of lose interest after the 15th reserve keyword for saying public static void int type main,   all this stuff, which is, you know, yeah, fine. It's a nice way of structuring object oriented code. But when you're used to Python, it just says, here's a function, here's its arguments,   forget your type hints and what's, you know, and public and private. And, like, this is just the core of the code in the most terse way. You know, reading this Java code is is is tiring. And I wonder if there is similarly, like, 1 of my hopes I I haven't quite decided on the title of the book, but I was thinking of something like lightweight application   architecture patterns or Pythonic application architecture patterns because I'm keen to show that, you know, just because you're adopting kind of Java reenterprise   y application architecture patterns doesn't mean that your code base is suddenly gonna gonna look like Java. And so, I'm really keen, especially now while the book is still in draft, to get people's feedback and to see, you know, like what feels heavyweight, what feels lightweight, what feels Pythonic, what doesn't, and see if we can really, as a community, build kind of the Python way of doing these things. Because I think that's something we're so good at is is kind of building conventions,   and deciding,   like, altogether,   you know, this is the Python way that we do things. Like, by convention, no 1 is forcing you, but by convention, we just use underscores and not camel case.  

And similarly, I think we can maybe build some conventions   around application architecture, around some of these classic patterns, around maybe type hints as well, whether or not there's something that needs to come into it. Around you were talking about dependency injection. What's the most light weight, Pythonic way of doing it? Do we even need it at all?   I really would like to see sort of more and more people join that conversation. And I think I'm looking forward to the kinds of solutions we're gonna come up with together, not just me and Bob in our book or or, you know, or not just the first draft of what me and Bob have in our book. Yeah. And I'll definitely add links in the show notes to people who want to read through the book in its current state and provide feedback and  

[00:49:31] Unknown:

start experimenting with the ideas there. And I'm also in wondering if you know of any examples of open source or visible applications  

[00:49:40] Unknown:

that are using the DDD approach and some of these patterns that we've talked about that people can look at as a reference to try and learn from or get involved with? Yeah. No. I'm afraid I don't have a good answer for that beyond the beyond the 1 I've written in the book and then a couple of other blog posts I've linked to. I don't have a good example of an open source application that's using DDD in that way. I think I think because, you know, it tends to be   about   modeling a domain with business experts and so it tends to be towards the closed source  

[00:50:08] Unknown:

world. Alright. Well, for anybody who wants to follow along with you or get in touch, I'll have you add your preferred contact information to the show notes. And so with that, I'm going to move us into the picks. And this week, I'm going to choose a book that I started reading with my kids called Dragon Pearl.   It's from the Rick Riordan publishing imprint, and it's actually a   sci fi,   sort of space faring journey that incorporates some different elements of Korean folklore and mythology. So it's very interesting. I've been enjoying that so far. And so with that, I'll pass it to you, Harry. Do you have any picks this week?  

[00:50:42] Unknown:

Awesome. Picks. Alright. Well, listen. You know, I have kids as well. So as you probably know from experience, I don't have any spare time to read books   or things like that. But I do know in in my very limited time, I I'll mention 2 things. 1 is that for the second time I'm making my way through a TV show called Tremay,   which is the TV show that David Simon, the creator of The Wire, made just after The Wire.   And it's about the city of New Orleans.   New Orleans in the in the aftermath of, hurricane Katrina. So it's about a city that's been devastated by a national disaster that's trying to rebuild itself. And in my view,   it's it's got everything that made The Wire great and none of that kind of, but it's so much more,   because it's uplifting instead of being just kind of depressing. So in the same way as in The Wire, it's talking about, you know, neighborhoods and characters and and, people from different classes and ethnicities.  

And it's talking about problems with corruption and the police and the way that, like, kind of institutions and democracy and government is dysfunctional.   But it's it's it's also, you know, so they're very much like these these really good diagnoses of the social problems and and and political problems of America.   But also, it's set in this amazing town,   called New Orleans, where this the musical   the musical and the culture around music, and the culture of the town itself is so strong   that everyone kind of pulls together or or really just just helps lift the spirit of that whole TV show. And I've   never seen a TV show illustrate so well   the way that, you know, each individual is part of a culture and how that   influences them and their life choices, but how their choices also turn back and contribute back to that community and contribute back to the the culture around them and help build it. I,   you know, like, I I tear up at every show pretty much and and and my whole life since watching it has been been trying to figure out how I can move my family   and my, over to New Orleans and convince my wife that this is gonna be a good idea and that, you know, I promise you, even though it's America, we're not gonna get shot.  

But it's a it's a it's a you know, we're getting there. We're getting there. She loves jazz too. So, you know, we've got a chance. The other thing I want oh, so you mentioned a book. Yeah. I'm reading a book called Why We Sleep, which has been absolutely fascinated.   I saw it recommended by Rafael Pianazzo, who's 1 of the contributors on the Pytest framework.   He was really enthusing about it, so much so that I I just thought, you know, no 1 can be this enthusiastic about a book without it being good. So I bought it, and I've not been disappointed. It's an amazing,   nonfiction,   written by a sleep scientist   about, you know, what sleep is, the our best understanding of of how it works and what it does.  

And, you know, I'm only sort of 3 or 4 chapters in, and I'm already becoming a sleep bore and and kind of going on about it to my friends and telling them the importance of sleep. But, you know, the the short gist of it is that there is it looks like a massive amount of of the modern world's population is chronically sleep deprived. Like, we are not getting enough sleep. We're, you know, we should be getting about 8 hours a night. And ideally, we should be getting 7 or 8 hours a night and a 20 a minute to an hour long nap in the middle of the day, and none of us are doing that. So as a result, we are sleep deprived. And being sleep deprived   affects you negatively   in pre in every single health   metric that you can think of. It's bad for your heart. It gives you cancer. It shortens your lifespan. It reduces your intelligence. It reduces your motivation.  

It reduces your ability to build relationships, to be empathetic, to control your emotions,   to enjoy   life. You know, it reduces your physical and psychological   health,   and that's very measurable. And so fascinating book, like an absolute polemic and a call to arms and saying, and I just make sure that we understand sleep is important. Let's get more of it, and let's try and understand it better. So, yeah, Why We Sleep.  

[00:54:29] Unknown:

I can't look up the author just now, but, fascinating stuff. Alright. It's definitely something I'll have to take a look at. Well, thank you very much for taking the time today to join me and discuss your experiences   of   working with domain driven design and building with it and, helping us learn more about why we might wanna consider incorporating it into our own applications.  

[00:54:49] Unknown:

So thank you for all of that, and I hope you enjoy the rest of your day. Such a pleasure being here. I I feel flattered to be on the podcast. Thank you very much to I hope this starts some conversations.   I hope to hear from some of your listeners   and, wishing you all the very best. Lovely day, lovely week, lovely month, lovely rest of the year, happy holidays. It's gonna be a beautiful summer. All this sort of stuff. Yeah. Yeah. Absolutely. Bye to buy it.  

[00:55:12] Unknown:

Alright. Thank you.