Michael Foord On Testing, Mock, TDD, And The Python Community

Hello, and welcome to podcast dot in it, the podcast about Python and the people who make it great.

When you're ready to launch your next app, you'll need somewhere to deploy it, so check out Linode. With private networking, shared block storage, node balancers, and a 200 gigabit network, all controlled by a brand new API, you've got everything you need to scale.

Go to podcastinit.com/linode

to get a $20 credit and launch a new server in under a minute. And visit the site at podcastinit.com

to subscribe to the show, sign up for the newsletter, and read the show notes.

Your host as usual is Tobias Macy. And today I'm interviewing Michael Ford mockingly about his career in Python. So, Michael, could you start by introducing yourself?

Hello, everyone. Yeah. I'm used to being mocked particularly from my career

in in Python. I'm sorry. I have a very strange voice today. I'm on the the tail end of a cold and sore throat, so I I sound quite different from normal. But, yeah, I'll I'll introduce myself. My name is Michael Ford. I've been developing with Python since about 2002 and commercially since 2006.

I'm probably best known back in the distant

past with from my work with IronPython.

I worked for a small firm in London, and we were the first to commercially use IronPython, so I ended up writing a book called IronPython in Action.

And then whilst there,

I at the same company, I became passionate about testing and wrote a tool called mock, which is now in the Python standard library as unit test dot mock, and I became a Python core developer. I maintained

unit test for quite a long time. I did a whole bunch of features. And, and I've had a whole other bunch of other adventures

with Python since as well. And do you remember how you first came across Python? So it was sheer serendipity,

really. At the time so this is back in sort of 2001,

2002, that kind of era. I was actually working in a builder's merchant selling bricks and timbers for a living. I was in Heavyside, Sayers. But I started this play by email game where everyone emails their orders in every week and, a computer processes the stall and sends you back reports of what's happened, your adventures and battles and things you've discovered. So the really interesting bit is the sort of diplomacy that happens by email in between. This is this is back in the sort of early ish days of the Internet. And a bunch of us, we had this alliance, and it was us against this other 1 big character. And we'd,

some of us said, hey, wouldn't it be really interesting? These reports are generated by a computer. The orders are,

a very simple syntax for a computer to understand. We could write some some software, some little bots

that could, that could automate all of this and maybe even fight each other. And, and we'd set up on for some reason, we sat on small talk as a language. And at the very last minute, someone said, hey, You should use Python for this. And we said, alright. And, somebody else wrote the parser,

regular expression parser that, understood the reports. And then on top of the data structure that that spit out, I did an implementation of, Dijkstra's

algorithm for path finding. I did

maps and,

all sorts of interesting stuff with them. And Python and programming became my passion again. I mean, not done any for for about a decade. I'm very bit before university. A bit of 68, 000 assembler on the Amiga, BBC Basic

on the 8 bit BBC

microcomputer. It was my first programming language.

But I just I just became passionate about about Python and,

got involved with the community because it was open source. I could start contributing, writing articles,

ended up writing some documentation for URL lib 2,

which got into the, the the Python documentation because the documentation at that time was pretty cool.

So that was my path to sort of discovering Python discovering the Python community, discovering open source, and starting to become passionate about this. And, and I

was lucky enough to manage to turn that into a career a few years later. And about how long was it from when you first started using Python until you decided to get involved as a core maintainer?

Well, so as a core maintainer, that happened,

a little bit after I'd started my career as a professional software engineer. I got this job with Resolver Systems in London, and we were building,

a spreadsheet application, a Windows desktop application

that had programmability

built into it right at the at the very core.

So on the IronPython was initially going to just be embedded as a calculation engine for the spreadsheet. But because,

the guy who ran the company, Giles Thomas, was a and the other people in the company were big fans of test driven development, They discovered that actually, a home Python is a dynamic language. It's way easier to test than,

than the static, like, the type languages that they were used to.

So, they wrote the whole application in Python. And when I joined, I was the first,

1 on the team with any Python experience.

And also,

they were in my interview, they were the first people I'd actually met programming Python before my, experiences

interaction with Python programs before then had all been online.

And after having been with them a couple of years, I went to

the, to PyCon in the US. There, I've met all of the developers. I'd already done stuff like contributing documentation,

to the to the Python project. But the,

we were using IronPython a lot. And,

and the issue with that,

we had with that was that there were lots of fixes to the standard library that we wanted to go into

Python because, there were lots of fixes that we wanted as a company to go into the Python standard library for iron Python compatibility.

But Microsoft at the time hadn't sorted out licensing agreements for their developers who were working on iron Python to be able to contribute changes

to the, to the Python standard library. So I was

kind of given commit rights in proxy for, as a representative

for IronPython even though I was just working with it, wasn't working on it. Because I as an individual,

I could make the changes. I could propose them. I could get them committed and which the the Microsoft guys couldn't. So that was how I first became,

a core contributor. But this was round about the time of the Python 2 to Python 3 transition. So my first real experience

of working with the Python core team was in the the development sprints after the pie PyCon. And I worked with, Trent Nelson in particular, the tokenized module. Trent Nelson had made this bet with,

with what the guy who maintained the, the Python build bots. The that he could get them all green on Python 3 during the sprint, which was a ridiculous claim. We got a whole beer on it. And then we ended up spending most of the sprint just working on the tokenized module,

because the in order to maintain API compatibility and

the fact that we were working with files opened as strings that we needed to open for text reading, that we needed to treat as bytes, we had great fun with it. So we did get tokenized done, but we didn't get, all of the build bots green.

And so that that was the story of me becoming a core contributor. I I was passionate about testing.

I'd started writing mock for our needs at,

Resolver,

by then. So I was, working with the community on the testing in Python mailing list on building,

a mock object library that would be really useful for Python.

And I I started to make some much needed changes in unit tests, adding test discovery,

updating,

and modernizing a lot of the assert methods and and some of the infrastructure around it. So that was my journey into becoming a Python core developer. Again, a lot of serendipity,

knowing some of the core developers already,

meeting them at the the

the the Python sprints. And as you've mentioned in passing a few times, 1 of the main focuses of your career has ended up being around testing and

usability,

and maintainability of tests.

And it's interesting how people can end up sort of focusing on some of these subproblems within software engineering. So what is it about that aspect of development that you have found so

interesting and compelling,

and how did you first get exposed to that aspect of building software?

Well, everyone ends up specializing to some extent.

Like every part of the world, the the world of software engineering and computers is is fractal.

As you zoom in on it, you sort of reveal computers, the various different character sets,

internally in computers

the various different character sets that are available and communicate those around in an unambiguous ways. But as soon as you get into the sort of details and the

sub aspects of Unicode,

right to left combining characters there,

it's just it's just an it's just extraordinary that there there's a very basic minimum level that every

developer should know, and Joel Swanson wrote a very good article on that, which is worth digging out. But it's it's an extraordinary world of its own, and every aspect of software engineering is like that. My particular passion for for testing, again, came out of those very early days in my first,

my first job as a professional programmer. I taught myself Python, and I taught myself programming

to a pretty good standard as a as a as a self taught programmer. I'm pretty proud of how well I'd understood programming, particularly,

object oriented programming, which I've never come across before. But it was really working on Resolver systems, working as part of a team, building real things, and and working together on the workflows around that, that I learned the the craft of engineering and how to be a software engineering, how to build big things, how to build things that are stable and robust,

and that are easy to change. And I think that ability to change, ability to understand, to maintain, and to change things

is is vital,

in order to have a maintainable system.

And for me, it was real we we followed fairly strict test driven development, And it was really clear to me that,

first of all, you couldn't fix and find all of the bugs. But there were a lot of bugs you could fix and find

with testing.

And what's more, once you've fixed a bug, if you've got a test for it, you can be certain that that bug doesn't come back. It was it's always been my experience

writing software that, you know, you sometimes end up fixing the same bug several times. And as soon as you write automated tests for those bugs, you can make sure that you don't bring them back. And the same with with a comprehensive test suite. You can't guarantee your product works, but you can guarantee that it has a certain a minimum level of quality. There's there is a threshold of quality that you can verify and assert with testing. And and it seemed to me that being able to do that, being able to know that the changes I just made to my software have added the functionality

I want them to,

and I haven't broken to the best of my knowledge existing functionality.

That seemed clearly

the right way to do software engineering. And also the approach of test driven development,

which whilst people couldn't get very purist about it, it can become a religious argument. But the approach that I think about testability,

which requires me to decompose my problem, into parts that I can reason about, think about,

that I can test in isolation, that aren't tightly coupled to each other. It forces me to think about design.

That approach to to design,

and to designing APIs, you you mentioned that I care about usability.

My my guiding principle,

particularly for API design, is that simple things should be simple,

but complex things should be possible. So often you get nice and simple APIs wrapping some other functionality that just don't let you do the more complex stuff. You have to go right down to the low level yourself.

All you see really heavily engineered APIs that expose everything. But the happy parts, the straightforward parts, not to be a single call with a single parameter is like 4 calls, and 1 of them has 6 parameters.

So I think there are very

there are nice principles you can take to to API design, to project

design, from thinking from the perspective of an of an outside user. And it so happens that in particular, the functional, the automated, the end to end testing, which is so lacking in many places, but which is so powerful in,

verifying product quality,

that requires you to think about the product from the outside. It requires you to think about the product as a user. And so you start to see usability issues in order to use this particular software that I'm testing. What's the minimum number of actions I have to perform

before I can achieve a simple task?

And quite often, products will require you to completely buy into the way they model the problem domain and set up sort of 8 or 9 different

conceptual things before you can even achieve a simple task. There ought to be a 1 button operation. So it's that sort of mindset is a useful way of thinking about product design, about API design, about feature quality, and it's also the way,

the the testing mindset function, the automated testing mindset. So all of these aspects of software engineering that I care about,

about quality, about having processes that that work for people as well as machines,

All of these things come together in in the way we test software.

So, yeah, that that is is my my passion. It's been my

focus through throughout my career, and and in particular, writing in the mock library

was about

creating a tool that didn't really exist at the time,

enabler

is to do effective

unit testing

whilst keeping components

isolated and being able to use mock objects.

It can also be terribly abused

but, but so can everyone. Yeah. That's definitely something worth talking a bit more about. But before we get there,

as you mentioned, when you first began working with Python, there didn't exist any tools such as mock.

And over the intervening years, there has been a lot of development of additional

frameworks

and techniques

available for doing these automated tests, both at the unit level and the integration level,

and end to end testing. And also, somewhat recently,

there's been a lot of popularity in in the area of quick check style tests,

such as what hypothesis

provides, where you just automatically generate random inputs to see what's going to break the API that you've built. So what have you seen

as being some of the evolution of the language and the surrounding ecosystem in terms of support for testing

and how that has developed over the course of your career? That's been a really interesting,

journey to watch. At the start of my career back in 2006,

we did full end to end end to end functional testing. We did full unit testing,

testing excessively and learning, sort of learning to find the balance. I mean, we had 3 times as much test code as production code, which for full test room development isn't that unusual, but we learned all the perils of tests that are very closely tied to the implementation so then become a nightmare to maintain as you refactor

been through all of that journey. But back then, that was very

unusual. And and through that period, we've seen the transition of, particularly the the more enterprise companies moving away from manual testing of products, moving away from no unit testing. So where unit testing is basically normal and where,

even where there are separate QA teams,

most companies have some

automated testing,

and, and a lot of there are still a lot of companies making that transition. I've helped a couple of companies

do that. I work when I while I work for Canonical,

we had a manual, testing QA team, and I actually wrote a a simple testing framework over the top of

unit test and selenium,

allowing,

QA teams to using very simple

scripts to automate

testing. And then more recently, I worked for Red Hat Ansible, and they're in the pros process

well along the way of moving the testing of their Ansible Tower, their enterprise product, enterprise web app from being

manually tested to to being automated testing. And and this is in 2, 000 and to using automated testing, and this is in 2018.

So the the really big change that I've seen is not so much the testing techniques and the testing styles

to a to a certain extent the tools, but really it's the infrastructure.

We now have this fairly well understood idea of a CICD pipeline, the continuous integration, continuous deployment. Lots of people tell me that continuous integration isn't suitable for,

shrink-wrap, desktop software, software on the release cycle rather than web apps. And well, we did continuous integration for a desktop product back in 2006, so I'm not terribly,

sympathetic.

So that is an argument on its own. The the big problem with waterfall

is that it's it is not that continuous integration, these techniques are not suitable. It's the waterfall is terribly hard to change.

Very strongly, waterfall process is very difficult to to change. And that, as far as I'm concerned, is the biggest demonstration of why they're such a bad idea. If your development process is very hard to change,

then you can't respond quickly

to changes.

And therefore,

a development process that was able to change, that was able to react more quickly, that was able to change priorities more quickly without burning development time might be a better idea, and that's the that's what's at the heart of that

job.

But alongside so alongside these developments of of standard concepts

of how we test and and why we test. We have the the all of the tooling, the equipment for that. We we have,

GitHub and and most of the the popular GitLab. You can gate check ins on test runs. That's very easy to integrate with tools like Jenkins.

We have Selenium, which is nice, which is very mature. We have a whole crop of new Java test

script based functional testing tools like NightWatch,

which are very powerful. So we have this whole ecosystem and infrastructure

around normal ways

to integrate testing into part of our standard

development workflows.

And a huge number of developers know how to use those, know how to do that. Pilot test is is very widely used. Even unit test with with test discovery,

it's still it's very easy to build build out the infrastructure

and the code to have these test suites. And that's a huge change, over the early days when,

when people were reinventing Jenkins for themselves. We kind of did that, built our own distributed test runner system.

It was,

code coverage tools now are much more standard than linters.

The the whole ecosystem and infra an environment around testing,

in Python but in programming in general has grown vastly

since those earlier days.

And

given the

improvements

in the tooling and the visibility of testing and the broader acceptance,

what are some of the areas that you see as still being lacking, whether

as far as how people approach the testing that they perform or the utilities that are available for building and executing the tests? So interestingly

at at Ansible, 1 of the things we we we hit the the same problem

that everyone does when you do end to end testing. It's nice and straightforward to build up Jenkins jobs that build your system, deploy your system, whether it be to cloud or to your own infrastructure,

and then run your tests and then collate and and report those tests. That's that's nice and easy and straightforward to to put in place. The difficulty

is that where you're,

manipulating your product or orchestrating

your product, your your system under test from the outside,

the test starts to take quite a long time. And a a single test testing a relatively

feature might take a minute or 2 minutes or 5 minutes. And,

because you really ideally, you want test isolation. So, in theory, ideally, you want a a blank system every time,

and you want to build up the state through the actions in the same way the user would. Except that if that takes 5 or 10 minutes per test and got a 100 tests or a few 100 tests, your test run pretty soon start taking hours. There are various

techniques and approaches to solving those various different

trade off and balances around,

not over testing,

only testing the specific features you want, not allowing tests to interfere with each other. But 1 of the simplest ways is just parallelize your tests. If we're running against the web app, if the if the bottleneck is actually our test runner and,

the the the network

request that we're making, then we could just run a bunch of these and combine the results. And and it and it seemed that Pytest has various

distributed test runs, but none of them seem to do the fairly straightforward thing we needed to do. So we we had to write a write our own. I think, really, the the,

some of these best practices,

some of the the more common answers to these trade offs and the techniques for for solving these problems of how do we do large scale functional testing

of systems

and platforms,

but not have it take

a great deal of time. How do we speed that up, get the results back quickly?

I don't think that's a problem that very I I I think there are some better solutions

to more cut there are some more common sort of patterns waiting to emerge

that we could that can, that can help answer those questions. It's a bit more it's still a bit of a case of biting in the trenches

with every different situation

for for answering those kind of questions. That that's

that's how I see it at the moment anyway. And

as you were discussing some of the challenges of these long running tests and the fact that they have expensive setup routines,

and then wanting to be able to parallelize. 1 thing that I was wondering about is the feasibility,

and I'm sure this varies with the language run time that you're using or the framework, but of being able to

essentially checkpoint

the memory

or the,

state of the data in the system at various branch points within the tests, so that you can have

common setup routines

that would then allow you to

spawn off,

sub tests based on the location in the overall flow that you're at, so that you can then sort of reduce duplicated effort that's necessary in terms of running the entire test suite. Yeah. That's a good thing. The, a common approach to that is to take so typically typically in a normal web app, not exclusively, but a lot of the state you want to build up in order to be able to test any particular feature or functionality

is really database

state. Is what what is the the system state? And and usually system state is modeled in the database, maybe message queues as well. So database fixtures are 1 way of checkpointing and freezing the system

and say, we can we have, a whole bunch of these tests that test the

these the smaller individual features like login, like creating users,

like setting permissions.

And we don't need to test these every time. So we can checkpoint

the database state of a system that's done and a whole bunch of standard operations that have these. We can preload these fixtures,

and then we can just run tests that test the the the stuff that we haven't tested, the new and interesting stuff. There's the the trouble then you run into is that your tests then become very coupled to your your specific data modeling. You have, perhaps, SQL dumps dumps are very specific that are very, very specific to the current model. So whenever you do a migration, whenever you update the model, whenever your system design changes, a whole bunch of tests become invalid. Now there are ways of applying migrations to this data,

but it's something that you have to to manage because the danger is actually that the test will carry on passing, but they're actually now testing against invalid system states. So there's a whole bunch of trade offs and things to check there when when taking that approach. The other way is to just accept that

we have a whole bunch of tests that are dependent on each other because you can't put a car in the garage without having opened the garage doors first. So we're going to write a test that checks the garage doors open, and then all of our tests that put cars in and out can just rely on that test. So you have 1 test that fails.

Implicitly, a whole bunch of tests fail, but, actually, that's the reality. If opening the garage doors fails, then all of those other tests are going to fail. So that's that's another alternative approach to to solve it. So

the the same difficulty, but that breaks test isolation.

Normally, you end up with some compromise

of of of various approaches for this. Larger tests that actually test a whole bunch of stuff in 1 go are, again, another approach. Large tests,

are bad because they're harder to understand. Ideally, 1 test should test 1 thing. But where things it

it this issue seems to be all about finding the the least bad way through a bunch of difficult and bad choices.

That that sounds like a a a good description of software engineering in general. Right. Right. I think that's very true. And

1 of the other options for being able to build up system state or at least approximate it is through the use of mocks. And

there are a number of different sort of philosophies in terms of how mocking can and should be done, and the patterns for creating mocks,

and how the mocks are used. And so 1 of the most widely used implementations

of that pattern is actually the library that you wrote, and was subsequently incorporated into the standard library.

So I'm wondering if you can talk a bit about your reason for creating it in the first place

and some of the potential pitfalls of using mocks within your tests?

So I guess I'll start with the caveat that,

1 way of looking at mock objects is to say that every time you need to use a mock,

say there's a bit of a slight personal failure

that this part of the system can't be tested in isolation without a mock object and minimize the number of mocks you use, and that's a good principle. But the the,

the basic theory

behind mocks is that,

your unit testing your unit testing is the implementation of behaviors. You want to test the behavior

of, some particular part of your system under test. You want to be able to in order to be able to test just that behavior, you want as little as the rest of the system under test to be,

to to be alive. You want to have to instantiate and put in place as little of the system under test in order to be able to just test the specific

functionality and behavior that you're testing.

But most components of your code don't work completely in isolation.

They're connected to the other bits. That's the the their reason for existing. So at the boundaries,

it's useful to be able to objects that will have the same behavior as parts of your system under test.

So they can be called, they can return data and values, but you can configure them and control their behavior

and afterwards interrogate them about how they've been used. And they aren't actually live bits of your system in under test, so you can avoid having to make network calls,

file system operations,

touching the database, all things that you shouldn't be doing inside a unit test that take time and that require big chunks of the rest of the system under test to be active. So having mock objects is very useful.

Back in 2006,

the the standard reference from mocks was, a work by Martin Fowler

on mock objects. And he defined a mock object itself to be an object that used the record and replay style of mocking. That was his definition of the the terminology.

And that's where you configure in advance

how you expect your mocks to be used. You call your code, and your mock either does nothing, in which case it works or it blows up because it was used in some way you didn't expect it to. And that was a that that was a great idea. It was revolutionary for its time. But but but, actually,

it means that every time you use a mock object, you have to specify all the ways it's going to be used upfront.

So it puts your tests a bit back to front. It puts all your asserts about what will happen

ahead of the actions that cause them. And it also means even if you're only interested in 1 particular piece of behavior, you have to specify all of the behavior, so it makes the test harder to read. And at the time in Resolver systems, we were creating custom mock objects for each of our tests. So we had a custom we had a mock cell. We had a mock worksheet.

We had a mock grid and and so on. The advantage of writing them all ourselves was we very tightly specified

the exact behavior of these mock objects. They only did exactly what we told them to do, so we knew they couldn't be used in inappropriate ways. But we wrote all of them by hand, and we had hundreds of these things. And and I thought Python is such a dynamic language

through the attribute access protocols. I've done the getattr, done the setattr,

and friends and the descriptor protocols.

We can completely

control the behavior of any object. So we can make 1 object that

is capable of mimicking any object, and it can record for itself how it's being used. So instead of having to configure it all up front about how it's going to be used,

afterwards, we can just make we can have some helpful assert methods on it, check that I've been called with this argument or these arguments or have been called twice, and it could be a nice simple object. And and the first implementation of mock was, like, 30 lines of code that replaced, I mean, literally thousands of lines of code in the resolver code base. And then the the the second problem that we wanted to solve was that, Python allows you to because it's, it's a dynamic language, we can dynamically change and modify

attributes and parts of the system under test at runtime.

And things like, for example, if we want to be able to mock out

the time, we want to be able to control the time,

that the system sees

so that, we can test that some time related behavior works. It's nice to be able to replace time dot time with, with our own custom function. And in Python, monkey patching is easy easy. Putting a new object

into a system in test, injecting it there is really easy. You can just do it as an attribute access. What's really hard is putting the system back the way it was. Because when you fetch an attribute off an object, you don't know,

did it come from the object instance?

Did it come from the class? Did it come from 1 of the multiple classes

that it inherits from? Was it fetched dynamically?

If the object is a type, it might have come from the inheritance hierarchy or the metaclass.

It might have been fetched as a property. So if you replace a method by setting an instance attribute, the way you restore the instance method is just delete the attribute you set and that re exposes

the the method. So monkey patching is easy, but un monkey patching is really hard.

So I wrote a a patch decorator, and then my colleague, Tom Wright, worked out, well, look,

the context manager API is entirely compatible with the way decorators work. So we can have patch can be a decorator

and or a context manager, and then you can be really flexible with how you use it. And what patch does is it ejects an object into your system

for a very specified scope, either during a method call or during,

the just the scope of a with block. And then when the when that scope is executed, whether the test succeeds or passed, it puts the system back how it was.

And and so this ability to to monkey patch the system under the test really straightforwardly

and to have a mock object that could pretend to be any other object

and could be configured and also interrogated how it was used just proved to be a really powerful combination, and

and gradually, it took over the world. There were a whole bunch of other mock libraries, most of them based on the record and replay style,

and it just turned out most people used mock. And in the end, it it became stable

and widely enough used

Yeah. I can remember

when I first came across the concept of mocking and was trying to leverage it in the Python eco system

being very stymied by that record and replay

paradigm

until I came across your work with the sort of canonical or eponymous

mock library, and that simplified all of the efforts along those lines. So, I definitely appreciate your work on that.

And 1 of the things that is

potentially hazardous when you are using mocks

is that,

situation that you described where you have tests that will silently pass when they shouldn't be because of the fact that maybe the mock is responding to a method call that no longer exists. And I know that 1 of the ways around that is

the introduction of the magic mock object that you created that will actually

copy all of the attributes and method signatures

from the object that you were trying to substitute for. So

what are some of the ways that you can work around some of the potential pitfalls of adding in these mock objects while still being able to easily create tests and,

prevent some of these

erroneous passing cases.

Yeah. You're correct, Aras. That is 1 of the greatest

dangers with, the mock objects is that when you replace part of your system under test with a mock object that can be called in any way, you can then have, as you as you say, test that pass, but your code is actually being used incorrectly, and the mock object is just letting it happen.

That's not actually the the job of magic mock. The difference between the mock and the magic mock is that the magic mock supports

all of the or almost all of the Python protocol methods, anything else that the Python protocols allow you to do. And getting that support, anything else that the Python protocols allow you to do. And getting that support,

it needed to be a separate type because there are some checks, like the check for call ability, that work by checking for existence of the done to call method. So you either need a mock that's callable or a mock that's not callable. And the same with the existence of some of the other protocol methods like under getitem, under setitem. So the magic mock is a separate type. There's a really lovely trick at the heart of magic mock, which I I really like. So I'm gonna

share it with you. Now, normally so if you create a mock object,

and, from a magic mock, you instantiate magic mock, You can set as an instrument as an instance attribute,

your mock dot under get item, and you can set that to either be a function with a signature compatible with, the get item or you can set it to be another mock up object. Now done to get item and done to set item, they're only ever actually looked up on the type. That's how Python works. If you subscribe an object as if it was a dictionary,

done the guess item is looked up on the object clerk on the object type.

Now,

so the first thing that the mock does is when you set up 1 of these magic methods on a a mock instance is it promotes it to the type for you. Now normally, that would mean that all mock objects would now be sharing the same protocol methods because you've set the type. And obviously, we can't have,

mock objects interfering with each other like that. So So whenever you instantiate

a magic mock in the

new method,

Python has this 2 phase object creation.

New creates the instance

and dunder init initializes the,

the freshly created instance. So it's very common to override dunder init, the initialization phase, but the instance has already been created at that point, and that's done in object dot under

new. So in mock,

dot under new,

every time you instantiate,

I think this is for for a default mock as well, not just for magic mock. It creates a new subclass of mock

and returns you an instance of that subclass. So now we can attach whatever we want to the type without it interfering with the other types. So with magic mocks, you can you can set

the magic methods, the protocol methods to be whatever you want or or you can have magic mock

create, mock objects there for you, which is what what it does without it interfering with any other magic mock

without any other magic mock instances.

And those magic methods live on the on the type. Now the auto specking is slightly different.

This what what this does is is say, I I have this mock object, but I want it I need it to pretend to be this other part of my API.

And there, the the auto spec function,

create auto spec function, which you can also trigger from patch the patch decorator just say auto spec equals true, is it will look at the object being replaced,

and it will create an a mock object that only has the same API, and it will even copy function signatures for you.

So then your mock up your the mocks you create can only be validly called in exactly the same way as your, system under test is called.

So it prevents those kind of kinds of bugs. It's not on by default because in order to do this, it has to do some object traversal,

which triggers property access and descriptors.

We can't really get around that because in order to know the API of what an attribute is, we have to fetch that attribute, and fetching the descriptor itself would do no good. We need to know what the descriptor returns. So it's an optional feature, but it does avoid a whole class of bugs.

And

in testing,

there is also the concept of stubs and fixed years, each of which have slightly different purposes within testing Right. In addition to mocks. So

what are your current

best practices

or heuristics for determining

when to use each of those types of substitution method for creating and building your tests? Well, that's a very interesting question. And actually, part of my

part of how mock evolved is a bit of a reaction

against

Martin Fowler's use of all of that top terminology

of stubs and mocks, and

all of that article, which

did the testing world a great amount of good, but I but I do

slightly

denigrating perhaps refer to it as the static typing of mocks because,

actually, mock pretty much does all of them.

And so I don't really need to think about which type of mock I'm using. Fixtures are slightly different. Fixtures are a way of sort of putting part of your system in, in place

to test

fixture is a way of putting part of your system under test in place, whether that be data or state or what,

the actual system itself and a way of sort of passing that to test. And they're very useful,

very, very useful concepts. There are other libraries for unit tests that do this, and then they're a standard feature of pytest.

You can get lost in fixture hell. I've been there as well.

So they can be overused.

But, yeah, they're they're very powerful testing concepts and very important.

You did mention earlier sort of some common anti patterns with mock. And I think 1 of the dangers with mock and patch is that,

so the ideal for code is for

the components of your code, that their

behavior to be tested completely in isolation

totally through the public API and without needing big parts of the rest of the system under test to be in place. That's the ideal.

A lot of people are working on legacy systems with no tests at all or with very difficult tests or they're writing tests after the code. So

there's a lot of coupling and a lot of difficulty,

to work around. And and the danger with patches, it does allow you to just sort of put in place a huge chunk of your system and then just sort of patch out various bits of it,

and then run some of your tests. And you're not really there writing unit tests. You're sort of writing integration tests,

And you're very tightly coupled to the implementation.

As you change and refactor your implementation,

those tests are gonna start failing or having weird behavior.

And, and like I said, if you start looking at as every mock that you need to put in place as being a bit of a failing of the code,

then you can very you can very easily see where you're having to to over mock, where you're,

really, you just want mocking to just be at the boundaries of the component itself. If you're starting to patch deep into other places in the system in order to be able to control behavior, it's a sign that you've got too much of this this coupling. You may not that that may be technical depth that you may not be able to,

address immediately, but it is a a warning sign and an an anti pattern when using mock objects.

And over the course of your career, you've worked in a number of languages

outside of the Python ecosystem

and built testing capabilities,

or test suites within some of these various different projects. So as somebody who has had this experience

and,

this view

and dedication to testing

over a number of years and a number of languages,

what is your perspective

on how the state of the art for testing in Python compares to some of the other languages and frameworks

and,

platforms

that other people are using? So my long my longest stint

of working and testing in another language was working with Go.

Go is a very interesting language, much smaller language than Python,

very different type of language, but a lot of similarities.

And, really, that that whole experience,

made me very grateful for how easy Python is to test as a dynamically typed language.

Go has a lot going for it, particularly the use of the way you can use interfaces for testability,

make it more pleasant than other languages.

C sharp is a nice language. It's hard to test.

Is Go I found to be a language I wasn't that fond of Go. A lot of people, a lot of my temporaries, a lot of people were very keen on it didn't particularly appear to me.

And it's genuinely true that having the compiler, having the compile type warnings, you do pick up a lot of bugs, a lot of typos that that you that you do require your tests to catch in Python. That's a it's a very real trade off, and the performance advantages of of,

languages

like Go and c sharp are are nice and real. It's hard to get around that. But still, Python

is a lot easier to test.

The the testing infrastructure

is is mature.

We still see growth. We still see change. We still see

development, but it's it's it's nice to be back in the solidly back in the Python world and and ensuring it's just the right way.

And as you mentioned earlier,

you came into

Python

at around the same time as the introduction

of Python 3. And as such, you were fairly early

in creating

a library that spanned

the 2 and 3 divide within a single code base. So

how

has that overall experience changed in the intervening years since you first

endeavored in that effort with the mock library.

So as far as I know, the the approach for supporting Python 2 and Python 3 compatibility,

from a single code base

in the subset of Python that works with Python 2 and Python 3 along with some other tricks for where you need Python 2 or Python specific shims.

Mock was the first

widely used library

to,

release Python 2 and Python 3 support using this approach as far as I know.

And in the early days of the transition,

the thought was that we would use tools like

2 to 3,

and the people would maintain in a single code base probably initially in Python 2.

And they would use the automated tool to generate the Python 3 version, and they would tweak their Python 2 source

so that the generated Python 3 works. Except you could never that was really hard to do.

So a lot of people just did did the conversion once and then maintain separate branches, separate code bases. But that was a lot of work as well. And this

so in mock,

mock was

whatever it was, 1500 lines of code

at the time, plus tests. Reaching back into my memory a little bit there. So,

not a a whole lot of code. And I thought it's gonna be way easier to do this

with a single code base. And there's there there there's just a,

whole bunch of little tricks you can use to get around the incompatibilities.

And so I did that, and a lot of those that that they're they're all individually mostly fairly obvious.

And

and other people started doing the same thing. Benjamin released the tool 6, which is trying to hop a lot of those tricks as helper functions.

And that's was

the the standard for a long time. It's now much more common for, there are a lot of projects that are Python 3 only.

But it it's also what I consider now, as demonstrated by Instagram,

to be, part of what I see as the best way to move a large production system from Python 2 to Python 3. And this is what Instagram did. I think it's a great story.

They had

a good team doing this. And they, and it relied on them having good tested tests. They had good end to end

function tests, of the whole system.

And so they, and the production system is running Python 2.7.

And so they

they got the tests running under Python 3 first, and they they got they they got and they got the tests and the code base. They work towards

getting all of the tests and the code running both under Python 2 and under Python 3 from a single code codebase. And the Python 2 tests were passing, obviously,

and, a bunch of the Python 3 tests were failing. And so they worked they they they gradually worked. They got all of the the tests passing under Python 3. So they're still running the production system in Python 2, but now the tests pass under Python 3 as well. And by the time they got to the point, we're right. We're now confident

that,

the system is stable with Python 3, we can now switch the production code over to running under Python 3, and then we can gradually start to take out all of these compatibility shims and move over to a Python 3 codebase only. So you have this awkward intervening period

where you're just using the Python 2 Python 3 subset, which is still

which is a subset. It is a little bit annoying, but it's it's fine. It works. There's nothing particular that you can't achieve that way, and then you can switch your system over

and, and get rid of all the compatibility

hacks.

And and that worked for Instagram is a great story. So so those early days where the community was still working out what are the best ways, how do we do Python 2 to 3 migrations, how do we do how do we

great technique for switching over

large production systems from Python 2 to Python 3. And and it's great that we're now seeing

many new projects

that are are Python 3. Finally,

finally, it's normal

years later.

And as somebody who has been in the software industry

for a while now, what are some of the notable

evolutions that you've seen in Python and the broader industry?

1 of the things that's been loveliest to see has been

the the genuine

push for

diversity

and radical inclusion within the Python community.

That's not to say there isn't an awful lot of work still to do,

but there's an awful lot of people out there doing an awful lot of work in this regard. We have Transcode,

which is a fixture of most Python conferences these days. We have Django Girls and My Ladies and the incredible work done by

all of those people. We've seen the number of

women speakers

at Highcon and women attendees.

Not that

go up a great deal, we're working on diversity,

other types of diversity as well. And I think that

I see that as a competitive competitive advantage for Python the language. If we can build and as we are building

as much as we're able,

on humans,

a community that is genuinely

diverse and and and compassionate, we will

we will have

more developers. We will have more contributors. We will have more people who enjoy and love Python and love the Python community. I mean, diversity and compassion are these incredible

competitive advantages honed by evolution to take and

more

and more thriving

in all sorts of niches as well as in the the general program language world and and with all sorts of diverse aspects of community. And that's something that's

that I hope

continues because only good things can will come out of it.

And

most recently in your career, you transitioned

to being full time in

offering training and consultation

in Python and software development and testing. So

where are you currently focus focusing your energy, and what are some of the grand plans that you have for the future?

So I get to advertise.

Yes. I I've made the transition. I was working with Red Hat Ansible up until,

April this year. I'm now a full time time freelance

software engineer, trainer, consultant,

and coach.

Agileabstractions.com

is my website.

I've been I've been teaching Python since 2011, mostly on behalf of David Beasley. And that's been the the most fun thing that I I I I enjoy teaching,

introduction

to Python for programmers.

So people with,

programmer experience getting them up to speed,

with intermediate level Python,

a 3 day course, and then Python mastery,

which is teaching Python from the inside out and taking people up to Python

in a in a 3 or 4 day course. And and, I really enjoy doing that. So I'm hoping to do more of that. I'll be teaching effective testing with unit test or pytest and mock as well, and TDD with Django is another course I offer.

Alongside that, my the the thing I've specialized

in in recent years is the end to end automated testing systems.

Currently, I'm working on a contract with, a firm called TestRail,

putting in place, some automated testing for their test management system. And that that's great. That's a that's a that's a lovely project. So I'm

looking to do

consulting or contracting,

around Python engineering. I enjoy

Python software engineering a great deal, coaching teams around

development process and practices, perhaps particularly with regard to testing, and helping companies

put in place or

extend and expand their automated testing systems is something that I really enjoy doing and hope to be doing more of in the future. So, yeah, michael@python.org

is the is my, email address if anyone's interested in in getting in in touch. Mostly working in the the UK and Europe, but I can do contracts in the

US as well. So automated

testing of systems

or Python teaching, training, that sort of thing. These are the things that I I love doing, and we're looking to do more. I'm hoping to to start putting together my own training courses, sort of

on-site here in,

here in Northampton.

That's, hopefully coming up in the not too distant future. Alright. And are there any other aspects

of your career

with testing or your current work or the mock library or anything along those lines that we should cover in greater depth before we close out the show?

You've let me talk quite a lot, and I enjoy talking about myself a great deal. So I'm very grateful. I think really my main takeaway is is I mean, Python and the Python community has been so good to me both

personally and,

in providing a career.

Many of my deep and abiding friendships are from within the community.

I love the open source ethos

and, and and the fact that

something that I enjoyed doing

from contributing, from building something with other people could then provide a career is I think I I think it's a beautiful thing, and and and that for me

came out of,

a passion for learning and for, and being able to contribute and find a place before that within the Python community. So I'd I'd love

to to to hope that that's still alive within the Python community, and I think it is. I see, people coming into the community

and finding ways to contribute. And,

so long may open source rate. Long may open source rate. Alright. Well, you've already mentioned some of the ways that people can get in touch with you, and I'll add those to the show notes. And so with that, I'll move us on into the picks. And this week, I'm going to choose a series of books that I've been enjoying with my kids that are,

the ology books. So there are things such as nightology,

dragonology,

monsterology,

And they're just this series of books that are very richly illustrated,

and they have a lot of little,

flaps

and interesting little tidbits of information

and whimsy

that are all purported to be these lost tomes that have been rediscovered.

So they're a lot of fun to read through, and they're an interesting way to explore some of the different subjects,

such as ones that they have on,

Egypt and mythology. So I I definitely recommend checking those out. They're a lot of fun.

And with that, I'll pass it to you. Michael, do you have any picks this week? So 1 of my great passions

is

perfumes.

And,

there are there are lots of lovely

masculine and unisex perfumes and even some of the

the adore. And, so I'm gonna tell you about an American perfume house and 2 of my favorite perfumes. And the American perfume house is called Imaginary Orphans,

and they do a beautiful summer perfume called falling into the sea. And it's, it's a nice light bright

citrus perfume as many, summer perfumes open. But then in the, the dry down, that's the, glad you're as you get the base notes coming out as the top notes sort of fade away, it fades down to the sort of sixties suntan lotion

smell and hence the name falling into the sea because gradually over time, it changes from the sort of citrus into the sort of reminiscent of sort of beaches and sun in the sixties, which is, so that's that's 1 of my favorite sort of perfumes. They also do another perfume, which is very much darker. It lists burnt match as 1 of its

notes. So it's all sort of going out in the nighttime perfume for me. It has the great advantage because it's quite a strong, quite dark perfume. It's like, if you don't have time to to have a proper wash, it sort of goes quite well over there. But it's called City on Fire. And it's it's very powerful, sort of slightly dark smell. And, and for me, the sort of the imagery of fire is is sort of passion, so city on fire. It's a it's a perfect metaphor for going out and enjoying the nightlife. So that's,

there you go. That's what that's that's that's 1 of my my passions.

Alright. Well, thank you very much for taking the time today to talk about your experience

in the Python community and your work in testing and building and maintaining the mock library. So,

thank you for that, and I hope you enjoy the rest of your day. Thank you, Tobias. It's great to meet you. Thank you.

The Python Podcast.init

Summary

Preface

Interview

Keep In Touch

Picks

Links

The Python Podcast.__init__