Test & Code

This episode is a replay of a 2021 interview I did with Michael Foord about the mock library that's part of unittest. Michael was a Python trainer and contractor. He specialized in teaching Python and end-to-end automated testing of systems. His passion was for simplicity and clarity of code, efficient lightweight processes, and for well-designed systems. As a Python core developer, he wrote part of Unitest and created the mock library, which became Unitest.mock. I just read that from his own introduction on Agile abstractions.

We lost Michael in January, and I'm really happy that I got to interview him. It wasn't easy to get this interview either. I started blogging in 2012 and soon after saw Michael Foord's name show up all over the place with respect to testing and especially with Python and testing, software testing. I asked him to come on the show to talk about unittest in 2016, but at the time he was working on Go and he recommended that I talk to Robert Collins instead, which I did and that was episode 19. But I still wanted to talk to Michael. I saw him in person for the first time in 2018. I saw him in the hallway and he was wearing a black coat and a black hat. And I think he had goggles on the hat. So I just assumed that he came from a steampunk conference or something. And he may have. I don't remember if I talked with him in 2018. But I did get a chance to in 2019.

He'd written an article about software practices. And I wanted to interview him about it. And we tried to do the interview in person at PyCon in 2019, but the audio just didn't turn out well. We agreed to reschedule. And it wasn't until 2021 that we finally got an interview done. And by that time, I thought it would be great to just talk to him about mock. So that's the focus of this interview. I'm very grateful that I did finally get the interview with Michael done. Through his writing and through talking with him, I've learned a lot. He helped shape my ideas around software development and testing.

I'm grateful that our paths crossed and I won't miss him.

The mock library that comes with Python as unittest.mock started with Michael Foord. In this episode, Michael and I talk about mocking, of course, but also testing philosophy, unit testing and what a unit is and TDD and even

where Michael's towel is and what color. Michael was instrumental in the building of testing tools for Python and he's got a lot of great advice about testing. I hope you enjoy the episode Michael Foord did i get the last name right you got it right yeah yeah it's the english version or the british version to four with two o's yeah that's right yeah double o i'm a double o so what why do is it just americans that have trouble with that.

The thing is the single i was so much more common people sort of assume it's dutch for some reason it's not it's just english back in the days when we used to have telephone books there was always a few in every telephone book so you know not common not rare whenever i see your last name i don't know if it's spelled the same but i think of Ford Prefect but did he spell it with one o or two he spelt it with one o's there's also there's also the hoopie fruit in um the work of douglas adams and i i love that i've always felt sort of an affinity with those works both because of Ford Prefect and because i am the hoopie fruit who knows where his towel is definitely so do you have your towel with you i don't have it with me but i know where it is you can know where it is yeah it's purple and it's upstairs it's one of these microfiber ones so you can pack it really small for traveling it's great perfect so one of the things okay so we want to talk about mocking eventually but what are you doing now you do you have like a consulting company or something right yeah contracting and training training so i started training python doing python training about 10 years ago now with david beasley teaching his practical python and advanced python mastery courses alongside my regular work you know i just do it a couple of times a year And I really love doing it and enjoying it. And it's a good trade to be in.

And then I worked for Red Hat. My recent career history is that my last job was with Red Hat on Ansible Tower, their enterprise web application for managing Ansible and for managing computer infrastructure with Ansible. And I worked on the test automation team there, helping them build out a test system. I worked for them for a year. And in fact, I really wanted to go self-employed. it was time to to do that and so i've done half contracting and half training and i really enjoy that i still teach david beasley's courses i've also taught some flask and some testing and, object-oriented theory for the royal ordinance survey object-oriented theory with python they wanted they didn't just want a python course they wanted something a bit more fancier because they're the royal ordinance survey so i did object-oriented theory with python that was great fun.

Contracting my most interesting project has been for the united kingdom atomic energy authority and that was from january to june last year and that was working on software for designing fusion reactors oh wow so normally yeah it's like it's like a dream job i would say he's actually a phd guy and it was his phd project so but so he'd already committed to make it open source before he sort of accepted the job with that with the uk so they are going to make it all open source his blueprint and i worked with him on turning it into an engineering project he had and at one point i mean he is phenomenally clever guy i mean i was literally actually removing descriptors off metaclass metaclass you know removing metaclasses off descriptors which you know just does not need to exist you know but but still you have to be really clever to make as much of a mess as that.

That was great fun yeah and it so the normal way that the fusion reactor design is done the academic world they're all silo everyone has their field of expertise but there's a whole series of things that need to be done you know they want to work out the magnetic containment the plasma containment chamber so they've got place they've got overall size power requirements positioning of the magnets the really cool thing they're doing now is breeder blankets.

Basically fusion reactors you put hydrogen in you ignite it with a laser enough heat and pressure to trigger fusion and then using magnets you shape and contain the plasma usually in a torus and then this produces a stream of neutrons which hit big metal absorber blocks around the side of the lining the the the reactor turns it into heat and from there it's normal turbine technology which is very well understood what they've done is they found that if they put lithium into these blankets that the neutron stream bombarding the lithium every now and then will hit the lithium split into two tritium atoms split into tritium which is the fuel for the reactor so it's a you need enough tritium which is very expensive to produce and radioactive or pretty dangerous an isotope of hydrogen and once you've got ignition it's then it's then self-feeding in terms of um of the tritium so that's a brilliant innovation anyway you used to design these things by sort of getting your specs sending to the first guy to do the first bit of design he'd take a month send you back a bunch of numbers which you send to the next guy so the whole process took months so you can't really do an iterative process of trying a bunch of things that way yeah and this this guy his genius is an interdisciplinary approach and he essentially wrote a.

One application that does all of these different things and it's for it's for initial design work it's not the detailed design but but you can now do an initial design you know with fancy open gl drawings which are beautiful and everyone thinks is the whole point but of course it's not it's all the numbers it spits out you can do that in 40 minutes so you can do iterative design processes so it's it's got it is and is going to revolutionize fusion reactor design it was a fantastic opportunity to to work on it and so much fun that's so cool um did you say that was open source stuff well i i emailed him recently to ask him if it was open source yet and he didn't reply to that bit so they were they were due to do it before now so i think the answer is no it's called blueprint there are some there is a paper he's released on it so yeah okay i'll be all over twitter with it when it comes out because it's a you know it's a it's a fantastic it's fantastic to play with yeah you can design fusion reactors in your living room i know you've got a site agile abstractions dot com is that the yeah that's my professional site a bit i haven't added any of the the projects i've worked on since 2019 and 2020 i don't think so it's a little out of date but that's my professional site yeah if somebody wanted to hit you up for training or something they could go there right yeah oh michael at python.org okay a python.org email yeah that's the instant credibility.

I got it because i helped out a lot on the web i was one of the webmasters on various mailing lists administration and um and yeah so and i asked for Michael and nobody else had taken it but it is a beautiful email address i'm very proud of it yeah so there was there's this there's a python feed thing i can't remember where it is python do you remember what that is python is that what you mean say that again i can feed do you mean planet python yeah planet python i i just remember that because i think that's the first time i ran across your name because planet python when i started blogging about python i heard somebody say well you got to get you got to get your your blog on planet python to get listed or people won't pay attention to it and then so i requested it and i think you replied and said okay it's there so it was one of the things i was looking after that was back in what i think of as the golden days of the python community back when python was about to explode with the web revolution and google adopting python and and prior to that python had mostly been used only by enthusiasts only by people who really loved the language which made a beautiful community full of passionate people really eager to to teach you you know and then.

Python just exploded and so back in back in the day having your blog on planet python you know you could get a you know a thousand views for a blog entry the glory days yeah now the other then i ran across your name next when i started researching testing stuff and you had a bunch of testing articles but then also your name is attached to the the mock library in unittest that's right that's right yeah i originally wrote what's now unittest.mock and maintained that as a library for for quite a while and that came out of my first program and gig for resolver systems in london i started with them back in 2006 and they were doing all extreme programming so it was all pair programming fully test driven development customer representative doing prioritization estimations tracking velocity all of this this kind of stuff we did that rigorously for four years and that was an amazing experience and it yeah it came out of that time and in particular i got a test a passion for testing as a way of ensuring product quality, in in programming what whereas before the sort of style of programming that i was used to do knowing that things worked was a real challenge you had to try everything and we can automate so much of that yeah so so yeah that's that i i i was passionate about python and i became passionate about testing in that time and so since i guess since then you.

You still incorporate testing within all your development processes then.

Yeah i mean we did test driven development rigorously we did extreme programming rigorously for four years so that was the first thing we do is write a functional test which exercises end to end the the feature that we're trying to add or demonstrates the bug and then we start writing unit tests and those are layered and we had a test to code ratio of at least three to one.

And we you know we over tested in all sorts of places we had testing coupled to the implementation so i learned a great deal about testing and the problems if you over test you couple your unit tests but you're testing the implementation so you're testing your private methods and then when you come to refactor and you've changed you want to change your code change the way you could call your code your tests tell you nothing useful and they're broken because even when you fixed your code again the tests are not going to show you the right thing they're testing old code functional tests your end-to-end tests if you've got good reasonable end-to-end coverage even just a set of smoke tests then you can refactor and you can be reasonably confident that sort of if you haven't broken functionality because with refactoring your end-to-end tests don't change so i think there's a lot more value in end-to-end testing i think there are dangers in over testing i like to say unit testing is about testing to the unit of behavior not the unit of implementation test through the public api if you can't test through the public api then your abstraction isn't right you know typically these sorts of things help you to avoid over over testing scripting tasks i won't test adding tests to big legacy projects is also very challenging particularly in the face of ongoing feature work and that's something you.

Have to sort of work to incorporate gradually and pragmatically because businesses you know you only get paid as a programmer if the business keeps going yeah so you have to uh you you have to incorporate building in tests and testing to give you keep sanity with um the um the the work of maintaining and extending the legacy project i really like that i'm going to steal that unit of behavior not unit of implementation i like that a lot because that's what you want to test right you want to test it does the right thing you don't want to test how it does it because in theory at least it would be very much nicer if you could change the how and your tests still tell you useful things and if you're only testing through the public api.

As long as you've got the public API right, you know, often you need to change that, then, you know, you're free to change the internals, and your tests still tell you useful things. Yeah. Now, and then, of course, as you know, there's always cases where you really want to beat up some algorithm bit in the middle, and so there is going to be some new tests around that. There are bugs where you can't replicate it without, you know, it's some timing issue or some file system quirk. And the easiest way of replicating is sticking some invalid data into some internal state, because you know what triggers the bug. And, you know, that's going to happen. You know, all generalizations are wrong, including this one. Yeah, okay, I'm definitely going to have to get this one transcripted, because we've got a lot of gems in here. The okay so mocking you got into my mocking and and stuff is a is is a natural part of i guess it depends on the year you were doing it extreme programming and and test driven development but but mocking is a little bit misunderstood by a lot of people including probably myself, and probably me.

So and and partly it's partly your fault uh-huh uh-huh alex gayback blames me, well there's i mean there's a with people start researching it they get into things like well mocks the article by i'm gonna get it wrong uh called mocks aren't stubs can't remember by martin fowler yes the static typing of that he defines various categories and types of mock ways you can use mock and he defines them as different objects and it's the defining them as they're fine as categories but they're fairly rigid definitions and i think by his definition mock is all of his types of mock except a mock.

Okay, yeah, I was curious about that. So I object to his definition. Okay, so in Python, we use the unittest mock library or often wrappers around it. So like pytest mock is a wrapper around the mock object thing so that you can use. It's basically context managers built around it, which is kind of cool. Also really cool that you can use them as context managers anyway right off the bat. I didn't know that at first. But yeah. so that's that's an interesting point and that was an innovation in mock that actually that came but it was from a guy at resolver systems it wasn't actually me although i get the credit for the guy called tom i forget his surname oh dear me poor lad good guy who discovered that so patches are decorator patch monkey patches things so you can inject a mock into any you know all sorts of places and the reason alex gainer blames me and i think possibly what you're about to say i might steal your thunders i mean patch makes it possible to test code that was essentially really hard to test before so the combination of mock and patch make it hard to make it possible to test code that's really hard to test which doesn't give you an incentive to write code that's easy to test and code that's easy to test is generally better code so patch and mock let you disguise the fact that you're writing terrible code. And that is definitely true.

But the innovative thing is that Patch, Patch which puts mocks into place, the thing that it does that's really powerful is it undoes it. It puts things back the way they were before. And monkey patching things is easy. Unmonkey patching them is hard. And there's a lot of logic in Patch that knows how to do that. So you can use it as a context manager with Patch.

Time.time as mock time yeah and that and then anything in but anything inside the context manager sees time.time as your mock object which you can then configure the return values, but then outside the context manager time.time is restored to what it was normally so the scope of the effect of the patch is limited by the context manager which is what context managers are great for you know a visible scope of effect and you can also use it as a decorator where during the function for the inside of the function that's decorated the patch is in place there and this was actually done in python 2.4 originally back before we had context managers so being able and it was tom who at resolver systems who realized that the the way that we do decorators is entirely compatible with context managers so patch can do both and patches are an interesting beast and there's actually in context lib now there's context decorator where you can write context managers that also work as decorators and that came out of what first happened happened in mock in result at resolver systems oh that's so cool python 2.5 a bit of python archaeology.

Yeah so if people have kind of missed it so far how do you describe a mock like to somebody that doesn't even know what it is okay right and this is so a mock object is an object that can pretend to be any other object essentially now the thing that martin fowler defined about mocks and what was common at the time so he talks about mocks and stubs and i forget the the another one but the the essential point about a mock is that it record you can record your expectations and then replay those and so you you create a mock object you configure the mock object and say how i think this is the martin fowler one and the java mocking libraries and the python mocking libraries at the time you create a mock object you say i want you to have these methods and i expect this method to be called it should return this then i expect this method to be called and then you do you run your code and you hit replay and it throws an exception if it was used in the wrong way oh okay so that's the record replay style of mocking and for me that puts the you you record your expectations on the mock and then you call your code and that's us about face you know what i want to do is i want.

Run some code and then i want to be able to make assertions that it was used in the right way because not all of the things that happen to it might be relevant it might just be one particular thing i want to assert that you were called with this argument i want to assert the mock might even just be going in there just for the purposes of returning a break and value stubbing out a system function that you don't want called in a unit test that's a very good use of mocking mocking external dependencies to return deterministic results for the purposes of testing to avoid external calls in your unit tests that's the classic and a great use of you might you might mock out api calls mock out network calls mock out file calls and there's some some support in the mock library for particularly for files yeah like for instance um i i think of an example which probably isn't very common but if my test if i've got a system on a logging system or something and if it finds something critical it's going to email a bunch of people during the test i don't want to actually email everybody but i can make sure that the the appropriate call to email the right people is called during the test with the right parameters yeah exactly that's the sort of thing that mocking is wrong and just to finish off the thought previously so instead of being record replay style mock is triple a what is it a range act assert you set up your code you call act which is a range then you call your code which is act and then you make the asserts.

So unittest.mock, what it does is you can configure it so that methods will return values, or a method call will have a side effect, raising an exception or calling another function, or you can give a sequence of values for multiple calls.

All sorts of ways you can configure mock objects. They can pretend to be any objects. They can pretend to be a dictionary. You can configure any of the magic methods or all of this kind of stuff. You put it in place, and then all of the calls are recorded on it. There's a and there's some convenient there's some convenient assert methods and convenient call methods so the reason i created it was for two reasons the first thing was that we were doing full test driven development and we were creating all of these little stubs objects in scattered throughout our code base with like a mock workbook a mock worksheet a mock cell a mock row a mock column you know and overall it added up to sort of hundreds maybe thousands of lines of code And initially, I was like, I can replace all of this with a Python object with a done-to-get-attra that responds to every attribute lookup. So the initial implementation was about 30 lines of code, and it was to replace all of these...

Mock objects in the resolver code base but a requirement from giles thomas who was the cto at the time was that we had to have a way of limiting the api so if we had accessed an attribute that shouldn't exist it would still raise an attribute error so that's where all of the spec stuff in mock came from um so that was where it originally came from it was motivated also by the desire of not wanting none of the existing mock frameworks all of them in python they were all this record replay style which i didn't like and there the testing in python community in python was particularly close-knit and fun and we got the the testing in python boff at.

Pycons which had a great run for for a good number of years it was a lovely community to be part of and and mock really evolved very rapidly with users in the testing in python mailing list and feedback from them and competition with other people writing libraries i was i was full of passion and activity in those days you know if any mock library came out with a feature i thought was good i'd have a new version out with that feature you know in a few days you know i answered every email about mocks on the testing in python email list showing them how to do it with my library you know i blitzed the competition well it's interesting the testing of enthusiasm so the the testing in python mailing list still exists has very very little traffic on it so a lot of people think maybe it's dead but it we still get i mean i still pay attention to it and replies go really you get really good replies if you ask a question there it's pretty good now i probably regret putting this in a podcast because suddenly people will start using it start using the testing in python mailing list folks right but yeah you've got we've got some really great smart people paying attention to it so but the okay so what is your relationship if i'm thinking of this as somebody coming in and going, maybe I should use mocks in my testing.

What do we tell people that haven't used this before? Because there is this, like you described, this test-driven development model, which is, okay, there's two huge, there's classical and MACUS, but obviously we're talking about MACUS TDD, which means we try to test every function in isolation with everything around it.

Right, right, right. The trouble then is you're not testing the wiring between your parts. The great advantage of doing that is that your tests run nice and fast, but you can have full coverage at the individual function and method level and still have no idea if your system

works because the wiring between your components is not tested at all. And the same effort expended just at the functional test level would give you confidence that the application actually works. The advantage of test-driven development and the reason it's called test-driven development the other way of putting it is i often talk about test first but the idea is that that the tests drive the design and this is what i love about test-driven development and this is what i take from it even if i'm not always religious about test first these days and it's that step of thinking about the design is that if you do test first the first step has to be not what's the solution let me bang out some code it's how do i call this how ought this to work what's the best the right api for me to be testing and so there's that it bakes in right at the start the first bit of thinking is how should this look like because the default otherwise is you bang out some code you think it works i'll add a test and what you're testing is whatever you happen to come up with not the best way of doing it and the other aspects of tdd the simplest thing that could possibly We work building up tests incrementally is that by evolving a design like this.

So long as you pay off the technical debt of doing the refactoring.

Incorporate the cost of refactoring into your estimates, that you actually come up with better solutions.

Basing your incremental approach on actual usage so those are lovely reasons to do test first and as you say if you're doing test first um if you're trying to maintain some level of isolation you're going to need some mocks but i think that's the question then is the whole the question then becomes what is my testing philosophy and and the specific question we're asking is how do i get started with mocks and so i think we can answer that much more simply but we can say look that The two things in unittest.mock library are the patch decorator, patch context manager, and the mock classes, the mock object. And actually, most of the time, probably, patch is going to create your mocks for you. So first, you need to use patch. And this often confuses people. So we can talk a little bit about that if you want.

But basically, you say with patch, and then the location. I'm going to patch out a method on a class.

It's module name.class.method as mock object. And then inside the context manager you can configure the value of the mock object we want we probably want to say mock method dot return value equals three and then after we've executed our code we simply say mock object dot assert called with and assert it was called with the right parameters so the very basics of using patch to inject a mock and making the asserts are quite straightforward but where should you do this well an obvious place is replace file access with a mock object you can if you patch back open the built-in open then you know it was called if you've got your results and you'll save time in your unittest patch out calls to time.time with a deterministic with something that's going to return you something deterministic that that system calls network calls database queries anything where you want to return pre-can deterministic results and you can avoid a real life network or if you're not testing your database access if you find if you're happy about the the database access you that's covered maybe the integration test level mock them out in the at the unit test level make your tests faster and less dependent on your underlying model this this kind of stuff is the place where mocking can give you a win yeah and these these external parts of your system well it's really the the system under test and i think one of the things people don't talk about a lot is the the test.

Architecture often mimics the people architecture so we well i mean like let's say i'm working on a i'm working with a database but i've got a database layer that some other team is working on i think it would be and and i'm i'm not responsible for the user interface i'm responsible for like this middle layer of stuff it's completely reasonable to then.

I think that there needs to be system-level tests, but as a team, I'm going to probably feed my API and stub out my dependencies or mock my dependencies. The principle is sometimes expressed as don't test the browser, which really only applies to web application development. You test the code you own, not the code you don't own, unless you have to.

Right. And then the other really depends, it doesn't matter what style of testing you're doing, whether you're doing a lot of test-driven development or tiny unit tests or even functional tests, people are going to eventually need to mock out their external stuff like API calls to external services. You're not going to, like an example that, I'm blanking on his name, but Harry Percival brought up is the credit card processing. You're definitely not going to hit the credit card API unless there's a debug one. But why not just mock that out or stub it out so the other thing that that harry brought up which i thought was a cool idea is any real third-party system not not some other team's system in within your own company but like a third party like a credit card processing or something that you really want to try to replace their api with wrap it in your own object or own module so that you have a limited set of API functions or entry points into that service, and then that's a natural place to mock or stub out those functions, is to hit those. Yeah, yeah. And if your API system is making network calls, then...

You can test this abstraction layer by mocking out the network calls and the calls to request or, you know, whatever, or the calls to their client. And so you can be sure that's worked, that's doing the right thing. And then you've got a nice abstraction layer where you can put your mocks and

completely replace that bit of the system. So you can test the two layers separately and be confident that it does work end to end. I know that the system under test calls the API correctly that we've provided, the abstraction layer that we've provided, and I know that that makes the correct API calls because we've tested that as well. So yeah, I like that approach. The other thing that I think is neat, and I wonder if this is one of the reasons why you get blamed for bad designs, is that I don't really have to understand dependency injection to use mock.

Yeah that was another motivation in python dependency injection typically boils down to adding extra parameters to your function signatures and i think function signatures are part of your api and messing with those as a way of managing dependencies is not necessarily ideal i might have softened on that i i kind of think i sometimes say this and i think i think it's a useful thing to say is that every time you use patch it's it's an admission of failure you know mocks mocks ought to be your last resort it ought to be possible to test your system and the parts of your system it ought to be testable and if you have to replace a bit of it inside the live system in order to test it then what you're saying is i couldn't design the system in such a way that i didn't need to do this so working to minimize your use of mock and patch means that you're going to get the best value out of it i think only the situations where mocking is really clearly the best approach rather than making it the tool that you turn to first because yeah there's a definite there's.

A i mean i've written code that where it's first it does this function call then it does that function call then it does another function call then it returns a result and i've mucked out all my dependencies i've mocked out this function call and the other function call and that function call and now i'm testing my mock objects and not my code right you know and that's crazy you know there and that's you've really tightly coupled your test to your implementation then you're testing completely testing your implementation details that's not a useful test you know really so look let's uh take that a bit further so let's say i have in order to test my credit card processing part of my system i i i have i've been using mocks or something with that what's the alternative how would i i i think having cleanly defined layers helps so you have a single point.

I mean, the other thing I might do is do this via config files, you know, or have another mechanism that loads a mock part of the machinery in place when I'm running under a dev config. You know, it's like, I want to be able to guarantee that when I'm running my tests, I can never send a credit card request. So I don't want to inject live things into the live system to make sure it doesn't. I want the system I stand up to not be capable of doing it.

So particularly for credit card or anything that's privacy or security critical, I would think about having machinery that does that for me. And it might put a mock API in place, and maybe that would use mock objects. But, you know, you'd have a shim layer. You'd have some shim machinery.

Well, that kind of reminds me of the database stuff. So one of the things that people often do in testing is to replace the live database or a file-based database with an in-memory database. Right. It's exactly a very, yes, yes. But a lot of the databases like Postgres and others have a memory feature. So you can just use the live database and just have it be in memory if you want, for instance.

So, yeah. yeah any any warning signs to give people the i like the idea of like look at patch first and and possibly look at like your external system like file like you gave a list which is good network system calls things like that and then the easiest isolating those like i guess i making sure that those aren't all over your system i wouldn't put like requests calls in every file of your your have it done in in a layer which is much easier to step out and test and and, have your calls go through that that's um so and here your sort of testing strategy starts to influence your design and i think in a good way yeah you know it's it's like putting the side effects into a separate function so as much as possible your functions are pure functions, which are then really easy to test and it's really easy then to stub out mock out the the bits of your code with the side effects it's the same concept same concept if the file writing happens in its own function then the rest of the function is much easier to test.

When you're teaching people, I see on your training site that you do teach testing training course. How much of that is around mocking, or is that just a small part of what you're teaching people? It's a part. It depends a little bit on the customer and what their priorities are. I maintained unit testing the standard library for quite a while. That was my other involvement with testing. I added test discovery. I helped break it up into a package. I think Benjamin And Peterson actually did the work on that one, but it was controversial. So I committed it under my name as the maintainer, and it happened. But even so, I still recommend PyTest for new projects, and I use PyTest for new projects. And I will teach people to use unittest, and there's often a section in courses I teach in general on Python. But generally, when I'm starting a new project, it's PyTest I turn to, and it's PyTest I teach. So do you use the with if you're using pytest do you use the pytest mock plugin or just use unittest mock directly i tend to use unittest mock directly that's possible usually because i teach mock as a separate section okay you know just because you know you can spend a day or two days on mocking depending on how much of the api you want to learn to use and how many different scenarios you want to cover and you know and i'm quite fond of unittest.mock but.

I'm not sure i'm not sure that's necessarily best practice with my test, okay the so i when i when i'm reaching for it i usually use the the the pytest mock plugin but it has like this mocker object that you can pull in and stuff like that but anyway the it's a it's a fixture right yeah it's a fixture that my one problem with pytest is how easy it is to get into fixture hell and i i even worked at one company where they had a cultural convention of providing stuff via fixtures so you would use fixtures instead of imports and so you'd look in your code and like your id no longer has a clue where things come from or where they are and they've got fixtures taking fixtures returning fixtures and you're trying to work out where something happens and you're following this graph of fixtures and you're up to the eighth level and you're like why why you know my life has descended into hell.

Okay so so fixtures are fantastic for you know limiting the scope of stuff and i love the scoping you know but it's like you use them sparingly you know there's like imports are great use imports not fixtures well okay i'll disagree with you on that uh but, fixtures fixtures are great as well i've just been in fixture hell but also just a reminder that there is no framework or strategy that can prevent you from writing really crappy code oh yeah i know you're right maybe maybe scala or haskell well okay so one of the things that that you hinted at i just want to like inject this in here uh dependency inject this into the conversation that.

With PyTest, you can put fixtures either in your test file or in a conf test file, and you can have one conf test file for every directory in your test structure. Oh, that was the other thing, wasn't it? It's like, this fixture I'm using, where the hell does it come from? Yeah, you can have... And then you can scatter it across your code base, like... And your fixtures can depend on any other fixture that is anywhere in its parent hierarchy. But I recommend people in a project to have one conf test file at the top. You can i love that feature it's like if i put a comf.py at the root of the project py test knows what the root of my project is now and you know a whole bunch of other things just work yeah so it's one of the nice things about py test yeah so that also and that also helps people if they they know if my fixture isn't in the file i'm looking at it's over here just helps your whole project yeah yeah pi test is really flexible but the the when i started looking at all these things i did look at unittest and i have used unittest some the i actually was annoyed with a lot of the people's complaints of unittest is the the thing that people say is it's too much boilerplate and with test discovery added to unittest there's not you don't have to do the like name equals main thing in your file there's not a lot of boilerplate i don't think but.

So yeah i mean it used to be if you used unit tests you had to write test collection yourself and test running and there's um there's a test result there's the test collectors, there's a whole load of machinery and and really test discovery was the minimum to kind of bring it up to the sort of a usable level for for for projects um a lot of the cruises in my head come from the it's a port of like j unit version five or something and j unit evolved a huge amount and unittest didn't so we have the non-pep8 style naming and pytest has funct tests as functions whereas with unit tests you inherit from test case and and you write test methods and you use the assert methods instead of the assert statement and honestly that's the boilerplate people are talking about generally and it's like yeah i like writing test functions in for pi test but i also like grouping tests together in classes you know so i don't mind grouping tests together in classes i don't mind the assert methods i was suspicious for years of the assert the pi test assert magic rewriting because the code if you get a you know the code that runs the test room they've rewritten the bytecode magically so they can tell the intermediate values at all the points so you can tell you the values of the variables in the expression that failed and that's amazing but it also means that.

You know, you're dependent on them having got it right. But like, yeah, after a bunch of years, I was convinced they got it right enough to let go of that. And using the bare assert statement is great. For tests is great. It's, yeah. But yeah, unittest is fine. You know, it works fine. It's a common style of testing that everyone's used to. And really the boilerplate boils down to, well, you need to inherit from test case and you call assert methods for doing your asserts. But, you know, not terrible.

The main thing is it doesn't have pytest fixtures right doesn't have picture fixtures parameterized tests and we have subtests in unit tests which are a nice addition that not everyone will be familiar with and um and so that's a context manager that allows you to have um a bunch of tests in a sub tests that each can have a separate name and tell you which condition failed with which parameters so yeah that's nice in unit tests but there's there's a bunch of stuff pytest plugins are really pretty easy to to write i um i did some contracting for a firm called gurock and they have a product called test rail and i wrote a py test plugin which will report tech which reports for them which reports test py test test results um and records them in test rail you know and that was that was way easier than i expected it to be it was that was great to do you know and unittest is much harder to extend and and really a consequence of using inheritance as an extension mechanism common for frameworks to provide something to subclass but like test results if you want to test results that spits out xml j unit xml and then you want another somebody else has written another test result subclass that reports.

You know orders your tests by time or whatever you have to use one or the other in unit tests you know it's composing you can't really compose the extensions so the way pi test does plugins with plugin points i actually had a version of unittest that did this unittest 2 and that i didn't merge that back into unittest in the end it became nose 2 which was popular for a while but i think that was jason pellegrin is that was is that his name it was his project it was he's it was a one-man project and you know he he couldn't keep it going unfortunately uh which was a shame but that made unittest behave very similar but i think the way py test does plugins is right and it's an interesting point about extension through inheritance as an extension mechanism from frameworks that it you know it's uh it lacks flexibility it's easier to understand at first but hook functions do make it easier in the long run.

Hook functions with events, I think that's probably something that we're becoming more familiar with as more of us are doing async programming. It is a different machinery, a different pattern.

Well, Michael, I need them just having a blast talking about testing with you. I need to wrap it up, but any calls to action for anybody? Calls to action? You know, or anything, drop your we already talked about your Agile abstraction site. Give to the homeless. But the other thing I can say, which is, I did write an article which is up on opensource.com, and I'll drop you the URL. You can maybe put it in the show notes. Oh, yeah. And that's, I think it's something like 30 things every developer wishes they didn't have to learn the hard way, which is some pithy wisdom about testing and developing my experience over the years. It's quite religious and dogmatic about the testing, but there's a lot of really good points, and some of them we've talked about, and a bunch of them we haven't. So I'll send you that URL. That's up on opensource.com. And I reckon that's an article worth reading. I love that article. Yeah, it's excellent. It's a huge brain dump on people of like some decent practices right off the bat. It's good. So thanks a lot for your time and we'll keep in touch. Thanks, Brian. I've really enjoyed it too.