198: Testing Django Web Applications - Carlton Gibson, Will Vincent
One of the most popular questions I get is, how do I test my web application? But I am starting a project and I'm thinking about this of, like, how do I make sure it's working right? So how do I how do I start testing with Django, and what do I need to think about?
Will:Hi. Welcome to a cross episode of Django Chat and Testing Code. This is Will Vincent joined by Carlton Gibson. Hi, Carlton.
Carlton:Hello, Will.
Will:And we have Brian Okken with us, the host of Test and Code. Hi, Brian.
Brian:Hello. Thanks for setting this up. So
Will:Yeah, so we're trying something new. We we want we both want to talk to each other and we both have podcasts. So we decided to do a joint podcast and talk about testing PyCon, Django, and, and do it together and see how it goes. For our listeners who may not know, you are the author of a book on pytest, and this is a Test and Code podcast. So that's sort of like the one sentence description.
Will:Right? Like, we wanna talk about specifically testing and Python today with you on our on our side of the fence.
Brian:So I just got back from PyCon, and you were, you were about to tell me before I I stopped you when we started recording. Your your podcast started somehow with a a hallway track.
Will:I run a site called learnjango.com and have some books teaching Django. Carlton just finished 5 years as a Django fellow. So one of 2 people paid to actually work on Django and do the releases. So that's kind of the the quick and dirty on us. But we met at, DjangoCon 2018 in San Diego, which was my first Django event.
Will:My first tech conference, real tech conference to be honest, too. And such a great time, and I was introduced to the concept of the hallway track because, of course, I was like, I'm gonna see all the talks in person. That's what you do. And then I realized all the poohbas were not in the talks at all. And I was like, that's a little rude.
Will:But then I realized there's this this hallway track thing and all these great conversations, you know, outside of the, the main event. And so this podcast came around as an idea of trying to mimic that and share it with others, because I came away saying, Wow, like, I had no idea, you know, how great the community was and all these things I learned. And is there some way to share that with others? And I was Nice. On my side, the genesis.
Will:And then I asked Carlton, if he would be my first guest. And he said, how about co host? And here we are. So
Brian:it was interesting that you bring that up because sometimes it's, awkward and to get started to a podcast start. You get somebody on the podcast and you and at first, you wanna, like, talk about what you're gonna talk about, and then you, you know, to try to get started. But in the hallway track, you can run into somebody and say, oh, hey. You know? Cool.
Brian:Hey. Good to meet you. Tell me what projects you're working on. And you could just start running and and you get a 15 minute conversation in with no stress at all. So
Carlton:But there is still a little tension, you know, because we're all as programmers, often we're introverted and, you know, there's still that sort of nervousness of approaching people sometimes.
Brian:The I remember the first time I was trying to impress everybody. I'm like, okay. I have to, like, talk about a project that sounds cool. Carlton, what's, is that your story too? Are you you're sticking with it?
Will:Or Did I make all that up? Or yeah. What was your side of the of our engagement? You know?
Carlton:Will gave an awesome talk on, authentication in with Django rest framework because it's quite complex. And, you know, it's a token auth and, you know, JWTs and session auth and all these different options. He gave a really good talk. And so, you know, I'd went to his talk and we we started speaking after that. And yeah.
Carlton:And then sometime later, it was, oh, you're thinking about doing a podcast. I said, well, that sounds really cool. But my favorite podcasts are ones where people kind of chat. It's very difficult. A solo person just by themselves.
Carlton:And so, you know, it's like, why don't we have, you know, we'll we'll add this name. I think you may even add the domain, Django chat or something. I was like, that's perfect. Let's just chat about Django. So Nice.
Carlton:Is it good and so so so so so so so so so so so
Will:Yes. Because you normally do this one. It's just you. Right? Like, I know you and Michael, Kennedy have, or how does that all work?
Will:Your your podcast network? Because you have a couple
Brian:Podcast network. I guess we do have a network. Right.
Will:You each have your you each you have your own and then you do one together. Isn't that Yeah.
Brian:So, he and I were kicking around the idea individually before we even met of doing podcasting way back when. And so he started, we started pie I started testing code, but it started out as the Python test podcast, and I just got tired of saying that. Testing code is easier. Although I've been I've had also people tell me that they they thought the podcast was testing code, instead of test and code. So I've been a little bit, conscious lately to try to enunciate.
Brian:So it's test and code. But, anyway, the and the testing is first because testing should be first.
Will:Oh, we can talk about that.
Brian:Michael started, talk Python to me around the same time, but both of us, I think, individually made a conscious choice to support each other, promote each other's podcast and everything. You know, 22 social networks, could have a larger audience than 1. We were doing the our individual thing for maybe a year. I can't remember the exact time frame, but he contacted me and said, hey. It'd it'd be kind of fun if we did a podcast together, and, and it of a news format sort of thing of, like, a few news items, every week.
Brian:So we just we tried it, and it and I love the the I love the format. It's a it's a really fun thing to be able to we talked about, like, 44 to 6 topics with 6 if we have a a guest host on. But everybody picks a couple topics to talk about, and it could be any anything, anything in Python related usually. Or sometimes mostly, we thought it'd be news most of the time, but sometimes it's just a cool project that we haven't covered yet, and it could be an existing older project. And, and a lot of people have given us feedback that they just, they just constantly have, like, a a list of things they wanna try out because of our podcast, which is nice.
Brian:But they they having another cohost gets you to do it every week.
Will:There's no way I'd be doing this if it was just me still. I mean, it's so much easier with a cohost to just sometimes you're like, I'm out of it this week. Like, you need to do the lifting or hey. Like, Carlton's been great about we started off weekly, and then we've been going every other week. And we take breaks sometimes because kind of the key thing is not to burn ourselves out.
Will:And we we have thousands of listeners, but it's not like our it's not a moneymaker for us. So we, you know, we wanna stay sane. And at this point, for me, it's frankly a chance to talk to interesting people and, like, have an organized chat with Carlton. Like, you know, everything else is secondary, like, you know, I guess, you know, marketing our stuff a bit and all the rest, but it's really just nice to talk shop since, you know, we're all sitting in our own little caves somewhere. You know, we're not like, I'm not overwhelmed with social stimulation with work.
Will:Oh, that's
Brian:a that's a good point. The the being willing to talk with somebody is like or having being being okay with asking somebody to talk with you is something that a podcast gives you that's really pretty cool because I I've reached out to people that I would have never just reached out in person and said, hey, you don't know me, but, can we talk for like an hour?
Carlton:Yeah. Can I have an hour of your massively valuable time? Can I just have that, please?
Brian:Yeah. Carlton, you just got done being a Django fellow. And I Yes. I guess I'm not I'm not involved with
Will:The Sai. The Sai.
Brian:I'm not involved with Django enough yet. I do a little Django, but I don't know what a Fellow is.
Carlton:Okay. So, Django is.
Will:Django just appears and updates itself. And that's.
Carlton:Well, it's quite a big project. And it's it's got a lot of history. And it just give you an idea on the Django issue tracker. There's 3 to 5 new trick tickets opened every single day. So there's like a 1,000 plus tickets every year.
Carlton:And, you know, that's gone on years years on years. The security reports that come in pretty much every week and they need triaging, they need looking at, they need resolving. We do. So So in 2022 with the security team, there were 10 CVEs handled through Django Security Program Processes reports. For instance, there's, you know, a constant stream of pull requests that need to be dealt with.
Carlton:There's releases monthly. There's major releases every 8 months. There's, you know, there's community handling that needs doing. And it turns out that on a project the size of Django's, it literally cannot be done just on volunteer effort alone. And so how long ago now?
Carlton:I don't know. Maybe 8 years ago, 9 years ago, there was the, the creative, the Django fellowship program where people were contracted, initially, Tim Grahame. And then I came I joined 5 years ago, Marish Feliciac 4 years ago. And Natalia, whose surname I don't know but really should now, has just started as the new fellow now. And that that, having a contractor, having someone who's people who are paid means that the tickets get triaged, means that the pull requests get reviewed, means that the releases go out, you know, on time every month, means that the security patches are done.
Carlton:And I honestly believe that the fellowship program is still the reason why Django is a been able to survive and not just it didn't die, essentially. It got to that kind of fruition of an open source project where the original contributors started to fade away. And why did Django not just collapse in on itself and fade away? Well, because of the fellowship program. And it's I honestly believe it's one of the, you know, the reasons why Django is sustainable and reliable and will continue now till probably the heat death of the universe.
Carlton:And then
Will:And Python's adopting it now too. Right?
Carlton:Yeah. Good point. I was just gonna say that was the the Python develop in residence program that they've now got is directly inspired by the success of the Jenga Fellowship program.
Brian:Okay. But, apparently, we just didn't like the name fellow.
Carlton:Well, there's already Python Fellows, so you can be a PSF, PSF fellow, which
Will:Naming is hard. Right?
Carlton:Yeah. So there's a community award, for contribution contributions to Python, which is the PSF Fellow Ships. And they're they're awarded quarterly, I believe. And there's a Django has a Django's the Django Software Foundation. And if you're an individual member of the Django Software and the Nations, that's the equivalent community award in the in the Django community.
Carlton:Okay. Nice. So that's everything I know.
Brian:And how many fellows are there at a time? And
Carlton:well, currently, there's 2 and there were 2. There was originally one until I came, so I think there might have been the original pilot. There might have been an overlap with it. Birkin did it for a few months. Yeah.
Carlton:Yeah. A little while. I don't know the the very dark history that I'd need to, ask others. But Tim Graeme did it for 4 years by himself, which was amazing. He did full time, 4 years by himself.
Carlton:I've always said so when Tim stepped down, you know, there was some conversation more to Carlton. Do you want to carry on? Do you want to go full time? So I did it part time for 5 years. I said I didn't really want to do it full time and I didn't want to be the only fellow because it's really hard.
Carlton:Like you're on the you're on the sort of front edge of the fire hose. You know, it's a it's a really big jet of traffic that comes at you. And a lot of it you have to say, sorry, this isn't a valid request or, you know, we're not going to add this. You have to close a lot of tickets as won't fix. And people get very upset when their idea is rejected.
Carlton:And some sometimes you get it on the nose. You know, people aren't necessarily meaning to take out on you, but they do. So one of the and so they have some support. It's important.
Brian:One of the things that Python has is this is well, we used to have the BDFL and now we have the steering console. Does Django have a similar sort of idea?
Carlton:Yes. So there is a technical board which was called technical board is now called the steering council. There's gonna be elections for the new the new version of that, coming up for the 5.6 series now. But if if in Django, we try and work by consensus. So normally, there's a maining list or that the forum where there's a discussion.
Carlton:And if there's not a clear consensus, then it can be you can we can ask the steering council to have a vote and say what the way it should be. That doesn't happen very often. We're quite good at, you know, reach trying to accommodate and reaching some consensus. But sometimes it does have to go to the store at Steering Council and they have to just make a decision.
Brian:Well, if you're a fellow, though, like, can you do you have enough power to just say, wow, I want this cool new feature. I just wanna put it in. No.
Will:When you're leaving, you can. Right, Carlton? You slipped a couple in.
Carlton:I I may have burnt some social capital, getting some things through quickly that needed to be done.
Will:What was it? There was one in particular. There was something that, you know, was one of those that this isn't controversial. That's why I'm mentioning it. There was there was something last
Carlton:I have no
Brian:I have no idea.
Carlton:I don't know. I've I've blanked it all.
Will:It's okay.
Brian:No comment. I'm on it.
Carlton:I'm on official detox. So I stopped like 2 weeks ago. So I think I'm in week 3 of stopping. And I and and so so the the DjangoCon Europe is coming up at the end of May. And I've said I'm not going to look at Django Django and I'm not going to look at track unless, you know, if someone sees me or mentions me, then that's fine.
Carlton:But I'm not just going to watch the notification because 5 years, every comment, every commit, every pull request, every ticket, every you just need to step away and cleanse the habit of checking that. So I don't really know what's going on in the moment, which is lovely. Well,
Will:I was just gonna add Brian was also Django's history, there were there were 2 BDFLs who stepped down actually before Guido did, but like Python, Django was set up as a nonprofit.
Brian:So the
Will:fact that it's a nonprofit is why there are some volunteers who run a board. We could talk about that. I just got off I just also left. I spent the last 3 years as treasurer on the board running the money. So, but that's what funds the fellowship program, and that's kind of how it's very community based, which generally works well.
Will:It means we go a little slower, but there's a there's a churn of people. So when someone gets burned out, like, you know, not burned out, but 5 years, Carlton needed a break. Like, as treasurer, 3 years, I needed a break. There's other people there, and, you know, we're still involved in the system. Like, Carlton just released a Django Skunk Works project, you know, last week.
Will:So, there's a recycling that keeps it fresh.
Brian:Well, one of the things we wanted to wanted to talk about was testing a little bit. And I'm really happy that you guys are here because I get a lot of questions about it. Since I run, test and code, wrote a book on testing, one of the most popular questions I get is, how do I test my web application?
Carlton:Okay.
Brian:And, I'm not a professional web developer. I'm a professional embedded software engineer. So, not my wheelhouse really, but I, so my my deflectiveness sometimes is just you run through the API instead of the, the front end. But that's not really that's not really fair, maybe. But, but Django's and I, I was the first time I interviewed you, I didn't have enough experience with Django to to ask decent questions also.
Brian:So there was technical I
Will:think same for me. I don't even wanna look at my responses.
Brian:Yeah. So but, but I have a, I'm not saying I'm a Django expert yet. I'm still learning Django. I'm in the I'm a I'm a Django newbie, but I am starting a project. And I'm thinking about this of, like, how do I make sure it's working right?
Brian:So I'm really glad. I don't know how to start this conversation other than how do I how do I start testing with Django, and what do I need to think about?
Will:I want Carlton to take that, but I wanna just ask you, are you so as someone who's, you know, you know how to program, you're new ish to Django, are you using a starter project? Have you, like, how are you starting? Are you, like, because I have I'm just yes. I'll leave you with that. What what are your what's the process for someone like you who wants to start Django and wants to know how to do it properly?
Brian:Well, I'm not sure. So I start I did I did a, like, starting Django, and I started the the tutorial. And the tutorials, I I read slow, and I'm very impatient. So I I'm I'm like, there's gonna be a long time. I to to to it reminded me of my first, first couple months at Hewlett Packard when I when I first started out.
Brian:So I was a compute pure computer science person. I got hired, to do, I was work working on satellite testing at first, but then, quickly moved into oscilloscopes. So doing scopes? No. Spectrum analyzers.
Brian:And one of the the one of my managers said, you really under should understand electrical engineering, and gave me this big Horowitz book or this this huge gray book of of and I'm starting, like, learning DC circuits and stuff, but I'm working with microwaves. And I'm like, there's gonna be it's gonna be so long before we get to microwaves. I'll just rather I'll I'll just rather the I'll cheat. And so what I did was I learned the I I I was just nice to people at work and learned the experts in the RF stuff, and I learned enough electrical engineering to talk to the electrical engineers. And so I kinda feel like that with Django.
Brian:I'm like, who do I need to find some experts in Django, and I'll just learn how to talk to them, and that will be good. So I have a start I found a starter project. There was a and I don't even remember the name, but somebody that knows what they're doing. And, I'm starting with their
Will:You're killing me. Can you tell me who it was? Well, I'm well, I'll I'll I'll speak out loud. So there's the big one is there's one called cookiecutter Django, which that's probably the default that people find. And then there's a host of slightly smaller ones.
Will:I run one of them. That's why I was sort of curious if you came across it. But there's, I think there's a gap in terms of people who are experienced programmers who want scaffolding to kind of jump in. There is there's only one really paid starter project for Django, SASPegasus that CoriZoo runs, which is fantastic. So Okay.
Will:The the sort of market question is, is there room for more? Or, you know, he's I think he's doing that pretty much full time. Sorry, I'm just I'm just I'm getting free market research from you because, you know, Jenga developers, all of us after some period of time have our own personal starter project, just like all the consultant you know, the agencies have their own. And so it's it's one of these standard, you know, you're having a pint and you're like, you know, what if I charge for my starter project? Like, how much work could it be?
Will:You know, and, you know, it's a it's the code is the least of it.
Brian:Well, the answer
Will:is probably that's kinda why I was
Brian:The answer is probably it's more work than you think, and it's probably good that it's
Will:Oh, for sure. Yeah. Yeah. But it might be worth it.
Brian:I think there's room. I think that there could be dozens of script. I maybe dozens would be confusing to the marketplace, but, but a handful of, I could easily there's different target audiences too. So, like, you were saying the, SaaS, Pegasus. That's a particular kind of thing that's as as far as I my brief looking at it, it's, targeting, paid, subscription based kind of projects, which not all projects are like that.
Brian:So your project might not be. So
Will:Yeah. Well, that's what I was gonna ask you. So you said testing. I mean, he, that project has a lot of things in it. It does have, you know, a lot of great things.
Will:The test the payments thing seems to be the one that lets people open up their wallet as opposed to, sorry, Carlton. I'm just like
Carlton:I like I like your MBA wisdom as you do the market analysis on this.
Will:Oh, God. As I sit in a psych ward co, you know, podcasting booth. But anyways, yeah, Zio is Carlton, testing. Where where should Brian start?
Carlton:Okay. So you should start with Django's test client. So Django in Django dot test has a, the module Django dot test. It has a client. And what you can do with that client is you can make a web request to a URL.
Carlton:So you can say get client dot get, you know, forward slash homepage, you know, whatever the URL to the path to your home page is. It might just be just slash. It might be nothing. It might be, you know, the index page. And then it will simulate the actual request, and it will it will run the the the the the request through the the website, the request response back.
Carlton:It will give you back the response, and it will also add a few extra features on the response so you can test things like, the template context. What would so you can get the HTML and you can say, did did the HTML include my header? Well, you might know that might be a bit hard if you're looking for a particular fragment of HTML. So you might say, was this template rendered with this context? So you can get that which template was rendered, which context was it, What was the status code?
Carlton:What was, you know, did it redirect to a to a login view, for instance? So say you've got an authenticated view and you want to check the authentication works, will you make a request to that with an anonymous user and in the cert that it was redirected to your login page, for instance. So that's should use the test card.
Brian:And does that go through does it does it go do you have to have a server running? Or
Carlton:No. That doesn't. So you can there's there's a level up from that. So that's that's the basics. That will do most of your cases where really you're not you don't need the browser behavior.
Carlton:You it just it talks it runs the test client will sort of use your Django app internally, so to speak, it will make a request to the Django app and get the response and you can test the HTML. But that's not the browser behavior. So there's a level up where if you need, like, you know, say you've got some JavaScript on the page, and you need to test that the JavaScript actually does function, then you would need to use a thing called Selenium or or Playwrights, the new the new kid on the block there. I, you know, I don't have much to say about which is better that a lot of people are liking Playwright, but I don't know if the integrations are as mature is all I can. So in Django's test, we we use selenium.
Carlton:There are lots of tests, and we use that for testing the admin. So for instance, in the admin, there's JavaScript to add a new inline related object in the in the you've got the admin, you've got the inline objects. You want to add a new one. There's a bit of JavaScript to do that. Does that work?
Carlton:Well, we have to use a selenium test for that because it's all JavaScript logic. And that does need that that will when you run that selenium test, Django will fire up, a live server. It's called live server test case, and it fires up a live server. It opens a Firefox window or whatever your Selenium driver is using. It clicks on the actual buttons as you as you describe in the test, and it runs it.
Carlton:So you can do that kind of more. That's like I guess they call it fully end to end testing or something like that. You can do that selenium testing, but those are difficult to write and they're more brittle because every time you change your CSS or, you know, the position of a button moves, sometimes it doesn't. The test breaks and you have to fix those. So I would say save those when you need them.
Carlton:But the first port of call is Django's test client.
Brian:But like a workflow. So let's say I'm I've got a, application where somebody has to log in, and it might be like there's different levels of log, like permissions and stuff. And I wanna think, okay. I've got a different kind of per person logging in, and what what do they have access to? Does this workflow work, go going through some stuff?
Brian:Can I do all of that with the test client, or do I need to do playwright or something like that?
Carlton:Pretty pretty much you can do most of it with the test client. Like, in in most cases, you would do so I you know, I'm thinking, you might have a a view where you start off in the foot, and you might you might just take you might test a sort of a view, a post, a form post view in in a kind of cup couple of tests together. You might break them up into separate functions or you might just go right. First of all, make a get request to the page to load the form. Then make a post request with the data, then check that, you know, that the the object was created.
Carlton:But you can you can authenticate with the test client. So you can give it, you know, log in with this particular user. And you can either do that by sending it to the login URL, or it's got a shortcut method where you can kind of say, look, log this user in, and then we'll do that, run the rest of the test. Because, you know, there's a lot there's a teeny bit of boilerplate going via the login view, and then back to the other view, which you might not want, might not want or need.
Brian:And then I'm assuming I'm hoping that there's like some, there's enough structure there that we can evaluate. I don't wanna do a workflow thing. I'm I'm saying, well, I assume that this kind of a user is already logged in there, and there's already data there and stuff, and I wanna just test this little thing. Is there, like, I'm assuming there's setup stuff you can do to it
Carlton:like Yeah. Yeah. So Django Django ships and some test cases which are based on unit tests. So, you know, normally in your test set, you define a setup function, and that's where you do your fixtures, where you populate the database with whatever you want. And you, you know, you said so what is it?
Carlton:You you you I can't remember the 3 a's, but you set it up and then you do some actions and then you assert the, what you're looking for.
Brian:A range. It's a range. Yeah.
Carlton:A range. There you are. Yeah. You arrange what you want in the test case and then you do some actions and then you assert that it was. And the Django test case as this case is provide some nice assertions like assert that query set is equal to or assert that.
Carlton:I don't know. A warning message was raised or assert that a redirect to a certain URL or these kind of things. It's got a lot you know, it's quite mature.
Brian:So like Python, Django, the default test framework is a unit test based thing.
Will:Well, it extends it. Yeah. It extends unit tests.
Brian:Okay.
Will:Django. Django is best defined. It extends.
Carlton:Yeah. Yeah.
Brian:Whereas, I don't think that Pytest Python extends it. It's just, it's just raw unit testing. But the, but the the 2021 developer survey from, JetBrains and Django, says that 39% only 36% of the people use unit tests and 39 use Pytest.
Carlton:Yep. Yep. You so you can do that as well. That's because everyone's read your book. Right?
Carlton:And I I do highly recommend that book as well. So, yeah, you can do that. So there's a there's a plug in for Pytest called Pytest Django, which gives you lots it gives you, for instance, a fixture for the client, and it gives you a fixture for, I know, the database access. So, pytest Django will not allow you to access the database unless you tell it. No, no, no.
Carlton:Here's a I think there's a marker or a fixture. You can use either. But if you use one of them, it lets you access the database. Okay. Whatever.
Carlton:I think there's even a live server fixture. So if you want to do these kind of selenium tests, I believe I'm not a 100% sure because by preference, I do use, Django's test cases. But But myself. But I have used I've used PytestPlenty, and it's it works just as well.
Brian:That was one of my questions. So I'm glad it works. It works just as well. It probably has similar sort of the similar test cases built in or something, hopefully.
Carlton:It's the Pytest the Pytest Django package has got the things that you're looking for when you're migrating away from the Django test cases.
Will:Okay.
Brian:The but I was just curious if you thought these numbers also were real. Because the Django's developer survey is like the people that would might even know that there's a JetBrains based survey out there.
Will:Well, there's So there's backstory on the surveys. So this is where this all sound really incestuous. So Django had a survey, like, 2015 and then hadn't had one for years. Meanwhile, Python has one which is very helpful to the community. So one of the things when I got when I got on the Django board in 2020 is I wanted to do the survey.
Will:So that was one of the things I worked on was it being in charge of that. So hopefully it happens this year in my absence, but, and we partnered with with JetBrains is, you know, so we fly we we, Django, fly blind because we don't track downloads, we don't track anything. Really, how does it being worked? It's what know, the fellow see an issue tracker. It's what's in the forums, and it's what people talk about in the hallways.
Will:We know that we're massively blind to a lot of things. So anyway, so the survey was brought about to try to shed some light. Now, we know that a lot more people use Django than take the survey. I think it was something like 8,000 people used it last took it last year, and this is a communication problem that, being on the board of Django has. How do people find out about anything about Django?
Will:It's like, well, if you go to the Django website, used to be go to the Twitter account, like, there's no email newsletter for Django. This is a whole separate discussion Carlton and I have had many times, I think even on this podcast, but there is there's definitely a lack of light being shown in terms of how are people actually using Django. I you know, anecdotally, the numbers I see in the survey match with what I my impression. Yeah. But, you know, I'm a American, you know, going to Django cons.
Will:Right? I'm not, like
Carlton:there's a there's a just a whole mass of people out there using Django in their, you know, real world situations who are you I think don't even know that pytest is a thing. Like, they're just it's like they go they download Django. They're using Django. Django says these are the testing tools. They use a testing tools if they test at all.
Brian:Okay. I was gonna say I'm just grateful if they test at all. So Yeah.
Carlton:Well, yeah, that's the thing. If they test at all, they're probably I would imagine there's a big chunk who aren't filling in the the survey that are just using the Django tools. But within the developer community, I think, yeah, it probably is represents.
Brian:Now one of the other another question I had was, early on in reading about Django, and I I'm not sure if this this changed. There was a notion of, fixtures that were like things that set up the database. Is that still the term used, or is that still a thing that people do? Or
Carlton:It is a thing. So you can you can populate the database, and then you can do this. You've got this command to dump data where it will dump it as JSON, files. And then you can load load them again with the load data command. It's quite you know, it's got, you know, all sorts of useful uses.
Carlton:But you can also specify those those dump data files as fixtures for a test case, and it will it will load them in kind of prior to to you running your Tesla as fixtures.
Will:Yeah.
Carlton:I don't think people use that as their first protocol. And I don't really recommend people using that as a first port of call. You're much better off in, setup in setup or setup class data, which is a class method that only runs once rather than once per test. Creating the the objects you need, using the ORM or using factory functions that you've, you know, written to speed up, much like a Pytest fixture that you're familiar with. You know, you create the fixture function that does that creates the objects.
Carlton:You would do the same, but you just instead of having it as a fixture function, which you can't ever find, you'd have it in the, the test case. So just above the test that's tested.
Brian:And is it is it difficult to, like, let's say I'm I'm, getting started with, testing my Django application and but I have a lot I mean, Django pulls in a lot of stuff. I want I kinda wanna, like, know if I've covered all of the the things I've added. Is coverage do people use coverage? Does it make sense?
Carlton:Yeah. No. And it works it works really well. Yes. There's a there's even a plug in which will help you cover your
Will:I think Ned maintains, you know, Ned Batchelder who runs coverage. I think he also runs the the, coverage Django plug in.
Carlton:But that will test that will test coverage of your templates, which is quite good because you might have a little bit of logic in your templates, like a for loop and then an if and then an else branch. And I didn't test the else branch.
Brian:Okay.
Will:I think, Brian, for for beginner or people new to Django, but who know what they're doing, one of the issues is you don't want to you don't need to test Jeng core Django thing. Django comes with all these tests itself. So there's how do you know what is a core Django behavior versus what is something you did? Right? So I think sometimes people end up trying to test Django behavior.
Will:So generally, Carlton, you're making a face?
Brian:No. No. No.
Carlton:No. No. I was nodding. I was nodding. I was I was lining up in some So I
Will:mean, so you you said earlier about, you know, test test first and, you know, test driven development. That's a whole discussion. Carlton and I both, come down on the side of kind of write your code and then test it. Or better yet, when something changes or when something breaks, then you add then you add the test. So that that's a, you know, sort of an in between system because anytime so I you know, in my books, I lead I have tests, and I show people how to do this.
Will:And I'm like, even the beginner's book, like, boom, we test, we're testing everything. When you've changed something, then you add a test. So just because you see Hello World, like, you know, you test, does it show Hello World? You don't need to test that the view got the temp. I mean, you can, but, like, it's only when it's something new.
Will:Like, I don't test the admin, for example. Like, I don't test that the admin works because it should work unless I've changed it. If I have changed it, which which Django lets you do, then I would run a test to see, oh, does does this customization, work as planned? So it's when you customize something you wanna test it.
Carlton:What do you like changing
Brian:the admin? So don't don't you doesn't just doing anything change the admin or do you have to, like, would you mean change how admin works or
Will:Well, so you can yeah. That that's that's a valid point. So you can there's built in ways to how you display your your data, whether you want it kind of how you want it listed. You people can and do abuse the admin, so you can kinda use it as a poor man's CMS, or maybe there's valid use cases for it. But when you're so there's there's there's changing kind of the the graphical layout of the admin, which Django gives you tools to do.
Will:You don't need to test that. But if you're I mean, Carlton, you've seen it used and abused. This is one of the things as a Fellow people people I mean, there's all sorts of thoughts on the admin and trying to make it be more than it is.
Carlton:The official line is that the admin is not your application. Right? So use the admin. The admin is a power tool, and it's wonderful. And it's one of Django secret sources and, you know, push it to its limits.
Carlton:But people when people are like, oh, can I open it up to the world? No. No. No. No.
Carlton:No. And can I customize this thing? No. Just write a custom view. It's gonna be much less work to write a custom view than it is to try and force the admin to do this funny thing that it was never designed to do.
Carlton:So there is a sort of there needs to be a limit to what the admin can do, not not least for maintainability.
Brian:That that's actually something that that I, the the for my one of my first exposures to Django was an internal project, not at the company I'm at now, a previous company that, was, the the user interface essentially was the admin pay at the admin level, page. And I was like, I I I don't wanna build something like that because that's what what one I'm consider I'm talking about, like, 20 year old Django probably or 15 year old Django. So it's even that's probably better now, than it was 15 years ago, of course. But still, if I just wanna have somebody enter some stuff, I don't want them to have to go through and look at the data tables and stuff. That's that's too much.
Will:You know There's built. There's permissions that the problem is Django lets you kinda do a lot and you can, like, you can have super you know, you have a super user and you can have different access. So you can give different people access to the admin and permissions it up a little bit. And then, you know, just you said, it's like, oh, do I really need to build my own custom thing for this when it's kinda working well? Like, you know, at some point, yeah, you do.
Will:That that's kinda how it plays out. Right? Like, it's just me. It's me and Carlton. Oh, now we're a team of 10.
Will:There's some nontechnical people. We really want them to be able to, like, answer customer support. So we have set on permissions so they can't touch this, but they can enter this. But they're still like, that's when you get into trouble. Or you start, like, saying, hey, the admin's slow because I'm trying to view, like, a 1000000 records within the admin.
Will:I mean, there's multiple Jengo Con talks about, you know, using and abusing the the admin. So it's a very natural progression. I mean, I I probably abuse it too a little bit, but it's more like in yeah. It's easy. Yeah.
Will:It happens, but you should avoid it.
Brian:So if if I'm so Django's big in my application if I do a relatively simple application, I'm changing just a little bit. I'm like using all this stuff and I'm not really changing how Django works. I'm just, building a website. So the I'm assuming my my approach probably would be test the thing, like the end user experience. The if somebody goes to this page and is that is that when you said that that's is that testing the views at that point or
Will:Yeah.
Brian:Was okay. Or the model Yeah.
Carlton:So well, so the what does the view do? The view is like the controller in the old MVC thing it get it get the request comes into the view and the views job is to turn that request into a response. Okay, so in order to do that, it probably goes to the ORM and it says can I have some records from the database, and then it grabs a template from the file system and it puts the 2 together? And it creates a, it creates a response. Yeah.
Carlton:So test what you want to test, you want to test what does that load. So you know, for any if you've created a nice view, create a a test which is that hits it and checks that it returns 200 and maybe that the template was correctly rendered or something. You know, that so that it could it it works. Because one day, you'll I don't know. You'll do something.
Carlton:And then you'll just break that view. And that you've got just that test that checked the 200, that it that it returned an okay response will tell you that you broke something. Now Yeah. You know, if if your view has complex logic and it could it might return a 404. So, you know, so if you go to somebody else's repo on GitHub, it goes 404.
Carlton:It just said, you know, if you haven't got the permission for it, it's a private repo. It just says 404. It doesn't exist. So lots of URLs are like that where if if you've got permission to access the the object, it will show you the response 200 and render the template. But if you haven't got permission, it will just say, oh, it didn't exist, or it will redirect you, or it will say 403.
Carlton:Like, forbid forbid it. So those kind of different responses that you might get, they're worth testing. And then they're quite high level. They're quite level most integration test level rather than unit test level, but you can spring those around. And they give you quite a lot of coverage and quite a lot of sort of juice for your money, if that makes sense.
Carlton:And they're quite easy to write. Anyway, Gomber.
Will:Well, I think it's yeah. I was gonna say with because I've I mean, like you, a book author, I've thought about how to present this to people. Because I think tests in a Django context, people have no idea where to start, in part because they don't understand how the web works, how Django works, let alone how do I test it. So it's just a lot of confusion.
Brian:Yeah.
Will:But you kind of write the same tests over and over and over again. Like, tests are super confusing until they're incredibly boring to the point where, like, we've like, chatGPT and all this stuff, like, throw in that at, like, what should I test would be a great use case actually, because a lot of times you're writing the same test over and over and over again. And so I was for, you know, for you or for someone who's, like, what test do I write? Well, you can look at open source projects. Like, if you go to the awesome Django repo, there's a list of some, see how they do it.
Will:You can if you type in, like, Django testing tutorial, you'll probably see, like, Django has docs, Mozilla has one, and then I have one that kinda gives you 80% of all you'll ever need. Like in my books, I show you how to do it. But, I mean, if you're doing CRUD and you're doing forms, you're really you're, like, test as Carlton said, testing the model, testing the view, testing the template, testing URL. Does it return what I want? Does it not return what's not expected?
Will:And you just kinda do that ad infinitum, at a certain point. So just, like, find an example of something like, if it's CRUD form stuff, which, you know, 90 whatever percent of the web is, you're kinda writing the same test over and over and over again.
Brian:Well, one of the things that
Will:Right, Carlton?
Carlton:Two sides.
Will:I mean, you know, it it depend and then when you have logic, it you know, at a base level, as your logic becomes more complex, yes, then then things change and, you know, you wanna test that out. But if you're just kind of the the basics of, like, does this page display the information I think it does in the way I think it does, like, it's kind of the same thing over and over again.
Brian:Well, I'm I'm gonna take that as a challenge then, because I don't like to write the same thing over and over again.
Carlton:Okay, but then you get to another level of Jango where you, for instance, you're into, you're dealing with the ORM. And so what people cut quite often do will, embed domain logic into manager methods or methods on the ORM. The themselves. So for instance, you know, I will quite often so I've got a project, a project model. And I don't it's a list of projects.
Carlton:And I users are only limited to which projects they'll they can access, you know, via an organization, via a team, or, you know, maybe they're the owner. And so on the project model, only the what's called the manager, which is the objects bit inject you know, project or objects dot filter is the ORM API. But that manager, I'll add a method called for user, and then I can pass in the current user. And then you're writing more of a unit test type of approach where you literally you've create a few instances, assign some of them to the user, create some for a different user, filter the the query set, and make sure only the users, projects have returned.
Brian:Okay.
Carlton:That kind of thing is is embedding your domain logic. Right? You're you're checking that that works. And then that that manager you'll then use in your view, and you don't necessarily need to test again that that it was correctly filtered because you know that the base query set was filtered to that you to the request user, for instance, just as an example.
Will:Yeah. We should also say you, as a guest and, you know, to get some free help, Adam Johnson, who's on the the tech the technical board at Django, has written 2 books, one of which is Speed Up Your Django Test, which I think addresses more of the concerns you have. Like, that's a more, I would say, intermediate advanced level book on testing. And he certainly knows his stuff. So, you know, you could do an hour on, like, how do you not write the same test twice with him?
Will:And
Carlton:I wanted to ask I wanted to ask you, Brian, though. So you would you are you gonna be using pytest? Because you're you're the expert on Pytest. So would you be using Pytest for your for testing your project?
Brian:Yeah. I mean, I
Will:okay.
Brian:I'm aiming on it. Yes.
Carlton:Okay. So so I have a a real question about Pytest. And you're the expert, and I really want to ask this. It's about managing fixtures. Because in the in the unit test world, the the the fixtures go the the arrange goes in the the setup method of the test case cast, and it's right next to it.
Carlton:In Pytest, they do the dependency injection thing where they do sort of the fixtures live somewhere else, and then they they get injected. And one thing I found as projects scale is that it becomes increasingly hard to track down where the fixtures come from. So do you have any kind of advice for, you know, managing your fixtures to keep them maintainable, to keep them trackable? Because I I sort of end up at the point where I'm, like, right clicking in the IDE and then the to go to definition. And I'm like, please take me to the right place because I'm just not sure.
Carlton:You know, as it scales up. Only if you've got I'm not I'm not a Pytest. You know, I use it, but I'm not like a pi test efficient order by choice. I'll use the unit test method. So I'm asking
Brian:Well, what are the things that I used to complain about? So there's the there's a couple ways you can do it. Well, the the most obvious way now is the dash dash fixtures command line flag. So if you if you run-in like, either as a whole, you can run, if you run all your test suites, but if you just run run as if you're gonna run all the test suites, like or your particular test and add the dash dash fixtures, command line, it tells you all the fixtures that are available and where they're defined.
Carlton:Okay. That's quite cool.
Brian:It used to be a very a fairly quick list. It still is. But but you could get more information if you did dash dash fixtures with, like, verbose, like dash v also.
Carlton:Mhmm.
Brian:But, and the verbose used to be the only way you could get the actual file name and line number where it's defined. And now by default, it gives you the file name and line number. So, it's really not hard to find them. But I also you don't have to put them somewhere else. Like, you can put them right in the test file.
Brian:If you don't wanna go hunt for them, don't put them somewhere else. So you can put them right there. So they're either they're either in the test file or they're in a conf test file somewhere.
Carlton:Yeah. But yeah. Okay. I mean, because one thing I've I did I have done is use test classes in the like, arrange my tests in test classes to keep them grouped together. And it's something that I'm used to and then use the Pytest runner.
Carlton:You know, and it all works the same. It works perfectly. And then I get I sometimes feel a bit embarrassed as if I'm doing it wrong or something as people.
Brian:I I mean, I I there's a lot of fixtures that I write that are only for one test. Because I because I want them I want the setup to be obvious that it's not part of the test. The the other thing that you get is if a fail you can assert within your fixture. And if an assert fails or any any, with pytest, any exception causes a failure. But if it's in a fixture, any exception uncaught exception causes an error.
Brian:So that distinction between error and failure is did my did the the thing that I'll go wrong happen in the setup or did that go happen in the test itself?
Carlton:That's quite a little that's a good good one. So would you would you kind of recommend not not, not over committing to being super dry and that you reuse a fixture every would you rewrite a fixture? Would you like so say you've got the same, you know, you you, different tests, different places in your project. You you need a project fixture. Would you would you be tempted to rewrite the project picture closer to where it's being used to avoid a bit of confusion?
Brian:No. I I I'll put it wherever wherever at some central point. So you can have a ConfTest file at all at any directory, or multiple ones. You can only have 1 Pytest to any file, but you can have multiple ConfTest files. And the and the ConfTest can it can go all the way up.
Brian:And the cool thing is you can start out at a file level and then maybe move it to a directory level if you wanna share it between tests, and then maybe you can move it up a directory, but the tests don't have to be rewritten. The the you just move where you've defined the fixture. It's it's not, it doesn't have to be see, the test I love that you don't have to rewrite the tests. With unit tests, you have to
Will:you have
Brian:to change the code. If you if you if you decide you want this fixture that fix fixture logic to be at the at the file level or a class level, that's a completely different thing. Actually, you only have class and function level. Right? I think there's
Carlton:Yeah. But you might you might move it to a shared module or to, you know, somewhere.
Brian:But the, the other thing I like is you can change the scoping of it. I mean, so you can say, like reset the database to be a completely clean, absolutely no changes in it, but that might be too time consuming at the around every function. So you could broaden it up and say, I want this to only happen at the beginning of the session. And, and then if you, if you've gotten bugs all over the place, you can go, well, that was a mistake. And you can, dial that down to like a module level or something.
Brian:And the test does the test still doesn't know about it. The test just says it gets the fixture stuff. The other thing I really like about fixtures is that the setup and tear down are in the same file or in the same function. They're just they're just divided by a yield keyword. Whereas a unit test, you've got to set up and tear down completely separated into 2 different functions.
Carlton:Yeah. No. Interesting. Yeah. Good.
Carlton:So it's you. So the issue there is just I think the answer to my question is is just maintain hygiene, you know, because I've been on projects where I've been a bit like, ah, where's that? Where are these? Where's this fixture? Where's that fixture?
Carlton:And I think it was just got a bit messy over time. It's all
Brian:both the but it's gotten really a lot better even just recently. Like, Versus Code just recently implemented in the last few months implemented, the ability to just, right click on a fixture and go to the definition.
Will:Yeah.
Brian:And and then also, being able to, do type hints in it. So if they're they're well type hinted, like, Versus Code and PyCharm will also, like, do com code completion for you, with your the returned object. So, but I'm I'm looking I probably will get you guys back on because I'm so I've got this project that I haven't written any tests for because I haven't actually ever yeah. I don't have any running code for it yet. But, but I it'll be interesting, interesting progression to build this thing up and find out, like, how I feel about the Django testing environment.
Brian:And I'm I'm I'm somebody that likes shiny new tools, so I'm sure I'm gonna try out, like, the Django test plus. I mean, it's got a plus in it. It's gotta be good. Yeah. And, and I've heard people talk about Factory Boy.
Brian:I'll probably, like, try that out and all these sorts of WYSIWYG. Not WYSIWYG, but whizbang features things that but I also don't like too many things because why pull in a a dependency
Will:You kinda mentioned all I mean, there's not that many. Like, they're right. Are there other other I mean, there's, like, maybe 1 or 2 other big ones, but there's not a 1000000 commonly used third party things with Django.
Brian:But there's probably a 1000000 uncommonly used ones. So it's, shit.
Will:But are they but why are they why are they uncommon, though? Right. Why are they uncommon?
Brian:I don't know. People don't have their own podcast to plug them on.
Carlton:One thing I will say is you can use Django, which I think with Django, with Pytest Django, with the plugin, you can use Django's test cases just with the pytest runner and it just works. And so, you know, you can experiment writing your test in the Django way and then you can mix in fixtures and you can mix in plain assert. One thing I really love about pytest is just being able to use plain assert rather than self dot assert. And what was it again? Because, you know, so many different asserts is can be hard to remember.
Brian:But there's stuff that, like, you can't it's gonna be hard to write as an assert. So some of the helper functions are helpful, especially around Django, to say, like, well, assert that the user is logged in or something. I don't you know, it might be easier as an assert helper function than it is as code. Yes. Yeah.
Will:Yeah. Well, what you're what you're verbalizing is very common in that, like, I wanna try out all the things, but, like, where do I even start? Because, you know, Django has 3rd party ecosystem, which is a huge strength. And you can go to djangopackages.org and see however many 1000 and 1000 there are. But, like, which ones do you wanna use first?
Will:Right? So if you went to the testing section, there's a lot. So there is a curated list, and again, I'm plugging my own stuff again here, but it's called Awesome Django that, I run with actually the person who runs Django packages now, that has a list of, you know, how how many is it? You know, a dozen. Right?
Will:With this idea of trying to say, okay. Well, those of us in the community, there are some packages that have been around for a little while, have a certain number of stars that are maybe a a good first place to look rather than just what's new and shiny, because that is you know, and there's also things like there's a Django newsletter and and stuff to try to both expose people to new things, but also remind people of existing things that are plugging along, like like Django debug toolbar, for example. Like, that's a package that is, you know I just did a post on, like, what do I think are the top, like, I think if I said 10 packages, 3rd party packages you'd use. Like, there's a whole discussion of this. I think everyone's got that probably top 5.
Will:But who maintains it and who uses it? Like, someone started it, and then there's this group called Jazz Band, like so the maintainability and the fact that Yeah. Positive things are still just like Django itself, like, there's a tremendous amount of stuff in the most recent release of Django 4.2. And yet from the outside, people say, oh, Django's kinda just Django. Like, right?
Will:Like, I don't know if you had that with your books. I mean, you're on you're on the 2nd edition. I'm on, like, the 7th edition, and I basically rewrite them all from scratch, and I'm, like, it's so much better. But, like, everyone's, like, oh, it's just still there. Right?
Will:So, like, it's hard to get people to appreciate, like, you know, just being still around is a huge sign of success generally for for a code repo, for a book, for a project.
Carlton:Yeah.
Will:But it doesn't seem like that maybe unless you've maintained one yourself.
Brian:Well, so the second edition of, the PyTorch book wasn't really because the pie test changed too much. I mean, it it had changed, so I I did add new things. But it was more around the teaching. So the, I I didn't I didn't feel that the first edition was an the appropriate ramp of difficulty. And so I the the main impetus for rewriting it was I'd like to start people out with the real basics and then introduce fixtures, try to do it gradually.
Brian:I think I put as a little bit more abrupt than maybe I should be, but, but that's fixtures coming in chapter 3. And then just gradually add on complexity so that by the end, they're pretty much I mean, if you really understand the whole Pytest book, I think you're a Pytest expert at that point. I'm not teaching a 100% of Pytest, but it's it's the 80% that or the 50%, 60% that 80% of the people need every day. And then some extra fun stuff. Like, I really want people to write plugins more, because it's a great way to and packages.
Brian:It's just a packaging packaging extensions, because that's how they should be sharing code. Even within even if I don't do a pipey package internal to a company, people should be sharing their fixtures with the plug in packaging system because it's there. Why not? But I do I see people trying to share, like, trying to share fixtures between projects in some wacky way, and I'm like, there's a built in mechanism. Just do that.
Brian:So, that might seem like a fancy thing, but you get into companies and people have those needs even if they're not writing their own open source project. So
Carlton:Yeah. And if you if you're the person who knows that then as well, then that's valuable too.
Brian:But the, we're kinda getting long, but I I do wanna come back because at some point, because there's a lot here. The the final thing is I wanna I'm guessing I don't wanna spend 80% of my time testing or thinking about testing. I wanna probably spend 80 to 90% thinking about my problem, and I only wanna spend, like, 10 or 20% in the test thought process. Does that jive with how I can develop Django or do I need to skew those percentages?
Carlton:I'd say that's fine. Like, yeah. Like like so you you'd mentioned about testing first. Right. So when you're developing your your sort of domain, you bits inside the view, the bits that are going to go into the view layer and they're going to be like the view is going to call out to those to do do its job.
Carlton:Then, you know, you could do test driven development and you can write nice little tight tight unit tests for those and you can build them as you go. And that's that's where you're having the fun. But the actual testing around the outside, Django is well tested. It works. You can lay down these simple endpoint tests that we've talked about, and they will give you confidence.
Carlton:But they're not. They're quick to write and they're not difficult. And Okay. Give you a lot of coverage for not much.
Brian:So people shouldn't be scared to try testing Django's projects. This isn't no. Okay. Good. I mean, I in
Will:my in my What what? I put yeah. I I put it, like I don't think I put it in the Hello World, but I put it on, like, the second project that you do. You know? I mean, it's like like, you know and again, not to I'll give you a copy.
Will:Like, you just peek at my book, just flip to the testing sections. But, like, I show how to test, like, a basic thing, how to test a CRUD, how to test something custom. That kind of gives you 80% of what you're gonna use in a normal setting.
Carlton:Okay. When a colleague of mine, when I was a junior, was very keen a senior colleague was very keen on writing tests. And I was like, oh, you always write the test. He said, well, if I don't write the test, I have to click it in the browser. Yeah.
Carlton:Like, it's like, so just write the little endpoint tests. And he was like, they just say it's quick than writing in a clicking in the browser. And every time I make an update, just,
Will:yeah, I've
Carlton:got a test. I know it works. And there's not I can keep coding as long as I don't break that test. And then yeah, once or twice, I'll check it in the browser.
Will:We didn't even talk about CI at all in this conversation, you know, all that.
Brian:Yes. We'll have to get back into that, hooking up CI and everything. I want
Will:Just GitHub actions.
Brian:I wanna clear my name, though, because I was disparaged earlier. Let's just a little bit, I'm joking. Is I think of, I think of testing first, when I'm thinking about coding, but I don't always write the test first.
Will:Yeah.
Brian:Yeah. I I just wanna make sure that test the idea of how am I gonna make sure it works is in my mind while I'm coding. That's that's my advice I give people is, just think about how how how am I gonna verify that this works. You can do it afterwards, but just make sure you verify it. That's one of the things I like about coverage because, if I forget, I can run coverage and go, oh, I didn't even cover I didn't run I don't test that at all.
Brian:So but that's good. I enjoy so your your part how do I find the podcast you guys at? It's djangochat?
Will:So we're djangochat.com. Okay. And yours is what's what's testing code?
Brian:Test I think it's testing code dot com. The and is spelled out. So that's the and.
Will:There you go. Yeah. But, yeah, we should we should do this again in the fall when, you know, you have a little more experience, getting your hands dirty. And I'm glad I'm so glad that just as a side note, Carlton has been mentioning that question around fixtures in Pytest for, like, years. So finally, like, I I saw him nodding.
Will:So some something new to think about on this issue because people have raised Pytest, and that's always been the bugaboo that you you were burned in some corporate projects on it. And so and I can't help you out on it. So
Brian:Yeah. Well, I mean, people the the the pros in your team learned it when there was no dash dash fixtures. Would that would give No.
Carlton:I'm gonna give that a go. I didn't I didn't know about this. So I'm gonna go and take pull out a Pytest project and give that a go right now.
Brian:Okay. Well, so fun talking with both of you, and I wish you all luck. And, I'm gonna go get some coffee and wake up. They they had me wake up at 5 in the morning to do this. Can you believe that?
Will:That's what it that's what it takes. Well and I'll just end with we are also taking we're taking a little bit of a break until the fall. We might slip in a special episode, but, we'll be back. We'll be doing repeat episodes, but Carlton's not saying no. So this is our last one for a little bit.
Will:Alright. So let's slide that in.
Brian:Good to talk about you.
Carlton:Thanks for coming on Chip Proline.
Will:Okay. Thanks, Brian. K. Bye.
Creators and Guests
