Test & Code

The standard pytest output is actually pretty good. It's come a long way since I started using pytest. There's nice percentages if even by default, you can see the test files that ran and all the test cases themselves show up as dots for passing. And and then a percent of the the test suite complete shows up on the right. And it's pretty nice, actually.

And there's ways you can customize it. There's all sorts of ways you can customize it. And we're gonna take a look at some plug ins that customize it for you. And we're gonna start with, looking at a couple pytest plug ins that will generate markdown reports, which is cool. Taking a look at the top pytest plug in list, there are a couple plug ins, listed.

At number 76. I'm looking at the February list. At number 76 is pytest-md pytest-md. That's a plug in for generating markdown reports for pytest results. And then down at number one zero two is pytest-md-report. That is a pytest plug in to generate test outcome reports with markdown table format.

So one does markdown and one does a markdown table. This is cool. That's what we're gonna look at today. Let's start with the first one. pytest-md.

That is a plugin generated by Rafael Pierzina, and it doesn't look like it's gotten a lot of work lately. If I look at PyPI, it shows that it supports up to Python three seven. Well, three eight is still deprecated. So I tried this out. So I'm I'm trying this on Python three thirteen two and using pytest eight point three point four is what I was trying these out on.

And it still works fine even though it's not list. It hasn't been updated. It's doing some pretty some pretty standard stuff that's been supported for with pytest for a long time. So and, also, Rafael was one of the is one of the pytest core maintainers. Well, he has in the past.

It's still kind of kind of fun. So what does pytest-md do? If you if you install it, it will PIP install it. And then you if you pass it in --md and then give it a file name like report dot m d, it does the normal stuff, but then it generates a report file. And this were like, it's it's just a it's a markdown format, and it lists kind of a summary of of how many tests passed and failed and skipped and everything.

Now you can get a verbose more verbose output with if you pass in -v, and that will give you more detail on each test passing or failing. Actually, if you do -v, it will list it'll list every test, even the the passing ones and tell you the times that are they ran. It's kind of fun, actually. And it's even incorporated with emoji. So he has another plug in called pytest emoji.

And if you have that on also and pass in --emoji and -v and to your report. Actually, it doesn't matter if you have the v or not. It'll put little emojis like a a happy face for passing. And for error, it's a little angry face. This is cool.

xpassed is a surprise. Nice. xfailed is a snooze or, like, sad. I don't know. Anyway, there's emojis that's kinda fun.

Like I said, it hasn't had a lot of a lot of love since 2019. I think it was 2019. Let's take a look at the history. Yeah. Last touched in July of twenty nineteen.

And Rafael did some some trainings and some some presentations around that time, so I don't know if this was associated with something like that. But, anyway, you can still take a look at it. It's kinda fun to look at the code, and I it's still it might be useful for you to generate some markdown report, so that's nice. Next up, let's take a look at pytest-md-report. That is it's had a little more activity.

So that, if I look at the release history, it was started back in 2020, and the most recent release was January of twenty twenty five. So that just just last month. So nice. What does this do? It is it's generating a report.

Oh, it's kinda neat. So the the the Rafael's pytest-md, it only reports to a file. It doesn't change the standard output. Now md-report does change the standard output. So if you have it installed and you pass in --md-report, at the end, it will list a a table, a markdown table with, let's see, file path, passed, failed, error, skipped, xfailed, xpassed, and then a subtotal for each of the files.

So it's got a it's got all these columns for the different types of things and the number of tests that are in each column, like the number of passes in a file. And it so each file is a row, and that's nice. And they're color coded. So the failing files are red. The passing the files that all of the, all the tests are passing are green.

And then you've got yellow for skipped xfailed and xpassed. And and then the total has the color, like, basically, if there's any failures, it's red, etcetera etcetera. If you you can have a a verbose report, but this is kinda cool. It doesn't it isn't tied to the -v. It's got its own verbose flag that you pass in.

And instead of, per file then, it will list it still it tells you what file it's in, but it adds another column, which is the function. So you get one row for every test function. So that's it's a bigger table, but it's pretty cool. It's actually a beautiful looking table. Nice.

There's a whole bunch of really cool flags for it too. If you it it'll the like, by default, the if something's zero, so, like, if a test file has zero xfails in it, it'll show up as a zero, but it'll be kind of a light gray zero. So it's it's pretty, you know, innocuous. But if you don't want any if you're not especially if you're not using color, you might not want any zeros in there at all so that it's easier to read. And you can you can say, like, set the md-report zeros to empty, and that'll, like, clear those out.

You can also, change the color so you can have control over color. Maybe if you don't want color at all, you can say report color never. And there's a lot of configurations for it. You can set up the configurations in in toml, pyproject dot toml, probably your any file too, I imagine. Yeah.

I'm sure you can yep. It it supports any and TOML and, setup dot CFG, but that's just because it's using using the the same setting system that pytest is. So so that's pretty cool. So what about if if you wanted a file? So by default, it's just printing it out, but you can pass it in a file, and it'll go to a file instead.

And if you wanna see it on both the file and the standard output, you there's a t flag for that too, so you can have that as well. And what if that markdown file is not quite the same format as you want? So, like, maybe you want GFM or the GitHub. Are those different? Anyway, there's a couple different flavors of markdown that you that it supports, which is nice.

The other thing is maybe you've got a system where you really don't aren't using xfails, so you don't really need the xfail and xpass column. So if that's the case, you can tell it to just exclude those columns. That's nice. Anyway, really kind of a really fun plug in. I really like this one.

So I think both neither of them are a lot of code, though. They're not super huge, and that's kinda one of the neat things about pytest plug ins. So this is kind of a fun thing to look at. If one of these is closer to what you need, you can take a look at how they're coding this and take a look. And since these are open source, you have access to it, of course.

So check those out. I highly recommend, checking out probably both of them, but especially pytest-md-report because I like those color coded tables. I also really like that the verboseness is separated. Actually, if you pass in -v, it will do the verbose also, but I think you can turn anyway, it's it's possible to not have your verbose pytest and still have a per function report table. So that's cool.

Anyway, I think let's let's this is fun talking about different ways you can report. So next episode, I'll probably do a different plugin that also does a different kind of report.