The Python Podcast.__init__

Share on social media:

[00:00:12] Unknown:

Hello, and welcome to podcast dot in it, the podcast about Python and the people who make it great.   When you're ready to launch your next app or want to try a project you hear about on the show, you'll need somewhere to deploy it. So take a look at our friends over at Linode.   With 200 gigabit private networking, scalable shared block storage, node balancers, and a 40 gigabit public network, all controlled by a brand new API, you've got everything you need to scale up.   And for your tasks that need fast computation, such as training machine learning models, they just launched dedicated CPU instances.   And they also have a new object storage service to make storing data for your apps even easier.  

Go to python podcast.com/linode,   that's l I n o d e, today to get a $20 credit and launch a new server in under a minute. And don't forget to thank them for their continued support of this show.   Having all of your logs and event data in 1 place makes your life easier when something breaks, unless that something is your Elasticsearch cluster because it's storing too much data.   ChaosSearch frees you from having to worry about data retention, unexpected failures, and   all for half the cost of running your own Elasticsearch cluster or using a hosted platform.   Try it out for yourself at pythonpodcast.com/chaossearch,   and don't forget to thank them for supporting the show.  

And you listen to this show to learn and stay up to date with the ways that Python is being used, including the latest in machine learning and data analysis.   For even more opportunities to meet, listen, and learn from your peers, you don't want to miss out on this year's conference season.   We have partnered with organizations such as O'Reilly Media, Corinium Global Intelligence,   Alexio, and Data Council.   Upcoming events include the data orchestration summit and Data Council in New York City.   Go to pythonpodcast.com/conferences   to learn more about these and other events, and take advantage of our partner discounts to save money when you register today.  



[00:02:13] Unknown:

Your host as usual is Tobias Macy. And today, I'm interviewing Ram Rahum about PySnooper, an alternative approach to debugging your Python projects. So, Ram, can you start by introducing yourself?  

[00:02:23] Unknown:

Sure. Hi, Tobias. I'm happy to be on the show. My name is Ram, and, I've been a long time Python developer.   Yeah. I've been   involved in open source for a long time. I would say I I've been programming Python for 10 years.   Most people know me in the Python community as an organizer of PiWebIL,   which is the Israeli   Python community.   We meet once in a couple of months   as a meet up in the evening where volunteers from our community give talks about the various topics,   sometimes Python related and sometimes more general software development related talks.   And now, over the years, I've worked in the small companies and big companies, worked for startups, and worked with a freelancer.  

And recently, I started working for a company called BlueVine.   So I actually only won't work there for 4 months. They're a company that does the Fintech and Israeli American startups.   And   so I'm I'm sorry getting to know in my new environment. So I've been working there. It's a good place to work if anyone we are looking for all kinds of software development and   all other kinds of positions, both in the Redwood City, California offices and the Tel Aviv offices. So if anyone might be interested in joining BlueLine,   feel free to send me your CV. I've got my   the careers page in my email in the show notes. So it's a pretty good company. They closed an investment round of $100, 000, 000 a few weeks ago So we are all very happy about that. Yeah, another thing I'd like to promote before I get talking about the my main thing is that I also give the Python workshops   inside business.  

I have a website there Python workshops that co where, where I go into detail about my services, but in brief   companies sometimes hire me to come to their offices and give courses. We get, like, 3 days or 5 days where I teach their teams the   where I improve their Python skills. Sometimes,   I get to teach teach beginners and sometimes more advanced people. I tailor the courses to their clients' demands. So it's a side business. I do if you might be interested and check on the website Python workshop dotco.  

[00:04:22] Unknown:

So that's pretty much about me. And do you remember how you first got introduced to Python?  

[00:04:26] Unknown:

Sure.   I was a poor college student into Technion, hyphen,   and I was reading Paul Grant's essays   for anyone who isn't familiar with Paul Grant.   He's the founder of a y combinator,   which is the most,   I would say it's the most well regarded startup accelerator.   And so I was reading his essays. He's famous for having all these interesting essays on his website.   I think it's I don't remember the address, but if you Google for Paul Graham essays, you'll get them. The   he he has very interesting essays about programming and startups and finance.   And I was very interested in him and that was 15 years ago when he was just starting y coordinate, which was just the first batch   before everyone realized he was a genius.  

So many startups like Airbnb and Dropbox   came out of there. So I was reading his essays, and in 1 of his essays, he off handedly mentioned that Python is a good language. So I sort of made a mental note to myself, Python   might be a language I would like to learn.   So a few years down the line, when I was getting back into programming, I had to ask you this Python thing to try.   And   I used to program when I was a kid. I used to program in CA in Pascara when I was in junior high, and I was always frustrated. But all the technical thing is that you have to remember all the details.   That was that was very difficult for me. So I figured I I hate being a newbie. You can being a newbie is so so annoying. So I figured, I'm gonna take this 1 language, and I'm gonna know it really well. And I really, really hope I wouldn't have to be a newbie in a language again. That was 10 years ago, and my plan kind of worked. And I kind of became a Python expert, and I really hope I I really hope I'm gonna be able to keep on the same language   for the remainder of my career.  



[00:06:12] Unknown:

Yeah. Well, with the trajectory that it's on right now, I think that it's a fairly safe bet, but you can never tell in the area of technology because the, the fates can be quite fickle. Yeah. I know lots of people, are falling in love with their Rust right now. So you have And so in the process of learning Python and using it for your day job and doing these trainings, you   ran into some of the shortcomings of debugging in the Python ecosystem. So I'm wondering if you can just start by discussing some of the existing state of the art for debugging in Python, some of the shortcuts that people might make, and some of the pain points that you were dealing with that led you down the road of creating PySnooper, and then we can talk a bit about what it is. So I've always liked using debuggers.  

[00:06:55] Unknown:

When ever since I started programming, I would always do all my work in a debugger.   I use a Wing IDE   and I know most people have never heard of it. Most people   who most people use PyCharm for their debugging. So it's basically the same kind of software. It's a it's a good program   to install, and   it it's the same same same old kind of debugger that most languages have.   And now I'm very used to using a debugger, and I use it all the time. And I set breakpoints and I step through the code and I have my debugger automatically stop on exceptions, but then I can travel up and down the stack and   and explore any kind of variables or run any kind of arbitrary code, which I find is very useful for figuring out what's happening in your program. So I've done this since forever and for me, this is programming,   but the 1 I started working with other people, especially when I started working being companies, I saw that most Python developers don't use debuggers.  

And that blew my mind. That really blew my mind. I mean,   I I I was thinking I'm gonna see professional developers who are very, very serious, but it turns out they they just most of them use print statements.   If they had a piece of code and it's not doing what they think it should be doing and they want to investigate,   they add print statements or a temporary   log   statements,   and then they see the the print output.   And then they and then they kind of have an idea of what their code is doing. Like, they could add a print statement at certain point in the code, and then they would know that that that certain point was reached.   And they would add more print statements in different in different parts, Then they would kind of try to figure out the flow of the program that they would maybe expose the certain variable in a print statement to know what it is.  

I've done this in the past. We've all done this, but I was a little bit surprised that this method is so common. And the reason it is so common   is because when you're working for a company, especially if it's a it's a big corporate thing,   the setup is usually very convoluted.   I mean, you're working on a project that was inherited to you from someone who inherited it from someone else. It's usually a mess. And the code that you're writing is not running on your computer. It's running on some kind of other machine that your computer is connected to.   And there is probably   a complex build process that shuffles your Python code around, and and most people don't really know how to connect their debugger,   be it PyCharm or Wing ID or any other program. Most people don't know how to get the debugger to work with whatever convo to set up their corporate has. So most people just resort to using print statements, which was very surprising.  

But using print statements for debugging, I think that's both   I have mixed feelings about this. I think it's both very beautiful and very ugly.   The ugly part, of course, is is how ridiculous it is because you have to, instead of these print statements, manually to your code and then see the output   and then do and then you usually realize you wanted to put the print statements somewhere else.   So you do a back and forth, building your project multiple times just to get the information you wanted, which to me is such frustrating process. So that's the other part. The nice part is that it always works. Like, it doesn't matter how convoluted your service, you can always print to stand it out or at least write to a file. And this is the reason everybody uses print statements   at some point or another because it's already resilient. It works everywhere. So I'm getting to a point about what what is Spice Nooper.  

So I was thinking maybe I mean, after many years of seeing people use print statements, I was thinking maybe I could do a compromise.   Maybe I could provide a solution that is as resilient as print statements and as easy to use,   but a little bit more powerful like a debugger. So that's what Python is. So I'm gonna describe how someone would use it and what it provides. Let's say you wanted to use Python to debug your code.   So, obviously, you do a pip install PySnooper and import PySnooper. And then you go to the function in which you you wanna understand what's going on, and you decorate it with a pipe number.   Snow decorator. And and what that does is it basically   does sort of automatic log to your function as if you put a print line on each and every line of your function.  

And this means that when you run your code, you're gonna get sort of output text dump of all the lines of your function at random   1 by 1. And anytime a variable got changed, you're gonna get a line saying variable x change, and its new value is this and that.   So sort of instead of going back and forth and adding print statements, it sort of just automatically   prints everything you might need. And then you can take this x output and look at it and sort of figure   out the play by play what's going on in your function. So that's that's the gist of the solution.  

[00:11:43] Unknown:

Yeah. As you mentioned, print statements are easy to get started with. They work all the time. But then there's also the added issue of you need to make sure that you actually go back in and delete them so that they're not running in production and potentially leaking information into log files or into just the standard output. So,   that's 1 of the areas we're using. Log statements are a little bit preferred because you can set the you know, set them as debug or error level so you can have a little bit more control about when they show up, but it's still littering your code base with a lot of extra information that is only useful in a particular context and might be useful down the road to expose it to people. But like you said, there's the back and forth of, oh, I want it here. No. Actually, I wanted it there. And, oh, I actually needed to know what this other variable was. And so as the program evolves, those log statements print statements become a lot less useful. So I'm wondering if you can talk through a bit now about what Pysnooper is and some of the problems that you're solving. Yeah. So Pyscooper, it doesn't aim to be, like,  

[00:12:38] Unknown:

strong, very advanced debugger. It aims to be a cute little tool that fill the hole in the market   as far as the buggers are concerned.   So, it's nothing to be a very, very powerful thing, just just something that answers a very specific need.  

[00:12:53] Unknown:

What was the rest of your question? Just wondering if you can, describe a bit more about sort of what PySmooper is and some of the issues with print based debugging, log based debugging that it addresses?  

[00:13:05] Unknown:

The main issues are that you have to go back and forth. You have to build your project multiple times. Lots of especially in big corporates, they the build process can be slow. Sometimes you can take a few minutes to build your projects,   and I'm sure that some of our listeners are working in a terrible corporate environments where it takes more than a few minutes to build your project. So having to do the back and forth of building your project multiple times, that's such a drag in Python will perhaps to avoid that. And, also, having to kind of guess the information that you want to expose,   having to write the string formatting   lines that kind of expose this variable and that variable, having to do it manually, you know, it's a real chore. So that's pretty much what the Python number saves you saves you from having to do. So for people who are feeling the pain of dealing with these print statements and log statements as their debugging mechanism  

[00:13:54] Unknown:

or who have been fighting with the built in PDB debugger for trying to figure out how do I actually get access to it, particularly if you're running in a web request and you don't wanna use some of these debugging tools that open a socket. What's the actual workflow for getting started with PySnooper and starting to use it for investigating problems in their code base? And what's the life cycle of a PySnooper statement once you incorporate it into the code? Is it something that you would then need to go back and remove like you would with print? So I stand  

[00:14:19] Unknown:

something that you would then need to go back and remove like you would with print? So I start from the end. Yes. You will need to go back and remove it. It's the same as print in this regard. And now what would be the workflow for someone who want to use pipe snopper? You could read the read me which is pretty short, but I'll try to make it shorter PIP install pipe snopper   import PySnooper in your code, and then add the PySnooper dot snoopdecorator   to your function. Now there are a bunch of arguments that optional arguments that I snooprecording that the snoop decorator can get,   but you don't really need all of them to get started. Just the the 3 things I said, build install Pythonoper,   import by Snooper and decorate your function Snoop by Snoop and   that's not is already working as we got more arguments. You can pass the for the first argument, you can pass the filing. And then instead of writing to standard error, it would write the same kind of text them to your file. And now a bunch of other useful arguments and you can set set us a different depth. The default is depth equals 1, and this means that only   only code lines in your function are gonna get traced. Now if you were to set that equals 2, and it will also go into any function that your function calls. If you were to set that equals 3, it would go first to go into any function, it's called by function that your function calls, and you get it. So so that is 1 use for sending there. I think I think there are 10 different arguments there that you can modify to kind of tweak it to your needs. But the but the very basic usage is just the 3 things I said at the top. Just keep install PySnooper  

[00:15:52] Unknown:

import PySnooper and decorate with the PySnooper dot snoop. So for somebody who is using Py Snooper, what are some of the types of information that it surfaces   and how would you incorporate that information into the overall debugging process of identifying   what's the control flow for the logic, what are some of the branch points, how do I figure out   what the values are given these different operations and understanding   how to actually fix the problem that you're trying to debug? And most of the information that's provided  

[00:16:24] Unknown:

is what I mentioned so far,   which is   the order in which the lines ran. Because every time a line runs,   it produces   a line in the text dump.   So it you'll basically see the runs the lines running 1 by 1. Every time a variable gets changed, you're gonna see the output   the value of the new variable as a new line.   So this is basically 90% of what you need to debug.   Now sometimes people will also like to call arbitrary code and maybe call a certain function   and and to get its value or at at a certain point. So we also have an argument for that called watch,   and it works basically the same as a debugger. You just put any arbitrary argument that you desire, and you're gonna see its value.  

Yeah. And if it if its value changes, you're gonna see the new value every time.   And now watch is 1 of the algorithms that we've got in Pythonoper. And if you go into the Pythonoper documentation,   you're gonna see the documentation both for watch and for all the other arguments that we have. It's it's a pretty short documentation, just a couple of Vimy files. I I potentially kept PySnooper as simple as possible.  

[00:17:28] Unknown:

And so I think that's a good opportunity to talk through a bit about how Pysnooper itself is actually implemented  

[00:17:35] Unknown:

and some of the evolution that it's gone through from when you first had the idea of it and to where it's gotten now and maybe some of the types of projects that you were testing it with to guide the direction of where it ended up? Okay. So these were a few questions in 1. I'm gonna back in 1 by 1. So you asked about how Fastenoper works. Right? Yes. Okay. So fast number works by using c's dot set trace. C's dot set trace is the mechanism that CPython provides, and that's basically any kind of debugger that you've ever used. Uses this dot set trace   and also any kind of coverage measuring to the platform also uses this to set risk   what is system ceteris and it's Python a it's Python function that you call it and you pass the function into it called a tracing function   And you can see Python, please use this tracing function. Please, for every line that runs in my code, I would like you to call this tracing function with information information such as which line ran and what kind of event happened,   as in just a regular code line or function returned or there was an exception,   and any other useful information like the exception that got raised or the or the value that the function return. Basically, any debugger or any on any   coverage measurement tool works by putting its own logic into that trace function. For example, if you were writing a debugger,   then some logic you might put in your trace function   is a is to see whether the current line is on the list of breakpoints that you've got. And if it's on the list of breakpoints,   then you should pause the program at this point. Or if you were writing a code coverage tool, you would look at the at the current line that's running, and you would mark that in your database   as a line that did run. So in the end, you   and I put my trace function. And my trace function,   what it does is it checks whether this is the a line that we should be tracing.  

It checks whether the variables   got changed from the previous line, and it outputs the line saying which line ran and if any variables change, which variables changed, and what the new values are. So that's basically how it works. So CPython does the heavy lifting, and I just do a little bit of logic around that. Now, sorry. I forgot the other parts of your question. Just curious about some of the ways that  

[00:19:55] Unknown:

the implementation of PySnooper has evolved since you first began working on it and having the idea about it and some of the types of programs that you were working on that influenced the direction that you took it?  

[00:20:06] Unknown:

It's it's interesting to to talk about   how how it evolved.   It mostly didn't change.   It mostly I mean, once I published it and it got very popular, We're gonna talk about it later. We got lots of contributors. And so people   added features and fixed bugs and polish it, but mostly the basic operation stayed the same,   and it mostly stayed stayed the same just   faster and stronger and better. You you know, there is 1 interesting thing to talk about here. I use the c. Settrace, and 1 of my contributors, Alex Hall, we also gonna talk about him later, he found that there is a sort of optimization. Actually, it's not it's actually optimization, but a more lower level way in Python to control the tracing function, which is something called f   underscore trace.  

Now, he added it to the code and it worked. And it also it allowed using a PySnooper as a context manager   also. And I didn't know how it did that. That was magic.   So I googled it, and I assumed I'm gonna I would find information about it in the Python documentation, but I didn't.   It was some sort of magic that he found from   from inspecting a different debugger. So so after after we published his PR,   at some point, I thought, why not add this to the Python documentation so other people are gonna know about that? So I made the PR to Python,   and I've I've contributed to CPython in the past, but it was years ago.  

And recently CPython moved to GitHub, which is a which is a great move in my opinion. So this was my first time contributing to CPython   on the GitHub. So I kinda learned the new process. If you got a bunch of bot to do things for you, it's it's pretty cool.   And, I wrote the documentation that explains what F Trace is for other people who might want to use it. And I wrote it, and we did a few back and forth with the the developers there for a few weeks.   And, eventually, it it was accepted. So that was part of the documentation. So I'm happy to have this resulted in a contribution to CPython itself. Yeah. I'm gonna I'm gonna move on to your other question. You asked what kind of project are used by paper on that determined how it's evolved. The funny thing is the first time I've actually used by paper was well after   I published it and it became famous. So I didn't even use it at all when I published it. I know it's pretty funny. It's like it's like it's a thing I wanted,   and I never used it until then. But, yeah, I just always use a debugger, and I meant for fasten up to be as a sort of a last resort tool when you can't use a debugger. And I always use the debugger, so I was good. Now   I I did use it, like, I don't know, maybe 10 times   in in the last couple of months. And I used it at my job,   and I've also used it when   debugging   scripts on my debugger ring ID. So it's it's very useful for these kind of times where something is convoluted, and you can't. It's like   I I write scripts for my IDE, and my IDE runs them in some convoluted way. And I don't know how to debug the scripts that my that my IDE runs. So if if I push a pass number decorator in there and I tell it to write output to 5, I know for a fact that I'm gonna get output in that file. So that's what I really like about the passcode for 3 d. It gets the output out no matter how how difficult it is. So and and when I use it for the first time, it was pretty much exactly what I wanted. And the only thing I was missing was that it should output the   absolute path of the file every   time it it outputs the lines of trend. So I added that as a quick feature and made a new release for that new feature. But other than that, it worked exactly like I wanted it to, so I was happy about that.  



[00:23:40] Unknown:

And so you had mentioned that some of your motivation for creating PySnooper was to address some of the issues around using print or log based debugging   and to make it easier to be able to get the full context of the application   as you're trying to debug it without having to do all this back and forth. And I'm wondering if you can just talk through some of the   initial goals that you had for the project   and some of them that you had to either abandon because they were technically infeasible or not useful   or maybe some of the,   initial assumptions that you had about how it was going to be used or how it was going to work that didn't quite play out how you had anticipated as you got further into the weeds of implementing it and using it and promoting it? Well, that's it. That's gonna be a fun question to answer.  



[00:24:28] Unknown:

You know, I've been doing software development for a long time, and, I learned a long time ago that when you start a project, you have a certain vision. You imagine something about how this project is gonna be. And once you start working on it, and especially once you let users use it, you   you have to pivot. You have to understand things that you thought were easy, actually difficult,   and things you thought are not worth it are actually worth it. And you end up changing your project. I've I've kinda gotten used to it and taken it for granted. Now with this specific project,   it kind of all played out exactly the way I wanted to, which I I guess I guess, I'm lucky about that. I mean I mean, I thought about this idea in the shower. I thought that that'd be a cool thing to build, and I had a sort of vision for what it's gonna be. And then I built it, and it was exactly what I wanted it to be. So I guess that the 11 in 100   in a in a software development career. So so I published it, and I was very happy that it got so popular. This is I mean, I've written a bunch of open source projects in the past, and I did have 1 that reached a similar similar level popularity like this 1. But, but most of the time, it doesn't happen. So I was very happy that that that happened, and the and the the project pretty much turned out exactly how I want it to be. So I was very happy about that. And then in terms of the actual  

[00:25:42] Unknown:

implementation, what are some of the edge cases or technical challenges that you encountered while working on it, either in how Python itself operates or in how PySnooper  

[00:25:53] Unknown:

interacts with the program that you're debugging? Sure. So I said before that I really like to use the debugger when I program. Now the irony is I couldn't choose the debugger when I was programming PythonOper.   Why? Because as I said before both PythonOper and a debugger they use the system search function   that CPython provides. Now if you're if you're using a debugger, then it's   it's telling Python to use its tracing function, and then you use Pine Sloper, and it overrides the tracing function set by a debugger. So the minute you start to use Pineslooper,   your debugger stops working. I mean, assuming you you're debugging Pineslooper. So that that was really annoying to me. I wanted to debug Python, but when I was developing it, I wanted to see why it's not doing what I thought it should be doing. And I just couldn't do it with the debugger, so I ended up using print statements, which is, yeah, pretty ironic. And I also wanted to use code coverage tools for my test. We have an extensive   test suite, which I'm very proud of, and I wanted to use the coverage measurement measurement to see which lines are being covered. And I can't do that because   because coverage measurement, you also use a system of Tetris,   which gets overridden by pass number.  

So what I would like to see, a change I would like to see in CPython, is that instead of having just   just 1 kind 1 kind of tracing function that you'd be able to have an entire stack of training   The t, for example, I'm running a code coverage measurement tool which set its own tracer and then I use ice number. It's gonna set its own tracer on top of the previous true tracer without   overriding it. So both tracers are gonna be active at the same time. That could be a good subject for for a pet.   Personally, I think I think I floated the ideal Python ideas   and the Python ideas, mainly, I mean. And, there wasn't much enthusiasm, which I can understand because because the people who develop debuggers and code coverage tools are a pretty niche crowd. So they run at a lot of interest, but that could be a change in thick c Python, which will make it easier to write the binding tools. But, yeah, it will also add complexity to the project, so I'm not entirely sure it will it would be worth it. May maybe it will be worth it to make a branch of work of CPython.  



[00:27:50] Unknown:

It would be used only to develop the binding tools for actual Python. But, yeah, that's still a niche use case. Yeah. It's 1 of the challenges of having a community that's as broad in scope as Python is that, as you said, for something that's so niche where there's only a small handful of people that are doing it, that's a small percentage of the overall Python community, which is still a fair number of people, but in the context of the people who are trying to   drive the language forward and ensure that it meets all the needs of everybody who's using it. It's difficult to get the focus and attention and buy in necessary   for being able to push forward a feature that's going to be useful for such a small group of people. And so, yes, interesting how that plays out in terms of open source and community management. Yeah. That's right. When I was younger and and and more stupid, I I would suggest lots of things to pass on ideas, and most of them will be stupid.  



[00:28:43] Unknown:

And once in a while, I hit hit on something that was approved. But but most of the time, I would suggest things, and everybody will knock them down. And it's especially Guido who would knock my things down,   which is really funny for an inexperienced developer. I keep posting the idea and Guido says, no. This is not good because of this and that. And then after he says that,   like, 5 more people after he say, no. It's not a good idea because of this and that. And I'm like, thank you very much. Guido already implied I was an idiot. Now you guys   you guys are so grateful for helping you. But I know I know that   as fast training as it was for me, they were doing the good thing by making sure that   everything that goes into CPython is goes without scrutiny because it affects a lot of people. So I think they were doing the good thing even though it was frustrating to me. But they when people do make suggestions on Fastenal or try to improve it, I try to be as nice as I can be. Some people are doing really great job in their PRs and their issues,   and they're really easy to work with. Some people, I have to correct them again and again and and correct their mistakes and teach them about how to contribute to open source. But I was I always try to do it in the way that's most encouraging. If they did something good, I would compliment them with something good that they did. And I I would thank them for trying to contribute because I know that not everybody   not everybody has contributed to open source, and for some people, it's a very frightening  

[00:30:00] Unknown:

prospect. So so I try to be as welcoming to newcomers as possible. Yeah. Open source community management is definitely a challenging beast and 1 that has a lot of different aspects to it. And then there's also a lot of different aspects to actually   contributing to an open source ecosystem where part of it might be contributing to the actual project itself in the case that you're discussing as far as proposing these peps or wanting to implement additional functionality in the Python language, or it also might just be in terms of building on top of it as you've done with Py Snooper. And another case of that is a project that I found a little while ago called Snoop, which builds on top of your work and, from what I was able to tell, adds some additional features in terms   of user experience and maybe some additional functionality. And so I'm curious   what your initial reaction was if and when you found out about it and maybe some of the   ideas that you have incorporated back into PySnooper itself after playing around with some of the functionality that they added   and, just your overall reaction to that project and the fact that you gained enough popularity that somebody is building another layer on top of what you've done. Right. So it's not just somebody. The guy who makes snoop is called Alex Hall, and he's the the top contributor to Pass Number.  



[00:31:16] Unknown:

So he he's implemented a lot of features, fixed a lot of bug, and he also helped do a lot of reviews for other people with related features and bugs. So he's been the most helpful contributor I've had. Now, there were a bunch of these features that you wanted to add that I felt were, you know, were nice to have, but they were not worth the added complexity   and the long term maintenance burden and that they would be great. Now we had to go back and forth about it.   And he wanted he just wanted to implement them, so he made his own fork. So I'm happy I'm happy he made his fork.   So so he could do what he want without me being a bad guy and telling him he can't do that. Yeah. I'm a little bit unhappy that they call it snoop because I think it might be confusing to people. I mean, we use by Snoop, the decorator is called Snoop. So having he his project called Snoop could   cause confusion in between the projects. So I'm I'm unhappy about that, but, yeah, that's his decision. And I don't have a trademark on Snoop, so I can't really enforce that. And so I'm I'm happy that he has his fork. And I don't remember whether I merged anything from it. It's possible that he submitted PRs at the first, got into snoop, so I I can't recall any of these.  

I do recall looking at the project with me and seeing that there were a few nice to have features, but, but nothing that is really a huge difference. And I think I think, there is a mistake that developers make,   they're really into that thing.   I mean, if you're in I mean, we were both working on Python, and I think that developers kind of tend to be really into their thing and   not appreciate   how different the real world is. I know I'm putting it in a way that's kind of difficult to understand. But   for most people who use PySlooper,   they would never touch the more advanced features. Never.  

Only I mean, most people when they use code   I mean, this is something I learned when I started working in a big company. Like, for me and you, if we use package like Python or Requests or Django, it is very clear to us that we're using a package that someone wrote, and it has features, and it has documentation. You can read about them. It has all kinds of things that you can use. But most people most programmers   don't really care about that. I mean, if they use PythonOperator because they saw some other developer use PyTorch, so they copy pasted his code   and do their code until the and ticked it until it worked, and then it worked. And they won't even bother to check the more advanced features. So the people who use the more advanced features, they are kind of a minority. So I'm not saying not not have advanced features. But realize that their benefit  

[00:33:40] Unknown:

compared with the maintenance burden isn't that great. That's what I'm saying. Yeah. 1 of the most important and most difficult jobs of anybody who's maintaining a project is understanding when to say no to a given feature. 1 of my favorite sayings is that open source is free as in puppy where somebody's can give you a puppy for free, but then you're on the hook to actually take care of it and make sure that it's well fed fed and well maintained throughout the entire rest of its life. Oh, that that's a nice metaphor. I never had that 1 before. So as you alluded to before a couple of times, 1 of the notable aspects of your work on Pie Snooper is that shortly after you launched it, it garnered a lot of attention   broadly   within various communities. And so I'm wondering if you can just talk about some of the strategy that you   used   to gain that level of popularity   and a little bit about how that visibility has affected the long term popularity and use of PySnooper   and maybe some of the future direction  

[00:34:36] Unknown:

of where you're planning on taking it? Right. So I'm guessing that many of our listeners might be interested in writing their own open source projects and maybe maybe   get them to be very popular. So I'm gonna share what I did and what I think about what contributed to popularity.   So the the first thing that's very important, you gotta have a read me. It really explains what your project is about and what kind of needs it's fulfilling. Now, it's going to have a good with me, a good tagline for for your report. And to understand, this is now a marketing effort. You now have to convince the user in, like, I don't know, 20 seconds or 30 seconds that you've got something interesting.   Show him something cool.  

When someone can I mentioned this in the previous census? People are less interested in your project   than you are.   And this means that if you're if they're going to your project top page, they have, like, 20 seconds or 30 seconds of attention. They are bored. They wanna do something else, but they're thinking maybe maybe you could entertain them. So it's really it's really important to get your message across. So with Spice number, I kind of described the tagline was, Spice number, never use print for debugging again, which I think kind of hits it home. I mean, people can relate to that. And I and I kind of made made the explanation in the reading of you're trying to debug and you wanna know the values of the variables. But getting a debugger is too difficult because you're in a corporate environment. And I I kinda described the exact scenario that I've seen play out a thousand times. And then I and then I had sort of a quick start example of how to use it instead of linking to to a documentation   and having someone go through a a complicated tutorial. So it's really important to to get that reading straight so people could be interested to try to learn more. So that's 1 aspect. The other aspect you gotta do is solve a launch. I mean, when you make it public, should they post to hacking news and post to Reddit and post to wherever ads you can think of. You should get a few of your friends to upload it in the first few minutes while you when you're posting. It's important. I would say this is the only kind of dark pattern that you I mean, it's kind of cheating, but you only need, like, 3 or 4 people to get to the front page to for a few minutes. And once you've got that, you're not really cheating anymore because if people like it on the front page, they will upload it. If they don't, they won't. At that point, it depends on how likeable your project is. With Pice Snover, you got to you got more points on hacking use. You got to them to be number 1 there. So lots of people started on GitHub. And once lots of people started on GitHub, then it got to the trending page on GitHub and then more people found it through there and more people started. And then it got to be number 1 on this on the trending page of GitHub, and then it's really exploded. I got lots of stars and contributors. I got the   I think at this point, I'm almost reaching 13, 000 stars, which is insanely   unproportional   to to to the project. Because I think, like, projects like CPython or or NumPy have, like, I don't know, like, I think, 30, 000 stars. So, obviously, my project is not,   half as complicated or half as, whatever as CPython.  

So I guess I was just just lucky to get all that attention. I you asked how it influenced the project. Well, I got lots of contributors.   Lots of lots of them have been easy to work with, like like Alex Hall with all super helpful pull requests and issues.   Lots of people, like, submitted issues for stuff that didn't work for them, and, obviously,   they obviously, no no 1 ever yelled at them before for not including information in their issues. So I would I would get just, like, just random, like, for for example, today or I think yesterday, just someone just said, hey. I got this exception using pipe number, And he just he just put it in with no context about what he was doing. And Alex and I had to kind of question him until he gave us the right information.   So so lots of people,   like, submit issues,   in a way that has no information.  

And   and, also, in the last few years, lots of people in Asia got into open source. So there's lots of people who just just   create issues in Chinese, just writing in Chinese   without a thought about whether I speak Chinese, and I don't.   So so that's that's been pretty funny. You know, as long as I'm thinking about these kind of issues,   when I started doing open source development,   mailing lists were a big thing. Like, every project had a mailing list, and I I personally hated it because it was weird that you had to subscribe to something to ask a question.   And so now there is no many list for Python support as there is in 1 for most projects.  

And so,   people just ask questions on the issue board,   but they but they we're having fun and I've got lots of contributors. I think I've got around 20 contributors.   So the popularity   improved improved the project, all the contributors came in and improved and polished the project. So so I'm very happy about how it turned out. And overall, what have been some of the most interesting or unexpected or challenging aspects  

[00:39:10] Unknown:

of creating and maintaining and promoting PySnooper   and maybe some of the,   opportunities that you've gained as a result of its popularity?  

[00:39:18] Unknown:

Yeah. I've had fun speaking about it in a few conferences.   To to be honest, I I think I mentioned most of the interesting or unexpected things above that I had to say, but   I I had fun speaking about it in a few conferences at Pyolovael   and Python Israel   and the in Python in the Czech Republic and also   in in Geopython in, Brazil, Switzerland.   So that's been fun.   And that's pretty much it. I I try to,   you know, I've been very tempted to, to maybe add more features and more fancy stuff. But I think that the strength of ice hopper   is that it's it's so,   small and unambitious.  

So   I I try to keep it small because if I if I start adding more ambitious features to it, I think it is gonna ruin the project.   So so that's pretty much for me. I mean, as far as I as far as I'm concerned, the only thing I'm interested with regarding this project is a long term maintenance and reliability.   Make sure it fits with the new Python version.   Once in a while, someone submits a feature request that it actually happened a week or 2 ago,   and, and I work on it. But mostly, I just the project is going along fine, and I don't tend to change it or to add any major features at any point. Are there any other aspects of your work on PySnooper  

[00:40:31] Unknown:

or the state of the art in debugging within the Python ecosystem or anything else related to your work that we didn't discuss yet that you'd like to cover before we close out the show? Yeah. Sure. And 1 thing I say, which is actually not my work,  

[00:40:44] Unknown:

you've mentioned PDB early in the episode.   Now I hate PDB with a passion. I hate it so much.   I I remember using it, like, 1 once or twice when I was starting my career, and I wanted to use q as a variable name just because sometimes they use random letters as a variable name when I'm debugging. And when I type q, it immediately x. And I was so angry when I never used PDB again. I was very angry about that. But I learned there is a different program called pudb,   which is a sort of similar to pudb as in it as in it's a debugger that you use in a shell, but it has a sort of graphical   interface in the shell, which just kind of reminds you of the bone and c interface from the past. It's really cool. It's basically like PDB, except it actually shows you your code. It has sort of a graphical interface, but it's still in the shell and doesn't require a GUI. It's like in Windows or whatever. So that's a pretty fun tool.  

Once in a while, when I get shared access to something and I wanted to bug it, I use QDB. So that that that's a pretty fun tool. I think I think everybody should at least give it a try because it's been useful to me, you know, so many times. Yeah. There's another project along the same lines that's called pdb plus plus or pdbpp  

[00:41:51] Unknown:

that will transparently work in place of anywhere you use pdb   for the built in module and just adds a lot of niceties   in terms of syntax highlighting and better visibility   of the   code that you're currently within the context of, but it's not quite as full featured as p u d b. So it's sort of a step in between. But there's definitely a lot of value to be had in terms of   understanding   how to use a traditional debugger for being able to step through the code and then being able to have things like your work on PySnooper available for being able to just quickly get a good overview of what is the overall logic flow and the variables that are there. So understanding what all is in the tool chest and when it's appropriate to use it is definitely an important skill for any engineer.   Right. Alright.  

So for anybody who wants to get in touch with you and follow along with the work that you're doing, I'll have you add your preferred contact information to the show notes. And with that, I'll move us into the pics. And this week I'm going to choose PyCon US. It's the largest PyCon event. It's the 1 that I've been to a number of times and plan to be at again for the spring.   So, they actually have the call for proposals is still open until December 20th if you're listening to this before then, and registration has been opened. So, I hope to see a lot of people there again.  

[00:43:07] Unknown:

And with that, I'll pass it to you, Ram. Do you have any picks this week? Actually, I wanna say something about your pick. I submitted the talk proposal to to back back on US. Really hope it gets accepted. So a talk I gave in Israel about the live calming and music synthesizer. So it's a really fun talk. So I really hope they can accept me. So I I hope that I'm gonna see you at Pittsburgh.   Regarding my pick, I'm I I would like to talk about something that is completely unrelated to programming, but I think it's a it's a fun thing that people should know about. It's called nonviolent communication.   Sort of a school of thought about expressing your emotions and interacting with other people.  

And, like, around 15 years ago, a friend of mine sent me a YouTube video of this guy explaining a non violent communication why what's what it's about. And that kind of video, yeah, kind of blew my mind and really changed the way I interact with people. If if if I put it in 1 sentence, it's about   expressing your emotions in a way that is completely   free of judgment and separated from your thoughts. Learning how to listen to your emotions in a way that's separate from your thoughts. You You you get to learn a lot of things when you're doing that. So I'm gonna link to the YouTube video   in the show notes, and hopefully, it's gonna blow your mind as much as it blew mine.  



[00:44:20] Unknown:

Alright. Well, thank you very much for taking the time today to join me and discuss your work on PySnooper and some of your experience   of   helping to grow the community around it and your experience of promoting and marketing an open source project. So it's it's definitely an interesting tool and 1 that I am sure I'll have cause to use in the near future. So thank you for all of your efforts on that front, and I hope you enjoy the rest of your day. Thank you very much for having me on, Tobias. It's been a pleasure.  

[00:44:49] Unknown:

Thank you for listening. Don't forget to check out our other show, the Data Engineering podcast at dataengineeringpodcast.com   for the latest on modern data management.   And visit the site at pythonpodcastdot   com to subscribe to the show, sign up for the mailing list, and read the show notes.   And if you've learned something or tried out a project from the show, then tell us about it. Email host at podcastinit.com   with your story.   To help other people find the show, please leave a review on Itunes and tell your friends and coworkers.