Summary
Python is an intuitive and flexible language, but that versatility can also lead to problematic designs if you’re not careful. Nikita Sobolev is the CTO of Wemake Services where he works on open source projects that encourage clean coding practices and maintainable architectures. In this episode he discusses his work on the DRY Python set of libraries and how they provide an accessible interface to functional programming patterns while maintaining an idiomatic Python interface. He also shares the story behind the wemake Python styleguide plugin for Flake8 and the benefits of strict linting rules to engender good development habits. This was a great conversation about useful practices to build software that will be easy and fun to work on.
Announcements
- Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
- When you’re ready to launch your next app or want to try a project you hear about on the show, you’ll need somewhere to deploy it, so take a look at our friends over at Linode. With the launch of their managed Kubernetes platform it’s easy to get started with the next generation of deployment and scaling, powered by the battle tested Linode platform, including simple pricing, node balancers, 40Gbit networking, dedicated CPU and GPU instances, and worldwide data centers. Go to pythonpodcast.com/linode and get a $60 credit to try out a Kubernetes cluster of your own. And don’t forget to thank them for their continued support of this show!
- This portion of Python Podcast is brought to you by Datadog. Do you have an app in production that is slower than you like? Is its performance all over the place (sometimes fast, sometimes slow)? Do you know why? With Datadog, you will. You can troubleshoot your app’s performance with Datadog’s end-to-end tracing and in one click correlate those Python traces with related logs and metrics. Use their detailed flame graphs to identify bottlenecks and latency in that app of yours. Start tracking the performance of your apps with a free trial at datadog.com/pythonpodcast. If you sign up for a trial and install the agent, Datadog will send you a free t-shirt.
- You listen to this show to learn and stay up to date with the ways that Python is being used, including the latest in machine learning and data analysis. For more opportunities to stay up to date, gain new skills, and learn from your peers there are a growing number of virtual events that you can attend from the comfort and safety of your home. Go to pythonpodcast.com/conferences to check out the upcoming events being offered by our partners and get registered today!
- Your host as usual is Tobias Macey and today I’m interviewing Nikita Sobolev about his work with DRY Python and Wemake Services
Interview
- Introductions
- How did you get introduced to Python?
- Can you start by sharing your overarching philosophies or design aesthetics for writing maintainable software?
- What is your process for starting a new project, beginning at the design phase?
- What are some of the challenges or shortcomings that you see in the "default" way that most developers write Python?
- What is DRY Python is and how does it help in addressing those concerns?
- What was your motivation for creating these projects?
- There are a number of different projects that are being built under the DRY Python umbrella. Can you list the ones that are currently active and outline how they fit together?
- What are some of the initial challenges that newcomers to the DRY Python libraries encounter?
- How do you approach the design of the API and developer experience to make these development approaches more accessible?
- What have you seen in terms of real world impact on the maintainability and extensibility of projects that you have built on top of the DRY Python components?
- In addition to DRY Python you are also involved with development of the wemake-python-styleguide. Can you describe that projects goal and how it got started?
- If you make the linting too restrictive then developers are likely to just ignore or disable it. What have you found to be the right balance to which rules will fail a build and which are just informational?
- Why do you push the responsibility for things like formatting onto the developer, rather than an autoformatter such as YAPF or Black?
- What are some of the other supporting technologies that you rely on during your development workflow?
- What are some of the elements that you think are missing in the common toolbox for Python developers?
- What tools are we lacking entirely?
- What are the cases where DRY Python is the wrong choice?
- What are your goals and plans for the future of DRY Python and the various Wemake libraries?
Keep In Touch
Picks
- Tobias
- Nikita
Closing Announcements
- Thank you for listening! Don’t forget to check out our other show, the Data Engineering Podcast for the latest on modern data management.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@podcastinit.com) with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers
- Join the community in the new Zulip chat workspace at pythonpodcast.com/chat
Links
- DRY Python
- Wemake Services
- wemake-python-styleguide
- Turbogears 2
- Dotenv Linter
- Returns
- Wemake Python Package Cookiecutter Template
- Test Driven Development
- Requirements Analysis
- RESTs
- Django Rest Framework
- Classes
- Monads
- Functors
- Scala
- Kotlin
- Haskell
- Punq dependency injection library
- Flake8
- Wemake Django Template
- Flake8 Baseline
- isort
- Nitpick
- Mypy
- Darglint
- Poetry
- Pip Dependency Resolver
- Hypothesis
- Schemathesis
- Pytest Auto Hypothesis
- Typescript
- Rust
- Elixir
- Zio Scala
- GitHub Sponsors
- Do Not Log blog post
The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA
Hello, and welcome to podcast dot in it, the podcast about Python and the people who make it great. When you're ready to launch your next app or want to try out a project you hear about on the show, you'll need somewhere to deploy it. So take a look at our friends over at Linode. With the launch of their managed Kubernetes platform, it's easy to get started with the next generation of deployment and scaling powered by the battle tested Linode platform, including simple pricing, node balancers, 40 gigabit networking, dedicated CPU and GPU instances, and worldwide data centers. Go to python podcast.com/linode today. That's l I n o d e, and get a $60 credit to try out our Kubernetes cluster of your own. And don't forget to thank them for their continued support of this show. You listen to this show to learn and stay up to date with the ways that Python is being used, including the latest in machine learning and data analysis.
For more opportunities to stay up to date, gain new skills, and learn from your peers, there are a growing number of virtual events that you can attend from the comfort and safety of your own home. Go to python podcast.com/conferences to check out the upcoming events being offered by our partners and get registered today. Your host as usual is Macy. And today, I'm interviewing Nikita Sobolev about his work with dry Python and WEMAK Services. So Nikita, can you start by introducing yourself? Hello. My name is Nikit Sabadev, and I am now CTO of we make services and core developer of dry Python. And do you remember how you first got introduced to Python?
[00:01:33] Unknown:
Yeah. It was in university. I was doing a side project, and I was rewriting some turbo gears 2 to Django. It was like 2010.
[00:01:44] Unknown:
And in terms of your overall approach to development, I know that you have a lot of opinions and a lot of structure and rigor to the way that you do your development, which is expressed in the open source work that you do and the high quality blog posts that you put out. So can you just start by sharing your overarching philosophies and design aesthetics for how you approach the writing of maintainable software?
[00:02:09] Unknown:
Oh, that's a really hard question, because there are a lot of connected topics I want to talk about. And, I guess I should start about speaking a bit of my own company and how we do things inside. There is a philosophy that's called a blame blameless environment. And, the main idea of this approach is to eliminate any human specific problems, and to move them to technical problems. So we don't blame people for mistake they do, and start blaming technology for this unfortunate event. And, as a part of this philosophy, we make a lot of tools that prevents errors from happening. I can give you some examples. For example, we had a really strange bug on our production server.
And I spent, like, 3 or 4 hours debugging what is going on. With no luck, I was really lost and no idea what to do. And then, by lack of chance, I found that I have a typo in my dotenv file. I had 1 key and right in the end of the file, I had this key written for another value. So we came up with dotenvlinter. To the linter that links your .env files for a lot of mistakes, bad, usages, and so. So we eliminated this problem at all. This problem cannot be repeated. Just, just don't work. So and that's how we create our new tools. And that's the main motivation for doing these tools. And then we can continue, speaking about each particular tool individually and in the whole picture in general. So Dry Python was made as an answer to the architectural question. How should you structure your code? How can you describe your contracts on functional level, on class levels, and so on? How can you easily express your business logic?
How can you switch some parts of your programs without rewriting the business logic and so on? So we took several steps in achieving this. And 1 step was, creating the returns returns library. It is inspired by the functional programming in general. And, it's way to express, things in Python, like to still be, Pythonic, but with some, functional programming, Shiny. It's kind of a mix. And, from this point, this whole journey started with the dry Python and, I'll be happy to answer you more specific questions about it. And so, in terms of your overall approach
[00:04:48] Unknown:
to starting a new project, what is your first step as far as approaching the overall design and architecture of the system and some of the guardrails and scaffolding that you do to ensure that you can generate a successful project and something that will survive the test of time and the changing landscape of technologies that you need to be able to
[00:05:11] Unknown:
run and interact with? Oh, I don't love mobile apprised and I love templates. So we make Python package templates. It is a tool to bootstrap in your projects, with the most more than and the most useful tools we use. So it's open source. You can try it for your next project. And the second part is to think about what architecture do you want to see in different ways. We use a lot of test driven development to prototype your APIs. We use a lot of, requirement analysis even for open source projects because there are still requirements. So why should we treat them differently? And that's probably it. That's the 2 most important parts here. Good technology and good requirements.
[00:05:57] Unknown:
And then as far as the overall challenges or shortcomings that you experience in the quote unquote default way that most developers write Python. I'm wondering what are some of the notable elements and some of the reasons that you tend to take a more functional or domain driven approach to building your projects?
[00:06:17] Unknown:
That's a very controversial topic right now because I see a lot of, ongoing suites of discussions about how, Python used to be and how it should be in the future. And I don't seem to agree with these points because from my point of view, Python was never actually a simple script language. It was always language to create complex applications. And there is a whole idea that you start something in Python prototype, it and release it to the market and then you rewrite it. No, it doesn't work this way. When you start prototyping Python, you keep Python for a long time, like for for several years, at least. So when you write Python code, you have to think about the growth of your project, the ongoing opportunities to refactoring, the new features you want to introduce somewhere unknowingly in the future and stuff like this. So, when we talk about, regular Python issues that I don't like, there are several of them. The first 1 is obviously the lack of typing. A lot of Python developers still don't understand what typing is and how it works. And, this is something I try to fight and try to persuade people that typing is a good thing. It does not harm your code. It makes it more reliable, more readable, and more stable. So, there is nothing wrong with typing. And I always give an example that you always know types that you're operating with. So when you just make them explicit, that's a good thing. There is nothing wrong with making something explicit. And the second thing that I don't like how Python developers, do is the architecture. Because a lot of tutorials, learning materials promotes very limited way of writing business logic, of writing some infrastructure code and stuff like this. The most known example of how things should not be done is, logic, you know, views or your models in Django. So that that's something that we historically have, but I don't believe that we should keep this idea. And then
[00:08:27] Unknown:
the dry Python project you mentioned starts with the returns package, but there are a number of other projects that fit together in that overarching umbrella. So I'm wondering how the dry Python set of libraries helps to address some of the concerns and shortcomings in how some people will architect their projects
[00:08:50] Unknown:
and how it contributes to building more maintainable and more scalable systems. Right now, it is, on the very early stage of its life. And returns library is the only library that should be used right now. So I want to make it clear from the beginning. And, returns just provides a bunch of primitives to work with your code and to, make some other things explicit. Like, for me, it's very strange when we don't have an opportunity to know if this function rises an exception or not. Returns make makes this simply explicit. So you should write, this function, returns this value and can write this exception.
It's very convenient because after that, you just start to see your patterns in your code and the flow, through different files, different classes and so on. And it also allows to enforce your AO contract because sometimes, a lot of people do, input output operations in very, very deep places and you just cannot switch this code easily with some other code. So we want to know where our IO happens and where it does not happen. So that's very important as well. The next library I want to talk about is REST. It's not released yet, but, it's going to replace Django REST framework kindly. It is another approach of working with HTTP REST APIs. And it has some explicit place to put your business logic in. Because right now, it's not really clear when and where to put your business logic. Should you go to serializers, cure sets, or maybe, your view sets. But, when we talk about, developing some new stuff, we have to think about it and we have to make some explicit places to put your business logic. So it will be clear,
[00:10:44] Unknown:
when and how to do it. And I know that you've also got some other projects in the form of classes as 1 of the ones that I was looking at and a couple of others that serve to expand that overall landscape. I'm wondering what your vision is for once these different projects are more fully can help other users of those set of libraries. Colossus is the way that it can help other users of those set of libraries? Classes is another project that is going to be publicly available soon
[00:11:17] Unknown:
for the production use. I know there are some people that already use it. So it's okay, but I don't recommend it right now. So it's kind of alpha version. So use it on your own risk. The thing about classes is that, it provides some ad hoc polymorphism for different Python structures. So you can create functions, for example, to JSON or from JSON that behaves differently based on what type do you provide. It's kind of single dispatch, but it is more strictly, typed. And you can now types, when running my py. Because right now, single dispatch is not typed. At all. So we just have no type information here in mypy check. And for developers who are first picking up the dry Python set of libraries,
[00:12:06] Unknown:
what have you found to be some of the initial challenges or points of confusion that they experience? And what have you found to be some useful resources or references for helping them pass those initial hurdles and to really appreciate the capabilities that they provide.
[00:12:23] Unknown:
Actually, it was the whole point of creating, dry Python returns in the first place because I wanted to hide all the complex to behind some really useful and simple abstractions. There's really no bad known functional jargon that people ask carry off. Like, what is monad, functor, and so on. We don't have these things in our returns library. You just don't have any references of monad and, in the monad library. So it's kind of strange, but, we think that a lot of developers should just understand, how easy it is to actually use these things without deep understanding of the modern concepts and mathematician concepts behind it. So I think that it does this job pretty well. We have very friendly user documentation that a lot of people really find helpful.
And, I'm grateful to a lot of people who viewed it, asked questions, proposed their suggestions. So thank you guys. You made this possible. But on the other hand, we still have to find some, attitude to this approach. Because a lot of people say that that's not Pythonic and that's not something Python developers should do. And, I don't have an answer for this because I don't think that there is something Pythonic or not, but not Pythonic. When Python was really simple and, not really popular, like 12 or 20 years ago. It might be some default ways of doing stuff, but right now Python is so huge and it used for millions of developers and, a lot of domain areas and a lot of our infrastructure areas. So there is no such thing as Pythonic anymore. So a lot of people do write Python differently. For example, if you look at Python code that is written by a web developer and a data scientist, that's completely different code. You just can't compare them. They're different. And so the same way I think about our own business code that we write. Because I believe that we should write business code that does not break and is easy to change. And when we talk about functional approach and, type safe functional programming, that's exactly what we're talking about. We can easily add steps. We can easily remove steps. We can easily type check the whole thing, and it doesn't add any significant mind complexity to this. So just cognitive complexity. You just write the same codes basically, and you compose well and you see the pipeline of what are you doing. And I think that's a great approach and a lot of people should
[00:15:09] Unknown:
try it. Yeah. I think some of the hang up of whether or not code is Pythonic or not is related to the fact that a lot of people attach their identity to the code that they write and they forget that the software in the end doesn't really matter because nobody cares about the lines of code. They care about the effect that it can produce and the value that it can provide to a business or an organization or an individual. And so whether something is exactly at 80 lines of length or whether you structure your if clauses in a certain way versus using chained function calls is irrelevant at the end of the day. And so I think that people definitely need to be a bit more open minded in terms of exploring different styles of structuring their code and different approaches to programming methodologies, whether that's object oriented or functional or composability
[00:15:59] Unknown:
or anything along these sort of myriad ways that we can approach the structuring of our logic. Exactly. Oh, 1 more thing about it. If you learn functional approach, you can read almost any other language that user said. You can read Scala, Kotlin, with Arrow. You can read Haskell with a bit of, syntax learning and so on. So that's just a great approach to learn something new and take a new skill for yourself. And the other element to
[00:16:29] Unknown:
the work that you're doing on the dry Python libraries and something that we alluded to already is the API design and the developer experience. And I'm wondering what your approach has been to make those libraries approachable for people who are coming from the more object oriented or procedural approach to Python projects and some of the ways that you structure the examples or documentation to give an easy on ramp to people to give them away to be effective and comfortable with those libraries and start to pick them up and run with them on their own? I try to showcase different problems
[00:17:08] Unknown:
and to show how you can solve them using our tools. For example, when we talk about returns, it solves a lot of different problems. Can, as I already said, show what code is pure and, which code is impure. You can replace exceptions with a primitive we provide called result. You can structure your async code different way using our new feature called future. You can use it for dependency injection using our requires context primitive and so on. And, I try to write about problems and to to highlight this solution. Like, you can also use this and solve this problem with, this approach. It is easy. It is is type safe. You can check it, during the,
[00:17:53] Unknown:
MyPy run. So make your own decision. Particularly, when you bring up dependency injection, that's something where a lot of Python developers will start to have an automatic reaction because they think that that is too, sort of enterprise or akin to how you would approach things in Java. And I'm wondering how you address those types of concerns where people just have an automatic reaction to the concepts that are being introduced because they associate them too much with some other type of programming language or approach to building software that they don't feel is native to the Python experience. I guess it is, the same approach I take for functional jargon. So I just try not to use these words explicitly
[00:18:37] Unknown:
until a person realized that actually, dependency injection is just passing arguments to functions. So there is nothing enterprise of Java in itself. You can just do it in any language.
[00:18:48] Unknown:
Yeah. Dependency injection in particular, I think gets a bad reputation because of the fact that there have been a number of attempts at creating libraries in Python for performing it that are just direct ports of the way that Java libraries are are written. And so it just becomes very cumbersome to try to use it and people will get burned by that experience, not realizing that there are much simpler ways to do it in Python that are still as effective in terms of getting the desired results of separating the context of execution from the specific implementations.
So I definitely appreciate the work that you've done in terms of making that approachable. But you have definitely seen libraries that are just a direct port of the Java syntax, and it's hideous to try and work with. Thank you.
[00:19:37] Unknown:
I also want to highlight when 1 of my most favorite libraries to work with dependency injection, it's called punk. And it is written by Bob the mighty. And I really like I recommend, the materials they write,
[00:19:51] Unknown:
blog posts they do, and all the library stuff they open source. This is a great value. I'll definitely have to take a look at that 1. And then in terms of the real world impact that you have seen on the maintainability and extensibility of the projects that you've built using these more functional approaches and the dry Python components and using domain driven design. How has that impacted the ability for you to deliver software in a fast and effective manner?
[00:20:23] Unknown:
I I I don't have any numbers to show, but, I can share my own experience and my emotions about it. Since I started to write our business code in functional ways and returns, I almost don't have any logical bugs in our business code. There are still bugs, like, bad requirements, some kind of Python specific ones. But we try to structure our logic, the same way we have our requirements done. So it kind of 1 to 1 mapping. So it's really easy to read, understand, comprehend, and to extend. So we don't have any problems with this part. And that's a great thing because we can now concentrate on something else and just say it's good enough, it works, and improve something something else.
[00:21:15] Unknown:
This portion of podcast.init is brought to you by Datadog. Do you have an app in production that is slower than you like? Is its performance all over the place, sometimes fast and sometimes slow? Do you know why? With Datadog, you will. You can troubleshoot your app's performance with Datadog's end to end tracing and, in 1 click, correlate those Python traces with related logs and metrics. Use their detailed flame graphs to identify bottlenecks and latency in that app of yours. Start tracking the performance of your apps with a free trial at python podcast.com /Datadog.
If you sign up for a trial and install the agent, Datadog will send you a free t shirt to keep you comfortable while you keep track of your apps. And then in addition to the work that you're doing on dry Python, you, as you said, also help run the we make services agency. And as part of that, you mentioned the cookie cutter template for Python packages. But another project that I've been using recently and really appreciate is the we make Python style guide. I'm wondering if you can give a bit of detail as to what that project is and your motivation for creating it. It's a really, really interesting history because I work, in a Russian university as a professor. So I,
[00:22:30] Unknown:
teach a lot of students, and I use Python as a main language to teach them different concepts, testing general programming, Python itself and so on. And I see a lot of bad code for the newcomers. And that's okay. I'm here to teach them way, they should write Python code. But I don't want to conduct code reviews and to write comments on each line of, the submission. And I want to automate this stuff. And that's where this ID comes from. To create a very strict linter that finds all possible mistakes in in a very, very bad code. And after that, I decided that we also want to use the same approach in our company as well. Because we want to standardize all the things. Not just sound things like Flaky does or like, Pepe does, but all the things. And, it serves a great value to us because we don't have any, kind of, for discussions about your coding style, about, should this be a problem or not. We have a strict policy. If Linter says that this is a violation, you should remove it or refactor it or do whatever you want. But, if Linter does not say anything, that's not a problem. And, if you still think that, this can be improved, you open a new issue in the repository file and say, like, I found this case. I don't think it's correct. It should be fixed like like this. You should work in this feature. We say, okay, and so fine. We will do it. And, in some time in the future, it will become,
[00:24:06] Unknown:
a new rule and you will have to obey it. Like, this how it works. And there's a very extensive set of rules that cover a broad set of categories of development styles and potential sources of bugs. And I know that it leans heavily on flake8 as the foundation with a number of other plugins and libraries incorporated. I'm wondering how long it's been in development and some of the, potential sort of contention that has come up in the process of adding new linting rules and defaults to how it operates?
[00:24:39] Unknown:
I guess it is, 3 years in the development right now. And it started as a kind of small set of rules and small set of plugins. But right now, I can say that it has the most rules and plugins in all Python static analysis family. Like, I guess, PyLink has less than we do, but, I don't have any numbers to show up so you have to take it on my word. There are several important milestones in the development that I want to talk about. The first 1 was the initial release because, some time ago, I considered that our company should have some something hidden. And, then I realized that, no, we should open source everything we do and be open source by default. But it was a hard decision to make some time ago. The the first important milestone. The second 1 is when we realized that this project is going to be, popular among other developers, that it is not our internal tool. And it is something that other people want. And we decided to build great documentation, about why we disallow this thing and why you should do differently and stuff like this. And it really gained a lot of attention during the October 1st, seasons. Because a lot of pipe, Python people decided to work on it. And we had, like, 20 or 30 contri new contributors in a week. So it's really, really, really great. Thank you guys for that. And, the 3rd important milestone is that it is bundled logically with different things we do. Like, we make Django templates, the template for Django applications, returns because they do complement each other and stuff like this. And when some person finds, for example, returns and they start asking questions. For example, how can I make this pipeline not long enough? Like, and to make this pipeline readable. I just give them a link like you should use this, starting analysis too, and it will be good. But they say, oh, okay. Fine. And, I also want to highlight 1 very important feature we're working on right now. It's called Baseline and it's addresses the most important problem with Static Analysis in any language, not just in Python. It is that you already have some code basically, and you want to, introduce some new tool into this old codes, possibly written in a different style, possibly with some bugs, complex, structures and so on.
And you do want to add it very easily. You don't want to refactor all your projects. You don't want to configure it for like a week. Just want to install it, run it once and to go from there. That's where baseline comes in handy. It is a concept when you remember all the bad parts, all the violation you have, all the bad parts of the code you have, write them into some database and then report on a new violations you have. Because all the violation are still there. You just cannot work on them because you don't have any time or you don't want to, stuff like this. But you want to find new ones In new code, you don't want to have this ugly patterns you had in in the past. So we're going to release it, I guess, this year. And, it turned out to be a very difficult feature because I thought initially that it is going to it was going to be released in March, but it is July right now.
It's still work in progress. So there are a lot of things to consider before releasing in public. Yeah. Delivery
[00:28:14] Unknown:
estimates are definitely 1 of the things that software developers have historically been very bad at. So I can commiserate with the sense that you wanted to release it 2 months ago and it's not quite ready yet. So and then to your point too about the documentation to it that say, this is why we think this rule is necessary. This is the approach that you can take to address it. And then, of course, being a flake 8 plug in, it means that you can disable the rules that you disagree with or you don't want to enforce for your own code base. But, and then your point about baseline, it somewhat addresses my next question, which is with linting, if you make it too restrictive, then chances are that developers are likely to either just disable it entirely or ignore it. And so I'm curious how you approach the balance of which rules you specify are going to fail the build and which are just going to be informational and serve as guidance for people who are writing the code. I don't Because if
[00:29:20] Unknown:
you choose the strictest Python linter, Because if you choose the strictest Python linter, you already know what you are doing. It just not like, some random plugin that has, like, 3 rules you want and 1 rule you don't want. It is a whole mindset, you install into your project. And, possibly, you can disable some rules that don't think this particular project. For example, you want to use a different naming button because that's how you do things in your company. It's okay. But, in most parts, these rules are complementary.
And if you disable something, it not going to be, reported, correctly in some other room. I'll give you an example. We have this concept built into our links are called, complexity waterfall. It's when you start with, analyzing, complexity of simple expressions and, then functions, and then methods, classes, models, packages, and so on. And when you find some different spots inside the function, you can report that this function is too complex. You have to refactor it into 2 functions or something like this. And if you switch some of complexity rules, this function is not going to be reported properly as a complex 1. So you don't have to refactor into 2 functions and, you break this complexity of the 4.
And, I guess that's not a reason why people use this software. So but I know that, there are some developers who disable just everything. Like they have this installed for some reason, but just, ignores all the codes from the VPS.
[00:31:04] Unknown:
Okay. I'll do that. Yeah. I've been using it for a couple of greenfield projects recently, and I keep most of them turned on, but there are a couple of them that I personally disagree with. Like, I think 1 of them is suggesting to not use f strings anywhere where I do personally find them useful.
[00:31:21] Unknown:
So I knew it. I I could tell you have strong feelings. Go ahead. I knew it because that's the most controversial we will have. So what what's your personal take on f strings? I'm kind of okay with them in other languages like JavaScript, for example. But in Python, I don't think that they're really required because we have, format method. We have old style formatting and just 1 more. I don't think that that's a reasonable feature to add into language. So but who am I? Just a user, not a developer.
[00:31:52] Unknown:
Fair enough. And then the other thing that is notable about the style guide is there's a section that talks about auto formatting and it has a sort of philosophical bent to it that you don't think that in general, they're a very good idea and that the responsibility of making sure that the code is formatted appropriately should lie with the developer. And so I'm wondering why you feel that auto formatter such as YAPF or black are sort of doing a disservice to the developer and why that responsibility should lie with the person writing the code and just following the guidance of the linter?
[00:32:32] Unknown:
I actually think that these tools are great because I use them a lot. I don't use, Yap. I use black for some code that I already have, and I know that's it it is ugly. For some reason, it is a legacy project, I have to work with, in the code I wrote myself 10 years ago and stuff like that. Yeah. I can use these tools once. I take some project, I run after Formatter and, I begin to check with everything else. What's the same way? Like, say it like this. But, I don't think that, active formatives are used correctly. Because developers tend to use it on the code they write right now. For example, they write some codes, it is ugly, it is not formatted properly.
And they are just trying and not formatted to fix it for them. I believe that, it is not a good approach because when you care about the code and you think about it and you have to pay attention to it, you have to pay attention for all the details. And just ignoring some kind of style issues, is not a proper way to work with the code. And you should try to automate this thing with your hands. Like, if you're trained properly to write beautiful code, you always do write beautiful code. You don't have to write an auto format and to run it every time. You just have to memorize with your fingers how to do it. And then there are a number of other supporting technologies that are
[00:34:09] Unknown:
bandit linter for identifying security issues. But what are some of the other supporting technologies that you rely on for your own development workflow?
[00:34:18] Unknown:
I actually use 1 out of format every time. It's called iSort. It's great because I'm too lazy to properly structure imports. So I use it every time, like 20 times a day, I guess. It's great. I also use, this little known link to called Nitpick. It is a tool to synchronize all your configurations across different projects. So we can ensure that all the projects are configured the same way. It's also included in the link Make Python style guide, but it has to be installed separately because not all people want to have the same configurations, for example, we do. But, the great thing about it, you can create your own configuration and distribute it as just a plain text file. Just can keep it on your GitHub or GIST as you wish. And I also really enjoy this little tool we rely on in links called Doctor Glint. It is a tool to link your docstrings.
It allows to check that you have all your parameters, documented, all your exceptions documented in the riser section, that's, you have yield, your return keywords documented, and stuff like this. It's really great. I really enjoy it. And, I also recommend to use it for your projects. Maybe even without, we make Python style guide. It's good in its own. Yeah. Documentation
[00:35:45] Unknown:
format is another potential area of contention. I'm wondering what your preferences are as far as do you follow the sphinx style docstrings? Do you use the Google or NumPy style? What are your particular feelings about how to structure the documentation
[00:36:01] Unknown:
strings in the functions that you're writing? I don't really think that it is important because it's, just a stylistic preference. The most important part here that you document these things. Like, anyhow. Choose what you or what you like the most, but just please do it.
[00:36:18] Unknown:
Fair enough. And then there are a number of other elements of the overall development experience that have been going through some evolutions recently. 1 of the notable ones being poetry gaining traction and the work being done with pip to improve its dependency resolution. What are some of the other elements of the so called modern Python development experience that you think are worth calling out and that you particularly either use or promote?
[00:36:48] Unknown:
I would say that's, for me, the most important part in the toolset is, Hypothesis. I really enjoy how Hypothesis allows you to test your programs differently. And, there are a lot of great, projects that rely on hypothesis to take it to another level. For example, you can use Schema Tesis to test your property based test, your, the Swagger documentation, and to ensure that it is also up to date with the actual code. Like not just correct swagger implementation, but that it is actually up to date with the code and, that it works the same way it declares. So it's really great and I really recommend everyone to use it in the API projects.
There is also a nice tool called Pytest Alter Hypothesis, I guess. There's a tool to auto generate, your property based tests from the type annotations. And it is also a great tool to allow, like, 2 lines of codes to generate, like, thousands,
[00:38:04] Unknown:
of tests for you. And what are some of the tools that you think are either not fully developed or lacking entirely in the Python ecosystem? And as a corollary to that, what are some of the other languages that you typically rely on when you decide that Python is the wrong choice? I actually
[00:38:22] Unknown:
use, TypeScript a lot. I guess, it is my second language in my daily life. I also enjoy Rust and Elixir. These 2 are very different, but, great languages. You just don't have to compare them. But I really recommend to have a look at both of them. I guess, Rust is kind of popular in the Python ecosystem. So but Elixir is not so popular. So you you can have a look at Elixir at least. And, I don't think that Python is actually lacking something. That's the main reason I use it as the main language because we have like everything. And so the ecosystem is really great. I can compare it to some other ecosystems. Ecosystems. I can say that, probably JavaScript has pretty much the same, maybe, more or less, the same ecosystem, but not Rust or Elixir or any other language like Haskell or even Go.
[00:39:22] Unknown:
And for people who are evaluating the dry Python libraries or your overall development philosophies, what are the cases where that set of approaches is the wrong choice and they might want to use some other set of packages or structure their code in a different manner?
[00:39:41] Unknown:
When you work with a lot of junior programmers. Because, these concepts are not so easy for junior developers. Because they have to wrap their head around just basic Python things. And when you add some extra complexity on top of it,
[00:39:58] Unknown:
it just doesn't work. And as you continue to develop the dry Python components and the set of libraries that you use and release for your work at we make services. What are some of the future developments or goals or upcoming plans that you have for the future of those libraries
[00:40:19] Unknown:
and, ecosystem components? And we want to create something similar to what, Xero is in Scala. And then I'll go for the next year, I guess, maybe 2 years, maybe 3 years. I don't know. But it is going to be something similar. It is going to be a set of libraries that resolve some different problems in different fields and, uses a lot of typing and functional programming and, creating some new cool stuff that you can use, independently and benefit from just a single part of the project. And you can use the whole ecosystem
[00:40:58] Unknown:
to build your functional stack on Python. And you have contributed a lot of time and energy to the open source community in the form of the libraries that you release, but also in the form of some very well thought out blog posts. Sometimes that people might have some, objections to. 1 of the ones I'm thinking of is your, somewhat contrarian stance that you should not log, which I know has more nuance in the post itself. But Yeah. I know. What is your what what is your overall approach to sustainability of the your work there and of your open source efforts? And what are some of your main motivations for continuing down that path? I can say that sustainability
[00:41:41] Unknown:
of the open source is broken right now. It's broken in a very serious way. And, sadly, I don't know any systematic path, to make it sustainable. I know a lot of, things are going on right now, like GitHub Sponsors, which is a great, initiative. There are a lot of developers who share their paths, how did they manage to make it sustainable. And, you maybe heard of the recent, story of Callep Prozzo. I guess, I pronounced his name correctly. I'm sorry, if not. And he shared his story about how he managed to make his open source work sustainable and, work full time on the open source project. And, it is a really motivational story. And, I really happy that we have such stories, but, it does not work for everyone, sadly.
Just you, you don't have any working mechanism to open source your project, make sure that people use it and to start benefiting from it. You have to implement your own ways, promoting yourself, promoting the work you do, promoting some ways of monetizing this work, and so on. So it just not the same as writing code. And, I spent last year negotiating with different companies on, mostly Russian companies on supporting the open source project. And I managed to get just 1 deal out of it. I've spent, like, a whole year, And I'm really grateful for the company, Rostecom, who supports us. And, it is a great example of how things should be done. Because, I know that a lot of companies do benefit from the work we do, and, we want to make sure that, the projects, they benefit from are sustainable. Because right now, it works, this way. I can dedicate, like, 4, 6 hours from my day job to work on my open source project because, internally, we rely on a lot of them. So I have a lacking opportunity to work, on this. But on the other hand, I can spend this time more productively working on our client projects and to earn more money from this.
And, right now, I'm just losing money, but building, an open source projects. And, that's a really sad choice
[00:44:17] Unknown:
1 have to take. So And are there any particular areas of contribution or engagement from the community that you're looking for in any of the open source work that you're doing that would be helpful to drive the projects forward? Yeah. Now we are always welcome new contributors,
[00:44:34] Unknown:
for the projects we do. We always try to make them, as easy to contribute as possible because we value the time of our contributors, and we try to document all the things. We try to make it as automated as possible. And to make this process, easy and pleasant. So,
[00:44:55] Unknown:
yes, if you want to contribute, you're more than welcome. That's pretty much it. Alright. Well, for anybody who wants to follow along with the work that you're doing or get in touch, I'll have you add your preferred contact information to the show notes. And so with that, I'll move us into the picks. And this week, I'm going to choose a book that I've been reading with my kids called the map to everywhere. It's just a very interesting and imaginative fantasy book that, been the first of a series of 4 so far. But the overall premise is that there's a stream of creation that has split off from the mainstream. And so it has all of these interesting interconnectedness between different realms and, worlds and just interesting little adventure book. Definitely recommend that for somebody who's looking for a new thing to read and, dive into. And so with that, I'll pass it to you, Nikita. Do you have any picks this week? Yes.
[00:45:44] Unknown:
We are actually organizing a conference in Russia. It's called, Russian Python Week. And it is going to be a new format for online conferences. And if you do, want to participate, you're more than welcome. We'll have some activities in English, but mostly in in Russian, because Russians do love, speaking Russian. It's something we are really famous for. And, join us. We will have some really great activities, including, discussing the communities, including discussing the future of Python,
[00:46:20] Unknown:
some games, some fun, and some knowledge. So I guess it's going to be some new industry standard for other conferences Alright. To follow. Well, I look forward to seeing how that plays out for you. Thank you again for taking the time today to join me and discuss the work that you've been doing with the dry Python libraries and the open source work that you're doing with we make services.
[00:46:42] Unknown:
Definitely some very well structured code and some useful tools there. So appreciate all the time and effort you've put into that, and I hope you enjoy the rest of your day. Thanks a lot for having me. I really enjoy your podcast as well. So thanks a lot for taking the time to record a lot of awesome guests and covering a lot of awesome topics. Happy to help.
[00:47:01] Unknown:
Thank you for listening. Don't forget to check out our other show, the Data Engineering Podcast at dataengineeringpodcast.com for the latest on modern data management. And visit the site of pythonpodcastdot com to subscribe to the show, sign up for the mailing list, and read the show notes. And if you've learned something or tried out a project from the show, then tell us about it. Email host@podcastinit.com with your story. To help other people find the show, please leave a review on Itunes and tell your friends and coworkers.
Introduction and Sponsor Message
Interview with Nikita Sobolev: Introduction and Background
Philosophy of a Blameless Environment
Dry Python: Origins and Tools
Starting a New Project: Design and Architecture
Challenges in Default Python Development
Dry Python Libraries: Addressing Python Shortcomings
Returns Library and Future Developments
Simplifying Functional Programming Concepts
API Design and Developer Experience
Dependency Injection in Python
Impact on Maintainability and Extensibility
We Make Python Style Guide
Balancing Linting Rules
Supporting Technologies and Tools
Documentation and Auto Formatting
Modern Python Development Experience
When Dry Python is the Wrong Choice
Future Developments for Dry Python
Sustainability of Open Source Work
Community Contributions and Engagement
Closing Remarks and Picks