Riding The Rising Tides Of Python

Hello, and welcome to podcast dot in it, the podcast about Python and the people who make it great.

When you're ready to launch your next app or want to try a project you hear about on the show, you'll need somewhere to deploy it. So take a look at our friends over at Linode.

With 200 gigabit private networking, scalable shared block storage, node balancers, and a 40 gigabit public network, all controlled by a brand new API, you've got everything you need to scale up.

And for your tasks that need fast computation, such as training machine learning models, they just launched dedicated

CPU instances.

They also have a new object storage service to make storing data for your apps even easier.

Go to python podcast.com/linode.

That's

linode

today to get a $20 credit and launch a new server in under a minute. And don't forget to thank them for their continued support of this show.

You listen to this show to learn and stay up to date with the ways that Python is being used, including the latest in machine learning and data analysis.

For even more opportunities to meet, listen, and learn from your peers, you don't want to miss out on this year's conference season.

We have partnered with organizations such as O'Reilly Media, Corinium Global Intelligence,

ODSC,

and Data Council.

Upcoming events include the software architecture conference, the Strata Data Conference, and PyCon US.

Go to python podcasts.com/conferences

to learn more about these and other events, and take advantage of our partner discounts to save money when you register today.

Your host as usual is Tobias Macy. And today I'm interviewing Pete Fine about his voyage on the rising tide of Python. So, Pete, can you start by introducing yourself? Sure. Thank you for having me. My name is Pete Fine. I'm a software engineer and consultant.

Work exclusively in Python,

for almost close to 20 years. And do you remember how you first got introduced to Python? Yeah. I first got introduced to Python 2.1

around 2, 001. I was working,

in the trading industry in Chicago, and a coworker put me on to it for, doing some interest rate modeling. And, shortly thereafter,

I started a startup as a CTO as a job search engine.

I was kinda really really cut my teeth in language at that point. And my understanding is that you have used Python exclusively

in your professional life. And I'm curious if there are any other programming languages that you have been exposed to or experimented with and taken any inspiration from as along your journey of working in Python or, just some of your views on the rest of the programming community as somebody who has been using Python professionally for so long? Sure. I'm somewhat unusual in being this experienced and only working in 1 language, and I mean that literally. Like, Python is the only programming language I know how to code in, and I can obviously read other things. But the advantage of being a language monologue is that it's allowed me to be an architectural

polyglots. I've worked on whole range of different different back end systems, front end systems.

And by focusing just in Python, it's allowed me kind of to to expand in other areas.

In terms of other influences,

I

studied computer science as an undergraduate, and the I did a little bit in high school, but the first class I did in the fall of 1996

was on Scheme, a lisp dialect using the structure interpretation of computer programs. It's a kind of famous book called The Wizard Book. And

despite having done basically no Lisp

since then,

that has more influence on how I think about programming and how I structure my code. And I I should should amend that a little bit to say that a year ago, I

retook this class,

with David Beasley, who's been a friend. I've done maybe about a half dozen classes with him, in Chicago,

where what I had done in, you

know, 13 or 14 weeks semester when I was 18 years old and kicked my butt, we did in 4 whole book in 4 days.

And,

was certainly certainly ex experience and kinda really, really interesting to go back to it. But yeah. And then I, you know, I've drawn on sort of other language paradigms,

as, you know, things have gotten popular. Erlang was kinda intellectual for a little bit and thinking about some things and, you know, async JavaScript. But,

really, it's just been it's been Python via Dave Beasley and and, the wizard book. And so you mentioned that at the beginning of your career in Python, you were working in training. You've worked as a CTO,

working in the web space. And my understanding is that now a lot of your consultancy is focused on the data engineering and data science area. So I'm wondering if you can just sort of give a brief overview of sort of the shape of your career and some of the main sort of verticals and problem domains that you've worked in. Oh my gosh. Sure. So like I said, I started out in trading,

doing interest rate modeling, like trying to

figure out, you know, how to price bonds,

which is where where I learned Python. And then spent about 6 years as the CTO of a job search engine vertical

vertical search start up, kinda similar to Indeed or Simply Hired if people know those.

And I feel like in some ways, I have been

repeating the lessons I learned during that time,

over and over again in whole or in part,

both in terms of web development, which I did for a number of years, Django, Flask, a bunch of other frameworks,

and then over the last, oh, 5 or 6 years moving into more intensively into data engineering,

really big data, supporting data scientists, data analysts,

and sort of bringing a software engineering approach to to data. And along that journey, what are some of the projects that you have been involved with which you're particularly proud of or that you think were sort of milestones

along your career? Sure. Certainly that startup and the fact that, you know, I got really burned out towards the end and

kinda exercise my options and got on with my life. And I'm still friends with the founder, which feels like as much of accomplishment

as anything. Since then, I on the open source side, I've done a number of projects.

The kind of most successful 1 was a library logging library called Twiggy, which has kind of an interesting backstory. I was at PyCon in

2009 and was hanging out on the the chat group and complaining about standard web logging as a lot of people do. And Jesse Knower, the conference chair was like, you should write a new 1. And I'm like, okay. I I just here's this guy. He's giving all of his time and energy to make this conference happen, and he's telling me that the most useful thing I can do,

right now is to write a new logging library.

So maybe I should listen to him. And once I got into it, it turned out to be actually a really interesting problem domain. I really enjoy,

kinda like system level utilities.

I joke that the kinda, like, code I like to write is, you know, you say if name equals main and then import Pete's code kind of things that cut across a whole range of different different problem spaces. And so it was fun to take a look at that problem space,

afresh.

I mean, everything up to that point had been based on Log 4 j, the Java logging library.

And we had learned a lot about how to structure applications in the intervening 15 or 16 years since Log4j came out. And, yeah, that library,

got some use. It's sort of an interest been an interesting

experience for me as an open source maintainer. I I really

I really try to write code that works perfectly.

You know, it does it does it does what it says on the box. It has, like, 95% test coverage, and it, you know, just does it does it totally perfectly because it's just both as an open source container. And then in my consulting business, there's no there's, like, no profit and support. It's not fun. It's not a good way to make money.

And so the problem with that libraries is I didn't know anybody was using it. It turned out there were, like, 3 or 4 third party extensions people had written and put up on PyPI,

and nobody nobody got in touch with me. It got packaged by Debian and Red Hat, and I didn't know until I was, like, randomly googling the project and found it in their repos.

And I think I got maybe 6 bug reports in 10 years.

And I'm sure maybe I should have left some more bugs in because if I had no people were work using it actively, I would have probably worked on it more more than I did. And so, yeah, it's still out there. Some people are still using it. I talked to the author of Struct Log, which is a add on for for standard with logging and said he said he had kind of been influenced by the the work I had done on that. And that felt like as much as an accomplishment as anything. I mean, that's that's success in my terms is I was really just hoping to reopen reopen the space and kinda try some new ideas out, and so I'm glad some people people ran with them. Yeah. And there have been a number of different points where people have tried to reinvent logging for Python, and there's it's interesting to see the different ways that it gets approached. So there's the work you've done. There's Structlog as you mentioned. A while ago, I had Itamar Twiner Trauering on to talk about Elliot, which is a different approach.

There's the logbook project from Armand Ronnicker. So there are a lot of different ways that people have tried to tackle the same problem. But 1 of the limiting things is that for the most part, you do, at some point, need to be able to be at least 1 way compatible with the built in logging library because because you need to be able to consume the you know, consume and propagate the log messages from your 3rd party dependencies as well. Yeah. And I think that's exactly been the challenge. It's just 1 of these system wide cross cutting concerns

that everybody has to use the same thing. It's like you can't have your own import system.

Right? It's just every

you have to support basically the whole universe of,

you know, 3rd party and extension packages and and have some kinda compatibility or monkey patching. I think that that has really

kinda kept us locked in on standard web logging, which for its time was kinda quite powerful, but it's sort of, I think, starting to show its age and the performance is not very good. And it's it's in some ways I think we're at the point where we've had a lot of ideas and just that somebody or a couple people kinda need to kinda get down and just sort of do the hard long work

of getting something that, you know, supports a bunch of these different different idioms, supports all these different libraries, and then doing the work of getting that mature, getting community uptake, and then working it into the standard of library. It's not unlike the packaging dilemma of the last I I wish I could say it's done, but it doesn't seem to be,

of the last, I don't know, 5 or 10 years. Is that right? This is another area where everybody's gotta use the same thing. The yeah. We've been fortunate that, some people from the,

PyPA, the Python packaging authority have really just been dedicated enough

to to kind of push that through because it was really critical. And logging is important, but it just you know, we can continue to write programs that use standard of logging and it's basically fine.

And so, yeah, I think,

it's tough

to to be motivated sometimes to to to do that over the long haul. And so I have a lot of respect for for people who who put that work in. I wish I had more time. And another interesting element of your work on Twiggy is, as you mentioned, there have only been a small handful of bug reports over the number of years that it's been available. And so it's an interesting

perspective on

the idea of when software is done because it if you look at some projects on GitHub, for instance, it's easy to see something that hasn't been updated in a year or 2 and assume, okay, this project is dead because nobody's maintaining it. But on the other hand, it could just be that the project is done and there's no need to do any, you know, updates or maintenance because it works as intended and there's nothing that has come along in the intervening years that has caused it to break. So it still functions as it did at the first time that it was written. So I'm curious if you have any particular thoughts on that. Yeah. It's an interesting

it's an interesting question. I run the Python meetup, here in Pittsburgh, and maybe about 6 months ago, we did an event I called, Pete Picks a Python Package. And so just trying to teach people that when I go out and I'm trying to find a 3rd party library to use, what am I looking at? And this is a a question. You know, the ecosystem is so large and so complex, and this was 1 of these issues is, you know, you you know, I go I look at the GitHub activity,

and is it done? Is it just is it dead? Are people not using it? And so,

you know, being distinguishing between, you know, projects that have open bug reports or open pull requests that are just aging and not getting respond to, that project might be dead. But something that just doesn't have bug reports and starts with yeah. It might still be still be active. You know, other things I I look at are,

are there documentation?

Are there tests?

I

like to look at some code and then just get a sense of kind of the feel of, like, what people are doing. I tend to do this when I'm doing interviews,

either, you know, when I've been hiring for a company or just I do this sometimes,

for clients. It's like I do technical interviewing. It's like, I like to see with some real code. And my favorite thing to actually look at is, utils.py

is is that you learn a lot about a project or a person's coding style about what they what they think utility functions are.

But, Yeah. I think that's I think it's a far more effective way of evaluating

a a technical peer or a project or than whiteboarding or kinda any other such silly thing. And it's like, what we do all day is write code. And so, like, let's let's see something. Let's talk about that. Yeah. The utils dot pie is an interesting thing as well because there are some projects where there's a utils.py that's dedicated to that project, or you might have an entire module that you PIP install for all of your utilities that you cargo cult across all of your different projects. And it's interesting how you figure out what are the dividing lines along that, and how do you package them so that you're not just bringing in a whole bunch of useless functions that have no bearing on the project at hand. Right. I'm

that it's kind of a whole line that doesn't, you know, 3 or 4 dozen kind of common recipes at this point. And so, yeah, it's just as the as the community and the ecosystem has grown

and the tooling

has gotten better for sharing code, it's just the the amount of work that we have to do redundantly

has has reduced so dramatically. And, you know, I think I think it's sometimes useful as particularly for people who are, like, coming up in their career or kind of earlier in their career to go through the exercise of writing these utilities or or, you know, certainly reading how these things are are written. But it's just the the amount of high quality code out there that you don't have to go figure out and make the same bugs over and over again compared to when I started in my career is just amazing to me. And just how much more we're able to get done. It's just really it's just I mean, this this was the point of free software. Like, this is the point of free software and open source is that we can make a

collective code commons that everybody benefits from. And it's amazing to see how how that's actually panned out, how well it's panned out. And that's an interesting thing too is that early on in the life cycle of a language and its community and ecosystem, it's very easy to have a significant impact on a broad range of people who are just discovering the language.

So for instance, Elixir is a fairly recent language that I've heard a lot of people having the same type of experience of, it's a new language, nobody has written the canonical ORM for it, so I can go ahead and be the 1 that writes that. Whereas, if you're coming into Python now, there are dozens of ORMs. So if you write another 1, it's just another ORM, so not as many people are going to see it or be as impacted by it. So I'm curious what your experience has been of somebody who joined the community fairly early, but up not at the beginning. And having been engaged in helping to support and grow the community through running these meetups,

how the sort of accessibility of the community and the ability to have an impact has changed over the years from when you started to where you are now. Sure. I think that's a that's a great question. I got first involved

in the Python community and open source at the Chicago Python user group. And I say user group these days and people look at me funny, but it predated Meetup and where it was basically a self hosted mailing list and a conference table, you know, in a coffee shop. And I remember being about 25 and going to a Meetup, and there are 12 or 13 people there. And 3 of them are the core subversion developers, which was the version control system everybody used before Git. And I'm I'm just, like, 25, and I'm holy crud. I'm, like, I got I type subversion, like, a 100 times a day, and I get to hang out with you folks. And that was just a really just a transformative experience for me,

that wait. Like, these these are just people, and they're just around, and they're just willing to talk to a young kid. And so I've really tried to kind of carry that spirit forward. I've both, you know, in sort of, giving talks of PyCon and sort of being helping involve that. And then when I had the opportunity to take up the the Pittsburgh meetup, which had gone dormant, just be able to to pay that back in a in a broader sense of accessibility. It's just the community gotten larger. We've become much more aware

of

what our community looks like and what we're doing to make it inclusive

or not for all sorts of people. So when I first attended PyCon in Dallas in 2007,

it was 96% men. And as a guy, like, it was terrible. It was just terrible. This wasn't good for anybody. And just the strides we've made on that front, you know, when I went went last year in Cleveland,

just looked around, it was like, you know, it was like 35% of women were it's just like people. It was like just people. I was like, wait. This just looks like this just looks like normal people.

And I really

appreciate the efforts the Python community has made on that front. You know, this this has been my home in open source for almost 20 years, and it sort of spoiled me. I go to other conferences or I I go to other language communities or whatever, and everybody is striving to be what Python is in terms of a community. And, I I really appreciate being able to play a small part in that. And what are some of the other ways that the growth of the community

and the way that it has evolved has impacted your career trajectory

from working in trading to where you are now working in data?

Sure. I mean, at a at a technical level, the changes I've seen over this time are just amazing to me. So when I started, there was no cloud. I remember as Amazon released this s 3 product, which all you could do with static storage. Like, there wasn't you wouldn't even, like, hang on. And I was like, oh, this is interesting. We can do something with this.

So

what's what's available now? There was no Web Story when I started. And I remember being at the Chicago Group in around 2007 or 2008,

and Ruby on Rails was eating our lunch.

And these guys come up from Kansas, and it was the first public demo of Django. We're all just like, oh my gosh. We still get to have careers. This is awesome. You know? And then more recently, the the growth of data as a a discipline has been,

certainly interesting for me. So I've sort of been working in that space since since I was doing the search engine in the early 2000.

And, it's just seeing that applied in just all these different domains that we can can make concrete decisions. And it's really in the last year or 2 that I feel like data engineering is a subspecialty,

has has kind of come into its own that, you know, like, I'm not a statistician. I'm not a data scientist. As I like to joke, it's been a really long time since anybody taught me calculus. And so I I, like, I kind of understand it, but just, like, don't don't let me near a machine learning model. Like, it's just it's not, you know,

I think. And so being able to bring,

a real, like, hard engineering focus to people coming from those backgrounds and to support them in the work they're doing has been really rewarding. I think career wise, the big change over this time has been just the specialization

across tech that for what was for me was 1 job in, you know, 2003. It's literally 5 or 6 different jobs now from web developer, back end developer,

data engineer,

dev ops. I mean, we used to call that sysadmins.

And, you know, it's just you kinda use like like, 1 person would do all of these things.

And,

the ability to focus,

I think, as the field has grown,

has just kind of generally improved the quality of of software and products we're able to produce. Yeah. I remember probably 6 or so years ago at this point was when everybody was using the term full stack engineer to mean somebody that had to do

web API to managing the database

and figuring out how the overall system architecture looked. And as you said, there's been a lot more specialization

as front end engineering is with

And

early on. And the complexity of the web environment has grown, particularly with trends in DevOps and the move to things like microservices

or,

using a lot of third party APIs to be able to stitch together an application. And then as you said, the rise of data as its own discipline

and re needing to actually put a lot more rigor and engineering around how you manage the life cycle of the data and the data flows to ensure that it continues to be available and consistent and have the necessary amount of cleanliness and rigor that is necessary to be able to gain useful analytics from it. Yeah. And the data side is still

evolving, which keeps things interesting to me. You know, you said that just as communities mature that there's maybe less opportunity to have an influence. And I think that's true to a degree, but just as these engineering disciplines have gotten more sophisticated,

there's an opportunity

to go through your own, you know, personal personal growth and as well as community development, work on those, you know, as as disciplines. And so, like, yeah, we're still in the early stages of what engineering software engineering means for data. And that's been interesting for me. And as a consultant, you kinda I mean, I think this is true in any part of the field, but, like, you kinda have to keep moving. So in 2000

12, I was doing a lot of Django, and I can still do that, but just there's a lot more people who do that now. And it's allowed me to kinda continue to move in new directions. And I think 1 of the things that's so rewarding about being a software engineer is if you like learning new stuff, if you're the kind of person who's curious

and self motivated, it's just such a such a wonderful field to work in. It's that just there's an infinite number of rabbit holes you can go exploring. And so I think I don't think that's gonna change. I mean, I think as we continue to

to help software eat the world, like, we're gonna find more and more things that like, even if it's all Python and, like, okay. Great. We've we've, like, we've solved Python. We haven't solved Python. But, you know, as we apply that to more and more domains,

there's gonna

continue to be opportunity

to growth. I mean, I'm, like, really curious to see what happens with IoT and, like, physical devices.

I've done done some consulting work on you know, there's a whole bunch of self driving car and, like, robotic start ups here in Pittsburgh. And it's been a lot of fun for me to do things that aren't the web. I like the web just fine, but just, you know, I've spent much of my career building the web or consuming the web or advertising on the web. And so go out and be like, wow. I get to work on things that move around in the real world or, you know, what? Just did a prod a little bit of project for a company making, heart monitors. It's like, wow. This is real stuff. Like, this is real stuff. And as we start to couple software with physical technology, the opportunities to, I think, really make a difference, like, really make a difference in meaningful ways in people's lives are just increasing. Yeah. And another area of growth is on the higher level piece of things of making programming easier and more accessible to people who don't actually want to do it as a career and just wanna be able to have some influence over the technology

that has been permeating their lives, whether it's the smartphone or some of the, smart home devices or their vehicles, and just being able to have some way of modifying

the logic in these devices to suit the specific needs that they have and not just have to take what's handed to them as a prepackaged good that they're not able to look under the hood for and moving programming and moving computational thinking and computational accessibility further and further up the ladder? Yeah. It's just a really just powerful trend generally.

1 of the meetups I I did here last year, there was a woman who, presented on, she works on the artificial pancreas project. And so this is an open source project to make a closed loop

pancreas system. So she has,

juvenile she had juvenile diabetes and the automatic insulin pumps that were on the market are just very primitive, like, very, very lagging and stuff like that. And so people hacked the

hacked the firmware

and wrote, you know, monitors that are kind of much more responsive.

And I'm like, this, like, this is the promise of free software. Like, this is what open software is about. It's people, like, building their own pancreas.

You know? It's just amazing to me. And,

I think the challenge is the community is just how do we

welcome these people in. It's just more and more people

get involved in, like, people who don't think of themselves primarily as coders. You know, there's a a pretty large community here in Pittsburgh of people who do computational art.

And they are they're artists first and programmers second, if that's a meaningful thing to say.

But, you know, programming is their paintbrush. And as more and more people get involved, it's just how do we continue to spread our culture and I mean that both in a technical and social sense, you know, basically.

How people don't, you know, people coming in don't know version control or testing or, kind of, all these kind of basic practices

that we've developed and how do we spread that. But then also just, you know, how do we spread our community values of accessibility, you know, free software stuff and that. And your note on things like version control and basic engineering best practices is something that is worth talking about as far as as you have moved through these different phases of your career and worked across these different verticals and problem domains. What are some of the constants that have been helpful throughout and some of the areas that you see as being valuable regardless of where you might end up working within the software realm? So, obviously, for me, Python

Python Python has been a constant.

I keep getting tempted to learn other languages, and I just kinda there's this joke in the Python community that's the 2nd best language for everything and, you know, that if you need to make a web page, maybe PHP is better. And if you need to crunch some data, maybe R is better. But if you need to crunch some data and put that on a web page, well, now you're doing 2 things. And, Python Python excels at that. And so so out in the real world, like, we're often doing more than 1 thing. It's been been really excellent. And so just that's been just the big tool in my toolbox for nearly 20 years. What else? People. It's just as as I get get a little bit older, just the, you know, software, it's full of humans.

And whether that's the clients or, you know, your employer, your bot the business people or it's the other developers, it's just it's it's, you know, it's it's surprising is that, you know, I think a lot of people get into technology because they're maybe a little less social, a little outgoing, and then, you know, you find that to really be successful that you do have to deal with the humans and that it's really rewarding after a time. Beyond that, I mean, just, you know, the I mean, just the availability

of what's out there in terms of best practices,

like logs or

Stack Overflow, which is is a relatively new development

relatively

in the course of my career. Again, I think sort of reflects that community is that we're just yeah. We're trying to make something better. And it's just that's what's so so inspiring to me about working this area is it just it feels like an area of the world where we can make some progress. We can make things better in in small ways. And on the other side of things, as you have moved across these different problem domains

from web to data

and working across the different communities that you've been involved with in your different roles? What have been some of the biggest differences and some of the biggest learning curves as you have moved into these different areas?

Sure. So 1 of the nice things I like about being a consultant is I do get to lot work in a lot of different

looking verticals, and it's it's fun for me to learn a little bit about people's businesses or people's particular problems.

It kinda keeps keeps things interesting. You know, in a on a technical side,

moving from the web to data oh my gosh. I feel like I mean, it's taken, I don't know, close to 20 years, but the web is largely a solved problem. I mean, not entirely, but just, you know, like, data best spec websites. Like, we find it took 20 years we finally got it,

and less so in some of the other areas. And so in in terms of the domain, there's a it's interesting. Like, a lot of web code or even, like, kind of systems code like dev, you know, dev ops systems, there's both the right answer in terms of that, you know, you can write a function in your web server and you can test it and you can, like, thoroughly test it. So, like, this is like there's an oracle. It's like, this is what this should do. And that somehow coupled with a bit of a loosey gooseiness about how that actually performs, it's like, well, the web server went down for a minute. Okay. Whatever. We just we don't care. Just doesn't matter, you know.

Or, hey. We, you know, we dropped a couple of requests or, like, that was just whatever. It's fine. And then on the data side, it's interesting is there's kind of the flip where

a lot of these problems that there is no right answer. It's just, you know, you crunch some data and just the the result you get is what you get and you don't know you know, it's there's not an algorithmic solution. It's not like it's 2 plus 2 equals 4. It's like, well, the trend line is up into the right. It's like, okay. How steep? How steep? We just you know, it is well, it's just what the data is. And so that that has on that side, the, you know, correctness is harder to define. And then sometimes on the engineering side, it's more important. So like I said, I did a small project for a company that makes heart monitors. And so some of the questions I like to ask when I'm doing data projects is, like, are we allowed to drop data on the floor? Sometimes that's just an engineering compromise that makes things easier. And so you go to somebody who's doing web analytics and you're like, yeah. We missed a few clicks, Whatever. And then you talk to somebody who's monitoring a patient's heart, and they're like, we can lose no data at all.

And that contrast,

is interesting to me between stuff on the web where, like, you can, in theory, get it right, but you don't have like, totally right, but you don't have to. And then on the data side, where there is no definitely right answer, but it's gotta work perfectly. And so how do you how do you resolve it? I don't know. We're still figuring it out. And so that's what that's what it keeps it interesting for me is, like, I like I like being able to go in and help people solve their problem and build them a platform that they can kinda continue to grow their business and iterate on, and I kinda go go do it for somebody else. So, you know, over the the course of, like, long experience and I don't know if I'm better at anybody else than this. I've just been doing it a long time, and I've made made the mistakes, I think. And to kinda spread that knowledge, you know, around and kind of apply it to a bunch of different situations has been interesting for me. And going back to Python itself,

what are some of the aspects of the language

or its overall ecosystem or the community which you feel are either lacking or they don't receive enough attention and that we should be spending more focus on and, more energy on either improving or filling any technical gaps or anything along those lines?

Sure. So

it's interesting to me. The growth of the language over the last, let's say, 5 years, the language itself, I think, has become

almost not unmanageable. I mean, because clearly, like, you know, the core Python team has continued to put out releases. But as a user of the language, I just I can't keep up anymore. So I knew Python 27 inside and out, like, inside and out. And with the transition to 3, I didn't really start using it on projects till maybe, like, 3:5. And by that point, there was so much new stuff in there that I feel like I had kinda lost touch a little bit with all of the new features. And just as the community of committers

to Python itself has gotten larger, I feel that, like, prom has accelerated. And it's I've talked to other people, you know, who've been around for as long as I haven't. They're kind of in a similar boat. It's just we're like, wait. What's going on? Or, like, you know. And it's particularly as they move to faster release cycles. You know, it's like, wait. There's all this crazy new stuff in typing and, you know, I just I was just looking at the release notes for the upcoming 3 8 release and multiprocessing

is now gonna support shared memory communication

between processes in addition to kinda just piping and copying data back and forth. I'm like, this is a game changer. Like, that's a huge technical innovation, and it's got, like, you know, a 1 sentence bullet point in the change log. And so

how we get better at keeping the user community

moving along with advances in the language itself, I think, is still a challenge,

as as the pace pace of innovation

increases. I don't have a good answer for that. The other thing in terms of as a community,

at PyCon this past year, I heard there was a lot of fretting about the challenges of sustainability in open source and just that, you know, maintainers are getting burned out and there's no good funding model.

And I'm just I'm not entirely sure what problem funding will solve, to be honest. And I I'm kind of unusual that that, you know, somebody getting I mean, I guess everybody likes money, but I just you know, somebody getting a couple $100 a year is not gonna make the difference between are they able to support support doing a project or not. So, like, what is what does sustainability

look like in this space? And, honestly, the biggest thing challenge I heard of is that is about project and product management. And so as a community, we've gotten really good at onboarding new developers and bringing new people in as contributors and there's, like, a lot of best practice that that people put together. We've all gotten really good at writing documentation, and I think Eric Holcher and the folks at Read the Docs have done a great job in just showing is important to us and giving us a platform we can do on that. And I think the next big challenge is is project management. As you know, I talked to a bunch of people who are maintainers of popular projects, and they're like, I spend all my time doing ticket triage. You know, we get we get, like, 800 bug reports a year. I gotta go figure out which of these actually need to be addressed and we'll, like, manage manage them through a workflow, reproducing cases. And that's something I don't think we

know how to do ourselves, and we certainly don't know how to teach it. And I think that's kinda coupled

with the people wanna work on their own projects. I mean, I I'm guilty of this as much as anybody else and I think this is a broader cultural thing is that you get credit or you get a promotion or, like, you know, people wanna see their name name up on the web. And, you know, you don't get that for being just having your name in the changes file. You get that for being, you know, like, the inventor of whatever, you know, like the inventor of date utils

or whatever it is, you know, like these major libraries that people use. And I don't know. I think we, as a consequence, are reinventing the wheel a lot in the open source space, and then, you know, the people who do sort of accidentally wind up with their hobby project becoming critical infrastructure just overwhelmed.

And I don't know

how I mean, like I said, I I probably could personally do a better job of of supporting other people's projects instead of working on mine, and I just I I think that's something maybe we all

need to think about. And then as somebody who has had these different shifts in your career

going from trading to the web to data engineering

and being involved in the community and follow along in these different industry trends, what are some of the currently evolving trends or any that you see as being upcoming, which you're keeps keeping a close eye on? And how do you anticipate them influencing the direction that you take into your career and the direction that the broader community might trend towards in the coming years? Sure. So definitely, like, we talked a little bit about, Python for everyone,

that that more and more people who might have been

spreadsheet jockeys are now learning to program or people who did data entry are now learning to automate parts of their job. And so, I'm looking forward this year, to doing some training, some classes here in Pittsburgh. I'm just kinda, like, not teaching people to program, but for people who have a little bit of basic experience, maybe did an online class and are looking to level up their game. Or for people who know Pandas but not Python. I feel like there's a lot of people who are coming from R, have gone through a data science boot camp, can use some of these science libraries,

but just, you know, in terms of structuring code, classes, how the object system works under the hood,

data cleanup and processing. I'm looking to you know, this is as both a career thing and just kinda spread my knowledge to do some teaching on that. I'm also, this year, gonna be writing a book called Principles of Data Engineering with Adam Westley about covering that. Just not specific technologies, but data software engineering practice as a whole. So those, I think, are gonna be big trends. I mean, just the continued growth of data is is astounding to me is that I mean, it just

every sales organization

on Earth is gonna turn into a data

just that this is gonna allow us to answer questions

more specifically

and kind of would like as opposed to just kinda sticking our finger in the wind and being like, I think we should go to New York. It's like, okay. Like, are people buying in New York? I think it's gonna be a really big trend. And then like I said, the IoT space, like, physical computing is gonna continue to really develop. And that's interesting. Super interesting to me. The 1 of the handouts at at PyCon

in the Schwag bag this past year was a circuit Python, which is a microcontroller made by Adafruit that you can now run Python on a everything from, like, supercomputers to these huge, you know, 1, 000 node clusters down to a $5 chip with 256

k of RAM or whatever, and it's the same language. It's super cool. It's super cool. And just yeah. I I you know, there's this this

saying from, you know, the early 2000 that software is gonna eat the world. And I think that's largely played out. It's that, you know, software has has kind of invaded all these different spaces. And from where I sit, it feels like Python's about to use software. It's that if you're a college student today,

that 5 years from now, if you're going out and starting a project, whether that's a startup or like a hobby project or an art project or whatever, odds are you're probably your first tool you're gonna reach for is gonna be Python. And that's that's really exciting for me as somebody who's been involved in this community, for so long. It's that I think it's gonna make a lot of things possible. Outside of that space, you know, I'm always kinda keeping an eye out for new languages and new technologies.

Rust looks really interesting to me as a programming language. I know there's a a bunch of kind of experienced Python folks I've been working with that are, really, really getting into it. Yeah.

Yeah. We'll see. And since you are living in Pittsburgh and engaged with the community there, and that's where PyCon is going to be for the upcoming 2 events. What are some of the things that people who are attending the conference should know about the area?

It's great. We just had a meetup this past week with,

the event coordinator, the, like, Python planner from the Python Software Foundation

and Emily Morehouse, who's conference chair, coming up,

for the next 2 years where they were gonna be in town to go visit the convention center, and they're like, hey. We wanna meet

with the community. When is your meetup? And I was like, you just tell me when you're coming, and that's what we'll have a meetup.

And so it was great. We had maybe about 30 people come out, maybe only 5 of them had been to a previous event, and so they people were interested in attending, volunteering, and just we're really hoping to be good hosts. You know, I moved I moved here from Chicago about 5 years ago and it's been a wonderful place to live, but it's not obvious

in some ways. It's, you know, like, 3 years ago when it was important, like, Portland is just an easy place to go have a conference. I mean, it's just there's food trucks, there's breweries. It's just, like, super easy. Pittsburgh

Pittsburgh is a wonderful place, but it's a little a little subtle. And so I'm interested in trying to help meet people's human needs as

conference attendees just to to recognize that we're more than just butts in seats with a laptop. And so,

we are looking to do everything from yeah. Provide guidance on, like, local restaurants, bars, coffee shops, and stuff like that. There's a fantastic fine arts scene here. The number of, like, world class museums within walking distance

of the convention center, including a 7 story museum just for Andy Warhol. It's like a 5 or 6 minute walk away. Kinda giving people advice on transit. I know, like, when I I go to travel conferences,

I always try to make sure to get out of the conference hotel or, like, the conference zone. Cause, like, when else are you gonna be in Atlanta? Or when else are you gonna be in Dallas or whatever? And, yeah. And, you know, beyond that, in terms of conference

content itself for people who haven't been, it's really a wonderful

event. It'll be about 35100 people. Event is capped at that size. I mean, they could sell more tickets but they're kind of deliberately keeping it small. There's, like, 2 days of tutorials beforehand,

3 days of conference proper including, like, I think it's 5 or 6 parallel tracks including, what they call Python Charloss.

Charloss is the Spanish language is Spanish for, conference talk. And so it is a full full track of talks in Spanish. I'm just like, I can't believe. It's just it's just amazing to me. And then, you know, afterwards, there's, 4 or 5 days of sprints where all these people, who are in town for this event and work on open source projects get together and hack on things for a couple days. So if you wanna make commit you know, if you wanna sit with the core committers of Django and, like, like, work on some Django patches, you wanna work on CPython itself, like, it's an opportunity to do that. And then beyond that, there are what are called hatchery events. And so PyCon's like a very progressive conference in terms of just being open to all the kind of weird stuff interesting stuff people

people wanna do. And so last year,

there was a traditional performing arts about the experience of programming. So play, spoken word. I I I had a big part in 1 of the plays, playing the role of a web API, which was hilarious.

And so, yeah, they've now formalized the process for,

doing these interesting things and it's called the Hatchery program. And so I know some of the things that are getting proposed is me and some friends are proposing

a gallery style installation on computational art. There's talks about a data science specific track, and we'll see what else comes up. But those proposals are due in a couple weeks. The CFP for conference talk like, conference talks proper, I think closes in, oh, just a few

days. So you should get on it

if you haven't. And then if you can't make this conference, there's a whole bunch of, regional events. So I attended both PIE Ohio with Columbus and PIE Gotham in New York, and they're like little mini PICONS and are also kinda really great way great way to partake. And always, like, I'm always looking forward to the hallway track, and so that is that is an event in and of itself of just people. There's lots of space to sit and drink coffee and just meet people and kinda find out about interesting stuff. That's really kind of a core part of the conference for me. Are there any other aspects of your experience of working in Python and with the community that we didn't discuss yet that you'd like to cover before we close out the show? I think 1 of the things is, you know, people people will come back to me and they'd be like, why why just Python? Like, why like like, why only language and why just Python? And so, for me,

some of that was just being, I think, in the the right place at the right time, but it just I love this language for its simplicity. It's just, you know, this is runnable pseudo code. If you and, you know, you and I were to get out a whiteboard and just rough out an algorithm, that's gonna look like what runnable Python code is. Pretty close. And so I I joked that I'm not smart enough for other things.

Like, I'm not smart I'm not smart enough to do Java. I'm not smart enough to do Scala. I'm definitely not smart enough to do Haskell. And that the problems I'm trying to solve are are pretty difficult

and having a language that's easy to work with and fits my brain makes doing those harder things possible.

And I think that's part of what's contributed to it just general growth, particularly among people who are coming into later in their careers that don't have formal training. They said it's just it's easy. It's an easy language to to get started in. And then the the community is great. I mean, that both in terms of the technical ecosystem, like, the number and quality of packages is incredible to me, and then and then the people. It's just I just continue to meet wonderful people and,

people who just kinda really care about what they do, and that's just really inspiring to be around. For anybody who wants to get in touch with you and follow along with the work that you're doing or hire you from your consultancy, I'll have you add your preferred contact information to the show notes. And with that, I'll move us into the picks. And this week, I'm going to choose the Matomo Analytics platform.

It's an open source and self hostable alternative to Google Analytics. I've been using it for the podcasts

for a while now, and it does a pretty good job of being able to track the data and have some usable reports. So for anybody who's looking for something to be able to track traffic on their applications, it's definitely worth a look. And so with that, I'll pass it to you, Pete. What do you have for picks this week? My pick this week would be fast API, which I have been using in a couple of projects recently. It's a Flask style restful API and just really, really a pleasure to work with. It's backed by a data layer called Pydantic,

which makes

modeling

requests and responses in different formats. And packing and unpacking JSON

are really trivial. And so it's it's easy to say, hey. I wanna switch this from, you know, passing arguments and query parameters to passing them in the body of the function. And it has kinda nice integration. So no RRM, but, you know, you can back with any data later. It's got, like, nice integration with SQLAlchemy, and it's both very fast, as the name might expect, works with asyncio, and just really been a pleasure to work with. And it's just kind of been nice to just bang these things out. So I'd recommend people check it out. Alright. Well, thank you very much for taking the time today and sharing your experiences working in Python and the trajectory that it has for your career. So I appreciate all of your time and effort on that front, and I hope you enjoy the rest of your day. Oh, thank you so much for having me. It's been a pleasure.

Thank you for listening. Don't forget to check out our other show, the Data Engineering Podcast at data engineering podcast dotcom for the latest on modern data management.

And visit the site at pythonpodcast.com

to subscribe to the show, sign up for the mailing list, and read the show notes.

And if you've learned something or tried out a project from the show, then tell us about it. Email host@podcastinit.com

with your story.

To help other people find the show, please leave a review on Itunes and tell your friends and coworkers.

The Python Podcast.init

Summary

Announcements

Interview

Keep In Touch

Picks

Closing Announcements

Links

The Python Podcast.__init__