Summary
One of the common complaints about Python is that it is slow. There are languages and runtimes that can execute code faster, but they are not as easy to be productive with, so many people are willing to make that tradeoff. There are some use cases, however, that truly need the benefit of faster execution. To address this problem Kevin Modzelewski helped to create the Pyston intepreter that is focused on speeding up unmodified Python code. In this episode he shares the history of the project, discusses his current efforts to optimize a fork of the CPython interpreter, and his goals for building a business to support the ongoing work to make Python faster for everyone. This is an interesting look at the opportunities that exist in the Python ecosystem and the work being done to address some of them.
Announcements
- Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
- When you’re ready to launch your next app or want to try a project you hear about on the show, you’ll need somewhere to deploy it, so take a look at our friends over at Linode. With the launch of their managed Kubernetes platform it’s easy to get started with the next generation of deployment and scaling, powered by the battle tested Linode platform, including simple pricing, node balancers, 40Gbit networking, dedicated CPU and GPU instances, and worldwide data centers. Go to pythonpodcast.com/linode and get a $100 credit to try out a Kubernetes cluster of your own. And don’t forget to thank them for their continued support of this show!
- Your host as usual is Tobias Macey and today I’m interviewing Kevin Modzelewski about his work on Pyston, an interpreter for Python focused on compatibility and speed.
Interview
- Introductions
- How did you get introduced to Python?
- Can you start by describing what Pyston is and how it got started?
- Can you share some of the history of the project and the recent changes?
- What is your motivation for focusing on Pyston and Python optimization?
- What are the use cases that you are primarily focused on with Pyston?
- Why do you think Python needs another performance project?
- Can you describe the technical implementation of Pyston?
- How has the project evolved since you first began working on it?
- What are the biggest challenges that you face in maintaining compatibility with CPython?
- How does the approach to Pyston compare to projects like PyPy and Pyjion?
- How are you approaching sustainability and governance of the project?
- What are some of the most interesting, innovative, or unexpected uses for Pyston that you have seen?
- What have you found to be the most interesting, unexpected, or challenging lessons that you have learned while working on Pyston?
- When is Pyston the wrong choice?
- What do you have planned for the future of the project?
Keep In Touch
Picks
- Tobias
- Kevin
Links
The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA
[00:00:13]
Unknown:
Hello, and welcome to Podcast Dot in It, the podcast about Python and the people who make it great. When you're ready to launch your next app or want to try a project you hear about on the show, you'll need somewhere to deploy it. So take a look at our friends over at Linode. With the launch of their managed Kubernetes platform, it's easy to get started with the next generation of deployment and scaling powered by the battle tested Linode platform, including simple pricing, node balancers, 40 gigabit networking, dedicated CPU and GPU instances, and worldwide data centers.
Go to python podcast.com/linode, that's l I n o d e, today and get a $100 credit to try out a Kubernetes cluster of your own. And don't forget to thank them for their continued support of this show. Your host as usual is Tobias Macy. And today, I'm interviewing Kevin Mazilevskiy about his work on Piston, an interpreter for Python focused on compatibility and speed. So, Kevin, can you start by introducing yourself?
[00:01:08] Unknown:
Sure. Hi. I'm Kevin Moduleski. I'm a software engineer. I spent most of my career doing back end engineering at Dropbox. Recently, I took a little bit of a detour into finance, and then I've been working on great start up projects, and then that led me back to doing Python performance as a startup which is what I'm currently doing. And do you remember how you first got introduced to Python? Yeah. It was at a summer internship in quantitative finance. I was doing a bunch of numerical analysis and it was all in Python, and I had to learn Python for that. And that was also my first introduction to Python performance problems.
I was waiting quite a bit for my analysis to run. So during that time, I was researching how to make it faster and just being a little disappointed that there weren't more options for things to do. This was also, you know, several years ago. So I learned a little bit about what you could do and also learn some stuff to distribute Python to multiple machines and that was my first introduction to Python.
[00:02:09] Unknown:
Now you've been working on the Piston project for a while now because I know that it started while you were at Dropbox. And as you mentioned, you've now decided to pursue continued development on it as a startup. I'm wondering if you could just give a bit more of a description about what the project is and how it got started.
[00:02:25] Unknown:
Yeah. Sure. So Pysen is a new implementation of Python that with the goal of being faster. And well, I say new implementation, but it's really a a fork of CPython, the reference implementation in terms of the technical implementation of it. In terms of how it got started, it got started when I was at Dropbox and Dropbox was spending a lot of money running Python and that amount of money kept growing quickly. So we decided that it made sense to put together a small team to try to reduce our Python server costs by making Python faster. And that's how it got started. That's the original goal. We worked on that. We decided to do it from scratch implementation and that's where it got started. And then eventually, because we started a completely new from scratch implementation, it took a couple years to get it working.
And in that time, Dropbox decided to move off of Python, or start transitioning off of Python. So the cost analysis became very different and the project was canceled. And then fast forward a couple of years, we had some ideas on how to tackle it in a more resource efficient way. So we decided to do that and we started a company to do it and that is what this incarnation of iViscen is.
[00:03:48] Unknown:
And I'm curious what your motivation is for taking the your time and energy and focusing on Piston and just the overall realm of Python optimization and what it is about it that interests you enough that you're willing to dedicate so much of your focus on that and build a company around it.
[00:04:06] Unknown:
Yes. I think a big part of it is is just a technical itch that I really wanna scratch. It just really bothers me that Python is not as fast as it could be. And I feel very attracted to that problem and feel like it should be solved. And if it hasn't been solved yet, then I wanna help solve it. So there's some some just personal motivation there. And then it also dovetails nicely with my overall career goals in terms of producing something that maybe a lot of developers use that could help people produce better software. And I think if we do a good job at this, it could help help a lot of people and have a really exciting.
[00:04:51] Unknown:
Another thing worth digging into is just the overall notion of performance because there are a number of different ways that you can approach that. And 1 of the common ways that people will discuss it in the context of Python is that its execution speed isn't as fast as other languages. But in terms of optimizing for developer throughput, it's actually very good for that because of the various capabilities of the language. And then in terms of the actual speed component, there are again other ways to slice it where it might be startup time, it might be focusing on optimization of tight loops, it might be, you know, reducing the overall execution time for a particular workload. I'm wondering if you can just discuss the primary use cases that you're focused on and the areas of improvement that you're have been able to see the biggest gains.
[00:05:44] Unknown:
So on the first point in terms of developer productivity, I definitely believe that Python is very good for y'all developer productivity. And when I explain what I'm working on to people who are not engineers, who who really don't know what Python is, I describe Python as a language that people like because it's very powerful but it's slow because of all those powerful features. And that does sort of reflect my view on its position. And I think we've seen a number of cases where people start projects in Python for developer productivity and then it grows very large and it becomes infeasible to rewrite it in another language.
So then people start caring about performance a lot more at that at that point. In terms of your other point, yeah, there are definitely a ton of different aspects of performance and that's 1 of the things that I feel like we were very careful about is deciding what exactly we wanna solve. That there are a lot of different things that you could try to improve. There's a lot of different use cases you can try to support. And depending on what you pick, I think you end up with very different solutions to them. And there's also this little bit of an urge by some people to do something that's very technically interesting and then later try to figure out what problems it solves.
But we started from a specific use case that we want to improve. And this is going back to how we got started in terms of wanting to improve web serving use cases. And so web serving, in my view, is characterized by a couple things. 1 is that the code bases tend to be quite large that or especially if there aren't a small number of hot spots that you really just need to optimize, you know, a couple loops and things will be much better. You have to optimize a wide swath of code. And web serving is also not very numerical in nature, So speeding up numerical computation doesn't really help it. People care a little bit about warm up time, but it's definitely not in the realm of, you know, command line script where it's basically all warm up time.
And yeah. So the things that end up mattering for performance for web serving look a little bit different and surprised us along the way that we spend most of our time improving attribute accesses, which was not really on our radar as a fundamental thing to improve but has actually been 1 of our biggest wins is making attributes way faster to access.
[00:08:17] Unknown:
And so there have been a number of different efforts over the years to improve the overall performance and speed capabilities of Python. You know, notably, PyPy is 1 of the most long running projects, but then there's also been Jython for trying to leverage some of the capabilities of the Java Virtual Machine. There's the Pigeon project from Microsoft that's trying to hook into CPython and expose the capability of JIT compiling using the dotnet runtime. I'm wondering what you see as the available space for Piston and just the overall Python ecosystem for another project that's focused on performance and execution speed?
[00:08:59] Unknown:
So there've been yeah. As you mentioned, a couple attempts to run Python on existing mature VMs such as the JVM or the the CLR. And I'm a little skeptical of those approaches based on my experience that looking at the things that we have improved, they are not things that would be provided by a VM. They're very language specific things. We've done a lot of very Python specific optimizations. And it's been nice to be able to express those in terms of a very simple JIT compiler that we have. And I'm pretty happy with that trade off. I don't think it means that it definitely wouldn't work. I think it's definitely I'm curious to see how these other projects go. But we definitely made a conscious choice to not build on top of an existing VM with this in.
High Fire is a super interesting example that they are quite successful in certain domains. And this kinda goes back to what I was saying earlier that there's a lot of different things you can try to optimize, a lot of different use cases that need different approaches. And PyPI has picked a set of approaches and sort of a thesis on how to make Python faster that I think makes it really good for certain domains and a little bit less applicable in other domains. So they are quite a bit faster for a lot of things, but then they seem to not always be a surefire win for things like web serving.
It's a little hard to know how much they've been adopted, but it seems like it's a they struggle a little bit in this domain. So it's not a good or bad thing. I think it's hard to build 1 thing that magically solves all of Python performance problems. So IPAD definitely exists and it's good for some things. We think it is not the right solution for web serving. So that is why we have done PISN because we think that need is being underserved right now.
[00:11:05] Unknown:
Another element of the current state of PyPI, there are attempts at addressing it in the form of things like the hpy project, but is the overall scope of compatibility with the existing ecosystem of Python libraries, particularly for implementations that you see or compiled languages and that use Python as a wrapper. And that's another element of what you're working on with Piston is this focus on compatibility. And so I'm wondering if you can just dig a bit more into that and some of the overall challenges that exist in that space, particularly as you try to improve the capabilities of the CPython implementation.
[00:11:44] Unknown:
Yeah. That's a really good point. I think that is a point of issue with PyPI that there's c extension support. It's much better these days but could still be really slow. So we definitely are taking the approach that compatibility with c extensions is very important that you can't write them off as something separate. They're part of the Python ecosystem. If you wanna build a Python implementation, you have to support this very important part of it. Personally, when it comes to projects to shrink the c API, I'm a little skeptical. I do very much dislike how much is accessible through the c API. And it sort of scares us that any change we make with Piston is theoretically observable because the CAPI exposes every single thing about your implementation.
So so that's a little scary but in our case, you can only see it if you really do some unsafe memory accesses and things like that. But so in general, I like the idea of having a smaller CPI and removing certain things. But I think a lot of the things that give implementations trouble are people actually using features that are quite expressive from the c API. And I don't think there's a simpler way to express these features that make it easier for other implementations to implement them. I think it's fundamentally that people are asking for very powerful features and you have to either decide to support it or not. So we ran into this with Piston the first version of Piston that we had a it wasn't quite an emulation layer, but we built an alternate implementation of the c API.
And we tried to put as few things in it as possible. As we were going through the extension modules that Dropbox used, we just had to add more and more to it until it looked very similar. We weren't able to remove a ton from the c API. So I think that experience makes me feel like it's going to be tough to reduce the surface area of it without eliminating the extensions that you were wanting to migrate in the first place. So I don't feel sure of that. That is my belief and that is definitely informing how we have been proceeding with this in in that we want to, as natively as possible, support this API. And that's what we've worked, see Python, and have made as few changes to it as possible.
[00:14:15] Unknown:
And I definitely wanna dig a bit more into the specifics of how you've built Piston. But before we get to that, going back to the hpy project and what you're discussing with the c API, I'm wondering if you have looked at all invested any of your time either contributing to or digging into that overall space.
[00:14:39] Unknown:
I have to say I'm not super familiar with it. I think I read the the overview, but I have not looked at the technical details and I have not dug into it to contribute to it myself. So I might be completely off based on very cursory understanding of it. I don't know about it in full detail.
[00:14:55] Unknown:
Now digging a bit more into the piston project itself, can you talk through the overall technical approach that you're taking with it and some of the changes that you've had to make to the CPython implementation to improve the overall execution speed?
[00:15:11] Unknown:
So we have a number of techniques that we use. The 1 that I think gets the most attention is that we added a JIT compiler to it. Something that just in time produces machine code. So we took the core interpreter loop of Python and if a function gets executed enough, we will generate machine code that is essentially a specialized version of the interpreter that just interprets that 1 function. And this is faster because you can strip away a lot of the dynamic behavior of the interpreter and you can also resolve a lot of things at compilation time at at the time that you jit the function so it doesn't have to figure it out every time it's run.
So this is a headline feature and often people will refer to an entire project like this as a JIT even though there's a lot of other things that happen with it. I've actually been a little bit on record of saying I didn't think that a Jit compiler would be that helpful for Python. And I'm not sure yet how much the evidence backs that up or not. We found that our JIT compiler gives us maybe 25% of our speed ups. It's a little hard to measure because a lot of techniques are sort of interwoven and it's hard to attribute speed ups to 1 or another. But I would guess that out of our 20% faster that we are, 5% of that is from the JIT. So that is the headline feature that we have, but not even our the primary source of our speed ups.
The main source for speed ups, as I mentioned earlier, is actually attribute accesses. I don't remember the exact numbers, but I think it might be, like, 8% of our speed up comes from 8 percentage points comes from speed and batch reboot accesses. So some of this is we took the work that has been happening in the recent versions of CPython to cache attribute accesses, and we just expanded on it a lot. We do it a lot more aggressively. We handle a lot more cases than CPython supports. And then with our JIT, we can also make a much lower overhead version of the cash lookup.
So it's not as sexy as some other things that we've tried, but just doing the sort of bread and butter optimization of caching has actually been quite a big percentage of our improvements that we cache some other things as well and it's these things up a lot. At eye level, we found a bunch of things that were worth caching because I think people have not looked at the web serving use case a ton. So there's been some low hanging fruits. I think people have put a lot of work into optimizing Python, but it's tended to be on simpler benchmarks that show certain types of behavior. So for example, 1 thing that we optimize is attribute misses.
That this is something that is not common in a standard language shootout benchmark. But in production code, there's a lot of sort of reflective dynamic code where we'll look up if an attribute exists on an object. And this is relatively expensive because it it creates an exceptional object and throws it, and then there's just a bunch of overhead that makes it more expensive than actually retrieving out the attribute. So we set all of that up. It wasn't super sophisticated. It was just I think people haven't looked at how fast that is and we made it a little bit faster. So there's some things like that too.
In general, I think our approaches have gotten less sexy over time. That our original approach with Pearson version 1 was to do it all from scratch. That we rebuilt everything, tried to find the best version of everything, we replaced reference counting with garbage collection, we added a full JIT. We did a lot of different things. And a lot of that ended up we had to get rid of it for compatibility reasons. And then when we started Piston version 2, we started with this technique called quickening or sometimes called quickening. I'm not sure if there's if it's called the same thing in every context. And we have this very fancy system to automatically produce traces of CPython. It was a c tracing JIT, and we were very proud of ourselves. It was so sexy.
But then once we actually started testing it out, it wasn't helping on macro benchmarks at all. It did a pretty good job on micro benchmarks, but it didn't really help with macro benchmarks at all. So that's an interesting lesson for us to sort of make sure to say benchmark focus. We kept it in because people will, I think, compare us on micro benchmarks, and so it's nice to have some micro benchmark specific ideas. But that is something that we built and didn't get a lot of mileage out of. So, yeah, in the end, it's mostly caching is the biggest single thing that we have. Right?
[00:20:04] Unknown:
And as you continue to dig into the performance capabilities of Python, I'm curious if you see a particular point where you might reach an asymptote of capabilities of the Python language or at least the CPython implementation of it and how far you're able to push it to gain improvements.
[00:20:23] Unknown:
Yeah. Definitely. I think in some areas, we're already seeming to bump up against that. That it seems like for at least this use case, the ceiling might not be as high as some people expect. So we're currently about 20% faster, but, you know, maybe the ceiling is 40 or 50% faster. It's not 4 times faster. Not unless you start changing some of the semantics of the language. So we do have some plans to do that. We think there's certain things within Python. There's certain features that everyone is paying for by having them included in Python, but in typical usage aren't being used. They tend to be debugging features that are always enabled.
So we might remove some of those or maybe have a separate non debug build up this in. So we are having to already reach for some ideas like that. We set up those debugging features, but to maintain semantic compatibility, you still need some amount of them in there, and there's only so far that you can optimize them out. So still unknown exactly how much we can optimize this, but we have just run into a lot of cases where people use a lot of the expensive features and you have to support them and there's only so fast you can make it. So for example, property objects.
In Python, there's this class called the property class. And when you have an attribute that's a property, accessing it will call a function for you to determine the value of the property. And we found that these are used fairly often and but often the the function to get it is a very simple function. It might just look up a single other attribute. So this is quite expensive and we've we're trying to think if there's some way to notice that not all the semantics of Python are invoked for this kind of function call. But so far, it looks like you have to do quite a bit of expensive bookkeeping to just implement the property function call. So we are running into cases where it seems like there's just a lot of expensive work being done.
[00:22:27] Unknown:
Another area that a lot of people have spent time discussing and working on is the global interpreter lock and the ability to expand execution across multiple cores for Python, particularly given the fact that so many computers these days have reduced the overall clock speed for an individual core but increased the number of cores available. And I'm wondering if that's an area that you're investigating at all.
[00:22:54] Unknown:
Yeah. We actually produce a version of Piston version 1 that did not have the global interpreter lock, and it was a fair bit slower. It did have enough scalability that if you run it on enough cores, it would make up for it, but it does make single threaded performance slower. I think 1 thing that people don't realize is that getting rid of the interpreter lock means getting rid of reference counting. I think people believe that you can do multi threaded reference counting by doing atomic operations on your reference counts. So on x86, that's at the lock ink instruction, but that's actually not sufficient to do, say, multi threaded reference counting.
You need a longer primitive that is not supported on processors that I know of that does multiple memory accesses in an atomic fashion. So it's possible that you can do it with hardware transactional memory on Intel Processors. But anyway, my general belief is that getting rid of the GIL, the Global Interpretor Lock, will first require getting rid of reference counting. And the first version of Piston did not have reference counting. We had a garbage collector, and we found that there are a lot of compatibility issues by having the garbage collector versus having reference counting.
That, in particular, the main semantic difference between the 2 is when your disruptors get called. That with reference counting, they get called predictably as soon as you lose the last reference to an object. And so this means that you're guaranteed that your destructors get called in a timely fashion. And so in some sense, you're not supposed to, but you can use destructors to clean things up for you. But when you switch to a garbage collector, the destructors are called at some unknown future time, and it might be too late, or the code might have been written assuming that it gets called immediately.
So we actually found that the Python standard library, the weak ref module, has some cases where it relies on timely destruction of objects. And so we had to actually patch the standard library to work with the garbage collector. And then we just kept running into issues of this in the Dropbox code base where people would accidentally rely on this. So that's the main issue of giving a preference counting, I think. The other issue is that reference counting is also very memory efficient. With garbage collection, typically, keep a fairly large overhead of memory available to do your allocation over with reference counting, that doesn't happen. Kerber's collection has other benefits of if you have a packing collector, then you can get better efficiency that way. But I think you definitely see that something like PyPI will tend to use quite a bit more memory than something like CPython.
So there's a long story. I think that the GIL removing the GIL will first require removing reference counting, and I think that that will be a huge change. I think that can only happen at a major version change of Python. And I don't know if there's even the appetite to do that. And I am skeptical given my experience with it. I I definitely do want a GIL list version and that's why we put the time in to build a version of it that didn't have the GIL. But I I think realistically, Python is gonna have the GIL for a long time and it's gonna it takes so much effort to remove it because of all these things that it will have to imply that I am not counting on it happening anytime soon.
[00:26:30] Unknown:
In terms of your overall work on Piston, I'm curious if you can discuss your approach to sustainability and the overall governance efforts of the Piston project itself and just your perspective on your relationship with the CPython implementation and the core team there and what your thoughts are on contributing some of the work that you're doing back upstream?
[00:26:55] Unknown:
Yeah. So we haven't thought about this a ton. We've mostly deal with those issues first. For the moment, we're a closed source. We're deal with those issues first. For the moment, we're closed source and so that is the way we're handling it is that, you know, there's not a lot of governance. It's just our team working on it. We plan to open source it at some point, and we haven't thought a lot about exactly the governance structure for that. I haven't looked into it at all. I wonder if there's some world in which this is also sort of under the umbrella of the PSF, the the Python Software Foundation. The 1 that I think sort of governs the CPython, code base.
I think it will live as separate code bases. I'm not really sure. I think people have asked about this and it seems like, well, well, I think it might be technically possible to have all of our stuff in the c Python code base. I think there might be a little bit of hesitation to do that of having all these extra dependencies that are conditionally enabled, a lot of conditionally enabled code. And I'm not ruling it out, but there has been some hesitation on those sorts of things. So it's not clear if this will have to exist as a separate project or how much they can get merged at some point. But we definitely have certain features that make a lot of sense just being merged into CPython, and at some point we'll do that. But there's a lot of things like our quickening implementation, for example, is where it has some pretty heavy weight dependencies.
Our our JIT, for example, right now only runs on x8664 bit. So we're not as portable as I think the CPython developers want their code to be. So it's an open question of exactly how much we can merge back, and at some point, we'll try to do that. And I think ideally, it our project diverges as little as possible from CPython. And I don't know exactly what it would mean, but keeping it close to that, I think, would be a good thing.
[00:29:04] Unknown:
In terms of the overall work that you've done both on the original implementation of Piston, but particularly with this v 2 approach of forking CPython and adding changes to the existing implementation. I'm wondering what you see as being the biggest challenges that you have experienced so far and some of the potential roadblocks that you foresee coming up in the, you know, months as far as the overall road map that you have for the project.
[00:29:32] Unknown:
Yeah. I think the biggest challenge is in getting adoption that we knew this going in. That if you look at something like PyPy, they seem very technically successful, but they also seem to not have super widespread adoption. So I think getting adoption is a first order problem with a even a very technical project like this. So that's why we decided to make it as adoptable as possible by making it a port, making this compatibility as compatible as possible. And even with that, we've seen people be a little hesitant to adopt it. And so we're in the process of getting more user feedback, understanding exactly what is making people hesitant.
There's certain technical things that we could do to maybe ease some of them, but getting more adoption, I think, is our main challenge at this point. It could be that people want us to be even faster and then they'll adopt it, but I don't think that that is exactly where people are at right now. But I suppose we'll find out by talking to more people. So I would definitely say adoption and we have a number of things that we could do around it and figuring out so we have to figure out which ones to do and then do them.
[00:30:47] Unknown:
I'm also interested in understanding how you are gathering use cases and examples of optimization problems and areas of improvement, and then how you approach understanding what are the performance impacts from the various aspects of the implementation and the language and how to target your development efforts to try and get the most impact for the amount of work that you're able to put in?
[00:31:15] Unknown:
So in terms of finding people to talk to, we have a Discord chat room now and some people have found us through that. And then proactively, we've been seeking out companies that we know are running web deployments on Python and trying to get their impressions of what their problems are. In terms of the day to day, we have a set of benchmarks that we have produced. We have a Django benchmark and a Flask benchmark because these are the 2 most popular Python web frameworks. And I believe they're a step forward in terms of benchmarks, but there's obviously a lot more that we can do. They're still very artificial in nature. So, ideally, we can find some projects or some team that's willing to contribute a benchmark that is more realistic.
But then on a day to day basis, we just look at the benchmark results as we look at the benchmark profiles to guide us on what to improve and where the different time is spent and things like that.
[00:32:17] Unknown:
As far as the people who have been using Piston either in your current incarnation where you're building a business around it and you're looking for people to adopt it because they have production use cases that they need to optimize, But also in your time at Dropbox, I'm wondering what you've seen as some of the most interesting or unexpected or innovative ways that Piston has been used.
[00:32:38] Unknown:
1 thing that was very unexpected was someone got Piston working on Windows through the Windows subsystem for Linux that we only produce builds build packages for Linux and we were planning on producing some of the Mac and we just weren't thinking about Windows. And then someone found us and let us know that it was working through on Windows. So that was a very nice surprise. I think, in general, the less nice surprise has been just how high the hurdle is to get someone to adopt a new implementation. Goes well with our choice to choose web serving because I think web serving as opposed to other parts of Python Python ecosystem has big players involved.
Whereas you don't have, you know, a large number of individual data scientists each making their own decisions. You might have a large company with a a platform team that's going to make this decision that will affect how they run Python. So we just need to get a smaller number of people who might be very interested to adopt it rather than getting a large mass of mildly interested people to try to adopt it. But even with that, even with having there being people that we think we can help quite a bit, it's still been a process to go from having a technical solution to something to going to have actual users who are using it on a day to day basis.
[00:34:01] Unknown:
In your own experience of building Piston and trying to build a business around it, what are some of the most interesting or unexpected or challenging lessons that you've learned there?
[00:34:12] Unknown:
I think adoption is still a big 1. And then as I mentioned earlier, we learned the hard way that you don't really get rewarded for having a very fancy system. You get rewarded for having something that solves a user's problem. So as I said earlier, we've been continually making Piston a little bit less sexy from a technical level as we've been optimizing more for solving what we hope are users actual problems. That's a little bit of a tough lesson to learn but I think it has really made our product a lot better. Because the other thing is as we've been thinking about business models for this, it's been tough to come up with a business model for this that there are number of open source business models, but they are a little bit tough for a project like this.
There's some closed source models, but they're not super common. So we thought it might be easier to monetize something that people wanted a lot, but it is seeming that is a tricky thing that we're still trying to figure out.
[00:35:15] Unknown:
And for people who are considering whether to use Piston for their workloads, what are the cases where it's the wrong choice and they would be better served either just staying with the CPython implementation or using something like PyPy or 1 of the other projects that's aimed at various aspects of performance?
[00:35:32] Unknown:
So we aim to be moderately helpful for all use cases. And so I think that implies a couple cases that this is maybe not the right choice. 1 is we're not quite there. There's certain use cases where due to certain practicalities, for example, it's it's a little bit tough to run AI workloads on Piston right now just due to some packaging minutiae. So we should we're hopeful to get that solved soon. But if you're running AI stuff, you currently have to use a sort of unsafe workaround to run that, and it's not really recommended at the moment. So it's just a temporary thing. I don't think it's fundamental to our approach. The other case I would say is that our approaches have been pretty general and try to help all code a decent amount.
But there are certain types of code, like, if you have highly numeric code, there are gonna be options out there that will speed up your type of code a lot. So if you have numeric code and you run it on Piston, you'll get a good speed up, but you could run it on something like Pypi or Cython or Numba or any of these things and get a much more dramatic speed up. So calling it a wrong choice is a little strong, but you could definitely get more bang for your buck if you work in certain specific fields where there's specialized tools just for that area.
[00:36:53] Unknown:
Given the amount of time that you've spent digging through the Python code base and working with people who are trying to optimize the performance of their projects, Are there any aspects of the language that you, in retrospect, regret and that you wish didn't need to be supported and would make your job easier for improving the execution capabilities of the language?
[00:37:17] Unknown:
That's a good question. I think there are a number of minor things. I mentioned earlier that there's a number of debugging features that are always enabled and I think we might remove some of those. Most people don't even know that these exist because they're very low level things that I I think most people don't use and we might remove some of them. There's certain other, kind of, things that are a little bit difficult which is that, for example, throwing an exception will basically take a snapshot of the execution state at that point. And in CPython, this is not expensive at all because they're already maintaining the structures to do this.
But it prevents optimizations where you only produce these sort of dynamic structures on the fly if they're requested. And we tried this with PISN v 1 where we would generate these on the fly, but we ended up generating them so often because you have to generate them for every exception that it tended to not be super valuable.
[00:38:18] Unknown:
And as you continue to work on Piston, what do you have planned for the near to medium term of the project, and what are you most excited about digging into?
[00:38:27] Unknown:
I think I'm most excited to see people use it and to understand what will get them to that point. I think we have a pretty compelling technical solution right now and it's interesting. I'm excited to go through the process of going from that to getting people to actually use it. So as I mentioned earlier, we're not a 100% sure what is stopping people from that. And I think as we figure that out, that will influence a lot of where we put our work. But I think some things that are probably all involved is well, first of all, we can always be faster and improving that will always make our options seem a little bit more compelling.
And then there are certain things around compatibility where, in theory, on paper, we are a 100% compatible with CPython because we fought from them and didn't change any any of the relevant things. But in practice, there's a number of compatibility issues that have come up. For example, there's kind of the equivalent of user agent parsing. So like like a web browser will identify itself as a certain user agent and all the web browsers kind of lie and pretend to be each other in order to get all the right features enabled. There's an extra equivalent thing where a Python implementation says what implementation it is, and then some compatibility code will determine what features to enable based on that. So I think we found a combination of, like, when to say we're CPython, when to say we're Piston that enables all the features but also is safe. And but I think there's a little bit more we can do on that front.
And then there's just some, like, packaging issues around getting prebuilt packages of the AI libraries and things like that. So an assortment of things that we wanna do, and they're mostly focused around either performance or making it easier for people to adopt it.
[00:40:19] Unknown:
Are there any other aspects of the work that you're doing on piston or just the overall space of performance optimization and improvements and the overall compatibility with the Python ecosystem that we didn't discuss yet that you'd like to cover before we close out the show?
[00:40:35] Unknown:
I'll just go on a little mini rant about that last thing I talked about which is the compatibility detection. I think it would work better if there were flags that an implementation exposed about do I support this, do I support that rather than having third party code look at your version string and determine what features are enabled or not. We're having some difficulty where there's all this code that's already deployed out there that checks for certain things in a very specific way. And we don't have the ability to change, for example, the code that is currently shipping with Ubuntu that might detect certain things about this or not.
So my wish would be that we move some more of a flag based detection system than a, like, implementation name plus version number based detection system.
[00:41:27] Unknown:
Well, for anybody who wants to follow along with the work that you're doing or try out Piston or get in touch, I'll have you add your preferred contact information to the show notes. And so with that, I'll move us into the picks. And this week, I'm going to choose the last week in AWS newsletter. It's a newsletter run by Corey Quinn, so it's full of lots of snark and humor, but it's also a great way to keep up to date with all of the various things that change in the Amazon Web Services catalog and various projects and information around that. So as somebody who uses AWS for a lot of my day to day work, it's a useful way to keep up to date. So definitely worth checking out if you happen to use anything from that ecosystem as well. And so with that, I'll pass it to you, Kevin. Do you have any picks this week?
[00:42:12] Unknown:
I think my pick will, in general, be meditation. I've been getting into meditation lately and I found it to really pay for itself essentially that I think you do meditation not just to get certain benefits, but it's also I do feel like meditation has made me more productive, like, more so than the time that I've invested in meditation and solve. So that's been very pleasant to see. And maybe in terms of a specific thing to look at, I use the Calm app, which I do enjoy. I've heard people also like the His Face app, and I don't know if there are other apps. But I like that it makes it very sort of digestible to just do a little bit of meditation.
[00:42:52] Unknown:
Definitely worth taking a look at, particularly given the hectic state of the world right now. So thank you very much for taking the time today to join me and share the work that you've been doing at Piston. It's definitely an interesting project and 1 that I've been keeping an eye on since it was first announced from your time at Dropbox. So I'm glad to see that you've been able to continue along with it, and I definitely look forward to seeing where you're able to take it going forward. So thank you again for all the time and effort you've put into that, and I hope you enjoy the rest of your day. Thanks, and thank you very much for having me. Thank you for listening. Don't forget to check out our other show, the Data Engineering Podcast at dataengineeringpodcast.com for the latest on modern data management.
And visit the site at pythonpodcast.com to subscribe to the show, sign up for the mailing list, and read the show notes. And if you've learned something or tried out a project from the show, then tell us about it. Email host@podcastinit.com with your story. To help other people find the show, please leave a review on Itunes and tell your friends and coworkers.
Hello, and welcome to Podcast Dot in It, the podcast about Python and the people who make it great. When you're ready to launch your next app or want to try a project you hear about on the show, you'll need somewhere to deploy it. So take a look at our friends over at Linode. With the launch of their managed Kubernetes platform, it's easy to get started with the next generation of deployment and scaling powered by the battle tested Linode platform, including simple pricing, node balancers, 40 gigabit networking, dedicated CPU and GPU instances, and worldwide data centers.
Go to python podcast.com/linode, that's l I n o d e, today and get a $100 credit to try out a Kubernetes cluster of your own. And don't forget to thank them for their continued support of this show. Your host as usual is Tobias Macy. And today, I'm interviewing Kevin Mazilevskiy about his work on Piston, an interpreter for Python focused on compatibility and speed. So, Kevin, can you start by introducing yourself?
[00:01:08] Unknown:
Sure. Hi. I'm Kevin Moduleski. I'm a software engineer. I spent most of my career doing back end engineering at Dropbox. Recently, I took a little bit of a detour into finance, and then I've been working on great start up projects, and then that led me back to doing Python performance as a startup which is what I'm currently doing. And do you remember how you first got introduced to Python? Yeah. It was at a summer internship in quantitative finance. I was doing a bunch of numerical analysis and it was all in Python, and I had to learn Python for that. And that was also my first introduction to Python performance problems.
I was waiting quite a bit for my analysis to run. So during that time, I was researching how to make it faster and just being a little disappointed that there weren't more options for things to do. This was also, you know, several years ago. So I learned a little bit about what you could do and also learn some stuff to distribute Python to multiple machines and that was my first introduction to Python.
[00:02:09] Unknown:
Now you've been working on the Piston project for a while now because I know that it started while you were at Dropbox. And as you mentioned, you've now decided to pursue continued development on it as a startup. I'm wondering if you could just give a bit more of a description about what the project is and how it got started.
[00:02:25] Unknown:
Yeah. Sure. So Pysen is a new implementation of Python that with the goal of being faster. And well, I say new implementation, but it's really a a fork of CPython, the reference implementation in terms of the technical implementation of it. In terms of how it got started, it got started when I was at Dropbox and Dropbox was spending a lot of money running Python and that amount of money kept growing quickly. So we decided that it made sense to put together a small team to try to reduce our Python server costs by making Python faster. And that's how it got started. That's the original goal. We worked on that. We decided to do it from scratch implementation and that's where it got started. And then eventually, because we started a completely new from scratch implementation, it took a couple years to get it working.
And in that time, Dropbox decided to move off of Python, or start transitioning off of Python. So the cost analysis became very different and the project was canceled. And then fast forward a couple of years, we had some ideas on how to tackle it in a more resource efficient way. So we decided to do that and we started a company to do it and that is what this incarnation of iViscen is.
[00:03:48] Unknown:
And I'm curious what your motivation is for taking the your time and energy and focusing on Piston and just the overall realm of Python optimization and what it is about it that interests you enough that you're willing to dedicate so much of your focus on that and build a company around it.
[00:04:06] Unknown:
Yes. I think a big part of it is is just a technical itch that I really wanna scratch. It just really bothers me that Python is not as fast as it could be. And I feel very attracted to that problem and feel like it should be solved. And if it hasn't been solved yet, then I wanna help solve it. So there's some some just personal motivation there. And then it also dovetails nicely with my overall career goals in terms of producing something that maybe a lot of developers use that could help people produce better software. And I think if we do a good job at this, it could help help a lot of people and have a really exciting.
[00:04:51] Unknown:
Another thing worth digging into is just the overall notion of performance because there are a number of different ways that you can approach that. And 1 of the common ways that people will discuss it in the context of Python is that its execution speed isn't as fast as other languages. But in terms of optimizing for developer throughput, it's actually very good for that because of the various capabilities of the language. And then in terms of the actual speed component, there are again other ways to slice it where it might be startup time, it might be focusing on optimization of tight loops, it might be, you know, reducing the overall execution time for a particular workload. I'm wondering if you can just discuss the primary use cases that you're focused on and the areas of improvement that you're have been able to see the biggest gains.
[00:05:44] Unknown:
So on the first point in terms of developer productivity, I definitely believe that Python is very good for y'all developer productivity. And when I explain what I'm working on to people who are not engineers, who who really don't know what Python is, I describe Python as a language that people like because it's very powerful but it's slow because of all those powerful features. And that does sort of reflect my view on its position. And I think we've seen a number of cases where people start projects in Python for developer productivity and then it grows very large and it becomes infeasible to rewrite it in another language.
So then people start caring about performance a lot more at that at that point. In terms of your other point, yeah, there are definitely a ton of different aspects of performance and that's 1 of the things that I feel like we were very careful about is deciding what exactly we wanna solve. That there are a lot of different things that you could try to improve. There's a lot of different use cases you can try to support. And depending on what you pick, I think you end up with very different solutions to them. And there's also this little bit of an urge by some people to do something that's very technically interesting and then later try to figure out what problems it solves.
But we started from a specific use case that we want to improve. And this is going back to how we got started in terms of wanting to improve web serving use cases. And so web serving, in my view, is characterized by a couple things. 1 is that the code bases tend to be quite large that or especially if there aren't a small number of hot spots that you really just need to optimize, you know, a couple loops and things will be much better. You have to optimize a wide swath of code. And web serving is also not very numerical in nature, So speeding up numerical computation doesn't really help it. People care a little bit about warm up time, but it's definitely not in the realm of, you know, command line script where it's basically all warm up time.
And yeah. So the things that end up mattering for performance for web serving look a little bit different and surprised us along the way that we spend most of our time improving attribute accesses, which was not really on our radar as a fundamental thing to improve but has actually been 1 of our biggest wins is making attributes way faster to access.
[00:08:17] Unknown:
And so there have been a number of different efforts over the years to improve the overall performance and speed capabilities of Python. You know, notably, PyPy is 1 of the most long running projects, but then there's also been Jython for trying to leverage some of the capabilities of the Java Virtual Machine. There's the Pigeon project from Microsoft that's trying to hook into CPython and expose the capability of JIT compiling using the dotnet runtime. I'm wondering what you see as the available space for Piston and just the overall Python ecosystem for another project that's focused on performance and execution speed?
[00:08:59] Unknown:
So there've been yeah. As you mentioned, a couple attempts to run Python on existing mature VMs such as the JVM or the the CLR. And I'm a little skeptical of those approaches based on my experience that looking at the things that we have improved, they are not things that would be provided by a VM. They're very language specific things. We've done a lot of very Python specific optimizations. And it's been nice to be able to express those in terms of a very simple JIT compiler that we have. And I'm pretty happy with that trade off. I don't think it means that it definitely wouldn't work. I think it's definitely I'm curious to see how these other projects go. But we definitely made a conscious choice to not build on top of an existing VM with this in.
High Fire is a super interesting example that they are quite successful in certain domains. And this kinda goes back to what I was saying earlier that there's a lot of different things you can try to optimize, a lot of different use cases that need different approaches. And PyPI has picked a set of approaches and sort of a thesis on how to make Python faster that I think makes it really good for certain domains and a little bit less applicable in other domains. So they are quite a bit faster for a lot of things, but then they seem to not always be a surefire win for things like web serving.
It's a little hard to know how much they've been adopted, but it seems like it's a they struggle a little bit in this domain. So it's not a good or bad thing. I think it's hard to build 1 thing that magically solves all of Python performance problems. So IPAD definitely exists and it's good for some things. We think it is not the right solution for web serving. So that is why we have done PISN because we think that need is being underserved right now.
[00:11:05] Unknown:
Another element of the current state of PyPI, there are attempts at addressing it in the form of things like the hpy project, but is the overall scope of compatibility with the existing ecosystem of Python libraries, particularly for implementations that you see or compiled languages and that use Python as a wrapper. And that's another element of what you're working on with Piston is this focus on compatibility. And so I'm wondering if you can just dig a bit more into that and some of the overall challenges that exist in that space, particularly as you try to improve the capabilities of the CPython implementation.
[00:11:44] Unknown:
Yeah. That's a really good point. I think that is a point of issue with PyPI that there's c extension support. It's much better these days but could still be really slow. So we definitely are taking the approach that compatibility with c extensions is very important that you can't write them off as something separate. They're part of the Python ecosystem. If you wanna build a Python implementation, you have to support this very important part of it. Personally, when it comes to projects to shrink the c API, I'm a little skeptical. I do very much dislike how much is accessible through the c API. And it sort of scares us that any change we make with Piston is theoretically observable because the CAPI exposes every single thing about your implementation.
So so that's a little scary but in our case, you can only see it if you really do some unsafe memory accesses and things like that. But so in general, I like the idea of having a smaller CPI and removing certain things. But I think a lot of the things that give implementations trouble are people actually using features that are quite expressive from the c API. And I don't think there's a simpler way to express these features that make it easier for other implementations to implement them. I think it's fundamentally that people are asking for very powerful features and you have to either decide to support it or not. So we ran into this with Piston the first version of Piston that we had a it wasn't quite an emulation layer, but we built an alternate implementation of the c API.
And we tried to put as few things in it as possible. As we were going through the extension modules that Dropbox used, we just had to add more and more to it until it looked very similar. We weren't able to remove a ton from the c API. So I think that experience makes me feel like it's going to be tough to reduce the surface area of it without eliminating the extensions that you were wanting to migrate in the first place. So I don't feel sure of that. That is my belief and that is definitely informing how we have been proceeding with this in in that we want to, as natively as possible, support this API. And that's what we've worked, see Python, and have made as few changes to it as possible.
[00:14:15] Unknown:
And I definitely wanna dig a bit more into the specifics of how you've built Piston. But before we get to that, going back to the hpy project and what you're discussing with the c API, I'm wondering if you have looked at all invested any of your time either contributing to or digging into that overall space.
[00:14:39] Unknown:
I have to say I'm not super familiar with it. I think I read the the overview, but I have not looked at the technical details and I have not dug into it to contribute to it myself. So I might be completely off based on very cursory understanding of it. I don't know about it in full detail.
[00:14:55] Unknown:
Now digging a bit more into the piston project itself, can you talk through the overall technical approach that you're taking with it and some of the changes that you've had to make to the CPython implementation to improve the overall execution speed?
[00:15:11] Unknown:
So we have a number of techniques that we use. The 1 that I think gets the most attention is that we added a JIT compiler to it. Something that just in time produces machine code. So we took the core interpreter loop of Python and if a function gets executed enough, we will generate machine code that is essentially a specialized version of the interpreter that just interprets that 1 function. And this is faster because you can strip away a lot of the dynamic behavior of the interpreter and you can also resolve a lot of things at compilation time at at the time that you jit the function so it doesn't have to figure it out every time it's run.
So this is a headline feature and often people will refer to an entire project like this as a JIT even though there's a lot of other things that happen with it. I've actually been a little bit on record of saying I didn't think that a Jit compiler would be that helpful for Python. And I'm not sure yet how much the evidence backs that up or not. We found that our JIT compiler gives us maybe 25% of our speed ups. It's a little hard to measure because a lot of techniques are sort of interwoven and it's hard to attribute speed ups to 1 or another. But I would guess that out of our 20% faster that we are, 5% of that is from the JIT. So that is the headline feature that we have, but not even our the primary source of our speed ups.
The main source for speed ups, as I mentioned earlier, is actually attribute accesses. I don't remember the exact numbers, but I think it might be, like, 8% of our speed up comes from 8 percentage points comes from speed and batch reboot accesses. So some of this is we took the work that has been happening in the recent versions of CPython to cache attribute accesses, and we just expanded on it a lot. We do it a lot more aggressively. We handle a lot more cases than CPython supports. And then with our JIT, we can also make a much lower overhead version of the cash lookup.
So it's not as sexy as some other things that we've tried, but just doing the sort of bread and butter optimization of caching has actually been quite a big percentage of our improvements that we cache some other things as well and it's these things up a lot. At eye level, we found a bunch of things that were worth caching because I think people have not looked at the web serving use case a ton. So there's been some low hanging fruits. I think people have put a lot of work into optimizing Python, but it's tended to be on simpler benchmarks that show certain types of behavior. So for example, 1 thing that we optimize is attribute misses.
That this is something that is not common in a standard language shootout benchmark. But in production code, there's a lot of sort of reflective dynamic code where we'll look up if an attribute exists on an object. And this is relatively expensive because it it creates an exceptional object and throws it, and then there's just a bunch of overhead that makes it more expensive than actually retrieving out the attribute. So we set all of that up. It wasn't super sophisticated. It was just I think people haven't looked at how fast that is and we made it a little bit faster. So there's some things like that too.
In general, I think our approaches have gotten less sexy over time. That our original approach with Pearson version 1 was to do it all from scratch. That we rebuilt everything, tried to find the best version of everything, we replaced reference counting with garbage collection, we added a full JIT. We did a lot of different things. And a lot of that ended up we had to get rid of it for compatibility reasons. And then when we started Piston version 2, we started with this technique called quickening or sometimes called quickening. I'm not sure if there's if it's called the same thing in every context. And we have this very fancy system to automatically produce traces of CPython. It was a c tracing JIT, and we were very proud of ourselves. It was so sexy.
But then once we actually started testing it out, it wasn't helping on macro benchmarks at all. It did a pretty good job on micro benchmarks, but it didn't really help with macro benchmarks at all. So that's an interesting lesson for us to sort of make sure to say benchmark focus. We kept it in because people will, I think, compare us on micro benchmarks, and so it's nice to have some micro benchmark specific ideas. But that is something that we built and didn't get a lot of mileage out of. So, yeah, in the end, it's mostly caching is the biggest single thing that we have. Right?
[00:20:04] Unknown:
And as you continue to dig into the performance capabilities of Python, I'm curious if you see a particular point where you might reach an asymptote of capabilities of the Python language or at least the CPython implementation of it and how far you're able to push it to gain improvements.
[00:20:23] Unknown:
Yeah. Definitely. I think in some areas, we're already seeming to bump up against that. That it seems like for at least this use case, the ceiling might not be as high as some people expect. So we're currently about 20% faster, but, you know, maybe the ceiling is 40 or 50% faster. It's not 4 times faster. Not unless you start changing some of the semantics of the language. So we do have some plans to do that. We think there's certain things within Python. There's certain features that everyone is paying for by having them included in Python, but in typical usage aren't being used. They tend to be debugging features that are always enabled.
So we might remove some of those or maybe have a separate non debug build up this in. So we are having to already reach for some ideas like that. We set up those debugging features, but to maintain semantic compatibility, you still need some amount of them in there, and there's only so far that you can optimize them out. So still unknown exactly how much we can optimize this, but we have just run into a lot of cases where people use a lot of the expensive features and you have to support them and there's only so fast you can make it. So for example, property objects.
In Python, there's this class called the property class. And when you have an attribute that's a property, accessing it will call a function for you to determine the value of the property. And we found that these are used fairly often and but often the the function to get it is a very simple function. It might just look up a single other attribute. So this is quite expensive and we've we're trying to think if there's some way to notice that not all the semantics of Python are invoked for this kind of function call. But so far, it looks like you have to do quite a bit of expensive bookkeeping to just implement the property function call. So we are running into cases where it seems like there's just a lot of expensive work being done.
[00:22:27] Unknown:
Another area that a lot of people have spent time discussing and working on is the global interpreter lock and the ability to expand execution across multiple cores for Python, particularly given the fact that so many computers these days have reduced the overall clock speed for an individual core but increased the number of cores available. And I'm wondering if that's an area that you're investigating at all.
[00:22:54] Unknown:
Yeah. We actually produce a version of Piston version 1 that did not have the global interpreter lock, and it was a fair bit slower. It did have enough scalability that if you run it on enough cores, it would make up for it, but it does make single threaded performance slower. I think 1 thing that people don't realize is that getting rid of the interpreter lock means getting rid of reference counting. I think people believe that you can do multi threaded reference counting by doing atomic operations on your reference counts. So on x86, that's at the lock ink instruction, but that's actually not sufficient to do, say, multi threaded reference counting.
You need a longer primitive that is not supported on processors that I know of that does multiple memory accesses in an atomic fashion. So it's possible that you can do it with hardware transactional memory on Intel Processors. But anyway, my general belief is that getting rid of the GIL, the Global Interpretor Lock, will first require getting rid of reference counting. And the first version of Piston did not have reference counting. We had a garbage collector, and we found that there are a lot of compatibility issues by having the garbage collector versus having reference counting.
That, in particular, the main semantic difference between the 2 is when your disruptors get called. That with reference counting, they get called predictably as soon as you lose the last reference to an object. And so this means that you're guaranteed that your destructors get called in a timely fashion. And so in some sense, you're not supposed to, but you can use destructors to clean things up for you. But when you switch to a garbage collector, the destructors are called at some unknown future time, and it might be too late, or the code might have been written assuming that it gets called immediately.
So we actually found that the Python standard library, the weak ref module, has some cases where it relies on timely destruction of objects. And so we had to actually patch the standard library to work with the garbage collector. And then we just kept running into issues of this in the Dropbox code base where people would accidentally rely on this. So that's the main issue of giving a preference counting, I think. The other issue is that reference counting is also very memory efficient. With garbage collection, typically, keep a fairly large overhead of memory available to do your allocation over with reference counting, that doesn't happen. Kerber's collection has other benefits of if you have a packing collector, then you can get better efficiency that way. But I think you definitely see that something like PyPI will tend to use quite a bit more memory than something like CPython.
So there's a long story. I think that the GIL removing the GIL will first require removing reference counting, and I think that that will be a huge change. I think that can only happen at a major version change of Python. And I don't know if there's even the appetite to do that. And I am skeptical given my experience with it. I I definitely do want a GIL list version and that's why we put the time in to build a version of it that didn't have the GIL. But I I think realistically, Python is gonna have the GIL for a long time and it's gonna it takes so much effort to remove it because of all these things that it will have to imply that I am not counting on it happening anytime soon.
[00:26:30] Unknown:
In terms of your overall work on Piston, I'm curious if you can discuss your approach to sustainability and the overall governance efforts of the Piston project itself and just your perspective on your relationship with the CPython implementation and the core team there and what your thoughts are on contributing some of the work that you're doing back upstream?
[00:26:55] Unknown:
Yeah. So we haven't thought about this a ton. We've mostly deal with those issues first. For the moment, we're a closed source. We're deal with those issues first. For the moment, we're closed source and so that is the way we're handling it is that, you know, there's not a lot of governance. It's just our team working on it. We plan to open source it at some point, and we haven't thought a lot about exactly the governance structure for that. I haven't looked into it at all. I wonder if there's some world in which this is also sort of under the umbrella of the PSF, the the Python Software Foundation. The 1 that I think sort of governs the CPython, code base.
I think it will live as separate code bases. I'm not really sure. I think people have asked about this and it seems like, well, well, I think it might be technically possible to have all of our stuff in the c Python code base. I think there might be a little bit of hesitation to do that of having all these extra dependencies that are conditionally enabled, a lot of conditionally enabled code. And I'm not ruling it out, but there has been some hesitation on those sorts of things. So it's not clear if this will have to exist as a separate project or how much they can get merged at some point. But we definitely have certain features that make a lot of sense just being merged into CPython, and at some point we'll do that. But there's a lot of things like our quickening implementation, for example, is where it has some pretty heavy weight dependencies.
Our our JIT, for example, right now only runs on x8664 bit. So we're not as portable as I think the CPython developers want their code to be. So it's an open question of exactly how much we can merge back, and at some point, we'll try to do that. And I think ideally, it our project diverges as little as possible from CPython. And I don't know exactly what it would mean, but keeping it close to that, I think, would be a good thing.
[00:29:04] Unknown:
In terms of the overall work that you've done both on the original implementation of Piston, but particularly with this v 2 approach of forking CPython and adding changes to the existing implementation. I'm wondering what you see as being the biggest challenges that you have experienced so far and some of the potential roadblocks that you foresee coming up in the, you know, months as far as the overall road map that you have for the project.
[00:29:32] Unknown:
Yeah. I think the biggest challenge is in getting adoption that we knew this going in. That if you look at something like PyPy, they seem very technically successful, but they also seem to not have super widespread adoption. So I think getting adoption is a first order problem with a even a very technical project like this. So that's why we decided to make it as adoptable as possible by making it a port, making this compatibility as compatible as possible. And even with that, we've seen people be a little hesitant to adopt it. And so we're in the process of getting more user feedback, understanding exactly what is making people hesitant.
There's certain technical things that we could do to maybe ease some of them, but getting more adoption, I think, is our main challenge at this point. It could be that people want us to be even faster and then they'll adopt it, but I don't think that that is exactly where people are at right now. But I suppose we'll find out by talking to more people. So I would definitely say adoption and we have a number of things that we could do around it and figuring out so we have to figure out which ones to do and then do them.
[00:30:47] Unknown:
I'm also interested in understanding how you are gathering use cases and examples of optimization problems and areas of improvement, and then how you approach understanding what are the performance impacts from the various aspects of the implementation and the language and how to target your development efforts to try and get the most impact for the amount of work that you're able to put in?
[00:31:15] Unknown:
So in terms of finding people to talk to, we have a Discord chat room now and some people have found us through that. And then proactively, we've been seeking out companies that we know are running web deployments on Python and trying to get their impressions of what their problems are. In terms of the day to day, we have a set of benchmarks that we have produced. We have a Django benchmark and a Flask benchmark because these are the 2 most popular Python web frameworks. And I believe they're a step forward in terms of benchmarks, but there's obviously a lot more that we can do. They're still very artificial in nature. So, ideally, we can find some projects or some team that's willing to contribute a benchmark that is more realistic.
But then on a day to day basis, we just look at the benchmark results as we look at the benchmark profiles to guide us on what to improve and where the different time is spent and things like that.
[00:32:17] Unknown:
As far as the people who have been using Piston either in your current incarnation where you're building a business around it and you're looking for people to adopt it because they have production use cases that they need to optimize, But also in your time at Dropbox, I'm wondering what you've seen as some of the most interesting or unexpected or innovative ways that Piston has been used.
[00:32:38] Unknown:
1 thing that was very unexpected was someone got Piston working on Windows through the Windows subsystem for Linux that we only produce builds build packages for Linux and we were planning on producing some of the Mac and we just weren't thinking about Windows. And then someone found us and let us know that it was working through on Windows. So that was a very nice surprise. I think, in general, the less nice surprise has been just how high the hurdle is to get someone to adopt a new implementation. Goes well with our choice to choose web serving because I think web serving as opposed to other parts of Python Python ecosystem has big players involved.
Whereas you don't have, you know, a large number of individual data scientists each making their own decisions. You might have a large company with a a platform team that's going to make this decision that will affect how they run Python. So we just need to get a smaller number of people who might be very interested to adopt it rather than getting a large mass of mildly interested people to try to adopt it. But even with that, even with having there being people that we think we can help quite a bit, it's still been a process to go from having a technical solution to something to going to have actual users who are using it on a day to day basis.
[00:34:01] Unknown:
In your own experience of building Piston and trying to build a business around it, what are some of the most interesting or unexpected or challenging lessons that you've learned there?
[00:34:12] Unknown:
I think adoption is still a big 1. And then as I mentioned earlier, we learned the hard way that you don't really get rewarded for having a very fancy system. You get rewarded for having something that solves a user's problem. So as I said earlier, we've been continually making Piston a little bit less sexy from a technical level as we've been optimizing more for solving what we hope are users actual problems. That's a little bit of a tough lesson to learn but I think it has really made our product a lot better. Because the other thing is as we've been thinking about business models for this, it's been tough to come up with a business model for this that there are number of open source business models, but they are a little bit tough for a project like this.
There's some closed source models, but they're not super common. So we thought it might be easier to monetize something that people wanted a lot, but it is seeming that is a tricky thing that we're still trying to figure out.
[00:35:15] Unknown:
And for people who are considering whether to use Piston for their workloads, what are the cases where it's the wrong choice and they would be better served either just staying with the CPython implementation or using something like PyPy or 1 of the other projects that's aimed at various aspects of performance?
[00:35:32] Unknown:
So we aim to be moderately helpful for all use cases. And so I think that implies a couple cases that this is maybe not the right choice. 1 is we're not quite there. There's certain use cases where due to certain practicalities, for example, it's it's a little bit tough to run AI workloads on Piston right now just due to some packaging minutiae. So we should we're hopeful to get that solved soon. But if you're running AI stuff, you currently have to use a sort of unsafe workaround to run that, and it's not really recommended at the moment. So it's just a temporary thing. I don't think it's fundamental to our approach. The other case I would say is that our approaches have been pretty general and try to help all code a decent amount.
But there are certain types of code, like, if you have highly numeric code, there are gonna be options out there that will speed up your type of code a lot. So if you have numeric code and you run it on Piston, you'll get a good speed up, but you could run it on something like Pypi or Cython or Numba or any of these things and get a much more dramatic speed up. So calling it a wrong choice is a little strong, but you could definitely get more bang for your buck if you work in certain specific fields where there's specialized tools just for that area.
[00:36:53] Unknown:
Given the amount of time that you've spent digging through the Python code base and working with people who are trying to optimize the performance of their projects, Are there any aspects of the language that you, in retrospect, regret and that you wish didn't need to be supported and would make your job easier for improving the execution capabilities of the language?
[00:37:17] Unknown:
That's a good question. I think there are a number of minor things. I mentioned earlier that there's a number of debugging features that are always enabled and I think we might remove some of those. Most people don't even know that these exist because they're very low level things that I I think most people don't use and we might remove some of them. There's certain other, kind of, things that are a little bit difficult which is that, for example, throwing an exception will basically take a snapshot of the execution state at that point. And in CPython, this is not expensive at all because they're already maintaining the structures to do this.
But it prevents optimizations where you only produce these sort of dynamic structures on the fly if they're requested. And we tried this with PISN v 1 where we would generate these on the fly, but we ended up generating them so often because you have to generate them for every exception that it tended to not be super valuable.
[00:38:18] Unknown:
And as you continue to work on Piston, what do you have planned for the near to medium term of the project, and what are you most excited about digging into?
[00:38:27] Unknown:
I think I'm most excited to see people use it and to understand what will get them to that point. I think we have a pretty compelling technical solution right now and it's interesting. I'm excited to go through the process of going from that to getting people to actually use it. So as I mentioned earlier, we're not a 100% sure what is stopping people from that. And I think as we figure that out, that will influence a lot of where we put our work. But I think some things that are probably all involved is well, first of all, we can always be faster and improving that will always make our options seem a little bit more compelling.
And then there are certain things around compatibility where, in theory, on paper, we are a 100% compatible with CPython because we fought from them and didn't change any any of the relevant things. But in practice, there's a number of compatibility issues that have come up. For example, there's kind of the equivalent of user agent parsing. So like like a web browser will identify itself as a certain user agent and all the web browsers kind of lie and pretend to be each other in order to get all the right features enabled. There's an extra equivalent thing where a Python implementation says what implementation it is, and then some compatibility code will determine what features to enable based on that. So I think we found a combination of, like, when to say we're CPython, when to say we're Piston that enables all the features but also is safe. And but I think there's a little bit more we can do on that front.
And then there's just some, like, packaging issues around getting prebuilt packages of the AI libraries and things like that. So an assortment of things that we wanna do, and they're mostly focused around either performance or making it easier for people to adopt it.
[00:40:19] Unknown:
Are there any other aspects of the work that you're doing on piston or just the overall space of performance optimization and improvements and the overall compatibility with the Python ecosystem that we didn't discuss yet that you'd like to cover before we close out the show?
[00:40:35] Unknown:
I'll just go on a little mini rant about that last thing I talked about which is the compatibility detection. I think it would work better if there were flags that an implementation exposed about do I support this, do I support that rather than having third party code look at your version string and determine what features are enabled or not. We're having some difficulty where there's all this code that's already deployed out there that checks for certain things in a very specific way. And we don't have the ability to change, for example, the code that is currently shipping with Ubuntu that might detect certain things about this or not.
So my wish would be that we move some more of a flag based detection system than a, like, implementation name plus version number based detection system.
[00:41:27] Unknown:
Well, for anybody who wants to follow along with the work that you're doing or try out Piston or get in touch, I'll have you add your preferred contact information to the show notes. And so with that, I'll move us into the picks. And this week, I'm going to choose the last week in AWS newsletter. It's a newsletter run by Corey Quinn, so it's full of lots of snark and humor, but it's also a great way to keep up to date with all of the various things that change in the Amazon Web Services catalog and various projects and information around that. So as somebody who uses AWS for a lot of my day to day work, it's a useful way to keep up to date. So definitely worth checking out if you happen to use anything from that ecosystem as well. And so with that, I'll pass it to you, Kevin. Do you have any picks this week?
[00:42:12] Unknown:
I think my pick will, in general, be meditation. I've been getting into meditation lately and I found it to really pay for itself essentially that I think you do meditation not just to get certain benefits, but it's also I do feel like meditation has made me more productive, like, more so than the time that I've invested in meditation and solve. So that's been very pleasant to see. And maybe in terms of a specific thing to look at, I use the Calm app, which I do enjoy. I've heard people also like the His Face app, and I don't know if there are other apps. But I like that it makes it very sort of digestible to just do a little bit of meditation.
[00:42:52] Unknown:
Definitely worth taking a look at, particularly given the hectic state of the world right now. So thank you very much for taking the time today to join me and share the work that you've been doing at Piston. It's definitely an interesting project and 1 that I've been keeping an eye on since it was first announced from your time at Dropbox. So I'm glad to see that you've been able to continue along with it, and I definitely look forward to seeing where you're able to take it going forward. So thank you again for all the time and effort you've put into that, and I hope you enjoy the rest of your day. Thanks, and thank you very much for having me. Thank you for listening. Don't forget to check out our other show, the Data Engineering Podcast at dataengineeringpodcast.com for the latest on modern data management.
And visit the site at pythonpodcast.com to subscribe to the show, sign up for the mailing list, and read the show notes. And if you've learned something or tried out a project from the show, then tell us about it. Email host@podcastinit.com with your story. To help other people find the show, please leave a review on Itunes and tell your friends and coworkers.
Introduction to Kevin Mazilevskiy and Piston
The Genesis of Piston at Dropbox
Motivations Behind Piston
Understanding Python Performance
Piston's Place in the Python Ecosystem
Challenges with Python Compatibility
Technical Approach and Innovations in Piston
Limits of Python Optimization
The Global Interpreter Lock (GIL) and Multithreading
Sustainability and Governance of Piston
Challenges and Roadblocks in Piston Development
Gathering Use Cases and Benchmarking
Lessons Learned and Business Challenges
When Not to Use Piston
Regrets and Challenges in Python Language Support
Future Plans and Excitement for Piston
Final Thoughts on Compatibility and Detection
Picks and Recommendations
