Summary
Functional programming is gaining in popularity as we move to an increasingly parallel world. Sometimes you want access to purely functional syntax and capabilities but you don’t want to have to learn an entirely new language. Coconut is here to help! This week Evan Hubinger explains how Coconut is a functional language that compiles to Python and can be mixed and matched with the rest of your program.
Preface
- Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
- I would like to thank everyone who supports us on Patreon. Your contributions help to make the show sustainable.
- When you’re ready to launch your next project you’ll need somewhere to deploy it. Check out Linode at www.podastinit.com/linode?utm_source=rss&utm_medium=rss and get a $20 credit to try out their fast and reliable Linux virtual servers for running your awesome app.
- Visit the site to subscribe to the show, sign up for the newsletter, read the show notes, and get in touch.
- To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
- Your host as usual is Tobias Macey and today I’m interviewing Evan Hubinger about Coconut, a functional language implemented as a superset of Python
Interview
- Introductions
- How did you get introduced to Python?
- Can you start by explaining what Coconut is and what problem you were trying to solve when you created it?
- Where did the name come from?
- How is Coconut implemented and what does the compilation process for Coconut code look like?
- How will I be able to debug my Python if I’m not the one writing it?
- The documentation mentions that Coconut itself is compatible with both Python 2 and 3, are there any caveats to be aware of in terms of mixing in standard Python syntax?
- Are there any performance optimizations that you have had to perform in order to make things like recursion and pattern matching work at reasonable speeds in the Python VM?
- Which functional languages have you taken inspiration from during the creation of Coconut?
- What are some of the most interesting or unexpected uses of Coconut that you have seen?
- What are some resources that you recommend for people who are interested in learning more about functional programming?
Keep In Touch
Picks
- Tobias
- Evan
- pyparsing is an awesome PyPI package you should check out
The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA
[00:00:14]
Unknown:
Hello, and welcome to podcast dot in it, the podcast about Python and the people who make it great. I would like to thank everyone who has donated to the show. Your contributions help us make the show sustainable. When you're ready to launch your next project, you'll need somewhere to deploy it. So you should check out linode@linode.com/podcast in it, and get a $20 credit to try out their fast and reliable Linux virtual servers for running your app or trying out something you hear about on the show. You can visit our site at www.podcastinnit.com to subscribe to the show, sign up for the newsletter, read the show notes, and get in touch.
To help other people find the show, you can leave review on iTunes or Google Play Music. Tell your friends and coworkers and share it on social media. Your host as usual is Tobias Macy, and today I'm interviewing Evan Hubinger about Coconut, a functional language implemented as a superset of Python. So, Evan, could you please introduce yourself?
[00:01:00] Unknown:
Yeah. Absolutely. I'm an undergraduate student at Harvey Mudd College studying math and computer science, and I am also a software engineering intern at Google this summer. Previously, I worked for Yelp. And do you remember how you first got introduced to Python? Yeah. So I actually first got introduced to Python at a middle school sort of summer camp thing. It wasn't the best introduction. It was the first time I'd ever done programming. It wasn't taught very well. It was just sort of they threw this stuff out there. I looked at it, but it got me going on the idea of programming and idea of Python. I just saw it, and I was like, this is so cool. This is so useful. And I started using it for all of my personal projects. I wrote, like, games. I tried to implement math stuff in Python, and I've worked with that for a while, and I really sort of came to see how how useful it was and how powerful it was, and I really haven't stopped writing Python code since. So, yeah, Python was my first language. That was how I first got introduced to it. It it was actually only later that I got into functional programming which would later be me to Coconut.
[00:02:03] Unknown:
And can you explain a bit about what Coconut is and the problem that you were trying to solve when you first created it?
[00:02:09] Unknown:
Coconut is a functional programming language that compiles to Python. So originally, like I was saying, you know, I got into functional programming and I wanted to be able to do functional programming in Python. There's a lot of stuff that you can do just straight away in Python. Python's got great support for higher order functions. Python's got great support for all of these functional programming concepts, it's got Lambda's, but there's a lot of stuff that's still, it doesn't quite work coming from something like Haskell. I wanted to be able to do pattern matching. I wanted to be able to use immutable algebraic data types. I wanted to have Taylor occursion elimination. A lot of these sorts of more complicated things were really frustrating trying to write that way in Python and not having them. And so I developed Coconut as a way to try and solve that problem in Python. So what Coconut does is it syntaxes a strict superset of Python 3. And on top of that, it adds support for a lot of these functional programming features like tail workers elimination, like algebraic data types, like pattern matching, and then it compiles that back to universal version independent Python.
[00:03:15] Unknown:
And when you first started conceiving of Coconut, do you know if some of the other tools out there like Funky and PyTools were already out there and available? And if so, what is it that was lacking in those that you decided was necessary when you were implementing Coconut?
[00:03:33] Unknown:
So there's a there's a lot of things that Coconut does differently. So first of all, Coconut is a as in new syntax but is a strict superset of Python 3. So that means that it can have room for making functional programming features more elegant like adding, you know, native support for partial application, for pan matching without having to try to fit within Python's existing syntax, but it still means that all of the stuff that you know from Python Python continues to be available in Coconut. Additionally, Coconut compiles to your Python source code instead of CPython bytecode. The reason for that is a couple of reasons. First is that Python source code has the ability to be version and implementation independent.
That's the big major reason. That means that you can use Coconut with PyPy, you can use Coconut Iron Python, Jython, and you don't have to be stuck with some particular version of the CPython VM. I guess the the second reason is that Python source code can also be used for other purposes, like, if you wanna use Python, Coconut, that's totally possible. Or if you integrate Coconut with other tools that take in Python source, it's always possible to do so because Coconut just compiles to that. And can you explain briefly a bit about the background for the name? So Python was named after Monty Python. Guido wrote about this. He was in an irreverent mood at the time. And so, coconut is named after Monty Python and the Holy Grail in which they bang coconuts together to,
[00:05:01] Unknown:
sort of mimic the sound of horses. And they have this sort of whole bit about it, and I thought it was great. And so I figured I would play on that or, coconut. Yeah. It's definitely 1 of the great things about the Python community is that continued irreverence that weaves through a lot of the different naming at and just various community aspects like the Python package index was originally referred to as the cheese shop because there's a famous Python sketch where there's a cheese shop that doesn't actually have any cheese. Yeah. So it's definitely great to see that that, tradition is being continued, even to today. And you're mentioning that Coconut is a compiled language in that it compiles down to Python syntax. So I'm wondering if you can explain a bit about how that compiler is implemented and what the overall process for compilation looks like as it runs through the Coconut runtime.
[00:05:51] Unknown:
So the Coconut compiler is written entirely in Python, in universal version independent Python. So just like Coconut's compiled Python, which runs on 2.6, 2.7, 3.2, and above, the Coconut compiler also runs on the same burns and it's just Python, which means you can run it on any Python implementation as well. And it leverages a tool called PyParsing to go through the OpenNet source code and then it does a parsing and compilation process on it to output Python source code. So there's a, you know, couple different ways that you can interface with it. You can interface with the coconut compile just via the command line and you say this is, some coconut file I wanna compile and it'll give you the Python file. You can also use it as an interpreter, on the fly, compiling code as you go or even in a Jupyter or I IPython,
[00:06:40] Unknown:
notebook. What are the benefits of implementing Coconut itself in pure Python as opposed to having it be written as sort of an extension to the CPython runtime or the PyPy runtime?
[00:06:52] Unknown:
So that's a good question. The same benefit that you get from compiling the Python in that it's version independent and it's implementation independent. However, it's definitely true that we would like the those things matter less for the compiler, and we would rather the compiler just be faster. There is actually some effort that I'm working on right now to try to get the compiler itself to perhaps also have a c extension to make it faster, but it's pretty quick as is, and the benefit of being written in Python is that it's version independent, and I like writing in Python, and so it's a lot easier to maintain and work on compiler because the whole thing is just Python. And given that it is straddling the 2 and 3 divide,
[00:07:31] Unknown:
are there limitations in terms of some of the capabilities that you're able to take advantage of in the compiler itself, particularly thinking about things like the async capabilities in the later versions of 3 or,
[00:07:44] Unknown:
any issues with the bytes versus Unicode with strings, things like that. 1 of the biggest things that Coconut does is that it is able to take Python 3 source code because Coconut syntax is a superset of Python 3 and includes things like built ins and imports, and it's able to take that and compile it to universal Python. And so in the process, Coconut does a lot of things like translating imports, creating these mocks of built ins to look like the Python 3 versions. A lot of including handling things like you were saying, like the Unicode bytes difference in Python 2 and Python 3. All of that is handled by Coconuts by the compiler and and the when it compiles the source code, it able to take care of all of that version universality, and all of that also applies to the compiler itself. However, there are some cases in which that's not possible. So you mentioned asyncio. You can't get asyncio on Python 2 really no matter what you do. And so, yeah, in that case, if you try to use an async dash statement, for example, Coconut will say, you can't do that. And it'll ask you to specifically pass a command line argument that says, I only am compiling for Python 3.6. And then if you do that, the compiled code then will only work on Python 3.6, but it'll let you write things like async that. And 1 of the difficulties that always comes about with transpiled languages
[00:09:05] Unknown:
is being able to debug the rendered source after it goes through the compilation step. So I'm wondering if there are any capabilities akin to a source map from JavaScript or anything like that that will make it easier for people to debug the final program as it's running?
[00:09:21] Unknown:
Yes. So that's a big 1 that is definitely a pain point. Right? That's how are you gonna debug this compiled Python if you don't have any insight into what it looks like? So what Coconut does, there are a couple of command line options that are helpful here. The first 1 is line numbers. So Coconut will add in comments line numbers from the Coconut source code that are, like, you were saying, like a source mapping from 1 to the other, so that then if you get an exception, if you do a pdb.settrace, and you're taking a look at Python code, you'll see the line numbers of the coconut code that created it. So that way you don't have to context switch when you're debugging, and you could always just stay in the coconut mindset. There's also a keyplans flag. We'll do the same thing, but instead of line numbers, it actually puts the entire line there. And I imagine that there are gonna be cases where a single line of coconut will map out to several lines of Python. So does it just tag it with the same
[00:10:13] Unknown:
line comment of the original code? Yeah. That's exactly right. There are many cases in which 1 line coconut will yield many lines of Python, But, yeah, they all all of those lines will get tagged with the same line number. And are there any sort of performance optimizations that you've had to make in the compilation step for things like tail call recursion and pattern matching because I know that in a lot of purely functional language implementations and run times, There are some pretty heavy optimizations in the actual compiler or in the VM that will account for those particularly functional capabilities that are necessarily present in Python. So I'm just curious,
[00:10:49] Unknown:
what the overall runtime performance and overhead looks like for those, and if there are any tweaks or tricks that you came across in order to make those run a bit faster in in the actual execution step. Yeah. So that's a great question. So there's a couple of things that Coconut does. The first 1 is, like you mentioned, tail co optimization. Coconut actually does this in 2 ways. There's tail recursion elimination and tail co optimization. So if you by this, always happen whenever you write write a function in Coconuts. If your function returns a call to itself, Coconuts can actually reword that into a wall loop, which is a lot faster than actually having to put a new function call on the stack every time. If it doesn't return a call to itself, but it does return a call to another function, Coconut can still optimize that away. It's a slightly more complex process, however. It involves trampolining, using exceptions, but it Coconut will always make sure that you never get a new, like, stack frame from a function call. If you make that function call in a tangent recursive style, you return that function call directly. So that's 1 implementation that Coconut does. Coconut also does a couple of other things with, build ins. So some of them, there are things you do in functional programming that you often don't do in other cases where you'll have things like I map a bunch of times and then I filter and then I reduce. Coconut can handle a bunch of those different operations together and it can streamline them really. So there's less stuff like that that Coconut will do in the compilation process. In general, though, Coconut just compiles the Python. And so Coconut is about as fast as Python because at the end of the day, what you're running is still Python. And for some of the other functional capabilities like algebraic data types, and I think you mentioned that there are immutable data structures.
[00:12:25] Unknown:
I guess if you could start by explaining a bit in particular about what algebraic data types are for somebody who's not familiar with them, and also the immutable data. 1, why it's useful and also some of the ways that you are able to implement that because as pretty much everybody who's ever used Python knows it by nature is not, purely immutable. There are some immutable types, but they are not usually used as such.
[00:12:52] Unknown:
Yeah. So great question. So algebraic data so for coconut algebraic data types and immutable data types, that's the same thing. Coconut algebraic data types are immutable. So first, I'll try to answer, what does it what does it mean for something to be an algebraic data type? So in Coconut, that means a couple of things. 1st, it means you can pattern match and deconstruct So, for example, I could define algebraic data type. It was a vector of 2 elements, x and y. And then I could define a function, f of vector x comma y. And if I call that function on a vector, would automatically look inside the vector, take out the components, and put them in x and y, since those were the bindings that I had inside of the pattern to that function. And so the algebraic nature really, is that you can pattern match and deconstruct against them. You can even sort of union them together, and use them for pattern matching that way and type checking. So that's really where the algebraic data nature comes from. The immutability, yeah, another big aspect of them. This all comes through Coconut's data statement. The way that the immutability comes in is actually through name, tuple, and slots.
In Python, pretty much at the end of the day, everything immutable have to inherit from either tuple or frozen set. And, of course, we're gonna use tuple in this case because it's the we're not we're not creating a set. We want to have different elements and we wanna be able to access them. And so behind the scenes, Coconut does, at some point, every time you create a data statement, there's gonna be a name tuple invocation there. There's gonna be some slot stuff. There's gonna be a bunch of there's usually a bunch of other stuff that Coconut fusses with as well. But the end result is that you get this, algebraic data type that is immutable, that can be deconstructed against, that can be pattern matched against, and all you have to do is just say that. And is there anything special that you do for handling cases where there are dictionaries or lists that are
[00:14:47] Unknown:
attributes of that data element. Because as some people might be aware, if you have a tuple that does contain either a dictionary or a list, despite the fact that the tuple itself is immutable, the contents aren't necessarily. So you can actually modify a dictionary that is contained within a tuple, which it will throw an exception, but it will also still modify the dictionary that's contained in there. So I'm wondering if that is still an edge case in the implementation with Coconut or if there were any particular tricks that you were able to put in that would help prevent people from doing things like that. So that's a great point.
[00:15:24] Unknown:
Coconut doesn't solve that. It want to have immutability. You have to make sure that at every step of the your data tree, every single object that you're using is a mutable object. However, Coconut makes it a lot easier to do that, to have everything be immutable because it makes it really easy to define new immutable data types using the data statement. So as long as you make sure that every data type you're using, all of your data types have attributes that correspond to other data types and all of that is immutable, then, yeah, then you'll find. But if you if you don't, if you mix in lists, if you mix in dictionaries,
[00:15:58] Unknown:
then, yes, that stuff will be mutable. And there are a number of different functional paradigms and sort of families of functional languages. So I'm curious which 1 or, which ones are you drawing the most inspiration from in the process of creating Coconut?
[00:16:14] Unknown:
I personally drew a lot of inspiration from Haskell. That was the language that I worked in for a while before, that sort of really got me interested in functional programming and got me to write Coconut. But there's other languages as well that haven't had an influence, namely, CoffeeScript, f sharp. I think that those are probably maybe the other 2 big ones, but the the largest influence is definitely Haskell. Of course, that's as far as functional languages go though. The biggest influence, of course, is is Python because the syntax is based on Python and the entire point is to try to take concepts from those languages and put them in Python in a way that makes sense to a Python developer. And so the goal is really not to so much look like Haskell, but to look like Python, but let you do the sort of things you can do in Haskell. Yes. As I was looking through the documentation, there are definitely a number
[00:17:10] Unknown:
of sort of syntax elements that are pretty foreign to a Python developer such as the pipelining syntax and the capabilities that that brings about. So for somebody who is interested in adding some coconut syntax or some of the coconut capabilities to their code base, what are some of the best ways for them to become familiarized with those functional applications, you know, particularly things like the pipelining or the parallelization capabilities that Coconut brings to bear?
[00:17:40] Unknown:
So, yeah, a lot of that stuff is covered in Coconut's tutorial. You can take a look at that. That's on the website and you can go through it. The tutorial is, I think, a pretty good introduction to a lot of that stuff. It's a it's a good introduction, I think, to functional programming in general also from a Python standpoint because, like I said, every single thing that you know in Python continues to work in Conan. So it means that you're not starting from a point like you would if you tried to learn Haskell where you don't know anything in the language. You're starting from point where you already know a lot of things in the language, and it's just giving you more tools to work with. And so I think it's a good way to get your feet wet with functional programming, as well as start using it in Python. Beyond that, sort of, if you want to learn functional programming more generally, there are lots of functional programming languages out there with great tutorials. I know, like I said, I started with Haskell and I really enjoyed learning Haskell. So definitely any of the above, all of those things are great resources.
[00:18:39] Unknown:
And what are some of the most common use cases and some of the most interesting implementations of Coconut that you've seen?
[00:18:48] Unknown:
So, yeah, there's a there's a couple of of good use cases and and people using Coconut. 1 of the really big ones that I've seen is there are a lot of people from the data science community that have started using it or the machine learning community as well because there's a lot of stuff that you do in those applications where you really want to be able to leverage functional programming principles. You wanna set up your data in a pipelining structure where you take your data and then you transform it at each step. You wanna be able to sort of compose all these different operations together using pipelining, partial application, function composition to create this sort of data pipeline that will transform it at each step. That sort of stuff is really, really easy to do in Coconut. And so for that reason, there's been a lot sort of attention on that regard. And 1 of the things that Coconut does to help with that is that, I mentioned this earlier, it supports Jupyter or iPython Notebooks.
So you can really easily use that existing workflow that a lot of people have and just plug Coconut right in there. 1 other, use case that I've seen is, of course, the sort of really hardcore functional use case. For that, 1 of the really nice things that Coconut has is, Coconut integrates with MyPy, so that if you really want Python to feel like a functional programming language, you can even have Coconut compile 3.6 style type annotations, including, like, variable type annotations, like a colon int equals 5, down to universal type comments, then Mypy can then analyze so that you can write functional code that's statically typed in Python, which is quite the, sort of turnaround.
But, yeah, it's lots of lots of really different cool use cases that people are starting to, to find with it. It's really
[00:20:38] Unknown:
evolving and growing and so it's it's really cool to see that. And what are some of the most challenging aspects that you've come across in the process of building up Coconut and trying to grow the community around it?
[00:20:50] Unknown:
So that's a great question. So I guess as far as challenges are concerned, though, there were a couple of meetups that people were doing earlier. I know in New York City. I I think some of those, I I I'm not sure what happened to it. It's a little bit hard to coordinate a lot of this stuff because so much of the development effort and whatnot, like the documentation is still all written by me and I live in California. So that's definitely an issue. The the really big issue though is just that, you know, like I like I mentioned, so much of it is still written by by me that there's the hit by a bus problem that if I, if I get knocked out, then the development sorta comes to somewhat of a stop. There are other people that are working on it, and that's changing pretty rapidly now. There's lots of people that are starting to give pull requests and look at and work with the code base and so that that problem, I think, will become resolved. But for right now, at least, I would say the sort of biggest issue is just we need more people to contribute because most of it is still at this point my code. And are there any,
[00:21:55] Unknown:
significant technical hurdles that you've had to overcome in the process as well?
[00:22:00] Unknown:
Well, yes, there's there's a there's actually a bunch of technical hurdles that in the process of writing Kokan, I have encountered. 1 big 1 that you will see if you peruse the FAQ will find a section that says help my Python segfault. So 1 of the things that you can do when writing iterators in Python using some of Coconut's more powerful iterator utilities that in pure Python you don't encounter very often. You can get into a situation where you create a recursive iterator that theoretically you expect to raise, like, a maximum recursion depth exceeded runtime error.
But you will actually get at a seg fault. Your Python will just crash out from under you. And this is can be very frustrating. So Coconut actually has a special decorator that sort of takes a shortcut around the issue and makes it so that you can still write recursive iterators that work correctly. But, yeah, there well, that was a that was a fun day when I managed to take vanilla Python without any c extensions, and it segfaulted on me. So that was maybe the biggest biggest thing that I came across.
[00:23:13] Unknown:
Yeah. That that's quite an achievement.
[00:23:16] Unknown:
Yeah. That's not supposed to happen in Python. That's not supposed to happen. Otherwise, I mean, there's there's certainly were lots of technical hurdles that were common in grading. Coconut, it's it's a fairly ambitious project. It aims to be able to both take all of these functional programming constructs and turn them into Python code as well as take Python 3 stuff and turn it into universal Python. So all of that, was definitely a big big challenge. Took me took me 2 years until I open sourced it, and it's been about a year since then. Yeah. You know, it all works. It's it's all it's all out there now. So
[00:23:51] Unknown:
Yeah. And I was noticing in the documentation that unlike some Python projects that took on the order of a decade to hit 1 0, you have pretty rapidly progressed past the 1 0 sort of milestone. So that's definitely a good indicator that, people should feel comfortable with, adopting it and implementing it in their own workflows.
[00:24:17] Unknown:
The 1.0 release, that was the 1 that I open sourced. That was the sort of first release when Coconut was made was publicized. Since then, we are now at 1.2.3. And, yes, I would say Coconut is definitely at a point where I use it for a lot of my own projects. I know a lot of other people are starting to integrate it into their own projects and their projects at work. I would definitely say that it's totally ready for production use.
[00:24:43] Unknown:
And I imagine the fact that the final actual running binary is still regular native vanilla Python code would probably make people feel a bit more comfortable actually using it in their workflow, for the technology that isn't as necessarily time tested and proven as the Python VM itself.
[00:25:03] Unknown:
Absolutely. Yeah. That means that, you know, first of all, you can inspect the code, and you can take a look at what's going on if you're confused about it or anything. And it means that, you know, whatever your existing workflow is, whatever your existing implementation, however you have it set up, you should be able to use Coconut on that existing workflow, on that existing implementation. And so it means it's pretty easy to just plug and play. It also means you don't have to switch over to it wholesale. You can have just a couple of files that are written in coconut, and that's totally fine because it just compiles to Python. And the inner operates with other Python files the same way that Python operates with itself. And so that means that, you know, it really is very easy to introduce Coconut into an existing project without causing any real complication.
[00:25:47] Unknown:
And with some transpiled languages, particularly things like CoffeeScript, the actual rendered output sometimes can be fairly opaque and obtuse to try and read through. So I'm wondering if there are any sort of tricks or anything that you implemented in that transpilation step to make sure that the final output is more human readable and easier to try and parse and understand without having to necessarily refer back to the original source?
[00:26:15] Unknown:
Unfortunately, the compiled output often is very difficult to parse, very difficult to read. It's kind of funny because, you know, you think you could never you can't write Python that is impossible to parse as Python, but, yeah, the compiler can, but that's okay. There's a lot of things that can alleviate that problem. For 1 thing, I mentioned line numbers and keep lines that can let you sort of take a look at what caused that specific line to be generated, as well as those we're working on some ability to try and make that better, to make that code look more readable, make more sense, that may be coming for the next version potentially.
In addition to that, before then, you can also just, like, if you write your code without using a lot of the Coconut's own new features, it will usually be pretty easy to parse because it can just translate Python into Python. But when you start using a lot of pattern matching, when you start using a lot of algebraic data types, It does get a little bit tricky, but I don't think it's that big of a problem. You know, at the end of the day, you're gonna be running it. But theoretically, you should be able to debug your code, write your code, everything just in the coconut system. And it's not every day that somebody decides to just start implementing their own language even if it is,
[00:27:32] Unknown:
targeting a sort of battle tested run time. So I'm wondering where you've got where where you got the background and the inspiration and motivation to decide to to go ahead and actually proceed with a project as ambitious as Coconut and implement your own language implement and write your own language implementation?
[00:27:51] Unknown:
So yeah. It's a big project. It actually I'll I'll give you a a slight historical note here. It actually was not my first programming language. So I wrote a programming language before, my sort of first attempt to try and do functional programming in Python. It was really bad. Would not recommend using it. It's called Rabbit. It's on my GitHub if you really want to take a there were so many things wrong with it. For 1, it was interpreted on top of Python. You can imagine how slow it was. But I learned a lot of things from writing Rabbit about compiler design, about language design, and it was really after doing that and being like, I see how I did this wrong, but maybe I can do it right. And that was really what maybe got me doing Coconut. It was certainly ambitious. You know, I said it took me 2 years before I open sourced it. But, you know, this was something I've been really passionate about, you know, enough that I'd already written a programming language. I was already really invested in it, and I really, enjoy functional programming, and I really enjoy Python, and I wanted to bring these 2 things that I like so much together. And so it really started out initially just as, you know, passion of sorts. And and so would you say that Coconut was the holy hand grenade that slew rabbit?
That that is a really good point. I like that. I should use that, at some point in my, in my document. I I'm assuming that rabbit was named after the, Monty Python and the Holy Grail movie as well. It was. It was. Yes. Yes. The rabbit of Kerbanugos. That's a thing. Alright.
[00:29:28] Unknown:
Are there any other questions or topics
[00:29:31] Unknown:
that we should cover before we start to close out the show? I think we've covered a lot of ground. I think we've covered most of the things that the people ask about that are that are useful, interesting.
[00:29:39] Unknown:
Well, for anybody who wants to follow-up and ask some more questions, I'll have you add your contact information to the show notes and they can find it there. With that, I'll bring us to the picks. So this week, I'm gonna choose the element tree library from the Python standard library because I was recently writing an XML formatter for Jinja so that you could pass a Python dictionary through it and get XML rendered output. And I was able to do it all with the built in standard library, element tree capabilities, and, a little bit from the, XML.dom.minidom.
So for anybody who does need to abuse themselves with a bit of XML, it's definitely worth taking a look at the capabilities built into the standard library itself. So with that, I'll pass it to you. Do you have any picks for us this week?
[00:30:30] Unknown:
Yes. There's this awesome PyPI package called Py parsing You may or may not know about. Py parsing, is the backbone of Coconut. It does all of the parsing behind the scenes, but it is a much more general library than that. I've used it for all sorts of things whenever I wanna just sort of take any sort of structured input and take something interesting out of it. I would highly recommend checking out Py parsing for any of your parsing or destructuring needs. It's a really, really great library.
[00:31:02] Unknown:
Yeah. I've had it come up in a few other episodes as well. Recently, the interview that I did about the UNDET project from Yelp, they, mentioned that they used Py parsing for actually implementing the tokenizing capabilities for doing the refactoring.
[00:31:18] Unknown:
So that's fascinating. So I I mentioned that I worded Yelp. OnDebt was, my hackathon project that I did with Pai Wei and Yalin. Who okay. This doesn't have to be on the podcast, but who did who was this? I interviewed Andrew Mason about it. Andrew Mason. Oh, yeah. He was on the he was on the the team. Yeah. Oh, that's so cool. Yeah. So, I worked with him, and, Yeah. Debt was, was part of that. That's that's that's awesome. It's funny how often some of these things come full circle where a little tidbit from 1 of the episodes will come around in another future episode as well. Yeah. Actually, I'll talk about that a little bit. I guess I have another pick. It would be on debt. We would highly recommend checking out Undet. I worked on Undet as my hackathon project at Yelp and, you know, ended up being open source, and I wrote the blog post for it that went up on the Yelp engineering blog. And undet is an awesome utility that leverages PyParsing to enable you to really easily refactor your code. So and it's written in Python, and you can write all of the,
[00:32:16] Unknown:
ways to do the replacements in Python. Part of the reason that I was able to, sort of work on UNTET was because of the knowledge of parsing that I had gotten from Coconut. So I would highly recommend checking out UNTET as well. Well, that's very cool to get that additional bit of background about these projects. So I appreciate that. And with that, I would like to thank you for joining me and telling me more about the coconut project. It's definitely very interesting and something that I'll be taking a closer look at, I'm sure. So I appreciate you taking the time out of your evening to tell me about it, and I hope you enjoy the rest of your day. Thank you so much.
Hello, and welcome to podcast dot in it, the podcast about Python and the people who make it great. I would like to thank everyone who has donated to the show. Your contributions help us make the show sustainable. When you're ready to launch your next project, you'll need somewhere to deploy it. So you should check out linode@linode.com/podcast in it, and get a $20 credit to try out their fast and reliable Linux virtual servers for running your app or trying out something you hear about on the show. You can visit our site at www.podcastinnit.com to subscribe to the show, sign up for the newsletter, read the show notes, and get in touch.
To help other people find the show, you can leave review on iTunes or Google Play Music. Tell your friends and coworkers and share it on social media. Your host as usual is Tobias Macy, and today I'm interviewing Evan Hubinger about Coconut, a functional language implemented as a superset of Python. So, Evan, could you please introduce yourself?
[00:01:00] Unknown:
Yeah. Absolutely. I'm an undergraduate student at Harvey Mudd College studying math and computer science, and I am also a software engineering intern at Google this summer. Previously, I worked for Yelp. And do you remember how you first got introduced to Python? Yeah. So I actually first got introduced to Python at a middle school sort of summer camp thing. It wasn't the best introduction. It was the first time I'd ever done programming. It wasn't taught very well. It was just sort of they threw this stuff out there. I looked at it, but it got me going on the idea of programming and idea of Python. I just saw it, and I was like, this is so cool. This is so useful. And I started using it for all of my personal projects. I wrote, like, games. I tried to implement math stuff in Python, and I've worked with that for a while, and I really sort of came to see how how useful it was and how powerful it was, and I really haven't stopped writing Python code since. So, yeah, Python was my first language. That was how I first got introduced to it. It it was actually only later that I got into functional programming which would later be me to Coconut.
[00:02:03] Unknown:
And can you explain a bit about what Coconut is and the problem that you were trying to solve when you first created it?
[00:02:09] Unknown:
Coconut is a functional programming language that compiles to Python. So originally, like I was saying, you know, I got into functional programming and I wanted to be able to do functional programming in Python. There's a lot of stuff that you can do just straight away in Python. Python's got great support for higher order functions. Python's got great support for all of these functional programming concepts, it's got Lambda's, but there's a lot of stuff that's still, it doesn't quite work coming from something like Haskell. I wanted to be able to do pattern matching. I wanted to be able to use immutable algebraic data types. I wanted to have Taylor occursion elimination. A lot of these sorts of more complicated things were really frustrating trying to write that way in Python and not having them. And so I developed Coconut as a way to try and solve that problem in Python. So what Coconut does is it syntaxes a strict superset of Python 3. And on top of that, it adds support for a lot of these functional programming features like tail workers elimination, like algebraic data types, like pattern matching, and then it compiles that back to universal version independent Python.
[00:03:15] Unknown:
And when you first started conceiving of Coconut, do you know if some of the other tools out there like Funky and PyTools were already out there and available? And if so, what is it that was lacking in those that you decided was necessary when you were implementing Coconut?
[00:03:33] Unknown:
So there's a there's a lot of things that Coconut does differently. So first of all, Coconut is a as in new syntax but is a strict superset of Python 3. So that means that it can have room for making functional programming features more elegant like adding, you know, native support for partial application, for pan matching without having to try to fit within Python's existing syntax, but it still means that all of the stuff that you know from Python Python continues to be available in Coconut. Additionally, Coconut compiles to your Python source code instead of CPython bytecode. The reason for that is a couple of reasons. First is that Python source code has the ability to be version and implementation independent.
That's the big major reason. That means that you can use Coconut with PyPy, you can use Coconut Iron Python, Jython, and you don't have to be stuck with some particular version of the CPython VM. I guess the the second reason is that Python source code can also be used for other purposes, like, if you wanna use Python, Coconut, that's totally possible. Or if you integrate Coconut with other tools that take in Python source, it's always possible to do so because Coconut just compiles to that. And can you explain briefly a bit about the background for the name? So Python was named after Monty Python. Guido wrote about this. He was in an irreverent mood at the time. And so, coconut is named after Monty Python and the Holy Grail in which they bang coconuts together to,
[00:05:01] Unknown:
sort of mimic the sound of horses. And they have this sort of whole bit about it, and I thought it was great. And so I figured I would play on that or, coconut. Yeah. It's definitely 1 of the great things about the Python community is that continued irreverence that weaves through a lot of the different naming at and just various community aspects like the Python package index was originally referred to as the cheese shop because there's a famous Python sketch where there's a cheese shop that doesn't actually have any cheese. Yeah. So it's definitely great to see that that, tradition is being continued, even to today. And you're mentioning that Coconut is a compiled language in that it compiles down to Python syntax. So I'm wondering if you can explain a bit about how that compiler is implemented and what the overall process for compilation looks like as it runs through the Coconut runtime.
[00:05:51] Unknown:
So the Coconut compiler is written entirely in Python, in universal version independent Python. So just like Coconut's compiled Python, which runs on 2.6, 2.7, 3.2, and above, the Coconut compiler also runs on the same burns and it's just Python, which means you can run it on any Python implementation as well. And it leverages a tool called PyParsing to go through the OpenNet source code and then it does a parsing and compilation process on it to output Python source code. So there's a, you know, couple different ways that you can interface with it. You can interface with the coconut compile just via the command line and you say this is, some coconut file I wanna compile and it'll give you the Python file. You can also use it as an interpreter, on the fly, compiling code as you go or even in a Jupyter or I IPython,
[00:06:40] Unknown:
notebook. What are the benefits of implementing Coconut itself in pure Python as opposed to having it be written as sort of an extension to the CPython runtime or the PyPy runtime?
[00:06:52] Unknown:
So that's a good question. The same benefit that you get from compiling the Python in that it's version independent and it's implementation independent. However, it's definitely true that we would like the those things matter less for the compiler, and we would rather the compiler just be faster. There is actually some effort that I'm working on right now to try to get the compiler itself to perhaps also have a c extension to make it faster, but it's pretty quick as is, and the benefit of being written in Python is that it's version independent, and I like writing in Python, and so it's a lot easier to maintain and work on compiler because the whole thing is just Python. And given that it is straddling the 2 and 3 divide,
[00:07:31] Unknown:
are there limitations in terms of some of the capabilities that you're able to take advantage of in the compiler itself, particularly thinking about things like the async capabilities in the later versions of 3 or,
[00:07:44] Unknown:
any issues with the bytes versus Unicode with strings, things like that. 1 of the biggest things that Coconut does is that it is able to take Python 3 source code because Coconut syntax is a superset of Python 3 and includes things like built ins and imports, and it's able to take that and compile it to universal Python. And so in the process, Coconut does a lot of things like translating imports, creating these mocks of built ins to look like the Python 3 versions. A lot of including handling things like you were saying, like the Unicode bytes difference in Python 2 and Python 3. All of that is handled by Coconuts by the compiler and and the when it compiles the source code, it able to take care of all of that version universality, and all of that also applies to the compiler itself. However, there are some cases in which that's not possible. So you mentioned asyncio. You can't get asyncio on Python 2 really no matter what you do. And so, yeah, in that case, if you try to use an async dash statement, for example, Coconut will say, you can't do that. And it'll ask you to specifically pass a command line argument that says, I only am compiling for Python 3.6. And then if you do that, the compiled code then will only work on Python 3.6, but it'll let you write things like async that. And 1 of the difficulties that always comes about with transpiled languages
[00:09:05] Unknown:
is being able to debug the rendered source after it goes through the compilation step. So I'm wondering if there are any capabilities akin to a source map from JavaScript or anything like that that will make it easier for people to debug the final program as it's running?
[00:09:21] Unknown:
Yes. So that's a big 1 that is definitely a pain point. Right? That's how are you gonna debug this compiled Python if you don't have any insight into what it looks like? So what Coconut does, there are a couple of command line options that are helpful here. The first 1 is line numbers. So Coconut will add in comments line numbers from the Coconut source code that are, like, you were saying, like a source mapping from 1 to the other, so that then if you get an exception, if you do a pdb.settrace, and you're taking a look at Python code, you'll see the line numbers of the coconut code that created it. So that way you don't have to context switch when you're debugging, and you could always just stay in the coconut mindset. There's also a keyplans flag. We'll do the same thing, but instead of line numbers, it actually puts the entire line there. And I imagine that there are gonna be cases where a single line of coconut will map out to several lines of Python. So does it just tag it with the same
[00:10:13] Unknown:
line comment of the original code? Yeah. That's exactly right. There are many cases in which 1 line coconut will yield many lines of Python, But, yeah, they all all of those lines will get tagged with the same line number. And are there any sort of performance optimizations that you've had to make in the compilation step for things like tail call recursion and pattern matching because I know that in a lot of purely functional language implementations and run times, There are some pretty heavy optimizations in the actual compiler or in the VM that will account for those particularly functional capabilities that are necessarily present in Python. So I'm just curious,
[00:10:49] Unknown:
what the overall runtime performance and overhead looks like for those, and if there are any tweaks or tricks that you came across in order to make those run a bit faster in in the actual execution step. Yeah. So that's a great question. So there's a couple of things that Coconut does. The first 1 is, like you mentioned, tail co optimization. Coconut actually does this in 2 ways. There's tail recursion elimination and tail co optimization. So if you by this, always happen whenever you write write a function in Coconuts. If your function returns a call to itself, Coconuts can actually reword that into a wall loop, which is a lot faster than actually having to put a new function call on the stack every time. If it doesn't return a call to itself, but it does return a call to another function, Coconut can still optimize that away. It's a slightly more complex process, however. It involves trampolining, using exceptions, but it Coconut will always make sure that you never get a new, like, stack frame from a function call. If you make that function call in a tangent recursive style, you return that function call directly. So that's 1 implementation that Coconut does. Coconut also does a couple of other things with, build ins. So some of them, there are things you do in functional programming that you often don't do in other cases where you'll have things like I map a bunch of times and then I filter and then I reduce. Coconut can handle a bunch of those different operations together and it can streamline them really. So there's less stuff like that that Coconut will do in the compilation process. In general, though, Coconut just compiles the Python. And so Coconut is about as fast as Python because at the end of the day, what you're running is still Python. And for some of the other functional capabilities like algebraic data types, and I think you mentioned that there are immutable data structures.
[00:12:25] Unknown:
I guess if you could start by explaining a bit in particular about what algebraic data types are for somebody who's not familiar with them, and also the immutable data. 1, why it's useful and also some of the ways that you are able to implement that because as pretty much everybody who's ever used Python knows it by nature is not, purely immutable. There are some immutable types, but they are not usually used as such.
[00:12:52] Unknown:
Yeah. So great question. So algebraic data so for coconut algebraic data types and immutable data types, that's the same thing. Coconut algebraic data types are immutable. So first, I'll try to answer, what does it what does it mean for something to be an algebraic data type? So in Coconut, that means a couple of things. 1st, it means you can pattern match and deconstruct So, for example, I could define algebraic data type. It was a vector of 2 elements, x and y. And then I could define a function, f of vector x comma y. And if I call that function on a vector, would automatically look inside the vector, take out the components, and put them in x and y, since those were the bindings that I had inside of the pattern to that function. And so the algebraic nature really, is that you can pattern match and deconstruct against them. You can even sort of union them together, and use them for pattern matching that way and type checking. So that's really where the algebraic data nature comes from. The immutability, yeah, another big aspect of them. This all comes through Coconut's data statement. The way that the immutability comes in is actually through name, tuple, and slots.
In Python, pretty much at the end of the day, everything immutable have to inherit from either tuple or frozen set. And, of course, we're gonna use tuple in this case because it's the we're not we're not creating a set. We want to have different elements and we wanna be able to access them. And so behind the scenes, Coconut does, at some point, every time you create a data statement, there's gonna be a name tuple invocation there. There's gonna be some slot stuff. There's gonna be a bunch of there's usually a bunch of other stuff that Coconut fusses with as well. But the end result is that you get this, algebraic data type that is immutable, that can be deconstructed against, that can be pattern matched against, and all you have to do is just say that. And is there anything special that you do for handling cases where there are dictionaries or lists that are
[00:14:47] Unknown:
attributes of that data element. Because as some people might be aware, if you have a tuple that does contain either a dictionary or a list, despite the fact that the tuple itself is immutable, the contents aren't necessarily. So you can actually modify a dictionary that is contained within a tuple, which it will throw an exception, but it will also still modify the dictionary that's contained in there. So I'm wondering if that is still an edge case in the implementation with Coconut or if there were any particular tricks that you were able to put in that would help prevent people from doing things like that. So that's a great point.
[00:15:24] Unknown:
Coconut doesn't solve that. It want to have immutability. You have to make sure that at every step of the your data tree, every single object that you're using is a mutable object. However, Coconut makes it a lot easier to do that, to have everything be immutable because it makes it really easy to define new immutable data types using the data statement. So as long as you make sure that every data type you're using, all of your data types have attributes that correspond to other data types and all of that is immutable, then, yeah, then you'll find. But if you if you don't, if you mix in lists, if you mix in dictionaries,
[00:15:58] Unknown:
then, yes, that stuff will be mutable. And there are a number of different functional paradigms and sort of families of functional languages. So I'm curious which 1 or, which ones are you drawing the most inspiration from in the process of creating Coconut?
[00:16:14] Unknown:
I personally drew a lot of inspiration from Haskell. That was the language that I worked in for a while before, that sort of really got me interested in functional programming and got me to write Coconut. But there's other languages as well that haven't had an influence, namely, CoffeeScript, f sharp. I think that those are probably maybe the other 2 big ones, but the the largest influence is definitely Haskell. Of course, that's as far as functional languages go though. The biggest influence, of course, is is Python because the syntax is based on Python and the entire point is to try to take concepts from those languages and put them in Python in a way that makes sense to a Python developer. And so the goal is really not to so much look like Haskell, but to look like Python, but let you do the sort of things you can do in Haskell. Yes. As I was looking through the documentation, there are definitely a number
[00:17:10] Unknown:
of sort of syntax elements that are pretty foreign to a Python developer such as the pipelining syntax and the capabilities that that brings about. So for somebody who is interested in adding some coconut syntax or some of the coconut capabilities to their code base, what are some of the best ways for them to become familiarized with those functional applications, you know, particularly things like the pipelining or the parallelization capabilities that Coconut brings to bear?
[00:17:40] Unknown:
So, yeah, a lot of that stuff is covered in Coconut's tutorial. You can take a look at that. That's on the website and you can go through it. The tutorial is, I think, a pretty good introduction to a lot of that stuff. It's a it's a good introduction, I think, to functional programming in general also from a Python standpoint because, like I said, every single thing that you know in Python continues to work in Conan. So it means that you're not starting from a point like you would if you tried to learn Haskell where you don't know anything in the language. You're starting from point where you already know a lot of things in the language, and it's just giving you more tools to work with. And so I think it's a good way to get your feet wet with functional programming, as well as start using it in Python. Beyond that, sort of, if you want to learn functional programming more generally, there are lots of functional programming languages out there with great tutorials. I know, like I said, I started with Haskell and I really enjoyed learning Haskell. So definitely any of the above, all of those things are great resources.
[00:18:39] Unknown:
And what are some of the most common use cases and some of the most interesting implementations of Coconut that you've seen?
[00:18:48] Unknown:
So, yeah, there's a there's a couple of of good use cases and and people using Coconut. 1 of the really big ones that I've seen is there are a lot of people from the data science community that have started using it or the machine learning community as well because there's a lot of stuff that you do in those applications where you really want to be able to leverage functional programming principles. You wanna set up your data in a pipelining structure where you take your data and then you transform it at each step. You wanna be able to sort of compose all these different operations together using pipelining, partial application, function composition to create this sort of data pipeline that will transform it at each step. That sort of stuff is really, really easy to do in Coconut. And so for that reason, there's been a lot sort of attention on that regard. And 1 of the things that Coconut does to help with that is that, I mentioned this earlier, it supports Jupyter or iPython Notebooks.
So you can really easily use that existing workflow that a lot of people have and just plug Coconut right in there. 1 other, use case that I've seen is, of course, the sort of really hardcore functional use case. For that, 1 of the really nice things that Coconut has is, Coconut integrates with MyPy, so that if you really want Python to feel like a functional programming language, you can even have Coconut compile 3.6 style type annotations, including, like, variable type annotations, like a colon int equals 5, down to universal type comments, then Mypy can then analyze so that you can write functional code that's statically typed in Python, which is quite the, sort of turnaround.
But, yeah, it's lots of lots of really different cool use cases that people are starting to, to find with it. It's really
[00:20:38] Unknown:
evolving and growing and so it's it's really cool to see that. And what are some of the most challenging aspects that you've come across in the process of building up Coconut and trying to grow the community around it?
[00:20:50] Unknown:
So that's a great question. So I guess as far as challenges are concerned, though, there were a couple of meetups that people were doing earlier. I know in New York City. I I think some of those, I I I'm not sure what happened to it. It's a little bit hard to coordinate a lot of this stuff because so much of the development effort and whatnot, like the documentation is still all written by me and I live in California. So that's definitely an issue. The the really big issue though is just that, you know, like I like I mentioned, so much of it is still written by by me that there's the hit by a bus problem that if I, if I get knocked out, then the development sorta comes to somewhat of a stop. There are other people that are working on it, and that's changing pretty rapidly now. There's lots of people that are starting to give pull requests and look at and work with the code base and so that that problem, I think, will become resolved. But for right now, at least, I would say the sort of biggest issue is just we need more people to contribute because most of it is still at this point my code. And are there any,
[00:21:55] Unknown:
significant technical hurdles that you've had to overcome in the process as well?
[00:22:00] Unknown:
Well, yes, there's there's a there's actually a bunch of technical hurdles that in the process of writing Kokan, I have encountered. 1 big 1 that you will see if you peruse the FAQ will find a section that says help my Python segfault. So 1 of the things that you can do when writing iterators in Python using some of Coconut's more powerful iterator utilities that in pure Python you don't encounter very often. You can get into a situation where you create a recursive iterator that theoretically you expect to raise, like, a maximum recursion depth exceeded runtime error.
But you will actually get at a seg fault. Your Python will just crash out from under you. And this is can be very frustrating. So Coconut actually has a special decorator that sort of takes a shortcut around the issue and makes it so that you can still write recursive iterators that work correctly. But, yeah, there well, that was a that was a fun day when I managed to take vanilla Python without any c extensions, and it segfaulted on me. So that was maybe the biggest biggest thing that I came across.
[00:23:13] Unknown:
Yeah. That that's quite an achievement.
[00:23:16] Unknown:
Yeah. That's not supposed to happen in Python. That's not supposed to happen. Otherwise, I mean, there's there's certainly were lots of technical hurdles that were common in grading. Coconut, it's it's a fairly ambitious project. It aims to be able to both take all of these functional programming constructs and turn them into Python code as well as take Python 3 stuff and turn it into universal Python. So all of that, was definitely a big big challenge. Took me took me 2 years until I open sourced it, and it's been about a year since then. Yeah. You know, it all works. It's it's all it's all out there now. So
[00:23:51] Unknown:
Yeah. And I was noticing in the documentation that unlike some Python projects that took on the order of a decade to hit 1 0, you have pretty rapidly progressed past the 1 0 sort of milestone. So that's definitely a good indicator that, people should feel comfortable with, adopting it and implementing it in their own workflows.
[00:24:17] Unknown:
The 1.0 release, that was the 1 that I open sourced. That was the sort of first release when Coconut was made was publicized. Since then, we are now at 1.2.3. And, yes, I would say Coconut is definitely at a point where I use it for a lot of my own projects. I know a lot of other people are starting to integrate it into their own projects and their projects at work. I would definitely say that it's totally ready for production use.
[00:24:43] Unknown:
And I imagine the fact that the final actual running binary is still regular native vanilla Python code would probably make people feel a bit more comfortable actually using it in their workflow, for the technology that isn't as necessarily time tested and proven as the Python VM itself.
[00:25:03] Unknown:
Absolutely. Yeah. That means that, you know, first of all, you can inspect the code, and you can take a look at what's going on if you're confused about it or anything. And it means that, you know, whatever your existing workflow is, whatever your existing implementation, however you have it set up, you should be able to use Coconut on that existing workflow, on that existing implementation. And so it means it's pretty easy to just plug and play. It also means you don't have to switch over to it wholesale. You can have just a couple of files that are written in coconut, and that's totally fine because it just compiles to Python. And the inner operates with other Python files the same way that Python operates with itself. And so that means that, you know, it really is very easy to introduce Coconut into an existing project without causing any real complication.
[00:25:47] Unknown:
And with some transpiled languages, particularly things like CoffeeScript, the actual rendered output sometimes can be fairly opaque and obtuse to try and read through. So I'm wondering if there are any sort of tricks or anything that you implemented in that transpilation step to make sure that the final output is more human readable and easier to try and parse and understand without having to necessarily refer back to the original source?
[00:26:15] Unknown:
Unfortunately, the compiled output often is very difficult to parse, very difficult to read. It's kind of funny because, you know, you think you could never you can't write Python that is impossible to parse as Python, but, yeah, the compiler can, but that's okay. There's a lot of things that can alleviate that problem. For 1 thing, I mentioned line numbers and keep lines that can let you sort of take a look at what caused that specific line to be generated, as well as those we're working on some ability to try and make that better, to make that code look more readable, make more sense, that may be coming for the next version potentially.
In addition to that, before then, you can also just, like, if you write your code without using a lot of the Coconut's own new features, it will usually be pretty easy to parse because it can just translate Python into Python. But when you start using a lot of pattern matching, when you start using a lot of algebraic data types, It does get a little bit tricky, but I don't think it's that big of a problem. You know, at the end of the day, you're gonna be running it. But theoretically, you should be able to debug your code, write your code, everything just in the coconut system. And it's not every day that somebody decides to just start implementing their own language even if it is,
[00:27:32] Unknown:
targeting a sort of battle tested run time. So I'm wondering where you've got where where you got the background and the inspiration and motivation to decide to to go ahead and actually proceed with a project as ambitious as Coconut and implement your own language implement and write your own language implementation?
[00:27:51] Unknown:
So yeah. It's a big project. It actually I'll I'll give you a a slight historical note here. It actually was not my first programming language. So I wrote a programming language before, my sort of first attempt to try and do functional programming in Python. It was really bad. Would not recommend using it. It's called Rabbit. It's on my GitHub if you really want to take a there were so many things wrong with it. For 1, it was interpreted on top of Python. You can imagine how slow it was. But I learned a lot of things from writing Rabbit about compiler design, about language design, and it was really after doing that and being like, I see how I did this wrong, but maybe I can do it right. And that was really what maybe got me doing Coconut. It was certainly ambitious. You know, I said it took me 2 years before I open sourced it. But, you know, this was something I've been really passionate about, you know, enough that I'd already written a programming language. I was already really invested in it, and I really, enjoy functional programming, and I really enjoy Python, and I wanted to bring these 2 things that I like so much together. And so it really started out initially just as, you know, passion of sorts. And and so would you say that Coconut was the holy hand grenade that slew rabbit?
That that is a really good point. I like that. I should use that, at some point in my, in my document. I I'm assuming that rabbit was named after the, Monty Python and the Holy Grail movie as well. It was. It was. Yes. Yes. The rabbit of Kerbanugos. That's a thing. Alright.
[00:29:28] Unknown:
Are there any other questions or topics
[00:29:31] Unknown:
that we should cover before we start to close out the show? I think we've covered a lot of ground. I think we've covered most of the things that the people ask about that are that are useful, interesting.
[00:29:39] Unknown:
Well, for anybody who wants to follow-up and ask some more questions, I'll have you add your contact information to the show notes and they can find it there. With that, I'll bring us to the picks. So this week, I'm gonna choose the element tree library from the Python standard library because I was recently writing an XML formatter for Jinja so that you could pass a Python dictionary through it and get XML rendered output. And I was able to do it all with the built in standard library, element tree capabilities, and, a little bit from the, XML.dom.minidom.
So for anybody who does need to abuse themselves with a bit of XML, it's definitely worth taking a look at the capabilities built into the standard library itself. So with that, I'll pass it to you. Do you have any picks for us this week?
[00:30:30] Unknown:
Yes. There's this awesome PyPI package called Py parsing You may or may not know about. Py parsing, is the backbone of Coconut. It does all of the parsing behind the scenes, but it is a much more general library than that. I've used it for all sorts of things whenever I wanna just sort of take any sort of structured input and take something interesting out of it. I would highly recommend checking out Py parsing for any of your parsing or destructuring needs. It's a really, really great library.
[00:31:02] Unknown:
Yeah. I've had it come up in a few other episodes as well. Recently, the interview that I did about the UNDET project from Yelp, they, mentioned that they used Py parsing for actually implementing the tokenizing capabilities for doing the refactoring.
[00:31:18] Unknown:
So that's fascinating. So I I mentioned that I worded Yelp. OnDebt was, my hackathon project that I did with Pai Wei and Yalin. Who okay. This doesn't have to be on the podcast, but who did who was this? I interviewed Andrew Mason about it. Andrew Mason. Oh, yeah. He was on the he was on the the team. Yeah. Oh, that's so cool. Yeah. So, I worked with him, and, Yeah. Debt was, was part of that. That's that's that's awesome. It's funny how often some of these things come full circle where a little tidbit from 1 of the episodes will come around in another future episode as well. Yeah. Actually, I'll talk about that a little bit. I guess I have another pick. It would be on debt. We would highly recommend checking out Undet. I worked on Undet as my hackathon project at Yelp and, you know, ended up being open source, and I wrote the blog post for it that went up on the Yelp engineering blog. And undet is an awesome utility that leverages PyParsing to enable you to really easily refactor your code. So and it's written in Python, and you can write all of the,
[00:32:16] Unknown:
ways to do the replacements in Python. Part of the reason that I was able to, sort of work on UNTET was because of the knowledge of parsing that I had gotten from Coconut. So I would highly recommend checking out UNTET as well. Well, that's very cool to get that additional bit of background about these projects. So I appreciate that. And with that, I would like to thank you for joining me and telling me more about the coconut project. It's definitely very interesting and something that I'll be taking a closer look at, I'm sure. So I appreciate you taking the time out of your evening to tell me about it, and I hope you enjoy the rest of your day. Thank you so much.
Introduction and Guest Introduction
Evan's Introduction to Python
What is Coconut?
Coconut vs Other Tools
Naming and Community Traditions
Coconut Compiler Implementation
Benefits of Pure Python Implementation
Handling Python Versions and Limitations
Performance Optimizations in Coconut
Algebraic and Immutable Data Types
Functional Paradigms Influencing Coconut
Learning Coconut and Functional Programming
Use Cases and Implementations
Challenges in Building and Growing Coconut
Technical Hurdles and Solutions
Coconut's Stability and Adoption
Readability of Compiled Output
Inspiration and Motivation for Coconut
Historical Note: Rabbit
Picks and Recommendations
