Functional Python with Matthew Rocklin and Alexander Schepanovsky

Hello, and welcome to podcast.init,

the podcast about Python and the people who make it great. You can subscribe to our show on Itunes, Stitcher, TuneIn Radio, or add our RSS feed to your pod catcher of choice.

You can also follow us on Twitter or Google Plus, and please give us feedback. You can leave a review on Itunes to help other people find the show, send us a tweet or an email, leave us a message on Google Plus or on our show notes, and you can also join our new community. Visitdiscourse.pythonpodcast.com

for your opportunity to find out about upcoming guests, suggest questions, and propose show ideas.

I would like to thank everyone who has donated to the show. Your contributions help us make the show sustainable.

For details on how to support the show, you can visit our site at python podcastdot

com. Linode is sponsoring us this week. Check them out at linode.com/podcastinit

and get a $20 credit to try out their fast and reliable Linux virtual servers for your next project.

I would also like to thank Hired, a job marketplace for developers and designers, for sponsoring this episode of podcast dot in it. Use the link hired.com/podcastin

it to double your signing bonus.

Your host today is Tobias Macy, and today I'm interviewing Matthew Rocklin and Alexander Shepanovsky

about their work on functional libraries for Python.

So could you guys please introduce yourselves? How about you go first, Alexander?

Hello, everyone. My name is Alexander Shpanovski,

and I'm a Python programmer.

And open source author and talks

giver.

K.

Alright. And how about you, Matthew?

Hi, everyone. First, thanks for having me on the show. My name is Matthew Rocklin. I'm a full time open source developer at Continuum Analytics. I usually develop into the numeric Python stack. So So libraries that run on top of NumPy and Pandas.

Previously, I did research in numerical methods and usability.

And how did you each get introduced to Python? How about you go again, Alexander?

Yeah. We can see a difference

between,

me and Matthew.

So I came from

web development

and, Matthew from,

numeric programming.

And I think this will,

influence

the rest of our,

podcast.

So

it was,

I believe,

2,007

And

we we were using,

Pearl back when,

and we were using our own

kind of

framework.

And,

I decided with,

we need to move on and use

some something

some frame framework, some open source framework.

And I looked around,

and

there were only

old things in

Pearl. So I looked

more around and

it was a choice at the at the time between Django and,

Ruby on Rails.

Not really a choice even between languages but between frameworks.

And

even though,

Ruby seemed

more interesting,

rails

seems too dirty. Like,

the last thing was that

we are passing all the local

variables from this, what we call, controller to a t templates.

And so,

I went with

Django and was introduced to Python.

And how about you, Matthew? How did you find Python?

Yeah. So I actually picked it up in graduate school. Previously, I just joined coursework.

Previously, I used C Sharp for software engineering and MATLAB for more data analysis and science work.

Particularly, I used MATLAB because

sort of had all the data science things for me, and also because my science collaborators,

you know, physicists, not computer scientists,

could understand it.

I sort of picked up Python just doing coursework as a fun side project.

And I was sort of very happily surprised when that it could do both software engineering

and have the API that a data science user could use. So I could build tools and share them with my science collaborators. And that sort of combination really, really stuck with me. So that was sort of my why I jumped into it, and I've just haven't left since.

To reiterate,

Alexander's point about 2 different communities, I think it's a very valid valid point that people come into Python from a lot of different a lot of different perspectives.

And we have, you know, we make different software. Sometimes it doesn't work well together. And very recently, it's been very exciting, actually, to see the web community and the data science community start to come together. So, you know, most scientists that I know now use Flask as sort of this, like, accessible thing. And a lot of the web developers I know are now using some of the more visual, tools for visualization inside of this data science world. So, again, it's very cool to see these 2 communities coming together.

Although in some cases, like with the case of Alexander and I, we've,

we've duplicated some efforts.

This is not necessarily

a bad thing. Absolutely not. No. It's great.

And can you guys first explain what functional programming is and how it differs from the procedural or object oriented programming that most Pythonistas are familiar with? And recognizing that functional programming is kind of an overloaded term with lot of subcomponents. So feel free to delve into that as well.

I don't think you should oppose functional programming and,

procedural

or object oriented 1. You should oppose functional programming and imperative.

So in imperative programming, you, and the difference is in how you think about your program and how you decompose it. In imperative programming, you,

think

of a program as a algorithm which is a series of

steps. So you decompose it in this into steps

which,

change the state of a world.

And in in functional programming, you think of a program as a function. So it's you have input and output and some

transformation

from 1 to to seconds.

And, you decompose it into a simpler function by

using function composition and things like that. So in imperative programming, if you get some steps and you unite them, you get

procedural programming.

And even if you add objects, you get a object oriented programming. But these are, like, add ons

on imperative programming.

And,

in functional programming,

it's usually associated with things like pure functions, small nodes, and

higher order functions and things like that, but

these are only,

the consequences of, a way of

thinking about the program and a way of decomposing it.

Because,

when you start to decompose

your big function to smaller functions, you start to see that,

not functions

are equally easy to decompose to.

So it's easier to decompose to viewer functions

and things like that. And you want

to

make,

to give special special attention to side effects

and probably

separate

side effects from the rest of the population.

And, so all these things that are usually associated with functional programming, these are just consequences

of the idea of functional programming.

So this is my understanding.

Yeah. So I have a I think that's a great definition. I think that

many developers will have different understanding of this of this term. I think a Haskell developer and a closure developer will will think about what functional programming is a little bit differently.

But I think the definition provided by Alex is Alexander is great. How I would define it actually is more of a historical definition. I would say that functional programming is a set of language features

that come from languages that descended from 2 root languages.

Lisp that originated around the same time as Fortran and Algol, and ML,

sort of an academic

language to prove theorems about type systems

that originated in the 19 seventies.

Lisp begat languages like common Lisp and Clojure, and ML begat languages like Haskell. And I would say that these languages have a lot of language features, as all languages do. And the language features that appeared only in these languages, but not in the sort of C and Fortran like languages that we're more familiar with, All of those features

sort of tend to be called

functional. So this might be function composition, functions as data, higher order functions,

but also things like macros or algebraic type systems,

sort of more fancy things, or more pedestrian things, like streaming iterators

and a strong reliance on core data structures.

Today, I would say that people often use the term functional programming

just as a reaction to

highly imperative or object oriented programming.

There's sort of this push towards simplicity. So when I read blog posts, I sort of I get a variety of there's a variety of terms that also can be lumped into this term of functional

programming. And if it's a Haskell programmer, it might be talking about algebraic type systems. If it's a closure programmer, it might be talking about macros.

And so there's a there's a large grab bag of of concepts here.

And how did you each get started with functional programming?

For me, it's

not like you you start with you don't start with functional programming

at first because you don't know what functional programming is. But, you start thinking

in a functional way sometimes.

And and, you probably

got introduced to some concept like function

or higher order functions. And

you can see

a practical advantage of, for example, higher order functions because you,

can write functions which are more custom customizable this way. And it could be easier when writing objects. You have less parts or less code.

Like, like this, you get introduced to these concepts or and

not only concepts, but

the specifics,

the application.

When I I also got introduced to Common Lisp,

I never actually used it for anything,

practical, but it was,

just played with it and extracted some

ideas from it. So this is my story with functional programming. I I later took a a look into ML and Haskell,

and I don't think,

that much practical,

especially Haskell. It's,

has too much hassle in it.

I think there are a lot of Haskell developers that would disagree with that, but I I I also agree with you. Yep. Even if you have less code, it not always,

means you

spend,

less time writing it and,

anybody else will spend less time reading it and understanding it or modifying this code. It's, there are huge amount of hassle with many things in Haskell, like IO.

Okay.

Matthew, how'd you get involved in functional programming?

Yeah. So I think,

like Alexander, I I probably

started learning functional concepts before I really identified them as being functional concepts. I think that was a really great point. When I sort of intentionally started learning functional programming, it was actually a social activity. A couple of my friends, Avi Newman, John Jacobson, and I wanted to get together and drink some beers. We wanted some activity to do. And so we decided to,

implement the same problem. It was, like, a job interview problem in a variety of languages.

So we did it in C and in Fortran and in Clojure and in Haskell and in languages like mod.

And it was actually really interesting to see

what what about our solutions was intrinsic to the problem that would stay the same across all these languages.

And what about our solutions was sort of a side effect of the language?

And it was just a really fascinating experience. I highly recommend grabbing a couple of programming friends and doing the same thing, especially if they're in different disciplines.

Neither Avi, John, or I had a common background.

John Jacobson, in particular, was quite keen on closure, and he sort of brought us in that direction. And as a result, I sort of screwed on the closure for the following year, and that had a really strong impact on my programming style even today. I don't think I've ever used functional programming in I've never used a

strictly functional language,

in practice,

but I took a lot of this a lot of the ideas that I learned from delving into functional programming into my everyday Python code. And I think there's really a lot of value to functional programming

even if you're not doing it explicitly. Or if you're not in Haskell or enclosure,

you can still use a lot of the concepts

in everyday everyday style.

And we've touched on it a bit, but what are some of the benefits of functional programming, and when might somebody want to use functional paradigms in their projects?

I may first add a small

remark on,

writing,

writing the same thing in different languages. There is a resource,

called, called Rosetta Code. It's a wiki with a

solution in different languages for different problems

and could be quite interesting to look sometimes. And about the benefits on functional programming. For me, it's,

like, you you can,

use this customization

with, which high order functions give you

without,

the immediate layer of, like, classes.

Classes and factories and things like that. You can achieve in object oriented paradigm, but it's usually

more hassle.

When also, if you think

not

about concrete,

specific features, but about,

thinking,

it could also be beneficial because the alternative is usually

imperative programming where you think of steps.

And, in functional programming, you think of, operations. And you think of data and you think of operations which you apply to it. And,

it

could be a higher level of thinking. So you might compress bigger algorithm

to fit into your mind. This could be pro this is probably the highest benefits.

But different features

could also be beneficial.

Yeah. So I totally agree with,

functional programming lets you decompose a problem into functions

that are pure and have clean inputs and outputs,

and how

we can solve most problems using just

simple data structures and functions that call each other and have very dependable inputs and outputs.

I would I generally classify this into the concept of modularity.

And I would say that if we had a more object oriented person on on the phone, they might also say that object oriented programming

also enables modularity. It also enables people to break down large problems into small problems.

I think the difference comes in the style in which 1 achieves that.

So I think that functional programming, there's maybe a tenant that you don't need an object, which is this marriage of state and functions.

And often, state and functions can live separately. And that's often sufficient.

So I think you can actually solve a large number of problems, very hard problems,

using just core data structures like lists and dictionaries and strings and numbers

and simple functions.

I think a lot of functional programming a lot of the resurgence recently has come out of sort of

a a backlash

or a reaction to the sort of heavy object oriented programming that came out of the nineties. Would you agree? What are your thoughts, Alexander? I can add to it, that's when you model your domain with in object oriented way, usually, you create these,

types which represent some,

data. And,

when this is, always a a new class, it's new types.

So you need to implement lots of operations to handle them. Right. If you model your data with, some data structures like, dates, lists, and so on, You already have lots of operation in language for them. So usually, it's it's less code and it's faster to model things.

Yeah. And in addition, not only is your code simpler, but your code can more easily interact with other code that you didn't anticipate.

For example, you know, if I just use dictionaries and lists, I can use the JSON library

immediately, trivially, without having to think about writing a toJson method. I think has a good example here of the,

of the HTTP request object in those languages.

It's usually some object that has lots of sub objects.

And if you wanna interact with an HTTP request object, you sort of need to have the documents

side by side as you're walking through this object. Well, instead, it could just be a nested

dictionary.

And then I could, you know, pull requests off of some socket.

I could dump them into a document store like MongoDB,

or I could serialize and put them to disk,

all without having to write any extra code to handle those interfaces.

So I think 1 major feature

of functional programming is just

a reliance on shared abstractions that are common to language core data structures.

But I think that there are also some other features that are also

associated to functional languages that are still also valuable. Like, algebraic types are something that, you know, came out of ML and help compilers ensure

correctness and improve speed. And these, you know, might be sort of strange if you're looking for it from Haskell.

But if a newer language like Julia, it's very accessible. They use algebraic types very effectively to improve performance.

Something like macros, which came out of Lisp,

can help library developers make more powerful user APIs.

This is something that's been very helpful in, say, the language R, which uses them to make very powerful user APIs,

even if they're a little bit strange for developers.

Immutability is something that's often referred to as a functional

concept,

and software transactional memory. They really help with concurrency. And these are there's lots of immutable data structures inside of Python or inside of external libraries within Python. And software transactional memory has recently been added to PyPI.

So there's lots of ideas that are come from these languages that are that are still useful,

in a variety of different contexts.

Yeah. And,

we have progress in

things like that, and, I can see little progress in, object oriented programming.

You may have alluded to it a bit earlier, but what is it about functional programming

that much intimidating if you don't tell people it's, It's not that much intimidating if you don't tell people it's it's fashion.

Yeah. And so you don't need to,

in, if

I go a bit aside Sure. When you,

market like a functional functional

programming support library for a language, you could

say, like,

so stop doing like that and

write code this way and use my library. And this is very weak proposition.

Or you can,

have some utils

which actually solve,

problems that

people have. Like, some

utils with, to work with sequences and

utils, to work with collections in the like red. And then, you say

my with my utils,

we are very customizable

because,

we

accept functions.

So, we have all the functions which accept functions so you can use them for many tasks with your sequences of collections.

And,

this way you're solving

some problems some problems people already have and,

this way people can Google your library and, Google it by Googling their problems not Googling

functional programming in Python,

which,

I think, far

less people do. And,

when people start using these

utilities for sequences and collections,

we start we start thinking of how we're we're formed with utilities,

which are high order functions. And, we start to look around in your library and

maybe introduce to our concepts like function composition.

So it

should not be really intimidating. And so back to why is why is it, besides its name. I think,

any anything that's,

not

people are not used to could be intimidating. Also, we have this,

thing that you can pack lots of things on single line and sometimes,

people abuse it and this makes the code

unreadable

especially for people who are not used to things like WAVs. This could be intimidating. Also,

some features like monads could be intimidating because it's really hard

to explain why do you need those. Because you don't need those in most languages.

And so this is my take. Maybe Matthew can add more. Yeah. No. I I totally agree

with, the right way to brand functional programming and how,

the again, the conversation isn't functional or not functional. It's,

here's here's good ways to write software and and less good ways to write software. And that's really where we should be focusing if we wanna have a

a pragmatic, if we want to have a positive effect. I would say that I think everyone learned some derivative of c historically.

And so if you knew a functional language, it was usually your second language. And as a result, I felt a little bit fringe or elite.

And this this impression of it being elite, I think, was very damaging to the ideas within functional programming. I mean, I think even within the the core Python community, there's still a little bit of hesitancy about any

concept that's that's called functional

because of this this bad history.

But in truth, I think a lot of the concepts

of functional programming are actually much simpler

than people think.

For example, I think that macros, which are maybe a little bit strange to people,

are simpler intrinsically

than, say, class inheritance in object oriented programming.

However, I say recently so you asked about the recent rise in popularity.

And I would say that recently, there's been some hardware changes, like multicore architectures have brought on parallel computing.

And there's been a lot of need around highly concurrent systems.

And for these, the fully stateful model of of imperative programming and mutation and state and object oriented

aren't particularly ideal.

And so sort of the functional language community was sort of off in the side, this very small minority saying, hey. We've actually got all of these nice tools

that are really well suited to handle

parallelism and concurrency

and the building of large systems.

And that really

that utility, that pragmatism,

really helped to reverse public opinion. So I think there's now maybe a sort of revitalization of functional thinking

to the point where most model languages coming out now include a decently large subset of functional concepts within them.

And this this really makes those concepts much more accessible to single language developers.

The fact that in Python, you can use functions as data or you can use higher order functions

that's accessible to someone who's just learning programming is is really powerful, I think.

Yeah. And I think too that part of it might be attributable to the recent rise in popularity

of JavaScript

because while it does have some object oriented capabilities,

it's generally used more in a functional aspect of passing functions around as variables and as data.

And

also,

you know, there's there's a new generation of developers coming up. And as you said, you know, a number of developers started with some

variant of the c programming language, and therefore, so a lot of the functional principles are a little bit more foreign.

But given that there has been a slight rise in function in the popularity of functional languages, like you said, because of the recent changes in CPU architectures,

then

the newer programmers who are getting started don't necessarily have that same shared history and are a little bit more willing to experiment.

And, also going back a little bit to what you were saying

about functional

paradigms being not as,

not as widely advertised or encouraged in Python.

1 thing that I've seen in a number of places, both in some of the official documentation and also in some company style guides, is, for instance, avoiding the use of the map filter and reduce functions in favor of,

list dictionary comprehensions.

And 1 thing that

it seems

it it seems that in some respects, that might be a disservice

because

I haven't tried it specifically, but it seems to me that the,

comprehensions wouldn't be as composable as some of the, like, map and reduce functionality

that are more intended to be a,

that are that are more intended to be used in a functional approach?

I'm actually totally fine,

making map and filter a little bit, forbidden. The way I think of it, I think I'm gonna steal some ideas that Alexandra has spoken about.

But think of all the ways that you use a for loop.

Right? You might use a for loop to apply some function onto some lists, or you might use a for loop to,

filter some lists by some predicate, or you might use a for loop to reduce some list to some single number, like a sum.

And if you're familiar with map and filter, you realize that, you know, 70% of your use cases of for loops fall into the cases of map, filter, or reduce. And so I think that map and filter are strictly better than a for loop.

But,

the core language designers

recognized this, and they elevated

these common patterns

to to syntax, to the level of syntax. They elevated it to the list comprehension. And when you get up to that level, that's that's even 1 step

better than map and filter.

I think that if list comprehension didn't exist, map and filter would be very, very useful. But because list comprehensions do

exist, they're not so much necessary.

I think what libraries, like what

funky and tools provide,

is they

provide a bunch of other functions like map and filter

that do the same sorts of things. They represent common patterns that people use,

but that haven't yet been elevated to the level of syntax.

So I would say, yes, don't use map and filter. Use list comprehension. They're usually more readable. But map and filter have cousins, and maybe it's good to look at those cousins.

I think, both map filter and, comprehensions

are fine.

The usual

the usual reason to limit

use of, some functions or some language

constructs is to

make your code base more

more simple like,

less, features in it. Mhmm. So either it could lead

to to less errors

if you have this code style but,

for me both is fine.

Regarding

with and,

when you,

go from my open filter which are

special purpose,

to this

list

of our comprehensions

via more general purpose. So this is,

try to

go from

some specific

solutions to,

DSL.

And

we have, overlaps

in what we provide.

But, I think,

you have you still have a usage for filter and for map.

Yeah. I really like that idea, actually.

Map and filter are good at

giving a name to a common pattern.

And so even if you use a list of operations or use for loops, it's nice to have that name. Oh, I'm doing a map now.

And map and filters, you say, are are more explicit

than our list comprehension.

Sometimes you can also

use a partial of map and or filter,

but this doesn't work with,

comprehensions.

Right. And that's part of what I was thinking of in terms of the composability

where map and filter are a little bit superior in that respect.

But like you said, Matthew, the

in in a lot of cases, the comprehensions can be more immediately

readable and clear as to what they're trying to do. But also, there are cases where you can get a little carried away using comprehensions

where, for instance, you're trying to do 2 or 3 different levels of filtering within 1 comprehension. And then

then it might be make more sense to

either

compose those comprehensions together and nest them a bit or to actually take advantage of the explicit filter call as Alexander was describing.

Actually, I think, comprehension language, could be expanded.

Like,

the most, thing I miss is a lot

in comprehension.

Yeah. When you assign something and then use it later in if or.

That would be very helpful. I agree.

Yeah.

Because,

now sometimes you need to just

duplicate an expression.

For example, in if and in,

in expression itself

before before.

And so

in

the common list, for example,

they have a very, very complicated

loop

macro which

yeah.

I

hear you. You heard about it. And so it does not look like the rest of the language,

but it

and it's it is

causes contradictions.

So some people hate it

but

for many people, it's very handy because you can make lots of and lots of things with it. And you can, do map and filter and,

aggregations and

do several things at once.

And in Python, you sometimes

need to fall back to common 4 because if you want to do several things at once, then

it's usually the

most efficient and

sometimes the only way.

And so we've

done a pretty good coverage of list comprehensions. What other aspects of the Python language lend themselves to being used in a functional manner, and where do they start to fall down?

Maybe the most noticeable

is the absence of

special

immutable

data structures.

But

you

you can usually

be okay with, like, immutable by

convention

which is

not very efficient but, works.

And you can mess it up or also.

But but still.

Yeah. So I would say that Python actually has a pretty good functional core. I would think ideas like streaming iterators,

functions as data, functions as first class objects,

really good core data structures.

There's the legacy things like, Lambda map and filter.

But higher order functions generally, these are all functional ideas that are really common in Python. That Python programmers do a very good job of using

in common style.

So to a large extent, Python is a functional language.

There are some things that doesn't that Python doesn't do,

like, as Alexander was saying, immutable data structures. Well, there's actually been some good work in this. There's I think, fn.py

has some

persistent data structure with structural sharing.

Kurt Smith has a project called Ozymandias,

where he's implementing persistent data structures,

at the sea level, actually, using Cython. And they're very efficient. It's still work in progress, but it's a fun project.

Some things I would love to see

is

better type systems, and there seems to be some push on this in Python 3.5. That's where the type modules coming out. Not doing anything yet, but it's still present.

And macros. I'd actually really love to see macros. As someone who builds user APIs, I would love to see macros.

There's actually a Python library that I came across recently

where

the that that that does provide some macro capabilities. So I'll see if I can find the link, and I'll put it in the show notes. K. There are actually a couple of good ones. There's macro py, and there's also code transformer

by Scott Sanderson and Joe Devnick at Quantropion.

These both do really cool things. They're manipulating the bytecodes. They're doing lots of cool things.

Nothing

will ever work without core language functionality

in the interpreter. Most of my users

are actually using the Python interpreter or like IPython or the Jupyter Notebook.

And it's really hard to provide macros,

without some kind of import hook

or some kind of decorator. Yeah.

There's also I don't know if you're familiar with it, but the highlighting project, which is the yes. We we had them on in a previous episode. And, anyway, the the the project creator,

was describing how he was able to back port some of the Python 3 capabilities

to Python 2 using macros that

he it was the yield from operator that he backported from Python 3 using high using a high macro.

But, yes, like you said, it does require some level of import to be able to inject that into the interpreter.

I also have a point on macros.

It's it's a contradictory 1 because

it's very fascinating to

to have this opportunity to extend the language,

but,

this is a very dangerous 1.

If you

add your markers to your language,

you

get this danger of,

degenerated into

this language into some dialects.

So some people will start using some macros and, our people our ones and when we don't just understand each other.

We can't read each other's code.

It

happens,

more with smaller languages, like scheme when you have a small language and when you

you guys can

build on it with macros.

But

for languages like Python, it's, could also be

dangerous.

The other alternative is that,

you don't use much macros.

Like, you have them, but,

you it's

you should be very

cautious about using them.

And I think in common Lisbon, enclosure,

we're

a bit adopting it.

Yeah. I think that's a totally valid point. And macros are definitely

a language feature that can destroy a language because it provides too much power to library developers

to separate themselves into their own domain specific language.

And along with macros, you must include some sort of community standard that you should never use a macro, and that's absolutely necessary. I think if you look at, again, languages like Clojure or Julia,

this has become the norm.

Very few people use macros. It's considered to be a faux pas.

So yeah. And also, you could also say that, you know, Python

provides lots of opportunities for us to segment ourselves. There's lots of, you know, very, you know, say, 2 powerful ideas in Python, like metaclasses,

that

that enable this sort of bad behavior already.

And monkey patching.

And monkey patching. Yeah. Where Python is very permissive in lots of ways. There's a way that's not permissive.

But as a library writer who serves,

user scientific user communities,

it's something I look at r, look at Julia, and I I just really want the ability to write macros. Because there's there's certain APIs that users love that are just impossible to write without

language support for macros.

But I agree. You know, I only want sort of 10 people in the world to be able to write Python macros. Everyone else not allowed.

And so can you each describe what your respective libraries provide in terms of functional capabilities and what their particular focuses are? And also wondering if they're distinct enough from each other that it would make sense to use them both in a single project.

The most obvious,

thing that, FANSI provides is

sequences

and collections and utilities.

And,

this is

within people use the most.

Because,

you don't need to get functional programming and stuff to use it. I already

talked a bit about it.

When,

Fantic has

some

extension

of fun tools, I guess,

with,

particles,

right particles, and,

composition and some other things.

And when we are just,

some

utilities for

lots of things, like string utilities.

Where

1 example is that I always hated how you need to first

in Python, first match the thing with regular expression and then write if it match then go

with

new it's like you always need at least 3 lines of codes to

use a regular expression.

And, I came before Python, I wrote Perl a lot and it's much easier there and,

usually it's 1 line. So

I added some things to

work with regular expressions.

When some, flow control utilities and some

primitives and

ironically even some utilities for you

to use with objects like cached property.

We we we now also have this cash property in Django,

I believe, from 1.6.

And,

this is really handy when

you have these objects like ORM

objects

and you need some properties which shouldn't be calculated

twice for for an instance.

And, when,

there is a debug debugging utilities

for functional program because,

when you debug a functional program,

you sometimes

can't go the way you usually do.

You can't

add print

because you

everything is happening on a single line

and or you

can't use watches because

and stepping because,

where

file is named variables.

You just pass everything to every 1

1 function to our function and like that.

So it has some debugging utilities like,

decorators,

logging and sprinting and timing

and things like that.

Yeah. So PyTools, that's t o l z,

is a collection of functions

that are good for managing iterators and functions and dictionaries.

These functions

are pure, composable, lazy, and tuned.

And they sort of follow the standard functions you'll find in a variety of languages.

So if you look at languages like Clojure or Scala

or in JavaScript, libraries like underscore or lodash,

or a variety of languages in Ruby and whatnot,

you'll find that

there's a bunch of functions that work on iterators. Things like map and filter, but there's far more, like groupby,

and sliding window, and repeat,

and memoize, and curry.

And you'll find these functions all over the place. And they end up being really useful when you

build an ecosystem

based on core data structures. So earlier in the show, we were talking about the value of just dealing with lists and dictionaries and iterators and normal functions,

and how this is really good for an ecosystem.

And once you start to

use only

these core data structures,

you find there's actually a lot of functions you often want, And there's there's not in the Python standard library, but there are lots of other standard libraries.

And so a lot of other languages sort of converge to this common, like, this common function set, and tools was just the instantiation of that common function set in Python.

You asked about how these libraries might relate to each other if they're different. Whatever you use both in the same same project.

I think that they probably overlap enough that it's not usually necessary to use in the same project.

They're also both simple enough that you could. They're both very lightweight dependencies.

I would say that tools is,

notable,

compared to the all the other functional libraries,

in that it's very low tech. It's usually just functions,

and it's also so it's very accessible.

You shouldn't ever need

to learn some new concept, the same with Funcie.

It's also notable in his performance. Tools has been heavily benchmarked,

largely thanks to Eric Welch,

who does most maintenance today. And Eric also rewrote PyTools in Cython,

which makes the all the iterator stuff

about as fast as you'd find in a compiled language like Java on dynamic data structures.

So tools has tended to be sort of

very much like what you expect if you're familiar with other languages,

and it's

very fast.

The 1 thing that

you

the 1

obvious difference that

with is that you use the the other iterator versions of

all the functions.

But,

yeah, the font is a bit different.

It, first, it has

2 versions for many functions, like, the list version and the other version.

And,

they are named differently for Python 2 and for Python

3. In Python 2, it's like

start if it starts with I, then it's iterative version, like map and imap.

And,

for Python 3, it's like map and l map. So,

in,

Python 3, by default, you don't have a list version of map and field and, like, wait, and

I don't

really like this change

because, this is

so confusing for many people.

You you, like, call this map of filter and,

you it does not actually

do

the the job. You pass this and,

when later, you can

get something from it. And, say,

you can,

forget with its iterator. You can try to add it to a list or you can

try to iterate it, twice,

which gives you an error in a different,

in a different,

point when you actually

get this error,

when you actually made it.

So I should not I don't think,

the iterator should go by default.

This for me it's more like optimization.

So the Python 2 way of doing things,

makes more sense to me.

Right. So this is actually this this difference so you're right. So everything in tools where possible

is an iterator,

even in Python 2. So if you import map from tools in Python 2, you'll get the equivalent of iter toolsimap.

And this is by design for 2 reasons.

1,

1 of the desires here is that tools should be minimally creative.

We didn't want to design an API or make those decisions.

The core Python library core Python language is going lazy. We decided to go lazy too. And second, this also probably comes down to

our backgrounds.

In web programming, I suspect that many times you want map. You actually have a small dataset, and you want that to be a list in memory.

Where for me, I do a lot of data processing. And for me, it's very valuable to have everything be lazy by default. I can stream through tens of gigabytes from disk on my machine,

and be very happy. And tools just works in a small space.

But that

I agree with you that in many cases, when I do work on, you know, just very small lists of 5 elements, I really do

find writing list, you know, list map sequence,

to be very annoying all the time.

Still, I see this, using the teratas

as an optimization.

So it shouldn't go by default.

And, this way I don't agree how Python 3

map and filter does things.

So you should go with lists by default and if it's too slow or too memory consuming when you switch to iterators.

It's like

premature optimization

and Python 3 forces you in it.

And,

I it's rather so we

are inherently more complex than lists.

They're

more hard to

handle.

So you write

a more harder to to handle code by default

and this is wrong.

It's also

very useful sometimes, though. I think if the whole community gets on board with the concept of lazy iterators,

then the whole ecosystem

is lazy by default. And that that has a lot of value, again, when you're when you're processing data, which for me is the common case. Again, I think this is the difference between the web community and the science community,

where it might be an optimization, but it's an optimization we want almost all the time. And it's it is quite tricky to make sure that,

all the libraries you're using everywhere

are being lazy by default.

It's sort of very easy in your own code just to use lists.

But if you're writing infrastructural code, it's gonna be used by many different projects at the same time, you sort of want to be lazy by default.

Because a lot of those users, again, in my community,

want the laziness. It's a high value.

But we can disagree. That's that's that's fun.

And so what inspired each of you to create your libraries?

I started from writing some

data utilities and

functional

utilities.

It was a a part of

a part of a web application.

Only later I extracted,

it just grow this part

over time. And then

I extracted it in the library,

and

it was obviously very

limited in

in in its in its scope because

I only added, the things that, I needed for a project.

So

it's not like a complete library.

So I looked at, Closure Standard Library

and Corio Library and,

on, underscore

JS library.

And, I

looked carefully what,

things could be

useful in Python and,

if, if I need to change them when I'm

moving them to Python.

And,

I implemented

all

functions from

ClosureCoreClosureCore

library and from underscore JS library,

which

made sense to to move.

And so

so, Fancy was born.

Later, I added some some features that I

needed myself

or some features that people asked

or

contributed.

The idea of,

Alivery also it's that it

aims to

limit its scope at some point. So,

like, you add to it,

but at some point,

it's complete.

Nowadays, we usually

most projects and libraries

were never complete.

We, like, go with new and new versions and you add and add features but I don't think it's,

a good idea for library like this. At some you need to limit scope at some point and,

save it. This is out of scope, like, immutable

structures,

out of fancy scope definitely. And,

some other

things need to stop at some point. So

Matthew?

Yeah. So my experience was very similar. I was writing a lot of Clojure at the time. And I I found that what I really liked about the language wasn't any of the fancier features. I just really liked that the whole ecosystem was based

on everyone using core data structures. And then again, I talked about, you know, HTTP requests were just nested nest addict. Database connections just took in dictionaries of iterators of dictionaries.

And this is I found to be really valuable. And so I took that same thinking into Python, but I found that a lot of the functions that I was used to using weren't present. And so I, again, looked at closure, I looked at underscore, I looked at Ruby's enumerable,

and I started implementing them in in in tools.

And I found that, you know, it's a really nice approach to use core data structures.

And if you have these a good set of functions, you know,

20 or so functions,

that actually encompasses

90% of what you want to do on those data structures for most programs.

And so I was able to lift up my thinking out of 4 loops, out of maps to more,

to more higher level concepts.

It was really interesting looking at all these languages and the set of names that they had converged to, the set of functions.

And so it was actually really an easy project to to build.

All the API was set out.

Actually, so at the time, right after building this, I started working on a few different projects.

It was actually the same algorithm, but in different different languages. And And it was really cool to use python underscore enclosure

and have sort of a lingua franca, a common language, among all of these different languages,

where I sort of depend on this API being present. I found that really valuable.

To go with Alexander's

point, I think that the scope of these libraries should be small.

I

most pull requests today, I I reject

just because they're a little bit weird.

I don't think they just stop. I think they should keep growing. I think we're gonna keep learning. But

I think both tools and Funcie have done a very good job of limiting their scope and of being correct.

You know, there there are almost no bug fixes. It's it's actually really cool.

They they just work, and everyone

understands and trusts everything in the libraries.

The reason is, let's say, implementation

of many,

most of functions is very easy,

very simple.

Yeah.

Sometimes it's,

it's

more complex because you optimize it, but

it's still usually very simple where

the most value is in, like, concepts and ideas.

Yeah. Yeah. So I think that,

it's very simple also because

they're communicating

off a very simple shared abstraction. They communicate off of lists or iterators or dicts.

So usually, bugs occur in software

because we have 2 things that are independently correct. But when they interact,

there's some complex behavior and and a bug arises.

And so, usually, a lot of bugs are tracking down, you know, this piece into this widget.

And all of the functions and tools and all the functions in Func

are completely isolated.

And we just trust the underlying Python mechanisms, iterators, dictionaries, etcetera,

to

be the shared substrate along which they communicate.

Which again,

that lack of interaction

really

makes developing these libraries

a pleasure.

And for anybody who isn't aware of it anyway, there's a functools module in the Python standard library that provides some methods that enable some functional paradigms. I'm wondering where that module falls short and how your respective libraries either augment or replace some of the functionality found therein.

1st, fun tools is, not like a library with the idea. It's like,

storage for very loosely connected things,

which, gets added to into Python,

over time. A more interesting 1 is,

iter tools,

which, actually has some structure and,

has

some utilities,

to work with sequences, which are more,

probably more popular and mono.

If you

you speak about fun tools when, for example, it lets,

even compose

function composition,

probably

very basic thing. So it's very easy.

I'll I'll agree with Alexander here that I'm looking through the fun tools API right now. And the things that I reasonably use are LRU cache in Python 3,

partial,

single dispatch,

and reduce, because it's been sort of evicted from the the core language.

And there's really not a whole lot there otherwise.

Things like composition,

pipe, curry,

aren't present. And these are sort of, like, standard things you'd expect to be in any functional

library.

So it's not really that it's not,

it's really not sort of the love

of the core developers.

I'd also say so I think functools and iter tools have a lot of really great functionality in them. But sometimes so as someone who tries to adhere to standards met by other communities,

sometimes they deviate a little bit from

from common from common interfaces. So for example, the group by function iter tools is not like the group by function in any other language. It's a

false cognate.

I think the LRU cache and functools,

I would much rather have seen

them

accept some mutable mapping as a as an input rather than bake in LRU

to the caching mechanism. There's just some ideas that I think it would have been better if the community had looked out to what other languages have been doing for these exact same concepts and taken a little bit more more inspiration.

That being said, I think it's also fine. I think that a lot of this code shouldn't be in the standard library. It should be in 3rd party libraries.

You know, packaging is good enough now

that, you know, fun c or tools or f on py can just be the standard functional programming library for Python. It doesn't really need to be baked in.

Yeah. That was 1 thing I was going to ask about is whether you thought it would be

useful or something that you would consider pushing for to try and get, you know, some or all of or at least some small portions of some of these

other functional languages baked into the standard library to either augment or replace the existing func tools. But I also know that there's a

common meme that the standard library is where modules go to die because it's a much slower rate of change, and it's not as easy to

introduce updates or modifications

to those modules once they do become part of the standard lib.

I think,

now that we have,

better packaging tools, like, where are still imperfect but,

where better.

It,

makes less sense to keep a big standard library library. So we we have obvious things

like requests

which is far better when building

or or lib and

2 or or even

slightly fixed

fixed in Python 3.

But still we don't have this request in standard library and I think it's okay. The main and the main reason is is better packaging tools. And,

this situation

permits,

each project

to develop separately and usually faster. We can see how even better packaging tools like in Node. Js community.

They lead to, like, lots of lots of packages and, they lead to smaller packages.

So the better your packaging tools, the more and the smaller packages you have. And,

we develop

the faster way. We don't need to to keep

everything in standard library.

It's a at this stage.

Yeah. I I agree. I think,

the only the only value that it would serve, I think, is the value of social proofing

that, okay, here's something that, you know, an external authority has sort of declared as good. But I think we've we've handled social proofing through things like GitHub stars. You know, both Funcie and tools are well downloaded. There are they have lots of stars. We both publish continuous integration tests, And that's sort of enough to convince people that they're worth worth doing. The other value would be with distribution.

But I think, I mean, at least in the scientific world, you know, sort of the, you know, anaconda distribution and, you know, grain of salt, I work for this company. But the anaconda screen has become sort of a de facto standard among numeric developers.

And, you know, requests is there. All of the libraries that you sort of think of as being standard

are distributed along with what people download when they install Python. And you can you can trust that it will be on the user's machine.

You guys have mentioned it a couple of times, the, fn.py

library.

And I'm wondering if you guys can compare and contrast that with your own work or if it makes sense to dig into that further.

If if you don't think it makes sense to dig into that further, then I'm happy moving on to the next question.

I looked at it and I,

read the code of it. And, it's a near interesting library, but I think of it more like

research project, not some practical tool. It's a bit like Haskell.

It

has many interesting

things in it. And

this is why I read it's it's code.

But

I never even tried to use, it's, in

in any real code.

I think that f n.py is a really cool project.

It has actually a lot of really cool stuff in it, like persistent data structures,

a lot of neat syntax tricks to make pretty lambdas and streams.

There's for very clearly some heavy sort of scholar or Haskell inter

influence in it.

I would say that

it differs from the 2 products we've written, and that its its scope is much, much larger.

So it's it's encompassing lots of fun ideas inside functional programming, Where funsci and tools are both restricting themselves to what is clearly pragmatic,

clearly Pythonic.

And you can sort of insert into a workplace

without trumpeting the sort of functional programming

idea.

But I think that for the conversation, we should probably have Alexei here,

and that's,

that's not present.

And so there are a number of concepts involved in functional programming, such as carrying function composition, immutable data, and pure functions. And we've covered a number of those, but, for any of them that we didn't cover, can you describe some of those concepts and explain any that you,

tried to incorporate into your libraries if we haven't already covered

them? I tried to incorporate

some practical and useful tools and,

still I have things like, function composition.

I have even,

helpers for

immutable data to use

common collections like dicts,

as immutable by convention collections

which I

I do doubt is useful but but still. So my approach not like

adding more concepts in the library is I don't

think I have

much more

than those. And current composition, maybe particles,

a bit immaterial data help us. That's it.

So I'll say a few things. So,

like Alexander,

tools has sort of the basic pure things, pure functions, core data structures, function composition, iterators.

But some of the concepts we haven't talked about are present, not in our libraries, but in other libraries that are accessible in the Python ecosystem. So I'll talk about those for a little moment.

There's things like dispatch. There's single dispatch and fun tools. There's a library called multiple dispatch that I also wrote that does

allows you to dispatch a function based on all the types of all the arguments,

which can be handy to write some generic functions.

There's things like macros.

So a macro is

something like a function

which

gets its full input unevaluated.

So for example, if I call a function f on the input 1+2,

the function doesn't get 1+2, it gets the evaluator result 3. So when I go into the function call f, I get the value 3, and f can work on 3.

Macros actually give the unevaluated result

1+2

as an abstract syntax tree to the function.

This can be useful if you want to make a nice plot function. You want the user to say you know, plot x equals y squared.

And you actually want to have,

that full expression being handed to the the macro

without being evaluated ahead of time.

So the common use case here is

the assert statement in Python. So with assert, we say assert x equals y.

And assert gets true or false. And if it's true, then it does nothing. If it's false, it raises an error. But if it's false and it raises an error, we actually wanna give a good error message to the user.

And we want to, you know, say, you know, x, you know, x, which happens at the value 3, doesn't equal y, which happens at the value 7.

And we sort of gotten around this in very hacky ways. If you look at the project Pytest, Pytest actually compiles your code a little bit to do this for you.

There's a really cool project called code transformer

by Joe Jevnick and Scott Sanderson,

that does a lot of sort of compilation

of of Python functions in a really cool way.

It's you know, I'd never use it pragmatically, but it's it's very neat. It's worth looking at. This is, like, immutable data structures with persistence

and structural sharing. This is another concept.

So if I have,

say, some some data structure like a dict, and I want to change some element of it, but

I've already handed that dict dict off to some other process running some other thread or some other coroutine.

I wanna make sure that that other

user of that data doesn't see my change results. So in Python, currently, I'd have to make a copy. This is what Alexander was talking about, immutable by convention. But if it's it's possible with sort of very fancy underlying data structures

to

make a slight copy of a data structurally dict or a list

that

even though logically they're completely separate, these 2 copies, 1 slightly changed,

in memory, they actually share a lot of the same structure. So this is persistent data structures with structural sharing.

There's a project by Kurt Smith called Ozymandias, which is sort of new and and rocky, but it does this on the sea level. So it's it's it's quite fast. There's general thinking about types.

And there's, again, some nice work from the the core language team to think about, you know, adding function annotations into Python 3. There's a whole types module, allows you to spell interesting types. So some some interesting work happening there as well.

So just to be clear, there's lots of functional programming happening in Python.

And

what Alexander and I have done is just sort of a small part of it. There's lots of other libraries to look at.

Maybe not a concept, but

but another touch on, functional programming

thing that you can do in Python is,

like,

utilities to to abstract away flow control. And, in fancy, I have some of those, like, in the some for

handling errors

and, some for,

like, making the retries or things like that. I also have some back ports from Python

3. Like, in Python 3.4,

we have, this, suppress

context manager. Like, you like, we've suppressed some exception and a block which,

ignores this exception. You can write it in Python too also and and

so I beg and

edit a few other things,

like this to.

And about immutable data,

it's, like, if you have them, by default, like, in closure in all where all

your collections are by default,

immutable and they share structure.

This is a very different thing when when you have this immutable collections as a separate library because,

in this situation

when we have separate library, it's a very special purpose tool which,

very small amount of people will use. And, I think, in in Python, it will be like that.

That's true. But I would I would suggest that

as long as we adhere to some of the interfaces, like mutable mapping or immutable mapping,

we can pass these things around and get some more coverage.

So for example, I think that Python has actually done a really good job about having some shared interfaces, like the file interface,

the underscore iter interface.

In the scientific world, we've got the array interface.

And these allow

a lot of more custom applications. So many people have written array libraries.

And they can all interoperate because they all use these these shared protocols.

And so I think that with things like immutable data structures,

the sort of hook into the Jamet

ecosystem is going to be with those shared protocols. They're gonna implement mutable mapping. They're gonna implement sequence.

Mhmm. Agree.

This,

protocols in in Python were very beneficial.

Very recently with

a big thing in, closure called transducers.

And,

when people started to

make their own transducers in all sorts of languages

and, this includes Python.

So there are probably several implementations

of, that for Python. And then

if you're looking

a bit closer

to it, it, does not actually makes much sense for Python because,

transducers

for

closure, we solve,

we we solve a problem that Python

had solved years ago with,

iterator protocol and with, things like that. So, protocols

like iterator protocols and

abstract base classes like sequence. They help a lot. Agree.

And, what are some of the resources that you guys have found to be most helpful when trying to learn and apply different functional paradigms to your programs?

I'm

in a way self taught programmer, so I just try things. But there there are probably this talk by Rich Hickey called Simple Made Easy,

which,

advocates,

the use in data instead of classes and generally

simpler things like, things with, those

less amount of parts. Not sure if you heard it, but

you probably should. I've heard of it a few times, but I haven't yet watched it. So I'll definitely take some time to do that.

Yeah. It's only,

half an hour, and it's it's really good. Has

other good,

talks, and many of them are very very opinionated. Like, he advocates things that he

doesn't closure.

And for

many languages these things they don't make that much sense or don't make sense at all. But still

we're usually interesting. But but if I chose the single thing,

single talk about functional programming,

this will be this symbol made easy. It's also very interesting to look at, functional programming languages

and, at the standard libraries,

like,

Closure and Haskell and ML, different sorts of ML. So these are my sources.

Yeah. I'll definitely repeat the suggestion for Simple Made Easy. I think it's a great talk. I've I I watch it every once in a while, honestly. It's a little shameful, but it's a great talk. Rich Hickey is a great speaker or he's a very opinionated speaker, and he distills his ideas very well. Whether you agree with him or not, up up to you. But SimpleMedes is certainly,

certainly worth reading.

I would I'd also like to repeat, I would say language tutorials.

I would say that in the last few years, there's been a good surge to

increase the friendliness of languages,

especially in the functional space. And so, you know, things like learn you Haskell. There's probably learn you an Erlang or something.

I would generally encourage

anyone who's interested in programming

to

gain a little bit of exposure to a lot of languages.

And this really helps you to understand much more about your own language. Just how when you learn a second natural language, like if you speak English and you learn French, it teaches a lot more about the structure of language. So you're not necessarily learning these languages to use them, but it's nice to learn go and see how that affects, you know, how you might use tornado or asyncio inside of python. Actually, in my most recent project, I was using

concurrent programming a lot. I sort of settled on tornado,

and before that I read lots of talks on go, or I listened to lots of talks on go,

I read all the ClosureCore async documentation. I read the asyncio documentation.

And

really by exposing yourself to a broad set of ideas from a VOD community, you really become a, I think, a much more well informed programmer. So learn functional programming

from many different languages,

but come back home and

write fairly normal Python. Just slightly

slightly different. If you are learning languages for concepts, then, I think it's better to find, the least,

least,

familiar language and with different paradigm. Like, you it's enough to learn, like, some Lisp and some ML for for functional programming. And then you better go with some

logical programming and, when something like stack based, like Forth, this will benefit you more than learning 2 ml type languages or 2 lisps. This is a a bit of aside from functional programming. That's more like an inter learning concepts.

Yeah. I can definitely agree that learning

different languages of the same general

family aren't necessarily going to give you any new they they may give you some new ideas, but they're not going to give you any new concepts

or,

drastically improve your worldview when it comes to programming. So for instance, I've had experience in programming both Ruby and Python. They're both pretty familiar once you've started to learn 1 of them. But I've started delving a little bit into the Elixir and Erlang languages, and that's definitely an extreme paradigm shift. So it's

not as, easy to pick them up because of some of the functional concepts involved.

Yeah. I'm quite excited about Elixir, actually. That's all the time on my list to to find a project to work in.

Yeah. Agreed. That and Rust are the 2, the 2 biggest ones on my radar at the moment.

So,

is there anything that I didn't ask that you guys think that I should have or anything else that you guys wanted to bring up before we move on? No. I think we've covered it pretty well. Okay. Great. Yeah. Another little point to this discussion about languages.

Sure.

Sometimes, even if you look at language with which you

immediately understand because

it's similar to whatever you know already. It's, still beneficial to look through its standard library or some ecosystem or its packaging tools

because, this could be different or or this could be better. So besides Python, I'm also using Node. Js and I look

at this

at where package manager

npm and,

look at, how we install libraries and how we handle, dependencies.

And, it's, it has a

very

significant difference. Like, you in single process, you you can have, several

several copies of a semi library of different versions.

This makes you go with dependencies,

entirely entirely

our way. You have some new problems, but you have solved some problems that you have in Python. And,

if

you say about, Python and Ruby, which are some somewhat similar but you can,

read this, like, enumerable

class and all its methods. And, it is

interesting to read at least first time. So I will actually say 1 thing. We should probably add a disclaimer that both Alexander and I, come from a similar conceptual background. We both largely agree on we both largely agree on things.

I think if you had another programmer who is, say, more familiar from the Haskell community, they would have very different opinions.

So,

1 shouldn't take our our words here as being, what what functional programmers think. There's lots of functional programmers that would disagree with a lot of what we just said. Functional programming is a very broad,

very broad term. And a functional a Haskell program would come on to this show and say

a completely different set of of

very valid points.

I agree. We could have added also some object oriented

advocates, like, coming from Java

or at least from Ruby.

Although there are a number of functional aspects to Ruby as well and and a number of people in that community who are embracing some of those functional paradigms, much the same as people in Python are.

Absolutely. Functional programming is,

becoming much more hip these days.

So for anybody who wants to follow you guys and keep up to date with what you're up to, what would be the best way for them to do that?

Alexander, how about you go first?

I think it's, Twitter,

Hackflow. And I also have a blog which I don't write often,

but still it's a hack hackflow.com.

Okay. And, Matthew, how about you?

For me, so my professional website is matthewrocklin.com,

matthewrocklin.com.

You go to slash blog, and I write fairly regularly there. I just wrote a thing on testing. And there's various links to all the projects I work on. I wrote tools years ago. I, and I wrote tools years ago, but I work on lots of other projects today.

For tools, you go to tools, t o 0 l z,

dot read the docs dot org. I'm also mrocklin@github

or Twitter or most things.

Great. So with that, I'll move on to the picks. And my pick today is going to be Datadog.

It's a software as a service platform that makes it really easy to get up and going with server metrics and has a number of different integrations for being able to

get some pretty detailed information out of a number of services you might be using. So for instance, if you're using Mongo or MySQL

or Nginx, you can add those integrations and start getting some pretty fine grained metrics about how those systems are running.

That makes it pretty easy to integrate with a pager service. So things like Opsgenie or

or PagerDuty,

as well as your chat tool that you might be using, for instance, HipChat or Slack.

So recently

added this into the environment of, where I just started working because we were using Zenos, which,

well, it's a good tool in its own right. It

doesn't seem to handle as well a cloud workload.

So Datadog just gave us a really quick and easy way to get some pretty solid metrics up and running and start integrating that with some of the other services that we're using.

And with that, I'll pass it to you, Alexander. What do you have for us for picks today?

I'll choose a TV show.

It's a an Expanse,

sci fi 1 because I I love everything about space. And, we don't have much

TV shows about space now nowadays.

But, and this looks good because it's based on a book. It's a it has a solid

story.

It goes great so far.

If you want

something more practical, then it could be I can type in this.

Okay. K. The more practical thing will will be,

the Revolut app. It's

an app which sends you bank cards which works in every country without any,

fees in bank accounts

and translates money. It's the best, cost you it it best rate you can

achieve. I've been traveling

last month and intend to travel

1 month

more, so it's very handy for me.

Yeah. So I've got, 3 picks. First is a library that I ran into just last week on a tools issue. It's Riemann, r I e m a n n. This is a a closure library, so sort of in inside the same theme,

for they got a really nice stream abstraction, and it's for monitoring distributed systems. If you go to that page, riemondotio,

there's a there's a video on the front, that I really enjoyed. So if you're watching this podcast, it's it's just some of these ideas. It's nice seeing some of those ideas in practice

in sort of real concrete problems. It's a it's a nice it's a nice video.

Yeah. That's a project that I've been keeping a close eye on myself with the intent of putting that into the infrastructures that I've worked

on. I have not yet had the opportunity to do so. I'll second that recommendation wholeheartedly.

Yeah. I just learned about it last week. I thought it was really cool. The second 1, more of sort of a humanities

pick, it was a movie 5 Dances. I think it's like a an independent film.

And if you're into dance or sort of human movement, it's it's really cool.

I think in the first 5 minutes, you'll figure out if you love it or hate it.

But it's just it's a bunch of dancers. There's a little bit of plot, but it's mostly them,

in a in a practice room going around.

So I think it's really cool, but, again, it's not for everyone.

The 3rd 3rd pick is actually a little bit self serving. What I've been working on for the last few months is a project called Distributed,

which is a dynamic task scheduler, so something like Make,

but for sort of data science, analytic operations.

And it's on distributed systems

using the concurrent futures interface.

So if you like concurrent futures, but you wanna use a full cluster,

check out distributed.

Again, it's sort of new. It's in beta, but I'd love to have some feedback on it, especially from, your your your viewership.

So that's it. I was also gonna recommend Ty, but we also mentioned that earlier on in the show. So Very cool. Well, thank you both for taking the time out of your day to join us and talk about functional programming and your work on some libraries to make that easier to do in Python. And thank you for having us. This is really fun. Yeah. Thank you too. Alright. Adios. Bye. Bye.

The Python Podcast.init

Summary

Brief Introduction

Interview with Alexander and Matthew

Keep In Touch

Picks

Links

The Python Podcast.__init__

Summary

Brief Introduction

Interview with Alexander and Matthew

Keep In Touch

Picks

Links

The Python Podcast.init