Morepath with Martijn Faassen

Hello, and welcome to podcast.init,

the podcast about Python and the people who make it great. I would like to thank everyone who has donated to the show. Your contributions help us make the show sustainable. When you're ready to launch your next project, you'll need somewhere to deploy it, so you should check out linode@linode.com/podcastinnit

and get a $20 credit to try out their fast and reliable Linux virtual service for running your next app. You can also visit our site at python podcast.com

to subscribe to the show, sign up for the newsletter, read the show notes, and get in touch. And to help other people find the show, you can leave a review on Itunes or Google Play Music and tell your friends and coworkers. We also have a community site at discourse.pythonpodcast.com,

where you can talk to other listeners and suggest ideas and ways to make the show better. Your host as usual is Tobias Macy. And today, I'm interviewing Martijn Fasen about the MorePath web framework. So, Martijn, could you please introduce yourself?

Hello. I'm, Martin Vasser. I'm, I call myself a web developer. I'm from the Netherlands. And I've been doing Python for a while since 1998 and,

yeah, I do various other things,

like, garden and play role playing games and

various other geeky and less geeky things.

What are some of your favorite role playing games?

Recently, I've been playing 13th age. That's,

it's kind of, I actually, in my whole sort of years years of playing role playing games, I never actually played any variety of d and d for some reason I managed to avoid that.

But 13 h is sort of a spin off fork of the d and d system and I've been enjoying sort of doing that. It has a lot of, wacky monsters and it's easy to run. So I've been enjoying that game and people started playing, Pathfinder as well, recently.

Yeah. I'm a fan of board games and tabletop games, so I'll have to take a look at that 1. And how did you first get introduced to Python?

Well, in 1998,

I was,

programming

an artificial life simulation in c plus and I read about some, some ways to analyze your c plus code base,

to see how, you know, the module dependencies were and whatnot. So I thought, okay. I read about these scripting languages in a magazine article. I've listed a whole bunch of them. And,

let's see whether I can write a little script to help me manage my code base. So, of course, I, tried out Pearl, which was the the big fish at the time. Had it on my Linux box.

I kinda liked

some of it. Like, I

I liked how it was so easy to create these data structures,

and that you didn't have to worry about manual memory management and such. But I couldn't quite get my head wrapped around it. And then

1 day

I,

printed the Python tutorial and then I read it in the evening and then I always joke that the next day I could program, in Python

not very well and not very idiomatic but I could, I could use Python and that sort clicked for me. And then I was looking for a job,

around that time, and I got in,

into contact with

the local university

and they were looking for someone who could fix,

their well, they had this DOS batch file driven database munging,

code going

on, and it broke. And the person who created it, was gone and would never return. And they wanted to someone to,

to fix it. So I said, so should I fix that that code? No. No. We want you to throw it away because it's completely unmaintainable.

And then I said, oh, so I have to write it a new, a new, version of it and, can I use Python? And of course, they was 1998. Nobody ever heard of Python. So they said python. We never heard of that. Is that a readable language? And I said, no. Yeah. Python is very readable.

And then a few weeks after I discovered Python, I was programming,

I had a job programming Python

and then I joined the, the Python news group and then I

shortly after started working with Python professionally, I saw this thread there in the news group where lots of people were lamenting that they had to use some other

languages in their day job. And, oh, they wish they could use Python. And I sort of stumbled

into it and,

basically haven't touched c plus since then and,

yeah, it was kind of funny how people were lamenting and I just sort of accidentally

happened to get a job doing Python and then I quickly rolled into web development. Soap was released the same year,

and I started getting into, doing web development with that and, sort of did a lot of stuff with Python over the years.

Yeah. It's definitely funny how things can work out like that where some people will be using a language for quite some time as their preferred language, but only as side projects and still aren't able to use it in their primary role and other people come into a primary role and find their favorite language. So

Yeah. Yeah. I just locked into it because I came I proposed Python to them and they said, okay. And then that was,

that was it. And then I I brought in Zope and, all kinds of, they I I would always try whenever we had meetings and said, oh, we have this problem, I said, oh, I can write a Python script for it until that became a joke.

So can you tell us a bit about what the Morpeth framework is and the problem that you were trying to solve when you created it?

Yeah. MobilePASS is is a Python

web framework, a server framework, like, so many other, like Flask or a Django or Pyramid. Besides

trying to actually solve actual problem, it's also an

actual problems, it's also an act of creativity,

as much as actually solving a problem.

I just sort of can't help myself creating new things once every while.

I'd been using web frameworks and helping create them for so many years. This was sort of, it's like, let's do it again and this time, for real,

kind of, a reaction. What I was trying to do with more path is is to create a web framework that would make it easy to create REST web services. That was sort of the 1 of the primary goals.

And then

REST sort of definition has eroded somewhat over the years.

Originally,

it had this whole notion of links between resources

and automated discovery very much part of the definition, but that's not how people use rest in practice.

So another way to,

to say the same thing is that Mopar tries to be good at helping you build hypermedia APIs, which is sort of the modern way to express

what rest used to mean. And my background is in is in Zope, which was sort of the big web framework,

from its first release in Python land, at least from its first release, in in 98,

for almost a decade afterwards. And Zoap

software is huge and convoluted and has all kinds of history in it, but it also has a lot of power. And

1 thing that I wanted to do is express the power of ZOAP, the things I've learned that worked well in ZOAP, and express it in a very small and flexible form. So that's why I call Morpath the microframework.

So that's sort of from the perspective

of my experience with web frameworks.

And you mentioned that 1 of the primary concerns of Morpeth is the ability for it to be RESTful, whatever for whatever definition of REST people want to apply to it. And as you mentioned, 1 of the original

ideas behind rest was the fact that there would be this linking between resources. So what are some of the ways that Morpeth makes it easy to create those

resources? And, also, is there any sort of built in support for some of the various

standards such as the,

JSON API or some of the standardized

rest methods that are outlined in, the

of course, I'm gonna blank on the name of the book.

RESTful Web Services, perhaps. I don't know. That's the 1. Yeah. I only read the first edition,

of that, but I have the new edition, but I haven't read it yet, actually.

I think it's it's got renamed as well. It's not really a new edition. It's it's kind of a partial rewrite, I believe.

So,

more path doesn't do anything specific about JSON API, which is slightly newer, I believe, in in in it's, sort of when it came together. But it tries to be it it's it's more low level than that. So it tries to be flexible enough to let you implement things like that, but it doesn't actually,

automate it itself. Could be part of an extension, but it's, that that's not currently,

something we

we do. At the time, I also played quite a lot with,

JSON LD,

which is sort of this semantic web link data solution.

And there is this emerging specification called Hydra that,

also lets you, sort of define a RESTful API, but I sort of came back from that because it's sort of still too much semantic web, and it's hard to wrap my head

around what they want to do sometimes.

It has nice features, but it's also has some drawbacks.

But what Moba tries to be is sort of flexible enough and small enough so that you can implement things like that on on on top of it. And,

since it allows you to write reusable and generic code, it is actually fairly easy to define libraries that provide some standard views, so that if you implement a certain kind of model, it will automatically generate,

that kind of front end for you.

And does it have any capability to be able to support some of the newer standards, such as GraphQL or Falcore?

No. And, actually,

that was kind of an interesting experience. So I spent some,

some years building a, a web framework that could do rest really well. Right? And then,

this whole emerging no rest movement,

started

sometime last year. I guess it's really started getting traction. And that gave me some pause,

because I think some of the sort of the criticisms of REST and the improvements to REST that, in particular GraphQL, which I have the most, sort of experience looking at. What it offers is

is actually you know, there's a lot of validity in the points they make. And, you know, as somebody who's been doing rest for, for a very long time, that sort of was an interesting thing

to have to deal with. And more path itself is not really good fit for for serving a GraphQL

back end because it's it's all about HTTP, and GraphQL is just not about HTTP. And

so

while, you know, a website

or web application and a GraphQL service could very well coexist,

they don't really, have much to do with each other. And so I started thinking, like, okay, well, if GraphQL is perhaps the future, then what am I doing developing more path? Right? That's kind of an interesting question to, to have the face as a developer after you spend a couple of years into a framework.

But, of course, REST is not gonna go away anytime soon. There is still plenty of, new development happening. And it turns out that more path is actually also quite a useful framework for developing any web application, including non REST web applications,

meaning just classic web applications that generate HTML on the server and provide dynamic content.

It works well for that as well, and that's certainly not going away anytime soon either. So whereas perhaps this whole future of doing web services is not supported by ModPath,

there's still plenty of things you can do with ModPath that are not going to go away anytime soon.

Yeah. As with anything in technology, there's always gonna be a long tail of

solutions

and

technologies that are used

by the large number of companies and individuals

even beyond the point where it stops being the most talked about thing. So there are a lot of cases where you hear people talking about the decline of a particular language or framework, and then you actually go start talking to people, and they're still using it in their to day, and they're perfectly happy with it. So Yes. Yes. And and I don't think that I mean, HTML generated by the server,

sitting by an HTTP server is not sort of the declining technology of of the of the day.

That's still extremely relevant.

How you implement web services. Well, it's interesting that there are now sort of new ideas again,

that, of course, arose in part because of our experience with with doing REST for for for so many years now. But since Molbot is just a it it's a good way to do any HTTP application, not just

web services.

I don't think, it's become irrelevant in the last year.

And I'm assuming that it speaks the standard WSGI protocol?

Yes. Yes. It's a WSGI framework. And do you have any plans for any modifications

to more path to make it more compliant with the new HTTP 2 specification?

No. I haven't really, thought a lot about that. I mean, the basics of HTTP even in HTTP 2 are are just still very similar and generally would be handled by a front end framework. So if that speaks HTTP 2,

then, you know, your whiskey should still work. But I haven't really looked into it very much. I don't know how h t p 2, would affect

whiskey. I mean, there is the whole notion of dealing I mean, it can deal better with with with streaming resources and with

lots of small accesses and reusing the same connection and all that stuff. But that's more something for HTTP server than than for a web application framework, I

would say. I have thought a little bit about sort of asynchronous

frameworks, like the whole HTTP

AIO framework, I believe it's called,

and and the whole Python 3 async infrastructure.

And I see some interesting potentials for the way more path does its routing and does its view lookup to make that async. I'm not sure whether there there would be a real significant benefit, but that would be interesting to explore. But I I've only thought about that so far. I haven't really done anything with it.

So the tagline on the documentation for Morpath is that it's a web microframework with superpowers. So I'm wondering if you can tell us a bit about what the superpowers are that it has and what makes it special or different that sets it apart from some of the other options in the Python ecosystem that people might already be using.

Right.

So Morpoth is, is a small web framework, but it tries to pack a lot of power in the small package. And

things that set Morpoth apart are

centered, I think, about around perhaps 3 topic areas, but maybe we can restrict that

to to 2.

1 thing that Morpah tries to do very well,

which is sort of something that I learned from from ZOAP, in particular the ZOAP 3 project,

is that it treats

configuration of code

as a as an actual explicit thing and

by that I mean that

many frameworks, many web frameworks in Python

sort of

implicitly

they have configuration systems in their

in in their code base. I'm a diabetic. What I mean by configuration system is,

for instance, a way to register views with routes. That's that's configuration

or to,

give a permission to some particular access. That's another sort of aspect of configuration or,

to perhaps register middleware

for your for your whole

pipeline so that whenever requests come in and they go out they are they they are handled. And there are actually quite a a few different things you can configure in a typical web framework.

And many web frameworks

have ad hoc mechanisms to do this. So they have 1 way to register routes,

and they have another way to,

perhaps,

plug in generic views, and they have another way to do middleware registration. And what more path path tries to do is that it makes that explicit.

And by making it explicit in the code, you can actually reason about it. So so that means that extensions and overrides,

if you have a piece of code that you want to reuse and you want to extend it or override it, which which happens a lot, is a general principle. It's not a, it's not a ad hoc sort of you the the framework doesn't have to offer an ad hoc API to do all these different tasks

of overriding or adding configuration,

to the system, a separate API. So sort of a general principle of how you do that. So that's 1 big part of, I think, what sets SmallPath

apart, shares that with,

with with with Zope and also Pyramid, which is also inspired by. So Pyramid also has a configuration system.

And the other

aspect

that makes more bark different from most web frameworks is that it,

is its

its publisher. And by publisher, I mean, sort of the system that turns an HTTP request with a URL in it to, an

view code that is being rendered and returns either a JSON result or an HTML,

a dynamically rendered HTML result. And the publisher mobile path is structured differently than what most web frameworks do. Most web frameworks are routing frameworks,

and you you define

routes. You define paths,

and that and when the path in the request matches the path in your route,

perhaps with some with some variables in it, then

you have a view function associated with it. And that gets to a request and then has to return a HTTP response with some result in it, some JSON or some HTML or whatever. And MobilePath is also a routing framework, but it routes it

sort of divides this routing into 2 steps.

First, it routes you to an instance of a Python object, which I call a model.

And then for that model, it looks up a view,

and the the view is responsible for taking the model and taking the taking this Python object and taking the request object and then returning the response. And it actually turns out that this is something that you do in routing frameworks, in any routing framework, not just small path anyway, but it's it's sort of, the framework doesn't know about it. It's left to the view functions to do this. So what you do in a Django view function or a Flask view function is typically you look up your your Python object instance from a database, and then immediately in that same function, you start

generating the response for it, based on it. And what more path does is separate that into 2 steps. And that gives it all kinds of, sort of interesting,

consequences

that it can exploit,

and automate that other frameworks just don't know about.

Going back to what you were saying about the fact that Morpath has a more standardized approach

to configuring the layout of the application, in particular, the middleware.

I'm wondering if it would be able to solve 1 problem that I've experienced recently, where, for instance, in Django, I was trying to figure out an easy way to hook into a particular function call in 1 of the middleware stacks so that I could cache

the call and return you know, short circuit the

response so that I don't have to go out to the database every time. But there isn't really an obvious way of doing that aside from just monkey patching the middleware itself. So I'm wondering if that would be something that would be easier to solve with the way that Morpath handles the layering of the application.

I think so. Part of the middleware story I stole from, from from pyramids, you give, credit there.

So has this concept called tweens, which are basically very simple middleware that gets a request, to get a request and have to return a response and then defer to underlying middleware.

And there is,

yeah, there's a sorting subsystem inside of MolPath that can do sorting by dependency. So it can does a topological sort and it's very easy to sort of insert middleware between 2 other middlewares

by just specifying your your configuration. This middleware needs to be under or over,

this other piece of middleware. And what you can do is you can write an application.

If you already have an application that somebody else is maintaining

for whatever reason,

that has middleware already and you want to insert your own middleware into this, then with Molotov, it's actually very simple. Subclass that application object and then insert sort of your own middleware there. And the original,

application

class is not affected by it. So if somebody else uses it, they don't have to worry about your middleware. But you and when you use it,

you use its subclass, and the middleware will be where you expect it to be. And that's a general principle that doesn't just apply to middleware, but also to views and routes and and and permissions and, so sort of the

everything that can be configured can be overwritten or extended in the same way just by, subclassing the,

the other more path application and then associating new configuration sort of statements in the form of Python decorators to it.

It can be difficult to convince somebody to migrate to a new framework,

especially if there is a lack of any supporting ecosystem for that framework. So I'm wondering what are some of the motivating factors for a developer to want to switch to more path if they've already got experience with 1 of the more widely used frameworks?

Oh, that's actually,

I've I've thought about that a lot and I don't have a great answer for it. This is not the first time I helped create a web framework.

I actually started the grok project back in 2006,

which was based on,

Zoom, technology, and it was a sort of an attempt to make that more approachable. And in 2, 000 6, when, that got started, it was still a lot easier to track people to your framework

than it is today.

Today, there are so many Python web frameworks around that. It's very hard to get any traction. And more path is in this difficult niche where on the 1 hand, it tries to present itself as a approachable framework.

But it sort of has all this power inside of it at the same time. And it's kinda hard to sell because the people who need powerful frameworks, they might be using Pyramid already. And that's fine. It's a great framework. It can do, more than more by path. It can probably in the end, it's it's a little bit less opinionated,

making it perhaps a little bit harder to figure out sort of how you,

want to organize it. But, you know, those users are happy already, so then I'm gonna switch to more path. On the other hand, if I try to convince somebody who's already using Flask or perhaps Django to say, okay, you should, you should consider more pop. They they they might not actually see some of the value that it brings because I don't have an experience.

This this whole, for instance, this configuration,

story. Or they maybe they have experienced the problems, but they don't realize there are actually solutions to some of these problems. Like, hey. I want to specifically override this thing. So now I need to monkey patch, and they just sort of or the framework needs to provide some specific API so I can do this specific thing to it,

and all these specific APIs are ad hoc,

then you might not actually realize that there are better ways to do it if you don't have experience with frameworks like ZOAP or

or Pyramid. So it's hard to sell sort of either way,

and plus yeah. It was just a very crowded field. But I would say if you if you are looking for flexibility,

if you're looking for,

for power in the form of reuse potential, we're using applications,

in various ways both through inheritance and composition. If you want something that's still small and manageable, it's a microframework

that you can plug into existing applications perhaps that need a web front then. That's sort of where more pop fits. And I also think it helps you organize your code better. So if you're interested in sort of more path encourages you to structure your code in a way that I think actually works better for web applications,

by by separating

the way you look up the model instance

and the way,

the the view is looked up. It actually encourages you to structure code in a way that will lead to

more readable code, I think. And then it can use that information. So it can, if you need a powerful, permission system or if you need a way to, automatically generate links without worrying about the the details everywhere. If you want that to be easy, then more pop is there for you as well. Concerning more about ecosystem, make more about the ecosystem, it actually small, but

there is actually quite a bit already out there. So, if you want database integration or or different authentication systems or different template languages, static resource serving. There's a lot of stuff out there already that that that you can use, because it actually is relatively easy to make,

a more path extension.

Yeah. From looking through the documentation, it seems that because of the fact that a large portion of the way that you would

add support for additional capabilities is just through subclassing and overriding a certain method that it would be fairly simple to wrap existing libraries so so that you just need to write a little bit of glue code. And then then you can leverage 1 of the existing libraries for supporting whatever functionality it is that you need. Whereas some of the other frameworks like Flask or Django might have a bit more boiler plate or specific

structure necessary to be able to actually bring in that additional capability.

Yeah. Usually, it's just if you want to use the capability, it's, it's by subclassing an application that sort of configures that, capability already. 1 interesting thing is that if you build a larger application or you've been to build a selection of larger application, you often end up in a lot of common code between them that sort of is edging towards becoming its own little framework. And and more path makes that transition from application application specific code and framework code very smooth.

So if you already have code in an application that starts like you want to factor it out into its own library and then you can reuse it in multiple places, The path towards that in more path is very

smooth as opposed to, I think,

frameworks which have yeah. But we just only have to go to a whole different way of configuring things or a different way of thinking.

So can you share a bit about what the internal architecture of Morpeth looks like and some of the challenges that you have faced or are facing while building it?

So I would say it falls apart in,

into 3 areas. 1 is the publisher, which does the routing and the Vue lookup. Then there's the configuration engine, which is in its own library called Dectate.

And there is the predicate dispatch system, which is used by the VUE lookup system and also by the, permission system and a few other areas in Morpah. That's in its own library called REG. And,

Morpaw is basically

a package that defines a whole bunch of directives based on on DECTADE on the DECTADE system to configure its publisher.

So you can add routes, and you can add views, and you can add permission rules and middlewares and things like that. And that's sort of what the mobile package really does besides the publisher implementation itself.

And challenges I had building it, perhaps it's more interesting to speak sort of in terms terms of,

of of history of of these these different,

subsystems because they actually go back quite a while in my thinking to before Bob Morpeth was actually born in 2013, I believe. I mean, the publisher, the way that

routing works to a model and,

the way if you look up works is it goes back to,

something that I did for Grok,

I think probably closer to 10 years ago now, where I realized that I was using SOAP. And SOAP doesn't use a routing system. And,

SOAP uses traversal.

And traversal

gets you to a view that renders a HTTP response

in a different way. What basically happens is that when you have a URL path like a/b/c,

what a traversal system does is break it apart into steps. So you have step a, step b, and step c. And then it has an object tree. So there's some kind of in memory object tree or perhaps generated on the fly of Python objects, and perhaps the outer object is,

in the root and it's called a.

And, the root object simply has an attribute called a or perhaps it has a an item like in a dictionary,

with the key a. And the routing system then just fetches that object. And then it goes looking for b and the object that finds them, and then perhaps it does another get item access, into that object. And it finds b and then looks further, and it does another getitem access to find c. And then in the end, it's like, okay. I have this object now that represents, I don't know, date on a calendar or

to do item in a in a in a database of to do items or whatever it might be in that tree. And then it looks up a view for it. And the view is then a function that takes that that instance, and it takes the request, and it returns the response. And that works very well if you're especially if you you have an object database like you have in ZOPA. So it's typically Zopa applications typically based on the Z to be, which is a very nice and powerful,

Python object database, which lets you just store arbitrary Python objects in 3 form without worrying too much about anything else. You just change attributes and you just know that they're gonna be, they're gonna be saved in the database. You can add objects. They're gonna be saved in the database. So it's like working with an in memory tree. But it doesn't work so well, this traversing approach, when you actually have a relational database because even in ORM, it's actually quite hard to turn a relational database into a tree of of objects in Python. There's all kind of sort of details like how do you navigate a relay in the a collection

attribute,

what names do you do see do these things get? And I experimented a little bit with that approach to do traversal through a relational database, but I realized that routing is actually a better approach when you need to deal with this kind of data. And but I like the as I like the aspects,

some aspects of the traversal system. I like that it was possible

to create a link for everything that you find

in that object tree really easily. If you have an object, it's actually relatively

easy to just walk back up the tree and reconstruct the path that was taken. And so for any object in the database, you can just easily create it in a hyperlink without having to know about how to do that everywhere in your code.

So, because that creates tight coupling between

the code that needs to make the link and

the the object that you're linking to and and I didn't like that. And

another thing that I like of this approach is that it separates this whole

model lookup from the view lookup because then you can become more flexible with the ways views are looked up. You can look up the view based on the class of the object that you find. So that for a base class, you define a view for base class, and it will automatically work for all the subclasses. So maybe you have a base class called document, and then you have a more specific document subclass, and it will just automatically share the views with the base class.

And,

it also helps you structure sort of your code when you separate those 2 concepts of

resolving to a model

and resolving a view for a model. So what I then sort of I had this sort of,

this this problem that I wanted the features I liked from traversal, but I also wanted the features I liked from routing. And that's how I ended up creating a library for grok called,

Traject, which tried to sort of square that circle and have the best of both worlds. And then

that sort of directly led into the publishing engine for more paths. So I already had some experience with this approach, and I liked it in in the context of the Grok web framework before I sort of double down on it in more path. The configuration engine goes back as already mentioned to,

my history with the, with the Soprep framework. And I already sort of saw the power of the configuration engine. That data is very much sort of my attempt to make it also more approachable to make it very easy to do particular common things you want me to configuration framework. A grok was already a reaction to the way soap configuration work. Because in soap 3, you had to configure things with XML files and you would always have to look at the XML file and your Python code and became sort of very cumbersome, at least to, to me and a bunch of other people. Some people like their approach to each their own. So Grok tried to integrate the configuration with Python code already, and that led to my,

that experience with Grok also led into,

the, more path configuration engine together with looking a lot of what Pyramid has done because Pyramid also builds on ideas from Zope. So, it was very interesting to see how Chris McDonough approached things, and I stole a lot of ideas from him. And then,

the reg library goes back to my experimentation

with the ZOAP component architecture. ZOAP has this powerful system for

looking up objects by their interface.

So when you have a, you say, I want something that can send email, and then you have this interface that describes how you can,

sort of that's an icon. I email sender interface, and then you can look 1 up and you can just plug in an arbitrary,

email sending concrete object into the system. You can register 1 and then you can just ask for the abstract object and the the interface of doing it and then that allows you to build more pluggable applications, which is something that Zoop is good at. And and that also works that way with Vuelookup where you say, okay, I have a document. Now give me something that can actually render it to an HTTP response. So I worked with these old component architecture for years, and,

I liked it. But I wanted to improve just a few tiny little things about its API

to make it sort of more easy, and to,

sort of consolidate a whole bunch of separate APIs into a smaller, simpler,

more generic API. So I propose that to the soap people, to to the other soap developers, but it never got any traction. And then I also was actually interested in how it was implemented because there's this whole looking and stuff up by, by interface and interfaces can can be, subclasses from other interfaces and, you know, extending other interfaces and then you want to find the right thing. There's some interesting algorithms going on there. So I also separately actually from the improving the API story, I played with creating my own implementation of the subcomponent architecture just to have an idea of how that works. It was basically just play at the time. And then when I started building more path, I thought, okay. Well, actually, I can put those 2 experiments together. I can create a better API on top of a reimplementation

of the component tech architecture. And that was the start of the reg library,

sort of a general library for plug ability for looking up things by their interface or actually in reg,

and by their, by by the class. So you can say, okay. I want something

that subclasses

this particular class.

Or I want to look I want to register something for a base class and then look up something for its subclass. It could do that. But then when I had created reg, I realized that actually this started looking very much like a generic function,

library like you have in,

in fun tools in Python, I believe, already,

like like a generic

dispatch system where you where you have a function that,

dispatches on on its arguments.

And basically, any Python method could be seen as a generic dispatch function already. When you have a method, then you call that method. Effectively, what implementation

of or maybe you call a method on an object, you just call it by its name. And what specific implementation of the method is used

is dependent on the class of the instance that you're actually calling the method on. So if you call a method on, that's called Foo on a on an instance of a class a, then it will

that method. And if you call it on b, then it will look at the method of b.

And you can sort of generalize that by saying, okay. Well, the first argument to the function, well, look at its class. And if the class is a, then we'll dispatch to this concrete function. If the class is b, we'll dispatch to another concrete function. And that's sort of single dispatch

on the first argument. And that makes it sort of possible to extend objects without changing their classes, to sort of extend classes without changing them, because you can add something that behaves like a method, but you it's just a Python function and it dispatches on its first argument. And you can generalize that, and that's what Reg,

then started doing by looking not just at the class of the first argument, but looking at the class of any of its arguments,

and that's what's called a generic function call. You can say, okay. I want to, call this function, but the concrete function that is being called depends on the classes of the of its arguments,

not just its first argument, but other arguments as well. And they have a resolution algorithm that determines which concrete reg function that you registered actually,

gets called. And then you can generalize that. I realized after a while that generic function calls were very nice, but I also wanted to, dispatch not just on the class of the argument, but also dispatch on the,

the value of an argument in some cases. And that occurs in more parts view lookup system where you say, okay. I want to now get a view for this object, and that's registered as a generic function. So if you can register a view for documenting, you can register a view for,

an animal class or whatever it might be. And then when Morpaz looks up the view, it will, look at the class of of the instance,

that it found through routing and then,

find the view that way. And with REG, you,

what you also wanna do in more path is you want to look at other aspects, like, for instance, the request method is is if this is a get request, then I want to look up a different view than if it's supposed to request. And if it's a put request, I want to do something else again. And in more path, there's also

a notion of a view names. The last segment of the path can actually identify a view by name. So maybe your view is called edit or or, delete or whatever it might be. And you you also want to dispatch on that. So then I generalized

generic dispatch into something that is called predicate dispatch, where you can basically dispatch a function not just on the class of its arguments but also on the value of its arguments and that's what's underlying more parts

now when it's exploited fully by its view, a subsystem, which is entirely based on, on on reg. And that was sort of an interesting

evolution going from, oh, just a better API and an alternative implementation to the soap component architecture to a predicate dispatch,

library, which is useful

by itself. If you have a need for such

a fairly obscure features, you can, you can download REG and you can use it

directly.

Yeah. It's interesting

how

dynamic dispatch or multiple dispatch in the Python ecosystem

isn't very popular. Part of that is because of the fact that it's not built into the core of the language. But there are a number of libraries

that

add that capability to the language, but they don't seem to be very widely used despite that fact. And 1 of my earlier experiences with learning programming was with the c plus plus language, which does support multiple dispatch and function overloading.

So I got some exposure to the power of that. And there are some cases where I'll be working on a project in Python and thinking about how being able to do that function overloading would have been useful. But in that particular instance, it wasn't necessarily

worth the extra weight of pulling in

another dependency to add that capability. But I can definitely see cases where it would be useful and beneficial to have that option. And it's interesting that there's also just sort of a lack of visibility into that capability for people who are getting started with Python, where they don't know that that is something that is done in other languages or frameworks?

Yeah. Yeah. It's quite common in some other languages and then, extending a class without actually having to modify it is quite a useful feature and without having the subclasses as well, especially in in a in a framework that that that aims to be extensible like like MobilePASS.

Reg, of course, as far as I know, doesn't have any users

outside of MobilePASS itself. But I I sometimes just split off stuff into its own library, not so much because I'm hoping it will get a huge amount of traction,

even though, of course, I'm in the back of my head. That would be nice. But because it sort of isolates the whole problem from the rest of the problem. So reg is just about doing predicate dispatch well, and,

it's just basically functions that you can register,

other functions for and say, okay. Call this 1 if my argument happens to be a subclass of this or if my argument, sorry, of an instance of a, of a class of this class or,

if it my argument has this value. And that's what it worries about. And I can sort of think about that in isolation and document it and make it work well and make it work fast and then and have other people contribute to it that way as well without having to worry about

web frameworks

for a little while. And I find that as a very useful sort of creative exercise that makes it easier to, to solve problems

in a good way. Reg is just 500 lines of code in the end.

It actually was bigger a little bit before, but we managed to shrink it down a a little bit further again. And, yeah, I like it. And if people wanna use it, then,

I'd be happy. But it's already worthwhile to to separate it into its own thing even without that.

Yeah. Definitely, it makes it much easier to reason about a project when you're able to break it into separate components. And it also encourages you to

think hard about whether or not you need to add a particular piece of coupling between 2 portions of the code base. So when you have it as a separate component, it discourages you from having that tight coupling, which makes it easier to isolate the responsibilities of the various pieces of the code,

as well as the sort of entire goal of object oriented programming and even functional programming

for that matter is code reuse.

And it's easy to get stuck in the trap of just writing it all in the same code base and ending up with that tight coupling. And you lose sight of that reuse option.

And by breaking it into

separate dependencies, even if they aren't used elsewhere,

it will at least keep you thinking about the fact of, well, I've got this piece. Maybe I can use it for this other project.

And then also thinking even within a single project, thinking hard about what are the responsibilities of this piece of code and does it need to be included here or could I extract it into something else?

Yeah. It makes APIs better if you, sort of think about them as their own thing and and and test them as their own thing and also document them as their own thing, it it just makes, it makes it,

easier to think about. I agree.

Going back to the automatic link generation

that Morpeth

is built around,

you mentioned that it helps you ensure that you don't end up with dead links in the website, but I'm wondering if there's any support for permalinks

or automatic redirects. So for when you refactor a project so that if you have links out on the internet to a particular location in the application, you, when somebody goes to visit it, they don't necessarily end up at a dead end path.

More parts of link generation system is primarily aimed to actually making it easy to do links and to reduce the, the tight coupling between pieces of code. So when you, when you make a link in, you know, the most simple way to create a link, the most naive way to create a link in a in a in a web application is to simply hard code the link,

interpolate,

a few of the, the variables, like maybe you're making a link to a user, and then you interpolate the user ID or something like that into that, link. And you just hard code it in your template or in your Python code somewhere. But that is sort of doing I'm lazy, so I don't want to actually have to, worry about what that link is when I'm actually creating the link. I just wanna create a link to user object no matter where it is, and how that link happens to be constructed. And then most web routing web frameworks then offer a way to give your route specific names. And then,

you can refer to the route name, and then you can interpolate. But you'd still need to know what variables go into that route. So you can refer to it by naming your template and say, okay. Give me a link to

the user route, but I know I also need to provide this user ID here. And then how do you and you need to extract that user ID from somewhere.

What Moab does instead is, okay. I have a user object. Just give me a link to it. And it knows itself how to get the user ID from the user. And that's sort of its primary goal. It's not so much to prevent broken links,

sort of in the long lifetime of a of a website, but to make it easy to make these links in your web application and to to provide a certain

lack of, motive, to make the more coupling between things more loose and to make it easier to write generic code as well because links often make it harder to write code reusable,

views. There are ways I mean, I do think Morpup

makes it easier to preserve links. It's still an explicit action you will need to do in your web application but you could at least, if you know you're breaking a link, if you know that the URL is is not there anymore

where it used to be there before. What you can do is, is just create a,

a dummy model, that does nothing else than just redirect whenever it's accessed or it's viewable to do nothing else and then redirect to the real place. And it would be easy to create a generic view for that where the dummy model

provides a way to get to the real model. And then the generic view that you write for for for this sort of redirect model just gets the real model first and then asks for a link to that so that you sort of don't have to write these redirect views all the time. You just implement an,

and the, the redirect models and put them on the routes, and then they will start redirecting. So that that's very easy to implement, but there's nothing in the framework itself that does that right now. Perhaps we should add add a little extension to help you do that because that's kind of a neat idea.

Another thing that I've thought about, and I haven't actually used the feature, but the feature is in there. MorePath has a way to configure a link prefix where sort of all the links that you generate in an application will have a particular

prefix, and you can also control the,

the server name in that system. And that's useful for situations where you have, a host or you have to you mount the application under sub URL or with virtual host. The hosting, it can also be useful. But it can also be useful when you have sort of if you had a complicated web application that consisted of multiple,

sub applications, you can also Mount applications in other applications in more part. You can sort of say, okay, this URL, everything

under application that's mounted here. So you can do composition, not just inheritance.

And if you live in a world where microservices

are important, then each application

could potentially

end up living on its own,

host eventually or at least on a on a different URL. And, Mobile App provides a way to evolve towards that. So you can start with a with a single application, and then you evolve it towards a bunch of applications that can Mount it into each other. And then you say, okay, well, actually you want to run this sub application as a, as a completely separate application

and, run it on in its own process. And then it's relatively easy to take that application and to, run it separately. And then in your original application, you could create a dummy application with a bunch of sort of dummy objects that when in your original application, you link to them automatically sort of linked to the application on the other host. Also in the other process. So it's a way to evolve your way towards microservices

if that's something that you consider useful. But that's just sort of more or less an idle thought I've had of how to exploit that link prefix,

feature in Moab.

I can see where that would also be useful for the case where you're versioning an API because a lot of times a restful

interface will have a a URL prefix of, you know, /apis/vone.

And then when you need to release a version 2, you can say this particular portion of the application has a URL prefix of v 2. So whenever somebody needs to use the newer version, it will automatically route over here, but we still have the previous version available at the old v 1 URL.

Yeah. Yeah. That that's also a matter of mounting. So is it the way you would do that is you would just mount version 1 and you mount version 2 separately into a into a root, and then you just develop them as separate applications. And then, yeah, you could do the redirect to another server if if you wanted to.

And another 1 of the things that I found interesting while I was looking through the documentation is that it seems that Morpeth has a different concept of what a model is than most of the other frameworks that I've dealt with in that they aren't necessarily associated with a back end database.

They can just be a plain Python object. So I'm wondering if you can explain how you arrived at that conception of models and some of the patterns that it allows for that might be different from the traditional database oriented model?

Well, first of all, of course, any routing framework could use any Python code as its models already. Right? If you are going to look up something,

you don't have to look it up in a relational database. You could just construct whatever instance you need, in whatever way you want on the fly in your view and then just represent it. Then, of course, you often don't do that in a sort of a routing to view framework like a Flask or a Django because you would just simply

start generating the view from whatever data that you, want to represent right away. So you wouldn't create that abstraction of a model first, and more pop does encourage you to do that, because you you have to to give it the URL. URL. But I think that's a good thing. I think these things do need abstracting out. So what you typically end up with is that many routes in a mobile app application do route to a model that's backed by a relational database.

But for things that are collections

of models, you have these model classes that are that are not directly backed by a relational database, but instead provide an API to let you interact with that relational database. So

it lets you do currying, the the collection,

perhaps parts of the collection if you wanna support batching.

It lets you insert new items in the database and then has an API to delete items perhaps. And then you provide views for that collection. And I think that collection abstraction is very useful to have in any application. It's just that, most web frameworks don't encourage you to create 1 because then you can give that collection permissions,

specifically.

You can also,

easily solve sort

make batching where you have like you want to display 10 items and have a next and a previous link you can solve that very easily by having a collection that represents a batch. And then if you want the link to the next batch, you simply construct the collection that represents the next batch and you link to it and the linking will, will work. So something that sometimes becomes a bit crafty or,

when you when you do batching sort of becomes very elegant, I think. So it encourages you to create these

model classes,

and I think that led leads to better code organization in the end. It leads to more maintainable code. And I think that's a good thing.

So what are some of the most interesting or unexpected uses of Morpeth that you've seen?

Well, I think the most unexpected thing is that I designed Morpeth explicitly as a framework that was good at doing REST. But it turns out that making a framework that's good at doing rest, which is basically being good at doing HTTP, also makes for a framework that works very well for just traditional dynamic web applications that, you know, generate,

HTML with server templates. So

1 of the, projects that that is very, very big that's designed that way is is is,

is a system called 1gov that's, used by Swiss

municipalities

to run their,

government websites.

And, that's that's a mostly server driven web application.

And, yeah, it turns out more more path works just fine for that as well. So that was a bit of a surprise. But in retrospect, yeah, being good at HTTP makes you being good at traditional web applications as well. That's

I mean, the other thing that that, sort of, keeps surprising me is, sort of, how 1 thing

leads to another in creativity. So, you once you,

make certain design decisions,

you know, sort of you get these follow on creative ideas and and it doesn't stop as quickly as I thought in in the beginning. So, you know, the the whole field of web frameworks in Python is sort of people tend to say they're all the same. And, of course, there are a lot of similarities

between Python web frameworks. But it turns out there's still a lot of convenience

features you can build into a web framework,

and I enjoyed the aspect of discovering that. So, the way Molbob deals with

variables in

in URLs or or URL parameters.

More pop sort of automatically converts the input, and it's surprising. So, like, you get an integer. It actually converts it to an integer for you. And doesn't leave it up to the user code to do this. And then you can also generate an HTTP error with the conversion failed automatically. And it's surprising how many web frameworks don't handle things like that. And by handling things like that in the framework, it's, you know, does less things that can the less things that the developer needs to, think about when they write web applications. Another aspect of that is HTTP errors. Many

because more path routes to a, to a model instance. If you cannot find the model instance, you just return

none from your route and then more path just generates a 4 4.

And when you look at something like Django, you have to actually use a specific API to do a database query that raises in 4 4 error if it doesn't find the, the item that you're looking for. And that just looks a lot more ugly that you solve this with the database lookup,

subsystem,

then and it does in more parts. So it can very adequately,

deal with those errors. So you don't have to deal with 44 errors in your own code anymore. It sort of falls out the way you organize your code. And those things are very easy to forget. Right? Because it works, and then now you need to start worrying about the errors. But those things are the things that you're not very inclined to do when you're developing code. So it's nice that it just works out of the box. And I discovered sort of a lot of little details with what a web framework can do for you that over time to sort of that fell out of easier

or earlier design decisions so that the whole permission system is also quite powerful because you can define permission where you can you can protect views with permissions, and then you can define permission rules where you say, okay. If I'm having a view and the view is,

for a document so if but you could define permission rules that know about models. And I think that's,

that's very powerful because you can then describe permissions,

rules for okay. For documents, I want they to have used this permission rule for another kind of model class, I want to use another permission rule and allows you to, to very flexibly write very powerful,

rules for when someone has a permission or not. And, again, that falls out of this separation of model and view that MoPub has and that MoPub has knowledge about this, and then it can do stuff with it that most web frameworks don't have because they don't have knowledge about models. So that was kind of an interesting journey of creativity, basically.

Yeah. The

authorization

capabilities of a lot of frameworks

are frequently somewhat lacking or difficult to apply.

And a lot of times

the sometimes the only way to apply the authorization capabilities is actually through an extension. So having that built into sort of the core of more path through the way that you interface with the models is definitely something that's fairly interesting and seems like it would make that problem a little easier to solve.

Yeah. And it's not a lot of coding more power to allow you to do that. It's it's surprising also, I think. But 1 1 thing that surprised me is how much power that I was used to from soap could be written down in just a few 1, 000 lines of code in the end. That that's, I guess, another out surprising outcome over this whole,

journey that really you don't need giant piles of code in order to, enable some very powerful features by using some very powerful concepts

underneath, like configuration and predicate dispatch.

So what do you have planned for the future of Morpath?

Well, 1 thing that we've been thinking about a long time is a 1 release, because more parts are stable has been stable for some years now. I mean, we've made some large internal changes, but it's not like it it wasn't already solid,

2 years ago.

But we need to do that 1 other release.

We hope to also create a

non documentation website, a little bit of a marketing website

along with that. So we'll see whether we, we manage to do that in the in the next year. 1 other thing we've been looking into recently is integration with more relational databases.

We have integration through the,

transaction module and the more transaction extension with the Speed SQL Alchemy, and that also gets you integration with the c 2 b. So that's transaction.

The transaction module is a general module that abstracts sort of the notion of a database transaction, and we integrate with that. But we're also looking into, Pony ORM because that looks like a nice ORM.

See whether we can provide some integration for that. And recently, I've been working on a tool while I completed a tool that's called,

a more dot, path info. I think it's called more

more dot path tool actually. And that provides a command line tool that gives you sort of a complete overview of what routes and exist in your system and what views exist in your system and sort of analyzes that because

1 of the aspects of of separating views from models is that sometimes it's a bit harder to reason about what's what's there, but this gives you basically just a CSV file that contains some information. And, again, that's possible because we have an abstract notion of that we have a concept of configuration so we can access the configuration database specifically and just, query that instead of a whole bunch of ad hoc APIs. And I've been working on a tool that sort of,

you can access you can do it use on the command line and then just tells you more about why you got the result that you see on your screen, like what model was used to, to render this response and what view is used.

That just gives you a little bit of debugging capability.

And

yeah. Then otherwise, we'll just have to to see. We've been working a little bit more on the internals of the reg library. It's become a little bit faster still.

That will be in a in a future release. Otherwise, I don't have any big plans. It also depends on what other people bring to the table. And, but I'm sure new ideas will arise eventually because they always do.

Are there any other topics that you think we should cover before we start to close out the show?

No. I think that's a reasonable overview. I mean, I I like talking about sort of the, I've blogged about it, the history of my experience with with ZOAP. So so I can say a lot about my experience with with Zope. So, so I can say a lot about that. I enjoyed your podcast about, and we need to interview with Eric Steele about Plone very much because I, even though I never actually worked on Plone, I worked very close to Plone for for years. So I have a bunch of blogs,

that I called my exit from ZOAP that sort of describe the history of ZOAP and my experiences with it. That might be interesting to people who, are curious, like, how frameworks not just arise, but also how they evolve and then perhaps die in the end. Because SOPE is sort of undergoing this protracted death for some, some years now, even though Plone is still very much alive. I was gonna say, that definitely sounds like an interesting topic, so I'll be sure to add some links to that in the show notes for anybody who wants to check it out.

Yeah. And then another sort of general topic that interests me is sort of what it means to be creative as a software developer because it's something that I can't help doing,

creating new things because I like just doing it. But I, I thought quite a bit about what works well when you're creating a library or a framework, how the process of this creativity works. Then, for instance, with, with the routing system, I wanted the routing system with the benefits of a traversal system. And then you get this sort of dialectic where you want to do both

at the same time. And that often

results in creative new solution and trying to unify 2 different sort of

things with each other. Another experience I had that somewhat like that is years ago, I created the LXML library that's quite popular in Python land for dealing

with with XML and XML,

with XPath and XLT, etcetera. And

when I created it, I sort of had these 2 poles. 1 is was this lippexmal 2 library that's very c and very low level and quite tricky to use, and you have to deal with,

manual memory management, etcetera. And on the other,

and I had element tree, which Frederick Land had created, already,

which had a nice Pythonic API for dealing with XML. And

so my contribution

after spending a few years messing around creating all kinds of half baked XML libraries for Python, they're never gonna need traction. Well, so the minimal contribution I could make is to say, okay. I take the element 3 API,

and I add it on top of libXml2,

and that's what I do. I don't invent new APIs where I don't have to. I don't implement XML pro centric libraries where I don't have to. I just unify them. And and that was my most successful open source project, I think, especially because, Stefan Benel,

took over the project after a year and has maintained it very capably,

since then and extended its capabilities quite a lot. But I I sort of realized, like, hey, I've created something new here by doing the minimum act of creation

a pause possible by just connecting 2 things together that are already there. And I find that that that's an interesting,

way to think about creativity and software as well. So that's sort of a topic that interests me, and, I like, giving talks about it once every while for Python conferences.

Alright. For anybody who wants to

follow you or keep in touch, what would be the best way for them to do that?

Well, I have a blog,

at blog.starteffect.com.

And,

I also have a Twitter account and I tweet fairly often. It's called Fassen. It's my last name, which you can probably find in this show notes.

And it's,

I I also have a GitHub account that has that name as well. So those would be, good ways to,

see what I'm up to. And, yeah, feel free to drop me an email as well, of course. Great. With that, I will move us into the picks. And my picks today are first, I'm gonna choose the Internet movie database,

the IMDB,

because it's a good way to see what some of the upcoming movies are and look up information and trivia about the various movies.

I've been using that as my sort of discovery mechanism since I don't really watch TV or see any of the commercials for upcoming movies. So every now and then, I'll browse through and see what it has in the sort of top scroll to see if there are any movies that I find interesting. And I also use that to keep track of the movies that I want to watch. So it's a good way to do that.

I always use it when I'm watching something and I recognize the actor or the actress. And it's like, who is this? I have no idea. I've seen them in something else. And then you go for the I mean, IMDB for that particular episode or whatever it is. And then you, look through the cast list and then you can, click on them and see, what other things they appeared in, which sort of solves 1 of these,

not very important problems that you have when you're watching something. It's like, I recognize that guy. Who is it? Yeah. I I definitely use it for that all the time as well. It's and and then my other pick today is just gonna be gyroscopes

because they're a fun little toy to play with, and it's a good way to experiment with the

weird physics that is angular momentum. So, with that, I will pass it to you. Do you have any picks for us today, Martin?

So I have a podcast,

pick,

that's has nothing to do with Python whatsoever, but it's about, it's by 2 game designers. It's quite well known. It's called, Ken and Robin Talk About Stuff.

They're both, role playing game designers.

And in this podcast, they talk about stuff, which doesn't just talk about they don't just talk about role playing games, but anything else that interest them, which

includes history and the occult and conspiracies and food and lots of other things, narrative,

principles of narrative and things like that. And, yeah, I find it's a very interesting geeky. Make sure to listen. And my other pick is

Vili, which is a Finnish yogurt,

that I'm making for myself these days. So I got into the fermentation hobby,

some years ago where I'm fermenting

weird stuff, food, and various strange drinks. And,

I've been enjoying Vily lately. Vily is this Finnish yogurt, which is is very easy to ferment,

because it's mesophilic, meaning it will do its job at room temperature. So you just take some

vili and you put it in milk and you wait for a day or maybe 2 days and then you have batch of new vili and then you take some vili from that and you put it in milk and then you get more yogurt. And has this very interesting consistency.

It's sort of

I would say,

it

part of what it it's a very mild yogurt because proteins I believe that are generated by the bacteria that create the yogurt also create this weird

ropy long proteins that give it sort of a,

very

slimy content can consistency

in in, it behaves very strange when you scoop it on a spoon. It's sort of a lot of it comes with it in,

threads below it. And there's a story that somebody was making ville in an open pot and, put too much milk in the pot. And then the next day, the pot was empty and all the ville was sitting next to it because it had pulled itself out

in the process of forming.

But it actually I mean, it's the mouth feel is not as disgusting as I make it sound now. It's actually very nice to eat. It's a very nice mild yogurt and it's very easy to make and it's kind of fascinating how you can just put a bit of yogurt into some other yogurt I mean, into some milk and then do nothing. And then you have more yogurt the next day. That's magical. I like that. It's very interesting. I'll have to look into that a bit.

Yeah. You spell that V and then double I l I. So it has a finished double I in there. Well, I really appreciate you taking the time to join me today and talk about more path. It's definitely an interesting framework with a lot of interesting concepts in it. So I'm sure I'll be looking more into it and maybe using it in a future project.

Thank you. Yeah, I enjoyed this, too. And thank you for the interview, as well.

The Python Podcast.init

Summary

Brief Introduction

Interview with Martijn Faassen

Keep In Touch

Picks

Links

The Python Podcast.__init__