Django, Channels, And The Asynchronous Web with Andrew Godwin

Hello, and welcome to podcast dot in it, the podcast about Python and the people who make it great.

When you're ready to launch your next app, you'll need somewhere to deploy it, so check out Linode. With private networking, shared block storage, node balancers, and a 200 gigabit network, all controlled by a brand new API, you've got everything you need to scale.

Go to podcastinit.com/linode

to get a $20 credit and launch a new server in under a minute.

And visit the site at podcastinit.com

to subscribe to the show, sign up for the newsletter, and read the show notes. Your host as usual is Tobias Macy. And today, I'm interviewing Andrew Godwin about Django Channels 2 and the ASCII specification for modern asynchronous web protocols. So, Andrew, could you start by introducing yourself?

Hello. Yes. I'm Andrew Godwin. I am a, shall we say, Django developer at large.

You may know my work on things like South for Django or Django migrations in the past and, as Tobias just said, on channels these days too.

And do you remember how you first got introduced to Python?

I

sort of do. Yes. It was back in sort of the, like, mid 2000, I think it was. And I was coming from PHP and was thinking, well,

PHP is great. I'd written a few things. I've released an open source library,

but I was like, I really want to

change language. Right? It's like I actually got my first,

vulnerability report with my my open source PHP project. I was like, ah, well, you know, at the time, it was, like, PHP is by nature unsafe, and, like, PHP 45 weren't the best. It's much better these days. And so I sort of looked around and I tried a few things. I tried a very early version of,

a few different things, but weirdly, TurboGears, 1 of the older Python web frameworks, was my first introduction to Python.

So I found TurboGears, and I was like, oh, this is great. It's sort of Python is like a good language. I sat down and learned it that way.

It was sort of solidified on my I think it was like the next year

or maybe later that year, I sat down with,

Simon Winterson. I went to work at the company he was working at the time,

and he very excitedly ran over. I was like, Andrew, you have to learn this Django thing. It's very exciting.

And so, his enthusiasm sort of sealed me into the Python web ecosystem and taught me, a lot of the Django I know today.

Yeah. It's not often that you hear turbo gears mentioned. And I remember when I was first getting into Python, I came across that as well and was very interested in it. And I never actually built anything with it, but it's something that I always periodically go back to and think about and wonder why it doesn't get as much attention these days as some of the other frameworks that everyone's talking about. Yeah. And it was a fascinating thing too. Right? Like, they have this very clear vision of, like,

there are 4 separate components, and then TurboClears is the glue between them.

I think maybe that you know, like, the templating framework was very XML based, which probably didn't work in their favor. But it was a it was a really nice project. And, like, I think 1 of the things it had I liked was it had good documentation I seem to remember as well all the time. Like, it's a great tutorial I got going really quickly.

And so now that you've been involved in the Django world for a while,

there are

a number of cases where people are looking for support for different web protocols

that don't fit nicely inside of the WSGI protocol. So can you just start with a brief overview of the problem that channels was set out to solve? No. Definitely. So it starts again, like, like, 5 or 6 years ago, like, when WebSockets,

is first being sort of thrashed out. The WebSocket specification

was

this sort of very developing protocol at the time. There's, like, 3 or 4 major revisions.

There were a few years there where

if you didn't have the most recent version of Chrome, it would be the wrong version. It wouldn't interoperate properly. I started off sort of looking into the protocol and working,

with, at the time hang on. Let me just think of the right thing here. It wasn't gevent. It was the other thing. Events. Yeah. So at the time, I was working with EventLit. So I took EventLit, which had an old copy of the protocol that didn't work anymore, and I sort of took that, upgraded it to the new version of the protocol. I was a little bit fascinated at the time by that. That sort of sparked my initial interest. And then later on, sort of what kind of background Django and this stuff too. So, Aymeric Augustin's work on

web sockets with Django, of course, was around this time. He did a few sort of interesting demos of that stuff.

So I sat down. I I looked back at the problem. I was like, okay. We have this protocol.

It's so unlike HTTP. There's no request response flow at all.

How could we possibly handle this in Django

without, like, resorting to hacks about like, oh, it's like fake web requests or it comes through as posts and gets or there's some kind of weird, like, long polling thing you do. And so I sat down, and I sort of sketched out the first version of what became channels 1,

back then

and sort of built it out. And this is very much sort of filling the hole that I had after migrations. And then so I did the migration support in Django, had about a year off just burnout, basically, from that project, and then this is sort of what filled that gap. And over time, I sort of made channels, played around with it. People seemed pretty interested as doing it. So I sort of built that up and developed it into the first version of channels as it was. And kind of from the beginning, it wasn't just about WebSockets. It was a sort of direct focus on

how can we have support for more than just HTTP.

Like, even back then, WebRTC was coming up as a protocol too.

And as the time goes on, I suspect we're gonna have more and more different protocols that the web is built upon. I think just today, there's announcement from, like,

I think it was Cloudflare. Like, oh, we've now support IPFS, which is sort of this weird

decentralized interplanetary file system kind of web thing. So, like, there's a lot of different protocols. The web continues to evolve.

It continues to be more, like, JavaScript based and more interactive, and, like, that, to me, was the driving force behind, like, making something like channels. And do you think that if we didn't have a strong story for native

asynchronous support in Django,

that it would eventually start to fade into the background and be replaced by some of the other asynchronous frameworks that have existed in Python for a while and with new ones that are starting to come in and be natively asynchronous

on top of the new built in async support with Python? I I was never quite sure about that 1. Right? Like, the first version of channels was

entirely separate. It ran on Python 2, so it had to basically work on a synchronous version of Python. And it's 1 of those things where I I was never quite sure

if

something would come to solve it.

I think

someone always steps in to help with stuff. And in this case, that ended up being me and a few other people for this 1. But I do think something would have happened eventually.

The question is if it would have been involving Django or a brand new thing. Like, we've had technology around for a long time. Like, Twist has been around forever doing asynchronous stuff on Python. But, also, you know, even in the last, like, 3 or 4 years, there's been

Tornado and other solutions like that and building on top of different c bindings. Like, there's not so many different approaches that

something would've come through eventually, but I think

and this is leading into channels too. Like, the idea of a core central Python async framework is really useful from a here is the

sort of guiding light everyone can go towards. Like, trying to have some kind of single way of solving this stuff. And I think too that with how mature and long running the Django project is and how many different

projects have been built on top of it, that by

having it eventually

fade out, it would introduce a lot of extra rework by people who have already invested so much time and effort into these existing applications that having a way to,

give them a migration path into this new world of asynchronous protocols while still supporting

the applications that they have been maintaining for all this time provides a lot of value to the community

and to all of the different businesses and people who have been building and maintaining these things. Yeah. Like, 1 of the things I've learned, like, as I've sort of done more and more open source maintenance is, like, the easy thing is not adding new features or making new projects. The hard thing is not adding new features or making new projects. The hard thing is, in fact,

trying to get people

from what they're on now to that new thing without a rewrite. Like, Django's had

over a decade of success in that arena, like, pretty much always backwards compatible for a few releases. There's a nice upgrade path. We try and do well by users as much as we can. And it's very easy to just, like, I could have gone away and done a whole separate framework from scratch and ignored all the learnings of the past. Maybe tried to make it a spiritual version of Django too. But I really wanted to try and integrate it. Like, take what's good about Django and add to it on top of that stuff as well. And that was the almost the reason why, like, why I kinda made more work for myself in that way, but, like, the end result is also very useful. Yeah. I know that I've been hearing about channels for a while, but it never really seemed to hit that critical mass of everybody's talking about it partly because of the fact that it is this sort of bolt on application that requires the an extra process with the,

Daphne server

and the Redis server for being able to manage the communications between the different processes.

And,

we're getting a bit deep into the weeds, about the differences between the 1 dot x and 2 dot x release, so I'll I'll leave it there for now. But for somebody who is familiar with

working in the Django web web framework as it is today,

and they have an either existing application or they're starting a new 1. What are the steps that are involved in integrating channels

back in the 1 dot x release and now with 2 dot x? Also, the key thing with 1 dot x, as you just mentioned, is that it required an extra component. So you replaced your web server, something like, say, like, gunicorn with Daphne, which is the WebSocket's sort of ASGI version 1 enabled web server. And then you also needed a way to transport stuff between

web server and Django. So channels 1, because it ran on an old version of Python, it couldn't be natively async inside the same process. So Daphne was running on Twisted. It was natively async in that sense, but Django wasn't. And so we ran them in separate processes, and Redis was our sort of bus between the 2 of them. That caused all sorts of issues,

not only with scaling that particular problem, but, like, just reliability and, like, adding an extra sort of moving piece to your deployment stack was not necessarily the best thing to do. In channels 2, it's a lot simpler.

Everything now runs in the 1 server, in the 1 process, like it always has done with Python. So you sort of just swap out your existing Python web server

for Daphne or for you for calling or 1 of the other

several, ASIO website as we now have. And so

as you mentioned, the transition from the 1 dot x to the 2 dot x version of channels was a significant amount of rework and redesign. So I'm wondering if you can discuss how the current system is architected

and how that differs from a standard Django web application that somebody might already be running today. Yeah. So, like, 1 of the key things with channels 2

was to

take some of the lessons from channels 1 and a lot of the basic design structures, but also try and incorporate, like, all the stuff that Python 3 had bought, like native async and that kind of thing. So the key thing is channels 2 is, by default,

noninvasive. Like, you can install it as a dependency. You can switch everything running on Daphne or another ASCII server,

and it'll just run as expected. All the defaults are gonna work as you expect. And then what you do is in places you want to have asynchronous support, so where you want WebSockets or long polling or another protocol,

you can then go into those places and write a consumer, which is sort of the asynchronous channels version of a view and say, okay. So this particular part of the site, this WebSocket thing, this has asynchronous code. This is gonna do polling. It's gonna do requests, whatever you want it to do, but you still write your views as well. So the idea is you wanna keep that traditional

normal Django world that everyone understands and is easy to debug and all that kind of stuff in 1 place.

And then the asynchronous stuff is not quite separate, but it's, you know, got a nice outline around it. Like, here is this asynchronous code.

And And because asynchronous code is harder to write and to debug, it helps keep that surface area smaller. There's no sort of trudging through loads and loads of weird asynchronous versions of bits of Django and, like, no need to rewrite all your views in an asynchronous fashion if you don't need to. A big part of my philosophy is

asynchronous code is harder to write and it's not always necessary.

Synchronous code is often just what you need and runs perfectly fine. So giving you the free option to choose between the 2 of them is kind of what I'm trying to aim for there. And when you're writing an application that's using channels, do you have any general guidelines

as to when you want something to be asynchronous versus synchronous? Or do you just start with the synchronous version and then do some profiling to determine that this is IO blocked and needs some additional parallelism

to increase the performance?

So it depends. There's 2 different answers to that question. The first 1 is, like, if it's a feature that absolutely requires, like, a WebSocket, like, oh, we're doing a live chat, then the answer is very clear up front. Like, the chat protocol has to be a WebSocket at that point. There are some other things too where it's like, oh, like,

we could change this to a a long poll rather than sort of a a short poll, or, oh, we found out we could do maybe some database requests here in parallel to improve it. But in general, if it's not super obvious, I would encourage building it synchronously first. And as you say, then doing some profiling or trying to just work out what your

sore spots are. Because sometimes you're not gonna get it right. Like, asynchronous code is strange.

It might seem obvious that you can just

take these queries and run them parallel, but it may not be much better. You may be wasting a lot of you may be wasting a lot of development time for not much gain. So in general, building synchronous first, building short polling first,

and then trying to understand

where your state is and what your usage patterns are and that kind of stuff, I think, is it is how I encourage it, unless you are already experienced with acing stuff and know, okay, I can jump in here and do this the right way. And in terms of synchronous versus asynchronous, when I was looking through the documentation,

there were a number of mentions of the Django ORM and requiring that bridge between the synchronous land and asynchronous land because of the synchronous nature of those database calls.

And I'm wondering

if there

are any limitations that that imposes by not having the ORM layer natively asynchronous

or if it doesn't really seem to

give a lot of benefit by trying to

enable asynchronicity in the database, and it's just, too much additional overhead

for what it would gain you. So there's a lot to be gained from syncing the database. Like, it's maybe 1 of the biggest areas most sites could optimize on. The problem is right now, it's still kind of a young field in Python,

and that means that the tooling is not there as necessary to support it. You know, things like maybe like prefetch related in Django, we can maybe optimize those you run in sort of a parallel fashion if there's more than 1 of them. But the problem is, ultimately,

like,

Python now has asynchronous database bindings for most

big databases,

but Django doesn't. And we have to sort of first take that layer and have an asynchronous version of Django's,

things like the query planner and the thing that makes all the model instances and all the sort of internal caches and connection handling. There's a lot of stuff inside there. In some ways,

the ORM is by far the biggest and most complex part of Django.

But I do think there's a lot to gain there. And, like, 1 of the things I'm we ought to focus on maybe the next year or so is

how do we

really make use of that potential? Like, how do we

take

asynchronous database access and not just make it possible, but make it easy? Like, how do we make good access patterns and API design

that encourages

safe database queries, but also making them fast?

Things like, you know,

if you naively just do every query in parallel,

you're gonna end up reading stuff before you've written it. Right? Like, there's a lot of things that you assume that come with synchronous building. You really want to sit back and think differently about asynchronous stuff. And, like, Django's job has always been to give you a nice, safe way to do stuff. So how do we get there? How do we make it so you can do things

in parallel easily and safely without it requiring a whole load of, like,

new reeducation on your part? And as we're talking about the asynchronous versus synchronous nature and some of the difficulties in being able to reason about what's happening in the

asynchronous aspects of the application.

I'm wondering what you have found to be some of the biggest stumbling blocks or points of confusion both for your own work on building these asynchronous applications and working with these asynchronous protocols and for other people who have been following in your footsteps and building their own applications on top of the newer versions of channels and the ASGI spec?

So, like, 1 of the biggest problems I certainly run into a lot myself is

just debugging this stuff. Like, it's so hard to understand

everything that's happening at once. Because often you have, like, maybe 10, 20 coroutines happening all at the same time. Putting a debugger into 1 of them is more difficult than it maybe could be.

And

also, there's some weird side effects. A lot of the time you'll get bugs where

things run

slightly too close and run over each other, and as soon as you put a print statement in, you put just enough delay and they rerun separately,

and the bug's gonna goes away. And so there's really sort of a almost quantum level of, like, if you look too closely and put too much debugging in, it just fixes itself,

which is really annoying to deal with until you realize what's happening. There's also other problems with, like, Python's async framework is still a little bit,

sort of patchy in places. There are problems like, no, Oh, it turns out you weren't yielding properly. And there's there's solutions to this in asyncio and and core Python, but there's still a few places.

3.7

helps a lot with a lot of those, and There's still some more we could do as a Python community to patch up async in Cayo and make it a little less, you know, rough around some edges.

And in general, just using channels and debugging that way, a lot of people run into problems

around understanding

how things are flowing through the system. Like, the trace back looked entirely different. Right? Like, there's all this channel stuff in the middle. Middleware works a bit differently too.

There's a bug we only fixed it, like, I think, last month where, for a long time, middleware is being

created every single request in channels,

and it wasn't being an item like this in Django. It's people like, oh, it's weird. My pages all work, but they're really slow. And it turns out they're doing a lot of work in their middleware's init method. And once we fixed it, that went away. But it's all sort of these slight differences where, like,

we are close enough to Django, but almost the uncanny valley of differences starts cropping up. Like, oh, this looks a bit differently or these are asynchronous APIs. And, like, especially if you're new to Python async,

understanding the intricacies of how to call async stuff can be a bit surprising. We try to help some of that in in channels in Django, but there's still some rough edges that, know, we just can't help with,

like, you just have to await things. If you don't await something, it's not gonna happen, except it might happen sometimes because, of course, that's how good bugs happen.

It never works except when it does.

Yeah. It's the the best bugs are ones where you're like, well, if I don't await, it works like 30% of the time, there's enough left in the scheduler to actually run the thing anyway. But the rest of the time, it doesn't get there. And it's just like, oh, it's the worst kind of heisenbug, basically.

Async programming is sort of a lot of those kind of, like, very unpredictable,

very, like, nonconsistent bugs. And debugging is mostly me just, like, running a thing 3 times, trying to get it roughly consistent to see what's happening.

Yeah. And as you were discussing the ORM layer, 1 of the other things I was thinking about of paralyzing things and not putting enough consideration into it is the risk of running out of database connections and some of the fun debugging sessions that leads to as well. Oh, totally. Like, we've already had problems with that. People just try and do that themselves. Right? Like, you just get way more connections when you're running async. Like, you can run, like, hundreds of things in parallel in 1 process.

And at some point, you start hitting, like, these core limits on database software or, like, you hit, like, file socket limits as well. We've had that crop up recently. It's a standard problem for most web stuff, but it happened much quicker than I expected. So, yeah, like, all these things, like, you you never really think about them normally with normal Python, but all the things come out of the woodwork to try and find you when you're running async. All things you never quite thought about too much.

Yeah. Everything is easy until you scale, and then weird stuff starts to happen, like network saturation or TCP collisions or any number of other fun things along those lines.

Exactly.

And

as we were discussing

the

asynchronous

interface

and some of the and some of the confusing aspects of building an application in that fashion,

what were some of the design considerations

that you worked through as you were building channels, both 1.xand2.x,

and some of the lessons that you learned from that initial implementation

as far as how to structure the interfaces that you expose and the internals of the application

to help reduce some of that confusion and give people,

launching off point to be able to get something up and running quickly? Well, so the main part of that is, like, safe defaults. Right? It's the very it's the very Jangief philosophy of the thing we give you the thing we give you by default shouldn't be too dangerous.

A big 1 of those is deadlock prevention.

The design of channels 1 and channels 2 is both

pretty specifically designed, so it's very hard to deadlock yourself.

Consumers can't wait on

a message for another consumer. Like, the easiest deadlock is 2 things both wait for each other and then just get deadlocked. So, like, there's a little bit of design there like, oh, you have to build everything as, like, it receives a message and then your code gets called. So everything's sort of reactive. It helps design quite a bit. There's some things you just can't solve though. Like, we just can't solve. You might run a synchronous function, and it blocks the whole process. That's a weakness of both this asynchronous implementation by itself. We can provide easy ways around it. So it's sort of

like, oh, here's a very easy way to stop this happening if you manage to get here and read the docs. And, like, we, you know, we press it in the tutorial, in the quick start guide, in the code examples,

like, hey, if you wanna call synchronous code, use this sync to async wrapper. It does most of the safety for you. But there are some cases we just can't stop that stuff. Other cases we try to make it safe are things like

especially in channels 2 being more familiar. So channels 2 uses

uses standard Django URL routing, whereas Django so whereas channels 1 didn't do that. And so it's trying to make it it's like if it's familiar to like if it's like Django,

then it's gonna be

what you expect and what as you expect. And if it's different, we try to make it obviously different. Like, trying to make that API feel a bit more dangerous

or feel a bit more unusual

so you might get an idea of how it's shaped or might have an idea to go away and look up how it works. And then try and reinforce that through, like, just examples and documentation.

And in the process of building channels

and the upgrade to channels too,

you have developed

a new protocol for asynchronous web applications

that in some ways mirrors the

WSGI protocol that we've had for a long time, and

we've mentioned it periodically

as being ASGI. So first, can you break that out as to what that stands for

and how it differs from the WSGI standard and the overall process that you went through for being able to develop the specification?

Totally. So, I mean, the joke is ASCII is just busy with an a in it. Right? So the the idea is ASCII, ASGI, stands for asynchronous server gateway interface.

In many ways, it's just literally, like it's meant to be a spiritual successor to WSGI, to whiskey. Right? And so having a similar name is a very obvious homage to that and meant to signal, hey. This is meant to be like that thing. There was a sort of previous thing that was called ASCII, ASCII 1 with channels 1. That wasn't really a specification as much like, oh, here's kinda how channels works. With channels 2, what I really try to do is take a long, hard look at WSGI

and what it did right and try and replicate that for the modern world. Right? Like,

there's been lots of discussions over the years about, oh, how can we make a new whiskey? How do we make, you know, like, the next version of it? There's been discussion on, like, the web special interest groups over the years. And it's always kinda petered out because

WSGI in some ways is such a strict thing. It's hard to take a thing and call it WSGI while making it asynchronous.

So my hope was to do like a half step. Here is a thing that is asynchronous. It's a little bit different to WSGI on purpose, but it has the same kind of through line of it's quite simplistic. You can write applications as a single object. It's meant to be easy to understand and have a basic sort of API that if you want, you can just code against raw in Python. That's only for a complex framework or anything like that. So that's kind of the, really, the the driving force behind having

ASCII. And, of course, like, the idea of whiskey was, you know, when it was invented, there were

web frameworks like TurboGears

that only ran on web server. Like, you had to use cherry pie. And so the idea was, like, Light Whiskey tried to make things more open,

gave you a choice of frameworks and a choice of servers that you mix and match. I want to have bring that forward to the asynchronous web world too. Like, I want to have that from the get go almost. Like, hey. You can develop here in Django. You can develop in Flask. You can develop in anything else. You can run on any of the other servers, and they all interoperate in the right way. And, like, having that standard,

giving people the option not just to use existing stuff but also to really open it up. So if you want to write a new web server, you can go for it. You're not limited to trying to convince a framework to come and say, hey, framework. Can you really add support for my thing? Like, no. You have this specification. It's quite loose. It gives you a lot of freedom on the implementation side, and it really lets you explore that stuff. And hopefully, it will really encourage more innovation with Python Frameworks and Python Servers as well. And

a little while ago,

at least a year ago, maybe even 2 at this point, you joined Corey Benfield on this podcast to discuss the early days of channels and also

the

then proposal of the WSGI 2 that was being discussed. And I'm curious if

ASGI in some ways has become that WSGI 2 or if there are still efforts being made to try and upgrade WSGI itself to support some of the

newer protocols such as h t HTTP 2 and things like that. If there has been pros on WSGI 2 recently, I certainly haven't heard about it. I think a lot of the discussion died about the time of that that old podcast.

In many ways, like, looking at the way things work in Python especially,

I was pretty convinced the best way to approach this wasn't to just discuss in a vacuum. It was to go away, implement it, get multiple implementations,

really

hash out that problem solving part of the process

with actual software and then come back with a real living specification and go, hey. We've done this. We've implemented it. It runs websites now. We know what the holes are. We understand this much better. That's kinda how whiskey works in a way, and that's kinda what I'm trying to do with with ASCII as well. I think at this point, it seems, certainly,

from my biased viewpoint,

ASCII is being seen more and more as a Wizzie successor.

I was up being asked just this morning on Twitter if there's gonna be a pep and when there's gonna be a pep about it. And, like, I have to really look into that and start start down that process as well, but that that is my intention at this point. Like, I am

now satisfied that

the thing I dreamt I sort of dreamt up as as ASCII 2 and has been refined with the helps in so many friendly people over the last 2 years

is ready to take on that mantle. Right? It seems stable.

Everyone I've talked to pretty much

mostly likes it, which for protocol, it's as good as good as you'll get. Like, no 1 ever likes everything perfectly.

As long as everyone's like, oh, it's fine. I'm I'm cool with that. That's that's as far as I can know, a success.

And so the next step really is, like, you know Yeah. Where do we go from here? Like, do

we, probably, you know, like, start the pet process, like, start that process of making it more official, like, as

more and more frameworks are

just

taking up the mantle, like,

we had a sort of small discussion at PyCon in the US with a few sort of server and framework authors about this, but I think the

Python 3 train continues apace.

And behind that, it pulls the async train and and thus the ASCII train too. So I I really think, like, the time is definitely

fast approaching for much, much more adoption of that, across the spectrum. So as he is the caboose on the async train? I I guess if analogy continues, yes. Exactly.

And have you been seeing a lot of buy in and other uses or implementations of ASCII with other frameworks beyond the work that you've done with channels too? I have. I mean, the most, obvious and impressive 1 is a lot of Tom Christie's work around Jangarith framework and Starlet and all his stuff. He's honestly done a lot of the heavy lifting with this stuff and really, like, done that, really important second full implementation of everything. Like, there's Uvicor, which is a standalone separate server.

He's got basically 2 different frameworks as well, on top of it. And, like, one's minimalist, one's quite much more fully featured.

He's been, like, giving me all this wonderful feedback on the tracker and also people he works with and other people around it. Like, that really is a great,

sort of thing I've been looking at. Like, oh, it really does work somewhere else. And, like,

the idea of having this middle where we can truly share across frameworks does actually seem to work in the end too. So that that's been particularly useful. I could give, him and the people who work on, the projects he he leads a lot of credit as well. And do you have any particular

hopes

for what the community

is going to do with ASCII or any updates or revisions

to the standard as it stands right now that you want to see implemented before ASGI gets cemented in the form of a PEP? I think at this point, it's mature enough that the PEP process would in itself thrash out the final details. There's a few rough edges that, you know, I'm always perpetually unsure about. Like, 1 of the things I can speak to as an open source maintainer is,

it has been about a decade of being unsure about everything.

So at some point, that's kinda come to the territory.

But, like, there there's, like, you know, the PEP process and all things surrounding it and also

other people implementing that specification

is that final thing we need to really seal it up and make it good. And 1 of the things is, like, we've made like, there is a version field in the ASCII spec. Right? We we made sure that, like, we don't wanna get stuck in a corner again and make it unacceptable.

The idea is to make it a bit more living than whisky was,

to adapt more to different flow of stuff. We we've separated out, for example,

the basic structure of an application, but how you sort of build a thing that receives events and sends stuff back from the protocols. Like, there's a HTTP spec. There's a WebSocket spec.

There's some sort of in development specs for other things too. And the idea is if something comes along that's new, we can hopefully make a protocol spec for it separate from changing the fundamental design principles of ASCII. And that's kind of part of that too. And do you have any particular plans for the future of channels itself? I do. So I posted a blog post, I think, at this point, quite a few months ago, about, like, what's next for Django and Channels. Right? And I think a big part of that is eventually

taking

things we've learned from Channels and putting them into Django.

I never want Django to do WebSockets. I think channels' future

is to become the WebSocket solution for Django,

But I think Django itself should do asynchronous HTTP.

Like, Django should gain

asynchronous web middleware, asynchronous ORMs, do long poly natively, all that stuff that Django is sort of more familiar with. And that's really the channel the challenge here is to work out

what that work is and how to tackle it. Like, there there's already sort of big buy in from the community. I posted the sort of rough plan online. There's, like, on the channel developers list. A lot of positive feedback on that. People are just so very keen for this to happen.

The next big step is to sit down and work out, like, how are we gonna run this thing? It's gonna be, like, 1 of the biggest changes to Django, probably the last 5 years at least. So, like, you know, how do we do such a big change? How do we make sense? Like, we don't have this sort of branch that never merged. It's often a big problem. Right? Like, a couple of products in Django's past have done amazing work on a branch,

and then the branch has never quite landed. Right? Like, for a long time, we've had a composite primary key branch. It was like, oh, you can have composite primaries foreign keys. But it's all it was always a little bit, like, not mergeable or something and, like, it just drifted away over time, so that was kinda sad. I think this is like, try to do that here and also, at the same time, control, like, you know, my particular commitment to this. Right? Like, I don't want

to over kick my over commit myself or become a single point of failure.

I'd love to try and bring on some,

like, new big Django contributors from this too. Like, this is

1 of the

best things to bring on, like, new people is to have this big new thing you can really sink your teeth into.

Django has been

not quite, but, like, there's been a lot of, like, you know, difficult,

like, security bug fixes for a long time and not a lot of brand new features. So this is, like, almost a nice piece of fresh ground for us to use for that too. So it's a lot of different facets to what it becomes, and it's also fluid. Right? Like, every DjangoCon or PyCon I go to, I try and talk to people. I try and get the feeling of, like, the room or the conference and try and understand who's where, and it sort of develops over time. But I think at this this point, the goal is to take channels, make it like sort of the WebSocket handler,

move some acing stuff into Django proper,

and then maybe think like Daphne. Maybe like if Daphne gets replaced by some other good web servers, just retire it. Like, if it's being sort of the same job as being done by other people, just say, hey, like,

this serve its purpose. It was a good reference server,

and we don't have the people to maintain it, but there's other ones you can move to. So, like, really, like, you know, I I'm not afraid to

be very frank with myself about

what has to continue, what does have to continue to. Like, what where is this what is best for Python as a whole as well as for Django?

And what have been some of the most interesting or unexpected projects that you've seen built on top of channels? Oh, this is fun. Every PyCon I go to, someone comes up with a new amazing thing I I've heard about.

So in there's obviously there's a slew of, things like chatbots or real time stuff. There's at least 1 person who has, like, an RPG they're making with it, like an online web game. That was pretty fun. I like that 1. They had some fun questions.

A lot of people doing, like, Uber for x. That's way too common, in fact.

Like like, every other week, I get a thing like, oh, I'm trying to, like,

plot stuff on a map and find what's what what's closest to somebody. I'm like, you're making Uber for x on Mhmm. So that's very common. But also the there's some fun things like, like, I I like the games the most. Like, you know, I I came to programming through games originally. That was my always my passion, like, with many other people as well. So I love seeing it used for, like, weird little games or, like, back ends and stuff like that. But there's been some good stuff too. I there's 1 that involves space, and space is always very interesting to me. I forget what it was, unfortunately. But,

yeah. Anything unusual like that, I'm very excited about. And are there any other aspects of the work that you've been doing with channels and the ASGI

specification

and your thoughts on the future paths of those and Django that you'd like to discuss further further that we didn't cover yet? I mean, a little bit. Let's get a little bit into the philosophy around this, right? Because sometimes people look at Django from the outside, but like, Oh, Django. It's this

venerable institution.

It's staffed and maintained by this who's who of Python developers, like, you know, that kind of stuff. But in reality,

a lot of the Django developers have moved on or changed to other frameworks or you know, it's been over a decade. And so the thing really here is to

not just make this something that's important, make it sustainable. Right? It's really important to me that how will we do this is something that is not gonna burn out more people. We're gonna, like, just churn through 1 or 2 more developers and, like, spit them out the other end and make them, like, just not program for a year. And, like, that's really the difficult part of this. And part of that too is, like, when when I say it's important for Python as a whole, I really mean it. Right? Like, Django

is important,

but I don't believe Django is the future of Python web forever,

and that's a foolish

perspective to have. I'd hope that there'd be something else that will complement it in the future if if not, start replacing it. But it's really the case of, like, how do we best serve that stuff? And it's really hard to do that because you don't know what the future is. Predicting the future is very difficult. Like,

I have tried and tried again over my career in both commercial software and open source to try and predict the future. It usually goes quite poorly. Sometimes it works, and people say, oh, Andrew, you're so good. Like, I I was lucky at that point. And so it's really like, how do we make it flexible in a way where

we don't get stuck again with, oh, we have whiskey.

We can't upgrade it, and, also,

everyone who knows about it is just not willing to work on it anymore.

And that's kind of the hole we were stuck in there for a while. So I I really think it's important to not just think about

how we build new specification protocols, but also who builds them and how we come together to organize and discuss and do that part of the open source movement that's often kind of ignored. Yeah. And I I have to say that

channels itself is definitely an impressive feat of engineering. But 1 of the things I was most pleased about while I was reading through for this show is the fact that you

implemented ASGI

as a separate concern and something that lives apart from channels itself and enables

everyone else to

have some focal point to build off of without everyone having to go and reinvent the wheel in their own special way and start to

fragment and fracture the

capabilities and approaches of the Python community because WSGI has been

very valuable

up to this point of providing a very standard implementation of a web interface, and it does have its shortcomings. And there are other projects such as Tornado in particular that allow you to take different approaches. But having 1 common

obvious way

of doing things gives everyone else a good reference point to either build off of or take as inspiration for their own work. Yeah. And it's 1 of those things, like, when you make a protocol like ASCII or anything similar to that, a lot of it's not making, like, oh, the obvious good choice. It's making the choice between 2 both bad options. Right? There's a lot of trade offs in the design of Husky. And this is true of whiskey too. We're like, well, we have this problem. When life the solution is good,

We've gotta pick 1 of them. And, like, that's the value of having

a specification

the value of having a specification like ASCII is that we've taken those choices,

we've made those choices, we've justified them in some cases, like, why we went down that path,

and consistency is more important than perceived best in this. Like, a protocol spec like this will never work for everyone. Like, ASCII is not gonna be perfect for every single implementation of

asynchronous web stuff. Someone will always fall outside the

sphere we've drawn for ourselves.

But our goal is to be there for, like, 95%.

Like, it's the good common use case where you can pull on that common idea and understanding of, like, here's all the docs, here's tutorials, here's the servers, here's the frameworks,

and you can operate inside that sphere. And if you get big enough or special enough or high traffic enough to do something differently,

at that point, you probably know enough that you can look at ASCII,

change 1 of our trade offs for your trade off, and go down a different path. But I think having that information there and having it separate is really important. Like, you know, just recently in the last couple of months, there's now a separate website for ASCII that says like, has the specs, has, like, some extension discussion,

has a list of things implemented to try and build it as

a definite more separate identity to Django.

So for anybody who wants to follow you and keep up with the work that you're doing, I'll have you add your preferred contact information to the show notes. And so with that, I'll move us into the picks. And this week, I'm going to choose a new podcast I started listening to recently called Nobody Listens to Paula Poundstone.

And as the name suggests, it's Paula Poundstone

waxing poetic about various topics and being her humorous self. I will warn that there has been some profanity in some of the episodes, so probably best not to listen with small children, but it is,

very enjoyable and amusing. So with that, I will pass it to you, Andrew. Do you have any pics this week? I have 1 particular pick this week, which I really love, which is a wonderful blog post. I'll I'll, send link to put in the show notes. It's, a literary appreciation of the Olsen time zone database, which among other things shows the separate time zones of World War 2. If like me, you're a big,

nerd about time zones and language and things, it's a wonderful breakdown of some of the really weird,

like, tiny exceptions to time zone rules. You're like, why did this happen? Like, why is there, like, a 3 minute offset of here? Why why does the war have a separate time zone? And it's really fascinating. That's definitely amusing. And, if anyone hasn't seen the falsehoods programmers believe about time, that that's always

good for some amusement

and agony. So I appreciate you taking the time today to join me

and discuss the work that you've been up to with channels. It's definitely a very interesting project. And as I mentioned, I appreciate the work you've done with the ASCII specification as well. So thank you for that, and I hope you enjoy the rest of your evening. Thank you for having me. It's been a great show. Thank you.

The Python Podcast.init

Summary

Preface

Interview

Keep In Touch

Picks

Links

The Python Podcast.__init__