Building GraphQL APIs in Python Using Graphene with Syrus Akbary

Hello, and welcome to Podcast Thought in It, the podcast about Python and the people who make it great.

When you're ready to launch your next app or want to try a project you hear about on the show, you'll need somewhere to deploy it, so check out Linode.

With 200 gigabit private networking, scalable shared block storage, node balancers, and a 40 gigabit public network, all controlled by a brand new API, you've got everything you need to scale up. Go to python podcast.com/linode

today to get a $20 credit and launch a new server in under a

minute. And to keep track of how your team is progressing on building new features and squashing bugs, you need a project management system designed by software engineers for software engineers.

Clubhouse lets you craft a workflow that fits your style, including per team tasks, cross project epics, a large suite of prebuilt integrations, and a simple API for crafting your own.

Podcast.init

listeners get 2 months free on any plan by going to python podcast.com/clubhouse

today and signing up for a free trial.

And visit the site at python podcast.com

to subscribe to the show, sign up for the newsletter, and read the show notes.

And keep the conversation going at python podcast.com/chat.

Registration for PyCon US, the largest annual gathering across the community, is open now. So don't forget to get your ticket, and I'll see you there. Your host as usual is Tobias Macy. And today, I'm interviewing Cirrus Akbari about Graphene, a Python library for building your APIs with GraphQL.

So, Cirrus, could you start by introducing yourself?

Yeah. Hello everyone. I'm Cirrus Lakbari. I've been working

in Python or in the startups for about, like, 10 years. And right now, I'm, like, trying to,

go through the entrepreneurship

path, and I'm trying to build a company.

And do you remember how you first got introduced to Python?

So I think it was, like, 10 years ago I was in college and, actually, I already have, like, programming experience, which was with PHP.

I guess we all have a a starting point. When I discovered Python, I saw, like, it matched very well with my train of thought.

So I just fall in love with it, and since then, I've been developing Python.

And so can you start by discussing a bit about what GraphQL

is and the benefit that it provides

versus a rest based API that people might be more familiar with?

Yeah. Sure. So, GraphQL is has, a specification

that helps us on the task of creating,

APIs

much easier and more reliably.

So how it compares or,

what what is the benefit of GraphQL versus a REST based API

It's basically because, like, GraphQL help us on,

creating or having a type safe API.

And then, like, in it help us to,

to retrieve,

much more data

at 1, request rather than REST API that, like, in general, we have to do, like, multiple HTTP requests for fetching all the data we need. So mainly, I will say, like, GraphQL

helps on implementing your API

from a domain driven architecture

rather than how things are modeled on your database. And that actually helps on making APIs

that makes our clients happier. That means, like, it's easier to query and to use.

Another area of work that has happened over the past couple of years is in putting forth specifications

such as open API and RAML

as a way to

try and standardize the way that rest APIs are built. So I'm wondering if you can discuss a bit about how GraphQL

compares to

those efforts

and whether you think those are still useful

in in comparison to what's possible with GraphQL.

So I think actually both OpenAPI

or Swagger and Ramble

are a really great step forward for rest APIs.

It provides a description format for for for it, and, actually, it let us know, like,

how we can, do, like, HTTP request for a REST APIs

and what kind of data we can expect to get back.

However, I think there are a few reasons that, like, put the GraphQL

at a better point

or as a better solution

than open API.

And these, reasons are,

first, are, like, the types are built in into the specification.

Therefore, you cannot really escape from typing even if you want. So if your,

graph field type is expecting to have, like, a field that is returning a string, you cannot suddenly return an object or a number or things like that.

With a swagger

or open API, it really depends on, like, how you implement it. So even if you document it, like, really well, like, you can still, like, escape from that.

Other advantage or other reason for, using GraphQL,

instead of REST is even because it provides a nested fetching mechanism.

What this means is you can actually, like, fetch

nested data. So if you wanna fetch a user and from a user, you wanna fetch their photos, for example, in REST, you will, this will involve, like, doing tons of HTTP request.

1 for the people model or the user's model,

other for the photos model, and so on. And With GraphQL, you only have to, like, do 1 HTTP request and you retrieve all data at once. So

these 2 are kind of, like, the the big factors. Over these 2, there there is another 1 that I think, it's quite important. Is that the fact that simplifies incredibly the effort required in the clients

to integrate the the data returned from GraphQL into their data flow. That's because, like, they are you stop,

the clients have to stop worrying about, like, how to glue or stitch data together on the client side, and it's on automatically

with, GraphQL clients.

And I know that when GraphQL

was first made public, there was a lot of

excitement in the JavaScript community because it made their lives easier in terms of consuming APIs. But

on the back end development side, it increased the complexity to a degree because of the amount of effort needed to be able to effectively respond to these GraphQL requests where there is potentially nesting and recursion.

I was wondering if you can discuss a bit about how

a GraphQL

API

is implemented and how Graphene simplifies that process overall for somebody who's trying to

create an interface that is conformant to the specification?

So, basically, like,

GraphQL,

with Graphene is easy to, to build. And the main reason is because, like, GraphQL really provides you the with the types and the structures that, like, GraphQL needs. So the only thing that you need to do from the Python side is just create, like, Python classes that, like, inherit from a a graph in, object type. And from there, like, you will need to implement, resolvers,

which specifies, like, how each field is gonna be resolved or, like, how

each field is gonna return data to the GraphQL endpoint. And from that, like, you stop worrying about, like, how, Graphene or GraphQL is doing all the things because, it's actually, like, a

a little like, it's handled everything by different work. So you, like, as a developer,

creating a GraphQL API, you don't worry about, like, how the nesting mechanism is being implemented or anything like that. You just worry about, like, how to create your types and how to retrieve fields,

within your types, and that's it. And then, like, Graphene provides all the all the structure and all the implementation

regarding GraphQL

to assure that, like, this,

matches the GraphQL classification.

And can you discuss a bit about what it is that Graphene is doing behind the scenes

and how it's implemented in terms of the actual library itself?

So first, like, what we're Graphene is doing is, first analyzes

the query that the client is making.

So it runs a parser. It gets, like, all the fields that, like, it's trying to get.

Knowing, like, how is the schema structure,

it validates this query against the schema. So for example, if you are trying to query a a type that doesn't exist or a field in a type that doesn't exist, Graphene will complain about it. So you will see an error and the query will not even get executed. So the first this is the first step, validation

of, syntactic gate first, the validation,

of the query, then, like, the validation against the schema. That means, like, we are able to run queries,

that matches our our structure.

And the last 1 is if these 2 passes is running the query.

For running the query, like, first, we take a look into, like, all the fields that you are fields that you are trying to request

and or the client is trying to request. And then, like, we try to look up all the

types from GraphQL, and we try to go into, like, how you implemented them, and we call the resolvers that you you created.

So this is more or less like how everything matches.

On scrapping, basically, like, we trace all the data. It, returns it back to the client. So it gets,

the client gets the data in the shape they expect.

And how has the implementation of Graphene changed since you first began working on it?

Actually, it changed a lot. At the beginning, just because I didn't have, like, enough, context on on GraphQL and how everything, like, works, I focused a lot on, like, how to create an API that was very easy to use, and that was kind of, like, my main goal.

However, the implementation, I have to say, was not very good. I was,

was bad.

But after, working on it, like, for a few months, I got a better idea of, like, how it should be implemented. And I completely repacked it from scratch. That means,

like,

starting from scratch, like, removing old code and and creating a new code base. But thanks to the API, basically, it still match how I think, like, a graph field should be created and used. Like, basically, the the implications for the developers were almost done. Basically, like, they were able to still use with the same exactly same syntax.

And then about, like, another year ago, I got, like, much better context into, like, how I should make the internal internal soft graphing.

And with that context, I decided to to refactor again from scratch,

which I I did. And

and, yeah, it was, we have, like, some

some small

breaking changes. But overall, like, the API and the way you create types, like, are is almost the same as as the first version of Proven.

So, yeah, I like, my recommendation here, like, for, whoever is creating, like, a framework that is intended to use by, like, more companies or more people is, like, focus on the API first.

Make as many tests as you can, especially on the less on the internals and more on the interface that you are serving developers.

And then as once you have that, it's very easy to refactor or to improve things as you gain context on on the project. Can you talk a bit about the

motivation

that you had for creating Graphene in the first place and some of the other libraries that you have built to accompany

it, particularly

the,

JavaScript implementation?

Yes. Sure. 1st, it was super funny because I think it was about 3 years ago, Facebook invited me to their offices. They were trying to hire me, and they were doing, like, a private show showing their, like, their new baby that they they were gonna expose to a wall. When I saw it, I saw certain potential, but I personally thought, like, was only gonna be useful for Facebook and not outside.

But, like, this was kind of, like, just from first sight.

After a week, the Facebook released first implementation.

And, with that, I got the opportunity to play with it. I also like no JavaScript, and I'm, like, super used to to the front end stack. So I start playing with it, and I saw, like, an enormous potential because I have been, like,

building APIs or dealing with APIs for more than, like, 6 years, when I started at that moment. And I saw, like, a lot of potential. And at the moment, there were there was no framework for using GraphQL in Python. And I saw

in that, like,

incredibly

great opportunity to build something that can make an impact in in the way we develop APIs.

I think you probably started working on this shortly after I gave up on trying to implement a GraphQL API because,

that was around the time frame when, as you said, it first came out publicly.

And I was in the process of greenfielding a new API oriented Flask application, but,

trying to figure out how to make it compliant with GraphQL

at that early stage was a bit bigger of a lift than I wanted to deal with. So I ended up just going with a traditional REST interface.

Completely. Actually,

like, at the beginning,

like, the process that,

I got to make Graphene cannot, like, successful is by, like, porting the JavaScript version completely into Python. This require, I think, was, like, more or less a month of effort.

But with that, like, we assure that, like, our implementation

was, like, 100% compliant with the spec. And, like, once we have that, like, it was much easier to, to start building on on top of that.

But yeah. Yeah. Like, actually, like, even the fact of porting, which

is less,

is not as hard as creating it from scratch because, like, you already have a base that you can start with. Even the start of porting the code, like, was not super easy. I completely understand. Like, starting with GraphQL, like, few years ago was was not really that easy.

And

in addition to the

base Graphene library, I know that you've also built some integrations for SQLAlchemy

and Django for tying into its ORM to

simplify the creation of APIs that interface with SQL databases. So I'm wondering if you can talk through a bit of the,

integration

involved there and some of the difficulties

of getting

these SQL libraries

to produce data structures that are easily represented in the GraphQL format?

So, actually, like, it was less challenging that I thought. 1st on Django and then on SQLAlchemy,

I didn't like the integrations of of them into Graphene because I felt like it was gonna be a very good point for making, like, developers to start using GraphQL in Python.

Basically, if you make, easier path for them to build GraphQL APIs so they can they don't have to worry about, like, how to map from SQL Alchemy models or Django models into GraphQL, then, like, that's a win for for them.

And it will be a win for because, like, they will use the library.

And, like, what was challenging about, like, making integrations

from Graphene or from SQLAlchemy or or Django or the Django models to to Graphene was

was actually just a little bit of introspection to see, like, how we can, like, look up the types that SQLAlchemy

is is providing, how we can, see, like, what are all the columns that, like, these these models have or or what are all the fields that, like, these models have and and try to map them, like, 1 to 1 to

to. But, like, actually, I will say, like, the integration,

it took, like

for SQLAlchemy, the first integration, I will say, took, like, 4 days or something like that. So it was not really that hard.

It it's like at the end, like, we are dealing with, like, Python types with,

Python objects, and and it's super easy to introspect them and and to go from 1 to to the other. So actually, like, graphing on the at the end rather than transforming

your,

your, SQL alchemy

instances, I will say. Rather than transforming it into GraphQL types, what it's doing is reusing your same types

and just kind of, like, matching the output

of, of these types to, like, what the GraphQL is scoring.

And in terms of ensuring that your implementation

in Graphene

is fully compliant with the GraphQL specification, is there any sort of publicly available

test case that you can run it through for validation?

Yes. Actually, like, these are are built into Scratch into both Graphene and GraphQL core, which is, like, the library that, like, powers, Graphene. I think that right now, there is about, like, 98%

or 99% of, like, test coverage. That means, like, almost all lines of code that you will be using within Graphene

are are tested.

With that, like, we just started

looking into, like, all the test cases that the GraphQL main implementation in JavaScript had and try to replicate them in in Python. So this way, we're gonna show, like, 100% to be 100% spec complaint. So, yeah, testing is like, having testing in general is, like, super, super

useful. And

what have you found to be some of the most complex or confusing aspects of building GraphQL

APIs both with and without Graphene?

So first, I think, like, there is because of, like, with GraphQL, you have to model your API

into a way that, like, your clients

you have to know, like, how your clients are gonna query this data. And I think that's a main

aspect into, like, how you build that GraphQL API.

First, you have to understand, like, how your data should be queried.

So first, you are not serving models

per se. You are serving kind of, like, connections on how these models relation between each other.

And with that, like, you have to spend, like, fair amount of time trying to analyze, like, how how your clients are are will be using your data in a in a very easy way. And that means kind of, like, introspecting, using your your application, if you have, like, a if your company is serving a a mobile application or your desktop app, and seeing, like, for each page how the data is used and served. Just start analyzing that the scenarios. And with that, like, you will start having an idea of how you should structure your schema and the relation between your types. Even in general, like, this this relation, like, maps

almost 1 to 1 with the how you shape things in the database.

In some cases, it doesn't. So it's very good that you're aware of all these cases.

You will say, like, these are the the first steps of in the involving building a GraphQL API.

But once you have a clear idea of, like, how should be the structure of your GraphQL schema,

then it's super easy to implement it. It's just like the created your Python objects,

build your resolvers, or reuse,

implementations that already exist,

and that's it.

And I know that GraphQL,

because of the fact that it's agnostic to how the data is retrieved

on the serving side,

it's possible to combine multiple data sources together in a given response. And so I'm wondering how you would structure that in Graphy and as far as being able

to retrieve from, their multiple databases

or federate the query out to maybe different microservices.

Yeah. So, basically, like, the first suspect is, everything is kind of, like, monolithic, and we have, like, 1 code base that, like, is serving all our our

GraphQL endpoints.

From that point of view, we can use, like, almost anything that we want. We can use

Django models with SQL, okay, new models, and both, like,

Redi's database and then, like, Django or SQLAlchemy,

and we can mix them, like, in the way we want, really. Like, it doesn't matter. We just have to know, like, how to relation them,

between the functions.

So, basically, like, how to retrieve, like, certain

radius

object from a from a Django object,

or a radius relation from a Django object or or things like that. But in general, like, how to do is, like, within the same code base is super, super easy.

What it gets more challenging is actually, like, when you have, like, multiple microservices

that are completely isolated be between each other regarding database and and regarding code base. And with that, actually, like, 1 of the things that we are working with RafiN is provide providing schema stitching. And this actually means, like,

being able

to match or to glue together

different GraphQL APIs into only 1. So at the end, like, there is only 1 GraphQL

API that is being queried by the client, which I think is super important.

And I know that 1 of the optimizations that GraphQL provides is primarily felt on the client side of not having to have multiple round trips over HTTP.

But because of the fact that you might be fetching data from multiple different systems in the back end, it, in some ways, just pushes that same complexity further down while shortening the overall round trips.

But because of the fact that 1

API request to a GraphQL endpoint may end up

being more complex to render, it can reduce the overall responsiveness

and throughput of the serving layer. So I'm wondering if you have encountered any situations where that's become a problem and any strategies that you have established to mitigate it.

Sure. Actually, like, I will say at the end, all the data that we need to serve, if it's for creating websites or application, it depends on, like, how we design the pages.

So, basically, for,

application

or or desktop app is trying to serve, like, a lot of data, that means, like, it's trying to serve a user and the name of a user and then, like, 100 photos, and then from each photo, like, 10 the first 10 comments, then, like, no matter what, you will need to request

or get all this content. Right?

So at the end, you will not end saving any time,

using rest over GraphQL.

Neither the other way, the only thing is with GraphQL rather than doing, like, 100 requests, you will do only 1. So you will you will save latency.

Another interesting

step into, like, how GraphQL is even faster than REST in these cases

is because there is 1 strategy called data loading. And with the data loader, actually, like, what you can do is

cache things if you already, like, serve them.

So for example, if you are requesting a photo that have a comment on it and this photo belongs

to user a, have a comment of user a and user b. With rest, for example, you will do, like, 3 requests. 1 for user a,

13 fields,

then, like, you will retrieve the photo, and then you will do another request for user a and user b again. With GraphQL,

all this is solved with data loader. But not just that. At the same time that you are requesting, for example,

3

same types at the same time, like you're, you're requesting 3 different users at the same time with the data loader strategy. What you can assure is rather than doing, like, 3

3 SQL queries, for example, under hood, you are only doing 1.

So all these strategies are actually, like, much harder to solve

from a purely REST point of view.

And what are some of the

capabilities

that are unique

to GraphQL APIs that wouldn't be possible when using a REST interface?

So unique capabilities, there are there are a few. 1 is, like, everything is typed, which I think

is is super good, especially if we are not in a typed language. Like, Python is not typed. Right now with Python 3, we have annotations with, I think,

they are a great step forward.

But in general, like, if that's, you know, types on top of our system and especially non type systems, like, gets a big benefit from from this. The second 1 is you have introspection

into your GraphQL schema,

and what that means is the clients can know, like, what is the structure of the API, and this have, like, a lot of benefits.

1 of these benefits is how the clients can auto generate very easily or look at the documentation

of of the types super easily. And the other 1, which I think is a great tool, is Graphical.

Graphical is an interface for querying, GraphQL APIs that actually let us introspect these GraphQL APIs while we are typing. And we have auto completion

of the fields. We can see, like, what we can query within, like, a type, and we can see also, like, what is all this recommendation for for the types that we are querying. So at the end, like, all these things, introspection,

type safe type safety

and

type sound, soundiness.

It helped us

a lot into, like, querying and and creating our GraphQL APIs.

And are there any particularly

interesting or useful or unexpected lessons that you have found to be particularly valuable in the process

of building Graphene and using it to create GraphQL APIs?

1st 1 is community is essential. I do think, like, getting in touch,

or getting feedback as soon as possible

from people using your framework is essential to make it go and actually helps a lot, adoption and, users using your your library, like, know that, like, you are aware of of their their thoughts on on how they think, like, the library should evolve. So I think that's 1 of the the thing that I learned from from Graphene. The other thing is, like, testing

is super, super, super important. It actually assures for big companies or companies in any size that they can use graphene reliably. The other thing that I thought

I might I could actually might be better if, if I thought about this, like, when I started this, like, how how can I make this a sustainable project? Because right now, like, over time, it it become a little bit harder to know, like, how

how now companies that are using Graphene, which are about, like, 5, 000 companies in the world, how they can, help the the ecosystem to grow in a way that is sustainable and it doesn't depend into only 1

1 actor but multiple.

And on the sustainability topic,

it seems that 1 of the approaches you've taken is to build the Quiver project

to help optimize the serving of GraphQL APIs

built up built with Graphene.

So can you discuss a bit about the what the Quiver project is and how it fits in with the other Graphene projects and how it fits in with your strategies for ensuring that you can continue to,

support your development efforts on graphene?

Sure. So Quiver was a first commercial solution based on on Graphene. What it is is a a step up on the GraphQL engine that is focused on on optimizing the the overhead of the GraphQL engine. So it actually can run at at almost native performance.

So that's kind of like the the first goal of Quiver. Right now it's being used by by 2 companies that are actually, like, paying for it. And we can talk a little bit later, like, what Quiver is doing, like, on their foot to

to optimize the API. But, basically, like, with Quiver, I'm I'm helping to sustain the the Graphene ecosystem and, and GraphQL in Python in general. And it's a way that actually, like, big companies can start paying or offering support. So, basically, like, small companies can can also take advantage of of the of the ecosystem.

Yeah. If you could dig a bit further into

some of the strategies that you're using for optimizing the API serving in Quiver, I think that would be interesting.

1 of the things that you have to know is I've been, like, super interested in template engines for, like,

a lot of years.

And, I think it was 8 years ago, I built, like, a template engine, which was called PyJade.

It was for, generating

HTML

code more or less more easily. And with that, I discovered basically, like, what kind of templates there are and and and so on. And I discovered there were 2 kind of templates.

The 1 that were fast and the 1 the ones that were were slow. So for example, if you look at the if you take a look into the Django template engine

and the Jinja 1, like, both have a very similar syntax. Right? But, like, Jinja performs from 10 to 20 times faster.

How they do that? They actually, like, compile like, Jinja itself compiles the template to Python code to remove completely the overhead of a template engine. So run running render on a Jinja template is almost the same as as calling a function that returns a string.

And with Django, actually, you have you have to run through the lexer parser and all this stuff.

So there is a lot of overhead. Having this experience, I started thinking into, like, how we can start optimizing

the

the GraphQL engine because for some cases, for example, there were some companies that were returning, like, 100, 000 objects

with a lot of fields in each, and it it was taking, like, few seconds while it it should take, like, 100 milliseconds or or something like that, basically, for just kind of, like,

the base JSON serializer.

So I started thinking, like, what I can do to to improve the speed. And based on this experience, on on the template

experience,

I I started researching with the idea of,

compiling the these GraphQL queries

to Python code to remove completely the overhead of, of GraphQL engine.

So, basically, with Quiver,

that means the running GraphQL queries

is the same as building

your own endpoints

at hand, doing all the fetching smart fetching mechanisms that, like, GraphQL is doing under it, but without the overhead of running the the GraphQL framework. So basically, with Quiver, you can see more or less a speed up of or of 5 to 20 to 20 times

faster

if, of course, you don't count the IO in your in your API.

And you're providing that as a managed service for people who need that performance improvement.

So wondering if you can discuss a bit how successful that is in terms of enabling you to spend a significant

amount of time focusing on Graphene and building its ecosystem.

Yeah.

So thanks thanks for that question because it's been, like, a little bit challenging to make it work, especially at the beginning.

But, like, a lot of companies reach show interest.

It was hard to know, basically, like, what kind of pricing should I put,

how I'm, like, super involved into, like, having all these open source. But at the same time, I'm worried of, like, open sourcing something and then, like, not being able to

to sustain, like, this project or or any other.

So what I decided to do is kind of, like, offer this as a kind of a paid solution.

So far, like, few companies are paying for it. It's not doesn't

help me, like, to reach, like, even, like, 20% of of my needs,

of, like, living in San Francisco, which I don't know if you know, but it's super expensive.

So it doesn't cover, like, most of the needs. But at the same time, I think, like, we just need a little bit more of time to get, like, more companies, or big companies to use it. I mean, I'm talking, like, with, really big companies that they are interested on trying quicker. And, for example, they are, like, performing about, like, 1 1000

to 2 1000 queries per second.

And in these cases, like, it's it's super

like, they can get a lot of benefit of of using Quiver.

But, like, my goal eventually is actually, like, open source it, but only

once I'm able to sustain

to sustain myself.

Basically, once I'm

I can verify that, like, big companies are up for sustaining this project.

Yeah. 1 licensing model that might be interesting to explore down the road is 1 of the ones where the,

source code at a particular release

reverts to open source after a period of something like 2 months, which,

allows people to take advantage of the project on their own terms while still providing incentive for someone to pay so that they can stay up to date with the latest releases and latest improvements

while still having that option?

I think that's, a quite interesting option. Right now, for example, like, I'm trying this to grow naturally. Like, I don't wanna push any companies to to use it or or to pay for it. Right now, other strategy that I'm following is I created that Patreon

page. Yeah. So people that are using Graphene or companies that are using Graphene and wants to sustain the project, they can do it, like,

without, like, getting anything in exchange.

I mean, in general, like, getting, like, probably more visibility, their logo, and the graphic page,

helping to to sustain the project. But, like, with a specific term of of,

this is for paying for or

or

anything is in the specific. It's just gonna be like, this is for support the project and making sure, like, it's sustained over time. So this is something that I'm I'm, still playing on, but,

I'm not sure. I'm not sure, like, how this will end and what will be the end path for, like, making graphene kinda like a fully

fully sustained and supported project.

However, I'm all into, like, sustaining it myself while I can.

And in terms of someone who

is interested in building a GraphQL API on top of Graphene, can you just talk through the workflow

of going from either an existing or a greenfield

application through to having an API available that is implemented on top of Graphene and serves as a GraphQL endpoint?

Yes. 1st, we'll need to know, like, basically, like, in which,

server frame which server framework are you using. Like, are you using Django? Are you using Flask? Are you using

or?

Like, basically, what kind of, like, server framework are you using? This will help, you to to get the integration of Graphene

in that framework much easier so you don't need to write, like, the implementation by hand.

And we have, like, implementations in normal almost

all,

graph

almost all server frameworks.

So it should be easy for you to integrate that side. And then, like, you will

it will be useful to know, like, basically what kind of data system are you using, if you are using any OEM, and looking looking at integrations of Graphene

into your OEM.

And just by following the documentation and see kinda like how you can, like, plug your own model to a graphene

graphene or GraphQL model.

That should be, like, that should be, like, super easy. So, basically, like,

just start downloading Graphene. You start downloading the Graphene integration within your server framework,

then you download the Graphene integration with your OEM framework, if you have any. And if not, you can do it by hand, the integration, and then you just run your server, and that's it. Like, in general, like, starting with a GraphQL server or plugging a GraphQL server between a existing Python server is super easy. It should take, like, at most 5 minutes.

And once you have the schema defined, it also mentions

that you can define the queries that are available.

So I'm wondering if it's possible for a given

client to be able to generate queries that don't specifically match what's defined on the server end,

or if the queries that get defined in Graphene are the only ways that are available for interacting with the API?

Yes. So first, when you query, the first thing that happens when you do a GraphQL query

is we

validate the query. We first check that this is synthetically

bad valid. That means, like, it matches the structure of a GraphQL query.

If this doesn't hap if that's not the case, then, like, we stop there at the execution and we throw an error. If for whatever reason is a valid GraphQL,

syntactically valid

GraphQL query,

then we check if the query can be run against our schema. Basically, if we are able to query the fields that are exposed within GraphQL.

If for whatever reason you're trying to query things that are not exposed, it will also throw an error. So you will not be able to query more than it than it's exposed to GraphQL.

So that's in the case, like, you are running,

you are running

GraphQL queries that, you know, like, matches your schema. Because in the case they don't match, then the the

request stops.

But in the case they match your schema,

let's say, like, you have a you have a type which it have a field that takes a lot to resolve. For whatever reason, resolving this field is super expensive. Let's say it does, like, a super complex SQL query under under a hood.

And you wanna limit basically, like, how many times you can do it or or things like that. There are a lot of mechanisms in Graphene that you can implement

to make sure,

like, first,

like,

things are not queried much more than they should. You can have, like, make defense mechanisms there. But other things that, like, in general, other companies does is they, they define

a set of all valid GraphQL queries, and if you try to go outside of these valid GraphQL queries, it will run error. So in general, even if the queries are dynamic,

they are dynamic in development mode. In production mode, they are static, and you cannot go over, like,

statically defined set of queries. So if you go or, like, if you try to do something dangerous trying to run like any other query, it will also, like, throw an error. So in this sense, there are, like, a lot of mechanisms that you can do to to stop.

Or even, like, you can implement a middleware and check, like, how many fields,

is a user, like, querying. And if it's more than they should, you can just stop right away.

What are some of the most interesting or surprising ways that you've seen Graphene and GraphQL

used?

So

1 of the most surprising is, or I would say most interesting is, like, using,

GraphQL or Graphene, like, to actually, like, scrape,

HTML data.

I think that was kind of like super interesting use case. It was not very surprising because actually I created it,

But, like, it was a funny way to play with GraphQL and

and show to people that actually, like, it can be used for

some more things or, like, fetching data

from databases.

In the case of a scraping, they they something that, like, it was not very common to see see before. But, like, there are a lot of other use cases that I was

it it was surprised to see. For example, like, at the beginning, Graphene was meant

to or GraphQL was meant to for a server to client communication. That means, like, from a server to either a mobile app or to a desktop app or, like, any kind of client that you can control.

But it was never thought to be

used at the beginning

in for server to server communication. That means, like, you have a microservice.

This microservice serves our GraphQL API and then you have other microservice

who wants to consume this GraphQL API.

So for that case, there are certain,

scenarios that were not covered. For example, the case of, like, fetching 100, 000 fields or 100, 000 object types. And this at the beginning was, a little bad surprise because

I realized

I have to make Rafi much more performant to make it happen. But then, like, once we

went over, like, the cases, like, we basically, like, downed the

the the time from,

I don't know, the next, I think it was, or something like that. So, yeah, this was, this have been, like, some interesting uses of of Graphene for sure. I particularly like your mention of using Graphene to serve web scraped data via the GraphQL API because it just showcases the fact that

the resolvers

for the given schema are just an arbitrary Python function. So as you mentioned, it doesn't necessarily have to retrieve data from a database. It can do That's right. Arbitrary API requests of its own, or it can, as you said, go out and scrape a website, or just anything that you can do programmatically with Python, you can potentially return as a data object within that GraphQL API. So

it helps expand the way that people

perceive what's possible.

Yeah. At the end, like, if if you are serving that data for sure and these data have relations between it, for sure, you can use this graphically in it. There are some challenging cases. For example, if you wanna do things that look like joins or or or aggregations in

in, GraphQL, I will say these are, like, challenging cases

to, to do it properly in GraphQL. But almost all our cases,

that means, like, fetching data,

and relations between data. You will see, like, in GraphQL and and Graphene, like, a very good opportunity to to play with GraphQL.

And what are some of the cases where it would make more sense to

use REST or some other more commonly used interface instead of GraphQL?

Some I think

GraphQL is very useful when you are in control of of the clients that are querying your data.

However, like,

sometimes, you are not in control of that client. That means, like, for whatever reason, your company is serving APIs that other companies are trying to consume.

In this case, like,

it depends if these other companies want to use GraphQL or not. And in some cases,

they still want to go through REST.

So for that scenario,

probably, like, you are still good with rest because you cannot change your clients. But whenever your company is it's dominates

or or can can dictate, like, how the clients behave

or how the clients are

are are are built, then, like, GraphQL is a very interesting scenario.

Because I feel, like, 1 of of the most useful things about it is you can create GraphQL APIs at a much faster pace or APIs in general at a much faster pace. And not just that, but, like, the front end incredibly

gets a benefit for from that,

in the sense of, like, their developing time or or things like that.

So for me, the only case when, like, I will use REST is if your if if you have, like, no

control over your clients and your clients want to still use REST, then you're you're tied.

Otherwise, GraphQL should be, like, a a good, option to to research.

And 1 of the nice things too about the fact that GraphQL is served up from a single endpoint is that it makes it easier

to

progressively

add support for GraphQL to an existing application that already uses REST. So if you're using something like the Django REST framework, but you want to support GraphQL QLA as well, you can do it in parallel

and the client can decide which interface they want to use until

the entirety of the functionality

has been encapsulated in that GraphQL endpoint. And you can then,

in theory, at least, depending on how the clients are distributed,

turn off that REST endpoint.

Yeah. Completely. Actually, like, when Graphene, we have all as well, like, integration with Django REST framework.

So you can map your Django REST framework models to GraphQL types automatically,

which I think is is super powerful. But 1 of the interesting cases for companies that they still need to have, to serve rest endpoints,

but they wanna research into GraphQL is building your rest endpoints on top of GraphQL queries. So basically, like, for the client, it will still look like a rest endpoint, but under the hood, this rest endpoint will do a GraphQL query

like in your implementation side. So

so this way, you got you are still kind of, like, using

a GraphQL for building your API structure, But this way, you provide a, easy wrapper for clients that that still depend on the rest, which I think is a super interesting use case.

Yeah. That that's pretty funny to have sort of

building the, GraphQL API alongside and then shunting it underneath as you move

the existing rest end points over to just use the GraphQL that you built.

Yeah. That's that's, I think there there is a a good opportunity for that. And I know, like, few people are working on, like, making that to happen with, with Graphene. We just need a little bit more time to to do it automatic in an automatic way. But in general, it should be like super super easy even if you do it by hand.

And for anybody who's interested in learning more about the GraphQL specification

and the ecosystem

that's being created around it, what are some references that you would recommend?

So, first, the graphql.org

website

is is great. Like, you have a a lot of resources and it describes, like, very well the different types that we have

in our GraphQL

schema.

So

so we can see, like, what is an object type, what is an interface, what is an union,

what is a field within a type, and and all these kind of, like, relations.

So that's useful.

Apart from this website, which have a little bit more of documentation on the graph on the GraphQL JavaScript

side, we have how to GraphQL,

which is a website that I have tutorials for using GraphQL in almost any language,

including Python.

And apart from that, in the Graphene Python with itself website,

we have documentation

and good tutorials to to get this started on, GraphQL and European.

And what do you have planned for the future of the Graphene libraries and ecosystem?

So it's it's very interesting, like, what we have planned ahead.

1 of the first thing that I thought is actually, like, moving into other to other languages. So they are available, for example, in JavaScript or Ruby, basically, with the same kind of, like, syntax of creating graphical types.

However, like,

there is a little bit of it's it's not easy to reach a lot of, like, different languages.

Right now, especially for Graphene and Python, we are working on getting up to date with the latest spec.

Right now, 1 of the most challenging things that we did at Graphene is actually, like, make it work same code base either if you are using Python 2 or Python 3.

But this brought some downsides. For example, like, if we do that, we can assure, like, that you can use asyncio,

but, like, we have to do a lot of things under it to make

it work as well with other

other,

async frameworks

like g event or something like that. 1 of the effort that we are doing is moving the GraphQL

core library, which is like what powers Graphene

up to date to, like,

Python 3 with the type annotations and all all this stuff. Other thing that, I think is super interesting

is being able to generate your GraphQL types from Python objects that are annotated.

So you just need to create like

a normal Python object

that is annotate, like, have annotations and it's saying, like, okay, this field is a string or this field is a integer or this field is a it's a relation to other object. So 1 interesting use case that we are researching on is regenerating these GraphQL types from annotated types in in GraphQL.

And this is more in the short term. From now to 2 months, like, we'll get, like, these things ready.

In the long term, though, just looking into, like, how I can, how we can make, like, graphene to be available in more languages. I'm looking in into

universal GraphQL,

which is kind of like a universal way for defining, like, GraphQL schemas that can be actually available in the language

or can be used in the language. But this is more kind of like long term planning. I have to do with other other things that are are less close

are not as close with Python, like web assembly, but I think have a lot of, potential to make, Python shine even more.

Yeah. As I was reading through the documentation

and looking at the schema definitions, I was thinking how it would be interesting to see how that would

work in terms of being able to autogenerate the schema from a simple decorator on the data classes that got introduced in Python 37.

Yes. Actually, I'm I'm super interested. I love the data classes syntax. I think it's super smart.

And, like, actually, the

the thing that we are researching into, like, generating GraphQL types from annotated types is completely based on the on the data classes from Python 3.7.

It will be great. I mean, I I don't remember, like, what's the URL of the issue where we are tracking this. But, like, if you search if you go to the echo repo and you search for type annotations, I think,

you will find there the issue.

And you can see, like, basically what is our research, but it goes

over

a very similar

syntax of using data classes, which I think I personally think they are they are great.

And are there any other aspects of GraphQL and its ecosystem or the work that you're doing with Graphy and its various libraries that you think we should discuss further before we close out the show?

I mean, in general, I will,

love that, like, people will start, like, playing more with, Python 3, like,

kinda, like, more in getting more involved in, like, the Python strict process of of Graphene or how we can make it, like, more com more, like, type safe and

with, static annotations with MyPy,

how we can make it, like,

a tighter integration with asyncio and all this stuff. But other than that, I think, like, we cover, like, most of

the graphing graphing stuff or graph field,

stuff in Python.

And for anyone who wants to follow the work that you're up to and get in touch, I'll have you add your preferred contact information to the show notes. And so with that, I'm gonna move this into the pics. And this week, I'm going to choose

the audible service.

It's so when we go on long car trips, it's great having audiobooks to listen to because it keeps everybody

happy and engaged.

So

that's been very valuable. And sometimes waiting

for certain books to be available on hold at the local library

ended up taking a bit too long. So having the Audible subscription

is great for being able to just get easy access to the specific book that you want, and it's a fairly reasonable cost. So that's been,

it's been a useful subscription to have for the past few months. And so

with that, I'll pass it to you, Cyrus. Do you have any picks this week?

Yes. I have the,

okay. I'm I'm super biased. I I have, the pick of, of the Patreon website of for group for Graphene.

Is gonna

help is gonna help the ecosystem move forward and and and basically, like, everyone that is using GraphQL, like, be sure that, like,

that the product is sustained and, like, is moving at a pace that, like, all

we need.

But, yeah, like, jokes aside, I so I will recommend, like, 2 things. 1 is, like, for people taking a look on WebAssembly.

I think it's,

the future

of computation in a lot of ways,

and I'm super excited about about it.

It's it's not only browser side. That's 1

1 of the things that I I wanted to do, and I have other 1 in mind that I just

I just forgot. But, yeah, let's let's, say WebAssembly, and and let's see if people are curious about it and and ask me any questions.

Well, I appreciate you taking the time today to discuss the work that you've put into Graphene. It's definitely great to see something of that level of polish available for Python developers

to

leverage some of the new

interfaces and new practices in the technology space. So thank you for that, and I hope you enjoy the rest of your day.

Thank you. Thank you for inviting me to Versus, and I hope you enjoy the rest of the day too. I hope to to stay in touch.

The Python Podcast.init

Summary

Preface

Interview

Keep In Touch

Picks

Links

The Python Podcast.__init__