Summary
Any time that your program needs to interact with other systems it will have to deal with serializing and deserializing data. To prevent duplicate code and provide validation of the data structures that your application is consuming Steven Loria created the Marshmallow library. In this episode he explains how it is built, how to use it for rendering data objects to various serialization formats, and some of the interesting and unique ways that it is incorporated into other projects.
Announcements
- Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
- When you’re ready to launch your next app or want to try a project you hear about on the show, you’ll need somewhere to deploy it, so take a look at our friends over at Linode. With 200 Gbit/s private networking, scalable shared block storage, node balancers, and a 40 Gbit/s public network, all controlled by a brand new API you’ve got everything you need to scale up. Go to pythonpodcast.com/linode to get a $20 credit and launch a new server in under a minute. And don’t forget to thank them for their continued support of this show!
- And to keep track of how your team is progressing on building new features and squashing bugs, you need a project management system designed by software engineers, for software engineers. Clubhouse lets you craft a workflow that fits your style, including per-team tasks, cross-project epics, a large suite of pre-built integrations, and a simple API for crafting your own. Podcast.__init__ listeners get 2 months free on any plan by going to pythonpodcast.com/clubhouse today and signing up for a trial.
- Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email hosts@podcastinit.com)
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers
- Join the community in the new Zulip chat workspace at pythonpodcast.com/chat
- You listen to this show to learn and stay up to date with the ways that Python is being used, including the latest in machine learning and data analysis. For even more opportunities to meet, listen, and learn from your peers you don’t want to miss the Strata conference in San Francisco on March 25th and the Artificial Intelligence conference in NYC on April 15th, both run by our friends at O’Reilly Media. Go to pythonpodcast.com/stratacon and pythonpodcast.com/aicon to register today and get 20% off
- Your host as usual is Tobias Macey and today I’m interviewing Steven Loria about Marshmallow, a Python serialization library that is agnostic to your framework and object mapper of choice
Interview
- Introductions
- How did you get introduced to Python?
- Can you start by describing what Marshmallow is and the history of the project?
- What are some of the capabilities that make it unique from other similar projects in the Python ecosystem?
- What are some of the main use cases for schematized serialization and deserialization?
- Can you walk through how a user would get started with Marshmallow, particularly for complex or nested schemas?
- Can you describe how Marshmallow is implemented?
- How has that design evolved since you first began working on it?
- How have the changes in the Python language and ecosystem impacted the requirements and use cases for Marshmallow?
- What are some of the most interesting or unexpected ways that you have seen Marshmallow used?
- What have been some of the most interesting, complex, or challenging aspects of building the Marshmallow project and community?
- What are lessons you’ve learned from maintaining marshmallow?
- What have been some of the benefits and drawbacks of keeping Marshmallow agnostic to any frameworks or object mappers?
- What are some of the edge cases that users of Marshmallow should be aware of?
- What are some of the little-known features of Marshmallow that you find most useful?
- What do you have planned for the future of Marshmallow?
Keep In Touch
Picks
- Tobias
- Sherlock BBC tv series
- Steven
- Greater Than Code podcast
Links
- Marshmallow
- Butterfly Network
- Biology
- ORM (Object Relational Mapper)
- ODM (Object Document Mapper)
- Webargs
- Avro
- Swagger/OpenAPI
- REST (REpresentational State Transfer)
- JSON-Schema
- Environs
- Django Rest Framework
- WTForms
- DynamoDB
- MongoDB
- Etsy’s boundary-layer for building Airflow DAGs from config files
- Toasted Marshmallow
The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA
Hello, and welcome to podcast dot in it, the podcast about Python and the people who make it great. When you're ready to launch your next app or want to try a project you hear about on the show, you'll need somewhere to deploy it. So say hi to our friends over at Linode. With 200 gigabit private networking, scalable shared block storage, node balancers, and a 40 gigabit public network, all controlled by a brand new API, you've got everything you need to scale up. Go to python podcast.com/linode, l I n o d e, to get a $20 credit and launch a new server in under a minute. And don't forget to thank them for their continued support of this show. And if you're like me, then you need a simple and easy to use tool to keep track of all of your projects.
Some project management platforms are too flexible, leading to confusion of workflows and days' worth of setup, and others are so minimal that they aren't worth the cost of admission. After using Clubhouse for a few days, I was impressed by the intuitive flow. Going from adding the various projects that I work on to defining the high level epics that I need to stay on top of and creating the various tasks that need to happen only took a few minutes. I was also pleased by the presence of subtasks, seamless navigation, and the ability to create issue and bug templates to ensure that you never miss capturing essential details. Listeners of this show will get a full 2 months for free on any plan when you sign up at python podcast.com/clubhouse.
So help support the show and help yourself get organized today. And don't forget to visit the site at pythonpodcast.com to subscribe to the show, sign up for the mailing list, and read the show notes. And don't forget to keep the conversation going at python podcast.com/chat. You listen to this show to learn and stay up to date with the ways that Python is being used, including the latest in machine learning and data analysis. For even more opportunities to meet, listen, and learn from your peers, you don't want to miss the Strata Conference in San Francisco on March 25th and the Artificial Intelligence Conference in New York City on April 15th, both run by our friends at O'Reilly Media.
Go to python podcast.com/strataconandpythonpodcast.com/aicon
[00:02:14] Unknown:
today to register and get 20% off. Your host as usual is Tobias Macy. And today, I'm interviewing Stephen Loria about Marshmallow, a Python serialization
[00:02:23] Unknown:
library that is agnostic to your framework and object mapper of choice. So, Steven, could you start by introducing yourself? Sure thing. I am what I'd I'd say a software hobbyist, meaning I write software a lot in my free time, and I'm doing that a lot more right now because I'm between jobs. But by the time that this airs, I'll be a software engineer at a company called Butterfly Network. And do you remember how you first got introduced to Python? I do. So it was in probably my junior or senior year of college. My major was in biology. So I was working in a lab that was studying vocalization patterns in California mice, which was really interesting research.
But as an undergrad, you end up doing undergraduate work, which is less exciting. So for me, that meant listening to these hours and hours of recordings of mice squeaking and calling to each other and classifying the vocalizations by hand. And it was very tedious. And I thought to myself, like, there there must be a better way. And it turned out there was. So, thankfully, my PI was, very generous in how I was able to use my time. So, I took the time to learn Python and end up ended up writing the script, that used supervised learning, to classify the mouse squeaks. So it reduced the time that I had to listen to these mouse squeaks from hours down to minutes. And from then on, I was, like, hooked on to Python. I'm like, wow. This can actually solve problems that I have.
[00:03:51] Unknown:
And somewhere along the line, you decided to start working on the Marshmallow project. So can you give a bit of about what Marshmallow is and the history of the project and how you started it? Yeah. So Marshmallow is
[00:04:06] Unknown:
a library for serialization and deserialization of data. So what does that mean? So serialization is taking application objects. Let's so say ORM objects or ODM objects, and then converting them down to simple types. So strings, numbers, things like this. So once you have those and you can persist them somewhere or transport them over HTTP, over a REST API, for example, and deserialization is the just the, reverse operation. So you take that structure of simple types, validate validate against the schema, and then convert it back to your application specific objects.
So a little bit about the history of, Marshmallow. I was working a few years ago on, what was a very typical CRUD app, with a with an HTTP API, and it was a sort of mixed multipage app, single page app. So there were some pages that were using server rendered Mako templates, and there were but there were interactive UI elements in JavaScript. And I noticed that there was a lot of duplication of hand coded serialization and formatting logic in our view layer. And it was at this around the same time that single page apps backed by rest APIs were, becoming sort of commonplace. And I came to this realization that, like, we use templating languages like Mako and Jinja to define the formatting for for HTML. Why don't we do the same thing for JSON? I mean, after all, JSON is a client facing presentation format just like HTML is. So, you know, why don't we have a Jinja for JSON? So I had used Django REST framework before, so I was familiar with its serialization module, and I wanted something similar in Flask.
We were building a Flask app at the time. So I started the building the pieces of what would a sort of Flask REST framework. But then eventually, I stopped and asked myself, like, well, why do we have so many Flask and Django plug ins? Like, why does, this package need to be, you know, Flask dash this or Django dash that? So what I really wanted was to build a serialization library that wasn't tied to any framework at all or any or or m at all. So that's when I started working on Marshmallow. And
[00:06:18] Unknown:
there are a number of other libraries in the Python ecosystem, both currently and I'm sure at around the same time that you start first started working on it, that also serve to serialize and deserialize or validate structures of these serialized objects. So can you give a bit of a compare and contrast between Marshmallow and some of the other projects that are available in the Python ecosystem?
[00:06:40] Unknown:
Sure. Whereas, I think a number of other similar projects try to focus on doing 1 thing well. We kind of ditched the Unix philosophy, and Marshmallow actually tries to do 3 things well. Tries to do validation, serialization, and deserialization. And to me, that's a benefit because these operations actually turn out to be very tightly coupled to each other, and Marshmallow really tries to exploit that relationship. And the upside for users, of course, is that there's less code. Like, you have 1 library to do all of those things and, typically, you know, 1 class to do all of those things for the same object type. So another thing that I think that differs from similar libraries is its class based approach. So as opposed to some libraries that use dictionaries to define schemas, Marshmallow uses classes, and this allows for easy code reuse and, configuration. And using classes also allows for modifying serialization behavior at runtime just by virtue of being able to create different instances of the schema.
So for example, like, if your HTTP API wants to allow for requesting a subset of a schema's fields, you can just pass those fields field names to the schema constructor and, limit the serialization output to those fields. This isn't to say you can't use the sort of simpler syntax. You could define your scheme as as dictionaries. That's still possible with you know, you can you could write a helper function that just gen generates schema classes. In fact, I have a couple projects that do this. Webargs, for example, does this so you can define schema classes just as key value pairs. But that that's that's not sort of built in. Like, I really wanted this way, this the sort of, like, DRF approach where you could reuse schema classes. And I think another sort of unique sell selling point for Marshmallow at this point is its maturity. So as the project stabilizes, the ecosystem around Marshmallow has continued to grow. There are more and more integrations with the popular web frameworks.
The majority of them, I think, are in Flask, but there's also ones for Pyramid and AIO, HTTP, and other frameworks as well, I'm sure. There's integrations for different ORMs and ODMs, such as SQLAlchemy and Peewee. And then there's also libraries that generate, specifications, such as Swagger or OpenAPI or JSON schema and things like this. So you have this combination of libraries that I think provides a lot of the needed batteries when you're using a a micro framework.
[00:09:09] Unknown:
And the class based aspect too makes it easier to not get locked into thinking of the only option for output being something like JSON, where it's easy visually to see the mapping between a Python dictionary and a JSON object, whereas you might want to output to something like XML or Avro or some other more sort of niche, serialized format outputs.
[00:09:33] Unknown:
Yeah. Exactly. So, so, actually, Marshmallow has a sort of a a feature for rendering to different formats. So Marshmallow does the job of serializing to those simple types. Basically, you're taking an object and serializing it down to a dictionary, but then there's this option to plug into a rendering module, we call it. So you can say, I want to take that that dictionary and render it to JSON or YAML or Avro or what what have you. And also some of these other output targets might support more rich type information as opposed to just the strings and ints and bullions that JSON will provide,
[00:10:12] Unknown:
particularly things like XML, where you can have your own schema definition that has these complex or nested types that are natively representable in the output format. Yep. Exactly. And so we've talked a bit about the idea of using serialization and deserialization in the context of a web framework for rest APIs. But what are some of the other main use cases for being able to have a schematized serialization and deserialization layer within an application?
[00:10:43] Unknown:
Yeah. Marshmallow was built first and foremost for building REST APIs. That's the use case that I wanted to solve for myself. But since Marshmallow has matured, it I've definitely been seeing a lot more different use cases that I had not expected. 1 common use case that I see is validating configuration, so validating, like, YAML files or, I have a related library called Envirens that that validates, environment variables. Another use case for schematized, serialization is, generating documentation. So since you're separating the data formatting from your application code, you can just take those those classes, and then you can target other specification formats like, OpenAPI and JSON schema, as I mentioned.
[00:11:28] Unknown:
And so for somebody who wants to get started with using Marshmallow, can you walk through the beginning steps of defining a schema and how that would evolve as they start to need something like nested objects and nested representations of those schemas?
[00:11:44] Unknown:
Yeah. So the Marshmallow API, like I said, is very class based. So what you define a schema subclass, this is most typically in, a schemas dotpy file, but that's not prescribed or anything. But you have the schema subclass, and then you define the shape of your internal data by instantiating these field objects from the marshmallow fields module. And then from there, you can also define options. So you there's a class meta nested class within the schema subclass that you can use to specify options that may, change the behavior of validation and serialization. So for example, if you want to if you want your serialized output to be an order dict instead of, a regular dict, you can just pass order equals true to that class meta options object. From there, you instantiate the schema, and the rest should be pretty familiar. So to serialize an object, you call schema dot dump. To deserialize an object, you call schema dot load. As far as the nesting use case, there's a nested field from which you can compose another schema, or you can even nest a schema within itself.
There's also, for more complex use cases, you may need pre and post processing. So you can add methods to your schema subclass and decorate them with the relevant decorator. So there you have preload, postload, pre dump, and post dump decorators,
[00:13:08] Unknown:
for that purpose. I've used Marshmallow for my own projects before, particularly in a Flask app, as you mentioned before, and it was definitely nice to be able to have at 1 location to be able to see what are the main objects that I care about that I'm going to be using as an IO interface for the front end. You know, these days, there's projects such as GraphQL that provide a similar approach, but for the majority of people who are still using REST, being able to just have that schema file of, these are the objects, these are the ways that they're represented and nested, and these are the types that I care about, is a good way to ensure effective communication between teams who are building these projects.
[00:13:49] Unknown:
Exactly.
[00:13:50] Unknown:
And so can you talk a bit more now about how Marshmallow itself is implemented and how that design has evolved since you first began working on it? Sure. So Marshmallow
[00:14:00] Unknown:
is very much implemented in the spirit of Django REST framework and WTF forms and even Django models, where you have this class based DSL for defining a schema. And then you have the field classes, which encapsulate the serialization and deserialization logic for each data type that you're dealing with. So for example, the daytime field class will know how to take a string and deserialize it into a daytime object, and do the reverse. Take a daytime and then serialize it to an ISO or RFC formatted daytime string. Then you have the validator or validate module in which you have a number of built in validators.
And validators are just implemented as callables, but, there's a number of common validation functions that come built in with Marshmallow in the validate module. And then the pre and post processing functionality, which I touched on, is implemented as decorators, which sort of tag schema methods so that they can be called at the right time during the invocation of, the load and dump methods. So you asked how the design has evolved. It's changed quite a bit. So pre 1 0, Marshmallow was a serialization only library. So I really only meant for Marshmallow to serve my purposes, and my purposes at the at the time were to to just dry up all of the data formatting logic in the app that I was working on. But then 1 0, for 1 0, we added deserialization and validation, and then that sort of brought in a a number of new use cases, which sorta we sort of, I guess, enhanced in in the 2.0 version. So the 2 point marshmallow 2.0 added the that pre pre and post processing API. It improved the symmetry between serialization and deserialization.
And now, we're trying to get out, Marshmallow 3, which is really how do I sum it up? I guess it's it's really focused on being strict in the right ways. So making strictness sort of the default without getting in the way. Have there been any particular changes
[00:16:04] Unknown:
in the Python language and library ecosystem that have either improved or added difficulties in terms of how you yourself build and use Marshmallow?
[00:16:15] Unknown:
Not too much, actually. Although, like I said, I mean, we Marshmallow started at really just as a way to serialize database backed objects for rest APIs, but it's really since we added that, deserialization and validation, it sort of evolved into something more general purpose. 1 trend that I have been seeing is to use Marshmallow as sort of a lower level target for building web frameworks on top of a micro framework. So rather than just being a a a library that you kinda add for serialization, it it's it's become a way to build sort of the Django REST framework equivalent for the other micro frameworks.
[00:16:53] Unknown:
And can you talk a bit more about some of the other ways that it's been used that you found to be particularly
[00:16:59] Unknown:
interesting or unexpected given your initial goals with the project? Yeah. I've seen a couple of projects that are using Marshmallow to underlie the implementation of an ORM. So I've actually seen a dyno DynamoDB and a mongodbmongoDBODM built using Marshmallow as sort of the the validation and serialization later. So I thought that was interesting. Some other use cases. I mentioned, validating configuration. So, Etsy has this this project for, creating airflow DAGs using YAML, and it uses Marshmallow to sort of validate the that YAML configuration. So 1 unexpected use case was 1 of my own with my WebRx project.
So WebRx is a library that basically can take request objects, parse the parse the pieces of them, so, you know, your form data or JSON data or, know, query strings. And then it allows you to sort of declaratively define your, request objects, request object pieces. So I I developed WebRx independently of Marshmallow, but it eventually became clear that they'd sort of do the same thing. Like, WebRx is just doing deserialization of request objects into application objects. So I actually, basically rewrote WebRx to use Marshmallow under the hood. So, yeah, that that was another interesting project that I've seen is from Lyft. They they built this sort of fork of toasted they called toasted marshmallow, which basically does just in time compilation of serialization, methods. So in the same way that, you know, Jinjo or Mako will take a template and compile it down to optimize Python code. Toasted marshmallow takes your objects and then sort of just in time compiles
[00:18:51] Unknown:
Python methods that efficiently serialize and deserialize objects. So it's a pretty interesting use case. Yeah. I remember reading the article that they wrote when they first released that, and, it was interesting looking at some of the optimizations that they made and just the the fact that they were dealing with so many different objects that that was even a requirement as far as the efficiencies that they gained in terms of being able to work with these objects and transferring them between the application layer and the network and IO layer. And so in terms of your own work on Marshmallow itself, what have you found to be some of the most interesting or complex or challenging lessons learned in the process of building the project and its community? I think I've run into
[00:19:36] Unknown:
most of the challenges that come with working on a volunteer project. So up until now, I was not using Marshmallow at my day job. Actually, this will likely change with my new position. But what that meant was that I worked on Marshmallow, in my own time and as I saw fit. And this had a lot of drawbacks and advantages. Like, on on 1 hand, I'm I'm not dogfooding it as much as someone who uses it in their day job, So I don't run into all of the same corner cases that other people might. But on the other hand, I I really have the liberty to focus on development of of pieces that I might not otherwise get to work on if I had, for example, deadlines. But, thankfully, I have 2 really great co maintainers right now, Jared and Jerome, who I believe do use Marshmallow in their day jobs, and they do run into all those corner cases that come along with running code in production. So, I think we complement each other well there. I I think another challenge I've run into is getting user feedback. As far as I know, there's no sort of Google Analytics for libraries.
So I've really had to sort of make a con conscious effort to, you know, figure out how how are people using a library. And sometimes this means emailing people who have written a blog post or things like that or just evaluating how similar libraries do certain things. So it it just takes that little bit of extra effort to, collect user feedback about an open source library.
[00:21:04] Unknown:
Yeah. That's a common thread with a lot of people who I've spoken with who work in open source, and it's an experience that I have with the podcast as well where you have this output that is write once, read many, and there's not any sort of barrier or authentication or identification layer of that read process. And, even once somebody's read it from 1 location, they may copy it to other locations, so there's no way to even ensure that the number of downloads that you're tracking is actually representative of the overall usage of that output. Yeah. And it's difficult to sort of figure out how people are you know, what what parts of the podcast or the library are people finding difficult or most interesting. It's it's difficult to to get to that level. Yeah. It's 1 of 1 of the benefits and the disadvantages of being open is that because there's no barrier to entry, it's easy for anybody to pick it up and make use of it. But at the same time, if they run into a stumbling block that they then decide they don't wanna use it anymore, they don't wanna listen anymore, there's no way for you to step in and say, oh, if you just do this, then it will make things easier. Or, you know, if you if they have a question and they're listening to the podcast of being able to clarify, because you're not there, you have no way of establishing that 2 way connection with them. You can't ensure that they have continued success and don't just decide to put it down and go on to something else. Yeah. Exactly. It's an unsolved problem, and I'm really interested,
[00:22:26] Unknown:
in anybody who has ideas of how how to solve that with when it comes to open source or, you know, any sort of open creative output. And
[00:22:35] Unknown:
in terms of keeping Marshmallow agnostic to any specific frameworks or object mappers, what have been some of the overall benefits of that approach and any drawbacks that you might have experienced in the process? Yeah. So I think 1 advantage
[00:22:50] Unknown:
that I kind of touched on was that it enables use cases that are that are outside of the web domain. So even though Marshmallow was built with HTTP APIs in mind, we've seen it used in a lot of different other context. And that also sort of allows Marshmallow to stay rather focused on what it's meant to do, and that therefore reduces maintenance bird burden, which does lead me to 1 drawback or, I guess, a consideration is that we really have to give extra thought about what goes into Marshmallow Core. So on 1 hand, you know, if you have n different web frameworks and ORMs that Marshmallow could integrate with, we don't wanna have to, you know, implement a a given feature n times. Like, that's just inefficient.
At the same time, we don't want to keep Marshmallow's API surface. We don't wanna increase that too much, so that can be a tricky balance. Yeah. It's definitely
[00:23:44] Unknown:
an interesting challenge because you want to be able to ensure that it continues to be useful and that people can leverage it for all their different use cases. But each time you add a feature or a new way of integrating, then it might break an existing integration that you don't know about or increase the burden of somebody who wants to get started with an integration because there are just more things for them to have to consider as they interface with their library or framework or object mapper. Exactly. Yeah. And as far as using Marshmallow, what are some of the edge cases that you and some of your users have come across that newcomers should be aware of? I think a lot of the issues
[00:24:25] Unknown:
newcomers have dealt with have to deal with the symmetry of serialization and deserialization. And earlier versions of marshmallow were actually not very good at this. Like, you'd you'd serialize an object with the schema. You'd find that you couldn't deserialize that data back to the same object. Marshmallow 2 improved on that quite a bit, and it we added options to account for use cases where the input and output keys may differ from the internal representation. So there's load from and, dump 2 parameters that allow you to specify these. And, actually, Marshmallow 3 improves on that even further by consolidating those into a single attribute called data key. So it meets that same use case.
[00:25:08] Unknown:
And are there any particular situations where you found that it doesn't really make sense for the additional overhead of defining and maintaining the schema for being able to serialize and deserialize from a given format or any cases where
[00:25:25] Unknown:
Marshmallow is just overkill for a particular use? I would say my advice on that is is the same as for any open source library, and that's to say, don't use it until you need it. There are definitely some micro microservices where you it would be just plain, overkill to to, define a schema class to, serialize your objects. So I'll just say, you know, don't use Marshmallow when until you actually need it. Once once you're at a point where you have, common format data formatting logic in more than 1 or 2 places, then that's when you, should start considering, like, do, should I use a a serialization library?
[00:26:04] Unknown:
And for cases where you have a, you know, a data source that might have variability in terms of the format of the object structure, but there is some sort of shared commonality between them. Is it possible to define a schema for the bits that you know are going to be unchanged, but still allow for that dynamicism in terms of maybe a sub attribute of the object that can just be rendered as a, you know, a regular JSON dot loads or XML load for creating that nested dictionary object without having to define that schema ahead of time while still having the advantage of the structure for the rest of the object. Sure. So I'm just thinking of cases where you maybe have a data output from any source where there are certain attributes that are guaranteed to be common across all the objects, but there might be some dynamic data that is either user generated or somehow randomized that you want to be able to capture the general structure of the object, but then still be able to load in that dynamic or changing piece of data without having to define the schema ahead of time or track those changes in your class object each in each case. I see. Yeah. I I have seen some some use cases like that where you actually have marshmallow field classes that are generated dynamically
[00:27:18] Unknown:
either at runtime or generated from actually some other data source. So, you could have, schema classes generated from, like, let's say, a JSON schema or even a SQL table. So yeah. I mean, I I think that's that's a that's a use case that could be that could be met with Marshmallow.
[00:27:36] Unknown:
And what are some of the features that you've found to be relatively unknown in the user community of Marshmallow that you yourself find particularly useful that you'd like to tell other people about, particularly if they haven't leveraged them themselves?
[00:27:51] Unknown:
I think a lot of the questions that come up on the issue tracker or on Stack Overflow where the question starts with, like, can marshmallow do x, I think a lot of those questions are answered by the pre and post processing decorators. Like, those add a lot of flexibility. So that's where you'd you'd plug in your sort of custom data formatting or logic. So that that meets a lot of use cases that come up. I think another feature that users might not know about is, the context, API. So schemas have this context dictionary attached to them that allows you to dynamically change the behavior based on the context in which a schema is used. So I don't know. Let's say you have you're instantiating a schema within a Flask view, and based on some property of the request, you wanna change the way something serialized. So you can set properties on that context dictionary, and that propagates to all of the fields within that within a schema. So you can change the way that those those fields format the data. Another 1 I can think of is you can override how to access attributes during serialization by overriding, the get attribute method of a schema. And this one's kind of buried in the docs, so I think not a lot of people know about it. But by by default, Marshmallow uses this general purpose get attribute function that is meant to work with most object mapping and collection types. But most users don't need such general purpose attribute access because most typically, you're serializing 1 category of object, like your model objects, for example. So an easy way to get a performance improvement is to just implement the get get attribute that works with your application's data types. You mentioned now that you're working on the 3 0 release of Marshmallow,
[00:29:33] Unknown:
but beyond that, what are some of the other features or improvements that you have planned for the future of Marshmallow?
[00:29:40] Unknown:
Yeah. So I think after Marshmallow 3, we really wanna focus on a few things. And 1 of those is performance. So up to version 3, we've really focused on getting the API right, and it's at a place that I'm pretty happy with now. So in future releases, I'd really like to focus our efforts on profiling and figuring out where we can make things faster. Marshmallow 3 will also be the last version to support Python 2. So version 4 will be Python 3 only. So we'll definitely be taking advantage of new syntax features, such as type annotations, f strings, and things like that within the marshmallow code base itself. Now whether we decide to embrace some of those Python 3 features, such as, like, data data classes and annotations as a way to generate schemas, I think that still remains to be discussed, but it's definitely on the table. But I would say I really hope that Marshmallow continues to actually get even leaner than it already is. So Marshmallow has a pretty small API surface. But if anything, I in the future, I'd rather slim down that API surface
[00:30:43] Unknown:
than, add new features. So that that's the plan moving forward. And are there any other aspects of the Marshmallow project or your work with building it and the community or serialization and deserialization
[00:30:56] Unknown:
in general and data validation that you think we should discuss further before we close out the show? I have learned a lot about maintaining an open source project in general during the while building Marshmallow. And 1 of those is the importance of just being friendly in the open source community. So right now, actually, my activities are really focused on sort of managing the project, attracting new contributors, and retaining new contributors. So a big part of that is making the project fun to work on or at the at the very least, making it not a negative experience. I remember GitHub did a survey a few years ago, and 1 of the findings from that was that people who experienced or witnessed a negative interaction stopped contributing because of it. So it's not even just about people acting negatively towards you. It's also just seeing a negative interaction towards others that made potential contributors want to leave. So we really need to be mindful of how we conduct ourselves in these communities.
[00:31:52] Unknown:
Yeah. It's definitely an important piece of advice, particularly because you don't have that context of visual social cues that are important when we're dealing with each other, and we have to try and just embody that in very 1 dimensional communication for the most part. So just being, as you said, mindful of how we do that is, important to ensure the continued success of our various projects and communities that we choose to support. Alright. So for anybody who wants to follow along with you or get in touch and follow the work that you're up to, I'll have you add your preferred contact information to the show notes. And with that, I'll move us into the picks. And this week, I'm going to choose the Sherlock series that the BBC put out a little while ago. I recently started watching through that with my wife, and we've been enjoying that pretty immensely. So some great acting, great storylines. Sherlock is a character that has been able to maintain relevance even into the modern age, so it's interesting seeing a sort of modern reimagining of Sherlock and, his overall interactions. So for anybody who's looking for a new show to watch or check out for a little bit, definitely recommend that 1. And with that, I'll pass it to you, Steven. Do you have any picks this week? Yeah. I can recommend a podcast called Greater Than Code, which is a podcast about the human side of software development, I guess. So they take a look at the social and psychological
[00:33:11] Unknown:
aspects of the tech industry. And it's really interesting that the panelists always bring really unique perspectives on things, and
[00:33:19] Unknown:
it's really mind expanding to listen to. Alright. I'll have to, take a look at that 1. So thank you very much for taking the time today to talk about the work that you've done with Marshmallow and for building the project. I found value a little bit on my own. So, appreciate all of that, and I hope you enjoy the rest of your day. Thanks for having me. This
[00:33:42] Unknown:
was fun.
Introduction and Sponsor Messages
Interview with Stephen Loria
Stephen Loria's Background and Introduction to Python
Introduction to Marshmallow
Comparison with Other Serialization Libraries
Serialization Formats and Use Cases
Getting Started with Marshmallow
Implementation and Evolution of Marshmallow
Unexpected Use Cases and Community Contributions
Challenges and Lessons Learned
Framework Agnosticism and Benefits
Edge Cases and Considerations
Advanced Features and Lesser-Known Capabilities
Future Plans for Marshmallow
Maintaining an Open Source Project
Contact Information and Picks