Summary
Movies are magic, and Python is part of what makes that magic possible. We go behind the curtain this week with Dhruv Govil to learn about how Python gets used to bring a movie from concept to completion. He shares the story of how he got started in film, the tools that he uses day to day, and some resources for further learning.
Preface
- Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
- I would like to thank everyone who supports us on Patreon. Your contributions help to make the show sustainable.
- When you’re ready to launch your next project you’ll need somewhere to deploy it. Check out Linode at www.podastinit.com/linode?utm_source=rss&utm_medium=rss and get a $20 credit to try out their fast and reliable Linux virtual servers for running your awesome app.
- Visit the site to subscribe to the show, sign up for the newsletter, read the show notes, and get in touch.
- To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
- Your host as usual is Tobias Macey and this week I am joined by Dhruv Govil to talk about how Python is used for making movies.
Interview
- Introductions
- How did you get introduced to Python?
- How did you get started in the film-making business?
- What are some of the ways that Python is used in the process of bringing a movie to completion?
- How much of the overall pipeline processing happens in Python vs just being used as a means of wiring together other programs.
- How much of the code that gets written is reusable between different projects?
- What is involved in testing data assets when they are submitted to the pipeline for the open format conversion process?
- What are some of the libraries that you have found to be most useful in your day-to-day work?
- Why do you think that Python is so widely used in the film industry and are there any other languages that you see being used in a similar manner?
- What are some of the areas where Python is used that you were most surprised by?
- Are there any portions of the process where you would like to be able to use Python but are unable due to performance or platform constraints?
- What are some of the most interesting projects that you have worked on and which are you most proud of?
- How does the work that is done by developers and technical contributors get reflected in the final credits?
- For anyone who is interested in working in the film industry as a technical contributor what advice do you have?
Keep In Touch
- Dhruv
- Website
- @DhruvGovil on Twitter
- dgovil on GitHub
Picks
- Tobias
- Dhruv
Links
- Udemy: Python for MayaUdemy
- Vancouver Film School
- Guardians of the Galaxy
- Cloudy w/ chance meatballs 2
- Blog Post: Python For Feature Film
- PyQT
- PySide
- Autodesk Maya
- Katana
- Nuke
- Cython
- Rez
- Alembic Geometry Storage Format
- Pixar Universal Scene Description
- Pyblish
- Open Color IO
- Edge of Tomorrow
- PyOpenGL
- Kraken
- Fabric Engine
- SIGGRAPH Convention
- Ray Tracing In A Weekend
- Mathematics for Computer Graphics
- Blender
The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA
[00:00:14]
Unknown:
Hello, and welcome to podcast dot in it, the podcast about Python and the people who make it great. I would like to thank everyone who supports us on Patreon. Your contributions help to make the show sustainable. When you're ready to launch your next project, you'll need somewhere to deploy it, so you should check out linode at ww w.podcastinnit.com/linode and get a $20 credit to try out their fast and reliable Linux virtual servers for running your app or experimenting with something that you hear about on the show. You can visit the site at www.podcastinit.com to subscribe to the show, sign up for the newsletter, read the show notes, and get in touch. To help other people find the show, please leave a review on Itunes or Google Play Music, tell your friends and coworkers, and share it on social media.
Your host as usual is Tobias Macy. And this week, I'm joined by Dhruv Govil to talk about how Python is used for making movies. So, Dhruv, could you please introduce yourself?
[00:01:01] Unknown:
Hi. I'm Dhruv Govil. I'm a lead pipeline developer working in animation and visual effects for feature films. I've been in this industry for about 5 years now. I'm currently working on the new Spider man movie, and I worked on movies like Guardians of the Galaxy and Claudio of the Transcend Meatballs 2. I also teach a course on Udemy, teaching Python for artists in the CG industry.
[00:01:26] Unknown:
And do you remember how you first got introduced to Python?
[00:01:28] Unknown:
I was an animation student, and I needed some tools to just work faster. And usually, I'd go on, websites like Creative Crash, where other TD's, developers would put up their scripts. But I couldn't find the exact tools that I wanted. So I figured, this can't be that hard to do, and I started teaching myself Python in my free time. Then I ended up making a few tools. I put them online. Eventually other people started using my tools, and were requesting more. So I slowly transitioned from animating to just writing tools in Python. I think the technical challenge of coding was more appealing to me than the animation itself.
[00:02:19] Unknown:
It sounds like that's how you first made your way into a full time role working in the filmmaking business? Or did you have a different entry point to get into the industry?
[00:02:28] Unknown:
No. That was pretty much it. I when I was younger, I saw, you know, the first Lord of the Rings movie, and that really just blew my mind. I think I was, like, 12 or something about then. And then I started just teaching myself, you know, CG software. When I graduated high school, I decided to move to Vancouver from India, because, Vancouver Film School at the time was 1 of the best film schools. After I graduated, I just got a job in film doing mostly camera work, so not really programming, but doing digital cameras and digital sets. While I was doing that, I spent a lot of time programming to make my workflow faster.
Eventually, the studio I was at, was about to close down, so I made the jump to my current studio at Sony as a full time developer. So I guess it's been a pretty straightforward path. I knew I wanted to work in film, and it, so far has worked out alright.
[00:03:29] Unknown:
Do you think that the work that you did in school prepared you well for working in industry? Or do you think that you would have been equally well suited if you had gotten into industry without getting the degree first?
[00:03:40] Unknown:
I think going through school helped a bit in terms of networking, for sure, because a lot of the jobs we get in this industry are just who you know at other studios and them recommending you. So that definitely helps. In terms of knowledge, my school didn't really teach me programming, but it was good to work with other artists because I would make tools for them, or I would help them learn 3 d software. So that skill set is something I don't think I'd have learned by myself if I hadn't gone to film school. But at the same time, none of the skills I currently use were taught to me in film school.
[00:04:21] Unknown:
And what are some of the ways that Python is used in the process of bringing movie from concept to completion?
[00:04:28] Unknown:
We use Python pretty much from start to finish on movies. We use it to import all the footage from the cameras, and every process in between uses Python. And when we deliver to the clients, we are still using Python to package everything up. The key part of the studio workflow is called the pipeline, where we take data from 1 department, like animation, and transfer it to the next department down, like lighting and then compositing, that's pretty much completely written in Python. Other places where we use Python is to make tools for artists in each of the departments. So we have tools completely built in Python again that will automatically generate character skeletons and their control rigs, which are the things that animators use to animate.
We have Python scripts to set up cloth and hair simulations, or ocean simulations, and things like that. And we use Python for the majority of our user interfaces. So a lot of artists like to work in node graphs or other technical views, and we use Python with PyQ to or PySide to develop those UIs.
[00:05:45] Unknown:
And how much of the actual pipeline processing happens in Python versus Python just being used as a means of transporting the information from 1 program to another?
[00:05:55] Unknown:
A lot of our pipeline code involves manipulating the host programs. So Python is usually an interpreter within a larger program like Autodesk Maya or Katana and Nuked by the Foundry. Python usually comprises the majority of the logic, whereas the host applications usually handle the actual processing of the data. This is, actually probably why Python is so popular, because it's a really great glue language, in that you can have all the performance critical code in c or c plus plus. And then you can use Python, which is really easy to understand and easy to write, controlling all the actual logic of how things get processed.
We do have a lot of code that is real time processing, completely written in Python. But again, it does use CLibs like NumPy under the hood.
[00:06:53] Unknown:
And does Cython get used much as well for increasing the performance of the Python code?
[00:06:58] Unknown:
Cython, unfortunately, isn't that popular in this industry. I don't think a lot of developers have a lot of familiarity with it. I have heard of some people using it to get, that critical performance, but I think most people prefer writing
[00:07:16] Unknown:
an extension in c and then using, like, a CFFI to interact with Python and vice versa. Couple of questions from there is, 1, I guess, how would you characterize the volume of code that goes into a given film project, And how much of that code is reused across projects versus being purpose built for a particular film?
[00:07:37] Unknown:
So the amount of code that gets written per project varies on the size of the project. Some of our movies are, you know, like just 200 shots. Some of them are 2, 000 shots. The larger movies often have more code involved, because they'll often have more demanding needs. And a lot of the development is done on a particular project, but then when the project is done, we usually ship that back to the facility level, so that other shows can use it. In terms of how much code is written, I would say we usually are working in the tens of thousands of lines of code per project.
Each, tool we build is usually, at a minimum, at least 5, 000 lines or so, just because of the amount of work that's needed. And the amount of code that does get reused between projects is pretty high. I wanna say about 80% of our code is stored on a common level that all projects can pull from. This is actually something I like to have because it means you can spin up new projects really quickly without having to reinvent the wheel. Some other studios prefer that every show is its own beast, and it does let them be a little more agile in their development, because they don't have to worry about affecting another project.
But at the same time, there's a lot of initial overhead to get to a workable point.
[00:09:15] Unknown:
And is there a lot of standard software engineering practices that gets used in the movie industry? Or is it a lot of sort of ad hoc scripting just to get something done quickly and get it out the door?
[00:09:29] Unknown:
It's a bit of both. A lot of the studios' pipelines were not necessarily built by actual developers, but rather artists who learned to code. So the problem is that a lot of the legacy code doesn't necessarily follow best practices for software development. But there's a recent shift, especially just because we have to work so quickly now to try and become more efficient, and that involves adopting a lot of the common practices that stand alone developers would use. So we're recently seeing a shift to moving all our code to Git, whereas earlier, the code may have been unversioned or was under SVN.
We're finally seeing a lot of code review in the studios, which is something I'm really grateful for, as opposed to just people doing the wild west, like checking in the code, releasing it to the wild, and then hoping it doesn't break a few hundred artists' workflow. And we're finally moving towards having version packages so that each project can control its release cycle. We're using projects like Res, which I'm not sure a lot of Python users are familiar with, but it's gaining popularity in the visual effects industry because it allows you to control your development and deployment versioning very easily inside of 1 package manager.
[00:10:53] Unknown:
And while I was reading through the blog post that you wrote about the ways that Python is used in the movie industry, 1 of the things that I noticed was the fact that you have a method for testing the data assets that get submitted the pipeline for the, open format conversion process. So I'm just curious what that testing process looks like and some of the tools that you use to, achieve that.
[00:11:17] Unknown:
Yeah. So a lot of studios will have their own solutions for this, but they generally follow the same guidelines where when an artist is done working in their department, they'll open a published UI of some sort, which will list the assets and their scenes that can be published for the next department down to pick up. When so then the artist will select which assets they wanna send through. And we run a suite of tests on each of those assets. So for example, if we're publishing a model, we'll check for things like non manifold geometry or lamina faces, which are basically big no no's when you're trying to render geometry.
Most render engines can't handle lamina faces, which is when 2 polygons on a piece of geometry are right on top of each other. Similarly, we have tests to make sure that animators, for example, don't have disabled animation that might kick in. And we end up writing out incorrect geometry for the next department to read. A lot of the studios have developed their own open source formats to allow for quick interchange between the various applications. So at ImageWorks, we co developed a format called Alembic with ILM. And Alembic is a geometry storage format that is used by pretty much every CG application today.
Similarly, Pixar has now put out USD, which is another open source format for storing scene hierarchies. And, generally, they're pretty flexible, but we need to make sure our data is compliant with them. So that's kinda what our testing entails. There is an open source project called Pyblish, written by Markus Ottuson, which is pretty similar to what you would find in most modern day studios. And again, it follows the same basic idea of you choose which assets you wanna publish, and per asset, it'll start running tests before it starts writing out to 1 of the open source formats.
[00:13:36] Unknown:
And you've mentioned a number of different libraries already, but what are some of the other libraries that you found to be most useful in your day to day work that people may not have heard of?
[00:13:46] Unknown:
The most common ones are NumPy and PyQ, which are fairly common even outside the CG industry. NumPy especially, because it lets us do heavy mathematical computation, while escaping Python's overhead and performance limitations. And pycute, just because it's such a great UI binding to the cute library, It just lets anyone spin up a new UI, very complex UIs in fact, without having to write a ton of code. We do have some less common libraries, especially for color correction. For example, we have OCIO, which is the open color IO library, which handles color transformations.
Other common libraries include, Alembic and USD, which I mentioned. But those are really popular, obviously, to write data out and share them between applications and studios. Other than that, for the most part, all our libraries are fairly common with other Python developers. We mostly just stick to the standard library and use things like YAML, JSON. Nothing too extravagant.
[00:15:02] Unknown:
You mentioned that 1 of the reasons that Python is so widely used in the industry is because of the fact it's already integrated to a lot of the 3 d publishing tools. I'm wondering if there are any other aspects of the language that you think lend itself well to working with film. And what are some of the other languages that you see being used in a similar manner?
[00:15:24] Unknown:
Python is probably the best language for film in my opinion, just because it's so easy to read and use, even for beginners. Like I said earlier, a lot of the early development in this industry was done by artists, not necessarily by actual developers. So they often attach themselves to easier to use languages, like Python. At the time, there was a lot of Perl. Thankfully, that's gone. But it was just easy for someone to jump in, write some code, get it to do what they want, and get out. The other nice thing that Python has going for it, other than just being easy to use, is that it has such a large community on the Internet.
It's really hard to find an issue in Python, where you can't just go on Stack Overflow and find an answer. The other nice thing about Python is CPython is a really great reference implementation, and it's really easy to write extensions in C and use the CFFI. So this makes it really nice for application developers to just stick it in their programs. Like I said before, Python, a lot of studios hedge their bets on Perl. Unfortunately, Perl is kind of cumbersome to use. It's not the most intuitive language, in my opinion. But it does offer a lot of scriptability, which I think is what people were looking for. And at the time, Python wasn't as on the radar as it is today.
But as soon as Python gained traction, we've started ditching Perl almost completely. Other popular languages that we're using now are c sharp. It's more popular in the game development world rather than film because a lot of game engines integrate it. For example, Unity is the primary scripting language is c sharp. Other languages that we use a lot of are c plus plus for our highly performing code, where we need threading or real time interaction with data. And recently, a popular language is Lua, because it has a really low overhead. And you can spin up a Lua interpreter with very very minimal memory usage.
Do what you want and then get out, whereas Python tends to be a little heavier and a little slower. And finally, the other language that is gaining a bit of traction recently is JavaScript, especially with the advent of Node. Js. Because in addition to the 3 d work we do, we do have a lot of SysOps and DevOp roles where they have to develop websites for the artist to manage their shots and see what the statuses are. We have industry equivalents to things like Jira and other ticket management software, but primarily aimed towards artists. So, yeah, those are pretty much the main languages today. I'm sure in the future, things like Rust or Lisp might even make a comeback.
But Rust is definitely something that people are really excited about.
[00:18:45] Unknown:
Yeah. I'm definitely pretty interested in Rust myself. I have to say, I'm a little surprised that you didn't, have any mention of Ruby being used because, generally, in terms of dynamic languages, it's often referred to in a in almost the same breath as Python as far as comparable languages. But also given the age of the film industry, it's understandable that Perl was a, early entrant to that arena as well.
[00:19:11] Unknown:
Yeah. Ruby was popular with a lot of our website developers for our internal web pages and things like that. I don't think it ever gained traction, really, among artists for some reason. I wasn't around back when they were deciding these languages, really. I did talk to the people who decided that Pearl was a good idea. And I think Pearl kinda shocked them at the time at how it didn't scale up to the artist's needs in terms of usability. So they kinda stayed away from that hemisphere. And then when Python gained traction, they were just like, this is it. Let's just stick with this since it works. We're also a fairly slow moving industry.
We don't experiment a lot in terms of the programming languages we use. So if we find something that works, we kind of stick with it. For example, we're not even using Python 3 at the moment. Everything we do is on Python 2.7, which I hope changes in the next few years, but as of now we're stuck on a fairly outdated version of Python.
[00:20:26] Unknown:
Yeah. Python 27 is still maintaining a lot of gravity, though that has been shifting recently where more and more projects are becoming either Python 3 compatible or Python 3 only as more as more modules get added to the standard library. But it's hard to give up an existing code base that's been worked on over the you know, over a number of years in Python 2 because it can be difficult to make that migration.
[00:20:54] Unknown:
Yeah. Totally. It's a hard migration to make. So I understand why people are sticking to Python 2.7. There are things I'd love to use from Python 3, like type annotations, because our libraries get quite large. As well as async, because I find myself writing more and more network libraries recently. And not having async is kind of a burden when developing that.
[00:21:23] Unknown:
Yeah. Are some capabilities for bringing some of that back, but, yeah, it's not as well implemented as it was done natively in Python 3. So for instance, there's the Tulip library for async IO in Python 27, and you can use mypy for annotating Python 27 code, but it has to be done in a separate file. So it adds a bit of extra friction to the process.
[00:21:48] Unknown:
Yeah. It's not as intuitive. But, yeah, I do use, MyPy quite a bit and PyCharm's type annotations, which are quite nice. But it would be great 1 day to just move to Python 3 and call it a day.
[00:22:05] Unknown:
And as you work on more projects, what are some of the areas where you have found Python being used that you were surprised by that you didn't expect to see it there?
[00:22:16] Unknown:
So Python, in my mind, isn't particularly suited to real time graphics. But a lot of my coworkers have surprised me by using it for that. In, Edger Tomorrow, for example, a coworker made a tendril rig for these aliens with these long tentacles. And it was completely written in Python, and it performed really well. Part of that is because a lot of his code calls out to c plus plus code in Autodesk Maya. But even then, it was far more performant than I thought it would be. Another coworker recently made a proof of concept tool with where he could do 3 d sketching.
So an artist could draw in 3 d space, kind of similar to the grease pencil tool in Blender. And all he really used was Python, NumPy, and some pyopengl libraries. And it performed really, really well. And that was really impressive to me, because in my head, anytime I wanna go real time, I have to go to c plus plus. But here are people using Python as a prototyping tool, And it works so well that most people wouldn't even know if you didn't convert it to C plus plus later. The other thing that surprised me when I started in this industry was how much Python was used for user interfaces. A lot of our user interfaces are just done with PyQute.
We have these massive node graphs with millions of nodes sometimes that are just done completely in Python. We have custom bezier curve widgets and other things, Again done in PyQute, because it's so easy for even non technical artists to put together a really complex UI. You can see a great example of an advanced node based UI in a tool called Kraken, which is a rigging front end to the fabric CG platform. Fabric is another framework of sorts for building CG software that is software agnostic, in that you build it for fabric and can plug it in to other applications.
But a lot of the UI for Kraken was done in PyQt, and it performed so well, you'd never really know that it was in C plus plus Or you would know that it was done without threading out the UI from the underlying logic.
[00:24:59] Unknown:
So what are some of the most interesting or exciting projects that you've worked on, and which are the ones that you're most proud of?
[00:25:06] Unknown:
When I was at my first studio, I was using Python a bit to increase my workflow speed. The studio had its own language called Parsley, which wasn't really used anywhere else since they had developed it. And it was kind of a pain to use. So since I was learning Python, I decided to try and make a little IDE for it using Python and PyQt. So I made a little IDE with a built in linter for that language. I also then tried to make a dynamic binding in Python to their language, and made a little interactive shell, again in Python. So it meant that I didn't have to necessarily use their language directly, and I could code completely in Python.
And it worked surprisingly well. And I think that's 1 of the projects I was most proud of, because I was pretty much a beginner to the language. But even then, it would this was a fairly advanced task, I think. And it held up pretty decently. Another project I'm actually excited about is a tool I made in Python again for compositing multiple artists' work together, which is a tool that I built, several feature films ago, but has been used on about 5 feature films now. And I'll be presenting this tool at SIGGRAPH, which is a CG convention later this year in LA.
But basically what the tool lets me do is it lets us split up a single shot between multiple artists, which traditionally was very difficult to do. But now each artist can work on their section of a single shot. They'll render out their images, which is fairly quick for them to do, and my tool will take all their images, take their depth data, which tells you how far in-depth each pixel is, and it'll combine this together so that each artist can get a fairly quick preview of what their shot looks like, combined with every other artist's work. So this has been pretty instrumental for a lot of our shots at Sony, where, you know, where, you know, movies are getting bigger every day.
So you have so you used to have movies where you'd have maybe 2 or 3 characters in a shot. Now you have movies with 300, 400 characters in a shot. And it's really cool seeing my tool let artists manage this kind of extended workflow.
[00:27:55] Unknown:
And how does the work that is done by developers and technical contributors on films get reflected in the final credits?
[00:28:02] Unknown:
So pipeline developers like myself are crude to the individual projects. So we get credited as the pipeline department for the films we work on. Or if the film doesn't have department divisions, we just get clubbed in with everyone else. We also have a group of facility level developers who aren't crewed to a specific show, and they get credited on every 1 of our animated films. They don't usually get credit for the visual effects films, since credit space is more limited there. Occasionally, the films have more limited credit space, so a lot of us don't get credited on those particular films, since credits are usually prioritized by the time spent on the project and your seniority.
But for the most part, every developer on a show will get credited, and all the facility level developers get their little bit of screen time as well.
[00:28:58] Unknown:
And how important is it to show up in the credits in terms of career advancement or mobility across teams and across employers?
[00:29:07] Unknown:
It's, not actually very important at all to be in the credits. A lot of people go uncredited all the time. Credits are more just something we have, so we can tell our friends and family, like, look. Hey. We actually worked on this movie. And, you know, people stay by after the end of a movie to see which of their friends worked on a particular movie. I have a lot of films where I wasn't credited, but they're still listed on my IMDB page. And most employers will look at your demo reel, where you still may have shots from movies that you weren't credited on.
Or in the case of developers who won't have a demo reel, since you can't really show code, They will just take you at your word that you worked on this particular project, and they'll definitely contact your old employer just to make sure that everything is above board. But, yeah, credits aren't super important other than just a nice memento from the movie.
[00:30:10] Unknown:
So for anybody who's interested in getting involved in that kind of work and working as a technical contributor for in the film industry, do you have any particular advice as far as know, material that they should become familiar with or any sort of degree program that they might wanna get involved in or just any other general advice?
[00:30:28] Unknown:
So I think that depends on the person. There are usually 3 kinds of developers I run into in this industry. The first are people who are a little more hardcore and they're interested in real time graphics or rendering. And those are people who tend to be in more low level languages, like C plus plus or Rust or something like that. And they understand a lot of math, Open GL, multi threading, things like that. For those people, I recommend books like mathematics for computer graphics or ray tracing in a weekend. It's a lot more technical, but it is something I know a lot of developers like doing.
For people who are more interested in being more involved with the artist, and therefore also being more involved on a particular movie, I think the best way is to start learning a 3 d application. There are educational versions of Autodesk Maya that are completely free. And Blender, which is a free open source 3 d application. I usually recommend people work through a few simple tutorials to understand the artist's workflows. And once they understand the artist's workflow a bit, they'll understand what common issues are and what kind of development is needed.
It also really helps when you're building tools for an artist, if you understand where they're coming from, rather than developing sort of in a vacuum. And last, but definitely not least, we have a lot of supporting players, like the guys who write the network infrastructure tools, who manage our databases, and generally more SysOps, DevOps kind of roles. So they use a lot of Python, and I think that's probably the most compatible with a lot of Python developers who didn't start out in film, because it lets you reuse a lot of your skills, and you still get to work in the film industry without having to relearn all these new skills, learning 3 d, etc. You can be a web developer working on a movie. You'll still get credited, It'll still be kinda cool, but you don't have to relearn 3 d and things like that.
[00:32:54] Unknown:
And, your mention of the databases that are being used made me wonder, what are the typical kinds of volumes of data that you're dealing with? Is it generally fairly substantial, or is it highly variant depending on the particular type and length of movie?
[00:33:10] Unknown:
It's, pretty variant on the type of movie. So the databases themselves are fairly lightweight, because we just use them to point to actual geometry data on disk, but it's not unheard of to be for even a single shot to be writing multiple terabytes of data. I'm sure we've hit a petabyte of data on some show somewhere, but I know myself I've generated terabytes and terabytes of data in the hundreds of terabytes range for a particular shot. So if you multiply that by, let's say, a 1000 shots in a movie, you're already probably at a petabyte per movie.
[00:33:53] Unknown:
I'm just glad I don't have to deal with that much data in my day job.
[00:33:57] Unknown:
Yeah, it's kinda nuts. And they also have to back up every movie that we've ever done in case we need to go back to something on there. So they have these massive disk storage. They have data from, you know, the early nineties that are still on tape that they occasionally have to take out and load up, which is kinda crazy, but it's cool that they're able to do that.
[00:34:25] Unknown:
Yeah. It's definitely seems to be a pretty complex industry with a lot of different moving parts. And despite the fact that it is, as you say, somewhat of a slow moving industry, I'm sure that there are still new things being introduced on a regular basis as artists and technical crews continue to push the limits of what's possible in film.
[00:34:45] Unknown:
Yeah. There's, there's definitely a lot of innovation going on. I guess when I say it's a slow moving industry, we are slow moving on our base layer of technology because we don't wanna change too much down there. But at the high level, we have a lot of innovation going on. Right now, something that's very popular is machine learning. I'm reading about other studios using machine learning to speed up simulations. Whereas earlier, we'd brute force a simulation where every vertice in a mesh in a geometry mesh would be calculated colliding with every other point on that geometry.
We're now able to say, okay, the artist changed this tiny parameter. How will this actually affect it, and how can we reuse the data we've already generated in the most efficient way possible? They're starting to use machine learning to aid animators to get better muscle deformations and better posing, so that the animator doesn't have to think about how muscles move. There's a lot of innovation in rendering technology right now, as we're shifting from using mainly CPU's to shifting onto the GPU. So shots that used to take 100 and 100 of hours to render 1 single frame can now render in a fraction of the time.
And of course, 1 of the biggest areas of innovation right now is VR. Virtual reality is becoming a huge part of the film industry now in not so obvious ways, but lots of movies will have supplementary content with virtual reality. So traditionally films would have tons of resources at their disposal because we never had to worry about real time. It's okay for a frame to take 200 hours to render in film, but for VR, you need to render a shot within a few milliseconds. So there's definitely a lot of development going into getting our shots and our assets working in much more constrained resources, whether we're shifting things to the GPU or shifting things to a lot of parallel evaluation, it's definitely gonna be a big push in the next few years to make our data much faster to process.
[00:37:27] Unknown:
So are there any questions or topics that we didn't cover that you think we should talk about before we start to close out the show?
[00:37:35] Unknown:
Nothing really off the top of my head. Your questions were pretty thorough, I think.
[00:37:40] Unknown:
So for anybody who wants to get in touch with you or follow what you're up to, I'll have you add your contact information to the show notes. And so with that, I'll bring us to the picks. And my pick this week is going to be the Firefox browser for Android. I started using that recently because a lot of Android apps have switched to having the embedded Chrome view. So if you're, for instance, loading you know, reading a newsletter in your Gmail app and then you try to click a link, it will open an embedded browser as opposed to opening it in another window. But if you use Firefox, you can actually set it so that it will just add the link to a queue. So you can just open a bunch of links at once, and then when you open the browser, it will pull them all in at the same time. So it's made it a lot easier for me to, you know, go through newsletters to keep up with what's going on in Python and other technical areas and, just has a lot of nice little design aspects to it that, give it an edge over the Chrome browser for Android, in my opinion. So if you haven't at least checked it out, I definitely recommend downloading it and playing around to see if you enjoy it as much as I do. And with that, I'll pass it to you. Do you have any picks for us today?
[00:38:50] Unknown:
Yeah. Something I find really cool recently is I've been using a lot of VR, to demo to my friends. I bought a virtual reality headset last year. And surprisingly, 1 of the things that people find really cool is Google Earth in VR. And Google recently had a giant update, so it's really impressive. You can fly around the world in Google Earth. You can go visit the Grand Canyon. You can go to France. You can go to India. And you can walk around. And it sounds kinda dumb, maybe, when someone's describing it, but just actually being able to see it and look at places at scale is something that I think everyone I've demoed it to found a really compelling experience.
So if anyone ever has access to a virtual reality headset, whether it's on a phone or an HTC Vive or an Oculus Rift, I definitely think Google Earth is surprisingly well worth checking out.
[00:40:05] Unknown:
Yeah. That definitely sounds really interesting being able to have sort of a first person perspective into it because it's all well and good to look at it on your computer screen. But as you said, it's not, it's not you're not viewing it at the appropriate scale and magnitude for being able to get a sort of impactful experience from it.
[00:40:24] Unknown:
Yeah. Definitely. I think that's 1 of the crazy things for virtual reality is, even if the graphics or the visual fidelity isn't that high, just being able to see things at a life size scale is really breathtaking to a lot of people. And it's something that a 2 d monitor just really can't convey to anyone just watching.
[00:40:49] Unknown:
Alright. Well, I appreciate you taking the time out of your day to tell me more about how Python is used in the film industry. It's definitely an interesting area and 1 that I haven't had any personal experience in, but it certainly seems like there's a lot of worthwhile challenges going on there. So thank you for that, and I hope you enjoy the rest of your evening.
[00:41:11] Unknown:
Cool. Thank you for having me on here.
Hello, and welcome to podcast dot in it, the podcast about Python and the people who make it great. I would like to thank everyone who supports us on Patreon. Your contributions help to make the show sustainable. When you're ready to launch your next project, you'll need somewhere to deploy it, so you should check out linode at ww w.podcastinnit.com/linode and get a $20 credit to try out their fast and reliable Linux virtual servers for running your app or experimenting with something that you hear about on the show. You can visit the site at www.podcastinit.com to subscribe to the show, sign up for the newsletter, read the show notes, and get in touch. To help other people find the show, please leave a review on Itunes or Google Play Music, tell your friends and coworkers, and share it on social media.
Your host as usual is Tobias Macy. And this week, I'm joined by Dhruv Govil to talk about how Python is used for making movies. So, Dhruv, could you please introduce yourself?
[00:01:01] Unknown:
Hi. I'm Dhruv Govil. I'm a lead pipeline developer working in animation and visual effects for feature films. I've been in this industry for about 5 years now. I'm currently working on the new Spider man movie, and I worked on movies like Guardians of the Galaxy and Claudio of the Transcend Meatballs 2. I also teach a course on Udemy, teaching Python for artists in the CG industry.
[00:01:26] Unknown:
And do you remember how you first got introduced to Python?
[00:01:28] Unknown:
I was an animation student, and I needed some tools to just work faster. And usually, I'd go on, websites like Creative Crash, where other TD's, developers would put up their scripts. But I couldn't find the exact tools that I wanted. So I figured, this can't be that hard to do, and I started teaching myself Python in my free time. Then I ended up making a few tools. I put them online. Eventually other people started using my tools, and were requesting more. So I slowly transitioned from animating to just writing tools in Python. I think the technical challenge of coding was more appealing to me than the animation itself.
[00:02:19] Unknown:
It sounds like that's how you first made your way into a full time role working in the filmmaking business? Or did you have a different entry point to get into the industry?
[00:02:28] Unknown:
No. That was pretty much it. I when I was younger, I saw, you know, the first Lord of the Rings movie, and that really just blew my mind. I think I was, like, 12 or something about then. And then I started just teaching myself, you know, CG software. When I graduated high school, I decided to move to Vancouver from India, because, Vancouver Film School at the time was 1 of the best film schools. After I graduated, I just got a job in film doing mostly camera work, so not really programming, but doing digital cameras and digital sets. While I was doing that, I spent a lot of time programming to make my workflow faster.
Eventually, the studio I was at, was about to close down, so I made the jump to my current studio at Sony as a full time developer. So I guess it's been a pretty straightforward path. I knew I wanted to work in film, and it, so far has worked out alright.
[00:03:29] Unknown:
Do you think that the work that you did in school prepared you well for working in industry? Or do you think that you would have been equally well suited if you had gotten into industry without getting the degree first?
[00:03:40] Unknown:
I think going through school helped a bit in terms of networking, for sure, because a lot of the jobs we get in this industry are just who you know at other studios and them recommending you. So that definitely helps. In terms of knowledge, my school didn't really teach me programming, but it was good to work with other artists because I would make tools for them, or I would help them learn 3 d software. So that skill set is something I don't think I'd have learned by myself if I hadn't gone to film school. But at the same time, none of the skills I currently use were taught to me in film school.
[00:04:21] Unknown:
And what are some of the ways that Python is used in the process of bringing movie from concept to completion?
[00:04:28] Unknown:
We use Python pretty much from start to finish on movies. We use it to import all the footage from the cameras, and every process in between uses Python. And when we deliver to the clients, we are still using Python to package everything up. The key part of the studio workflow is called the pipeline, where we take data from 1 department, like animation, and transfer it to the next department down, like lighting and then compositing, that's pretty much completely written in Python. Other places where we use Python is to make tools for artists in each of the departments. So we have tools completely built in Python again that will automatically generate character skeletons and their control rigs, which are the things that animators use to animate.
We have Python scripts to set up cloth and hair simulations, or ocean simulations, and things like that. And we use Python for the majority of our user interfaces. So a lot of artists like to work in node graphs or other technical views, and we use Python with PyQ to or PySide to develop those UIs.
[00:05:45] Unknown:
And how much of the actual pipeline processing happens in Python versus Python just being used as a means of transporting the information from 1 program to another?
[00:05:55] Unknown:
A lot of our pipeline code involves manipulating the host programs. So Python is usually an interpreter within a larger program like Autodesk Maya or Katana and Nuked by the Foundry. Python usually comprises the majority of the logic, whereas the host applications usually handle the actual processing of the data. This is, actually probably why Python is so popular, because it's a really great glue language, in that you can have all the performance critical code in c or c plus plus. And then you can use Python, which is really easy to understand and easy to write, controlling all the actual logic of how things get processed.
We do have a lot of code that is real time processing, completely written in Python. But again, it does use CLibs like NumPy under the hood.
[00:06:53] Unknown:
And does Cython get used much as well for increasing the performance of the Python code?
[00:06:58] Unknown:
Cython, unfortunately, isn't that popular in this industry. I don't think a lot of developers have a lot of familiarity with it. I have heard of some people using it to get, that critical performance, but I think most people prefer writing
[00:07:16] Unknown:
an extension in c and then using, like, a CFFI to interact with Python and vice versa. Couple of questions from there is, 1, I guess, how would you characterize the volume of code that goes into a given film project, And how much of that code is reused across projects versus being purpose built for a particular film?
[00:07:37] Unknown:
So the amount of code that gets written per project varies on the size of the project. Some of our movies are, you know, like just 200 shots. Some of them are 2, 000 shots. The larger movies often have more code involved, because they'll often have more demanding needs. And a lot of the development is done on a particular project, but then when the project is done, we usually ship that back to the facility level, so that other shows can use it. In terms of how much code is written, I would say we usually are working in the tens of thousands of lines of code per project.
Each, tool we build is usually, at a minimum, at least 5, 000 lines or so, just because of the amount of work that's needed. And the amount of code that does get reused between projects is pretty high. I wanna say about 80% of our code is stored on a common level that all projects can pull from. This is actually something I like to have because it means you can spin up new projects really quickly without having to reinvent the wheel. Some other studios prefer that every show is its own beast, and it does let them be a little more agile in their development, because they don't have to worry about affecting another project.
But at the same time, there's a lot of initial overhead to get to a workable point.
[00:09:15] Unknown:
And is there a lot of standard software engineering practices that gets used in the movie industry? Or is it a lot of sort of ad hoc scripting just to get something done quickly and get it out the door?
[00:09:29] Unknown:
It's a bit of both. A lot of the studios' pipelines were not necessarily built by actual developers, but rather artists who learned to code. So the problem is that a lot of the legacy code doesn't necessarily follow best practices for software development. But there's a recent shift, especially just because we have to work so quickly now to try and become more efficient, and that involves adopting a lot of the common practices that stand alone developers would use. So we're recently seeing a shift to moving all our code to Git, whereas earlier, the code may have been unversioned or was under SVN.
We're finally seeing a lot of code review in the studios, which is something I'm really grateful for, as opposed to just people doing the wild west, like checking in the code, releasing it to the wild, and then hoping it doesn't break a few hundred artists' workflow. And we're finally moving towards having version packages so that each project can control its release cycle. We're using projects like Res, which I'm not sure a lot of Python users are familiar with, but it's gaining popularity in the visual effects industry because it allows you to control your development and deployment versioning very easily inside of 1 package manager.
[00:10:53] Unknown:
And while I was reading through the blog post that you wrote about the ways that Python is used in the movie industry, 1 of the things that I noticed was the fact that you have a method for testing the data assets that get submitted the pipeline for the, open format conversion process. So I'm just curious what that testing process looks like and some of the tools that you use to, achieve that.
[00:11:17] Unknown:
Yeah. So a lot of studios will have their own solutions for this, but they generally follow the same guidelines where when an artist is done working in their department, they'll open a published UI of some sort, which will list the assets and their scenes that can be published for the next department down to pick up. When so then the artist will select which assets they wanna send through. And we run a suite of tests on each of those assets. So for example, if we're publishing a model, we'll check for things like non manifold geometry or lamina faces, which are basically big no no's when you're trying to render geometry.
Most render engines can't handle lamina faces, which is when 2 polygons on a piece of geometry are right on top of each other. Similarly, we have tests to make sure that animators, for example, don't have disabled animation that might kick in. And we end up writing out incorrect geometry for the next department to read. A lot of the studios have developed their own open source formats to allow for quick interchange between the various applications. So at ImageWorks, we co developed a format called Alembic with ILM. And Alembic is a geometry storage format that is used by pretty much every CG application today.
Similarly, Pixar has now put out USD, which is another open source format for storing scene hierarchies. And, generally, they're pretty flexible, but we need to make sure our data is compliant with them. So that's kinda what our testing entails. There is an open source project called Pyblish, written by Markus Ottuson, which is pretty similar to what you would find in most modern day studios. And again, it follows the same basic idea of you choose which assets you wanna publish, and per asset, it'll start running tests before it starts writing out to 1 of the open source formats.
[00:13:36] Unknown:
And you've mentioned a number of different libraries already, but what are some of the other libraries that you found to be most useful in your day to day work that people may not have heard of?
[00:13:46] Unknown:
The most common ones are NumPy and PyQ, which are fairly common even outside the CG industry. NumPy especially, because it lets us do heavy mathematical computation, while escaping Python's overhead and performance limitations. And pycute, just because it's such a great UI binding to the cute library, It just lets anyone spin up a new UI, very complex UIs in fact, without having to write a ton of code. We do have some less common libraries, especially for color correction. For example, we have OCIO, which is the open color IO library, which handles color transformations.
Other common libraries include, Alembic and USD, which I mentioned. But those are really popular, obviously, to write data out and share them between applications and studios. Other than that, for the most part, all our libraries are fairly common with other Python developers. We mostly just stick to the standard library and use things like YAML, JSON. Nothing too extravagant.
[00:15:02] Unknown:
You mentioned that 1 of the reasons that Python is so widely used in the industry is because of the fact it's already integrated to a lot of the 3 d publishing tools. I'm wondering if there are any other aspects of the language that you think lend itself well to working with film. And what are some of the other languages that you see being used in a similar manner?
[00:15:24] Unknown:
Python is probably the best language for film in my opinion, just because it's so easy to read and use, even for beginners. Like I said earlier, a lot of the early development in this industry was done by artists, not necessarily by actual developers. So they often attach themselves to easier to use languages, like Python. At the time, there was a lot of Perl. Thankfully, that's gone. But it was just easy for someone to jump in, write some code, get it to do what they want, and get out. The other nice thing that Python has going for it, other than just being easy to use, is that it has such a large community on the Internet.
It's really hard to find an issue in Python, where you can't just go on Stack Overflow and find an answer. The other nice thing about Python is CPython is a really great reference implementation, and it's really easy to write extensions in C and use the CFFI. So this makes it really nice for application developers to just stick it in their programs. Like I said before, Python, a lot of studios hedge their bets on Perl. Unfortunately, Perl is kind of cumbersome to use. It's not the most intuitive language, in my opinion. But it does offer a lot of scriptability, which I think is what people were looking for. And at the time, Python wasn't as on the radar as it is today.
But as soon as Python gained traction, we've started ditching Perl almost completely. Other popular languages that we're using now are c sharp. It's more popular in the game development world rather than film because a lot of game engines integrate it. For example, Unity is the primary scripting language is c sharp. Other languages that we use a lot of are c plus plus for our highly performing code, where we need threading or real time interaction with data. And recently, a popular language is Lua, because it has a really low overhead. And you can spin up a Lua interpreter with very very minimal memory usage.
Do what you want and then get out, whereas Python tends to be a little heavier and a little slower. And finally, the other language that is gaining a bit of traction recently is JavaScript, especially with the advent of Node. Js. Because in addition to the 3 d work we do, we do have a lot of SysOps and DevOp roles where they have to develop websites for the artist to manage their shots and see what the statuses are. We have industry equivalents to things like Jira and other ticket management software, but primarily aimed towards artists. So, yeah, those are pretty much the main languages today. I'm sure in the future, things like Rust or Lisp might even make a comeback.
But Rust is definitely something that people are really excited about.
[00:18:45] Unknown:
Yeah. I'm definitely pretty interested in Rust myself. I have to say, I'm a little surprised that you didn't, have any mention of Ruby being used because, generally, in terms of dynamic languages, it's often referred to in a in almost the same breath as Python as far as comparable languages. But also given the age of the film industry, it's understandable that Perl was a, early entrant to that arena as well.
[00:19:11] Unknown:
Yeah. Ruby was popular with a lot of our website developers for our internal web pages and things like that. I don't think it ever gained traction, really, among artists for some reason. I wasn't around back when they were deciding these languages, really. I did talk to the people who decided that Pearl was a good idea. And I think Pearl kinda shocked them at the time at how it didn't scale up to the artist's needs in terms of usability. So they kinda stayed away from that hemisphere. And then when Python gained traction, they were just like, this is it. Let's just stick with this since it works. We're also a fairly slow moving industry.
We don't experiment a lot in terms of the programming languages we use. So if we find something that works, we kind of stick with it. For example, we're not even using Python 3 at the moment. Everything we do is on Python 2.7, which I hope changes in the next few years, but as of now we're stuck on a fairly outdated version of Python.
[00:20:26] Unknown:
Yeah. Python 27 is still maintaining a lot of gravity, though that has been shifting recently where more and more projects are becoming either Python 3 compatible or Python 3 only as more as more modules get added to the standard library. But it's hard to give up an existing code base that's been worked on over the you know, over a number of years in Python 2 because it can be difficult to make that migration.
[00:20:54] Unknown:
Yeah. Totally. It's a hard migration to make. So I understand why people are sticking to Python 2.7. There are things I'd love to use from Python 3, like type annotations, because our libraries get quite large. As well as async, because I find myself writing more and more network libraries recently. And not having async is kind of a burden when developing that.
[00:21:23] Unknown:
Yeah. Are some capabilities for bringing some of that back, but, yeah, it's not as well implemented as it was done natively in Python 3. So for instance, there's the Tulip library for async IO in Python 27, and you can use mypy for annotating Python 27 code, but it has to be done in a separate file. So it adds a bit of extra friction to the process.
[00:21:48] Unknown:
Yeah. It's not as intuitive. But, yeah, I do use, MyPy quite a bit and PyCharm's type annotations, which are quite nice. But it would be great 1 day to just move to Python 3 and call it a day.
[00:22:05] Unknown:
And as you work on more projects, what are some of the areas where you have found Python being used that you were surprised by that you didn't expect to see it there?
[00:22:16] Unknown:
So Python, in my mind, isn't particularly suited to real time graphics. But a lot of my coworkers have surprised me by using it for that. In, Edger Tomorrow, for example, a coworker made a tendril rig for these aliens with these long tentacles. And it was completely written in Python, and it performed really well. Part of that is because a lot of his code calls out to c plus plus code in Autodesk Maya. But even then, it was far more performant than I thought it would be. Another coworker recently made a proof of concept tool with where he could do 3 d sketching.
So an artist could draw in 3 d space, kind of similar to the grease pencil tool in Blender. And all he really used was Python, NumPy, and some pyopengl libraries. And it performed really, really well. And that was really impressive to me, because in my head, anytime I wanna go real time, I have to go to c plus plus. But here are people using Python as a prototyping tool, And it works so well that most people wouldn't even know if you didn't convert it to C plus plus later. The other thing that surprised me when I started in this industry was how much Python was used for user interfaces. A lot of our user interfaces are just done with PyQute.
We have these massive node graphs with millions of nodes sometimes that are just done completely in Python. We have custom bezier curve widgets and other things, Again done in PyQute, because it's so easy for even non technical artists to put together a really complex UI. You can see a great example of an advanced node based UI in a tool called Kraken, which is a rigging front end to the fabric CG platform. Fabric is another framework of sorts for building CG software that is software agnostic, in that you build it for fabric and can plug it in to other applications.
But a lot of the UI for Kraken was done in PyQt, and it performed so well, you'd never really know that it was in C plus plus Or you would know that it was done without threading out the UI from the underlying logic.
[00:24:59] Unknown:
So what are some of the most interesting or exciting projects that you've worked on, and which are the ones that you're most proud of?
[00:25:06] Unknown:
When I was at my first studio, I was using Python a bit to increase my workflow speed. The studio had its own language called Parsley, which wasn't really used anywhere else since they had developed it. And it was kind of a pain to use. So since I was learning Python, I decided to try and make a little IDE for it using Python and PyQt. So I made a little IDE with a built in linter for that language. I also then tried to make a dynamic binding in Python to their language, and made a little interactive shell, again in Python. So it meant that I didn't have to necessarily use their language directly, and I could code completely in Python.
And it worked surprisingly well. And I think that's 1 of the projects I was most proud of, because I was pretty much a beginner to the language. But even then, it would this was a fairly advanced task, I think. And it held up pretty decently. Another project I'm actually excited about is a tool I made in Python again for compositing multiple artists' work together, which is a tool that I built, several feature films ago, but has been used on about 5 feature films now. And I'll be presenting this tool at SIGGRAPH, which is a CG convention later this year in LA.
But basically what the tool lets me do is it lets us split up a single shot between multiple artists, which traditionally was very difficult to do. But now each artist can work on their section of a single shot. They'll render out their images, which is fairly quick for them to do, and my tool will take all their images, take their depth data, which tells you how far in-depth each pixel is, and it'll combine this together so that each artist can get a fairly quick preview of what their shot looks like, combined with every other artist's work. So this has been pretty instrumental for a lot of our shots at Sony, where, you know, where, you know, movies are getting bigger every day.
So you have so you used to have movies where you'd have maybe 2 or 3 characters in a shot. Now you have movies with 300, 400 characters in a shot. And it's really cool seeing my tool let artists manage this kind of extended workflow.
[00:27:55] Unknown:
And how does the work that is done by developers and technical contributors on films get reflected in the final credits?
[00:28:02] Unknown:
So pipeline developers like myself are crude to the individual projects. So we get credited as the pipeline department for the films we work on. Or if the film doesn't have department divisions, we just get clubbed in with everyone else. We also have a group of facility level developers who aren't crewed to a specific show, and they get credited on every 1 of our animated films. They don't usually get credit for the visual effects films, since credit space is more limited there. Occasionally, the films have more limited credit space, so a lot of us don't get credited on those particular films, since credits are usually prioritized by the time spent on the project and your seniority.
But for the most part, every developer on a show will get credited, and all the facility level developers get their little bit of screen time as well.
[00:28:58] Unknown:
And how important is it to show up in the credits in terms of career advancement or mobility across teams and across employers?
[00:29:07] Unknown:
It's, not actually very important at all to be in the credits. A lot of people go uncredited all the time. Credits are more just something we have, so we can tell our friends and family, like, look. Hey. We actually worked on this movie. And, you know, people stay by after the end of a movie to see which of their friends worked on a particular movie. I have a lot of films where I wasn't credited, but they're still listed on my IMDB page. And most employers will look at your demo reel, where you still may have shots from movies that you weren't credited on.
Or in the case of developers who won't have a demo reel, since you can't really show code, They will just take you at your word that you worked on this particular project, and they'll definitely contact your old employer just to make sure that everything is above board. But, yeah, credits aren't super important other than just a nice memento from the movie.
[00:30:10] Unknown:
So for anybody who's interested in getting involved in that kind of work and working as a technical contributor for in the film industry, do you have any particular advice as far as know, material that they should become familiar with or any sort of degree program that they might wanna get involved in or just any other general advice?
[00:30:28] Unknown:
So I think that depends on the person. There are usually 3 kinds of developers I run into in this industry. The first are people who are a little more hardcore and they're interested in real time graphics or rendering. And those are people who tend to be in more low level languages, like C plus plus or Rust or something like that. And they understand a lot of math, Open GL, multi threading, things like that. For those people, I recommend books like mathematics for computer graphics or ray tracing in a weekend. It's a lot more technical, but it is something I know a lot of developers like doing.
For people who are more interested in being more involved with the artist, and therefore also being more involved on a particular movie, I think the best way is to start learning a 3 d application. There are educational versions of Autodesk Maya that are completely free. And Blender, which is a free open source 3 d application. I usually recommend people work through a few simple tutorials to understand the artist's workflows. And once they understand the artist's workflow a bit, they'll understand what common issues are and what kind of development is needed.
It also really helps when you're building tools for an artist, if you understand where they're coming from, rather than developing sort of in a vacuum. And last, but definitely not least, we have a lot of supporting players, like the guys who write the network infrastructure tools, who manage our databases, and generally more SysOps, DevOps kind of roles. So they use a lot of Python, and I think that's probably the most compatible with a lot of Python developers who didn't start out in film, because it lets you reuse a lot of your skills, and you still get to work in the film industry without having to relearn all these new skills, learning 3 d, etc. You can be a web developer working on a movie. You'll still get credited, It'll still be kinda cool, but you don't have to relearn 3 d and things like that.
[00:32:54] Unknown:
And, your mention of the databases that are being used made me wonder, what are the typical kinds of volumes of data that you're dealing with? Is it generally fairly substantial, or is it highly variant depending on the particular type and length of movie?
[00:33:10] Unknown:
It's, pretty variant on the type of movie. So the databases themselves are fairly lightweight, because we just use them to point to actual geometry data on disk, but it's not unheard of to be for even a single shot to be writing multiple terabytes of data. I'm sure we've hit a petabyte of data on some show somewhere, but I know myself I've generated terabytes and terabytes of data in the hundreds of terabytes range for a particular shot. So if you multiply that by, let's say, a 1000 shots in a movie, you're already probably at a petabyte per movie.
[00:33:53] Unknown:
I'm just glad I don't have to deal with that much data in my day job.
[00:33:57] Unknown:
Yeah, it's kinda nuts. And they also have to back up every movie that we've ever done in case we need to go back to something on there. So they have these massive disk storage. They have data from, you know, the early nineties that are still on tape that they occasionally have to take out and load up, which is kinda crazy, but it's cool that they're able to do that.
[00:34:25] Unknown:
Yeah. It's definitely seems to be a pretty complex industry with a lot of different moving parts. And despite the fact that it is, as you say, somewhat of a slow moving industry, I'm sure that there are still new things being introduced on a regular basis as artists and technical crews continue to push the limits of what's possible in film.
[00:34:45] Unknown:
Yeah. There's, there's definitely a lot of innovation going on. I guess when I say it's a slow moving industry, we are slow moving on our base layer of technology because we don't wanna change too much down there. But at the high level, we have a lot of innovation going on. Right now, something that's very popular is machine learning. I'm reading about other studios using machine learning to speed up simulations. Whereas earlier, we'd brute force a simulation where every vertice in a mesh in a geometry mesh would be calculated colliding with every other point on that geometry.
We're now able to say, okay, the artist changed this tiny parameter. How will this actually affect it, and how can we reuse the data we've already generated in the most efficient way possible? They're starting to use machine learning to aid animators to get better muscle deformations and better posing, so that the animator doesn't have to think about how muscles move. There's a lot of innovation in rendering technology right now, as we're shifting from using mainly CPU's to shifting onto the GPU. So shots that used to take 100 and 100 of hours to render 1 single frame can now render in a fraction of the time.
And of course, 1 of the biggest areas of innovation right now is VR. Virtual reality is becoming a huge part of the film industry now in not so obvious ways, but lots of movies will have supplementary content with virtual reality. So traditionally films would have tons of resources at their disposal because we never had to worry about real time. It's okay for a frame to take 200 hours to render in film, but for VR, you need to render a shot within a few milliseconds. So there's definitely a lot of development going into getting our shots and our assets working in much more constrained resources, whether we're shifting things to the GPU or shifting things to a lot of parallel evaluation, it's definitely gonna be a big push in the next few years to make our data much faster to process.
[00:37:27] Unknown:
So are there any questions or topics that we didn't cover that you think we should talk about before we start to close out the show?
[00:37:35] Unknown:
Nothing really off the top of my head. Your questions were pretty thorough, I think.
[00:37:40] Unknown:
So for anybody who wants to get in touch with you or follow what you're up to, I'll have you add your contact information to the show notes. And so with that, I'll bring us to the picks. And my pick this week is going to be the Firefox browser for Android. I started using that recently because a lot of Android apps have switched to having the embedded Chrome view. So if you're, for instance, loading you know, reading a newsletter in your Gmail app and then you try to click a link, it will open an embedded browser as opposed to opening it in another window. But if you use Firefox, you can actually set it so that it will just add the link to a queue. So you can just open a bunch of links at once, and then when you open the browser, it will pull them all in at the same time. So it's made it a lot easier for me to, you know, go through newsletters to keep up with what's going on in Python and other technical areas and, just has a lot of nice little design aspects to it that, give it an edge over the Chrome browser for Android, in my opinion. So if you haven't at least checked it out, I definitely recommend downloading it and playing around to see if you enjoy it as much as I do. And with that, I'll pass it to you. Do you have any picks for us today?
[00:38:50] Unknown:
Yeah. Something I find really cool recently is I've been using a lot of VR, to demo to my friends. I bought a virtual reality headset last year. And surprisingly, 1 of the things that people find really cool is Google Earth in VR. And Google recently had a giant update, so it's really impressive. You can fly around the world in Google Earth. You can go visit the Grand Canyon. You can go to France. You can go to India. And you can walk around. And it sounds kinda dumb, maybe, when someone's describing it, but just actually being able to see it and look at places at scale is something that I think everyone I've demoed it to found a really compelling experience.
So if anyone ever has access to a virtual reality headset, whether it's on a phone or an HTC Vive or an Oculus Rift, I definitely think Google Earth is surprisingly well worth checking out.
[00:40:05] Unknown:
Yeah. That definitely sounds really interesting being able to have sort of a first person perspective into it because it's all well and good to look at it on your computer screen. But as you said, it's not, it's not you're not viewing it at the appropriate scale and magnitude for being able to get a sort of impactful experience from it.
[00:40:24] Unknown:
Yeah. Definitely. I think that's 1 of the crazy things for virtual reality is, even if the graphics or the visual fidelity isn't that high, just being able to see things at a life size scale is really breathtaking to a lot of people. And it's something that a 2 d monitor just really can't convey to anyone just watching.
[00:40:49] Unknown:
Alright. Well, I appreciate you taking the time out of your day to tell me more about how Python is used in the film industry. It's definitely an interesting area and 1 that I haven't had any personal experience in, but it certainly seems like there's a lot of worthwhile challenges going on there. So thank you for that, and I hope you enjoy the rest of your evening.
[00:41:11] Unknown:
Cool. Thank you for having me on here.
Introduction and Guest Introduction
Dhruv Govil's Background and Career Path
Education and Industry Preparedness
Python in Film Production
Code Volume and Reusability in Film Projects
Software Engineering Practices in the Film Industry
Testing Data Assets in Film Production
Useful Libraries in Film Production
Languages Used in the Film Industry
Unexpected Uses of Python
Interesting and Exciting Projects
Credits and Career Advancement
Advice for Aspiring Developers in Film
Data Volumes in Film Production
Innovation and Trends in the Film Industry
Closing Thoughts and Contact Information
