Summary
Location is an increasingly relevant aspect of software systems as we have more internet connected devices with GPS capabilities. GIS (Geographic Information Systems) are used for processing and analyzing this data, and fortunately Python has a suite of libraries to facilitate these endeavors. This week Sean Gillies, an author and contributor of many of these tools, shares the story of his career and contributions, and the work that he is doing at MapBox.
Brief Introduction
- Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
- I would like to thank everyone who has donated to the show. Your contributions help us make the show sustainable.
- When you’re ready to launch your next project you’ll need somewhere to deploy it. Check out Linode at linode.com/podcastinit and get a $20 credit to try out their fast and reliable Linux virtual servers for running your awesome app.
- You’ll want to make sure that your users don’t have to put up with bugs, so you should use Rollbar for tracking and aggregating your application errors to find and fix the bugs in your application before your users notice they exist. Use the link rollbar.com/podcastinit to get 90 days and 300,000 errors for free on their bootstrap plan.
- Visit our site to subscribe to our show, sign up for our newsletter, read the show notes, and get in touch.
- To help other people find the show you can leave a review on iTunes, or Google Play Music, and tell your friends and co-workers
- Join our community! Visit discourse.pythonpodcast.com for your opportunity to find out about upcoming guests, suggest questions, and propose show ideas.
- Your host as usual is Tobias Macey
- Today I’m interviewing Sean Gillies about writing Geographic Information Systems in Python.
Interview with Sean Gillies
- Introductions
- How did you get introduced to Python?
- Can you start by describing what Geographic Information Systems are and what kinds of projects might take advantage of them?
- How did you first get involved in the area of GIS and location-based computation?
- What was the state of the Python ecosystem like for writing these kinds of applications?
- You have created and contributed to a number of the canonical tools for building GIS systems in Python. Can you list at least some of them and describe how they fit together for different applications?
- What are some of the unique challenges associated with trying to model geographical features in a manner that allows for effective computation?
- How does the complexity of modeling and computation scale with increasing land area?
- Mapping and cartography have an incredibly long history with an ever-evolving set of tools. What does our digital age bring to this time-honored discipline that was previously impossible or impractical?
- To build accurate and effective representations of our physical world there are a number of domains involved, such as geometry and geography. What advice do you have for someone who is interested in getting started in this particular niche?
- What level of expertise would you advise for someone who simply wants to add some location-aware features to their application?
- I know that you joined Mapbox a little while ago. Which parts of their stack are written in Python?
- What are the areas where Python still falls short and which languages or tools do you turn to in those cases?
Keep In Touch
Picks
- Tobias
- Sean
Links
- GDAL
- SWIG
- QGIS
- Shapefiles
- Shapely
- Fiona
- Raster File
- GEOS
- Rasterio
- PostGIS
- RTree
- GeoPandas
- GeoJSON
- Orthorectification
- Mapbox
- SCONS
- Mapnik
The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA
Hello, and welcome to podcast.onet, the podcast about Python and the people who make it great. I'd like to thank everyone who has donated to the show. Your contributions help us make the show sustainable. When you're ready to launch your next project, you'll need somewhere to deploy it, so you should check out Linode at linode.com/podcastin it and get a $20 credit to try out their fast and reliable Linux virtual servers for running your awesome app. You'll want to make sure that your users don't have to put up with bugs, so you should use Rollbar for tracking and aggregating your application errors to find and fix the bugs in your application before your users notice they exist. Use the link rollbar.com/podcastinit to get 90 days and 300, 000 errors tracked for free on their Bootstrap plan.
And you can visit our site to subscribe to our show, sign up for our newsletter, read the show notes, and get in touch. To help other people find the show, you can leave a review on Itunes or Google Play Music and tell your friends and coworkers. You can also join our community at discourse.pythonpodcast.com to find out about upcoming guests, suggest questions, and propose show ideas. Your host as usual is Tobias Macy. And today, I'm interviewing Sean Gillies about writing geographic information systems in Python. So, Sean, could you please introduce yourself?
[00:01:19] Unknown:
Yeah. Hi. I'm Sean, and I work at Mapbox building a mapping platform for public and private enterprise. And I've been a, a member of the open source GIS community 1, 001. And, not formally trained as a GIS practitioner but came from a background in science and remote sensing and kind of taught myself how to use the software and python kind of as the field grew. So it was interesting to be able to kind of get in at the ground floor of open source GIS and Python and GIS.
[00:01:56] Unknown:
And how did you first get introduced to Python?
[00:01:59] Unknown:
I was introduced by Zope. And this was about 2, 000, I believe. The company I was working for wanted an internet and a CMS and we found Zope. I can't remember the other things that we tried. But as soon as we found Zope and found its customization story, its permissions, its configuration, we were hooked on it. I mean, if you use Zope in those days, it was pretty clear that it was ahead of its time. I think it had features that other kinds of CMSs wouldn't have for years. And we enjoyed Zope. We profited a lot from it. And in order to extend it to write custom applications for our company, that's how I learned Python. So, I learned Python to Extenso.
[00:02:48] Unknown:
So, can you start by describing what geographic information systems are and what kinds of projects might take advantage of them?
[00:02:55] Unknown:
Geographic information systems have been changing quite a bit in the past few years. The constant though, is that a geographic information system is data models and representations of data that treat a location as a first class property of the system and have tools for relating spatial objects to each other and visualizing the relationships. So often this means making maps. But there's a lot of Mapless GIS as well. So if you have if your business or your enterprise has, spatial information, whether it's static assets like buildings or property or roads, pipelines, such things. And you want to be able to query your system for the the relationships to these things, so distances of new projects to other existing infrastructure, to roads, maybe do things like view shed analysis, say build, structures and you want to see their impact on adjacent structures. Let's see, watershed analysis, this type of thing. I'll just say there's many applications then for geographic information systems in businesses, whether it's, you know, science, business, emergency response.
[00:04:08] Unknown:
Yeah. I think my first introduction to really doing much with GIS is I was working at a company that had a number of sensors deployed at various locations throughout a couple of cities. And so I imagine that 1 of the things that's probably growing some of the current advancements in GIS technologies and applications is the fact that there are so many more connected sensors being used as the sort of Internet of Things continues to grow?
[00:04:35] Unknown:
Yeah. Certainly. And, additionally, how would I phrase it? I would say consumer consumer GPS devices and, and programs to use them have also changed the amount of, GIS data that's available to us as well. I mean, since I've started in the GIS field, it's changed from a state where really only large enterprises could collect and use GIS information. So this would be national, regional, and local governments who did, you know, could pay surveyors to go do stuff, you know, large development projects that would pay to go out and survey sites and collect data. And it's changed now so that you can, with your own smartphone, can go out and survey the field. You can map streets. You can map parks. You can map the locations of trees. And the amount of data has just grown immensely. A lot of this ends up in community curated free sites like OpenStreetMap as well.
[00:05:30] Unknown:
And you mentioned it briefly, but can you give a bit of background in how you got involved in GIS and location based computation?
[00:05:37] Unknown:
Yeah. At at the same time I was discovering Zope, this was in 2000, I was working for a company in Fort Collins, Colorado called Information Integration and Imaging. And it's, we called ourselves a GIS shop and it was about 12 people at the time. And we did I was working there doing satellite and aerial image analysis and some Linux system administration. So at the time, the work I did in 2000 was largely in Perl and in scripting languages for our GIS software. 1 of these was called Easi, the other was called Avenue. So this was the scripting language for ArcView, ESRI software at the time. So at the same time, this is also the same time that Python and GIS came together. So in I think it was January 11, 2000 that, Bruce Dodson, who was an employee for ESRI in Canada, he embedded Python in ESRI's ArcView program and announced it on an email to a, GIS like special interest group and said, Here, I've embedded Python in ArcView, which previously had its own scripting language.
And he said, you know, here's the download. Is there anybody else interested in it? And not too long after that, I went and looked through JITL, which is a popular open source GIS library written in C. And I went back in its subversion repo and found that on March 5, 2000 Frank Warmerdam, the author of JITL, he committed Python bindings to the JITL library to that repo. And these bindings were written let's see. The bindings were generated using a program called SWIG which was pretty new at the time and this was originally written by David Beasley. And what I loved about swig and what the whole community loved about this is that it brought the enormous power of the JITA library immediately to Python programmers. So features that were previously only available in the desktop GIS programs that we'd been using, So like ArcView and other software at the time. This stuff with the help of SWIG became immediately available to Python programmers. So there was a I think people really felt at the time kind of a bit of a rush because you had applications that other people wrote that you had to use and suddenly you had the power to write your own applications, say as to do different things, to do smaller things, to do, I guess, in some ways, even larger things.
[00:08:06] Unknown:
So I have some experience using the QGIS application, which is an open source application that allows you to do some geographic modeling, which also has some Python bindings exposed. I'm just curious what kinds of operations or routines you would script with Python in an application such as QGIS or the Esri suite.
[00:08:26] Unknown:
Yeah. As so as you mentioned, there's, the desktop applications. I think Python and GIS kind of started as a bit of a, perhaps a bit of a revolution against desktop applications. So people discovered that we could write programs that would do the stuff that we would ordinarily do in our desktop applications. So you could then at this point say with, Python and GDEL, you could take a directory full of shapefiles. And this was 1 of the large geospatial data 1 of the popular geospatial data formats of the time and continues to be popular. So you could take a directory full of shapefiles and you could convert them all to a different format or you could transform their coordinates to show them in a different map projection or you could augment or reduce the the properties and the feature of these. And so this is the kind of stuff we did originally with Python and GIS. Not too long after, I think developers of you know, both Arcview and QGIS realized that we could use Python as well, they could use Python as well to extend their own desktop applications. So I think in QGIS, for example, you can in a map view, say you have your layout, you've opened data, you're viewing a map and the features in it. You can open a console, and you can then write Python code to, let's say, explore the features you're looking at at the time so you can query what's in your map window. You can change your map windows.
So this would be to give you a little more a little more power and a little more flexibility and, you know, saving on pointing and clicking in in the toolbar. So kind of a a complement to the the toolbars in these applications.
[00:10:03] Unknown:
So it's almost like being able to have a visual REPL for your data as you explore it in the Python console. Yeah, exactly. That's interesting. So you've been involved in creating and or contributing to a number of the canonical tools for building JS systems in Python. So I'm wondering if you can list off at least some of them and describe how they fit together for some sort of your typical workflow or, you know, what their relationships are.
[00:10:28] Unknown:
I think, like, the beginning and the end for a lot of GIS projects is reading and writing the data to formats. So to get them from whatever their source is from your partners that are providing you with the data, at the end of your job, you probably have to give a final product in a GIS data format to another partner or send it to another system. And the library for doing this is generally the open source 1 is called GDAL. And this is the Geo Data Abstraction Library. So g d a l. GDAL is the library that allows us to read and write data formats. So that's the beginning of the end of our jobs. It has a data model inside too, an abstract data model for these things. So for, gridded data, say, you know, aerial imagery or satellite imagery, this is generally stored in some sort of a raster file.
And the data model for this is that you have a dataset that contains 1 or more imagery bands. And these are like channels, say, in like a photoshop image. So we call them bands in GIS. And these bands are have a number of rows and columns they all have a uniform data type. For the whole data set there's generally a coordinate system and projection information and some kind of measure of the extents of the image. And those things allow you to put the imagery and analyze it in the context of other spatial data. So you could, you know, say, warp it and drape it on a globe, or you could drape it over a terrain map or something like that. For geographic features, generally, the data model in GEOS or in GDAL is that you have a feature is a thing that combines a geometry, a spatial geometry. So this is a 0, 1 or 2 dimensional geometric figure. So a point, a line or a polygon and the number of associated attributes. The, so Python bindings for this, JITL has its own Python bindings.
And I've been using those for a long time. Recently I've written a couple new libraries to provide an abstraction that I prefer for the JITO library And 1 of these is called Fiona and this is for the vector side of the problem. Vectors being the vector lines. And you can think of this as being if you're new to the field, you can think of these as being like SVG like the same sense of vector. The other 1 is called Rasterio, which is a funny name for the raster IO operation that's inside JITL. So this is the part that the Python library that allows you to read and write bands from an image dataset. And rosterio is meant to operate along with, numpy. So it reads data off a disk, loads it into a numpy array. NumPy arrays then become the representation for your data that you operate on. And then you can take these arrays and you can write them back out to GIS formats. Now for, spatial algorithms, there's a library called GEOS. This is also written in C and it's a port of a Java library called the Java Topology Suite. And I think 1 of my more successful open source GIS projects is 1 called Shapely. And this is Python bindings to the Geos library. And what it gives you as a Python programmer is the same sort of features that you would normally have in a GIS database, like say post GIS or Oracle spatial, something like this, and gives you those same kind of tools outside the context of a database so that, you can create Python objects and then compare them to see does this geometry is it contained within another geometry? Or what is the distance between them? Or say if you want to dilate them or erode them erosion being like kind of a negative dilation. It has all these algorithms in it. So kind of standard GIS stuff like you can do in a desktop GIS, buffering especially being 1 of the mainstays in GIS. You can do these now in Python without actually having to your data can come from, you know, any kind of Python source. It doesn't actually have to come from a GIS file or be created in a desktop GIS.
So say, if you have tabular data with locations latitude and longitude, you can now read this table. You can create shapely points out of them and you can, say, find the bounding boxes of points. You can assemble the points into lines. All this kind of stuff that you would ordinarily have to have an actual GIS software to do, you can now do entirely in Python. I think the last piece that you need for a GIS application in Python is you need spatial indexes because, GIS data sets are very large and you want to be able to quickly find subsets of them, say from a road database or from a file that contains all the roads for a city. You'd want to select the roads that, are within, say a particular small fraction of your of your dataset. And the tool for this generally there's a there's an R tree.
So it's a tree data structure where R is for rectangle and a spatial index library for generating these R trees. And we have a Python binding for this as well, which is called Rtree.
[00:15:33] Unknown:
Yeah. In terms of using Polygons and managing the relationships, 1 of the applications in a project that we had a little while ago was I had county data for US state. And then the application wanted to be able to click a button and then automatically highlight in a list all of the neighboring counties. So I loaded all of those polygon coordinates into a post GIS database. And then I was able to select on all any of the other polygons that shared any portion of a border with that selected county. So being able to do that purely in Python, I can definitely see as being pretty useful.
[00:16:09] Unknown:
Yeah, it is. On the other hand though, I've met, among my colleagues at Mapbox. I know plenty of people that still I think there's something to be said for the power of PostGIS as a as a workspace for spatial operations and largely because of the powers of, the spatial queries in it. I think that the, perhaps that's something that as a Python programmer, you don't have access to quite as readily as you do in PostGIS. But I think like, you know, maybe you saw that yourself when you once you've got your data loaded into PostGIS, you're kind of limited with how you can, interact with it. But certainly the, you know, the query infrastructure in the database maybe probably exceeds that of what we have in Python.
[00:16:52] Unknown:
Yeah. For being able to productionalize the use of the data, I think that Post JS is still a very useful tool, but for being able to do some exploration of the data and do some exploratory analysis, I can definitely see being able to just do a straight in Python without having to, you know, set up a post GIS database and load load in all the information, but just to be able to, you know, have the data on your desktop and just start exploring it with a lower barrier to entry, I can imagine being very useful.
[00:17:19] Unknown:
Yeah. And if you're, you know, if you're familiar with software like Pandas, there's also a, a geospatial extension to pandas as well, so called geo pandas, where it, adds, you know, a new geometry type to your pandas data frame. So you can do pandas like operations to find intersections between your objects, filter, and sort, etc. So there's another option there that's kind of maybe that's kind of a middle ground between writing your own custom program in Python or using, an existing application like post post GIS to do your work in. You have pandas as well. So there's there's quite a few options for analysis these days.
[00:17:57] Unknown:
So what are some of the unique challenges that are associated with trying to model geographic features in a manner that allows for effective computation?
[00:18:05] Unknown:
I think the challenge the challenge for me has been finding good abstractions for for data and representations of data. Alright. I'll just back up. The challenge for me has been finding good abstractions for gridded data and for geographic features. Because without good abstractions, we're kind of struggling to make good programs and useful programs. I think we have those now so this is a challenge that I'm happy to say is pretty well solved. I think that nd arrays from numpy are excellent representations of gridded data and they have all the power of scipy behind them. And then for geographic features, the good abstraction to use now is, GeoJSON type dictionaries in Python. So this is kind of like having, it's like having a literal syntax for geographic features in the form of a of a Python dictionary, grade way. So you had like say if you had a, a using in kind of a fine grained way. So you had like say if you had a, a polygon you needed to or a polygon type feature. You had a bunch of accessors for changing its its, properties if you had a bunch of accessors for changing the rings of the polygon or accessing coordinates within those rings. Now as a Python programmer, you can largely just treat a geographic feature as a literal Python dictionary And there's a lot of software that now supports this. So in Shapely, you can do any kind of geometric object that you create in Shapely or that's the product of an operation in Shapely. You can get it as a G adjacent type feature. ESRI software in ArcMap does this as well. And it's QGIS as well. So it's like a common shared abstraction now for geographic features.
[00:19:49] Unknown:
What are some of the problems that only arise as you start to scale the amount of land area that you're covering that aren't really present when you're dealing with smaller geographic features?
[00:19:59] Unknown:
Well, 1 nice thing about, GIS problems is that they in some ways, they scale linearly with your area. So much so that it's still in the business of buying and selling imagery, say from an aerial photography company where they fly planes with a camera and make ortho photos over a city or a satellite image provider. It's still very common to buy and sell imagery in in the currency of square kilometers or square miles. So this kind of shows how the amount of data still increases linearly with the area. 1 way in which the complexity does change is that as you zoom out this is actually reducing a scale in the cartographic sense but as you zoom out, you see there's more types of features. And so I think that's where the complexity comes in.
There's more kinds of things in your information system that you have to deal with. So, if you're zoomed in over land, you may not be dealing with, with water features But as you zoom out, you do have water. You have land. You have more land use. You have more kinds of roads, more kinds of transportation networks.
[00:21:02] Unknown:
And I know too that 1 of the problems that starts to become apparent as you cover significant amount of land areas that you start having actually deal with the curvature of the earth in your computations when you're trying to, for instance, do so any sort of route planning or, distance computation?
[00:21:18] Unknown:
Yeah. Certainly.
[00:21:19] Unknown:
So mapping and cartography have a really long history with an ever evolving set of tools. And I'm wondering what our digital age brings to this time honored discipline that was previously either impossible or at least impractical.
[00:21:33] Unknown:
I think 2 of the major changes here are scale and the ability to freely move between scales and also currency of information. So back when maps were on paper, if your enterprise had a map maker and a cartographer, you could have very nice maps but you were limited by how by this person's production. How fast could they prepare maps, Right? If they were doing them by hand, you know, it was kind of a painstaking process to make a good map. It still is a painstaking process for digital cartographers. Now we're in a situation where maps can be made very quickly and very cheaply which gives people who want a just good enough map to get to the grocery store or to find their way into new town or to deliver a package. They have access to this, which wasn't previously available. I think we can also collect data and turn it around and get it into maps faster than ever before. So for example, in Mapbox, we use OpenStreetMap as a data source in our maps. And if you go in OpenStreetMap and trace the footprint of the house you live in committed to OpenStreetMap, this then shows up in the Mapbox maps like 5 minutes later. So this is not possible before we had computers. This wasn't possible in the time of paper maps. And then for the freedom to move around in scales and maps, I think that many people were fascinated by this 'powers of 10' book. But be able to go from images of the earth or views of the earth on a very small scale, say your yard, and then step out to a view of your neighborhood, to the city, to the state. And then to be able to zoom out further to where you can indeed see the curvature of the Earth and the entire planet. These are the kind of pictures that we had from orbit and from satellites. And be able to, say, then browse to the other side of the Earth and zoom back in. I mean, this was not possible before we had computers. And I think that's probably 1 of the biggest changes.
I remember seeing the program that became Google Earth when I worked at, at iCube in about 2, 002. So So we got this demo from a company that was called Keyhole at the time and they brought it and we looked at our computers. We loaded it on our computers and were just astounded by the kind of vision you got of the globe. And we didn't really have enough data to, make it super compelling everywhere, but we had a few global imagery datasets. And it was just mind blowing to just pan around the planet and zoom in and out. Kind of an incredible thing that, I think we're starting to take for granted these days in some ways. I mean, I think the even the excitement over Google Earth has maybe faded a little bit, but it's still just an impressive achievement.
[00:24:13] Unknown:
Yeah. I think that as mapping becomes more commoditized and common, you don't have as much of a connection to them as when they were handcrafted pieces of, in some cases, artwork that a lot of times are actually very expensive to be able to even have, particularly if you wanted them to be, you know, relatively accurate. So at 1 time, we gain a lot of utility from them, but at the same time, I think we sort of lose some of the connection with the people who actually spend the time creating them.
[00:24:41] Unknown:
Indeed. There are still practitioners of fine cartography in the world. I'm drawing a blank on the name. You may have seen this, an article recently about a man, I believe he lives in Oregon. I can't believe I'm spacing his name. But he has these fantastically detailed you know, conventional wall maps that he makes using digital data, but then a lot of, I think, photoshopping and illustrator when he gets right down to it. So it's like, you know, it's like the classic classic maps made by hand but backed up by digital data. And I think that what he's working for there and what he argues is that these maps made by hand still give you much more of a sense of the place than your typical digital map which I think is kind of a fair critique of digital maps that, what we're making at Mapbox and the kind of maps that you see from Google Maps as well. We're beyond just good enough. I mean, I think that we're making very compelling maps now but a map actually made by hand can be deliberately designed to produce a feeling in you as the viewer.
You can draw a map in such a way that it compels you to want to visit the places that are on it. Even by cartographers, this is what they do. This is their craft. They study how to do this. They study how to depict mountains and rivers and towns in such a way that actually kind of makes them intriguing and appealing if they want to do that rather than just reveal where they are in relation to each other. But I'm optimistic that they, I don't think that digital map making is completely at odds with that kind of craft. And certainly people practicing the craft now they use digital data as much as they can because you certainly if you make a wall map of the United States, you can't compile it all from from other paper maps or from tables of locations, right? You've got to have the digital data to get yourself started. Then you go from there with the actual craft.
[00:26:34] Unknown:
Yeah. I think there's definitely still a lot of room for creativity and adding a personal touch to maps. And another area that this is showing is that I think that having these digital tools available makes it easier for people to create maps of nonexistent places. So people who are interested in exploring some realms from literature or, you know, sci fi or fantasy have been be able to create some pretty interesting maps that are fairly realistic looking, but of the places that don't even actually exist. Mhmm. So in order to be able to add some additional accuracy to mapping and cartography, there's also data involving elevation and contours. So I'm wondering how often that comes up in your work and what sorts of challenges that poses.
[00:27:21] Unknown:
Yeah. Elevation is an interesting problem for GIS. So in my particular application, which is processing aerial imagery and satellite imagery, you can't use aerial photography or satellite imagery well with other data sources unless you account for the terrain that, you know, you're shooting the imagery over. So and this is because curvature of the Earth, like say for very large very small cartographic scale imagery but imagery that's kind of global in scale. Also for aerial photography because you have change of perspective. So you do need to account for the terrain that you're imaging. And this is generally done with the help of what we call a digital elevation model. So for your imagery, you would want to be able to know the feature for the feature in every pixel in your image, you would want to know its elevation. And then using the elevation model, you can then warp the imagery to, in effect, flatten it out and would be called orthorectification to correct for the imagery. And this is, as you can guess it kind of depends upon the accuracy of your elevation models. But yeah, the and elevation models have kind of, you know, for a long time they were a little bit coarse and it was hard to do this well but, you know, now with, now that we have satellites with radar and the good quality elevation models are more accessible, So it's easier to correct for elevation. Now of course in applications like on your phone, if you do you know, if you have a fitness app or a running app and you want to see your recent course. You went out on a hard run and you did some hills and you'd like to show this to somebody and you'd like to show them your course with the hills. You'd like to drape it actually on the elevation because it's kind of, you know, it tells the story of how hard you worked on your run or the hill you were proud of, of summoning.
And doing this with terrain a few years ago was computationally very expensive. That's all changed now that we have WebGL in our browsers. So it's quite commonplace now to be able to display in 3 d, at least with the elevation model that you have and drape GIS features on the train and do this in the browser as well or in your phone.
[00:29:30] Unknown:
So in order to be able to build accurate and effective representations of the physical world, there are a number of different domains involved, including geometry and geography and various other mathematical disciplines. So I'm wondering what advice you have for somebody who's interested in getting started in this particular niche.
[00:29:47] Unknown:
Thanks to programs we have like PostGIS, Python libraries like Shapely, Fiona, Rostario, I think the GIS concerns are largely solved for a person who knows Python or a developer that's comfortable in, in a relational database. I think that maybe the bit of GIS expertise that's useful to know well, there's probably some. I think 1 would be a little bit of fluency in data formats because unfortunately there's many, many formats and I think many of them are historical so if you go and you see historical data it could be in some fairly obscure formats. So a little fluency in how to convert between them and maybe the various features or limitations of different formats is a good thing to know. I think a good place to learn those would be the user communities around your favourite GIS software so it might be the JITL community or the Phosphorg user communities which have a lot of experience in how to interpret formats, how to convert between them, the pluses and minuses of using different ones. Another bit of expertise that's good to know would be some knowledge about how map projections work. So I think I've seen if you look on Stack Overflow you'll see questions every so often where someone will say I have some latitude and longitude points in a table and I want to find other points within a particular radius of these points Right? And so these points might be tens or a couple 100 miles apart And now this starts to be the distance where you do have to take curvature of the Earth seriously. So you can't just say, it's nonsensical to say, 'Find me all the other points that fall within, let's say, 1 degree radius of these points I have. So at this point you need to have, first of all, the understanding that the world is a sphere and that, a lot of the algorithms that we have in GIS systems are not necessarily going to handle spherical geometry. And this is especially in the case of open source GIS. So you'll need to transform your coordinates from latitude and longitude on the earth, say a spherical or elliptical earth, ellipsoidal earth. You'll need to transform these to a co ordinate system which is flat and then do your geometry operations in a flat coordinate system. And then you'll need to transform back then, let's say for display on a globe or to put back into a table that expects latitude and longitude. You'll need to convert back then from your flat coordinate system your plain coordinate system, back into, geographic coordinates.
[00:32:16] Unknown:
As you mentioned, you're currently working at Mapbox. So I'm wondering, what are the uses of Python within Mapbox?
[00:32:22] Unknown:
So, in my team in particular, we use Python to process all the new imagery we we acquire. So to read them from their native formats, do color correction on them, to mosaic them into larger images that then we tile up and, and then store to be served up as tiles in your browser. So that's done in Python. So all of our pixels, if you look at a Mapbox map with satellite imagery, every single 1 of those pixels goes through a number of Python programs and probably spends some of its time in a, in a NumPy array or some of its time inside a, rasterio function call. We use the, AWS's Python library. So we use moto, we use, the AWS command line all over the place for deploying. We use Python and build scripts. Yeah, we use SCONs for building node libraries, but particularly, Mappnic. So, you know, to render datasets into tiles, there's a library called Mappnic and its Python bindings are built with scones. There was a time in the past when Mapbox was just getting started that, we considered using Python and, twisted in in particular instead of Node, but ended up not going down that path.
[00:33:26] Unknown:
So what are the areas where Python still falls short for GIS systems and which languages or tools do you turn to in those cases?
[00:33:34] Unknown:
Well, because because GIS data is large and some of the algorithms are pretty computationally expensive, Python is maybe too slow still. You know, CPython is too slow still for a lot of applications. So generally, the libraries I use so Shapely, Restario, Fionna these have a core written in C and then, a Python wrapper outside that. So we use a lot of C, a lot of C plus plus JavaScript of course because at some point in the end, in a lot of cases in GIS, you want to see results in a browser. You wanna be able to share the maps you've made immediately with other users. And the web and browsers is how we do this.
[00:34:12] Unknown:
So are there any other topics or questions that we didn't cover that you think we should have? No. I don't think so. So for anybody who wants to follow what you're up to and keep in touch, what would be the best way for them to do that?
[00:34:24] Unknown:
Let's see. I I like emails. It's pretty easy to find my email. You can find it on my, on my Mapbox page. I'm also a Twitter user, so that's a good place to find me there. And my handle there is sgillies.
[00:34:37] Unknown:
So with that, I will move us on into the picks. So for my pick today, I'm gonna choose the Roku Streaming Stick. I had built myself of as a way to simplify the interface and get a bit better performance when playing, you know, doing video playback. So, definitely recommend checking those out. They've got a pretty good set of different channels that you can pick from, and they seem to be the best option if you want to be able to use different video providers such as Amazon and Google all in the same interface without having to pick 1 or the other and go with their particular proprietary offering. So with that, I will pass it to you. What do you have for us today, Sean?
[00:35:24] Unknown:
So 1 of my favorite things to do is to cook and eat. I'm living in France right now and I'm thinking a lot about a book I left at home called the Tacopedia. And this is a tribute to Mexican taco culture. Fun book. I learned a lot of things from it. You know, I thought I knew about tacos living in Colorado but in fact I didn't actually know very much about tacos at all. And I learned that this crockpot chicken stewed in salsa tacos that my friends made that I thought was kind of a midwestern thing. I learned that it's actually like a real taco thing. So this is the, I'm not sure I can pronounce the name very well. Well. Polodashebraba. And that I also learned from the Tacopedia the Taco al Pastor which is the marinated pork often with pineapple and cilantro. It originates with Lebanese immigrants to Mexico and so people accustomed to eating rotisserie lamb and pitas adapted themselves to pork and tortillas. And I'm experimenting with this a little bit because my city has a strong Mediterranean and Arab culture. So there's a lot of lamb here. There's a particular sausage made from lamb with chilies called merguez And so I've been dabbling in, kind of like taco al pastor but made with merguez sausage. And then I think my other pick is I've been listening a lot lately to, an album by an artist named Strome called Racine Kare. And this is funny because I'm not generally a big fan of popular music and stromae is about as popular as you can get in Europe. There's a couple of songants of stromes and I think, in particular Papa Ute. And this is playing somewhere in France probably every moment of the day.
You can skip through the radio station and you'll find that this song is on. But the album's brilliant. I didn't really know that I liked dance music at all. It's really good. Totally recommend to listen.
[00:37:07] Unknown:
Well, I really appreciate you taking the time to tell us all more about JS and how it's used in Python and some of the different challenges and applications for it. It was very interesting, and I'm sure that our listeners will appreciate it as well. Yeah. Thank you, Tobias.
Introduction to Sean Gillies and GIS
Sean's Introduction to Python and Zope
Understanding Geographic Information Systems (GIS)
The Impact of Consumer GPS and IoT on GIS
Sean's Journey into GIS and Python Integration
Key Tools and Libraries for GIS in Python
Challenges in Modeling Geographic Features
Scaling Geographic Data and Earth Curvature
Digital Age Advancements in Cartography
Elevation and Contour Data in GIS
Getting Started in GIS and Python
Python's Role at Mapbox
Limitations of Python in GIS
Conclusion and Contact Information