Visit our site to listen to past episodes, support the show, and subscribe to our mailing list.
Summary
Erik Tollerud is an astronomer with a background in software engineering. He leverages these backgrounds to help build and maintain the AstroPy framework and its associated modules. AstroPy is a set of Python libraries that provide useful mechanisms for astronomers and astrophysicists to perform analyses on the data that they receive from observational equipment such as the mountain observatory that Erik was preparing to visit when we talked to him about his work. If you like Python and space then you should definitely give this episode a listen!
Brief Introduction
- Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
- Subscribe on iTunes, Stitcher, TuneIn or RSS
- Follow us on Twitter or Google+
- Give us feedback! Leave a review on iTunes, Tweet to us, send us an email or leave us a message on Google+
- I would like to thank everyone who has donated to the show. Your contributions help us make the show sustainable. For details on how to support the show you can visit our site at pythonpodcast.com
- I would also like to thank Hired, a job marketplace for developers, for sponsoring this episode of Podcast.__init__. Use the link hired.com/podcastinit to double your signing bonus.
- Linode is sponsoring us this week. Check them out at linode.com/podcastinit and get a $10 credit to try out their fast and reliable Linux virtual servers for your next project
- We are recording today on November 2nd, 2015 and your hosts as usual are Tobias Macey and Chris Patti
- Today we are interviewing Erik Tollerud about AstroPy
Interview with Erik Tollerud
- Introductions
- How did you get introduced to Python?
- What was the inspiration to create AstroPy and what kinds of astronomical research can it be used for?
- Can you tell us what AstroPy’s modeling functions are and give us examples of where they might be used?
- Are there any considerations that you need to be aware of when writing software libraries for practitioners of the hard sciences that would be different if the target audience were software engineers?
- What are some of the most interesting applications that AstroPy has been used for?
- Are there open data sets that are available for people outside of academia to do analysis of astronomical data using AstroPy?
- Have there been any useful discoveries made in this way?
- Could you please tell us about AstroPy’s Virtual Observatory capabilities?
- What are some interesting use cases for AstroPy’s Cosmological calculations?
- Are there other libraries available that provide similar capabilities, perhaps in other languages? What makes AstroPy unique among them?
- Can AstroPy consume data directly from telescopes and other observational apparatus?
- The amount of data generated from observing astronomical phenomena must be immense. What are some of the tools used to manage that data and how does AstroPy interface with them?
- How might AstroPy be used to prove or disprove the cold dark matter hypothesis?
- What are some of the architectural choices that have been made to allow for the AstroPy library to serve as the core for a number of other add-ons?
- Does AstroPy provide a common data format to allow for easy interoperability between the various addons?
- I noticed that AstroPy adheres to the PSF code of conduct, as well as having adopted an enhancement proposal process modelled after PEPs. Can you explain why that is important and what kind of an impact it has had on the community around AstroPy?
Picks
- Tobias
- Chris
- Erik
Keep In Touch
Links
The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA
Hello, and welcome to podcast. In it, the podcast about Python and the people who make it great. You can subscribe to our show on iTunes, Stitcher, or TuneIn Radio, or add our RSS feed to your podcatcher of choice. You can also follow us on Twitter or Google plus and please give us feedback. Leave us a review on iTunes, send us a tweet, send us an email, or leave us a message on Google plus You can also leave a comment on the show notes on our episode on our site at pythonpodcast.com. I would like to thank everyone who has donated to the show. Your contributions help us make the show sustainable. For details on how to support the show, you can visit our site at python podcast.com.
I would also like to thank Hired, a job marketplace for developers, for sponsoring this episode of podcast dot init. Use the link hired.com/podcastinit to double your signing bonus. Linode is also sponsoring us this week. Check them out at linode.com/podcastinit and get a $10 credit to try out their fast and reliable Linux virtual servers for your next project. We are recording today on November 2, 2015, and your host, as usual, are Tobias Macy and Chris Patti, though Chris might join late today. Today, we are interviewing Eric Tollerud about Astropy. Eric, could you please introduce yourself?
[00:01:25] Unknown:
Hi. Yes. I'm Eric Tollerud. I'm 1 of the coordinators of the Astropy project.
[00:01:30] Unknown:
And how did you get introduced to Python?
[00:01:33] Unknown:
Well, I guess I started Python in as a as an undergraduate student, actually, when I was doing a electronics course, where I where I happened upon a software defined radio project called the USRP, and it was, after I spent a little bit of time just sort of fiddling around with it, I realized what you're really supposed to do is use this weird language called Python to program, program the radio. And that forced me to learn at least just a Python to get along. And ever since then, I've used it for essentially all my work.
[00:02:07] Unknown:
And what was the inspiration to create AstroPi, and what kinds of astronomical research could it be used
[00:02:14] Unknown:
for? Well, the inspiration to create AstroPi sort of came from the fact that, 4 4 years ago, I guess now, 4 or 5 years ago, there had been a lot of discussion about the problem that there were a lot of little libraries to doing to do a lot of standard things in astronomy. And all of these libraries were, you know, overlapping somewhat in functionality, mostly incompatible, each programmed by sort of 1 person who was writing it as they have the time. And this was clearly a huge waste of resources. So we essentially got together and said, well, maybe we should just work together on a project.
And since then, it has sort of morphed into a much larger beast that is even supported by some, some actual funded astronomical institutions, universities, and so on, to do pretty much the basics of most kinds of astronomy research that's available today. So it's sort of a underlying library, you might say. It's not it's not aimed at 1 the at least the core part of it is not aimed at 1 specific type of Astro Pi research. It's meant to be able to to be the underpinnings of just about any kind of research.
[00:03:26] Unknown:
So that's interesting that the impetus for its creation wasn't just because somebody saw a need in the space and created a library. It was actually because too many people saw a need in the space and were creating disparate libraries. And so it sounds like AstroPi was created as a way to unify all of those and provide
[00:03:53] Unknown:
Yeah. Exactly. And also as a sort of a learning opportunity because it was it was right around the time so, up until the last year or 2, really, a lot of astronomy research is done using this, this this programming language called IDL, which is sort of sort of comparable to MATLAB, but has somewhat different in in very subtle ways, but then is known almost entirely only by astronomers in certain key parts of the defense industry. And that was what a lot of astronomers did their did all their day to day research in. But there were a variety of reasons why more and more people started realizing, well, this is not the best choice. So, they've been switching a lot to Python recently.
And that's that's what that's what sort of led to this getting critical mass was that previously was sort of a hobby thing where a few random astronomers used it. But there has been a shift recently from this ideal language to most, most at least new scientists working in Python.
[00:04:55] Unknown:
Yeah. So it definitely sounds like Python would provide a pretty big boost, both in terms of being more accessible to more people because of the syntax of Python itself, but also being accessible to people outside of the realm of astronomy. So maybe people from other sciences or even just the average lay person who doesn't have the time or inclination to learn IDL specifically for doing astronomical research?
[00:05:21] Unknown:
Yeah. Absolutely. And and the the other thing is that, you know, Python is, of course, free and open source. IDL is not. Ideally, you have to pay a large licensing fee for each installation. So by having it be Python, a lot more, you know, less less well funded universities and colleges could actually take part in astronomical research.
[00:05:42] Unknown:
Yeah. I was curious about that because when you likened it to MATLAB, I know that 1 of the complaints I've heard a number of times about MATLAB is the licensing costs. And so I was curious whether IDL had the same stigma, which it sounds like it does. Yep. Absolutely.
[00:05:59] Unknown:
So first off, guys, I apologize for joining late. My company's hair was on fire, and I had to put it out.
[00:06:07] Unknown:
No problem.
[00:06:08] Unknown:
So my question is, can you tell us what AstroPi's modeling functions are and give us an example of where they might be used?
[00:06:19] Unknown:
Yeah. So the the the functionality in AstraPy have, surrounding modeling. So this is the astropi dot modeling subpackage. I guess I haven't said that yet. Astropy is built around, a variety of subpackages that are all that all live inside the Astropy namespace that sort of cover various different functionality that astronomers often want. So so the modeling subpackage, is based around fitting basically, generically fitting models to data. So a very a more concrete example might be a spectrum. So, you you take the you take the light from, say, a star. You you pass it through, a a prism or a grating or something like that. You spread its light out in a CCD, and then you can see there are there are, bright areas, which are correspond to, say, emission lines.
And this is interesting for a variety of reasons in astronomy that maybe we don't have time to dwell on right now. But the key is you often want to fit particular, quantitative models to that spectral line. So you need to take a 1 dimensional representation of how much light there is as a function of wavelength, and then you wanna fit some model to it, say, a Gaussian or a Lorentzian. And the AstraPi modeling, subpackage is meant to do exactly that and make that sort of 2 or 3 lines of code.
[00:07:41] Unknown:
And so when you refer to it as a subpackage, does that mean that you can actually install the different sub packages independently of each other so that you can only install the bits that are actually being used or does it all get pulled in at once when you install AstraPi?
[00:07:57] Unknown:
Well, so modeling is a sub package that is the second 1. So it all gets pulled in at once when you install AstraPi. We have other different class of, projects, which are sort of loosely coupled, you might say, to the AstroPi core library, which are called affiliated packages. So they are separately installable. You can PIP install those. They all depend on AstraPy and are sort of part of a wider community of people who agree to sort of work together and share interfaces, but they aren't part of the AstroPi core library, whereas modeling is. Modeling is a subpackage that you've actually imported as astropy. Modeling.
[00:08:36] Unknown:
And this leads into a question I have about, I know that in academia and a number of the hard sciences, there isn't as much of a focus of rigor put on the act of software engineering and managing libraries and dependencies and versioning, etcetera. So I'm curious if there are any considerations that you need to be aware of and that you had to account for when writing libraries like AstroPi for practitioners of the hard sciences that would be different if the target audience were software engineers?
[00:09:05] Unknown:
Oh, absolutely. This is probably the hardest thing of the whole project is trying to sort of convince astronomers who are generally not terribly well versed in software. You know, many of them have had no formal education in software even though probably 80 to 90% of everyone's job is, well, is is either writing papers or writing code that goes into the papers they're about to write. You know, data sort of data analysis code, yet, easily half of them have no formal training at all in software and just had to sort of pick it up as they go along. You know, this is common in in a lot of the hard sciences, but it's particularly acute in astronomy because we are we're we're very software oriented.
We have we generally we generally a relatively small fraction of us work with actual, like, lab apparatus. Most of us take data from the telescope and then spend months analyzing that, you know, and that's all in the software. Yet many of us don't have, formal software training. I I do, actually, but most, many astronomers do not. So we have to figure out how to write tools that are modern tools using modern techniques while still sort of hiding that the more complex side of that. Like, for example, object oriented programming, many astronomers don't really know what that means. So we have to sort of sneak it in there, right, which Python is very good at. Python, everything is an object, and you can just interact with things in sort of intuitive ways.
But we often have to write code hiding the fact that it's object oriented and instead just tell them there's this thing you can call, and it's sort of like a function, but not exactly.
[00:10:40] Unknown:
And so it sounds like the majority use case for AstroPi and its associated modules is going to be in more of an imperative script style approach as opposed to doing, like you said, OO or functional programming?
[00:10:54] Unknown:
It's a mix. So it's some like that. Recently, there's there's been a a shift, I would say, because of the advent of the iPython notebook, which you may or may not be familiar with. Yes. You are familiar. Okay. So, the IPython notebook has made it so that, you know, that it's it's sort of looks like a script, but it really is is done sort of in between a script and an interactive workflow. And this this appeals to a lot of the the way a lot of astronomers operate. And I think there's been a gradual shift more in that direction just in the last, again, couple of years. But it is it is true that for most astronomers, it's gonna be more like a script or, you know, sort of very loose sort of functional programming, rather than anything anything more advanced or more complex than that. There there are a few exceptions. Like, I I use object oriented programming all the time, But that's, like I say, because I have the background to do it. And many astronomers just have either don't have the experience or don't have the time to learn how to do that.
[00:11:59] Unknown:
Yeah. There's a project that I'm aware of who we're actually going to be interviewing next week called Software Carpentry that I know focuses on teaching academics and people in the hard sciences some of the best practices around software engineering such as version control and some of the ideas around software design patterns. So it's definitely interesting talking to somebody who both works in the hard sciences and also maintains a library meant for that target audience and hearing some of the challenges that go along with maintaining a library to make it usable and accessible to people who don't have that hardcore software background.
[00:12:38] Unknown:
Yeah, absolutely. And that's, like I say, that's 1 of the hardest things is how to make it both, you know, easier to maintain by using these more advanced organizational techniques and useful to people who know them while still at the same time sort of softly falling back on more simple methods when it makes sense. That's sort of it's a very, difficult edge to walk along, but it's what we try to do.
[00:13:04] Unknown:
And you also mentioned IPython Notebook, and I'm curious if there are any libraries or additional modules that provide any tighter integration between Astropy and the notebook itself?
[00:13:16] Unknown:
So we've more tried to take the the the effort to go the other direction. So in there are a lot of objects in Astropy that interact well with notebooks. So if you just type the name of the object and execute the cells, for example, we have a table, a table class that will sort of printably format itself, and and it uses some of the hooks that ifython Notebooks have written, to make that possible. So we don't have, like, an interface library or anything like that. We just have things that work well in the notebook and then gracefully degrade into a slightly less pretty appearance when you run them not in notebook context.
[00:13:55] Unknown:
And I imagine that the ability to render inline images in the notebook would become pretty handy in astronomy because I'm assuming that there would be some visualization of that data and perhaps even imagery associated, you know, being pulled from the telescopes.
[00:14:10] Unknown:
Yep. Absolutely. Yeah. That's that's 1 of the places. In fact, I this has been something I've been working on just this last week is trying to develop more. So there's there are there are sort of full phone fully functioning astronomical image viewers that are aimed at particular astronomy formats. And it's a little difficult right now to get those to interface well with the notebook, but the project that I've been working on just recently is to do exactly that. Use some of these hooks to connect the the, the very astronomy specific tools that already exist to, of the wider notebook environment.
[00:14:48] Unknown:
You mentioned Gaussian analysis earlier. I've heard of Gaussian analysis in the context of image processing in terms of sharpening images. Is that what you folks use Gaussian analysis for in astronomy?
[00:15:01] Unknown:
So Gaussians appear in many different places in astronomy just because they have convenient statistical properties. What I was talking about before was taking a so not not a 2 dimensional Gaussian, which is used sometimes in astronomy, but rather a 1 dimensional Gaussian because a a spectrum, usually, you, you take you you you take an image of a spectrum that's projected onto a 2 d, imager, but then you collapse it essentially down into a 1 dimensional thing that just says how much light came in as a function of color or wavelength. And you fit a Gaussian into that, that 1 dimensional thing, not a 2 dimensional Gaussian.
[00:15:40] Unknown:
Gotcha. So what what are some of the most interesting applications that AstraPy has been used for?
[00:15:49] Unknown:
Yeah. Let's see. So AstraPy is a a nuts and bolts library in a lot of ways. So, for example, some of the beautiful images you see on Hubble's site of images taken by the Hubble Space Telescope have certainly been processed with, either directly or someone who took those observations, planned those observations using AstroPi. But, you know, the it it's it's hard to so AstroPi is very much a nuts and bolts library, so it's a little hard to say what the specific application is that AstroPi has been used for. And sort of Right. Most people who work on Python work in Python and do astronomy have used AstroPi a little bit here and there for some part of the process.
[00:16:29] Unknown:
I'm sure it's also true that as a library writer, people who are using your library for cool things don't necessarily report back. So it's probably hard for you to even be able to to say. Right?
[00:16:40] Unknown:
Absolutely. It's a huge challenge. In academia, we have this extra problem that a lot of your career advancement in academia is driven by writing papers. And, you know, especially in astronomy, which is not you know, even though we use software all the time, software is not necessarily thought of as being an integral piece of doing astronomy. You need people to cite it whenever they use AstroPi, and we're probably massively undersighted, given how many people we know are using it relative to how many citations we've gotten. And that's the thing that that gets you career advancement in academia in a lot of ways.
[00:17:17] Unknown:
And so, are there open datasets that are available for people outside of academia to do analysis of astronomical data using AstroPi?
[00:17:25] Unknown:
Yes. Absolutely. So there are there are some open datasets that are actually integrated into our tutorials website. So it's just tutorials.astropi.org. And there's a couple of those tutorials that just have datasets that you that that we've we've put up there, basically, that someone someone has observed, say, the spectrum of, a star. Actually, that's 1 of them, as I was mentioning earlier. Someone has observed the spectrum of the star, and they uploaded it onto 1 of our, data server we're using. And when you run through the tutorial, you actually download that dataset, and you process that dataset using AstroPi tools. And there are there are many other kinds of open data that are available, in astronomy that that some of which can be accessed directly from AstroPi, primarily through the AstroQuery affiliated package.
There's something like 20 different, tools that are part of AstroQuery that access 20 different places where you can download data. By and large, it's mostly open. This is sort of the culture in astronomy that most data that gets taken after a certain period of time, usually 18 months, maybe 2 years, becomes open so that anyone can use it at any time.
[00:18:39] Unknown:
And are you aware of any useful or breakthrough discoveries that have been made by people outside of academia using AstroPi to analyze that information?
[00:18:50] Unknown:
Yeah. So that goes back to the same problem before of almost certainly, yes. But it's a little hard to tell, You know, they they use AstraPi as part of the the project. You know, they as part of making an significant discovery that they wrote up in a journal paper, They said they used AstroPi, but it's hard to say whether they use AstroPi to download the data or they used AstroPi to reprocess the data into a form that was easier for them. So it's a it's a little hard to say, to give a very specific example of that. But there have certainly been significant discoveries more recently.
[00:19:27] Unknown:
Could you please tell us about Astropay's virtual observatory capabilities?
[00:19:33] Unknown:
Sure. So so the virtual observatory is a very specific thing. So virtual observatory is a standard that was developed by an international consortium of, organizations that do astronomy, basically. And Astro Pi provides and they they set a bunch of standards essentially to how to access astronomy specific datasets. Right? Like, a specific standard way to download, spectra of stars, for example. So that was the example I was giving before. And AstroPi essentially provides an interface layer that lets you use those standards. And that's that's mostly what it what it is.
Now to be honest, that functionality has not been used a huge amount because the standard has not been as widely adopted as I think the original thought was it was going to be. But it is there for data sources that use, the VO, as it's called.
[00:20:32] Unknown:
Gotcha. What are some interesting use cases for Astropie's cosmological calculations?
[00:20:39] Unknown:
Yeah. So astropi.cosmology is another 1 of the sub packages in AstroPi. And it's essentially intended to be a way to do many of the standard cosmological calculations that are done by sort of, regular, you know, in anyone anyone who's either doing actual astronomical research or who's, say, an undergrad who just wants to understand the relationship between redshift and look back time. So the, the universe is expanding. So that's, that's just a fact that we know about the universe. And there's a specific relationship between, how quickly it's expanding and how far away something is. So that's the redshift, distance relation.
And the astropody.cosmology gives you a way to just and and oh, and the the calculations involved in figuring out exactly how to do that connection, can be very complicated. You know, there are pages pages of complex integrals on in textbooks that you learn in sort of the graduate level, cosmology course. But astropi.cosmology has sort of locked these away in functions that just have relatively straightforward names like redshift to look back time. So you can ask, for example, how, essentially, how old the universe is by simply asking what was the look back time to redshift of infinity. And anyone could just download AstraPy and do this and get a rough sense of how, how the universe is expanding.
[00:22:10] Unknown:
And are there other libraries available that provide similar capabilities, perhaps in other languages? And what makes AstroPi unique among them?
[00:22:19] Unknown:
Yeah. So there are there are a lot of there are a lot of different versions of, the various pieces that go into AstroPi. I don't know that there is specific a library that has all of these pieces put together in 1 place. There is a very standard library called Astron in IDL, that rather obscure language that is known mostly to astronomers, and parts of the defense industry, that has a lot of the same functionality, although often expressed in very different ways. IDL, for example, only very recently supported object oriented programming. So as you might imagine, the interfaces look totally different from Python where it's baked in from the beginning. But there are there are a number of libraries in a variety of different languages that have been used in the past by astronomers that do a lot of this stuff. And moreover, even in Python, there are a lot of overlapping libraries that do some of the functionality that's in Astropy.
What what Astropy has that I think makes it unique is that it's in Python, which is a very efficient language to do sort of, quick data work, which is what most of what astronomers are doing is quick data analysis. And it provides sort of a 1 stop shop. Right? Instead of having to pull together 6 different libraries to do basic tasks that, you know, you you you know, you you've learned you need to know how to do as a graduate student, let's say, in astronomy or even an undergraduate. But, you don't wanna have to code up all of those little pieces yourself. AstroPizer buys you 1 place to do it with 1 consistent interface and, high quality documentation so that you don't have to spend a lot of time messing around trying to figure out just how it works before you can even get your research done.
[00:24:04] Unknown:
So, can AstroPi consume data directly from telescopes and other observational apparatus?
[00:24:11] Unknown:
Yes. So it it again, it it depends on since AstroPi is a a library, sort of a plumbing level library, AstroPi, the core the core library, AstroPi, doesn't necessarily take in data and provide you with, for example, you know, analyzed, reduced, as we call it in astronomy, reduced data just straight out of the box. But it does provide a lot of tools that let you take data in some weird obscure format that the telescope output, and put it in a more generally useful format that then you can feed into other tools. In fact, I'm going to a telescope tomorrow, and and I plan to use Astrobee for exactly that purpose.
[00:24:53] Unknown:
And I'm sure the amount of data generated from observing astronomical phenomena must be pretty immense. And I'm wondering what are some of the tools that are used to manage that data, and how does AstroPi interface with them?
[00:25:05] Unknown:
Yeah. So so there's definitely a lot of, a lot of different levels of either big data, as you might call it, or just large amounts of data in the sense of you take an image once every few minutes. Each image is, you know, a few gigapixels in size or something like that, and you have to figure out how to keep up with this. AstraFi is designed around some of these sorts of workflows to make this at least conceivable. And 1 of the main tools we use is Cython. So So Cython, lets you essentially write C code, in Python like syntax. So that's you write code that you can compile down to something very quickly or something that runs very quickly, while still writing it in a Python like syntax. And it provides a lot of very smooth ways of interfacing between the Python level and the Cython level. So we often use this for pieces of the code that we know are gonna be very computationally intensive when you feed the output of observations into it. So we, you know, we try to write as much as possible in Python, then we profile the code. We say, oh, this is a spot that is running really slow because there's a 4 loop. And, you know, this this 4 loop has to go over every single pixel in the imaging data.
So let's put that in the c level code where the 4 loops will run much more quickly, and let's do that with Cython.
[00:26:34] Unknown:
How might AstraPi be used to prove or disprove the cold dark matter hypothesis?
[00:26:41] Unknown:
So AstraPi, again, on its own is not, particularly useful as as a tool for proving particular hypotheses, but it gives us a toolbox you can use to to work towards understanding what's going on with you with your astronomical topic of interest. So in the specific case of cold dark matter, for example, you might take you might need to take observations of stars in a satellite galaxy near us, so a small small galaxy that orbits around the Milky Way. It turns out if you take spectra of those spectra of those stars, you see that they're moving much more quickly than they would be if there were no or if the only matter there was the stars themselves. So the fact that they're moving that much more quickly means there's a lot of mass, and that mass is dark matter.
And in fact, it turns out that cold dark matter, as opposed to some other hypotheses like warm dark matter, behaves much much different on the scales of these tiny galaxies. So you can analyze datasets. And in fact, this is a lot of what I do is analyze datasets of spectra of small galaxies or of stars in small galaxies. And by analyzing these and determining how fast the stars are going, something I do with Astro Pi, you can ask whether or not these galaxies are consistent with cold dark matter or some other form of dark matter or no dark matter at
[00:28:06] Unknown:
all. And what are some of the architectural choices that have been made to allow for the AstroPi library to serve as the core for such a large number of other add ons? And does AstroPi provide a common data format to allow for easy interoperability between those add ons?
[00:28:22] Unknown:
Yeah. So some of the we'll start with some of the architectural choices. So AstroPi has been it was not originally written in this way, because we as it when it started, we were just trying to build a unified library around some of these core ideas. But it turned out that a lot of the stuff we had to do in this process, like writing good documentation, efficiently and quickly, and, tools for making Cython work very smoothly with Python, Cython, which I mentioned earlier, turned out to be things where people would then come to us and be like, oh, that's that's cool. You know, I wanna write this affiliate package that does this, particular very niche domain specific, task in astronomy, but I wanna be able to use all of those other things that I see you're using.
So what we ended up having to do is sort of take those pieces and separate them out into a different package, which we call AstraPy helpers, that can be installed separately from AstraPi and can be used either, alongside AstraPi or just by itself as a way to make it easier for these other, packages to write code that looks very similar to AstroPi code so that when someone else comes along and wants to contribute or work on the code, they they they recognize as being similar to the kind of code they've seen before. So that's that's sort of the main architectural choice that's made the difference is that we, we we try to share both code style and sort of code support tools, like testing, documentation, and, compilation when quality or when when fast code is needed.
Right. So that's that's those are the key architectural pieces. Another another level that fortunately can be somewhat separated from that is common data formats. So this is 1 of the main things that Astrappi is trying to do. So, for example, we have a class that is meant to represent, celestial coordinates. You wanna know where that star is in that spot in the sky and that galaxy is in that other spot in the sky. And I don't wanna have 6 different ways of of, representing that point in the sky. So AstroPipe provides a Skye Award class that does exactly that. And the idea is all of these affiliated packages, when they need a Skye Coordinate, a position in the sky, they use the Skye Award class, and they don't have to write it themselves. They can just use the AstroPipe version.
Similarly, we have representations of 2 dimensional datasets, like the kind you get when you use an image around a CCD, and even spectra, you know, the amount of light as a function of color, that is also a very commonly used object or thing, really, that's used in astronomy for a for a wide variety of different purposes.
[00:31:20] Unknown:
So it sounds like rather than having a common data format, it's more that that there are a common set of functionality that the add on packages leverage. And that's how they are able to interoperate with the other add ons is through the through the library functions that are exposed as part of the core rather than having to adhere to a particular dictionary format, for example, for exchanging data between them.
[00:31:45] Unknown:
Yeah. Exactly. And that's partly because in terms of actual on disk files, there's been a long history of a of a shared data format that almost all of astronomy excuse me. That a shared data format that almost all of astronomy, has used for the last 20 or 30 years, even back to when most data was stored on magnetic tapes. Astronomers all agreed at that time to share this format that's called FITS. So we didn't have to reinvent the format. We just had to write Python level, tools that would use those sort of on disk files and let the code interact directly with other code.
[00:32:28] Unknown:
Well, that's certainly convenient.
[00:32:30] Unknown:
Definitely.
[00:32:33] Unknown:
And I noticed that AstroPi adheres to the PSF code of conduct as well as having adopted an enhancement proposal process modeled after PEPs. Can you explain why that's important and what kind of an impact it has had on the community around AstroPi?
[00:32:47] Unknown:
I think they're slightly different beasts. So let me let me start with, the PEP like, procedure. So we we call them 8 for AstroPi proposals for enhancement. So the idea there is that the the PEP that that has been used by Python is a good way of getting, consensus because you can present a well formulated idea and then, well, hopefully, a well formulated idea. And then other people can look at that whole that idea as a whole and then decide whether they think that's worth doing for large changes in Python. And we basically face the exact same need in Astropy. For example, that Skype award class I was talking about that represents a particular point in space, that is required for a wide range of tasks in astronomy. Almost everything you need to know where it was that the telescope was pointed, for example.
So that affected a lot of people who might be wanting to use this functionality. So it was good to have a procedure that was sort of outside of the core group of developers, that people could just look and say and see, here's what the coordinate stuff is gonna look like. What do you think about that? And get feedback on it, through that mechanism. So we don't use it for smaller changes. So Python definitely, even relatively small changes often go through the PEP, the PEP sequence. H Suite reserved mostly for large things like the layout of an entire major subpackage or adopting a code of conduct, for example, that is sort of wide ranging across what, at least, we think, the whole AstraPi community.
Whereas smaller changes like adding another function to do some significant task, but just, you know, that's a a localized change in the code, we instead use pull requests on GitHub. So that's that's the the 8 process. Now on on the question of the code of conduct, yeah. So we so this is this is, we consider this pretty important in part because Astrappi is more than just a code library. It's also intended to be a community of people who agree on how we should go about developing this code library. And inevitably, when you have any at all significantly sized group of people, conflict comes up from time to time.
And that conflict can take the form of, disagreements about how to do the code or, you know, even harassment at a conference we might we might be organizing. And we wanted to make sure at the very beginning that we had a clear set of standards that we could all agree on so that if problems did come up, we could address them immediately. We could say, you know, look. You you know, not you know, you just sort of did something we kinda thought was sketchy, but rather, look. There is this code of conduct. You agreed to it, and you violated it.
[00:35:42] Unknown:
Definitely, it's important to have a specified body of behavior that you can point to as concrete examples of what is and is not acceptable because otherwise it's too easy to fall into the, you know, I said this, you said that line of argument which doesn't get anybody anywhere. So having clearly defined rules upfront can greatly reduce and sometimes eliminate that that, whole line of argument that might come up. Yeah. Absolutely. And I'm curious if you evaluated a number of other codes of conduct in the space and if so, what made the PSF code of conduct stand out among them?
[00:36:25] Unknown:
So the we actually started with the PSF code of conduct as, in part because we were, we're about to have a conference, an actual a fully organized conference on Python or on on, Python and astronomy. And we we knew because that would be a larger group, there was a distinct possibility for needing this sort of code of conduct. And it was just a really good starting point. And what we ended up doing was starting from the PSF code of conduct as 1 that was fairly relevant for us as a group of Python coders, and actually had a whole session at the conference on sort of crowdsourcing, you might say, the code of conduct. And the code of conduct we now use that's listed on the Astro Pi website came out of this, what what we call an unconference session, actually, where, a lot of the people involved or at the conference helped write it, had a lot of discussion about what we thought were the important elements and which which were relevant for us and which weren't given that we were both astronomy and Python instead of just 1 or the other.
[00:37:28] Unknown:
Yeah. It's definitely valuable to have that kind of feedback and buy in during the process of creating the code of conduct because then you don't have a single person or a small group of people saying this is how the code of conduct has to be and it's actually buy in from the whole community which greatly reduces the amount of pushback that you would get if it needs to be employed. Because you can point to it and say, hey, we all agreed on this, everybody worked together. As opposed to somebody arguing, oh, well I didn't agree to it. It's just something that was imposed upon me. So that definitely seems like it would greatly improve the effectiveness of it. That's what we hope.
So are there any questions that we didn't ask that you think we should have or any other topics around AstroPi that you would like to bring up before we move to the PIX? Oh, I actually, I I would like to mention, 1 piece of functionality that I think,
[00:38:18] Unknown:
is important in Astropie that I haven't had a chance to talk about too much here, and that's the the units package. So what what units basically does is provide a way to associate a physical unit, like kilometers or, you know, minutes or any any sort of physical quantity energy, with a number or a NumPy array. And this has turned out to be an extremely valuable, tool for astronomy, certainly, but in fact, as we've talked to other members of other communities that are sort of allied, like physicists who use this, Python, for example, It it it turns out this is something that a lot of people want, and I think I think it's worth considering. We haven't made a firm decision on this yet, but it's worth considering whether or not, our units package should be, separated out of Astro Pi to be shared with the wider community and, moreover, whether we should have a larger effort to agree on 1 unit package that does the same sort of, the same sort of stuff that that the, AstroPi units and quantity packages do.
[00:39:28] Unknown:
Yeah. That definitely sounds like it would be immensely valuable for any experimentation or analysis on physical quantities. I know that when I was taking physics classes in college, that was, you know, in chemistry classes, that was definitely 1 of the things that you really had to focus on is making sure that you're doing the proper conversions between different types of units and remembering how the different types of units cancel each other out depending on whether it's you know, meters per second squared multiplied by distance, for instance, and how how that results in, you know, meters per second. And I can definitely see how that would be valuable outside of just astronomy. And I'm curious if in the library itself, if what you're doing under the covers is actually defining new types that have their own sets of behavior for being able to do those kinds of conversions?
[00:40:19] Unknown:
Yeah. They're basically classes that, inherit from the NumPy array class, the ndarray, I guess it's called. So that's that's how they work under the hood, but they don't really have to. It's more sort of defining an interface. And there are some people who've, you know, sort of hacked around on it. I don't think any, so far, public packages, but who've, shown that you can basically, get away with using some of that machinery to, driving this combination of a physical quantity, and an and a number or an array of numbers. And that that makes it both an interface, a way to connect different codes written by different people, maybe in very different communities, and a powerful tool for doing exactly what you just said, actually, keeping track of your units and not having to worry yourself about exactly what is the conversion from, you know, feet to kilometers or something like that.
[00:41:19] Unknown:
Very interesting. So, is there anything else that you'd like to bring up?
[00:41:24] Unknown:
No. I don't nothing's going to mind.
[00:41:28] Unknown:
Well, with that, we will move into the picks. And for my first pick today, I'm going to choose the the Open Culture account on Twitter. It's a Twitter feed that just has a lot of really interesting content. So 1 of the things that they listed today was it had a brief article and a link to recordings of readings of all of C. S. Lewis' Chronicles of Narnia. You know, there are a few things having to do with Frida Kahlo and some of her work. And it's just a lot of really interesting bits of popular culture from within the last century or so that you can gain free access to. So a lot of really good content there, definitely worth following.
Another project that I've been using for a little while is a Chrome plugin called CitizenX and what that does is it will actually analyze the sites that you visit. And based on where each of the different companies or sites are located, it will calculate your algorithmic citizenship. So for instance, if you are viewing a lot of sites that are based out of Germany, then it will calculate your primary algorithmic citizenship as being in Germany or the United States, for instance. And it will actually show you the breakdown of where each of the different sites are located and what your percentage is in terms of your citizenry. So definitely an interesting view of data in terms of how we actually use the web.
And my last pick today is a Python library called piprot, which you pass it your requirements. Txt files and it will actually call out to the Python package index and determine how far out of date all of your installed libraries are and show you what the most up to date versions are. So for instance, if all you're looking to do is run through all of your dependencies and pin new versions, new updated versions and your requirements dot text, then PipRock gives you an easy way to do that. And with that, I will pass it to you, Chris.
[00:43:40] Unknown:
Okay. Since I'm kinda running around like a chicken with my head cut off, today, I'm gonna keep my pick simple. I'm gonna do a a podcast trifecta. My first pick is an excellent podcast. It's called The Illusionist. It's part of that Radiotopia group that's that 99 Percent Invisible that I've gushed on about ad infinitum, started. And this is this very, very witty, funny, Britisher lady, talking about language in really, really amusing ways. It's short. It's edifying. It's interesting, and it's incredibly entertaining. I recommend it. My next pick is a podcast called Criminal.
And pardon me, that is a podcast about the law and the people who break it, but from a really interesting perspective. Yeah. I'll leave it at that, but it's definitely not your average sort of criminals are bad and k perspective. It's it's definitely very thought provoking. Again, short and easy listen. And my 3rd pick is a bit more in-depth and a bit more intense. It's called hardcore history, and it means what it says. The current episode is 3 hours and 20 minutes long, but each episode or series of episodes is just so incredible in terms of its depth. And he has a really great way of relating events in history to the modern era, like the current episode is about, the Spartans versus the Persians, the famous battle of the 300.
And he talks about how, Spartans spoke laconically and essentially likened it to, you know, if if Clint Eastwood and Batman had a love child, that's the way he would speak kinda thing. So it's it's it's interesting stuff. Really, really well written, well produced, very in-depth. I can't recommend it hardly enough. That's it for me. Eric, what kind of picks do you have for us?
[00:45:46] Unknown:
Well, I figure I should I've I have to start as being an astronomer with pointing everyone to, 1 of what I see as 1 of the most inspirational sites you can find on on the web, which is the Hubble site. So this is, hubblesight.org. A great place to go for some of the some of the best and most iconic imagery from the Hubble Space Telescope, which really helps you sort of understand at least helps me, maybe I should say, understand, how how vast and beautiful the cosmos is, and, you know, helps me understand even, you know, since I can be often deep in the weeds writing code, it helps me understand that the reason why I'm deep in the weeds writing this code is because this is the sort of thing that we can get out of it.
So that's dot org. Another thing inspired by, in fact, this this, that that history podcast, that you were just mentioning there, Chris, is something called The Great Courses. So these are a series of, audiobooks that are courses on some, interesting, topic. And I've just recently completed the, the great course on his the history of the ancient world. So I'm, I've been a long time history buff, but I've really found this to be an excellent overview of, ancient history, or ancient history is sort of defined as the beginning of of, urban civilization up until, roughly speaking, the fall of the Roman Empire.
Actually, the the course has an interesting perspective on where the end of ancient history is, but I won't spoil it for you in case you wanna see it yourself.
[00:47:30] Unknown:
Excellent. So for anybody who wants to keep in touch with you and follow what you and the AstroPi project are up to, what would be the best way for them to do that?
[00:47:40] Unknown:
The best place is, always astropi.org, which has a, a variety of links there for how you how you contact for specific needs. A more general thing, if you wanna just keep up with what the project is doing and people might be using Astropy for, is either of the Astropy mailing list. So we have 1 that's just called Astropy, which is primarily for users or, it's even more wise than just Astropy, the Python library. It's really for, anyone who has just about any sort of question about using Python to do astronomy. And if you're more interested in the details of Astropay, the library, we have an Astropay dev mailing list.
Again, both of the the links for both of these, you can find right away just at the AstroPi website, astropi.org.
[00:48:31] Unknown:
Great. Well, we really appreciate you taking the time to join us today and talk to us and particularly taking time out of your preparation to go and do some observations. And I just wanna say thank you, and I hope you enjoy your trip and the rest of your evening.
[00:48:47] Unknown:
Great. Thanks. Thank you for having me. This has been great.
Introduction and Guest Introduction
Eric Tollerud's Journey with Python
Inspiration and Purpose of AstroPy
AstroPy's Modeling Functions
Challenges in Software Engineering for Astronomy
Integration with IPython Notebook
Applications and Use Cases of AstroPy
Virtual Observatory Capabilities
Cosmological Calculations with AstroPy
Architectural Choices and Common Data Formats
Code of Conduct and Community Impact
AstroPy Units Package
Picks and Recommendations