Summary
Content discovery and delivery and how it works in the digital realm is one of the most critical pieces of our modern economy. The blockchain is one of the most disruptive and transformative technologies to arrive in recent years. This week Jeremy Kauffman explains how the company and platform of LBRY are combining the two in an attempt to redefine how content creators and consumers interact by creating a new distributed marketplace for all kinds of media.
Preface
- Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
- I would like to thank everyone who supports us on Patreon. Your contributions help to make the show sustainable.
- When you’re ready to launch your next project you’ll need somewhere to deploy it. Check out Linode at www.podastinit.com/linode?utm_source=rss&utm_medium=rss and get a $20 credit to try out their fast and reliable Linux virtual servers for running your awesome app.
- Visit the site to subscribe to the show, sign up for the newsletter, read the show notes, and get in touch.
- To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
- Your host as usual is Tobias Macey and today I’m interviewing Jeremy Kaufman about LBRY, a new marketplace for media built on peer to peer storage and blockchain technologies.
Interview
- Introductions
- How did you get introduced to Python?
- What is LBRY and how did the idea for it get started?
- What, if any, mechanisms are there for content owners to address piracy?
- Is the LBRY blockchain purpose built for the protocol and application or is it using something like Ethereum under the covers?
- In order to support a large scale distributed marketplace, the crypto coin that you are using will need to be able to support large transaction volumes so how have you architected it in order to achieve that capability?
- What technologies are you leveraging to facilitate the content distribution mechanism?
- One of the current problems with Bitcoin mining is that as the complexity of the proofs has increased and dedicated operations have moved to ASICs it has become less feasible for an individual to take part. Is there any provision for that situation built into the LBRY blockchain or does it not matter due to the capabilities for individual users to earn coins by participating as part of the storage network?
- What led to the decision to use Python for the initial implementation?
- For people who are participating in the LBRY network, what is the mechanism for them to convert their earned LBC into fiat currency?
- How much of the overall LBRY stack is using Python and what other languages are you taking advantage of?
- What is the business plan for LBRY the company and what do you have planned for the future of LBRY?
Keep In Touch
- Jeremy
- @jeremykauffman on Twitter
- LBRY
Picks
- Tobias
- Jeremy
Links
The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA
[00:00:14]
Unknown:
Hello, and welcome to podcast dot in it, the podcast about Python and the people who make it great. I would like to thank everyone who supports us on Patreon. Your contributions help to make the show sustainable. When you're ready to launch your next project, you'll need somewhere to deploy it, so you should check out linode at ww w.podcastinit.com/linode and get a $20 credit to try out their fast and reliable Linux virtual servers for running your app or experimenting with something that you hear about on the show. You can visit the site at www.podcastinit.com to subscribe to the show, sign up for the newsletter, read the show notes, and get in touch. To help other people find the show, please leave a review on Itunes or Google Play Music. Tell your friends and coworkers and share it on social media. Your host, as usual, is Tobias Macy. And today, I'm interviewing Jeremy Kaufman about LBRY, a new marketplace for media built on peer to peer storage and blockchain technologies. So, Jeremy, could you please introduce yourself?
[00:01:02] Unknown:
That's right. I'm Jeremy Kaufman, the CEO of LBRY. And when forced to describe LBRY in 1 sentence, well, if I'm talking to my grandmother, I say it's like a community controlled YouTube. I guess my grandmother does know what YouTube is. But if I'm, gonna explain it to your audience, I would say it is an open source protocol providing decentralized digital content discovery, distribution, and access or purchase. And we're I'm sure we're gonna dive into that in a lot more detail exactly what that means.
[00:01:34] Unknown:
And how did you first get introduced to Python?
[00:01:37] Unknown:
So Python is Python is the language that we use, to to provide our distributed network for, the daemon layer of library. I started using Python myself in probably I was thinking it's my either my junior or senior year of college, which would have been, like, 2006, 2007. I was a computer science and and physics student at RPI in upstate New York.
[00:02:01] Unknown:
So you briefly gave a high level overview of what library is, but I'm wondering if you can give a bit more detailed description of it as well as how the idea for it got started, particularly given the, sort of confluence of different technologies that it is using under the covers.
[00:02:17] Unknown:
Yeah. So it's library can be, slightly tricky to get your head around, although I think I'm getting hopefully, I'm getting better at describing it. Fundamentally, what library is is it's a technology. It's a protocol. We, in fact, picked something that was 4 letters because library uses a a URL structure like HTTP, l b r y, and library URLs resolve to a piece of digital content, or potentially some other things related to that, like a list of things like that, or a a publishing identity. And library is basically what some people have called a fat protocol. It's a protocol that does a little bit more than a thin protocol like HTTP. So that is library as an implementation handles things like that would typically be handled at the layer above, like, at the application level. Libraries handling them at the protocol level. So including things like, you know, payments and managing a digital catalog and so on. But, basically, we've created a a a what we hope is as simple as can be a simple set of APIs to handle digital content discovery and distribution in a completely decentralized, censorship resistant, and basically peer to peer and market driven manner. And I think that there is a second half of that question that I didn't answer, but the answer to the first half was so complex that I'm just gonna take a pause there and then see, see where we go.
[00:03:37] Unknown:
Yeah. Sure. So the second part of the question was wondering how you got the idea for it and how you got it started.
[00:03:43] Unknown:
Yeah. So the idea for it. The the idea that, you know, no 1 really has original ideas. Everyone just, like, combines ideas and tweaks them a little bit and so on and just it's just layers and layers of this stuff. It's very much that. Right? Like, so library is an original idea in the sense that this combination of things, I don't think, has existed before, at least not in this way. But I'd say, actually, I think definitely not in this way. But at the same time, it's basically a combination of a couple of different things. 1 of which was absolutely I I've been a fan of of peer to peer and distributed technologies for a long time. So, part of the inspiration for library was BitTorrent, which is just a remarkable and really wonderful technology.
Another big part of the piece was blockchain technology and Bitcoin, which, succinctly, the real innovation there is that we can reach consensus on a state of affairs in a decentralized manner. So it's not just about money. What blockchain technology lets us do is maintain a decentralized state for any number of things. 1 thing that that can work really well for is a a catalog of digital content. So now we can fix some of the problems that exist with previous decentralized technologies. They have bad incentives. They have poor discovery mechanisms. Other some infringement problems, which I think are also related to the previous 2. And so now, by basically combining these 2 things, as well as some other, little twists and innovations, we can create a an end to end system, for for a decentralized digital marketplace.
1 particular inspiration was so beyond just knowing about these technologies was there's this great there's a conversation between Julian Assange, and Eric Schmidt, the former CEO of Google. It's funny that I need to describe who Eric Schmidt is, but, I know who Julian Assange is. But I may I was both. Anyway, it's like a massive conversation. It's like 8 hours long. And Assange being Assange published the whole thing online. But in 1 part of it, Assange is talking about this decentralized DNS system inspired by Bitcoin, like coupling a name to a a piece of text. And and that was also something that got me thinking about this possibility.
[00:05:40] Unknown:
Yeah. There's definitely a lot of stuff to dive into there. 1 thing that you did briefly touch on is the idea of bad actors in the network and piracy in distributed content delivery systems. And reading through the FAQ, I know that you have at least a philosophical stance on how that can be addressed in the library protocol. But I'm wondering if you can explain a bit about what mechanisms are present for content owners to address piracy for anybody who does start distributing content that isn't necessarily owned by the person doing the distribution and some of the other network and market effects that are in play to help mitigate that from happening in the first place?
[00:06:17] Unknown:
Yeah. Yeah. Absolutely. So this is a this question gets a little bit complex because there's library as a protocol. And in that ex extent library, you know, like HTTP, to some extent, asking what library does at a protocol level, to some extent, is asking, like, what does HTTP do? The answer is, well, it's a protocol. Right? There's no there's no centralized registry of of infringing HTTP domains. Well, there are blacklists and things like this, but this is how we handle it. We handle it at this level. And so, for example, in the library browser that we provide, we provide search results. You can use library browser to search content that's been published to the library network. Those search results are filtered, and we respond to DMCA requests and and filter those.
We will also be maintaining a list of of of URLs that we know to be infringing. And so clients that, you know, are interested in being on the right side of the law will follow that list, or they will be facing, very significant penalties. In fact, breaking the law when you're using library, sort of 1 of 1 of the extra incentives to not use library in a bad way, is that the penalties are actually much more severe. When you are caught infringing, like, downloading something off of BitTorrent, it's a it's a civil infraction. You just you you infringe on someone's copyright by accessing it. When you profit off of copyright infringement, it is a criminal infraction. The clients that will be publishing, like the official clients, will respect the blacklist, won't rehost stuff. If you choose to break that rule, you're facing jail time. And it's not like library is designed to obfuscate who you are in that fashion. Right?
So there, you know, there's a continuum of how possible it is to power things where the most impossible is to show you something on a screen in my house, and then, like, right, you've got com completely in the clear. I think pirating stuff on library is hard it's gonna be it's gonna be harder for people to do it than it is on BitTorrent. If you're the kind of person who just wants to infringe and doesn't care about paying for stuff, you'll just you'll just keep using the places that are best for you for doing that. So we're trying to capture what's the wins from those technologies. Again, BitTorrent's a wonderful technology, but, unfortunately, it never really caught on as a as a way of too much mainstream distribution. We we wanna capture that mainstream.
[00:08:30] Unknown:
Yeah. I don't really have anything to add to that because it's such a
[00:08:34] Unknown:
that that that's a deep rabbit hole to go down, so we'll avoid that 1. It is. A lot of these are deep rabbit holes to go down. And if you can't tell already, I will, spit a lot of words at you. So
[00:08:44] Unknown:
So another aspect of LBRY, the protocol is the fact that, as you mentioned, it's built on a blockchain, which if some for people who aren't as familiar with the blockchain technology specifically, like you mentioned, it is the same underlying technology as what Bitcoin is using as well as other so called altcoins such as Ethereum and Litecoin and a number of others, including the name coin system that you alluded to. So I'm wondering if the blockchain that library is using is using is built on top of something like Ethereum using smart contracts or if you developed your own blockchain purpose built for the specific use of LBRY?
[00:09:21] Unknown:
Okay. So you gave me a there are a couple of things there that, inclined me to to get up on a soapbox a little bit, but I'll start by just giving the straight answer, which is that, library has a separate blockchain. That blockchain is what's used to maintain the library content catalog, so the the listing of what's available in library. As a protocol so library as a protocol is larger than the blockchain. Library you know, the blockchain is part of the protocol. But it's not every the it's not like everything that the set of APIs does inherently involves the blockchain.
And as a protocol, it's acceptable to pay for things. However, 2 parties want a handshake, in terms of payment, so it's not some kind of thing where you have to use the LBRY blockchain to pay for everything. But 1 of the interesting things about blockchain is that, in terms of how to think about it, I think a lot of people start thinking about it wrong because they look at Bitcoin into them like Bitcoin is money, and so they think that or or some something like money. And therefore, Bitcoin as a technology is somehow related to that, and that's wrong. Bitcoin is just a it's just a new kind of database technology. I will frequently say it's the world's crappiest database technology. It it has it has terrible throughputs, long confirmation times, you know, all these downsides in terms of a database except for 1, which is distributed consensus.
And so if you've got all of these downsides, it's almost like anti scaling properties. And so, like, the notion that everything would ever be on 1 blockchain, it's like saying, we're gonna use 1 database to power all of the world's websites, and also that database is fundamentally bad at scaling. I was like, that doesn't make any sense. Right? This so the word kinda altcoin, it's like saying, like, are there alt database technologies? You're you're gonna have different databases for different applications. There's there's nothing about blockchain that says, let's do everything on top of 1 block. You mentioned that
[00:11:07] Unknown:
at the protocol layer, there's no requirement to use the library coin or the library blockchain for settlement of payment for content that's being distributed through the network. And when I was reading through the FAQs, I did see that there was an allowance for 3rd party settlements. So I'm wondering if you can talk about that a bit and how those 3rd party settlements would then be recorded as part of the distributed ledger.
[00:11:32] Unknown:
Actually, the first part of that is the settlement doesn't necessarily have to be recorded as part of the the ledger. The only hard requirement for usage of libraries that the library blockchain acts as essentially a URL scheme and a metadata storage system. It act it's the it's the catalog. And the only reason that you would need library credits is is for entries in that catalog. That's the only, like, hard reason. Inside that metadata is, you know, a pointer to other resources. That pointer could be to any number of things. It could be to HTTP. It could be to BitTorrent. It could be to FTP. We have our own, sort of DHT like system that it it allows for marketized data transactions as well, and that's the default system. But all this stuff is it's designed to be very open ended. The only hard requirement is, for that catalog. Now that said, like, our client doesn't support that we've released, like, our our, you know, sort of MVP.
That it doesn't support paying via Bitcoin. But the API is, you know, you're broadcasting request. You're offering this. If the other guy says, I have that, and I'll accept that, you know, you could say you had 5 squirrels. And if the guy says, yeah. I'll take 5 squirrels. I mean, it's up to the rest. You guys figure it out from there how that's gonna happen. You know?
[00:12:40] Unknown:
So if somebody does settle out of band from the blockchain and doesn't use the library currency for that transaction. I know that 1 of the other layers of library is the peer to peer content distribution network. So for somebody who does settle out of band and doesn't necessarily record that transaction in the ledger or use the, provided application or the private provided UI and APIs, I guess, in what manner would that distribution then take place? Is that again just in the hands of the parties who are transacting?
[00:13:12] Unknown:
Yeah. It's it's basically that. In terms of the metadata, there's basically like, how many fields are relevant here? You've got 1 part of the metadata is the sources, which is basically saying these are the way this is the way or the ways that you can access this. Then another part is basically saying what the price is if there is 1. And, obviously, stuff can be free. We also 100% support and encourage that. And then, you know, how you're gonna make that payment to that person, which is then gonna if it's assuming the content requires a payment, then also how you're gonna get the key for it. Yeah. It's it's very open ended, and it's gonna be whatever those 2 parties are willing to kinda get a handshake on.
[00:13:49] Unknown:
So in order to be able to support such a large scale and distributed marketplace, the blockchain that you're using under the covers will need to be able to support large transaction volumes, which I know is something that is a problem with the current implementations of Bitcoin, which leads to slow transactions and a inherent limit on the number of transactions that can happen on the blockchain. So what are some of the ways that you've architected your implementation in order to be able to achieve that scalability?
[00:14:18] Unknown:
I will again I'll give the real answer, then I I have just a little bit of a soapbox answer to this 1 that's related to that same 1 as before. And it's all these are all kind of interrelated in terms of the mistakes that people make in terms of thinking about blockchain. So the answer is, fundamentally, if you want to do a large amount of volume, you have to do stuff at the second layer. You know, so we're looking at stuff like, lightning network or whichever 1 of these seems to be the most promising, but we've been monitoring a lot of them, and we've been in contact with the teams that have created them. And we're gonna as we reach those limits, which we're not at, we we will, be looking at dealing with, you you know, using something like that. So this is much more about programming than cryptocurrencies. So so your people probably aren't, super familiar with, like, all this Bitcoin drama. But, like, Bitcoin is just and all of these that, you know, blockchain, it's it's fundamentally an inefficient technology.
You know, blockchain is saying do something in replicate across every single device that's participating as a miner in the network. And and if you do something on 2, 000, 000 machines, it's always going to be more expensive than doing it on 1 machine. And just people forget to trace it all the way back to what's actually happening. And And so, you know, like, there's, like, people complain about Bitcoin transaction fees. And it's like, the only reason we have any you know, the kind of network security that we have now is because for every bit transaction on the Bitcoin network, it's being subsidized to the tune of, like I think it's, like, $5. Every transaction is costing $5 in real resources.
Like, that's how much computing power and so on is used. So so blockchain and microtransactions, that never fundamentally works. You're going to always have to move that stuff to a second layer because microtransactions are the worst thing to do 2, 000, 000 times in in Replica. You know?
[00:16:06] Unknown:
And can you dig a bit more into what a Lightning Network is and what that's going to enable in terms of the overall volume?
[00:16:13] Unknown:
Yeah. Absolutely. And so Lightning Network is an example of 2nd layer solutions where 2nd layer solutions is basically there's an ability for nodes to act as things that are like, you know, other kinds of institutions in a in a typical financial system. So, you know, they're basically providing credit and and settling transactions. And so, you know, they they may observe a large number of transactions and group them up and bundle them so that rather than pushing a a 1, 000 transactions to the blockchain, they can do it in, you know, 10. And so that saves space on the blockchain and makes everything more efficient. And, Lightning Network is an example of this solution where it involves committing and showing that you have a certain amount of currency, opening up a channel and advertising it, and then people can use that channel and in a trustworthy way know that that payment will ultimately settle.
[00:17:02] Unknown:
So and that's all I'm gonna say about that before I, embarrass myself. Going to another layer of the stack is the content distribution mechanism that you've built up, which I know you said that you were at least inspired by a BitTorrent. So I'm wondering if you can describe a bit how that, how that's implemented and some of the challenges that you faced in the process of getting that rolled out and also that another aspect of that being how you are encouraging people to take place in that distribution network.
[00:17:31] Unknown:
Yeah. Absolutely. So this is a whole another piece of library. And this is actually the piece most relevant to Python. Alright? All that blockchain stuff, blockchain is a, you know, c, c plus plus. The peer to peer data network is so this is basically the default publishing method that we use in in the library application and in the library daemon. It's very similar to BitTorrent in the sense that you're using a DHT pointer to find hosts that have the content. The difference from BitTorrent is that in this model, the downloader and the host have a chance to negotiate.
So, basically, data is provided in a complete market like fashion. Let me try to explain that. Basically, you're looking for a chunk of data. When something when something is published to the library network in this model, it gets encrypted. It gets broken up into a bunch of 2 megabyte chunks. Each of those 2 megabyte chunks have a hash. Essentially, the library protocol creates a marketplace where there's a price, for each of those hashes. Now we're talking about really, really small prices, fractions and fractions of a penny. In fact, from for many, many data hashes right now, the price is simply free, because the default behavior of our clients is to host things for free. And so more than enough people are running it at free that most people aren't paying data prices. But it actually creates and this is, I think, a very powerful idea, the idea of literally a marketplace for data, data as a commodity.
That is that, like, if we know a chunk of of data hashes to a certain value, you know, we can essentially create a marketplace for that data and, in a decentralized way. You know, you've got the it it's like I think it becomes a very efficient way to host things. You have the power of market incentives to to provision hosting as efficiently as possible.
[00:19:18] Unknown:
And just as a brief aside for you you've mentioned DHT a couple of times, which I'm assuming to mean distributed hash tables for people who just haven't heard that, TLA, which again stands for 3 letter acronym.
[00:19:30] Unknown:
Oh, yeah. I'm I'm I'm bad about that. And please, yeah, interrupt me. I could that's a that's another just wonderful technology, man. You learn how that 1 works, and you're like, man, that guy was a that was a smart guy. He thought of that. You know, the idea of how you can basically, you know, you announce you're looking for a hash and it can route to someone that has it without any kind of any kind of centralization, any kind of catalog or anything like that.
[00:19:53] Unknown:
Yeah. Distributed hash tables are pretty mind blowing. And, also, 1 of the things that you run into there, particularly as you grow the amount of data in the network, is the possibility of hash collisions. So I don't know if you have any if if you have done any engineering effort on the part of trying to account for and, either prevent or, resolve hash collisions in the data network?
[00:20:19] Unknown:
That's true. That's a good question. I mean, I miss I actually and I don't know off the top there, and that's an example of, like, I should probably know the size of our, you know, hash space off the top of my head, but I don't. I don't know how many bits it is. So I don't know, I'm I'm sure that this is something that they've, that we've thought about, and it's also something that I'll you know, to by the oh, it's also 1 of those problems that I call the it's in the good set of problems. Like, we by the time you have that problem, it's like, great. We were we were really successful. Well, you know, we can evolve this, if it did happen to be something that was Absolutely. As long as you're not using Shaw 1 or MD 5, then Oh, yeah. No. No. Definitely not. So a couple of things,
[00:20:58] Unknown:
going back to the blockchain technology is that for people who have been following what's going on with Bitcoin mining, when it first started off, it was entirely feasible to, you know, run a mining client on your laptop or your desktop at home just using CPU capacity. But as the necessary computations have become more complex and the value of the bitcoins has gone up, people have been making that their entire business purpose. And so there have been application specific integrated circuits or ASICs developed specifically for the purpose of Bitcoin mining. So it's gotten to the point where your average person doesn't really stand a chance of trying to obtain any currency in that manner unless they're taking part of a mining pool. And so I'm wondering what if any, engineering effort you've put into the library blockchain to mitigate that eventuality.
And if you don't have anything in place from the mining perspective, how do you see the monetization of the data as an alternative to that for people who do want to get involved in the library community and the library blockchain?
[00:22:02] Unknown:
Yeah. So, well, right off the bat, first, I'll say that, like, the the hosting is is the way better way regardless. That's true today. And then, actually, I will also go ahead and say, I probably should have said this at some point already that, like, like, for those people who are interested in library, you can go and check it out right now, especially since your since your audience is programmers, I'll point them to l b r y dot I o slash quick start, as an intro guide that teaches you about the library API and then gives you a good introduction to to the technology. Although, there's just a bunch of good resources on our website in general. But if you're if you're using it and you want to to, make credits, way better to contribute your disk space and bandwidth than to compete as a miner.
Mining, now we have made choices here to try to, combat this, so that, you know, that is we we chained a number of different, hashing algorithms together, and we picked ones, that, you know, were not matching anything existing. Library is not being ASIC mined right now. I think it would be fairly expensive to do so. I think eventually I'm not, yeah, I'm not sure at at what point that would be possible. But already, you're looking at you know, it's already mining pools and the kinds of people who are buying GPUs and so on that are mining the credits. So it's already something where if you're mining on your own local machine, it's probably not gonna be that profitable. You know, in terms of, in terms of of of just kind of getting something, the another reason another easy way to get some stuff is just to come use it right now, because we give a bunch of free credits out, in this stage, especially to technical users, because we wanna get them testing the technology, vetting the technology, and so on.
[00:23:43] Unknown:
Yeah. And on that point, I was experimenting with it as I was preparing for the show. And in the process, I published the most recent episode that I released, at the time of this recording, and I'm, probably going to be releasing the rest of the back catalog onto library. So if anybody does want to experiment with it and use that and use the podcast in it, show as a means of seeing how the content distribution and content discovery works, then, by the time this gets released, those episodes should be up and, able to be accessed.
[00:24:17] Unknown:
That's awesome. Yeah. We'll, I'll have to check them out on there and make sure, everything's, going smoothly. What'd you think? You wanna tell me about the the process here on the show a little bit? You wanna give me a little a few a little live review?
[00:24:29] Unknown:
So when I was first getting started, I was just using the API approach for trying to publish, and, it's definitely verbose when you're trying to type it all out manually. But from a interaction perspective and a UX perspective, it's all pretty seamless. You know, the the most difficult startup piece was just getting the library credit to be able to, publish onto the blockchain because of the what you were saying of requiring to contribute some of the library currency as a means of reserving the namespace that you want to publish to. But once I got that out of the way by visiting your Slack channel, it was very easy and painless to publish on the network. To my knowledge, I haven't had anybody download it yet, but I also just did it a couple of hours ago. So I'll let you know once I find out more about the, the other end of the transaction.
[00:25:16] Unknown:
Alright. Oh, did you did you set a price for it? So it's, is it free or am I I I it's I'm gonna go and check I am gonna go and check it out. So I'm gonna go and check it out. So I'm gonna go and check it out. I I experimented with setting it at 10¢, but I'll also, release them for
[00:25:27] Unknown:
I'll also, make it available for free for people who don't want to pay.
[00:25:31] Unknown:
Okay. Okay. Alright. Well, well, I'm I am glad it worked. You know, we first announced library to the public, sort of to the public. Like, we were trying to be kinda, like, quiet about it, which didn't work, by the way. But when this was, like, last July, and then, like and library had, like, been run on a total of probably, like, 8 computers at that point, like, a very small number, and it just didn't work at all. And we've been a significant amount of the effort over the last 6 months has been, like, getting the technology to the point where, like, it actually does all the stuff that we had wanted it to do the entire time. So it's still thrilling to me every time someone says, like, oh, yeah. Like, I watched the video. It just, like, it just worked. I'm like, yes. We We were not there. Like, 6 weeks ago, that was not always the case. So
[00:26:16] Unknown:
Yeah. It's definitely a great feeling when people are using stuff that you've been building and, dedicating a lot of time and energy and effort into. Indeed. Indeed. You know, I think that the, you know, the potential here is absolutely tremendous.
[00:26:28] Unknown:
And, you know, that's part of why our team is putting so much energy into this is because this is gonna happen. Right? Like, it's just the idea that the power of blockchain to to build some of these decentralized applications, it's just such a good solution to a lot of, a lot of problems. So I think we're gonna see solutions beyond just digital content in this area, and and, certainly, it's gonna happen in digital content. If it's not us, it'll be someone else. It's gonna happen. It's, like, it's guaranteed, in my opinion.
[00:26:56] Unknown:
Yeah. Just as a anecdotal reference to the power of the blockchain and the sort of the mind share that it's gaining, I was reading an article a little while ago about a number of banks who were, working on establishing an international, sort of, agreement where they were going to try using a blockchain for managing distributed ledgers for transactions between the various banks. So, it's definitely an interesting technology that is going to be spreading, far beyond where people can even think of right now.
[00:27:28] Unknown:
A lot of the bank's usages, though, in my opinion, are are they're exactly the wrong ones. They're exactly the ones that aren't aren't leveraging the real power of blockchain. The the power of blockchain is that distributed consensus, and that comes from essentially a massive waste of resources. It comes from doing things 2, 000, 000 times instead of doing them once. If you're con if you're this group of banks, you can just make an agreement, and you and you can just run your own network. Like, I I don't the the a private blockchain is like it's not I don't it it's like an oxymoron. It doesn't make it's it's you're throwing you're getting rid of the whole, insight of what blockchain was.
[00:28:11] Unknown:
So switching gears again, as you mentioned, the at least reference implementation of library is written in Python. So I'm wondering if you can explain a bit about what the decision making process was that led to that and what some of the, benefits and challenges of that choice have been.
[00:28:28] Unknown:
Yeah. Since this is a Python podcast, I wish that the answer was something like, you know, we really studied 20 different languages and did a really detailed analysis that Python was the winner. But it was, basically, Jimmy was the person who wrote this layer of library, and he liked Python. And, and I was like, Python's fine with me. And at this point, you know, there's only it's really only a couple of people, and so it started getting written Python. I mean, I think Python is a good choice. Like, I don't regret Python as a choice in any way, but it wasn't, like, an especially careful choice. Python is a good choice because, like, there's, you know, there's an incredible amount of libraries. It's easy to be productive, you know, very quickly.
A lot of the stuff, similar things, you know, like, there's existing DHT implementations already written in Python. You know? So it's it's and, you know, BitTorrent was originally written in Python. So, you know, there is a it has also kind of established itself as a good, you know, choice for problems in this domain. Yeah. There are also a lot of really good networking libraries.
[00:29:27] Unknown:
And as I was running the library daemon on my terminal, I was noticing that you're making use of Twisted for handling that. So
[00:29:35] Unknown:
Yeah. Right. And so, I mean, I don't know how you feel. I didn't write that much of this part particular, but I have given twisted a bit of a dive. What what are your thoughts on Twisted here? Just just just thumbs up, thumbs down, mixed feelings. So Twisted
[00:29:49] Unknown:
as a library and as an implementation is definitely widely regarded and has a number of years of maturity that it benefits from. It was also 1 of the first movers in terms of asynchronous networking in Python and even among other languages. So it definitely has a lot of good lessons embedded into it. And also the fact that it includes so many different protocols within the 1 library makes it very easy to interoperate between those protocols, which lends a lot of power to it. I think that as more asynchronous networking libraries have been added to Python, both the standard library and with things like tornado, It has been, sort of losing ground in terms of popularity just because of the fact that the API is sometimes a little bit difficult to wrap your head around.
But as a piece of technology, once you do have it in place, it's definitely pretty bulletproof because of the number of years that it's been run-in so many different production contexts.
[00:30:49] Unknown:
Well, that's a that's a way better answer than I would have given. So and then and you've touched on a lot of things that I would say, as well in terms of, like, it did it provided a lot, and I agree that it's, like, a very solid technology. And it's it it it got us here, but it does also seem like it is it's it's sliding a bit, and, it can also, and maybe this was, per perhaps us making mistakes in the beginning days as well, but it can just sometimes it just feels like it creates a tangled mess.
[00:31:18] Unknown:
You know? Yeah. 1 of the, complaints that I have seen leveraged against it is the sort of nested callback architecture, which in my understanding there are some, alternative APIs that has introduced at least, in the past few years to address some of that. But, yeah, it's definitely There there It's definitely possible to get to the point of having, of being in callback hell.
[00:31:41] Unknown:
Yeah. Yes. It's actually gotten better in the span of I mean, again, this may have been us just being behind the curve in terms of how we did some stuff originally. But it's it's gotten better, like, in the span of this project existing. So
[00:31:53] Unknown:
And 1 of the things that I did notice when I was installing the Python project is that you're still using Python 2. So I don't know if that's related to the fact that you chose twisted or if there are any other, dependencies that are keeping you on Python 2 at this time.
[00:32:08] Unknown:
Yeah. I think that's why I mean, I would definitely like to be on Python 3, but it's just not it's it's not gonna be a priority for us to make that switch, I don't think, too soon.
[00:32:18] Unknown:
And how much of the rest of the sort of infrastructure around library is built using Python, whether at the protocol layer or even just at supporting infrastructure?
[00:32:30] Unknown:
So it's basically that piece, the daemon level, the part that runs in the background, the operating system level, that provides a set of APIs that most, clients and users are gonna use. That is, however, the biggest and most important, piece that we have. The they're you know, we're a company, and so as a company, you end up having any number of various code bases. But, really, there's 3 big ones, or or ones. There's in terms of thinking about the ecosystem, there's 3. There's the blockchain level, which is c, c plus plus. A lot more work was done there in the early days than is done now. Then you've got the daemon level. So blockchain is like the database. You got the daemon level, which is doing all of the data exchanges, the rules, the handshakes, provides the APIs. It's like it does a ton.
And then you've got the level above that, which is, like, your applications clients. And our our sort of proof of concept MVP browser, is written in, it's just JavaScript and CSS. It's just a Electron application.
[00:33:34] Unknown:
As you mentioned, there is a business built around library. So I'm wondering if you can explain a bit what the business model and business plan is for that for the company and some of the things that you've got planned for the future of the company and the protocol.
[00:33:49] Unknown:
Yeah. Absolutely. So there's 2 factors. 1 is a a lot of the things that other open source companies have done, which is just generally being experts at your technology, being able to provide value added services, all kinds of things like that. But really a big part of it is, this this nature of what I, it's kind of a jargony term, fat protocols, protocols that do more and contain a kind of, a built in kind of token that, is used to to use the protocols. And so we have, we have retained 10% of the credits that will come into existence.
And so part, part of our business is to make the library protocol as successful as we can possibly make it and, retain some of the value in the form of the credits that we kept. We have been funded by the Boston Area VCs and some angels from a number of different places, and we have, no plans, to sell any of our credits for some time. So that's not something we're actively doing, but we have retained them for the future. So if library is a success, you know, 3, 5, 10 years down the road, that's part of where our return will come from.
[00:34:59] Unknown:
And as you mentioned, and as is the case to my understanding with all of the different block chains is there is an upper limit in terms of the number of coins that will ever come into existence because of the nature of how the computations are performed, where they'll ask ask them to towards a final value. So once that does happen, I guess, how does the currency continue to be in circulation, and what happens at the blockchain level once the number of coins has reached a maximum?
[00:35:30] Unknown:
So at that point, it's driven by transaction fees. This is also a point that no blockchain has actually reached in reality. In the case of the LBRY blockchain, you're looking at you know, it's around 20 years from now. So it's, it's it's a somewhat theoretical question, but the answer the the current theoretical answer is transaction fees.
[00:35:51] Unknown:
So from my understanding of this is that the coin or currency level is somewhat independent of the actual ledger. And so even once the maximum number of coins has been mined, then the ledger will still be able to continue on, as far as maintaining the tree of transactions. Is that correct?
[00:36:13] Unknown:
That's exactly right.
[00:36:15] Unknown:
So there's definitely a lot here, a lot of things that we've kind of skated over the surface of because digging into any 1 of these things could be a full podcast or more on its own. But are there any other topics that you think we should cover before we start to close out the show?
[00:36:33] Unknown:
No. I don't I I can't think of anything else. It it was definitely wonderful to get the chance to to talk about library. Great.
[00:36:40] Unknown:
So for anybody who wants to follow you and library and follow what you guys are up to, I'll have you add your preferred contact info to the show notes. And so with that, I'll bring us to the picks. And my pick this week is going to be the book neurotribes, which I started reading recently. I'm about a quarter of the way into it so far, and it's very well done. It's a book by, 1 of the wired authors, and it covers sort of a broad swath of the culture and history and sort of, you know, established research on the subject of autism. So it's a really interesting book, covers a lot of ground. So do you have any picks for us today?
[00:37:26] Unknown:
Very interesting. Yeah. I'm I'm gonna throw 1 out since since you went with something, to read. I got something to read as well. A little bit related, to what we talked about today, but not too much. It's 1 of the best, kinda most thought provoking pieces that I've read or had pointed my way in the last, I'd say, even the last couple of years. It's in the title might not make it sound that way. It's called and just Google this title. You'll find it. Crystals and mud in property law. It's like a it's like maybe, like, a 15 page PDF, from, like, maybe someone at Yale or Yale press or something like this. I couldn't even tell you who it's by. I've I've read it a couple of times. I couldn't even tell you it's by. It's not my I'm bad.
But it deals with, it came up the other day, and I was just reading it again. And it and it reminded me again just how good it was. It's dealing with the trade off between, hard, rules, and so a lot of blockchain systems are about how great hard rules are gonna be, but also the that, your hard rules have problems. And I don't know if you've you guys have talked about the DAO, that whole Ethereum scandal thing, where that was an example of hard rules got out of control. And so the the this fundamental trade up between hard rules and softer sort of more human systems and how, our law systems and our desire kinda can oscillate between these 2 extremes and why that happens and so on. And it's just very interesting, and it wasn't wasn't something that I had even even thought about or had concepts for before. And so I'll I'll point it out now. Great.
[00:38:54] Unknown:
Well, I really appreciate you taking the time out of your day to join me and talk about library and the work that you're doing there. It's definitely a very interesting project and I'm definitely excited to see where it leads to in the future. So I appreciate that, and I hope you enjoy the rest of your day.
[00:39:09] Unknown:
Thank you very much, Tobias. You too.
Hello, and welcome to podcast dot in it, the podcast about Python and the people who make it great. I would like to thank everyone who supports us on Patreon. Your contributions help to make the show sustainable. When you're ready to launch your next project, you'll need somewhere to deploy it, so you should check out linode at ww w.podcastinit.com/linode and get a $20 credit to try out their fast and reliable Linux virtual servers for running your app or experimenting with something that you hear about on the show. You can visit the site at www.podcastinit.com to subscribe to the show, sign up for the newsletter, read the show notes, and get in touch. To help other people find the show, please leave a review on Itunes or Google Play Music. Tell your friends and coworkers and share it on social media. Your host, as usual, is Tobias Macy. And today, I'm interviewing Jeremy Kaufman about LBRY, a new marketplace for media built on peer to peer storage and blockchain technologies. So, Jeremy, could you please introduce yourself?
[00:01:02] Unknown:
That's right. I'm Jeremy Kaufman, the CEO of LBRY. And when forced to describe LBRY in 1 sentence, well, if I'm talking to my grandmother, I say it's like a community controlled YouTube. I guess my grandmother does know what YouTube is. But if I'm, gonna explain it to your audience, I would say it is an open source protocol providing decentralized digital content discovery, distribution, and access or purchase. And we're I'm sure we're gonna dive into that in a lot more detail exactly what that means.
[00:01:34] Unknown:
And how did you first get introduced to Python?
[00:01:37] Unknown:
So Python is Python is the language that we use, to to provide our distributed network for, the daemon layer of library. I started using Python myself in probably I was thinking it's my either my junior or senior year of college, which would have been, like, 2006, 2007. I was a computer science and and physics student at RPI in upstate New York.
[00:02:01] Unknown:
So you briefly gave a high level overview of what library is, but I'm wondering if you can give a bit more detailed description of it as well as how the idea for it got started, particularly given the, sort of confluence of different technologies that it is using under the covers.
[00:02:17] Unknown:
Yeah. So it's library can be, slightly tricky to get your head around, although I think I'm getting hopefully, I'm getting better at describing it. Fundamentally, what library is is it's a technology. It's a protocol. We, in fact, picked something that was 4 letters because library uses a a URL structure like HTTP, l b r y, and library URLs resolve to a piece of digital content, or potentially some other things related to that, like a list of things like that, or a a publishing identity. And library is basically what some people have called a fat protocol. It's a protocol that does a little bit more than a thin protocol like HTTP. So that is library as an implementation handles things like that would typically be handled at the layer above, like, at the application level. Libraries handling them at the protocol level. So including things like, you know, payments and managing a digital catalog and so on. But, basically, we've created a a a what we hope is as simple as can be a simple set of APIs to handle digital content discovery and distribution in a completely decentralized, censorship resistant, and basically peer to peer and market driven manner. And I think that there is a second half of that question that I didn't answer, but the answer to the first half was so complex that I'm just gonna take a pause there and then see, see where we go.
[00:03:37] Unknown:
Yeah. Sure. So the second part of the question was wondering how you got the idea for it and how you got it started.
[00:03:43] Unknown:
Yeah. So the idea for it. The the idea that, you know, no 1 really has original ideas. Everyone just, like, combines ideas and tweaks them a little bit and so on and just it's just layers and layers of this stuff. It's very much that. Right? Like, so library is an original idea in the sense that this combination of things, I don't think, has existed before, at least not in this way. But I'd say, actually, I think definitely not in this way. But at the same time, it's basically a combination of a couple of different things. 1 of which was absolutely I I've been a fan of of peer to peer and distributed technologies for a long time. So, part of the inspiration for library was BitTorrent, which is just a remarkable and really wonderful technology.
Another big part of the piece was blockchain technology and Bitcoin, which, succinctly, the real innovation there is that we can reach consensus on a state of affairs in a decentralized manner. So it's not just about money. What blockchain technology lets us do is maintain a decentralized state for any number of things. 1 thing that that can work really well for is a a catalog of digital content. So now we can fix some of the problems that exist with previous decentralized technologies. They have bad incentives. They have poor discovery mechanisms. Other some infringement problems, which I think are also related to the previous 2. And so now, by basically combining these 2 things, as well as some other, little twists and innovations, we can create a an end to end system, for for a decentralized digital marketplace.
1 particular inspiration was so beyond just knowing about these technologies was there's this great there's a conversation between Julian Assange, and Eric Schmidt, the former CEO of Google. It's funny that I need to describe who Eric Schmidt is, but, I know who Julian Assange is. But I may I was both. Anyway, it's like a massive conversation. It's like 8 hours long. And Assange being Assange published the whole thing online. But in 1 part of it, Assange is talking about this decentralized DNS system inspired by Bitcoin, like coupling a name to a a piece of text. And and that was also something that got me thinking about this possibility.
[00:05:40] Unknown:
Yeah. There's definitely a lot of stuff to dive into there. 1 thing that you did briefly touch on is the idea of bad actors in the network and piracy in distributed content delivery systems. And reading through the FAQ, I know that you have at least a philosophical stance on how that can be addressed in the library protocol. But I'm wondering if you can explain a bit about what mechanisms are present for content owners to address piracy for anybody who does start distributing content that isn't necessarily owned by the person doing the distribution and some of the other network and market effects that are in play to help mitigate that from happening in the first place?
[00:06:17] Unknown:
Yeah. Yeah. Absolutely. So this is a this question gets a little bit complex because there's library as a protocol. And in that ex extent library, you know, like HTTP, to some extent, asking what library does at a protocol level, to some extent, is asking, like, what does HTTP do? The answer is, well, it's a protocol. Right? There's no there's no centralized registry of of infringing HTTP domains. Well, there are blacklists and things like this, but this is how we handle it. We handle it at this level. And so, for example, in the library browser that we provide, we provide search results. You can use library browser to search content that's been published to the library network. Those search results are filtered, and we respond to DMCA requests and and filter those.
We will also be maintaining a list of of of URLs that we know to be infringing. And so clients that, you know, are interested in being on the right side of the law will follow that list, or they will be facing, very significant penalties. In fact, breaking the law when you're using library, sort of 1 of 1 of the extra incentives to not use library in a bad way, is that the penalties are actually much more severe. When you are caught infringing, like, downloading something off of BitTorrent, it's a it's a civil infraction. You just you you infringe on someone's copyright by accessing it. When you profit off of copyright infringement, it is a criminal infraction. The clients that will be publishing, like the official clients, will respect the blacklist, won't rehost stuff. If you choose to break that rule, you're facing jail time. And it's not like library is designed to obfuscate who you are in that fashion. Right?
So there, you know, there's a continuum of how possible it is to power things where the most impossible is to show you something on a screen in my house, and then, like, right, you've got com completely in the clear. I think pirating stuff on library is hard it's gonna be it's gonna be harder for people to do it than it is on BitTorrent. If you're the kind of person who just wants to infringe and doesn't care about paying for stuff, you'll just you'll just keep using the places that are best for you for doing that. So we're trying to capture what's the wins from those technologies. Again, BitTorrent's a wonderful technology, but, unfortunately, it never really caught on as a as a way of too much mainstream distribution. We we wanna capture that mainstream.
[00:08:30] Unknown:
Yeah. I don't really have anything to add to that because it's such a
[00:08:34] Unknown:
that that that's a deep rabbit hole to go down, so we'll avoid that 1. It is. A lot of these are deep rabbit holes to go down. And if you can't tell already, I will, spit a lot of words at you. So
[00:08:44] Unknown:
So another aspect of LBRY, the protocol is the fact that, as you mentioned, it's built on a blockchain, which if some for people who aren't as familiar with the blockchain technology specifically, like you mentioned, it is the same underlying technology as what Bitcoin is using as well as other so called altcoins such as Ethereum and Litecoin and a number of others, including the name coin system that you alluded to. So I'm wondering if the blockchain that library is using is using is built on top of something like Ethereum using smart contracts or if you developed your own blockchain purpose built for the specific use of LBRY?
[00:09:21] Unknown:
Okay. So you gave me a there are a couple of things there that, inclined me to to get up on a soapbox a little bit, but I'll start by just giving the straight answer, which is that, library has a separate blockchain. That blockchain is what's used to maintain the library content catalog, so the the listing of what's available in library. As a protocol so library as a protocol is larger than the blockchain. Library you know, the blockchain is part of the protocol. But it's not every the it's not like everything that the set of APIs does inherently involves the blockchain.
And as a protocol, it's acceptable to pay for things. However, 2 parties want a handshake, in terms of payment, so it's not some kind of thing where you have to use the LBRY blockchain to pay for everything. But 1 of the interesting things about blockchain is that, in terms of how to think about it, I think a lot of people start thinking about it wrong because they look at Bitcoin into them like Bitcoin is money, and so they think that or or some something like money. And therefore, Bitcoin as a technology is somehow related to that, and that's wrong. Bitcoin is just a it's just a new kind of database technology. I will frequently say it's the world's crappiest database technology. It it has it has terrible throughputs, long confirmation times, you know, all these downsides in terms of a database except for 1, which is distributed consensus.
And so if you've got all of these downsides, it's almost like anti scaling properties. And so, like, the notion that everything would ever be on 1 blockchain, it's like saying, we're gonna use 1 database to power all of the world's websites, and also that database is fundamentally bad at scaling. I was like, that doesn't make any sense. Right? This so the word kinda altcoin, it's like saying, like, are there alt database technologies? You're you're gonna have different databases for different applications. There's there's nothing about blockchain that says, let's do everything on top of 1 block. You mentioned that
[00:11:07] Unknown:
at the protocol layer, there's no requirement to use the library coin or the library blockchain for settlement of payment for content that's being distributed through the network. And when I was reading through the FAQs, I did see that there was an allowance for 3rd party settlements. So I'm wondering if you can talk about that a bit and how those 3rd party settlements would then be recorded as part of the distributed ledger.
[00:11:32] Unknown:
Actually, the first part of that is the settlement doesn't necessarily have to be recorded as part of the the ledger. The only hard requirement for usage of libraries that the library blockchain acts as essentially a URL scheme and a metadata storage system. It act it's the it's the catalog. And the only reason that you would need library credits is is for entries in that catalog. That's the only, like, hard reason. Inside that metadata is, you know, a pointer to other resources. That pointer could be to any number of things. It could be to HTTP. It could be to BitTorrent. It could be to FTP. We have our own, sort of DHT like system that it it allows for marketized data transactions as well, and that's the default system. But all this stuff is it's designed to be very open ended. The only hard requirement is, for that catalog. Now that said, like, our client doesn't support that we've released, like, our our, you know, sort of MVP.
That it doesn't support paying via Bitcoin. But the API is, you know, you're broadcasting request. You're offering this. If the other guy says, I have that, and I'll accept that, you know, you could say you had 5 squirrels. And if the guy says, yeah. I'll take 5 squirrels. I mean, it's up to the rest. You guys figure it out from there how that's gonna happen. You know?
[00:12:40] Unknown:
So if somebody does settle out of band from the blockchain and doesn't use the library currency for that transaction. I know that 1 of the other layers of library is the peer to peer content distribution network. So for somebody who does settle out of band and doesn't necessarily record that transaction in the ledger or use the, provided application or the private provided UI and APIs, I guess, in what manner would that distribution then take place? Is that again just in the hands of the parties who are transacting?
[00:13:12] Unknown:
Yeah. It's it's basically that. In terms of the metadata, there's basically like, how many fields are relevant here? You've got 1 part of the metadata is the sources, which is basically saying these are the way this is the way or the ways that you can access this. Then another part is basically saying what the price is if there is 1. And, obviously, stuff can be free. We also 100% support and encourage that. And then, you know, how you're gonna make that payment to that person, which is then gonna if it's assuming the content requires a payment, then also how you're gonna get the key for it. Yeah. It's it's very open ended, and it's gonna be whatever those 2 parties are willing to kinda get a handshake on.
[00:13:49] Unknown:
So in order to be able to support such a large scale and distributed marketplace, the blockchain that you're using under the covers will need to be able to support large transaction volumes, which I know is something that is a problem with the current implementations of Bitcoin, which leads to slow transactions and a inherent limit on the number of transactions that can happen on the blockchain. So what are some of the ways that you've architected your implementation in order to be able to achieve that scalability?
[00:14:18] Unknown:
I will again I'll give the real answer, then I I have just a little bit of a soapbox answer to this 1 that's related to that same 1 as before. And it's all these are all kind of interrelated in terms of the mistakes that people make in terms of thinking about blockchain. So the answer is, fundamentally, if you want to do a large amount of volume, you have to do stuff at the second layer. You know, so we're looking at stuff like, lightning network or whichever 1 of these seems to be the most promising, but we've been monitoring a lot of them, and we've been in contact with the teams that have created them. And we're gonna as we reach those limits, which we're not at, we we will, be looking at dealing with, you you know, using something like that. So this is much more about programming than cryptocurrencies. So so your people probably aren't, super familiar with, like, all this Bitcoin drama. But, like, Bitcoin is just and all of these that, you know, blockchain, it's it's fundamentally an inefficient technology.
You know, blockchain is saying do something in replicate across every single device that's participating as a miner in the network. And and if you do something on 2, 000, 000 machines, it's always going to be more expensive than doing it on 1 machine. And just people forget to trace it all the way back to what's actually happening. And And so, you know, like, there's, like, people complain about Bitcoin transaction fees. And it's like, the only reason we have any you know, the kind of network security that we have now is because for every bit transaction on the Bitcoin network, it's being subsidized to the tune of, like I think it's, like, $5. Every transaction is costing $5 in real resources.
Like, that's how much computing power and so on is used. So so blockchain and microtransactions, that never fundamentally works. You're going to always have to move that stuff to a second layer because microtransactions are the worst thing to do 2, 000, 000 times in in Replica. You know?
[00:16:06] Unknown:
And can you dig a bit more into what a Lightning Network is and what that's going to enable in terms of the overall volume?
[00:16:13] Unknown:
Yeah. Absolutely. And so Lightning Network is an example of 2nd layer solutions where 2nd layer solutions is basically there's an ability for nodes to act as things that are like, you know, other kinds of institutions in a in a typical financial system. So, you know, they're basically providing credit and and settling transactions. And so, you know, they they may observe a large number of transactions and group them up and bundle them so that rather than pushing a a 1, 000 transactions to the blockchain, they can do it in, you know, 10. And so that saves space on the blockchain and makes everything more efficient. And, Lightning Network is an example of this solution where it involves committing and showing that you have a certain amount of currency, opening up a channel and advertising it, and then people can use that channel and in a trustworthy way know that that payment will ultimately settle.
[00:17:02] Unknown:
So and that's all I'm gonna say about that before I, embarrass myself. Going to another layer of the stack is the content distribution mechanism that you've built up, which I know you said that you were at least inspired by a BitTorrent. So I'm wondering if you can describe a bit how that, how that's implemented and some of the challenges that you faced in the process of getting that rolled out and also that another aspect of that being how you are encouraging people to take place in that distribution network.
[00:17:31] Unknown:
Yeah. Absolutely. So this is a whole another piece of library. And this is actually the piece most relevant to Python. Alright? All that blockchain stuff, blockchain is a, you know, c, c plus plus. The peer to peer data network is so this is basically the default publishing method that we use in in the library application and in the library daemon. It's very similar to BitTorrent in the sense that you're using a DHT pointer to find hosts that have the content. The difference from BitTorrent is that in this model, the downloader and the host have a chance to negotiate.
So, basically, data is provided in a complete market like fashion. Let me try to explain that. Basically, you're looking for a chunk of data. When something when something is published to the library network in this model, it gets encrypted. It gets broken up into a bunch of 2 megabyte chunks. Each of those 2 megabyte chunks have a hash. Essentially, the library protocol creates a marketplace where there's a price, for each of those hashes. Now we're talking about really, really small prices, fractions and fractions of a penny. In fact, from for many, many data hashes right now, the price is simply free, because the default behavior of our clients is to host things for free. And so more than enough people are running it at free that most people aren't paying data prices. But it actually creates and this is, I think, a very powerful idea, the idea of literally a marketplace for data, data as a commodity.
That is that, like, if we know a chunk of of data hashes to a certain value, you know, we can essentially create a marketplace for that data and, in a decentralized way. You know, you've got the it it's like I think it becomes a very efficient way to host things. You have the power of market incentives to to provision hosting as efficiently as possible.
[00:19:18] Unknown:
And just as a brief aside for you you've mentioned DHT a couple of times, which I'm assuming to mean distributed hash tables for people who just haven't heard that, TLA, which again stands for 3 letter acronym.
[00:19:30] Unknown:
Oh, yeah. I'm I'm I'm bad about that. And please, yeah, interrupt me. I could that's a that's another just wonderful technology, man. You learn how that 1 works, and you're like, man, that guy was a that was a smart guy. He thought of that. You know, the idea of how you can basically, you know, you announce you're looking for a hash and it can route to someone that has it without any kind of any kind of centralization, any kind of catalog or anything like that.
[00:19:53] Unknown:
Yeah. Distributed hash tables are pretty mind blowing. And, also, 1 of the things that you run into there, particularly as you grow the amount of data in the network, is the possibility of hash collisions. So I don't know if you have any if if you have done any engineering effort on the part of trying to account for and, either prevent or, resolve hash collisions in the data network?
[00:20:19] Unknown:
That's true. That's a good question. I mean, I miss I actually and I don't know off the top there, and that's an example of, like, I should probably know the size of our, you know, hash space off the top of my head, but I don't. I don't know how many bits it is. So I don't know, I'm I'm sure that this is something that they've, that we've thought about, and it's also something that I'll you know, to by the oh, it's also 1 of those problems that I call the it's in the good set of problems. Like, we by the time you have that problem, it's like, great. We were we were really successful. Well, you know, we can evolve this, if it did happen to be something that was Absolutely. As long as you're not using Shaw 1 or MD 5, then Oh, yeah. No. No. Definitely not. So a couple of things,
[00:20:58] Unknown:
going back to the blockchain technology is that for people who have been following what's going on with Bitcoin mining, when it first started off, it was entirely feasible to, you know, run a mining client on your laptop or your desktop at home just using CPU capacity. But as the necessary computations have become more complex and the value of the bitcoins has gone up, people have been making that their entire business purpose. And so there have been application specific integrated circuits or ASICs developed specifically for the purpose of Bitcoin mining. So it's gotten to the point where your average person doesn't really stand a chance of trying to obtain any currency in that manner unless they're taking part of a mining pool. And so I'm wondering what if any, engineering effort you've put into the library blockchain to mitigate that eventuality.
And if you don't have anything in place from the mining perspective, how do you see the monetization of the data as an alternative to that for people who do want to get involved in the library community and the library blockchain?
[00:22:02] Unknown:
Yeah. So, well, right off the bat, first, I'll say that, like, the the hosting is is the way better way regardless. That's true today. And then, actually, I will also go ahead and say, I probably should have said this at some point already that, like, like, for those people who are interested in library, you can go and check it out right now, especially since your since your audience is programmers, I'll point them to l b r y dot I o slash quick start, as an intro guide that teaches you about the library API and then gives you a good introduction to to the technology. Although, there's just a bunch of good resources on our website in general. But if you're if you're using it and you want to to, make credits, way better to contribute your disk space and bandwidth than to compete as a miner.
Mining, now we have made choices here to try to, combat this, so that, you know, that is we we chained a number of different, hashing algorithms together, and we picked ones, that, you know, were not matching anything existing. Library is not being ASIC mined right now. I think it would be fairly expensive to do so. I think eventually I'm not, yeah, I'm not sure at at what point that would be possible. But already, you're looking at you know, it's already mining pools and the kinds of people who are buying GPUs and so on that are mining the credits. So it's already something where if you're mining on your own local machine, it's probably not gonna be that profitable. You know, in terms of, in terms of of of just kind of getting something, the another reason another easy way to get some stuff is just to come use it right now, because we give a bunch of free credits out, in this stage, especially to technical users, because we wanna get them testing the technology, vetting the technology, and so on.
[00:23:43] Unknown:
Yeah. And on that point, I was experimenting with it as I was preparing for the show. And in the process, I published the most recent episode that I released, at the time of this recording, and I'm, probably going to be releasing the rest of the back catalog onto library. So if anybody does want to experiment with it and use that and use the podcast in it, show as a means of seeing how the content distribution and content discovery works, then, by the time this gets released, those episodes should be up and, able to be accessed.
[00:24:17] Unknown:
That's awesome. Yeah. We'll, I'll have to check them out on there and make sure, everything's, going smoothly. What'd you think? You wanna tell me about the the process here on the show a little bit? You wanna give me a little a few a little live review?
[00:24:29] Unknown:
So when I was first getting started, I was just using the API approach for trying to publish, and, it's definitely verbose when you're trying to type it all out manually. But from a interaction perspective and a UX perspective, it's all pretty seamless. You know, the the most difficult startup piece was just getting the library credit to be able to, publish onto the blockchain because of the what you were saying of requiring to contribute some of the library currency as a means of reserving the namespace that you want to publish to. But once I got that out of the way by visiting your Slack channel, it was very easy and painless to publish on the network. To my knowledge, I haven't had anybody download it yet, but I also just did it a couple of hours ago. So I'll let you know once I find out more about the, the other end of the transaction.
[00:25:16] Unknown:
Alright. Oh, did you did you set a price for it? So it's, is it free or am I I I it's I'm gonna go and check I am gonna go and check it out. So I'm gonna go and check it out. So I'm gonna go and check it out. I I experimented with setting it at 10¢, but I'll also, release them for
[00:25:27] Unknown:
I'll also, make it available for free for people who don't want to pay.
[00:25:31] Unknown:
Okay. Okay. Alright. Well, well, I'm I am glad it worked. You know, we first announced library to the public, sort of to the public. Like, we were trying to be kinda, like, quiet about it, which didn't work, by the way. But when this was, like, last July, and then, like and library had, like, been run on a total of probably, like, 8 computers at that point, like, a very small number, and it just didn't work at all. And we've been a significant amount of the effort over the last 6 months has been, like, getting the technology to the point where, like, it actually does all the stuff that we had wanted it to do the entire time. So it's still thrilling to me every time someone says, like, oh, yeah. Like, I watched the video. It just, like, it just worked. I'm like, yes. We We were not there. Like, 6 weeks ago, that was not always the case. So
[00:26:16] Unknown:
Yeah. It's definitely a great feeling when people are using stuff that you've been building and, dedicating a lot of time and energy and effort into. Indeed. Indeed. You know, I think that the, you know, the potential here is absolutely tremendous.
[00:26:28] Unknown:
And, you know, that's part of why our team is putting so much energy into this is because this is gonna happen. Right? Like, it's just the idea that the power of blockchain to to build some of these decentralized applications, it's just such a good solution to a lot of, a lot of problems. So I think we're gonna see solutions beyond just digital content in this area, and and, certainly, it's gonna happen in digital content. If it's not us, it'll be someone else. It's gonna happen. It's, like, it's guaranteed, in my opinion.
[00:26:56] Unknown:
Yeah. Just as a anecdotal reference to the power of the blockchain and the sort of the mind share that it's gaining, I was reading an article a little while ago about a number of banks who were, working on establishing an international, sort of, agreement where they were going to try using a blockchain for managing distributed ledgers for transactions between the various banks. So, it's definitely an interesting technology that is going to be spreading, far beyond where people can even think of right now.
[00:27:28] Unknown:
A lot of the bank's usages, though, in my opinion, are are they're exactly the wrong ones. They're exactly the ones that aren't aren't leveraging the real power of blockchain. The the power of blockchain is that distributed consensus, and that comes from essentially a massive waste of resources. It comes from doing things 2, 000, 000 times instead of doing them once. If you're con if you're this group of banks, you can just make an agreement, and you and you can just run your own network. Like, I I don't the the a private blockchain is like it's not I don't it it's like an oxymoron. It doesn't make it's it's you're throwing you're getting rid of the whole, insight of what blockchain was.
[00:28:11] Unknown:
So switching gears again, as you mentioned, the at least reference implementation of library is written in Python. So I'm wondering if you can explain a bit about what the decision making process was that led to that and what some of the, benefits and challenges of that choice have been.
[00:28:28] Unknown:
Yeah. Since this is a Python podcast, I wish that the answer was something like, you know, we really studied 20 different languages and did a really detailed analysis that Python was the winner. But it was, basically, Jimmy was the person who wrote this layer of library, and he liked Python. And, and I was like, Python's fine with me. And at this point, you know, there's only it's really only a couple of people, and so it started getting written Python. I mean, I think Python is a good choice. Like, I don't regret Python as a choice in any way, but it wasn't, like, an especially careful choice. Python is a good choice because, like, there's, you know, there's an incredible amount of libraries. It's easy to be productive, you know, very quickly.
A lot of the stuff, similar things, you know, like, there's existing DHT implementations already written in Python. You know? So it's it's and, you know, BitTorrent was originally written in Python. So, you know, there is a it has also kind of established itself as a good, you know, choice for problems in this domain. Yeah. There are also a lot of really good networking libraries.
[00:29:27] Unknown:
And as I was running the library daemon on my terminal, I was noticing that you're making use of Twisted for handling that. So
[00:29:35] Unknown:
Yeah. Right. And so, I mean, I don't know how you feel. I didn't write that much of this part particular, but I have given twisted a bit of a dive. What what are your thoughts on Twisted here? Just just just thumbs up, thumbs down, mixed feelings. So Twisted
[00:29:49] Unknown:
as a library and as an implementation is definitely widely regarded and has a number of years of maturity that it benefits from. It was also 1 of the first movers in terms of asynchronous networking in Python and even among other languages. So it definitely has a lot of good lessons embedded into it. And also the fact that it includes so many different protocols within the 1 library makes it very easy to interoperate between those protocols, which lends a lot of power to it. I think that as more asynchronous networking libraries have been added to Python, both the standard library and with things like tornado, It has been, sort of losing ground in terms of popularity just because of the fact that the API is sometimes a little bit difficult to wrap your head around.
But as a piece of technology, once you do have it in place, it's definitely pretty bulletproof because of the number of years that it's been run-in so many different production contexts.
[00:30:49] Unknown:
Well, that's a that's a way better answer than I would have given. So and then and you've touched on a lot of things that I would say, as well in terms of, like, it did it provided a lot, and I agree that it's, like, a very solid technology. And it's it it it got us here, but it does also seem like it is it's it's sliding a bit, and, it can also, and maybe this was, per perhaps us making mistakes in the beginning days as well, but it can just sometimes it just feels like it creates a tangled mess.
[00:31:18] Unknown:
You know? Yeah. 1 of the, complaints that I have seen leveraged against it is the sort of nested callback architecture, which in my understanding there are some, alternative APIs that has introduced at least, in the past few years to address some of that. But, yeah, it's definitely There there It's definitely possible to get to the point of having, of being in callback hell.
[00:31:41] Unknown:
Yeah. Yes. It's actually gotten better in the span of I mean, again, this may have been us just being behind the curve in terms of how we did some stuff originally. But it's it's gotten better, like, in the span of this project existing. So
[00:31:53] Unknown:
And 1 of the things that I did notice when I was installing the Python project is that you're still using Python 2. So I don't know if that's related to the fact that you chose twisted or if there are any other, dependencies that are keeping you on Python 2 at this time.
[00:32:08] Unknown:
Yeah. I think that's why I mean, I would definitely like to be on Python 3, but it's just not it's it's not gonna be a priority for us to make that switch, I don't think, too soon.
[00:32:18] Unknown:
And how much of the rest of the sort of infrastructure around library is built using Python, whether at the protocol layer or even just at supporting infrastructure?
[00:32:30] Unknown:
So it's basically that piece, the daemon level, the part that runs in the background, the operating system level, that provides a set of APIs that most, clients and users are gonna use. That is, however, the biggest and most important, piece that we have. The they're you know, we're a company, and so as a company, you end up having any number of various code bases. But, really, there's 3 big ones, or or ones. There's in terms of thinking about the ecosystem, there's 3. There's the blockchain level, which is c, c plus plus. A lot more work was done there in the early days than is done now. Then you've got the daemon level. So blockchain is like the database. You got the daemon level, which is doing all of the data exchanges, the rules, the handshakes, provides the APIs. It's like it does a ton.
And then you've got the level above that, which is, like, your applications clients. And our our sort of proof of concept MVP browser, is written in, it's just JavaScript and CSS. It's just a Electron application.
[00:33:34] Unknown:
As you mentioned, there is a business built around library. So I'm wondering if you can explain a bit what the business model and business plan is for that for the company and some of the things that you've got planned for the future of the company and the protocol.
[00:33:49] Unknown:
Yeah. Absolutely. So there's 2 factors. 1 is a a lot of the things that other open source companies have done, which is just generally being experts at your technology, being able to provide value added services, all kinds of things like that. But really a big part of it is, this this nature of what I, it's kind of a jargony term, fat protocols, protocols that do more and contain a kind of, a built in kind of token that, is used to to use the protocols. And so we have, we have retained 10% of the credits that will come into existence.
And so part, part of our business is to make the library protocol as successful as we can possibly make it and, retain some of the value in the form of the credits that we kept. We have been funded by the Boston Area VCs and some angels from a number of different places, and we have, no plans, to sell any of our credits for some time. So that's not something we're actively doing, but we have retained them for the future. So if library is a success, you know, 3, 5, 10 years down the road, that's part of where our return will come from.
[00:34:59] Unknown:
And as you mentioned, and as is the case to my understanding with all of the different block chains is there is an upper limit in terms of the number of coins that will ever come into existence because of the nature of how the computations are performed, where they'll ask ask them to towards a final value. So once that does happen, I guess, how does the currency continue to be in circulation, and what happens at the blockchain level once the number of coins has reached a maximum?
[00:35:30] Unknown:
So at that point, it's driven by transaction fees. This is also a point that no blockchain has actually reached in reality. In the case of the LBRY blockchain, you're looking at you know, it's around 20 years from now. So it's, it's it's a somewhat theoretical question, but the answer the the current theoretical answer is transaction fees.
[00:35:51] Unknown:
So from my understanding of this is that the coin or currency level is somewhat independent of the actual ledger. And so even once the maximum number of coins has been mined, then the ledger will still be able to continue on, as far as maintaining the tree of transactions. Is that correct?
[00:36:13] Unknown:
That's exactly right.
[00:36:15] Unknown:
So there's definitely a lot here, a lot of things that we've kind of skated over the surface of because digging into any 1 of these things could be a full podcast or more on its own. But are there any other topics that you think we should cover before we start to close out the show?
[00:36:33] Unknown:
No. I don't I I can't think of anything else. It it was definitely wonderful to get the chance to to talk about library. Great.
[00:36:40] Unknown:
So for anybody who wants to follow you and library and follow what you guys are up to, I'll have you add your preferred contact info to the show notes. And so with that, I'll bring us to the picks. And my pick this week is going to be the book neurotribes, which I started reading recently. I'm about a quarter of the way into it so far, and it's very well done. It's a book by, 1 of the wired authors, and it covers sort of a broad swath of the culture and history and sort of, you know, established research on the subject of autism. So it's a really interesting book, covers a lot of ground. So do you have any picks for us today?
[00:37:26] Unknown:
Very interesting. Yeah. I'm I'm gonna throw 1 out since since you went with something, to read. I got something to read as well. A little bit related, to what we talked about today, but not too much. It's 1 of the best, kinda most thought provoking pieces that I've read or had pointed my way in the last, I'd say, even the last couple of years. It's in the title might not make it sound that way. It's called and just Google this title. You'll find it. Crystals and mud in property law. It's like a it's like maybe, like, a 15 page PDF, from, like, maybe someone at Yale or Yale press or something like this. I couldn't even tell you who it's by. I've I've read it a couple of times. I couldn't even tell you it's by. It's not my I'm bad.
But it deals with, it came up the other day, and I was just reading it again. And it and it reminded me again just how good it was. It's dealing with the trade off between, hard, rules, and so a lot of blockchain systems are about how great hard rules are gonna be, but also the that, your hard rules have problems. And I don't know if you've you guys have talked about the DAO, that whole Ethereum scandal thing, where that was an example of hard rules got out of control. And so the the this fundamental trade up between hard rules and softer sort of more human systems and how, our law systems and our desire kinda can oscillate between these 2 extremes and why that happens and so on. And it's just very interesting, and it wasn't wasn't something that I had even even thought about or had concepts for before. And so I'll I'll point it out now. Great.
[00:38:54] Unknown:
Well, I really appreciate you taking the time out of your day to join me and talk about library and the work that you're doing there. It's definitely a very interesting project and I'm definitely excited to see where it leads to in the future. So I appreciate that, and I hope you enjoy the rest of your day.
[00:39:09] Unknown:
Thank you very much, Tobias. You too.
Introduction and Guest Introduction
What is LBRY?
Jeremy's Journey with Python
Deep Dive into LBRY Protocol
Inspiration and Idea Behind LBRY
Addressing Piracy and Bad Actors
Blockchain and Third-Party Settlements
Scalability and Lightning Network
Content Distribution Network
Mining and Monetization
User Experience and Feedback
Python and Twisted in LBRY
Business Model and Future Plans
Blockchain Longevity and Transaction Fees
Closing Remarks and Picks
