The Python Podcast.__init__

Share on social media:

[00:00:13] Unknown:

Hello, and welcome to podcast dot in it, the podcast about Python and the people who make it great.   When you're ready to launch your next app or want to try a project you hear about on the show, you'll need somewhere to deploy it. So take a look at our friends over at Linode.   With the launch of their managed Kubernetes platform, it's easy to get started with the next generation of deployment and scaling powered by the battle tested Linode platform,   including simple pricing, node balancers,   40 gigabit networking,   dedicated CPU and GPU instances, and worldwide data centers.  

Go to python podcast.com/linode,   that's l I n o d e, today and get a $100 credit to try out a Kubernetes cluster of your own. And don't forget to thank them for their continued support of this   show. Your host as usual is Tobias Macy. And today, I'm interviewing Leon Tom about Riven, a flow based visual scripting environment for Python. So, Leon, can you start by introducing yourself? I'm Leon Tham. I am from Germany, and I am a student currently studying computer science in Switzerland.  

[00:01:13] Unknown:

I finished high school last year. So I'm in just started my bachelor.   So there's a long way to go for me. Hopefully exciting way to go. Yeah. And I'm very happy to be here. Thank you for the invitation.  

[00:01:24] Unknown:

Absolutely.   And do you remember how you first got introduced to Python?  

[00:01:28] Unknown:

Yes, I do. Actually, Python was the first programming language that I approached   about 5 to 6 years ago, I guess.   I was just doing some, like, small   high school projects with a few friends and Python seemed like the perfect language. So Python introduced me to the world of programming.   So I started with Python and now for this project going to talk about today,   I kind of came back to Python and   I love Python. It's really great. The more I work with it, the more impressed I am about the design of the language and   how well it works.  



[00:02:04] Unknown:

So you mentioned that Python was the first language that you started using. What are some of the other languages that you've experimented with since you first got started in the space?  

[00:02:13] Unknown:

Over the last couple of years, I tried to do, like, as much different stuff as possible. So I   did a lot of   smaller projects using languages like Java, a little bit of web development   with JavaScript,   but mostly c plus plus because I don't know. For some reason, I decided at some point, I want   to dig a little deeper into   into using c plus plus for mainly building desktop desktop applications.   So I   got introduced into all this through the Qt framework. So this is where I learned the Qt framework   in c plus plus for building large desktop applications.   And this is what I use the most during the last years.  

From language perspective, I don't regret to coming back to Python.  

[00:03:00] Unknown:

Yeah. That c plus plus experience,   particularly with the QT framework, is definitely useful with building a Riven because I know that that's at least the wrapper for the project. So I'm wondering if you can just give a bit more of an overview about what it is that you're building with Riven and what your inspiration was and motivation was for creating it and spending your time on it. Basically, Riven  

[00:03:20] Unknown:

is a   small   environment for flow based visual scripting in Python.   So what it is, it it implements the idea of flow based visual scripting using a more diagram like,   representation to represent parts of your script of your code.   And it just provides with a software   that makes it   kind of intuitive. So basically,   to give a little picture how this works, you have, like, a scene and you can place blocks inside the scene and you can connect these blocks with connections to define relations between these blocks.   And this is how you define your program structure.  

ArriveIn is just a software that makes it easy to create these nodes and run these flows inside an application.   And this is an idea. It's not so common, but I got introduced to it when I as most people do,   when I use Unreal Engine a few years ago for game development.   And they are using this extensively, this parodying of low based visual scripting.   I worked with this for about half a year, and I really started to like it. And it was asking myself why this is not more common in   in more different areas. And I did a little bit of research, and I found   a few other quite interesting systems. But I also found that it's not very common. So I was asking myself why this was so underrepresented   in software. So I decided   to start building my own editors. And Riven is basically the a very   sort of, Pythonic implementation of this whole idea.  



[00:04:54] Unknown:

You mentioned the   overall paradigm of flow based scripting and being able to build programs from these networks of individual building blocks.   I'm wondering if you can talk through a bit of the overall use case or some of the particular problem domains where this approach is popular.  

[00:05:13] Unknown:

I think the best way is to give examples of   the established   implementations   that some of them differ from Riven, some are quite similar. So in Unreal Engine, for example, I mentioned Unreal Engine, and they have created such a pretty big environment for using these,   this parading   for creating   flows using this flow based, these flow base diagrams to create code. And this does not just apply on code. So basically you can define the game mechanics for for a game that you're building with unreal engine just using these   block structures   by building these networks.  

They have a huge material editor where you have   every node presents some kind of effect that you apply on an image or some kind of texture.   It can be a plain texture. It can be height map. And in the end, you just put everything together in a target node, and it creates a full material out of that.   That's a good example.   Node RED, for example, Node RED is a software that is originally I think it was developed by IBM. It's a browser based, Node JS based   software for managing IoT connections.   That was quite inspirational for the Aiven project.   Shows also a very interesting use case, I think, because it's kind of like   IFFT,   but   it's called IFTTT.  

It's a little bit like IFTTT, but   if this then that. So it's a little bit like this if this then that thing   where you have a block that can   be triggered and the event happens. For example, you receive an email somewhere or some file on your file system is changed, and then you   can do lots of stuff   up to writing a Twitter post or something like that.   And it's very easy to use and yet it provides   an interface that you can where you can still program and add your own source code. So you have these function blocks where you basically can put JavaScript code inside it. And that's kind of the idea   that I thought is very powerful and that I tried to implement in Riven. So   with Riven, it's really hard to   explain to someone what the possible use cases of something like this would be because there are just too many. And I feel like I'm not even qualified for listing the use cases because it   repeatedly happened in the past that someone came to me and said, hey, I'm using this for this and that.  

And wouldn't it be useful to optimize it in this or that way? And I was like, yeah, I didn't think about that   application at all, but that's actually a great idea.   So   if I wanted to categorize it pretty much anything that has to do with web APIs   and workflow   automation, stuff like that. Machine learning, it has very big potential for applications with flow based visual scripting, I guess.   Things like that.  

[00:08:05] Unknown:

Yeah. I can definitely see a parallel to the machine learning space or the data processing space because all of those frameworks   structure the workflows as a directed acyclic graph. So and a lot of them will provide some visual means of being able to view all the different nodes within that graph.   And that's very analogous to the flow based system where you have these different building blocks, and you can see the interconnects between them to see this output is going into the input of this other block, and then that's providing an output to another 1. So you can have these fan in or fan out topologies.   And, you can see what are all of the structures that get you from point a to point b and the potential branching points. And so I can definitely see where that's useful   in that arena, but I can also see a lot of other places where that would be valuable. You mentioned web APIs. You can see what is the actual network   of   the, you know, potential paths that you might take where you have the root of the API for a particular object. And then you can see, okay, this branches into the specific   instance of that object. And then that instance of that object has these other methods off of that. So it's definitely an interesting way to think about the way the program flows rather than just trying to piece that together from looking at the textual information in your editor, for example.  



[00:09:20] Unknown:

Yeah, absolutely.   1 thing that's closely related to what you mentioned is   also,   I think there's a lot of potential for real time,   If you can implement some real time access to your data, for example, if you have, like, a web API or you have some network system that is basically projected as a graph   in your flow and you have real time data input, then this could be really useful for getting an overview of what's going on. So you have the static income of data,   and you can just update the flow accordingly and   basically create a small projection of your network or what your web API is doing. If you have that kind of real time access to your data, I feel like this is something that hasn't really been done yet with flow based visual scripting, but I see lots of potential for that.  



[00:10:10] Unknown:

That   also is an interesting thing to dig into more is the way that information propagates throughout the program as you're interacting   with the   overall flow in the network diagram of your program logic.   And, you know, I know that from digging through the documentation that you have a couple of different ways of allowing things to propagate depending on whether it's a data oriented flow or a   the trade   the trade offs of those 2 different approaches and some of the ways that that helps you think about how to structure the program and the ways that the information flows throughout it.  

[00:10:47] Unknown:

The dualism that you were mentioning is the dualism between pure data flows and execution flows. And I think, originally,   the first thing that was invented or first implementations of flow based visual   programming always focused on pure data flows where you have   basically 1 type of connection for nodes, and every connection basically represents a you you can think of it sort of as a pipe   where you have a data flow through it. So from 1 node just passes data to another node, and this other node is notified when data comes in.   But there's also a different idea that basically allows control structures,   which is execution connections. So you have a different type of connection   that is, for example,   implemented in the blueprint editors inside the blueprint editor of Unreal Engine.  

Most of the nodes are active, so they have these execution inputs and outputs. And these execution connections, they are basically like simpler data connections because they don't transmit any data. They just transmit signals. So they just trigger the activation of certain parts of your   script or certain notes and everything that comes afterwards.   And there was a 2 different approaches from an algorithmic point of view. So   in Riven, I wanted to enable both because both can be very useful.   In general,   the data flow approach is the more powerful 1, but sometimes it makes more sense to introduce execution connections. So I wanted to   enable both, and you can choose   the algorithm   that is used to execute the flow. So you basically say whether you are using a pure data flow or whether you have execution connections and it uses different algorithms.  

And basically,   I'm not   sure. It's hard to categorize it and to make a strict distinction for what you can do with the 1 approach and what you can't do because I repeatedly saw   people   using the 1 approach that I would consider   not the optimal 1 for doing the other thing. So it really works.   You can totally build huge data flows that only pretty much rely on execution connections.   And you can definitely do anything that you can do with execution connections using few data flows. It's a little bit different in the way the script is executed, but it's not a fundamental change   because what both have in common is a more logical thinking to the human mind.  

You have these blocks, and every block represents a certain procedure, whether this is just, like, 1 operation. Like, think of it as a variable assignment or maybe there's, like, a big program written into this node. It doesn't matter. It represents 1 procedure.   And when a signal comes in somewhere, whether this is data provided or whether this is just a sig a trigger through an execution connection,   This node is notified and updates itself, and then the procedure starts and it computes.   I feel like this is a more human way of thinking as   execution of a bigger script that you have these step by step components and you just define   using these flow diagrams, you define the flow of information   through your program. And to me, usually kind of the more   intuitive way to think   of an application of a program.  

I think which is reasonable because these kinds of flow diagrams look a lot like mind maps or something like that.   And there's a reason why those are called mind maps. So it's kind of a more intuitive way of thinking how data flow flows through your application. And I think that's also why, especially as you mentioned for data processing, this has, for sure, huge potential.  

[00:14:32] Unknown:

And in terms of the Riben project itself, I'm curious some of the things that you're using it to build for your own personal use cases.  

[00:14:39] Unknown:

1 example would be during the semester, I had, like, linear algebra as you as you do when studying computer science.   I had linear algebra last semester, and   we learned all this, like, matrix operation stuff. Right?   So at some point, someone,   1 of my student colleagues,   approached me and asked me whether it would be possible to make this a little more intuitive using this diagram structure from my software   by representing   an operation on a matrix by a node.   And I thought that is a really great idea. So I basically just took   the functions and methods from NumPy and SciPy and   put them into notes and which is a very quick process. And I just made a basic widget   that displays the matrix   inside the node. And I added this to all these nodes. So you can build your flow and build a chain of matrix operations, and you can change something at the beginning and everything that's dependent on that updates immediately.  

It is a very playful way of finding similarities between matrix operations and   depending on what type of matrix, you can use or you're using, you will find, like, symmetry somewhere. And   this is really cool   Because this was also inspired by things like Node RED. I   also use this for smaller   IoT based stuff. I got this running,   I don't know, about, like, 8 months ago or something on my Raspberry Pi. So the software on my Raspberry Pi, and   I did some very rudimentary basic   network connections to, yeah, to catch some events if when some sensor is receiving something, I do this and that. And this is very intuitive using this kind of flow structure.   The yeah. There's a lot that   other people have done that I didn't even try.  

The hardest thing is to say for what this could be used because it's not something that is specifically designed to be used   for, for a specific domain. You could optimize it, but Riven itself is kind of a general purpose editor.   So it just provides you with the   implementation of this idea of flow based visual scripting,   and you can pretty much build anything you want using it, as long as Riven has the features that you need.  

[00:16:57] Unknown:

I'm wondering if there are any particular projects that maybe you have implemented   in sort of the quote unquote standard approach of just having text in a buffer and saving it and then just executing it using whatever   the build tool is for the programming language, whether it's just invoking it with Python or building the c plus plus application or what have you, and then having revisited   that with Riven   to rebuild it. And I'm curious if that's the case,   how that made you think differently about how to approach the problem or how to structure the various components of the logical flow.  

[00:17:34] Unknown:

So I think 1 very important thing to mention is that   at least for Riven, the idea is not at all to provide   a full alternative to conventional coding, to textual coding. It's not at all the idea to replace textual coding in some way. It's just a different approach that   requires a little bit different kind of thinking for different applications.   So what you would not want to do I mean, you could do it, but I think this quickly gets pretty messy is   taking a script that you have written in source code and basically creating notes for   all the atomic   operations that you're doing, and then create the same thing   using the flow structure. I did this myself just to try it out with, like, the basic examples, like sorting algorithms, stuff like that. But it very quickly gets very messy, and I don't think this is where the strength of this   idea   is. It's more like the other way around, I think. Because   1 good example, which also showed me how full this idea can be,   is I was trying to build a small game in Unreal Engine 4 a few years ago, and   I was using these blueprints to create the game mechanics and to also to compute the level. I was procedurally generating a random   maze,   and you were just running through this maze and there were like objects following you and stuff like that. And I had to write a script to procedurally   generate that maze.  

And I couldn't really find an algorithm online that fitted my needs. So I just wrote this thing myself in a huge script. And not only I realized after   actually was working   that I could have never done that in pure source code back. This was pretty huge.   But what was much more interesting to me   is that   usually when I just coded something using text, I don't think I could have written something like that. And usually when I did write something,   it was much harder to get into that code again when I revisited it afterwards.   I could not just, like, look at it and know what this code was doing, how this worked. I had to read it   again. Like, I had to reread   most part of the code to   get a sense for what, program is actually doing.  

This was kinda new to me. And this is also kind of what inspired me   to dig a little deeper into that and try to optimize that   because I felt like there was a lot of brainpower   basically that was unused   if you're using a paradigm or a representation of your way  

[00:20:21] Unknown:

that the   way that the program is structured and displayed and the editing experience   being a means of   maintaining context and making it easier to   jump into a piece of code. I'm also put in mind of things like literate programming,   which is enabled by things like Jupyter Notebooks and the n b dev project   or in e in the Emacs ecosystem   using org mode with embedded code. And I'm curious if you've ever experimented with some of the literate programming concepts   and any parallels that you might draw between that and this flow based paradigm.  

[00:20:55] Unknown:

So I don't really have experience with those systems.   I know they exist and they are popular. Like, these Jupyter notebooks are really cool tools for playing around, but also for doing scientific work. Right? So   I don't know much about the software behind it, but I was thinking about combining these systems with Riven. So I've had a few conversations with people from scientific areas who are using,   for example, Jupyter Notebooks for, yeah, for day also for data science and stuff. I haven't really tried it, and it's technically, it's not something easy to do. But integrating these kinds of systems like Jupyter Notebooks into   an environment like Riven,   I think could   be the best thing because you can basically combine   2 very different but very intuitive approaches   of structuring your program or structuring your computation and your data processing.  

And I'm not sure how I would go about combining these, but I feel like there are different ways. I made a few concepts, and it it feels like this could be something that I'm definitely gonna take a closer look at in the future to combine these systems.  

[00:22:07] Unknown:

Taking a bit more into Riven now, can you give a bit of an overview about how it's implemented and some of the ways that it has evolved or shifted or the goals of the project have changed since you first began working on it?  

[00:22:19] Unknown:

The development was pretty wild because   at any point in time, Riven was always like a prototype of of something.   So   in the beginning, I had implemented a few very different,   visual scripting editors in the past over the last few years. And Riven   kind of is, like, the result of that because after those   and with the last 1, with the biggest 1, I took part in the youth science competition in Germany, and this was a very great experience. And after that, I sat down and for a few months, I just made concepts and I came up with the idea for Aiven, and I thought this could have a lot of potential to combine it with with Python in this way. But, of course, like, the software has evolved   drastically   always.  

It's not that the concept has changed, but it's rather that the concept has become more and more clear to me, I guess.   Because in the beginning, I wasn't really   sure how exactly or at which level of of abstraction this would be very useful.   And I also thought that this would be primarily directed towards   execution flows or flows and stuff like that. While now I would actually say this is a much more powerful for data flow programming.   It has evolved in a bunch of ways.   I have basically always tried to add more features to made it to make it more intuitive   in use. So at the beginning, it was kind of a sketchy process to create new notes and execute them. And meanwhile, I think it's it's quite intuitive. So there is Riven. And besides Riven, there is the Riven node manager then which is a second application that you use to create these nodes now. And it basically provides a small code editor   and a bit of intuitive graphical user   interface to create these nodes, define initial inputs, define initial outputs, give a description, a title, a color, stuff like that.  

These things I introduced   over time because I felt like this was really an important thing to have.   And, like, I always went a little further month by month. So   1 of the first big things that I implemented is Riven console, which is basically a an application   that runs   a back end version of Riven   without any graphical user interface. So once you have made a flow and you have saved it, you can then run it in Riven console without any graphic user interface and without any cute dependencies.   So you can, for example, run this on a server to, I don't know, do some database processing overnight or something like that. This was really cool.   After that, I introduced things like a console, a built in interpreter.  

So inside Riven, you can click on a node and on such a block called node,   and you can   add a reference to this node to the console.   You just have a reference to the object in the scope of inside the scope of the interpreter, and you have access   therefore to all the internal data   through this console. And this is really intuitive because you have full access to the whole API. You can add and delete inputs and outputs   generation,   which is kind of a very commonly discussed   thing.   Because when you think of a flow based, which is scripting editor, you're asking yourself, does it export code? So when I have a flow, can I export   all my work into 1   single Python file to run it anywhere, to embed it in my software   or somewhere?  

And   while I was kind of reluctant   to add this for a long time,   because   this is not really the idea of Riven,   Because Riven itself is not really a software for developing software,   but rather an exploration   software. So when you have a note package for, say, OpenCV. Right? So you have a note package with a lot of notes that just apply effects on OpenCV effects on images, then you can place these notes and play around with them, and then you can discover new I don't know, some new effects or something. I think this   is something that it's really useful for.  

And therefore, it's not really the goal to export a code that would be much more inefficient than what you would   you could write as a human.   But nonetheless, it would be something very nice to have. Right? So I think it was, like, 2 months ago or something.   It's not yet in the official release, actually. It's just in the master branch on on GitHub.   It's a prototype   implementation of a code generation. So   basically, it runs a short semantic analysis on all the nodes,   and then it shows you a list of the dependencies that the nodes use to specify which sources should be included in the target file and which shouldn't.   And then you can just, like, click generate code and it generates the whole source code out of this, which basically means it creates a file where all the clouds definitions of the nodes used   are included,   as well as a very minimal implementation of the back end to execute the flow   and a function that or main routine that just very explicitly creates all the objects and creates the connections and builds up the flow. And then you can add functions and you can also modify the flow.  

I can't really say where this is going in the future, but   I'm, I'm pretty stoked currently   about where this is right now because like 5 months ago or something, I wasn't imagining   to have these kinds of features included in 5 months. The fundamental concept never changed drastically. I just added features that I think are really useful.  

[00:28:08] Unknown:

As you work on building Riven, how much of it are you able to actually use on itself where you can dogfood the Riven development   with Riven as the editing environment?  

[00:28:19] Unknown:

This is actually a question that I get   many times because it's a funny thought. No. I'm not. Because, again, Riven is not at all, like, something that you would   use to build conventional software.   But it's a funny idea. And do we have, like, a few friends and me, we had ideas of, like,   creating   kind of splash splash   screens while the flow is loading   while this flow that you see in the splash screen is basically the code that's executed creating the flow and stuff like that.   So, yeah, maybe in the future, but so far, Riven has just implemented   by hand in text.  



[00:28:59] Unknown:

And so for people who are using Riven to build a script or build a project with it, I'm curious if you can just talk through the overall process of getting started and the workflow of   designing the program and creating the various nodes within the execution structure and just some of the ways that they should be thinking about how to use Riven to fulfill their goals?  

[00:29:22] Unknown:

There's not really a very clear   or solid path   that you go when developing notes for Raiven.   Usually, I create, like, a basic skeleton of the notes that I want to use in the note manager, and I save it as a note package. And then I start driving, and I import this note package.   And then I   place these notes and play around with them and repeatedly just add functionality   to the notes. So   I mentioned the interpreter reference feature   where you have access to internal data and also to the API so you can dynamically just add inputs and outputs and stuff like that.   And there's also a   the implementation is kind of sketchy, but it works surprisingly well. There's a source code editing feature. So you can   of 1 individual component of 1 node that is instantiated and placed in the flow,   you can   reprogram   method implementations of this node.  

And this way, it's pretty efficient to develop new nodes and try out different things. And then when I'm done, when I know how I want to implement   this note, I just take the code from Riven and put it inside the note manager in inside the note package.   So that's kind of the overall process   to write new notes.   When creating the script, I don't know. It probably really comes down to what you're trying to do. I usually just use this for   playing around and   exploration, and this just involves a lot of   playful   trying out things. And there's not really a very solid procedure that I follow to create to solve some kind of problem. This is more like a tool for exploration.  



[00:31:06] Unknown:

Another interesting aspect of working in this kind of environment   is being able to integrate with the broader Python ecosystem ecosystem for things like code quality or   implementing some sort of testing or how you handle things like version control.   And I'm wondering if you   have dealt with any of that or if it's still just in a prototype stage so you haven't dug too much into the sort of code quality and correctness aspects of how to manage a more long running workflow for somebody building with Riven?  

[00:31:41] Unknown:

The latter. So I haven't really spent a lot of time trying to implement things like version control, because   all those are tools that you apply   on conventional source codes to keep your code clean. But in Riven, you're not coding using source code or at least, like, you have this kind of source code layer that which is the the code implementation of your notes,   but you use this kind of an assumption that this   code is correct and works. And   Riven just extends this layer by introducing the graphical structures, and then you can play around with that. And   and it just adds something to this code layer, but you're not really in strong interaction with the code anymore. So I'm not focusing   yet on these tools to provide, like, version control or something,   which are more directed towards,   yeah, as I said, like, code quality and stuff. As you said, it's   still in the very early prototype stage, so   I'll see what the future brings.  



[00:32:43] Unknown:

Another question,   which is   a little bit of an outgrowth of the previous 1, is how you manage things like collaboration   on a Riven project, like being able   to define a set of nodes and then share that with somebody else to be able to   either   add to that library of nodes and append to the existing flow structure or just being able to reuse some of the nodes that you've defined and, you know, build up a library   or, you know, a team or a individual who   wants to be able to reuse Riven for different problems in sort of adjacent domains?  

[00:33:19] Unknown:

So far, I haven't set up a particular system for this. I would like to in the future if this receives enough usage and   a lot of users users would benefit from that. So far, it's basically just through the GitHub community. So   basically, you're just, like, forking the project on GitHub or something, and you're adding your notes and   everyone can see them. And   if those notes are   compatible with normal Riven projects,   then I usually try to integrate   those into the main repository   to add them as node packages. This is what happened, for example, with the OpenCV notes, this OpenCV note package to apply open CV effects on images.   I just created, like, 2 or 3 very basic notes for this, but then someone from I don't know. I think, like, Istanbul or something   created, like, 20,   30 more notes, and I saw that on GitHub. So I just quickly integrated them into the main repository.  

So far, there's not really a system setup, but I would like to create, like, a cloud based built in system in the future where you could where you can, like, upload your notes and also your flows to the cloud and everyone can just download them. This would be really nice, and that's definitely something I'll think about in the future.  

[00:34:37] Unknown:

In terms of the ways that you're using it or projects that you've seen other people build with Riven, what are some of the most interesting or innovative or unexpected ways that you've seen it used?  

[00:34:47] Unknown:

I think it's used in very different ways. I'm not sure I'm seeing everything at GitHub, but   some of the ways that were pretty unexpected was, for example, last year someone contacted me. It was a data scientist from the Netherlands. Someone contacted me and wrote me an email that he's using this for   like, database   processing for big data. And   he requested a way   to run a flow without   a graphic user interface on the server after he sufficiently tested the flow and know it works so that he can just do all the processing overnight on the server. So this is basically how our admin console was born. And this is something that I   didn't see coming at all.  

There's a lot going on in the machine learning fields, because apparently, a lot of people see a lot of potential for this regarding machine learning.   So   for example, someone is currently creating a note package for   the most common scikit learn   methods.   So scikit learn is this popular library for machine learning in Python. And for data science purposes, he works project and is creating a bunch of scikit learn notes for this, stuff like that that I didn't expect in the beginning. Because in the beginning, I was still pretty focused on these   popular editors like Node RED and the editors from Unreal Engine,   which are basically very different kinds of use cases.  

So   it was quite interesting to see that people are actually using this for things that I didn't consider at all in the beginning.   And I'm pretty sure there's a lot more   to come. Like, someone is currently integrating this in into Unreal Engine actually to have a more Unreal Engine like way of   automizing a workflow.   I think they want to use it for as a automation   tool and stuff like that is   pretty cool and wasn't intended in the beginning during the design of this project.  

[00:36:42] Unknown:

That's pretty cool that Unreal Engine was some of your original inspiration for building this project, and now some of your ideas are getting factored back into that tool. Yes. Absolutely. This is this is very exciting because I didn't even try this myself  

[00:36:56] Unknown:

because I assumed that this would require a lot of additional configuration.   But apparently this is possible. There are 2 popular projects implementing Python into the unreal engine. There's unreal Python and unreal engine Python. The 1 is from Epic Games and the other 1 is 3rd party.   And both are very useful for, like, workflow automation. So if you're   making a game in a small   team and you have a group of developers and designers, then you want to have some workflow of getting your assets that you created in some modeling software like also the Ismaya Blender   into your software and you maybe want to do some computations on it on your textures or something. For this kind of workflow optimization, Python is really useful.  

And I can see that this would be even much more intuitive using something like using something that has this kind of flow based structure structure.   So   I'm very happy to see that shifting   towards unreal engine   and seeing it getting used there because I as I said, I I didn't expect this to happen, cavalarious.  

[00:38:04] Unknown:

In your experience   of building the Riven project and using it for creating some of your own projects out of it, what are some of the most interesting or unexpected or challenging lessons that you've learned in that process?  

[00:38:16] Unknown:

Mostly technical details.   This   obviously is the biggest software project that I did so far. So it's just a lot of   insight into how you should structure a project, and there's a lot of trial and error involved.   Very technical details usually, how you should distribute   or how you should decentralize internal parts of your software.   Currently, I'm really getting into the threading part because I will talk about that later probably, but,   I'm adding, like, threading features for Bryban software.   So the lessons I learned is mostly   regard   the technical details. But what I wasn't used to up to this point until a few weeks ago where this kind of started to take off a little bit on GitHub   was really getting in touch with the community. So   there were a few things happening. Like   there were 1 or 2 short articles about me and the project in Chinese technical journal, which caused a lot of   Chinese people to   visit the GitHub repository   which caused the GitHub repository   to move up to the GitHub trending page for 1 or 2 days   which then caused also a lot of Western   GitHub users to visit the repository. And this way, I just got into touch with a lot more people who were really interested in the project. And   that was just in general, like, a great experience to see so many people interested in the project I'm building   and eager to join the discussion on flow based visual scripting and and stuff like that. So this was kind of just a great experience.  



[00:39:54] Unknown:

As you continue to work on Riven and use it for your own purposes and work with other people who are using it for different projects, what are some of the things that you have planned for the future of it in the near to medium term?  

[00:40:06] Unknown:

A lot.   There is a lot happening right now. So   about 2 months ago, I   started to split the Bribant project into 2 projects   because I always had this idea in my mind that the ultimate goal for me would be   to create something like a   small framework that you could use to easily optimize   such editors.   And what I'm developing right now based on the Riven project so based on the Riven software, I'm right now developing a small framework,   a cube based framework for Python   that   just makes it very easy to create   these editors and to optimize them. So   say you have a bunch of note packages that work well in Riven, you can then create your own editor using Riven. This thing is will be called Riven Core. You can use Riven Core to build your own editor   and really optimize it for your specific domain.  

That's very exciting because I was skeptical   if this would work, if this would be intuitive in use.   But so far, it looks pretty good. And I'm actually currently   adding a bunch of features that are not part of Riven.   Riven in the end will be built on top of Riven Core.   But Drive and Core will be very free in the way you use it, not very   restricting in application.   So   for example, 1 thing that I'm developing right now that I'm especially   enthusiastic about is a threading   system.   So I don't know much about threading at all, but I really wanted to add this. So I'm just really playing   playing around with this currently a lot.  

Actually, already when I started planning the project, I knew that   if I would want to do this in a very professional way, then   I would have to add this at some point.   Then it's basically the idea to keep the abstract logic of the flow in a separate thread. So all your graphic user interface runs in the main thread, and then you have a separate thread where all the abstract logic is executed.   At first, may seem like just some minor performance improvement.   This is actually   crucial for enabling   real time   data processing applications. Whenever you have some kind of static flow of information into your software, say, you have microphone that is recording or you have a camera or any data that's coming in and you do some real time data processing on this data, then you want to have this kind of threaded separation   of responsibilities.  

And it's a hard thing to implement because Riven Core is not only going to provide the management for the abstract logic, but it's also going to provide the graphical user interface for the flows and everything that's contained   because this is effectively   the biggest part of the code.   And therefore, I have to make sure that the whole communication between the internal   abstract objects and the widgets is implemented   somehow for that safe.   And I need to add some kind of intuitive API so that you can easily update your widgets   once your   data computation   has updated. So   Riven core is   will just be like a simple framework to use to create these editors and it just provides you with   a lot of stuff that can be hard to implement. Things like save and load, at first, a prototype for this code generation I was talking about. Things like that that you can, like, make pictures of the flows, which is also already,   part of Riven. So   when you have, like, a flow, you can make a high res you can render a high resolution picture of the flow to use it in a presentation, for example, things like that. And all this will just be done by Riven Core and it will be very easy to create these optimized editors.  

And I feel like there's a lot of potential for this. I already have   a bunch of ideas   for optimized editors, and I actually cannot wait   to to start working on those. So   the first release of all this, this is going to lead to the release of Riven 3. And the first release of all this is probably going to   take place somewhere during the next 3 months, I would say. But, yeah, there is a lot to come.  

[00:44:32] Unknown:

Well, for anybody   who wants to get in touch with you or follow along with the work that you're doing, I'll have you add your preferred contact information to the show notes. And with that, I'll move us into the picks.   And this week, I'm going to choose the project pyinfra.   I started using it recently for   prototyping deployment of a new system that I'm working on at work. I've been pretty happy with it so far, just being able to use   pure Python for being able to build up the logic of getting the server set up and configured as opposed to all the YAML that I've been dealing with for years. So definitely worth checking out if you're working on deployment or management of any systems.   I actually had the author of that project on the podcast a while ago, so I'll have a link to that episode as well. And with that, I'll pass it to you, Leon. Do you have any picks this week? Yeah.  



[00:45:18] Unknown:

So I would actually like to promote a book that I read a few months ago by Lawrence Krauss, who's a theoretical physicist. It's called A Universe From   And it basically   deals with the idea.   How can there be a big bang or something   when there's nothing before?   And   therefore, this is more like it contains a bunch of philosophical ideas, but it's still very scientific.   It's   quite phenomenological   in a way. In my eyes, it contains   a lot of   very important   scientific thinking. So if you're somehow into, like,   theoretical physics or something like that, I can absolutely just recommend this book. It really blew me away because   this kind of thoughtical   scientific thinking was very impressive and   kind of   unique. Yeah. I really enjoyed it. So if you're like somehow into that,   these kinds of books, I can absolutely recommend that.  



[00:46:17] Unknown:

Well, thank you very much for taking the time today to join me and share the work that you've been doing on Riven. It's definitely a very interesting project and a different way of thinking about programming. So I'm excited to see some of the directions that you're able to take it in. So I appreciate all the time and energy you've put into that, and I hope you have a good rest of your day. Thanks for having me. It was a pleasure.   Thank you for listening. Don't forget to check out our other show, the Data Engineering Podcast at data engineering podcast dot com for the latest on modern data management.   And visit the site at pythonpodcast.com   to subscribe to the show, sign up for the mailing list, and read the show notes.  

And if you've learned something or tried out a project from the show, then tell us about it. Email host@podcastinit.com   with your story.   To help other people find the show, please leave a review on Itunes and tell your friends and coworkers.