Summary
With the proliferation of messaging applications, there has been a growing demand for bots that can understand our wishes and perform our bidding. The rise of artificial intelligence has brought the capacity for understanding human language. Combining these two trends gives us chatbots that can be used as a new interface to the software and services that we depend on. This week Joey Faulkner shares his work with Rasa Technologies and their open sourced libraries for understanding natural language and how to conduct a conversation. We talked about how the Rasa Core and Rasa NLU libraries work and how you can use them to replace your dependence on API services and own your data.
Preface
- Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
- I would like to thank everyone who supports us on Patreon. Your contributions help to make the show sustainable.
- When you’re ready to launch your next project you’ll need somewhere to deploy it. Check out Linode at podastinit.com/linode and get a $20 credit to try out their fast and reliable Linux virtual servers for running your awesome app. And now you can deliver your work to your users even faster with the newly upgraded 200 GBit network in all of their datacenters.
- If you’re tired of cobbling together your deployment pipeline then it’s time to try out GoCD, the open source continuous delivery platform built by the people at ThoughtWorks who wrote the book about it. With GoCD you get complete visibility into the life-cycle of your software from one location. To download it now go to podcatinit.com/gocd. Professional support and enterprise plugins are available for added piece of mind.
- Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email hosts@podcastinit.com)
- To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
- Your host as usual is Tobias Macey and today I’m interviewing Joey Faulkner about Rasa Core and Rasa NLU for adding conversational AI to your projects.
Interview
- Introductions
- How did you get introduced to Python?
- Can you start by explaining the goals of Rasa as a company and highlighting the projects that you have open sourced?
- What are the differences between the Rasa Core and Rasa NLU libraries and how do they relate to each other?
- How does the interaction model change when going from state machine driven bots to those which use Rasa Core and what capabilities does it unlock?
- How is Rasa NLU implemented and how has the design evolved?
- What are the motivations for someone to use Rasa core or NLU as a library instead of available API services such as wit.ai, LUIS, or Dialogflow?
- What are some of the biggest challenges in gathering and curating useful training data?
- What is involved in supporting multiple languages for an application using Rasa?
- What are the biggest challenges that you face, past, present, and future, building and growing the tools and platform for Rasa?
- What would be involved for projects such as OpsDroid, Kalliope, or Mycroft to take advantage of Rasa and what benefit would that provide?
- On the comparison page for the hosted Rasa platform it mentions a feature of collaborative model training, can you describe how that works and why someone might want to take advantage of it?
- What are some of the most interesting or unexpected uses of the Rasa tools that you have seen?
- What do you have planned for the future of Rasa?
Keep In Touch
Picks
- Tobias
- Joey
Links
- Rasa Technologies
- Rasa NLU
- Rasa Core
- SpaCy
- yt-project
- Chatbot
- Word2Vec
- State Machine
- Recursive Neural Network
- MITIE
- Support Vector Machine
- Scikit Learn
- wit.ai
- LUIS
- Dialogflow
- Keras
- Reinforcement Learning
The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA
[00:00:14]
Unknown:
Hello, and welcome to podcast dot in it, the podcast about Python and the people who make it great. I would like to thank everyone who supports the show on Patreon. Your contributions help to make the show sustainable. When you're ready to launch your next project, you'll need somewhere to deploy it, so you should check out linode@podcast init.com/linode and get a $20 credit to try out their fast and reliable Linux virtual servers for running your app. And now you can deliver your work to your users even faster with the newly upgraded 200 gigabit network in all of their data centers. If you're tired of cobbling together your deployment pipeline, then it's time to try out GoCD, the open source continuous delivery platform built by the people at Thoughtworks who wrote the book about it. With GoCD, you get complete visibility into the life cycle of your software from 1 location. To download it now, go to podcastinit.com/gocd. Professional support and enterprise plug ins are available for added peace of mind. You can visit the site at podcastinnit.com to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions, I would love to hear them. You can reach me on Twitter at podcastinit or email me at host@podcastinit.com.
To help other people find the show, please leave a review on Itunes or Google Play Music. Tell your friends and coworkers and share it on social media. Your host as usual is Tobias Macy. And today, I'm interviewing Joey Faulkner about Rasa Core and Rasa NLU for adding conversational AI to your projects. So, Joey, could you please introduce yourself?
[00:01:31] Unknown:
Hi. Yeah. I'm I'm Joey and I do AI research for Rasa.
[00:01:36] Unknown:
And do you remember how you first got introduced to Python? I do because it was actually surprisingly recently. So,
[00:01:41] Unknown:
it it was maybe 3 years ago when I was just starting my PhD. And, I just turned up on my 1st day. I haven't done much coding before, and my supervisor was like, yeah, we're gonna be working with simulations. So you're gonna have to use, this thing that's known as y t, which is like this awesome, simulation library. This, like, really clever, very Pythonic and awesome. I had no idea what was going on. So I think I spent the 1st 3 weeks trying to work out how to PIP install stuff because I was so useless and, yeah. So that so it was, it was kind of a trial by fire.
And like looking back, the way I approached Python was like in a way where I was being very stupid about it, and I probably should have just read something before I got started.
[00:02:23] Unknown:
Well, there's something to be said for just jumping in feet first and,
[00:02:27] Unknown:
seeing what happens as you go through it. Yeah. Yeah. I I think I think the, the 1 moment that sticks with me is is when I went to my supervisor and I was like, oh, yeah. This, this thing isn't my code isn't working unless I run it in pseudo. And he was just like, what? Like it just didn't make any sense. And, yeah. So I I I still have no idea what I was doing back then. But yeah.
[00:02:49] Unknown:
And so as we mentioned at the intro, you're gonna be talking to us about the work that you're doing at Rasa Technologies. And so I'm wondering if you can just start by explaining the goals of Rasa as a company and highlighting the projects that you've open sourced in the pursuit of those goals. Yeah. Absolutely.
[00:03:07] Unknown:
So broadly, what we're doing at Rasa is making conversational AI, which I sent a lot of people know as chatbots. And our goals are to firstly, what what we want is to have a a set of libraries such that people, of open source library such that people can make their own chat bots and see what's going on all the way through the process of making a chat bot. So all the way from text to actions produced by the chat bot, people can see exactly what's going on with the data. But but more broadly, we wanna make software that can actually produce like real conversations. Because we looked at the at the market and at this what was being used by a lot of companies and we thought that actually, like, there was a there was a way we could step forward with this.
[00:03:52] Unknown:
And so the 2 projects that are at least being promoted on your main website are rasa core and rasa n l u. So I'm wondering if you can just do a brief rundown of how those 2 libraries relate to each other and what are the reasons somebody might want to use 1 over the other.
[00:04:11] Unknown:
Absolutely. Yeah. Yeah. So actually, Vasco and Vasco and you feed into 1 another. So as I said before, what a chatbot does is essentially go from text to actions. And the pipeline for doing that is you start off by sending your text into RAS NLU, the NLU standing for natural language understanding, where the text goes, it's classified into an intent and entities. So for example, if you were talking to a a weather forecast bot, you could say something like, what's the weather gonna be like in Scotland tomorrow?
And the intent there would be to request for the weather that would, and the end and you also get entities, which are bits of information that the user is trying to pass to the bot. And in that case, you'd have like location being Scotland and the date being tomorrow. So that's Rasa NLU. Basically, we're we're providing an open source framework to go from these the text to the intents and entities. But then Rasa Core is what we know, what we call is the dialogue manager, which goes from these intents and entities to the actions. So in this really simple case of the forecast spot, it would take the fact that you're requesting the weather and the entities, and it would do a series of actions. The first of which being something like query and querying an API saying like, what is the weather in Scotland tomorrow? And then, and if you've got a successful return, then the it would commit another action, which which would be like, tell the person about what the what the weather is gonna be. So message back. So basically, between Rasa and all you and Rasa Core, you have the full chatbot framework to make a bot.
[00:05:47] Unknown:
And just stepping back a bit, just to do a sort of high level discussion of chatbots in general. What do you see as the main driving force for the popularity of that particular sort of style of technology? And what do you think its main benefit is to people who are using them?
[00:06:10] Unknown:
I think there are 2 reasons why chatbots are becoming so popular at the moment. And the first of which is is just technologically, it's it's now possible to do things that even 5 years ago would be it seemed to be impossible. Like in machine learning, there have been a lot of really great step forwards. 1 of the biggest being word2vec, which is, it's a way of creating sort of like, meaning out of words without having to have a very like, explicit definition of how language works, which is awesome. So yeah. So I think I think, the first part is just the capability of making these chat bots and making them work well is is 1 of the reasons why this has become so big recently. But I think second secondarily, just how much people use messenger apps now. Com and compared to like 5, 10 years ago is like huge. Like people most of the capabilities of somebody's phone are used in in messenger.
So before, when you were like, oh, I need to make a website because everybody's on the Internet on Google phone and stuff. That that that make it sense then. But now, since everybody's talking on Facebook, Messenger, and and stuff like this, like, you can see where the people would want to, move into chat bots as it's it's a very natural way of doing user experience. So instead of some somebody going on to your website and trying to work out how to do something by how you've designed it and something that may seem really normal and obvious to you, like, clicking through menus and stuff like that, may be completely alien to someone else. But through natural language and just requesting things from a service is much easier. Just it's much more natural to just ask a service a question than to try and use its own interface, I suppose.
[00:07:55] Unknown:
Yeah. And as you said, 1 of the main reasons why it is feasible today is because of the fact that we do have these, machine learning algorithms that are able to make it feel natural when you're conversing with these bots. Because previously, if you wanted to be able to interact with, you know, a a a textual interface to a program, it was either via the command line or you had to memorize a very limited set of inputs that the bot would be able to understand in order to be able to get a useful output. Whereas with the word2vec algorithm that you mentioned, we now have the capability of being able to do semantic matching between the, you know, the intent behind what that person is trying to do even if they aren't, phrasing it in the exact structure that the bot is expecting or using the exact same words as what it was originally programmed to accept as an input.
[00:08:48] Unknown:
Yeah. Yeah. Absolutely. So I I I think I don't know if people would agree with me, but I think that, Word2Vec has had, like, a huge, huge impact on on NLP in general, specifically in chatbot, in in the chatbot space. It is really awesome to see what you can do with it.
[00:09:05] Unknown:
And we've started to briefly touch on how the interaction model changes when you're going from a traditional conversational bot framework that uses state machines to determine where you are in a conversational flow. And just curious how that particular interaction model changes when you bring in the capabilities of Rasa core.
[00:09:29] Unknown:
Base basically, the way when we were starting out, we were looking at sort of like dialogue managers and what the state of the art was and that what a lot of people use are these state machines, which which have you in a set of discrete states. So say, if we were in the the weather bot state, as well, let's stick with the weather forecast example. So you could, you would start off in the, Greek state where if you said hi, then the bot would say hi back. And then if you ask for the weather, then you move into a new state, which was the weather state. And now you're talking about the weather and stuff like that. And, basically, we don't didn't really think that this is how conversations work. Like, if if you and I were talking about about this, then now we're in the Python state. But if we were talking about sports, we you'd ask the question about sports time being the sports state, but we'd forget that we were ever talking about Python in the first place. So, and basically, we were just like, it doesn't really it's not really how conversations work, and we wanna create something which has the capacity to have real conversations.
I mean, without getting too into details, we, use what's known as a recursive neural network, which, I mean, deep learning is used for everything now. But, it's essentially like a deep learning plat platform with which what we do is we scan over the entire conversation you have with the user and try and predict the next action, not only through what the person has said, what the other person has said to you, but also the history of the of the conversation. And sometimes it won't be relevant and sometimes it will be, but what it means is that, firstly, you're never having to just sit there and define what the conversation can be. Like, what are the finite states of this conversation?
Which which is which is good because you you just have to think, right. Well, how should the bot respond in this con in this in this certain state, and it'll then learn like around that. What the what the actions are as a function of what your history is and what the person has said. But what it basically means is that we don't have to have these linear interactions with users where it's like, okay, now tell me this, now tell me that. We, the bot can learn to go in different ways and to have sort of like a a more conversation, more of a real conversation sort of thing.
But 1 of 1 of the things that we think is really awesome about the fact that we've now transferred this into, because of your networks And new, well, neural networks in general is that instead of having to define the state state machine before as before and like define what the conversation is, What what you can do now is, instead of having to teach the model by telling it what the state machine is, you can teach the model simply by speaking to it. So say, you say hello to it. It'll come up with a prompt and say, the bot thinks that it should say hello back. Should it? Yes. And you respond yes. And then it'll update the parameters of the model such that it it can gains more confidence in how it should respond to users, which is basically a way of bootstrapping training data. So instead of ever ever having to fully define what the bot does, you simply teach it to do stuff by, instruction essentially on the command line.
[00:12:37] Unknown:
And so it sounds like it's using a very interactive super proceed through proceed through the conversational flow that you're trying to teach it to be able to support, and it just is is able to then curate that information and feed it through its neural net to train the model at that point. But 1 of the things that I'm curious about as well is in order for it to even understand what an appropriate prompt is at the beginning of that interaction flow, do you ship a sort of rudimentary model with the library so that you have some sort of starting point? Rather so that it doesn't need to start from ground 0 of not necessarily even understanding what the lang what, you know, what language is and what a conversation is supposed to look like?
[00:13:28] Unknown:
Yeah. Absolutely. So, basically, what we what we do is we ship a few examples of, like, how people would make these bots. Like, just as as most likely do just in the examples folder. But yeah. I should have also said you you define sort of like the domain of things that your bot can do. So you have a set of actions that you that you can write yourself such as like a query API action or like a response action. And then basically, from this domain, you all you all you're doing is you're telling it what actions it can do, what intents it can handle, what entities it should have, and this, and what's what slots it should have. And what these slots are are just a way of storing information in the in the bot. And basically, all all of these things I just said, the intent, actions, entities, and slots, all go in as features to know on that network. So so after you've defined those 4 things, that is the point at which you start bootstrapping training data with the online learning model, as we call it. So then at at that point, you can start teaching it by instruction.
[00:14:32] Unknown:
And in order to be able to parse the natural language input, my understanding is that you're leveraging libraries such as spaCy under the hood. Though looking at the documentation, it appears that you swap that out for some other alternatives. So, at this point, it would be good to do a bit of a deep dive on how the Rasa natural natural language understanding and Rasa core libraries are implemented and maybe talk about how that design has evolved as you have continued to work on and, update and improve the libraries.
[00:15:07] Unknown:
Yeah. Absolutely. So to start with, resnou had sort of like 1 model where we said, okay, we'll take the the word vectors from I think I think we started on spaCy, but it might might have been, Mitty. I'm not sure. But it would just be like, okay. So intent classification, specifically is what. And what actually, no. For for both of them. So to start with, you would take your your word vectors from from spaCy, let's say, and you would just do intent classification using a, simple classifier from, scikit learn, but there are a lot of cases where that doesn't work out perfectly. So for example, we we, wanted to make it good for German speakers to use, Rasa as well. And that that's fine in the sense that there are still still word vectors for German. But, there's a unique problem with well, I don't know if it's unique, but it's it's it's not a problem in English language, which it is a problem in the German language where there are these horrendous compound words where they'll they'll take like 4 or 5 words and jam them together into 1 word.
And of course, since that that word will just not appear much in training data, that they won't have, end up having sort of like converge word, word vectors for that. Which means that you'll lose like a like a a big part of a sentence, which will then, which will normally be a really important part of the sentence to classify what people are, trying to talk about. So what what we did and what is sort of what, it's sort of, This is just sort of an example of how we're trying to change, how we've changed the design of it is, by, implementing a mod a module, which you looks for, series of letters within these compound words and then uses them as extra features onto the classifier.
But yes, a lot of people won't need that. So what we decided to do was make it a lot more modular and it's such that you could say, okay, I want these features, but I don't want these other features by sort of and like defining a pipeline, out of all these features. But what this allow allows us to have is a a library where people can make their own featurizers and add them in and it's it it all works much much more smoothly. So I guess I guess sort of like having the fluidity of people being able to to find their own modules of Rasa and Wu is definitely is definitely a big a big part of how, how it's changed.
[00:17:38] Unknown:
And in the documentation, it calls out, services such as wit.ai or luis or dialogue flow as sort of the main contenders to the capabilities that the Russell libraries provide. And so I, I would just like to see what are the motivations for somebody to use the Russell libraries instead of 1 of those hosted services for being able to provide that conversational interface to their projects?
[00:18:09] Unknown:
Mhmm. So, I mean, like, there's there's there's the first reason which I'm I'm sure a lot of, your listeners will agree with of of, like, just having things that are open source is generally, like, pretty good and pretty cool. But that's not, that's not that compelling a reason I understand. But for me, somebody who comes from a machine learning background, like it, it's very it would be very frustrating like black boxes in general. Like things that you put in, things where you put in data and you get out an answer and you don't know what it's doing. Are incredibly frustrating. Because if they work well, that's fine. And you just, you just carry on with your life, but as soon as there's an error in it or you think there's an error in it, there's nothing you can't do anything about it. It's not like you can go halfway through the the wit.ai or lewis classifier and say, okay. So why is this sentence not being classified badly? So that's that's 1 of the main things. We we we thought we could we could bring a service where where people who wanted to know what, wanted to debug their their, sentences would know why, these things were going wrong.
But I also think that it's kind of a I don't wanna call it scam, but it's it's it's a scam level like because you you have what your when you're using something with with.ai or lewis or something, the the first thing you'll do is you'll train you'll send them a bunch of training data of sort of like sentences and it tends an entities and you'll, you'll give it to them and be like, okay, now make me a, a classifier of these things. And and for for the first part, it'll be free to you and if as long as you go and stay under a few queries, per, I I don't know the exact limits, then you'll you'll get this all for free and that's fine.
But, and then you'll start paying when people start using it in production. But it's kind of a scam because you are you are giving them stuff that's worth value anyway. Like like, 1 of 1 of the most expensive things in machine learning is labeled training data and you're, you're giving them all of your training data and for for nothing, and for a service which, so there was a paper out recently that, if you if you talk to anyone in our company, they'll they'll tell you about because, we we were we were ranked we came second out of 4 where they where they use these API services in terms of our recall. So so you you if if I was goo if I was Amazon or or Google or then I I would say I would say to you, well, the reason why you keep your date, we we are closed source is because we have very, we have proprietary algorithms and we don't wanna, show show what we're doing because because that's where our worth is. But in fact, we we've shown that that with our with our model, which you can literally, get cloned onto your hard drive and we can never take it away from you. You can get similar, if if not better results than a lot of these API services. So yeah.
[00:21:07] Unknown:
Yeah. The factor of being able to control your data is definitely a big win in terms of virtually any service, particularly when you're trying to deploy this in a business context because customer data is 1 of the most valuable differentiators that you can have. And there was a podcast I listened to a little while ago that talked about the concept of data capital and how that's becoming 1 of the biggest differentiators as we continue into this new era of artificial intelligence and machine learning because that's what allows you to leverage your relationships and your core competencies into something that your customers are actually going to want to use.
So being able to have an open source tool chain to be able to collect and, improve with the data that you're receiving is definitely highly valuable. 1 of the other things that got called out in the documentation is the fact that by using 1 of those services, you're sort of forced into, using the generalized, conversational models that they're using because they have to be able to support such a wide variety of use cases. Whereas with Rasa, because you know exactly what it is that you're trying to do with it, you can create a more finely tuned model based on the training data and the interactions that you're using within the context of your project.
And so, a little earlier you mentioned that 1 of the most expensive aspects of building and deploying a learning system is in obtaining the labeled training data. So I'm wondering if you can just talk about some of the biggest challenges that users of the Ross libraries are facing in terms of being able to collect and curate that, data for being able to build the models that they're going to use within the Rasa libraries?
[00:23:08] Unknown:
Yeah. So it's it's actually very human things which are the the biggest problems in trying to train in trying to get this training data. So I know I know personally that like like I'll I'll be making a bot and I'll be using the, the the the sort of like training interface that Rascal, has at where where you type a message, it, classifies the intent and entities, and then you go through the actions like that. And it's and it's very, sort of, like, addictive. You just keep going, and you you try typing things in different ways. But the way I think about saying something, especially textually, is very different to how other people would. Oh, and that's true for everyone. So so you can be training your your model, your bot for ages, and and then somebody else can come along and type a message and you'd be like, oh, I'd never say that to to mean that. And, but, you know, a user would. So, and so but but we, obviously, the the way that, Rasa, I know you and Rasa Core are designed is to generalize across, sort of like how, different people speak and different people say things. So, I think generally the the the the the biggest challenges you'll have is having conversations with people who aren't you with the bot sort of thing. So so, generating an o u that comes from people who aren't so close to the project.
Because this has the problem of if your bot isn't working properly, then you can't just get somebody new to to use it. Because if it has a bug, they're like, oh, I don't know what's going on now. But 1 of the ways that you can get around this is something that we know, we call sort of like, Wizard of Oz And, that the way you can generate training data and sort of debug your conversational script, even though it's not script as, otherwise, we're just using state machine, is to is where you find a new user and the the users sends messages, but on the other end, instead of there being a bot, it's just you. So you, you pretend to be the bot, but through text and if, and it works in a couple of ways. So first of all, you get new NLU data, which is very valuable, but also it's, if you truly like restrict yourself to the domain of I can do this action, I can do this other action, and you find that when somebody says something, you don't know what to do with it, Then you've either realized this the the limit of the scope of your bot or that you need to extend your domain.
So so yeah. So so that's 1 way of getting around this curating the this, training data.
[00:25:40] Unknown:
And so when you have a system that's running already and you encounter 1 of those, sort of colloquialisms or, you know, speech idioms that is unfamiliar to the bot, is there a way for being able to flag that as misunderstood input so that you can then store that in a system to be able to go through and classify for being able to feedback into the training set?
[00:26:08] Unknown:
Yes. At different stages. So if if you have your bot in production, then you you have to sort of like, go through conversations and say, if if there was a problem, you look through it and you say, there was a problem here. And then, you you put that back into the the NLU data and stuff like that. When you're putting the bot into production, it it isn't quite as simple because the the bot is just simply operating and you the the bot classifies intents and entities and and those actions and sometimes those actions will go wrong. But, our hope is that that, users, sort of developers would have a testing mechanism in place that you wouldn't go ahead and show you, make your bot public before testing it on sort of like 10:10 to a 100 people who haven't used it before. And you can go through their conversations and say, right. Did this go okay? Did did this go wrong? Sort of thing. So sort of like a soft launch.
And then you would fold that back into your NLU and your core data and sort of like debug what what the problems were there. But on at the stage when you're first starting and developing it, even just on the Rasa core command line interface, it'll tell you when what it believes the classification of certain text is. And, you will then sort of like, have to tell the the bot that, no, that was wrong. And it and it'll automatically go into, NLU training data, which you can then use to, update your model.
[00:27:38] Unknown:
This is 1 of the cases where having a machine learning or AI driven conversational system can be more beneficial than, for instance, the state machine is because it provides you with more ways and more opportunities for being able to, surface to the user that you didn't understand what their input was. Because if you're using a state machine and somebody enters a, you know, command that is completely out of the bounds of what the state machine was programmed to be able to handle, then there's a good chance that it will either just get into an unknown state or just become unresponsive. Whereas with the conversational AI, if somebody's saying something and it's not able to properly classify it, then you can then program the bot to just come back and say that to the user to say, I'm sorry, but I don't quite understand what it is that you're trying to ask me for. Can you please, you know, rephrase it or something so that you can at least then continue the flow of the dialogue and the user, you know, might not necessarily have the optimal experience, but they're at least getting the feedback that whatever, way that they're asking a question or issuing a command isn't understood by the bot. And then, later on, you can then take that data and then feed it back through so that in future, you know, those particular speech idioms can then be incorporated into the system.
[00:28:59] Unknown:
Right. Exactly. Yeah. Yeah.
[00:29:02] Unknown:
1 of the other big challenges when dealing with any sort of conversational interface is being able to support multiple languages, particularly given that a lot of the work being done in this space is generally fairly, centered around English language input. So I'm wondering what are some of the biggest challenges that you faced on that front, and what are some of the ways that you're working on supporting, additional language inputs and, conversational capabilities for the bot?
[00:29:37] Unknown:
So, it's a great question. And it's it's part part of the reason why Rasa NLU is well, well, why why we've split it into Rasa NLU and Rasa Core. So the the idea is that, since we're going from text to intent and entities in Rasa NLU, the intent and entities should be, sort of like, invariant to what language they're in. So the the transformation from text to intents and entities is definitely very language specific. But when you go from intents and entities to actions, that should happen the same way no matter what language you're in, is the idea. But but obviously, you can change that if if you'd, if that's what your bot needs to do. But yeah, it's a great question. And the the way we designed rational you, which which leverages, very, very much the, word vectors is that essentially if, subject to a pull request, you can just implement a new tokenizer and, load some word vectors, some converge word vectors from the new language. And the intent classification and the entity recognition should work very well at that point.
So especially, languages that are sort of like similar to English and German and stuff. It it should it should work out of the box with these, new word vectors. But which we're trying to make it sort of more general, looking at sort of languages that don't, follow this, the same the same sort of like patterns of, words as tokens as, what sort of English and German stuff does. But it's obviously very hard, and there is, sadly, a very small amount of research in those areas. But but it will be an awesome space to get into, so I I really hope that.
[00:31:22] Unknown:
And what are some of the biggest challenges that you have been, confronted with both in the history of the Rasa project and things that you're facing currently and into the future as far as being able to, build and improve on the tools that you're creating and the sort of, growth of the company and, the pursuit of its goals?
[00:31:47] Unknown:
So, the the the main, I I think 1 of the problems we've come up against is, at least in the past, was people seeing, you know, the big names sort of like IBM, Microsoft, Google, all of them producing these APIs. And then this sort of like tiny company of of, 10 people. And then they're just like, okay, so why shouldn't I choose the the first 3, and why should I choose this sort of like small company? And to start with, it was like, come on. Surely these these these guys, product isn't actually, like, at that level. But, it's over time sort of, like, through through our users, but also through sort of, like, the the, academic research of of people of Rasa and these other APIs.
It's it's kind of starting to show that that we we can compete with these with the big, companies. And we're sort of like approaching things in in in cool ways. So yeah. So so firstly, building that trust that we have a good service. That was a problem in the past. I I think people are starting to to believe. But but for now, I I I think sort of the problem we face is sort of reaching more developers and saying, look, this is, like, legitimate. We are giving you, chatbot software, where you can develop it yourself, you can fork it, and you can make it private. We just we actually just want people to use it and trying to communicate to people that that we we have a awesome product and that they should they they should use it, but at a developer level, not just that, sort of, like, the corporate level.
And I'm trying trying it's sort of trying to build a, we have we have lots of contributors on rasandlu, which is like super awesome. And we have, like, a really active get a community, which which is great. And part of the reason why we love being open source, and trying to move that onto Rasa core as well, is a interesting problem because sort of Rasa core isn't the most standard way of approaching dialogue, but we think it's really great. And we think that if people start using it, they're gonna wanna keep using it.
[00:33:50] Unknown:
I've done a number of shows in the past about different projects that have a conversational component to it, whether it's via text interface such as obstroid for being able to interact with your systems infrastructure or Calliope and Mycroft for providing a voice voice oriented personal assistant capability. And I'm wondering what would be involved for those various projects to replace the, dialogue components of their systems with the capabilities that Rasa provides and what would be the benefit to the end user by for those projects to take that
[00:34:29] Unknown:
step. Specifically on the voice activated, that this is this is sort of an act active research area for us because we have been, focusing, I don't know you towards, like actual types, text to action. So there are a lot of extra things that extra problems that people have with, voice classification because, you know, there are st there are stutters and there are uhs and that there there's a lot more scope from this classification of words. So it's it's definitely an interesting area that we'd like to look into. But but Yeah. For for for now, I'm not a 100% certain how Russell and you would cope with a a voice to a voice to text to actions, transition. But it would be really interesting to find out.
But again, part of the reason why we we split up Rasa into Rasa NLU and Rasa Core is that if Calliope or Mycroft have a sort of like voice to intent pipeline already in place, then what we can offer is sort of like a dialogue manager, which which doesn't which doesn't have to have explicit states and which can be simply trained by by instruction, which can lead to sort of like much more nonlinear and interesting conversations that people could have with a, with the voice assistant. So, I mean, we'd obviously love love it if they, if, if they got involved and, Yeah. And Obstroid, I I haven't looked at the specifics of Obstroid, but again, it we we could we could we can offer an NLU service, which which interpolates really well over different text and can learn with with, small amount of training data.
And again, a, dialogue manager, which which can handle sort of, like, interesting conversations and conversations which aren't just sort of, like, intent to action, intent to action. Sort of, like, build up a a history and build up a real actual conversation.
[00:36:25] Unknown:
Yeah. 1 of the things that came up in particularly the obstroid as well as the Calliope conversations is the difficulty of maintaining context in an interaction with a user because particularly the bots that are focused on group chat and interactive capabilities are generally just keyword or regular expression driven where it's you issue a command, and then it either takes an action or issues a response, and that's essentially the end of the interaction with the bot. And so being able to provide context and create a continuous dialogue with the user so that you can get a much richer in, interface and experience is 1 of the challenges that is currently presented with the, general way that those are being implemented.
So it definitely sounds like Rasa NLU and Rasa Core for being able to maintain the flow of the conversation and not necessarily have to do it via a state machine or explicit state, would definitely be a big win for those projects.
[00:37:31] Unknown:
Yeah. Yeah. Well, it would be, again, it would be totally awesome if if they if they used it. And if if you're listening, feel free to shoot me an email because it would be it would be really cool if we can get Rasa Corp helping, sort of like helping run these bots.
[00:37:46] Unknown:
And on the comparison page on the Rasa website for comparing the hosted Rasa platform with the open source capabilities, it mentions 1 of the features as being the collaborative model training. So I'm wondering if you can just describe a bit about how that works and what are some of the reasons that somebody might want to take advantage of that as opposed to just running itself hosted?
[00:38:12] Unknown:
Yeah. Absolutely. When when we're looking at sort of like commercial applications of Rasa, the amount of training data that's required is, is general generally massive because, not only are there a bunch of intense people want, but there are loads of sort of like, the both the conversations are more complex and also the the, the room for error is is much much worse. Like, if you get something wrong, it's like it's like a actual cost, and that's that really sucks. So we're we're we're trying to make it a lot easier to create this training data. So we create interfaces for people to use use Rasa and train Rasa even if they're not a developer.
So so we we don't only have to have, sort of like, Python programmers teaching the bot how to work or sort of adding intents, adding entities or actions and stuff like this. So that's what we kind of mean by collaborative model training that we can have a bunch of people, all sort of like side by side trying to, improve the model and improving the model as they go essentially.
[00:39:16] Unknown:
And 1 of the things that I was curious about as well is when you are running Rasa embedded in a particular application, particularly for things such as Calliope or Mycroft that are intended to be run on a Raspberry Pi, I'm wondering what are the limitations that, Rasa can handle in terms of, system resources while still being able to run, you know, at an acceptable level and acceptable, level of expressiveness.
[00:39:46] Unknown:
Mhmm. So, in terms of RAS NLU, there is a quite a large well, a large ish memory, requirement through spaCy, simply just, because of the library needing to have access rapidly to every every words, word vector. So you have about 2 gigabytes of mem of RAM taken up as soon as you load base e, which, you will do with SNLU. But I'm I presume they have the same problem anyway. But, yeah. In terms of Rasa core, it's it's, incredibly light. It's a Keras model with, to well, typically the the weights the the weights will, it isn't as deep as the deepest known that we can go. So, like, just simply the storage of those weights and the architecture is actually very low. So so an implementation of brass core shouldn't hog the resources too much.
[00:40:43] Unknown:
And what are some of the most interesting or unexpected uses of the rasa libraries and tool chains that you've seen?
[00:40:51] Unknown:
Yeah. I I actually just came back from a Python conference in, Pycon d, in Karlsruhe. And, I was talking to somebody afterwards, about, after a talk I gave about Rasa tools. And he told me that he was using Rasa to to sort of like run a chat bot, which helped, HIV patients in South Africa get, get from their houses to sort of like the doctors and HIV clinics and stuff. And I just thought that was like really awesome. Like, it's it's it's super cool that since since we have this open source framework and like we don't expect people to pay for for for their usage, they feel like they can do charitable stuff with it. And the the and sort of like the the fact that this person was like we were we were enabling this person to do something which was which was so charitable, was with, and and so useful for his local community was was really really great.
[00:41:46] Unknown:
Yeah. That's definitely 1 of the great things about open source in general is the amount of leverage that it can provide with a minimal amount of investment, you know, particularly monetarily and the ways that people can use that to improve their communities and themselves.
[00:42:06] Unknown:
Yeah. Absolutely. And that's why that's 1 of the 1 of the reasons why I I wanted I got involved with RAS in the first place because I like like most of most of all, when when sort of like, we're working really hard to get something getting something out there and it's just like, ah, what's the point? But it's just like, we want this we want the software to exist, and we want people to have the ability to use it because AI AI is kind of scary. Like, there are a lot of things that AI can do, which, which kind of scare people and can really change the structure of our society. And the idea that, that the capacity to use it is all behind closed doors for a few massive companies is really scary concept.
So the fact that we can make Rasa and give it to people is yeah. I'm I I like, it's some it's something that that really drives me to to work more, I guess.
[00:42:57] Unknown:
Yeah. And that's a great point is, as you said, because of the fact that the public face of artificial intelligence is generally bound up with these large organizations because of the fact that they have all of this customer data that they can use to train these massive and complex models. It makes it seem very foreign and unattainable for the everyday person to be able to try and even comprehend what an artificial intelligence system is or does. And by providing the tools to anybody to be able to experiment with it and see what are some of the ways that it can be used outside of these large organizations that have their own particular agendas, it makes it much more approachable and human scale and, valuable to, you know, the everyday person.
[00:43:52] Unknown:
Yeah. Yeah. Absolutely. Absolutely.
[00:43:55] Unknown:
And what are some of the things that you have planned for the future of Rasa in terms of improvements or enhancements or new features?
[00:44:03] Unknown:
So 1 of 1 of the things that we think would be really cool would be something that's known as reinforcement learning. So instead of you going directly, instead of you going through and hard coding in your, in your bot, like, when you have 1 search result, then you have to do this, this thing. And when do you have 3 search results? You should probably list them before you do anything. What we wanna do is something known as reinforcement learning. So, what that is is where you don't tell the bot anything, and you you give it certain tasks such as, so in the example I've just used, say you're querying an API and you get 3 different results and it has a confidence of 0.6%. It's the first, it's not 0.6. It's the first 1, 0.2. It's the second 1 and 0.2. It's the third 1. And the bot could either sort of like take the first 1 or suggest all 3 and ask for clarification from the user.
What we'd wanna do is we'd want to make it so the bot could learn what the correct action would be in that case. Simply through trying different things out. So so if if the user gets annoyed after you've picked the first 1 because you didn't clarify it was the wrong case, then the bot will be told, ah, that wasn't confident enough. So you should change the model slightly. And what this means is that for like much much more scalable, projects, projects will go out to a lot of people. You can, you obviously give it the, give it the, the bones of how conversation should go, but, but there can be specific flows within the, the larger flow, which are, tuned by simply how these are interact with the bot, which is more like how humans learn in general and how conversations go in general.
[00:45:39] Unknown:
And are there any other topics that you think we should cover before we start to close out the show?
[00:45:44] Unknown:
No. I I think I think,
[00:45:46] Unknown:
that's kind of gone over most of them. And so for anybody who wants to get in touch with you or follow the work that you're up to both personally and with Rasa, I'll have you add your preferred contact information to the show notes. And so with that, I'll bring us to the picks. And, let's see. My pick this week is a book that I've been reading, the information architecture book from O'Reilly, which is the 1 with the polar bear. It's definitely an interesting book. It's largely oriented around user experience design and UI information architecture, but it's definitely worthwhile to read it and consider how those same concepts can be applied to the software and systems infrastructure that we use as we're developing new projects.
So it's definitely giving me a lot to think about, and I think it's worth a read. And so with that, I'll pass it to you, Joey. Do you have any picks for us this week?
[00:46:45] Unknown:
Yeah. So so 1 thing that I would recommend to anyone is a, a Facebook group, known as dog spotting. So, they're essentially, what dog spotting is is a collection of maybe I think it's about 600,000 people now, And, what they do is it's it's sort of like a game where if you see a dog on the street, you take a picture of the dog, and then you get points for it. And then people sort of, like, comment on it being like, oh, there's, oh, you get an extra point because the dog has a hat or something like this. And you just get the funniest, stuff going on there. And it's a really good way of sort of, like, not think not looking at a sublime screen for too long, and just being able to see pictures of few dogs. So yeah, that's what I'd what I'd recommend.
Oh, and another thing, which which is really cool, and is sort of like an illustrative thing of why Rasa is, being open source is like good is that we have, that that somebody released a an interface. Somebody sort of released an open source interface where you can label your own training data called the RAS NLU trainer. And it, means that you don't have to go through like a JSON and label it, and sort of like worry about where your curly brackets are. And it's really awesome. So, yeah. I'll attach the link for that.
[00:48:05] Unknown:
Well, I really appreciate you taking the time out of your day to join me and talk about the work that you're doing with Rasa. It's definitely very interesting, and I'm definitely gonna be following up with some of the past guests such as Obstroid and Calliope to make sure that, your work is on their radar to see if there's any way that they can benefit from it. So, thank you again for your time, and I hope you enjoy the rest of your day.
[00:48:27] Unknown:
Awesome. Thanks so much for having me on. It was, really fun.
[00:48:36] Unknown:
Thanks for listening. Don't forget to go to podcast init.com/linoche to get your $20 credit to run your continuous delivery pipeline with Go CD, which you can download at podcast init.com/gocd. If you enjoyed the show, then don't forget to go to podcastminute.com/itunes and leave a review.
Hello, and welcome to podcast dot in it, the podcast about Python and the people who make it great. I would like to thank everyone who supports the show on Patreon. Your contributions help to make the show sustainable. When you're ready to launch your next project, you'll need somewhere to deploy it, so you should check out linode@podcast init.com/linode and get a $20 credit to try out their fast and reliable Linux virtual servers for running your app. And now you can deliver your work to your users even faster with the newly upgraded 200 gigabit network in all of their data centers. If you're tired of cobbling together your deployment pipeline, then it's time to try out GoCD, the open source continuous delivery platform built by the people at Thoughtworks who wrote the book about it. With GoCD, you get complete visibility into the life cycle of your software from 1 location. To download it now, go to podcastinit.com/gocd. Professional support and enterprise plug ins are available for added peace of mind. You can visit the site at podcastinnit.com to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions, I would love to hear them. You can reach me on Twitter at podcastinit or email me at host@podcastinit.com.
To help other people find the show, please leave a review on Itunes or Google Play Music. Tell your friends and coworkers and share it on social media. Your host as usual is Tobias Macy. And today, I'm interviewing Joey Faulkner about Rasa Core and Rasa NLU for adding conversational AI to your projects. So, Joey, could you please introduce yourself?
[00:01:31] Unknown:
Hi. Yeah. I'm I'm Joey and I do AI research for Rasa.
[00:01:36] Unknown:
And do you remember how you first got introduced to Python? I do because it was actually surprisingly recently. So,
[00:01:41] Unknown:
it it was maybe 3 years ago when I was just starting my PhD. And, I just turned up on my 1st day. I haven't done much coding before, and my supervisor was like, yeah, we're gonna be working with simulations. So you're gonna have to use, this thing that's known as y t, which is like this awesome, simulation library. This, like, really clever, very Pythonic and awesome. I had no idea what was going on. So I think I spent the 1st 3 weeks trying to work out how to PIP install stuff because I was so useless and, yeah. So that so it was, it was kind of a trial by fire.
And like looking back, the way I approached Python was like in a way where I was being very stupid about it, and I probably should have just read something before I got started.
[00:02:23] Unknown:
Well, there's something to be said for just jumping in feet first and,
[00:02:27] Unknown:
seeing what happens as you go through it. Yeah. Yeah. I I think I think the, the 1 moment that sticks with me is is when I went to my supervisor and I was like, oh, yeah. This, this thing isn't my code isn't working unless I run it in pseudo. And he was just like, what? Like it just didn't make any sense. And, yeah. So I I I still have no idea what I was doing back then. But yeah.
[00:02:49] Unknown:
And so as we mentioned at the intro, you're gonna be talking to us about the work that you're doing at Rasa Technologies. And so I'm wondering if you can just start by explaining the goals of Rasa as a company and highlighting the projects that you've open sourced in the pursuit of those goals. Yeah. Absolutely.
[00:03:07] Unknown:
So broadly, what we're doing at Rasa is making conversational AI, which I sent a lot of people know as chatbots. And our goals are to firstly, what what we want is to have a a set of libraries such that people, of open source library such that people can make their own chat bots and see what's going on all the way through the process of making a chat bot. So all the way from text to actions produced by the chat bot, people can see exactly what's going on with the data. But but more broadly, we wanna make software that can actually produce like real conversations. Because we looked at the at the market and at this what was being used by a lot of companies and we thought that actually, like, there was a there was a way we could step forward with this.
[00:03:52] Unknown:
And so the 2 projects that are at least being promoted on your main website are rasa core and rasa n l u. So I'm wondering if you can just do a brief rundown of how those 2 libraries relate to each other and what are the reasons somebody might want to use 1 over the other.
[00:04:11] Unknown:
Absolutely. Yeah. Yeah. So actually, Vasco and Vasco and you feed into 1 another. So as I said before, what a chatbot does is essentially go from text to actions. And the pipeline for doing that is you start off by sending your text into RAS NLU, the NLU standing for natural language understanding, where the text goes, it's classified into an intent and entities. So for example, if you were talking to a a weather forecast bot, you could say something like, what's the weather gonna be like in Scotland tomorrow?
And the intent there would be to request for the weather that would, and the end and you also get entities, which are bits of information that the user is trying to pass to the bot. And in that case, you'd have like location being Scotland and the date being tomorrow. So that's Rasa NLU. Basically, we're we're providing an open source framework to go from these the text to the intents and entities. But then Rasa Core is what we know, what we call is the dialogue manager, which goes from these intents and entities to the actions. So in this really simple case of the forecast spot, it would take the fact that you're requesting the weather and the entities, and it would do a series of actions. The first of which being something like query and querying an API saying like, what is the weather in Scotland tomorrow? And then, and if you've got a successful return, then the it would commit another action, which which would be like, tell the person about what the what the weather is gonna be. So message back. So basically, between Rasa and all you and Rasa Core, you have the full chatbot framework to make a bot.
[00:05:47] Unknown:
And just stepping back a bit, just to do a sort of high level discussion of chatbots in general. What do you see as the main driving force for the popularity of that particular sort of style of technology? And what do you think its main benefit is to people who are using them?
[00:06:10] Unknown:
I think there are 2 reasons why chatbots are becoming so popular at the moment. And the first of which is is just technologically, it's it's now possible to do things that even 5 years ago would be it seemed to be impossible. Like in machine learning, there have been a lot of really great step forwards. 1 of the biggest being word2vec, which is, it's a way of creating sort of like, meaning out of words without having to have a very like, explicit definition of how language works, which is awesome. So yeah. So I think I think, the first part is just the capability of making these chat bots and making them work well is is 1 of the reasons why this has become so big recently. But I think second secondarily, just how much people use messenger apps now. Com and compared to like 5, 10 years ago is like huge. Like people most of the capabilities of somebody's phone are used in in messenger.
So before, when you were like, oh, I need to make a website because everybody's on the Internet on Google phone and stuff. That that that make it sense then. But now, since everybody's talking on Facebook, Messenger, and and stuff like this, like, you can see where the people would want to, move into chat bots as it's it's a very natural way of doing user experience. So instead of some somebody going on to your website and trying to work out how to do something by how you've designed it and something that may seem really normal and obvious to you, like, clicking through menus and stuff like that, may be completely alien to someone else. But through natural language and just requesting things from a service is much easier. Just it's much more natural to just ask a service a question than to try and use its own interface, I suppose.
[00:07:55] Unknown:
Yeah. And as you said, 1 of the main reasons why it is feasible today is because of the fact that we do have these, machine learning algorithms that are able to make it feel natural when you're conversing with these bots. Because previously, if you wanted to be able to interact with, you know, a a a textual interface to a program, it was either via the command line or you had to memorize a very limited set of inputs that the bot would be able to understand in order to be able to get a useful output. Whereas with the word2vec algorithm that you mentioned, we now have the capability of being able to do semantic matching between the, you know, the intent behind what that person is trying to do even if they aren't, phrasing it in the exact structure that the bot is expecting or using the exact same words as what it was originally programmed to accept as an input.
[00:08:48] Unknown:
Yeah. Yeah. Absolutely. So I I I think I don't know if people would agree with me, but I think that, Word2Vec has had, like, a huge, huge impact on on NLP in general, specifically in chatbot, in in the chatbot space. It is really awesome to see what you can do with it.
[00:09:05] Unknown:
And we've started to briefly touch on how the interaction model changes when you're going from a traditional conversational bot framework that uses state machines to determine where you are in a conversational flow. And just curious how that particular interaction model changes when you bring in the capabilities of Rasa core.
[00:09:29] Unknown:
Base basically, the way when we were starting out, we were looking at sort of like dialogue managers and what the state of the art was and that what a lot of people use are these state machines, which which have you in a set of discrete states. So say, if we were in the the weather bot state, as well, let's stick with the weather forecast example. So you could, you would start off in the, Greek state where if you said hi, then the bot would say hi back. And then if you ask for the weather, then you move into a new state, which was the weather state. And now you're talking about the weather and stuff like that. And, basically, we don't didn't really think that this is how conversations work. Like, if if you and I were talking about about this, then now we're in the Python state. But if we were talking about sports, we you'd ask the question about sports time being the sports state, but we'd forget that we were ever talking about Python in the first place. So, and basically, we were just like, it doesn't really it's not really how conversations work, and we wanna create something which has the capacity to have real conversations.
I mean, without getting too into details, we, use what's known as a recursive neural network, which, I mean, deep learning is used for everything now. But, it's essentially like a deep learning plat platform with which what we do is we scan over the entire conversation you have with the user and try and predict the next action, not only through what the person has said, what the other person has said to you, but also the history of the of the conversation. And sometimes it won't be relevant and sometimes it will be, but what it means is that, firstly, you're never having to just sit there and define what the conversation can be. Like, what are the finite states of this conversation?
Which which is which is good because you you just have to think, right. Well, how should the bot respond in this con in this in this certain state, and it'll then learn like around that. What the what the actions are as a function of what your history is and what the person has said. But what it basically means is that we don't have to have these linear interactions with users where it's like, okay, now tell me this, now tell me that. We, the bot can learn to go in different ways and to have sort of like a a more conversation, more of a real conversation sort of thing.
But 1 of 1 of the things that we think is really awesome about the fact that we've now transferred this into, because of your networks And new, well, neural networks in general is that instead of having to define the state state machine before as before and like define what the conversation is, What what you can do now is, instead of having to teach the model by telling it what the state machine is, you can teach the model simply by speaking to it. So say, you say hello to it. It'll come up with a prompt and say, the bot thinks that it should say hello back. Should it? Yes. And you respond yes. And then it'll update the parameters of the model such that it it can gains more confidence in how it should respond to users, which is basically a way of bootstrapping training data. So instead of ever ever having to fully define what the bot does, you simply teach it to do stuff by, instruction essentially on the command line.
[00:12:37] Unknown:
And so it sounds like it's using a very interactive super proceed through proceed through the conversational flow that you're trying to teach it to be able to support, and it just is is able to then curate that information and feed it through its neural net to train the model at that point. But 1 of the things that I'm curious about as well is in order for it to even understand what an appropriate prompt is at the beginning of that interaction flow, do you ship a sort of rudimentary model with the library so that you have some sort of starting point? Rather so that it doesn't need to start from ground 0 of not necessarily even understanding what the lang what, you know, what language is and what a conversation is supposed to look like?
[00:13:28] Unknown:
Yeah. Absolutely. So, basically, what we what we do is we ship a few examples of, like, how people would make these bots. Like, just as as most likely do just in the examples folder. But yeah. I should have also said you you define sort of like the domain of things that your bot can do. So you have a set of actions that you that you can write yourself such as like a query API action or like a response action. And then basically, from this domain, you all you all you're doing is you're telling it what actions it can do, what intents it can handle, what entities it should have, and this, and what's what slots it should have. And what these slots are are just a way of storing information in the in the bot. And basically, all all of these things I just said, the intent, actions, entities, and slots, all go in as features to know on that network. So so after you've defined those 4 things, that is the point at which you start bootstrapping training data with the online learning model, as we call it. So then at at that point, you can start teaching it by instruction.
[00:14:32] Unknown:
And in order to be able to parse the natural language input, my understanding is that you're leveraging libraries such as spaCy under the hood. Though looking at the documentation, it appears that you swap that out for some other alternatives. So, at this point, it would be good to do a bit of a deep dive on how the Rasa natural natural language understanding and Rasa core libraries are implemented and maybe talk about how that design has evolved as you have continued to work on and, update and improve the libraries.
[00:15:07] Unknown:
Yeah. Absolutely. So to start with, resnou had sort of like 1 model where we said, okay, we'll take the the word vectors from I think I think we started on spaCy, but it might might have been, Mitty. I'm not sure. But it would just be like, okay. So intent classification, specifically is what. And what actually, no. For for both of them. So to start with, you would take your your word vectors from from spaCy, let's say, and you would just do intent classification using a, simple classifier from, scikit learn, but there are a lot of cases where that doesn't work out perfectly. So for example, we we, wanted to make it good for German speakers to use, Rasa as well. And that that's fine in the sense that there are still still word vectors for German. But, there's a unique problem with well, I don't know if it's unique, but it's it's it's not a problem in English language, which it is a problem in the German language where there are these horrendous compound words where they'll they'll take like 4 or 5 words and jam them together into 1 word.
And of course, since that that word will just not appear much in training data, that they won't have, end up having sort of like converge word, word vectors for that. Which means that you'll lose like a like a a big part of a sentence, which will then, which will normally be a really important part of the sentence to classify what people are, trying to talk about. So what what we did and what is sort of what, it's sort of, This is just sort of an example of how we're trying to change, how we've changed the design of it is, by, implementing a mod a module, which you looks for, series of letters within these compound words and then uses them as extra features onto the classifier.
But yes, a lot of people won't need that. So what we decided to do was make it a lot more modular and it's such that you could say, okay, I want these features, but I don't want these other features by sort of and like defining a pipeline, out of all these features. But what this allow allows us to have is a a library where people can make their own featurizers and add them in and it's it it all works much much more smoothly. So I guess I guess sort of like having the fluidity of people being able to to find their own modules of Rasa and Wu is definitely is definitely a big a big part of how, how it's changed.
[00:17:38] Unknown:
And in the documentation, it calls out, services such as wit.ai or luis or dialogue flow as sort of the main contenders to the capabilities that the Russell libraries provide. And so I, I would just like to see what are the motivations for somebody to use the Russell libraries instead of 1 of those hosted services for being able to provide that conversational interface to their projects?
[00:18:09] Unknown:
Mhmm. So, I mean, like, there's there's there's the first reason which I'm I'm sure a lot of, your listeners will agree with of of, like, just having things that are open source is generally, like, pretty good and pretty cool. But that's not, that's not that compelling a reason I understand. But for me, somebody who comes from a machine learning background, like it, it's very it would be very frustrating like black boxes in general. Like things that you put in, things where you put in data and you get out an answer and you don't know what it's doing. Are incredibly frustrating. Because if they work well, that's fine. And you just, you just carry on with your life, but as soon as there's an error in it or you think there's an error in it, there's nothing you can't do anything about it. It's not like you can go halfway through the the wit.ai or lewis classifier and say, okay. So why is this sentence not being classified badly? So that's that's 1 of the main things. We we we thought we could we could bring a service where where people who wanted to know what, wanted to debug their their, sentences would know why, these things were going wrong.
But I also think that it's kind of a I don't wanna call it scam, but it's it's it's a scam level like because you you have what your when you're using something with with.ai or lewis or something, the the first thing you'll do is you'll train you'll send them a bunch of training data of sort of like sentences and it tends an entities and you'll, you'll give it to them and be like, okay, now make me a, a classifier of these things. And and for for the first part, it'll be free to you and if as long as you go and stay under a few queries, per, I I don't know the exact limits, then you'll you'll get this all for free and that's fine.
But, and then you'll start paying when people start using it in production. But it's kind of a scam because you are you are giving them stuff that's worth value anyway. Like like, 1 of 1 of the most expensive things in machine learning is labeled training data and you're, you're giving them all of your training data and for for nothing, and for a service which, so there was a paper out recently that, if you if you talk to anyone in our company, they'll they'll tell you about because, we we were we were ranked we came second out of 4 where they where they use these API services in terms of our recall. So so you you if if I was goo if I was Amazon or or Google or then I I would say I would say to you, well, the reason why you keep your date, we we are closed source is because we have very, we have proprietary algorithms and we don't wanna, show show what we're doing because because that's where our worth is. But in fact, we we've shown that that with our with our model, which you can literally, get cloned onto your hard drive and we can never take it away from you. You can get similar, if if not better results than a lot of these API services. So yeah.
[00:21:07] Unknown:
Yeah. The factor of being able to control your data is definitely a big win in terms of virtually any service, particularly when you're trying to deploy this in a business context because customer data is 1 of the most valuable differentiators that you can have. And there was a podcast I listened to a little while ago that talked about the concept of data capital and how that's becoming 1 of the biggest differentiators as we continue into this new era of artificial intelligence and machine learning because that's what allows you to leverage your relationships and your core competencies into something that your customers are actually going to want to use.
So being able to have an open source tool chain to be able to collect and, improve with the data that you're receiving is definitely highly valuable. 1 of the other things that got called out in the documentation is the fact that by using 1 of those services, you're sort of forced into, using the generalized, conversational models that they're using because they have to be able to support such a wide variety of use cases. Whereas with Rasa, because you know exactly what it is that you're trying to do with it, you can create a more finely tuned model based on the training data and the interactions that you're using within the context of your project.
And so, a little earlier you mentioned that 1 of the most expensive aspects of building and deploying a learning system is in obtaining the labeled training data. So I'm wondering if you can just talk about some of the biggest challenges that users of the Ross libraries are facing in terms of being able to collect and curate that, data for being able to build the models that they're going to use within the Rasa libraries?
[00:23:08] Unknown:
Yeah. So it's it's actually very human things which are the the biggest problems in trying to train in trying to get this training data. So I know I know personally that like like I'll I'll be making a bot and I'll be using the, the the the sort of like training interface that Rascal, has at where where you type a message, it, classifies the intent and entities, and then you go through the actions like that. And it's and it's very, sort of, like, addictive. You just keep going, and you you try typing things in different ways. But the way I think about saying something, especially textually, is very different to how other people would. Oh, and that's true for everyone. So so you can be training your your model, your bot for ages, and and then somebody else can come along and type a message and you'd be like, oh, I'd never say that to to mean that. And, but, you know, a user would. So, and so but but we, obviously, the the way that, Rasa, I know you and Rasa Core are designed is to generalize across, sort of like how, different people speak and different people say things. So, I think generally the the the the the biggest challenges you'll have is having conversations with people who aren't you with the bot sort of thing. So so, generating an o u that comes from people who aren't so close to the project.
Because this has the problem of if your bot isn't working properly, then you can't just get somebody new to to use it. Because if it has a bug, they're like, oh, I don't know what's going on now. But 1 of the ways that you can get around this is something that we know, we call sort of like, Wizard of Oz And, that the way you can generate training data and sort of debug your conversational script, even though it's not script as, otherwise, we're just using state machine, is to is where you find a new user and the the users sends messages, but on the other end, instead of there being a bot, it's just you. So you, you pretend to be the bot, but through text and if, and it works in a couple of ways. So first of all, you get new NLU data, which is very valuable, but also it's, if you truly like restrict yourself to the domain of I can do this action, I can do this other action, and you find that when somebody says something, you don't know what to do with it, Then you've either realized this the the limit of the scope of your bot or that you need to extend your domain.
So so yeah. So so that's 1 way of getting around this curating the this, training data.
[00:25:40] Unknown:
And so when you have a system that's running already and you encounter 1 of those, sort of colloquialisms or, you know, speech idioms that is unfamiliar to the bot, is there a way for being able to flag that as misunderstood input so that you can then store that in a system to be able to go through and classify for being able to feedback into the training set?
[00:26:08] Unknown:
Yes. At different stages. So if if you have your bot in production, then you you have to sort of like, go through conversations and say, if if there was a problem, you look through it and you say, there was a problem here. And then, you you put that back into the the NLU data and stuff like that. When you're putting the bot into production, it it isn't quite as simple because the the bot is just simply operating and you the the bot classifies intents and entities and and those actions and sometimes those actions will go wrong. But, our hope is that that, users, sort of developers would have a testing mechanism in place that you wouldn't go ahead and show you, make your bot public before testing it on sort of like 10:10 to a 100 people who haven't used it before. And you can go through their conversations and say, right. Did this go okay? Did did this go wrong? Sort of thing. So sort of like a soft launch.
And then you would fold that back into your NLU and your core data and sort of like debug what what the problems were there. But on at the stage when you're first starting and developing it, even just on the Rasa core command line interface, it'll tell you when what it believes the classification of certain text is. And, you will then sort of like, have to tell the the bot that, no, that was wrong. And it and it'll automatically go into, NLU training data, which you can then use to, update your model.
[00:27:38] Unknown:
This is 1 of the cases where having a machine learning or AI driven conversational system can be more beneficial than, for instance, the state machine is because it provides you with more ways and more opportunities for being able to, surface to the user that you didn't understand what their input was. Because if you're using a state machine and somebody enters a, you know, command that is completely out of the bounds of what the state machine was programmed to be able to handle, then there's a good chance that it will either just get into an unknown state or just become unresponsive. Whereas with the conversational AI, if somebody's saying something and it's not able to properly classify it, then you can then program the bot to just come back and say that to the user to say, I'm sorry, but I don't quite understand what it is that you're trying to ask me for. Can you please, you know, rephrase it or something so that you can at least then continue the flow of the dialogue and the user, you know, might not necessarily have the optimal experience, but they're at least getting the feedback that whatever, way that they're asking a question or issuing a command isn't understood by the bot. And then, later on, you can then take that data and then feed it back through so that in future, you know, those particular speech idioms can then be incorporated into the system.
[00:28:59] Unknown:
Right. Exactly. Yeah. Yeah.
[00:29:02] Unknown:
1 of the other big challenges when dealing with any sort of conversational interface is being able to support multiple languages, particularly given that a lot of the work being done in this space is generally fairly, centered around English language input. So I'm wondering what are some of the biggest challenges that you faced on that front, and what are some of the ways that you're working on supporting, additional language inputs and, conversational capabilities for the bot?
[00:29:37] Unknown:
So, it's a great question. And it's it's part part of the reason why Rasa NLU is well, well, why why we've split it into Rasa NLU and Rasa Core. So the the idea is that, since we're going from text to intent and entities in Rasa NLU, the intent and entities should be, sort of like, invariant to what language they're in. So the the transformation from text to intents and entities is definitely very language specific. But when you go from intents and entities to actions, that should happen the same way no matter what language you're in, is the idea. But but obviously, you can change that if if you'd, if that's what your bot needs to do. But yeah, it's a great question. And the the way we designed rational you, which which leverages, very, very much the, word vectors is that essentially if, subject to a pull request, you can just implement a new tokenizer and, load some word vectors, some converge word vectors from the new language. And the intent classification and the entity recognition should work very well at that point.
So especially, languages that are sort of like similar to English and German and stuff. It it should it should work out of the box with these, new word vectors. But which we're trying to make it sort of more general, looking at sort of languages that don't, follow this, the same the same sort of like patterns of, words as tokens as, what sort of English and German stuff does. But it's obviously very hard, and there is, sadly, a very small amount of research in those areas. But but it will be an awesome space to get into, so I I really hope that.
[00:31:22] Unknown:
And what are some of the biggest challenges that you have been, confronted with both in the history of the Rasa project and things that you're facing currently and into the future as far as being able to, build and improve on the tools that you're creating and the sort of, growth of the company and, the pursuit of its goals?
[00:31:47] Unknown:
So, the the the main, I I think 1 of the problems we've come up against is, at least in the past, was people seeing, you know, the big names sort of like IBM, Microsoft, Google, all of them producing these APIs. And then this sort of like tiny company of of, 10 people. And then they're just like, okay, so why shouldn't I choose the the first 3, and why should I choose this sort of like small company? And to start with, it was like, come on. Surely these these these guys, product isn't actually, like, at that level. But, it's over time sort of, like, through through our users, but also through sort of, like, the the, academic research of of people of Rasa and these other APIs.
It's it's kind of starting to show that that we we can compete with these with the big, companies. And we're sort of like approaching things in in in cool ways. So yeah. So so firstly, building that trust that we have a good service. That was a problem in the past. I I think people are starting to to believe. But but for now, I I I think sort of the problem we face is sort of reaching more developers and saying, look, this is, like, legitimate. We are giving you, chatbot software, where you can develop it yourself, you can fork it, and you can make it private. We just we actually just want people to use it and trying to communicate to people that that we we have a awesome product and that they should they they should use it, but at a developer level, not just that, sort of, like, the corporate level.
And I'm trying trying it's sort of trying to build a, we have we have lots of contributors on rasandlu, which is like super awesome. And we have, like, a really active get a community, which which is great. And part of the reason why we love being open source, and trying to move that onto Rasa core as well, is a interesting problem because sort of Rasa core isn't the most standard way of approaching dialogue, but we think it's really great. And we think that if people start using it, they're gonna wanna keep using it.
[00:33:50] Unknown:
I've done a number of shows in the past about different projects that have a conversational component to it, whether it's via text interface such as obstroid for being able to interact with your systems infrastructure or Calliope and Mycroft for providing a voice voice oriented personal assistant capability. And I'm wondering what would be involved for those various projects to replace the, dialogue components of their systems with the capabilities that Rasa provides and what would be the benefit to the end user by for those projects to take that
[00:34:29] Unknown:
step. Specifically on the voice activated, that this is this is sort of an act active research area for us because we have been, focusing, I don't know you towards, like actual types, text to action. So there are a lot of extra things that extra problems that people have with, voice classification because, you know, there are st there are stutters and there are uhs and that there there's a lot more scope from this classification of words. So it's it's definitely an interesting area that we'd like to look into. But but Yeah. For for for now, I'm not a 100% certain how Russell and you would cope with a a voice to a voice to text to actions, transition. But it would be really interesting to find out.
But again, part of the reason why we we split up Rasa into Rasa NLU and Rasa Core is that if Calliope or Mycroft have a sort of like voice to intent pipeline already in place, then what we can offer is sort of like a dialogue manager, which which doesn't which doesn't have to have explicit states and which can be simply trained by by instruction, which can lead to sort of like much more nonlinear and interesting conversations that people could have with a, with the voice assistant. So, I mean, we'd obviously love love it if they, if, if they got involved and, Yeah. And Obstroid, I I haven't looked at the specifics of Obstroid, but again, it we we could we could we can offer an NLU service, which which interpolates really well over different text and can learn with with, small amount of training data.
And again, a, dialogue manager, which which can handle sort of, like, interesting conversations and conversations which aren't just sort of, like, intent to action, intent to action. Sort of, like, build up a a history and build up a real actual conversation.
[00:36:25] Unknown:
Yeah. 1 of the things that came up in particularly the obstroid as well as the Calliope conversations is the difficulty of maintaining context in an interaction with a user because particularly the bots that are focused on group chat and interactive capabilities are generally just keyword or regular expression driven where it's you issue a command, and then it either takes an action or issues a response, and that's essentially the end of the interaction with the bot. And so being able to provide context and create a continuous dialogue with the user so that you can get a much richer in, interface and experience is 1 of the challenges that is currently presented with the, general way that those are being implemented.
So it definitely sounds like Rasa NLU and Rasa Core for being able to maintain the flow of the conversation and not necessarily have to do it via a state machine or explicit state, would definitely be a big win for those projects.
[00:37:31] Unknown:
Yeah. Yeah. Well, it would be, again, it would be totally awesome if if they if they used it. And if if you're listening, feel free to shoot me an email because it would be it would be really cool if we can get Rasa Corp helping, sort of like helping run these bots.
[00:37:46] Unknown:
And on the comparison page on the Rasa website for comparing the hosted Rasa platform with the open source capabilities, it mentions 1 of the features as being the collaborative model training. So I'm wondering if you can just describe a bit about how that works and what are some of the reasons that somebody might want to take advantage of that as opposed to just running itself hosted?
[00:38:12] Unknown:
Yeah. Absolutely. When when we're looking at sort of like commercial applications of Rasa, the amount of training data that's required is, is general generally massive because, not only are there a bunch of intense people want, but there are loads of sort of like, the both the conversations are more complex and also the the, the room for error is is much much worse. Like, if you get something wrong, it's like it's like a actual cost, and that's that really sucks. So we're we're we're trying to make it a lot easier to create this training data. So we create interfaces for people to use use Rasa and train Rasa even if they're not a developer.
So so we we don't only have to have, sort of like, Python programmers teaching the bot how to work or sort of adding intents, adding entities or actions and stuff like this. So that's what we kind of mean by collaborative model training that we can have a bunch of people, all sort of like side by side trying to, improve the model and improving the model as they go essentially.
[00:39:16] Unknown:
And 1 of the things that I was curious about as well is when you are running Rasa embedded in a particular application, particularly for things such as Calliope or Mycroft that are intended to be run on a Raspberry Pi, I'm wondering what are the limitations that, Rasa can handle in terms of, system resources while still being able to run, you know, at an acceptable level and acceptable, level of expressiveness.
[00:39:46] Unknown:
Mhmm. So, in terms of RAS NLU, there is a quite a large well, a large ish memory, requirement through spaCy, simply just, because of the library needing to have access rapidly to every every words, word vector. So you have about 2 gigabytes of mem of RAM taken up as soon as you load base e, which, you will do with SNLU. But I'm I presume they have the same problem anyway. But, yeah. In terms of Rasa core, it's it's, incredibly light. It's a Keras model with, to well, typically the the weights the the weights will, it isn't as deep as the deepest known that we can go. So, like, just simply the storage of those weights and the architecture is actually very low. So so an implementation of brass core shouldn't hog the resources too much.
[00:40:43] Unknown:
And what are some of the most interesting or unexpected uses of the rasa libraries and tool chains that you've seen?
[00:40:51] Unknown:
Yeah. I I actually just came back from a Python conference in, Pycon d, in Karlsruhe. And, I was talking to somebody afterwards, about, after a talk I gave about Rasa tools. And he told me that he was using Rasa to to sort of like run a chat bot, which helped, HIV patients in South Africa get, get from their houses to sort of like the doctors and HIV clinics and stuff. And I just thought that was like really awesome. Like, it's it's it's super cool that since since we have this open source framework and like we don't expect people to pay for for for their usage, they feel like they can do charitable stuff with it. And the the and sort of like the the fact that this person was like we were we were enabling this person to do something which was which was so charitable, was with, and and so useful for his local community was was really really great.
[00:41:46] Unknown:
Yeah. That's definitely 1 of the great things about open source in general is the amount of leverage that it can provide with a minimal amount of investment, you know, particularly monetarily and the ways that people can use that to improve their communities and themselves.
[00:42:06] Unknown:
Yeah. Absolutely. And that's why that's 1 of the 1 of the reasons why I I wanted I got involved with RAS in the first place because I like like most of most of all, when when sort of like, we're working really hard to get something getting something out there and it's just like, ah, what's the point? But it's just like, we want this we want the software to exist, and we want people to have the ability to use it because AI AI is kind of scary. Like, there are a lot of things that AI can do, which, which kind of scare people and can really change the structure of our society. And the idea that, that the capacity to use it is all behind closed doors for a few massive companies is really scary concept.
So the fact that we can make Rasa and give it to people is yeah. I'm I I like, it's some it's something that that really drives me to to work more, I guess.
[00:42:57] Unknown:
Yeah. And that's a great point is, as you said, because of the fact that the public face of artificial intelligence is generally bound up with these large organizations because of the fact that they have all of this customer data that they can use to train these massive and complex models. It makes it seem very foreign and unattainable for the everyday person to be able to try and even comprehend what an artificial intelligence system is or does. And by providing the tools to anybody to be able to experiment with it and see what are some of the ways that it can be used outside of these large organizations that have their own particular agendas, it makes it much more approachable and human scale and, valuable to, you know, the everyday person.
[00:43:52] Unknown:
Yeah. Yeah. Absolutely. Absolutely.
[00:43:55] Unknown:
And what are some of the things that you have planned for the future of Rasa in terms of improvements or enhancements or new features?
[00:44:03] Unknown:
So 1 of 1 of the things that we think would be really cool would be something that's known as reinforcement learning. So instead of you going directly, instead of you going through and hard coding in your, in your bot, like, when you have 1 search result, then you have to do this, this thing. And when do you have 3 search results? You should probably list them before you do anything. What we wanna do is something known as reinforcement learning. So, what that is is where you don't tell the bot anything, and you you give it certain tasks such as, so in the example I've just used, say you're querying an API and you get 3 different results and it has a confidence of 0.6%. It's the first, it's not 0.6. It's the first 1, 0.2. It's the second 1 and 0.2. It's the third 1. And the bot could either sort of like take the first 1 or suggest all 3 and ask for clarification from the user.
What we'd wanna do is we'd want to make it so the bot could learn what the correct action would be in that case. Simply through trying different things out. So so if if the user gets annoyed after you've picked the first 1 because you didn't clarify it was the wrong case, then the bot will be told, ah, that wasn't confident enough. So you should change the model slightly. And what this means is that for like much much more scalable, projects, projects will go out to a lot of people. You can, you obviously give it the, give it the, the bones of how conversation should go, but, but there can be specific flows within the, the larger flow, which are, tuned by simply how these are interact with the bot, which is more like how humans learn in general and how conversations go in general.
[00:45:39] Unknown:
And are there any other topics that you think we should cover before we start to close out the show?
[00:45:44] Unknown:
No. I I think I think,
[00:45:46] Unknown:
that's kind of gone over most of them. And so for anybody who wants to get in touch with you or follow the work that you're up to both personally and with Rasa, I'll have you add your preferred contact information to the show notes. And so with that, I'll bring us to the picks. And, let's see. My pick this week is a book that I've been reading, the information architecture book from O'Reilly, which is the 1 with the polar bear. It's definitely an interesting book. It's largely oriented around user experience design and UI information architecture, but it's definitely worthwhile to read it and consider how those same concepts can be applied to the software and systems infrastructure that we use as we're developing new projects.
So it's definitely giving me a lot to think about, and I think it's worth a read. And so with that, I'll pass it to you, Joey. Do you have any picks for us this week?
[00:46:45] Unknown:
Yeah. So so 1 thing that I would recommend to anyone is a, a Facebook group, known as dog spotting. So, they're essentially, what dog spotting is is a collection of maybe I think it's about 600,000 people now, And, what they do is it's it's sort of like a game where if you see a dog on the street, you take a picture of the dog, and then you get points for it. And then people sort of, like, comment on it being like, oh, there's, oh, you get an extra point because the dog has a hat or something like this. And you just get the funniest, stuff going on there. And it's a really good way of sort of, like, not think not looking at a sublime screen for too long, and just being able to see pictures of few dogs. So yeah, that's what I'd what I'd recommend.
Oh, and another thing, which which is really cool, and is sort of like an illustrative thing of why Rasa is, being open source is like good is that we have, that that somebody released a an interface. Somebody sort of released an open source interface where you can label your own training data called the RAS NLU trainer. And it, means that you don't have to go through like a JSON and label it, and sort of like worry about where your curly brackets are. And it's really awesome. So, yeah. I'll attach the link for that.
[00:48:05] Unknown:
Well, I really appreciate you taking the time out of your day to join me and talk about the work that you're doing with Rasa. It's definitely very interesting, and I'm definitely gonna be following up with some of the past guests such as Obstroid and Calliope to make sure that, your work is on their radar to see if there's any way that they can benefit from it. So, thank you again for your time, and I hope you enjoy the rest of your day.
[00:48:27] Unknown:
Awesome. Thanks so much for having me on. It was, really fun.
[00:48:36] Unknown:
Thanks for listening. Don't forget to go to podcast init.com/linoche to get your $20 credit to run your continuous delivery pipeline with Go CD, which you can download at podcast init.com/gocd. If you enjoyed the show, then don't forget to go to podcastminute.com/itunes and leave a review.
Introduction and Guest Introduction
Joey Faulkner's Journey to Python
Overview of Rasa Technologies
Rasa Core and Rasa NLU
The Popularity and Benefits of Chatbots
Improving Conversational AI with Rasa Core
Initial Model and Training Data
Implementation and Evolution of Rasa Libraries
Comparison with Hosted Services
Challenges in Collecting Training Data
Handling Misunderstood Input
Supporting Multiple Languages
Challenges and Growth of Rasa
Integration with Other Projects
Collaborative Model Training
System Resource Limitations
Interesting Uses of Rasa
Future Plans for Rasa
Closing Remarks and Picks
