Summary
Wouldn’t it be nice to have a personal assistant to answer your questions, help you remember important tasks, and control your environment? Meet Kalliope, a Python powered, modular, voice controlled automation platform. This week Nicolas Marcq and Thibaud Buffet explain how they started the project, what makes it stand out from other open source and commercial options, and how you can start using it today.
Preface
- Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
- I would like to thank everyone who supports us on Patreon. Your contributions help to make the show sustainable.
- When you’re ready to launch your next project you’ll need somewhere to deploy it. Check out Linode at www.podastinit.com/linode?utm_source=rss&utm_medium=rss and get a $20 credit to try out their fast and reliable Linux virtual servers for running your awesome app.
- Need to learn more about how to scale your apps or learn new techniques for building them? Pluralsight has the training and mentoring you need to level up your skills. Go to www.pythonpodcast.com/pluralsight?utm_source=rss&utm_medium=rss to start your free trial today.
- Visit the site to subscribe to the show, sign up for the newsletter, read the show notes, and get in touch.
- To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
- If you work with data for your job or want to learn more about how open source is powering the latest innovations in data science then make your way to the Open Data Science Conference, happening in London in October and San Francisco in November. Follow the links in the show notes to register and help support the show in the process.
- Your host as usual is Tobias Macey and today I’m interviewing Nicolas Marcq and Thibaud Buffet about Kalliope, a modular always-on voice controlled personal assistant designed for home automation.
Interview
- Introductions
- How did you get introduced to Python?
- What is the Kalliope project and how did it get started?
- How does Kalliope compare to commercial options such as Amazon Alexa and Google Home, as well as other open source projects such as Mycroft or Jasper?
- The majority of voice assistant projects that I have seen default to interacting in English, whereas Kalliope is multi-lingual. What led you to that design choice and how is that implemented?
- One of the perennial questions around voice assistants is privacy, so how does Kalliope work to mitigate the issues associated with having an always on device listening in people’s homes?
- How is Kalliope architected internally and how has the design evolved over time?
- What are some of the most difficult or challenging aspects of building Kalliope and its associated projects?
- What are some of the most interesting uses of Kalliope that you are aware of?
- What are some of the most notable features or improvements that you have planned for the future of Kalliope?
- How has the choice of Python as the implementation worked for you, and if you were to start over today do you think you would make the same decision?
Keep In Touch
- Nicolas
- Thibaud
Picks
- Tobias
- Nicolas
- Thibaud
Links
- Snowboy
- Mycroft
- Mycroft Interview
- Amazon Alexa
- Google Home
- Jasper
- Kalliope
- TTS
- STT
- CMU Sphinx
- Abstract Base Class
- MQTT
- RxPy Interview
The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA
[00:00:14]
Unknown:
Hello, and welcome to podcast dot in it, the podcast about Python and the people who make it great. I would like to thank everyone who supports us on Patreon. Your contributions helped to make the show sustainable. When you're ready to launch your next project, you'll need somewhere to deploy it, so you should check out Linode at www.podcastinit.com/linode and get a $20 credit to try out their fast and reliable Linux virtual servers for running your next app. Do you need to learn more about how to scale your apps or learn new techniques for building them? Pluralsight has the training and mentoring you need to level up your skills. Go to www.podcastinit.com/pluralsite to start your free trial today.
You can visit the site to subscribe to the show, sign up for the newsletter, read the show notes, get in touch, and support the show. To help other people find the show, please leave a review on Itunes or Google Play Music, tell your friends and coworkers, and share it on social media. If you work with data for your job or want to learn more about how open source is powering the latest innovations in data science, then make your way to the Open Data Science Conference happening in London in October and San Francisco in November. Follow the links in the show notes to register. Your host as usual is Tobias Macy. And today, I'm interviewing Nicolas Marc and Thibault Buffet about Calliope, a modular always on voice controlled personal assistant designed for home automation.
So, Nicolas, could you start by introducing yourself?
[00:01:30] Unknown:
Sure. So I'm Nicolas, and I'm, IT engineer engineer and, also, And,
[00:01:38] Unknown:
that's it. And, Thibault, how about yourself? Hi, everyone.
[00:01:42] Unknown:
I'm Thibault Buffet. I'm also an IT engineer, mostly a software engineer. I'm used to develop in Java, but I know it today. We are talking about Python, but, my expertise is more on Java. And, yeah, I'm also a beta geek. We are developing Kalliopo since since last year. So yeah.
[00:02:05] Unknown:
And, Nikola, do you remember how you first got introduced to Python?
[00:02:09] Unknown:
I was a trainee in a IT company during my studies, and I started working on a project, with Jumbo Framework. And I remember same year, the first Raspberry Pi has been released, and so Python was supposed to be present, for this device. So I started calling from with Python.
[00:02:27] Unknown:
And, Thibault, how about yourself?
[00:02:29] Unknown:
To be honest, it was a long time ago I wanted to give it a try to Python. And I take the opportunity when we start playing around with Kalliopi, and Nicolas started to to introduce me to Python. And it was the best opportunity to give it a try, so I did it.
[00:02:48] Unknown:
So I gave a brief overview of what the Calliope project is, but I'm wondering if you guys can give a bit more detailed description of the project and how it got started and what inspired you to start working on it.
[00:03:01] Unknown:
Yes. So at the beginning, we felt, on, an article on the Internet that was talking about a little library called Snowboy. Snowboy, it's a is a smaller engine, which is always listening for an artwork, and, it run a script once the the world is recognized. So we directly source about, the journey's like program. We started looking around for existing project. We found, we found some interesting thing, but, for all of them, it was not exactly what we had in mind. So we started, good coding more more our project. And, actually, it was even not a a real project at the beginning, but just a smaller bank on scripts that answer your sentence when you say something. And to answer what CayoPay is, it's a simple framework to help you that can help you to build, your own personal assets like, Alexa or Google. Tivo, maybe you want to add something?
[00:04:03] Unknown:
Well, no. Basically, you did, everything. You talk about everything. But, yeah, at the beginning, it was, as Nicolas said, really, just a a way to have fun. And we were joking about it at a coffee time, in, at work. Finally, a little about step by step, it become more important, and some people try some people get interested by the project. A little community helped us also to test the project and, to, to continue and give ideas. So we continue playing around with the with this. Finally, we are here 1 year later.
[00:04:49] Unknown:
And as you mentioned, Calliope is operating in the similar spaces, the Amazon Alexa and Google Home, as well as some of the other open source projects such as Mycroft or Jasper. So I'm wondering if you can do a bit of comparison describe why somebody might want to use Calliope in place of any of those other projects.
[00:05:07] Unknown:
Well, to be honest, I don't know well other projects. But, from what we saw so far, k u p is more like a framework. Each user has to work a little bit before being able to use it. It's not really everything is done out of the box like Google Home or Alexa, but we ask yes. We ask the user to release this bot. In most case, in other project, we have seen, also only English is supported and, your orders are are coded. So for for 1 order, you can earn only 1 action. And with KU Pay, you can select the sentence and of your choice, and, you can attach 1 or more action to it. That's why we have created, KDP. It's to be free for for support.
[00:05:54] Unknown:
Yeah. Yeah. And you you said the most important, but, the IDB and Clarioper is really to provide a framework, a com a core framework to handle, all many components many different components. And we access all all development around modularity. But I think we can talk a bit later about modularity because it is free. It is really the art of CardioPay. The core of CardioPay is, focused on modularity. And I think it's why, in this way, we are
[00:06:28] Unknown:
no more different from other projects. Yeah. 1 of the things that I was attracted to with Calliope is the fact that it does have that modularity and the ability to plug together different inputs and outputs, which seemed a little bit more straightforward and easier to reason about than some of the other projects I've looked at.
[00:06:46] Unknown:
Yeah. And it allow us to to add some some cool stuff, like, we can handle multiple languages because of it. We can make sure that, we use a TTS or STT, which are cloud free, which are self hosted or not. Or we can also add, different neurons and etcetera. So, yeah, the modularity is really what Caliope is really good at. And the design was we thought about the design earlier earlier when we started the project. And I think now it's what people are coming and the community is giving interest to the project, it's because of this modularity.
[00:07:33] Unknown:
And 1 of the other things that I thought was really interesting and noteworthy in the Calliope project is the fact that particularly given how it's a fairly new, entry into the market, it also supports a number of different languages, whereas the majority of the other assistants, particularly the open source ones, focus entirely on supporting English language input and output. So I'm wondering why was that an important feature from the outset, and what are some of the challenges and architecture that you've built around it that, enabled you to provide that capability?
[00:08:08] Unknown:
A little bit pretty long well. Yes. At the beginning, we just wanted to have something where we are free to select what we say to robots and, what's about to us. And, from that came, the. But it was not a more main target at the beginning, to be honest. It's just because we have selected this design, which or the language, but, it was not the main idea at the beginning. But, yes, in most case, with other projects, you need to say a particular sentence that is hardcoded into the scope. And the bot will answer another sentence uploaded to what we had in mind was something where Kariope provides information, source, audio, and you can compose elsewhere with it like you want. And so it's because of this design, we are.
So can you pay? Yes. Today, can you pay support the the multibillion language? Actually, is a TTS engine or the STT engine is supported best by CaluPay than your language is supported.
[00:09:10] Unknown:
Yeah. Yeah. You said everything. Because because of the design, we can handle, all languages that STT or TTS are providing. You can even imagine that, giving asking question to Calliope in English, and she can answer in French, for example. So, yeah, it's very, flexible. Yeah. You can it can adapt to, well, it can adapt to all users' need. So Yeah. Because the user has, has the opportunity to, to define orders, matching orders, and, to define answers. It makes it very flexible. But, the opposite the worst point is that the user has to spend a little bit of time to
[00:09:54] Unknown:
define all these orders and answers. I think that is. So, yes, to be honest, the the fact was we haven't planned to be multilingual at the beginning. Just, we wanted to be able to select sentence, as an input and to select what can you be with the answer to us. And so because of these facts, we are pretty much.
[00:10:15] Unknown:
And with any of the voice assistants, 1 of the major questions that always comes up is the idea of privacy because of the fact that you have this device in your home that is potentially always listening. And, so I'm just curious how Calliope works to mitigate some of the issues and and perceptions associated with having that device always listening and, to try and respect people's privacy, particularly given that it's an open source project. Yeah. Well, privacy
[00:10:42] Unknown:
privacy was first feature asked by the community. So magic rob trigger, called Snowboy. So so Snowboy engine was already offline. So we have just implemented some offline STT and TTS. So we have CMU SEX for the STT and because you're a for the TTS. And a user can also implement, their own engines as well and propose their propose them to the community if they want.
[00:11:10] Unknown:
Yeah. Again, it's because of the design of Kariope. If you want to use, if you want to use Google, if you want to use Bing, TTS or STT, you still can use it. Give a give a give a fuck to your privacy. But, in all case, we we because of the design and the modularity, you can choose to you can choose, STT or TTS you want. It makes it makes it flexible, makes it, yeah, private if you self hosted if you want. So yeah. It's not because of Kallopy.
[00:11:48] Unknown:
It's because of the design of Kariope, which makes a big a big difference. We provide by default, when you install Kariope, we provide some STG and DTS so you can select 1 of them. And, some of them are community community based, so you have to install them before using it. But, yes, in most case, users, use Google TTS, Google STDs. Sorry. How can you be by default?
[00:12:15] Unknown:
And for people who are using the local TTS and STT engines, what are some of the trade offs versus using the Google or Bing hosted ones? I think if you use,
[00:12:33] Unknown:
very well. Best place. That's really the case, if you work, in French or German or something like that. So I just recommend to try. And if you really want to to add something offline, yes, you have to choose to use that. That's that's really good if you are about the English.
[00:12:53] Unknown:
And you've mentioned, that the sort of core of Calliope is very modular, which enables you to have that pluggability of the input and outputs as well as the different capabilities and the triggered actions, which, in the parlance that I saw in the documentation are neurons, which will form synapses, which I think is very clever. So I'm wondering if you can just describe a bit about how Calliope is built, in order to be able to facilitate that modularity and what the plug in interfaces look like for somebody who wants to create, additional extensions to work with Calliope?
[00:13:26] Unknown:
There is no interface in Python. So we just provide, abstract, places. You have to implement this class in your main class for your own. So you have to respect the documentation and your 2 x, method, we provide. And so, yes, actually, with Python, we can load dynamically cases, and this is what we do with just when we read neurons and around the the Yammer description. We try to load the the exact exact claim name as a module in Python. So we just provide a path. And from this path, we try to run the the further, and, that's it. And everything work work like that. So when you write STT under, TTS or even a trigger, we just you just need to put it into the right folder and can you maybe try to load the modular promises for them.
[00:14:24] Unknown:
And for somebody who wants to get Calliope installed to start using it, what does that process look like? Of course, recognizing that it's gonna be a little different depending on the platform that you're deploying it to.
[00:14:34] Unknown:
Yes. It was the main target for KUPs, servers. Right? Of course. So for us, this platform is easy to inspect. You just need to, to close the image. Like, you do for other, Raspberry Pi image. If you want to install it manually, we provide the documentation, for the platform. So Debian and, Ubuntu. And, it's not really hard. You just need to follow the doc exactly exactly. That's it. And, also, after that, on certain KUVs instead, you can just follow the quick start documentation and download, what we call starter kits with, simple configurations for English or French or Italian. And then we can start and you perform this program. And you have a basic configuration, basic settings, and also a couple of, synapse in the rain to just understand the concepts, how how can you go wrong?
[00:15:32] Unknown:
What have been some of the most difficult or challenging aspects of building Calliope and the associated projects that go along with it as well as trying to raise awareness in the community for, you know, gaining contributions?
[00:15:45] Unknown:
Well, it's the most part, it's most difficult part for us is to make KUP user friendly. We know it's not the case, actually because we ask the user to configure it to the a YAML configuration, to the YAML, descriptor. So we don't have an interface like a certain project. It's not like Alexa. You just have to download the and then so your skill is ready. With KDP, it's not, everything ready out of the box. You need to work a little bit. And this is the the most important part of that for us. We will try to have something ready user friendly. Yeah. But in the same time, it's not what the community
[00:16:27] Unknown:
asked for. It's not user friendly. We know it, but we want to focus on the first hand on, on maturity of the soft and also on, on, the community needs. It was not the first request community had. So, basically, we we know we have to do it some more, some way, but it's not a priority in in the first place. We'll do it later, and maybe, adding an an an an interface over Kallio Pico to to be able to control and to build all the settings parameters and, of the brain with neurons and synapses. But it's not a priority even if it's we know it's it will be a big challenge in the future.
[00:17:13] Unknown:
And given the fact that it is so pluggable and there are decent number of extensions that have been contributed back by the community as well as being built into the core of Calliope. What are some of the most interesting or innovative uses of Calliope that you're aware of?
[00:17:28] Unknown:
Actually, we are not really close of the community about this part. We don't know much about, how user are using KDP. In most of case, we know that people use KDP like Google or Amazon assistance. We have a page on the website to receive a project and for an example made by the community. So we are waiting for new for stuff from the community. We'll chat, and, that's it. Yes. We don't know, yeah, what kind of use.
[00:17:54] Unknown:
Yeah. We have some users who who made some, who contribute with, making some neurons. So we can imagine that they are using them them at home, like some Uber neurons or, Google Maps neurons or things like that. So, I guess, they are using it at home. And once on the chat, someone said they they are also using Caliope to impress friends and girlfriends. So kind of fun.
[00:18:22] Unknown:
Are there any sort of, integrations or additional capabilities that you'd like to be able to build into Calliope but haven't been able to do so because of any sort of technical or just complexity reasons?
[00:18:34] Unknown:
Not really. Not really. Actually, in the next, release, we will integrate, MQTT protocol. So can you be we'll be able to discuss with, other IoT projects? But we haven't had technical program so far. Maybe you have something something in mind.
[00:18:52] Unknown:
We lack of time. That's the most the most problem we have. We have lot of ideas for the future, but at the moment, we we are mainly focused on making Cardiopay more mature implementing the community needs. We didn't face any big issue Or maybe no. We didn't face big issue. We just need more time to to make all we want to do. Yeah. More time to code.
[00:19:27] Unknown:
So now that you've been working on the project for a while, are you still happy with the choice of Python as the language of implementation? And if you were to start it all over again today, do you think you'd make the same decision?
[00:19:39] Unknown:
Yes. I think Python was a good choice. 1st, because, I personally had some experience some experiences before with Python because of a professional project. Secondly, because, Python is now a mature language with a lot of libraries like speech, recognition, wrapper, and, of course, because of the Raspberry Pi because of the Raspberry Pi. I'm not saying that it is, the only language available on this device, but, it's the most appreciated by the community. So, yes, I will start again with Python Client.
[00:20:13] Unknown:
Same for me. Are there any other aspects of the project that you'd like to talk about or any other topics that you think we should discuss?
[00:20:20] Unknown:
Yes. Maybe I can't discuss about the near future with a new version, which should be released as this, next weekend. And so that will bring some new capabilities like, so, like I said, mQTT senior and. So Kaguya will be able to interact with those IoT project, like, home assistant, support of community seniors with community, you know, like, community So the community can work on stuff like on stuffs that, like, communication protocol receiver, like for the lower or ZigBee. We have also added, memory. So KiloPay can store variables during the execution of our steps, in order to be used later via Amazon steps. Amazon's nice stuff. It's a neuro dynamic, so you can run deleted task. You can ask, can you pay to remind you to stop duty in 3 minutes? And, we made, an Android application that is already available, and we hope we can reuse it on Apple Store soon.
[00:21:26] Unknown:
Yeah. That was 1 of the things that I thought was, very interesting when I first came across the project is the fact that you do have the Android and the web applications available for being able to interface with Calliope so that even if you're not necessarily in the same room, you can still benefit from the capabilities of it for automation or just, performing various tasks.
[00:21:46] Unknown:
Yeah. Yeah. Yeah. So so application, you can already do that. We plan to add something like a community cell client server, architecture. So you can have 1 call can installed somewhere and a multiple client, in a musical room. Yeah. Multiple speaker all all over your home or your house.
[00:22:08] Unknown:
And so you can, yeah, talk to Kallopy everywhere in your home. So yeah. We also plan to add some some feature on the Android Android app, like, geo localization. And, so you could start karaoke from home depending the position you have on the map, things like that. So, yeah, we have a lot of ideas. You are 1 of the of the users. You said you tried Kalliopi. What what did you think about the project? And or did you try it? Did you do you still use it at home, for example, or something like that?
[00:22:48] Unknown:
Yeah. So I've started using it a little bit. So far, I've gotten it the point where it's installed. I'm using it as a way to introduce my 7 year old son to programming, trying to get him involved in, doing some Python development. So we worked together on getting it set up on a Raspberry Pi, which for the most part was fairly straightforward. There are a few things that I missed from the documentation, but I work as a systems administrator for my day job, so I was able to figure it out fairly readily. Once once I have a few spare moments, I'm going to go through and clean up the bash history so that I can just write out an install script for the, Raspberry and distribution. But the project itself is definitely interesting. I like the fact that it's modular and has all the has the ability to chain the different modules together.
And I also thought it was interesting that there's the I forget the specific plug in, but it allows you to chain the inputs and outputs of multiple neurons together so that you can potentially create some sort of a conversational interface for Calliope where if it, for instance, doesn't hear you correctly, it will ask you for clarification. Or if you, you know, ask a question and then it provides an output, you can then provide, an additional input, and it will be able to maintain the context so that you can sort of build up a chain of reactions.
[00:24:00] Unknown:
Reset network transmitter. Yeah. Yeah. You can link all neurons together. So all those things are definitely
[00:24:06] Unknown:
very interesting. Yeah. Cool. Yeah. Okay. Great. 7 years old. Sorry. You're calling. Right. Okay. I think, yes, so design efforts, every every day, actually, I'm currently referring to doing something to provide flexibility to the user. And what about, technology? We are a huge fan of, Ansible projects. I use this every day at work, and it's a concept of providing a framework where the user do not have to code but only describe what he wants. It is exactly what we wanted for KOB, an abstraction of, code execution. So we are maybe a little bit, did a copy of Ansible for that, and, it was a target. And so installation of of community stuff is based on a Ansible library. So a user made something like an error and create what we call, the t n a DNA file, and inside, it's a you have a description that is under the by, Constable. And after that, what can I say about the technology inside Kupi? That's it. We use a lot of library. It's not a big project about that. We don't cover stuff. We just, did a big wrapper of a lot of library.
[00:25:33] Unknown:
Yeah. It's 1 of the great things about the Python ecosystem is that because there are so many different uses of the Python language across a number of different industries, there are some so many different things you can pull from, and you just need to worry about writing the glue and the business logic to tie it tie it all together rather than being required to build everything from the ground up. Yes. Exactly. Exactly. So security for us
[00:25:55] Unknown:
now is to provide, something stable. And, I mean, like, the last time version broke everything, you know, like, in the base of so if you install it right now, it will not work because of a new version of libraries. Yeah. We need to work a bit more on consistency.
[00:26:14] Unknown:
But, maybe, yeah, we can talk a bit more about the community because we talked about, the design aspect of Kariope, but we also focused a lot on community, and, we wanted to to bring a a really big open source community. So that's why you can if you want to contribute to Kalliopay, you can still do it on your on your own GitHub project, for example, with your GitHub page. And you can still host all your code, and you can manage your own code, your neuron, your own sign on, if you want. Still, the community community the users will still be able to install it using, CardioPay with CardioPay install and, pointing the light the link of the GitHub project on GitHub.
So, yeah, we really focus on the on the community, and we want, yeah, people to, contribute and to increase because we really believe that, it is project. If, people are interested in, contributes. It can be, yeah, like, providing lot of ideas, providing lot of functionalities, interesting functionalities, and plugins, TTS, STT, everything. So, yeah, we're also really focused on the community, sharing, brain and neurons and everything. So yeah. And you can control and manage everything from the, Kallio PSCLI, drive everything, install in a row, and I'll remove it.
So, yeah, it's also We are really
[00:27:57] Unknown:
we are glad to have this community because I made I made a lot of, neurons and, STT wrapper or TTS wrapper. And so and they also test the new version before releases. So it's important for us. KDP project. I want to go on to a new user to go on the website and see, brain examples, see past examples, and, maybe call their home level to share it to the community.
[00:28:29] Unknown:
So for anybody who wants to follow what you guys are up to and get in touch and maybe ask questions about Calliope or any of the other projects you're involved in, then I'll have you add your preferred contact information into the show notes. And so with that, I'll move us to the picks. And this week, I'm gonna be picking the, Kiwi Crate service, which is a subscription service where every month they'll send you a box with a few different projects for your kids to be able to learn and explore different concepts. So, for instance, last month, I we got 1 where it was projects to, look at UV light and, you know, what it is and how you can use it to, you know, make things glow and look interesting and just sort of diving into that area. And then most recently, it was, secret agent kit where it was talking about different kinds of ciphers and being able to pass coded messages and things like that. So it was just, interesting way to get kids involved in learning new and interesting things, and it's presented well, and it's also fairly affordable. So it's a interesting thing to keep your kids learning on a monthly build a case. How can you build? Because,
[00:29:40] Unknown:
I to to build a case on Knew build because, actually, I, I don't use it in production at my home. So it's a secret. But, soon it would be fixed up because, I just bought, the I don't remember the name. It's, Ress, the reed speaker. It seems to be the best speaker you can buy, online right now, and I tried it. It's, it's a really good stuff. Can use it in a room when I add 20 meters of distance of branch. And so, yes, my next target is to have
[00:30:16] Unknown:
a real KPI production at my home. Alright. Do you have any picks for us this week, Thibault?
[00:30:22] Unknown:
We were talking with, Nicolas about refactoring, some part of Kalliopi, and we were very interested in, libraries. We have to still, dig in. But, we are really interested in reactive extension, managing reactive streams in Python. So I will recommend to have a look on it. Yeah. I think we will add it maybe maybe soon, maybe later in KOP.
[00:30:52] Unknown:
We have to to raise a new version of KEROP as we can.
[00:30:58] Unknown:
Yeah. Yeah. Reactive x is definitely an interesting framework, and it seems like it would be a really good fit for something like Calliope where you're trying to chain together a bunch of different neurons to, you know, perform a sequence of actions. I actually interviewed, 1 of the maintainers of that a while back on the show. So I'll add a link to that to the show notes as well.
[00:31:16] Unknown:
Yeah. Because now we are facing some issues with, managing asynchronous, stream with Kalliopi 1. We have to negate neurons, and we have to manage, neurotransmitters, etcetera.
[00:31:31] Unknown:
So, yeah, maybe we would have to think about it. I feel like your reactive stream could be a good solution to manage that. Great. Well, I appreciate the both of you taking time out of your day to join me and talk to me about the work you're doing with Calliope. It's definitely an interesting project and 1 that I plan to do some more tinkering with and, continue working on. So thank you both for your time and for your, efforts in building Calliope and turning it into what it is today. Thanks to you. Thanks to you, Tobias, for inviting us, and sorry for English. Not a problem. It was wonderful.
Hello, and welcome to podcast dot in it, the podcast about Python and the people who make it great. I would like to thank everyone who supports us on Patreon. Your contributions helped to make the show sustainable. When you're ready to launch your next project, you'll need somewhere to deploy it, so you should check out Linode at www.podcastinit.com/linode and get a $20 credit to try out their fast and reliable Linux virtual servers for running your next app. Do you need to learn more about how to scale your apps or learn new techniques for building them? Pluralsight has the training and mentoring you need to level up your skills. Go to www.podcastinit.com/pluralsite to start your free trial today.
You can visit the site to subscribe to the show, sign up for the newsletter, read the show notes, get in touch, and support the show. To help other people find the show, please leave a review on Itunes or Google Play Music, tell your friends and coworkers, and share it on social media. If you work with data for your job or want to learn more about how open source is powering the latest innovations in data science, then make your way to the Open Data Science Conference happening in London in October and San Francisco in November. Follow the links in the show notes to register. Your host as usual is Tobias Macy. And today, I'm interviewing Nicolas Marc and Thibault Buffet about Calliope, a modular always on voice controlled personal assistant designed for home automation.
So, Nicolas, could you start by introducing yourself?
[00:01:30] Unknown:
Sure. So I'm Nicolas, and I'm, IT engineer engineer and, also, And,
[00:01:38] Unknown:
that's it. And, Thibault, how about yourself? Hi, everyone.
[00:01:42] Unknown:
I'm Thibault Buffet. I'm also an IT engineer, mostly a software engineer. I'm used to develop in Java, but I know it today. We are talking about Python, but, my expertise is more on Java. And, yeah, I'm also a beta geek. We are developing Kalliopo since since last year. So yeah.
[00:02:05] Unknown:
And, Nikola, do you remember how you first got introduced to Python?
[00:02:09] Unknown:
I was a trainee in a IT company during my studies, and I started working on a project, with Jumbo Framework. And I remember same year, the first Raspberry Pi has been released, and so Python was supposed to be present, for this device. So I started calling from with Python.
[00:02:27] Unknown:
And, Thibault, how about yourself?
[00:02:29] Unknown:
To be honest, it was a long time ago I wanted to give it a try to Python. And I take the opportunity when we start playing around with Kalliopi, and Nicolas started to to introduce me to Python. And it was the best opportunity to give it a try, so I did it.
[00:02:48] Unknown:
So I gave a brief overview of what the Calliope project is, but I'm wondering if you guys can give a bit more detailed description of the project and how it got started and what inspired you to start working on it.
[00:03:01] Unknown:
Yes. So at the beginning, we felt, on, an article on the Internet that was talking about a little library called Snowboy. Snowboy, it's a is a smaller engine, which is always listening for an artwork, and, it run a script once the the world is recognized. So we directly source about, the journey's like program. We started looking around for existing project. We found, we found some interesting thing, but, for all of them, it was not exactly what we had in mind. So we started, good coding more more our project. And, actually, it was even not a a real project at the beginning, but just a smaller bank on scripts that answer your sentence when you say something. And to answer what CayoPay is, it's a simple framework to help you that can help you to build, your own personal assets like, Alexa or Google. Tivo, maybe you want to add something?
[00:04:03] Unknown:
Well, no. Basically, you did, everything. You talk about everything. But, yeah, at the beginning, it was, as Nicolas said, really, just a a way to have fun. And we were joking about it at a coffee time, in, at work. Finally, a little about step by step, it become more important, and some people try some people get interested by the project. A little community helped us also to test the project and, to, to continue and give ideas. So we continue playing around with the with this. Finally, we are here 1 year later.
[00:04:49] Unknown:
And as you mentioned, Calliope is operating in the similar spaces, the Amazon Alexa and Google Home, as well as some of the other open source projects such as Mycroft or Jasper. So I'm wondering if you can do a bit of comparison describe why somebody might want to use Calliope in place of any of those other projects.
[00:05:07] Unknown:
Well, to be honest, I don't know well other projects. But, from what we saw so far, k u p is more like a framework. Each user has to work a little bit before being able to use it. It's not really everything is done out of the box like Google Home or Alexa, but we ask yes. We ask the user to release this bot. In most case, in other project, we have seen, also only English is supported and, your orders are are coded. So for for 1 order, you can earn only 1 action. And with KU Pay, you can select the sentence and of your choice, and, you can attach 1 or more action to it. That's why we have created, KDP. It's to be free for for support.
[00:05:54] Unknown:
Yeah. Yeah. And you you said the most important, but, the IDB and Clarioper is really to provide a framework, a com a core framework to handle, all many components many different components. And we access all all development around modularity. But I think we can talk a bit later about modularity because it is free. It is really the art of CardioPay. The core of CardioPay is, focused on modularity. And I think it's why, in this way, we are
[00:06:28] Unknown:
no more different from other projects. Yeah. 1 of the things that I was attracted to with Calliope is the fact that it does have that modularity and the ability to plug together different inputs and outputs, which seemed a little bit more straightforward and easier to reason about than some of the other projects I've looked at.
[00:06:46] Unknown:
Yeah. And it allow us to to add some some cool stuff, like, we can handle multiple languages because of it. We can make sure that, we use a TTS or STT, which are cloud free, which are self hosted or not. Or we can also add, different neurons and etcetera. So, yeah, the modularity is really what Caliope is really good at. And the design was we thought about the design earlier earlier when we started the project. And I think now it's what people are coming and the community is giving interest to the project, it's because of this modularity.
[00:07:33] Unknown:
And 1 of the other things that I thought was really interesting and noteworthy in the Calliope project is the fact that particularly given how it's a fairly new, entry into the market, it also supports a number of different languages, whereas the majority of the other assistants, particularly the open source ones, focus entirely on supporting English language input and output. So I'm wondering why was that an important feature from the outset, and what are some of the challenges and architecture that you've built around it that, enabled you to provide that capability?
[00:08:08] Unknown:
A little bit pretty long well. Yes. At the beginning, we just wanted to have something where we are free to select what we say to robots and, what's about to us. And, from that came, the. But it was not a more main target at the beginning, to be honest. It's just because we have selected this design, which or the language, but, it was not the main idea at the beginning. But, yes, in most case, with other projects, you need to say a particular sentence that is hardcoded into the scope. And the bot will answer another sentence uploaded to what we had in mind was something where Kariope provides information, source, audio, and you can compose elsewhere with it like you want. And so it's because of this design, we are.
So can you pay? Yes. Today, can you pay support the the multibillion language? Actually, is a TTS engine or the STT engine is supported best by CaluPay than your language is supported.
[00:09:10] Unknown:
Yeah. Yeah. You said everything. Because because of the design, we can handle, all languages that STT or TTS are providing. You can even imagine that, giving asking question to Calliope in English, and she can answer in French, for example. So, yeah, it's very, flexible. Yeah. You can it can adapt to, well, it can adapt to all users' need. So Yeah. Because the user has, has the opportunity to, to define orders, matching orders, and, to define answers. It makes it very flexible. But, the opposite the worst point is that the user has to spend a little bit of time to
[00:09:54] Unknown:
define all these orders and answers. I think that is. So, yes, to be honest, the the fact was we haven't planned to be multilingual at the beginning. Just, we wanted to be able to select sentence, as an input and to select what can you be with the answer to us. And so because of these facts, we are pretty much.
[00:10:15] Unknown:
And with any of the voice assistants, 1 of the major questions that always comes up is the idea of privacy because of the fact that you have this device in your home that is potentially always listening. And, so I'm just curious how Calliope works to mitigate some of the issues and and perceptions associated with having that device always listening and, to try and respect people's privacy, particularly given that it's an open source project. Yeah. Well, privacy
[00:10:42] Unknown:
privacy was first feature asked by the community. So magic rob trigger, called Snowboy. So so Snowboy engine was already offline. So we have just implemented some offline STT and TTS. So we have CMU SEX for the STT and because you're a for the TTS. And a user can also implement, their own engines as well and propose their propose them to the community if they want.
[00:11:10] Unknown:
Yeah. Again, it's because of the design of Kariope. If you want to use, if you want to use Google, if you want to use Bing, TTS or STT, you still can use it. Give a give a give a fuck to your privacy. But, in all case, we we because of the design and the modularity, you can choose to you can choose, STT or TTS you want. It makes it makes it flexible, makes it, yeah, private if you self hosted if you want. So yeah. It's not because of Kallopy.
[00:11:48] Unknown:
It's because of the design of Kariope, which makes a big a big difference. We provide by default, when you install Kariope, we provide some STG and DTS so you can select 1 of them. And, some of them are community community based, so you have to install them before using it. But, yes, in most case, users, use Google TTS, Google STDs. Sorry. How can you be by default?
[00:12:15] Unknown:
And for people who are using the local TTS and STT engines, what are some of the trade offs versus using the Google or Bing hosted ones? I think if you use,
[00:12:33] Unknown:
very well. Best place. That's really the case, if you work, in French or German or something like that. So I just recommend to try. And if you really want to to add something offline, yes, you have to choose to use that. That's that's really good if you are about the English.
[00:12:53] Unknown:
And you've mentioned, that the sort of core of Calliope is very modular, which enables you to have that pluggability of the input and outputs as well as the different capabilities and the triggered actions, which, in the parlance that I saw in the documentation are neurons, which will form synapses, which I think is very clever. So I'm wondering if you can just describe a bit about how Calliope is built, in order to be able to facilitate that modularity and what the plug in interfaces look like for somebody who wants to create, additional extensions to work with Calliope?
[00:13:26] Unknown:
There is no interface in Python. So we just provide, abstract, places. You have to implement this class in your main class for your own. So you have to respect the documentation and your 2 x, method, we provide. And so, yes, actually, with Python, we can load dynamically cases, and this is what we do with just when we read neurons and around the the Yammer description. We try to load the the exact exact claim name as a module in Python. So we just provide a path. And from this path, we try to run the the further, and, that's it. And everything work work like that. So when you write STT under, TTS or even a trigger, we just you just need to put it into the right folder and can you maybe try to load the modular promises for them.
[00:14:24] Unknown:
And for somebody who wants to get Calliope installed to start using it, what does that process look like? Of course, recognizing that it's gonna be a little different depending on the platform that you're deploying it to.
[00:14:34] Unknown:
Yes. It was the main target for KUPs, servers. Right? Of course. So for us, this platform is easy to inspect. You just need to, to close the image. Like, you do for other, Raspberry Pi image. If you want to install it manually, we provide the documentation, for the platform. So Debian and, Ubuntu. And, it's not really hard. You just need to follow the doc exactly exactly. That's it. And, also, after that, on certain KUVs instead, you can just follow the quick start documentation and download, what we call starter kits with, simple configurations for English or French or Italian. And then we can start and you perform this program. And you have a basic configuration, basic settings, and also a couple of, synapse in the rain to just understand the concepts, how how can you go wrong?
[00:15:32] Unknown:
What have been some of the most difficult or challenging aspects of building Calliope and the associated projects that go along with it as well as trying to raise awareness in the community for, you know, gaining contributions?
[00:15:45] Unknown:
Well, it's the most part, it's most difficult part for us is to make KUP user friendly. We know it's not the case, actually because we ask the user to configure it to the a YAML configuration, to the YAML, descriptor. So we don't have an interface like a certain project. It's not like Alexa. You just have to download the and then so your skill is ready. With KDP, it's not, everything ready out of the box. You need to work a little bit. And this is the the most important part of that for us. We will try to have something ready user friendly. Yeah. But in the same time, it's not what the community
[00:16:27] Unknown:
asked for. It's not user friendly. We know it, but we want to focus on the first hand on, on maturity of the soft and also on, on, the community needs. It was not the first request community had. So, basically, we we know we have to do it some more, some way, but it's not a priority in in the first place. We'll do it later, and maybe, adding an an an an interface over Kallio Pico to to be able to control and to build all the settings parameters and, of the brain with neurons and synapses. But it's not a priority even if it's we know it's it will be a big challenge in the future.
[00:17:13] Unknown:
And given the fact that it is so pluggable and there are decent number of extensions that have been contributed back by the community as well as being built into the core of Calliope. What are some of the most interesting or innovative uses of Calliope that you're aware of?
[00:17:28] Unknown:
Actually, we are not really close of the community about this part. We don't know much about, how user are using KDP. In most of case, we know that people use KDP like Google or Amazon assistance. We have a page on the website to receive a project and for an example made by the community. So we are waiting for new for stuff from the community. We'll chat, and, that's it. Yes. We don't know, yeah, what kind of use.
[00:17:54] Unknown:
Yeah. We have some users who who made some, who contribute with, making some neurons. So we can imagine that they are using them them at home, like some Uber neurons or, Google Maps neurons or things like that. So, I guess, they are using it at home. And once on the chat, someone said they they are also using Caliope to impress friends and girlfriends. So kind of fun.
[00:18:22] Unknown:
Are there any sort of, integrations or additional capabilities that you'd like to be able to build into Calliope but haven't been able to do so because of any sort of technical or just complexity reasons?
[00:18:34] Unknown:
Not really. Not really. Actually, in the next, release, we will integrate, MQTT protocol. So can you be we'll be able to discuss with, other IoT projects? But we haven't had technical program so far. Maybe you have something something in mind.
[00:18:52] Unknown:
We lack of time. That's the most the most problem we have. We have lot of ideas for the future, but at the moment, we we are mainly focused on making Cardiopay more mature implementing the community needs. We didn't face any big issue Or maybe no. We didn't face big issue. We just need more time to to make all we want to do. Yeah. More time to code.
[00:19:27] Unknown:
So now that you've been working on the project for a while, are you still happy with the choice of Python as the language of implementation? And if you were to start it all over again today, do you think you'd make the same decision?
[00:19:39] Unknown:
Yes. I think Python was a good choice. 1st, because, I personally had some experience some experiences before with Python because of a professional project. Secondly, because, Python is now a mature language with a lot of libraries like speech, recognition, wrapper, and, of course, because of the Raspberry Pi because of the Raspberry Pi. I'm not saying that it is, the only language available on this device, but, it's the most appreciated by the community. So, yes, I will start again with Python Client.
[00:20:13] Unknown:
Same for me. Are there any other aspects of the project that you'd like to talk about or any other topics that you think we should discuss?
[00:20:20] Unknown:
Yes. Maybe I can't discuss about the near future with a new version, which should be released as this, next weekend. And so that will bring some new capabilities like, so, like I said, mQTT senior and. So Kaguya will be able to interact with those IoT project, like, home assistant, support of community seniors with community, you know, like, community So the community can work on stuff like on stuffs that, like, communication protocol receiver, like for the lower or ZigBee. We have also added, memory. So KiloPay can store variables during the execution of our steps, in order to be used later via Amazon steps. Amazon's nice stuff. It's a neuro dynamic, so you can run deleted task. You can ask, can you pay to remind you to stop duty in 3 minutes? And, we made, an Android application that is already available, and we hope we can reuse it on Apple Store soon.
[00:21:26] Unknown:
Yeah. That was 1 of the things that I thought was, very interesting when I first came across the project is the fact that you do have the Android and the web applications available for being able to interface with Calliope so that even if you're not necessarily in the same room, you can still benefit from the capabilities of it for automation or just, performing various tasks.
[00:21:46] Unknown:
Yeah. Yeah. Yeah. So so application, you can already do that. We plan to add something like a community cell client server, architecture. So you can have 1 call can installed somewhere and a multiple client, in a musical room. Yeah. Multiple speaker all all over your home or your house.
[00:22:08] Unknown:
And so you can, yeah, talk to Kallopy everywhere in your home. So yeah. We also plan to add some some feature on the Android Android app, like, geo localization. And, so you could start karaoke from home depending the position you have on the map, things like that. So, yeah, we have a lot of ideas. You are 1 of the of the users. You said you tried Kalliopi. What what did you think about the project? And or did you try it? Did you do you still use it at home, for example, or something like that?
[00:22:48] Unknown:
Yeah. So I've started using it a little bit. So far, I've gotten it the point where it's installed. I'm using it as a way to introduce my 7 year old son to programming, trying to get him involved in, doing some Python development. So we worked together on getting it set up on a Raspberry Pi, which for the most part was fairly straightforward. There are a few things that I missed from the documentation, but I work as a systems administrator for my day job, so I was able to figure it out fairly readily. Once once I have a few spare moments, I'm going to go through and clean up the bash history so that I can just write out an install script for the, Raspberry and distribution. But the project itself is definitely interesting. I like the fact that it's modular and has all the has the ability to chain the different modules together.
And I also thought it was interesting that there's the I forget the specific plug in, but it allows you to chain the inputs and outputs of multiple neurons together so that you can potentially create some sort of a conversational interface for Calliope where if it, for instance, doesn't hear you correctly, it will ask you for clarification. Or if you, you know, ask a question and then it provides an output, you can then provide, an additional input, and it will be able to maintain the context so that you can sort of build up a chain of reactions.
[00:24:00] Unknown:
Reset network transmitter. Yeah. Yeah. You can link all neurons together. So all those things are definitely
[00:24:06] Unknown:
very interesting. Yeah. Cool. Yeah. Okay. Great. 7 years old. Sorry. You're calling. Right. Okay. I think, yes, so design efforts, every every day, actually, I'm currently referring to doing something to provide flexibility to the user. And what about, technology? We are a huge fan of, Ansible projects. I use this every day at work, and it's a concept of providing a framework where the user do not have to code but only describe what he wants. It is exactly what we wanted for KOB, an abstraction of, code execution. So we are maybe a little bit, did a copy of Ansible for that, and, it was a target. And so installation of of community stuff is based on a Ansible library. So a user made something like an error and create what we call, the t n a DNA file, and inside, it's a you have a description that is under the by, Constable. And after that, what can I say about the technology inside Kupi? That's it. We use a lot of library. It's not a big project about that. We don't cover stuff. We just, did a big wrapper of a lot of library.
[00:25:33] Unknown:
Yeah. It's 1 of the great things about the Python ecosystem is that because there are so many different uses of the Python language across a number of different industries, there are some so many different things you can pull from, and you just need to worry about writing the glue and the business logic to tie it tie it all together rather than being required to build everything from the ground up. Yes. Exactly. Exactly. So security for us
[00:25:55] Unknown:
now is to provide, something stable. And, I mean, like, the last time version broke everything, you know, like, in the base of so if you install it right now, it will not work because of a new version of libraries. Yeah. We need to work a bit more on consistency.
[00:26:14] Unknown:
But, maybe, yeah, we can talk a bit more about the community because we talked about, the design aspect of Kariope, but we also focused a lot on community, and, we wanted to to bring a a really big open source community. So that's why you can if you want to contribute to Kalliopay, you can still do it on your on your own GitHub project, for example, with your GitHub page. And you can still host all your code, and you can manage your own code, your neuron, your own sign on, if you want. Still, the community community the users will still be able to install it using, CardioPay with CardioPay install and, pointing the light the link of the GitHub project on GitHub.
So, yeah, we really focus on the on the community, and we want, yeah, people to, contribute and to increase because we really believe that, it is project. If, people are interested in, contributes. It can be, yeah, like, providing lot of ideas, providing lot of functionalities, interesting functionalities, and plugins, TTS, STT, everything. So, yeah, we're also really focused on the community, sharing, brain and neurons and everything. So yeah. And you can control and manage everything from the, Kallio PSCLI, drive everything, install in a row, and I'll remove it.
So, yeah, it's also We are really
[00:27:57] Unknown:
we are glad to have this community because I made I made a lot of, neurons and, STT wrapper or TTS wrapper. And so and they also test the new version before releases. So it's important for us. KDP project. I want to go on to a new user to go on the website and see, brain examples, see past examples, and, maybe call their home level to share it to the community.
[00:28:29] Unknown:
So for anybody who wants to follow what you guys are up to and get in touch and maybe ask questions about Calliope or any of the other projects you're involved in, then I'll have you add your preferred contact information into the show notes. And so with that, I'll move us to the picks. And this week, I'm gonna be picking the, Kiwi Crate service, which is a subscription service where every month they'll send you a box with a few different projects for your kids to be able to learn and explore different concepts. So, for instance, last month, I we got 1 where it was projects to, look at UV light and, you know, what it is and how you can use it to, you know, make things glow and look interesting and just sort of diving into that area. And then most recently, it was, secret agent kit where it was talking about different kinds of ciphers and being able to pass coded messages and things like that. So it was just, interesting way to get kids involved in learning new and interesting things, and it's presented well, and it's also fairly affordable. So it's a interesting thing to keep your kids learning on a monthly build a case. How can you build? Because,
[00:29:40] Unknown:
I to to build a case on Knew build because, actually, I, I don't use it in production at my home. So it's a secret. But, soon it would be fixed up because, I just bought, the I don't remember the name. It's, Ress, the reed speaker. It seems to be the best speaker you can buy, online right now, and I tried it. It's, it's a really good stuff. Can use it in a room when I add 20 meters of distance of branch. And so, yes, my next target is to have
[00:30:16] Unknown:
a real KPI production at my home. Alright. Do you have any picks for us this week, Thibault?
[00:30:22] Unknown:
We were talking with, Nicolas about refactoring, some part of Kalliopi, and we were very interested in, libraries. We have to still, dig in. But, we are really interested in reactive extension, managing reactive streams in Python. So I will recommend to have a look on it. Yeah. I think we will add it maybe maybe soon, maybe later in KOP.
[00:30:52] Unknown:
We have to to raise a new version of KEROP as we can.
[00:30:58] Unknown:
Yeah. Yeah. Reactive x is definitely an interesting framework, and it seems like it would be a really good fit for something like Calliope where you're trying to chain together a bunch of different neurons to, you know, perform a sequence of actions. I actually interviewed, 1 of the maintainers of that a while back on the show. So I'll add a link to that to the show notes as well.
[00:31:16] Unknown:
Yeah. Because now we are facing some issues with, managing asynchronous, stream with Kalliopi 1. We have to negate neurons, and we have to manage, neurotransmitters, etcetera.
[00:31:31] Unknown:
So, yeah, maybe we would have to think about it. I feel like your reactive stream could be a good solution to manage that. Great. Well, I appreciate the both of you taking time out of your day to join me and talk to me about the work you're doing with Calliope. It's definitely an interesting project and 1 that I plan to do some more tinkering with and, continue working on. So thank you both for your time and for your, efforts in building Calliope and turning it into what it is today. Thanks to you. Thanks to you, Tobias, for inviting us, and sorry for English. Not a problem. It was wonderful.
Introduction to Calliope and Guests
Nicolas and Thibault's Backgrounds
Genesis of the Calliope Project
Comparison with Other Voice Assistants
Multilingual Support in Calliope
Privacy and Offline Capabilities
Modularity and Plugin Interfaces
Challenges and Community Contributions
Future Integrations and Features
Reflections on Using Python
Upcoming Features and Releases
Community Focus and Contributions
Closing Remarks and Future Plans
