Summary
Software engineers are frequently faced with problems that have been fixed by other developers in different projects. The challenge is how and when to surface that information in a way that increases their efficiency and avoids wasted effort. DeepCode is an automated code review platform that was built to solve this problem by training a model on a massive array of open sourced code and the history of their bug and security fixes. In this episode their CEO Boris Paskalev explains how the company got started, how they build and maintain the models that provide suggestions for improving your code changes, and how it integrates into your workflow.
Announcements
- Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
- When you’re ready to launch your next app or want to try a project you hear about on the show, you’ll need somewhere to deploy it, so take a look at our friends over at Linode. With 200 Gbit/s private networking, scalable shared block storage, node balancers, and a 40 Gbit/s public network, all controlled by a brand new API you’ve got everything you need to scale up. And for your tasks that need fast computation, such as training machine learning models, they just launched dedicated CPU instances. Go to pythonpodcast.com/linode to get a $20 credit and launch a new server in under a minute. And don’t forget to thank them for their continued support of this show!
- You listen to this show to learn and stay up to date with the ways that Python is being used, including the latest in machine learning and data analysis. For even more opportunities to meet, listen, and learn from your peers you don’t want to miss out on this year’s conference season. We have partnered with organizations such as O’Reilly Media, Dataversity, Corinium Global Intelligence, and Data Council. Upcoming events include the O’Reilly AI conference, the Strata Data conference, the combined events of the Data Architecture Summit and Graphorum, and Data Council in Barcelona. Go to pythonpodcast.com/conferences to learn more about these and other events, and take advantage of our partner discounts to save money when you register today.
- Your host as usual is Tobias Macey and today I’m interviewing Boris Paskalev about DeepCode, an automated code review platform for detecting security vulnerabilities in your projects
Interview
- Introductions
- Can you start by explaining what DeepCode is and the story of how it got started?
- How is the DeepCode platform implemented?
- What are the current languages that you support and what was your guiding principle in selecting them?
- What languages are you targeting next?
- What is involved in maintaining support for languages as they release new versions with new features?
- How do you ensure that the recommendations that you are making are not using languages features that are not available in the runtimes that a given project is using?
- For someone who is using DeepCode, how does it fit into their workflow?
- Can you explain the process that you use for training your models?
- How do you curate and prepare the project sources that you use to power your models?
- How much domain expertise is necessary to identify the faults that you are trying to detect?
- What types of labelling do you perform to ensure that the resulting models are focusing on the proper aspects of the source repositories?
- How do you curate and prepare the project sources that you use to power your models?
- How do you guard against false positives and false negatives in your analysis and recommendations?
- Does the code that you are analyzing and the resulting fixes act as a feedback mechanism for a reinforcement learning system to update your models?
- How do you guard against leaking intellectual property of your scanned code when surfacing recommendations?
- What have been some of the most interesting/unexpected/challenging aspects of building the DeepCode product?
- What do you have planned for the future of the platform and business?
Keep In Touch
Picks
- Tobias
- Redwall Series by Brian Jacques
- Boris
- Artifical Intelligence
- Get outside
Closing Announcements
- Thank you for listening! Don’t forget to check out our other show, the Data Engineering Podcast for the latest on modern data management.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@podcastinit.com) with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers
- Join the community in the new Zulip chat workspace at pythonpodcast.com/chat
Links
The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA
Hello, and welcome to podcast.init, the podcast about Python and the people who make it great. When you're ready to launch your next app or want to try a project you hear about on the show, you'll need somewhere to deploy it. So take a look at our friends over at Linode. With 200 gigabit private networking, scalable shared block storage, node balancers, and a 40 gigabit public network, all controlled by a brand new API, you've got everything you need to scale up. And for your tasks that need fast computations, such as training machine learning models, they just launched dedicated CPU instances. Go to python podcast.com/linode, that's l I n o d e, today to get a $20 credit and launch a new server in under a minute. And don't forget to thank them for their continued support of this show.
And you listen to this show to learn and stay up to date with the ways that Python is being used, including the latest in machine learning and data analysis. For even more opportunities to meet, listen, and learn from your peers, you don't want to miss out on this year's conference season. We have partnered with organizations such as O'Reilly Media, Dataversity, Corinium Global Intelligence, and Data Council. Upcoming events include the O'Reilly AI Conference, the Strata Data Conference, the combined events of the Data Architecture Summit in Graphorum, and Data Council in Barcelona. Go to python podcast.com/conferences today to learn more about these and other events and take advantage of our partner discounts to save money when you register.
Your host as usual is Tobias Macy, and today I'm interviewing Boris Paskolev about Deep Code, an automated code review platform for detecting security vulnerabilities in your projects. So, Boris, can you start by introducing yourself?
[00:01:48] Unknown:
Hi. My name is Boris Paskliff. I'm the CEO and, cofounder of Deep Code. We're currently based in Zurich, Switzerland.
[00:01:56] Unknown:
And so can you start up explaining about what the Deep Code project is and some of the story of how it got started?
[00:02:02] Unknown:
So, ultimately, what Deep Code does is learns from the global development community every single issue that was ever fixed and how it was fixed, and combine this knowledge of all development, almost like crowdsourcing the development knowledge, to prevent every single user from repeating those, mistakes that are already known. In addition, we actually have predictive algorithms to understand issues that may not have been fixed but could actually appear in software development. And where we started? So, ultimately, the start the idea started by, the other 2 cofounders. They actually spent more than 6 years in researching the space of program analysis and learning from big quote, which means, like, billions of lines of quote that they're available out there. And they did that in ETH Zurich, which is what we call the MIT of Europe, and they're 1 of the foremost experts in the world in that space. They have hundreds of publication in the space. And, yeah, and when they finished the research, our CTO, he published his PhD. It was super well awarded, and, well, we decided that it totally makes sense, to actually build it into a platform and revolutionize how our software development works.
[00:03:10] Unknown:
And was there any particular reason for focusing specifically on security defects in code and how to automatically resolve or detect them?
[00:03:20] Unknown:
Actually, security was a later add on add on. We actually did that in, 2019. It was just started this year, and we published a specific paper on that. The platform itself is not targeting or anything. Specifically, it can any issues that's fixed, being that a bug performance, you name it, can be detected. Security was just a a nice add on feature that, that that we did, and it's, was pretty novel as well.
[00:03:44] Unknown:
So in terms of the platform itself, can you talk through a bit about how it's implemented and the overall architecture for the actual platform and how it interacts with the user's code base?
[00:03:55] Unknown:
So pretty much what it does is there's 2 steps both in the learning and analyzing code. The first step is, we we we take your code. We analyze it quickly. We convert it. We use standard parsing of each language, And then we actually, do a data log extraction of semantic facts about the codes. We built a customized internal representation about the various interaction, every single object, how the object propagates and interacts with functions, getting into other objects, then how they change, etcetera. And and and and this knowledge represents pretty much the intent and how the program functions. Right? And then we do that for every single version of the program. So we see over time when people commit code and change code, how that changes.
And that gives us, the delta, what is changing and how people are fixing things. Right? Then we do that extremely fast, and we do that over this, hundreds of thousands of repositories, obviously, and, like, billions of lines of code. And then we actually, identify trends. This is where our machine learning kicks in, and it identifies trends, how and, how people fix things, what is the most common things, are there specific weird cases, etcetera. And this is where the this is how we build the global knowledge, as we call it.
[00:05:03] Unknown:
For the languages that you're currently supporting, I noticed that you're focusing, at least for the time being, on Python and JavaScript. And I believe there are 1 or 2 others. And I'm wondering what your criteria was for selecting the languages that you were targeting for evaluation
[00:05:19] Unknown:
and automated fixing and just some of the other languages that you're thinking about targeting next? Yep. So pretty much we started with the the most popular languages out there. I mean, there's different charts, but kind of the, the standard suspects are obviously, Python, Java, JavaScript. Then following down that line, we're looking into obviously c sharp, PHP will come, c and c plus plus. And down the list, I mean, we're getting more and more requests for various languages. So it's a combination of the ranking of the language and popularity as well as specific, customer requests, specifically, big companies if they're asking for very very specific.
[00:05:53] Unknown:
Given the dynamic nature of things like Python and Java script, I'm wondering what some of the difficulties you faced are as far as being able to statically analyze the languages and avoid any cases where there might be things like monkey patching going on or maybe some sort of code obfuscation?
[00:06:13] Unknown:
Yeah. I mean, so, since we we don't do the the typical static analysis here, we're doing actually a a a static semantic analysis, and we do that in context. Particular object and then you put it into an array and then the object comes out, we still know that it's the exact same object. So that kind of gets us closer to a dynamic analysis, as well. So that's kind of some of the the the the features that allow us to stopically analyze and identify much more complex issues that are that are close to dynamic interprocedural analysis, if you say. And and this allows you to get much much higher accuracy. You don't have the false positives that are tools will, will throw you there as well identify issues that, classical syntax static analysis will not be able to see at all.
[00:07:02] Unknown:
Another thing that can potentially complicate that matter is the idea of 3rd party dependencies and how that introduces code into the overall run time. And I'm wondering how you approach that as well, particularly as those dependencies are updated and evolved.
[00:07:17] Unknown:
Pretty much for dependencies, we scan the dependencies code if if the code is included in your repository. We don't have what the there are many other services out there that have a list of dependencies in their version version saying which 1 might be having issues or not. We don't do that because that's pretty much static databases that, that do that. But we do look at how they actually call specific APIs. So if you're actually calling, you have a dependency and you're calling some kind of a function, from it, we actually gonna identify how you're calling the function, telling you, hey. You're not calling the function the right way, or the 3rd parameter that you're passing is not the the right 1, etcetera, etcetera. But, specifically, what dependencies you incorporate, we don't actually look at it. I mean, we can tell you you're importing things more than once or you're importing something, you know, if you're not using it. Things like this, we can have, as well. And that's kind of the scope that we go into.
[00:08:08] Unknown:
Another thing that introduces complexities is as languages themselves evolve and introduce new capabilities or keywords. And I'm wondering how you keep up with those release cycles and ensure that your analyzers and your recommendation engines are staying up to date with those capabilities. And then also on the other side, ensuring that any recommendations that you provide in your code reviews match the target runtime for the code base as it stands. So for instance, if somebody wrote a Python project that's actually using Python 2, then you don't end up suggesting fixes that rely on Python 3
[00:08:44] Unknown:
features? So so on the first 1, when the language has changed and evolved, which is, again, pretty common these days. So there there's 2 things. Right? First of all is, are the parsers supporting the new feature? Right? Because there's we have to get the latest version of the parsers. If the parsers are supporting it, that's great. If the parsers are not supporting it, then we have to do our own extensions until the parsers start supporting it. Because we pretty much use standard parsers with minimum extensions only when needed. Right? So this is the standard feature. If there's something fundamentally different about the language, right, this is where we might actually, have to extend our internal representation to support that. But, like, like, it's something, like, really fundamental. But that's the rarely, we rarely see that in specific languages. That's more happening if you had a new language. Right?
So that's kind of the 2, the 2 major branches, when new things comes in. But for the majority of things, there's very little that we have to do, but, extending the to the latest parser. And the second question that you ask is kind of the Python versions, Python version 2 versus 3. So we don't specifically differentiate that, but we wanna give you suggestions, we'll tell you, hey. For Python version 3, we have to be doing this. Right? If you're doing a part if you are in Python version 2, obviously, you can just say ignore these suggestions. And you can actually create set of rules and saying, okay. This is all the set of rules that are Python version 3. Just ignore them. You can put that into config file. And until you move to Python version 3, you can just ignore all version 3 rules.
[00:10:08] Unknown:
And there it also gets a little bit more difficult for within Python 3 versions. For instance, if your code is targeting Python 3.5, you don't wanna, suggest fixes that incorporates things such as f strings or data classes. And I'm curious how you approach that as well or if it's more just based on, what the user specifies in their config as far as the runtime that they're using.
[00:10:31] Unknown:
That is correct. So we don't have any very strong changes in that space. The place that helps in that is we actually all the suggestions we provide are contextually based. So, usually, you can actually see what's happening before and after specific issues. Right? And if they are version specific, then you actually will not get the the recommendation because it looks different in your case. That doesn't cover all the cases, obviously. I think you're rightfully asking that questions, and we don't have a great solution for that. We leave it to the developer to actually when they see a suggestion, say, nope. I don't care about that. Clearly, as I said, we can do it through the ignoring rules, but those changes are rare. I mean, they do happen, and we've seen cases where the developer said, yep. I don't care about this yet. I haven't updated. And that happens. But we usually target most of our suggestions and learning since it's automated. It gets the learning from the latest version. So if a large percentage of development community is moving to the latest version, then they're making changes related to it, and you'll be getting suggestions for that as well. Can you describe a bit more about the overall workflow for somebody who's using Deepgram and how it fits into their development process?
Yeah. So the most standard 1 that we we envision and we see it's most popular out there, it's a it's a developer tool that lives on the git. So pretty much you log in with your Git account, Git for Bitbucket, whatever that is. You see a list of repositories that you wanna analyze, and you subscribe them. Right? 1 once the repository is subscribed, you're getting 2 things. First of all, every time you do a pull request, we actually analyze it and we tell you, during this diff, are you introducing any new issues? Right? So that's number 1. That's continually monitoring the new code being generated.
The second piece is, continuously monitoring on your old code because old code could all also age. Right? As the development community changes, new security vulnerabilities are uncovered, etcetera, something that you've written 2 years ago might actually not secure anymore, and you actually wanna get pinged for that because very few people actually test or look into their call from 2 years ago. So that will give you a ping as well saying, hey. Yeah. This function here, the call has to be updated to a new encryption, for example, to make sure it's secure. So those are the 2 major pieces. Again, fully lives into the git. And in addition to that, we obviously offer an API and a command line interface so you can really integrate our solution anywhere you want. It could be as part of continuous integration. We actually have that in GitHub already that, if you finish the pull request before the merge, it can tell you, hey. We analyzed it. There's no critical stuff. Please proceed. Or, hey. Here's 1 critical stuff, critical issue. Look at it.
But, yeah, the API and command line interface allows you to build, like, script within minutes, a checker at any point on your workflow because developers and different companies or setups have very different development workflows, and they might want it at different stages. If you're having a QA team, if having continuous integration, if you're having a continuous, delivery versus, individual builds every day or month, whatever that is.
[00:13:23] Unknown:
And then in terms of the model itself, can you describe a bit about the overall process that you're using for training and some of the inputs that you use as far as curating the projects that you're using as references to ensure that they're of sufficient quality and that you're not relying on something that is maybe using some nonstandard conventions?
[00:13:44] Unknown:
Yep. So 2 points on this. So we do have a custom curation. It takes a lot of different things, how active the project is, how many contributors, how many stars, etcetera, etcetera. So that's that's continuously updating. As and this is mainly done because there's a lot of tail objects, in the Gits, like, that haven't been touched for, like, 2 years or have 1 developer only that never touches it. So that there's kind of a long tail of such projects. So we just don't wanna waste time to analyze them. The machine learning automatically actually seeds out such a kind of a poison pills, in a way. Kind of random developer have fixed something in a wrong way. Right? And this is where it comes in with the probability that we the probability we assign to every single suggestions that we have, which is based on how many people have fixed it this way. Is there a trend a lot of people fixing it? How many counterexamples there are, and how many actual such issue exists in the open source community today? Right? So based on that, we can automatically see issues. Because when you fix something wrongly, it's very unlikely that there's many people that can fix it the same wrong way. That only happens, for example, if somebody publishes a wrong solution in core and nobody catches it. And that can happen for, like, 1, 2 weeks, but usually that gets resolved immediately and then our knowledge base will automatically update.
[00:14:59] Unknown:
In terms of the amount of domain expertise that's necessary for identifying those falsely trying to detect, I'm curious if you're using, sort of expert labeling techniques where you have somebody going through and identifying the faults that were created and the associated fixes, or if you're relying on more of an unsupervised learning model for being able to build the intelligence into your engine?
[00:15:23] Unknown:
Yeah. So it's mainly, unsupervised learning. We actually do have, some labeling which is based on how severe the issue is. So we have categorization of critical warnings and info type of suggestions. So pretty much, we have to actually categorize which ones are critical. And this is when our team does that, but that's on a type of issue. So, like, within 2 hours, you can label, like, hundreds of thousands of different suggestions. So it's a it's a pretty quick process. So it's very minimal supervision that we have to do. Everything else is pretty much fully automatic. What we do, we automatically detect the type of issue. Is it the security? Is it a bug? Is it performance, etcetera? We use a number of techniques there. We have an NLP on the commands, obviously, looking at the specific call and semantically what it does, because what we do, we have a predictive algorithm that infers the usage of specific functions and objects. So we actually know what they do and what setting they're being used.
[00:16:20] Unknown:
And you mentioned too that for the pull request cases, you're relying on parsing the diffs of what's being changed. And I'm curious if there are cases where the diff just doesn't provide enough context about the overall intent of the code and any approaches that you have for being able to mitigate some potential false positives or false negatives where you missed something because of the fact that the code is only changing maybe 1 line, but you need the broader context to understand what's being fixed.
[00:16:51] Unknown:
Okay. So so yeah. Sorry. So maybe I didn't clarify that correctly. So we do analyze the whole, the whole tree. Like, we always do the full analysis. Right? The but usually, the the semantic the semantic changes are only within the diff, and we actually show you what it is. So if a change that you make on this line of code is causing a security issue somewhere else, we'll actually catch that. Absolutely. I mean, we cannot analyze anything small than that because our internal representation requires the context of what's happening. So we have to analyze every single function procedure to see what it is. So we will analyze everything, but usually the changes are happening only in the diffs because they are focusing there. But it could be in a different part of the code base, as well, where the the issue comes from.
In terms of, you mentioned false positives and false negatives again. So there's a number of techniques to lower that. I mean, we have kind of a record high accuracy rate compared to any of the existing tools today. And that's mainly based on the contextual analysis. So we actually know in what cases the problem is there and on the fact that we actually have kind of usually thousands of thousands of examples, so it's pretty accurate what it is. And we're not doing a syntax based comparison. We're doing a semantic comparison. So we're not looking as what you're doing in the specific lines of codes because without knowing the semantic details about it, you actually could be very wrong. But looking at semantically gives you the considerably high accuracy rate.
[00:18:13] Unknown:
And in terms of identifying those false positives and false negatives, if you do identify maybe a false positive, and is there any way for the users to be able to label it as such so that it can get fed back into your machine learning model so that you can prevent that from happening in the future and just any other sort of feedback mechanisms that are built in for users to be able to feed that back into your model to improve it over time?
[00:18:38] Unknown:
Yep. So we have 2 ways. First of all, is you can ignore rules for your own. You can say, it's, hey. This rule, I don't like, and you can decide if you wanna do this for a project or in general. And the second 1 and the second is you can actually have a kind of a thumbs up and thumbs down with a command saying, yep. I don't like this because blah. Right? So this is the the 2 main mechanisms that that we look at it is clearly for open source. We actually get the feedback automatically if an issue was fixed or not. Right? And that, as I said earlier, we'll look at how many of the issues exist, in in the code base out there and how many of these type of issues have been fixed, which is part of our, probability assessment if an issue is, should be should be flagged or not.
[00:19:19] Unknown:
And in terms of the code that you're analyzing, I'm wondering, again, how that feeds back into your models, particularly in the case where somebody might be scanning a private repository and if there are any sort of, intellectual property in terms of algorithms or anything along those lines and preventing that from getting fed back into your model so that it gets surfaced as a recommendation on somebody else's project?
[00:19:43] Unknown:
Yep. So we do not learn from private code, At least that do not become part of the public knowledge. Right? We have a special function that, you can learn from your private code and that becomes your own knowledge. That's usually is for larger companies with larger code bases. If you when we analyze code, we don't learn from that code. Right? We only learn from the open source repositories. And depending on the licensing, there's some, open source repositories that you can see but you cannot use. Right? So for those, we're not gonna ever create the suggestions, suggestion examples coming from there. We'll still count them as how many times we've seen that issue and or that it should be in fixed, but we never show it as an example for fixing. The examples will only come from a fully open source,
[00:20:27] Unknown:
projects. And in terms of the overall challenges or anything that was particularly interesting or unexpected that you've come across in the process of building and growing the Deep Code project and the business around it, what has been something that was sort of notable in your experience?
[00:20:46] Unknown:
Wow. That's a interesting question. I think the 1 that it's, more shocking shocking is the number of different technology innovations that we have to do. Like, I mean, we create new versions of the platform a lot. Like, we're actually literally about to release a new 1. In a matter of weeks, we released it to some pilot customers already that considerably increases the coverage, while maintaining the same high accuracy. So but yeah. So it's really like we have to come up with new things all the time. I mean, we have half of our team is focusing on really inventing new stuff. We do publish about half of them, because those, those are pretty interesting findings.
Some of them we keep internally because, obviously, they're a bit, proprietary, and over time, they come out, obviously. So yeah. So it's really the sheer volume of new things that you have to build. Like, there's so many modules when our CTO starts drawing the whole architecture. Like, it's takes hours and it's a bunch of small boxes. And each 1 in its own, it's kind of a different innovation that came up, and that's, that's really interesting. And I was not expecting that, and I was not expecting that 2 years ago when I started looking into it. And when I look at it today, we're still doing a lot of that. And when I look at the roadmap, a lot of new things are coming in that space as well. So that is, yeah, quite interesting, and explains why they have never been a platform so far that really goes deep into understanding code in that way and then being able to learn from such a large set of big code out there in a extremely fast way.
[00:22:13] Unknown:
In terms of the platform itself and its capabilities, what are the some of the overall limitations and some of the cases where you might not want to either use it or avoid some of the recommendations that it might come out with just because of some of the artifacts of the code that you're trying to feed through it?
[00:22:30] Unknown:
Fair question. So no limitations in general. It's fully scalable, can support any language. That's the the the best piece of the architecture. Specific carriers that you don't wanna use it, we haven't found 1 yet. I mean, ultimately, that's part of the basic building blocks. Maybe when we start delivering some more more higher level architectural analysis, Some of those spaces might come up, but, that's due to come. But from the basic building blocks, finding bugs and issues in your code, yeah, there's a we haven't find any specific areas where they are. I mean, some projects might have a little bit higher false positive rate versus another for specific reasons, as you mentioned, the Python version, for example, using Python version 2, and we're giving you a lot of Python version 3 suggestions.
But out on the day, there's no industry or language or focus specific.
[00:23:17] Unknown:
And another thing that is a potential challenge are cases where the code base itself is quite large. I'm wondering if run into any issues where you've hit an upper limit in terms of just your deployed platform for being able to parse and keep the entirety of that structure semantically in mem you know, in the working set, and any strategies that you've developed to be able to work around that?
[00:23:41] Unknown:
The platform as designed can literally handle anything. And, yeah, millions of lines of code in seconds. So, I mean, think about it. We are learning from billions of lines of code. And in order to do that efficiently, we've built some pretty efficient algorithms to actually do that. And so we haven't seen like, and we finalize some pretty big code bases, any issues. Like, in usually, we're like, wow. So we are on average when I compare it to other tools, 10 to oftentimes 100 times faster, in the analysis space. So, yeah, I think that scalability is definitely not an issue. I mean, it's happened a couple of times that we ran out of hard disk space because of caching, but since when the cloud was pretty fast for us to add more. Yeah. I was just thinking in terms of some of the
[00:24:24] Unknown:
sizes of mono repos for the Googles and Facebooks of the world where it takes, you know, potentially hours to actually clone the entire history of the project and some of the workarounds that they've had to do. But I'm sure that, you know, that that's the sort of 1 tenth of 1 percent case that code is even of that, scale. But I was just curious if you had ever run into something like that. Yeah.
[00:24:47] Unknown:
But you're right. The cloning is the slow part. So on those large issues, large repositories, usually cloning takes a while, and then analysis takes much, much faster, in our case. So that's really now we actually separate it, and we show that we are cloning so people know why it is slow. But yeah. So cloning just takes sometimes fast, slow, especially if you do it over the network, in the cloud and it's a large repo. But then the analysis is, yeah, much much faster than the cloning.
[00:25:13] Unknown:
What are some of the other user experience tweaks that you've ended up having to introduce just to improve the overall receipt of your product to make sure that users are able to take the full advantage of it?
[00:25:26] Unknown:
I mean, the areas where we've, toiled a little bit specifically, explanations, trying to actually explain to the customer what the issue is. We've actually had to release yeah. That was another new engine just for that, because people are saying, yeah, that's a bit confusing. And yeah. So we actually have to build on that. UI perspective as well, people understanding what's what. That's obviously work in progress. On the website, specifically explaining to customers that their call is secure, that we don't use it. We're not gonna display it as you rightfully ask to other customers. We're not gonna use it for anything else. We're not gonna store it. I think there's other companies that had issues with that. So we're very diligent in, in that. But, yeah, those are kind of the the major areas out there.
[00:26:09] Unknown:
And looking forward, what are some of the features or improvements that you have planned for the platform and for the business?
[00:26:16] Unknown:
So key 1 is in it's our internal main KPI that for this year is, the number of actual issues, like recall, that we can find. So that's, as I mentioned, it's gonna be coming up very soon, so expect something like 4 to 5 time increase in the number of issues that we can detect. So that's, that that's that's pretty exciting. I mean, it's, other things that we're looking at is we're doing automatic call fixing. We're starting to look into that, right now, but that's likely it's early 2020 release. So being able to kind of give you suggestions how to fix it automatically so you don't have to even write the code or try to understand it. We don't recommend that, obviously, but capability is gonna be there. The other 1 is, as I mentioned, trying to analyze the code on a more a more architectural level, like semantic level, and describe what it does. I think that's, another big 1. I mean, we're toying with some more interesting stuff like test case generation, automatic fully automatic as well, but that's more of a yeah. We have to see the results how commercially viable that will be.
We have many different spaces. We have quite a long road map map of of cool things that will come up. And on purely operational stuff, getting more integrations, obviously. People are asking for different integrations, so we are gonna be releasing than rather than having somewhere else. And, hopefully, if that spans out well, we'll kind of open it up so anybody can do with any ID integrations because there's quite a list of IDs out there.
[00:27:45] Unknown:
Yeah. Being able to identify some architectural patterns in ways that the code can be internally restructured to improve it either in terms of just the, understandability of it or potentially the scalability or extensibility would definitely be interesting. And also what you were mentioning as far as test cases, either identifying where a test case isn't actually performing the assertion that you think it is or cases where you're missing a test and being able to surface at least a stub of suggesting how to, encompass that missing piece of functionality in, verifying it?
[00:28:21] Unknown:
Correct. Yeah. So in the test case specifically, the the area that we're looking at is find the test case out there that it's most suitable for exactly what you're doing because that's human human generated already and to allow you to maintain it in the long run, which is pretty much the the main Achilles' heel for the all the current test case, automatic generations out there. And then adjust it a little bit so it fits perfectly for you. So that's really the the focus area that we're going in that space, which is pretty exciting. As I said, if it pans out to work, it will be an amazing product as well and nice add on.
But, yeah, the platform is tuned in a way that we can build multiple products, and we're just scratching the surface of what will come out. So there are some other tools that are operating in this space, at least tangentially
[00:29:04] Unknown:
or, you know, at surface value might appear to be doing something that's along the same lines of what you're doing, most notable being the Kite project. And I'm wondering if you can provide some compare and contrast between your product and Kite and any others that are that you're tracking that are in a similar space.
[00:29:21] Unknown:
Yep. So Kite is a great tool. It's a great ID integration. They have some great inline suggestions. The again, the the the main differentiation between Kite or any other similar tool that is doing static analysis is they look at the code in a at a much shallower level. Right? They actually try to tell, hey. It looks like based on what you're typing, a lot of other people are typing this. Right? Which is almost like treating the call as regular text. Right? There's it's syntax. Right? While we're actually doing semantic analysis, we're saying is is actually you're typing this and the third parameter, what you're passing in is not right. Actually, you the object you're passing in is a int and it has to be a long or whatever that is. Right? So that's kind of the the the main differentiation. So they have good suggestions that it's mainly kind of auto complete. So you're a bit faster to type. They they go a bit deeper and kind of getting kind of the linked or type of suggestions, as well. But, again, gives you a higher false positive rate, obviously, because it's a it doesn't go deeper to understand the issue and doesn't give you the contextual analysis as well. So that's kind of the main thing. So the accuracy, like, recall, and and accuracy is the 2 main things that are measured. So we can find considerably more things, and the accuracy rate will be considerably higher. So that's what it kind of the main, differentiation out there. But we do love Kite, by the way. They have amazing UI, amazing design, and amazing community behind them. So,
[00:30:37] Unknown:
great too as well. Are there any other aspects of the work that you're doing at Deep Code or just the overall space of automated
[00:30:46] Unknown:
fixes and automated reviews that we didn't discuss yet that you'd like to cover before we close out the show? Yeah. I don't wanna go too deeper into things that are more experimental because, those will take time, and I don't wanna get people too excited because some of them might take, years to to be ready. But the space is ripe. That's pretty much I have to say. And,
[00:31:05] Unknown:
yeah. And there'll be a lot of new things coming up. And so developers should be extremely excited what's coming up. And for anybody who wants to follow along with you or get in touch, I'll have you add your preferred contact information to the show notes. And so with that, I'll move us into the picks. And this week, I'm going to choose a book series that I read a while ago and that I'm probably going to be revisiting soon that's called the Redwall series by Brian Jacques that focuses on a bunch of, woodland animal characters and just all of it's it's a very elaborate and very detailed world and series that he built up with a lot of complex history. So, definitely worth checking out if you're looking for a new book or set of books to read, and they all stand alone nicely. You don't have to read them all in any particular order, but altogether, they give you a much broader view of, sort of his vision for that space. So definitely recommend that. And so with that, I'll pass it to you, Boris. Do you have any picks this week?
[00:31:59] Unknown:
Yeah. For me, picks this week, in general, the AI space has been going great. I mean, everybody knows there's no real AI. It's much machine learning, but, there's a couple of new areas coming in that space, and that's very exciting. It's pretty much applying machine learning to everything or big data. So that's lovely. But in that contrast, because we all do that every day and that's our passion here at DeepCult, my favorite, because do a little bit less of that and do some sports and go outside.
[00:32:26] Unknown:
That's always a good recommendation and something that bears repeating. So thank you for taking the time today for joining me and describing the work that you're doing with Deep Code. It's definitely an interesting platform, and I'll probably be taking a look at it myself. So thank you for all of your work on that, and I hope you enjoy the rest of your day. Thank you very much. You too. Thank you for listening to the show. If you want to hear more and you don't want to wait until next week, then check out my other show, the Data Engineering Podcast, with deep dives on databases, data pipelines, and how to manage information in the modern technology landscape. Also, don't forget to leave a review on Itunes to make it easier for others to find
Introduction and Sponsor Message
Interview with Boris Paskolev: Introduction to Deep Code
Deep Code's Functionality and Origins
Supported Languages and Challenges
Deep Code's Workflow and Integration
Training Models and Ensuring Quality
Challenges and Innovations in Deep Code
Platform Limitations and Scalability
Future Features and Improvements
Comparison with Other Tools
Closing Remarks and Picks