Summary
Algorithmic trading is a field that has grown in recent years due to the availability of cheap computing and platforms that grant access to historical financial data. QuantConnect is a business that has focused on community engagement and open data access to grant opportunities for learning and growth to their users. In this episode CEO Jared Broad and senior engineer Alex Catarino explain how they have built an open source engine for testing and running algorithmic trading strategies in multiple languages, the challenges of collecting and serving currrent and historical financial data, and how they provide training and opportunity to their community members. If you are curious about the financial industry and want to try it out for yourself then be sure to listen to this episode and experiment with the QuantConnect platform for free.
Announcements
- Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
- When you’re ready to launch your next app or want to try a project you hear about on the show, you’ll need somewhere to deploy it, so take a look at our friends over at Linode. With 200 Gbit/s private networking, scalable shared block storage, node balancers, and a 40 Gbit/s public network, all controlled by a brand new API you’ve got everything you need to scale up. And for your tasks that need fast computation, such as training machine learning models, they just launched dedicated CPU instances. Go to pythonpodcast.com/linode to get a $20 credit and launch a new server in under a minute. And don’t forget to thank them for their continued support of this show!
- And to keep track of how your team is progressing on building new features and squashing bugs, you need a project management system designed by software engineers, for software engineers. Clubhouse lets you craft a workflow that fits your style, including per-team tasks, cross-project epics, a large suite of pre-built integrations, and a simple API for crafting your own. With such an intuitive tool it’s easy to make sure that everyone in the business is on the same page. Podcast.init listeners get 2 months free on any plan by going to pythonpodcast.com/clubhouse today and signing up for a trial.
- You listen to this show to learn and stay up to date with the ways that Python is being used, including the latest in machine learning and data analysis. For even more opportunities to meet, listen, and learn from your peers you don’t want to miss out on this year’s conference season. We have partnered with organizations such as O’Reilly Media, Dataversity, and the Open Data Science Conference. Coming up this fall is the combined events of Graphorum and the Data Architecture Summit. The agendas have been announced and super early bird registration for up to $300 off is available until July 26th, with early bird pricing for up to $200 off through August 30th. Use the code BNLLC to get an additional 10% off any pass when you register. Go to pythonpodcast.com/conferences to learn more and take advantage of our partner discounts when you register.
- The Python Software Foundation is the lifeblood of the community, supporting all of us who want to run workshops and conferences, run development sprints or meetups, and ensuring that PyCon is a success every year. They have extended the deadline for their 2019 fundraiser until June 30th and they need help to make sure they reach their goal. Go to pythonpodcast.com/psf today to make a donation. If you’re listening to this after June 30th of 2019 then consider making a donation anyway!
- Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email hosts@podcastinit.com)
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers
- Join the community in the new Zulip chat workspace at pythonpodcast.com/chat
- Your host as usual is Tobias Macey and today I’m interviewing Jared Broad and Alex Catarino about QuantConnect, a platform for building and testing algorithmic trading strategies on open data and cloud resources
Interview
- Introductions
- How did you get introduced to Python?
- Can you start by explaining what QuantConnect is and how the business got started?
- What is your mission for the company?
- I know that there are a few other entrants in this market. Can you briefly outline how you compare to the other platforms and maybe characterize the state of the industry?
- What are the main ways that you and your customers use Python?
- For someone who is new to the space can you talk through what is involved in writing and testing a trading algorithm?
- Can you talk through how QuantConnect itself is architected and some of the products and components that comprise your overall platform?
- I noticed that your trading engine is open source. What was your motivation for making that freely available and how has it influenced your design and development of the project?
- I know that the core product is built in C# and offers a bridge to Python. Can you talk through how that is implemented?
- How do you address latency and performance when bridging those two runtimes given the time sensitivity of the problem domain?
- What are the benefits of using Python for algorithmic trading and what are its shortcomings?
- How useful and practical are machine learning techniques in this domain?
- Can you also talk through what Alpha Streams is, including what makes it unique and how it benefits the users of your platform?
- I appreciate the work that you are doing to foster a community around your platform. What are your strategies for building and supporting that interaction and how does it play into your product design?
- What are the categories of users who tend to join and engage with your community?
- What are some of the most interesting, innovative, or unexpected tactics that you have seen your users employ?
- For someone who is interested in getting started on QuantConnect what is the onboarding process like?
- What are some resources that you would recommend for someone who is interested in digging deeper into this domain?
- What are the trends in quantitative finance and algorithmic trading that you find most exciting and most concerning?
- What do you have planned for the future of QuantConnect?
Keep In Touch
- Jared
- @jaredbroad on Twitter
- Alex
- AlexCatarino on GitHub
- @AlexCatx on Twitter
- QuantConnect
- @QuantConnect on Twitter
- Website
Picks
- Tobias
- Good Omens book and miniseries
- Jared
- Chernobyl HBO Series
- Alex
Links
- QuantConnect
- LEAN algorithm engine
- Alpha Streams
- Google Spanner
- PyCharm
- Visual Studio Code
- IronPython
- NumPy
- SymPy
- Pandas
- PythonNet
- Tensorflow
- Keras
- Udemy
The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA
Hello, and welcome to podcast dot in it, the podcast about Python and the people who make it great. When you're ready to launch your next app or you want to try a project you hear about on the show, you'll need somewhere to deploy it. So take a look at our friends over at Linode. With 200 gigabit private networking, scalable shared block storage, node balancers, and a 40 gigabit public network, all controlled by a brand new API, you've got everything you need to scale up. And for your tasks that need fast computation, such as training machine learning models or running your CI pipelines, they just launched dedicated CPU instances. In addition to that, they just launched a new data center in Toronto, and they've got 1 opening in Mumbai at the end of 2019.
Go to python podcast.com/linode, that's l I n o d e, today to get a $20 credit and launch a new server in under a minute. And don't forget to thank them for their continued support of the show. And to keep track of how your team is progressing on building new features and squashing bugs, you need a project management system that can keep up with you that's designed by software engineers for software engineers. Clubhouse lets you craft a workflow that fits your style, including per team tasks, cross project epics, a large suite of pre built integrations, and a simple API for crafting your own. With such an intuitive tool, it's easy to make sure that everyone in the business is on the same page.
Podcast.init listeners get 2 months free on any plan by going to python podcast.com/clubhouse today and signing up for a free trial. And you listen to this show to learn and stay up to date with the ways that Python is being used, including the latest in machine learning and data analysis. For even more opportunities to meet, listen, and learn from your peers, you don't want to miss out on this year's conference season. We have partnered with organizations such as O'Reilly Media, Dataversity, and the Open Data Science Conference. Coming up this fall is the combined events of Graph Forum and the Data Architecture Summit.
The agendas have already been announced, and super early bird registration is available until July 26th where you can get up to $300 off, or the early bird pricing for $200 off is available through August 30th. Use the code b n l l c to get an additional 10% off any pass when you register. Go to python podcast.com/conferences to learn more about this and the other conferences and take advantage of our partner discounts when you register. And, also, the Python Software Foundation is the lifeblood of the community, supporting all of us who want to run workshops and conferences, run development spritz or meetups, and they also ensure that PyCon is a success every year. They have extended the deadline for their 2019 fundraiser until June 30th, and they need help to make sure they reach their goal.
Go to python podcast.com/p s f 2019 today to make a donation. And you can visit the site at python podcast.com to subscribe to the show, sign up for the mailing list, read the show notes, and get in touch. And if you have any questions, comments, or suggestions, I'd love to hear them. And to help other people find the show, please leave a review on Itunes and tell your friends and coworkers. Your host as usual is Tobias Macy. And today, I'm interviewing Jared Broad and Alex Catarino about QuantConnect, a platform for building and testing algorithmic trading strategies on open data and cloud resources. So, Jared, could you start by introducing yourself? Hi, Tobias. Nice to meet you. My name is Jared Broad, and I'm the CEO and founder of Conkonet.
[00:03:36] Unknown:
We were founded about 2012. And since then, we've been building this, community and cloud platform to give people the most powerful algorithmic trading services possible. And, Alex, could you introduce yourself as well? Yeah. Hi. I'm Alex. I'm the lead at QuantConnect,
[00:03:54] Unknown:
and I was involved a lot in the, Python integration with QuantConnect.
[00:03:59] Unknown:
So, I'm here because of that. And going back to you, Jared, do you remember how you first got introduced to Python?
[00:04:05] Unknown:
Yeah. I've been in in in and out of Python for many years now, but, it was really the the community of Kong Konnect that drove me personally mostly into Python. So it's been strongly demanded by our community and, I've had to learn it from the ground up, which has been a a fascinating and and humble experience. But,
[00:04:27] Unknown:
yeah. No. It's been it's been a great journey. And, Alex, do you remember how you first got introduced to Python? Yeah. It started late
[00:04:33] Unknown:
2015 when we start work on the integration. Yeah. And it's been, great learning, how powerful Python is. I also help to translate some of the algorithms to from c sharp to to Python, and, I learned a lot in this process.
[00:04:50] Unknown:
And so, Jared, going back to you, you mentioned a little bit about what QuantConnect is and some of, when it got started. But I'm wondering if you can just give a broader overview about what you're building with the QuantConnect business and platform and how it got started and just your overall mission for the company.
[00:05:08] Unknown:
Sure. QuantConnect really at its core is a a global community of engineers, quants, scientists, mathematicians, just people who have a passion for the market and, have ideas that they want to test. And so, we our mission really was to break open this really complicated and hard to access, technology and financial data so that individual people could get access to it and test their ideas and then deploy powerful investment strategies to the markets. It's been we started in in 2012, and the world's evolved a lot since then. And, now we're we've built this technology. We've we've offered this platform, and the community is the biggest community in the world of these quants.
And we are looking to provide them a way to, monetize their efforts and, make the most of their ideas. And so that's what we're we're launching, this year, which is called Alpha Streams. And it's a way for them to really, make the most of of their efforts.
[00:06:18] Unknown:
And I know that there are a few other businesses that are in a similar space as far as being a way for average users to be able to build and test algorithms for running against financial data and possibly actually executing trades with it. So I'm curious if you can talk through some of the other businesses or platforms, whether open source or proprietary, that are operating in that similar space and maybe characterize your position in the market and how you compare to some of the other options?
[00:06:52] Unknown:
Sure. We we, I would say that we're the only company that has stayed 100% focused on the quants and the community. We that motive that, has driven our core business decisions, our technology design. And so as a result, we're the only 1 in the market who actually didn't become a hedge fund. So we've focused on developing the platform and the technology to be the best possible for the Quant community. And so as a result, we now support, equities, forex, options, crypto markets. You can do, live trading on, 6 different brokerages now, and it's all co it's all hosted in our colocated servers in New York. So we provide, you know, like, professional quality execution, live trading, live data, and it's all built and focused on this quant community. And this really, this came to a peak with Alpha Streams because we were being approached by a lot of quantitative fund hedge funds who were saying they wanted access to the community.
And we didn't want to compromise on those values that we put the community members first. So when we were looking at how we could do that, we were trying to find a way that the the quant could do we could do the best for the quant. And that's really how Alpha Streams came about. It's a, the world's first alpha market. It's a way for these quants to design algorithms and put them into a marketplace. And then I have dozens of hedge funds from all around the world reviewing the the alpha signals and, having the ability to license those algorithms and get distribution that they've never had before.
[00:08:39] Unknown:
And so I know that the core of the platform is largely implemented in c sharp, and that that's where a lot of your background lies, and that you have added support for Python as an implementation language. But I'm wondering if you can just start by talking through what the main ways are that you and your customers end up using Python both, within the QuantConnect platform itself and just more generally in terms of the algorithmic nature of the problem domain that you're working in? We are, we are constantly humbled
[00:09:13] Unknown:
by by the 1, 000, 000 different ways that the community uses, the platform. And mostly, our job is to make it as versatile and flexible as possible. So we we do that for them, and then, they they can use it to test basically anything. So the the platform data ranges from tick all the way up to daily resolution. And so users are developing, sort of fast I wouldn't say high frequency, but they're developing fast strategies, intraday strategies that are trading on momentum or or some sort of intraday signals, all the way up to portfolios and long term rebalancing kinds of strategies.
And the the beauty of having all the different asset types that are available in the platform, you can combine strategies and asset types to generate the best returns. So if you've got an idea that could use equities and options, you can merge them together. Or if you want to go to crypto assets when the market is going crazy, you can add crypto data to your algorithm and move between the crypto markets and the equity markets. It's really fascinating.
[00:10:24] Unknown:
And for anybody who isn't familiar with the space of quantitative trading and some of the just overall requirements in terms of general knowledge of financial data and algorithmic strategies. I'm wondering if you can just talk through, just what is involved overall in being able to build and test these algorithms.
[00:10:47] Unknown:
Well, we have a lot of material for someone who is new. We have been working a lot of on this material for the past 2 years. And now we have dozens of tutorials in, in a strategy library with hundreds of academic papers implementation. So, our users, have access to full, working algorithms that can that they can, take as an inspiration and also follow all the process that in these, paper implementations. And we also have a boot camp, which is a lit interactive, step by step tutorial. And on top of that, we have the an active community that share knowledge and ideas in the in a forum.
And just add to that, no matter whether you are beginner beginner or professional, we have the tools, and then that attributes, powerful, awaiting trading strategies.
[00:11:43] Unknown:
Now in terms of QuantConnect itself, I'm wondering what the overall architecture of the platform looks like and the various components that play into each other for being able to provide a way to access the data that's necessary for testing these strategies and, just some of the overall challenges and complications involved both from the, nature of being able to run potentially untrusted user code and also being able to collect and maintain all of the data that's necessary for being able to do the historical backtesting?
[00:12:21] Unknown:
It's incredibly challenging, to be honest. We are always breaking every API that we use. So, we we add a new brokerage, and we find bugs in the brokerage API within a day. We add like, we swapped database technology to use Google Spanner a while ago, and we broke Google Spanner. We have we have to file bug reports in Google Spanner. Yeah. So, overall, the the the core of the the sort of user facing experience is an online coding environment. So you you go to QuantConnect, you sign up, it's completely free, and the first thing you're greeted with is it's a coding environment very similar to, PyCharm or or or Visual Studio Code, and, 80% of the community are are Python users. So it presents you with a Python algorithm to get started, and you can go and you can design, 1 of these algorithms in this online editor.
So behind the scenes, we have all of the we've built web versions of all of the normal tech that you would have for a Code Editor. So you've got debugging. You've got a builder, a compiler. You've got interactive IntelliSense. So all of the, the aspects of running IntelliSense, you know, and and but it's all in the cloud version and made for scale with with thousands and thousands of people. So it's it's a very fun challenging problem. And then, once you've built an algorithm and you go to deploy it into a backtest, You're running your algorithm through lean.
Lean is our, open source project. It's a massive algorithmic trading system that allows you to build an algorithm and test it. And we, the lean engine, handles everything else. So behind the scenes, it's getting the data you need. It's synchronizing that data. It's, managing your portfolio, so the state of your algorithm, and managing all of the models so that we can do, virtual fills in in simulation on all these different asset classes and and virtual transactions and modeling all of the fees and the slippage and the portfolio values. So lean the lean engine handles that, and it handles it incredibly fast because, really, the the users are running on terabytes of data underneath. And so we've got these large clouds underneath, supplying that data. And behind the scenes, the users are requesting terabytes of data. So we have a large financial data library, and we maintain it and curate it. And we have a, redundant systems in multiple locations, which are generating, curating, and, providing that data to the community. So it it's kind of a large engine that really just lets the the users in the community, focus on building and designing and testing their ideas, and the QuantConnect team handles everything else.
[00:15:36] Unknown:
And so you mentioned that the core of the platform is the lean engine, and I know that it's been released as an open source technology. So I'm wondering what your overall strategy is and your motivation for providing that as a freely available resource for people to be able to use and analyze for their own purposes.
[00:15:59] Unknown:
Yeah. We released, Lean in 2014, 2015. And it was kind of scary at the time because it was our first open source release. And like the algorithmic trading industry, everything's so secretive and it kind of goes against the grain. So it was actually probably 1 of the best decisions we made. Algorithmic trading is incredibly sensitive to the platform, the infrastructure that you're running on. And by open sourcing it and letting the users touch it, inspect it, we were able to give them a lot of confidence and faith in the platform and how it's built and how it's architected. We were able to rally this global community of engineers and and brilliant minds to come and work on the platform. So there's over 90 contributors from all around the world who have been part of making Lean possible. It's kind of weird, but, a lot of our team was found through the open source. It's people who are working on the project for fun on their own time. And we say, hey, you know, come and join QuantConnect and we'll do it full time together. So we just bring the people on board, and they they join the team full time. And it's been a great way to meet, new team team members. So, yeah, it's it's been an interesting journey with the open source. And given that the lean engine is written in c sharp, but you're using it as a mechanism for executing Python code. I'm wondering what the bridge looks like and just the overall technology stack that you've compiled for being able to,
[00:17:29] Unknown:
enable that interaction and be able to run those Python algorithms, particularly in terms of managing dependencies, within this c sharp core runtime?
[00:17:39] Unknown:
Yeah. The original implementation was based on Aeropython. It was in product it dates back in 2014. It was in production for about 1 year, but it has, poor library support, just the standard library. And the community members were looking for NumPy, Sampy, and Pandas. They they love Pandas. And, eventually, machine learning libraries. So we start looking for replacement in 2, 000 15, 16. And we tried, out of code generation by reading the c, sharp code and then creating a Python library in Leap for the it wrote 1, 000 of classes, and automatically porting c sharp to Python, but it was really, really, really slow. So we we we couldn't use it.
So, despite in NumPy, we looking to, make a c c plus plus version of green. We would would bound it directly, the Python, c API, invoke it from the c plus plus library that would be Lean. It, this, solution requires a serialization across the memory domain, passing the data from c to Python and, on on the other, direction. And it is always the lowest part of the the transport for anything from, beyond basic types. Civilization is the core limitation, for this kind of, interop operations. Anytime a complex type needed to transmit between the c sharp and the Python, it requires authorization to string to work properly.
So, in in our research, we came across a Python net, which also a no brain source project. It's led by Dennis Akiarov. And Python net is it was a great finding. It provides, the key bindings into Python c API into net applications. So it can easily consume, the Python objects. So with, Python Net, we're able to import Python classes into Lean as algorithms. It supports, all, our versions of Python from 2 to 3.6. That that's the version that we are using, right now, and it support most of the libraries. So now, it supports NumPy, SciPy, TensorFlow, Keras.
For instance, ELEAN, we deploy model, the the portfolio of an algorithm, providing the profits, losses, the slippage models like Gerald mentioned before. And PythonX handles all the types conversion between the 2 language very well.
[00:20:34] Unknown:
And because of the fact that you're bridging these 2 different language run times, I imagine that that has at least some impact on the overall latency and performance characteristics of the execution of the algorithms. And since the financial space is highly time dependent as far as the execution of trades to make sure that you're getting the price that you think that you're getting. I'm curious what your approach has been to managing those performance impacts and maintaining an appropriate level of latency?
[00:21:07] Unknown:
Well, with NetBeat, we stuck with Pythonet and, looking to, optimize the PythonNet bridge, bridge between the the 2 languages because it's it's it's already, the direct binding through the c, the Python's API. So it's already the FESO, FESUS option available. Right? So we're just looking to improving, Pythonet and improving also the speed of the in a way. In live trading, being slightly fast. It's not the steel it's not designed for, high frequency trading, like like Gerald has mentioned. But 99% of the world is more than enough speed.
So, since links can analyze and respond to milliseconds, there are other bottlenecks like the broker JPI that are much more important, to to the latency.
[00:22:04] Unknown:
And have there been any complications or edge cases that you've had to deal with in terms of the, communication layer between Python and the, dot net CLR for, you know, any sorts of bugs that are introduced. And then the other curiosity I have is management of the 3rd party libraries that you bring in as far as any upgrades trying to avoid breaking the, user submitted algorithms that might be dependent on particular features of whatever, version is available at the time of implementation for things like NumPy or TensorFlow.
[00:22:43] Unknown:
It it is a problem, though, and, it's been an interesting 1 to manage. So behind the scenes, we have a versioning system kind of like GitHub that lets us deploy different versions of our infrastructure. So we can actually for, the startup hedge funds that they're using, QuantConnect, we can actually pin the version of the infrastructure to a certain version, and then they can go and they can run their fund on a specific version. But for the bulk of the community, it's, better. We found to give them the latest version of QualcommX code because it has the most bug fixes.
Just, simply we we get, we iterate very, very quickly, and we we fix the open source and, and and fixes bugs very, very quickly. And so deploying those fixes to production is the best we can do for the for the bulk of the community. But for a lot of people who are, you know, managing, assets and and doing sort of production, sensitive deployments every day to live trading, we've got this pinned infrastructure system that we make available to them.
[00:23:57] Unknown:
Well, for the for the libraries, we we we have our our every time that we add new libraries or update the our the framework, the the the basic Docker, container that we have, we test, it against a bunch of, algorithms as a kind of unit test to see if, it's not broken. But we try to keep, the the versions, cost, constant in time and just make, an upgrade when we when we see, its fit. So we are always looking for the the state of the art of, each library.
[00:24:43] Unknown:
And for the usage of Python itself, you mentioned that there are something like 80% of your users who use that versus some of the other run times that you support. And I'm wondering what are the benefits of the Python language specifically that brings in so many people in the Quant community. And I'm also wondering what the practicality is for machine learning techniques, because I know you mentioned that people are leveraging TensorFlow and Keras. And I know that a lot of the utility of machine learning is based on being able to recognize and strategies are on that front. Right.
Strategies are on that front.
[00:25:31] Unknown:
Right. The main benefits of Python, it's called Clarity in, Brevity and along alongside with the data process capability. Our members actively use Pandas as it's a great library to deal with metrics prices that represent the history of our portfolio. Also, it benefits as as we mentioned, it was the main reason to to work on the with that, all sorts of libraries that comes, with Python, that, is great for machine learnings and which is a hot up today. On the type of strategies, we can't really disclose, but, simple strategies like see, what what's the predictability of, a a get a open a gap in the open prices. People can plug this into, the the machine learning and see, how this can, be used as a a predictor.
[00:26:31] Unknown:
And you mentioned that the other recent product addition is this Alpha Streams platform where users in your community can submit their algorithms for being able to be licensed by other people to execute their trades on. And so I'm wondering what it is about Alpha Streams that makes it unique in the market and how it benefits the users of your platform, and just what the overall adoption and feedback has been.
[00:27:00] Unknown:
Alpha Streams is, at its core, this market where users can deploy their algorithms into, an environment where institutions can review them and potentially license them. But really to to fully understand it, you have to go back to how the industry works today. So you have thousands of hedge funds doing things in a very old fashioned way. They're not quite the same. They're not quite keeping up with the open source community. They, they hire recruiters. They they bring in these quants who are new to the firm. They pay expensive recruiter fees. They train them up and they get them work on new datasets. And maybe in 6 months or so, the quant is going to come up with a cool idea that they're going to put into production. And so there's this very long search process where these institutions are looking for an idea and looking for an algorithm to trade. And because there's no central way, central platform, where they can compare these, algorithms, they can't trust anything that comes from outside the firm. So everybody's repeating everything they do. They they bring in data. They redo the same experiments that every other hedge fund has done. So with Alpha Streams, it's it's flipping the entire industry on its head. We are looking to automate this entire search process.
So if if we were to design the perfect hedge fund today, what would it look like? And so in our minds, it would tap into a globally distributed community where the ideas are tested and vetted and evaluated based on merits, Not where you are in the world and not where which school you came from. So with Alpha Streams, if your idea is good, you can put it into QuantConnect. You can get it tested and verified independently by us, and then it can be distributed to thousands of hedge funds. And so that idea of a meritocracy in Quant Finance is, at its core, is that's what Quant Finance is about. But in practice, there's just so much politics and paperwork and overhead that goes into into running the fund that, it doesn't actually happen in practice.
So we've built an API where these institutions can come in and search the marketplace for funds, for strategies which are listed. And the community can quickly and easily deploy their algorithms to be listed in the marketplace and, be considered to be licensed. It's it's really an exciting concept that flips the entire industry on its head. And
[00:29:30] Unknown:
the other thing that is notable about the work that you're doing is the efforts that you're putting into to foster and grow the community around QuantConnect and quantitative finance. And so I'm wondering what your overall strategies are and just the overall effort that you're putting into building and maintaining this community and how it plays into your overall product design?
[00:29:55] Unknown:
It's a tough 1. It's a very hard subject. So the community members are grappling with, high order math. Like, they have to know, pretty advanced coding. They're grappling with a new API, our API, and and and terabytes of data that they've never seen or managed before. So it's, it's definitely a challenge that we are learning and growing from on a daily basis. But, we we just do our best to, make the the platform transparent so that you can see as much as possible to give the community, the tools that they're familiar with, like debugging, and then, and and sort of code inspection and the ability to to build algorithms in a familiar environment.
And then, we just try to, really empower and highlight those people who are doing an awesome job. So, we we give people free live trading as much as possible. We give them free data so they can go and backtest on on massively powerful VMs in the cloud, on on terabytes of data. And, it's just, it's 1 of those things. We as a company are constantly evolving and constantly figuring out the answers to that 1. Recently, actually, just a couple of days ago, we deployed something that's awesome. And, it's a parameter detection system.
So in quantitative research, there's certain ways where you can fall into common pitfalls. And if you're not careful, you can, design an algorithm that'll work perfectly on historical data and be extremely overfitted and be useless for live trading, and it could be very risky for you in live trading. So we've built a parameter detection system and a backtest counting system so that we can give people warnings. Hey. You're using 30 parameters in this algorithm. You know, did you know that with 30 parameters, you could paint the Mona Lisa? You could come up with a variable that'll literally paint the Mona Lisa, and and you could do it with a handful of variables. So, you know, try and try and avoid using so many parameters in your algorithm. So it's just a a continuous process that we're going through. I think that something like that is great that you have this feedback mechanism, particularly for people who are maybe just doing this as a hobby project to just dabble in what's involved in these higher order mathematics
[00:32:24] Unknown:
or trying to experiment with data science and data analytics. And so, I'm also wondering what the broad categories are of the types of users who interact with QuantConnect and what it is that keeps them engaged and continuing to work on and help you evolve the platform?
[00:32:47] Unknown:
Well, we are constantly amazed by the creative of our users. I work with, support, and I see what they are doing. They and it's incredible. And, they push us to develop the new technology every day, like German recognition, with the parameters count. By the way, that part, we we have done that with Python. It's a Python script that detects the parameters. Most of what we do, it's, intellectual property of the users. So we can discuss in details, like, in the examples that I I gave before. But they are there's, also work.
For example, Marine, is built on open source Android application, to the API so people can control their live algorithm, on the go. Or James has released an open source optimizer. You can, run generic optimizations with Lean that we still don't support, but, eventually, we will do in the future.
[00:33:49] Unknown:
And and to give you more back To give you and to give you more background, the QuantConnect community is just so diverse. It's so hard to put them into 1 bucket. So, we have all the core STEM graduates, so science, technology, mathematics, chemistry, physics, anybody who's gone through that sort of programming and rigorous training. But then, there's there's about a third who are financial professionals. They might be working in the fund industry and doing this on the side or dreaming about starting their own fund. Or, there's, academics and traders and, just thousands of people from all over the world. There's about half of the community here in the US, but the other half are just spread out in in UK, Russia, China, India.
It's fascinating.
[00:34:37] Unknown:
And what have been some of the most interesting or innovative or unexpected ways that you have seen people interacting with QuantConnect and using it and, just sort of building in terms of the types of strategies or tactics that they're leveraging?
[00:34:55] Unknown:
Yeah. So it's it's tough for us to talk about the individual strategies that people are building, but, broad strokes, there's everything from some fast to machine learning to portfolios. But then people also we've opened up an API to QuantConnect itself. And so, a a Netherlands, programmer has created a mobile application for Android, and it's actually got a full user interface for controlling, starting, and stopping your live trading algorithms. So we're a fairly small company and we're, we'd like to keep the whole company lean and, and, it's awesome to see the community sort of pick up areas where we haven't built yet. And they're just running with it and building epically, beautiful applications.
And, James, I think his name last name is Christchurch, but I only know his Githand his Githandel. He's built a genetic optimizer. So you can actually just plug the the open source lean algorithmic trading engine into his genetic optimizer, and it'll go and it'll run, a batch genetic optimization on 100 or thousands of of lean strategies that'll run through it to find, the right parameters for your algorithm. So it's it's really awesome to see the community do that sort of stuff. It's all open source, and and, we we are constantly blown away by it. And for somebody who's interested
[00:36:20] Unknown:
in getting started with QuantConnect and quantitative finance and algorithmic trading in general. I'm wondering what the onboarding process looks like for building on QuantConnect and just some of the resources that you have found to be useful for people who are new to the area and want to start experimenting with it on their own time? We try to make this process as easy as possible, but ultimately, it is gonna be a little tricky
[00:36:47] Unknown:
because we're trying to introduce people who might come from a just a pure programming background to all of this quantitative world. But the first thing you do when you sign up, is you've got that coding environment, and we we put you into something we call boot camp. And boot camp is kind of like Udemy, where you have, a a little documentation and you have to do, we call them like a micro tutorial. You have to do the smallest possible step to move your algorithm towards production and getting it to run a back test. And so, we plan to build out the boot camp, library and and have hundreds of different boot camp tutorials where people can go through these micro steps and learn how to build really powerful algorithm algorithms. We also have a boatload of documentation so that people can go and read videos, and the community is is very engaged.
[00:37:44] Unknown:
We're very grateful for, the community and all their help to to help other community members. And looking broadly at the industry, I'm wondering what the trends are in quantitative finance and algorithmic trading that you find most exciting and the ones that you find most concerning as you look forward? I'm probably fairly biased.
[00:38:05] Unknown:
I don't I don't find too much concerning. But, I am very excited about, the move for finance to become more like Silicon Valley, especially in the last 2 to 3 years. You're seeing these large institutions start to open up. And that's something that we've been doing for years now. And we're we're happy to see these these beasts, you know, these large machines starting to come around to these ideas. And then, the the other big trends that we believe is is happening is this, you're no longer you're no longer judged by a single idea.
You're judged by how fast you evolve and how fast your ideas can can iterate and evolve. And the financial markets are are driving that. The the markets are changing faster than they ever have before, and that's forcing people to change and automate faster, better, and new unique ideas more than they ever have before. And so that automation is really part of the key of QuantConnect's Alpha Streams. We see that the individual funds can't compete against thousands and thousands of quants in the community. And so by pairing a fund with the community, we think that they can be equipped to evolve fast enough to keep up with the markets.
And it's really the core of the Alpha Stream vision. And
[00:39:31] Unknown:
in terms of the future of QuantConnect, what do you have planned in terms of overall features or growth for the platform or plans that you have for helping to foster the community and help them thrive?
[00:39:49] Unknown:
We are investing every day a lot in our education, in the platform, making it better for the community. And ultimately, all of this is to drive licensing and and get these community members connected with funds so that they can earn revenue. And so that's really what inspires us. We want to get these users, we wanna give them as much research about cutting edge quantitative commute, techniques so they can make the best live trading algorithms, so they can get, licensed and distributed to funds. So really grow this to the largest quantitative community of alga traders and developers and coders in the world and, pair them with the best technology possible. And in the process, what we're doing is we're making the industry more efficient.
You've got all of these institutions all repeating the same thing, and it's incredibly inefficient. So by, making the platform better, we can make the whole industry more efficient and, empower people from all around the world based on the merit of their ideas. And that's that's really the core of our passion. And are there any other aspects
[00:40:57] Unknown:
of QuantConnect, any of its various aspects, or the overall space of quantitative finance that we didn't discuss yet, but that you'd like to cover before we close out the show? Yeah. I think, the industry,
[00:41:10] Unknown:
needs as much brain as possible, to foster, more innovation, and that's it. Oh, I suppose,
[00:41:17] Unknown:
we're recruiting. We are looking for people who love hard challenges, and we have epically hard challenges. So if if you're interested in solving how do we get debugging for thousands of people when they're backtesting terabytes of data and, you know, figuring out how to do that in a multi language environment, and worrying about memory leaks and pinning and cross domain serial is typically hard problems. So if that sounds like you, then check out our open source.
[00:41:49] Unknown:
Good point. I was also thinking about the data quality. Maybe, people can come data scientists can come up to help us to improve our data. That's also important, especially now that we are going to have, alternative data from, other markets that's not just price data. So it's it's very important to To to further
[00:42:11] Unknown:
to to give some background to that, we have terabytes of this price data. So futures, options, equities, forex, crypto, and really though to to build an algorithm, there's a little bit more than just price data that that's required. So most people, they say, hey, when we'll just apply this technical indicator to this price data and that'll be our strategy. But most of the time, because that, strategy because all of that price data is open, all of those technical strategies that might have worked in the 19 seventies, 19 eighties, they don't really work in live trading anymore.
So the whole, financial industry is moving towards using alternative data in their strategies. 1 other big thing I think that's happening at the moment that's really important for the community to understand is this move from focusing on price data alone to focusing on a core hypothesis. So when people are designing an algorithm, they're they're pulling in much more of the scientific method. So instead of just saying, I'm gonna hack around with the starter until I find something that fits, they first need to start with an idea. And that idea has to be something fundamental that'll move the markets.
So, for example, you might say that, when there's more sunshine, you're going to have more oranges produced, and that is going to cause a surplus of supply of oranges, and so orange juice futures contracts are probably going to fall. And so there's a definite cause and effect of that, relationship between the sunshine and the orange juice. And it might be weak and it might just be a hypothesis, but then you need to go and you test your idea and you see if it has merit. And so you might pull in things like, if the Federal Bank, increases interest rates, then the mortgage rates are going to go up. And so, thus, any real estate investment ETFs are probably going to go down, because they won't be able to buy as much with the capital that they have.
And so there's lots of different cause and effects, whether it be in the real world, like looking around you and seeing how things interact, or in sort of the virtual world, in the financial markets and how different mechanisms interact with each other. But really to to to monetize those things, your algorithm needs to be connected to what's called these days as alternative data. It's data which is not just the price data of the markets. It's it's a signal that's about the world. You know. It's a signal about sunshine and and the current sunshine hours in Florida or, you know, the current federal interest rates, and using that as a way that your algorithm can start to trade and act on those signals. And so, at QuantConnect, recently, we've started this project, to work with alternative data vendors and get them to import their data into the QuantConnect repository.
And that way our community can go and design these epically powerful algorithms on not just price data, but all of this alternative data that covers, you know, social sentiment, Twitter, you know, jobs numbers, real world data like weather, temperature, everything. As much data as we can find, we're pulling into QuantConnect right now. So it's a really, challenging project in terms of engineering, but also it's going to empower the community to make such amazing strategies.
[00:45:58] Unknown:
Yeah. That's definitely something that is, as you said, challenging and complicated, but also interesting in terms of just the breadth and scope of the problem domain and how that, how the level of sophistication that is required has grown since the financial markets first became, able to be interacted with in a computational fashion just because of the increasing globalization and inter interconnectedness of our various financial markets, as, you know, transportation and communication catapults us further into the 21st century and these, just global economies and yeah. It's a fascinating area. So it's definitely interesting to hear about how you're trying to, help provide your users the ability to leverage that additional data to be more effective in their strategies.
[00:46:54] Unknown:
Yeah. You've heard the the the meme, basically, that there's just what is it? Data's growing at double the rate each year or something like that. There's thousands of terabytes of data being added every day, and then it's it's so true. And you're seeing even, you know, mainstream government websites totally opening up their datasets. And so it's becoming this challenge to import all of that data and and analyze it and and be put it into a usable form so that you can run an algorithm on it. And, our job is we build say we build in 1 dataset. With that 1 dataset, you can do a 1, 000 different algorithms.
And that's the the brilliant thing we love seeing. You know, if you're a, agriculture engineer, or you've worked in that industry, you might be familiar with, you know, orange juice and sunshine. Or if you're a solar engineer, you might go and be familiar with the solar industry and how sunshine now is gonna affect electricity generation. And so there's just thousands of different ways that people with their own unique mindsets and and and abilities can write completely different algorithms on exactly the same dataset. Now that's that's the power of a global community like QuantConnect because it's not about our ideas or what we do.
[00:48:11] Unknown:
All we do is put 1 data set up and then individual people from all over the world make a 1, 000 different ideas from that. And it's so cool to see. Yeah. Definitely excited to see the work that you're doing, and I'm curious to see how the incorporation of those alternative datasets are going to help, enrich the capabilities of the people using your platform. And so for anybody who wants to follow along with the work that you're doing or get in touch, I'll have you add your preferred contact information to the show notes. And so with that, I'll move us into the picks. And this week, I'm going to choose Good Omens, both the book and the miniseries that was recently released.
It's definitely a great storyline. I watched the first episode of the miniseries recently, and I think they did a very good job of that. So I'm excited to watch the rest of it. So, if you're looking for a new piece of fiction to pick up and engage with, I definitely recommend that. And so with that, I'll pass it to you, Jared. Do you have any picks this week?
[00:49:07] Unknown:
I got into the I think it's HBO series Chernobyl. It's it's following the, like a story form documentary on the Chernobyl disaster. And, it's amazing. I love it so much. Highly recommend it.
[00:49:24] Unknown:
I'll have to take a look at that. Alex, do you have any picks this week?
[00:49:29] Unknown:
Yes. For, this week, I'm going to pick it the the series, the 100, because it talks about, the future of the humanity and the dangers of, artificial intelligence. So I think it's, fitting to the to what we'll be talking about.
[00:49:49] Unknown:
Alright. Well, thank you both for taking the time today to join me and discuss the work that you're doing on QuantConnect and in the quantitative finance realm. It's interested in, experimenting with. So thank you for your time and energy on that, and I hope you enjoy the rest of your day. Thank you very much, Tobias. It was great to chat. Thanks, Tobias.
[00:50:15] Unknown:
Bye.
Introduction to QuantConnect
QuantConnect's Mission and Community
Market Position and Competitors
Python Integration and Usage
Platform Architecture and Challenges
Open Source Strategy
Python and .NET Integration
Performance and Latency Management
Alpha Streams Platform
Community Building and Engagement
User Creativity and Contributions
Getting Started with QuantConnect
Industry Trends and Future Directions
Future Plans for QuantConnect
Call for Talent and Data Quality
Alternative Data and Market Strategies
Closing Remarks and Picks