Overview
- Open-source data science projects are a great way to prepare yourself for the future.
- Explore the open-source data science projects from computer vision, natural language processing and time series
Introduction
What you learn today may become completely obsolete tomorrow.
The development of Machine Learning, Deep Learning architectures, and frameworks is growing at a cut-throat speed which makes it really hard to remain relevant in this field. My colleague advised me to refer and work on some of the open-source data science projects. And this advice has been the secret sauce ever since! Working with these projects helped me to stay up-to-date with industry trends and assisted me in my career.
The Computer Vision industry is ditching the normal 2D object detection and adopting the newer 3D object detection. The phenomenal applications of NLP are seeing new heights with the open-source of the DeText library by Linkedin. This list goes on and on. Here, I’ve put together a list of the top open-source data science projects that were created or released in July 2020.
This is part of the monthly project series where we bring out the best data science projects open-sourced on GitHub.
If you want to check out the previous projects, We’ve put them together in the form of a free course. They’re structured by the domain (computer vision projects, NLP projects, etc.) so you can focus on the project you want.
And if you’re new to GitHub, make sure you’re enrolled in this free introduction to Git and GitHub course.
Table of Contents
- Transcoder – FB’s Translator for programming languages
- Navigate your way with Complex YOLO V4
- MMDetection3D – Power dive into 3D Object Detection
- DeText – Deep Neural Text Understanding Framework
- Sktime – Sklearn for time-series
Transcoder – FB’s Translator for programming languages
Let’s start off with one of the most powerful projects implemented by Facebook Research. Imagine the power of being able to convert one programming language to another without having the knowledge of either language. Sounds magical, doesn’t it?
Transcoder is basically a transcompiler, meaning it is used to convert source code from one high-level language to another such as C++, Python, and Java.
Traditionally, we use transcompilers to convert codebase of languages that have become obsolete or depreciated. Moreover, these usually rely on handwritten rules which require a lot of manual work, the code lacks readability which ultimately leads to projects being expensive.
But with the introduction of Transcoder, this space has been revolutionized. Transcoder leverages the recent developments in unsupervised machine translator to train a fully unsupervised neural transcompiler. This open-source project has demonstrated that it can successfully translate functions between C++, Java, and Python with high accuracy. Below is an example of the conversion of functions in C++, Java into Python.
Image Source : https://github.com/facebookresearch/TransCoder
Navigate your way with Complex YOLO V4
YOLO short for You Only Look Once, has been our favorite real-time architecture when it comes to real-time object detection requiring high accuracy, faster computation, and even parallel processing. Believe me, this architecture won’t let you down. The following articles contain more information about this architecture –
- Playing with YOLO v1 on Google Colab
- A Friendly Introduction to Real-Time Object Detection using the Powerful SlimYOLOv3 Framework
- A Practical Guide to Object Detection using the Popular YOLO Framework – Part III (with Python codes)
This intriguing open-source project is the PyTorch implementation of the paper: “Complex-YOLO: Real-time 3D Object Detection on Point Clouds” based on the latest architecture – YOLOv4.
With the introduction of self-driving cars and other autonomous vehicles, the importance of Lidar based 3D object detection has become very crucial but they come with their own set of challenges.
The challenge is the capability of being able to inference from highly sparse 3D data in real-time. Complex Yolo is a state-of-the-art 3D object detection system that handles these challenges very well with high accuracy.
The following is an example video that I have taken from the repository –
Image Source: https://github.com/maudzung/Complex-YOLOv4-Pytorch
MMDetection3D – Power dive into 3D Object Detection
This is another great addition to the family of awesome open-source projects by OpenMMLab such as MMDetection and MMClassification.
Object Detection has been a consistent problem in the computer vision community but mostly in the direction of 2D object prediction. But as we move into the future we may want additional information regarding the object such as its size, orientation, and position for which 3D object detection becomes very crucial. In addition, 3D object detection can have significant applications in the field of robotics, self-driving vehicles, image retrieval, and augmented reality.
You can use the MMDetection3D – object detection toolbox which is based on PyTorch, to work in the area of 3D object detection. Here’s an example video I have taken from the GitHub repository.
Image source: https://github.com/facebookresearch/TransCoder
DeText – Deep Neural Text Understanding Framework
Let me give you two reasons for falling in love with this library. Firstly, it intelligently unifies a lot of NLP tasks, and secondly its moto – Relax like a sloth, let DeText do the understanding for you. 😛
DeText is the state of the art framework for Deep Text understanding for NLP related tasks such as ranking, classification, and language generation. The Linkedin Engineering team developed it.
Let me ask you a basic question – What is the most common thing you do when you open up your browser? More or less you type in your query in a search bar. At the backend, a lot of complex NLP tasks take place to show you the best results based on your query and recommendation.
NLP has been revolutionized with the advancements in the deep learning NLP technology like BERT. This provides new capabilities in this growing space but there’s no one size that fits all framework. That’s where DeText comes in. With DeText, users can change NLP models depending on the type of task and leverage the models to make search and recommendation systems better than before.
Linkedin uses this framework for its own search and recommendation framework.
Image Source: https://github.com/linkedin/detext
CML – DevOps for Machine Learning
The majority of top projects have been related to the field of Deep Learning and now we will delve into a project that is more on the side of data engineering.
If you ask a computer engineer he’ll definitely let you know the importance of CI/CD or the Continuous Integration and Delivery in the development workflow. Imagine being a part of a machine learning development team, many of the team members will be alongside you and making changes side-by-side on the model, data, etc. so how will you manage these updates?
CML or Continuous Machine Learning is an open-source data science project for implementing CI/CD in machine learning projects. A great advantage of using this library is that you can use it to automate parts of your development workflow such as model training and evaluation, monitor changing datasets, and comparing ML experiments across project history.
I personally find this project very useful and would recommend you to use it in your development workflow. Here’s a great tutorial video to get you started.
Video Source: https://github.com/iterative/cml
Sktime – Sklearn for time-series
If you have been working on time-series data then we have a bonus for you! Till now, a lot of the time series tasks were spread across different libraries. But sktime brings everything into one place.
Sktime also synonymous as the Sklearn for time-series data is a Python machine learning toolbox for time series algorithms.
The main advantage of this library is that it unifies all the time-series operations under one umbrella and provide sklearn compatible tools. Currently, it supports Forecasting, Time series classification, and Time-series regression.
Start by using the following command!
pip install sktime
End Notes
To summarize, we have looked into some of the most recent and amazing open-source data science projects available. Also, we observe that the computer vision community is clearly working in the direction of 3D object detection which has a lot of potential for future applications. With DeText going public, we expect to see NLP reach new heights.
The following are some article to check out other open-source projects-
- 7 Open Source Data Science Projects you Should Add to your Resume
- 6 Open Source Data Science Projects to Impress your Interviewer
- 6 Open Source Data Science Projects to Try at Home!
Are there any projects you feel I should include in this article? Or did you find any of the above projects useful in your work? I would love to hear from you in the comments section below!
Product Growth Analyst at Analytics Vidhya. I'm always curious to deep dive into data, process it, polish it so as to create value. My interest lies in the field of marketing analytics.
