Overview
- ICLR is an annual global machine learning conference where the top minds in the ML and DL fields present their research papers
- This year’s event saw 937 submissions, double that of last year. 337 papers were accepted
- We look at the three best papers and provide links to other resources
Introduction
As a data scientist, it’s very important to keep yourself updated with the latest developments in the community. Reading research papers from the top researchers should be of the highest priority! And I can assure you there is no better conference that brings all the top minds under one roof than the The International Conference on Learning Representations (ICLR).
ICLR 2018 concluded yesterday in Vancouver, Canada, and was suitably represented by a whole host of top research papers, including entries by DeepMind and Facebook. This year, the ICLR community received 935 papers for review (double that of last year) and 337 papers were accepted into the final conference.
In this article, we have listed the best three papers, as chosen by the ICLR committee, and provided you the link for other resources which you will find handy.
Let’s look at the 3 best papers!
On the convergence of ADAM and Beyond
This paper was submitted by Google New York. In this, the researchers have proposed a new take on popular optimization algorithms like Adam and RMSProp. The team claims that gradient boosting, while being an effective neural network technique, fails to converge to an optimal solution in non-convex settings. This paper investigates the likely cause of failures and goes on to provide an example of where Adam does not converge to the optimal solution.
The team’s analysis suggests ways on how the convergence issues can be fixed, and proposes new variants of the Adam algorithm. These variants not only fix the convergence issues but have been proven to improve the performance as well!
You can view the research paper here.
Spherical CNNs
This paper was published by researchers from the University of Amsterdam. In this paper, they have introduced the building blocks for constructing spherical Convolutional Neural Networks (CNNs). What pushed this research? Well CNNs were already the algorithm of choice for problems involving 2D images. But those previous methods used to fail when 3D objects came into the picture. Examples of 3D spherical problems include omnidirectional vision for drones, robots, and autonomous cars, molecular regression problems, and global weather and climate modelling.
The researchers have demonstrated the computational efficiency, numerical accuracy, and effectiveness of spherical CNNs when applied to 3D model recognition.
You can view the research paper here.
Continuous adaptation via meta-learning in nonstationary and competitive environments
This paper was a join effort between researchers from UC Berkeley, OpenAI, UMass Amherst and CMU. In our quest towards true artificial intelligence, the ability to continuously learn and adapt from limited experience in non-stationary environments is considered a massive milestone. The researchers have developed a gradient-based algorithm that adapts in complex dynamically changing scenarios.
Not only that, they’ve also developed a new multi-agent competitive environment that they’re called RoboSumo.
You can view the research paper here.
Other Resources
This leaves 334 more papers that were covered at the conference! You can find the full session list here. Each session includes the corresponding research paper.
If you like consuming your content in the form of videos, the recordings are available on ICLR’s Facebook page here.
Let us know your thoughts on this conference in the comments section below!
Subscribe to AVBytes here to get regular data science, machine learning and AI updates in your inbox!
Senior Editor at Analytics Vidhya.Data visualization practitioner who loves reading and delving deeper into the data science and machine learning arts. Always looking for new ways to improve processes using ML and AI.
