Reinforcement learning could be about to get a whole lot simpler thanks to the release of a new library called ‘RLlib’. According to the authors, the goal of this library is to help the users in breaking down of the various components that go into reinforcement learning. This will make them more scalable and easier to reuse and integrate.
RLlib has been designed to support multiple deep learning frameworks and can be accessed using a simple Python API. It’s currently available with the below RL algorithms:
- Proximal Policy Optimization (PPO)
- The Asynchronous Advantage Actor-Critic (A3C)
- Deep Q Networks (DQN)
- Evolution Strategies
According to the official RLlib page, this library aims to provide both performance and composability:
Performance
- High performance algorithm implementions
- Pluggable distributed RL execution strategies
Composability
- Integration with the Ray Tune hyperparam tuning tool
- Support for multiple frameworks (TensorFlow, PyTorch)
- Scalable primitives for developing new algorithms
- Shared models between algorithms
The authors published a paper on this last month which can be accessed here. In the paper, they put forward the following:
“We argue for building composable RL components by encapsulating parallelism and resource requirements within individual components, which can be achieved by building on top of a flexible task-based programming model. We demonstrate this principle by building Ray RLlib on top of Ray and show that we can implement a wide range of state-of-the-art algorithms by composing and reusing a handful of standard components. This composability does not come at the cost of performance — in our experiments, RLlib matches or exceeds the performance of highly optimized reference implementations.”
You can access the GitHub library for RLlib here.
Our take on this
Next to deep learning, reinforcement learning is the most followed topic these days. RL learning has been previously described as a “data hungry” field. So scalability issues are inevitable with the amount of the data that is required to create RL models. RLlib will help in this regard as it aims to break down and scale the process for the user.
Senior Editor at Analytics Vidhya.Data visualization practitioner who loves reading and delving deeper into the data science and machine learning arts. Always looking for new ways to improve processes using ML and AI.
