The browser is the most common end-point consumption of Deep learning models. It is also the most ubiquitous platform for programming available. The maturity of the client-side JavaScript ecosystem across the deep learning process e.g. dataframe support (arrow.js), WebGl accelerated learning frameworks (tensorflow.js), declarative interactive visualisation (vega-lite.js), have made it easy to start playing with Deep Learning in the browser.
This hack session is designed to learn how to do deep learning in the browser. We will focus on exploring three particular use cases where deep learning models can be used for inference, training and explanation in the browser directly.?
Introduction: We will start with a quick introduction to javascript libraries including tensorflow.js for the session. We will introduce and use a browser based interactive notebook – observablehq.com to run all our code.
Model Inference: Inference is the most common use case and the browser allows you to bring your DL model to the data. It also allows you test how the model works, when executed on the edge. We start with a simple application to load a pretrained model for image classification and run it on the browser. We will understand some browser specific optimization – like weight quantization needed to manage models on the client side.
Rapid Prototyping: Training of DL models is now possible in the browser itself, if done smartly. We will build a rapid prototyping image classification example which allows the user to play with transfer learning to build a model specific for a user- generated image input. We will understand some nuances on data loading, preferred layers and memory management required for browser training.
Explorable Explanations: Explaining the DL model and allowing the users to build intuition on the model helps in generating insight. We will build an explorable explanation for our image model, which allows the user to explore the feature space and threhold boundaries using interactive visualisations to understand what the model has learnt.
Conclusion & Way Forward: We will end the session with how we see the ecosystems of tools for DL in the browser emerging and making it easy for everyone to start doing this.
To follow along the hack session – you would need a desktop with a modern browser (e.g. Chrome, Firefox) and a Github id to save your work on notebook.
Amit Kapoor teaches the craft of telling visual stories with data. He conducts workshops and trainings on Data Science in Python and R, as well as on Data Visualisation topics.
His background is in strategy consulting having worked with AT Kearney in India, then with Booz & Company in Europe and more recently for startups in Bangalore. He did his B.Tech in Mechanical Engineering from IIT, Delhi and PGDM (MBA) from IIM, Ahmedabad.