Overview
- MIT’s CSAIL researchers have unveilved an approach that automates certain parts of image editing, including object detection
- The approach is called Semantic Soft Segmentation (SSS)
- It combines the color and texture of images with information produced by a trained neural network
Introduction
Fixing corrupt or bad images, filling the gaps in existing images, translating the background from one image to another – these are all applications of computer vision that have transcended imagination to become a reality this year. And now researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) have thrown their hat into this ring with their latest study.
They have unveiled an approach called ‘Semantic Soft Segmentation (SSS)’ that uses machine learning to automate certain parts of the image editing process, including detecting objects! What takes an expert editor several minutes (or even hours) and involves tweaking and analyzing images pixel-by-pixel can now be done in a matter of seconds thanks to SSS. The below image shows how the algorithm works to detect objects in a given image:
SSS works by analyzing the texture and color of the given image. It then combines these attributes with data provided by a trained neural network model that information about what kind of objects are present in the image.
As mentioned in the research paper (link below), the algorithm “generates soft segments that correspond to semantically meaningful regions in the image by fusing the high-level information from a neural network with low-level image features fully automatically”. This makes tasks like parsing objects, editing backgrounds, etc. quite trivial and removed the need for expertise (at least as far as casual users are concerned!).
While the current version of SSS is heavily focused on static images, there is an acknowledgement by the researchers that this will be fine-tuned for video applications in the future.
I have mentioned a few resources below to help you get acquainted with this study and also try it out by yourself:
Also do check out the short video below which shows SSS in it’s full glory:
Our take on this
Another week, another breakthrough study in computer vision. Deep learning has carved a niche for itself in this field and you can expect to see more and more of these projects coming soon, especially in video editing. CGI effects that we see in movies could easily be done using techniques like SSS (once they have been improved a bit more).
Should expert artists be worried? Deep learning does have the potential to generate a decent work of art but that human touch and intuition continues to elude even the finest works of machines.
If this field interests you, you can take our Computer Vision using Deep Learning course which aims to help you dip your toes and come out a master in this exciting and upcoming field.
Subscribe to AVBytes here to get regular data science, machine learning and AI updates in your inbox!
Senior Editor at Analytics Vidhya.Data visualization practitioner who loves reading and delving deeper into the data science and machine learning arts. Always looking for new ways to improve processes using ML and AI.
