Overview
- A group of Microsoft Interns have used AI to add intelligent insights to screenshots on Windows 10
- The application is called Snip Insights and has been open sourced on GitHub
- It can convert images to editable text, recognize faces and places in screenshots, etc.
Introduction
Imagine this – your manager asks you to take any product in your organization and add the power of AI to it to make it a commercial release. That’s just what a group of Microsoft interns have done. Challenged by their manager to integrate AI into a widely used tool, these interns took up the challenge and transformed the way you take screenshots on Windows 10.
The name of this application is Snip Insights and has been developed under the Microsoft Garage project umbrella. It is an open-source desktop application that enables Windows users to get intelligent insights from a simple screenshot! It uses Cloud AI services to perform many tasks, including converting images to translated text, automatically detecting and tagging your image content, etc.
Snip Insights has the capability of taking a scanned image of a document, read it, and convert it into text. Additionally, it can also recognise celebrities, other famous personalities, landmarks and places that are captured within a screenshot. Imagine you see a cool pair of sunglasses online. Instead of wondering where you buy them and searching e-commerce sites, you can take a screenshot and Snip Insights will list down places where you can buy them. How cool is that?
When the manager laid down the challenge, the interns were given tons of resources to figure out how to use AI. It was daunting challenge from a technical viewpoint but through trial and error, they managed to bring down their options to 2 tools – Windows Snipping Tool and Snip. They decided to go with the latter as it offered an easy route to building Snip Insights as an independent app. Snip is built on the WPF platform using C#.
You can read the official Microsoft blog post about it here and access their GitHub repository here.
Our take on this
While some of these functionalities are offered by Pinterest, Snip Insights offers other features packed into one release. This will save a lot of time and effort for people looking to quickly edit text in an image (or even a PDF!), search for products online, etc. It’s a very interesting example of how AI can change the way you use everyday applications.
They have also open sourced the project on GitHub in the hope that developers and even students around the world can take inspiration and improve on these efforts.
Subscribe to AVBytes here to get regular data science, machine learning and AI updates in your inbox!
Senior Editor at Analytics Vidhya.Data visualization practitioner who loves reading and delving deeper into the data science and machine learning arts. Always looking for new ways to improve processes using ML and AI.
