Google’s New Machine Learning Service Makes Videos Searchable

How about a readymade machine learning system that would let you extract actionable insights from the videos you have stored in Google Cloud Storage without you having to learn anything about computer vision? Well, Google intends to do exactly that with its Cloud Video Intelligence REST API.

Simply upload a video and let Google scan it for the presence of more than 20,000 entities. What are these entities? An entity is basically nothing more than a noun – think a cat, or dog, or car… So, while on a broader level the API will be able to detect what the video is all about, on a more granular level, it will label all the elements it finds in each scene of that video.

According to a release by Google, the machine learning model is now able to distinguish between 180 different types of fruit (from apples to jujube), 229 airplane models, 667 vehicle models, and more than 200 types of buildings (church, supermarket, stadium, etc.).

The system will create an intelligent catalog of your videos which you can use to search for content just like you would have searched a document. So, if you want to find all the videos that feature a birthday cake, all you need to do is simply search for that term.

How can an enterprise use Google’s video intelligence API?

The immediate use case that comes to mind is that of a media house. Publishers often have hundreds of petabytes of video content just lying around in storage, and searching their vast library for a specific type of entity is no easy task.

Uploading these videos in Google Cloud Storage and looking up a specific term like ‘football’ will not only tell you how many videos contain the shot a football but also exactly where in those videos that entity has appeared. Nobody needs to watch the entire video just to look for one tiny scene. So, if an enterprise has a large library of video content, it can make sense of all that content within seconds.

If you are moderator looking for inappropriate user-generated content, you can instantly sift through tons of data. You can also build a stream of recommended content using the labels created by the video intelligence API and matching them with a user’s viewing history.

Google has also released a video transcription service for this tool, but that is in private beta mode at the moment.