A computer is only as powerful and intelligent as we build it to be. Engineers and tech enthusiasts all around the world have been trying to tackle one of the biggest challenges with AI: teaching these systems to understand the content of videos as well as humans do.
Not at all a simple task, but the team behind a certain navy blue social media platform just might crack this case before anyone else does.
Facebook Details Its Newest Video-Focused AI
Facebook has announced that it’s launching a new project, appropriately named “Learning from Videos,” which is designed to recognize and understand audio, textual, and visual components of videos that are publicly posted to the platform.
If you’ve ever watched a YouTube video with auto-generated closed captions on, then you’d know that AI typically doesn’t do a very good job at registering this information. Facebook’s latest initiative, should it go as planned, could very well be one of the biggest breakthroughs we’ve seen so far in the development of machine learning.
Facebook says this project will help it improve its core systems and power entirely new applications. The post continues:
By learning from global streams of publicly available videos spanning nearly every country and hundreds of languages, our AI systems will not just improve accuracy but also adapt to our fast-moving world and recognize the nuances and visual cues across different cultures and regions. And by helping AI researchers break away from the reliance on labeled data, we can improve AI-powered products and create entirely new experiences.
How is this possible? Facebook says it’s using Generalized Data Transformations (GDT), a self-supervised system that “learns the relationships between sounds and images.”
So far, Facebook has only applied GDT to the recommendation system for Instagram Reels, but claims to already see significant improvements. The self-supervised models learn themes from the videos, group similar themes together, and filters out near-duplicates before recommending them to you.
Whose Content Will Be Used to Train GDT?
If you use Facebook or Instagram and post videos set to Public, then your content may be used to help train Facebook’s latest AI project. A Facebook spokesperson told VentureBeat that its data policy states that the platform “uses the information we [Facebook] have to support research and innovation.” By using either app, you agree to these terms.
What’s Next for Facebook?
“People should be able to recall specific moments from their vast bank of digital memories just as easy as they capture them,” Facebook says. The hope is that as the AI is continuously worked on and upon, the platform’s systems will grow to become more reliable, efficient, and personalized.
It does seem as though the efforts of Facebook’s engineering team have produced the desired results so far. Their hate speech-detecting AI, for instance, is reportedly massively reducing hateful content on the platform.
