AI Recognition Technology is Transforming Global Streaming Platforms

[baoshi.rao]

The article explores the applications of AI recognition technology, primarily in video recommendations and targeted advertising, and shares insights into its future development.

Honestly, there are just too many high-quality online variety shows these days—it's impossible to keep up! My time has been completely consumed by workplace dramas, detective shows, language programs, and acting competitions. Today, let’s talk about these streaming platforms.

I’ve always believed that Chinese internet companies excel in "implementation." Compared to their international counterparts, domestic tech giants always introduce innovative features that lead the world. Over the weekend, while watching iQIYI’s The Rap of China, I noticed a new feature called "Scan."

There’s a scan button next to the video. At first, I wasn’t sure what it did. After clicking it, I realized it scans the content, performs facial recognition, and identifies the people in the video while providing related encyclopedia information.

Wow—this perfectly solves a common viewer question: "Who is this person?" Technically, this is quite challenging. Unlike 1:1 facial recognition at airports, where you face the camera directly, here the system must identify people based on partial side profiles, like recognizing Dan Zong and Chong Zai even when they aren’t looking at the camera.

In my view, innovations like "Scan" do more than just enhance user experience. This small feature reflects the intense technological competition among global streaming platforms and how AI connects people, scenes, and objects through recognition.

"Scan" is just one example of AI’s role in video. In reality, AI permeates every stage of video production. For AI to be fully utilized, the first step is recognizing video content—without this, nothing else is possible.

Let’s examine AI’s most complex application: creative production. In July, Netflix disclosed in its earnings report that it’s researching AI-generated trailers. Soon, Netflix will be able to quickly and efficiently produce high-quality trailers for vast content libraries. This reduces editing costs while personalizing trailers to boost click-through rates and watch time.

Back in 2016, 21st Century Fox and IBM used the Watson system to generate a trailer for the thriller Morgan, but Netflix’s approach scales massively. The underlying principle involves two steps:

1. Recognition: The system analyzes the full video, identifying actors (distinguishing leads from extras), scenery, objects (e.g., "park," "close-up at home"), and even emotions in dialogue and music.
2. Generation: Using predefined templates (e.g., "emotional" or "hilarious" styles for a Shen Teng comedy), AI assembles trailers tailored to user preferences.

The first step—recognition—is foundational. Once a video is "understood," the results can be reused for multiple applications. This is why streaming giants heavily invest in R&D. For instance, iQIYI has made strides in computer vision, with research published at conferences like ICCV and winning competitions. It also co-hosted the Multi-Modal Video Person Recognition Challenge with PRCV2018 and ACMMM, releasing the largest celebrity video dataset (iQIYI-VID) to advance industry-academia collaboration.

What’s multi-modal recognition? Simply put, it combines facial, voice, posture, and scene recognition to improve accuracy—even in low light, with heavy makeup, or side profiles. This cutting-edge AI tech powers features like "Scan."

After recognition, streaming platforms—bridging creators and viewers—unlock new possibilities. For viewers, AI enhances experiences:

• "Scan" identifies actors.
• iQIYI’s AI Radar displays star info.
• "Watch Only Them" (Tencent Video, iQIYI) lets fans binge their idol’s clips (e.g., Kun Kun’s basketball rap).
• Future ideas: "Skip Them" to exclude disliked scenes (e.g., flashbacks in Go Go Squid! or Chen Ke in The Thunder).
• Special optimizations: Tencent’s Kids Mode, iQIYI’s AI sign-language clips, Bilibili’s smart subtitle blocking.

For creators, AI aids in:

1. Casting: Systems like iQIYI’s ArtLink and Youku’s FishBrain match actors to roles (e.g., Zhang Xueying in Summer’s Desire, Liu Haoran in With You, Lei Jiayin in The Longest Day in Chang’an).
2. Editing: AI acts as an assistant—tagging scenes, emotions, and dialogue in reality shows (e.g., The Rap of China), slashing editing time by instantly locating specific clips.

Major platforms like iQIYI, Tencent, and Youku all invest in such tools (iQIYI Smart Edit, Tencent Cloud Edit, Youku’s Whale Edit).

Of course, AI's capabilities go beyond enhancing user experiences—its most critical role is helping platforms generate revenue.

How does AI help streaming platforms make money? Two key applications stand out: video recommendations and ad placements. Underlying both is the unifying concept of "matching"—connecting content with users and ads with users.

When users quickly find content they enjoy, they're less likely to cancel subscriptions. When users discover products they like, advertisers keep spending, keeping the entire ecosystem running. According to Netflix, 80% of its viewership comes from its recommendation engine. The platform aims for users to start watching within one minute to prevent them from closing the app.

Beyond standard recommendations based on user preferences and placement weights, Netflix uses AI to analyze video imagery and generate personalized thumbnails. For instance, if a show features multiple actors, Netflix identifies characters and uses the viewer's favorite actor as the thumbnail. This approach also applies to scenes and color schemes. For example, Stranger Things can generate countless thumbnails with different characters and scenes.

AI can also gauge users' moods based on viewing history and time, recommending tailored content. A movie might have both "romantic" and "thrilling" thumbnail variants. If you watch a horror film late at night, the platform might suggest a romantic cover to ease your nerves.

In China, platforms like iQiyi have rolled out personalized posters, identifying and pushing preferred visuals to users. In March last year, iQiyi's homepage introduced personalized long-video recommendations, offering a unique experience for each user.

Beyond subscriptions, ads are a major revenue stream for streaming platforms. Previously, mid-roll ads were randomly inserted, harming user experience and ad performance. Now, leaders like YouTube and iQiyi use AI to place ads at optimal moments, minimizing disruption. Recently, platforms have focused on aligning ads with video content. For example, iQiyi's "Band-Aid" ads use AI to analyze scenes, characters, emotions, and dialogue, displaying brand ads as overlays during playback. This contextual approach boosts both user experience and conversion rates. During the hit drama Story of Yanxi Palace, such ads were widely used and praised by advertisers.

iQiyi's new "Scan" feature hints at a new era of "TV shopping." By auto-identifying content and linking to e-commerce, viewers can instantly shop for items seen in shows—no more searching for "celebrity同款" (same as celebrity). This creates a seamless connection between viewers, products, and scenes.

Globally, YouTube tested personalized ad content in 2017 with its Director Mix tool, allowing advertisers to tailor ads to different users. For example, a soup brand saw a 55% boost in awareness using this method.

AI's role in streaming hinges on video recognition technology. Netflix and iQiyi exemplify this trend: heavy R&D investment (e.g., Netflix's $1M recommendation engine contest), followed by data-driven applications to enhance user experience and content production. The ultimate goal? Monetization through better recommendations, ad tech, and e-commerce integration.

In 2019, with Apple TV+ and Disney+ entering the market, competition intensified. Beyond original content and licensing, the battleground is AI—specifically, breakthroughs in video recognition and practical applications. In the 5G era, seamless streaming and advanced AI will squeeze every last drop of users' time. Are you ready?