AI + Short Videos: The Next Wealth Creation Trend?
-
Recently, a short video using AI face-swapping and dubbing quietly went viral online.
In the video, two actors from a domestic short drama were transformed into European and American appearances through AI face-swapping and dubbing, and even spoke fluent English.
Although AI face-swapping and dubbing are not technically groundbreaking, as a new strategy for short drama globalization, they have sparked widespread discussions.
In November, the globalization of short dramas has become one of the hottest content trends.
On one hand, the overseas short drama app "ReelShort" under Chinese Online has gone viral abroad, even surpassing TikTok to top the U.S. iOS entertainment charts, with revenues approaching those of the streaming giant Netflix.
According to business and technology media Techcrunch, on November 11 alone, ReelShort's net revenue after deducting app store fees was $459,000 (approximately RMB 3.28 million). By November 17, ReelShort's cumulative net revenue had exceeded $22 million (approximately RMB 160 million).
On the other hand, as domestic regulations on the micro-short drama industry further tighten and competition intensifies, many companies and entrepreneurs are shifting their focus overseas.
Going global with short dramas has become a major trend. However, constrained by production costs, filming resources, and other factors, many players' first step into overseas markets often involves testing the waters with AI-translated domestic short dramas (i.e., dubbed dramas).
Now, AI face-swapping short dramas offer another option, seemingly capable of both reducing costs and achieving visual localization. Online rumors suggest that by using AI face-swapping, domestic short dramas can save hundreds of thousands in production costs while attracting overseas audiences.
From dubbed dramas to AI face-swapping and dubbing, have the first batch of players using AI for overseas short dramas made money? What are the current applications of AI in short dramas? "Top AI Players" spoke with frontline practitioners about this.
As the domestic leader in exporting short dramas, ReelShort initially had poor ROI (Return on Investment) when it entered the European and American markets. According to industry insiders, this was related to ReelShort's content model at the time—simply and crudely translating domestic short dramas for overseas audiences without considering cultural background differences.
Through discussions with frontline practitioners, "Top AI Player" found that most are pessimistic about the performance of domestic short dramas adapted for overseas markets through AI face-swapping or dubbing. Whether it's translated dramas or AI face-swapping with dubbing, they are merely 'changing the skin but not the bones,' failing to address the core issue of cultural differences.
So, who is actually in the business of AI face-swapping?
"Customers coming to us are all pursuing extreme cost performance." Youfang Technology CEO Fang Wei said this when talking about the company's latest short drama business.
According to the source, the company entered China's short drama market in May this year, initially focusing on traffic investment. Recently, with the explosive popularity of short dramas overseas, their earlier experience in digital human business enabled the team to quickly accommodate a surge in client demand for AI face-swapping and voice-changing services.
"The short drama business has grown severalfold. While the unit price isn't high, the increased demand has significantly improved overall revenue," said Fang Wei. Compared to the rumored AI face-swapping quotes of 50,000 to 100,000 yuan for 100 minutes, he revealed to "Top AI Player" that AI face-swapping with voice dubbing has now become so competitive that it costs only a few dozen yuan per minute. This means the cost of AI face-swapping and voice-changing for a 100-minute short drama can now be controlled within 10,000 yuan.
The underlying logic is quite simple. At present, when the overseas market effect of AI face-swapping is still unclear, only by lowering the price can more hesitant players be attracted to try it out.
As for whether there is a difference in the effect of AI face-swapping at different prices, from the perspective of Fang Wei, who comes from a technical background, the price difference does not affect the final video effect unless there are multiple characters appearing in the video, which increases the difficulty of face-swapping.
The specific pricing will vary depending on factors such as the difficulty of face-swapping and the refinement level of the video.
According to Fang Wei, the industry generally uses open-source DFL for training face models in face-swapping, HeyGen for lip-syncing, while some also employ open-source algorithms like wav2lip.
HeyGen charges dozens of yuan per minute, with a common workaround being to repeatedly register new accounts using different email addresses, as each account comes with 1 minute of free usage. Therefore, the overall cost for technical providers is actually quite low.
DFL, short for DeepFaceLab, is a renowned open-source face-swapping project. The DFL face-swapping process mainly consists of five stages: converting video to images, extracting faces, training the model, replacing faces, and synthesizing the video.
Among these, the most critical steps are extraction, training, and synthesis.
Additionally, "Top AI Player" has observed that some AI professionals use AI painting tool Midjourney to generate multi-angle virtual character images for training face models.
"The advantage of this approach is avoiding copyright concerns. If using real human images, authorization would be required," explained Fang Wei.
HeyGen is an AI video and digital human generation platform that supports features like lip-sync, one-click translation, and multilingual video content creation.
Compared to DFL face-swapping, HeyGen has a lower entry barrier. Earlier, many netizens' creative works such as "Guo Degang speaking English" and "Taylor Swift speaking Chinese" helped HeyGen gain rapid popularity in China.
Beyond creating AI face-swapping short dramas, as a technology provider, Youfang Tech also assists clients in developing cross-border short drama platforms and provides related drama resources. Their currently served overseas short drama platform Jowo has recently achieved daily revenues ranging from five thousand to ten thousand US dollars.
Jowo primarily targets the US market, but most dramas available are AI-translated domestic productions, leading many platforms to face challenges with poor traffic conversion. Zilong, Jowo's ecosystem director, told "Top AI Players" that the company has been attempting to introduce high-quality content from copyright holders, but faces tough competition from industry leaders.
Currently, the short drama overseas expansion track is crowded with "super players." In addition to leading online literature companies going global, such as Chinese Online, New Reading Era, and Dianzhong Technology, there are also short drama companies like Jiuzhou Culture that have successfully established a profitable short drama model domestically, as well as platforms like Kuaishou and iQiyi that possess video production and distribution capabilities.
Therefore, Youfang Technology has recently started investing in locally produced short dramas overseas. Fang Wei admitted frankly, "We were really forced by the shortage of content and had to dive into it."
In their prediction, if the content were of higher quality and the traffic conversion were better, the daily turnover performance would be even better.
Jowo platform's newly released hit drama
When discussing the much-curious "overseas short drama user profile," Fang Wei revealed that the majority are women over 26 years old with strong purchasing power. "A single drama costs $30, and some people can spend over $800 in a single day."
In Fang Wei's view, although the quality of dramas currently on Jowo isn't high, there are indeed viewers who watch and pay for them. Compared to Southeast Asia, the purchasing power of consumers in the U.S. market is evident.
According to earlier reports from Xia Guang She, the popularity of ReelShort has spurred many short drama teams to focus on the European and American markets, with the U.S. being the most profitable country in terms of paid content.
However, genres that perform well in the Chinese market—such as rebirth, CEO romances, and sweet romances—often have lackluster performance in Western markets. Industry insiders summarize that Western audiences prefer werewolf themes, strong female revenge stories, and visually impactful works.
For example, ReelShort's first platform hit short drama 'Fated to My Forbidden Alpha' is a werewolf-themed series.
Correspondingly, the production costs for short dramas that adopt localized overseas production, like 'Fated to My Forbidden Alpha', have also risen significantly. According to short drama industry investor 'Titan', the production cost for a 70 to 90-episode urban romance drama by a purely overseas team ranges between $130,000 to $150,000, while the per-episode cost for popular werewolf-themed hit dramas can even rise to $2,100 to $2,300.
In China, the cost of short dramas can typically be controlled between 100,000 to 300,000 RMB.
Meanwhile, industry sources indicate that even without considering costs, due to high barriers in overseas filming and difficulties in resource coordination, leading players like ReelShort can only maintain an update frequency of two dramas per month, with the entire platform having only 21 short dramas in reserve.
Content is in short supply, and the overseas market is booming. For small and medium-sized players like Jowo, using AI to translate short dramas and even employing AI for face and voice swapping seems more like a 'last resort' to seize market opportunities.
Fang Wei told 'Top AI Player' that after starting to produce content, he found that friends involved in short drama production mostly stick to traditional production methods, with 'almost no involvement of AI.'
Regarding this new trend of AI face-swapping short dramas, Fang Wei believes it fundamentally comes down to cost issues. "If the content quality is poor, the conversion rate will suffer, making it not worth doing. When costs are low, people are still willing to give it a try."
Currently, based on client feedback he's received, the advertising performance of AI face-swapping short dramas is "mediocre at best"—far from the rosy claims circulating online about "saving hundreds of thousands in production costs while capturing overseas audiences."
From ChatGPT and Midjourney to the trending AI video generation tools like Runway and Pika, generative AI is gradually transforming traditional content creation.
Although the application of AIGC technology in the short drama field is still in its early stages, a considerable number of practitioners are exploring its potential. In addition to the previously mentioned AI translation and AI face-swapping, scriptwriting and distribution of short dramas are also areas where AI is making an impact.
"For scriptwriting, ChatGPT4 is generally used, while some teams also employ domestic models like Wenxin Yiyan," said Fang Wei. His peers in scriptwriting utilize ChatGPT to boost efficiency, with completed scripts being sold commercially. Even custom script orders now incorporate ChatGPT-assisted creation.
The parent company of ReelShort, a leading player in overseas short drama production, Chinese Online, launched its Zhongwen Xiaoyao Large Model on October 13 this year. This AI model provides comprehensive creative cycle assistance for writers - from story conception and plot arrangement to content writing, character dialogues, illustration production, and content evaluation.
Chinese Online stated during a late November research session that "Chinese Xiaoyao" has been utilized for short drama script creation. As the large model continues to upgrade, its applications will extend to producing high-quality dialogues and scripts.
In terms of distribution, Youfang Technology is internally developing a "matrix system" that currently enables over 3,000 accounts to automatically maintain profiles, post comments, and upload videos. Fang Wei revealed that to reduce editing costs for short drama distribution, they are developing an "automatic slicing" feature with large model participation for content delivery.
Coincidentally, to support the globalization of short dramas, Jiuzhou Culture (hereinafter referred to as Jiuzhou) has recently independently developed an in-house AIGC system. Wang Jiacheng, the founder of Jiuzhou, stated in an interview with the media that this AIGC system can automatically generate subtitles for micro-short dramas, enabling batch production.
Meanwhile, Jiuzhou has divided the AIGC system into three phases: Moss1.0 automatically generates multilingual subtitles; Moss2.0 addresses the challenges of cross-language dubbing; and Moss3.0 enables AI to directly generate short drama content based on scripts. Unlike the aforementioned AI face-swapping technology, Moss3.0's envisioned capability of AI directly generating short dramas requires the use of AI video generation technology.
Since the second half of this year, the field of AI video generation has developed rapidly. From Runway Gen2 to the open-source Stable Video Diffusion, and the dark horse Pika, it's clear that after making waves in text and image generation, many AI entrepreneurs have now set their sights on video as the next mainstream medium.
When it comes to AI-generated short drama videos, it's not just Jiuzhou. Recently, a large model video generation team called "CreativeFitting (Jingying Technology)" has also set its goal on "using AI to generate short drama content based on scripts." The team has raised millions of dollars in funding and plans to enter the overseas short drama market, developing an AI short drama App.
According to 36Kr, an AI-generated short drama video by CreativeFitting has gained millions of views on social media platforms.
Regarding the prospects of AI short dramas, the lead investor, NineSigma Venture Capital, believes that the advantage of AI short dramas lies in improving video generation capabilities, thereby reducing production time and costs, but the key is whether the AI generation technology is refined.
In practical applications, whether it's yesterday's star product Gen2 or today's hot favorite Pika, AI-generated videos currently suffer from output instability issues such as screen jitter, flickering, frame drops, as well as two major challenges: controllability and video duration limitations. The industry still has a long way to go before AI can fully automate the production of high-quality short dramas.
What's certain is that AI technology is already widely used in the production process of short dramas, with AIGC involved from content creation to distribution and promotion. With the "one AI day equals one human year" speed of AI development, perhaps we'll soon be watching short dramas entirely created by AI.