There’s a new kid on the block. Kling, the system that can ‘hallucinate’ video fakery from text prompts, has come crashing into the AI zeitgeist, with notable AI creators and influencers claiming that the system is a serious rival to OpenAI’s Sora. Let’s have a look at what Kling is, how it works, and how you can use it. Kling is a generative model for text-to-video synthesis. Named after the character in the TV series Firefly, Kling doesn’t need human input to hallucinate realistic-looking video, and its videos look excellent. Kling is available to use either via the Dreamstudio app or online using nightcafe.ai. You can also use NightCafe’s NFT site to produce a video and receive a free NFT. Kling is still in beta and only works for English text to English video. How does it work? Kling was developed on a very large pre-trained model called DALL-E 2. This AI model knows a lot about the relationship between text and visual content creation. To generate video from a text input, the system works by generating ‘codes’ that are fed into a version of the BigBird AI model.
Kuaishou Technology, the ‘duke’ of the short video creation apps industry in China, has given birth to Kling, not only competing on an equal footing with Douyin – the Chinese version of TikTok – but also an ‘avant-garde’ model with 400 million daily active users that has reached benchmark figures. As an innovation born in the ferment of such intense competition, Kling is not only an AI video model, but also a technological leap forward, which is set to provide a new dimension for the toolkit of creators.
Left: an input text. Right: the resulting 1080p video produced by Kling, an AI system still in a trial phase of development that can convert text prompts into entirely convincing clips. All images courtesy of The University of California, Berkeley/Shakir Mohamed.It achieves this astounding capability by employing a 3D Variational Autoencoder (VAE), which is trained to recognise lifelike expression, movement and adherence to the laws of physics – all coupled with a 3D spatiotemporal joint attention mechanism.
You don’t need to have a Chinese phone number to try Kling yourself, though it might seem that way at first glance (requiring one to register an account). Users are now employing clever workarounds, including using burner phone numbers (via the KwaiCut app), or transcription of app pages via Google Translate, to start using the universe of Kling. For those with the patience to enter, there is much to experiment with.
From lethal car chases and first-person-shooter thrills to Westeros-style magic worlds, Kling has it covered. Whether you’re aiming for ‘intermediate’ fidelity or the cutting edge of AI-generated creativity, Kling accommodates, adapts… and delivers – with approximate race/skin colour being just one of your potential issues.
Kling’s arrival perpetrates a healthy nervousness among filmmakers and hobbyists, because simply put it makes other options obsolete overnight. It puts pressure on OpenAI and similar industry giants to think fast and adapt. Immediately, there’s a tangible response. Amateurs, YouTubers, startups and even huge Hollywood studios will reset their agendas as the race to match – or top – Kling’s quality and resolution accelerates the evolution of AI video models. We can only imagine how the competition will heat up. But two things are clear: Kling is not business-as-usual; far from just an AI tool, it’s a filmmaking innovation catalyst.
Kling currently limits access to users with a Chinese phone number, but the demand for wider availability suggests his audience could become international. But should other AI video model providers step up to the plate to fill the gaps in Kling’s offerings, and eventually dominate the scene? We’ll have to wait and see – but the implications for AI filmmaking and content creation are significant.
Kling stands today as the pinnacle of AI-generated video manipulation. And it stands as testimony that the future of artificial intelligence imagery is not simply to emulate reality – it is to craft it in ways never before thought possible. Welcome to the new world. The future of video creation, enabled by Kling and others like it, is bright. It is bold. It is endless.
Although this requirement for an authentic Chinese phone number sounds like a barrier to entry, it’s also a symbol of what Kling can do for you when you gain access – either through a workaround or when it eventually becomes accessible everywhere. Your phone, then, becomes your way into Kling’s unlimited sources of creative possibility. Since it’s likely going to become a part of your creative workflows as this AI video-maker continually improves, it’s worth knowing how to use it and that it’s there to help.
More Info:
© 2024 UC Technology Inc . All Rights Reserved.