Note to the reader:
We wrote this comparison the day OpenAI released Sora (its own text to video tool), so this already feels outdated. But I guess that shows you how quickly this industry is moving and why it’s important to work with a video team who are staying ahead of the game.
Sora looks like a massive step forward in the text to video arena, although it’s not open to public use yet. But as a bonus we’ve included a Sora example at the end to show you how far and how fast things are moving.
Pika vs Runway
Generative photography has seen a whole bunch of companies and models competing for our attention (Dalle-3, Midjourney and Imagen are probably the three most talked about) – and we’re already seeing that come into play when looking to generate mood boards and pitch docs to flesh out ideation and creative concepts.
But AI video? It’s still very much the new kid on the block.
To give you an idea, this is where we were at about a year ago, in early 2023:
There are tools like Synthesys and Visla if you want to create digital avatars (still spooky in my opinion!) and turn scripts into video (pulling in existing stock to make a video). And let’s not forget Meta’s Emu Video (an AI model) – a gen AI tool with huge potential for both Instagram and Facebook. But despite big strides over the last two years, gen AI for video is still very limited.
That being said, it didn’t stop me wanting to put my prompting skills to the test after spending an evening watching back-to-back sci-fi and attempting to recreate a scene from Interstellar.
Ambitious, sure.
Sora aside, the two gen AI video tools that are really making moves now are Runway and Pika. Both are impressive AI text to video generators creating video using prompts (text, image, or both), and both are relatively easy to get started with.
Considering I’m still a newbie, I decided to go with a basic text prompt to see how far I could get even with the most basic of descriptions.
PROMPT: Aliens in space, satellites, meteor shower, planets and the sun.
(Told you)
Runway generated video
Pika generated video
Despite my simple prompting, I was impressed. No aliens, but the rest was there in some shape or form. 3D assets, relatively high production value – albeit a little glitchy here and there. The thing that’s really impressive of course is just being able to type a few sentences and get video. And text responses from GPT-4 in comparison to GPT-3 seem to be more accurate and useful which helps with the overall output.
But who’s doing it better right now?
Prompting:
Pika shines when it comes to prompting and customisation, responding better and attempting to follow prompts more closely. Even when you throw in more ideas, it’ll attempt to incorporate them. Pika also offers various input options, such as text and image prompts, which gives users the freedom to experiment with different ideas and see them come to life. This versatility makes Pika Labs a great choice for users looking for more control and input in the creative process.
Runway, although super cinematic, isn’t as flexible or customisable. It’s harder to get your prompts to align with what you’re asking for. But you do get a dynamic mix of camera shots that replicate more natural human movement, adding a level of realism to the outputs.
Animation:
I’m putting it out there now. When it comes to replicating human animation and movement, neither are that good (yet). And they both distort faces. It’s really freaky.
We found that Pika’s animations of human motions are less cinematic but more adventurous. It attempts something more exciting, although the outputs aren’t particularly accurate or smooth.
Runway also does the weird, distorted thing, but it does produce high-quality animations with a wide range of human motions, keeping things accurate when it comes to physics and body proportion.
That’s why we stuck to the space theme – no faces.
Camera:
Now both are pretty decent at incorporating zooms, pans and rotation. I’d say Runway pips this one generally as it’s able to be more dynamic – plus, you can combine different camera movements, making things much more interesting.
In conclusion, Pika Labs and Runway Gen 2 each have their strengths and weaknesses. While Runway excels in cinematic human motion and offers dynamic camera shots, Pika showcases greater creativity and flexibility in responding to user prompts.
But here’s the thing: the knowledge required to make a great video still rings true. Use great camera kit, know your way around production tools, tell a story, suggest a strong narrative flow. All of this is still very much key to creating a beautiful video.
Gen AI video models still have a way to go – although it’s going to be really interesting to see what happens with Sora this year. It’s already in the hands of red testers.
Talking of which, this is the kind of thing Sora’s capable of:
What these models are achieving right now is being able to break down the barrier to entry when it comes to video production. They’re enabling everyone to be the director of their own ‘stories’ and to bring out the creator in each of us (and for free, if you keep it basic). In my opinion, that can only be a good thing.
Written by Sian Evans, Head of Production at Definition