AI Boosted Drones To Capture More Emotional Cinematic Shots

Not Going Away

We live in a world of drones now. These remote aerial vehicles, many of the more high-tech versions equipped with such things as heat sensors, thermal infrared video cameras, cameras capable of live feeds and GPS trackers, are the new modus operandi of 21st-century surveillance.

Don’t like it. Tough. You better get used to it as they aren’t going away.

Growing in intelligence on what seems like a weekly basis, and powered by AI that is only set to continue throughout the decade and beyond, this technology will be transformative.

And that transformation could be a boon for sectors like the film industry: when used right drones can enhance video production ten-fold, making an inane shot look like it was dreamed up by Janusz Kaminski. To do that, however, countless hours of practice are required in order to use the very costly equipment correctly for the perfect take, which isn’t easy.

The Mood of Drone Flights

“Sometimes you just want to tell the drone to make an exciting video,” said Rogerio Bonatti, a doctoral candidate in the Robotics Institute at Carnegie Mellon University (CMU), in an interview with CMU’s Aaron Aupperlee. Bonatti, together with researchers from CMU, the University of Sao Paulo (USP) and Facebook AI Research, are collaborating on an AI model which enhances the cinematographic mood of drone flights for a specific shot.

“We are learning how to map semantics, like a word or emotion, to the motion of the camera,” added Bonatti.

So how did the team manage to map semantics?

After collecting hundreds of drone videos representing different camera angles, flight paths and speeds, the team then asked thousands of viewers to describe how the videos made them feel in an emotional sense. From this, the data was fed into the AI model which began to learn what shots induced certain kinds of emotions in the viewers. The drone could then be told to use those kinds of techniques on command, which were not commands in the usual imperative sense but more, for example, to make an “interesting,” “nervous,” or “enjoyable” video on request.

“I was surprised that this worked,” said Bonatti. “We were trying to learn something incredibly subjective, and I was surprised that we obtained good quality data.”

Opening Doors

That an AI model can induce something totally subjective is a thing in and of itself, and can bring the rise to more intuitive systems, not just in cinematography, but other creative industries, too.

“This opens the door to many other applications, even outside filming or photography,” added Bonatti.

With further explorations into utilizing the same approach to orchestrate robot behaviour, which the team has already designed a model for, this new drone AI technology could mean film creatives may have a new rival in fashioning that perfect Stanley Kubrick panoramic shot.

Leave a Comment