This magical AI from scientists at the Weizmann Institute of Science and NVIDIA which promises magical video editing powers

Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér.

Today we are going to have a look at this magical AI from scientists at the Weizmann Institute of Science and NVIDIA which promises magical video editing powers.

So let’s see 4 of my favorite things it can do together.

One, we can take this unremarkable video of a car passing by, and my goodness, look at that.

It has completely revamped the scene: first,it made sure that it is nighttime, and has given the car a nice neon cyberpunk look.

So, this took quite a bit of work for someone,right?

Well, not quite!

To achieve this, all we need to do is give
the AI a video and two text prompts.

One for the foreground, and one for the background.

And if we are looking for an example that
is a little less out there, we can also ask

that the car is made rusty instead.

So cool!

Two, once again, if we did not show up in
the right outfit for a day, or would like

to see what we would look like wearing something
else before buying it, look no more.

Yes, this is also just one text prompt away.

I particularly like this example because this
outfit has lots of wrinkles and geometry changes,

and the new technique can follow them really
well.

Now, if you look close, you see that there
is also a price to be paid for this.

Look, the new synthesized results are not
as detailed as the original video.

Now, we said that it can do foreground and
background editing separately.

And that was the foreground.

But what about the background?

Oh yes, let’s make this environment a little
foggier.

Look at that!

I love this one.

Even the shadows have become foggier.

So cool!

And, the shadows are even taken away for the
nighttime shot because of the different lighting

setup.

I would like these results a great deal for
images, but for video?

Such an amazing result for video.

Wow.

Now, of course, this kind of layered editing
concept can be applied to images too, and

the results are outstanding.

For instance, three if we have this boring
cake, we can reimagine it if it were made

of oreo instead, or if we wish to let our
imagination run wild, or we are light transport

researchers, we can ask the AI to make it
out of ice.

Loving it.

Now, if you aren’t hungry enough yet, here
is a piece of bread that can be converted

to the legendary red velvet bread too, or,
once again, ice.

Four, the authors call these semi-transparent
effects where we can choose a small part of

an already existing image, add a text prompt,
for instance, fire, and then, bam!

We suddenly have a fire-breathing bear, or
a beautiful cup of coffee.

This technique can truly do so much.

Now I would like to send huge thank you to
the authors of the paper because the source

code for this project is also available.

So, AI-driven layered image editing is possible
for free, and for everyone!

And just imagine what this technique will
be able to do just a couple more papers down

the line.

What a time to be alive!

So, what do you think?

What would you use this for?

Let me know in the comments below!

Thanks for watching and
for your generous support, and I’ll see

you next time!