Artificial Intelligence Restores Einstein’s Photos

Today we are going to take these old low-quality photos, and ask an AI what it thinks they should look like.

And yes, we are going to see a restored version of this photo of Einstein, and this one too.

And the results are absolutely amazing.

In fact, I could hardly believe some of the results so I had to check them over and over again.

So, what is the problem here? Well, we have one input photo, which is a blurry mess, and when we try to restore it with a previous method, the output is…also a blurry mess.

This was DFDNet, a research paper from 2020.

Surely, something better must be out there by now.

And, yes there is! This is GFP-GAN, a technique from 2021.

This is significantly better, but the hair is full of artifacts, and the photo itself is still not that sharp.

Now, have a look at this.

This is GPEN, also a paper from 2021, and this is much better than its predecessors.

But if you think that this image has detail, now have a look at the new technique.

Whoa, this is so much better.

The hair has tons of detail, the glasses finally look right, and now that we are comparing the two side by side, we see an interesting effect.

The previous method created an idealized image where a lot of wrinkles and other high-frequency effects are now gone.

And believe it or not, this aspect of the new AI is controllable.

I’ll tell you about it in a moment.

Now, this new technique is so good, we can even use it on old photos, and get this, even movies.

Due to copyright issues, I can only show you still images from here, but I made sure to put the full videos in the video description so you Fellow Scholars can have a look.

I will note that I have found some temporal coherence issues, in other words, some jumpy behavior, but it was very mild.

I think many people wouldn’t even notice that an AI restoration algorithm was at work here.

And I know what you are waiting for.

Now, it’s Einstein time.

I will say in advance that we are going to talk about what this new AI thinks Einstein looked like.

We cannot be for sure.

Now hold on to your papers and let’s see.

Whoa.

This is incredible image restoration, and the technique delivers amazingly well over and over again.

Now, once again, we cannot say that this is exactly what Einstein looked like, but I feel like with every paper, we get closer and closer.

And here, I will try my best to show you the strengths and weaknesses of this algorithm so you can decide for yourself.

Now, I promised that we are going to talk about identity preservation.

So let’s have a look together at this.

We can use this parameter before restoration, this preserves the identity of the subject, at the cost of some additional quality.

Or, if we are looking for a super-high quality output, and are okay with the results not looking like exactly the same person, we can also do this.

Yes, this gives us a super-high quality image, but the magnitude of the changes can be such that we get a different person.

Now, my first thought was that well, what is the point of this? Why would anyone use this? But then it struck me.

Are you thinking what I am thinking? Look! Got it? We can use this on synthetic characters created by an AI.

These are sometimes highly imperfect images, and in this case, we can supposedly get a super high quality version of this human.

Let’s see! Wow, now that’s what I call a huge difference.

Restoration happened, the facial features are a little different, but this is a synthetic character, in this case, we might decide that we don’t mind this tradeoff so much.

Now, for the case when we pull the slider to the right to have slightly lower quality, but preserve the identity of the image better.

This is obviously super useful.

However, how do we know if these results are truly good? We cannot compare this with high-fidelity photos of Einstein, the whole point of this technique is that such high-quality photos don’t exist.

Well, one way is to take an image of someone we know, delete parts of it, have it restore it, and compare to the original.

Now, there will be some failure cases from previous techniques that look a tiny bit frightening, so if you are not ready for that, stop the video now.

Otherwise, let’s see.

Whoa.

To say that some of these previous methods were not so good at this would be an understatement.

Now let’s see the new technique.

Wow! This is so far beyond the previous techniques, it is hard to believe that such progress can be made in just one paper.

But, if we take the time to look through the results, it delivers over and over again.

And at this point, there is no doubt in my mind that we are going to see people who are not with us anymore in all their glory on our screens.

That is incredible.

So, how long do we have to wait for such an image? Well, not long at all.

This new technique can pull this off not in minutes, and not even in seconds.

It does this 14 times every second.

Wow.

But you know what? I have good news – you Fellow Scholars can try it right now online! Let the experiments begin! So, what do you think? Whose image would you like to restore? Isaac Asimov anyone? Let me know in the comments below! Thanks for watching and for your generous support, and I’ll see you next time!