What on earth is operation Night Watch?

Prepare to have your mind blown

Recently I attended a talk: AI at the Rijksmuseum: revealing the secret life of art at The Next Web conference. Presented by Robert G. Erdmann — Senior Scientist & Full Professor — Rijksmuseum & University of Amsterdam

He demonstrated how the Rijksmuseum has been using AI to restore the Nightwatch painting by Rembrandt

Wikipedia image of the night watch — trimmed version

The Nightwatch was painted in 1642 and later in 1715, the painting was trimmed so that it would fit between two doors at Amsterdam’s City hall. Those trimmed pieces were lost, luckily another painter had made a copy of the entire piece including the missing pieces.

However, it did not look exactly the same, the artist had different techniques and colour pallets so the Rijksmuseum has been using AI to ‘merge’ the two and produce a painting that is as close to the original as it can be. They used 3 different AIs to identify the structural differences, painting techniques and style. Later on, they were able to restore the missing pieces in the museum.

But that's not all!

He also demonstrated how AI can be used on other art pieces to identify things like:

• What kind of paper/canvas they used
• Any scratches or changes in the paper
• Separate the ink from paper
• Arrange and sort the whole art collection at the museum into similar patterns without any information about the paintings themselves.
• Search an individual painting for sections

I am going to expand on that last point because this blew my mind. Using the AI you can search the entire painting, for example, you can search for a red block with white specs and it will highlight those areas for you an example can be seen below.

After searching for red with white specs it highlights the areas on the right

After zooming in we can see it has successfully highlighted a red area with white specs

This blew my mind because as a developer by day and I have coded a search bar many times on a website and every time you have to tell it where to look and it will only match words rather than the overall subject. So to be able to search for something so complex as specs in a painting and not have to tell it anything is simply amazing.

This just shows how far technology has come and I am sure the Rijksmuseum and others will continue to use AI to restore artwork, identify more information about the artwork, and allow people to take a closer look and find out more than you would by reading a plaque on the wall.

If you would like to read more what on earths? you can do so here.

I also write an artist of the month interview every month, which you can check out here.