What on earth is operation Night Watch?

Recently I attended a talk: AI at the Rijksmuseum: revealing the secret life of art at The Next Web conference. Presented by Robert G. Erdmann — Senior Scientist & Full Professor — Rijksmuseum & University of Amsterdam

He demonstrated how the Rijksmuseum has been using AI to restore the Nightwatch painting by Rembrandt

Wikipedia image of the night watch — trimmed version

The Nightwatch was painted in 1642 and later in 1715, the painting was trimmed so that it would fit between two doors at Amsterdam’s City hall. Those trimmed pieces were lost, luckily another painter had made a copy of the entire piece including the missing pieces.

However, it did not look exactly the same, the artist had different techniques and colour pallets so the Rijksmuseum has been using AI to ‘merge’ the two and produce a painting that is as close to the original as it can be. They used 3 different AIs to identify the structural differences, painting techniques and style. Later on, they were able to restore the missing pieces in the museum.

But that's not all!

He also demonstrated how AI can be used on other art pieces to identify things like:

  • What kind of paper/canvas they used
  • Any scratches or changes in the paper
  • Separate the ink from paper
  • Arrange and sort the whole art collection at the museum into similar patterns without any information about the paintings themselves.
  • Search an individual painting for sections

I am going to expand on that last point because this blew my mind. Using the AI you can search the entire painting, for example, you can search for a red block with white specs and it will highlight those areas for you an example can be seen below.

After searching for red with white specs it highlights the areas on the right
After zooming in we can see it has successfully highlighted a red area with white specs

This blew my mind because as a developer by day and I have coded a search bar many times on a website and every time you have to tell it where to look and it will only match words rather than the overall subject. So to be able to search for something so complex as specs in a painting and not have to tell it anything is simply amazing.

This just shows how far technology has come and I am sure the Rijksmuseum and others will continue to use AI to restore artwork, identify more information about the artwork, and allow people to take a closer look and find out more than you would by reading a plaque on the wall.

If you would like to read more what on earths? you can do so here.

I also write an artist of the month interview every month, which you can check out here.




Digital Artist: I write artist interviews, tips and tricks and my life as an artist on the side.

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Galleries in the Blockchain Era

The Pandemic Prolongs Performing Art Fears

The Powerhouse Museum: A Community Cultural Hub and Museum of Tomorrow, For Today


Contemporary Art and Controversies

WK 11 — Artist OTW — Student Choice

FRESCO and DSL Collection Reach A Global Strategic Partnership to Offer the Public Access To Art

A Moment in NYC

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Melissa Gilbert

Melissa Gilbert

Digital Artist: I write artist interviews, tips and tricks and my life as an artist on the side.

More from Medium

On the Origin of Kevin

Michael Hobbes on how all of American politics is just the War On Christmas now

In kindergartens and schools, children are taught to work from a pattern.

Knowing the search, enjoying the party (Revisiting Search Party, season 1)