With this innovative system, all the user has to do is approach the canvas or wall with the device and wave it around. The system does the rest!

By Lola Gayle, Editor-at-large

While many are worried that robots will one day take over their jobs, artists for the most part have had little to fear from automatons. However, perhaps they should all take a little more notice as researchers have invented a “smart” spray paint can that can robotically reproduce photographs as large-scale murals.

041216-robotic artist 2
Researchers have invented a ‘smart’ paint spray can that robotically reproduces photographs as large-scale murals. Wojciech Jarosz

I don’t think we’ll be seeing an army of robotic graffiti artists just yet, but the new computerized technique developed by a team from ETH Zurich, Disney Research Zurich, Dartmouth College and Columbia University could eventually find uses in digital fabrication, digital and visual arts, artistic stylization and other applications.

The “smart” spray system is a novel adaptation to computer-aided painting, which originated in the early 1960s and is a well-studied subject among scientists and artists, according to a statement from the researchers.

Because spray paint is efficient and affordable, large-scale spray painted murals have become common in modern urban culture. However, the logistics and technical difficulty of covering such a large “canvas” can be vexing even for the most skilled artists.

The computer-aided system can be attached to an ordinary can of spray paint and is able to track the can’s position and recognize what image it “wants” to paint. All the user has to do is approach the canvas or wall with the pre-programmed device and simply wave it around. The system does the rest, automatically operating the can’s on/off button to reproduce the specific image as a spray painting.

Their prototype system includes two webcams, QR-coded cubes for tracking, and a small actuation device for the spray can, attached via a 3D-printed mount. Paint commands are sent via radio directly connected to a servo-motor operating the spray nozzle. A real-time algorithm running on a nearby computer determines the optimal amount of paint of the current color to spray at the spray can’s tracked location.

Amazingly, the human half of the equation doesn’t even need to be aware of the image beforehand. They just have to watch and see as the painting reveals itself through their movements.

041216-robotic artist 3
The device and real-time optimization enables spray painting murals of any input photograph. Romain Prévost, Alec Jacobson, Wojciech Jarosz, Olga Sorkine-Hornung/Computers & Graphics, 55:108–117, April 2016.

The researchers tested the automated painting system on large sheets of paper which they then assembled into mural-size paintings. While the current system only supports painting on flat surfaces, one potential benefit of the new technique over standard printing is that it may be usable on more complicated, curved painting surfaces, the researchers said.

“Typically, computationally-assisted painting methods are restricted to the computer,” said co-author Wojciech Jarosz, an assistant professor of computer science at Dartmouth who previously was a senior research scientist at Disney Research Zurich. “In this research, we show that by combining computer graphics and computer vision techniques, we can bring such assistance technology to the physical world even for this very traditional painting medium, creating a somewhat unconventional form of digital fabrication. Our assistive approach is like a modern take on ‘paint by numbers’ for spray painting. Most importantly, we wanted to maintain the aesthetic aspects of physical spray painting and the tactile experience of holding and waving a physical spray can while enabling unskilled users to create a physical piece of art.”

A study detailing this work is published in the journal Computer & Graphics. A supplemental video is available here.

Originally written for STEAM Register.