I have long been fascinated by mosaics, and the idea that it’s possible to create an image from atomic units – individual tiles – that look nothing like the image itself. Georges Seurat and Chuck Close took the mosaic concept and substituted painted dots or squares on canvas in order to form images. Contemporary artists like Tara Donovan and Gregory Hayes use pushpins and dollops of paint to do the same.

It was inspired by artists like these that I pursued this latest project: a mosaic maker. The basic idea was to take a bunch of tiles made with traditional media – in this instance, watercolor on paper – and then use software to algorithmically generate high quality mosaics of anything. It works for portraits, like this:

… and for more abstract images, like this:

The process starts by creating a whole bunch of the raw material of mosaics, the tiles. Over a few years, I hand painted about a thousand of them, working to create some variation between and within the tiles to make it possible to capture more detail in the source image. I then scanned them all so the tiles could be cut, copied, and rearranged digitally. Individually, the tiles look like this:

Up close, the assembled mosaics look like this. (This is Frederick Douglass’s right eye and nose.)

The tile matching approach is pretty straightforward. After dividing the target image up into a grid of cells, the basic algorithm is, for each cell in the target image, for each painted tile, compute the mean squared error. Then, minimize the total mean square error across the image, subject to some constraints, for instance not having the same tile be adjacent to itself. This all takes a few hundred lines of Python code, and is very much akin to what software to create photomosaics does.

You can think of the algorithm as cutting the target image up into a bunch of squares, and then trying to find the best painted tile to match each square. As you do this, the target image emerges, but composed of the mosaic tiles, which can be visualized like this:

It was fun to experiment with different sets of mosaic parameters, e.g. using more vs. fewer tiles, laying tiles out on the diagonal vs. on a grid structure, and constraining the layouts in various ways. Here are five portraits, each in a different “style”.

As for what’s next, I may experiment with other types of tiles, e.g. pencil or wood. I also have ambitions to clean up the code and get everything running as a web service. We’ll see.