A Digital Skylight

For 2017’s complicated art project, I continued with the theme from 2015: simulating real life with digital displays. But this year, instead of recreating artwork digitally, I attempted to recreate the sky.

The concept was pretty simple: a digital skylight. I’ve been thinking about making one ever since I visited a frozen yogurt shop in London years ago, where some clever person had used hundreds of diffused LEDs to create a digital sky on the shop's ceiling. I was also inspired by the painted ceiling above the shops at the Venetian Hotel in Las Vegas, and by work on "personal skies" by the Japanese industrial designer and artist Naoto Fukasawa.

The idea appealed to me for a couple of reasons. First, being unconstrained by having a direct path through to the roof of the house, I could put the skylight anywhere. It could be in an “impossible” location, like on the first floor of a two-story house, if I wanted. And second, the digital skylight could show “impossible" skies – faster, slower, shifted in time and space, or fantastical.

Here’s a photo of what it looks like. (Scroll to the end of this post for a video of the final product in action.)

It's worth saying that this image is straight out of the camera. There's no Photoshopping or other post-processing here.

The Display

I really wasn’t sure whether the whole thing would work, so I chose to install it in my closet, which is out of the way. To create the skylight window, I modified a trap door into the attic, enlarging the opening, adding drywall, and painting it, so it looked just like a finished skylight would. With the help of a friend (thanks, Harrison!), I then managed the incredibly awkward task of getting a 65” TV through the opening and into the attic.

To make access to the attic and to the back of the TV possible, I mounted the TV in a wood frame on a set of rails, which made it easy to slide in and out of place. I also ran an IR repeater to the TV, since the IR receiver on the TV itself became obscured. 

Before settling on the TV-in-the-attic approach, I considered a couple of other setups. One involved building a hemispherical projection surface in the attic – basically a big white half dome – and front projecting imagery onto it. Another was using a mirror mounted in the skylight opening at a 45 degree angle and large, flat projection screen some distance from the window, mounted vertically. Both of these approaches would have created somewhat convincing parallax, where the skylight window would shift over the sky as the viewer moved. The virtual image would also have been pushed back a meter or two, creating more realistic focusing cues. In the end, I judged both approaches too complicated to be worth it for a v1.
 

Collecting Skies

The real fun began with capturing different skies. Over a period of a year, I ended up with half a dozen flavors of sky from a handful of locations.

Time-lapse – These ended up being my favorite, since they’re nice to look at, clearly recognizable as skies, but also “impossible”. Real skies and clouds just don't change this quickly. The approach here was pretty simple: I mounted my digital SLR on a tripod, pointed it straight up, and set my camera to take a frame per second for a few hours. These captures resulted in imagery like this:

(The actual imagery is obviously much better than this and the other looping GIFs in this post. It's all mastered at 1080p or 4K and 30 or 60 frames per second.)

Night skies – Night skies were cool, because with a bright lens and longish exposure times, I could reveal stars that aren't visible to the naked eye. I especially liked when skies included both stars and clouds, which resulted in neat layered effects, like this:

Rain – One illusion I was eager to convincingly recreate was raindrops on the “glass” of the digital skylight. To do that, I built a free-standing structure out of wood to hold a piece of plexiglass in place 5-6 feet off the ground – basically, a window mounted horizontally, on stilts. When it rained, I’d leave the whole thing outside with a camera shooting up through the plexiglass. The images look like this:

Live – Here, I just pointed a Nest Cam Outdoor (a great product, by the way) at the sky and piped the imagery through to the skylight. This ended up being pretty boring, both because I live in Northern California, and so there is often nothing but empty blue sky to look at, and because, of course, I could see the same thing just by going outside. 

Far more interesting than using live sky imagery from above my own house was using imagery from somewhere else I care about. For instance, a friend in London, where I lived for several years, set up a sky camera for me from their apartment, so I can enjoy the London sky whenever I want.

(If you live in the American Southwest and are reading this: you have the best clouds and I would love to buy you a Nest Cam if you’re willing to point it at the sky for me. Please email me.)

Impossible Skies – These were really fun, and involved making creative use of After Effects to modify imagery I'd captured. A few of my favorites included Mars, Kaleidoscope clouds, and the "Ansel Adams" sky.

I learned a few things about what kind of imagery works and what doesn't. First, playback speed of the time-lapses turned out to matter a lot. Too fast, and the skylight was distracting and, in cases where there was strongly directional cloud movement, vertigo-inducing. Too slow, and it was boring. 

The focal length of the lens used to capture the imagery mattered a lot too. Too wide, and the skylight looked wrong. Clouds and other features looked "miniature", because too much of the sky was "squished" into a small window. Too telephoto, and there wasn't enough to see. In the end, I captured almost everything with either a 35mm or 50mm prime lens, which captured more of the sky than you'd really be able to see through the skylight if it were real, but not so much that it looks wrong.

Getting the Light Right

One surprise, which in retrospect should not have been a surprise at all, was that the character of the light emitted by an LCD display simulating a skylight is very different from that from a real skylight.

Blue or mostly blue skies were especially problematic. A real skylight obviously lets in sunlight, and sunlight is white. And, while you can see the blue sky through a skylight, it’s not primarily the blue sky that’s making the light. But with a digital skylight, to create the appearance of the blue sky, the display obviously has to create blue light – and that's all it's doing. So instead of flooding the room with nice white light, like a real skylight does, the digital version casts a weird blue into the room.

To compensate, I first tried mounting hidden strips of white LEDs in the gap between the top of the skylight opening and the display, the thought being to create lots of white light from the skylight itself, and to wash out the blue coming from the display. This approach turned out to not work at all, since the LEDs were reflected in the display, totally ruining the illusion.

The second approach I took was just to flood the room with diffuse white light, which I did using a pair of Philips Hue LED strips hidden above the shelves. This approach worked well enough, and because the HUE strips give you full control over color, I could dial in exactly the right white balance, and also create some neat effects.

All Together

The pieces all came together pretty nicely. Here's what the skylight looks like in action:

For Version 2.0

Version 1.0 turned out better than I expected, but there are a bunch of improvements I'd like to make. For instance:

OLED for true blacks – Because I used an LCD panel, the skylight can't display true black, like in a night sky. That's because even on the best LCD displays, some of the backlight bleeds through, causing black to end up looking like a glowing gray. An OLED panel, like those made by LG, would solve this.

Live time-lapse – If I had more time, I would have written some software to create time-lapses from recent imagery. For instance, the skylight could always display the last 15 minutes of clouds over the house, but accelerated 10x.

"Ok, Google" and automation – The whole system is pretty manual right now. I decide what I want to display, futz a bit with the Mac that drives the whole thing, and then the video displays until I tell it to stop. Some amount of automation, including the ability to control the skylight and lights with voice commands, would be a big improvement.

Parallax and accurate focusing cues – In person, the dead giveaway of the digital skylight is that the sky seems "too close". You move in the room, and the sky doesn't move behind the skylight opening. There's no parallax. And your eyes, focusing on something only a few feet away, tell you, "that's not the real sky." Alas, the display technology to properly address these issues is likely decades away. Until then, the TV-in-the-attic-approach will have to do.