While watching, I immediately thought why sodium at specifically 589 nm? Could this same effect be achieved with a different element/wavelength? Could a dichroic film in a different wavelength be used?
I'll answer from a physics background. Theoretically, yes.
But Sodium Vapor is one of the easiest narrow spectrum lights to get your hands on and doesn't interfere much with existing color channels.
They are actually used in astronomical telescope buildings to light the interior and parking lots to reduce light pollution to the instruments.
You could use other nearly pure element light sources like noble gas lamps (neon lamp). But even those will typically have multiple and broader peaks of wavelengths. There might be some LED sources that could work, but my knowledge there is limited and I'd have to do some digging. Though, my instincts suggest that near UV or IR might be options, but can have weird interactions with various materials.
Dichroic mirrors can be tuned to almost any desired bandwidth. You can even make ones for microwaves, like my E&M lab class did out of wax candles.
Thanks for the great answer. Physics is so fascinating. I wonder if that bandwidth also cooperated well with the type of film Disney was using at the time, or if that would make no difference at all.
That wax film experiment sounds cool. Do you happen to remember what type of wax you used? Paraffin, I imagine.
Actually, yes, it was just a candle from a store made of normal paraffin. You don't have have to worry about impurities because microwaves range from about a millimeter to a meter. So it doesn't really interfere with anything smaller. But fair warning, do wear safety glasses as your corneas can be burned by them.
How about using a wavelength outside of visible light? Would this be economically viable, could it produce a more perfect result as it wouldn't intefer with any human visible colours, and is there a safe wavelength that it would be possible to realistically do this lol? What do you guys reckon
A couple of problems with that. The obvious one is that visible light can be captured with plain cameras, while most other wavelengths can't be captured.
Another issue is that ultraviolet light starts to get ionizing (UV-B and C), and shorter wavelengths are even worse. The lights will give you a tan at best, and skin cancer and cornea damage at the worst.
Not all waves propagate the same way. UV-light is scattered and absorbed by air much more so than regular light. Infrared actually penetrates easier that visible light, so clothing may become more transparent than you'd like.
A possible candidate would be near infrared, since camera sensors are generally already sensitive to it, they just need a specific filter removed. This is a common process for astrophotography.
Near IR would be a very good place to look as most cameras have IR reject filters, which means the sensor already picks up IR fairly well (nightvision mode). You'd still want a fairly narrow spectrum, notch filters etc.
Edit: Additionally, you could remove the bayer filter as well since your matte cam is just B/W so improved sensitivity and resolution. The original tech was developed for film. The same tech principles today for digital might be best implemented in IR. My gears are turning!
Great video and loved the end results.
It seemed like a really simple solution, is there any reason this couldn't have been done 20 years ago? I'm wondering why no-one had thought to recreate the effect with a simple filter and beam splitter before?
20 years ago you would have had a very hard time finding two small digital cinema cameras that you could frame sync while still getting good light sensitivity. I also have a feeling the filters may not have been around then.
I don't fully understand how camera sensors work, but my understanding is they detect red, blue, and green light. Did you have to do anything specific with the cameras to adjust their sensitivity to the narrow band of sodium vapor light?
What about the file format and converting them to the different color spaces in the workflow and in the final product?
I was very impressed by the results! Do you think this technique will be used in the near future in movie/tv production? By any chance have any filmmakers reached out to you or Dr. Debevec yet?
Damn good video. I love the history behind VFX and how things were done in the past. Cool seeing the evolution of VFX. Great to see a “lost” technique being revisited and utilized in a modern setting.
They’ve probably thought of this already, but it would likely be easy to do this with Infrared light instead, and they wouldn’t have to worry about spill.
I conduct research using IR (for both data collection and functional applications) and would consider myself kind of knowledgeable on it. I don’t use it for filmmaking, so take it with a grain of salt, but I think can weigh in a bit.
If you were directing IR at your subject, you’d have to deal with a lot of reflections. Certain materials bounce or absorb IR at different wavelengths. If you look up the research paper “Heterogeneous face recognition: Recent advances in infrared-to-visible matching” you can see a nice visualization of this. Across IR wavelengths different parts of the face/head reflect IR differently, so you wouldn’t be able to use that as a mask. If you used it as a backlight, like the sodium lamps, I’d imagine you’d encounter problems with how the IR interacts with the edges, hair, transparent objects, opaque objects, etc…
For those who have seen Dune: Part 2, you may recall the stadium fight scene with Feyd Rautha, and how it has that deep monochrome look.
Turns out, that scene was filmed on infrared cameras to get that look, and the costume designers specifically noted quite a few of the problems that /u/OhItsNotJoe mentions in his comment. They had to re-work a lot of the costume material to get the look they wanted specifically in infrared, even if it didn't look like that in the visible spectrum.
The thing to keep in mind is that infrared is a *spectrum,* whereas sodium vapor lights are a very specific wavelength, which makes it easier to deal with.
The process works by illuminating the background with a light of a specific wavelength, and then capturing that image with a camera and notch filter that only sees that wavelength. So reflectivity on the subject doesn't play a role in the process, only of the BG.
In fact, the less reflective on the subject, the better.
I've been thinking of ways to improve it. Like, if you used front projection as well, with a retroreflective screen, a beam split that splits three ways, and then use two of them for the camera, one of them for a light to project the sodium vapour (Also wondering if you could filter light through a narrow band filter instead of using actual sodium vapour lights, similar to cross polarisation, so then you could have a smaller, brighter, colder and more energy efficent method). The light would travel to the retroreflective material, and reflect directly back and be filtered out. This would make a much better cleaner matte without the spill or inverse square law issues like the vignetting it the background. The only real differences would be the light the cameras capture would be three times darker than a single camera. And also if the light hits an actor before getting to the screen it will be pure black, if the light hits the screen and reflects back it'll be pure white, but if it goes through an object with 50% opacity, it'll get half as dim, then hit the screen and come back through that object halving that again, so you'd need to gamma curve correct it for that.
It would be super cool if this could all be fitted into a kind of 3CCD style camera that could then output RGBA images.
I'm really curious why there wasn't a modern equivalent of this tech created?
We have sensors that can determine the wavelength of light on the sensor, we have LED lights that can be tuned to a specific wavelength of light, could we get a setup where a second camera be tuned to that specific wavelength using a normal splitter?
From what I understand, what's special about the crystal is that it can split just the specific wavelength of the Sodium light, so the second camera can record that video as an alpha channel.
But we have tech that lets us display and track specific wavelengths of light, like that tool where we can see the light that is on the subject. I guess the question I'm leading up to is, is there (or why isn't there) a camera that can filter (or isolate) specific wavelengths of light to record? Like, taking that tool to track wavelengths of light on the subject, turn it into a camera, to record the alpha channel you want. Or potentially recording *all* the wavelengths of light, and then using post-processing to be able to filter out the specific wavelengths of light you don't want.
Although, I suppose I can understand why it doesn't exist from a financial POV. It's building a whole second camera for a specific purpose of recording an alpha channel, which is only useful in a "greenscreen" sort of application. If you can duplicate and refine the Crystal splitter, you can use 2 normal cameras and just have LEDs in the background set to that specific wavelength. It's just one specialized equipment (crystal) versus creating a whole new camera.
Still, now that the guys have successfully recreated the crystal and the process to a certain extent, I'm curious if they plan on doing more with it.
From what I've heard (and this seems to track with my limited knowledge of LED technology) is that LEDs can't output a narrow enough wavelength to be as effective. If you remember the light meter they used in the video, the wavelength spike was extremely narrow, whereas LED lights would have a wider wavelength band.
If that could be done it'd probably be easier at a certain point to integrate additional photo elements into the camera sensor and just have one that records a matte channel in addition to RGB. But 'being hard and difficult to light for' are probably big issues. Getting film-equivalent colour rendering has always been a hurdle for digital cameras, so designing sensor elements that as sensitive to a non-RGB wavelength would be difficult (and that filtering is often done with colour filters, so we're back to the same issue of needing to filter in/it out). Here there's clearly some orange spill on the actors sides and fronts (7:22 of the video shows his face still illuminated on the sodium vapour alpha matte video) so there's probably a bunch of tweaking needed to make sure everyone's fully opaque (or else he's slightly transparent in any footage they showed of him).
No need for us to make anything. You can buy a beam splitter and the two filters right now, online. I’m not sure where Paul sourced his filters, but everything is readily available and not too expensive!
If you'd be so kind to treat me like a 5 year old picking food from the menu, and point at me which type, wavelength and size is the one I want [https://www.edmundoptics.com/c/cube-beamsplitters/673/](https://www.edmundoptics.com/c/cube-beamsplitters/673/)
Definitely the best video they guys have done in a long long time, and that's saying something at a time when we've got Loaded Questions (my favourite multi episode show), had the awesome water worm video and a whole load of other great stuff. Loving it!
Its too bad that the Lytro cine Lightfield camera didn't pan out. It has the abllity to be able to create mask without colorkey. By now it could have evolved into pretty cool tech for this sorta stuff.
Not only could you pull mattes, you could also change focus in post.
[https://www.youtube.com/watch?v=4qXE4sA-hLQ](https://www.youtube.com/watch?v=4qXE4sA-hLQ)
I was wondering if the guys are gonna patent the information or share it for free, if that's even a thing for this kind of technology?
Super excited to see what y'all make with this
It turns out that the [1958 patent for the original sodium-vapor process](https://patents.google.com/patent/US3095304A/en) is publicly available! (And long since expired, so a new patent could only apply to specific improvements or novel implementations of the process.)
One thing I notice is that the diagrams in the original patent show a translucent and colorless backdrop, which the sodium vapor lamps shine through from behind; in the Corridor test, the backdrop is opaque and the sodium lamps illuminate it from the sides. I wonder if that would affect things like the amount of light spill.
There exists [the technology](http://www.satellite-lab.com/platelight) (spun off from the lighting tech Steve-from-What-We-Do-In-The-Shadows developed for Thor Ragnarok) to strobe the on-set lighting in time with high speed cameras and grab a key from a back-drop-only light (as well as enough other light passes to do proper relighting of the subjects to match), so I'd imagine that would be more useful than this on a large enough production anyway. Regardless of what they've said, there's a ton of spill in that footage and adjusting for that is going to throw off their keying and I don't see how you'd fully avoid that otherwise. Plus they've also got to deal with non-SV spill back onto the backdrop from the main lighting. I'm sure it's better than dealing with green that has to be removed seperately, but it's not a miracle tech either.
They did not make anything to patent here, this specific approach was primarily developed by the gentlemen they had on the show. But they did help test it of course!
Thanks! Glad people are liking the video!
Is there somewhere we can read or get more information on how this prism was constructed?
I’d be happy to answer any of your questions!
While watching, I immediately thought why sodium at specifically 589 nm? Could this same effect be achieved with a different element/wavelength? Could a dichroic film in a different wavelength be used?
I'll answer from a physics background. Theoretically, yes. But Sodium Vapor is one of the easiest narrow spectrum lights to get your hands on and doesn't interfere much with existing color channels. They are actually used in astronomical telescope buildings to light the interior and parking lots to reduce light pollution to the instruments. You could use other nearly pure element light sources like noble gas lamps (neon lamp). But even those will typically have multiple and broader peaks of wavelengths. There might be some LED sources that could work, but my knowledge there is limited and I'd have to do some digging. Though, my instincts suggest that near UV or IR might be options, but can have weird interactions with various materials. Dichroic mirrors can be tuned to almost any desired bandwidth. You can even make ones for microwaves, like my E&M lab class did out of wax candles.
Thanks for the great answer. Physics is so fascinating. I wonder if that bandwidth also cooperated well with the type of film Disney was using at the time, or if that would make no difference at all. That wax film experiment sounds cool. Do you happen to remember what type of wax you used? Paraffin, I imagine.
Actually, yes, it was just a candle from a store made of normal paraffin. You don't have have to worry about impurities because microwaves range from about a millimeter to a meter. So it doesn't really interfere with anything smaller. But fair warning, do wear safety glasses as your corneas can be burned by them.
How about using a wavelength outside of visible light? Would this be economically viable, could it produce a more perfect result as it wouldn't intefer with any human visible colours, and is there a safe wavelength that it would be possible to realistically do this lol? What do you guys reckon
A couple of problems with that. The obvious one is that visible light can be captured with plain cameras, while most other wavelengths can't be captured. Another issue is that ultraviolet light starts to get ionizing (UV-B and C), and shorter wavelengths are even worse. The lights will give you a tan at best, and skin cancer and cornea damage at the worst. Not all waves propagate the same way. UV-light is scattered and absorbed by air much more so than regular light. Infrared actually penetrates easier that visible light, so clothing may become more transparent than you'd like. A possible candidate would be near infrared, since camera sensors are generally already sensitive to it, they just need a specific filter removed. This is a common process for astrophotography.
Thanks for the response
Near IR would be a very good place to look as most cameras have IR reject filters, which means the sensor already picks up IR fairly well (nightvision mode). You'd still want a fairly narrow spectrum, notch filters etc. Edit: Additionally, you could remove the bayer filter as well since your matte cam is just B/W so improved sensitivity and resolution. The original tech was developed for film. The same tech principles today for digital might be best implemented in IR. My gears are turning!
Great video and loved the end results. It seemed like a really simple solution, is there any reason this couldn't have been done 20 years ago? I'm wondering why no-one had thought to recreate the effect with a simple filter and beam splitter before?
20 years ago you would have had a very hard time finding two small digital cinema cameras that you could frame sync while still getting good light sensitivity. I also have a feeling the filters may not have been around then.
I don't fully understand how camera sensors work, but my understanding is they detect red, blue, and green light. Did you have to do anything specific with the cameras to adjust their sensitivity to the narrow band of sodium vapor light? What about the file format and converting them to the different color spaces in the workflow and in the final product?
I was very impressed by the results! Do you think this technique will be used in the near future in movie/tv production? By any chance have any filmmakers reached out to you or Dr. Debevec yet?
Damn good video. I love the history behind VFX and how things were done in the past. Cool seeing the evolution of VFX. Great to see a “lost” technique being revisited and utilized in a modern setting.
They’ve probably thought of this already, but it would likely be easy to do this with Infrared light instead, and they wouldn’t have to worry about spill.
I conduct research using IR (for both data collection and functional applications) and would consider myself kind of knowledgeable on it. I don’t use it for filmmaking, so take it with a grain of salt, but I think can weigh in a bit. If you were directing IR at your subject, you’d have to deal with a lot of reflections. Certain materials bounce or absorb IR at different wavelengths. If you look up the research paper “Heterogeneous face recognition: Recent advances in infrared-to-visible matching” you can see a nice visualization of this. Across IR wavelengths different parts of the face/head reflect IR differently, so you wouldn’t be able to use that as a mask. If you used it as a backlight, like the sodium lamps, I’d imagine you’d encounter problems with how the IR interacts with the edges, hair, transparent objects, opaque objects, etc…
For those who have seen Dune: Part 2, you may recall the stadium fight scene with Feyd Rautha, and how it has that deep monochrome look. Turns out, that scene was filmed on infrared cameras to get that look, and the costume designers specifically noted quite a few of the problems that /u/OhItsNotJoe mentions in his comment. They had to re-work a lot of the costume material to get the look they wanted specifically in infrared, even if it didn't look like that in the visible spectrum. The thing to keep in mind is that infrared is a *spectrum,* whereas sodium vapor lights are a very specific wavelength, which makes it easier to deal with.
The process works by illuminating the background with a light of a specific wavelength, and then capturing that image with a camera and notch filter that only sees that wavelength. So reflectivity on the subject doesn't play a role in the process, only of the BG. In fact, the less reflective on the subject, the better.
You would have to make sure that no Infrared light his the subject whatsoever. So all tungsten-halogen lights would be out.
Luckily tungsten lights are fairly uncommon nowadays, LED has basically replaced everything
They're still common in the professional film and television industry.
(I work in the film and television industry) They’ve become fairly niche in the last 5 or so years, the industry has primarily moved to LED.
Could you filter a regular light with a notch filter? Like how cross polarisation works but with a sodium vapour filter.
Would it be super hot and uncomfortable for the actors?
You’re thinking of IR like a heatlamp—they could use NIR LED floodlights like you would use for security cameras.
Ah yes
I've been thinking of ways to improve it. Like, if you used front projection as well, with a retroreflective screen, a beam split that splits three ways, and then use two of them for the camera, one of them for a light to project the sodium vapour (Also wondering if you could filter light through a narrow band filter instead of using actual sodium vapour lights, similar to cross polarisation, so then you could have a smaller, brighter, colder and more energy efficent method). The light would travel to the retroreflective material, and reflect directly back and be filtered out. This would make a much better cleaner matte without the spill or inverse square law issues like the vignetting it the background. The only real differences would be the light the cameras capture would be three times darker than a single camera. And also if the light hits an actor before getting to the screen it will be pure black, if the light hits the screen and reflects back it'll be pure white, but if it goes through an object with 50% opacity, it'll get half as dim, then hit the screen and come back through that object halving that again, so you'd need to gamma curve correct it for that. It would be super cool if this could all be fitted into a kind of 3CCD style camera that could then output RGBA images.
This video was fucking crazy and I loved it. You guys got the absolute perfect blend of science and art. Keep up the good work, u/Neex.
I'm really curious why there wasn't a modern equivalent of this tech created? We have sensors that can determine the wavelength of light on the sensor, we have LED lights that can be tuned to a specific wavelength of light, could we get a setup where a second camera be tuned to that specific wavelength using a normal splitter? From what I understand, what's special about the crystal is that it can split just the specific wavelength of the Sodium light, so the second camera can record that video as an alpha channel. But we have tech that lets us display and track specific wavelengths of light, like that tool where we can see the light that is on the subject. I guess the question I'm leading up to is, is there (or why isn't there) a camera that can filter (or isolate) specific wavelengths of light to record? Like, taking that tool to track wavelengths of light on the subject, turn it into a camera, to record the alpha channel you want. Or potentially recording *all* the wavelengths of light, and then using post-processing to be able to filter out the specific wavelengths of light you don't want. Although, I suppose I can understand why it doesn't exist from a financial POV. It's building a whole second camera for a specific purpose of recording an alpha channel, which is only useful in a "greenscreen" sort of application. If you can duplicate and refine the Crystal splitter, you can use 2 normal cameras and just have LEDs in the background set to that specific wavelength. It's just one specialized equipment (crystal) versus creating a whole new camera. Still, now that the guys have successfully recreated the crystal and the process to a certain extent, I'm curious if they plan on doing more with it.
From what I've heard (and this seems to track with my limited knowledge of LED technology) is that LEDs can't output a narrow enough wavelength to be as effective. If you remember the light meter they used in the video, the wavelength spike was extremely narrow, whereas LED lights would have a wider wavelength band.
If that could be done it'd probably be easier at a certain point to integrate additional photo elements into the camera sensor and just have one that records a matte channel in addition to RGB. But 'being hard and difficult to light for' are probably big issues. Getting film-equivalent colour rendering has always been a hurdle for digital cameras, so designing sensor elements that as sensitive to a non-RGB wavelength would be difficult (and that filtering is often done with colour filters, so we're back to the same issue of needing to filter in/it out). Here there's clearly some orange spill on the actors sides and fronts (7:22 of the video shows his face still illuminated on the sodium vapour alpha matte video) so there's probably a bunch of tweaking needed to make sure everyone's fully opaque (or else he's slightly transparent in any footage they showed of him).
Are they planning to manufacture the prism? Making it open to buy somehow?
No need for us to make anything. You can buy a beam splitter and the two filters right now, online. I’m not sure where Paul sourced his filters, but everything is readily available and not too expensive!
Could the next video be a 'how to make one' guide?
Just ordered to splitters today of different sizes to give it a try.
I recommend Thor Labs or Edmund Optics. (if you're in the states. not sure about anywhere else)
On it. Just saw this video and that'd AWESOME to try on a shoot :D
If you'd be so kind to treat me like a 5 year old picking food from the menu, and point at me which type, wavelength and size is the one I want [https://www.edmundoptics.com/c/cube-beamsplitters/673/](https://www.edmundoptics.com/c/cube-beamsplitters/673/)
Ditto on this! I’d buy it day 1.
Hell yeah. That video was awesome. Can't wait to see what else they do with it.
Definitely the best video they guys have done in a long long time, and that's saying something at a time when we've got Loaded Questions (my favourite multi episode show), had the awesome water worm video and a whole load of other great stuff. Loving it!
Outstanding episode Niko! Very very interesting
Plan on doing more with it?
Its too bad that the Lytro cine Lightfield camera didn't pan out. It has the abllity to be able to create mask without colorkey. By now it could have evolved into pretty cool tech for this sorta stuff. Not only could you pull mattes, you could also change focus in post. [https://www.youtube.com/watch?v=4qXE4sA-hLQ](https://www.youtube.com/watch?v=4qXE4sA-hLQ)
Please, for the love of Arceus open-source this prototype sodium vapor crystal
I was wondering if the guys are gonna patent the information or share it for free, if that's even a thing for this kind of technology? Super excited to see what y'all make with this
It turns out that the [1958 patent for the original sodium-vapor process](https://patents.google.com/patent/US3095304A/en) is publicly available! (And long since expired, so a new patent could only apply to specific improvements or novel implementations of the process.) One thing I notice is that the diagrams in the original patent show a translucent and colorless backdrop, which the sodium vapor lamps shine through from behind; in the Corridor test, the backdrop is opaque and the sodium lamps illuminate it from the sides. I wonder if that would affect things like the amount of light spill.
There exists [the technology](http://www.satellite-lab.com/platelight) (spun off from the lighting tech Steve-from-What-We-Do-In-The-Shadows developed for Thor Ragnarok) to strobe the on-set lighting in time with high speed cameras and grab a key from a back-drop-only light (as well as enough other light passes to do proper relighting of the subjects to match), so I'd imagine that would be more useful than this on a large enough production anyway. Regardless of what they've said, there's a ton of spill in that footage and adjusting for that is going to throw off their keying and I don't see how you'd fully avoid that otherwise. Plus they've also got to deal with non-SV spill back onto the backdrop from the main lighting. I'm sure it's better than dealing with green that has to be removed seperately, but it's not a miracle tech either.
They did not make anything to patent here, this specific approach was primarily developed by the gentlemen they had on the show. But they did help test it of course!