107 - Visualization in fires with Matt Hoehler

Hello everybody, welcome to the fire science show. Many times I've mentioned the power of observation in fire science and how much you can learn from just looking at the fires and perhaps even recalling them. And if you are, just like me, a person who loves to stick their phone into fires and record pretty pictures, this episode may be actually for you, because in this episode I invited Dr Matt Hayler from NIST, with whom we discuss what distinguishes just taking pretty pictures of fires from real scientific observations or even perhaps visual experiments, visualizations of fires and how we use them in fire science to understand compartments, assemblies and building elements and other elements in fire. Matt certainly has experience in burning down cameras. So do I, and we are just two geeks who love to talk about visualizations. So in this episode you are going to learn about the basic stuff how to record fires. You will learn about the magical blue light illumination technique used by Matt to investigate structural elements engulfed in flames and how to record their surfaces even though there is flame between the camera and the building element. You will learn about 360 videos and how to use water to protect your valuable gear and not get it burned down, which is a very appreciated note. So, yeah, get your camera ready and when you snap a very nice picture of fire of maybe you have one in your collection remember you still have like two days as the episode airs to submit them to IFSS fire picture competition that's going on for the IFSS conference in Tsukuba. So if you have pictures, please submit them and don't miss out on that. And for now that's enough of rambling. Let's spin the intro and jump into the episode. Welcome to the Firesize Show. My name is Vojci Wimczynski and I will be your host, and before we jump into the episode, i would like to appreciate the sponsor of this podcast, ofr Consultants, thanks to whom all of this content is free and open access to anyone. I had the pleasure to visit OFR on their team day event last week and it was really nice to meet all the people and see the fantastic engineering and scientific research that's being carried out there. I've realized that the slogan pre-eminent fire engineering to help protect people, property and environment is not just few words but something that's really deep in their DNA, and I appreciate it very much. It's pleasure to work with you guys and I'm really thankful that this partnership already continues for almost half a year. Thank you so much for that, and if anyone would like to learn more about OFR or perhaps join their team they're always looking up for new fire engineers check it out at OFRconsultantscom. And now back to the episode on fire visualizations. Hello everybody, welcome to the Firesize Show. I'm today here with Matt Hayler from NIST. Hello, matt, good morning. And we're gonna talk about Imaginga visualization recording fires Put in camera in front of a fire. Let's agree on that Sounds good. I am a big fan of your work in the world of developing new technologies to record fires and so many times in the podcast we've discussed that actually observing and having a recording of a fire is absolutely full of information you would not even expect there to be. It's just priceless. I even had episodes with Professor Guillemorraine in which we praised recording and it's a basic tool for myself in my laboratory. But the thing is not as easy as just putting camera towards fire, glassy sights towards fire and press record and you're done. I guess that's my experience. It's always painful. How was it for you? When did you start playing with recording fires and when have you advanced to the next level?

I'm a lateral transition over into fire science in general. I spent about 10 or 15 years doing earthquake engineering in large scale testing in earthquakes before I moved to NIST. I'm a structural engineer by training so, like with fire, in earthquake engineering, video is incredibly important because you're talking about complex transient phenomena and you can measure the heck out of something, but you don't always look at the thing that you think is going to be important, and so when you have visualizations and videos, you can always go back and see what's going on. When I moved to NIST in 2014 and started working in the National Fire Research Lab there we had a group already driving ahead on the large scale testing, but came up again and again that we needed more capabilities to do measurement in general for large structural fire tests, and it was clear to me that one of the first places to start was in visualization and just capturing images of fires. I didn't have any formal background in photography. The people who come into this and do have that background are always very jealous of that. I always enjoyed photography. I used to do black and white photography in high school, but never really in a research context, and really at the beginning it was just like probably everybody else listening to your podcast does, setting up camera and trying things out. But we really started getting interesting, and this was pretty early on. We wanted to get closer to the fires that we were looking at, and one of the early experiments where we were doing this, we were doing some cross laminated timber tests with NRC Canada. This was 2017 and maybe even a little before that, and we wanted to really get some good images from the inside of the fire compartments. But these were very large 10 to 20 megawatt fires. It was really hard to get up close to them. We started experimenting with ways to get the cameras closer to the fire.

We will get to the technicalities of how to get closer, because this is the most interesting for me as well, but first, because it may sound like we're talking about a very trivial thing. I don't think it's trivial at all. I think failed so many times while doing it. That's an important disclosure. Let's give listeners an idea of what can you get outside of recording, what type of measures you can actually do with simple video recordings.

Well, there's a lot. So some of the actually also some of the early work we had as we were looking at measurement for fire, we had a couple of thrust areas that were going in parallel. One of the first one was actually looking at digital image correlation, which a formal measurement technique of using cameras to resolve deformations and strains using visual markers or patterns sprayed onto surfaces, and this technique is used a lot outside of the fire realm, And so we started early on experimenting with doing digital image correlation in fires, and there were some really neat advantages of these measurements. Again, we were focused on deformations and strains, trying to capture them.

Very structural thing when you just spray a bunch of dots on your wall and you have a fixed camera and then you just see how the dots move, which indicates the strains. Yeah, cool, cool. We have that in our lab, that's great. Okay, very structural engineering thing, yeah.

And it's interesting because I come at this very differently than a lot of my fire science colleagues. Usually, most of the things that I'm interested in the fire, i don't want to see the fire at all. I'm very interested in the object that's inside of the fire. So I want to know how something is moving or deforming in time. And it means I do take a different approach. I mean, in most of the techniques that I'm applying I am trying to not see the fire in most of my cases. And it's interesting because you were talking about some of the more simple things. You can go through sets of photos of the different photographers in our labs And that Bundy, who's also in the National Fire Lab. He's a big photography fan and like shooting fires. But if you look at his sets of images and mine, if you know us, you can always tell who's shooting the pictures, because I usually tend to deemphasize the fire when I'm shooting and expose the context. I want to see the experimental setup, i want to see the thing that's in the fire, and so you'll see less flame structure in my photos and you'll see more of the context, because if you look at somebody else's photos who's a fire science person, you really often see a big focus on the flame structure and you lose the context, and so it really that's. The interesting thing that you can do with image, whether it's a video system or still image system, is you can pull the things into focus that are relevant to you in a way that you can't with other measurement systems.

Let's go on what you can do with video. So the displacements. now you've mentioned flame structures, So for sure, I think that you can capture what else you can get from videos.

From videos. It's really about figuring out what happens between point A and point B. You want to get that time dimension to what's going on. I mean, we may talk about this more later, but that's really how we got into 360 video cameras in the fire because we had purchased one of the early GoPro rigs. It was one of those with like six separate GoPro cameras on them and you go in and you take a snapshot of a space and then you have to manually stitch all of those images together, which was very labor intensive. And we do that before a fire to kind of capture the fuel load. Then we pull the camera out, we run our fire and then afterwards we go back in and take another 360 image of what had happened in between. But you can't. When you do that, you don't see how you got from point A to point B. And that's really the benefit of the video, as you can see what's happening in between.

I can tell you that we've been doing a lot of work in this field as well With a friend from Imperial College London, matt Bonner. I don't know, i see a pattern here about the people who image fires and their first names. Maybe that's a prerequisite, i don't know. With Matt we've been shooting videos of flames actually in different experiments And what we have been trying to get is an approximation of flame value. That led to approximation of the heat release rate, which was kind of successful, i would say. I mean from a set of two sports cameras getting plus or minus 100 kilowatt a size of a fire, i would say that's not bad, that would be very good, that's not bad at all. In different experiments which we've done in our lab, from video capture we tried to get mean flame heights, maximum flame heights, some sort of flame frequencies. So the thing is that you can actually quite reliably measure when you have a video at a constant frame rate, shooting pointed at your fire. And here I would argue these are very scientific measurements, that these are not just observations.

Absolutely. I think the ability to mine images, whether it's a video or a still image, or the types of information that you're talking about is critical, and this is not something that I've worked in. But you can go beyond that and start to look at multispectral imaging. We've done a lot of work where we'll just shoot either in raw or in RGB red, green, blue and then we'll strip down to just the blue channel. You can start then looking at the different frequency spectra And you can also glean information about what's going on in different areas just by looking at different wavelengths of light, and so you can do work in that area. And another thing which is very important I don't know if we'll really get into it today, but certainly with video, audio is absolutely essential. I mean, here we are doing a podcast and it's all about what we're hearing, and I think people lose an incredible amount of information if they're shooting video and don't pay attention to the audio, because it adds a whole nother layer to what you're seeing and how you're experiencing it. But in the same way that you talked about mining the visual images for data, you can use the audio signals to try to figure out, for example, out in the wildland how quickly a fire is moving toward you, based on the amplitude of what you're hearing. You can do frequency analysis. So having good audio to accompany video, in particular, is something that we've put a lot of thought into.

I remember your talk from NFPE in San Antonio where you were showing the 360 setup and showing this shotgun microphone you've used. That was very interesting And we will definitely come back to that. Just to finish what you can do, i'll give you one more example. It also touches the wavelength thing you've mentioned. So a friend from Germany, professor Lucas Arnold. He was doing something they've called LED tomography And what that meant is that they were placing in an experiment stripes of RGB led lights nothing fancy, just a stripe of RGB LEDs And they used a normal DSLR camera, like a Canon camera, to shoot, you know, these LEDs through the layer of smoke And by comparing the change of light intensity and actually the change in shape of the LED diode shot by the camera through the smoke, they were able to figure out the average smoke density on the way to the camera And even the scattering rate, because the shape of the diode changed. And all of this with a simple stripe of LEDs, very normal to use those today. That sounds incredible. I mean, there are papers on that. It's amazing, and here I would say this is a really high level science.

Absolutely, and you have the added benefit that you can see the context of what you're looking at just because you have the image to go along with it.

So now let's move in depth. Let's try to structure, as you've started about CLT's and seeing inside the fire. I think that's a direction that we should follow in this episode. I also agree that many of us maybe too much focus on the flame structure where it's not. That I mean it's beautiful but it's not the most important thing in your experiments Often, especially when you're doing build environment related fire experiments or maybe even more in wildfire science. I've done both. So let's try the CLT's and your first tries with trying to image in a 10, 20 megawatt experiment setting, that's 20 megawatts in a compartment fire. That's going to destroy your imaging setup immediately. You cannot even put the camera on the stick inside, take a picture and move out because that's already going to be damaged. So tell me what was your approach and how did you succeed in that Got?

back a little bit before we get to that. So when we those cameras all those all the we burned up. We burned up many, many cameras, as I'm sure a lot of people who are listening to this have done as well. It's part of the process. So again, early on when we were kind of looking at measurement science needs for structure fires, i had a great postdoc from Stanford who then continued on at NIST for a while. He's since moved on in this, now working for the insurance industry guy by the name of Chris Smith, very bright guy. He and I did kind of a survey of all the different technologies that we might want to apply into structural fire engineering, and two of the ones that we started with first were digital image correlation and then also laser based techniques And it's interesting because there's an evolution here The DIC systems. One of the challenges that we faced with that system and this leads to another topic we may talk about was that the flames would get in the way of the structured patterns, the dots on the things we were imaging.

So DIC is the digital image correlation.

Digital image correlation, but again with the more important than the digital image correlation was the technique that we used to try to enhance our image quality, which led to the blue light imaging or narrow spectrum illumination imaging, because we were not able to see the pattern targets well enough with the cameras, with ambient light, because we were overpowered by the flames. So that was one kind of thrust area that was going on And in parallel to that we started taking measurements. This was with triangulation lasers trying to measure distance, single point distance to an object in a fire, and this will get to your question about a CLT. We started these experiments with you know, others had looked at time of flight lasers and red spectrum triangulation lasers trying to measure distance and fire, and we had been looking around and we found some 405 nanometers of blue light lasers, which are used for steel and glass manufacturing to look at surface deformations where you have a lot of heat energy. And the reason that these blue lasers are used is because they're illuminating the object, which is giving off a lot of red glow and infrared radiation, and they're working at the 400 nanometer wavelength. So it was a little bit away from where the energy from the fire was, and so we started using blue lasers. At the same time, we were using blue light in the DIC imaging. So you see, there was kind of a correlation there. We were trying to move away from the high energy portion of the fire in terms of the emitted radiation, and the problem that we had with these lasers these were expensive lasers is that we needed to get them within two meters of the object for them to work, and the radiant energy from the fire would just burn out your sensor within seconds, right, you have? you know, even at two or five kilowatts per meter squared, it was too much for the sensors, and so we needed a solution to protect that, and what we came up with was to basically put the laser behind a curtain of water, right, and so we created basically like a fish tank, you can. You know, we did some calculations and figured out, with a very thin film of water, even like 20, 30 millimeters of water If you flow that water, you can absorb the radiative energy from the fire, because water is really good at absorbing infrared radiation, right, and the other thing that it does is it passes visible light, and so these lasers that we're using were 400 nanometers, so they're at the bottom of the visible spectrum So we could shoot the laser through them but we could absorb the energy from the fire. So we started building devices for to create these water curtains in front of the lasers and we were having great success. We could take a $10,000 laser, put it right up against essentially, which was a home-built furnace, where you were seeing about 50 to 75 kilowatts per meter squared at that interface, and we could knock that down below one kilowatt per meter squared and put this laser right at the window of this furnace that we were using and shoot into the fire doing this And we could leave the laser there indefinitely. And we're like this is, you know, it turned out to be an incredibly powerful way for us to move these laser sensors closer to the fire. So that was kind of the backdrop. We had been doing that for a year or so and, you know, making some progress with these lasers. And then this project with the cross-laminated timber came along And we wanted to get the cameras closer to the fire. Right, originally we had set up far away with cameras on tripods, you know. Sometimes we put a piece of polycarbonate or some kind of sacrificial layer there that we could start to see if it started getting too hot. You'd see deformations. But we wanted to get much closer to the fire And it dawned on us we could take this same approach that we were using with the lasers and just put the cameras behind a thin layer of water and move them a lot closer to the fire. And so that's what we started doing, and early on, all we were doing is we were taking sports action cameras. We used a lot of GoPro's just because they were ruggedized and ready to go, and we just put them in a beaker of water and slide that beaker right up into the doorway of a fire, and when the water started to boil, we'd pull that out and pull the SD card out of the camera. And we found that we could, you know, get 10 to 20 minutes of run time very close to these large fires by just submerging the camera in water.

And again, how much would that be like liters?

It was maybe two or three liters of water And it's still, honestly for a lot of people. You know we'll get into the systems that we use now, which use for circulating water systems and or flowing open open water systems, which are very complicated But for a lot of applications, just taking a borosilicate a Pyrex and a bucket of water and proper alignment. You have to worry about reflections of light on the back of the glass and sort of the things like that. There's little details, but you can usually get fantastic performance and you don't need an expensive system And it really lets you get up close to fires for a duration of time. So we started. You know, that was kind of where we started. Again, that was a long, long way to get here, But we had started with the lasers, realized the benefit of water for visualization in fire in terms of protecting the camera and then transfer that over to the cameras. And so that was. And it's interesting because when you talk to people about the cameras and the fire, they say, oh, you put the camera in the water to keep it cool. And yes, we do, but that's not the primary reason. It's an optical band pass filter, Right, Okay, It's just really good at absorbing radiant energy.

So let's for a second stop on nature of light. So light is multiple wavelengths, from blue to red, and the thing in fires is that you have mostly carbon, that's emitting radiation at a certain spectra that are related to the carbon nucleus structure and where its electrons are around the atom. It's actually quite fascinating. You can work through that, through just knowing how a carbon atom looks like, and you can pretty much figure out which spectra you'll have. That's a brilliant part of science. Anyway, in fires, in the temperatures that we are talking about, you'll have illumination more towards the red end of the light wave than towards the blue end. Essentially, you will get very little of the blue colored light, which could be at 400 ish nanometers. you'll get a lot of reds, a lot of, perhaps, yellows, and you'll get a ton of infrared which is beyond the visible wavelengths. That's correct Just above. So you end up with a ton of wavelengths you don't like because you cannot see them. You cannot see an infrared which you would like to filter out, so you can emphasize on the ones that allow you to see what's happening beyond the fire, which is the blues and to some extent green, probably light that it just illuminates your room and it's not overpowered by fires. When you shoot fire, you usually end up with a white blob of light. I've learned why, just like two, three years ago, when I've picked astrophotography and I started imaging objects like the moon or planets. And you very quickly learn what is the well of your silicon chip, which means how many photons it can store until it gives you saturation. Maximum saturation. It's just white and it does not take any further information anymore. So you end, when in shooting fire you have overexposed image, you see white blob. It means your chip, your camera, whatever you're using, has taken the maximum amount of photons in that particular pixel and it will not take anymore. So you basically lose all the information that would be there because you don't know. It's like in Chernobyl It's 4.6 and not great, not bad, not terrible, right When it was so much more. So actually the whole fun around this is to make conditions around your camera that allow it to work in the range where you do not overfill this silicon chip with photons and you start to keep meaningful data within the spectrum that the camera covers. So I guess now you said it's a bandwidth filter for you You were cutting out the noise so you get more of the meaningful photons hitting your silicon.

That's absolutely right. Yeah, and the water we're using really to chop off the infrared, those longer wavelengths above the 7800.

Because cameras are sensitive for that.

Cameras are sensitive to that. Not only can the imaging detectors pick that up when you can oversaturate in the red, but it's going to overheat your sensor and destroy sensors.

A typical DSLR cameras or GoPro's. What they have is they have a filter in front of the silicon chip. I also know that from astrophotography because for us imaging the spectrum emitted by hydrogen, which is 652, and that's very red, that's the most important one you want to shoot when you're shooting the sky objects And that's also a cut a lot by the typical filter you would have in front of DSLR. So the first thing astrophotographer does after buying a very, very expensive camera is to take down the player, making it useless for that every day.

And you talk about learning by doing, but that's something in addition to thinking about the energy that the thing that you want to photograph is giving off. And, like I said for a slide, some people are more interested in the flame structure. I usually want to do away with the flame structure. You have to think about that. You have to think about the filters, as you said, that are in front of your imaging sensor, and then you also need to think about the quantum efficiency of the camera. You know, i learned the hard way that that plays a big role.

Yeah, and to finish out, the filter and the sensor and the challenge you are facing. Literally the challenge is that your filter is on top of your sensor, it's literally at the sensor And if it's filtering out this wavelength it means it turns this photons into heat. So it just ridiculously hits up the silicon chip you have on your camera, leading to damage and the malfunction of the chip, because the sensitivity is in the function of temperature. That's also the thing you learn very hardly at some point, that the sensitivity and it also has quantum jumps. So after certain temperatures it's very differently sensitive than at lower ones. So if you want to do science with it, you have to make sure it's predictable. You have to make sure that the conditions through your whole imaging are the same in terms of efficiency of your silicon chip and sensitivity to light. So basically, blocking the things you don't need and focusing on the ones you need is actually critical, and this is a hell of a technical explanation of why someone put the camera into a fishbowl.

But that's the thing. That's exactly right, you know it's funny. And then talk about learning the hardware For anybody who's listening and trying this. We, like I said, we were shooting a lot with GoPro's and for a series of experiments we did on cold form steel shear wall structures I wanted to look at the cracking of the drywall in the decay phase to look at how the flame was passing from one compartment to the other. And we had set up a GoPro in a fishbowl like right up against the compartment and we were combining this with the narrow spectrum blue light illumination technique. And I went through and you know we'd done this a number of times and I set up the experiment and I ran through like 30, 30 large scale tests And then I didn't even have time because we were on a tight schedule to look at any of the video data And I went back and started processing the GoPro data and found out that GoPro chops off a lot of the image sensitivity around 400 nanometers. All of my video was garbage from that And so all of the all of the stuff that I wanted to look at was lost. It was unfortunate, and you know. So I discovered and yeah, i don't even know. In that case it might have just been the chip sensitivity on the particular GoPro I think we're using the Hero 5s or the 6 at the time or there may have been a blue filter on there to cut out the UV light. But you definitely need to pay attention to the sensor, any filters in front of that sensor and then how you're illuminated in the processing.

Because I also have struggled like the fancier the camera. The more expensive camera you buy, the more hidden inside AI based image processing things are. That's the reason why you can now take an iPhone and take a beautiful picture of your wife in a very complex scenery, in the very difficult light settings, and it's going to look like a professional photo taken 20 years ago, where 20 years ago you'd have to put so much effort into setting that. Because that's what the camera manufacturers want us to do. Have the convenience of taking and shooting go. You know, take beautiful pictures with minimum effort, but this means there's a hell of processing happening behind the scenes that you probably do not want.

It's a lot of the things that we want to be doing in fire, where you really a simpler system is often a better system. You know and I think you would mention this a while back Sometimes it's really advantageous to go with an inexpensive camera that you can sacrifice, not only because it's cost effective, but just the data that you get out of it is less filtered and less modified, so it's a little easier to understand what you're working with.

We also found that it's usually easy to recover the SD card, even from very badly damaged cameras.

I've been astounded. I've boiled the cameras for half an hour and still recovered the SD card from them. So usually the weak point is the seals on the battery case. The seals will usually give out and the battery itself.

It can like pouch and yeah, all the fun. I find immense joy in talking to someone who likes to traumatize video cameras for the science. Okay, Matt, you've mentioned near blue illumination system. That's one of the most fascinating techniques I've seen in fire science because it is magic So efficiently. advanced technology is indistinguishable from magic, and this one is a type of that. So let's tell the listeners about the magic of the blue light. Sure.

Yeah, and again, this is I'm not going to claim this is anything necessary that we originated, i guess, just kind of the context that we're using it is different. People use blue light in digital image correlation all the time as a way for illumination, but not for the same purposes, and people for years have been using different types of filters on fire to enhance certain parts of the spectrum. Really, our application of it came because, again, we were working with these lasers down at 405 nanometers as a way and we found there was a lot less disturbance down there and we were having this issue that when we were trying to do digital image correlation and photograph these patterns. We have two challenges. One is that the flames in between the cameras and the thing that we want to image would just saturate our camera sensor, so all we would see, like you said, is white, so that's not good. The other problem that you face is that when we're dealing with metal targets and they start to heat, they start to glow and they start to emit radiation themselves and it changes the color contrast between the pattern and then the background, which is a whole another challenge in and of itself. And so in that case, with digital image correlation. We are using visual based cameras. So we're in this 400 to 700 nanometer wavelength range, and so we thought, how can we pull out more of the structure through the fire? And we thought, well, if we have less energy from the fire down around 400 nanometers, and we just hit it with a pulse function of a lot of energy down at that wavelength and then filter the heck out of our image, so we take all the rest of those frequencies out. What we should be left with is illumination of the target And let's see what we can do with those images Like what can we work with then? And so that's all we're doing, and it's the most basic of things. We started with a 200, maybe a $100 high power LED light that we purchased online somewhere and just started with, you know, 5, 10, 20 kilowatt fires and trying to image through them, and we were like this is incredible. We were able to completely just blend out the fire, so you could, in the visual spectrum you'd be unable to see a target through maybe like a quarter meter of flames, and you would illuminate it with this blue light in this case, and you look at the images after the fact and it would be like there's no flames there. It was always kind of eerie because you still have these refraction effects right, these shrink effects of the density changes. So you'd be looking at an object and you'd see it apparently distorting and moving, but there was nothing that you could see in terms of flames in the structure. But we realized this was a really inexpensive and efficient way to enhance our visualization and fires.

I always found that the nearest thing to you know CSI series where they would take a camera and then zoom in and hands like turn off distortions and it would magically happen on the screen like very, very surreal and unreal. And here in fires it's really is like that. If you have a camera looking at a fire and you shoot the whole spectrum, it's white blob. You put a blue filter in it, You suddenly don't see anything because you're cutting most of the visible spectrum. And then you turn on the blue light And what you see is just the blue light which passes through the fire for the exact same reason why the fire is glowing in the red. So it means this is the spectrum the fire works at or the suit particles work at. So they don't also do not absorb the light at this spectrum, because absorbing and emitting is pretty much the same thing. It's just opposite direction. So they don't absorb your blue light, So they are basically translucent to that light. And if you cut all the other light they emit and absorb, you end up with just the thing that you want to see. Transparent fire. That's a brilliant technique, so where have you put that into use? You mentioned CLT. Was it there?

Yeah. So two things. First of all and I got a lot of questions originally when we started, you know, when we did this the first couple of times, we kept getting questions Well, is this something firefighters can use? And the answer is no right. For the same reason, this works really well in low soot fires Because, like you said, it's the soot particles that are emitting the radiation and glowing in the red and absorbing. If you are in a very sooty fire, it's not going to let you see through smoke and soot. But we do a lot of work in our lab using natural gas surrogate fires. Right, we'll burn something, we'll get the energy signature from that And then we'll recreate that energy signature with a natural gas burner, because it has the advantage that it's repeatable, right. Right, if something happens, we can turn it off. We can turn off the energy source immediately. So it works really well in our context.

So you would burn an armchair, put it into this database, which I'm going to talk with Matt Bundy soon Fantastic. And then you would just take a natural burner, take the heat release rate, put it into the burner control and just repeat it as many times as you like, but now with the ability to have not much soot and It's more repeatable, it's safer And it has the benefit for us And you have an off button and it works much better with visualization techniques as well.

So it has a lot of advantages. So just up front, we can't start putting narrow spectrum illumination on some sort of firefighter gear and have this work in a fire. It's not going to work. But it does work in clean burning fires where the problem is the visual obstruction from the radiation. So that's one thing. And then you were saying where does this work? well, the applications where it works well are, again, clean burning fires. And so when we built up the NFRL, the National Fire Research Lab, to include the structural fire testing, we knew we were going to be doing a lot of testing of structural members engulfed in fire. So these weren't furnace tests but these were large compartment fires, maybe with natural gas fire source or other fuel sources, and we really wanted to be able to see the structural component. So one application is where you're trying to see something within a relatively clean burning compartment fire or a well-ventilated compartment fire where you're not obstructed by the smoke. The other thing where it's proven to be really useful and we did some neat work with John Gales up at York University is looking at timber or really any material where you have a thin flame front And you want to see what's going on behind that. So he at the time and his students wanted to look at char around structural connectors, how does the char progress and what happens with it, and so the flame was an obstruction to seeing what they wanted to see. But you apply the blue glide technique and you really only have a centimeter or three of flames in front of the timber And that's very easily to make that disappear using the narrow spectrum illumination technique.

That's fantastic, and the surprising uses of the blue light.

There was a serendipitous thing that we discovered and using it that I think is you know, would provide a lot of benefit to the research community is we were looking at explosive spalling of concrete. When you have relatively freshly cast concrete, it has a lot of moisture in it. When it's heated rapidly it can get that water inside the concrete expands and then the concrete can burst and expose reinforcement, which is a bad thing. We started imaging these concrete blocks placed in fire to look at really the expansion and then the explosion of the concrete. What we realized is that as the concrete heats, the leachates, the water in the concrete gets pushed out preferentially along the microcracks in the concrete and it contains phosphates and other minerals that glow in the blue light. You end up seeing all of the microcracking in the concrete before fracture occurs as heating is going. You have this predictor. It was a really neat way to look at fracture in concrete because you were getting basically fluorescent imaging overlaid on top of the advantage of having the blue light. I think that there is a study waiting to be done out there on concrete spalling and fracture using the blue light imaging. That could be really really useful. I was also. I was really hoping. I don't know if anybody's done this yet, but I think another very relevant application we'd be looking at for sahad fires, where again you have you don't have the obstruction from smoke, you have a lot of flame, but you're very interested in knowing what's happening with the siding of the building or how the materials behind the flames are degenerating. That's a perfect application for this.

I've actually completed the whole setup for that. And then COVID happened and this was put on the stop, but we still have the theater lights and the filters and everything. Perhaps we should go back to that. I also think again, this orientation where you have a thin flame layer which eventually prohibits you from just imaging what's behind the flame, because there's a flame obstructing, but indeed this could be something where this light could really shine. Wow, what a point.

Yeah, absolutely. Then the other application and I've talked to a couple of people about this just doing furnace testing, where you're particularly furnace testing of structural components Often you're using a gas heating system there and you're interested in the component that you're putting in there. You could very easily have one or two ports of a furnace be dedicated to a blue, lighter and narrow spectrum illumination and another port where you have your camera and your optical filters and improve the visualization, because a lot of the difficulty and scene inside of a furnace is just caused by the radiation. You see a lot of glowing things, but it's often hard to see the thing that you're really interested in studying the behavior of, especially, that you will have furnace walls that will also heat up and start to glow.

Then you get more radiation.

Those are all applications, very practical applications, where this can be applied. As a side note, we actually have built the same water filters that we use for the camera, as we put them in front of the blue light so that we can move the blue lights right up there Anytime you need to knock radiation out but transmit visible light. you can do this. In essence, you can put a curtain of water between your furnace port and your illumination source and then you can get it right up there. You're not losing power by having to put the illumination far away from the source. You can get it right up close to the thing that you want to illuminate.

In this application and, by the way, there are papers on that and I'll try to link them in the show notes so you can actually read much more in depth on these techniques we're talking about We were using this theater type light, led, like focus lights with blue color. I wonder does there exist some sort of strobe light You would have in a camera that would just do a flash for the second that you take picture, but very, very strong, of this particular wavelength?

The short answer is I think there's a lot more to do on this topic. Actually, we're always looking for students interested in these topics. I think somebody could do a whole postdoc on these sorts of things. There's two fields which I think would be very interesting. One we started to look at it was looking at a strobe light or looking at a time synchronized pulse of the lighting and the camera in order to further reduce the effect of flame structure. Actually, the way that we do it in a lot of our techniques, we'll time average over like a second or five, and we just average the effects of the flame structure and then filter it out. That's a quick and dirty way to do it. But the other way to do it would be to pulse your light and your camera triggering at the same frequency and then overlay the images and you could play around with that to enhance visibility. That's really nice. I think there's a lot that could be done there. The other thing which we haven't done and there's some safety challenges with this is there's no reason you couldn't use a coherent light source, a laser, instead of using an LED light like we do, which is a diffuse light source. then you could have a very targeted amount of energy at the wavelength you want and basically just hyper illuminate a very small area and then filter. That would give you a very high intensity illumination for when you have intense fires. I've looked around a little bit. I haven't found any really good commercial products, plus a lot of time. We're using these in an open lab space and a near ultraviolet laser of that intensity shooting across your lab would be a serious eye hazard as well. You need to do that in a controlled space.

I've never thought about that actually, but again, my astrophotography experience. you don't even have to average, You can use some median filters. There are specific Kaplan filters where you can do very fancy stuff to really filter out.

You could spend a whole career just doing these things.

I love you. I've picked a golden hobby to get away from fire and I have something to do that's absolutely unrelated to fire.

When you're ready for a break, you take a sabbatical and you can come out, and we'll spend six months letting our imaginations go crazy in our lab.

Okay, let's move forward. The blue light magic thing is fantastic. Let's move on to 360 cameras, because that's a hell of an invention for fire, a very interesting one. If I remember, you've put them into burning compartments. You've barrowed one in the forest and burned the forest down. Tell us more how you developed the 360 technique and what was your initial goal in developing this type of videos.

We kind of touched upon that before. That was one of those moments. I don't know how long you've been doing science and engineering, but that was really one of those things that I'd been thinking about for a year or three, but I didn't know it and there was a need to do this. But when I realized that it could be done and how it could be done, it kind of all came together in the course of an hour. It was during the cross-laminated timber experiments, where we were shooting these compartments in 360, pulling the cameras out, and then we would move our 2D cameras into the doorway in a bowl of water. We were getting these really nice upfront videos and then after the fire we'd go in and we'd document the damage in 360 and I was like why the heck don't I just leave the camera in the fire? I know it can be done. I know I can do this with a 2D camera. I know we've got the camera technologies. at the time We were using these big, almost the size of a volleyball camera cubes, but cameras were getting the 360 cameras every couple months. They were getting smaller and better. I knew the camera technology would very soon be at the point where we'd have a little camera a hand camera that we can now and now they exist. Actually, I think I was sitting. I was sitting waiting for my daughter at a birthday party once and I had a sketch pad there and I just started sketching up concepts. For how could I put a 360 camera in the fire and leave it there? We need the water to absorb the radiation, so we need a certain volume of water. We need it to be circulating, because it's going to heat up very quickly when it's 1,000, 1,400.

C in the compartment.

How do we get that out of there? What's going to happen if we start to get boiling in there? Are we going to get air bubbles in there? How do we deal with those sorts of things? The structure itself that the camera supported on needs to not overheat, so that has to be water cooled as well. and just started going and pretty quickly came to a solution which is essentially it's like a fishbowl, like one of those classic round fish bowls, sitting on top of a two-inch steel pipe, and you have water flowing in keeping the stand, which is the pipe, cool. It also serves as your optical filter for the radiation and then you have a hidden line inside the center of your stand that forces the hot water, after it's absorbed the radiation, back out again. That's where it started. I built a prototype, i think, for under 500 bucks worth of parts, and we just started trying and when we failed a lot we broke a lot of glass and burned up a lot of cameras but kept iterating on that design to the point now where we're at. We have essentially a setup which we use in the lab which is on rollers. We can roll it into the middle of a very intense compartment fire capture video for a few minutes and then either leave the camera in there or pull it back out again. It's just a really neat perspective to see what's going on in a fire.

You called it Bob.

We called it Bob. I toyed with a bunch of names for this for a long time, but it became Bob one because my kids know when I forget the name of somebody or something I always call it Bob. My memory is pretty bad. I thought that was my own personal reason for it. But the other is our technicians kept calling it the bubble, go get the bubble, go get the bubble. I was like this can be the burn observation bubble or Bob That stuck People like saying go put Bob in the fire.

But it's not a fishbowl. the bowl that you use It's a chemical thing.

It's a spherical chemical reactor. Actually, it was one of those things. I was walking through our hallways at NIST and I looked in a chemistry lab and they had this glass reactor which is like a sphere, with an opening at the bottom and a flange on it. These are used for chemical processes and these are manufactured out of borosilicate glass.

There are meant to be heated by a burner.

They're meant to be heated by a burner and you can purchase them off the shelf in various sizes for a couple of hundred dollars. That's where we started, although, again, trial and error, we have failed a lot of those because of the thermal shock when you have a flash over. We were failing a lot of those domes. Ultimately, we've switched over to a quartz glass, which have to be custom made and they're more expensive. They're triple or quadruple the cost, but they're very insensitive to thermal shock.

We broke ours as well, partially because you will have a large gradient from the exposed side and unexposed side, I guess.

That was one of the things I learned early on. This is what I love about working at NIST, but at any facility When we were doing the laser work, we were putting borosilicate glass in front of our water cooling just because we didn't want the smoke and the soot leaking out into the laps. We had a window and we were very successful in running these furnace tests. Then we'd come back the next morning and there'd be shattered glass all over the floor and we'd have to replace the borosilicate glass window. It turned out that one of the high temperature glass fracture experts in the world was in the building next door to us. I called them over after a test and I said look at this, what are we doing wrong? His answer was you're shadowing the edge of the glass. We had a metal retainer ring that was clamping the glass up against our wall. What was happening is that part of the glass was getting less hot than the exposed surface. Then, when we let it cool overnight and you have edges if they weren't properly ground edges then it would induce a fracture and the glass would blow up during the cooling phase. What we did is we removed that plate which caused the thermal shadow onto the glass and we precision ground all of the edges of the glass and the problem went away. So this was a case where it was better to have higher uniform exposure rather than partially shielding it. And so when I designed the Bob the first time, one of my goals was to make sure that I had as little thermal gradient as possible anywhere. I'd rather have the entire glass exposed uniformly, and so that's been a problem that others have been encountered. If you make too massive of a clamp around the glass, you can get fracture, and also the details of things like the edge, grind on the glass or how anything that's rigid comes into contact with these materials, as they're heating, becomes very important, and so you figure out over time okay, i have to bevel this edge, or I need to put in a ceramic gasket in here, and over time you eliminate those problems.

So now you have an observatory that you can literally put inside any burning fire, a flasher of a fire, forest fire, whatever, whatever size of fire you have. Very easy for you now, as it's on its own tracks, you can move it in out. Brilliant. Tell me how the microphones get to play with that.

Yeah, the microphones could spend a whole episode on that. So originally, when we started using the cameras, we just used the onboard audio And, as you can imagine, the audio on the camera when it's underwater is muffled. You don't hear anything, and it was, you know. After shooting even one of those, we realized that the experience of looking at a fire in 360, particularly if you put it in a head mounted display like in your Oculus headsets without the audio it just doesn't feel like you expect it to feel. And so the first thing that we started doing when we were burning fires in the lab, we would take two shotgun microphones and set them up in an XY pattern. So basically we put them at a distance from the Bob unit and we would have them cross in an X, which essentially gives you stereo sound in that cross pattern, if you can kind of imagine that. So if you're putting on a headset afterwards and viewing this 360 video, you have biornal sound. You know sound in each ear, left right, left, right But it doesn't track your head movement right, it's not true surround sound. But if something happens on your left side, you hear that. It's on your left right, or if it's on the right, you can hear, it's on your right, and so that was kind of our first iteration of using microphones. What we've found, though, is the places that we've been shooting, most of the time, which is in a compartment. Fire directionality doesn't matter, because sound bounces all around the room, as you know very well, and so in those cases, we often just set up one or two capsule condenser microphones just to get a sound, Just to get a sound, and that works really well in a room fire.

What sounds are you looking for? like cracking glass, The things?

yeah, and it's interesting. The fidelity of sound that you need is not as critical as just having something. It's. I've found it's the growth and decay of the fire. That kind of the rise of kind of an ambient noise sound as things are coming is very interesting. You also hear the pulsing in a compartment fire. You hear kind of the pulsing effects from the wind generated for this And you can, when you can see visually how the fire is growing and decaying and you can hear that at the same time. That connection is really important. And then if you have something happen we've had smoke detector batteries explode and that sort of thing You know, when you see something happen instantaneously in the fire, to hear that sound at the same time your brain thinks, okay, yeah, that makes sense. And where it was really useful was when we took the system out into the woods. We're working with the US Forest Service and the New Jersey Forest Fire Service and a group from University of Edinburgh. They were doing prescribed burns. We thought, hey, let's take this thing out into the woods and, you know, bury a water tank underground and put the pumps underground and we'll see if we can capture some wildland prescribed burns. In those cases we just buried microphones at, you know, a couple meters out from the camera, underground, and that was really powerful to see. It was a calm day when we were doing the experiments. So, you know, the video starts out and you see blue skies, you can kind of hear the sounds of the forest And then, off in the distance, you start to see the fire coming and you have this slow build of the rush of the wind And then you start to hear the crackle of the flames as the trees start to ignite. I mean, that's it's hugely important in understanding and appreciating what's going on in these fires.

One aspect that we haven't touched on here is that these are great tools for scientists, of course, but also, i think, especially the 360 ones immersed experience. I think this could be great training toolset.

Oh, absolutely.

Yeah, have you done anything like that?

So we you know, nist itself has not developed any training. Actually, it's interesting, the majority of the requests that we get for the 360 videos actually come from documentary filmmakers a lot who are working on climate science or on fire science. You know they'll use the videos as part of their work. Just their bloody good damn videos, because they're good videos, and the other thing is training. You know anybody who wants our videos, we provide them. We provided them to different organizations that develop training on their own, and what NIST has tried to do is, when we shoot a new video, we put it up on a website. So we're trying to curate these things And you know, my goal was always to have, just like a collection of fires, of outdoor fires in different terrains, to have a number of different type of compartment fires that anybody, like you said, for either for education or training, can go in and they can pull these things down and use them for whatever purpose that they want.

Fantastic, madzi. see, we've discussed whether we have an hour to fill with video imaging and visualization, and it's already almost past. So, from one amateur cinematographer to another, if a person goes and tries to shoot their first, fire like what's the most basic errors they can make, what's the most painful ones and how to avoid them?

Yeah, now I think I think, as you know as well, you often need to. I forget how you said this earlier. You got to turn off some of the smart features of the cameras, absolutely, as well as possible As raw as possible, you know, even to the point of shooting in raw. I shoot mostly in jpeg because I'm lazy and I don't want to go and do the post-processing. But no, it's much more about the camera settings. I think you know some of the important things.

Motion blurs like things like that right.

Yeah, turning off a lot of the automated settings, fixing your white balance to a single value, taking down your exposure levels I think I usually shoot it like minus two depends on the camera that we're shooting on. You can play around with the ISO, the ISO speed as well. You don't want to get too grainy of an image but it depends on the thing that you're shooting. But really turning off the automated features, turning down the exposure, which often means that before the fire starts things tend to be very dark. So when we shoot in the laboratory we'll use a lot of high-powered white light to illuminate the setup, because we're going to take all of the camera settings way, way down in anticipation of the fire growing later. Because, as you said, you don't want to start oversaturating the camera sensor once that fire is giving up all the energy. And sometimes I'll even throw some neutral density filters on there. These are basically flat filters that will just dim all of the frequencies uniformly.

Like a smoked glass, like a smoked glass. Exactly, we've learned the same thing very tough. You want your image If you know there will be fire in there. You want your image when there's no fire to be really dark, because once the fire comes in, you'll have a very quick rise of brightness of the object And you'll eventually end up with a white spot where the fire is. But when you start very dark, there's a good chance you will be able to capture. The second thing is you mentioned white balance. This is also something you perhaps do not want to have on auto, because you want to have this fixed, because once the fire comes, the camera will try to compensate for the new brightness, finding a new white balance zero, and this means it will change the Like. You can also end up with having just a picture of a fire and nothing else around, and when you're looking about everything around the fire, that's not a great outcome either, right.

Well, no, that's absolutely right. I mean, that's going back to some of the things earlier. One that's another advantage of shooting with the natural gas fires If the video or the images are really important, sometimes we'll even just put a burner out in the setup. We'll crank the fire up if you have any ability to do this, and check our settings at the maximum fire size before we do the actual experiments, that's really beneficial if you have the ability to do that, and it really depends again on the application. When I'm shooting, i tend to bring more of the context of what's going on in the room and not see the flames as much, and others may want to see the flame structure more, and so you can play around with your settings to bring out and emphasize the features that are important to what it is you're looking at.

When we have a new colleague in the office and we're doing Cosmo test in the building and they are burdened with the role to do the imaging of the test, i always end up with like 20 pictures of my trays burning and not really the picture of the smoke systems in the building, which is sometimes. But that's the context.

There's two types, There's two philosophies. One is the people if we hand our cameras over to. There's the people that really like flame structure, And then there's people who like to shoot more of the context, And then there's also I find there's a dichotomy. There's some people like to be very precise and take one or two pictures of everything they want to see, And other people are kind of of the philosophy I'll shoot 100 photos of everything and sort it out later, which is a way to do things, but it can make the post-processing very difficult when you have a hundred different versions of every single set up that you're testing.

Fantastic, Matt. let's wish every amateur fire scientist photographer a beautiful set of not overshot pictures, perfect white balance and great images, And, once you have them, submit them to IFSS picture contest, which I think is open as we are shooting this interview And there's a fierce competition between amateur fire photographers. That's the place that. that's the fire national geographic picture contest. I guess Are you submitting.

Absolutely. I have several times in the past. I do not know if I will submit this year or not, but we participated several times and I agree it's a great venue to share your work with the community.

I mean it's nice. It's just Golden Roll with Winston. That's probably very discouraging. He's got some good photos too. You've got to give him that. Yeah, I certainly do, especially when it's a picture of a newly found combustion process. Anyway, Matt, it was a huge pleasure to talk about visualization with you in the Fire Science show. Thank you, Thank you so much for coming, Thank you so much, And that's it. Before we started, Matt didn't believe me that we are going to record an hour long podcast episode about recording fires with cameras, But indeed we have not even finished everything we had on the list. We've missed the lighters, we've missed more advanced surface mapping buttons. So much to cover in visualizations. But it's just fun when you, when you have your soulmate who loves burning cameras, just like I do. Well, I don't like burning cameras, I like taking pictures of fires, and perhaps the cameras sometimes die during the process. That's, that's a little price to pay for pretty pictures, But hey, that's that moves fire science onwards. I hope from this episode you've realized that using visualization is not just about having pretty content for social media or beautiful posters, but it truly is science and something that really helps us understand fire phenomena in better depth and learn more from the scarce experiments that we are able to do, And whoever tried ever recording fires for scientific use. I bet you acknowledge the issues and challenges that are with here And I hope you really appreciate the little nuggets of knowledge that Matt brought to the table. I definitely took some for myself and we are improving both our 360 and blue light setups that we have built in. It be inspired by Matt and we will continue with that. So, yeah, oh, and by the way, if you have pretty pictures like really submit them to IFSS picture competition. You can find info in the website. The deadline is 30th June, So just a few days. If you're listening, go ahead. Fire science shows on the schedule. You should make it. Perhaps there will be an extension. However, I kind of doubt that. They rarely do extensions in IFSS. But yeah, it's a nice competition. Let's have someone else and Golnir win it. I guess Golnir will win again. They have just two good fire pictures at Berkeley, but yeah, still a challenge and fun to participate. Anyway, thank you for being here. Appreciate you. See you here next Wednesday. Bye.