Mrinal Kanti Bhowmik, Ph.D. (Engg.)
Department of Computer Science and Engineering
Tripura University (A Central University)


Night Vision in Adverse Weather Conditions

Night Vision in Adverse Weather Conditions


  • Challenges :
    1. He is actively involved in infrared based object background segmentation at night time adverse weather environments such as foggy, rainy and dusty conditions etc. as shown in Figure. 1. The background model-based background segmentation methods are mostly pixel-level approaches but in infrared FIR imaging, the pixel-intensity based methods are not well suited. There have many key issues that are related to object detection at night using an FIR camera as shown in Figure. 2, such as the following: (i) Flat Cluttered Background: The infrared radiation signal must travel from the target to the camera sensor among adverse atmospheric particles and is attenuated due to scattering; the loss of radiation along the way produces a blurred flat region. In addition, with the thermal sensors, because of large variations in the surface, which includes hot and cool objects such as buildings, vehicles, animals, humans, and light poles, the foreground objects and the background scene become indistinguishable; (ii) Temperature Polarity Changes: Thermal temperature adjustment during the maiden appearance of a moving object in a video sequence causes illumination-type effects in the background model from the current video frame and, therefore, yields false classifications. (iii) Background Dynamics: Outdoor scenes are affected by movement in the background, e.g., due to waves or swaying tree leaves.


    (a) (b) (c) (d) (e) (f)
    Figure. 1. Sample frames of the created dataset at night time (a), (b) a visual frame and the corresponding thermal frame, respectively, under dust conditions; (c), (d) a visual frame and the corresponding thermal frame, respectively, under rain conditions; (e), (f) a visual frame and the corresponding thermal frame, respectively, under fog conditions.


    (a) (b) (c)
    Figure. 2. Key Challenges under thermal imaging in outdoor uncontrolled adverse environments (a) flat cluttered background; (b) temperature polarity changes; (c) background dynamics.


  • Scope:
    1. The literature of benchmark datasets are contains of very limited video sequences in different adverse weather conditions. Thus, it is difficult to evaluate the robustness of object detection methods under atmospheric conditions, especially for night vision, because more than half of object-related accidents occur at night. In contrast, our motivation might be providing a new dataset comprising of several adverse weather conditions with large number of video sequences compared to existing datasets.


  • Dataset:
    1. Therefore, we are designing a standard night-vision video dataset that is based on several atmospheric-weather-degraded conditions and covers many real-world scenarios. The considered atmospheric conditions are dust aerosols, fog aerosols, rain aerosols, and a low-light environment, under which we utilize a thermal camera. The dataset is naming as ‘Tripura University Video Dataset at Night time (TU-VDN)’. The TU-VDN dataset provides a realistic diverse set of outdoor videos in night vision that were captured via a thermal modality. The current dataset consists of 60 video sequences that were captured under various atmospheric conditions. The key features of the designed dataset are as follows:

      (i) Each frame contains multiple types of moving objects, e.g., pedestrians, various types of vehicles, bicyclists, motorbikes, trains, and pets;

      (ii) The night video clips were captured under three outdoor atmospheric scenarios, namely, dust, rain, and fog, which produce flat regions in thermal scenes. In addition, the captured scenes are mostly in urban areas, which correspond to larger surface variations due to the presence of hot and cool objects such as houses, warehouses, office buildings, streets, and residents. Therefore, areas with varied background and adverse weather conditions produce thermal characteristics that lead to an increased flat cluttered region in the target area;

      (iii) A conventional challenge is encountered, namely, a dynamic background due to shaking trees, since the whole dataset was recorded in an outdoor environment;

      (iv) The key issue with the FIR camera is thermal temperature adjustment during the maiden appearance of a moving object in a video sequence, which causes illumination-type effects in the background model from the current video frame;

      (v) Motion-camera-based videos are captured by mounting the camera on a moving vehicle, where the camera and objects are moving and shaking simultaneously.


  • Our Proposed Algorithms to mitigate the challenges :
    1. If the background emits the same amount of thermal radiation as objects, e.g., a cluttered background, the foreground and background regions will be indistinguishable. We investigated the performance of a perceptual discrimination salient-feature-based methodology on a flat cluttered background. Most methods that are used for foreground object segmentation in video sequences that were captured by thermal or visual cameras are composed of two modules: feature extraction and maintenance of the background model. However, finding a satisfactory reference or background model for background subtraction is difficult when there are several real-time objects in thermal frames. Therefore, we working on a satisfactory background segmentation model that uses the novel Akin-based Local Whitening Boolean Pattern (ALWBP) salient features. The salient features handle flat cluttered regions (as shown in Figure. 3) and background model handle background dynamics (as shown in Figure. 4) and temperature polarity changes (as shown in Figure. 5) to increase false-negative ratios.


    Figure. 3.Classification of moving objects from a flat cluttered background.


    Figure. 4. Reduction of false classification in a frames sequences due to background dynamics.


    Figure. 5. Reduction of false classification in a frames sequences due to temperature polarity changes.


  • Featured Article(s) in the Proposed Domain:


    1. Anu Singha, Mrinal Kanti Bhowmik,"Salient Features for Moving Object Detection in Adverse Weather Conditions during Night Time", IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), IEEE, IF-4.046 (SCI), 2019, DOI: 10.1109/TCSVT.2019.2926164.

    2. Anu Singha, Mrinal Kanti Bhowmik,"TU-VDN: Tripura University Video Dataset at Night Time in Degraded Atmospheric Outdoor Conditions for Moving Object Detection", Proceedings of 26th IEEE International Conference on Image Processing (ICIP) [Tier II Conference], Taipei, Taiwan, September 22-25, 2019. (Accepted)











Total no of visitors: 6165

© 2019 | DR.Mrinal Kanti Bhowmik

Designed By : Santanu Das Contact
santanud803@gmail.com DR. Mrinal Kanti Bhowmik
+91 8837401073 mrinalkantibhowmik@tripurauniv.in
+91 9436129933
Admin Login