Journal of information and
communication convergence engineering
(J. Inf. Commun. Converg. Eng.)

Systems based on outdoor image processing are used in several fields, including intelligent transportation systems, outdoor object recognition systems, and remote sensing systems. However, unlike normal images, images obtained during harsh weather conditions containing atmospheric particles such as haze and smog exhibit a significant lack of visibility owing to light scattering and absorption. These hazed images may hinder the execution of image processing systems and reduce their reliability [1,2]. Therefore, single-image dehazing (SID) methods are required to restore outdoor images obtained during bad weather.

Early dehazing algorithms used traditional image processing methods based on histogram equalization (HE) [3] or algorithms that dehaze the same scene by capturing images with different degrees of depth and polarization [4]. However, HEbased methods do not reflect the physical characteristics of haze; moreover, methods that use multiple images have limitations that require additional research to overcome. Accordingly, SID has been actively studied in recent dehazing research as an alternative to HE [2].

As a classical SID method, He et al. [5] proposed the dark channel prior (DCP) because haze-free regions generally have color channels of very low intensity. Because DCP can remove haze simply and effectively, several improved algorithms [6-8] based on it have been proposed. In particular, guided image filtering (GIF) [6] applies a guided filter to the transmittance map of the DCP to effectively preserve the image quality in hazy images, whereas weighted GIF (WGIF) [7] incorporates edge-aware weighting into GIF to prevent the halo effect and preserve edge regions. Additionally, Tarel et al. [8] proposed fast visibility restoration (FVR) using a modified median filter to reduce the long soft mapping time of the DCP. In recent years, SID has been studied using machine-learning models. For example, a color attenuation prior (CAP)-based method [9] was used to modify a depth map as a linear model for hue, saturation, and value (HSV) through training, thereby producing results without noise. However, the CAP method slightly lowers the image visibility owing to soft edge processing; ICAP [10] suggests a new depth map through a secondary mode to compensate for it. Dana. et al. [11] proposed an efficient depth-map estimation method by defining haze lines within color clusters, and Ngo et al. [12] proposed the machine-learning approach based on the quad decomposition method and the Batcher network to compensate for background noise and color distortion. The SID methods developed to date have demonstrated excellent performance in terms of visibility and edge preservation; however, dehazing based on haze depth is difficult, resulting in an excessive increase in saturation and contrast in light haze regions. Furthermore, dehazing the entire image causes severe color distortion in the sky region and disturbs the perspective of natural images.

In this study, we set up a depth map based on pixel brightness to preserve the visibility and perspective in hazy images. We used the correlation between the value (V) and depth of the haze in the HSV color space to describe the spatial sense of the hazy image and calculated the deviation in V of the airlight to define the haze concentration (HC). Based on the defined HC, a weight map was designed to preserve the sky and light-haze regions; this map was applied to the depth map defined by the CAP for dehazing. The dehazing method proposed in this study provides highly visible and natural results by emphasizing the contrast of nearby objects in the image while softening the sky region.

The remainder of this paper is organized as follows. Section 2 describes related research, and Section 3 describes the proposed method. Section 4 presents the experimental results and analysis and Section 5 presents our conclusions.

A. Atmospheric Scattering Model

Numerous studies have used the atmospheric scattering model proposed by Narasimhan and Nayar [13] for dehazing [5-7,9]. The atmospheric scattering model [13] defines the hazy image to be dehazed, I, as follows:

where x is the spatial position of the pixel in the image, J is the scene brightness of the image without haze, A is the airlight, and t is the transmittance. t is generally defined as follows:

where β denotes the atmospheric scattering coefficient and d denotes the depth of the scene. Here, d(x) has a large value close to ∞ when the medium is extremely far away from the observer. In this case, as per Eq. (2), t has a small value close to zero; therefore, A has a value similar to I [5], which can be shown by the following equation:

To estimate A using Eq. (3), we need to know the position of a sufficiently large d(x). Generally, as the haze region is located far away from the observer, it has a sufficiently larged(x) [2]; based on this, we can estimate the depth map of the scene to calculate A.

Furthermore, if d is known, t can be calculated using Eq. (2). Based on this, the SID methods [5-7,9] estimate A and t or d, substitute them into Eq. (1), and output a restored image of the original J.

B. Color Attenuation Prior (CAP)

The CAP [9] describes the relationship between brightness and saturation in a hazy image based on statistical and atmospheric scattering model equations. According to this algorithm, the denser the haze concentration, the stronger is the effect of the additional airlight, resulting in a decrease in saturation and an increase in brightness. Because the haze concentration is affected by the scene depth d(x) according to Eq. (2), the brightness and saturation of the hazy image have the following correlation:

where c(x) denotes the concentration of haze, and v(x) and s(x) represent the brightness and saturation of the scene in the HSV color space, respectively. Based on this equation, a new linear model d_c(x) is proposed using machine learning for the CAP[9]. The depth map d_c(x) is represented as follows with respect to v(x) and s(x):

Here, a₀, a₁, and a₂ represent parameters, and ε(x) denotes a variable representing the model’s random error. As the parameters can vary according to the image, Zhu et al. [9] designed 500 training datasets consisting of hazy images and their corresponding scene depths to enable estimation by generalizing Eq. (5). The trained datasets were used to measure the scene depth and airlight by matching 500 clear images to the depth map of Eq. (5), with random brightness and saturation inputs. The designed 500 training data sets estimated parameters a₀, a₁, and a₂ using the MLE supervised learning model, and Eq. (5) can be expressed as a generalized depth equation for brightness and saturation. The parameters estimated through this process were generalized and defined as a₀ = 0.12178, a₁ = 0.95971, and a₂ = 0.780245.

Moreover, ε(x) can be calculated using a Gaussian distribution model N(0, σ²) with a mean of 0 and a distribution of σ. Zhu et al. [9] expressed it using parameters a₀, a₁, and a₂ as follows:

Here, using the values a₀, a₁, and a₂, σ was calculated as 0.041337.

A. Estimation of Airlight A

Fig. 1 illustrates the process of extracting A. According to Eq. (3), A has a value similar to I(x) in an image with an extremely dense haze, and d(x) has a large value. Similarly, many existing studies have estimated the airlight A using I(x) according to the definition of the atmospheric scattering model in Eq. (3) [14]: The well-known and effective method is to detect the airlight A using a set of the brightest higher-level pixels [1-2,5-7,9]. For example, He et al. [5] detected the brightest pixel in the local area of the DCP, and Zhu et al. [9] detected a set of the most brilliant pixels in the depth map defined in Eq. (5). The proposed method also detects airlight A in the HSV space using the depth map of the CAP [9].

Figure 1. Depth map and position of the Airlight. (a) Depth map, (b) Median filtering, (c) Position of A.

Fig. 1(a) shows the depth map d_A to output A. In Fig. 1 (a), A represents the largest value (brightest pixel) in the depth map.

Because A is calculated using pixel alignment, as in Eq. (7), it can be determined using the variables V(x) and S(x) according to the pixel position in Eq. (5). Therefore, the following equation was applied by approximating a₁ and a₂ in Eq. (5) to 0.95 and 0.7, respectively:

Moreover, the depth map d_A may contain objects brighter than the haze region, such as the t-shirt shown in Fig. 1. Therefore, a 7×7 window is used to perform median filtering of the depth map d_A [15, 16]. Fig. 1 (b) shows the median filtering results for d_A. Because A is located at the brightest pixel of d_A, we define A using the sorting matrix $d_A^{sort}$ of d_A as follows:

where w and h are the width and height of the entire image I, respectively, and x_A is the spatial coordinate where A is located. Specifically, xA represents the top 0.1%-pixel position of d_A, and A represents the I(x) image when the haze is very dense. I(x) corresponding to A is represented by the dot in Fig. 1 (c).

B. Haze Concentration (HC)

Fig. 2 shows the effect of dehazing based on the haze deviation H and HC.

Figure 2. V-map and H-map of a hazy image, and the output images of the conventional and proposed methods: (a) original, (b) V map, (c) H map, (d) hard haze image: WGIF[7], (e) light haze image: CAP[9], (f) proposed.

Generally, the haze concentration increases with distance, as described in Section 2 and shown in Fig. 2 (a), where all regions are affected by haze. However, most conventional filters apply the same equation to the haze region [5-9], which leads to the problems shown in Fig. 2 (d) and (e). Specifically, they show patterns in which the haze remains, as shown in (d), or the contrast in the short range becomes severe, as shown in (c).

Therefore, we calculate the deviation of V and airlight to estimate the effect of the haze added to each pixel and define HC. As shown in Fig. 2 (f), the proposed method improves visibility while preserving close objects.

Fig. 2 (b) shows the brightness V of the HSV space shown in Fig. 2 (a). In (b), the bright region represents the added haze and the black dot at the top represents V(x_A), the brightness value of A.

As mentioned above, the airlight A with a distance A represents the set of the brightest high-level pixels for the scene [1-2,5-7,9]. That is, the brightness V(x) of a scene generally shows a smaller difference than the brightness V(x_A) of A when the haze is thicker. Thus, we calculate the haze deviation H(x) as follows:

where x_A is the spatial coordinate of A, and H is as shown in Fig. 2 (c). According to Eq. (9), H(x) has higher values in closer regions and lower values in farther regions. The road illustrated in Fig. (c) has a lower value when it is the farther away, and the closest object, the bicycle, has the highest white color.

However, because Eq. (9) is defined according to the general airlight assumption, it can generate errors in scenes where highly bright objects are present. For example, Fig. 2 (c) shows a very low value for yellow clothes. Therefore, this study applies the DCP [5] assumption that in pixels without haze, the brightness of at least one of the three RGB channels contains a dark value close to 0 [5] and the airlight [1-2, 5-7, 9] assumption that the haze area comprises a set of the brightest pixels to obtain the maximum and minimum values of H(x) and defines the difference between them as the haze concentration (HC).

HC is calculated using the maximum and minimum values of H as shown below.

where max(H) and min(H) denote the H(x) values of the least and most dense haze, respectively. According to Eq. (10), HC represents the difference between the pixel with almost no haze and that with heavy haze, that is, street haze concentration.

In images, the sense of space felt based on the far and near fields is represented, whereas in hazy images, the far field shows denser haze than other objects, often resulting in images in which only the far field is affected by haze. Owing to this sense of space, the HC has a large value in the far field and normal images, thus degrading the image reconstruction performance. However, as an exception, an image with extremely dense haze, where even close objects are heavily affected by the haze, shows a low HC. In other words, the HC shows a tendency to gradually decrease in the order of distantview, general, and dense images.

Although an actual haze dataset that can accurately determine haze intensity is necessary to verify the accuracy of HC calculations, no study has yet suggested such a dataset. Therefore, this study compares the results of the Synthetic Objective Testing Set (SOTS) of RESIDE (REalistic Single Image DEHazing) [17] and Dense-HAZE [18].

SOTS provides indoor and outdoor composite haze image data. This study uses indoor SOTS data as images of general density because they provide 50 scene images of haze density intensifying in levels 1-10. Moreover, Dense-HAZE provides a dataset of 55 actual dense haze images, enabling the checking of the HC characteristics in thick haze images. The HC results for each dataset are presented in Fig. 3.

Figure 3. HD Results of RESIDE and Dense-HAZE: (a) Average HC for indoor, (b) Average HC for Dense-HAZE.

Fig. 3 (a) shows a visualization of 500 indoor datasets by averaging 50 images of levels 1-10. Indoor refers to a hazy image with a low density, whereas Dense-HAZE refers to a high-density hazy image. Fig. 3 shows the HC trends for these images.

Fig. 3 (a) shows the tendency of HC to decrease as the haze density increases, matching the HC characteristics described. Moreover, the HC calculation results for the indoor image shows that all datasets exceeded 0.4; thus, the mean HC values in Fig. 3 (a) are all 0.68 or higher. By contrast, for dense haze, all images except one show 0.6 or less. These results confirm that HC tends to have a lower value in images with higher concentration.

C. Weight-reflected Depth Map

In this study, hazy images were classified into three categories according to the HC for dehazing in terms of preserving the sense of space, as shown in Fig. 4, where (a) is a scene with haze only in the far field, (b) is a scene with severe haze, and (c) is a normal scene. The haze deviation in Fig. 4 shows the H map, where the two dots represent min(H) and max(H), respectively. They are represented visually and numerically in terms of deviations for easy comparison.

Figure 4. HC-based hazy image classification and haze deviation H-map.

When the deviations shown in Fig. 4 were substituted into Eq. (10), Cases 1, 2, and 4 show HC = 0.8095, 0.4431, and 0.63918, respectively. Case 1 shows a high value of max(H) and the largest HC because the object has almost no haze. In contrast, Case 2, with dense haze in all regions, has the lowest HC, with max(H) showing a dark value of less than or equal to 0.5. Case 3 shows a value between those of Cases 1 and 2.

We classified hazy images according to their HC and designed a weight map G that reflects this condition. The initial value of G was set as a zero matrix with weights applied only to H(x) to satisfy this condition.

First, data statistics were used to set a reference value for the HC. In Fig. 3, all 54 scenes, except one in Dense-HAZE [18], exhibit HC values of 0.6 or less. Based on these results, the dense haze images from DenseHAZE and LIVE [19] were simulated to set the reference value for the actual dense haze image. LIVE is an actual hazy image dataset with images of various densities and sizes, from which 45 dense hazy images were visually selected.

The HC calculation of the dense haze datasets of Dense-HAZE and LIVE showed 10 out of 100 images having values of 0.8 or higher, and 91 having 0.6 or lower. Analysis of images with values of 0.8 showed that nine types of exceptional photos contained close-range objects with very little haze, generating significant deviations between images with dense haze and those with little haze.

Second, clustering was performed to determine the criteria for images with large deviations. Similar to the above statistical method, the HC values of all 465 actual haze images from LIVE were calculated, and the k-means clustering algorithm [20,21] was implemented to classify them into two groups. The medians of the two classified clusters were 0.86362 and 0.5660, respectively, and the two clusters were classified into images with high deviation and normal images. These measured results were similar to the reference values obtained in the first step.

Therefore, based on the results of the two steps, the criterion was set to 0.8 or more for images with high haze deviation and 0.5 or less for images with dense haze. In this case, the reference value 0.5 is a value which compensates for the criterion for dense haze images to prevent excessive visibility enhancement for some general images.

Case 1. HC ≥ 0

For a highly deviated fog image, distant regions close to the sky and regions close to the camera must be preserved to preserve the spatial information. This problem is solved by defining H(x), excluding the region to be preserved, as G(x) according to Eq. (11).

where θ₁ and θ₂ represent the near and far regions that need to be preserved.

Case 2. HC ≤ 0

In Case 2, the entire image, except for the airlight region is dehazed because a high reconstruction performance is required even for close-distance regions. G in Case 2 is calculated by specifying the airlight region to be preserved through min(H) as follows:

If extremely dense haze is observed in Case 2, G can cause noise in the image because a weight is assigned to the haze. Therefore, we compensate for this by multiplying Eq. (13) by HC.

Case 3. 0.5 < HC < 0

In normal images, because the damage in the close-distance region is the main factor, it is specified as the region to be preserved. G in Case 3 is set as follows:

where β is the scattering coefficient in Eq. (2), which is set using μ(H), the mean value of H. Eq. (15) can be applied to any hazy image, and in Case 3, β has a value of 1. Therefore, it is used as the correction constant for G. The scene depth D(x) reflecting the weight G(x) is given by:

where k is the sensitivity constant. Fig. 4 shows the calculated G map, the depth map D, and the dehazing results for different values of k.

The results in Fig. 5 show that Eq. (16) is useful only when the haze is extremely dense, as in Case 2; in fact, Case 2 showed excellent visibility for most values of k, whereas Cases 1 and 3 exhibited darkening of the preserved regions, resulting in image losses, as shown by the magnified images. The cause of this problem can be identified by comparing Cases 1 and 2 in Fig. 5 (a) and (e), respectively. For images that require strong dehazing, such as those in Case 2, emphasizing the G region using Eq. (16) increases the visibility. However, for Case 1, the additional emphasis on G causes a loss of image quality, because it has already been dehazed by d_c. Therefore, the depth maps in Cases 1 and 3 were estimated by calculating 1−G, which is the opposite of Case 2. Because 1−G represents the preserved region as opposed to G, we used it to calculate the depth maps for Cases 1 and 3 as follows:

Figure 5. Weight map G, depth map D, and output results based on different values of k: (a) G (k = 0.7), (b) D (k = 0.7), (c) Result. (k = 0.4), (d) Result (k = 0.7), (e) Result. (k = 1).

where k₁ and k₂ denote the sensitivities of the weights. We set k₁ as positive and k₂ as negative to ensure that the preserved region is not overly affected even if V(x) increases.

Fig. 6 shows the dehazing results for Case 1.

Figure 6. Dehazing results for Case 1 when k₁ = 0.5 and k₂ = -0.7: (a) 1 − G, (b) Depth map D, (c) Output result.

The results in Fig. 6 show that, in Case 1, the image of the close-distance foreground is preserved, and the visibility increases slightly in the far field.

D. Dehazing

We have estimated the airlight A, atmospheric scattering coefficient β, and depth d through the above-described process. To reconstruct J, we need to calculate t, which is calculated by substituting β and d into Eq. (2). Eq. (18) shows the atmospheric scattering model for J [13].

where $\max (0.1,\min (1,t(x)+0.1))$ allows t to have a value between 0.1 and 1; for max(A), we use the brightest channel value of A.

To verify the dehazing performance of the proposed method, we compared it with the GIF [6], WGIF [7], FVR [8], and CAP [9] methods. The test platform comprised a desktop computer with a 7.2 GHz CPU (AMD Ryzen 7) and 16 GB of memory, and the experiments were performed on MATLAB R2023a. In the experiments, the parameter of the proposed method was set as k = 0.7 for Case 2. For Cases 1 and 3, respectively, k₁ was set to 0.5 and 0.3, and k₂ to −0.7 and −0.4. For comparison, the parameters in Eq. (5) of the CAP [9] used a₀ = 0.12178, a₁ = 0.95971, and a₂ = 0.780245, which are the parameters generated by the author.

Both visual and objective evaluations were performed for the proposed method and conventional methods.

A. Visual Evaluation Using Real-world Images

Fig. 7 shows the visual performance of each method with the LIVE [19] dataset by dividing it into Cases 1, 2, and 3 according to the haze concentration in the image. Cases 1, 2, and 3 show images with haze only in the far-field, images with dense haze concentrations, and normal images, respectively. Normal images are classified into three categories, shown in sub-figures (c), (d), and (e) in Fig. 7, to evaluate the dehazing performance for cases where there is a sky region and bright objects.

Figure 7. Output images of various dehazing methods for real-world hazy images: (a) Output images of Case 1, (b) Output images of Case 2, (c) Well-known hazy sample images, (d) Images with sky region, (e) Images with bright objects.

The results in Fig. 7 show that GIF produced dark images for all images with halo effects in (b) and (e) and significant noise for the sky in (b) and (d). WGIF, an enhanced version of GIF, showed higher contrast than the other methods but showed a loss of close object information in (c) and (d) owing to an excessive increase in saturation. Furthermore, noise occurred in the sky regions, as shown in (b) and (d). Among the conventional methods, FVR showed the best results, as shown in Fig. (a), and the CAP exhibited the best performance, as shown in Fig. (b)-(d). However, FVR produced more unnatural results than the proposed method, owing to excessive edge emphasis and loss of perspective. Moreover, CAP showed haze in (a), (b), and (c) owing to weak dehazing and showed a strong halo effect, especially as seen in (e).

Furthermore, the proposed method produced the most natural results for all images. Color and spatial senses were restored successfully in (a), and visibility was improved in the images in (b) and (c) while preserving close-distance objects. Furthermore, in cases (b) and (d), the sky region was best preserved, and the entire image was reconstructed without noise. Finally, in (e), which has a bright object, the proposed method yielded the most natural results with the lightest halo effect.

B. Objective Evaluation Using Ground Truth

We used the O-HAZE [22] and I-HAZE [23] datasets as test images for quantitative evaluation. O-HAZE and I-HAZE provide real-world hazy and ground-truth images (GT), respectively, using professional hazing machines for the same indoor and outdoor scenes. Therefore, they can be used to calculate the quantitative evaluation metrics peak signal-to-noise ratio (PSNR) [24] and structural index similarity measure (SSIM) [25].

Fig. 8 shows the ground truth and hazy images of the datasets along with the corresponding images of the datasets. In Fig. 8, unlike normal hazy images, the hazy images have a bluish tint due to artificial fog spraying. Consequently, the road, windows, and walls in the images (a), (b), and (c) are recognized as bright objects, requiring high performance for airlight extraction.

Figure 8. Output images of various methods for the O-HAZE and I-HAZE datasets: (a) TEST 1, (b) TEST 2, (c) TEST 3.

The proposed method yielded the best results when the resulting images were compared, as shown in Fig. 8. GIF and WGIF produced very dark close-distance objects owing to excessive dehazing, and all conventional methods except FVR showed poor visibility, especially in (c). The FVR showed relatively good results; however, the edges were over-enhanced, resulting in a more severe haze in (a).

For the resulting images in Fig. 8, we calculated the PSNR and SSIM values for quantitative evaluation and compared them, as shown in Table 1.

Table 1 . PSNR and SSIM results of various dehazing methods.

Image	Method	PSNR[dB]	SSIM
Test 1	GIF	36.95	0.76
	WGIF	39.24	0.76
	FVR	27.40	0.67
	CAP	37.34	0.75
	Proposed	39.65	0.78
Test 2	GIF	28.98	0.50
	WGIF	27.28	0.50
	FVR	38.73	0.53
	CAP	36.12	0.63
	Proposed	43.45	0.68
Test 3	GIF	19.68	0.11
	WGIF	20.55	0.14
	FVR	36.11	0.27
	CAP	26.59	0.29
	Proposed	40.91	0.43

As shown in Table 1, the CAP with relatively weak dehazing showed good results in terms of SSIM, whereas the FVR showed good results in terms of PSNR, indicating high visibility. Moreover, for test 1, WGIF showed very similar PSNR values to the proposed method, but the proposed method performed better in SSIM and Fig. 8.

Based on natural dehazing, the proposed method achieved the best results in both visual and quantitative evaluations.

In this study, we proposed a visibility enhancement method that reflects the spatial sense of the natural reconstruction of hazy images. We defined the haze concentration using the inverse relationship between the distance between objects and the haze concentration in hazy images. We also set a weight map that reflects the perspective by dividing the images into images with a large sense of space, images with dense haze, and normal images according to the defined HC. The output weight map was applied to the depth map defined by the CAP to preserve objects close to the sky. Consequently, in the output image of the proposed dehazing method, the contrast of close objects with haze was emphasized and the sky region was softened, providing a natural and highly visible result.

As a result of evaluating the performance of the proposed method, both objective and visual evaluations showed that the proposed method produced the most natural results; in particular, it successfully preserved the sky region and close objects.