Regular paper

Split Viewer

Journal of information and communication convergence engineering 2023; 21(1): 90-97

Published online March 31, 2023

https://doi.org/10.56977/jicce.2023.21.1.90

© Korea Institute of Information and Communication Engineering

Phase Differences Averaging (PDA) Method for Reducing the Phase Error in Digital Holographic Microscopy (DHM)

Hyun-Woo Kim 1, Jaehoon Lee 1, Arun Anand 2, Myungjin Cho 3*, and Min-Chul Lee1* , Member, KIICE

1Department of Computer Science and Networks, Kyushu Institute of Technology, Iizuka, Fukuoka 820-8502, Japan
2Department of Physics, Sardar Patel University, Vallabh Vidyanagar, Anand 388120, India
3School of ICT, Robotics, and Mechanical Engineering, IITC, Hankyong National University, Kyonggi-do 17579, South Korea

Correspondence to : Myungjin Cho1 (E-mail: mjcho@hknu.ac.kr), Min-chul Lee2 (E-mail: lee@csn.kyutech.ac.jp, Tel: +81-948-29-7699)
1School of ICT, Robotics, and Mechanical Engineering, IITC, Hankyong National University, Kyonggi-do 17579, Republic of Korea
2Department of Computer Science and Networks, Kyushu Institute of Technology, Fukuoka 820-8502, Japan

Received: August 8, 2022; Revised: October 14, 2022; Accepted: October 14, 2022

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Digital holographic microscopy (DHM) is a three-dimensional (3D) imaging technique that uses the phase information of coherent light. In the reconstruction process of DHM, a narrow region around the positive or negative sideband from the Fourier domain is windowed to avoid noise due to the DC spectrum of the hologram spectrum. However, the limited size of the window also degrades the high-frequency information of the 3D object profile. Although a large window can have more detailed information of the 3D object shape, the noise is increased. To solve this trade-off, we propose phase difference averaging (PDA). The proposed method yields high-frequency information of the specimen while reducing the DC noise. In this paper, we explain the reconstruction algorithm for this method and compare it to various conventional filtering methods including Gaussian, Wiener, average, median, and bilateral filtering methods.

Keywords Digital holographic microscopy, Noise reduction, Phase error, Three-dimensional imaging

Light sensors, including the human eye, and digital arrays cannot detect phase information directly, but can only detect the intensity of light. For this reason, the interference characteristics of light have been used for obtaining phase information. In particular, holography is a technology that uses interference to detect the phase of light. It utilizes the phase difference between a reference beam and an object beam to record three-dimensional (3D) information for objects. This technique has been studied extensively since it was first proposed in 1948 [1]. Digital holography (DH) uses the same basic principles as holography, but uses image sensors instead of films for recording. Unlike films that cannot be reused after recording, DH can record images without storage limitations. Recently, DH has been applied to quantitative phase imaging (QPI) [2,3], microscopy [2-10], 3D image encryption [11], and 3D object recognition [12]. Among these applications, as digital holographic microscopy (DHM) can obtain the thickness profile of an object with high resolution, it is used for disease diagnosis [13,14], 3D profiling of microstructures [15-17], and microbial research [18, 19]. In DHM, one of the sidebands (corresponding to object information) in the Fourier domain is windowed to obtain the depth information. In this process, when a wide window is used, high frequency information of the object is obtained. However, the information in the DC spectrum, as well as the noise, also increases, leading to phase error. In contrast, when a narrow phase region is masked, the phase error decreases, but high frequency information about the object is lost. Furthermore, in interferometers, it is difficult to adjust the interference fringe density and set the size of the window used to mask the sideband. In this paper, we propose a phase error filtering method for DHM, which can address this ambiguity. We refer to this filtering method as phase differences averaging (PDA).

A. Phase Error in the Processing of Digital Holographic Microscopy

Fig. 1(a) and (b) show the 2D and 1D Fourier spectra of the recorded hologram, respectively. To obtain depth information from the recorded hologram, one of the sideband terms is windowed and cropped. As shown in Fig. 1(b), Rc is the size of the cropped region, and the green area under the +fo term is the noise induced by the DC spectrum. To reduce this unwanted phase in image processing, we have proposed PDA. Our proposed method crops the 2D Fourier spectrum from narrow to wide regions in regular intervals to reconstruct the correct phase information by reducing the effect of the DC spectrum and high-frequency noise.

Fig. 1. Fourier spectrum of the recorded hologram. (a) 2D and (b) 1D. fx and fy are spatial frequencies along x-axis and y-axis respectively, A is amplitude, and Rc is the size of the window used for cropping.

Fig. 2 shows a schematic of the PDA. Fig. 2(a) shows the Fourier spectrum of the recorded hologram, and Fig. 2(b) shows each DC shifted +fo term after cropping the sideband around +fo with different-sized windows. Zero padding was applied, as shown in Fig. 2(b), to obtain images of the same resolution via inverse Fourier transform on the sidebands windowed using different sizes. Fig. 2(c) shows the phase calculated using the cropped Fourier spectrum. The average of these obtained phase profiles was then computed to yield the high-frequency thickness information of the specimen while reducing the effect of the DC spectrum. The PDA process is very simple as it selects the center of the sideband in the Fourier domain and crops it from the smallest to the largest window size in regular intervals, regardless of the trade-off between the phase error and high-frequency information of the specimen.

Fig. 2. Schematic diagram of the PDA. (a) Fourier spectrum of the recorded hologram, (b) windowed and cropped region around +fo using windows of different sizes, (c) obtained phase differences using each window.

B. Mathematical Formula of the Phase Differences Averaging

A detailed explanation of PDA implementation is as follows. Two holograms are recorded, one with the object in the field of view (object hologram) and one with the medium surrounding the object in the field of view (reference hologram), to obtain the change in phase of the wavefront caused by the spatially-varying optical path length of the object. In addition, this method cancels out the noise caused by the surface of the mirror, beam splitter, object lens, and image sensor. They are analyzed separately by applying different sized windows (from smallest to largest), and their phase difference is computed. The computed phase difference corresponding to any one of the windows can be written as [20]

Δ ϕ x , y = 2 π λ n s n m h x , y , h x , y = λ Δ ϕ x , y 2 π Δ n where ,   Δ n = n s n m ,

where ns and ns are the constant refractive indices of the specimen and surrounding medium, respectively; h is the thickness information of the specimen; λ is the vacuum wavelength of the source; and Δn is the difference in the refractive index. Here, h is one thickness value obtained with a particular masking window (Eq. (2)). Thus, the thickness information with the smallest masking window size is h1, and numbers are assigned in order of increasing window size. The average of all the thickness information with different window sizes is [20]

h P D A x , y , t = 1 N d i = 1 N d λ Δ ϕ i x , y , t 2 π Δ n

In this study, we wished to crop the phase region as wide as possible, so the fringe pattern was as narrow as possible. For this reason, we used modified Mach-Zehnder interferometry using two spherical waves.

Fig. 3 shows the experimental setup used for the demonstration of PDA. The setup used a 532 nm laser diode module (3 mW output power). To avoid overlapping of red blood cells (RBCs), we prepared thin blood smears as specimens by dripping blood on a glass slide and spreading it thinly. Both the object and reference beams were magnified by 40 × (0.65 NA) objective lenses. In this experiment, we used constant reflective indices for the RBCs and the surrounding medium of 1.42 and 1.34 (reflective index of the blood plasma), respectively [13]. Holograms were recorded using a CMOS sensor (Basler, acA2500-14uc) with 2590(H) × 1942(V) pixel resolution, and a pixel size of 2.2 μm (H) × 2.2 μm (V). The CMOS sensor was placed at the image plane. The beam splitter in front of the sensor was tilted to make the fringe pattern as dense as allowed by the sampling criteria.

Fig. 3. Experimental setup. L: lens, P: pinhole, M: mirror, BS: beam splitter, OL: objective lens

A. Comparison of the Phase Differences Averaging with the Unfiltered 3D Profile of the Object

Fig. 4 shows the unfiltered 3D profile of the object (red blood cell) obtained using the smallest phase region (90(H) × 90(V) pixels) and the largest phase region (660(H) × 660(V) pixels). In this study, we used a Goldstein phase-unwrapping algorithm [21] to obtain continuous phase information from the wrapped phase. As mentioned earlier, when the phase region is small, as shown in Fig. 4(a), the approximate shape information of the object and the noise decrease; when the region is wide, as shown in Fig. 4(b), the shape information of the object includes details and the noise increases. Therefore, we applied filtering using the proposed method. In this study, the window size of the data used for reconstruction of the proposed method was increased at intervals of 30 pixels horizontally and vertically from 90(H) × 90(V) pixels to 660(H) × 660(V) pixels. As a result, we obtained a total of 20 thickness datapoints with the same image resolution.

Fig. 4. 3D profile of the object was obtained by (a) the narrowest phase region and (b) the widest phase region, and the results of applying (c) the PDA, (d)-(h) the results of the 5 different conventional denoising algorithms.

As shown in Fig. 4(c), the noise in the 3D profile is reduced. In addition, we compared the proposed method with different conventional filtering methods. Figs. 4(d)-(h) show the results obtained using the conventional filters: a Gaussian filter (σ = 2), a Wiener filter (9(H) × 9(V) pixel filter size), an average filter and a median filter (5(H) × 5(V) pixels filter size), and a bilateral filter (σ = 2), respectively, applied to Fig. 4(b). The size of each conventional filter was chosen to retain as much high-frequency information as possible. As a result of comparing each 3D profile, no significant difference was found between the result of a Gaussian filter and the PDA. For numerical comparison, we calculated the signal-to-noise ratio (SNR) and mean squared error (MSE) of each 3D profile [22]. To compare only the noise coming from the DC spectrum, the comparison target of SNR and MSE uses the 3D profile of the smallest cropped phase region.

Table 1 shows the results of the numerical comparison between the PDA and various conventional filters. In Table 1, the MSE and SNR of the Gaussian filter show the best results among the conventional filtering methods. However, the Gaussian filtering results also show a higher MSE and lower SNR than PDA. Among the conventional filters compared in this section, the Gaussian filter, which shows the most effective value, is compared in more detail in the next subsection.

Units for magnetic properties

MSE SNR (dB)
Unfiltered 0.4827 19.5095
PDA 0.1882 23.5991
Gaussian 0.2340 22.6533
Wiener 0.2838 21.8156
Average 0.2781 21.9044
Median 0.3162 21.3470
Bilateral 0.4795 19.5388


B. Comparison with Ordinary Filtering Methods Using Statistical Analysis

We used statistical analysis to determine the advantages of the proposed method. For statistical analysis, we randomly selected 20 different RBCs and obtained thickness profile data using small to large masking windows. The results obtained with PDA were compared with those obtained with unfiltered, Gaussian, and median filtering (Fig. 5).

Fig. 5. Comparison of the PDA with (a) the unfiltered data, (b) median filtering, (c) Gaussian filtering

The horizontal line in Fig. 5 indicates the result obtained using PDA. For the unfiltered method, it can be seen that the larger the size of the masking window, the lower the SNR, as shown in Fig. 5(a). However, the variation increases with a larger masking window. For single-window filtering, the SNR is highest for small window sizes, but it also removes high-frequency object information. It can be seen that the SNR of PDA is the same as that of a single filter window of 270(H) × 270(V) pixels. Above this window size, the single-filteringwindow SNR decreases, while the PDA SNR remains high. As the fringe density changes, the filtering window size should also change. This is automatically done when using PDA. In addition, from Fig. 5(b) and 5(c), it can be seen that a larger sigma value of the Gaussian filter and larger filter size of the median filter results in better SNR values. However, these filtering operations smooth the obtained depth profile from phase differences, resulting in the loss of high-frequency information. PDA also considers this aspect. The variances of the Gaussian and median filtering are much larger than the variance obtained using PDA. The variance value is related to the stability of filtering, which means that there is a possibility that random noise may remain. To verify this, the correlation between SNR and MSE was used in this study. Fig. 6 shows the correlation between SNR and MSE.

Fig. 6. Correlation between SNR and MSE.

Fig. 6 shows the correlation between SNR and MSE for different RBCs generated using the same method. The vertical axis represents the SNR value and the horizontal axis represents the MSE value. As shown in Fig. 6, It can be seen that PDA provides a higher SNR with lower mean square error, while retaining high spatial frequency information of the object.

As shown in Fig. 7, when the sigma value of the Gaussian filter increases, a lower error and better SNR result. However, Gaussian filtering also suppresses higher spatial frequency information of the object. Among the randomly selected RBCs, 10% of RBCs have a serious phase error, and even if the sigma value of the filter increases, these errors lead to a lower SNR (Fig. 7).

Fig. 7. Comparison between the PDA and the Gaussian filtering using a correlation between SNR and MSE.

Fig. 8 illustrates the advantages of our proposed method. PDA can provide high-resolution, high-SNR thickness profiles compared to a single masking window. This is the strength of the proposed method. The RBCs in Fig. 8 have a flat surface, unlike the donut-shaped RBCs in Fig. 4. This is because the pressing force was strong in the process of making the specimen. However, this did not affect the experimental results.

Fig. 8. Example of PDA advantages over ordinary filtering method.

In this study, we proposed a new filtering method for reducing the phase error from the DC spectrum in DHM and compared our proposed method with general filtering methods. The proposed method showed strength in terms of stability compared to other filtering methods, and it had the advantage of obtaining 3D data with a high spatial frequency and SNR. Filtering methods using a single masking window either provided a low spatial frequency (small window) or high noise (large window) 3D profile of the object. However, our proposed method did not require the size of the filter to be determined, and it had the advantage of being able to filter effectively, even in severe phase error conditions. The proposed method retained the detailed (high-frequency) information of the object while removing unwanted noise. Furthermore, because the proposed method averaged the thickness information after phase unwrapping of windowed sidebands of 20 different sizes, it also eliminated the phase error that occurs after phase unwrapping. Off-axis DHM combined the object and reference beams at an angle, leading to the separation of the DC spectrum and sidebands. However, in many cases, especially while using low temporally coherent sources such as LEDs, to have a larger usable field of view (where interference fringes exist), the angle between the object and reference beams must be very small, leading to low-density fringes. In such cases, the sideband information overlapped with the DC spectrum, resulting in low lateral resolution owing to the smaller size of the filtering window used. The proposed method is very useful under such conditions. In this study, the width interval of the windowed sideband is an arbitrary value, and the height data also used an arbitrary number of data points for the average. As the width interval became narrower and the number of height data points used for the average increased, the filtering effect was expected to be excellent. However, because there is a trade-off relationship between processing time and filtering quality, it is important for the user to select an appropriate value.

This work was supported by the Japan-Korea Basic Scientific Cooperation Program between JSPS and NRF, Grant number (JPJSBP 120228811), and this work was supported under the framework of the international cooperation program managed by the National Research Foundation of Korea(NRF-2022K2A9A2A08000152, FY2022).

  1. D. Gabor, A new microscopic principle, Nature, vol. 161, pp. 777-778, May., 1948. DOI: 10.1038/161777a0.
    Pubmed CrossRef
  2. T. Pitkäaho and A. Manninen and T. J. Naughton, Focus prediction in digital holographic microscopy using deep convolutional neural networks, Applied Optics, vol. 58, no. 5, pp. A202-A208, Jan., 2019. DOI: 10.1364/AO.58.00A202.
    Pubmed CrossRef
  3. J. A. Picazo-Bueno and M. Trusiak and V. Micó, Single-shot slightly off-axis digital holographic microscopy with add-on module based on beamsplitter cube, Optics Express, vol. 27, no. 4, pp. 5655-5669, Feb., 2019. DOI: 10.1364/OE.27.005655.
    Pubmed CrossRef
  4. S. Shin, and Y. Yu, Lensless reflection digital holographic microscope with a fresnel-bluestein transform, Journal of the Korean Physical Society, vol. 74, no. 2, pp. 98-101, Jan., 2019. DOI: 10.3938/jkps.74.98.
    CrossRef
  5. T. O'Connor and A. Doblas and B. Javidi, Structured illumination in compact and field-portable 3D-printed shearing digital holographic microscopy for resolution enhancement, Optics Letters, vol. 44, no. 9, pp. 2326-2329, May., 2019. DOI: 10.1364/OL.44.002326.
    Pubmed CrossRef
  6. Z. Zhong, and H. Zhao, and L. Cao, and M. Shan, and B. Liu, and W. Lu, and H. Xie, Automatic cross filtering for off-axis digital holographic microscopy, Results in Physics, vol. 16, 102910, Mar., 2020. DOI: 10.1016/j.rinp.2019.102910.
    CrossRef
  7. T. O'Connor and A. Anand and B. Javidi, Field-portable microsphereassisted high resolution digital holographic microscopy in compact and 3D-printed Mach-Zehnder Interferometer, OSA Continuum, vol. 3, no. 4, pp. 1013-1020, Apr., 2020. DOI: 10.1364/OSAC.389832.
    CrossRef
  8. J. Dong, and A. K. Yetisen, and X. Dong, and F. Pöller, and M. Jakobi, and Z. Liu, and F. S. Bloise, and A. W. Koch, Low-pass filtering compensation in common-path digital holographic microscopy, Applied Physics Letters, vol. 117, no. 12, 121105, Sep., 2020. DOI: 10.1063/5.0019209.
    CrossRef
  9. L. Huang, and L. Yan, and B. Chen, and Y. Zhou, and T. Yang, Phase aberration compensation of digital holographic microscopy with curve fitting preprocessing and automatic background segmentation for microstructure testing, Optics Communications, vol. 462, 125311, May., 2020. DOI: 10.1016/j.optcom.2020.125311.
    CrossRef
  10. H. W. Kim, and K. Inoue, and M. Cho, and M. C. Lee, A study on real-time modification of the refractive index of a surrounding medium using a uniform microsphere in digital holographic microscopy, in in Proceedings of the 2020 3rd International Conference on Electronics and Electrical Engineering Technology, Kitakyushu, Japan, pp. 44-48, 2020. DOI: 10.1145/3429536.3429539.
    CrossRef
  11. M. L. Piao and H. Y. Wu and N. Kim, 3D image encryption based on computer-generated hologram, in in Digital Holography and Three-Dimensional Imaging. Optical Society of America, Bordeaux, France, pp. W3A-21, 2019. DOI: 10.1364/DH.2019.W3A.21.
    CrossRef
  12. I. Shevkunov, and V. Katkovnik, and D. Claus, and G. Pedrini, and N. V. Petrov, and K. Egiazarian, Spectral object recognition in hyperspectral holography with complex-domain denoising, Sensors, vol. 19, no. 23, p. 5188, Nov., 2019. DOI: 10.3390/s19235188.
    Pubmed KoreaMed CrossRef
  13. A. Anand, and V. K. Chhaniwal, and N. R. Patel, and B. Javidi, Automatic identification of malaria-infected RBC with digital holographic microscopy using correlation algorithms, IEEE Photonics Journal, vol. 4, no. 5, pp. 1456-1464, Oct., 2012. DOI: 10.1109/JPHOT.2012.2210199.
    CrossRef
  14. F. Yi and I. Moon and Y. H. Lee, Three-dimensional counting of morphologically normal human red blood cells via digital holographic microscopy, Journal of Biomedical Optics, vol. 20, no. 1, 016005, Jan., 2015. DOI: 10.1117/1.JBO.20.1.016005.
    Pubmed CrossRef
  15. B. Kemper, and G. Von Bally, Digital holographic microscopy for live cell applications and technical inspection, Applied Optics, vol. 47, no. 4, pp. A52-A61, Feb., 2008. DOI: 10.1364/AO.47.000A52.
    Pubmed CrossRef
  16. T. Colomb, and N. Pavillon, and J. Kühn, and E. Cuche, and C. Depeursinge, and Y. Emery, Extended depth-of-focus by digital holographic microscopy, Optics Letters, vol. 35, no. 11, pp. 1840-1842, Jun., 2010. DOI: 10.1364/OL.35.001840.
    Pubmed CrossRef
  17. S. Shin, and Y. Yu, Fine metal mask 3-dimensional measurement by using scanning digital holographic microscope, Journal of the Korean Physical Society, vol. 72, no. 8, pp. 863-867, Apr., 2018. DOI: 10.3938/jkps.72.863.
    CrossRef
  18. J. Zakrisson and S. Schedin and M. Andersson, Cell shape identification using digital holographic microscopy, Applied Optics, vol. 54, no. 24, pp. 7442-7448, Aug., 2015. DOI: 10.1364/AO.54.007442.
    Pubmed CrossRef
  19. D. Roitshtain, and N. A. Turko, and B. Javidi, and N. T. Shaked, Flipping interferometry and its application for quantitative phase microscopy in a micro-channel, Optics Letters, vol. 41, no. 10, pp. 2354-2357, May., 2016. DOI: 10.1364/OL.41.002354.
    Pubmed CrossRef
  20. A. Anand and V. Chhaniwal and B. Javidi, Tutorial: common path selfreferencing digital holographic microscopy, APL Photonics, vol. 3, no. 7, 071101, Jul., 2018. DOI: 10.1063/1.5027081.
    CrossRef
  21. R. M. Goldstein and H. A. Zebker and C. L. Werner, Satellite radar interferometry: Two-dimensional phase unwrapping, Radio Science, vol. 23, no. 4, pp. 713-720, Jul., 1988. DOI: 10.1029/RS023i004p00713.
    CrossRef
  22. R. C. Gonzalez, and R. E. Woods Digital image processing, 4th ed., New York, NY: Pearson, 2002.

Hyun-Woo Kim

received his B.S. from Jeju National University, Jeju, Korea, in 2011 and M.S from Korea University, Seoul, Korea in 2015. He is currently a doctoral student at Kyushu Institute of Technology in Japan. He worked as a student researcher and a researcher at Korea Institute of Science and Technology, Korea, from 2011 to 2017. His research interests include autostereoscopic 3D display, integral imaging, 3D computational reconstruction, digital holography, and digital holographic microscopy.


Jaehoon Lee

received his B.S. degree from Hankyong National University, Anseong, South Korea, in 2018, and his M.S. degree from Kyushu Institute of Technology, Fukuoka, Japan, in 2020, where he is currently pursuing a Ph.D. His research interests include integral imaging, three-dimensional (3D) computational reconstruction, night vision, photon counting, digital holographic microscopy, information security and 3D visualization.


Arun Anand

received his Ph.D. in applied physics from the M. S. University of Baroda, India, in 2003. From 2002 to 2008, he worked as a Scientist at the Institute for Plasma Research, Gandhinagar, India. Presently he is a Professor in the Department of Physics, Sardar Patel University, India. He has more than 100 publications including peer-reviewed journal articles, conference proceedings and invited conference papers. His research interests include 3-D microscopes for cell imaging and identification, biomedical optical systems, digital holography and optical instrumentation.


Myungjin Cho

received his B.S. and M.S. in Telecommunication Engineering from Pukyong National University, Pusan, Korea, in 2003 and 2005, respectively, and M.S. and Ph.D. in electrical and computer engineering from the University of Connecticut, Storrs, CT, USA, in 2010 and 2011, respectively. Currently, he is a full professor at Hankyong National University, Korea. He worked as a researcher at Samsung Electronics in Korea from 2005 to 2007. His research interests include 3D display, 3D signal processing, 3D biomedical imaging, 3D photon counting imaging, 3D information security, 3D object tracking, 3D underwater imaging, digital holographic microscopy, photon counting, and 3D visualization of objects under inclement weather conditions.


Min-Chul Lee

received his B.S. in Telecommunication Engineering from Pukyong National University, Busan, Korea, in 1996, and his M.S. and Ph.D. from Kyushu Institute of Technology, Fukuoka, Japan, in 2000 and 2003, respectively. He is an associate professor at Kyushu Institute of Technology, Japan. His research interests include medical imaging, blood flow analysis, 3D display, 3D integral imaging, digital holographic microscopy, and 3D biomedical imaging.


Article

Regular paper

Journal of information and communication convergence engineering 2023; 21(1): 90-97

Published online March 31, 2023 https://doi.org/10.56977/jicce.2023.21.1.90

Copyright © Korea Institute of Information and Communication Engineering.

Phase Differences Averaging (PDA) Method for Reducing the Phase Error in Digital Holographic Microscopy (DHM)

Hyun-Woo Kim 1, Jaehoon Lee 1, Arun Anand 2, Myungjin Cho 3*, and Min-Chul Lee1* , Member, KIICE

1Department of Computer Science and Networks, Kyushu Institute of Technology, Iizuka, Fukuoka 820-8502, Japan
2Department of Physics, Sardar Patel University, Vallabh Vidyanagar, Anand 388120, India
3School of ICT, Robotics, and Mechanical Engineering, IITC, Hankyong National University, Kyonggi-do 17579, South Korea

Correspondence to:Myungjin Cho1 (E-mail: mjcho@hknu.ac.kr), Min-chul Lee2 (E-mail: lee@csn.kyutech.ac.jp, Tel: +81-948-29-7699)
1School of ICT, Robotics, and Mechanical Engineering, IITC, Hankyong National University, Kyonggi-do 17579, Republic of Korea
2Department of Computer Science and Networks, Kyushu Institute of Technology, Fukuoka 820-8502, Japan

Received: August 8, 2022; Revised: October 14, 2022; Accepted: October 14, 2022

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Digital holographic microscopy (DHM) is a three-dimensional (3D) imaging technique that uses the phase information of coherent light. In the reconstruction process of DHM, a narrow region around the positive or negative sideband from the Fourier domain is windowed to avoid noise due to the DC spectrum of the hologram spectrum. However, the limited size of the window also degrades the high-frequency information of the 3D object profile. Although a large window can have more detailed information of the 3D object shape, the noise is increased. To solve this trade-off, we propose phase difference averaging (PDA). The proposed method yields high-frequency information of the specimen while reducing the DC noise. In this paper, we explain the reconstruction algorithm for this method and compare it to various conventional filtering methods including Gaussian, Wiener, average, median, and bilateral filtering methods.

Keywords: Digital holographic microscopy, Noise reduction, Phase error, Three-dimensional imaging

I. INTRODUCTION

Light sensors, including the human eye, and digital arrays cannot detect phase information directly, but can only detect the intensity of light. For this reason, the interference characteristics of light have been used for obtaining phase information. In particular, holography is a technology that uses interference to detect the phase of light. It utilizes the phase difference between a reference beam and an object beam to record three-dimensional (3D) information for objects. This technique has been studied extensively since it was first proposed in 1948 [1]. Digital holography (DH) uses the same basic principles as holography, but uses image sensors instead of films for recording. Unlike films that cannot be reused after recording, DH can record images without storage limitations. Recently, DH has been applied to quantitative phase imaging (QPI) [2,3], microscopy [2-10], 3D image encryption [11], and 3D object recognition [12]. Among these applications, as digital holographic microscopy (DHM) can obtain the thickness profile of an object with high resolution, it is used for disease diagnosis [13,14], 3D profiling of microstructures [15-17], and microbial research [18, 19]. In DHM, one of the sidebands (corresponding to object information) in the Fourier domain is windowed to obtain the depth information. In this process, when a wide window is used, high frequency information of the object is obtained. However, the information in the DC spectrum, as well as the noise, also increases, leading to phase error. In contrast, when a narrow phase region is masked, the phase error decreases, but high frequency information about the object is lost. Furthermore, in interferometers, it is difficult to adjust the interference fringe density and set the size of the window used to mask the sideband. In this paper, we propose a phase error filtering method for DHM, which can address this ambiguity. We refer to this filtering method as phase differences averaging (PDA).

II. THEORY

A. Phase Error in the Processing of Digital Holographic Microscopy

Fig. 1(a) and (b) show the 2D and 1D Fourier spectra of the recorded hologram, respectively. To obtain depth information from the recorded hologram, one of the sideband terms is windowed and cropped. As shown in Fig. 1(b), Rc is the size of the cropped region, and the green area under the +fo term is the noise induced by the DC spectrum. To reduce this unwanted phase in image processing, we have proposed PDA. Our proposed method crops the 2D Fourier spectrum from narrow to wide regions in regular intervals to reconstruct the correct phase information by reducing the effect of the DC spectrum and high-frequency noise.

Figure 1. Fourier spectrum of the recorded hologram. (a) 2D and (b) 1D. fx and fy are spatial frequencies along x-axis and y-axis respectively, A is amplitude, and Rc is the size of the window used for cropping.

Fig. 2 shows a schematic of the PDA. Fig. 2(a) shows the Fourier spectrum of the recorded hologram, and Fig. 2(b) shows each DC shifted +fo term after cropping the sideband around +fo with different-sized windows. Zero padding was applied, as shown in Fig. 2(b), to obtain images of the same resolution via inverse Fourier transform on the sidebands windowed using different sizes. Fig. 2(c) shows the phase calculated using the cropped Fourier spectrum. The average of these obtained phase profiles was then computed to yield the high-frequency thickness information of the specimen while reducing the effect of the DC spectrum. The PDA process is very simple as it selects the center of the sideband in the Fourier domain and crops it from the smallest to the largest window size in regular intervals, regardless of the trade-off between the phase error and high-frequency information of the specimen.

Figure 2. Schematic diagram of the PDA. (a) Fourier spectrum of the recorded hologram, (b) windowed and cropped region around +fo using windows of different sizes, (c) obtained phase differences using each window.

B. Mathematical Formula of the Phase Differences Averaging

A detailed explanation of PDA implementation is as follows. Two holograms are recorded, one with the object in the field of view (object hologram) and one with the medium surrounding the object in the field of view (reference hologram), to obtain the change in phase of the wavefront caused by the spatially-varying optical path length of the object. In addition, this method cancels out the noise caused by the surface of the mirror, beam splitter, object lens, and image sensor. They are analyzed separately by applying different sized windows (from smallest to largest), and their phase difference is computed. The computed phase difference corresponding to any one of the windows can be written as [20]

Δ ϕ x , y = 2 π λ n s n m h x , y , h x , y = λ Δ ϕ x , y 2 π Δ n where ,   Δ n = n s n m ,

where ns and ns are the constant refractive indices of the specimen and surrounding medium, respectively; h is the thickness information of the specimen; λ is the vacuum wavelength of the source; and Δn is the difference in the refractive index. Here, h is one thickness value obtained with a particular masking window (Eq. (2)). Thus, the thickness information with the smallest masking window size is h1, and numbers are assigned in order of increasing window size. The average of all the thickness information with different window sizes is [20]

h P D A x , y , t = 1 N d i = 1 N d λ Δ ϕ i x , y , t 2 π Δ n

III. EXPERIMENTAL SETUP

In this study, we wished to crop the phase region as wide as possible, so the fringe pattern was as narrow as possible. For this reason, we used modified Mach-Zehnder interferometry using two spherical waves.

Fig. 3 shows the experimental setup used for the demonstration of PDA. The setup used a 532 nm laser diode module (3 mW output power). To avoid overlapping of red blood cells (RBCs), we prepared thin blood smears as specimens by dripping blood on a glass slide and spreading it thinly. Both the object and reference beams were magnified by 40 × (0.65 NA) objective lenses. In this experiment, we used constant reflective indices for the RBCs and the surrounding medium of 1.42 and 1.34 (reflective index of the blood plasma), respectively [13]. Holograms were recorded using a CMOS sensor (Basler, acA2500-14uc) with 2590(H) × 1942(V) pixel resolution, and a pixel size of 2.2 μm (H) × 2.2 μm (V). The CMOS sensor was placed at the image plane. The beam splitter in front of the sensor was tilted to make the fringe pattern as dense as allowed by the sampling criteria.

Figure 3. Experimental setup. L: lens, P: pinhole, M: mirror, BS: beam splitter, OL: objective lens

IV. EXPERIMENTAL RESULT

A. Comparison of the Phase Differences Averaging with the Unfiltered 3D Profile of the Object

Fig. 4 shows the unfiltered 3D profile of the object (red blood cell) obtained using the smallest phase region (90(H) × 90(V) pixels) and the largest phase region (660(H) × 660(V) pixels). In this study, we used a Goldstein phase-unwrapping algorithm [21] to obtain continuous phase information from the wrapped phase. As mentioned earlier, when the phase region is small, as shown in Fig. 4(a), the approximate shape information of the object and the noise decrease; when the region is wide, as shown in Fig. 4(b), the shape information of the object includes details and the noise increases. Therefore, we applied filtering using the proposed method. In this study, the window size of the data used for reconstruction of the proposed method was increased at intervals of 30 pixels horizontally and vertically from 90(H) × 90(V) pixels to 660(H) × 660(V) pixels. As a result, we obtained a total of 20 thickness datapoints with the same image resolution.

Figure 4. 3D profile of the object was obtained by (a) the narrowest phase region and (b) the widest phase region, and the results of applying (c) the PDA, (d)-(h) the results of the 5 different conventional denoising algorithms.

As shown in Fig. 4(c), the noise in the 3D profile is reduced. In addition, we compared the proposed method with different conventional filtering methods. Figs. 4(d)-(h) show the results obtained using the conventional filters: a Gaussian filter (σ = 2), a Wiener filter (9(H) × 9(V) pixel filter size), an average filter and a median filter (5(H) × 5(V) pixels filter size), and a bilateral filter (σ = 2), respectively, applied to Fig. 4(b). The size of each conventional filter was chosen to retain as much high-frequency information as possible. As a result of comparing each 3D profile, no significant difference was found between the result of a Gaussian filter and the PDA. For numerical comparison, we calculated the signal-to-noise ratio (SNR) and mean squared error (MSE) of each 3D profile [22]. To compare only the noise coming from the DC spectrum, the comparison target of SNR and MSE uses the 3D profile of the smallest cropped phase region.

Table 1 shows the results of the numerical comparison between the PDA and various conventional filters. In Table 1, the MSE and SNR of the Gaussian filter show the best results among the conventional filtering methods. However, the Gaussian filtering results also show a higher MSE and lower SNR than PDA. Among the conventional filters compared in this section, the Gaussian filter, which shows the most effective value, is compared in more detail in the next subsection.

Units for magnetic properties.

MSE SNR (dB)
Unfiltered 0.4827 19.5095
PDA 0.1882 23.5991
Gaussian 0.2340 22.6533
Wiener 0.2838 21.8156
Average 0.2781 21.9044
Median 0.3162 21.3470
Bilateral 0.4795 19.5388


B. Comparison with Ordinary Filtering Methods Using Statistical Analysis

We used statistical analysis to determine the advantages of the proposed method. For statistical analysis, we randomly selected 20 different RBCs and obtained thickness profile data using small to large masking windows. The results obtained with PDA were compared with those obtained with unfiltered, Gaussian, and median filtering (Fig. 5).

Figure 5. Comparison of the PDA with (a) the unfiltered data, (b) median filtering, (c) Gaussian filtering

The horizontal line in Fig. 5 indicates the result obtained using PDA. For the unfiltered method, it can be seen that the larger the size of the masking window, the lower the SNR, as shown in Fig. 5(a). However, the variation increases with a larger masking window. For single-window filtering, the SNR is highest for small window sizes, but it also removes high-frequency object information. It can be seen that the SNR of PDA is the same as that of a single filter window of 270(H) × 270(V) pixels. Above this window size, the single-filteringwindow SNR decreases, while the PDA SNR remains high. As the fringe density changes, the filtering window size should also change. This is automatically done when using PDA. In addition, from Fig. 5(b) and 5(c), it can be seen that a larger sigma value of the Gaussian filter and larger filter size of the median filter results in better SNR values. However, these filtering operations smooth the obtained depth profile from phase differences, resulting in the loss of high-frequency information. PDA also considers this aspect. The variances of the Gaussian and median filtering are much larger than the variance obtained using PDA. The variance value is related to the stability of filtering, which means that there is a possibility that random noise may remain. To verify this, the correlation between SNR and MSE was used in this study. Fig. 6 shows the correlation between SNR and MSE.

Figure 6. Correlation between SNR and MSE.

Fig. 6 shows the correlation between SNR and MSE for different RBCs generated using the same method. The vertical axis represents the SNR value and the horizontal axis represents the MSE value. As shown in Fig. 6, It can be seen that PDA provides a higher SNR with lower mean square error, while retaining high spatial frequency information of the object.

As shown in Fig. 7, when the sigma value of the Gaussian filter increases, a lower error and better SNR result. However, Gaussian filtering also suppresses higher spatial frequency information of the object. Among the randomly selected RBCs, 10% of RBCs have a serious phase error, and even if the sigma value of the filter increases, these errors lead to a lower SNR (Fig. 7).

Figure 7. Comparison between the PDA and the Gaussian filtering using a correlation between SNR and MSE.

Fig. 8 illustrates the advantages of our proposed method. PDA can provide high-resolution, high-SNR thickness profiles compared to a single masking window. This is the strength of the proposed method. The RBCs in Fig. 8 have a flat surface, unlike the donut-shaped RBCs in Fig. 4. This is because the pressing force was strong in the process of making the specimen. However, this did not affect the experimental results.

Figure 8. Example of PDA advantages over ordinary filtering method.

V. Conclusion

In this study, we proposed a new filtering method for reducing the phase error from the DC spectrum in DHM and compared our proposed method with general filtering methods. The proposed method showed strength in terms of stability compared to other filtering methods, and it had the advantage of obtaining 3D data with a high spatial frequency and SNR. Filtering methods using a single masking window either provided a low spatial frequency (small window) or high noise (large window) 3D profile of the object. However, our proposed method did not require the size of the filter to be determined, and it had the advantage of being able to filter effectively, even in severe phase error conditions. The proposed method retained the detailed (high-frequency) information of the object while removing unwanted noise. Furthermore, because the proposed method averaged the thickness information after phase unwrapping of windowed sidebands of 20 different sizes, it also eliminated the phase error that occurs after phase unwrapping. Off-axis DHM combined the object and reference beams at an angle, leading to the separation of the DC spectrum and sidebands. However, in many cases, especially while using low temporally coherent sources such as LEDs, to have a larger usable field of view (where interference fringes exist), the angle between the object and reference beams must be very small, leading to low-density fringes. In such cases, the sideband information overlapped with the DC spectrum, resulting in low lateral resolution owing to the smaller size of the filtering window used. The proposed method is very useful under such conditions. In this study, the width interval of the windowed sideband is an arbitrary value, and the height data also used an arbitrary number of data points for the average. As the width interval became narrower and the number of height data points used for the average increased, the filtering effect was expected to be excellent. However, because there is a trade-off relationship between processing time and filtering quality, it is important for the user to select an appropriate value.

ACKNOWLEDGMENTS

This work was supported by the Japan-Korea Basic Scientific Cooperation Program between JSPS and NRF, Grant number (JPJSBP 120228811), and this work was supported under the framework of the international cooperation program managed by the National Research Foundation of Korea(NRF-2022K2A9A2A08000152, FY2022).

Fig 1.

Figure 1.Fourier spectrum of the recorded hologram. (a) 2D and (b) 1D. fx and fy are spatial frequencies along x-axis and y-axis respectively, A is amplitude, and Rc is the size of the window used for cropping.
Journal of Information and Communication Convergence Engineering 2023; 21: 90-97https://doi.org/10.56977/jicce.2023.21.1.90

Fig 2.

Figure 2.Schematic diagram of the PDA. (a) Fourier spectrum of the recorded hologram, (b) windowed and cropped region around +fo using windows of different sizes, (c) obtained phase differences using each window.
Journal of Information and Communication Convergence Engineering 2023; 21: 90-97https://doi.org/10.56977/jicce.2023.21.1.90

Fig 3.

Figure 3.Experimental setup. L: lens, P: pinhole, M: mirror, BS: beam splitter, OL: objective lens
Journal of Information and Communication Convergence Engineering 2023; 21: 90-97https://doi.org/10.56977/jicce.2023.21.1.90

Fig 4.

Figure 4.3D profile of the object was obtained by (a) the narrowest phase region and (b) the widest phase region, and the results of applying (c) the PDA, (d)-(h) the results of the 5 different conventional denoising algorithms.
Journal of Information and Communication Convergence Engineering 2023; 21: 90-97https://doi.org/10.56977/jicce.2023.21.1.90

Fig 5.

Figure 5.Comparison of the PDA with (a) the unfiltered data, (b) median filtering, (c) Gaussian filtering
Journal of Information and Communication Convergence Engineering 2023; 21: 90-97https://doi.org/10.56977/jicce.2023.21.1.90

Fig 6.

Figure 6.Correlation between SNR and MSE.
Journal of Information and Communication Convergence Engineering 2023; 21: 90-97https://doi.org/10.56977/jicce.2023.21.1.90

Fig 7.

Figure 7.Comparison between the PDA and the Gaussian filtering using a correlation between SNR and MSE.
Journal of Information and Communication Convergence Engineering 2023; 21: 90-97https://doi.org/10.56977/jicce.2023.21.1.90

Fig 8.

Figure 8.Example of PDA advantages over ordinary filtering method.
Journal of Information and Communication Convergence Engineering 2023; 21: 90-97https://doi.org/10.56977/jicce.2023.21.1.90

Units for magnetic properties.

MSE SNR (dB)
Unfiltered 0.4827 19.5095
PDA 0.1882 23.5991
Gaussian 0.2340 22.6533
Wiener 0.2838 21.8156
Average 0.2781 21.9044
Median 0.3162 21.3470
Bilateral 0.4795 19.5388

References

  1. D. Gabor, A new microscopic principle, Nature, vol. 161, pp. 777-778, May., 1948. DOI: 10.1038/161777a0.
    Pubmed CrossRef
  2. T. Pitkäaho and A. Manninen and T. J. Naughton, Focus prediction in digital holographic microscopy using deep convolutional neural networks, Applied Optics, vol. 58, no. 5, pp. A202-A208, Jan., 2019. DOI: 10.1364/AO.58.00A202.
    Pubmed CrossRef
  3. J. A. Picazo-Bueno and M. Trusiak and V. Micó, Single-shot slightly off-axis digital holographic microscopy with add-on module based on beamsplitter cube, Optics Express, vol. 27, no. 4, pp. 5655-5669, Feb., 2019. DOI: 10.1364/OE.27.005655.
    Pubmed CrossRef
  4. S. Shin, and Y. Yu, Lensless reflection digital holographic microscope with a fresnel-bluestein transform, Journal of the Korean Physical Society, vol. 74, no. 2, pp. 98-101, Jan., 2019. DOI: 10.3938/jkps.74.98.
    CrossRef
  5. T. O'Connor and A. Doblas and B. Javidi, Structured illumination in compact and field-portable 3D-printed shearing digital holographic microscopy for resolution enhancement, Optics Letters, vol. 44, no. 9, pp. 2326-2329, May., 2019. DOI: 10.1364/OL.44.002326.
    Pubmed CrossRef
  6. Z. Zhong, and H. Zhao, and L. Cao, and M. Shan, and B. Liu, and W. Lu, and H. Xie, Automatic cross filtering for off-axis digital holographic microscopy, Results in Physics, vol. 16, 102910, Mar., 2020. DOI: 10.1016/j.rinp.2019.102910.
    CrossRef
  7. T. O'Connor and A. Anand and B. Javidi, Field-portable microsphereassisted high resolution digital holographic microscopy in compact and 3D-printed Mach-Zehnder Interferometer, OSA Continuum, vol. 3, no. 4, pp. 1013-1020, Apr., 2020. DOI: 10.1364/OSAC.389832.
    CrossRef
  8. J. Dong, and A. K. Yetisen, and X. Dong, and F. Pöller, and M. Jakobi, and Z. Liu, and F. S. Bloise, and A. W. Koch, Low-pass filtering compensation in common-path digital holographic microscopy, Applied Physics Letters, vol. 117, no. 12, 121105, Sep., 2020. DOI: 10.1063/5.0019209.
    CrossRef
  9. L. Huang, and L. Yan, and B. Chen, and Y. Zhou, and T. Yang, Phase aberration compensation of digital holographic microscopy with curve fitting preprocessing and automatic background segmentation for microstructure testing, Optics Communications, vol. 462, 125311, May., 2020. DOI: 10.1016/j.optcom.2020.125311.
    CrossRef
  10. H. W. Kim, and K. Inoue, and M. Cho, and M. C. Lee, A study on real-time modification of the refractive index of a surrounding medium using a uniform microsphere in digital holographic microscopy, in in Proceedings of the 2020 3rd International Conference on Electronics and Electrical Engineering Technology, Kitakyushu, Japan, pp. 44-48, 2020. DOI: 10.1145/3429536.3429539.
    CrossRef
  11. M. L. Piao and H. Y. Wu and N. Kim, 3D image encryption based on computer-generated hologram, in in Digital Holography and Three-Dimensional Imaging. Optical Society of America, Bordeaux, France, pp. W3A-21, 2019. DOI: 10.1364/DH.2019.W3A.21.
    CrossRef
  12. I. Shevkunov, and V. Katkovnik, and D. Claus, and G. Pedrini, and N. V. Petrov, and K. Egiazarian, Spectral object recognition in hyperspectral holography with complex-domain denoising, Sensors, vol. 19, no. 23, p. 5188, Nov., 2019. DOI: 10.3390/s19235188.
    Pubmed KoreaMed CrossRef
  13. A. Anand, and V. K. Chhaniwal, and N. R. Patel, and B. Javidi, Automatic identification of malaria-infected RBC with digital holographic microscopy using correlation algorithms, IEEE Photonics Journal, vol. 4, no. 5, pp. 1456-1464, Oct., 2012. DOI: 10.1109/JPHOT.2012.2210199.
    CrossRef
  14. F. Yi and I. Moon and Y. H. Lee, Three-dimensional counting of morphologically normal human red blood cells via digital holographic microscopy, Journal of Biomedical Optics, vol. 20, no. 1, 016005, Jan., 2015. DOI: 10.1117/1.JBO.20.1.016005.
    Pubmed CrossRef
  15. B. Kemper, and G. Von Bally, Digital holographic microscopy for live cell applications and technical inspection, Applied Optics, vol. 47, no. 4, pp. A52-A61, Feb., 2008. DOI: 10.1364/AO.47.000A52.
    Pubmed CrossRef
  16. T. Colomb, and N. Pavillon, and J. Kühn, and E. Cuche, and C. Depeursinge, and Y. Emery, Extended depth-of-focus by digital holographic microscopy, Optics Letters, vol. 35, no. 11, pp. 1840-1842, Jun., 2010. DOI: 10.1364/OL.35.001840.
    Pubmed CrossRef
  17. S. Shin, and Y. Yu, Fine metal mask 3-dimensional measurement by using scanning digital holographic microscope, Journal of the Korean Physical Society, vol. 72, no. 8, pp. 863-867, Apr., 2018. DOI: 10.3938/jkps.72.863.
    CrossRef
  18. J. Zakrisson and S. Schedin and M. Andersson, Cell shape identification using digital holographic microscopy, Applied Optics, vol. 54, no. 24, pp. 7442-7448, Aug., 2015. DOI: 10.1364/AO.54.007442.
    Pubmed CrossRef
  19. D. Roitshtain, and N. A. Turko, and B. Javidi, and N. T. Shaked, Flipping interferometry and its application for quantitative phase microscopy in a micro-channel, Optics Letters, vol. 41, no. 10, pp. 2354-2357, May., 2016. DOI: 10.1364/OL.41.002354.
    Pubmed CrossRef
  20. A. Anand and V. Chhaniwal and B. Javidi, Tutorial: common path selfreferencing digital holographic microscopy, APL Photonics, vol. 3, no. 7, 071101, Jul., 2018. DOI: 10.1063/1.5027081.
    CrossRef
  21. R. M. Goldstein and H. A. Zebker and C. L. Werner, Satellite radar interferometry: Two-dimensional phase unwrapping, Radio Science, vol. 23, no. 4, pp. 713-720, Jul., 1988. DOI: 10.1029/RS023i004p00713.
    CrossRef
  22. R. C. Gonzalez, and R. E. Woods Digital image processing, 4th ed., New York, NY: Pearson, 2002.
JICCE
Sep 30, 2023 Vol.21 No.3, pp. 185~260

Stats or Metrics

Share this article on

  • line

Related articles in JICCE

Journal of Information and Communication Convergence Engineering Jouranl of information and
communication convergence engineering
(J. Inf. Commun. Converg. Eng.)

eISSN 2234-8883
pISSN 2234-8255