Journal of information and communication convergence engineering 2024; 22(3): 256-266
Published online September 30, 2024
https://doi.org/10.56977/jicce.2024.22.3.256
© Korea Institute of Information and Communication Engineering
Correspondence to : Hee-Cheol Kim (E-mail: heeki@inje.ac.kr)
Institute of Digital Anti-Aging Healthcare, Inje University, Gimhae 50834, Republic of Korea
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Sasang constitutional medicine (SCM) is one of the best traditional therapeutic approaches used in Korea. SCM prioritizes personalized treatment that considers the unique constitution of an individual and encompasses their physical characteristics, personality traits, and susceptibility to specific diseases. Facial features are essential for diagnosing Sasang constitutional types (SCTs). This study aimed to develop a real-time artificial intelligence-based model for diagnosing SCTs using facial images, building an SCTs prediction model based on a machine learning method. Facial features from all images were extracted to develop this model using feature engineering and machine learning techniques. The fusion of these features was used to train the AI model. We used four machine learning algorithms, namely, random forest (RF), multilayer perceptron (MLP), gradient boosting machine (GBM), and extreme gradient boosting (XGB), to investigate SCTs. The GBM outperformed all the other models. The highest accuracy achieved in the experiment was 81%, indicating the robustness of the proposed model and suitability for real-time applications.
Keywords Sasang constitutional medicine, Sasang constitution type, Machine learning, Feature extraction
In oriental medicine, the human body is viewed as a microcosm of the universe, and diseases are diagnosed and treated based on yin-yang and five-element theories, which state that health and longevity are achieved through a harmonious relationship between spiritual, mental, and physical states [1]. Korean Sasang constitutional medicine (SCM) differs from traditional Chinese medicine in that it focuses on internal traits such as body type, temperament, symptoms, and pathology, rather than exterior similarities in disease diagnosis and treatment [2]. SCM is a traditional Korean oriental medicine that categorizes people into four groups based on various phenotypic traits. The Sasang constitutional theory was established by Je-Ma Lee [1] and has been widely used as a diagnostic and supportive tool in traditional Korean medicine. This theory states that individuals can be categorized into four groups based on their physiological and pathological characteristics: Tae-Yang (TY), So-Yang (SY), Tae-Eum (TE), and So-Eum (SE)[2]. Genetic research has shown that the human genome directly influences the complex processes involved in health and disease. Studies have been conducted to analyze the Sasang types, which significantly impact particular diseases, such as metabolic syndrome, which occurs mostly in the TE type, and irritable bowel syndrome (IBS), which is predominant in the SE type [3,4]. This study was conducted to analyze the significance of voice in categorizing the four constitution types and to extract more useful information from physical quantities to determine a person’s health status in traditional Korean medicine [5]. The facial features considered in this study for classifying Sasang types include the forehead, nose, ears, eyes, and mouth. Prediction of metabolic syndrome using SCTs is a high-risk factor. Each SCT is associated with various disease susceptibilities and risks for metabolic syndrome [6]. S. Yang [7] conducted a study to classify the Sasang constitution by determining the correlation with the sound spectrogram. Additionally, they discovered a correlation between the Sasang constitution and the sound spectrum through an in-depth analysis of the pitch and reading flow associated with the Sasang constitution. The Sasang topology is crucial in diagnosing and prescribing treatment for patients and assists in making the right decision for a medical physician. Predicting disease symptoms based on the SCM is crucial for administering appropriate treatments to mitigate potential risks.
In this study, we propose a facial feature-based hybrid machine learning model to predict the distribution of SCTs on the face of an individual. We conducted a series of experiments to develop this real-time model by fusing manual and pretrained model features. The face is a mirror of the entire body, revealing many facts such as emotions, health status, personality traits, feelings, age, and gender. We used a deep learning-based face detector to extract manual features and identify and crop the face according to its detected coordinates. Furthermore, a 68-point facial landmark was placed on the cropped face to capture possible geometrical and local features. The geometrical method found thirty-two significant features, including the angle, distance, and distance ratio. Manual and self-learning features were merged to obtain the final useful features. These features were used to train various machine-learning models. The contributions of this study are as follows:
We proposed a real-time Sasang constitution diagnosis distribution prediction model.
We attempted to adopt self-learning and feature-engineering methods to extract facial features.
Various machine-learning models were utilized to evaluate this pipeline on our custom dataset, and their performances were assessed using various evaluation techniques.
We implemented and tested it on a real-time system.
The remainder of this paper is organized as follows. Section 2 provides a brief overview of related studies. The subsequent section describes the data collection and the proposed method. In Section 4, we present the experimental results and discuss the proposed model. The final section presents conclusions and directions for future work.
The concept of SCM is crucial in examining the psychological and physical characteristics of the Sasang constitutional types (SCTs). People with various types of genetic information have different personality traits, susceptibility to certain diseases, and responses to certain drugs. Personalized medication or tailored medicine [8], in which SCTs have different susceptibilities to medical responses, helps use more efficient and useful medication based on their traits. Tailored medicine contributes to building a relationship between clinical profiles and personal molecules, helping physicians make correct and effective decisions. Kim conducted a study [9] to analyze the serum levels of oxidative and hormone stress based on the SCTs theory. This study analyzed the distribution of SCTs. They concluded that serum cortisol levels in the SE and SY groups were significantly higher than those in the TE group in both males and females. Inheritance of genomic variance significantly contributes to the development of different diseases [10]. Voice features are considered significant cues for analyzing SCTs, and various studies have been conducted to extract significant features from voices. However, this remains challenging owing to the high variation in individual voice feature stability. This challenge was overcome by introducing the repeatability concept and a Gaussian mixture model to extract additional voice features from every frame [11]. The study was conducted to analyze different kinds of disease vulnerability and medicine response using bioelectrical impedance analysis and the Meyers–Briggs type indicator (MBTI) of a person based on their respective personality traits [12]. The face and voice of a human being are highly considered when identifying SCTs, owing to their unique features. TY types have bright eyes and broad foreheads; SY types have thin and small lips, and narrow and sharp chins; TE types have flat faces, heavy lips, large mouths, noses, eyes, and ears; and SE types have narrow and long faces, small mouth, nose, eyes, and ears [13]. This study was conducted to analyze the connection between varieties of skin dampness prompted by sweat across Sasang types. Significant changes were observed before and after sweating, particularly in the TE and SE types, before and after sweating [14]. SCM concept theory states that patients are treated with natural medications based on SCTs. Skin diseases (skin outbreaks) are cured by herbal medications, such as scutellarin baicalinase and Raphanus sativus, and these medicines are especially useful for the TE type. The combination of scutellarin and Raphanus (SR) is widely used in wrinkle care beauty products [15]. SCTs use morphological features and measurement techniques based on facial appearance. The study concluded that TE has a wide bigonial breath and the longest face length, SY has long brows, and SE has big eyes [16]. Lee [17] conducted a study to elaborate on the structure and function of the body based on the Sasang quadrifocal scheme. The kidney, liver, spleen, and lungs are the main organs that control the functions of the human body, such as water-food metabolism and energyfluid metabolism. Understanding the function of the human body is necessary for leveraging SCM. It also highlights the interlinks between the main body organs, their traits, and SCTs. The TY type has a progressive attitude with more evolved lungs and fewer evolved liver regions. The TE type has a commercial, dedicated, and bearable nature, more mature liver, and less mature lung. Similarly, SE types were categorized as pessimistic and apprehensive. This type of person has more mature kidneys and less mature spleens. SY has an ambiguous mentality and boring personality; this type of person has fewer and more mature splenic areas. Atopic dermatitis is a common allergic disease in both young and infant populations. They conducted a series of experiments and concluded that SCM was effective in mitigating AD symptoms of atopic dermatitis within a short period [18]. Certain chronic diseases include diabetes mellitus (DM), metabolic disturbance (MD), metabolic syndrome, obesity, subclinical hypothyroidism, and obstructive sleep apnea [19-24]. Nonalcoholic fatty liver disease (NAFLD) is a common condition. SCTs, particularly SY and TE, are independently associated with NAFLD. The risk factor of SY is less for NAFLD as compared to TE [25,26]. Accurate evaluation and recognition of SCTs by medical communities significantly impact the diagnosis and treatment of cardiovascular disease (CVD). The proportion of CVD within the SCTs was significantly different between patients with and without MetS. The clinical and analytical features showed that in the TE type, the prevalence of type 2 diabetes mellitus and metabolic syndrome was higher than that in the SE and SY types. Metabolic syndrome is the main factor that increases the risk of CVD [27].
Recently Park et al. [28] developed a metabolic syndrome prediction model based on machine learning and SCTs. Six types of state-of-the-art machine learning techniques were applied to a few potential clinical features by considering the presence or absence of SCTs and performing a comparative analysis. The overall performance and sensitivity of each predictive model were better with the SCTs than without. Lee et al. conducted a long-term follow-up study [29], to investigate the association between SCTs and hypertension (HTN) prevalence using survival analysis. SCTs are categorized using hybrid diagnostic methods such as question area response, body shape, voice, and facial features. Survival analysis showed that the SE and SY subtypes were less likely to develop HTN than the TE subtype. A prospective cohort study was conducted to understand metabolic syndrome risk prediction using the Cox proportional hazards survival analysis method [30]. This study showed that SCTs significantly improved the performance of a metabolic syndrome predictive model. This study, conducted by Cho et al. [31], aimed to predict type 2 diabetes mellitus based on the SCTs and presented several conclusions. The TE type is independent of obesity and a strong risk factor for type 2 diabetes mellitus. This study, conducted by Nam et al. [32], aimed to improve algorithms for SCTs and traditional medical examination techniques. The metabolite markers used to classify SCT models are helpful for doctors to scientifically examine medications. Researchers have analyzed metabolomics to acquire marker metabolites in the serum and urine based on various SCTs. The quantities of lactate, triglycerides, glutamate, and glycolic acids in the urine and fatty acids in the serum were higher in the TE group than in the SE and SY groups. Hemophilia is a common disease associated with genetic disorders and inappropriate clothing. This study aimed to identify the symptoms of the disease for the prevention and treatment of hemophilia. The study concluded that the occurrence of hemophilia was higher in the TE type than in the SE and SY types [33]. Therefore, to improve the life expectancy of patients with hemophilia, they should be monitored daily for diabetes and blood pressure of the TE type [34].
SCM has a long history and serves as a clinical classification system that categorizes people into one of four distinct groups [35], similar to Ayurveda, the TCM of India [36]. Each type reflects the characteristics of the group [37]. SCM has been used by people from countries other than Korea, including China, the United States, and Vietnam [35]. Each constitution type may be associated with a particular set of physical and mental characteristics as well as an increased risk of certain diseases. A study comparing the Sasang constitution types of Korean and Japanese individuals based on facial features highlighted distinct differences between the two groups [38]. Here, we provide a summary of the characteristics and potentially associated diseases of each constitution type.
This constitution type is characterized by a predominance of Tae Yang energy and is associated with a strong, active, and energetic personality. They have large and angular heads, large and sharp eyes, thick and thin eyebrows, moderately black lips, and normal noses [5]. It is associated with an increased risk of respiratory, gastrointestinal, and cardiovascular diseases.
This constitution type is characterized by a predominance of so-yang energy and is associated with a calm, gentle, and introverted personality. They have oval-shaped heads, large round eyes, dark eyebrows, moderately black lips, and sharp noses [6]. It is associated with an increased risk of nervous, digestive, and immune disorders.
This constitution type is characterized by a balance of the Tae-Yang and Tae-Eum energies and is associated with a balanced, moderate personality. They have round rectangular heads, small thin eyes, soft thick eyebrows, thick lips, and blunt noses [6]. It is associated with an increased risk of metabolic syndrome, diabetes, and obesity. is associated with an increased risk of IBS, digestive disorders, and immune system disorders.
This constitution type is characterized by a balance between So-Yang and So-Eum energies and is associated with a sensitive and emotional personality. They have an inverted triangle-shaped head, weak and narrow eyes, light pale eyebrows, thin and pale lips, and normal noses [8].
This section discusses the data preprocessing technique and experimental setup used in this study. Developing an automatic real-time system that classifies SCTs based on facial features is useful for identifying disease symptoms along with other physical and clinical features.
Fig. 3 shows the workflow diagram of the SCTs module and the real-time system. The SCTs module includes four phases: pre-processing, handcraft-based feature extraction, computation-based (CNN) feature extraction, and classification using AI models, as shown in Fig. 1(a). The real-time application included a mobile application that captured an image and sent it to the AI model. The AI model predicts the Sasang constitution probability of a face, as shown in Fig. 1(b).
The experiment used Windows 10 with a 64-bit operating system specification with an 11th Gen Intel (R) Core (TM) i7-11700 @ 2.50 GHz, 2.50 GHz processor, and 16 GB RAM. The Anaconda version 4.10.1 framework was used, and the Jupiter Notebook, an open-source web application, was used as an environment ID. Python version 3.8.8) was used as a programming language. The entire study was conducted in a smart computing lab using the specifications and software mentioned above.
We used a custom facial image dataset from an ITV company in South Korea to experiment. The dataset contains 400 labeled facial images of Korean people, including men and women, with various faces. Company experts labeled the entire dataset into four classes based on facial SCTs, body shape, and clinical features. The classes were Yang (TY), So-Yang (SY), So-Eum ( ) and Tae-Eum (TE).
To understand the SCTs topology, company experts define rules and provide samples of facial images with their types, as shown in Fig. 2. The face images in the dataset are shown in Figs. 1 and 4 with their respective labels. The rules are defined by considering facial image organs, such as the head, eyebrows, eye, nose, and lips, as shown in Table 1) based on the corresponding Sasang constitution facial attributes. Moreover, the company provides traditional geometric measurement ratios between different organs in facial images, as shown in Fig. 5 and Table 2.
Table 1 . Facial attributes of Sasang Topology
Attributes | Tae-Yang (TY) | So-Yang (SY) | So-Eum (SE) | Tae-Eum (TE) |
---|---|---|---|---|
Head | Large and angular | Oval shape | Inverted triangle | Round Rectangle |
Eyebrows | Thick and Thin | Dark | Light pale | Soft and thick |
Eye | Big, Sharp | Round, large | Weak and narrow | Small and thin |
Nose | Normal | Sharp | Normal | Blunt |
Lips | The lips line is black | Moderate | Thin and Pale | Thickness (steamed) |
Table 2 . Handcrafted extracted features are based on landmark key points
Gender | Case | Face (W) | Face (H) | Forehead (W) | Forehead (H) | Eyebrows (W) | Eyebrows (H) | Eye (W) | Eye (H) | B/w Eyebrows (W) | Nose (H) | Nose (W) | Philtrum (H) |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Male | SY | 13.2 cm | 19.2 cm | 11.6 cm | 4.5 cm | 4.1 cm | 1.0 cm | 2.8 cm | 0.9 cm | 3.4 cm | 4.0 cm | 4.2 cm | 1.8 cm |
SE | 13.4 cm | 18.7 cm | 11.8 cm | 4.3 cm | 4.0 cm | 0.9 cm | 2.6 cm | 0.8 cm | 3.5 cm | 4.0 cm | 4.3 cm | 1.8 cm | |
TE | 14.7 cm | 20.3 cm | 12.6 cm | 4.6 cm | 4.7 cm | 1.0 cm | 2.8 cm | 0.8 cm | 3.6 cm | 4.2 cm | 4.3 cm | 2.0 cm | |
TY | 14.0 cm | 19.4 cm | 11.9 cm | 4.4 cm | 4.4 cm | 1.0 cm | 2.9 cm | 0.8 cm | 3.6 cm | 4.3 cm | 4.3 cm | 1.8 cm | |
Female | SY | 13.6 cm | 18.3 cm | 11.7 cm | 4.4 cm | 4.4 cm | 0.8 cm | 3.1 cm | 1.1 cm | 3.6 cm | 3.9 cm | 4.0 cm | 1.6 cm |
SE | 12.7 cm | 17.9 cm | 10.9 cm | 4.6 cm | 4.1 cm | 0.7 cm | 2.7 cm | 0.8 cm | 3.3 cm | 3.4 cm | 3.9 cm | 1.6 cm | |
TE | 14.3 cm | 18.6 cm | 11.4 cm | 4.2 cm | 4.5 cm | 0.8 cm | 2.9 cm | 0.8 cm | 3.4 cm | 4.0 cm | 4.0 cm | 1.6 cm | |
TY | 13.4 cm | 18.4 cm | 11.3 cm | 4.3 cm | 4.2 cm | 0.7 cm | 2.7 cm | 0.9 cm | 3.4 cm | 3.8 cm | 4.1 cm | 1.6 cm |
Data pre-processing is a crucial step in developing a model. The model training performance depends on the input data arrangement; therefore, we should consider feeding more accurate and well-organized data into the model.
In this study, we used Haar Cascade’s frontal face [36] library with an XML file and a dlib library to identify and detect the coordinates of the face image. The face coordinates were used to crop the exact locations of all detected faces and remove the remaining parts from the images. All images were resized according to the requirements of the pre-train model VGG19.
This section discusses the feature extraction method based on feature engineering and automatic learning of features from face images. Facial landmark features and the transfer learning-based model VGG19 were considered once extracted and fused to obtain a single feature vector for training and testing.
The Haar Cascade classifier is an object detector that provides coordinate values for each face. A facial landmark is a set of 68 points that are equally distributed on the edges of the face and facial organs, as shown in Fig. 6. A facial landmark detector was used to detect and track facial images. A combination of landmark points and facial coordinates was used to calculate the handcrafted features, as shown in Table 3. A 17-point facial landmark and a 12-point eye landmark were used to measure the facial geometry and components of each organ. The distance formula is used to determine the length between each keypoint.
Table 3 . Average value of measured face organ by constitution type
Feature ID | Description | Formula |
---|---|---|
f1 | Length to width ratio of face | |
f2 | Length to jawline ratio of face | |
f3 | Length to temple ratio of face | |
f4 | Length to width ratio of the left eye | |
f5 | Length to width ratio of the right eye | |
f6 | Nose height and width ratio | |
f1-f15 | The angle between right side landmark points | |
f16-f23 | The angle between left side landmark points | |
f24-f32 | Distance between the left side and right side of the landmark point. |
Similarly, the angle between the key points and other parameters was measured. The mouth and nose length-width ratios were also considered. The eye and mouth are the most important parts of the analysis of SCTs. The OpenCV library provides eye landmarks to extract eye features [38]. The 6-point eye landmark represents the attributes of each eye. The aspect ratio of the eye (ARE) is calculated as follows:
Eq. (1 provides the ratio between the length and width of an eye, and Eqs. (2) and (3) provide the center of each eye. An equation can determine the center area between both eyes based on Eq. (4).
Therefore, based on these features, we obtain 12 powerful features, and this feature vector is fused with the facial landmark feature vector. The overall feature vector length obtained from the feature engineering method is 32, as listed in Table 3.
Deep learning architectures (DLA), particularly CNN-based models, have become popular. It is widely used to solve image classification problems. In this study, the pretraining model extracts the facial features already trained on the ImageNet dataset. VGG19 is a powerful pretrained model for image analysis. VGG19 is a series of convolutional neural network layers followed by one or more dense layers.
Specific features were extracted from the custom dataset via feature extraction. This model provides a feature vector that is used with a manually extracted feature to identify SCTs. Moreover, we also used other pre-trained models such as Mobile Net, RestNet52, inceptionv3, and RestNet101 for comparison and found that VGG19 provides a more effective feature result than the other models.
The proposed model is specifically designed for facial recognition of SCTs based on their personality and facial features. We analyzed facial characteristics from various angles to classify these types. However, similar to other models, this model has limitations. It can only be applied to the facial recognition of Korean individuals for medical purposes, considering their personality traits. Some studies [37, 38] have suggested that due to facial feature resemblances between Korean, Chinese, and Vietnamese individuals, this model may also apply to people from these countries.
This section presents the results of this study. We performed experiments using various techniques to build an effective automatic SCT classification system and selected the most suitable technique. We evaluated the performance of our model using the classification accuracy, confusion matrix, area under the curve (AUC), recall, precision, F1 score, and receiver operating characteristic (ROC) curve.
The confusion metric analysis provides a broad overview of the model performance, particularly the main diagonal values that provide the accuracy of the model. The accuracy, precision, recall, and F1-score were used to evaluate the performance of our proposed model based on confusion metric parameters. However, Fig. 7 shows the information based on the actual and predicted values. Four classifiers were selected, and a comparative analysis was performed. Precision is the ratio between the number of SCTs that are truly identified and all SCTs. In comparison, the F1-score is the representation between the predicted and actual outputs of the SCTs classification.
Table 4 illustrates the performance of the different machine learning models, and Fig. 8 shows the ROC curves of the respective models. Precision, recall, F1 score, and accuracy were the evaluation techniques used to analyze the output results of these models. The accuracies of RF, MLP, XGB, and GBM were 73.8, 76.5, 78.6, and 80.6%, respectively. ROC curves are graphical representations of the model. Fig. 9 shows a graphical representation of the various models and provides detailed information regarding the models.
Table 4 . Performance of different machine learning models
Classifier | Precision | Recall | F1-score | Accuracy |
---|---|---|---|---|
RF | 80% | 81.75% | 80.75% | 73.8% |
MLP | 82.25% | 80.25% | 79.25% | 76.5% |
XGB | 80.5% | 79.5% | 79.75% | 78.6% |
GBM | 81.9% | 85 % | 83% | 80.6% |
A person’s appearance is the most important factor in differentiating the constitution. The external appearance of the body shape is reflected by facial features and shape. To investigate the proper SCTs, we developed a real-time AI-based Sasang constitution prediction model. The practical use of facial characteristics in the Sasang constitution realtime system based on AI was tested several times by our laboratory members who noticed the response of real-time results. Fig. 9 shows the real-time AI model results of our lab members, demonstrating its strong performance.
We analyzed facial features from a different perspective to classify SCTs. This study aimed to develop an automatic computer-based face recognition Sasang constitution diagnostic model to help diagnose a person based on personality and traits. Although diagnosing the SCT remains challenging, this model allows stakeholders to identify the type accurately. Based on specific constitutional types, medical stakeholders can easily recommend medicines that align with the SCM concept. We used different machine learning approaches such as RF, MLP, XGBM, and GBM to identify SCTs. Moreover, among all the classifiers, the GBM yielded the best results. The output of our proposed model is a numerical value as a confusion matrix and a graphical representation as a ROC curve. Fig. 7 shows the actual and predicted values on the y- and x-axes, respectively. In the main diagonal, the actual and predicted values are the same, indicating the accuracy of the model. Table 4 shows the accuracy, F1-score, recall, and precision of the GBM model, which were 83, 83, 85, and 81.9%, respectively. As shown in Fig. 8, the ROC curves show graphical representations of each output classifier.
This study aimed to classify SCTs based on physiological and pathological features using transfer learning and facial landmarks, which will help to develop a computer-based SCT classification system. The face is the most prominent and representative part of an individual. This revealed a significant amount of information related to an individual. Age, sex, face shape, facial organ size, color, and personality traits were the key factors that helped identify the correct SCTs. SCTs help accurately predict or diagnose a patient, doctor, and images of Korean people, including males and females of different age groups. Furthermore, the data size is increased using traditional data augmentation techniques, and SCTs are important in this study. To extract features, we adopted feature extraction methods, and the feature engineering method obtained from both feature extraction techniques was combined to train the model. Additionally, a transfer-learningbased method was employed. The features resized all images to 256 × 256, according to the requirements, and features were extracted from the facial images for classification by a physicist. In Korea, SCM is popular and is extensively used in the medical domain. The proposed model is highly effective and robust for real-time applications.
We used a custom dataset containing 400 faces. Moreover, we analyzed various machine learning models for these feature values. We selected four machine learning models and performed a comparative analysis based on the results. The performance of the GBM was excellent among the classifiers, and the overall accuracy of this model was 80.6%. We computed the precision, recall, and F1-Score to analyze the model performance. Because of the deep analysis, we are confident that our proposed model is more efficient and robust than state-of-the-art machine-learning models. In the future, this work can be extended and its performance can be improved using additional data. Moreover, an advanced artificial neural network classifier can be used for better results.
This research was supported by the MSIT (Ministry of Science ICT), Korea, under the National Program for Excellence in SW, and supervised by the Institute of Information & Communications Technology Planning & Evaluation (IITP) in 2022.
The authors declare no conflicts of interest. The funders had no role in the study design; collection, analyses, or interpretation of data; writing of the manuscript; or decision to publish the results.
Abdullah
Abdullah is a PhD student at the College of Science and Engineering, James Cook University in Australia. He completed his master’s degree at Institute of Digital Anti-aging Healthcare from Inje University Republic of Korea and bachelor’s in electrical and computer engineering from Pakistan. His research interest includes artificial intelligence, medical imaging, computer vision, and deep learning.
Shah Mahsoom Ali
Shah Mahsoom Ali has completed a BS in Software Engineering from Abbottabad University of Science and Technology, Pakistan. He is currently a master's student in the Computer Engineering department at Inje University. His research interests include Artificial Intelligence, computer vision, Medical Imaging and deep learning.
Hee Cheol Kim
Hee-Cheol Kim earned a BSc from the Department of Mathematics and an MSc from the Department of Computer Science at SoGang University in Republic of Korea. He completed his Ph.D. in Numerical Analysis and Computing Science at Stockholm University in Sweden. He is a professor and Head of the Institute of Digital Anti-aging Healthcare at Inje University, Republic of Korea. His research interests include machine learning, text mining, and bioinformatics.
Journal of information and communication convergence engineering 2024; 22(3): 256-266
Published online September 30, 2024 https://doi.org/10.56977/jicce.2024.22.3.256
Copyright © Korea Institute of Information and Communication Engineering.
Abdullah 1, Shah Mahsoom Ali 2, and Hee-Cheol Kim3*
1Department of College of Science and Engineering, James Cook University, Cairns, QLD 4870, Australia
2Department of Computer Engineering, Inje University, Gimhae 50834, Republic of Korea
3Institute of Digital Anti-Aging Healthcare, Inje University, Gimhae 50834, Republic of Korea
Correspondence to:Hee-Cheol Kim (E-mail: heeki@inje.ac.kr)
Institute of Digital Anti-Aging Healthcare, Inje University, Gimhae 50834, Republic of Korea
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Sasang constitutional medicine (SCM) is one of the best traditional therapeutic approaches used in Korea. SCM prioritizes personalized treatment that considers the unique constitution of an individual and encompasses their physical characteristics, personality traits, and susceptibility to specific diseases. Facial features are essential for diagnosing Sasang constitutional types (SCTs). This study aimed to develop a real-time artificial intelligence-based model for diagnosing SCTs using facial images, building an SCTs prediction model based on a machine learning method. Facial features from all images were extracted to develop this model using feature engineering and machine learning techniques. The fusion of these features was used to train the AI model. We used four machine learning algorithms, namely, random forest (RF), multilayer perceptron (MLP), gradient boosting machine (GBM), and extreme gradient boosting (XGB), to investigate SCTs. The GBM outperformed all the other models. The highest accuracy achieved in the experiment was 81%, indicating the robustness of the proposed model and suitability for real-time applications.
Keywords: Sasang constitutional medicine, Sasang constitution type, Machine learning, Feature extraction
In oriental medicine, the human body is viewed as a microcosm of the universe, and diseases are diagnosed and treated based on yin-yang and five-element theories, which state that health and longevity are achieved through a harmonious relationship between spiritual, mental, and physical states [1]. Korean Sasang constitutional medicine (SCM) differs from traditional Chinese medicine in that it focuses on internal traits such as body type, temperament, symptoms, and pathology, rather than exterior similarities in disease diagnosis and treatment [2]. SCM is a traditional Korean oriental medicine that categorizes people into four groups based on various phenotypic traits. The Sasang constitutional theory was established by Je-Ma Lee [1] and has been widely used as a diagnostic and supportive tool in traditional Korean medicine. This theory states that individuals can be categorized into four groups based on their physiological and pathological characteristics: Tae-Yang (TY), So-Yang (SY), Tae-Eum (TE), and So-Eum (SE)[2]. Genetic research has shown that the human genome directly influences the complex processes involved in health and disease. Studies have been conducted to analyze the Sasang types, which significantly impact particular diseases, such as metabolic syndrome, which occurs mostly in the TE type, and irritable bowel syndrome (IBS), which is predominant in the SE type [3,4]. This study was conducted to analyze the significance of voice in categorizing the four constitution types and to extract more useful information from physical quantities to determine a person’s health status in traditional Korean medicine [5]. The facial features considered in this study for classifying Sasang types include the forehead, nose, ears, eyes, and mouth. Prediction of metabolic syndrome using SCTs is a high-risk factor. Each SCT is associated with various disease susceptibilities and risks for metabolic syndrome [6]. S. Yang [7] conducted a study to classify the Sasang constitution by determining the correlation with the sound spectrogram. Additionally, they discovered a correlation between the Sasang constitution and the sound spectrum through an in-depth analysis of the pitch and reading flow associated with the Sasang constitution. The Sasang topology is crucial in diagnosing and prescribing treatment for patients and assists in making the right decision for a medical physician. Predicting disease symptoms based on the SCM is crucial for administering appropriate treatments to mitigate potential risks.
In this study, we propose a facial feature-based hybrid machine learning model to predict the distribution of SCTs on the face of an individual. We conducted a series of experiments to develop this real-time model by fusing manual and pretrained model features. The face is a mirror of the entire body, revealing many facts such as emotions, health status, personality traits, feelings, age, and gender. We used a deep learning-based face detector to extract manual features and identify and crop the face according to its detected coordinates. Furthermore, a 68-point facial landmark was placed on the cropped face to capture possible geometrical and local features. The geometrical method found thirty-two significant features, including the angle, distance, and distance ratio. Manual and self-learning features were merged to obtain the final useful features. These features were used to train various machine-learning models. The contributions of this study are as follows:
We proposed a real-time Sasang constitution diagnosis distribution prediction model.
We attempted to adopt self-learning and feature-engineering methods to extract facial features.
Various machine-learning models were utilized to evaluate this pipeline on our custom dataset, and their performances were assessed using various evaluation techniques.
We implemented and tested it on a real-time system.
The remainder of this paper is organized as follows. Section 2 provides a brief overview of related studies. The subsequent section describes the data collection and the proposed method. In Section 4, we present the experimental results and discuss the proposed model. The final section presents conclusions and directions for future work.
The concept of SCM is crucial in examining the psychological and physical characteristics of the Sasang constitutional types (SCTs). People with various types of genetic information have different personality traits, susceptibility to certain diseases, and responses to certain drugs. Personalized medication or tailored medicine [8], in which SCTs have different susceptibilities to medical responses, helps use more efficient and useful medication based on their traits. Tailored medicine contributes to building a relationship between clinical profiles and personal molecules, helping physicians make correct and effective decisions. Kim conducted a study [9] to analyze the serum levels of oxidative and hormone stress based on the SCTs theory. This study analyzed the distribution of SCTs. They concluded that serum cortisol levels in the SE and SY groups were significantly higher than those in the TE group in both males and females. Inheritance of genomic variance significantly contributes to the development of different diseases [10]. Voice features are considered significant cues for analyzing SCTs, and various studies have been conducted to extract significant features from voices. However, this remains challenging owing to the high variation in individual voice feature stability. This challenge was overcome by introducing the repeatability concept and a Gaussian mixture model to extract additional voice features from every frame [11]. The study was conducted to analyze different kinds of disease vulnerability and medicine response using bioelectrical impedance analysis and the Meyers–Briggs type indicator (MBTI) of a person based on their respective personality traits [12]. The face and voice of a human being are highly considered when identifying SCTs, owing to their unique features. TY types have bright eyes and broad foreheads; SY types have thin and small lips, and narrow and sharp chins; TE types have flat faces, heavy lips, large mouths, noses, eyes, and ears; and SE types have narrow and long faces, small mouth, nose, eyes, and ears [13]. This study was conducted to analyze the connection between varieties of skin dampness prompted by sweat across Sasang types. Significant changes were observed before and after sweating, particularly in the TE and SE types, before and after sweating [14]. SCM concept theory states that patients are treated with natural medications based on SCTs. Skin diseases (skin outbreaks) are cured by herbal medications, such as scutellarin baicalinase and Raphanus sativus, and these medicines are especially useful for the TE type. The combination of scutellarin and Raphanus (SR) is widely used in wrinkle care beauty products [15]. SCTs use morphological features and measurement techniques based on facial appearance. The study concluded that TE has a wide bigonial breath and the longest face length, SY has long brows, and SE has big eyes [16]. Lee [17] conducted a study to elaborate on the structure and function of the body based on the Sasang quadrifocal scheme. The kidney, liver, spleen, and lungs are the main organs that control the functions of the human body, such as water-food metabolism and energyfluid metabolism. Understanding the function of the human body is necessary for leveraging SCM. It also highlights the interlinks between the main body organs, their traits, and SCTs. The TY type has a progressive attitude with more evolved lungs and fewer evolved liver regions. The TE type has a commercial, dedicated, and bearable nature, more mature liver, and less mature lung. Similarly, SE types were categorized as pessimistic and apprehensive. This type of person has more mature kidneys and less mature spleens. SY has an ambiguous mentality and boring personality; this type of person has fewer and more mature splenic areas. Atopic dermatitis is a common allergic disease in both young and infant populations. They conducted a series of experiments and concluded that SCM was effective in mitigating AD symptoms of atopic dermatitis within a short period [18]. Certain chronic diseases include diabetes mellitus (DM), metabolic disturbance (MD), metabolic syndrome, obesity, subclinical hypothyroidism, and obstructive sleep apnea [19-24]. Nonalcoholic fatty liver disease (NAFLD) is a common condition. SCTs, particularly SY and TE, are independently associated with NAFLD. The risk factor of SY is less for NAFLD as compared to TE [25,26]. Accurate evaluation and recognition of SCTs by medical communities significantly impact the diagnosis and treatment of cardiovascular disease (CVD). The proportion of CVD within the SCTs was significantly different between patients with and without MetS. The clinical and analytical features showed that in the TE type, the prevalence of type 2 diabetes mellitus and metabolic syndrome was higher than that in the SE and SY types. Metabolic syndrome is the main factor that increases the risk of CVD [27].
Recently Park et al. [28] developed a metabolic syndrome prediction model based on machine learning and SCTs. Six types of state-of-the-art machine learning techniques were applied to a few potential clinical features by considering the presence or absence of SCTs and performing a comparative analysis. The overall performance and sensitivity of each predictive model were better with the SCTs than without. Lee et al. conducted a long-term follow-up study [29], to investigate the association between SCTs and hypertension (HTN) prevalence using survival analysis. SCTs are categorized using hybrid diagnostic methods such as question area response, body shape, voice, and facial features. Survival analysis showed that the SE and SY subtypes were less likely to develop HTN than the TE subtype. A prospective cohort study was conducted to understand metabolic syndrome risk prediction using the Cox proportional hazards survival analysis method [30]. This study showed that SCTs significantly improved the performance of a metabolic syndrome predictive model. This study, conducted by Cho et al. [31], aimed to predict type 2 diabetes mellitus based on the SCTs and presented several conclusions. The TE type is independent of obesity and a strong risk factor for type 2 diabetes mellitus. This study, conducted by Nam et al. [32], aimed to improve algorithms for SCTs and traditional medical examination techniques. The metabolite markers used to classify SCT models are helpful for doctors to scientifically examine medications. Researchers have analyzed metabolomics to acquire marker metabolites in the serum and urine based on various SCTs. The quantities of lactate, triglycerides, glutamate, and glycolic acids in the urine and fatty acids in the serum were higher in the TE group than in the SE and SY groups. Hemophilia is a common disease associated with genetic disorders and inappropriate clothing. This study aimed to identify the symptoms of the disease for the prevention and treatment of hemophilia. The study concluded that the occurrence of hemophilia was higher in the TE type than in the SE and SY types [33]. Therefore, to improve the life expectancy of patients with hemophilia, they should be monitored daily for diabetes and blood pressure of the TE type [34].
SCM has a long history and serves as a clinical classification system that categorizes people into one of four distinct groups [35], similar to Ayurveda, the TCM of India [36]. Each type reflects the characteristics of the group [37]. SCM has been used by people from countries other than Korea, including China, the United States, and Vietnam [35]. Each constitution type may be associated with a particular set of physical and mental characteristics as well as an increased risk of certain diseases. A study comparing the Sasang constitution types of Korean and Japanese individuals based on facial features highlighted distinct differences between the two groups [38]. Here, we provide a summary of the characteristics and potentially associated diseases of each constitution type.
This constitution type is characterized by a predominance of Tae Yang energy and is associated with a strong, active, and energetic personality. They have large and angular heads, large and sharp eyes, thick and thin eyebrows, moderately black lips, and normal noses [5]. It is associated with an increased risk of respiratory, gastrointestinal, and cardiovascular diseases.
This constitution type is characterized by a predominance of so-yang energy and is associated with a calm, gentle, and introverted personality. They have oval-shaped heads, large round eyes, dark eyebrows, moderately black lips, and sharp noses [6]. It is associated with an increased risk of nervous, digestive, and immune disorders.
This constitution type is characterized by a balance of the Tae-Yang and Tae-Eum energies and is associated with a balanced, moderate personality. They have round rectangular heads, small thin eyes, soft thick eyebrows, thick lips, and blunt noses [6]. It is associated with an increased risk of metabolic syndrome, diabetes, and obesity. is associated with an increased risk of IBS, digestive disorders, and immune system disorders.
This constitution type is characterized by a balance between So-Yang and So-Eum energies and is associated with a sensitive and emotional personality. They have an inverted triangle-shaped head, weak and narrow eyes, light pale eyebrows, thin and pale lips, and normal noses [8].
This section discusses the data preprocessing technique and experimental setup used in this study. Developing an automatic real-time system that classifies SCTs based on facial features is useful for identifying disease symptoms along with other physical and clinical features.
Fig. 3 shows the workflow diagram of the SCTs module and the real-time system. The SCTs module includes four phases: pre-processing, handcraft-based feature extraction, computation-based (CNN) feature extraction, and classification using AI models, as shown in Fig. 1(a). The real-time application included a mobile application that captured an image and sent it to the AI model. The AI model predicts the Sasang constitution probability of a face, as shown in Fig. 1(b).
The experiment used Windows 10 with a 64-bit operating system specification with an 11th Gen Intel (R) Core (TM) i7-11700 @ 2.50 GHz, 2.50 GHz processor, and 16 GB RAM. The Anaconda version 4.10.1 framework was used, and the Jupiter Notebook, an open-source web application, was used as an environment ID. Python version 3.8.8) was used as a programming language. The entire study was conducted in a smart computing lab using the specifications and software mentioned above.
We used a custom facial image dataset from an ITV company in South Korea to experiment. The dataset contains 400 labeled facial images of Korean people, including men and women, with various faces. Company experts labeled the entire dataset into four classes based on facial SCTs, body shape, and clinical features. The classes were Yang (TY), So-Yang (SY), So-Eum ( ) and Tae-Eum (TE).
To understand the SCTs topology, company experts define rules and provide samples of facial images with their types, as shown in Fig. 2. The face images in the dataset are shown in Figs. 1 and 4 with their respective labels. The rules are defined by considering facial image organs, such as the head, eyebrows, eye, nose, and lips, as shown in Table 1) based on the corresponding Sasang constitution facial attributes. Moreover, the company provides traditional geometric measurement ratios between different organs in facial images, as shown in Fig. 5 and Table 2.
Table 1 . Facial attributes of Sasang Topology.
Attributes | Tae-Yang (TY) | So-Yang (SY) | So-Eum (SE) | Tae-Eum (TE) |
---|---|---|---|---|
Head | Large and angular | Oval shape | Inverted triangle | Round Rectangle |
Eyebrows | Thick and Thin | Dark | Light pale | Soft and thick |
Eye | Big, Sharp | Round, large | Weak and narrow | Small and thin |
Nose | Normal | Sharp | Normal | Blunt |
Lips | The lips line is black | Moderate | Thin and Pale | Thickness (steamed) |
Table 2 . Handcrafted extracted features are based on landmark key points.
Gender | Case | Face (W) | Face (H) | Forehead (W) | Forehead (H) | Eyebrows (W) | Eyebrows (H) | Eye (W) | Eye (H) | B/w Eyebrows (W) | Nose (H) | Nose (W) | Philtrum (H) |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Male | SY | 13.2 cm | 19.2 cm | 11.6 cm | 4.5 cm | 4.1 cm | 1.0 cm | 2.8 cm | 0.9 cm | 3.4 cm | 4.0 cm | 4.2 cm | 1.8 cm |
SE | 13.4 cm | 18.7 cm | 11.8 cm | 4.3 cm | 4.0 cm | 0.9 cm | 2.6 cm | 0.8 cm | 3.5 cm | 4.0 cm | 4.3 cm | 1.8 cm | |
TE | 14.7 cm | 20.3 cm | 12.6 cm | 4.6 cm | 4.7 cm | 1.0 cm | 2.8 cm | 0.8 cm | 3.6 cm | 4.2 cm | 4.3 cm | 2.0 cm | |
TY | 14.0 cm | 19.4 cm | 11.9 cm | 4.4 cm | 4.4 cm | 1.0 cm | 2.9 cm | 0.8 cm | 3.6 cm | 4.3 cm | 4.3 cm | 1.8 cm | |
Female | SY | 13.6 cm | 18.3 cm | 11.7 cm | 4.4 cm | 4.4 cm | 0.8 cm | 3.1 cm | 1.1 cm | 3.6 cm | 3.9 cm | 4.0 cm | 1.6 cm |
SE | 12.7 cm | 17.9 cm | 10.9 cm | 4.6 cm | 4.1 cm | 0.7 cm | 2.7 cm | 0.8 cm | 3.3 cm | 3.4 cm | 3.9 cm | 1.6 cm | |
TE | 14.3 cm | 18.6 cm | 11.4 cm | 4.2 cm | 4.5 cm | 0.8 cm | 2.9 cm | 0.8 cm | 3.4 cm | 4.0 cm | 4.0 cm | 1.6 cm | |
TY | 13.4 cm | 18.4 cm | 11.3 cm | 4.3 cm | 4.2 cm | 0.7 cm | 2.7 cm | 0.9 cm | 3.4 cm | 3.8 cm | 4.1 cm | 1.6 cm |
Data pre-processing is a crucial step in developing a model. The model training performance depends on the input data arrangement; therefore, we should consider feeding more accurate and well-organized data into the model.
In this study, we used Haar Cascade’s frontal face [36] library with an XML file and a dlib library to identify and detect the coordinates of the face image. The face coordinates were used to crop the exact locations of all detected faces and remove the remaining parts from the images. All images were resized according to the requirements of the pre-train model VGG19.
This section discusses the feature extraction method based on feature engineering and automatic learning of features from face images. Facial landmark features and the transfer learning-based model VGG19 were considered once extracted and fused to obtain a single feature vector for training and testing.
The Haar Cascade classifier is an object detector that provides coordinate values for each face. A facial landmark is a set of 68 points that are equally distributed on the edges of the face and facial organs, as shown in Fig. 6. A facial landmark detector was used to detect and track facial images. A combination of landmark points and facial coordinates was used to calculate the handcrafted features, as shown in Table 3. A 17-point facial landmark and a 12-point eye landmark were used to measure the facial geometry and components of each organ. The distance formula is used to determine the length between each keypoint.
Table 3 . Average value of measured face organ by constitution type.
Feature ID | Description | Formula |
---|---|---|
f1 | Length to width ratio of face | |
f2 | Length to jawline ratio of face | |
f3 | Length to temple ratio of face | |
f4 | Length to width ratio of the left eye | |
f5 | Length to width ratio of the right eye | |
f6 | Nose height and width ratio | |
f1-f15 | The angle between right side landmark points | |
f16-f23 | The angle between left side landmark points | |
f24-f32 | Distance between the left side and right side of the landmark point. |
Similarly, the angle between the key points and other parameters was measured. The mouth and nose length-width ratios were also considered. The eye and mouth are the most important parts of the analysis of SCTs. The OpenCV library provides eye landmarks to extract eye features [38]. The 6-point eye landmark represents the attributes of each eye. The aspect ratio of the eye (ARE) is calculated as follows:
Eq. (1 provides the ratio between the length and width of an eye, and Eqs. (2) and (3) provide the center of each eye. An equation can determine the center area between both eyes based on Eq. (4).
Therefore, based on these features, we obtain 12 powerful features, and this feature vector is fused with the facial landmark feature vector. The overall feature vector length obtained from the feature engineering method is 32, as listed in Table 3.
Deep learning architectures (DLA), particularly CNN-based models, have become popular. It is widely used to solve image classification problems. In this study, the pretraining model extracts the facial features already trained on the ImageNet dataset. VGG19 is a powerful pretrained model for image analysis. VGG19 is a series of convolutional neural network layers followed by one or more dense layers.
Specific features were extracted from the custom dataset via feature extraction. This model provides a feature vector that is used with a manually extracted feature to identify SCTs. Moreover, we also used other pre-trained models such as Mobile Net, RestNet52, inceptionv3, and RestNet101 for comparison and found that VGG19 provides a more effective feature result than the other models.
The proposed model is specifically designed for facial recognition of SCTs based on their personality and facial features. We analyzed facial characteristics from various angles to classify these types. However, similar to other models, this model has limitations. It can only be applied to the facial recognition of Korean individuals for medical purposes, considering their personality traits. Some studies [37, 38] have suggested that due to facial feature resemblances between Korean, Chinese, and Vietnamese individuals, this model may also apply to people from these countries.
This section presents the results of this study. We performed experiments using various techniques to build an effective automatic SCT classification system and selected the most suitable technique. We evaluated the performance of our model using the classification accuracy, confusion matrix, area under the curve (AUC), recall, precision, F1 score, and receiver operating characteristic (ROC) curve.
The confusion metric analysis provides a broad overview of the model performance, particularly the main diagonal values that provide the accuracy of the model. The accuracy, precision, recall, and F1-score were used to evaluate the performance of our proposed model based on confusion metric parameters. However, Fig. 7 shows the information based on the actual and predicted values. Four classifiers were selected, and a comparative analysis was performed. Precision is the ratio between the number of SCTs that are truly identified and all SCTs. In comparison, the F1-score is the representation between the predicted and actual outputs of the SCTs classification.
Table 4 illustrates the performance of the different machine learning models, and Fig. 8 shows the ROC curves of the respective models. Precision, recall, F1 score, and accuracy were the evaluation techniques used to analyze the output results of these models. The accuracies of RF, MLP, XGB, and GBM were 73.8, 76.5, 78.6, and 80.6%, respectively. ROC curves are graphical representations of the model. Fig. 9 shows a graphical representation of the various models and provides detailed information regarding the models.
Table 4 . Performance of different machine learning models.
Classifier | Precision | Recall | F1-score | Accuracy |
---|---|---|---|---|
RF | 80% | 81.75% | 80.75% | 73.8% |
MLP | 82.25% | 80.25% | 79.25% | 76.5% |
XGB | 80.5% | 79.5% | 79.75% | 78.6% |
GBM | 81.9% | 85 % | 83% | 80.6% |
A person’s appearance is the most important factor in differentiating the constitution. The external appearance of the body shape is reflected by facial features and shape. To investigate the proper SCTs, we developed a real-time AI-based Sasang constitution prediction model. The practical use of facial characteristics in the Sasang constitution realtime system based on AI was tested several times by our laboratory members who noticed the response of real-time results. Fig. 9 shows the real-time AI model results of our lab members, demonstrating its strong performance.
We analyzed facial features from a different perspective to classify SCTs. This study aimed to develop an automatic computer-based face recognition Sasang constitution diagnostic model to help diagnose a person based on personality and traits. Although diagnosing the SCT remains challenging, this model allows stakeholders to identify the type accurately. Based on specific constitutional types, medical stakeholders can easily recommend medicines that align with the SCM concept. We used different machine learning approaches such as RF, MLP, XGBM, and GBM to identify SCTs. Moreover, among all the classifiers, the GBM yielded the best results. The output of our proposed model is a numerical value as a confusion matrix and a graphical representation as a ROC curve. Fig. 7 shows the actual and predicted values on the y- and x-axes, respectively. In the main diagonal, the actual and predicted values are the same, indicating the accuracy of the model. Table 4 shows the accuracy, F1-score, recall, and precision of the GBM model, which were 83, 83, 85, and 81.9%, respectively. As shown in Fig. 8, the ROC curves show graphical representations of each output classifier.
This study aimed to classify SCTs based on physiological and pathological features using transfer learning and facial landmarks, which will help to develop a computer-based SCT classification system. The face is the most prominent and representative part of an individual. This revealed a significant amount of information related to an individual. Age, sex, face shape, facial organ size, color, and personality traits were the key factors that helped identify the correct SCTs. SCTs help accurately predict or diagnose a patient, doctor, and images of Korean people, including males and females of different age groups. Furthermore, the data size is increased using traditional data augmentation techniques, and SCTs are important in this study. To extract features, we adopted feature extraction methods, and the feature engineering method obtained from both feature extraction techniques was combined to train the model. Additionally, a transfer-learningbased method was employed. The features resized all images to 256 × 256, according to the requirements, and features were extracted from the facial images for classification by a physicist. In Korea, SCM is popular and is extensively used in the medical domain. The proposed model is highly effective and robust for real-time applications.
We used a custom dataset containing 400 faces. Moreover, we analyzed various machine learning models for these feature values. We selected four machine learning models and performed a comparative analysis based on the results. The performance of the GBM was excellent among the classifiers, and the overall accuracy of this model was 80.6%. We computed the precision, recall, and F1-Score to analyze the model performance. Because of the deep analysis, we are confident that our proposed model is more efficient and robust than state-of-the-art machine-learning models. In the future, this work can be extended and its performance can be improved using additional data. Moreover, an advanced artificial neural network classifier can be used for better results.
This research was supported by the MSIT (Ministry of Science ICT), Korea, under the National Program for Excellence in SW, and supervised by the Institute of Information & Communications Technology Planning & Evaluation (IITP) in 2022.
The authors declare no conflicts of interest. The funders had no role in the study design; collection, analyses, or interpretation of data; writing of the manuscript; or decision to publish the results.
Table 1 . Facial attributes of Sasang Topology.
Attributes | Tae-Yang (TY) | So-Yang (SY) | So-Eum (SE) | Tae-Eum (TE) |
---|---|---|---|---|
Head | Large and angular | Oval shape | Inverted triangle | Round Rectangle |
Eyebrows | Thick and Thin | Dark | Light pale | Soft and thick |
Eye | Big, Sharp | Round, large | Weak and narrow | Small and thin |
Nose | Normal | Sharp | Normal | Blunt |
Lips | The lips line is black | Moderate | Thin and Pale | Thickness (steamed) |
Table 2 . Handcrafted extracted features are based on landmark key points.
Gender | Case | Face (W) | Face (H) | Forehead (W) | Forehead (H) | Eyebrows (W) | Eyebrows (H) | Eye (W) | Eye (H) | B/w Eyebrows (W) | Nose (H) | Nose (W) | Philtrum (H) |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Male | SY | 13.2 cm | 19.2 cm | 11.6 cm | 4.5 cm | 4.1 cm | 1.0 cm | 2.8 cm | 0.9 cm | 3.4 cm | 4.0 cm | 4.2 cm | 1.8 cm |
SE | 13.4 cm | 18.7 cm | 11.8 cm | 4.3 cm | 4.0 cm | 0.9 cm | 2.6 cm | 0.8 cm | 3.5 cm | 4.0 cm | 4.3 cm | 1.8 cm | |
TE | 14.7 cm | 20.3 cm | 12.6 cm | 4.6 cm | 4.7 cm | 1.0 cm | 2.8 cm | 0.8 cm | 3.6 cm | 4.2 cm | 4.3 cm | 2.0 cm | |
TY | 14.0 cm | 19.4 cm | 11.9 cm | 4.4 cm | 4.4 cm | 1.0 cm | 2.9 cm | 0.8 cm | 3.6 cm | 4.3 cm | 4.3 cm | 1.8 cm | |
Female | SY | 13.6 cm | 18.3 cm | 11.7 cm | 4.4 cm | 4.4 cm | 0.8 cm | 3.1 cm | 1.1 cm | 3.6 cm | 3.9 cm | 4.0 cm | 1.6 cm |
SE | 12.7 cm | 17.9 cm | 10.9 cm | 4.6 cm | 4.1 cm | 0.7 cm | 2.7 cm | 0.8 cm | 3.3 cm | 3.4 cm | 3.9 cm | 1.6 cm | |
TE | 14.3 cm | 18.6 cm | 11.4 cm | 4.2 cm | 4.5 cm | 0.8 cm | 2.9 cm | 0.8 cm | 3.4 cm | 4.0 cm | 4.0 cm | 1.6 cm | |
TY | 13.4 cm | 18.4 cm | 11.3 cm | 4.3 cm | 4.2 cm | 0.7 cm | 2.7 cm | 0.9 cm | 3.4 cm | 3.8 cm | 4.1 cm | 1.6 cm |
Table 3 . Average value of measured face organ by constitution type.
Feature ID | Description | Formula |
---|---|---|
f1 | Length to width ratio of face | |
f2 | Length to jawline ratio of face | |
f3 | Length to temple ratio of face | |
f4 | Length to width ratio of the left eye | |
f5 | Length to width ratio of the right eye | |
f6 | Nose height and width ratio | |
f1-f15 | The angle between right side landmark points | |
f16-f23 | The angle between left side landmark points | |
f24-f32 | Distance between the left side and right side of the landmark point. |
Table 4 . Performance of different machine learning models.
Classifier | Precision | Recall | F1-score | Accuracy |
---|---|---|---|---|
RF | 80% | 81.75% | 80.75% | 73.8% |
MLP | 82.25% | 80.25% | 79.25% | 76.5% |
XGB | 80.5% | 79.5% | 79.75% | 78.6% |
GBM | 81.9% | 85 % | 83% | 80.6% |
Byung-Hyun Lim, Ismatov Akobir, and Ki-Il Kim, Member, KIICE
Journal of information and communication convergence engineering 2024; 22(3): 199-206 https://doi.org/10.56977/jicce.2024.22.3.199Yongseok Yoo
Journal of information and communication convergence engineering 2023; 21(4): 281-286 https://doi.org/10.56977/jicce.2023.21.4.281Dmitriy Ogay, Eun-Gyung Kim*,
The Korea Institute of Information and Commucation Engineering 2014; 12(2): 116-121 https://doi.org/10.6109/jicce.2014.12.2.116