Research on Improving Fetal Health Prediction Model Using Optimal Fetal Feature Selection Technique

Ⅰ. Introduction

The society has witnessed a rising concern over fetal health due to the increasing rates of congenital anomalies which impact to the persisting issue of low birth rates. Congenital anomalies refer to structural or functional anomalies in fetuses which potentially impose challenges on individuals and families^[1,2]. The increase in the incidence of birth defects is associated with various factors, including inadequate fetal health management, genetic factors, and an aging population structure. According to the Korean Statistical Information Service (KOSTAT) for the year 2021, the birth rate has decreased by 10.3% comparing to 2020. This figure, recording a birth rate of 0.81 births per woman, is notably unprecedented since the KOSTAT has started compiling birth rate data in 1970^[3]. The KOSTAT has also reported that the estimated population decline has been advanced from 2032 to 2029^[4]. The use of fetal health big data analysis aims to ensure that healthy childbirth and alleviate any concerns, even in the case of elder women, by identifying the presence of potential risk factors throughout the pregnancy period and offering early prevention and treatments.

In this paper, we proposed a methodology to improve the fetal health prediction using sequential backward selection (SBS) algorithmto seek the optimal number of features. By using SBS, the algorithm iteratively removes one feature at a time to identify the most relevant features for fetal health prediction. For better classifying performance, K-fold validation was used after training the dataset with SBS algorithm to make up suboptimal condition to achieve the best possible health prediction outcomes. The proposed methodology has shown improvements in all areas of performance metrics including accuracy, compared to the latest research that selected features based on exploiting correlation coefficient-based feature reduction technique.

Ⅱ. References

2.1 Light Gradient Boosting Machine

Light Gradient Boosting Machine (LightGBM) is a variation of the gradient boosting method as well as a tree-based learning algorithm known for its distinctive features: Gradient-based One-Side Sampling (GOSS) and Exclusive Feature Bundling (EFB)^[5]. Specifically, LightGBM grows tree leaf-wise, where trees are split to maximize data loss instead of aiming for a balanced tree structure. As a result, the tree levels become deeper and asymmetrical, as depicted in figure 1, but it minimizes the prediction error loss more effectively than other boosting algorithms^[6]. LightGBM offers advantages in constructing fast and powerful prediction models for large-scale datasets. By leveraging the characteristics of gradient boosting algorithms, LightGBM aims to minimize errors and maximize prediction performance.

Fig. 1.

Leaf-wise tree growth.

2.2 Extra Tree Classifier

The Extra Trees Classifier is one of the ensemble techniques in a machine learning model that utilizes the Extremely Randomized Trees (ExtraTrees) algorithm to perform classification tasks. Extra Trees Classifier exhibits high performance on various datasets by randomly selecting features and performing random splits, enabling faster learning compared to Random Forests^[7]. Moreover, it possesses the property of preventing overfitting, allowing for good performance with relatively fewer hyperparameter tunings.

2.3 Sequential Backward Selection Algorithm

Sequential Backward Selection (SBS) is one of the feature selection algorithms, where the algorithm repeatedly trains the model by removing one feature at a time from the entire feature set after initially training the model with all features^[8]. In the SBS algorithm, each feature is removed one by one based on the performance difference in accuracy before and after the removal, aiming to obtain the model with the best performance at that point. However, the overall performance may vary depending on the order of feature removal which could lead to suboptimal result and less reliability. Thus, additional validation process is needed to achieve higher reliability and optimal performance.

2.4 Dataset

In this study, we used childbirth monitoring device data provided by the University of California Irvine (UCI) ^[9]. The dataset consists of 22 features extracted from Cardiotocograms (CTG) examinations, including fetal heart rate, fetal movement, uterine contractions, abnormal short-term variability, and the average value of short-term variability, as shown in table 1. The sample size in the dataset is 2126 and are classified into three categories: normal, suspicious, and pathological cases of fetuses (1, 2, 3) as shown in table 2. The categorical data in CTG dataset are in numerical values but data normalization is required for better predicting performance due to the ranges in data across features.

Ⅲ. Proposed Fetal Health Prediction Model

3.1 Data Imbalance

As shown in figure 2, the number of fetuses classified as normal is 1655, suspicious cases are 295, and pathological cases are 175 which shows a significant data imbalance in the dataset ^[10]. The data imbalance among different classes can lead to overfitting on the training dataset, resulting in high accuracy with only training sets but poor predictive performance on test sets. Thus, the aim is to fix the data imbalance problem among the datasets presented in table 2 to achieve more balanced and reliable classification results.

In this paper, random resampling technique was applied to improve the imbalanced distribution of datasets^[11]. Random resampling is a method that generates a new version of the training dataset with a different class distribution from the existing imbalanced classes to reduce the difference in sample numbers between majority and minority classes. Within random resampling, there are two types of technique to resolve data imbalance: under-sampling and over-sampling. As in figure 3, under-sampling involves removing samples from the majority class at a predefined interval to balance the class distribution and over-sampling increases the number of samples in the minority class by randomly duplicating data from the majority class at a predefined interval or by creating duplicate instances of the minority class data^[12].

Fig. 2.

Distribution of fetal health classification classes

Fig. 3.

Under-sampling and over-sampling.

In this paper, the distribution of the normal class was under-sampled by an amount equal to the sum of suspicious and pathological classes. Specifically, the samples from the normal fetus class were randomly selected and removed, resulting a total of 471 samples in normal fetus class. The suspicious and pathological fetus classes had fewer samples compared to the normal fetus class. Thus, over-sampling was performed to generate samples equal to the number of samples in the normal fetus class. As shown in table 3, all classes' datasets were balanced to have exactly 471 samples each to overcome the issue of overfitting that could arise due to the data imbalance.

3.2 Data Normalization

In this paper, after fixing the data imbalance, 80% of the dataset was divided to training dataset and the rest of 20% to test dataset. However, due to the large ranges of values in the samples across the features, the data was normalized using the robust scaler technique to prevent performance degradation ^[13]. The robust scaler uses the median and Interquartile Range (IQR) of the data to normalize each data point by subtracting the median from each data point and dividing it by the IQR as shown in equation (1). Additionally, robust scaler is less affected by outlier values compared to other normalization techniques, making it an optimal normalization method for datasets with many outliers. Thus, CTG dataset was scaled with robust scaler to preserve the overall shape and characteristics of the data which accurately reflected and preserved the distribution of the original dataset.

where x, x_med, and Qi stands for data point, median of the dataset, i-th quartile respectively.

3.3 Analysis of Sequential Backward Selection with K-fold validation

In this paper, we employed SBS feature selection algorithm with LightGBM model to select the necessary numbers of features for fetal health prediction. The SBS algorithm iteratively removes one feature at a time to obtain the optimal subset of features. The results from training the LightGBM model, as shown in figure 4, indicate that the accuracy of the model increases when the number of features is reduced from approximately 21 to 10. However, at 10 features, the accuracy decreases until 8 features and increases again to a level close to 10 features at 7 features. When reducing the number of features to 5 or below, there is a sharp average decline of about 2.5% in accuracy, leading to the exclusion of 5 or below features from the feature selection process.

Considering accuracy alone, the optimal number of features appear to be 10, 7, and 6. At 10 features, the accuracy is 98.59%, showing a marginal difference of 0.36% compared to 7 features. However, the processing time in the feature with 10 was 13.1801 seconds, which is approximately 2.57 times longer than 5.124 seconds in the 7 features. The model with 7 features is more computationally efficient with only at least accuracy loss . Thus, we can say the optimal number of features to 7 for computational time and concrete accuracy.

Fig. 4.

Prediction result of accuracy vs. number of features

In this paper, we used K-fold validation to overcome the issue of SBS algorithmnot selecting the overall optimal feature set based on the order of feature removal. K-fold validation divides the data into K subsets, or folds, and iteratively trains and evaluates the model multiple times to average the performance. K-fold validation was employed where the dataset was divided to 10 different folds, enabling the LightGBM model to be trained and evaluated 10 times. In each iteration, one fold was used as the validation set, while the remaining nine folds were used as the training set to train the model. The accuracy obtained from the 10 iterations was then averaged to calculate the final performance, as shown in table 5.

Ⅳ. Experimental Results

4.1 Proposed Framework of the Model

In this paper, the proposed fetal health prediction model’s framework is demonstrated as depicted in figure 5. From the fetal health dataset, the data were pre-processed through random sampling to fix data imbalance and normalized by robust scaling method. Then 80% of the data was divided to training and the rest of 20% to test set. From the training set, sequential backward selection feature algorithm was used to select the important number of features to ensure best accuracy for machine learning models. The K-Fold cross validation was used for training and testing the classifier performance. Overall, the machine learning model was tested through test set and evaluated the model’s performance through confusion matrix.

4.2 Confusion Matrix and Model Evaluation

The confusion matrix presents the comparison between the model's predicted results and the actual class labels, categorizing them into True Positive (TP), True Negative (TN), False Positive (FP), and False Negative (FN), as shown in figure 6. It is observed that the number of correctly classified samples for the normal, suspicious, and pathological classes is 89, 92, and 94, respectively, while a total of 8 samples were misclassified.

Fig. 5.

Proposed framework of the fetal health model

Using this confusion matrix, the model's performance can be evaluated and compared using metrics such as accuracy, precision, recall, and F1 Score. Among these classification evaluation metrics, accuracy is calculated by dividing the number of correctly classified samples by the total number of samples, as shown in equation (2). Recall, on the other hand, measures the proportion of correctly predicted positive observations among all positive observations in the actual class, as calculated in equation (3). Precision is the ratio of correctly predicted positive observations to the total predicted positive observations, as shown in equation (4). Lastly, the F1 Score is the weighted average of precision and recall, calculated according to equation (5).

Fig. 6.

Confusion matrix of LightGBM.

By utilizing these classification evaluation metrics, the model's performance can be evaluated and analyzed, providing valuable insights into its classification performance for each model.

4.3 Experimental Results

This paper compared and analyzed the results of fetal health prediction models using LightGBM with SBS proposed and without. At comparison evaluation in table 6, four classification metrics such as accuracy, precision, recall, and F1 Score, were used to evaluate the effectiveness of these models.

Based on the results in table 6, Proposed model can achieve better performance with an accuracy of 97.2%, recall of 0.972%, precision of 0.972% and F1 of 97.2% comparing to conventional model in predicting fetal health.

Furthermore, when comparing the results of this study with the research on the extra tree model ^[14], all four classification evaluation metrics have been improved as shown in table 7.

Ⅴ. Conclusions

In the paper, proposed framework of the fetal health prediction can improve computational time and classification accuracy by selecting optimal number of features in SBS algorithm with K-fold validation. In the proposal, the data obtained from Cardiotocograms (CTG) examinations was used. At first stage as data pre-processing, we applied random resampling to resolve the data imbalance problem in three classes. Specifically, Under and over-sampling for the normal fetus class and for the suspicious and pathological fetus classes can prevent overfitting and data loss problem caused by imbalanced data distribution. Secondly, the data was normalized using the robust scaler to ensure accurate results and to preserve the overall structure and characteristics of the data while performing scaling. Lastly, SBS algorithm and K-folder cross validation were proposed to enhance the accuracy and reliability by selecting the optimal number of features through removing one by one procedure of SBS method. It has been simulated with five different ML models, and the results indicated that the LightGBM model showed the best performance according to the confusion metrics and model evaluation.

This proposed research holds the potential to detect early signs of fetal health risks during pregnancy and provide timely information for necessary treatments, thereby reducing the occurrence of congenital abnormalities and fetal mortality rates.