Regular paper

Split Viewer

Journal of information and communication convergence engineering 2022; 20(4): 295-302

Published online December 31, 2022

https://doi.org/10.56977/jicce.2022.20.4.295

© Korea Institute of Information and Communication Engineering

LSTM Model-based Prediction of the Variations in Load Power Data from Industrial Manufacturing Machines

Rita Rijayanti 1, Kyohong Jin 2, and Mintae Hwang1* , Member, KIICE

1Department of Information and Communication Engineering, Changwon National University, Changwon 51140, South Korea
2Department of Electronic Engineering, Changwon National University, Changwon 51140, South Korea

Correspondence to : Mintae Hwang (E-mail: professorhwang@gmail.com, Tel: +82-55-213-3832)
Department of Information and Communication Engineering, Changwon National University, Changwon 51140, South Korea

Received: October 12, 2022; Revised: December 9, 2022; Accepted: December 11, 2022

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

This paper contains the development of a smart power device designed to collect load power data from industrial manufacturing machines, predict future variations in load power data, and detect abnormal data in advance by applying a machine learning-based prediction algorithm. The proposed load power data prediction model is implemented using a Long Short-Term Memory (LSTM) algorithm with high accuracy and relatively low complexity. The Flask and REST API are used to provide prediction results to users in a graphical interface. In addition, we present the results of experiments conducted to evaluate the performance of the proposed approach, which show that our model exhibited the highest accuracy compared with Multilayer Perceptron (MLP), Random Forest (RF), and Support Vector Machine (SVM) models. Moreover, we expect our method's accuracy could be improved by further optimizing the hyperparameter values and training the model for a longer period of time using a larger amount of data.

Keywords Load Power, Smart Power Device, Industrial Manufacturing Machine, LSTM Model, Abnormal Data Detection

Electrical power has become a basic requirement over the last few decades. Hence, instability in power system voltage constitutes a primary concern for power generation utilities, regardless of whether power is being used for household activities, education, office equipment, or production activities in manufacturing. In modern manufacturing, electricity supports efficient and effective production operations in which machines primarily function automatically using sensor technology and applications. Thus, electricity is a critical primary resource for a wide variety of general activities.

However, this naturally involves some unavoidable problems. The use of electrical power to operate various equipment inherently involves some load-balancing issues. Occasional instability in power generation or distribution systems is the main problem that arises owing to variations in load power, which increases excessive current and voltage. This leads to damage to machines and equipment and negatively impacting machine lifetimes and performance as well as production quality and labor productivity [1].

In particular, problems with load power tend to emerge in modern manufacturing environments. Assets and equipment must be appropriately monitored to solve these problems [2]. One key problem is that load power consumption requires multiple parts and simultaneous measurements [3].

Numerous devices have been developed to detect load power data to monitor the condition of machines in cases of sudden instability. Moreover, hardware designed to detect load power consumption has been extensively developed. However, no existing method has adequately resolved this issue.

Information about load power is required to identify power usage characteristics and to analyze and evaluate machine equipment, machine lifetime, manufacturing performance, production quality, and labor productivity in a manufacturing environment. Hence, in this study, we propose a model to predict changes in load power data to mitigate these issues by utilizing information on daily load power data consumption from industrial manufacturing machines. We also designed smart devices to measure and transmit load power data from an industrial machine in a manufacturing facility as a hardware solution.

The remainder of this study is organized as follows. Section II briefly reviews the relevant literature. Section III describes the proposed method to predict load power data. Section IV describes the implementation and results. Finally, Section V presents our conclusions.

Several studies have considered load power data, focusing on topics such as handling load power data due to unstable load power, inaccurate load power data readings, and the common problem of handling load power with big data [4]. Additionally, methods to predict the use of load power data for public interest have recently begun to attract attention [5]. Other studies have considered methods to deal with the predictive problem using the concept of Variational Mode Decomposition and dynamic adjustment Backpropagation (BP) to improve the accuracy of electricity consumption data owing to the redundant information and trend components contained in the original power load data [6]. Although, the authors identified a number of weaknesses, they noted that further study was required to improve accuracy. Other studies have reported on the characteristics of load power consumption data with respect to time, which typically involves several types of problems that are difficult to mitigate. Thus, an accurate prediction model is required to deal with electricity resources, particularly for business owners and the government [7-8].

In this study, we developed a model to predict changes in load power conditions in the manufacturing industry using a Long Short-Term Memory (LSTM) model to monitor the condition of load power data usage in industrial manufacturing machines. The proposed method is designed to monitor the conditions of connected machines with sufficient accuracy, contribute to maintaining machine lifetimes, and reduce maintenance costs, as well as to maximize production performance. Notably, this work is based on previous research related to the design of smart devices and applications for managing the load power of machines in industrial manufacturing facilities [9].

A. Smart Device

Two smart devices were developed to support this research, which were designed to detect machine load power data in an industrial manufacturing environment. The smart device consists of an Energy Metering Device (EMD) designed to collect load power from the machine and communicate with a Smart Power Device (SPD).

The SPD is designed to collect data from the EMD with additional information such as vibration and humidity. The results are displayed on a small screen attached to the SPD device to be stored in a database. Fig. 1 shows an implementation of the SPD device used to obtain the data. The SPD is intended to be installed with current and voltage metering devices, and can also function as an EMD when applied to some machines.

Fig. 1. Implemented Smart Devices.

B. Load Power Data

The load power data measures an electrical load in watts, a unit of power. The load power requires significant equipment to detect electricity at any given moment. In this study, the load power consumption data focused on the periodic usage patterns of industrial manufacturing machines over a given number of days and certain combinations of information that appeared periodically, such as frequency and humidity. Each machine displayed a unique pattern of information during the repetition period.

Fig. 2 illustrates the database design as a relational database structure consisting of three entities, including Device_ info, which contains information about the device, Model_ list, which explains all machine models, and Power_data, which contains information related to the status condition of the electrical machine. In this study, we used the Amazon Web Service (AWS) to host a server with a MariaDB database. In addition, the data shown here were obtained from the SPD.

Fig. 2. Relational Database Structure.

C. LSTM Model

In technical terms, the fundamental concept of the LSTM architecture is that of the Recurrent Neural Network (RNN). RNN models are considered a special case of neural networks and learn to predict the next step in a sequence based on the previously observed steps. RNN models use sequential observations and learn from earlier stages to forecast or predict future trends.

Data must be maintained during the initial stage while predicting the next sequence. In an RNN, the hidden layer acts as the internal storage to store the information collected during the initial stages of processing sequential data. RNN models are “recurrent” because they perform a similar task for each sequence element while utilizing information captured earlier to predict future values. One key challenge with RNN models is that they retain only a small number of initial steps within the data sequence; thus, they are not suitable for retaining longer sequences. This limitation has motivated the development of other types of recurrent network such as LSTM structures [10].

LSTM models were developed for improved performance in situations in which an RNN model may be unsuitable, such as missing gradients in backpropagation. In contrast, LSTM models can address this problem more effectively. In addition, both LSTM and RNN methods perform well with time-series data [11]. Each LSTM comprises a set of cells or modules in which data stream is captured and stored accordingly. The cells resemble a transport pathway connecting each module to convey past data and gather each module for the present prediction. The different gates in each cell enable data to be disposed, filtered, or added to the following cells. The gates in LSTM cells are divided into three types, including forget gates, memory gates, and output gates. These gates are considered filters and function as part of the neural network. The gates enable the cells to allow data to pass through or discharge the value. However, this disposal is optional.

A proposed predictive model was created using the LSTM algorithm, which uses a sequence of past data to predict electricity consumption. To evaluate the performance of the proposed approach, we conducted an experiment with an example dataset, and the experimental result demonstrates that the proposed method performed well as a solution for time-series data. The architecture of the proposed LSTM network is illustrated in Fig. 3.

Fig. 3. Architecture of LSTM model.

To construct this predictive model, we first collected the data, then reformatted the date-time columns, and checked unique data from datasets. Subsequently, we considered the energy distribution and energy with respect to time, and then resampled the data and constructed the model. Then, we tested the data and finally generated the output prediction. During the training process, embedding vectors that represent the characteristics and relationships of the profiles were learned according to the voltages that appear in the sequence.

The LSTM learns to recognize the patterns that appear in the energy sequence, and thus to predict future consumption based on a learned embedding vector. We collected data generated from a load power sequence over several weeks, along with two additional input vectors consisting of the current and the temperature for each day. With three LSTM layers and a SoftMax layer, the network predicts an expected future load profile sequence for the next period.

We began by preparing the training data. The data were measured and collected from the smart devices, and some data were missing owing to system and network connection failures. However, estimating the missing values is beyond the scope of this work; therefore, we did not attempt to do so. The Flask framework and the REST API were used to develop a user in terface to display the prediction results. We adopted Flask because it can be implemented using the Python programming language to facilitate integration with existing predictive models. In addition, Flask is capable of handling issues that occur from the server-side, handling RESTful requests, and securing cookies on the client-side.

A. Experimental Setup

Table 1 illustrates the experimental environment and settings implemented to develop a predictive model of load power data consumption in an industrial manufacturing environment based on the architecture of the proposed model, as described in Fig. 3.

Table 1 . Experimental setup

NoDescriptionValues
1EnvironmentsTensorflow 1.8.0
Phyton 3.5
2OptimizerGradient Descent
3Lost functionseq2seq loss
4Dropoutprobability: 0.2
5BatchBatch size: 32
6RegularizationL2 regularizer
7Gradient ClippingO
8Learning Rate10-2
9SoftmaxO


The dataset used to train the model comprised real data from tests performed in an enterprise environment using an SPD installed with current and voltage meters on the machine. After the results were recorded, the current and voltage were sent over a network to be stored in a database.

To train the model, we utilized periodic power patterns of industrial machines for load power data consumption. Some daily consumption patterns repeatedly appeared in the load power data, and several specific combinations of information occurred periodically. Although each machine exhibited unique information patterns and repeated periods, the consumption load data were transformed into a predefined sequence.

Fig. 4 illustrates the output of the load power data received from the SPD to the MariaDB database. These data comprise the results of tests performed by the company in the form of detailed information.

Fig. 4. Sample of load power data from SPD.

Creating a cleaned master dataset was a crucial step in constructing the proposed model. Specialized cleaning data were used to check unique data from the datasets. The results of data collected directly from the machines in the database cannot be directly used to create machine learning models because these raw data are likely to include many null or zero values, and we observed a similar trend in the data collected in this work. The raw data needs to be initially cleaned by deleting or modifying a number of data columns, including deleting duplicate values and outliers, which is commonly referred to as data normalization. After cleaning the master data, the cleaned data are used for a lower processing time and a more precise projection and to simplify the functionality of the algorithm, improve performance, perform energy distribution, and consider energy with respect to time to predict the result.

An explanation of the detailed range, normal range, and units of each data sample that may appear is provided in Table 2.

Table 2 . Detailed information of load power data

MeasurementRangeNormal RangeUnit
id0~65535--
device_id0~65535-
r_voltage0~655.35 V200V ~ 240 V0.01 V
s_voltage0~655.35 V200V ~ 240 V0.01 V
t_voltage0~65.535 A1A ~ 5 A0.001 A
r_current,0~65.535 A1A ~ 5 A0.001 A
s_current,0~65.535 A1A ~ 1 A0.001 A
t_current,0~65.535 A1A ~ 1 A0.001 A
r_active_power-32.768~32.767 kW0.22~1.100.001 kW
s_active_power-32.768~32.767 kW0.22~1.100.001 kW
t_active_power-32.768~32.767 kVAR-0.010~0.0100.001 kVAR
r_reactive_power-32.768~32.767 kVAR-0.010~0.0100.001 kVAR
s_reactive_power-32.768~32.767 kVAR-0.010~0.0100.001 kVAR
s_reactive_power-32.768~32.767 kVAR-0.010~0.0100.001 kVAR
r_apparent_power-32.768~32.767 kVA0.22~1.100.001 kVA
s_apparent_power-32.768~32.767 kVA0.22~1.100.001 kVA
t_apparent_power-32.768~32.767 kVA0.22~1.100.001 kVA
r_power_factor-1.000~+1.0000.980~1.0000.001
s_power_factor-1.000~+1.0000.980~1.0000.001
t_power_factor-1.000~+1.0000.980~1.0000.001
frequency45.00~65.00 Hz58~62 Hz0.01 Hz
temp-128~127 °C10~40 °C1 °C
datetime---


B. Experimental Results

The following section explains the results of the predictive model. The proposed LSTM network generates load power sequence data with five features, including date and time, voltage, current, temperature, and frequency, each day. These five features representatively generate information related to the load power data.

This model implements an average value for the movement of the load power data in the form of a centralized time series measured based on a real dataset for each second. To describe load power consumption in a sequential manner, we adopted an LSTM architecture as an effective and reliable approach for sequential time-series signal decomposition.

In this study, the prediction model applied seventy-five percent (75%) of the pre-processed form parameter dataset as the training dataset. In addition, twenty-five percent (25%) of the total data were used for the test dataset. The results were subsequently compared with the original load power data.

Based on the collected load power data, the model was applied to predict future variations in the load power parameters, while the prediction technology was developed to provide the information in the form of a graph on a web page. The proposed machine-learning-based load power data prediction process is divided into seven steps.

1) Step 1 - Data Initialization

Initialization data are collected by locating initial values for variable data that are implemented in the model as an initial state construction. Raw data collected from SPD and EMD in the previous process were used in this step.

2) Step 2 - Reformatting the date and time columns

The date and time of the data must be reformatted as required by the software required format (month, year, date, time, and week) to verify the distribution of the number of years or months and date.

3) Step 3 - Checking unique data from datasets

In this step, unique data are checked as to whether the data are within an allowed tolerance or include in an abnormal condition. Data considered as indicating an abnormal condition cannot be used for training. Therefore, data cleaning is required to solve this problem. This is also useful to verify the condition of the data distribution in terms of the time required to reproduce the correct prediction result.

4) Step 4 - Resampling data

Resampling methods use a data sample to improve accuracy and quantify the uncertainty of a population parameter. In this case, the sample comprised cleaned load power data obtained from the manufacturing facility.

5) Step 5 - Create the model

This stage presents the process of creating an LSTM model adjusted to the environment described in Table 1 and Fig. 3.

6) Step 6 - Test data

This stage comprises a process used to test the model using a prepared dataset.

7) Step 7 - Prediction Output

In this stage, the prediction results are displayed in the form of a graph representing the actual conditions and the prediction results.

To evaluate the proposed method, we tested MLP, RF, and SVM models on the same dataset with the environment adapted to the proposed model for comparison. The MLP model included two hidden layers. In addition, the number of units, dropout, and activation in each layer was determined carefully. SVM models include three import hyperparameters: kernel type, C, and γ. The kernel type determines how the original data are mapped to a high-dimensional space belonging to the Radial Basis Function, which is a frequently used kernel function in SVM. RF classifiers are ensemble models of several decision trees, where the n_estimators parameter determines the number of decision trees. The maximum features and depth of each decision tree are restricted.

To compare the proposed model with MLP, RF, and SVM more objectively, the parameters of these models were optimized using the respective hyperparameters. We used a negative cross-validation score as an objective function for the SVM model, whereas the same objective function as in the LSTM model was implemented for the MLP.

Fig. 5 illustrates the intermediate results generated through the seven-step prediction process. The results were grouped into states of usage energy and energy distribution, where the results were consistent because the data cleaning had been performed before processing these steps. As previously described, the data cleansing was performed because the estimation of missing values was considered beyond the scope of this the present work.

Fig. 5. Load power data from SPD.

Fig. 6 illustrates the final result of predicting load power consumption from step seven of the prediction process for the industrial manufacturing machine environment, where the accuracy varied with the number of epoch iterations.

Fig. 6. Output Load Power Prediction Graph (LSTM, MPL, RF, and SVM).

The performance of the proposed model and that of the MLP, RF, and SVM models were evaluated quantitatively in terms of the Mean Absolute Percentage Error (MAPE). MAPE is a comparative indicator that eliminates the impact of size. MAPE is defined by the following equation, where n is the number of observations in the test set, yt is the actual value, and y’t is the predicted value. [12]

MAPE=1n t=1 n yty 't yt*100%.

Fig. 7 shows that the proposed LSTM model achieved the best performance. The SVM was nearly as accurate as the proposed model. However, its prediction results were slightly lower than those of the proposed model. The MLP models encountered roughly the same limitations as the SVM; nonetheless, MLP performed slightly worse than SVM. Moreover, the prediction of RF fluctuated considerably. The accuracy of these methods can be improved by extending the training time, because accuracy is roughly proportional to the number of iterations, up to a point of diminishing returns.

Fig. 7. MAPE of the four models in eight months.

In this study, we developed a method to obtain information related to changes in load power data and predictions based on an LSTM network in an industrial manufacturing environment. Smart devices including the SPD and EMD were developed and installed in manufacturing machines to detect the load power consumption of each machine. The data obtained from these smart devices were sent to MariaDB for storage.

A sequence of machine load power data, including information such as voltage, current, temperature, and frequency information, was implemented as embedding vectors representing the characteristics and profile relationships of the electrical load. Using the analyzed embedding vector sequence, the LSTM network predicts the obtained data pattern as sequence data. The experimental results demonstrate that the proposed model exhibited the best prediction results with the highest accuracy using the collected dataset compared with MLP, RF, and SVM models. Furthermore, these results are expected to support manufacturing operations and management in monitoring the daily load power consumption of industrial machines in manufacturing facilities to reduce significant equipment damage due to unstable load power issue conditions by considering various appropriate preventative actions. Furthermore, we developed a user interface to display the prediction results using the Flask framework and the REST API.

In future research, other machine learning algorithms must be considered to determine which method may exhibit the best performance in detecting load power data consumption in an industrial manufacturing environment, considering that excluding missing or unique values from the dataset may limit network performance owing to the time intervals between data points in the sequence.

Therefore, the methods must be developed to estimate or deal with missing values, as well as inconsistent time intervals; thus, consistency must be further considered based on time-series data.

This work was supported by Gyeongnam SW Convergence Cluster 2.0 under the contract and the "Leaders in INdustry-university Cooperation +" Project by the Ministry of Education and National Research Foundation of Korea.

  1. D. Mao and K. Potty and J. Wang, The impact of power-electronicsbased load dynamics on large-disturbance voltage stability, in Proceeding of IEEE Power & Energy Society General Meeting (PESGM), Portland: OR, USA, pp. 36-39, 2018. DOI: 10.1109/PESGM.2018.8586221.
    CrossRef
  2. Y. Li and D. Han and Z. Yan, Long-term load forecasting based on data-driven linear clustering method, Journal of Power System and Automation, vol. 6, no. 2, pp. 306-316, Mar., 2018. DOI: 10.1007/s40565-017-0288-x.
    CrossRef
  3. J. Hao, and Y. Liu, and H. Gu, and D. Yang, and R. Wang, and J. Lei, and M. Deng, and Z. Huang, Short-term power load forecasting for larger consumer based on tensorflow deep learning framework and clusteringregression model, in Proceeding 2018 2nd IEEE Conference on Energy Internet and Energy System Integration (EI2), Beijing, China, pp. 1-6, 2018. DOI: 10.1109/EI2.2018.8582583.
    CrossRef
  4. Z. Wang and J. Qin and C. Cao, Research on prediction of dynamic load complementary based on big data, in Proceedings of 2019 International Conference on Computer Network, Electronic and Automation (ICCNEA), Xian, China, pp. 369-373, 2019. DOI: 10.1109/ICCNEA.2019.00074.
    CrossRef
  5. K. Zhou and Y. Zhou and K. Deng, A circular recursion algorithm power load forecasting model and application, in Proceedings of 2019 IEEE 3rd Conference on Energy Internet and Energy System Integration (EI2), Changsha, China, pp. 2870-2873, 2019. DOI: 10.1109/EI247390.2019.9061824.
    Pubmed KoreaMed CrossRef
  6. F. Kuang, and D. Huang, Power load prediction method based on VMD and dynamic adjustment BP, in Proceedings of 2019 CAA Symposium on Fault Detection, Supervision and Safety for Technical Processes (SAFEPROCESS), Xiamen, China, pp. 160-164, 2019. DOI: 10.1109/SAFEPROCESS45799.2019.9213431.
    Pubmed CrossRef
  7. A. Boltunov, and S. Vasiliev, and V. Karpenko, and A. Voloshin, and E. Voloshin, Short-term load forecasting system for smartgrids based on personal power units, in Proceedings of 2019 Modern Electric Power Systems (MEPS), Wroclaw, Poland, pp. 1-3, 2019. DOI: 10.1109/MEPS46793.2019.9394995.
    CrossRef
  8. T. Gong, and S. Tian, Analysis and forecasting for power load of office building taking crowd behavior into account, in Proceeding of 2019 IEEE 8th Joint International Information Technology and Artificial Intelligence Conference (ITAIC)), Chongqing, China, pp. 997-1000, 2019. DOI: 10.1109/ITAIC.2019.8785742.
    KoreaMed CrossRef
  9. R. Rijayanti, and A. B. Wahyutama, and B. Lee, and J. Kim, and B. Yoo, and M. Hwang, Design of smart devices and applications for managing the load power of industrial manufacturing machine, in Proceedings of The 17th International Conference on Multimedia Information Technology and Applications (MITA2021), Jeju, Korea, pp. 47-50, A2021.
  10. J. Schmidhuber. “Habilitation”, Postdoctoral Thesis, The Technical University of Munich, Munich, 1993, [online] Available: https://people.idsia.ch/~juergen/onlinepub.html
  11. S. A. Ludwig, Comparison of time series approaches applied to greenhouse gas analysis: ANFIS, RNN, and LSTM, in Proceeding of IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), New Orleans: NA, USA, pp. 1-6, 2019. DOI: 10.1109/FUZZIEEE.
    CrossRef
  12. C. Deb, and F. Zhang, and J. Yang, and S. E. Lee, and K. W. Shah, A review on time series forecasting techniques for building energy consumption, Renewable and Sustainable Energy Reviews, vol. 74, pp. 902-924, Jul., 2017. DOI: 10.1016/j.rser.2017.
    CrossRef

Rita Rijayanti

received her B.S. degree from the Department of Informatics Engineering, Pasundan University, Bandung, Indonesia in 2007 and M.S. degree from the Department of Informatics Engineering, Langlangbuana University, Bandung Indonesia in 2015. She is currently pursuing the Ph.D. degree from the Department of Information and Communication Engineering at Changwon National University, Changwon, Korea. Her current research interests include IT security, web-based applications, machine learning, and smart manufacturing.


Kyohong Jin

received his B.S., M.S., and Ph.D. degrees from the Department of Computer Engineering, Pusan National University, Pusan, Korea in 1991, 1993, and 1997, respectively. He was a senior research member at ADD (Agency for Defense Development) from 1997 to 2000. He has been with the Department of Electronics Engineering, Changwon National University, Changwon, Korea, since 2004, where he is currently a Professor. His current research interests include communication protocols, Internet of things, machine learning, and smart factories


Mintae Hwang

received his B.S., M.S., and Ph.D. degrees from the Department of Computer Engineering, Pusan National University, Pusan, Korea in 1990, 1992, and 1996, respectively. He was a senior research member of the PEC (Protocol Engineering Center) at ETRI (Electronics and Telecommunications Research Institute) from 1996 to 1998. He has been with the Department of Information and Communication Engineering, Changwon National University, Changwon, Korea, since 1999, where he is currently a Professor. His current research interests include communication protocols, Internet of things, machine learning, web/mobile applications, and smart cities.


Article

Regular paper

Journal of information and communication convergence engineering 2022; 20(4): 295-302

Published online December 31, 2022 https://doi.org/10.56977/jicce.2022.20.4.295

Copyright © Korea Institute of Information and Communication Engineering.

LSTM Model-based Prediction of the Variations in Load Power Data from Industrial Manufacturing Machines

Rita Rijayanti 1, Kyohong Jin 2, and Mintae Hwang1* , Member, KIICE

1Department of Information and Communication Engineering, Changwon National University, Changwon 51140, South Korea
2Department of Electronic Engineering, Changwon National University, Changwon 51140, South Korea

Correspondence to:Mintae Hwang (E-mail: professorhwang@gmail.com, Tel: +82-55-213-3832)
Department of Information and Communication Engineering, Changwon National University, Changwon 51140, South Korea

Received: October 12, 2022; Revised: December 9, 2022; Accepted: December 11, 2022

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

This paper contains the development of a smart power device designed to collect load power data from industrial manufacturing machines, predict future variations in load power data, and detect abnormal data in advance by applying a machine learning-based prediction algorithm. The proposed load power data prediction model is implemented using a Long Short-Term Memory (LSTM) algorithm with high accuracy and relatively low complexity. The Flask and REST API are used to provide prediction results to users in a graphical interface. In addition, we present the results of experiments conducted to evaluate the performance of the proposed approach, which show that our model exhibited the highest accuracy compared with Multilayer Perceptron (MLP), Random Forest (RF), and Support Vector Machine (SVM) models. Moreover, we expect our method's accuracy could be improved by further optimizing the hyperparameter values and training the model for a longer period of time using a larger amount of data.

Keywords: Load Power, Smart Power Device, Industrial Manufacturing Machine, LSTM Model, Abnormal Data Detection

I. INTRODUCTION

Electrical power has become a basic requirement over the last few decades. Hence, instability in power system voltage constitutes a primary concern for power generation utilities, regardless of whether power is being used for household activities, education, office equipment, or production activities in manufacturing. In modern manufacturing, electricity supports efficient and effective production operations in which machines primarily function automatically using sensor technology and applications. Thus, electricity is a critical primary resource for a wide variety of general activities.

However, this naturally involves some unavoidable problems. The use of electrical power to operate various equipment inherently involves some load-balancing issues. Occasional instability in power generation or distribution systems is the main problem that arises owing to variations in load power, which increases excessive current and voltage. This leads to damage to machines and equipment and negatively impacting machine lifetimes and performance as well as production quality and labor productivity [1].

In particular, problems with load power tend to emerge in modern manufacturing environments. Assets and equipment must be appropriately monitored to solve these problems [2]. One key problem is that load power consumption requires multiple parts and simultaneous measurements [3].

Numerous devices have been developed to detect load power data to monitor the condition of machines in cases of sudden instability. Moreover, hardware designed to detect load power consumption has been extensively developed. However, no existing method has adequately resolved this issue.

Information about load power is required to identify power usage characteristics and to analyze and evaluate machine equipment, machine lifetime, manufacturing performance, production quality, and labor productivity in a manufacturing environment. Hence, in this study, we propose a model to predict changes in load power data to mitigate these issues by utilizing information on daily load power data consumption from industrial manufacturing machines. We also designed smart devices to measure and transmit load power data from an industrial machine in a manufacturing facility as a hardware solution.

The remainder of this study is organized as follows. Section II briefly reviews the relevant literature. Section III describes the proposed method to predict load power data. Section IV describes the implementation and results. Finally, Section V presents our conclusions.

II. RELATED WORKS

Several studies have considered load power data, focusing on topics such as handling load power data due to unstable load power, inaccurate load power data readings, and the common problem of handling load power with big data [4]. Additionally, methods to predict the use of load power data for public interest have recently begun to attract attention [5]. Other studies have considered methods to deal with the predictive problem using the concept of Variational Mode Decomposition and dynamic adjustment Backpropagation (BP) to improve the accuracy of electricity consumption data owing to the redundant information and trend components contained in the original power load data [6]. Although, the authors identified a number of weaknesses, they noted that further study was required to improve accuracy. Other studies have reported on the characteristics of load power consumption data with respect to time, which typically involves several types of problems that are difficult to mitigate. Thus, an accurate prediction model is required to deal with electricity resources, particularly for business owners and the government [7-8].

In this study, we developed a model to predict changes in load power conditions in the manufacturing industry using a Long Short-Term Memory (LSTM) model to monitor the condition of load power data usage in industrial manufacturing machines. The proposed method is designed to monitor the conditions of connected machines with sufficient accuracy, contribute to maintaining machine lifetimes, and reduce maintenance costs, as well as to maximize production performance. Notably, this work is based on previous research related to the design of smart devices and applications for managing the load power of machines in industrial manufacturing facilities [9].

III. PROPOSED MODEL

A. Smart Device

Two smart devices were developed to support this research, which were designed to detect machine load power data in an industrial manufacturing environment. The smart device consists of an Energy Metering Device (EMD) designed to collect load power from the machine and communicate with a Smart Power Device (SPD).

The SPD is designed to collect data from the EMD with additional information such as vibration and humidity. The results are displayed on a small screen attached to the SPD device to be stored in a database. Fig. 1 shows an implementation of the SPD device used to obtain the data. The SPD is intended to be installed with current and voltage metering devices, and can also function as an EMD when applied to some machines.

Figure 1. Implemented Smart Devices.

B. Load Power Data

The load power data measures an electrical load in watts, a unit of power. The load power requires significant equipment to detect electricity at any given moment. In this study, the load power consumption data focused on the periodic usage patterns of industrial manufacturing machines over a given number of days and certain combinations of information that appeared periodically, such as frequency and humidity. Each machine displayed a unique pattern of information during the repetition period.

Fig. 2 illustrates the database design as a relational database structure consisting of three entities, including Device_ info, which contains information about the device, Model_ list, which explains all machine models, and Power_data, which contains information related to the status condition of the electrical machine. In this study, we used the Amazon Web Service (AWS) to host a server with a MariaDB database. In addition, the data shown here were obtained from the SPD.

Figure 2. Relational Database Structure.

C. LSTM Model

In technical terms, the fundamental concept of the LSTM architecture is that of the Recurrent Neural Network (RNN). RNN models are considered a special case of neural networks and learn to predict the next step in a sequence based on the previously observed steps. RNN models use sequential observations and learn from earlier stages to forecast or predict future trends.

Data must be maintained during the initial stage while predicting the next sequence. In an RNN, the hidden layer acts as the internal storage to store the information collected during the initial stages of processing sequential data. RNN models are “recurrent” because they perform a similar task for each sequence element while utilizing information captured earlier to predict future values. One key challenge with RNN models is that they retain only a small number of initial steps within the data sequence; thus, they are not suitable for retaining longer sequences. This limitation has motivated the development of other types of recurrent network such as LSTM structures [10].

LSTM models were developed for improved performance in situations in which an RNN model may be unsuitable, such as missing gradients in backpropagation. In contrast, LSTM models can address this problem more effectively. In addition, both LSTM and RNN methods perform well with time-series data [11]. Each LSTM comprises a set of cells or modules in which data stream is captured and stored accordingly. The cells resemble a transport pathway connecting each module to convey past data and gather each module for the present prediction. The different gates in each cell enable data to be disposed, filtered, or added to the following cells. The gates in LSTM cells are divided into three types, including forget gates, memory gates, and output gates. These gates are considered filters and function as part of the neural network. The gates enable the cells to allow data to pass through or discharge the value. However, this disposal is optional.

A proposed predictive model was created using the LSTM algorithm, which uses a sequence of past data to predict electricity consumption. To evaluate the performance of the proposed approach, we conducted an experiment with an example dataset, and the experimental result demonstrates that the proposed method performed well as a solution for time-series data. The architecture of the proposed LSTM network is illustrated in Fig. 3.

Figure 3. Architecture of LSTM model.

To construct this predictive model, we first collected the data, then reformatted the date-time columns, and checked unique data from datasets. Subsequently, we considered the energy distribution and energy with respect to time, and then resampled the data and constructed the model. Then, we tested the data and finally generated the output prediction. During the training process, embedding vectors that represent the characteristics and relationships of the profiles were learned according to the voltages that appear in the sequence.

The LSTM learns to recognize the patterns that appear in the energy sequence, and thus to predict future consumption based on a learned embedding vector. We collected data generated from a load power sequence over several weeks, along with two additional input vectors consisting of the current and the temperature for each day. With three LSTM layers and a SoftMax layer, the network predicts an expected future load profile sequence for the next period.

IV. IMPLEMENTATION AND RESULTS

We began by preparing the training data. The data were measured and collected from the smart devices, and some data were missing owing to system and network connection failures. However, estimating the missing values is beyond the scope of this work; therefore, we did not attempt to do so. The Flask framework and the REST API were used to develop a user in terface to display the prediction results. We adopted Flask because it can be implemented using the Python programming language to facilitate integration with existing predictive models. In addition, Flask is capable of handling issues that occur from the server-side, handling RESTful requests, and securing cookies on the client-side.

A. Experimental Setup

Table 1 illustrates the experimental environment and settings implemented to develop a predictive model of load power data consumption in an industrial manufacturing environment based on the architecture of the proposed model, as described in Fig. 3.

Table 1 . Experimental setup.

NoDescriptionValues
1EnvironmentsTensorflow 1.8.0
Phyton 3.5
2OptimizerGradient Descent
3Lost functionseq2seq loss
4Dropoutprobability: 0.2
5BatchBatch size: 32
6RegularizationL2 regularizer
7Gradient ClippingO
8Learning Rate10-2
9SoftmaxO


The dataset used to train the model comprised real data from tests performed in an enterprise environment using an SPD installed with current and voltage meters on the machine. After the results were recorded, the current and voltage were sent over a network to be stored in a database.

To train the model, we utilized periodic power patterns of industrial machines for load power data consumption. Some daily consumption patterns repeatedly appeared in the load power data, and several specific combinations of information occurred periodically. Although each machine exhibited unique information patterns and repeated periods, the consumption load data were transformed into a predefined sequence.

Fig. 4 illustrates the output of the load power data received from the SPD to the MariaDB database. These data comprise the results of tests performed by the company in the form of detailed information.

Figure 4. Sample of load power data from SPD.

Creating a cleaned master dataset was a crucial step in constructing the proposed model. Specialized cleaning data were used to check unique data from the datasets. The results of data collected directly from the machines in the database cannot be directly used to create machine learning models because these raw data are likely to include many null or zero values, and we observed a similar trend in the data collected in this work. The raw data needs to be initially cleaned by deleting or modifying a number of data columns, including deleting duplicate values and outliers, which is commonly referred to as data normalization. After cleaning the master data, the cleaned data are used for a lower processing time and a more precise projection and to simplify the functionality of the algorithm, improve performance, perform energy distribution, and consider energy with respect to time to predict the result.

An explanation of the detailed range, normal range, and units of each data sample that may appear is provided in Table 2.

Table 2 . Detailed information of load power data.

MeasurementRangeNormal RangeUnit
id0~65535--
device_id0~65535-
r_voltage0~655.35 V200V ~ 240 V0.01 V
s_voltage0~655.35 V200V ~ 240 V0.01 V
t_voltage0~65.535 A1A ~ 5 A0.001 A
r_current,0~65.535 A1A ~ 5 A0.001 A
s_current,0~65.535 A1A ~ 1 A0.001 A
t_current,0~65.535 A1A ~ 1 A0.001 A
r_active_power-32.768~32.767 kW0.22~1.100.001 kW
s_active_power-32.768~32.767 kW0.22~1.100.001 kW
t_active_power-32.768~32.767 kVAR-0.010~0.0100.001 kVAR
r_reactive_power-32.768~32.767 kVAR-0.010~0.0100.001 kVAR
s_reactive_power-32.768~32.767 kVAR-0.010~0.0100.001 kVAR
s_reactive_power-32.768~32.767 kVAR-0.010~0.0100.001 kVAR
r_apparent_power-32.768~32.767 kVA0.22~1.100.001 kVA
s_apparent_power-32.768~32.767 kVA0.22~1.100.001 kVA
t_apparent_power-32.768~32.767 kVA0.22~1.100.001 kVA
r_power_factor-1.000~+1.0000.980~1.0000.001
s_power_factor-1.000~+1.0000.980~1.0000.001
t_power_factor-1.000~+1.0000.980~1.0000.001
frequency45.00~65.00 Hz58~62 Hz0.01 Hz
temp-128~127 °C10~40 °C1 °C
datetime---


B. Experimental Results

The following section explains the results of the predictive model. The proposed LSTM network generates load power sequence data with five features, including date and time, voltage, current, temperature, and frequency, each day. These five features representatively generate information related to the load power data.

This model implements an average value for the movement of the load power data in the form of a centralized time series measured based on a real dataset for each second. To describe load power consumption in a sequential manner, we adopted an LSTM architecture as an effective and reliable approach for sequential time-series signal decomposition.

In this study, the prediction model applied seventy-five percent (75%) of the pre-processed form parameter dataset as the training dataset. In addition, twenty-five percent (25%) of the total data were used for the test dataset. The results were subsequently compared with the original load power data.

Based on the collected load power data, the model was applied to predict future variations in the load power parameters, while the prediction technology was developed to provide the information in the form of a graph on a web page. The proposed machine-learning-based load power data prediction process is divided into seven steps.

1) Step 1 - Data Initialization

Initialization data are collected by locating initial values for variable data that are implemented in the model as an initial state construction. Raw data collected from SPD and EMD in the previous process were used in this step.

2) Step 2 - Reformatting the date and time columns

The date and time of the data must be reformatted as required by the software required format (month, year, date, time, and week) to verify the distribution of the number of years or months and date.

3) Step 3 - Checking unique data from datasets

In this step, unique data are checked as to whether the data are within an allowed tolerance or include in an abnormal condition. Data considered as indicating an abnormal condition cannot be used for training. Therefore, data cleaning is required to solve this problem. This is also useful to verify the condition of the data distribution in terms of the time required to reproduce the correct prediction result.

4) Step 4 - Resampling data

Resampling methods use a data sample to improve accuracy and quantify the uncertainty of a population parameter. In this case, the sample comprised cleaned load power data obtained from the manufacturing facility.

5) Step 5 - Create the model

This stage presents the process of creating an LSTM model adjusted to the environment described in Table 1 and Fig. 3.

6) Step 6 - Test data

This stage comprises a process used to test the model using a prepared dataset.

7) Step 7 - Prediction Output

In this stage, the prediction results are displayed in the form of a graph representing the actual conditions and the prediction results.

To evaluate the proposed method, we tested MLP, RF, and SVM models on the same dataset with the environment adapted to the proposed model for comparison. The MLP model included two hidden layers. In addition, the number of units, dropout, and activation in each layer was determined carefully. SVM models include three import hyperparameters: kernel type, C, and γ. The kernel type determines how the original data are mapped to a high-dimensional space belonging to the Radial Basis Function, which is a frequently used kernel function in SVM. RF classifiers are ensemble models of several decision trees, where the n_estimators parameter determines the number of decision trees. The maximum features and depth of each decision tree are restricted.

To compare the proposed model with MLP, RF, and SVM more objectively, the parameters of these models were optimized using the respective hyperparameters. We used a negative cross-validation score as an objective function for the SVM model, whereas the same objective function as in the LSTM model was implemented for the MLP.

Fig. 5 illustrates the intermediate results generated through the seven-step prediction process. The results were grouped into states of usage energy and energy distribution, where the results were consistent because the data cleaning had been performed before processing these steps. As previously described, the data cleansing was performed because the estimation of missing values was considered beyond the scope of this the present work.

Figure 5. Load power data from SPD.

Fig. 6 illustrates the final result of predicting load power consumption from step seven of the prediction process for the industrial manufacturing machine environment, where the accuracy varied with the number of epoch iterations.

Figure 6. Output Load Power Prediction Graph (LSTM, MPL, RF, and SVM).

The performance of the proposed model and that of the MLP, RF, and SVM models were evaluated quantitatively in terms of the Mean Absolute Percentage Error (MAPE). MAPE is a comparative indicator that eliminates the impact of size. MAPE is defined by the following equation, where n is the number of observations in the test set, yt is the actual value, and y’t is the predicted value. [12]

MAPE=1n t=1 n yty 't yt*100%.

Fig. 7 shows that the proposed LSTM model achieved the best performance. The SVM was nearly as accurate as the proposed model. However, its prediction results were slightly lower than those of the proposed model. The MLP models encountered roughly the same limitations as the SVM; nonetheless, MLP performed slightly worse than SVM. Moreover, the prediction of RF fluctuated considerably. The accuracy of these methods can be improved by extending the training time, because accuracy is roughly proportional to the number of iterations, up to a point of diminishing returns.

Figure 7. MAPE of the four models in eight months.

V. CONCLUSIONS

In this study, we developed a method to obtain information related to changes in load power data and predictions based on an LSTM network in an industrial manufacturing environment. Smart devices including the SPD and EMD were developed and installed in manufacturing machines to detect the load power consumption of each machine. The data obtained from these smart devices were sent to MariaDB for storage.

A sequence of machine load power data, including information such as voltage, current, temperature, and frequency information, was implemented as embedding vectors representing the characteristics and profile relationships of the electrical load. Using the analyzed embedding vector sequence, the LSTM network predicts the obtained data pattern as sequence data. The experimental results demonstrate that the proposed model exhibited the best prediction results with the highest accuracy using the collected dataset compared with MLP, RF, and SVM models. Furthermore, these results are expected to support manufacturing operations and management in monitoring the daily load power consumption of industrial machines in manufacturing facilities to reduce significant equipment damage due to unstable load power issue conditions by considering various appropriate preventative actions. Furthermore, we developed a user interface to display the prediction results using the Flask framework and the REST API.

In future research, other machine learning algorithms must be considered to determine which method may exhibit the best performance in detecting load power data consumption in an industrial manufacturing environment, considering that excluding missing or unique values from the dataset may limit network performance owing to the time intervals between data points in the sequence.

Therefore, the methods must be developed to estimate or deal with missing values, as well as inconsistent time intervals; thus, consistency must be further considered based on time-series data.

ACKNOWLEDGMENTS

This work was supported by Gyeongnam SW Convergence Cluster 2.0 under the contract and the "Leaders in INdustry-university Cooperation +" Project by the Ministry of Education and National Research Foundation of Korea.

Fig 1.

Figure 1.Implemented Smart Devices.
Journal of Information and Communication Convergence Engineering 2022; 20: 295-302https://doi.org/10.56977/jicce.2022.20.4.295

Fig 2.

Figure 2.Relational Database Structure.
Journal of Information and Communication Convergence Engineering 2022; 20: 295-302https://doi.org/10.56977/jicce.2022.20.4.295

Fig 3.

Figure 3.Architecture of LSTM model.
Journal of Information and Communication Convergence Engineering 2022; 20: 295-302https://doi.org/10.56977/jicce.2022.20.4.295

Fig 4.

Figure 4.Sample of load power data from SPD.
Journal of Information and Communication Convergence Engineering 2022; 20: 295-302https://doi.org/10.56977/jicce.2022.20.4.295

Fig 5.

Figure 5.Load power data from SPD.
Journal of Information and Communication Convergence Engineering 2022; 20: 295-302https://doi.org/10.56977/jicce.2022.20.4.295

Fig 6.

Figure 6.Output Load Power Prediction Graph (LSTM, MPL, RF, and SVM).
Journal of Information and Communication Convergence Engineering 2022; 20: 295-302https://doi.org/10.56977/jicce.2022.20.4.295

Fig 7.

Figure 7.MAPE of the four models in eight months.
Journal of Information and Communication Convergence Engineering 2022; 20: 295-302https://doi.org/10.56977/jicce.2022.20.4.295

Table 1 . Experimental setup.

NoDescriptionValues
1EnvironmentsTensorflow 1.8.0
Phyton 3.5
2OptimizerGradient Descent
3Lost functionseq2seq loss
4Dropoutprobability: 0.2
5BatchBatch size: 32
6RegularizationL2 regularizer
7Gradient ClippingO
8Learning Rate10-2
9SoftmaxO

Table 2 . Detailed information of load power data.

MeasurementRangeNormal RangeUnit
id0~65535--
device_id0~65535-
r_voltage0~655.35 V200V ~ 240 V0.01 V
s_voltage0~655.35 V200V ~ 240 V0.01 V
t_voltage0~65.535 A1A ~ 5 A0.001 A
r_current,0~65.535 A1A ~ 5 A0.001 A
s_current,0~65.535 A1A ~ 1 A0.001 A
t_current,0~65.535 A1A ~ 1 A0.001 A
r_active_power-32.768~32.767 kW0.22~1.100.001 kW
s_active_power-32.768~32.767 kW0.22~1.100.001 kW
t_active_power-32.768~32.767 kVAR-0.010~0.0100.001 kVAR
r_reactive_power-32.768~32.767 kVAR-0.010~0.0100.001 kVAR
s_reactive_power-32.768~32.767 kVAR-0.010~0.0100.001 kVAR
s_reactive_power-32.768~32.767 kVAR-0.010~0.0100.001 kVAR
r_apparent_power-32.768~32.767 kVA0.22~1.100.001 kVA
s_apparent_power-32.768~32.767 kVA0.22~1.100.001 kVA
t_apparent_power-32.768~32.767 kVA0.22~1.100.001 kVA
r_power_factor-1.000~+1.0000.980~1.0000.001
s_power_factor-1.000~+1.0000.980~1.0000.001
t_power_factor-1.000~+1.0000.980~1.0000.001
frequency45.00~65.00 Hz58~62 Hz0.01 Hz
temp-128~127 °C10~40 °C1 °C
datetime---

References

  1. D. Mao and K. Potty and J. Wang, The impact of power-electronicsbased load dynamics on large-disturbance voltage stability, in Proceeding of IEEE Power & Energy Society General Meeting (PESGM), Portland: OR, USA, pp. 36-39, 2018. DOI: 10.1109/PESGM.2018.8586221.
    CrossRef
  2. Y. Li and D. Han and Z. Yan, Long-term load forecasting based on data-driven linear clustering method, Journal of Power System and Automation, vol. 6, no. 2, pp. 306-316, Mar., 2018. DOI: 10.1007/s40565-017-0288-x.
    CrossRef
  3. J. Hao, and Y. Liu, and H. Gu, and D. Yang, and R. Wang, and J. Lei, and M. Deng, and Z. Huang, Short-term power load forecasting for larger consumer based on tensorflow deep learning framework and clusteringregression model, in Proceeding 2018 2nd IEEE Conference on Energy Internet and Energy System Integration (EI2), Beijing, China, pp. 1-6, 2018. DOI: 10.1109/EI2.2018.8582583.
    CrossRef
  4. Z. Wang and J. Qin and C. Cao, Research on prediction of dynamic load complementary based on big data, in Proceedings of 2019 International Conference on Computer Network, Electronic and Automation (ICCNEA), Xian, China, pp. 369-373, 2019. DOI: 10.1109/ICCNEA.2019.00074.
    CrossRef
  5. K. Zhou and Y. Zhou and K. Deng, A circular recursion algorithm power load forecasting model and application, in Proceedings of 2019 IEEE 3rd Conference on Energy Internet and Energy System Integration (EI2), Changsha, China, pp. 2870-2873, 2019. DOI: 10.1109/EI247390.2019.9061824.
    Pubmed KoreaMed CrossRef
  6. F. Kuang, and D. Huang, Power load prediction method based on VMD and dynamic adjustment BP, in Proceedings of 2019 CAA Symposium on Fault Detection, Supervision and Safety for Technical Processes (SAFEPROCESS), Xiamen, China, pp. 160-164, 2019. DOI: 10.1109/SAFEPROCESS45799.2019.9213431.
    Pubmed CrossRef
  7. A. Boltunov, and S. Vasiliev, and V. Karpenko, and A. Voloshin, and E. Voloshin, Short-term load forecasting system for smartgrids based on personal power units, in Proceedings of 2019 Modern Electric Power Systems (MEPS), Wroclaw, Poland, pp. 1-3, 2019. DOI: 10.1109/MEPS46793.2019.9394995.
    CrossRef
  8. T. Gong, and S. Tian, Analysis and forecasting for power load of office building taking crowd behavior into account, in Proceeding of 2019 IEEE 8th Joint International Information Technology and Artificial Intelligence Conference (ITAIC)), Chongqing, China, pp. 997-1000, 2019. DOI: 10.1109/ITAIC.2019.8785742.
    KoreaMed CrossRef
  9. R. Rijayanti, and A. B. Wahyutama, and B. Lee, and J. Kim, and B. Yoo, and M. Hwang, Design of smart devices and applications for managing the load power of industrial manufacturing machine, in Proceedings of The 17th International Conference on Multimedia Information Technology and Applications (MITA2021), Jeju, Korea, pp. 47-50, A2021.
  10. J. Schmidhuber. “Habilitation”, Postdoctoral Thesis, The Technical University of Munich, Munich, 1993, [online] Available: https://people.idsia.ch/~juergen/onlinepub.html
  11. S. A. Ludwig, Comparison of time series approaches applied to greenhouse gas analysis: ANFIS, RNN, and LSTM, in Proceeding of IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), New Orleans: NA, USA, pp. 1-6, 2019. DOI: 10.1109/FUZZIEEE.
    CrossRef
  12. C. Deb, and F. Zhang, and J. Yang, and S. E. Lee, and K. W. Shah, A review on time series forecasting techniques for building energy consumption, Renewable and Sustainable Energy Reviews, vol. 74, pp. 902-924, Jul., 2017. DOI: 10.1016/j.rser.2017.
    CrossRef
JICCE
Dec 31, 2022 Vol.20 No.4, pp. 235~325

Stats or Metrics

Share this article on

  • line
  • mail

Journal of Information and Communication Convergence Engineering Jouranl of information and
communication convergence engineering
(J. Inf. Commun. Converg. Eng.)

eISSN 2234-8883
pISSN 2234-8255