Journal of information and communication convergence engineering 2024; 22(1): 44-55
Published online March 31, 2024
https://doi.org/10.56977/jicce.2024.22.1.44
© Korea Institute of Information and Communication Engineering
Correspondence to : Siva S (E-mail: sivaraju.reva@gmail.com)
Department of Computer Science and Applications, Reva University, Bangalore 560064, India
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
High-utility itemset mining (HUIM) is a dominant technology that enables enterprises to make real-time decisions, including supply chain management, customer segmentation, and business analytics. However, classical support value-driven Apriori solutions are confined and unable to meet real-time enterprise demands, especially for large amounts of input data. This study introduces a groundbreaking model for top-N high utility itemset mining in real-time enterprise applications. Unlike traditional Apriori-based solutions, the proposed convolutional sequential embedding metrics-driven cosine-similarity-based multilayer perception learning model leverages global and contextual features, including semantic attributes, for enhanced top-N recommendations over sequential transactions. The MATLAB-based simulations of the model on diverse datasets, demonstrated an impressive precision (0.5632), mean absolute error (MAE) (0.7610), hit rate (HR)@K (0.5720), and normalized discounted cumulative gain (NDCG)@K (0.4268). The average MAE across different datasets and latent dimensions was 0.608. Additionally, the model achieved remarkable cumulative accuracy and precision of 97.94% and 97.04% in performance, respectively, surpassing existing state-of-the-art models. This affirms the robustness and effectiveness of the proposed model in real-time enterprise scenarios.
Keywords High utility itemset mining, Semantic sequential deep learning, MLP, Cosine similarity, Top-N HUI Recommendation
In recent years, software computing, big data analytics, and decentralized computing technologies have emerged. These technologies, in sync with gigantic digital data, have helped enterprises understand user behavior, consumer perceptions, and preferences to improve business decisions. Consumers’ purchase behavior and transaction analyses have helped enterprises understand supply chain demands, highutility items, and periodic demand patterns, thereby playing a decisive role in business communities, including e-commerce, manufacturing, and supply chain industries (e.g., the global value chain). However, identifying a set of high-utility items for the aforementioned digital data or transaction details, remains challenging. Advanced computing technologies, such as pattern mining [1-4], have enabled the identification of high-utility itemsets for business decisions. Typically, in sync with the business ecosystem, pattern mining technologies exploit existing transaction details to understand consumer preferences and socioeconomic behavior, thus helping enterprises improve their marketing mix decisions. They also help consumers identify the intended product or service that meets the respective demands [5]. In other words, the pattern-mining technique can help both enterprises and consumers with certain optimistically identified sets of products or items for corresponding use. In recent years, machine learning (ML) and artificial intelligence (AI) have played decisive roles in pattern mining-based recommendation systems [5]. An improved computing ecosystem called natural language programming (NLP) can also identify the target output by learning raw input patterns or transaction details [5]. These techniques exploit different input patterns and associated features to make prediction decisions [5], learning over different periodic and sequential patterns, as well as frequent itemsets.
In business ecosystems, pattern analysis methods have gained widespread attention for market basket analysis and business intelligence (BI). BI approaches exploit sequential or periodic transaction details (or data) to identify the frequent itemsets with high utility values. This helps in performing personalized predictions for users. In BI ecosystems, the identification of high-utility itemsets (HUIs) enables the segmentation of N demanding products and services. Classical approaches often exploit intercommodity associations and frequency of item purchases or transactions, to identify HUIs for further prediction [6-8]. Notably, an item is stated to be a frequent itemset when its frequency is greater than a predefined threshold, called the support value [9]. Different approaches have been proposed for frequent high-utility itemset mining (FHUIM); however, merely applying a predefined threshold over large nonlinear features or patterns cannot yield optimal accuracy for generalization. Techniques such as the Apriori [6][7] method determine frequent itemsets by assessing iterative level-based searches to identify HUIs. Notably, these methods employ the downward closure method, in which an a priori characteristic is applied to prune redundant or less redundant items. Apriori-based methods ensure that itemsets possessing a low support value do not become an HUI. However, the iterative estimation of the support value can be computationally demanding, especially over a large search space, which limits the robustness of Apriori-based methods. In recent years, several other approaches such as equivalence class clustering and bottomup lattice traversal (ECLAT) [2], frequent pattern-growth (FP-Growth) [3][10], and hyper-structure mining (Hmine) [11] were proposed to improve pruning and the associated data structure for HUI identification. These approaches that employ frequent itemset mining, only consider the frequency of the itemsets in relation to other items rather than their respective utility or co-occurrence probability. This limits their utility in contemporary business decisions, where identifying co-occurrence items is as important as identifying HUIs. The literature indicates that other details, such as transaction counts, co-purchased items, their frequency, and high profit value, can help identify HUIs in value-based itemset predictions [12]. Unlike Apriori or frequency-based solutions, high-utility itemset mining (HIUM) [13,14] techniques which employ both volume as well as profit per unit for HUI estimation, have gained widespread attention. These methods aim to improve both accuracy and computational costs to improve scalability [12]. Many state-of-the-art studies have applied the utility factor, which includes total and unit profits of the itemset, in estimating HUIs from a large search space based on transaction details. These approaches perform better than Apriori methods; However, their suitability for generalization to larger dynamic transactions is disputable [12-15]. Studies have revealed that pruning insignificant items from transaction search spaces can minimize computational cost and delay and improve learning-based prediction accuracy [12]. This prompted the development of transaction-weighted utility (TWU) which focuses on improving pruning, whereby the upper threshold is estimated to prune an itemset. To improve accuracy and reduce the complexity over large inputs, two-phase methods have been proposed [16-19], which are hypothesized to be more effective than single-phase solutions [20,21]. In two-phase methods, highly correlated items are first identified. Subsequently, their utility is assessed to label them as HUIs. Despite claims of higher accuracy, the associated computational cost remains a challenge, particularly in iterative database scanning and level-wise utility estimation [20]. The use of a list structure (utility list) performs better. However, the utility value remains proportional to the length of the itemset, making the computation more complex. To alleviate this, the concept of average utility was recently proposed, which focuses on reducing the impact of length on HUI prediction [22-24], applying the average support value to perform HUI estimation. Interestingly, none of these methods can exploit the itemset probability or contextual details among the sequential or periodic transactions in performing HUI itemset prediction.
In this paper, a novel convolutional sequential embedding metric-driven cosine-similarity-based multi-layer perception learning model is proposed for top-N high-utility itemset recommendations. Unlike traditional deep learning solutions, such as convolutional neural network (CNN)-based top-N HUIM models that apply local features for prediction, the proposed model exploits contextual or global features, including semantic features, over sequential transactions to perform top-N recommendations. First, semantic features are extracted from the transaction details encompassing the itemsets and their respective co-occurrence probabilities using the semantic CNN (SCNN) model. The extracted semantic features are processed by a multilayer perceptron (MLP), which retrieves the linear relationship among the itemsets and their corresponding co-occurrence probabilities. Subsequently, the cosine-similarity method is applied over the MLP-predicted linear associations to perform top-N HUI predictions. In MATLAB-based simulations over different datasets, the proposed HUIM model achieved a precision of 0.5632, mean absolute error (MAE) of 0.7610, hit rate (HR)@K of 0.5720, and normalized discounted cumulative gain (NDCG)@K of 0.4268. Additionally, it exhibited an average MAE of 0.608 on different datasets and latent dimensions, achieving a cumulative performance accuracy and precision of 97.94% and 97.04%, respectively. Relative performance characterization revealed that the proposed top-N HUIM model surpasses other state-of-art methods, including CNN, in different federated learning environments, which confirms the robustness of the proposed model and its suitability for real-time enterprise purposes.
The remainder of this paper is organized as follows. Section II discusses related works, followed by the research questions in Section III. Section IV presents the overall proposed model and its implementation. The simulation results and relevant inferences are provided in Section V. Section VI discusses the overall research contributions in the context of future scope. The references used in this study are provided at the end of the manuscript.
Pattern mining [4] methods were initially designed based on the frequency of itemsets [4]. Frequent itemset mining methods exploit threshold conditions such as support value and profit score to perform HUI estimation. However, applying standalone threshold-based pruning alone cannot yield a robust solution. A few improved methods such as ECLAT [2], FP-Growth [3], Apriori [4], HMine [11], and HUIM have been designed in recent years for HUI estimation. An Apriori-based itemset mining method was proposed in [6], where a level-wise search method is used to help estimate frequent itemsets. However, the computational cost associated with a large search space and iterative pruning make these methods laborious. The algorithm of FP-growth [3] applies a tree-structure to detect HUIs, whereby an FP-tree structure is first obtained by traversing across data space, searching for frequent itemsets over the tree structure. An improved FP growth method called HMine [11] has been developed, with an additional supplementary pointer-based hyperlink to represent items with a high frequency across the search space. ECLAT [2] was designed by using a vertical database structure called “Transaction ID list.” Unlike conventional methods in which a unit pointer is applied to detect each data element, the ECLAT method exploits the transaction ID to minimize the scanning cost, applying a support count value to each itemset to prune the search space for HUI estimation. Other methods [6,26-29] have used the support value and/or mean profit score for HUI estimation.
Unlike frequency-based itemset mining methods, HUIM methods have desirable performance [13,14] because of their ability to exploit and learn over a large transaction volume, and the corresponding profit makes them suitable for HUI estimation. Two-phase HUI methods [16] begin by identifying itemsets with higher frequencies, followed by an estimation of their utility for final HUI prediction. However, these methods have been criticized for their reliance on a standalone threshold when dealing with non-linear sequential data, which is often inadequate for HUI estimation [16]. Thus, the authors in [16] designed a transaction-weighted utility (TWU) function to minimize the iterative data-scanning cost. This TWU method was further enhanced in [17] by introducing a flexible upper threshold and the capability to skip high-utility itemsets, thereby improving the efficiency of the search space. The authors employed a twophase method with pruning called the isolated itemset discarding strategy (IIDS) to improve delay performance. Other tree-based methods include incremental high-utility pattern (IHUP) [18], HUP-tree [30], UP-Growth [19], UP-Growth+ [31], mining utility(MU)-Growth [32], and projection-based (PB) indexing approach [33]; however, the computational costs involved and lack of co-occurrence probability remain unexplored. In [20], an HUI miner was proposed using a utility-list (UL) data structure. The UL contains details of the itemset required for pruning, followed by HUI identification. However, despite claims of time efficiency, it cannot address HUI identification using correlated itemsets. In [21], HUI-Miner was designed to reduce the number of joins between the utility and its efficiency functions. The authors employed estimated-utility co-occurrence pruning (EUCP) on a matrix structure called the estimated-utility co-occurrence structure (EUCS). The EUCP encompasses the TWU values of the two item sets arranged in the EUCS matrix. The estimated itemsets were used to prune low-significance items without estimating the utility value. A number of pruning algorithms have been developed to enhance HUIM [34]; however, they fail to address semantic relatedness among itemsets over the search space. The authors in [35] designed efficient itemset mining (EFIM) with predefined upper bounds, considering subtree and local utility factors. To reduce scanning costs, they used the transaction-merging concept. The HMiner [36] model was applied using utility information storage with allied pruning. Approaches such as BAHUI [37], the HUIM-BPSO sign [38], MinHUIs [39], and FHM+ [40] have also been used for HUI estimation.
The two-phase average-utility (TPAU) method [22] applies an average utility-based upper threshold condition, whereby a level-wise search method is applied to enhance the time efficiency. The projection-based average-utility (PBAU) [23] method applies an indexing structure. By applying PBAU, a hard upper limit is defined using the prefix concept [23] to reduce search costs. In [41], a tree-based high-average-utility itemset (HAUI) mining method was designed. The HAUIGrowth model [42] was developed using a tree-based method that efficiently reduces iterative data scanning, and the HAUI miner was designed as a one-phase concept [43] by applying an average utility-based list structure. The efficient high average-utility pattern mining (EHAUPM) in [44] was designed by amalgamating two upper thresholds: looser upper-bound utility (LUB) and revised tighter upper bound (RTUB). Mining of high average-utility (MHAI) [45] retains suitable HUIs based on a high average-utility itemset (HAI) list structure. A closed high-utility itemset (CHUI) with DGU, REG, RML, and DGM was designed in [51] to retain decisive itemsets over the input data structure. In [52], CHUI-Miner was designed as a single-phase model by applying the EU-list structure. This reduces unexpected iterative search costs. The CHIU-Miner, called EFIM-closed [53], was developed with two strictly defined upper thresholds with forward-backward examination. This method uses local and subtree utility values to prune the search space. The CLS-Miner [54] was designed with supplementary coverage and LBP. Despite numerous efforts [46-50], no significant work has examined the probability of coexisting items as HUIs for top-N HUI predictions [55]. Van et al. [56] used FP growth to examine the association between available features to append new features with a certain threshold to perform HUI estimation. In [56], sequential-to-sequential learning methods were applied to top-N target balanced recommendations. In [57], a deep learning method was applied to top-N recommendations, considering the user’s interest level and frequency. A deep reinforcement learning model was used in [58] for top-N recommendations. Similarly, interest-related item set learning and similarity measures were applied to perform top-N recommendations [59]. In [60], a trust-aware sequential recommendation model that exploits frequency information was designed. Unfortunately, there has been no viable effort that considers sequential co-occurrence probability or the semantically connected co-occurrence feature to perform top-N HUI recommendations. This was the key driving force in this research.
This section discusses the proposed convolutional sequential embedding-driven cosine similarity-based MLP learning model for top-N HUI recommendations. As the name indicates, the proposed system comprises three key components: a semantic sequential convolutional encoding (SSCE) also called semantic CNN (SCNN), MLP, and cosine similarity for top-N HUI predictions. The SSCE model comprises a multidirectional filtering-assisted semantic embedding framework that learns over sequential input items or transactions to generate a semantic embedding matrix. In sync with the enterprise application environment, transaction details (user preferences) and allied frequent itemset patterns (buying patterns) are considered. In this manner, unlike conventional deep learning approaches [64-67] that apply local item-wise embedding metrics for learning, the proposed model employs both item-level and corresponding group-level (co-occurrence probability) information to perform top-N HUI prediction or recommendation. This approach enables the model to achieve a higher accuracy while maintaining low computational cost and delays during the scanning of search space (or feature space) to support estimation. An MLP network is applied in conjunction with an adaptive model optimizer (ADAM) learning model to perform training and obtain linear associations among the items, item frequency, and associated co-occurrence probabilities over the sequential transaction inputs. To refine the predicted results and increase accuracy, MLP was deployed in conjunction with the cosine similarity method, to help predict (accurately) the top-N HUIs for recommendation. The proposed model encompasses the following phases:
Semantic sequential convolutional encoding (SSCE),
Multilayer perceptron (MLP) learning
Cosine similarity-driven top-N HUI prediction.
The detailed discussion of these (functional) components is provided as follows:
Let the transaction data U have L consecutive itemsets. The objective of this work is to identify the top-T items with both high frequency and co-occurrence probability over sequential transaction data S^{u}. Here, the transaction dataset has input items S_{1}^{u}, ..., S_{|Su|}^{u}. Therefore, the proposed SSCE model first converts input transactions into an equivalent semantic word-embedding form. More specifically, the Word2Vec embedding method is applied to convert sequential input transactions into equivalent word-embedded metrics (P_{M}). This metric states the probability of occurrence of each itemset in the transaction data. Once semantic (word-embedded) metrics, also called semantic features, are obtained for the input transaction data, multidirectional convolutional filtering (MDCF) is applied to retrieve the contextual features over the itemsets along with the corresponding co-occurrence probability. Subsequently, the retrieved contextual details are used to perform learning using the MLP network, which, in conjunction with cosine similarity, predicts the top-N HUIs. A schematic of the implementation of the proposed model is shown in Fig. 1.
By applying the aforementioned Word2Vec semantic word-embedding method over the input sequential itemsets and their occurrences across the transaction data, an embedding matrix that possesses embedded information related to the itemsets and their occurrence probability is obtained. Here, the embedding matrix for the input sequential transactions is obtained from the semantic feature space by inputting L-traversed itemsets into a neural network. Let, i denote an item; then the corresponding embedding matrix representing the latent feature is obtained as
In addition to the above derived itemset embedding, the proposed model applies a supplementary embedding model to derive latent information for u users,
Let
where ⊙ refers to the inner product or multiplication function, whereas ⌀c(·) represents the activation function.
We applied a max-pooling layer to the extracted embedding metrics or vector c^{k} (3). This helps retain the high-resolution features with maximum values, retrieved over the features generated by the convolutional filter. Thus, the proposed method retrieves the set of vectors or significant features over the deployed n filters that eventually yield the output as
Similar to horizontal filter-driven feature extraction (4), we also applied vertical convolutional filters. Let,
The retrieved feature vector (5) shows that the result is the same as the weighted sum over L rows of the embedding metrics E, obtained by examining the weights
In (6), E^{l} represents the lth row of E. Thus, the vertical filters help in learning the embeddings of the L subsequent itemsets having the same value as the weighted sum, in identifying the latent information of L itemsets. Vertical filters generate an entity-level sequential itemset pattern by using weighted sums over the latent information of the traversed itemsets.
Traditional deep learning-based recommendation models use a single weighted sum to predict an itemset. By contrast, we used
Horizontal and vertical filters were used to extract the global features from the global average max-pooling layer, thereby applying the max function to estimate the global features (8).
The proposed model generates a final feature vector by applying a nonlinear-activation-driven fully connected layer. The output of the fully connected layer is given by (9).
In (9), W_{h} and W_{h} represent the weight metrices, whereas
As stated earlier, the features from both the horizontal and vertical filters (9) were projected onto a fully connected neural network. The MLP network was applied as a fully convolutional neural network (FCNN) to represent the nonlinear relationship between the itemsets and the associated cooccurrence probability. The deployed MLP network is a multilayer feed-forward neural network (MFNN) that learns nonlinear itemsets, aligning them with co-occurrence probability interactions. In this study, the combined embedding feature (9) was fed into the MLP to obtain the latent embedding feature in (10).
In (10),
In (11),
The MLP training model over latent metrics was designed based on the loss between the predicted and measured itemset relationships. In this work, the cost function (12) was used to train the network for the prediction of target itemsets q using cost function p (12).
In (12),
In this study, a normalized cross-entropy loss function (14) was applied to perform the learning.
In (14), max(R) denotes the highest rating of the itemsets across the input latent embedding matrix. We applied MLP to represent a nonlinear relationship between the itemsets and their co-occurrence (utility). In the MLP network, let the input embedding matrices for the itemsets be
In (15-16) the rectified linear unit (ReLU) is represented by the activation function f(*), whereas
Notably, unlike traditional top-N prediction models, where the learning model (e.g. MLP) predicts the top N selected items, the proposed model applies cosine similarity to the initially predicted items. This helps identify the optimal set of top-N items with high inter-item similarity and corresponding co-occurrence probability. This can be of great significance to enterprises, for highly accurate HUI prediction and inventory management. Thus, by applying this method, the proposed model performs top-N HUI predictions for enterprises or applications.
This study proposes a robust convolutional sequential embedding metric-driven cosine similarity-assisted multilayer perception learning model for top-N high-utility itemset recommendation. Unlike classical approaches, the proposed model exploits both sequential itemset frequency and co-occurrence probability information to perform top-N HUI predictions. First, the sequential transaction data are input into word embedding using the Word2Vec method. Subsequently, the retrieved semantic embedding matrix and multidirectional filters encompassing the horizontal and vertical filters are used to derive global features. The composite embedded matrix features are projected as input onto the MLP layer, which, in conjunction with cosine similarity, performs top-N HUI prediction and recommendations. The deep learning model was executed at an initial learning rate of 0.0001. The overall proposed model was developed using the MATLAB software tool, performing simulation over central processing units arranged using 8 GB RAM and a 3.2 GHz processor. To assess efficacy, different benchmark datasets were applied [61,62]. The dataset encompasses sequential transaction details and itemsets. The proposed model enables intensity estimation over sequential itemsets, as expressed in (18).
In (18), the numerator represents the frequency
In the above equations, N denotes the list of N predicted (search) itemsets (i.e.,
Table 1 . Simulated results over different datasets
Dataset | d | Prec. | Recall | MAP | MAE |
---|---|---|---|---|---|
Tmall | 4 | 0.341 | 0.014 | 0.126 | 0.983 |
8 | 0.420 | 0.024 | 0.174 | 0.734 | |
16 | 0.451 | 0.042 | 0.204 | 0.832 | |
32 | 0.563 | 0.059 | 0.234 | 0.721 | |
64 | 0.498 | 0.059 | 0.293 | 0.701 | |
Gowalla | 4 | 0.174 | 0.039 | 0.199 | 0.863 |
8 | 0.192 | 0.059 | 0.201 | 0.723 | |
16 | 0.198 | 0.098 | 0.223 | 0.758 | |
32 | 0.210 | 1.235 | 0.224 | 0.634 | |
64 | 0.218 | 1.249 | 0.300 | 0.698 | |
Repeat_Buyer | 4 | 0.356 | 0.032 | 0.167 | 0.899 |
8 | 0.380 | 0.052 | 0.198 | 0.983 | |
16 | 0.404 | 0.159 | 0.201 | 0.799 | |
32 | 0.428 | 0.210 | 0.223 | 0.652 | |
64 | 0.439 | 0.245 | 0.247 | 0.678 | |
Kaggle | 4 | 0.319 | 0.087 | 0.199 | 0.857 |
8 | 0.334 | 0.042 | 0.199 | 0.699 | |
16 | 0.3452 | 0.073 | 0.259 | 0.694 | |
32 | 0.3569 | 0.099 | 0.299 | 0.621 | |
64 | 0.3452 | 0.139 | 0.301 | 0.6873 |
Among the state-of-the-art approaches considered for top-N itemset recommendation, a few methods [57-60] have applied deep learning methods for latent feature learning and prediction; however, state-of-the-art methods can be laborious and time consuming. For instance, the authors in [57] applied a CNN followed by a denoising autoencoder (DAE) to perform top-N recommendation, with the CNN applied for latent feature extraction, followed by fractional maximization and DAE-based top-N recommendation. The use of successive deep models can incur high computational costs, exhaust memory, and delay performance. In addition, in observing the empirical simulation results over different datasets, the proposed approach showed a relatively lower error value than \existing methods [57]. The highest precision obtained by DMLR-DAE [57] was 0.32. By contrast, the proposed method achieved the highest precision of 0.5632, which is higher than that of the other existing methods. The MAE obtained using the DMLR-DAE [57] was 0.611 at d = 10 and 0.609 at d = 30. In contrast, the proposed method achieved an average MAE of 0.7610 for the different benchmark datasets.
In addition, the hit rate was estimated for the number of target HUI predicted in the top N predicted lists. Mathematically, HR is measured using (22).
In (22), |T| denotes the number of itemsets and associated co-occurrence probabilities or interactions in the test set. In addition, the normalized discounted cumulative gain was applied to assess the hit position by assigning higher scores to hits (especially fort the top K ranks). Mathematically, NDCG@K is obtained as described previously (23).
In (23), r_{i} denotes the ranked relevance of the target item at the i-th position (hence, r_{i} = 1). Otherwise, r_{i} = 0. In addition, the root mean square error (RMSE) was measured using (24), where T states the total number of test ratings, whereas R^{s} (i, j) represents the real rating, with the measured or predicted rating being R^{s} (i, j).
The proposed model was simulated using embedding dimensions of k = {4, 8, 16, 32, 64}, where k represents the latent embedding dimension. The results (Table 2) infer that with increasing k, HR also increases. This implies that with increasing latent dimensions, both HR and NDCG increase. The mean HR ratio was 0.57923. In contrast, average NDCG was 0.4268 and RMSE was 0.9143. A higher HT rate indicates a higher accuracy and reliability of the solution, whereas, a lower RMSE indicates a better performance of the proposed model. In comparison with state-of-the-art methods [63], the results confirm the superiority of the proposed model for top-N HUI recommendations. The existing method [63] achieved an HR@64 of 0.5126; in contrast, the average HR@K of 0.5720, of the proposed method was significantly higher than that of the state-of-the-art methods. This confirms the robustness of the proposed model for top-N realtime HUI predictions.
Table 2 . Performance over the different latent embedding dimensions
Latent Dimensions (d) | Data | HR@K | NDCG@K | RMSE |
---|---|---|---|---|
4 | Tmall | 0.453 | 0.283 | 1.409 |
Gowalla | 0.682 | 0.410 | 1.389 | |
Repeat Buyer | 0.543 | 0.312 | 1.004 | |
Kaggle | 0.459 | 0.321 | 0.953 | |
8 | Tmall | 0.486 | 0.299 | 1.091 |
Gowalla | 0.672 | 0.461 | 1.077 | |
Repeat Buyer | 0.578 | 0.532 | 0.997 | |
Kaggle | 0.613 | 0.523 | 0.096 | |
16 | Tmall | 0.482 | 0.299 | 0.987 |
Gowalla | 0.677 | 0.498 | 0.988 | |
Repeat Buyer | 0.578 | 0.487 | 0.902 | |
Kaggle | 0.689 | 0.512 | 0.904 | |
32 | Tmall | 0.501 | 0.374 | 0.938 |
Gowalla | 0.698 | 0.460 | 0.874 | |
Repeat Buyer | 0.582 | 0.490 | 0.921 | |
Kaggle | 0.600 | 0.498 | 0.184 | |
64 | Tmall | 0.512 | 0.377 | 0.880 |
Gowalla | 0.701 | 0.481 | 0.871 | |
Repeat Buyer | 0.564 | 0.499 | 0.911 | |
Kaggle | 0.510 | 0.412 | 0.910 |
Additionally, we compared the proposed model in terms of top-N recommendation accuracy. Table III presents the comparative results of different state-of-the-art techniques. The information in the confusion matrix was used to assess prediction accuracy.
Observing the results, clearly unlike traditional deep learning-based approaches [64-67], where feature extraction is based on CNN, in the proposed method, semantic feature extraction is followed by improved learning-driven feature extraction, which is subsequently processed using cosinesimilarity to obtain top-N HUI prediction. Using this approach, the proposed model refined the predicted (top-N HUI) output, achieving higher accuracy, which can easily be visualized in Fig. 4.
Similarly, in terms of cumulative precision, which is derivedfrom the confusion matrix, the proposed itemset top-N HUI prediction model yielded higher precision (97.04%) than the existing method [68] (87%), clearly indicating that the proposed model is more robust compared to the state-of-the art models for enterprise HUI prediction. Conclusions alongwith inferences are presented in the following section.
Table 3 . Accuracy Prediction
Source | Techniques | Accuracy (%) |
---|---|---|
[64] | Federated NN, CNN | 71.68 |
[65] | CNN | 70.00 |
[66] | Random Forest, NNC and XGBoost | 88.84 |
[67] | NN, Logistic Regression | 89.00 |
[68] | Association rule as feature selection over CNN for top-N itemset prediction | 91.00 |
Proposed | convolutional sequential semantic embedding driven MLP with Cosine similaritybased top-N HUI recommendation | 97.94 |
Most existing HUIM methods often face limitations due to high computational costs, delays, and reduced accuracy when processing extensive sequential transaction data prevalent across various industries. Moreover, the pruning costs and lack of contextual details representing co-occurrence probability limit the efficacy of available state-of-the-art methods. Unlike HUIMs that rely on support value and unit price threshold for pruning, deep learning-based HUI identification is a better alternative. This approach, however, necessitates the exploitation of contextual details, including semantic sequential embedding features, to perform HUI prediction. Motivated by this, this study proposed a novel and robust convolutional sequential semantic embeddingdriven multi-layer perceptron learning environment, in sync with cosine similarity, to predict top-N HUI recommendations. In the proposed method, first, two filters are applied along the horizontal and vertical directions simultaneously in convolutional sequential deep learning, to extract the semantic embedding matrix over the transaction details. The use of multiple convolutional filters allows for the retention of a substantial amount of semantic information for further learning and classification. Using the extracted semantic features, the MLP neurocomputing model, which is designed using a ReLU regulation layer and ADAM non-linear optimization function, obtains the linear relationship among the itemsets available across the search space. The proposed MLP model was executed in conjunction with a cosine similarity function to predict the top-N HUI for further recommendations. The use of a semantic embedding matrix with MLP learning and cosine similarity measures helped refine the top-N HUI itemset predictions, which can be highly accurate for any enterprise solution(s). The proposed model exhibited the highest precision of 0.5632, MAE of 0.7610, HR@K of 0.5720, and NDCG@K of 0.4268. Additionally, it exhibited an average MAE of 0.608 over four different datasets, indicating robustness for real-time HUI predictions. The higher cumulative accuracy (97.94%) and precision (97.04%) confirm the efficacy and suitability of the proposed model for real-time enterprise solutions.
Siva S
He received his Master of Computer Applications (M.C. A) from the University of Madras, India, in 2004. With over 16 years of experience in software research and development, he is currently employed as a senior engineering manager in a company in Bangalore, India, where a healthcare mobile solution is being developed using IoT and AI. Additionally, he is pursuing his Ph.D. at REVA University, Bangalore, India, and holds membership in IEEE.
Dr. Shilpa Chaudhari
Currently, she is working as an Associate Professor at the Department of CSE, MSRIT, Bangalore. She has been a technology educator and corporate trainer since 1999. Over the last 18 years, she has held various academic positions in technical institutes in Maharashtra and Karnataka. Her areas of research and teaching include network security, RTOS, computational intelligence, wireless networks, and embedded system development. She is an active professional member of the Computer Society of India (CSI), having been a Life member since 2013, and also holds membership with IEEE Member (#94611631), Bangalore Section.
Journal of information and communication convergence engineering 2024; 22(1): 44-55
Published online March 31, 2024 https://doi.org/10.56977/jicce.2024.22.1.44
Copyright © Korea Institute of Information and Communication Engineering.
Siva S ^{1*} and Shilpa Chaudhari^{2 }
^{1}Department of Computer Science and Applications, Reva University, Bangalore 560064, India
^{2}Department of Computer Science and Engineering, M S Ramaiah Institute of Technology, Bangalore 560054, India
Correspondence to:Siva S (E-mail: sivaraju.reva@gmail.com)
Department of Computer Science and Applications, Reva University, Bangalore 560064, India
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
High-utility itemset mining (HUIM) is a dominant technology that enables enterprises to make real-time decisions, including supply chain management, customer segmentation, and business analytics. However, classical support value-driven Apriori solutions are confined and unable to meet real-time enterprise demands, especially for large amounts of input data. This study introduces a groundbreaking model for top-N high utility itemset mining in real-time enterprise applications. Unlike traditional Apriori-based solutions, the proposed convolutional sequential embedding metrics-driven cosine-similarity-based multilayer perception learning model leverages global and contextual features, including semantic attributes, for enhanced top-N recommendations over sequential transactions. The MATLAB-based simulations of the model on diverse datasets, demonstrated an impressive precision (0.5632), mean absolute error (MAE) (0.7610), hit rate (HR)@K (0.5720), and normalized discounted cumulative gain (NDCG)@K (0.4268). The average MAE across different datasets and latent dimensions was 0.608. Additionally, the model achieved remarkable cumulative accuracy and precision of 97.94% and 97.04% in performance, respectively, surpassing existing state-of-the-art models. This affirms the robustness and effectiveness of the proposed model in real-time enterprise scenarios.
Keywords: High utility itemset mining, Semantic sequential deep learning, MLP, Cosine similarity, Top-N HUI Recommendation
In recent years, software computing, big data analytics, and decentralized computing technologies have emerged. These technologies, in sync with gigantic digital data, have helped enterprises understand user behavior, consumer perceptions, and preferences to improve business decisions. Consumers’ purchase behavior and transaction analyses have helped enterprises understand supply chain demands, highutility items, and periodic demand patterns, thereby playing a decisive role in business communities, including e-commerce, manufacturing, and supply chain industries (e.g., the global value chain). However, identifying a set of high-utility items for the aforementioned digital data or transaction details, remains challenging. Advanced computing technologies, such as pattern mining [1-4], have enabled the identification of high-utility itemsets for business decisions. Typically, in sync with the business ecosystem, pattern mining technologies exploit existing transaction details to understand consumer preferences and socioeconomic behavior, thus helping enterprises improve their marketing mix decisions. They also help consumers identify the intended product or service that meets the respective demands [5]. In other words, the pattern-mining technique can help both enterprises and consumers with certain optimistically identified sets of products or items for corresponding use. In recent years, machine learning (ML) and artificial intelligence (AI) have played decisive roles in pattern mining-based recommendation systems [5]. An improved computing ecosystem called natural language programming (NLP) can also identify the target output by learning raw input patterns or transaction details [5]. These techniques exploit different input patterns and associated features to make prediction decisions [5], learning over different periodic and sequential patterns, as well as frequent itemsets.
In business ecosystems, pattern analysis methods have gained widespread attention for market basket analysis and business intelligence (BI). BI approaches exploit sequential or periodic transaction details (or data) to identify the frequent itemsets with high utility values. This helps in performing personalized predictions for users. In BI ecosystems, the identification of high-utility itemsets (HUIs) enables the segmentation of N demanding products and services. Classical approaches often exploit intercommodity associations and frequency of item purchases or transactions, to identify HUIs for further prediction [6-8]. Notably, an item is stated to be a frequent itemset when its frequency is greater than a predefined threshold, called the support value [9]. Different approaches have been proposed for frequent high-utility itemset mining (FHUIM); however, merely applying a predefined threshold over large nonlinear features or patterns cannot yield optimal accuracy for generalization. Techniques such as the Apriori [6][7] method determine frequent itemsets by assessing iterative level-based searches to identify HUIs. Notably, these methods employ the downward closure method, in which an a priori characteristic is applied to prune redundant or less redundant items. Apriori-based methods ensure that itemsets possessing a low support value do not become an HUI. However, the iterative estimation of the support value can be computationally demanding, especially over a large search space, which limits the robustness of Apriori-based methods. In recent years, several other approaches such as equivalence class clustering and bottomup lattice traversal (ECLAT) [2], frequent pattern-growth (FP-Growth) [3][10], and hyper-structure mining (Hmine) [11] were proposed to improve pruning and the associated data structure for HUI identification. These approaches that employ frequent itemset mining, only consider the frequency of the itemsets in relation to other items rather than their respective utility or co-occurrence probability. This limits their utility in contemporary business decisions, where identifying co-occurrence items is as important as identifying HUIs. The literature indicates that other details, such as transaction counts, co-purchased items, their frequency, and high profit value, can help identify HUIs in value-based itemset predictions [12]. Unlike Apriori or frequency-based solutions, high-utility itemset mining (HIUM) [13,14] techniques which employ both volume as well as profit per unit for HUI estimation, have gained widespread attention. These methods aim to improve both accuracy and computational costs to improve scalability [12]. Many state-of-the-art studies have applied the utility factor, which includes total and unit profits of the itemset, in estimating HUIs from a large search space based on transaction details. These approaches perform better than Apriori methods; However, their suitability for generalization to larger dynamic transactions is disputable [12-15]. Studies have revealed that pruning insignificant items from transaction search spaces can minimize computational cost and delay and improve learning-based prediction accuracy [12]. This prompted the development of transaction-weighted utility (TWU) which focuses on improving pruning, whereby the upper threshold is estimated to prune an itemset. To improve accuracy and reduce the complexity over large inputs, two-phase methods have been proposed [16-19], which are hypothesized to be more effective than single-phase solutions [20,21]. In two-phase methods, highly correlated items are first identified. Subsequently, their utility is assessed to label them as HUIs. Despite claims of higher accuracy, the associated computational cost remains a challenge, particularly in iterative database scanning and level-wise utility estimation [20]. The use of a list structure (utility list) performs better. However, the utility value remains proportional to the length of the itemset, making the computation more complex. To alleviate this, the concept of average utility was recently proposed, which focuses on reducing the impact of length on HUI prediction [22-24], applying the average support value to perform HUI estimation. Interestingly, none of these methods can exploit the itemset probability or contextual details among the sequential or periodic transactions in performing HUI itemset prediction.
In this paper, a novel convolutional sequential embedding metric-driven cosine-similarity-based multi-layer perception learning model is proposed for top-N high-utility itemset recommendations. Unlike traditional deep learning solutions, such as convolutional neural network (CNN)-based top-N HUIM models that apply local features for prediction, the proposed model exploits contextual or global features, including semantic features, over sequential transactions to perform top-N recommendations. First, semantic features are extracted from the transaction details encompassing the itemsets and their respective co-occurrence probabilities using the semantic CNN (SCNN) model. The extracted semantic features are processed by a multilayer perceptron (MLP), which retrieves the linear relationship among the itemsets and their corresponding co-occurrence probabilities. Subsequently, the cosine-similarity method is applied over the MLP-predicted linear associations to perform top-N HUI predictions. In MATLAB-based simulations over different datasets, the proposed HUIM model achieved a precision of 0.5632, mean absolute error (MAE) of 0.7610, hit rate (HR)@K of 0.5720, and normalized discounted cumulative gain (NDCG)@K of 0.4268. Additionally, it exhibited an average MAE of 0.608 on different datasets and latent dimensions, achieving a cumulative performance accuracy and precision of 97.94% and 97.04%, respectively. Relative performance characterization revealed that the proposed top-N HUIM model surpasses other state-of-art methods, including CNN, in different federated learning environments, which confirms the robustness of the proposed model and its suitability for real-time enterprise purposes.
The remainder of this paper is organized as follows. Section II discusses related works, followed by the research questions in Section III. Section IV presents the overall proposed model and its implementation. The simulation results and relevant inferences are provided in Section V. Section VI discusses the overall research contributions in the context of future scope. The references used in this study are provided at the end of the manuscript.
Pattern mining [4] methods were initially designed based on the frequency of itemsets [4]. Frequent itemset mining methods exploit threshold conditions such as support value and profit score to perform HUI estimation. However, applying standalone threshold-based pruning alone cannot yield a robust solution. A few improved methods such as ECLAT [2], FP-Growth [3], Apriori [4], HMine [11], and HUIM have been designed in recent years for HUI estimation. An Apriori-based itemset mining method was proposed in [6], where a level-wise search method is used to help estimate frequent itemsets. However, the computational cost associated with a large search space and iterative pruning make these methods laborious. The algorithm of FP-growth [3] applies a tree-structure to detect HUIs, whereby an FP-tree structure is first obtained by traversing across data space, searching for frequent itemsets over the tree structure. An improved FP growth method called HMine [11] has been developed, with an additional supplementary pointer-based hyperlink to represent items with a high frequency across the search space. ECLAT [2] was designed by using a vertical database structure called “Transaction ID list.” Unlike conventional methods in which a unit pointer is applied to detect each data element, the ECLAT method exploits the transaction ID to minimize the scanning cost, applying a support count value to each itemset to prune the search space for HUI estimation. Other methods [6,26-29] have used the support value and/or mean profit score for HUI estimation.
Unlike frequency-based itemset mining methods, HUIM methods have desirable performance [13,14] because of their ability to exploit and learn over a large transaction volume, and the corresponding profit makes them suitable for HUI estimation. Two-phase HUI methods [16] begin by identifying itemsets with higher frequencies, followed by an estimation of their utility for final HUI prediction. However, these methods have been criticized for their reliance on a standalone threshold when dealing with non-linear sequential data, which is often inadequate for HUI estimation [16]. Thus, the authors in [16] designed a transaction-weighted utility (TWU) function to minimize the iterative data-scanning cost. This TWU method was further enhanced in [17] by introducing a flexible upper threshold and the capability to skip high-utility itemsets, thereby improving the efficiency of the search space. The authors employed a twophase method with pruning called the isolated itemset discarding strategy (IIDS) to improve delay performance. Other tree-based methods include incremental high-utility pattern (IHUP) [18], HUP-tree [30], UP-Growth [19], UP-Growth+ [31], mining utility(MU)-Growth [32], and projection-based (PB) indexing approach [33]; however, the computational costs involved and lack of co-occurrence probability remain unexplored. In [20], an HUI miner was proposed using a utility-list (UL) data structure. The UL contains details of the itemset required for pruning, followed by HUI identification. However, despite claims of time efficiency, it cannot address HUI identification using correlated itemsets. In [21], HUI-Miner was designed to reduce the number of joins between the utility and its efficiency functions. The authors employed estimated-utility co-occurrence pruning (EUCP) on a matrix structure called the estimated-utility co-occurrence structure (EUCS). The EUCP encompasses the TWU values of the two item sets arranged in the EUCS matrix. The estimated itemsets were used to prune low-significance items without estimating the utility value. A number of pruning algorithms have been developed to enhance HUIM [34]; however, they fail to address semantic relatedness among itemsets over the search space. The authors in [35] designed efficient itemset mining (EFIM) with predefined upper bounds, considering subtree and local utility factors. To reduce scanning costs, they used the transaction-merging concept. The HMiner [36] model was applied using utility information storage with allied pruning. Approaches such as BAHUI [37], the HUIM-BPSO sign [38], MinHUIs [39], and FHM+ [40] have also been used for HUI estimation.
The two-phase average-utility (TPAU) method [22] applies an average utility-based upper threshold condition, whereby a level-wise search method is applied to enhance the time efficiency. The projection-based average-utility (PBAU) [23] method applies an indexing structure. By applying PBAU, a hard upper limit is defined using the prefix concept [23] to reduce search costs. In [41], a tree-based high-average-utility itemset (HAUI) mining method was designed. The HAUIGrowth model [42] was developed using a tree-based method that efficiently reduces iterative data scanning, and the HAUI miner was designed as a one-phase concept [43] by applying an average utility-based list structure. The efficient high average-utility pattern mining (EHAUPM) in [44] was designed by amalgamating two upper thresholds: looser upper-bound utility (LUB) and revised tighter upper bound (RTUB). Mining of high average-utility (MHAI) [45] retains suitable HUIs based on a high average-utility itemset (HAI) list structure. A closed high-utility itemset (CHUI) with DGU, REG, RML, and DGM was designed in [51] to retain decisive itemsets over the input data structure. In [52], CHUI-Miner was designed as a single-phase model by applying the EU-list structure. This reduces unexpected iterative search costs. The CHIU-Miner, called EFIM-closed [53], was developed with two strictly defined upper thresholds with forward-backward examination. This method uses local and subtree utility values to prune the search space. The CLS-Miner [54] was designed with supplementary coverage and LBP. Despite numerous efforts [46-50], no significant work has examined the probability of coexisting items as HUIs for top-N HUI predictions [55]. Van et al. [56] used FP growth to examine the association between available features to append new features with a certain threshold to perform HUI estimation. In [56], sequential-to-sequential learning methods were applied to top-N target balanced recommendations. In [57], a deep learning method was applied to top-N recommendations, considering the user’s interest level and frequency. A deep reinforcement learning model was used in [58] for top-N recommendations. Similarly, interest-related item set learning and similarity measures were applied to perform top-N recommendations [59]. In [60], a trust-aware sequential recommendation model that exploits frequency information was designed. Unfortunately, there has been no viable effort that considers sequential co-occurrence probability or the semantically connected co-occurrence feature to perform top-N HUI recommendations. This was the key driving force in this research.
This section discusses the proposed convolutional sequential embedding-driven cosine similarity-based MLP learning model for top-N HUI recommendations. As the name indicates, the proposed system comprises three key components: a semantic sequential convolutional encoding (SSCE) also called semantic CNN (SCNN), MLP, and cosine similarity for top-N HUI predictions. The SSCE model comprises a multidirectional filtering-assisted semantic embedding framework that learns over sequential input items or transactions to generate a semantic embedding matrix. In sync with the enterprise application environment, transaction details (user preferences) and allied frequent itemset patterns (buying patterns) are considered. In this manner, unlike conventional deep learning approaches [64-67] that apply local item-wise embedding metrics for learning, the proposed model employs both item-level and corresponding group-level (co-occurrence probability) information to perform top-N HUI prediction or recommendation. This approach enables the model to achieve a higher accuracy while maintaining low computational cost and delays during the scanning of search space (or feature space) to support estimation. An MLP network is applied in conjunction with an adaptive model optimizer (ADAM) learning model to perform training and obtain linear associations among the items, item frequency, and associated co-occurrence probabilities over the sequential transaction inputs. To refine the predicted results and increase accuracy, MLP was deployed in conjunction with the cosine similarity method, to help predict (accurately) the top-N HUIs for recommendation. The proposed model encompasses the following phases:
Semantic sequential convolutional encoding (SSCE),
Multilayer perceptron (MLP) learning
Cosine similarity-driven top-N HUI prediction.
The detailed discussion of these (functional) components is provided as follows:
Let the transaction data U have L consecutive itemsets. The objective of this work is to identify the top-T items with both high frequency and co-occurrence probability over sequential transaction data S^{u}. Here, the transaction dataset has input items S_{1}^{u}, ..., S_{|Su|}^{u}. Therefore, the proposed SSCE model first converts input transactions into an equivalent semantic word-embedding form. More specifically, the Word2Vec embedding method is applied to convert sequential input transactions into equivalent word-embedded metrics (P_{M}). This metric states the probability of occurrence of each itemset in the transaction data. Once semantic (word-embedded) metrics, also called semantic features, are obtained for the input transaction data, multidirectional convolutional filtering (MDCF) is applied to retrieve the contextual features over the itemsets along with the corresponding co-occurrence probability. Subsequently, the retrieved contextual details are used to perform learning using the MLP network, which, in conjunction with cosine similarity, predicts the top-N HUIs. A schematic of the implementation of the proposed model is shown in Fig. 1.
By applying the aforementioned Word2Vec semantic word-embedding method over the input sequential itemsets and their occurrences across the transaction data, an embedding matrix that possesses embedded information related to the itemsets and their occurrence probability is obtained. Here, the embedding matrix for the input sequential transactions is obtained from the semantic feature space by inputting L-traversed itemsets into a neural network. Let, i denote an item; then the corresponding embedding matrix representing the latent feature is obtained as
In addition to the above derived itemset embedding, the proposed model applies a supplementary embedding model to derive latent information for u users,
Let
where ⊙ refers to the inner product or multiplication function, whereas ⌀c(·) represents the activation function.
We applied a max-pooling layer to the extracted embedding metrics or vector c^{k} (3). This helps retain the high-resolution features with maximum values, retrieved over the features generated by the convolutional filter. Thus, the proposed method retrieves the set of vectors or significant features over the deployed n filters that eventually yield the output as
Similar to horizontal filter-driven feature extraction (4), we also applied vertical convolutional filters. Let,
The retrieved feature vector (5) shows that the result is the same as the weighted sum over L rows of the embedding metrics E, obtained by examining the weights
In (6), E^{l} represents the lth row of E. Thus, the vertical filters help in learning the embeddings of the L subsequent itemsets having the same value as the weighted sum, in identifying the latent information of L itemsets. Vertical filters generate an entity-level sequential itemset pattern by using weighted sums over the latent information of the traversed itemsets.
Traditional deep learning-based recommendation models use a single weighted sum to predict an itemset. By contrast, we used
Horizontal and vertical filters were used to extract the global features from the global average max-pooling layer, thereby applying the max function to estimate the global features (8).
The proposed model generates a final feature vector by applying a nonlinear-activation-driven fully connected layer. The output of the fully connected layer is given by (9).
In (9), W_{h} and W_{h} represent the weight metrices, whereas
As stated earlier, the features from both the horizontal and vertical filters (9) were projected onto a fully connected neural network. The MLP network was applied as a fully convolutional neural network (FCNN) to represent the nonlinear relationship between the itemsets and the associated cooccurrence probability. The deployed MLP network is a multilayer feed-forward neural network (MFNN) that learns nonlinear itemsets, aligning them with co-occurrence probability interactions. In this study, the combined embedding feature (9) was fed into the MLP to obtain the latent embedding feature in (10).
In (10),
In (11),
The MLP training model over latent metrics was designed based on the loss between the predicted and measured itemset relationships. In this work, the cost function (12) was used to train the network for the prediction of target itemsets q using cost function p (12).
In (12),
In this study, a normalized cross-entropy loss function (14) was applied to perform the learning.
In (14), max(R) denotes the highest rating of the itemsets across the input latent embedding matrix. We applied MLP to represent a nonlinear relationship between the itemsets and their co-occurrence (utility). In the MLP network, let the input embedding matrices for the itemsets be
In (15-16) the rectified linear unit (ReLU) is represented by the activation function f(*), whereas
Notably, unlike traditional top-N prediction models, where the learning model (e.g. MLP) predicts the top N selected items, the proposed model applies cosine similarity to the initially predicted items. This helps identify the optimal set of top-N items with high inter-item similarity and corresponding co-occurrence probability. This can be of great significance to enterprises, for highly accurate HUI prediction and inventory management. Thus, by applying this method, the proposed model performs top-N HUI predictions for enterprises or applications.
This study proposes a robust convolutional sequential embedding metric-driven cosine similarity-assisted multilayer perception learning model for top-N high-utility itemset recommendation. Unlike classical approaches, the proposed model exploits both sequential itemset frequency and co-occurrence probability information to perform top-N HUI predictions. First, the sequential transaction data are input into word embedding using the Word2Vec method. Subsequently, the retrieved semantic embedding matrix and multidirectional filters encompassing the horizontal and vertical filters are used to derive global features. The composite embedded matrix features are projected as input onto the MLP layer, which, in conjunction with cosine similarity, performs top-N HUI prediction and recommendations. The deep learning model was executed at an initial learning rate of 0.0001. The overall proposed model was developed using the MATLAB software tool, performing simulation over central processing units arranged using 8 GB RAM and a 3.2 GHz processor. To assess efficacy, different benchmark datasets were applied [61,62]. The dataset encompasses sequential transaction details and itemsets. The proposed model enables intensity estimation over sequential itemsets, as expressed in (18).
In (18), the numerator represents the frequency
In the above equations, N denotes the list of N predicted (search) itemsets (i.e.,
Table 1 . Simulated results over different datasets.
Dataset | d | Prec. | Recall | MAP | MAE |
---|---|---|---|---|---|
Tmall | 4 | 0.341 | 0.014 | 0.126 | 0.983 |
8 | 0.420 | 0.024 | 0.174 | 0.734 | |
16 | 0.451 | 0.042 | 0.204 | 0.832 | |
32 | 0.563 | 0.059 | 0.234 | 0.721 | |
64 | 0.498 | 0.059 | 0.293 | 0.701 | |
Gowalla | 4 | 0.174 | 0.039 | 0.199 | 0.863 |
8 | 0.192 | 0.059 | 0.201 | 0.723 | |
16 | 0.198 | 0.098 | 0.223 | 0.758 | |
32 | 0.210 | 1.235 | 0.224 | 0.634 | |
64 | 0.218 | 1.249 | 0.300 | 0.698 | |
Repeat_Buyer | 4 | 0.356 | 0.032 | 0.167 | 0.899 |
8 | 0.380 | 0.052 | 0.198 | 0.983 | |
16 | 0.404 | 0.159 | 0.201 | 0.799 | |
32 | 0.428 | 0.210 | 0.223 | 0.652 | |
64 | 0.439 | 0.245 | 0.247 | 0.678 | |
Kaggle | 4 | 0.319 | 0.087 | 0.199 | 0.857 |
8 | 0.334 | 0.042 | 0.199 | 0.699 | |
16 | 0.3452 | 0.073 | 0.259 | 0.694 | |
32 | 0.3569 | 0.099 | 0.299 | 0.621 | |
64 | 0.3452 | 0.139 | 0.301 | 0.6873 |
Among the state-of-the-art approaches considered for top-N itemset recommendation, a few methods [57-60] have applied deep learning methods for latent feature learning and prediction; however, state-of-the-art methods can be laborious and time consuming. For instance, the authors in [57] applied a CNN followed by a denoising autoencoder (DAE) to perform top-N recommendation, with the CNN applied for latent feature extraction, followed by fractional maximization and DAE-based top-N recommendation. The use of successive deep models can incur high computational costs, exhaust memory, and delay performance. In addition, in observing the empirical simulation results over different datasets, the proposed approach showed a relatively lower error value than \existing methods [57]. The highest precision obtained by DMLR-DAE [57] was 0.32. By contrast, the proposed method achieved the highest precision of 0.5632, which is higher than that of the other existing methods. The MAE obtained using the DMLR-DAE [57] was 0.611 at d = 10 and 0.609 at d = 30. In contrast, the proposed method achieved an average MAE of 0.7610 for the different benchmark datasets.
In addition, the hit rate was estimated for the number of target HUI predicted in the top N predicted lists. Mathematically, HR is measured using (22).
In (22), |T| denotes the number of itemsets and associated co-occurrence probabilities or interactions in the test set. In addition, the normalized discounted cumulative gain was applied to assess the hit position by assigning higher scores to hits (especially fort the top K ranks). Mathematically, NDCG@K is obtained as described previously (23).
In (23), r_{i} denotes the ranked relevance of the target item at the i-th position (hence, r_{i} = 1). Otherwise, r_{i} = 0. In addition, the root mean square error (RMSE) was measured using (24), where T states the total number of test ratings, whereas R^{s} (i, j) represents the real rating, with the measured or predicted rating being R^{s} (i, j).
The proposed model was simulated using embedding dimensions of k = {4, 8, 16, 32, 64}, where k represents the latent embedding dimension. The results (Table 2) infer that with increasing k, HR also increases. This implies that with increasing latent dimensions, both HR and NDCG increase. The mean HR ratio was 0.57923. In contrast, average NDCG was 0.4268 and RMSE was 0.9143. A higher HT rate indicates a higher accuracy and reliability of the solution, whereas, a lower RMSE indicates a better performance of the proposed model. In comparison with state-of-the-art methods [63], the results confirm the superiority of the proposed model for top-N HUI recommendations. The existing method [63] achieved an HR@64 of 0.5126; in contrast, the average HR@K of 0.5720, of the proposed method was significantly higher than that of the state-of-the-art methods. This confirms the robustness of the proposed model for top-N realtime HUI predictions.
Table 2 . Performance over the different latent embedding dimensions.
Latent Dimensions (d) | Data | HR@K | NDCG@K | RMSE |
---|---|---|---|---|
4 | Tmall | 0.453 | 0.283 | 1.409 |
Gowalla | 0.682 | 0.410 | 1.389 | |
Repeat Buyer | 0.543 | 0.312 | 1.004 | |
Kaggle | 0.459 | 0.321 | 0.953 | |
8 | Tmall | 0.486 | 0.299 | 1.091 |
Gowalla | 0.672 | 0.461 | 1.077 | |
Repeat Buyer | 0.578 | 0.532 | 0.997 | |
Kaggle | 0.613 | 0.523 | 0.096 | |
16 | Tmall | 0.482 | 0.299 | 0.987 |
Gowalla | 0.677 | 0.498 | 0.988 | |
Repeat Buyer | 0.578 | 0.487 | 0.902 | |
Kaggle | 0.689 | 0.512 | 0.904 | |
32 | Tmall | 0.501 | 0.374 | 0.938 |
Gowalla | 0.698 | 0.460 | 0.874 | |
Repeat Buyer | 0.582 | 0.490 | 0.921 | |
Kaggle | 0.600 | 0.498 | 0.184 | |
64 | Tmall | 0.512 | 0.377 | 0.880 |
Gowalla | 0.701 | 0.481 | 0.871 | |
Repeat Buyer | 0.564 | 0.499 | 0.911 | |
Kaggle | 0.510 | 0.412 | 0.910 |
Additionally, we compared the proposed model in terms of top-N recommendation accuracy. Table III presents the comparative results of different state-of-the-art techniques. The information in the confusion matrix was used to assess prediction accuracy.
Observing the results, clearly unlike traditional deep learning-based approaches [64-67], where feature extraction is based on CNN, in the proposed method, semantic feature extraction is followed by improved learning-driven feature extraction, which is subsequently processed using cosinesimilarity to obtain top-N HUI prediction. Using this approach, the proposed model refined the predicted (top-N HUI) output, achieving higher accuracy, which can easily be visualized in Fig. 4.
Similarly, in terms of cumulative precision, which is derivedfrom the confusion matrix, the proposed itemset top-N HUI prediction model yielded higher precision (97.04%) than the existing method [68] (87%), clearly indicating that the proposed model is more robust compared to the state-of-the art models for enterprise HUI prediction. Conclusions alongwith inferences are presented in the following section.
Table 3 . Accuracy Prediction.
Source | Techniques | Accuracy (%) |
---|---|---|
[64] | Federated NN, CNN | 71.68 |
[65] | CNN | 70.00 |
[66] | Random Forest, NNC and XGBoost | 88.84 |
[67] | NN, Logistic Regression | 89.00 |
[68] | Association rule as feature selection over CNN for top-N itemset prediction | 91.00 |
Proposed | convolutional sequential semantic embedding driven MLP with Cosine similaritybased top-N HUI recommendation | 97.94 |
Most existing HUIM methods often face limitations due to high computational costs, delays, and reduced accuracy when processing extensive sequential transaction data prevalent across various industries. Moreover, the pruning costs and lack of contextual details representing co-occurrence probability limit the efficacy of available state-of-the-art methods. Unlike HUIMs that rely on support value and unit price threshold for pruning, deep learning-based HUI identification is a better alternative. This approach, however, necessitates the exploitation of contextual details, including semantic sequential embedding features, to perform HUI prediction. Motivated by this, this study proposed a novel and robust convolutional sequential semantic embeddingdriven multi-layer perceptron learning environment, in sync with cosine similarity, to predict top-N HUI recommendations. In the proposed method, first, two filters are applied along the horizontal and vertical directions simultaneously in convolutional sequential deep learning, to extract the semantic embedding matrix over the transaction details. The use of multiple convolutional filters allows for the retention of a substantial amount of semantic information for further learning and classification. Using the extracted semantic features, the MLP neurocomputing model, which is designed using a ReLU regulation layer and ADAM non-linear optimization function, obtains the linear relationship among the itemsets available across the search space. The proposed MLP model was executed in conjunction with a cosine similarity function to predict the top-N HUI for further recommendations. The use of a semantic embedding matrix with MLP learning and cosine similarity measures helped refine the top-N HUI itemset predictions, which can be highly accurate for any enterprise solution(s). The proposed model exhibited the highest precision of 0.5632, MAE of 0.7610, HR@K of 0.5720, and NDCG@K of 0.4268. Additionally, it exhibited an average MAE of 0.608 over four different datasets, indicating robustness for real-time HUI predictions. The higher cumulative accuracy (97.94%) and precision (97.04%) confirm the efficacy and suitability of the proposed model for real-time enterprise solutions.
Table 1 . Simulated results over different datasets.
Dataset | d | Prec. | Recall | MAP | MAE |
---|---|---|---|---|---|
Tmall | 4 | 0.341 | 0.014 | 0.126 | 0.983 |
8 | 0.420 | 0.024 | 0.174 | 0.734 | |
16 | 0.451 | 0.042 | 0.204 | 0.832 | |
32 | 0.563 | 0.059 | 0.234 | 0.721 | |
64 | 0.498 | 0.059 | 0.293 | 0.701 | |
Gowalla | 4 | 0.174 | 0.039 | 0.199 | 0.863 |
8 | 0.192 | 0.059 | 0.201 | 0.723 | |
16 | 0.198 | 0.098 | 0.223 | 0.758 | |
32 | 0.210 | 1.235 | 0.224 | 0.634 | |
64 | 0.218 | 1.249 | 0.300 | 0.698 | |
Repeat_Buyer | 4 | 0.356 | 0.032 | 0.167 | 0.899 |
8 | 0.380 | 0.052 | 0.198 | 0.983 | |
16 | 0.404 | 0.159 | 0.201 | 0.799 | |
32 | 0.428 | 0.210 | 0.223 | 0.652 | |
64 | 0.439 | 0.245 | 0.247 | 0.678 | |
Kaggle | 4 | 0.319 | 0.087 | 0.199 | 0.857 |
8 | 0.334 | 0.042 | 0.199 | 0.699 | |
16 | 0.3452 | 0.073 | 0.259 | 0.694 | |
32 | 0.3569 | 0.099 | 0.299 | 0.621 | |
64 | 0.3452 | 0.139 | 0.301 | 0.6873 |
Table 2 . Performance over the different latent embedding dimensions.
Latent Dimensions (d) | Data | HR@K | NDCG@K | RMSE |
---|---|---|---|---|
4 | Tmall | 0.453 | 0.283 | 1.409 |
Gowalla | 0.682 | 0.410 | 1.389 | |
Repeat Buyer | 0.543 | 0.312 | 1.004 | |
Kaggle | 0.459 | 0.321 | 0.953 | |
8 | Tmall | 0.486 | 0.299 | 1.091 |
Gowalla | 0.672 | 0.461 | 1.077 | |
Repeat Buyer | 0.578 | 0.532 | 0.997 | |
Kaggle | 0.613 | 0.523 | 0.096 | |
16 | Tmall | 0.482 | 0.299 | 0.987 |
Gowalla | 0.677 | 0.498 | 0.988 | |
Repeat Buyer | 0.578 | 0.487 | 0.902 | |
Kaggle | 0.689 | 0.512 | 0.904 | |
32 | Tmall | 0.501 | 0.374 | 0.938 |
Gowalla | 0.698 | 0.460 | 0.874 | |
Repeat Buyer | 0.582 | 0.490 | 0.921 | |
Kaggle | 0.600 | 0.498 | 0.184 | |
64 | Tmall | 0.512 | 0.377 | 0.880 |
Gowalla | 0.701 | 0.481 | 0.871 | |
Repeat Buyer | 0.564 | 0.499 | 0.911 | |
Kaggle | 0.510 | 0.412 | 0.910 |
Table 3 . Accuracy Prediction.
Source | Techniques | Accuracy (%) |
---|---|---|
[64] | Federated NN, CNN | 71.68 |
[65] | CNN | 70.00 |
[66] | Random Forest, NNC and XGBoost | 88.84 |
[67] | NN, Logistic Regression | 89.00 |
[68] | Association rule as feature selection over CNN for top-N itemset prediction | 91.00 |
Proposed | convolutional sequential semantic embedding driven MLP with Cosine similaritybased top-N HUI recommendation | 97.94 |