Journal of information and communication convergence engineering 2022; 20(2): 79-89

Published online June 30, 2022

https://doi.org/10.6109/jicce.2022.20.2.79

© Korea Institute of Information and Communication Engineering

Component-Based System Reliability using MCMC Simulation

Sampa ChauPattnaik1*, Mitrabinda Ray 1, Mitalimadhusmita Nayak 2, and Srikanta Patnaik1

1Department of Computer Science and Engineering, Siksha ‘O’ Anusandhan (Deemed to be) University, Bhubaneswar 751030, India
2Department of Mathematics, Siksha ‘O’ Anusandhan (Deemed to be) University, Bhubaneswar 751030, India

Correspondence to : Sampa ChauPattnaik (E-mail: sampa.chaupattnaik@gmail.com, Tel: +91-88-9565-6648)
Department of Computer Science and Engineering, Siksha ‘O’ Anusandhan (Deemed to be) University, Bhubaneswar 751030, India.

Received: January 28, 2021; Revised: April 23, 2022; Accepted: May 10, 2022

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

To compute the mean and variance of component-based reliability software, we focused on path-based reliability analysis. System reliability depends on the transition probabilities of components within a system and reliability of the individual components as basic input parameters. The uncertainty in these parameters is estimated from the test data of the corresponding components and arises from the software architecture, failure behaviors, software growth models etc. Typically, researchers perform Monte Carlo simulations to study uncertainty. Thus, we considered a Markov chain Monte Carlo (MCMC) simulation to calculate uncertainty, as it generates random samples through sequential methods. The MCMC approach determines the input parameters from the probability distribution, and then calculates the average approximate expectations for a reliability estimation. The comparison of different techniques for uncertainty analysis helps in selecting the most suitable technique based on data requirements and reliability measures related to the number of components.

Keywords Component Based Systems, MCMC, System Reliability, Discrete Time Markov Chain

Software reliability is necessary for system developers to improve application performance. Delivering commercial-off-the-shelf (COTS) products is a critical challenge for all industries. Software reliability analysis at the architectural level is based on the following models: state-based [1-5], path-based [21,29], and additive [32].

Software reliability estimation occurs at the code and design levels. In architecture-based software reliability analysis, the input parameters include the software architecture of the application, transition probabilities between components, reliability of individual components, and average execution time of the components [6]. During the design phase, the major challenge is that the software architecture and component parameters may not be well known. Thus, an uncertainty analysis is required for the architecture-based software reliability model. The goal is to investigate how the inaccuracy of the operational profile [7] directly influences the transition probabilities between the components. The failure of the components is unknown and depends on the test procedures. Because the failure time data are uncertain, the system reliability computed from the parameters is also uncertain. The reliability growth of a component depends on its fault detection rate. A fault detection rate may be given by one of the software reliability growth models [8-11], and the location of faults that caused these failures can then be identified based on several accurate methods.

Motivation: Software reliability prediction based on architectural models depends on i) appropriate model considerations and ii) accurate parameter values. Practically, there is considerable uncertainty in parameters because uncertainty occurs in the estimation of the operational profile and component reliabilities. A major challenge in reliability prediction at the architectural level is the presence of uncertainties owing to the lack of information in relation to the proposed artifact, such as operational profile, implementation details, and failure behavior.

The objectives of this paper are:

  • To determine the uncertainty parameters that arise through the model such they are properly represented in the reliability analysis.

  • To monitor the uncertainty results such that they help decision-makers make further decisions.

  • To analyze the system reliability by considering the effect of the individual reliability of a component using some uncertain manner.

We present uncertainty in architecture-based software reliability based on a Markov chain Monte Carlo (MCMC) simulation and moment techniques. The methodology addresses the parameter uncertainty problem and shows its effect on system reliability.

The remainder of this paper is organized as follows. In Section II, we summarize the background study. This explains the different approaches to uncertainty analysis for component-based systems. Section III describes the uncertainty analysis technique based on the method of moments, Monte Carlo (MC) simulation, and Markov chain Monte Carlo (MCMC) simulation. Section IV explains the result analysis using different uncertainty methods in architecture-based software reliability analysis. Section V provides a comparative study of different methods for uncertainty analysis, and the paper concludes in Section VI.

In the last decade, several architecture-based software reliability models have been proposed [8, 12, 13, 18, 35]. In the early phase, software reliability analysis can identify the critical components that improve the reliability of an application. Software reliability estimation in the design phase causes uncertainty in the component reliabilities. Several researchers [14-24] have used different techniques to analyze uncertainty in architecture-based software systems. To evaluate the overall system reliability, we used different models, such as Markov chains and fault trees. Parameters such as component reliabilities, transition probabilities between two components, and average execution time of components are used in these models, generally collected from field data, systems with similar function data, or by guessing. The exact values of the parameters may be unavailable at the design stage (as coding has not begun). Hence, the analyst can formulate a relative estimation for the values of the parameters by considering the estimation from the actual system implementation to validate the assumption prepared in the design stage. Therefore, the values of these parameters may be inaccurate. This leads to uncertainties in system reliability.

Reliability assessment occurs during the early stages of the software development life cycle using software architecture. However, the prediction of component reliabilities at the architectural level is difficult because uncertainties arise in the system and its individual components [15]. authors considered a component-based robotics test bed, the controller component of SCR over, which is a NASA JPL mission data system (MDS). They identified different uncertainty factors of components, such as the recovery and failure probabilities. Their results showed that the reliability of the component increased with the recovery probability. In addition, if the failure probability increases, the component reliability decreases.

For architecture-based reliability estimation, accuracy may depend on parameters such as environmental factors and system usage. Researchers have handled this problem by integrating uncertainties in architecture assessment models and analytically solving them with simulations. This method illustrates the mean and variation of an attribute. The case study used was on the anti-lock braking system (ABS) of a car [25]. Architecture-based software reliability analysis can identify the critical components early, which leads to a cost-effective reliability improvement of the application. However, at this phase, estimating architectural and component parameters is not entirely possible with certainty.

The problem of uncertain component reliabilities has occurred in software application reliability estimation [26]. The authors considered the levels of confidence in architectural parameters using minimal test and simulation data based on the principle of multinomial distribution confidence intervals for reliability estimation. K. Go et. al. [27] conducted a large-scale case study on architecture–based software reliability evaluation to determine potential limitations and challenges for future analysis. The study of uncertainty in architecture–based software reliability found that the MCapproach is more effective than the moments technique. The MCtechnique results indicate the following: (1) fewer parameters yield the most variance in system reliability and (2) the reliability of components has a greater effect on system reliability than the probability of transition for a given operational profile.

Software reliability models consist of several parameters that are typically determined from the test data of the component. As point estimates concentrate on random data variations, uncertainties in the measured parameters were observed. The parameter causing uncertainty underestimates the uncertainty in terms of overall system reliability [16]. Here, the authors aimed to measure the uncertainties with their correlated parameters in the software reliability modeling of a single component and multiple components. A major problem in the uncertainty analysis of software reliability modelling is the lack of available failure data. Using the views of experts and historical data from earlier projects will help resolve this problem. An uncertainty analysis was performed by merging the concept of maximum entropy (MEP) with the Bayesian method. System-level uncertainty analysis can be conducted using the MCmethod to demonstrate the impact of uncertainty parameters on many system-level components.

In component-based software development systems, an inadequate selection of components results in low-quality software. The authors disagree with the assessment methods in which explicit uncertainty methods are represented using probability distributions. They provide details of the Bayesian model used to capture the uncertainties in the simultaneous assessment of two attributes, thereby confining the dependencies between them [28]. To handle the uncertainty in the system reliability, a MCsimulation method is used [29], where reliability is determined by the variance, and the accuracy increases by increasing the sample size. To estimate the reliability of a system, the method of moments is used for uncertainty analysis in architecture-based software reliability models [30]. The method of moments shows that the reliability evaluation occurs by the moments of component reliabilities, where the accuracy can be improved by considering the high level of Taylor expansion. The software consists of components and modules.

Software reliability assessment is the primary area of software-based systems. Software reliability estimation is based on historical data and a presumed distribution curve, which is inclined to uncertainties. A fault tree [31] is used for reliability estimation in component-based systems; and fuzzy set theory is used in fault tree analysis to quantify the basic uncertainty results. In recent years, for complex architecture systems, such as flights, a highly reliable system is required for safety purposes [32]. This leads to uncertainty in failure occurrences. The confidence interval and probability density functions were used to compute the uncertainty in the system reliability variance. The evaluations show that the uncertainty varies with the system or execution time.

The behavior of the system is affected by unavoidable components, which are considered uncertain. Uncertainty is categorized into two types: i) aleatory uncertainty: stochastic process behavior (i.e., randomness from inherent variability in the system) and ii) epistemic uncertainty: vagueness owing to a lack of knowledge and information about the system [33]. The causes and aspects of uncertainty have been studied in various applications such as engineering risk modeling and reliability assessments. Although many different types of uncertainty exist, they are usually categorized as aleatory or epistemic. Aleatory uncertainty, also known as stochastic uncertainty, arises owing to the randomness of the behavior of a software system. The probability distribution is commonly used to solve the aleatory uncertainty problem with sufficient data. For example, an application using a reliability-based design is based on probability theory. Epistemic uncertainty occurs because of a lack of knowledge or information regarding a software system, particularly during the design of a complex system. It is not based on probabilistic theory.

Methods such as interval analysis, possibility theory, and evidence theory have been used to model epistemic uncertainty. This paper is based on epistemic uncertainty because we apply uncertainty analysis during the design phase and sufficient data are unavailable for estimating system reliability. Epistemic uncertainty analysis is based on probability theory as well as fuzzy sets. This paper contains probability distributions, such as beta and gamma distributions.

To conduct reliability estimation with uncertainty analysis in a system, the following steps are followed.

  • Build the software architecture obtained from a specification and design document.

  • Estimate the component reliability

  • Design the software model (using the software growth model or fault injection)

  • Use the uncertainty techniques for the uncertainty para- meters (component reliability, transition probability, etc.). I.e., in this step, different uncertainty methods (method of moments, MC simulation, or MCMC simulation) are applied.

  • Obtain the system reliability.

Software reliability is estimated using an architecture-based model, and we include the dynamic behavior of the software architecture, i.e., a component dependency graph (CDG) [6], transition probabilities of two components, and nature of software failure (reliability of components or failure rate). For a given application software, the uncertainty in software reliability is caused by these parameters. Software architecture is defined as the interaction of different components. We used a path-based model to represent the architecture–based software reliability model [8, 15, 16, 26]. The path-based method uses the CDG. The components are represented as nodes, and the interactions between the components are represented as edges. Transfers between components are assumed to be a discrete-time Markov chain (DTMC) with transition probability Pr = [prij], where prij is the transmission from component i to component j. Component failure behavior estimates the reliability of each component. (CRi).

Several methods have been used to estimate component reliability. To develop component failure data, software reliability growth models (SRGM) [8, 15, 16, 26, 34] were applied to each component obtained during testing. Software reliability growth models cannot always be used because of the scarcity of failure data. Another manner of estimating the reliability of a component is to use a fault injection technique [13], which isa simulation basis. Therefore, the estimation of component reliability estimated may be inaccurate, which stimulates the use of uncertainty analysis. The methodology used for uncertainty analysis was obtained using a software architecture with the failure behavior of the components.

In this study, we used a path-based model [35] for reliability estimation. For the reliability assessment, we considered the software architecture with failure behavior. The reliability of a system is the probability of reaching the terminal component through the execution path along the components. The overall system reliability is calculated as follows:

SR=m1,nCRn,

where, m= k=0qk= IDq1. This denotes the (1,n)th value of matrix m, i.e., reaching the end component n by starting from component 1. In the DTMC method, the transition probability is represented by Pr=qc01, where q = (r - s) × (r - s) is a sub-stochastic matrix (minimum one-row summation <1), s × s is the identity matrix ID=1001, 0 = s × (r - s) is the matrix of 0s, and c = (r - s) × s. In addition, a Markov process with r states has s absorbing states.

Several techniques are used for uncertainty analysis. Here, we present some techniques, such as the method of moments [30], MC simulation [29], and MCMC simulation method [36].

A. Moments

This is an approximate approach that generates moments of system reliability from the moments of component reliabilities. The component reliability estimation using this method is inaccurate because of the higher-order Taylor series expansion.

For method of moments:

  • a) The link between system reliability (SR) and component reliabilities (CR1, CR2, ..., CRn) is represented by SR= f(CR1, CR2, ..., CRn) using the path-based model.

  • b) The system reliability is determined using Taylor series.

  • c) The moments are applied for the component reliabilities and transition probabilities.

  • d) The approximate mean and variance of the system reliability are calculated using the parameter moments using the given equations.

The method of moment is expressed as a Taylor series function. Thus, the 1st order moment is given by the equation

SR»r0+ i=0nriCRi-E CRi + i=1nj=1nrp ij Pr ij -E Pr ij ,

where, SR denotes the reliability of the components, and E[CR] is the expected reliability of components.

r0=fECR1, ECR1, ECR1,  ECR1, ri=CRCR1CRi=ECRi, for i = 1, 2, ...n, and rpij=CRPrijCRi=ECRi,Pri,j=EPri,j for i = 1, 2, ...n

The mean of the system reliability is E [CR]≈r0


Then, the variance of the system reliability is as follows.

VCR=σ2 i=1nri2VCRi+ k=1n i=1nrpij2VPrij+2 k=1n i=1n j=i+1nrpkirpkjCov Pr ki , Pr kj

The variance estimation shows the confidence in system reliability. Thus, if the variance decreases, the overall system reliability increases. Now, the efficiency of the expected reliability and variance of reliability is improved using high-order Taylor expansions.

B. Monte Carlo Technique

MCsimulation is a fairly accurate method for estimating system reliability. This method represents a well-defined distribution of the parameters of the software reliability model. The MCtechnique for a component-based system is as follows.

Algorithm: Uncertainty Analysis using MC for a Component Based System

Input: component reliability of the ith component (CR (i)), transition probability between two components (Prij), average execution time of n components; n: number of components

Output: System Reliability

  • Initialize CRi =X[i].

    X[i]= random variables, X ∈ [0,1]

  • Identify the system reliability using a path-based model (PR) with input parameters CR(i) and Prij:

    PR=CR1× i=2 n1CRi Pr 1,i ×CRn j=2 n1pr1,j×prj,n

  • Calculate the mean for each component (CRi) using the formula m=IDq1= u=0qu, where m is the mean number of visits from component I to component j, ID is the identity matrix, and q is the transition matrix explained below.

  • Calculate the variance of each component CRi using the formula v = m(2mdig - ID) - msqr , where v is the variance of each component, mdig is the diagonal of matrix m, and msqr is the square of the matrix m.

  • Determine the system reliability:

    SR= j=1 n1CRjm1,j+12CRjm1,jlogCRj2ν1,jCRn,

    where, m1,j and v1,j are the visit mean and variance, respectively, of each component from component 1 to component j.

  • Assign probability distributions to Prij and RCi. (The pro- bability distributions of the input parameters are assigned based on theoretical assumptions. Here, we use the beta and gamma distributions for the component reliability and transition probability, respectively.)

    fCRi=Γxi+yiΓxi+ΓyiCRixi11CRiYi1

    fpri1,pri2,pri3,.....prin=ΓAi1+Ai2++AinΓAi1+ΓAi2+.....Γ..in) j=1nprijAij1

  • Repeat Steps 3, 5, and 5 with the new values of CR (i) and Prij obtained from the probability distribution.

  • Repeat Steps3, 4, 5, and 6 until the required number of values is produced to obtain a lower variance and better system reliability.

    MCsimulation process

    ** i= no. of column and j= no. of rows

MCsimulation is a method for determining the probability of possible outcomes. Predict a model is difficult owing to the use of random variables. This is also referred to as a multiple probability simulation. MCMC is subset of MC. Thus, MCMC is an MC, but not all MCs are MCMC. The main objective of the MCMC method is to build a Markov chain. The Markov chain is produced using independent samples (with a negative correlation) to reduce the variance.

C. Markov Chain Monte Carlo

MCMC is an approximation technique for system reliability estimation. Therefore, the objective of this simulation was to use Bayesian inference [37] for posterior distributions. MCMC allows the user to approximate aspects of the posterior distributions. The MCMC method produces MCsimulations depending on the Markov property, and the simulation convergence point obtained is the posterior distribution [36]. The MCMC process is stochastic. A stochastic process {s(T)} has a Markov property (Mp). The Markov property is represented as Pr{s(T + 1) = n|s(0) = 0, s(1) = ...s(T) = m) = Pr{s(T + 1) = n|s(T) = m} for T = 0, 1, 2, .... This expression show that state s at time T + 1 only depends on the current state s at time T, and is not dependent on past states s(T−1), ..., T(1).

Algorithm: Component Based System Uncertainty Analysis using MCMC

Input: component reliability of the ith component (CR (i)), transition probability between two components (Pri,j), average execution time of n components

Output: variance of each component (Ci), System Reliability

  • Initialize CRi =X[i]

    X[i]= random variables, X ∈ [0,1]

  • Identify the system reliability using the path-based model (PR) with input parameters CR(i) and prij:

    PR=CR1× i=2 n1CRi Pr 1i ×CRn j=2 n1pr1j×prjn

  • Calculate the mean for each component (CRi) using the formula m=IDq1= u=0qu, where m is the mean number of visits from components i to j, ID is the identity matrix, and q is the transition matrix explained below.

  • Calculate the variance of each component CRi using the formula v = m(2mdig - ID) - msqr ,where v is the variance of each component, mdig is the diagonal of matrix m, and msqr is the square of matrix m.

  • Determine the system reliability

    SR= j=1 n1CRjm1,j+12CRjm1,jlogCRj2ν1,jCRn,

    where, m1,j and v1,j are the visit mean and variance, respectively of each component visit from component 1 to j component.

  • Apply the Markovian property discussed above to the transition probabilities (Prij). We consider n-step transition probabilities for computing these transition probabilities to form a Markov chain.

    Prijn= k=0MPrikm×Prkjnm  i=0, 1, 2, ...M;

    j=0, 1, 2 ...M; and any m=1, 2,..., n-1;

    n=m+1, m+2, ...

  • Apply the MCalgorithm to estimate the system reliability.

  • Repeat Steps 3 and 4 to determine a convergence posterior distribution.

We used the Chapman–Kolmogorov equations for the n-step transition probabilities [19].

We discuss the uncertainty analysis of the components with a waiting queue simulator application [6] as a case study. The architecture of the application consists of the following components: an ArrivalGenerator, EventList, ServiceFacility, ScheduleManager, QueuingFacility, and Measurement recorder. The interactions between the components were evaluated using the scenarios. The architecture of the waiting queue application is represented by a graph called the component dependency graph (CDG). A CDG consists of components and the transition probabilities between the components. The interaction between the components forms possible execution paths from the starting component to the end component. Thus, it follows a path-based model. Here, we consider the graph traversal with three structures: sequences, branches, and looping.

Fig. 1 shows a CDG graph represented with nodes and edges. The nodes and edges consist of information such as component name, component reliability, transition probability from one component to another, average execution time of each component, and transition reliability from one component to another. Assuming each component’s reliability is 1, the average execution times between components are shown in Table 1. The average execution time was evaluated using the following formula:

Table 1 . Reliabilities and average execution times of components

Index (CI)Reliability (CRi) (Assuming 1 for all components)Name (CNi)Average Execution Time (AvgETi)
11Schedule_Manager5
21Arrival_Gen2
31Service_Facility7
41EventList14
51QueuingFacility5
61Measurement0.8
71End0

Fig. 1. CDG of the queue management application.
AETi= i=1 sPr sk×T CRi CRiSk,

Where AETi is the average execution time of each component CRi, Prsk is the probability of executing scenario sk, T(CRi) is the execution time of CRi, i.e., the total number of active times in the execution of scenario sk. Table 2 lists the transition probabilities between the components of the waiting queue system.

Table 2 . Transition Probability between Components (Ci and Cj)

Pr12 = 0.05Pr13 = 0.215Pr14 = 0.475Pr15 = 0.26
Pr21 = 1
Pr31 = 0.1787Pr34 = 0.215Pr36 = 0.0363Pr37 = 0.57
Pr41 = 0.5725Pr43 = 0.215Pr45 = 0.2125
Pr51 = 0.3575Pr54 = 0.2125Pr57 = 0.43
Pr63 = 1


The values in Table 2 were obtained from the CDG in Fig. 2. The transition probabilities between two components (pri,j) are evaluated as follows. It is the probability of transitioning between two components (Ci, Cj), computed as the interactions between two components during the reliability analysis [6]. The sum of transaction probability from any component is always 1, i.e., d=1nPrdn=1.

Fig. 2. Path-based structure from the CDG for the waiting queue application.

Fig. 2 shows the corresponding DTMC-based execution path from starting component C2 to end component C7. We consider the structure based on the architecture of the application shown in Fig. 1. Fig. 2 consists of 3 types of path structure: a) sequence, b) branching, and c) looping. Hence, we apply the path reliability [35] for Fig. 2 and estimate the system reliability as follows. Fig. 2 obtains the expression for system reliability using the path-based approach, and its expression is as follows:

PR=CR2×CR1Pr211Pr12×Pr23×CR3Pr21×Pr231Pr23×Pr32×Pr36×CR6Pr21×Pr23×Pr361Pr63×Pr36×Pr34×CR4Pr21×Pr23×Pr36×Pr341Pr63×Pr23×CR5Pr21×Pr23×Pr36×Pr34×Pr451Pr63×Pr23×CR7Pr21×Pr23×Pr36×Pr34×Pr451Pr63×Pr23

The above expression is used in the method of moments to compute the mean and variance of the system reliability.

We demonstrate that the uncertainty analysis for the application has six components, and its architecture is described by the DTMC presented in Fig. 2. To compute the method of moments using Taylor series, we consider the nonzero transition probabilities Prij given in Table 2 as the mean values. Table 3 shows the mean component reliabilities E[Rei] and variance of the component reliabilities V[Rei]. We found that the uncertainty of component reliabilities results in uncertainty in software reliability. For this reason, we used the mean values of the component reliabilities and variances shown in Table 3 in the equations describing the method of moments in Section III. We observe from Table 4 that, if Taylor’s expansion order is greater, then the accuracy increases, and the variance is less. Thus, the system reliability increases. This indicates that, if the variance is less, the system is more reliable.

Table 3 . Mean and variance of the components

Component Index(CI)Expected components reliability E[Rei]Variance of components V[Rei]
10.98090.0565
20.98910.0240
30.98920.0324
40.97310.0041
50.98930.1230
60.98900.3450
70.9700.0565

Table 4 . Results of the mean and variance of the system reliability for the waiting queue

1storder Taylor expansion2nd order Taylor expansion
mean calculation value0.7780.758
variance calculation value0.000223620.0002162


A. Waiting Queue Management Case Study based on MC Simulation Method

The uncertainty analysis for the case study evaluates the system reliability using the formula SR = f(CRi, Prij). We obtained the results by varying the component reliability from 0.95 to 0.98, assuming that E[CR1] = E[CR7] = 1 and V[CR1]=V[CR7]=0. From Fig. 2, we obtain the mean of the component reliability and variance of each component’s reliability. The system reliability was 92%. Table4 presents the evaluated values of E[CRi] and V[CRi] without the MC simulation method.

Similarly, applying the MCmethod for the component reliability varies in the range [0.95, 0.98]; the mean of the component reliability and variance of the component reliability are described in Table 6. The MCmethod was executed several times (approximately 100 times), and we obtained an overall system reliability of 93%. The MCmethod was described in Section III. Hence, the values of alpha(x) = 10 and beta(y) = 2.675 were set for further calculations.

Table 5 . Results for mean and variance of each component’s reliability

Component #Mean of component Reliability E[CRi]Variance of Component Reliability, V[CRi]
11.0000
20.98280.9496
30.82680.4239
40.71300.9769
50.84310.5565
60.74610.3645
71.0000

Table 6 . Results for mean and variance of each component’s reliability using MC Simulation

Component #Mean of component Reliability E1[CRi]Variance of Component Reliability, V1[CRi]
11.0000
20.98370.9823
30.89970.4079
40.96410.9515
50.93350.5197
60.80210.3401
71.0000


The results also show that if the variance of the components is small, the system reliability increases.

B. Example of MCMC Method

The case study used for the MCmethod was also used for the MCMC method. Here, the MCMC applied for component reliability varies in the range [0.95 to 0.98], and the results are discussed below. Table7 shows the mean and variance of component reliability using the MCMC method. The n-step Markov chain is important for defining the probability of every component of a system at a random time. We start with any component as well; we reach the same probability distribution with a higher system reliability. The MCMC procedure consists of two steps.

Table 7 . Results for mean and variance of each component’s reliability using MCMC simulation

Component #Mean of component Reliability E[CRi]Variance of Component Reliability, V[CRi]
11.0000
20.23440.0234
30.00950.0093
40.00260.0021
50.00140.0014
60.24780.0114
71.0000

  • Use the Markov chain process (for this n-step transition probability).

  • Apply the MCsimulation method.

  • The overall system reliability obtained using the MCMC method is 96.8%, i.e., ≈ 97%.

The suitable selection of the method used for the analysis of uncertainty in software reliability for a given application depends on the requirements of the data, acquired reliability estimate, and performance of the solution. In this section, we explain the three methods used for the uncertainty analysis. We found that the method moment provided a system reliability of 92%. The MCsimulation showed that the overall system reliability was 93%. Similarly, the MCMC simulation provided a system reliability of 97%, with a lower variance than that of the MCmethod. Table 8 shows the variance of each component using the MC and MCMC methods. We found that the variance of each component decreased in the MCMC method compared to the MC method. Method of moments does not require a probability distribution. In method of moments, random numbers are not required. To increase the accuracy of method of moments, a high-order Taylor function is used.

Table 8 . Comparison results of variance of each component using MCMC and MCsimulation

Component IDVariance of components using MCmethodVariance of components using Markov Chain method
100
20.98230.0234
30.40790.0093
40.95150.0021
50.51970.0014
60.34010.0114
700


MCsimulation increases the accuracy by expanding the simulation numbers randomly generated. MC simulation includes a collection of reliability methods, such as moments, probability distributions, system reliability, and lower variance, to improve confidence in the system reliability estimation. In the MCsimulation method, uncertainty arises because of the operational profile and reliability of the components. Compared with the MCsimulation method, the MCMC simulation is better. In the MCMC simulation, the Markov chain was used for the probability distribution. This enables the quantification of the variance and accuracy of the reliability system estimation is high. This implies that the results obtained from different uncertainty methods help decision makers to make better decisions.

From [6], we considered the component dependency graph for the calculation of system reliability using the MCMC technique. In [6], the component reliability, average execution time, and transition probability between two components from the scenarios of the application were evaluated. However, we considered the evaluation of the mean and variance of each component based on the random reliability of each component. (In [6], the reliability of each component was fixed at 1). Table 9 lists comparisons between the results from previous studies and the proposed method.

Table 9 . Comparisons with previous studies

Sl. No.InvestigatorTitleInput parametersApproachFindings
1.[6]A scenario-based reliability analysis approach for component-based software.Component dependence graph, component reliability, average execution time, and transition probabilityThey suggest an application reliability with a scenario-based reliability analysis algorithm using the component dependency graphThis method helps discover the critical components and interfaces, as well as study the sensitivity of the application reliability to these components.
2.Proposed ModelComponent Based System Reliability using MCMC SimulationComponent dependence graph, component reliability, average execution time, and transition probability.Evaluation of the application reliability using the mean and variance of each component. The MCMC simulation method is then applied.This technique is used to quantify the variance of each component, and the estimated system reliability accuracy is high.

Architecture-based software reliability methods use several models (such as Markov chains and fault trees) to predict the reliability of an application. We analyzed the uncertainty in parameters, transition probabilities between components, and the reliability of individual components that propagate the software reliability estimation. Using these methods, we obtain the values for the mean and variance of the system reliability, which are considered for the uncertainty owing to their operational profile and component reliabilities. We presented the waiting queue management application architecture and its reliability calculation method with input values such as component reliability and transition probability between components. The estimated value of the system reliability using MCMC simulation provides more information than the MC simulation, or even method of moments. Consequently, the system has a higher confidence in reliability estimation with a smaller variance. In the MCMC method, owing to the “memory lessness” property, knowing the history of the process does not improve future predictions. The system reliability was analyzed by considering the uncertainty of the component, as shown in Table 3. From this, we can conclude that before applying the uncertainty measure, the reliability is 0.93, and by applying the uncertainty method to the components, the system reliability changes to 0.92 for method of moments, 0.93 for MC, and 0.97 for MCMC. Thus, we conclude that the overall system reliability is insignificantly affected, even if the individual components are subjected to uncertainty.

We consider that the uncertainty analysis of software reliability is important, primarily for predicting software reliability during the analysis stage in the life cycle. Sensitivity analysis provides intuition about the input uncertainty that controls the output uncertainty towards effective resources, which can be performed on the component, and transition probabilities can be considered in the future scope of this paper.

  1. R. C. Cheung, A user-oriented software reliability model, , IEEE transactions on Software Engineering, vol. SE-6, no. SE-2, pp. 118-125, Mar., 1980. DOI: 10.1109/TSE.1980.234477.
    CrossRef
  2. B. Littlewood, Software reliability model for modular program structure, IEEE Transactions on Reliability, vol. R-28, no. 3, pp. 241-246, Aug., 1979. DOI: 10.1109/TR.1979.5220576.
    CrossRef
  3. J. -C. Laprie, Dependability evaluation of software systems in operation, IEEE Transactions on software engineering, vol. SE-10, no. 6, pp. 701-714, Nov., 1984. DOI: 10.1109/TSE.1984.5010299.
    CrossRef
  4. P. Kubat, Assessing reliability of modular software, Operations research letters, vol. 8, no. 1, pp. 35-41, Feb., 1989. DOI: 10.1016/0167-6377(89)90031-X.
    CrossRef
  5. S. S. Gokhale, and K. S. Trivedi, Reliability prediction and sensitivity analysis based on software architecture, in Proceedings of the 13th International Symposium on Software Reliability Engineering, MD, USA: Annapolis, pp. 64-75, 2002. DOI: 10.1109/ISSRE.2002.1173214.
    CrossRef
  6. S. Yacoub and B. Cukic and H. H. Ammar, A scenario-based reliability analysis approach for component-based software, IEEE transactions on reliability, vol. 53, no. 4, pp. 465-480, Dec., 2004. DOI: 10.1109/TR.2004.838034.
    CrossRef
  7. J. D. Musa, Operational profiles in software-reliability engineering, IEEE software, vol. 10, no. 2, pp. 14-32, Mar., 1993. DOI: 10.1109/52.199724.
    CrossRef
  8. A. L. Goel, Software reliability models: Assumptions, limitations, and applicability, IEEE Transactions on software engineering, vol. SE-11, no. 12, pp. 1411-1423, Dec., 1985. DOI: 10.1109/TSE.1985.232177.
    CrossRef
  9. K. Goseva-Popstojanova and A. P. Mathur and K. S. Trivedi, Comparison of architecture-based software reliability models, in Proceedings of the 12th International Symposium on Software Reliability Engineering, Hong Kong, China, pp. 22-31, 2001. DOI: 10.1109/ISSRE.2001.989455.
    CrossRef
  10. M. R. T. Lyu, Software reliability theory, in Encyclopaedia of Software Engineering. DOI:10.1002/0471028959.sof329.
  11. J. D. Musa and A. Iannino and K. Okumoto, Software reliability, in Advances in computers, vol. 30, Academic Press Inc., pp. 85-170, 1990.
    CrossRef
  12. S. S. Gokhale, Architecture-based software reliability analysis: Overview and limitations, IEEE Transactions on dependable and secure computing, vol. 4, no. 1, pp. 32-40, Jan-Mar., 2007. DOI: 10.1109/TDSC.2007.4.
    CrossRef
  13. A. L. Goel, and K. Okumoto, A Markovian model for reliability and other performance measures of software systems, in 1979 International Workshop on Managing Requirements Knowledge (MARK), New York: NY, pp. 769-774, 1979. DOI: 10.1109/MARK.1979.8817248.
    CrossRef
  14. S. K. Chandran and A. Dimov and S. Punnekkat, Modeling uncertainties in the estimation of software reliability-a pragmatic approach, in 2010 Fourth International Conference on Secure Software Integration and Reliability Improvement, Singapore, Jun, pp. 227-236, 2010. DOI: 10.1109/SSIRI.2010.22.
  15. L. Cheung, and L. Golubchik, and N. Medvidovic, and G. Sukhatme, Identifying and addressing uncertainty in architecture-level software reliability modelling, in 2007 IEEE International Parallel and Distributed Processing Symposium, CA, USA: Long Beach, pp. 1-6, Mar., 2007. DOI: 10.1109/IPDPS.2007.370524.
  16. Y. S. Dai, and M. Xie, and Q. Long, and S. -H. Ng, Uncertainty analysis in software reliability modeling by bayesian analysis with maximum-entropy principle, IEEE Transactions on Software Engineering, vol. 33, no. 11, pp. 781-795, Oct., 2007. DOI: 10.1109/TSE.2007.70739.
    CrossRef
  17. W. W. Everett, Software component reliability analysis, in Proceedings 1999 IEEE Symposium on Application-Specific Systems and Software Engineering and Technology. ASSET'99 (Cat. No. PR00122), TX, USA: Richardson, pp. 204-211, 1999. DOI: 10.1109/ASSET.1999.756770.
    CrossRef
  18. L. Yin and M. A. Smith and K. S. Trivedi, Uncertainty analysis in reliability modelling, in Annual Reliability and Maintainability Symposium. 2001 Proceedings. International Symposium on Product Quality and Integrity (Cat. No. 01CH37179), PA, USA: Philadelphia, pp. 229-234, 2021. DOI: 10.1109/RAMS.2001.902472.
  19. M. A. Haque, and N. Ahmad, Modified Goel-Okumoto Software Reliability Model Considering Uncertainty Parameter, in Mathematical Modeling, Computational Intelligence Techniques and Renewable Energy, vol. 1405, pp. 369-379, 2022. DOI: 10.1007/978-981-16-5952-2_32.
    CrossRef
  20. Z. Liu and S. Yang and R. Kang, Software belief reliability growth model based on uncertain differential equation, IEEE Transactions on Reliability, pp. 1-13, Mar., 2022. DOI: 10.1109/TR.2022.3154770.
    CrossRef
  21. R. Dhaka and B. Pachauri and A. Jain, Two-Dimensional Software Reliability Model with Considering the Uncertainty in Operating Environment and Predictive Analysis, in Data Engineering for Smart Systems, vol. 238, pp. 57-69, 2022. DOI: 10.1007/978-981-16-2641-8_6.
    CrossRef
  22. C. Zhang, and A. Mostashari, Influence of component uncertainty on reliability assessment of systems with continuous states, International Journal of Industrial and Systems Engineering, vol. 7, no. 4, pp. 542-552, Apr., 2011.
    CrossRef
  23. S. V. Dhople and Y. C. Chen and A. D. Domínguez-García, A set-theoretic method for parametric uncertainty analysis in Markov reliability and reward models, IEEE Transactions on Reliability, vol. 62, no. 3, pp. 658-669, Sep., 2013. DOI: 10.1109/TR.2013.2270421.
    CrossRef
  24. P. Ju, and H. Li, and X. Pan, and C. Gan, and Y. Liu, and Y. Liu, Stochastic dynamic analysis for power systems under uncertain variability, IEEE Transactions on Power Systems, vol. 33, no. 4, pp. 3789-3799, Jul., 2018. DOI: 10.1109/TPWRS.2017.2777783.
    CrossRef
  25. Meedeniya, and I. Moser, and A. Aleti, and L. Grunske, Architecture-based reliability evaluation under uncertainty, in Proceedings of the joint ACM SIGSOFT conference--QoSA and ACM SIGSOFT symposium--ISARCS on Quality of software architectures--QoSA and architecting critical systems--ISARCS, pp. 85-94, Jun., 2011. DOI: 10.1145/2000259.2000275.
    CrossRef
  26. L. Fiondella, and S. S. Gokhale, Software reliability with architectural uncertainties, in 2008 IEEE International Symposium on Parallel and Distributed Processing, FL, USA: Miami, pp. 1-5, 2008. DOI: 10.1109/IPDPS.2008.4536436.
    CrossRef
  27. K. Go and M. Hamill and X. Wang, Adequacy, accuracy, scalability, and uncertainty of architecture-based software reliability: Lessons learned from large empirical case studies, in 2006 17th International Symposium on Software Reliability Engineering, NC, USA: Raleigh, pp. 197-203, 2006. DOI: 10.1109/ISSRE.2006.11.
    CrossRef
  28. Gashi and P. Popov and V. Stankovic, Uncertainty explicit assessment of off-the-shelf software: A Bayesian approach, Information and Software Technology, vol. 51, no. 2, pp. 497-511, Feb., 2009. DOI:10.1016/j.infsof.2008.06.003.
    CrossRef
  29. K. Goseva-Popstojanova, and S. Kamavaram, Assessing uncertainty in reliability of component-based software systems, in 14th International Symposium on Software Reliability Engineering, ISSRE 2003, CO, USA: Denver, pp. 307-320. DOI:10.1109/ISSRE.2003.1251052.
    CrossRef
  30. Goseva-Popstojanova, and S. Kamavaram, Uncertainty analysis of software reliability based on method of moments, in 13th Int’l Symp. Software Reliability Engineering, pp. 143-144, 2002.
  31. D. K. Mohanta, and D. S. Roy, Importance and uncertainty analysis in software reliability assessment of computer relay, in Proceedings of the Institution of Mechanical Engineers, Part O: Journal of Risk and Reliability, vol. 225, no. 1, pp. 50-61, Jun., 2011. DOI: 10.1177/1748006XJRR341.
    CrossRef
  32. Y. Wang, and X. Gao, and Y. Cai, and M. Yang, and S. Li, and Y. Li, Reliability evaluation for aviation electric power system in consideration of uncertainty, Energies, vol. 13, no. 5, p. 1175, Mar., 2020. DOI: 10.3390/en13051175.
    CrossRef
  33. A. K. Verma and S. Ajit and D. R. Karanki, Uncertainty analysis in reliability/safety assessment, in Reliability and Safety Engineering, pp. 457-491, Sep., 2015. DOI: 10.1007/978-1-4471-6269-8_13.
  34. Goševa-Popstojanova, and K. S. Trivedi, Architecture-based approach to reliability assessment of software systems, Performance Evaluation, vol. 45, no. 2-3, pp. 179-204, Jul., 2001. DOI: 10.1016/S0166-5316(01)00034-7.
    CrossRef
  35. C. -J. Hsu, and C. -Y. Huang, An adaptive reliability analysis using path testing for complex component-based software systems, IEEE Transactions on Reliability, vol. 60, no. 1, pp. 158-170, Mar., 2011. DOI: 10.1109/TR.2011.2104490.
    CrossRef
  36. D. V. Ravenzwaaij and P. Cassey and S. D. Brown, A simple introduction to Markov Chain Monte-Carlo sampling, Psychonomic bulletin & review, vol. 25, no. 1, pp. 143-154, Mar., 2018. DOI: 10.3758/s13423-016-1015-8.
    Pubmed KoreaMed CrossRef
  37. B. Zhou and H. Okamura and T. Dohi, Markov chain Monte Carlo random testing, in Advances in Computer Science and Information Technology, vol. 6059, Heidelberg, German.
    CrossRef
  38. F. S. Hillier, and G. J. Lieberman, A special algorithm for the assignment problem, in Introduction to Operations Research, 9th ed., McGraw-Hill Education, pp. 342-346, 2008.
  39. D. Hamlet and D. Mason and D. Woitm, Theory of software reliability based on components, in Proceedings of the 23rd International Conference on Software Engineering. ICSE 2001, ON, Canada: Toronto, pp. 361-370, 2001. DOI: 10.1109/TR.2011.2104490.[40] S. Krishnamurthy and A. P. Mathur, “On the estimation of reliability of a software system using reliabilities of its components,” in Proceedings of The Eighth International Symposium on Software Reliability Engineering, Albuquerque: NM, USA, pp. 146-155, 1997. DOI: 10.1109/ISSRE.1997.630860.
    CrossRef
  40. L. Shooman, Structural models for software reliability prediction, in Proceedings of the 2nd International Conference on Software engineering, pp. 268-280, Oct., 1976.
  41. Xie, and C. Wohlin, An additive reliability model for the analysis of modular software failure data, in Proceedings of Sixth International Symposium on Software Reliability Engineering. ISSRE'9, Toulouse, France, pp. 188-194, 1995. DOI: 10.1109/ISSRE.1995.497657.
    CrossRef
  42. F. Zhang, and X. Zhou, and J. Chen, and Y. Dong, A novel model for component-based software reliability analysis, in 2008 11th IEEE High Assurance Systems Engineering Symposium, Nanjing, China, pp. 303-309, 2008. DOI: 10.1109/HASE.2008.41.
    CrossRef

Sampa Chau Pattnaik

received her Bachelor’s in Computer Science from B.J.B.(Auto.) College (Utkal University), Orissa, India in 2002. She received her Master’s in Computer Science& Engineering from Utkal University Bhubaneswar, Orissa, India in 2008. Now, she continues her research (Ph.D.) at Siksha ‘O’ Anusandhan Deemed to be University. Her research interests include software engineering and reliability testing.


Mitrabinda Ray

received her Ph.D. degree from NIT, Rourkela. Now, she is as an associate professor at Siksha ‘O’ Anusandhan Deemed to be University, Bhubaneswar, in the department of Computer Science and Engineering. Her area of interest is software testing, reliability, and estimation. She has published several papers in different journals and conferences.


Mitali Madhusmita Nayak

received her Ph.D. degree from NIT, Rourkela. Now, she is as an associate professor at Siksha ‘O’ Anusandhan Deemed to be University, Bhubaneswar, in the department of Computer Science and Engineering. Her area of interest is software testing, reliability, and estimation. She has published several papers in different journals and conferences.


Mitrabinda Ray

received her Ph.D. degree from Siksha ‘O’ Anusandhan Deemed to be University. She is an associate professor with the Department of Mathematics, Institute of Technical Education and Research at Siksha ‘O’ Anusandhan. She has over 15 years of teaching experience and 10 years of research experience. Her research interests are in the fields of Numerical Analysis, Inventory management, Fuzzy optimization, and Decision science.


Srikanta Patnaik

is a Professor in the Department of Computer Science and Engineering, Siksha ‘O’ Anusandhan Deemed to be University, Bhubaneswar, India. He received his Ph. D. (Engineering) on Computational Intelligence from Jadavpur University, India in 1999. He has supervised more than 25 Ph. D. and 60 Master theses in the field of Computational Intelligence, Machine Learning, Soft Computing Applications, and Re-Engineering. Dr. Patnaik has published around 100 research papers in international journals and conference proceedings. He has authored 2 textbooks,42 edited volumes, and a few book chapters as a guest writer; published by leading international publishers such as Springer-Verlag and Kluwer Academic. Dr. Patnaik is the Editors-in-Chief of the International Journal of Information and Communication Technology and International Journal of Computational Vision and Robotics published by Inderscience Publishing House, England as well as Editors-in-Chief of the Book Series on “Modeling and Optimization in Science and Technology” published by Springer, Germany. He is also an Associate Editor for the International Journal of Telemedicine and Clinical Practices (IJTMCP) and International Journal of Granular Computing, Rough Sets and Intelligent Systems (IJGCRSIS)


Article

Journal of information and communication convergence engineering 2022; 20(2): 79-89

Published online June 30, 2022 https://doi.org/10.6109/jicce.2022.20.2.79

Copyright © Korea Institute of Information and Communication Engineering.

Component-Based System Reliability using MCMC Simulation

Sampa ChauPattnaik1*, Mitrabinda Ray 1, Mitalimadhusmita Nayak 2, and Srikanta Patnaik1

1Department of Computer Science and Engineering, Siksha ‘O’ Anusandhan (Deemed to be) University, Bhubaneswar 751030, India
2Department of Mathematics, Siksha ‘O’ Anusandhan (Deemed to be) University, Bhubaneswar 751030, India

Correspondence to:Sampa ChauPattnaik (E-mail: sampa.chaupattnaik@gmail.com, Tel: +91-88-9565-6648)
Department of Computer Science and Engineering, Siksha ‘O’ Anusandhan (Deemed to be) University, Bhubaneswar 751030, India.

Received: January 28, 2021; Revised: April 23, 2022; Accepted: May 10, 2022

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

To compute the mean and variance of component-based reliability software, we focused on path-based reliability analysis. System reliability depends on the transition probabilities of components within a system and reliability of the individual components as basic input parameters. The uncertainty in these parameters is estimated from the test data of the corresponding components and arises from the software architecture, failure behaviors, software growth models etc. Typically, researchers perform Monte Carlo simulations to study uncertainty. Thus, we considered a Markov chain Monte Carlo (MCMC) simulation to calculate uncertainty, as it generates random samples through sequential methods. The MCMC approach determines the input parameters from the probability distribution, and then calculates the average approximate expectations for a reliability estimation. The comparison of different techniques for uncertainty analysis helps in selecting the most suitable technique based on data requirements and reliability measures related to the number of components.

Keywords: Component Based Systems, MCMC, System Reliability, Discrete Time Markov Chain

I. INTRODUCTION

Software reliability is necessary for system developers to improve application performance. Delivering commercial-off-the-shelf (COTS) products is a critical challenge for all industries. Software reliability analysis at the architectural level is based on the following models: state-based [1-5], path-based [21,29], and additive [32].

Software reliability estimation occurs at the code and design levels. In architecture-based software reliability analysis, the input parameters include the software architecture of the application, transition probabilities between components, reliability of individual components, and average execution time of the components [6]. During the design phase, the major challenge is that the software architecture and component parameters may not be well known. Thus, an uncertainty analysis is required for the architecture-based software reliability model. The goal is to investigate how the inaccuracy of the operational profile [7] directly influences the transition probabilities between the components. The failure of the components is unknown and depends on the test procedures. Because the failure time data are uncertain, the system reliability computed from the parameters is also uncertain. The reliability growth of a component depends on its fault detection rate. A fault detection rate may be given by one of the software reliability growth models [8-11], and the location of faults that caused these failures can then be identified based on several accurate methods.

Motivation: Software reliability prediction based on architectural models depends on i) appropriate model considerations and ii) accurate parameter values. Practically, there is considerable uncertainty in parameters because uncertainty occurs in the estimation of the operational profile and component reliabilities. A major challenge in reliability prediction at the architectural level is the presence of uncertainties owing to the lack of information in relation to the proposed artifact, such as operational profile, implementation details, and failure behavior.

The objectives of this paper are:

  • To determine the uncertainty parameters that arise through the model such they are properly represented in the reliability analysis.

  • To monitor the uncertainty results such that they help decision-makers make further decisions.

  • To analyze the system reliability by considering the effect of the individual reliability of a component using some uncertain manner.

We present uncertainty in architecture-based software reliability based on a Markov chain Monte Carlo (MCMC) simulation and moment techniques. The methodology addresses the parameter uncertainty problem and shows its effect on system reliability.

The remainder of this paper is organized as follows. In Section II, we summarize the background study. This explains the different approaches to uncertainty analysis for component-based systems. Section III describes the uncertainty analysis technique based on the method of moments, Monte Carlo (MC) simulation, and Markov chain Monte Carlo (MCMC) simulation. Section IV explains the result analysis using different uncertainty methods in architecture-based software reliability analysis. Section V provides a comparative study of different methods for uncertainty analysis, and the paper concludes in Section VI.

II. BACKGROUNG STUDY

In the last decade, several architecture-based software reliability models have been proposed [8, 12, 13, 18, 35]. In the early phase, software reliability analysis can identify the critical components that improve the reliability of an application. Software reliability estimation in the design phase causes uncertainty in the component reliabilities. Several researchers [14-24] have used different techniques to analyze uncertainty in architecture-based software systems. To evaluate the overall system reliability, we used different models, such as Markov chains and fault trees. Parameters such as component reliabilities, transition probabilities between two components, and average execution time of components are used in these models, generally collected from field data, systems with similar function data, or by guessing. The exact values of the parameters may be unavailable at the design stage (as coding has not begun). Hence, the analyst can formulate a relative estimation for the values of the parameters by considering the estimation from the actual system implementation to validate the assumption prepared in the design stage. Therefore, the values of these parameters may be inaccurate. This leads to uncertainties in system reliability.

Reliability assessment occurs during the early stages of the software development life cycle using software architecture. However, the prediction of component reliabilities at the architectural level is difficult because uncertainties arise in the system and its individual components [15]. authors considered a component-based robotics test bed, the controller component of SCR over, which is a NASA JPL mission data system (MDS). They identified different uncertainty factors of components, such as the recovery and failure probabilities. Their results showed that the reliability of the component increased with the recovery probability. In addition, if the failure probability increases, the component reliability decreases.

For architecture-based reliability estimation, accuracy may depend on parameters such as environmental factors and system usage. Researchers have handled this problem by integrating uncertainties in architecture assessment models and analytically solving them with simulations. This method illustrates the mean and variation of an attribute. The case study used was on the anti-lock braking system (ABS) of a car [25]. Architecture-based software reliability analysis can identify the critical components early, which leads to a cost-effective reliability improvement of the application. However, at this phase, estimating architectural and component parameters is not entirely possible with certainty.

The problem of uncertain component reliabilities has occurred in software application reliability estimation [26]. The authors considered the levels of confidence in architectural parameters using minimal test and simulation data based on the principle of multinomial distribution confidence intervals for reliability estimation. K. Go et. al. [27] conducted a large-scale case study on architecture–based software reliability evaluation to determine potential limitations and challenges for future analysis. The study of uncertainty in architecture–based software reliability found that the MCapproach is more effective than the moments technique. The MCtechnique results indicate the following: (1) fewer parameters yield the most variance in system reliability and (2) the reliability of components has a greater effect on system reliability than the probability of transition for a given operational profile.

Software reliability models consist of several parameters that are typically determined from the test data of the component. As point estimates concentrate on random data variations, uncertainties in the measured parameters were observed. The parameter causing uncertainty underestimates the uncertainty in terms of overall system reliability [16]. Here, the authors aimed to measure the uncertainties with their correlated parameters in the software reliability modeling of a single component and multiple components. A major problem in the uncertainty analysis of software reliability modelling is the lack of available failure data. Using the views of experts and historical data from earlier projects will help resolve this problem. An uncertainty analysis was performed by merging the concept of maximum entropy (MEP) with the Bayesian method. System-level uncertainty analysis can be conducted using the MCmethod to demonstrate the impact of uncertainty parameters on many system-level components.

In component-based software development systems, an inadequate selection of components results in low-quality software. The authors disagree with the assessment methods in which explicit uncertainty methods are represented using probability distributions. They provide details of the Bayesian model used to capture the uncertainties in the simultaneous assessment of two attributes, thereby confining the dependencies between them [28]. To handle the uncertainty in the system reliability, a MCsimulation method is used [29], where reliability is determined by the variance, and the accuracy increases by increasing the sample size. To estimate the reliability of a system, the method of moments is used for uncertainty analysis in architecture-based software reliability models [30]. The method of moments shows that the reliability evaluation occurs by the moments of component reliabilities, where the accuracy can be improved by considering the high level of Taylor expansion. The software consists of components and modules.

Software reliability assessment is the primary area of software-based systems. Software reliability estimation is based on historical data and a presumed distribution curve, which is inclined to uncertainties. A fault tree [31] is used for reliability estimation in component-based systems; and fuzzy set theory is used in fault tree analysis to quantify the basic uncertainty results. In recent years, for complex architecture systems, such as flights, a highly reliable system is required for safety purposes [32]. This leads to uncertainty in failure occurrences. The confidence interval and probability density functions were used to compute the uncertainty in the system reliability variance. The evaluations show that the uncertainty varies with the system or execution time.

III. UNCERTAINTY ANALYSIS

The behavior of the system is affected by unavoidable components, which are considered uncertain. Uncertainty is categorized into two types: i) aleatory uncertainty: stochastic process behavior (i.e., randomness from inherent variability in the system) and ii) epistemic uncertainty: vagueness owing to a lack of knowledge and information about the system [33]. The causes and aspects of uncertainty have been studied in various applications such as engineering risk modeling and reliability assessments. Although many different types of uncertainty exist, they are usually categorized as aleatory or epistemic. Aleatory uncertainty, also known as stochastic uncertainty, arises owing to the randomness of the behavior of a software system. The probability distribution is commonly used to solve the aleatory uncertainty problem with sufficient data. For example, an application using a reliability-based design is based on probability theory. Epistemic uncertainty occurs because of a lack of knowledge or information regarding a software system, particularly during the design of a complex system. It is not based on probabilistic theory.

Methods such as interval analysis, possibility theory, and evidence theory have been used to model epistemic uncertainty. This paper is based on epistemic uncertainty because we apply uncertainty analysis during the design phase and sufficient data are unavailable for estimating system reliability. Epistemic uncertainty analysis is based on probability theory as well as fuzzy sets. This paper contains probability distributions, such as beta and gamma distributions.

To conduct reliability estimation with uncertainty analysis in a system, the following steps are followed.

  • Build the software architecture obtained from a specification and design document.

  • Estimate the component reliability

  • Design the software model (using the software growth model or fault injection)

  • Use the uncertainty techniques for the uncertainty para- meters (component reliability, transition probability, etc.). I.e., in this step, different uncertainty methods (method of moments, MC simulation, or MCMC simulation) are applied.

  • Obtain the system reliability.

Software reliability is estimated using an architecture-based model, and we include the dynamic behavior of the software architecture, i.e., a component dependency graph (CDG) [6], transition probabilities of two components, and nature of software failure (reliability of components or failure rate). For a given application software, the uncertainty in software reliability is caused by these parameters. Software architecture is defined as the interaction of different components. We used a path-based model to represent the architecture–based software reliability model [8, 15, 16, 26]. The path-based method uses the CDG. The components are represented as nodes, and the interactions between the components are represented as edges. Transfers between components are assumed to be a discrete-time Markov chain (DTMC) with transition probability Pr = [prij], where prij is the transmission from component i to component j. Component failure behavior estimates the reliability of each component. (CRi).

Several methods have been used to estimate component reliability. To develop component failure data, software reliability growth models (SRGM) [8, 15, 16, 26, 34] were applied to each component obtained during testing. Software reliability growth models cannot always be used because of the scarcity of failure data. Another manner of estimating the reliability of a component is to use a fault injection technique [13], which isa simulation basis. Therefore, the estimation of component reliability estimated may be inaccurate, which stimulates the use of uncertainty analysis. The methodology used for uncertainty analysis was obtained using a software architecture with the failure behavior of the components.

In this study, we used a path-based model [35] for reliability estimation. For the reliability assessment, we considered the software architecture with failure behavior. The reliability of a system is the probability of reaching the terminal component through the execution path along the components. The overall system reliability is calculated as follows:

SR=m1,nCRn,

where, m= k=0qk= IDq1. This denotes the (1,n)th value of matrix m, i.e., reaching the end component n by starting from component 1. In the DTMC method, the transition probability is represented by Pr=qc01, where q = (r - s) × (r - s) is a sub-stochastic matrix (minimum one-row summation <1), s × s is the identity matrix ID=1001, 0 = s × (r - s) is the matrix of 0s, and c = (r - s) × s. In addition, a Markov process with r states has s absorbing states.

Several techniques are used for uncertainty analysis. Here, we present some techniques, such as the method of moments [30], MC simulation [29], and MCMC simulation method [36].

A. Moments

This is an approximate approach that generates moments of system reliability from the moments of component reliabilities. The component reliability estimation using this method is inaccurate because of the higher-order Taylor series expansion.

For method of moments:

  • a) The link between system reliability (SR) and component reliabilities (CR1, CR2, ..., CRn) is represented by SR= f(CR1, CR2, ..., CRn) using the path-based model.

  • b) The system reliability is determined using Taylor series.

  • c) The moments are applied for the component reliabilities and transition probabilities.

  • d) The approximate mean and variance of the system reliability are calculated using the parameter moments using the given equations.

The method of moment is expressed as a Taylor series function. Thus, the 1st order moment is given by the equation

SR»r0+ i=0nriCRi-E CRi + i=1nj=1nrp ij Pr ij -E Pr ij ,

where, SR denotes the reliability of the components, and E[CR] is the expected reliability of components.

r0=fECR1, ECR1, ECR1,  ECR1, ri=CRCR1CRi=ECRi, for i = 1, 2, ...n, and rpij=CRPrijCRi=ECRi,Pri,j=EPri,j for i = 1, 2, ...n

The mean of the system reliability is E [CR]≈r0


Then, the variance of the system reliability is as follows.

VCR=σ2 i=1nri2VCRi+ k=1n i=1nrpij2VPrij+2 k=1n i=1n j=i+1nrpkirpkjCov Pr ki , Pr kj

The variance estimation shows the confidence in system reliability. Thus, if the variance decreases, the overall system reliability increases. Now, the efficiency of the expected reliability and variance of reliability is improved using high-order Taylor expansions.

B. Monte Carlo Technique

MCsimulation is a fairly accurate method for estimating system reliability. This method represents a well-defined distribution of the parameters of the software reliability model. The MCtechnique for a component-based system is as follows.

Algorithm: Uncertainty Analysis using MC for a Component Based System

Input: component reliability of the ith component (CR (i)), transition probability between two components (Prij), average execution time of n components; n: number of components

Output: System Reliability

  • Initialize CRi =X[i].

    X[i]= random variables, X ∈ [0,1]

  • Identify the system reliability using a path-based model (PR) with input parameters CR(i) and Prij:

    PR=CR1× i=2 n1CRi Pr 1,i ×CRn j=2 n1pr1,j×prj,n

  • Calculate the mean for each component (CRi) using the formula m=IDq1= u=0qu, where m is the mean number of visits from component I to component j, ID is the identity matrix, and q is the transition matrix explained below.

  • Calculate the variance of each component CRi using the formula v = m(2mdig - ID) - msqr , where v is the variance of each component, mdig is the diagonal of matrix m, and msqr is the square of the matrix m.

  • Determine the system reliability:

    SR= j=1 n1CRjm1,j+12CRjm1,jlogCRj2ν1,jCRn,

    where, m1,j and v1,j are the visit mean and variance, respectively, of each component from component 1 to component j.

  • Assign probability distributions to Prij and RCi. (The pro- bability distributions of the input parameters are assigned based on theoretical assumptions. Here, we use the beta and gamma distributions for the component reliability and transition probability, respectively.)

    fCRi=Γxi+yiΓxi+ΓyiCRixi11CRiYi1

    fpri1,pri2,pri3,.....prin=ΓAi1+Ai2++AinΓAi1+ΓAi2+.....Γ..in) j=1nprijAij1

  • Repeat Steps 3, 5, and 5 with the new values of CR (i) and Prij obtained from the probability distribution.

  • Repeat Steps3, 4, 5, and 6 until the required number of values is produced to obtain a lower variance and better system reliability.

    MCsimulation process

    ** i= no. of column and j= no. of rows

MCsimulation is a method for determining the probability of possible outcomes. Predict a model is difficult owing to the use of random variables. This is also referred to as a multiple probability simulation. MCMC is subset of MC. Thus, MCMC is an MC, but not all MCs are MCMC. The main objective of the MCMC method is to build a Markov chain. The Markov chain is produced using independent samples (with a negative correlation) to reduce the variance.

C. Markov Chain Monte Carlo

MCMC is an approximation technique for system reliability estimation. Therefore, the objective of this simulation was to use Bayesian inference [37] for posterior distributions. MCMC allows the user to approximate aspects of the posterior distributions. The MCMC method produces MCsimulations depending on the Markov property, and the simulation convergence point obtained is the posterior distribution [36]. The MCMC process is stochastic. A stochastic process {s(T)} has a Markov property (Mp). The Markov property is represented as Pr{s(T + 1) = n|s(0) = 0, s(1) = ...s(T) = m) = Pr{s(T + 1) = n|s(T) = m} for T = 0, 1, 2, .... This expression show that state s at time T + 1 only depends on the current state s at time T, and is not dependent on past states s(T−1), ..., T(1).

Algorithm: Component Based System Uncertainty Analysis using MCMC

Input: component reliability of the ith component (CR (i)), transition probability between two components (Pri,j), average execution time of n components

Output: variance of each component (Ci), System Reliability

  • Initialize CRi =X[i]

    X[i]= random variables, X ∈ [0,1]

  • Identify the system reliability using the path-based model (PR) with input parameters CR(i) and prij:

    PR=CR1× i=2 n1CRi Pr 1i ×CRn j=2 n1pr1j×prjn

  • Calculate the mean for each component (CRi) using the formula m=IDq1= u=0qu, where m is the mean number of visits from components i to j, ID is the identity matrix, and q is the transition matrix explained below.

  • Calculate the variance of each component CRi using the formula v = m(2mdig - ID) - msqr ,where v is the variance of each component, mdig is the diagonal of matrix m, and msqr is the square of matrix m.

  • Determine the system reliability

    SR= j=1 n1CRjm1,j+12CRjm1,jlogCRj2ν1,jCRn,

    where, m1,j and v1,j are the visit mean and variance, respectively of each component visit from component 1 to j component.

  • Apply the Markovian property discussed above to the transition probabilities (Prij). We consider n-step transition probabilities for computing these transition probabilities to form a Markov chain.

    Prijn= k=0MPrikm×Prkjnm  i=0, 1, 2, ...M;

    j=0, 1, 2 ...M; and any m=1, 2,..., n-1;

    n=m+1, m+2, ...

  • Apply the MCalgorithm to estimate the system reliability.

  • Repeat Steps 3 and 4 to determine a convergence posterior distribution.

We used the Chapman–Kolmogorov equations for the n-step transition probabilities [19].

IV. IMPLEMENTATION OF UNCERTAINTY ANALYSIS

We discuss the uncertainty analysis of the components with a waiting queue simulator application [6] as a case study. The architecture of the application consists of the following components: an ArrivalGenerator, EventList, ServiceFacility, ScheduleManager, QueuingFacility, and Measurement recorder. The interactions between the components were evaluated using the scenarios. The architecture of the waiting queue application is represented by a graph called the component dependency graph (CDG). A CDG consists of components and the transition probabilities between the components. The interaction between the components forms possible execution paths from the starting component to the end component. Thus, it follows a path-based model. Here, we consider the graph traversal with three structures: sequences, branches, and looping.

Fig. 1 shows a CDG graph represented with nodes and edges. The nodes and edges consist of information such as component name, component reliability, transition probability from one component to another, average execution time of each component, and transition reliability from one component to another. Assuming each component’s reliability is 1, the average execution times between components are shown in Table 1. The average execution time was evaluated using the following formula:

Table 1 . Reliabilities and average execution times of components.

Index (CI)Reliability (CRi) (Assuming 1 for all components)Name (CNi)Average Execution Time (AvgETi)
11Schedule_Manager5
21Arrival_Gen2
31Service_Facility7
41EventList14
51QueuingFacility5
61Measurement0.8
71End0

Figure 1. CDG of the queue management application.
AETi= i=1 sPr sk×T CRi CRiSk,

Where AETi is the average execution time of each component CRi, Prsk is the probability of executing scenario sk, T(CRi) is the execution time of CRi, i.e., the total number of active times in the execution of scenario sk. Table 2 lists the transition probabilities between the components of the waiting queue system.

Table 2 . Transition Probability between Components (Ci and Cj).

Pr12 = 0.05Pr13 = 0.215Pr14 = 0.475Pr15 = 0.26
Pr21 = 1
Pr31 = 0.1787Pr34 = 0.215Pr36 = 0.0363Pr37 = 0.57
Pr41 = 0.5725Pr43 = 0.215Pr45 = 0.2125
Pr51 = 0.3575Pr54 = 0.2125Pr57 = 0.43
Pr63 = 1


The values in Table 2 were obtained from the CDG in Fig. 2. The transition probabilities between two components (pri,j) are evaluated as follows. It is the probability of transitioning between two components (Ci, Cj), computed as the interactions between two components during the reliability analysis [6]. The sum of transaction probability from any component is always 1, i.e., d=1nPrdn=1.

Figure 2. Path-based structure from the CDG for the waiting queue application.

Fig. 2 shows the corresponding DTMC-based execution path from starting component C2 to end component C7. We consider the structure based on the architecture of the application shown in Fig. 1. Fig. 2 consists of 3 types of path structure: a) sequence, b) branching, and c) looping. Hence, we apply the path reliability [35] for Fig. 2 and estimate the system reliability as follows. Fig. 2 obtains the expression for system reliability using the path-based approach, and its expression is as follows:

PR=CR2×CR1Pr211Pr12×Pr23×CR3Pr21×Pr231Pr23×Pr32×Pr36×CR6Pr21×Pr23×Pr361Pr63×Pr36×Pr34×CR4Pr21×Pr23×Pr36×Pr341Pr63×Pr23×CR5Pr21×Pr23×Pr36×Pr34×Pr451Pr63×Pr23×CR7Pr21×Pr23×Pr36×Pr34×Pr451Pr63×Pr23

The above expression is used in the method of moments to compute the mean and variance of the system reliability.

We demonstrate that the uncertainty analysis for the application has six components, and its architecture is described by the DTMC presented in Fig. 2. To compute the method of moments using Taylor series, we consider the nonzero transition probabilities Prij given in Table 2 as the mean values. Table 3 shows the mean component reliabilities E[Rei] and variance of the component reliabilities V[Rei]. We found that the uncertainty of component reliabilities results in uncertainty in software reliability. For this reason, we used the mean values of the component reliabilities and variances shown in Table 3 in the equations describing the method of moments in Section III. We observe from Table 4 that, if Taylor’s expansion order is greater, then the accuracy increases, and the variance is less. Thus, the system reliability increases. This indicates that, if the variance is less, the system is more reliable.

Table 3 . Mean and variance of the components.

Component Index(CI)Expected components reliability E[Rei]Variance of components V[Rei]
10.98090.0565
20.98910.0240
30.98920.0324
40.97310.0041
50.98930.1230
60.98900.3450
70.9700.0565

Table 4 . Results of the mean and variance of the system reliability for the waiting queue.

1storder Taylor expansion2nd order Taylor expansion
mean calculation value0.7780.758
variance calculation value0.000223620.0002162


A. Waiting Queue Management Case Study based on MC Simulation Method

The uncertainty analysis for the case study evaluates the system reliability using the formula SR = f(CRi, Prij). We obtained the results by varying the component reliability from 0.95 to 0.98, assuming that E[CR1] = E[CR7] = 1 and V[CR1]=V[CR7]=0. From Fig. 2, we obtain the mean of the component reliability and variance of each component’s reliability. The system reliability was 92%. Table4 presents the evaluated values of E[CRi] and V[CRi] without the MC simulation method.

Similarly, applying the MCmethod for the component reliability varies in the range [0.95, 0.98]; the mean of the component reliability and variance of the component reliability are described in Table 6. The MCmethod was executed several times (approximately 100 times), and we obtained an overall system reliability of 93%. The MCmethod was described in Section III. Hence, the values of alpha(x) = 10 and beta(y) = 2.675 were set for further calculations.

Table 5 . Results for mean and variance of each component’s reliability.

Component #Mean of component Reliability E[CRi]Variance of Component Reliability, V[CRi]
11.0000
20.98280.9496
30.82680.4239
40.71300.9769
50.84310.5565
60.74610.3645
71.0000

Table 6 . Results for mean and variance of each component’s reliability using MC Simulation.

Component #Mean of component Reliability E1[CRi]Variance of Component Reliability, V1[CRi]
11.0000
20.98370.9823
30.89970.4079
40.96410.9515
50.93350.5197
60.80210.3401
71.0000


The results also show that if the variance of the components is small, the system reliability increases.

B. Example of MCMC Method

The case study used for the MCmethod was also used for the MCMC method. Here, the MCMC applied for component reliability varies in the range [0.95 to 0.98], and the results are discussed below. Table7 shows the mean and variance of component reliability using the MCMC method. The n-step Markov chain is important for defining the probability of every component of a system at a random time. We start with any component as well; we reach the same probability distribution with a higher system reliability. The MCMC procedure consists of two steps.

Table 7 . Results for mean and variance of each component’s reliability using MCMC simulation.

Component #Mean of component Reliability E[CRi]Variance of Component Reliability, V[CRi]
11.0000
20.23440.0234
30.00950.0093
40.00260.0021
50.00140.0014
60.24780.0114
71.0000

  • Use the Markov chain process (for this n-step transition probability).

  • Apply the MCsimulation method.

  • The overall system reliability obtained using the MCMC method is 96.8%, i.e., ≈ 97%.

V. COMPARISON STUDY

The suitable selection of the method used for the analysis of uncertainty in software reliability for a given application depends on the requirements of the data, acquired reliability estimate, and performance of the solution. In this section, we explain the three methods used for the uncertainty analysis. We found that the method moment provided a system reliability of 92%. The MCsimulation showed that the overall system reliability was 93%. Similarly, the MCMC simulation provided a system reliability of 97%, with a lower variance than that of the MCmethod. Table 8 shows the variance of each component using the MC and MCMC methods. We found that the variance of each component decreased in the MCMC method compared to the MC method. Method of moments does not require a probability distribution. In method of moments, random numbers are not required. To increase the accuracy of method of moments, a high-order Taylor function is used.

Table 8 . Comparison results of variance of each component using MCMC and MCsimulation.

Component IDVariance of components using MCmethodVariance of components using Markov Chain method
100
20.98230.0234
30.40790.0093
40.95150.0021
50.51970.0014
60.34010.0114
700


MCsimulation increases the accuracy by expanding the simulation numbers randomly generated. MC simulation includes a collection of reliability methods, such as moments, probability distributions, system reliability, and lower variance, to improve confidence in the system reliability estimation. In the MCsimulation method, uncertainty arises because of the operational profile and reliability of the components. Compared with the MCsimulation method, the MCMC simulation is better. In the MCMC simulation, the Markov chain was used for the probability distribution. This enables the quantification of the variance and accuracy of the reliability system estimation is high. This implies that the results obtained from different uncertainty methods help decision makers to make better decisions.

From [6], we considered the component dependency graph for the calculation of system reliability using the MCMC technique. In [6], the component reliability, average execution time, and transition probability between two components from the scenarios of the application were evaluated. However, we considered the evaluation of the mean and variance of each component based on the random reliability of each component. (In [6], the reliability of each component was fixed at 1). Table 9 lists comparisons between the results from previous studies and the proposed method.

Table 9 . Comparisons with previous studies.

Sl. No.InvestigatorTitleInput parametersApproachFindings
1.[6]A scenario-based reliability analysis approach for component-based software.Component dependence graph, component reliability, average execution time, and transition probabilityThey suggest an application reliability with a scenario-based reliability analysis algorithm using the component dependency graphThis method helps discover the critical components and interfaces, as well as study the sensitivity of the application reliability to these components.
2.Proposed ModelComponent Based System Reliability using MCMC SimulationComponent dependence graph, component reliability, average execution time, and transition probability.Evaluation of the application reliability using the mean and variance of each component. The MCMC simulation method is then applied.This technique is used to quantify the variance of each component, and the estimated system reliability accuracy is high.

VI. Conclusion

Architecture-based software reliability methods use several models (such as Markov chains and fault trees) to predict the reliability of an application. We analyzed the uncertainty in parameters, transition probabilities between components, and the reliability of individual components that propagate the software reliability estimation. Using these methods, we obtain the values for the mean and variance of the system reliability, which are considered for the uncertainty owing to their operational profile and component reliabilities. We presented the waiting queue management application architecture and its reliability calculation method with input values such as component reliability and transition probability between components. The estimated value of the system reliability using MCMC simulation provides more information than the MC simulation, or even method of moments. Consequently, the system has a higher confidence in reliability estimation with a smaller variance. In the MCMC method, owing to the “memory lessness” property, knowing the history of the process does not improve future predictions. The system reliability was analyzed by considering the uncertainty of the component, as shown in Table 3. From this, we can conclude that before applying the uncertainty measure, the reliability is 0.93, and by applying the uncertainty method to the components, the system reliability changes to 0.92 for method of moments, 0.93 for MC, and 0.97 for MCMC. Thus, we conclude that the overall system reliability is insignificantly affected, even if the individual components are subjected to uncertainty.

We consider that the uncertainty analysis of software reliability is important, primarily for predicting software reliability during the analysis stage in the life cycle. Sensitivity analysis provides intuition about the input uncertainty that controls the output uncertainty towards effective resources, which can be performed on the component, and transition probabilities can be considered in the future scope of this paper.

Fig 1.

Figure 1.CDG of the queue management application.
Journal of Information and Communication Convergence Engineering 2022; 20: 79-89https://doi.org/10.6109/jicce.2022.20.2.79

Fig 2.

Figure 2.Path-based structure from the CDG for the waiting queue application.
Journal of Information and Communication Convergence Engineering 2022; 20: 79-89https://doi.org/10.6109/jicce.2022.20.2.79

Table 1 . Reliabilities and average execution times of components.

Index (CI)Reliability (CRi) (Assuming 1 for all components)Name (CNi)Average Execution Time (AvgETi)
11Schedule_Manager5
21Arrival_Gen2
31Service_Facility7
41EventList14
51QueuingFacility5
61Measurement0.8
71End0

Table 2 . Transition Probability between Components (Ci and Cj).

Pr12 = 0.05Pr13 = 0.215Pr14 = 0.475Pr15 = 0.26
Pr21 = 1
Pr31 = 0.1787Pr34 = 0.215Pr36 = 0.0363Pr37 = 0.57
Pr41 = 0.5725Pr43 = 0.215Pr45 = 0.2125
Pr51 = 0.3575Pr54 = 0.2125Pr57 = 0.43
Pr63 = 1

Table 3 . Mean and variance of the components.

Component Index(CI)Expected components reliability E[Rei]Variance of components V[Rei]
10.98090.0565
20.98910.0240
30.98920.0324
40.97310.0041
50.98930.1230
60.98900.3450
70.9700.0565

Table 4 . Results of the mean and variance of the system reliability for the waiting queue.

1storder Taylor expansion2nd order Taylor expansion
mean calculation value0.7780.758
variance calculation value0.000223620.0002162

Table 5 . Results for mean and variance of each component’s reliability.

Component #Mean of component Reliability E[CRi]Variance of Component Reliability, V[CRi]
11.0000
20.98280.9496
30.82680.4239
40.71300.9769
50.84310.5565
60.74610.3645
71.0000

Table 6 . Results for mean and variance of each component’s reliability using MC Simulation.

Component #Mean of component Reliability E1[CRi]Variance of Component Reliability, V1[CRi]
11.0000
20.98370.9823
30.89970.4079
40.96410.9515
50.93350.5197
60.80210.3401
71.0000

Table 7 . Results for mean and variance of each component’s reliability using MCMC simulation.

Component #Mean of component Reliability E[CRi]Variance of Component Reliability, V[CRi]
11.0000
20.23440.0234
30.00950.0093
40.00260.0021
50.00140.0014
60.24780.0114
71.0000

Table 8 . Comparison results of variance of each component using MCMC and MCsimulation.

Component IDVariance of components using MCmethodVariance of components using Markov Chain method
100
20.98230.0234
30.40790.0093
40.95150.0021
50.51970.0014
60.34010.0114
700

Table 9 . Comparisons with previous studies.

Sl. No.InvestigatorTitleInput parametersApproachFindings
1.[6]A scenario-based reliability analysis approach for component-based software.Component dependence graph, component reliability, average execution time, and transition probabilityThey suggest an application reliability with a scenario-based reliability analysis algorithm using the component dependency graphThis method helps discover the critical components and interfaces, as well as study the sensitivity of the application reliability to these components.
2.Proposed ModelComponent Based System Reliability using MCMC SimulationComponent dependence graph, component reliability, average execution time, and transition probability.Evaluation of the application reliability using the mean and variance of each component. The MCMC simulation method is then applied.This technique is used to quantify the variance of each component, and the estimated system reliability accuracy is high.

References

  1. R. C. Cheung, A user-oriented software reliability model, , IEEE transactions on Software Engineering, vol. SE-6, no. SE-2, pp. 118-125, Mar., 1980. DOI: 10.1109/TSE.1980.234477.
    CrossRef
  2. B. Littlewood, Software reliability model for modular program structure, IEEE Transactions on Reliability, vol. R-28, no. 3, pp. 241-246, Aug., 1979. DOI: 10.1109/TR.1979.5220576.
    CrossRef
  3. J. -C. Laprie, Dependability evaluation of software systems in operation, IEEE Transactions on software engineering, vol. SE-10, no. 6, pp. 701-714, Nov., 1984. DOI: 10.1109/TSE.1984.5010299.
    CrossRef
  4. P. Kubat, Assessing reliability of modular software, Operations research letters, vol. 8, no. 1, pp. 35-41, Feb., 1989. DOI: 10.1016/0167-6377(89)90031-X.
    CrossRef
  5. S. S. Gokhale, and K. S. Trivedi, Reliability prediction and sensitivity analysis based on software architecture, in Proceedings of the 13th International Symposium on Software Reliability Engineering, MD, USA: Annapolis, pp. 64-75, 2002. DOI: 10.1109/ISSRE.2002.1173214.
    CrossRef
  6. S. Yacoub and B. Cukic and H. H. Ammar, A scenario-based reliability analysis approach for component-based software, IEEE transactions on reliability, vol. 53, no. 4, pp. 465-480, Dec., 2004. DOI: 10.1109/TR.2004.838034.
    CrossRef
  7. J. D. Musa, Operational profiles in software-reliability engineering, IEEE software, vol. 10, no. 2, pp. 14-32, Mar., 1993. DOI: 10.1109/52.199724.
    CrossRef
  8. A. L. Goel, Software reliability models: Assumptions, limitations, and applicability, IEEE Transactions on software engineering, vol. SE-11, no. 12, pp. 1411-1423, Dec., 1985. DOI: 10.1109/TSE.1985.232177.
    CrossRef
  9. K. Goseva-Popstojanova and A. P. Mathur and K. S. Trivedi, Comparison of architecture-based software reliability models, in Proceedings of the 12th International Symposium on Software Reliability Engineering, Hong Kong, China, pp. 22-31, 2001. DOI: 10.1109/ISSRE.2001.989455.
    CrossRef
  10. M. R. T. Lyu, Software reliability theory, in Encyclopaedia of Software Engineering. DOI:10.1002/0471028959.sof329.
  11. J. D. Musa and A. Iannino and K. Okumoto, Software reliability, in Advances in computers, vol. 30, Academic Press Inc., pp. 85-170, 1990.
    CrossRef
  12. S. S. Gokhale, Architecture-based software reliability analysis: Overview and limitations, IEEE Transactions on dependable and secure computing, vol. 4, no. 1, pp. 32-40, Jan-Mar., 2007. DOI: 10.1109/TDSC.2007.4.
    CrossRef
  13. A. L. Goel, and K. Okumoto, A Markovian model for reliability and other performance measures of software systems, in 1979 International Workshop on Managing Requirements Knowledge (MARK), New York: NY, pp. 769-774, 1979. DOI: 10.1109/MARK.1979.8817248.
    CrossRef
  14. S. K. Chandran and A. Dimov and S. Punnekkat, Modeling uncertainties in the estimation of software reliability-a pragmatic approach, in 2010 Fourth International Conference on Secure Software Integration and Reliability Improvement, Singapore, Jun, pp. 227-236, 2010. DOI: 10.1109/SSIRI.2010.22.
  15. L. Cheung, and L. Golubchik, and N. Medvidovic, and G. Sukhatme, Identifying and addressing uncertainty in architecture-level software reliability modelling, in 2007 IEEE International Parallel and Distributed Processing Symposium, CA, USA: Long Beach, pp. 1-6, Mar., 2007. DOI: 10.1109/IPDPS.2007.370524.
  16. Y. S. Dai, and M. Xie, and Q. Long, and S. -H. Ng, Uncertainty analysis in software reliability modeling by bayesian analysis with maximum-entropy principle, IEEE Transactions on Software Engineering, vol. 33, no. 11, pp. 781-795, Oct., 2007. DOI: 10.1109/TSE.2007.70739.
    CrossRef
  17. W. W. Everett, Software component reliability analysis, in Proceedings 1999 IEEE Symposium on Application-Specific Systems and Software Engineering and Technology. ASSET'99 (Cat. No. PR00122), TX, USA: Richardson, pp. 204-211, 1999. DOI: 10.1109/ASSET.1999.756770.
    CrossRef
  18. L. Yin and M. A. Smith and K. S. Trivedi, Uncertainty analysis in reliability modelling, in Annual Reliability and Maintainability Symposium. 2001 Proceedings. International Symposium on Product Quality and Integrity (Cat. No. 01CH37179), PA, USA: Philadelphia, pp. 229-234, 2021. DOI: 10.1109/RAMS.2001.902472.
  19. M. A. Haque, and N. Ahmad, Modified Goel-Okumoto Software Reliability Model Considering Uncertainty Parameter, in Mathematical Modeling, Computational Intelligence Techniques and Renewable Energy, vol. 1405, pp. 369-379, 2022. DOI: 10.1007/978-981-16-5952-2_32.
    CrossRef
  20. Z. Liu and S. Yang and R. Kang, Software belief reliability growth model based on uncertain differential equation, IEEE Transactions on Reliability, pp. 1-13, Mar., 2022. DOI: 10.1109/TR.2022.3154770.
    CrossRef
  21. R. Dhaka and B. Pachauri and A. Jain, Two-Dimensional Software Reliability Model with Considering the Uncertainty in Operating Environment and Predictive Analysis, in Data Engineering for Smart Systems, vol. 238, pp. 57-69, 2022. DOI: 10.1007/978-981-16-2641-8_6.
    CrossRef
  22. C. Zhang, and A. Mostashari, Influence of component uncertainty on reliability assessment of systems with continuous states, International Journal of Industrial and Systems Engineering, vol. 7, no. 4, pp. 542-552, Apr., 2011.
    CrossRef
  23. S. V. Dhople and Y. C. Chen and A. D. Domínguez-García, A set-theoretic method for parametric uncertainty analysis in Markov reliability and reward models, IEEE Transactions on Reliability, vol. 62, no. 3, pp. 658-669, Sep., 2013. DOI: 10.1109/TR.2013.2270421.
    CrossRef
  24. P. Ju, and H. Li, and X. Pan, and C. Gan, and Y. Liu, and Y. Liu, Stochastic dynamic analysis for power systems under uncertain variability, IEEE Transactions on Power Systems, vol. 33, no. 4, pp. 3789-3799, Jul., 2018. DOI: 10.1109/TPWRS.2017.2777783.
    CrossRef
  25. Meedeniya, and I. Moser, and A. Aleti, and L. Grunske, Architecture-based reliability evaluation under uncertainty, in Proceedings of the joint ACM SIGSOFT conference--QoSA and ACM SIGSOFT symposium--ISARCS on Quality of software architectures--QoSA and architecting critical systems--ISARCS, pp. 85-94, Jun., 2011. DOI: 10.1145/2000259.2000275.
    CrossRef
  26. L. Fiondella, and S. S. Gokhale, Software reliability with architectural uncertainties, in 2008 IEEE International Symposium on Parallel and Distributed Processing, FL, USA: Miami, pp. 1-5, 2008. DOI: 10.1109/IPDPS.2008.4536436.
    CrossRef
  27. K. Go and M. Hamill and X. Wang, Adequacy, accuracy, scalability, and uncertainty of architecture-based software reliability: Lessons learned from large empirical case studies, in 2006 17th International Symposium on Software Reliability Engineering, NC, USA: Raleigh, pp. 197-203, 2006. DOI: 10.1109/ISSRE.2006.11.
    CrossRef
  28. Gashi and P. Popov and V. Stankovic, Uncertainty explicit assessment of off-the-shelf software: A Bayesian approach, Information and Software Technology, vol. 51, no. 2, pp. 497-511, Feb., 2009. DOI:10.1016/j.infsof.2008.06.003.
    CrossRef
  29. K. Goseva-Popstojanova, and S. Kamavaram, Assessing uncertainty in reliability of component-based software systems, in 14th International Symposium on Software Reliability Engineering, ISSRE 2003, CO, USA: Denver, pp. 307-320. DOI:10.1109/ISSRE.2003.1251052.
    CrossRef
  30. Goseva-Popstojanova, and S. Kamavaram, Uncertainty analysis of software reliability based on method of moments, in 13th Int’l Symp. Software Reliability Engineering, pp. 143-144, 2002.
  31. D. K. Mohanta, and D. S. Roy, Importance and uncertainty analysis in software reliability assessment of computer relay, in Proceedings of the Institution of Mechanical Engineers, Part O: Journal of Risk and Reliability, vol. 225, no. 1, pp. 50-61, Jun., 2011. DOI: 10.1177/1748006XJRR341.
    CrossRef
  32. Y. Wang, and X. Gao, and Y. Cai, and M. Yang, and S. Li, and Y. Li, Reliability evaluation for aviation electric power system in consideration of uncertainty, Energies, vol. 13, no. 5, p. 1175, Mar., 2020. DOI: 10.3390/en13051175.
    CrossRef
  33. A. K. Verma and S. Ajit and D. R. Karanki, Uncertainty analysis in reliability/safety assessment, in Reliability and Safety Engineering, pp. 457-491, Sep., 2015. DOI: 10.1007/978-1-4471-6269-8_13.
  34. Goševa-Popstojanova, and K. S. Trivedi, Architecture-based approach to reliability assessment of software systems, Performance Evaluation, vol. 45, no. 2-3, pp. 179-204, Jul., 2001. DOI: 10.1016/S0166-5316(01)00034-7.
    CrossRef
  35. C. -J. Hsu, and C. -Y. Huang, An adaptive reliability analysis using path testing for complex component-based software systems, IEEE Transactions on Reliability, vol. 60, no. 1, pp. 158-170, Mar., 2011. DOI: 10.1109/TR.2011.2104490.
    CrossRef
  36. D. V. Ravenzwaaij and P. Cassey and S. D. Brown, A simple introduction to Markov Chain Monte-Carlo sampling, Psychonomic bulletin & review, vol. 25, no. 1, pp. 143-154, Mar., 2018. DOI: 10.3758/s13423-016-1015-8.
    Pubmed KoreaMed CrossRef
  37. B. Zhou and H. Okamura and T. Dohi, Markov chain Monte Carlo random testing, in Advances in Computer Science and Information Technology, vol. 6059, Heidelberg, German.
    CrossRef
  38. F. S. Hillier, and G. J. Lieberman, A special algorithm for the assignment problem, in Introduction to Operations Research, 9th ed., McGraw-Hill Education, pp. 342-346, 2008.
  39. D. Hamlet and D. Mason and D. Woitm, Theory of software reliability based on components, in Proceedings of the 23rd International Conference on Software Engineering. ICSE 2001, ON, Canada: Toronto, pp. 361-370, 2001. DOI: 10.1109/TR.2011.2104490.[40] S. Krishnamurthy and A. P. Mathur, “On the estimation of reliability of a software system using reliabilities of its components,” in Proceedings of The Eighth International Symposium on Software Reliability Engineering, Albuquerque: NM, USA, pp. 146-155, 1997. DOI: 10.1109/ISSRE.1997.630860.
    CrossRef
  40. L. Shooman, Structural models for software reliability prediction, in Proceedings of the 2nd International Conference on Software engineering, pp. 268-280, Oct., 1976.
  41. Xie, and C. Wohlin, An additive reliability model for the analysis of modular software failure data, in Proceedings of Sixth International Symposium on Software Reliability Engineering. ISSRE'9, Toulouse, France, pp. 188-194, 1995. DOI: 10.1109/ISSRE.1995.497657.
    CrossRef
  42. F. Zhang, and X. Zhou, and J. Chen, and Y. Dong, A novel model for component-based software reliability analysis, in 2008 11th IEEE High Assurance Systems Engineering Symposium, Nanjing, China, pp. 303-309, 2008. DOI: 10.1109/HASE.2008.41.
    CrossRef
JICCE
Sep 30, 2022 Vol.20 No.3, pp. 143~233

Stats or Metrics

Share this article on

  • line
  • mail

Journal of Information and Communication Convergence Engineering Jouranl of information and
communication convergence engineering
(J. Inf. Commun. Converg. Eng.)

eISSN 2234-8883
pISSN 2234-8255