Reliability Engineering and System Safety 83 (2004) 311–321

www.elsevier.com/locate/ress

The combined use of data and expert estimates

in population variability analysis
Enrique Lopez Droguetta,*, Frank Groenb, Ali Moslehb
a
Department of Production Engineering, Federal University of Pernambuco, Rua Academico Helio Ramos, S/N,
Cidade Universitária, 50740-530 Recife, PE, Brazil
b
Reliability Engineering Program, University of Maryland, 2100 Marie Mount Hall, College Park, MD 20742, USA
Received 15 August 2003; accepted 11 October 2003

Abstract
Bayesian population variability analysis, also known as the first stage in two-stage Bayesian updating [IEEE Trans. Power Appar. Syst.
PAS-102 (1983) 195] or hierarchical Bayes [Bayesian reliability analysis, 1991], is an estimation procedure for the assessment of the
variability of reliability measures among a group of similar systems. Variability distributions resulting from this form of analysis find
application as generic prior distributions in system-specific Bayesian reliability assessments. This paper presents an extension of the
Bayesian approach to population variability analysis, which concerns the introduction of sources estimates (e.g. engineering judgment) as
one of the forms of evidence used in the construction of population variability distributions. The paper presents the model, and illustrates its
behavior by means of a practical example.
q 2003 Elsevier Ltd. All rights reserved.
Keywords: Bayes theorem; Population variability; Prior distribution; Likelihood function; Expert opinion

1. Introduction about of the system’s reliability characteristics corresponds

A common problem in the field of reliability and risk is the
assessment of the failure rate, or failure probability, of a
specific system or component. The conventional Bayesian
assessment procedure consists of the updating, or condition-
ing, of some prior probability distribution based on evidence
obtained for the specific system. The assessment procedure
results in a posterior distribution, which reflects the updated
state of uncertainty about the failure rate or probability [2,3].
In the presentation of his two-stage Bayesian procedure,
Kaplan [1] introduced a method for the construction of prior
distributions using industry-wide, partially relevant evi-
dence. This partially relevant evidence consists of data
obtained from systems or applications similar to the system
and application of interest. Due to differences in design and
operating conditions, such systems are believed to exhibit
similar, yet different reliability behaviors. Prior construction
takes place based on the notion that in absence of evidence
directly relevant to the system of interest, the uncertainty
* Corresponding author. Fax: þ 55-81-3271-8728x30.
E-mail address: ealopez@ufpe.br (E.Lopez Droguett).
0951-8320/$ - see front matter q 2003 Elsevier Ltd. All rights reserved.
doi:10.1016/j.ress.2003.10.007
to the variability of reliability characteristics observed
among similar systems. The reader may refer to Siu and
Kelly [4] for a comprehensive practical review of Bayesian
parameter estimation.
Solutions to the problem, also referred to as population
variability analysis, hierarchical Bayes, or non-homo-
geneous analysis, typically involve the use of parametric
distribution models such as Lognormal [1] or Gamma [5 – 7]
to describe the variability. Bayes Empirical Bayes methods
[8], which include Kaplan [1], Pörn [7], and Hora [6], treat it
as a Bayesian inference problem over the parameters of
these distribution models. The approach is computationally
demanding, which in the past has led to the discretization of
the parameter space [1] or the use of conjugate variability
models [6,7].
Alternative methods that do not fall within this class of
fully Bayesian methods have been proposed as well. In
Frohner [9], the variability is constructed by superimposing
posterior distributions obtained for the individual popu-
lations. Vaurio [5] presents a moment-matching procedure
for the selection of a single Gamma distribution. It has been
recognized however that these approaches have some severe
312 E.Lopez Droguett et al. / Reliability Engineering and System Safety 83 (2004) 311–321

conceptual limitations, since, in their attempts to make their
approaches computationally less demanding, they ‘cut out
the heart of the problem’ [10].
Another method is that of Hofer [11], which was
presented as a way to avoid an alleged flaw of the Bayesian
methods [1,6,7]. The flaw consists of a tendency to return
distributions that tend towards those obtained under a
common failure rate assumption, and which depend only on
the total observed number of failures and total observation
time rather than the data separated by the individual systems
[12]. The flaw is attributed to the fact that the likelihood
model represents an incorrect chance mechanism.
The authors of this paper dispute however the claim that
the original formulation of the variability distribution model
[1,6,7] is wrong [11]. The probabilistic model underlying
the original formulation, clearly illustrated by Pörn [7], fits
exactly the description characterized by Hofer [11] as the
‘right chance mechanism’. The behavioral tendency arises
only in cases where the posterior densities are not proper.
Since the exercises in this paper are limited to proper
posterior densities, the issue is not further considered.
In the aforementioned approaches, the population
variability distribution is developed based on run-time
data. However, failure rate or probability of failure
estimates or distributions may also be available and
constitute another valuable source of information,
especially in situations where actual data is limited or
missing [13]. Therefore, if we use X to denote the reliability
measure of interest (e.g. failure rate, failure probability), the
assessment of the population variability distribution of X is
based on the following types of information
† E0 : the analyst prior state of knowledge such as
engineering experience with the system or component;
† E1 : exposure data (run-time or demand-based data) from
operating experience with similar systems in similar
applications;
† E2 : estimates or distributions about the reliability
measure X from various sources such as reliability
databases and engineering judgments.
Type E2 information usually consists of estimates from
experts opinions with engineering knowledge about the
design, manufacturing, and operation of the system to
estimates based on observed performance of the same class
of systems in similar applications. Note that this type of
information are usually a point estimate, referred to as the
‘best estimate’, or a range of values centered about a best
estimate. Alternatively, a distribution can be provided
expressing the range of values for the reliability measure
with mean or median representing the best estimate of the
source. As we discuss in the following sections, we also
consider an assessment of the analyst’s confidence in the
accuracy of the provided estimate.
As an example, Table 1 shows actual information
gathered for normally operating motor-driven pumps
comprising of estimates from four different sources as
well as run-time data from eight different plants. The
question is now what can we say about the population
variability distribution of X based on the available
information sets E0 ; E1 ; and E2 :
Mosleh and Apostolakis [14] originally addressed the
problem by means of a Bayesian population variability
methodology for assessing population variability curves
based on both exposure data and expert estimates infor-
mation. The methodology is however limited because it
decoupled exposure data and expert estimates, i.e. for single
systems, it allows only one type of information to be used.
Furthermore, applications of the methodology were carried
out using a discretized parameter space.
In this paper, we present an extension of the Bayesian
population variability analysis method that addresses the
limitations of the Mosleh and Apostolakis approach.
In particular, it concerns a capability to assess population
variability distributions based on both exposure data and

Table 1
Generic information used in estimating the failure rate distribution for motor-driven pumps (normally operating)

Data source System type Estimate Assigned range factor Number of failures Operating time (h)

Estimates
Source 1 Pump 3.0 £ 1025 5
Source 2 Alternating pump 2.1 £ 1025 3
Source 3 Motor-driven pump 2.0 £ 1025 10
Source 4 Cooling systems pumps 2.53 £ 1025 10
Plant data
Plant 1 Centrifugal charging pump 0 7.60 £ 104
Plant 2 Service water pump 0 1.52 £ 105
Plant 3 Component cooling pump 0 7.40 £ 104
Plant 4 Service water pump 2 7.40 £ 104
Plant 5 Component cooling pump 0 4.80 £ 104
Plant 6 Service water pump 3 7.60 £ 104
Plant 7 RHRSW/EECW pump 9 1.02 £ 104
Plant 8 Service water pump 2 4.80 £ 104

Failure mode: failure during operation.

 E.Lopez Droguett et al. / Reliability Engineering and System Safety 83 (2004) 311–321 313

estimates information, but allowing for decoupled and a given family of parametric distribution models. In
coupled situations, i.e. it is possible to handle both types of situations where the true variability distribution is uncertain,
information when available from the same system. this uncertainty is expressed in the form of a probability
The extension involves the use of non-conjugate distribution over the members of that family of models.
variability models, i.e. choices of the population variability If we use wðxÞ ¼ wðxlu1 ; …; ur Þ to denote the parametric
distribution and the likelihood function for the evidence that variability distribution model, then a probability distribution
do not lead to a variability model that can be evaluated pðu1 ; …; ur Þ over the parameters of the model can be used to
analytically, specifically constructed for reliability describe the uncertainty about the population variability
measures of distinct nature such as failure rate and failure distribution. The estimated population variability density is
probability. Numerical integration is therefore required to taken as
compute likelihood functions. Furthermore, the extension ð ð
operates over the continuous, rather than the discretized, p^ ðxÞ ¼ ··· wðxlu1 ;…; ur Þpðu1 ;…; ur Þdu1 ···dur ð1Þ
u1 ;…;ur
variability model parameter space.
Together, these factors make solution of the problem The estimated density function therefore consists of a
computationally demanding, to the extent that it has been weighted mix of distributions of the chosen model, as
described as overwhelmingly difficult [6]. However, opposed to being formed by a single ‘best’ distribution
advances in computer technology have made practical chosen from the set of distributions possible within the
application of the approach feasible on personal computers, definition of the model, e.g. a Maximum Likelihood
evidenced by the fact that the extension is part of a software estimator.
tool developed by the authors, recent versions of which have Population variability distributions are used as ‘generic’
seen applications in the space and nuclear industries. prior distributions in system-specific Bayesian assessment
The remainder of this paper is organized as follows. procedures [1], which is justified according to the discussion
Section 2 provides a brief overview of the representation of in Deely and Lindley [8]. Let pðu1 ; …; ur lE0 ; E1 ; E2 Þ be a
population variability distributions, their use in system- probability distribution over the parameters of the varia-
specific assessment problems, and the presentation of the bility model wðxlu1 ; …; ur Þ; conditional on E0 ; E1 ; and E2 ;
extended Bayesian model for the integrated treatment of and let E0 ; E1 ; and E2 not include any information relevant
exposure data and estimates information. Section 3 then to the system of interest. Then the system-specific
presents the likelihood construction based on a mix of data distribution over X; conditional on E0 ; E1 ; E2 and system-
and sources estimates. Section 4 consists of a discussion of specific evidence E3 is given by
the implementation of the procedure, which includes the
specification of an informed prior. Example applications are pðxlE0 ;E1 ;E2 ;E3 Þ
given in Sections 5 and 6. ð
PðE3 lx;E0 ;E1 ;E2 Þ wðxluÞpðulE0 ;E1 ;E2 Þdu
u   
¼Ð ð ð2Þ
2. Integrated treatment of exposure data and estimates x PðE3 lx;E0 ;E1 ;E2 Þ wðxluÞpðulE0 ;E1 ;E2 Þdu dx
u   
in population variability assessments 
where u ¼ {u1 ; …; ur }: The posterior distribution of the

The failure rate of a system is an often used character- population variability parameters based on information
ization of our expectations regarding the system’s ability to types E0 ; E1 ; E2 is developed by applying the Bayes’
perform its intended function. As discussed by Singpurwalla theorem
[3], failure rates are not objective entities, but rather are PðE1 ; E2 lu; E0 Þp0 ðulE0 Þ
expressions of our personal uncertainty about the system’s pðulE0 ; E1 ; E2 Þ ¼ ð   ð3Þ

failure behavior over time. These expressions can be PðE1 ; E2 lu; E0 Þp0 ðulE0 Þdu
u   
formulated for individual systems within their particular 
applications, even when the system or application is unique. where PðE1 ; E2 lu; E0 Þ is the likelihood of the information,

Given that we can associate failure rates with individual and p0 ðulE0 Þ is prior probability distribution on u:
 
systems, we can also consider the variability of failure rates Assuming that exposure data E1 and sources estimates E2
among a population of systems. A representation of this are independent, the likelihood function becomes
variability, in the form of a probability distribution, is PðE1 ; E2 lu; E0 Þ ¼ PðE1 lu; E0 ÞPðE2 lu; E0 Þ where the first
  
referred to as the population variability distribution. The member of the right side is the likelihood of the exposure
assessment of such population variability distributions, data, and the second is the likelihood of the estimates
referred to as population variability analysis, is the topic of information. Moreover, writing this equation in terms of the
this paper. likelihoods of the information for each system we have
As is common in population variability analysis Y
n
procedures, we will assume that the true population PðE1 ; E2 lu; E0 Þ ¼ PðE1i lu; E0 ÞPðE2i lu; E0 Þ ð4Þ
variability distribution can be described by a member of  i¼1
 
314 E.Lopez Droguett et al. / Reliability Engineering and System Safety 83 (2004) 311–321
where PðE1i lu; E0 Þ and PðE2i lu; E0 Þ is the probability of
 
observing evidence E1i and E2i ; respectively, for the ith out
of n systems assuming that the set of parameters of the
population variability curve is u:

Note that the reliability measure for the ith system, xi ; is
not known exactly. All we know is that xi is one of possibly
many values of variable X: In addition, according to our
model, X is distributed according to wðxluÞ; with u also
 
being unknown. Therefore, we calculate the probability of
observing the information E1 and E2 by allowing the
reliability measure to assume all possible values, i.e. by
averaging PðE1i ; E2i lu; E0 Þ over the distribution of X

ð the likelihood function is based on a Poisson model to solely
PðE1i ; E2i lu; E0 Þ ¼ PðE1i lu; E0 ÞPðE2i lu; E0 ÞwðxluÞdx ð5Þ
 x   
which can be substituted in Eq. (4) to obtain the likelihood
function of the total information.
By writing the likelihood as in Eq. (5), the estimate and
the exposure data are coupled. In other words, the source’s
estimate of the reliability measure is for the same system for
which exposure data have been observed. Obviously, two
special cases of Eq. (5) arise when only one type of
information is available for the ith system.
Ð In fact, in case of
exposure data only, theÐlikelihood is x PðE1i lu; E0 ÞwðxluÞ
 
dx; and it becomes x PðE2i lu; E0 ÞwðxluÞdx when just
 
estimate information is available for the ith system.
It is important to note that the likelihood as written in
Eq. (4) also allows for the decoupled treatment of
information types E1 and E2 : For example, given exposure
data for system i and an estimate for system j; the likelihood
for these two pieces of evidence can be written as
PðE1i lu; E0 ÞPðE2j lu; E0 Þ
 
ð ð  † Type E1 (exposure evidence): number of failures and
¼ PðE1i lu; E0 ÞwðxluÞdx PðE2j lu; E0 ÞwðxluÞdx
x   x  
where source’s estimate (e.g. engineering judgment) is for
the jth system, being different from the ith system for which
exposure data have been gathered. Therefore, it can be said
that the approach presented by Mosleh [14] is a special case
of the procedure presented in this paper.
In Section 3, the general mixed likelihood presented in
this section is specialized to match specific applications in
population variability assessment, in particular when the
reliability measure corresponds to the failure rate and
exposure data is in the form of run-time data, and then for
the case of failure probability and exposure data given as
demand-based data.
3. Mixed likelihood model
To perform a population variability analysis of a
reliability measure we need to specify an appropriate
probability distribution to describe the underlying varia-
bility of the measure of interest, wðxluÞ; as well as construct

the likelihood function PðE1 ; E2 lu; E0 Þ: While the specifica-

tion of the probability distribution describing the variability
may be guided by the nature of the reliability measure (e.g.
Gamma or Lognormal distribution for failure rate, Beta
distribution for probability of failure), the likelihood
construction is an evidence-driven process, i.e. it is
dependent on the type of available evidence.
In general, current population variability procedures only
utilize exposure data to construct the likelihood function.
For example, in assessing the failure rate variability, Kaplan
[1], Frohner [9], Vaurio [5], and Pörn [7] provide different
formulations for the population variability assessment but
account for run-time data (number of failures and the time to
observe them). However, there are some situations where
exposure data of similar systems is incomplete or absent.
This is particularly true for newly installed and/or highly
reliable systems where complementary field data infor-
mation is generally scarce: similar operational systems
might be available, but field data for some units may be
lacking, i.e. no failures have been observed. Another
situation of interest concerns the lack of standardized and
systematic data collection systems, which leads to useless
observed field data on similar units. In situations like these,
however, operational experience is available and constitute
a valuable source of additional information to supplement
incomplete or lacking exposure data on similar systems.
Indeed, in this section we specialize the ‘mixed’
likelihood function given by Eq. (5) for different forms of
exposure data, sources’ estimates, and nature of the
reliability measure of interest. We start by formalizing the
form of information types E1 and E2 as follows
exposure (time in service or number of demands). For
example, {ðki ; ti Þ; i ¼ 1; 2; …; n} or {ðki ; di Þ; i ¼
1; 2; …; n} where ki is the number of failures, ti is the
time to observe the ki failures in the i system, di in the
total number of demands, and n is the total number of
systems.
† Type E2 (sources’ estimates): it comes in the form of
estimates of possible values of a reliability measure. We
consider that the analyst’s consults n independent sources
and that his measure of confidence in each source is
represented by the multiplicative error model [15]. The
evidence is then in the form of {ðlpi ; si Þ; i ¼ 1; …; n};
where is the estimate provided by the ith source, and si is
the logarithmic standard deviation of representing the
uncertainty of source i; which represents the analyst’s
subjective measure of confidence in the ith expert.
The various versions of the mixed likelihood functions
based on a particular reliability measure (e.g. failure
rate, probability of failure) and different forms of exposure
data (see Table 2), are developed in Sections 3.1
and 3.2. Sections 3.1 and 3.2 present the development of
 E.Lopez Droguett et al. / Reliability Engineering and System Safety 83 (2004) 311–321 315

Table 2 Eq. (9) as

 
Mixed likelihood models  dlpi 
Pðmi lsi ; l; E0 Þ ¼ Pðlpi lsi ; l; E0 Þ
ðki ; ti Þ ðlpi ; si Þ ðki ; di Þ dmi 

Gamma Gamma– Poisson–Lognormal

and provided that ldlpi =dmi l ¼ emi ;
Lognormal Lognormal–Poisson– Lognormal
"   #
1 1 mi 2 ln li 2 mi
Lognormal–Binomial–Lognormal Pðmi lsi ; l; E0 Þ ¼ pffiffiffiffi m exp 2 e
Beta Beta –Binomial–Lognormal 2psi e i 2 si
therefore
the Lognormal –Poisson– Lognormal and the Lognormal – "   #
Binomial – Lognormal models, respectively. We also show 1 1 mi 2 ln l 2
Pðmi lsi ; l; E0 Þ ¼ pffiffiffiffi exp 2 ð10Þ
that the data- and expert-based likelihoods are special cases 2psi 2 si
of the general mixed likelihood models. Alternative
formulations such as the Gamma – Poisson – Lognormal which is a Normal distribution with mean equals to l:
and Beta – Binomial –Lognormal models follow straightfor- As we only know that l is one of the possible values of
ward and are not developed here. the failure rate represented by its population variability
distribution wðlluÞ; we average the likelihood given by
3.1. Lognormal –Poisson – Lognormal model 
Eq. (5) over all possible values of l in order to calculate the
probability of the data unconditional on the unknown value
Let us assume that we are interested in assessing the of l
population variability of an item failure rate, l; and the
available sources of evidence are run-time data {ðki ; ti Þ; i ¼ Pðki ; ti ; mi ; si ln; t; E0 Þ ¼ Pi
1; …; n} and estimates on the unknown L given by different ð
sources ðlpi ; si Þ where lpi and si are defined as before. We ¼ Pðki lti ; l; E0 ÞPðmi lsi ; l; E0 Þwðlln; tÞdl ð11Þ
l
further consider that the population variability of the
unknown L is given by a Lognormal distribution, i.e. where we have considered run-time data and estimates
"   # evidence as independent. Note also that the ith expert
1 1 ln l 2 n 2 estimate lpi is for the same ith system for which run-time
wðlly ; tÞ ¼ pffiffiffiffi exp 2 ð6Þ
2plt 2 t data ðki ; ti Þ have been observed.
Replacing Eqs. (6) and (10) into Eq. (11):
Therefore, the posterior of the variability distribution
ð 1
parameters in Eq. (3) can be written as Pi ¼ Pðki lti ; l; E0 Þ
l 2p s i tl
pðn; tlE0 ; E1 ; E2 Þ ( "    #)
1 mi 2 ln l 2 ln l 2 n 2
PðE1 ; E2 lE0 ; n; tÞp0 ðn; tlE0 Þ exp 2 þ dl ð12Þ
¼ ð ð ð7Þ 2 si t
PðE1 ; E2 lE0 ; n; tÞp0 ðn; tlE0 Þdn dt
n t One possible solution to Eq. (12) can be obtained if we write
( "    #)
For the ith system, the likelihood of type E1 information, 1 mi 2 ln l 2 ln l 2 n 2
PðE1i lu; E0 Þ; can be constructed in the following way. If we exp 2 þ as
 2 si t
know the failure rate li ¼ l of each system, we can use the
" !2 #
Poisson distribution to estimate the likelihood of observing 1 m0i 2 ln l
ki failures in Ti : exp 2
2 s0i
ðltÞki e2ti l
Pðki lti ; l; E0 Þ ¼ ð8Þ In fact, it can be shown that
Gðki þ 1Þ
n m
vffiffiffiffiffiffiffiffiffiffiffiffiffiffi þ 2i
Following the multiplicative error model proposed by u 1 t 2 s
Mosleh [15], the likelihood function for the ith source s0i ¼ u
u 1 and m0i ¼ i
ð13Þ
t 1 1 1
estimate, PðE2i lu; E0 Þ; can be written in terms of a þ 2 þ 2
 t2 si s2i t
Lognormal distribution with median ln l; i.e.
"   # Thus, the likelihood function given by Eq. (12) is
p 1 1 ln lpi 2ln l 2 ð ðlt Þki e2ti l
Pðli lsi ; l;E0 Þ ¼ pffiffiffiffi p exp 2 ð9Þ Pi ¼ i 1
2psi li 2 si l Gðki þ 1Þ 2psi tl

where we have considered an unbiased expert (for the case ( " !2 #)

1 m0i 2 ln l ðmi 2 nÞ2
of biased expert, please refer to Mosleh [14]). Note that if exp 2 þ 0 dl
we consider the transformation mi ¼ ln lpi ; we can write 2 s0i s i þ t2
316 E.Lopez Droguett et al. / Reliability Engineering and System Safety 83 (2004) 311–321

which after some algebraic manipulation becomes The population variability of the failure probability is
" # considered to follow a Lognormal distribution
1 ðmi 2 nÞ2
exp 2 "
2 s0i 2 þ t2 ð tiki lki 21 e2ti l   #
Pi ¼ 1 1 ln q 2 n 2
2psi t l Gðki þ 1Þ
wðqln; tÞ ¼ pffiffiffiffi exp 2 ð16Þ
2pqt 2 t
" !2 #
1 m0i 2 ln l
exp 2 dl ð14Þ In order to evaluate the posterior distribution of the
2 s0i
population variability parameters, we need to evaluate
Note that the previous equation can also be written as Eq. (7), but the construction of the likelihood function given
a product of a Gamma and a Lognormal distributions. by Eq. (5) should now take into account the demand-based
Indeed exposure data instead of run-time data. Following the same
! argument as in the previous case, the likelihood for the
0 1 ðmi 2 nÞ2 demand-based data is given by a Binomial distribution as
si exp 2 ð tki þ1 lðki þ1Þ21 e2ti l follows
2 s0i þ t2 i 1
I¼ pffiffiffiffi pffiffiffiffi 0
2psi tti l Gðki þ 1Þ 2psi l Gðni þ1Þ
Pðki lni ;q;E0 Þ ¼ qki ð12qÞni 2ki ð17Þ
" !2 # Gðki þ1ÞGðni 2ki þ1Þ
1 m0i 2 ln l
exp 2 dl and by a Lognormal distribution (Eq. (10)) for the sources
2 s0i
estimates, where we just need to write the failure probability
q in place of the failure rate l: For the ith system, the mixed
Substituting the expressions for m0i and s0i given by Eq. (13), likelihood function can then be written as
we have
! Pi ðki ; di ; mi ; si ln; t; E0 Þ ¼ Pi
1 ðmi 2 nÞ2
exp 2
2 s2i þ t2 ð 1ð
Pi ¼ sffiffiffiffiffiffiffiffiffiffiffiffi Gaðllki þ 1; ti Þ ¼ Pðki ldi ; q; E0 ÞPðmi lsi ; q; E0 Þwðqln; tÞdq ð18Þ
pffiffiffiffi 1 1 l c q
2psi tti þ 2
s2i t where c is a normalization factor given by
0 1 "   #
ð1 1 1 ln q 2 n 2
mi vffiffiffiffiffiffiffiffiffiffiffiffiffiffi pffiffiffiffi exp 2
B t 2 þ s2 u
n C c¼ dq
B u 1 C 2pqt 2 t
LNBll 1 i
; u Cdl ð15Þ 0
@ þ 1 t 1 1 A
s2i t2 þ Replacing Eqs. (16), (17), and (10) into Eq. (18), the
t2 s2i
likelihood for the ith system becomes
The total likelihood is then obtained by replacing Eq. (15) !
into Eq. (4). Note that by doing this, in case of sources 1 ðmi 2 nÞ2
exp 2 ð
estimates (e.g. engineering judgments), we have con- 1 2 s2i þ t2 Gðni þ 1Þ
sidered the sources to be independent. However, the Pi ¼ sffiffiffiffiffiffiffiffiffiffiffiffi
c pffiffiffiffi 1 1 q Gðki þ 1ÞGðni 2 ki þ 1Þ
mixed likelihood model can easily be extended to 2psi t þ 2
2
consider dependent sources by means of copula func- si t
tions. For instance, in the context of experts opinions, " !2 #
Jouini and Clemen [16] propose the use of Archimedean ki ni 2ki 1 1 m0i 2 ln q
 q ð1 2 qÞ pffiffiffiffi 0 exp 2 dq
copulas for the treatment of dependent experts, and 2psi q 2 s0i
Droguett and Mosleh [17] develop a methodology for the
treatment of dependent sources of information based on which, after some algebraic manipulation, can be written as
copula functions in the context of model uncertainty. !
1 ðmi 2 nÞ2
Gðni þ 1Þexp 2
3.2. Lognormal – Binomial –Lognormal model 1 2 s2i þ t2
Pi ¼ sffiffiffiffiffiffiffiffiffiffiffiffi
c pffiffiffiffi 1 1
Assume now that we are interested in assessing the Gðni þ 2Þ 2psi t þ 2
failure probability of an item, qð0 # Q # 1Þ: We consider s2i t
two types of information: demand-based data ð Gðni þ 2Þ
qðki þ1Þ21 ð1 2 qÞðni 2ki þ1Þ21
{ðki ; di Þ; i ¼ 1; …; n} and estimates on the unknown Q q Gðki þ 1ÞGðni 2 ki þ 1Þ
given by different sources ðqpi ; si Þ where qpi is the source’s " !2 #
1 1 m0i 2 ln q
estimate on Q and si is the logarithmic standard deviation of £ pffiffiffiffi 0 exp 2 dq ð19Þ
estimate qpi : 2psi q 2 s0i
 E.Lopez Droguett et al. / Reliability Engineering and System Safety 83 (2004) 311–321 317

Note that Eq. (19) can also be written as a product between a In case of the Lognormal variability model, priors
Beta and a Lognormal distributions: pLN ðx50 Þ and pLN ðEFÞ specified over the median x50 ¼ en
! and error factor EF ¼ e1:645t : In doing so, we recognize but
1 ð m i 2 nÞ 2 ignore the fact that EF is naturally limited to values higher
exp 2
2 s2i þ t2 than one, on the basis that typical values of the error factor
Pi ¼ sffiffiffiffiffiffiffiffiffiffiffiffi are normally well above one. The prior density over the
pffiffiffiffi 1 1
cðni þ 1Þ 2psi t þ 2 model’s parameter space is then found by applying the
s2i t
standard density transformation
ð  
 Beðqlki þ 1; ni 2 ki þ 1ÞLNðqlm0i ; s0i Þdq ð20Þ  ›ðx50 ; EFÞ 

pðx50 ; tÞ ¼ pLN ðx50 ÞpLN ðEFÞ 
›ðx50 ; tÞ 
q

¼ pLN ðx50 ÞpLN ðEFÞ £ 1:645 e1:645t ð22Þ

4. Implementation of procedure

The mixed likelihood models described in Section 3 have 4.2. Variability measures
been incorporated into a Bayesian data analysis tool
developed by the authors. In this section, we summarize The likelihood functions and prior distributions have
the elements of the implementation other than the likelihood been incorporated in a Bayesian inference procedure in
models. which the posterior density pðulEÞ is computed. The

Bayesian inference is performed using a Markov Chain
4.1. Prior specification Monte Carlo Method (MCMC), which allows samples to be
generated from a continuous unnormalized density [21].
As discussed earlier in this paper, population variability The MCMC method, which is frequently applied to
analysis can be used to construct prior distributions for Bayesian Inference problems [22], results in a sample set
system-specific reliability analyses based on data and expert S ¼ {u1 ; …; um }; representing the posterior density over the
judgments available for systems that have similar, though  
parameters of the variability distribution model wðxluÞ; u ¼
not necessarily equal reliability behavior. Application of  
{u1 ; …; un }:
this technique does however not remove the need to Given S; the estimated population variability density is
specify any prior distribution: the first stage of the computed as
analysis requires a prior distribution to be defined over the
ð 1 Xm
parameter space of the chosen variability model. This prior p^ ðxÞ ¼ wðxluÞdu ¼ wðxlui Þ ð23Þ
therefore represents the prior belief about the variability u   m i¼1 

distribution, as opposed to the prior uncertainty about the
reliability measure of the system of interest. The corresponding mean and variance are computed as
Kaplan [1] proposes that the prior be specified in the ð 1 Xm
form of a discrete prior distribution. Specifically, he m^x ¼ x^pðxÞdx ¼ mðu Þ ð24Þ
x m i¼1  i
suggests the construction of a discrete grid
ð 1 Xm
ni ¼ n1 ; …; nI tj ¼ t1 ; …; tJ ð21Þ s^2x ¼ ðx 2 m^x Þ2 p^ ðxÞdx ¼ s2 ðui Þ ð25Þ
x m i¼1 
where n and t are the parameters of the variability model,
where mðuÞ and s2 ðuÞ are the mean and variance of
followed by the assignment of a probability distribution  
wðxluÞ:
over the elements of the grid. This technique has been 
Furthermore, the generated results include uncertainty
applied in actual risk studies, e.g. Seabrook [18], and is part
bounds of the cumulative variability density
of algorithms such as those integrated into RISKMAN [19].
The use of non-informative priors in two-stage Bayesian ðx
PðxÞ ¼ pðxÞdx ð26Þ
analyses is discussed for instance in Refs. [7,20]. x¼0
The algorithm developed by the authors involves the
specification of an informed, continuous prior over the in the form of z-percentiles Pz ðxÞ; defined as
parameter space of the variability model. The analyst is z
PrðPðxÞ , Pz Þ ¼ ð27Þ
required to provide initial estimates in terms of a central 100
values and the extent of variability in the population The z-percentiles are determined by finding the value P^ z ðxÞ
variability distribution. Using a motivation similar to the for which a for a fraction z=100 of the samples ui [ S
one underlying the formulation of the error model discussed 
ðx
in Section 3, these estimates take the form of Lognormal ^
wðxlui Þ , Pz ðxÞ ð28Þ
distributions. x¼0 
318 E.Lopez Droguett et al. / Reliability Engineering and System Safety 83 (2004) 311–321

Table 3
Run-time data and engineering judgment data used in the example
application

Failures Exposure (h) Estimates EF

0 918.6 6.4 £ 1025 5

0 336.2 1.3 £ 1024 5
0 352.8 2.0 £ 1024 5
0 905.2 2.7 £ 1024 5
0 931.2 3.5 £ 1024 5
0 307.9 4.3 £ 1024 5
0 885.2 5.3 £ 1024 5
0 522.0 6.4 £ 1024 5
0 451.5 7.7 £ 1024 5
1 694.8 9.2 £ 1024 5
0 374.2 1.1 £ 1023 5
1 464.3 1.3 £ 1023 5
Fig. 1. Plots of the various estimation results.
0 167.1 1.6 £ 1023 5
These bounds provide the analyst with a basis to assess the 1 479.7 1.9 £ 1023 5
2 654.8 2.3 £ 1023 5
uncertainty associated with the estimated population 2 637.7 2.9 £ 1023 5
variability distribution. 3 860.2 3.7 £ 1023 5
These measures are illustrated in Fig. 1. Fig. 1a 1 285.6 5.0 £ 1023 5
represents the joint posterior density over the parameters 2 286.4 7.5 £ 1023 5
u: Fig. 1b shows how each point u corresponds to a unique 5 340.6 1.6 £ 1022 5
 
distribution wðxluÞ: Averaging over these distributions leads

to the population variability distribution p^ ðxÞ: Furthermore,
by considering the corresponding cumulative distributions, The result of the population variability analysis is shown in
it is possible to construct the uncertainty bounds around PðxÞ Fig. 2, and the corresponding estimated parameter density in
as shown in Fig. 1c. Fig. 3. The Fig. 2 shows the estimated cumulative
population variability density function, along with the
corresponding 5 and 95% uncertainty bounds, as defined
5. Example application
in Eq. (28), as well as the distributions from which the
sample data was generated.
In this section we illustrate and discuss the use of the
The Fig. 2 shows that the estimated variability
mixed likelihood models by means of an example. We start
distribution forms a reasonable approximation of the
by supposing that we are interested in assessing the
true distributions, particular for higher values of the
population variability of the failure rate of a particular
failure rate. At lower failure rates, the estimated
class of systems. Furthermore, it is assumed that the failure
distribution deviates more significantly from the theor-
rate variability is distributed according to a Lognormal
etical distribution however. Furthermore, the uncertainty
distribution.
bounds found at the lower failure rate values are wider
In order to have a reference to check the behavior of
than they are for higher values.
the mixed model, run-time data as well as engineering
The behavior can be explained by the fact that this part
estimates type evidence were generated from a lognormal
of the distribution concerns those systems for which no
distribution, Eq. (6), with x50 ¼ en ¼ 1:0 £ 1023 h21 and
t ¼ 1:4: The data used for the example is shown in
Table 3. The data was analyzed using the Lognormal –
Poisson– Lognormal model.
Initially, an analysis was performed based on just the
run-time data. The data, shown in Table 3 contains 18
failures observed during a total of 10,856 h of operation. No
data was observed for 11 out of the 20 systems. The prior
distribution was chosen as
pðx50 ; EFÞ ¼ pLN ðx50 l1023 ; 100ÞpLN ðEFl10; 10Þ
where pLN ðxlx50 ; EFÞ is the lognormal density
"   #
1 1 ln x 2 ln x50 2
pLN ðxlx50 ; EFÞ ¼ pffiffiffiffi exp 2
ln EF 2 ln EF=1:645
2px
1:645 Fig. 2. Cumulative population variability density based on run-time data only.
 E.Lopez Droguett et al. / Reliability Engineering and System Safety 83 (2004) 311–321
Fig. 3. Posterior distribution of the population variability parameters for
run-time data only.
failures were observed, and for which no good estimate of
the failure rate can be obtained. This statement is
supported by the results of a second analysis, in which
engineering judgments, in the form of estimates, were
added to the pool of evidence. The resulting variability
distribution is shown in Fig. 4. The figure shows a good
correspondence between the estimated and theoretical
distributions, both for low and high values of the failure
rates. Furthermore, the uncertainty is less, particularly at
the lower failure rates.
These examples show that by considering the uncertainty
bounds around the estimated population variability distri-
bution, it is in fact possible to distinguish between regions of
the distribution where we believe to have a good estimate
and those where we have a poor estimate. The amount of
uncertainty is a reflection of the evidential strength of the
available data. As shown in the example, engineering
judgments can be used to supplement data, in case few or no
failures were observed.
As discussed earlier, the extension presented in this
paper involves the use of non-conjugate models and
Fig. 4. Cumulative population variability density based on combination of
run-time data and engineering judgments.
319
the consideration of a continuous parameter space, where
earlier papers would either resort to discretization of the
parameter space or the use of conjugate models in order to
reduce computational load. However, during the exper-
iments described in this section, the non-conjugate
Lognormal – Poisson – Lognormal likelihood model,
applied to the data of Table 3, was computed at an
average rate of approximately 300 evaluations/s on a
1.7 MHz Pentium IV computer. This is sufficiently fast to
allow Bayesian population variability analysis to be
performed in a practical setup.
To justify the increased computation time, the run-
time data of Table 3 was analyzed using the conjugate
Gamma– Poisson model [7]. The prior in this case was
chosen as
pðm; 1Þ ¼ pLN ðml2:7 £ 1023 ; 100ÞpLN ð1l2:5; 10Þ
where m represents the mean a=u of the Gamma
distribution, and 1 is the standard error, defined as
s 1
1¼ ¼ pffiffi
m a
The results of this analysis was compared with the
results obtained for the lognormal variability model,
bearing in mind that the data was generated from a
lognormal distribution. Results of the comparison are
shown in Fig. 5. The figure shows the estimated
population variability distribution and uncertainty bounds
for both cases.
As expected, the estimated population variability distri-
bution generated using the lognormal variability model is a
far better estimate of the theoretical distribution than the
result obtained using the Gamma model. The latter estimate
is especially inaccurate for lower values of the failure rate,
where a significant underestimation of failure rate values
can be observed. We conclude therefore that the choice of
the distribution model should not be solely based on
mathematical tractability.
Fig. 5. Comparison of population variability distribution estimates
generated using Lognormal– Poisson (LP) and Gamma–Poisson (GP)
models.
320 E.Lopez Droguett et al. / Reliability Engineering and System Safety 83 (2004) 311–321

Table 4
Mean and percentiles of the expected population variability distribution for
the motor-driven pumps

Value Expected population variability

Mean 6.05 £ 1025

1st 3.16 £ 1027
5th 1.38 £ 1026
10th 2.67 £ 1026
50th 1.61 £ 1025
90th 7.79 £ 1025
95th 1.36E £ 1024
99th 4.85 £ 1024
6. Example: estimating the failure rate distribution of
motor-driven pumps
In a final example, the procedure is applied to a
realistic data set obtained for motor-driven pumps, see
Table 1. In this section, we go back to the initial question
posed in Section 1: what can be said about these pumps
given the available information. This question is
addressed by developing a failure rate distribution based
on the sources estimates information and on the run-time
data in Table 1. To do so, the ‘mixed’ Lognormal –
Poisson– Lognormal is employed.
Table 1 lists estimates obtained from four different
sources as well as run-time data for several pumps such as
centrifugal charging pump and service water pumps. Note
that the estimates and the run-time data are decoupled, i.e.
the sources estimates are for systems different from the ones
for which the exposure evidence was obtained. Moreover,
the different error factors associated to the sources estimates
reflect the different degrees of confidence of the analyst in
the accuracy of the estimates.
Upon solving the Lognormal–Poisson–Lognormal based
on the available information, the resulting expected failure
rate population variability distribution has a 5th and 95th
percentiles of 1.38 £ 1026 and 1.36 £ 1024, respectively,
with a mean of 6.05 £ 1025. Table 4 shows these and other
Fig. 7. Expected 5th, 50th, and 95th percentile curves of the failure rate
population variability for the motor-driven pumps.
percentiles of the expected variability curve, and Figs. 6 and 7
show the corresponding PDF and the CDF curves, respect-
ively. Also shown in Fig. 7 are the 5th and 9th percentile
curves of the failure rate population variability CDF.
7. Concluding remarks
The models presented in this paper represent an
extension of the fully Bayesian population variability
assessment method. The proposed extension allows for the
assessment of population variability distributions based not
only on exposure data but also on estimates from various
sources such as engineering judgments and reliability
databases about a reliability characteristic of similar
systems. Furthermore, the proposed models are capable to
deal with different types of reliability characteristics, such
as failure rate and probability of failure, and run-time as well
as demand-based field data. It has been shown that the use of
estimates information can lead to a reduction of the
uncertainty on the population variability distributions,
particularly in situations where few or no failures have
been observed.
Even though the extension results in the use of non-
conjugate models, and thus an increase in the compu-
tational requirements, the implementation of the extension
used in the examples in Sections 5 and 6 has shown that
the numerical solution of these models is practically
feasible. A comparison of results obtained using the non-
conjugate extension, and a conjugate population varia-
bility model, showed that the results were significantly
different, indicating that the choice of a variability
distribution model should not be based solely on
mathematical tractability.

References

[1] Kaplan S. On a ‘two-stage’ Bayesian procedure for determining

Fig. 6. Expected failure rate population variability PDF for the motor- failure rates from experimental data. IEEE Trans Power Appar Syst
driven pumps. 1983;PAS-102(1):195–9.
 E.Lopez Droguett et al. / Reliability Engineering and System Safety 83 (2004) 311–321
[2] Martz H, Waller. Bayesian reliability analysis. Florida: Krieger
Publishing Company; 1991.
[3] Singpurwalla ND. Foundational issues in reliability and risk analysis.
SIAM Rev 1988;30(2):264 –82.
[4] Siu NO, Kelly DL. Bayesian parameter estimation in probabilistic risk
assessment. Reliab Engng Syst Safety 1998;62(1 –2):89–116.
[5] Vaurio J. On analytic empirical Bayes estimation of failure rates, plus
discussion. Risk Anal 1987;7:329–38.
[6] Hora SC, Iman RL. Bayesian modeling of initiating event frequencies
at nuclear power plants. Risk Anal 1990;10. p. 102–9.
[7] Pörn K. The two-stage Bayesian method used for the T-book
application. Reliab Engng Syst Safety 1996;51(2):169–79.
[8] Deely JJ, Lindley DV. Bayes Empirical Bayes. J Am Stat Assoc 1981;
76(376):833– 41.
[9] Frohner F. Analytic Bayesian solution of the two-stage Poisson-type
problem in probabilistic risk analysis. Risk Anal 1985;5(3):217–34.
[10] Kaplan S. The two-stage Poisson-type problem in probabilistic risk
analysis. Risk Anal 1985;5(3):227–30.
[11] Hofer E. On two-stage Bayesian modeling of initiating event fre-
quencies and failure rates. Reliab Engng Syst Safety 1999;66(1):97–9.
[12] Hofer E, Hora SC, Iman RL, Peschke J. On the solution approach for
Bayesian modeling of initiating event frequencies and failure rates.
Risk Anal 1997;17(2):249– 52.
[13] Mosleh A, Apostolakis G. The assessment of probability distributions
from expert opinions with an application to seismic fragility curves.
Risk Anal 1986;6(4):447–61.
321
[14] Mosleh A, Apostolakis G. The development of a generic data base
for failure rates. International Topical Meeting on Probabilistic
Safety Methods and Applications, San Francisco, CA; 1985. p. 48-
1–48-10.
[15] Mosleh A. Bayesian modeling of expert-to-expert variability and
dependence in estimating rare event frequencies. Reliab Engng Syst
Safety 1992;38(1–3).
[16] Jouini MN, Clemen RT. Copula models for aggregating expert
opinions. Oper Res 1996;44(3):444–57.
[17] Droguett EL, Mosleh A. A framework for quantifying model
uncertainty. Technical Report. University of Maryland; 1999.
[18] Pickard, Lowe, Garrick, Inc. Seabrook station risk management and
emergency planning study, prepared for Public Service Company of
New Hampshire and Yankee Atomic Electric Company, PLG-0432;
December 1985.
[19] RISKMAN, Integrated Quantitative Risk Assessment Package. New-
port Beach, CA: PLG Inc.
[20] Meyer W, Hennings W. Prior distributions in two-stage Bayesian
estimation of failure rates. In: Kafka P, Schüller GI, editors.
Proceedings of the ESREL’99, 10th European Conference, vol. 2.
The Netherlands: A.A. Balkema; 1999. p. 893–8.
[21] Chib S, Greenberg E. Understanding the Metropolis-Hastings
algorithm. Am Stat 1995;49(4):327–35.
[22] Gilks WR, Richardson WR, Spiegelhalter DJ. Markov chain Monte
Carlo in practice. London: Chapman & Hall; 1996.