Journal of the Operational Research Society (2004) 55, 1090–1101 r 2004 Operational Research Society Ltd.

d. All rights reserved. 0160-5682/04 $30.00

www.palgrave-journals.com/jors

A multi-objective approach to determine

alternative targets in data envelopment analysis
MP Estellita Lins*, L Angulo-Meza and AC Moreira da Silva
Operational Research, Production Engineering, COPPE/UFRJ, Brazil
The choice for radial projections of classic data envelopment analysis (DEA) models, resulting in a number of
projections onto the Pareto-inefficient portion of the frontier, has been seen lately as a disadvantage in DEA. The search
for a non-radial projection method resulted in developments such as preference structure models. These models consider
a priori preference incorporation, using weights in the search for the most preferred efficient target, although presenting
some implementation difficulties. In this paper, we propose a multi-objective approach that determines the bases for a
posteriori preference incorporation, through individual projections of each variable (input or output) as an objective
function, thus allowing one to obtain a target at every extreme-efficient point on the frontier. This multi-objective
approach is shown to be equivalent to the preference structure models, yet presenting some advantages, such as the
mapping of the possible weights, assigned to partial efficiencies of an observed unit, in order to reach a specific target.
Journal of the Operational Research Society (2004) 55, 1090–1101. doi:10.1057/palgrave.jors.2601788

Keywords: data envelopment analysis; multi-objective linear programming

Introduction Kornbluth6 noticed that the DEA model could be

From a managerial point of view, there could be other
projections that are preferable to the radial projections of
classic data envelopment analysis (DEA) models. Moreover,
radial projections often lead us to Pareto-inefficient portions
of the frontier. These two issues have a common source: the
equi-proportional variation of input or output levels,
proposed by Farrell1 as a means of measuring efficiency.
Alternatives to radial projections exist in DEA literature.
One of the main approaches is through models that
incorporate preferences using preference structures, as
proposed by Thanassoulis and Dyson2 and Zhu3 or by the
specification of an ideal point.2
The appropriateness of using multi-objective linear
programming (MOLP) in a DEA context is supported by
the concept of Pareto efficiency, which DEA and MOLP
share. Both of them search for a set of different units that are
better in at least one aspect (or objective function) than the
others, thus characterizing an efficient facet of the feasible
solution space of a problem.4
Credit for the very first work integrating DEA and MOLP
is due to Golany,5 who proposed an interactive multi-
objective procedure (IMOLP—interactive MOLP) to deter-
mine efficient output levels. The algorithm consists of
sequential solutions to a set of related linear programming
problems in which the objective function is to maximize a
weighted sum of the former objectives.
*Correspondence: MP Estellita Lins, Rua Belisário Távora 80, ap 506
Laranjeiras, Rio de Janeiro RJ, Brazil.
E-mail: estellit@iis.com.br
expressed as a multi-objective linear fractional programming
problem. The objective function of the model has the same
expression as in the CCR multipliers model, but applied to
maximize efficiency of every DMU, instead of one at a time,
the restrictions remaining unchanged.
Also, Joro et al,7 observing the problem of characterizing
efficient facets, made a structural comparison of the CCR8
and BCC9 models with the reference point approach10 to
solving multi-objective problems.
Another methodology based upon a multi-objective
model was proposed by Halme et al,11 known as Value
Efficiency Analysis. In this methodology, a multi-objective
model is used to determine an existing or virtual (combina-
tion of existing) DMU preferred by the decision-maker,
which is called the Most Preferred Solution (MPS). Once the
MPS is identified, an efficient frontier is determined in a
DEA-like manner. Joro12 made an extension of the Value
Efficiency Analysis method to determine targets for ineffi-
cient DMUs.
Li and Reeves13 presented a multi-objective model that
considers two additional efficiency measures: the minimiza-
tion of the sum of the DMU distances to the frontier
(minisum) and the minimization of the largest distance
(minimax), besides the maximization of the classical
efficiency in DEA.
The Multi-Objective Simplex approach proposed in this
paper, not only allows calculation of the coordinates of all
alternative extreme efficient DEA targets, but also enables
putting forward a new mapping of the weight range given to
each input/output improvement onto those targets. This
 MP Estellita Lins et al—Data envelopment analysis 1091

promotes insights into the DM preferences, regarding
0 and
I 0. The model is:
variables: R
X X
variations in input or output intensity. We propose an Maximize wrþ fr  w
i ji
additional reference point approach that allows a better r2R0 i2I0
analyst-user interaction environment, benefiting from a 0 1
better visualization of the DEA targets. X X
þ e@ s
i þ srþ A ð1Þ
The content of this paper is organized in the following
i2I 0 r2R0
way: the next section comprises a revision of two models to
determine targets in DEA using the preference structure, Subject to
their description, and the disadvantages of this approach. In X
n

the following section, the first of DEA-MOLP models is fr yrj0  lj yrj ¼ 0; r 2 R0 ; ð2Þ
j¼1
introduced: MORO (Multiple Objective Ratio Optimiza-
tion), which optimizes the ratios between observed and X
n
target inputs (or outputs) of a DMU. We include a concise ji xij0  lj xij ¼ 0; i 2 I0 ; ð3Þ
value judgement-driven classification of MOLP methods, j¼1

which helps understand the relationship among the models X

n
discussed here. Also, a straight comparison between DEA- lj yrj  srþ ¼ yrj0 ; r 2 R0 ð4Þ
MOLP and preference structure models is shown, revealing j¼1

their formal equivalence. In the ensuing section, we propose

X
n
the use of indifference regions to map the range of weights lj xij þ s
i ¼ xij0 ; i 2 I0 ð5Þ
onto the target set. Then, we present a second DEA model: j¼1
MOTO (Multiple Objective Target Optimization), which
directly optimizes the target values, thus promoting the use fr X1 8r 2 R0 ; ð6Þ
of one interactive MOLP approach: the Pareto Race. Final
jip1 8i 2 I0 ð7Þ
sections are devoted to a case study using the MORO model
and conclusion. lj X0 8j ð8Þ

Non-radial models to determine targets in DEA
Classic DEA models, such as constant returns to scale (CRS)
or variable returns to scale (VRS) yield targets obtained as
the equi-proportional reduction of input values, when input-
oriented, or the equi-proportional increase in output values,
in the output-oriented case (without including slacks), thus
providing radial efficiency measures, according to the
Farrell1 measure of efficiency.
From a managerial point of view, there could be several
interesting alternative targets to be explored (characterized
by input and output levels), besides the radial approach.
Several researchers are concerned with the determination of
targets through different approaches, the most popular one
being based on preference structures. These models require a
priori information from the user or decision-maker about
s þ
i ; sr X0 8i 2 I 0 e 8r 2 R0 ; e40 ð9Þ
The decision-maker’s preference structure is expressed
through the weights wrþ and w i for the factors fr and ji,
where fr reflects the increment in the value of output r, and
ji reflects the decrease in the value of input i, in order to
reach the target.
Thus, the objective function consists of maximizing the
weighted sum of the ratios (considering output increments
and input reductions) between observed and projected data.
Projections are considered for sets of inputs and outputs that
can be improved, R0 and I0, through constraints (2) and (3).
For the non-discretionary variables R
0 and
I 0, we have
constraints (4) and (5).
Bounds imposed on f (6) and j (7) prevents the levels of
outputs from decreasing and those of inputs from increasing.
If fr* ¼ ji* ¼ 1, 8rAR0 , 8iAI0, and srþ * ¼ s

i * ¼ 0 8rAR0,

the relative importance of variables, expressed in the form of

8iAI 0. then the DMU j0 is Pareto efficient. Otherwise, the
weights. DMU j0 is inefficient and the target is given by
Two alternative preference structure models are as yrj0 ¼ fr yrj0 8r 2 R0 ð10Þ
follows.
yrj0 ¼ yrj0 þ srþ  8r 2 R0 ð11Þ

Thanassoulis and Dyson model xij0 ¼ ji xij0 8i 2 I0 ð12Þ

Thanassoulis and Dyson2 proposed assigning weights to
changes in the variables in order to implement a preference
structure. For a DMU j0, outputs and inputs are classified
into two categories: variables that we want to change, set R0
for the outputs and I0 for the inputs; and unchangeable
xij0 ¼ xij0  s 8i 2 I0 ð13Þ
i
In real-life applications, this model may be used
interactively, by testing different sets of weights (wrþ and
wi ) and allowing the decision-maker to search for appro-
1092 Journal of the Operational Research Society Vol. 55, No. 10

priate DMU targets. It is important to remark that this will allow knowledge of weight mixes that will lead to
model does not indicate the efficiency score for the DMU. interesting solutions from the decision-maker’s perspective.

Zhu model Multi-objective model for ratio optimization (MORO)

Based on Russell’s efficiency measure, Zhu presented a 3 and with dominance (MORO-D)
model to determine targets that also incorporate a preference Considering an efficient DMU, we propose here that each
structure: output to be increased (or input to be reduced) can be
X s Xm
treated as a separate objective function within a Multiple
Maximize wrþ fr  w
i ji
r¼1 i¼1
ð14Þ Objective Linear Programming context. Thus, the following
Subject to model considers as objective functions the independent
optimization of the increment/decrease of each input/output,
X
n
in order to obtain an efficient target for the DMU j0:
lj yrj  srþ ¼ fr yrj0 ; r ¼ 1; . . . ; s ð15Þ
j¼1 max f1 ð20Þ

X
n ...
lj xij þ s
i ¼ ji xij0 ; i ¼ 1; . . . ; m ð16Þ
j¼1 max fs

ji ; fr free 8i; r; s þ
i ; sr X0 8i; r ð17Þ min j1 ð21Þ

X
s X
m ...
where wrþ  w
i ¼1
r¼1 i¼1 min jm
The sum of weights condition prevents the user from Subject to
inadvertently testing a redundant weight set that is equal to a X
n
previous set multiplied by a constant amount. Although the fr yrj0 ¼ yrj lj 8r ¼ 1; . . . ; s ð22Þ
j¼1
Zhu model does not specify restrictions for the non-
discretionary variables, this can be considered by assigning X
n
zero weights to respective selected inputs/outputs in the ji xij0 ¼ xij lj 8i ¼ 1; . . . ; m ð23Þ
objective function. The major difference in the Zhu model is j¼1

the unboundedness of ji and fr, allowing output reduction

fr ; ji ; lj X0 8r; i; j
and/or input increment, that is, the observed DMU is not
necessarily dominated by the target DMU, like in the This DEA-MOLP model will be referred to as MORO
Thanassoulis and Dyson model. CRS – Multiple Objective Ratio Optimization, as it
In real-life cases, from a managerial point of view, an optimizes ratios between inputs (outputs) of the target
input increment (or output reduction) to reach the efficient DMU and the observed DMU, under constant returns to
frontier could be feasible and preferable. For example, in the scale (CRS). To consider variable returns to scale (VRS), we
labour field where the managers would prefer a solution that add the convexity restriction:
implies hiring instead of dismissing workers, which could X n
produce a negative impact on a firm’s public image. lj ¼ 1 ð24Þ
If fr* ¼ ji* ¼ 1 and srþ * ¼ s
i * ¼ 0 then the DMU j0 is
j¼1

Pareto efficient. Otherwise, the DMU is inefficient and the
target is given by
yrj0 ¼ fr yrj0 þ srþ  8r
xij0 ¼ ji xij0  s
i 8i
As in the previous model, the decision-maker can search
interactively for more appropriate targets for the DMU,
testing different weight sets.
There is an important drawback when determining DEA
targets in both the Thanassoulis and Dyson and the Zhu
models: the arbitrary choice of weights to be given to factors
fr and ji. We have no guidance for establishing weights
before testing the models; just the trial and error experience
The model including this restriction (24) will be referred to
as MORO VRS version. For both versions, the decision
ð18Þ variables are fs, lj, jm.
When the problem results in optimal solutions
ð19Þ fr* ¼ ji* ¼ 1 8r, i, then DMU j0 lies on the Pareto efficient
(non-dominated) frontier. The targets are:
yrj0 ¼ fr yrj0 8r ð25Þ
xij0 ¼ ji xij0 8i ð26Þ
The final value of fr* and ji* depends on the decision-
maker’s choice among the non-dominated solutions of the
problem (in multi-objective linear programming we get
efficient solutions, also known as non-dominated solutions).
 MP Estellita Lins et al—Data envelopment analysis 1093

When we require dominance of the target over the
observed DMU, we will add the following restrictions:
fr X1 8r ¼ 1; . . . ; s ð27Þ
jip1 8i ¼ 1; . . . ; m ð28Þ
The resulting DEA-MOLP model will be called MORO-D
(Multi-objective Ratio Optimization with Dominance), both
CRS—considering expressions (20)–(23) and VRS—adding
the convexity restriction (24). While desirable, dominance
can be dismissed in some real-world cases as Zhu3 pointed
out. Indeed, besides the labour field example given in the
previous section about Zhu model, there are cases where
trade offs over the frontier allow a large increase of outputs,
compensating for a small increase in one input.
In solving the MORO and MORO-D models, we can
provide the decision-maker with a complete set of extreme
efficient targets for the observed DMU, supporting the
implementation of value judgements.
Classification of methods for solving MOLP problems
In a MOLP problem an optimal solution, that is, one that
optimizes all objectives simultaneously, is generally impos-
sible to find. So, the process of solving MOLP problems
consists of finding non-dominated solutions, that is, solu-
tions in the feasible set of the decision variable space, which
cannot be altered to improve one objective function value
without deteriorating at least one other. This concept is
analogous with the Pareto–Koopmans efficiency.
According to Clı́maco et al,14 there are several possible
classification criteria for MOLP methods. We will adopt
here the following classification, which is based on the stage
at which value judgements are incorporated. Three classes
resulted:
(i) Methods with a priori preference aggregation. These
methods usually transform the MOLP problem into a
single objective one (LPP) through an equivalent
objective function, whose optimal solution is a non-
dominated solution for the original MOLP problem. This
function, known as the scalarizing function, expresses the
DM’s preferences. There are three kinds of scalarizing
functions, which consist of: (a) the optimisation of one
objective function restricting the others; (b) the weighted
sum of all objective functions; and (c) the use of a
distance function or reference point approach.
(ii) Methods characterized by a progressive preference
Malivert,20 among others, undertook the problem of
finding the efficient set in MOLP.
In the following three sections, we will develop MORO
model equivalence to preference structures and indifference
regions, using the multi-objective simplex method to solve
DEA-MOLP models, a method that belongs to the third
class and gives all the extreme points of the efficient set. We
have run both the ADBASE software21 and the DEA-
MOLP simplex implemented by Frota Neto.22 After that, we
will implement one interactive reference point method: the
Pareto Race,18 to implement the MOTO model. It is worth
remarking that, although the authors did not mention it,
preference structure models could be considered as weighted
sum a priori preference aggregation MOLP models.
Comparison of the multi-objective and preference
structure models
Let us consider the following example in order to make
comparisons among the MORO and MORO-D models with
the Zhu, and the Thanassoulis and Dyson model:
In Table 1, we consider seven DMUs that use input X to
produce output Y. We applied Linear Programming soft-
ware (LINDO) to run the preference structure models,
testing different weights for j and f until the complete range
Table 1 Original data set
Variable
DMU X Y
A 2 12
B 4 15
C 6 6
D 2 9
E 3 15
F 1 7
G 2 4
X vs Y
16
14 E B
12 A
10
D
Y

8
articulation, where the decision-maker maintains a 6 F C
continuous interaction with the problem. For an
4 G
extended discussion of the several classification alter-
2
natives for interactive methods, readers are referred to
Steuer.15 0

(iii) Methods that generate efficient solutions and incorpo- 0 1 2 3 4 5 6 7

rate preferences a posteriori. Geoffrion (1968),16 Evans X

and Steuer,17 Zeleny,18 Ecker et al19 and Armand and Figure 1 CRS frontier.
1094 Journal of the Operational Research Society Vol. 55, No. 10

Table 2 MORO model (VRS)

Actual value Objective function value Target value

DMU Y X Max f Min j Y X

A 12 2 0.58333 0.5 7 1
1 1 12 2
1.25 1.5 15 3
B 15 4 0.46667 0.25 7 1
0.8 0.5 12 2
1 0.75 15 3
C 6 6 2.5 0.5 15 3
2 0.33333 12 2
1.16667 0.16667 7 1
D 9 2 0.77778 0.5 7 1
1.33333 1 12 2
1.66667 1.5 15 3
E 15 3 0.46667 0.33333 7 1
0.8 0.66667 12 2
1 1 15 3
F 7 1 1 1 7 1
1.71429 2 12 2
2.14286 3 15 3
G 4 2 1.75 0.5 7 1
3 1 12 2
3.75 1.5 15 3

X vs Y X vs Y
16 16

14 E B 4 E
14 B

12 A 12 A
10 10
D D
8
Y

6 F C F
6 C

4 G 4 3 G

2 2

0 0
0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7
X X

Figure 2 VRS frontier for DMU D. Figure 3 CRS frontier, projections for DMU G.

of basic efficient solutions was obtained. For the multi-
objective models, we used ADBASE,21 which gives all
efficient, basic solutions, although we can also find non-basic
solutions, as will be done when applying MOTO model.
The results concerning CRS, for the Zhu model as for the
MORO model, show that any point on the efficient CRS
frontier (Figure 1) serves as a target for every efficient or
inefficient DMU, given that f and j are unbounded. In this
sense, there is equivalence between solutions for both Zhu
and MORO models.
The same equivalence is true under VRS, given that Zhu
and MORO models yield the same set of efficient solutions.
Results from ADBASE for the MORO VRS model show
(Table 2) that the target set is unique, no matter which
DMU is being projected. This set is composed of all extreme
efficient DMUs (F, A and E), corresponding to basic non-
dominated solutions in the MOLP problem.
In Figure 2, we present the alternative targets for DMU
D. For example, if the DM plans to increase the input X of
DMU D by 50%, then he should consider a required output
increase of 66.7%, taking DMU E as a possible efficient
target, according to Table 2.
Non-extreme efficient solutions were not attained, the
reason for which will become clear through the concept of
indifference regions, in the respective section.
The MORO-D model solved in ADBASE gave efficient
basic solutions that were compared to those obtained by the
Thanassoulis and Dyson model, considering CRS and VRS.
For the CRS frontier (Figure 3), we obtained different sets
of targets for each projected DMU, shown in Table 3.
 MP Estellita Lins et al—Data envelopment analysis 1095

Table 3 MORO-D model (CRS)

Actual value Objective function value Target value

DMU Y X Max f Min j Y X

A 12 2 1.166667 1 14 2
1 0.857143 12 1.71
B 15 4 1.866667 1 28 4
1 0.535714 15 2.14
C 6 6 7 1 42 6
1 6
D 9 2 1.555556 1 14 2
1 0.642857 9 1.29
E 15 3 1 0.741935 15 2.23
1.4 1 21 3
F 7 1 1 1 7 1
G 4 2 1 0.285714 4 0.57
3.5 1 14 2

The set of possible targets (basic feasible solutions) for X vs Y

16
DMU G (Figure 3) are the efficient points 3 and 4. It is
worth noting that the targets for every DMU are propor- 14 E B

tional to input and output values for the DMU F, which 12 A

defines the efficient frontier under CRS. 10
Adding the convexity condition to the MORO-D D
8
Y

formulation, we get the VRS frontier. Thanassoulis and F

6 C
Dyson program was solved for DMU G, and resulted in the
targets corresponding to DMUs F and A, shown in Figure 4. 4 4 G

We obtained an identical set of basic efficient solutions using 2

MORO-D. 0
The Non-Pareto efficient frontier cannot be attained at all, 0 1 2 3 4 5 6 7
unlike the classic models that would yield point 4 in Figure 4 X
as the input oriented projection of DMU G.
Figure 4 VRS frontier, projections for DMU G.
Table 4 shows results for the MORO-D (VRS) program:
the values for f and j, the targets and the strictly positive l’s
corresponding to the reference set of DMUs.
Results from DEA-MOLP models, compared to those MOLP models yielded all the non-dominated solutions in
from Zhu and Thanassoulis and Dyson models, pointed out one run, while preference structure models required the
the equivalence between target sets for the DMUs. DEA- specification of new weighting sets in every new run,

Table 4 MORO-D model (VRS)

Actual value Objective function value Target value Lambdas

DMU Y X Max f Min j Y X lA lE lF

A 12 2 1 1 12 2 1
B 15 4 1 0.75 15 3 1
C 6 6 2 0.333333 12 2 1
2.5 0.5 15 3 1
1.166667 0.166667 7 1 1
D 9 2 1.333333 1 12 2 1
1 0.7 9 1.4 0.4 0.6
E 15 3 1 1 15 3 1
F 7 1 1 1 7 1 1
G 4 2 3 1 12 2 1
1.75 0.5 7 1 1
1096 Journal of the Operational Research Society Vol. 55, No. 10

sometimes failing to obtain a new target, because various X

s X
m

weighting sets can lead to the same target solution. This can Maximize wrþ fr  w
i ji ð31Þ
r¼1 i¼1
be seen with the aid of the indifference regions in the
corresponding section. Subject to
X
n
fr yrj0 ¼ yrj lj 8r ¼ 1; . . . ; s
Formalizing equivalence j¼1

In the following paragraphs we are going to formalize the X

n
ji xij0 ¼ xij lj 8i ¼ 1; . . . ; m
equivalence between the models analysed. Conceptually, this j¼1
arises from a result obtained by Geoffrion,16 which states the
equivalence between an LP model with multiple objective fr X1 8r ¼ 1; :::; s
functions and an LP model whose sole objective function
ji p1 8i ¼ 1; . . . ; m
consists of a convex linear combination of the former objective
functions. He showed that, given a multi-objective model: fr ; ji ; lj X0 8r; i; j
Max CX ð29Þ Ifwrþ , w
are strictly greater than zero, then there will be no
i

S.t. slacks, and we can assume:

yrj0 ¼ fr yrj0 8r 2 R0 ð32Þ
AXpb
xij0 ¼ ji xij0 8i 2 I0
XX0
If we admit any component of wrþ or w i equal to zero
X* is an efficient solution for this model if and only if ( y*, (the unchanged variables case in the Thanassoulis and
y*40, Syi* ¼ 1, such that X* is optimal for Dyson model), then, we should use
Maxy CX ð30Þ Xn
lj yrj Xfr yrj0 ; r ¼ 1; . . . ; s
S.t. j¼1

AXpb X
n
lj xij pji xij0 ; i ¼ 1; . . . ; m
XX0 j¼1

Kornbluth23 stated that, given a MOLP problem, for
every weighting set there is a non-dominated basic solution.
Cohon24 added that weights should be strictly positive in
order not to generate weakly efficient solutions, but
Romero25 showed that, while possible from a theoretical
point of view, this is not likely to happen in practice. In fact,
this happens in the presence of multiple optimal solutions
and add such slacks (multiplied by e) to the objective
function. As we can see, the MORO model considering
dominance is equivalent to the Thanassoulis and Dyson
model, in which all variables are considered to improve. The
Zhu model includes slacks in constraints, in order to let ji,
fr be free, but not within the objective function, because wrþ
or w
i is not allowed to be equal to zero.

and can be avoided if we substitute a small infinitesimal

amount for a zero coefficient in the objective function.
As we will apply this theorem to a known vector y in order
to obtain an optimal solution for model (30), it is easy to see
P
that imposing restriction yi* ¼ 1 will not be necessary.
The equivalence between models (29) and (30) implies
that, for any given efficient solution X*, we can use the
associated y* to form a weighted function y*CX*. This
linear function is an example of a utility function for the
decision-maker, where y* represents the trade-offs between
objectives, with X* as an optimal solution. In the case of the
Zhu model, and Thanassoulis and Dyson model, this y*
vector is equivalent to the weighting vector composed of wrþ
and wi , given to variables j and f.
We can easily show that the MORO model, presented in
expressions (20)–(23), including dominance (27) and (28)
may be put in the form of a preference structure model using
(30) and substituting wrþ and w i for y:
Obtaining indifference regions
Let us consider that a given projection on the frontier can be
achieved through various different sets of weights. Thus,
each weight can be allowed to vary inside a limited
(indifference) region, while projecting a given observed unit
onto the same point on the frontier. As we will show, the
indifference regions can be used to map the range of weights
(j and f), given to each objective function, onto all the
possible targets. These weights indicate the ‘importance’
given to a variable improvement, therefore revealing the
preference structure that leads to a given target.
According to Kornbluth,23 a solution x* is efficient in
relation to any set of weights y that satisfy the Kuhn–Tucker
conditions. Each set of weights y characterizes one
indifference region: the interval where the weights y can
vary and maintain the efficient solution x*.
 MP Estellita Lins et al—Data envelopment analysis 1097

Table 5 MORO model (VRS)

Actual value Objective function value Target value

DMU Y X Max f Min j Y X

C 6 6 2.5 0.5 15 3
2 0.333333 12 2
1.166667 0.166667 7 1

The indifference regions can be determined by imposing θ1=0 0.16667 0.25 θ1=1
the optimality conditions on the coefficient of DMU j in the
objective function y*Cx, under the canonical form, that is,
Sol.3 Sol.2 Sol.1
zjcjX0. Once y*Cx is the sum of q objective functions, the A
F E
coefficient for each DMU j is
      Figure 5 Indifference regions.
y1 z1j  c1j þ y2 z2j  c2j þ þ yq zqj  cqj

where the optimality condition for a given non-dominated

solution is solutions. For the second solution, corresponding to DMU
      A, we obtained: 0.166667py1p0.25, and for the third
y1 z1j  c1j þ y2 z2j  c2j þ þ yq zqj  cqj X0 solution (DMU F): 0py1p0.16667. Then, imposing the
P optimality conditions, we can represent the indifference
and yiX0, i ¼ 1,y,q and qi ¼ 1yi ¼ 1.
regions of weights y1 (and y2 ¼ 1–y1) for the three DEA
We will implement these conditions to find the indiffer-
extreme-efficient solutions 1 (DMU E), 2 (DMU A) and 3
ence regions of the following MORO VRS model, applied to
(DMU F), as shown in Figure 5.
DMU C, with data from Table 1:
It is worth noting that an efficient, but non-extreme-
Max f efficient, solution can be expressed as a convex linear
Min j combination of some extreme-efficient solutions (for exam-
Subject to ple: A and E in Figure 5), corresponding to contiguous
indifference regions, and can be attainable with a single
6f  12l1  15l2  6l3  9l4  15l5  7l6  4l7 ¼ 0
value of y1 ¼ 0.25 (and y2 ¼ 0.75). In this case, a preference
6j þ 2l1 þ 4l2 þ 6l3 þ 2l4 þ 3l5 þ l6 þ 2l7 ¼ 0 structure model would exhibit multiple optimal solutions,
which are not shown by usual LP software.
l1 þ l2 þ l3 þ l4 þ l5 þ l6 þ l7 ¼ 1 Establishing the following equivalence: w1þ
y1 e w 1
y2,

f; j; lj X0 we have a preference structure where w1þ X0.25 and

w1þ þ w 1 ¼ 1 will lead us to the extreme-efficient solution
This program yielded three non-dominated solutions, E. A preference structure where 0.16667pw1þ p0.25 and
shown in Table 5, corresponding to the DMUs E, A and F, w1þ þ w 1 ¼ 1 will lead us to the solution A, and, ultimately,
in Figure 4. to a preference structure where w1þ p0.16667 and
Imposing the optimality conditions, zjcjX0, we obtained w1þ þ w 1 ¼ 1 will lead us to the solution F.
the following inequalities system for the first non-dominated
solution (DMU E in Figure 4):
0:5y1  0:1667y2 X0 Interactive method using the MOLP model for target
optimization (MOTO)
0:1667y2 X0
Instead of the ratio, we can consider each objective function
1:5y1 þ 0:5y2 X0 as the projected value (target), that is, the ratio multiplied by
y1  0:16667y2 X0 the observed output or input. With this simple change in the
objective functions in MORO (20 and 21), we obtain the
1:3333y1  0:3333y2 X0 MOTO model:
max y1j0 f1 ð33Þ
1:8333y1  0:16667y2 X0

y1 þ y2 ¼ 1 ...

After solving this system, we obtained the region max ysj0 fs

0.25py1p1. Using the same procedure, we determined the
indifference regions for the other two non-dominated min x1j0 j1 ð34Þ
1098 Journal of the Operational Research Society Vol. 55, No. 10
...
min xmj0 jm
S.t.
X
n The number of possible efficient solutions to be assessed
fr yrj0 ¼ yrj lj 8r ¼ 1; . . . ; s
j¼1
X
n approach.
ji xij0 ¼ xij lj 8r ¼ 1; . . . ; m
j¼1
MOTO model presents:
 The same variables and restrictions as MORO model.
 The possibility of imposing restrictions on f and j, as
presented in (27) and (28) for MORO-D, resulting in the
MOTO-D model, under CRS or VRS.
 If f*r ¼ j*i ¼ 1 8r, i, then DMU j0 is a Pareto-efficient
DMU. Otherwise, the targets are:
ynrj0 ¼ fr yrj0 ; r ¼ 1 . . . s
xnij0 ¼ ji xij0 ; i ¼ 1; . . . ; m
Visualization of the target for each variable, instead of the
value of the increment or reduction (f or j) given by the
MORO model, becomes more informative and understand-
able, improving the application of interactive solution
methods. One of them was applied in this paper: the Pareto
Race,26 whose metric is based on a distance to a reference or
ideal point, implemented in the VIG software, where the
visualization of the objective function values is made
through bar charts on the computer screen.
A change proposed by the expert/decision-maker in an
objective function, and its consequences for other objective
functions are easily visualized. With this approach, one can
know directly from the screen which values (input/output)
can be obtained, given the changes proposed. It is important
Figure 6
to notice that interactive methods not only show extreme
efficient points, but also non-extreme efficient points, that is,
interactive approaches can explore the whole non-dominated
set.
ð35Þ is, however, unlimited. This method should, therefore, be
used in a complementary way to the simplex multi-objective
ð36Þ Let us look at an example using the MOTO–VRS model
to analyse the efficiency of DMU C (previous example). In
Figure 3, we observed three possible targets for C, the
extreme efficient points (DMUs) A, E and F. The points
between them, inside the segments A–E and A–F are also
efficient and can be explored using this interactive approach.
Figure 6 shows the interface of VIG, with an efficient
solution chosen arbitrarily as a target for DMU C.
This solution corresponds to a virtual DMU, obtained by
convex linear combination of DMU A (lA ¼ 0.9177545) and
DMU E (lE ¼ 0.0822455), which is the target for DMU C:
Y ¼ 12.2467 and X ¼ 2.08225 as seen in Figure 6 (in the
MORO model the solutions visualized for this same target
would be f ¼ 2.04112 and j ¼ 0.347042).
The case study
Dental, as well as other branches of medical assistance, is a
right guaranteed by the Brazilian Constitution. Thus, the
Public Health System is supposed to provide free preventive
and conclusive medical care for every Brazilian citizen.
Although the Public Health System performs as a unified
system, the way it is managed varies according to the region.
In Rio de Janeiro city, the Federal Government supports
public hospitals and clinics, which are managed by the local
administration. Both Federal Government and local man-
agers are interested in the assessment of the efficiency of the
local units—geographical districts that constitute the muni-
Interactive method for MOTO-VRS to DMU C.

Figure 7 Geographical regions of Rio de Janeiro municipality.
cipality, shown on the map (Figure 7). These efficiencies
were assessed through the MORO-D model.
The final formulation considered the number of dentists
and potential target demand (which comprises both the
number of children and pregnant women belonging to low-
income families) as inputs and the total number of
preventive plus conclusive dental procedures as the output,
according to data shown in Table 6.
The following DMUs revealed a 100% efficiency using the
BCC Output oriented model: (III) Rio Comprido, (IV)
Botafogo, (V) Copacabana, (IX) Vila Isabel, (XIX) Santa
Cruz, (XX) Ilha do Governador, (XXI) Paquetá, (XXIII)
Santa Tereza, (XXV) Pavuna and (XXVI) Guaratiba.
The number of possible targets for each inefficient DMU
according to MORO-D/VRS formulation varies, depending
on the enveloping frontier and on the DMU input–output
vector itself. The Multi-objective Simplex algorithm based
on Guerreiro27 was implemented, resulting in a number of
targets for the inefficient units, ranging from three (Portuária
and Centro) to 19 (Penha). These targets, apart from any
further analysis, are already a useful tool for the Decision-
Makers. Table 6 also shows the number of targets for each
unit. Notice that efficient DMUs still project only on
themselves.
Take, for instance, the targets for DMU (II) Centro
shown in Table 7:
These three targets belong to the efficient frontier. The
DM could choose to dismiss/relocate 11 dentists and keep
production constant (target 1), or relocate nearly nine
dentists and increase production to 30 615 procedures (target
MP Estellita Lins et al—Data envelopment analysis 1099
2). Target 3 implies reducing the target population, which is
not so straightforward, but could mean arranging for local
population to be assigned to clinics in another district. The
three targets can also be expressed as the following convex
linear combination of the inputs/ outputs from the DMUs
(IV) Botafogo, (V) Copacabana and (XXI) Paquetá.
25 7
5667  0; 2377 þ 2612  0; 2284
83629 22477
5 10; 21
þ 205  0; 5339 ¼ 2053
3507 26884
25 5 11; 767
5667  0; 3383 þ 205  0; 6617 ¼ 2053
83629 3507 30615
25 5 10; 83
5667  0; 2918 þ 205  0; 7082 ¼ 1798
83629 3507 26884
It is possible, therefore, to explore non-basic feasible
solutions, as any convex linear combination of the targets
1–3.
We can note that any other target aiming to increase the
number of procedures (as an output oriented model would
try to do) will also decrease the number of dentists (once the
new target should be expressed itself as a convex linear
combination of the three targets presented) in order to be
1100 Journal of the Operational Research Society Vol. 55, No. 10

Table 6 Original data and number of alternative targets

DMUs Inputs Output MORO-D Results

Districts Number of dentists Target population Preventive plus conclusive procedures No. of targets

I Portuária 6 5720 22 105 3

II Centro 21 2053 26 884 3
III Rio Comprido 17 7529 66 836 1
IV Botafogo 25 5667 83 629 1
V Copacabana 7 2612 22 477 1
VI Lagoa 31 8892 44 985 14
VII São Cristóvão 12 8604 42 101 9
VIII Tijuca 13 5705 35 818 9
IX Vila Isabel 38 7023 91 465 1
X Ramos 22 50 726 34 340 18
XI Penha 19 34 172 58 424 19
XII Inhaúma 21 13 320 63 294 14
XIII Méier 50 25 721 103 245 13
XIV Irajá 20 16 652 71 818 9
XV Madureira 25 35 332 92 136 10
XVI Jacarepaguá 46 42 185 101 144 13
XVII Bangu 44 81 238 170 336 12
XVIII Campo Grande 39 56 396 146 758 12
XIX Santa Cruz 46 49 542 253 913 1
XX Ilha do Governador 32 13 973 122 375 1
XXI Paquetá 5 205 3507 1
XXII Anchieta 10 19 521 47 231 4
XXIII Santa Tereza 6 3493 23 416 1
XXIV Barra da Tijuca 21 6293 30 488 14
XXV Pavuna 8 30 258 42 655 1
XXVI Guaratiba 11 15 181 59 033 1

Table 7 Alternative targets for DMU Centro

Variables No. of dentists Target population No. of procedures

Observed DMU II Centro 21 2053 26 884

Targets 1 10.21 2053 26 884

2 11.76 2053 30 615
3 10.83 1799 26 884

efficient. We recall, indeed, that oriented models in practice
frequently yield non-oriented projections.
Under the MORO model without dominance, the (basic
feasible solution) targets for DMU (II) Centro would
correspond to each one of the efficient DMUs. Even so,
this DMU could hardly avoid dentists being relocated,
unless a huge increase in the production and decrease in
target population were attained.
Next, we will focus on the DMU (VI) Lagoa, which has 14
targets, according to MORO-D. We will select three from
among a group of six that present an unchanged target
population. The first one implies reduction in the number of
dentists from 31 to roughly nine, while also keeping the
number of procedures constant. A second target is
characterized by 18 dentists, but with an increase of 62%
in the number of procedures. Yet a third target implies no
reduction in dentists, but an increase of 120% in the number
of procedures. All of them behave strictly as Pareto-efficient
points on the frontier. So, it is up to the DM to decide which
one would be more adequate.
Conclusions
This paper proposes the use of a multi-objective linear
programming approach to determine non-radial projections
for inefficient DMUs onto the Pareto efficient frontier. We
established the equivalence between the efficient solutions set
obtained by the multi-objective (MORO and MORO-D)
models and the optimal solutions set obtained in models
with preference structure (Thanassoulis and Dyson, and
Zhu). We pointed out the advantages presented by the
proposed multi-objective models: they allow a posteriori
preference incorporation and do not require a priori

specifications of the weights given to each objective (input or
output improvement) and yield indifference regions that
map weight ranges onto the target alternatives.
For each set of weights, this method finds an efficient
point or tangential point of a family of segments y1 f1
(x) þ y2 f2 (x) ¼ l to a polygon, in the case of two objectives.
In the generalized case, parallel hyperplanes will be
tangential to a convex polyhedron set at an extreme-point.19
This is why the Thanassoulis and Dyson and Zhu preference
structure models will always give at least one extreme-
efficient point as a target.
We also proposed the use of an interactive approach
(Pareto race) applied to solve the MOTO model, exploring
the non-extreme efficient solutions.
We presented a case study where the MORO-D model
was implemented, illustrating the diversity of Pareto-efficient
solutions that could possibly be obtained.
Finally, we consider that both approaches could rather be
integrated, as the mapping of the weights onto the efficiency
regions can guide the DM on his/her interactive search for
the ultimate target. After obtaining extreme points and
indifference regions, the DM could explore non-extreme
efficient solutions through interactive methods.
References
1 Farrell MJ (1957). The measurement of productive efficiency.
J Roy Stat Soc Ser A 120: 253–281.
2 Thanassoulis E and Dyson RG (1992). Estimating preferred
target input–output levels using data envelopment analysis. Eur
J Opl Res 56: 80–97.
3 Zhu J (1996). Data envelopment analysis with preference
structure. J Opl Res Soc 47: 136–150.
4 Stewart TJ (1996). Relationships between data envelopment
analysis and multicriteria decision analysis. J Opl Res Soc 47:
654–665.
5 Golany B (1988). An interactive MOLP procedure for the
extension of DEA to effectiveness analysis. J Opl Re Soc 39:
725–734.
6 Kornbluth JSH (1991). Analysing policy effectiveness using
cone restricted data envelopment analysis. J Opl Res Soc 42:
1097–1104.
7 Joro T, Korhonen P and Wallenius J (1998). Structural
comparison of data envelopment analysis and multiple objective
linear programming. Mngt Sci 44: 962–970.
8 Charnes A, Cooper WW and Rhodes E (1978). Measuring
efficiency of decision making unit. Eur J Opl Res 2: 429–444.
MP Estellita Lins et al—Data envelopment analysis 1101
9 Banker RD, Charnes A and Cooper WW (1984). Some models
for estimating technical and scale inefficiencies in data envelop-
ment analysis. Mngt Sci 30: 1078–1092.
10 Wierzbicki A (1980). The use of reference objectives in multi-
objective optimization. In: Fandel G and Gal T (eds). Multiple
Objective Decision Making, Theory and Application. Springer-
Verlag, New York.
11 Halme M et al. (1998). A value efficiency approach to
incorporating preference information in data envelopment
analysis. Mngt Sci 45: 103–115.
12 Joro T (1998). Models for identifying target units in data
envelopment analysis: comparison and extension. IIASA, Interim
Report IR-98-055.
13 Li X-B and Reeves GR (1999). A multiple criteria approach to
data envelopment analysis. Eur J Opl Res 115: 507–517.
14 Clı́maco J, Antunes CH and Alves MJG (1996). Programac¸ão
Linear Multiobjetivo: Métodos Interactivos, ‘Software’ e Apli-
cac¸ões. Faculdade de Economia da Universidade de Coimbra e
INESC.
15 Steuer RE (1986). Multiple Criteria Optimization: Theory,
Computation, and Application. Krieger Publishing: Malabar, FL.
16 Geoffrion AM (1968). Proper efficiency and the theory of vector
maximisation. J Math Anal Appl 22: 618.
17 Evans JP and Steuer RE (1973). A revised simplex method for
multiple objective programs. Math Programming 5: 54–72.
18 Zeleny M (1973). Compromise programming. In: Cochrane JL
and Zeleny M (eds). Multiple Criteria Decision Making.
University of South Carolina Press, Columbia, pp 262–301.
19 Ecker JG, Hegner NS and Kouada IA (1980). Generating all
maximal efficient faces for a multiple objective linear program.
J Optim Theory Appl 30: 353–381.
20 Armand P and Malivert C (1991). Determination of the efficient
set in multiobjective linear programming. J Optim Theory Appl
70: 467–489.
21 Steuer RE (1992). ADBASE, Department of Management
Science and Information Technology, University of Georgia,
Athens, GA.
22 Frota Neto (2002). Implementac¸ão e Análise de Programac¸ão
Multiobjetivo na Soluc¸ão de Problemas de Análise Envoltória de
Dados. M.Sc. dissertation, COPPE/UFRJ, Brazil.
23 Kornbluth JSH (1974). Duality, indifference and sensitivity
analysis in MOLP. Opl Res Quart 25: 599–614.
24 Cohon JL (1978). Multiobjective Programming and Planning.
Academic Press: New York.
25 Romero C (1993). Teorı́a de la Decisión Multicriterio:
Conceptos, técnicas y aplicaciones. Alianza Editorial: Madrid.
26 Korhonen P and Wallenius J (1988). A Pareto Race. Naval Res
Logist 35: 615–623.
27 Guerreiro J et al. (1985). Programac¸ão Linear. McGraw-Hill:
Portugal.
Received September 2002;
accepted April 2004 after three revisions
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.