informs

Vol. 38, No. 3, MayJune 2008, pp. 187201

issn 0092-2102
eissn 1526-551X
08
3803
0187

doi 10.1287/inte.1080.0349
2008 INFORMS

Improving Maintenance Decision Making in

the Finnish Air Force Through Simulation
Ville Mattila, Kai Virtanen

Systems Analysis Laboratory, Helsinki University of Technology, FIN-02015 HUT, Finland

{ville.a.mattila@tkk., kai.virtanen@tkk.}

Tuomas Raivio

Gaia Consulting Oy, Systems Analysis Laboratory, Helsinki University of Technology,

FIN-02015 HUT, Finland, tuomas.raivio@gaia.

We used discrete-event simulation to model the maintenance of ghter aircraft and improve maintenance-related
decision making within the Finnish Air Force. We implemented the simulation model as a stand-alone tool that
maintenance designers could use independently. The model has helped the designers to study the impact of
maintenance resources, policies, and operating conditions on aircraft availability. It has also enabled the Finnish
Air Force to advance the operational capability of its aircraft eet. We designed the model to simulate both
normal and conict operating conditions. The main challenge of the project was the scarcity and condentiality
of data about the ghter aircraft, their maintenance, and various operational scenarios, especially during conict
situations.
Key words: simulation: applications; military: defense systems; reliability: availability, maintenance/repairs.
History: This paper was refereed. Published online in Articles in Advance June 4, 2008.

likely to consider. The model describes the essential features of ight operations and maintenance
including planned and unplanned maintenance, air
bases, aircraft repair shops, and maintenance personnel. Moreover, it describes both normal and conict
operating conditions.
Some earlier studies on military operations also
applied simulation to consider the effects of reliability
and maintainability on aircraft operational capability.
For example, Balaban et al. (2000) and Ciarallo et al.
(2005) developed simulation models for availability
of cargo and mobility aircraft, respectively. Upadhya
and Srinivasan (2004) built a simulation model for
availability of generic aircraft and helicopters in combat operations. Rodrigues et al. (2000) used a simulation model to assess the spare-parts management for
A-4 aircraft. Kang et al. (1998) considered two simulation models for managing spare-parts and component repairs. In a recent paper, Kladitis et al. (2007)
used simulation to analyze the impact of a new
avionics system on the availability of B-52H bombers.
However, these models either considered different

ghter aircraft typically requires several hours

of maintenance per hour of ight activity. This
maintenance involves a diversity of operating policies, processes, people, and materials. In a eet of
ghter aircraft, these elements form a complex maintenance system. The system performance directly
affects aircraft availability, i.e., the number of aircraft
that can be used in ight missions. Ability to assess
how maintenance-related decisions or operating conditions affect the system is critical in maintaining the
eets operational capability.
We used discrete-event simulation, which has been
widely applied in studying manufacturing systems
(Law and Kelton 2000), to model the maintenance
of the ghter aircraft eet in the Finnish Air Force
(FiAF). It lends itself to analyzing a maintenance system because manufacturing and maintenance share
common features, such as workforce considerations,
tasks times, material-handling delays, and equipment
reliability. We also found simulation to be a suitable
method for modeling the FiAF maintenance system
because it enabled us to study the system from many
aspects that the FiAF maintenance designers were
187

188

Mattila, Virtanen, and Raivio: Improving Maintenance Decision Making in the FiAF Through Simulation

types of ight operations than our model did or considered a more narrowly dened problem; they did
not take a system-wide view of maintenance. Pohl
(1991) used operational test data to devise a simulation model for operations of the F-15E aircraft. The
model described maintenance in much the same way
as ours did but limited the discussion to consideration
of a xed-size squadron in a single air base. To the
best of the authors knowledge, no previous simulation models in the open literature have considered the
maintenance of a eet of ghter aircraft at the depth
of the model that we present in this paper.
Our primary challenges in constructing the model
were scarcity and condentiality of data. In particular,
no data were available for modeling certain elements
of conict conditions such as battle-damage probabilities or repair-time distributions. Some condential
data, which FiAF could not share with the authors,
included parts of the contingency plans on conicttime maintenance policies. We found two approaches
useful in overcoming these challenges. First, in situations where data were unavailable, we asked subject
matter experts from different organizational levels
to provide their opinions. Second, we designed the
model such that the condential information was
included in the input data; the maintenance designers
who performed the corresponding simulation analysis could thus handle the condential data independently. Implementing the model as a stand-alone
tool with a graphical user interface (GUI) facilitated
our second approach because it made the model
approachable to the designers. The scarcity of data
also affected the validation of the model. We were
able to perform limited comparisons between the simulation output and actual performance data from the
maintenance system. Therefore, we used subject matter experts on multiple occasions to assess the underlying assumptions as well as the model output.
We introduced the model in the FiAF units that perform aircraft maintenance; it has enabled these units
to address many maintenance-related issues. Examples include the forecasting of aircraft availability, the
analysis of the resource requirements for international
operations, and the feasibility study of a readjusted
periodic maintenance program. The project has also
provided FiAF with new knowledge about possible
applications of simulation techniques. For example,

Interfaces 38(3), pp. 187201, 2008 INFORMS

the Finnish Army subsequently devised a simulation

model for the maintenance system of newly acquired
transport helicopters with collaboration from FiAF.

FiAF Aircraft Maintenance

The FiAF aircraft eet consists of Boeing F-18 Hornet ghters, BAe Hawk Mk51 jet trainers, and other
types of aircraft used in transportation, air surveillance, ight training, and liaison duty. We considered the ight operations and maintenance of the F-18
Hornet aircraft during normal and conict conditions
(conict refers to a situation in which the aircraft
eet is involved in an actual engagement with an
enemy). However, because detailed Hornet information is classied, we discuss Hawk maintenance in
this paper. At the modeling level, we found that the
maintenance principles and the appearance mechanisms of unexpected failures are very similar; in general, they differ only in model parameters. Hence, the
principles we report here apply to the Hornet as well.
The FiAF aircraft eet has three primary operational
units that are called air commands (Figure 1). Within
each air command, a ghter squadron is responsible
for aircraft ight operations and specic maintenance
activities. Each air command also has a separate repair

FiAF

Headquarters
Air commands

Air command 1

Other units

Air command 2

Headquarters

Air command 3

Fighter squadron

Depot-level
repair shops

Air commands
repair shop
Other units

Figure 1: The primary operational units for ight operations and aircraft
maintenance of FiAF include three air commands that are further divided
into ghter squadrons and repair shops. Separate, depot-level repair
shops perform the most demanding maintenance.

Mattila, Virtanen, and Raivio: Improving Maintenance Decision Making in the FiAF Through Simulation

189

Interfaces 38(3), pp. 187201, 2008 INFORMS

shop for more complex maintenance tasks. Depotlevel maintenance units of the national aerospace
defense industry perform the most demanding maintenance. The organization that Figure 1 shows remains
essentially the same during both normal and conict conditions, although the decentralization of the
units during a conict may change their geographic
locations.
Normal Conditions
During peacetime, the activities of an air command
are centralized at a single air base. The general goals
of aircraft maintenance are to assure that sufcient
numbers of aircraft are available for training and possible reconnaissance ight operations at all times, and
to preserve the long-term operating condition of the
entire eet. An air command should also be able to
raise the level of preparedness when necessary.
Daily aircraft maintenance consists of ightmission-related inspections. The aircraft that perform
ight missions undergo a preight inspection before
the rst mission, whereas a turnaround inspection is
performed after each mission. In these inspections,
the aircraft are checked according to given specications and the necessary replenishments are made.
The aircraft periodically undergo more elaborate
maintenance. The frequency of periodic maintenance
is based on accumulated ight hours. The maintenance intervals as well as the number and contents of periodic maintenance types depend on the
type of aircraft. The Hawk undergoes six different
types of periodic maintenance that are referred to
Maintenance activity
Preight inspection
Turnaround inspection
Periodic maintenance
Type I
Type II
Type III
Type IV
Type V
Type VI
Failure repairs

Timing

as type I, II,


,VI maintenance. Unplanned maintenance is performed in case of a failure. Some failures
are noncriticalthe aircraft are repaired only during
the next periodic maintenance; however, some failures
must be addressed immediately. A repair typically
involves diagnosing the defect cause and repairing or
replacing the failed component.
The above activities are categorized into different
maintenance levels and the aircraft maintenance units
are categorized according to their capability to perform the activities (Table 1).
The organizational-level (OR-level) maintenance
mainly includes turnaround and preight inspections,
minor periodic maintenance such as type I maintenance, and minor failure repairs such as simple
component changes. The ghter squadron operates
the OR-level maintenance unit, which is located in
the main air base of the air command during normal
conditions. Intermediate level (IN-level) maintenance
includes more complicated periodic maintenance and
failure repairs. The air commands repair shop, which
is also located in the main air base, performs IN-level
maintenance. Depot-level (DE-level) repair shops,
which are not located within the main air base, handle
major periodic maintenance.
Conict Situations
In a conict situation, the aircraft are exposed to battle
damage or may be destroyed during ight missions.
Any of the maintenance units may handle battledamage repairs during a conict depending on their
capability and the type of repair. These repairs require
Maintenance unit

Before rst ight of the day

After each ight

Fighter squadron
Fighter squadron

Every 50 ight hours

Every 125 ight hours
Every 250 ight hours
Every 500 ight hours
Every 1,000 ight hours
Every 2,000 ight hours
Unplanned, as required

Fighter squadron
Air commands repair shop
Air commands repair shop
Depot-level repair shops
Depot-level repair shops
Depot-level repair shops
Fighter squadron/Air commands
repair shop

Maintenance level
OR (Organizational-level)
OR
OR
IN (Intermediate level)
IN
DE (Depot-level)
DE
DE
OR/IN

Table 1: The maintenance of aircraft is categorized into different maintenance levels. Each maintenance unit
performs the maintenance of a specic level.

Mattila, Virtanen, and Raivio: Improving Maintenance Decision Making in the FiAF Through Simulation

190

Interfaces 38(3), pp. 187201, 2008 INFORMS

personnel with specic skills or materials that are

rarely needed during normal conditions.
The overall goals of aircraft maintenance change
as maintenance needs increase. The emphasis is on
assuring the availability of aircraft in high-intensity
operations and the restoration of failed or damaged
aircraft to a mission-capable condition in the shortest
possible time. If necessary, periodic maintenance can
be temporarily suspended. However, aircraft performance or reliable operation must never be reduced
severely. Changes in ight intensity and in the maintenance workload are difcult to anticipate because
they depend on the evolution of the conict.
In a conict situation, the FiAF air commands move
their units from the main air base to one or more
decentralized air bases to protect the air bases from
the enemy. The organization of the air force and the air
commands remains largely the same. The decentralized air bases are located in diverse areas that are typically sparsely inhabited; they utilize public roads as
a runway. They can typically support the ight operations and certain maintenance activities of a given
number of aircraft. The maintenance activities that are
allocated to an air base generally depend on the level
of infrastructure that is readily available at the location. For example, a given air base may support all
activities that occur in the main air base of an air command during peacetime. This type of air base would
have facilities for all of the OR- and IN-level maintenance. In turn, an air base may support the OR-level
only or merely the daily maintenance of the aircraft.
Decentralization changes the operating environment of the maintenance units if some of the infrastructure is inferior to that found in the main air
base. For example, the hangars in a decentralized
Air command

base may provide less space for larger equipment.

Because of the changes, the durations of different
maintenance tasks can be increased. Decentralization
can also increase the logistic delays involved in transferring materials, tools, and equipment between warehouses and air bases.

The Simulation Model

We constructed a simulation model that describes the
ight operations and maintenance of ghter aircraft
during normal and conict conditions. The model
has three air commands, each with a specic number of aircraft. The aircraft carry out ight missions,
which bring about different maintenance needs. In
the simulation, maintenance is carried out in facilities
within the air commands and in one DE-level facility
that represents the DE-level repair shops of the actual
maintenance organization. The model input data dictate the exact conguration of the ight operations
and maintenance and also govern whether normal or
conict conditions are simulated. The model output
consists of aircraft availability and other performance
measures such as queuing times, resource utilization,
and attained ight intensity.
The Structure of the Air Commands
In the simulation model, the functional entities of
an air command include the ghter squadron and
an IN-level maintenance facility that represents the
air commands repair shop. We model two types
of maintenance facility in the ghter squadron, one
for ight-mission-related inspections and one for
other OR-level maintenance (Figure 2). The DE-level
maintenance facility in the simulation model operates
separately from the air commands.

IN-level facility

Fighter squadron

Class 1 air base

Facility for daily
maintenance

Class 2 air base I

OR-level facility

Class 2 air base II

Class 2 air base III

Figure 2: In the simulation model, we divide an air command into an IN-level maintenance facility and a ghter
squadron that consists of OR-level maintenance facilities and facilities for daily maintenance.

Mattila, Virtanen, and Raivio: Improving Maintenance Decision Making in the FiAF Through Simulation
Interfaces 38(3), pp. 187201, 2008 INFORMS

Each air command can operate in up to four air

bases. The class 1 air base corresponds to the main
peacetime air base of an air command. It includes
a facility for daily maintenance as well as OR- and
IN-level facilities. The other three, which are class 2
air bases, represent alternate bases that include facilities for daily and OR-level maintenance. In a simulation of normal conditions, an air command uses the
class 1 air base. However, in a conict simulation, the
air command typically operates in both class 1 and
class 2 air bases. It has up to four facilities for daily
maintenance, four OR-level facilities, and an IN-level
facility.
The Simulation Logic of Flight Operations and
Maintenance Needs
From an individual aircraft perspective, the simulation consists of daily ight operations, daily maintenance, periodic maintenance, and failure and damage
repairs (Figure 3).
First, an aircraft waits in its home air base until
it is assigned to a ight mission. We determine the
number of aircraft required for a mission and the
duration of a mission randomly from suitable probability distributions (as we discuss later), and select
the required numbers of aircraft from the air bases
of the air command. As an additional criterion, we
select those aircraft that have waited the longest. If a
mission is generated when no aircraft are available in
the air command, the model records the mission as
noncompleted in the output.

Wait for
flight
mission

Carry out
flight
mission

Failed,
damaged, or in
need of periodic
maintenance?

No

Facility for
daily
maintenance

Yes
OR-level
facility
IN-level
facility
DE-level
facility

Figure 3: In the simulation, an aircraft waits in its home air base until it
is assigned to a ight mission. After completing the mission, it undergoes
any necessary maintenance activities and then returns to wait for the next
mission.

191

The model assesses the need for maintenance after

a ight mission. It does not include aborting a mission because of failure or battle damage because
the missions are described as time delays with no
specied objectives. We model periodic maintenance,
failure repairs, and damage repairs using different
maintenance activities that we characterize depending on the type of activity. Periodic maintenance is
performed on the basis of cumulative ight hours and
a predetermined maintenance interval. Time between
failures is measured in ight hours. To model battle
damage, pass-fail probabilities are used to determine
the type of damage sustained during the mission.
Failures are mutually exclusive, i.e., only one type of
failure can occur at a time. This also applies to different types of battle damage. All types of maintenance
needs can, however, be realized during a mission. For
example, an aircraft may sustain both battle damage
and failure.
Each type of maintenance activity is assigned to
a unique facility where the activity is always carried out. Typically, lower-level activities, such as those
that correspond to type I periodic maintenance, are
assigned to OR-level facilities, and higher-level activities to IN- and DE-level facilities. If an aircraft
has multiple maintenance needs, maintenance is performed in the highest-level facility required by the
activities. Aircraft are not transferred between facilities. An aircraft that requires maintenance is immediately transferred to the selected facility and will
remain unavailable for ight duty until the maintenance has been completed.
Aircraft daily maintenance involves turnaround
inspections. All aircraft that return from a ight mission and do not require maintenance, or that return
from maintenance in one of the OR-, IN-, or DE-level
facilities, undergo a turnaround inspection. After an
inspection, an aircraft returns to wait for the next
ight mission. We did not model the preight inspections because test simulations indicated that their
effect on the performance measures of interest is
negligible.
The Simulation Logic of Aircraft Maintenance
The aircraft downtime consists of the maintenance
in OR-, IN-, and DE-level facilities. The simulation
model considers aircraft that are in a turnaround

Mattila, Virtanen, and Raivio: Improving Maintenance Decision Making in the FiAF Through Simulation

192

Transfer to the
maintenance
facility

Interfaces 38(3), pp. 187201, 2008 INFORMS

Wait for
material
delivery

Wait for
available
mechanics

Maintenance

Transfer to
home base

Figure 4: The total maintenance delay is the sum of the transfer delay to
the maintenance facility, possible waiting time for materials and personnel, duration of maintenance, and the transfer delay back to the home
base.

inspection to be available for ight duty. Thus, the

inspection time does not affect aircraft availability.
The time in maintenance in OR-, IN-, and DE-level
facilities involves the duration of the actual maintenance and logistic delays as Figure 4 illustrates.
The transfer delays to and from a maintenance facility are specic to the facility. For example, the transfer
time to a DE-level facility, which is not located within
the air command, is typically longer than the transfer
time to OR- or IN-level facilities.
A set of material requirements characterizes each
type of maintenance. The materials are modeled as
generic items, which can represent, for example, spare
parts, equipment, or tools. A maintenance activity
cannot begin until the necessary materials are available in the maintenance facility. The need for materials is assessed when an aircraft arrives in a facility.
The maximum maintenance crew size and the probability distribution of the duration expressed in maintenance man-hours, both of which are dened separately for each activity, characterize the maintenance
delay. After a possible wait for materials, a mechanics crew gathers to carry out the maintenance. If all
mechanics in the maintenance facility are busy, the
aircraft waits in a rst-in-rst-out queue until one
becomes available. The number of mechanics allocated to the crew is, by default, the maximum crew
size. If the number of available personnel is less than
the maximum crew size, all available mechanics are
allocated. Finally, the net duration of the maintenance
is its duration in maintenance man-hours divided by
the allocated number of mechanics.
The logic for turnaround inspections differs slightly
from the maintenance in other facilities. The wait
for available materials and personnel and the
actual duration of maintenance determine the total
maintenance delay. Because the transfer delay is negligible, the model does not include it.

GUI
(VBA)

Simulation
parameters file
(Excel)

Initial state
file (Excel)

Input

Input
The simulation
model
(Arena)
Output
Simulation
results file
(Excel)

Figure 5: We implemented the simulation model such that simulation

parameters are either fed through the GUI or through the simulation settings le. The initial state of simulation is dened in the initial state le.
The simulation output is written in a results le.

Implementation
We implemented the simulation model using the
Arena software (Kelton et al. 1998); Arena is based
on the SIMAN language (Pegden et al. 1995) and
is intended for construction and analysis of discrete-event simulation models. Figure 5 depicts the
implementation.
The simulation uses a GUI that we implemented
using Visual Basic for Applications (VBA) (Seppanen
2000).
The model input data consist of the simulation
parameters and the initial system state. The simulation parameters dene characteristics of the air commands, maintenance needs, and ight operations. The
initial state denes all the data needed to initialize
the system, e.g., the accumulated ight hours and the
location of each aircraft. The output includes aircraft
availability and various ight and maintenance statistics. All external les of the model are Excel spreadsheets; this makes it easy to manage several sets of
input data and to postprocess the model output.

Distribution Selection and Estimation

of Simulation Parameters
An inherent part of the modeling is dening the
model input data as a function of the operating
conditions and air base structure. As we discussed
above, we used Hawk Mk51 unclassied data for the

Mattila, Virtanen, and Raivio: Improving Maintenance Decision Making in the FiAF Through Simulation
Interfaces 38(3), pp. 187201, 2008 INFORMS

simulation parameters of operations during normal

conditions. We also used these parameters as a starting point for determining the parameters in conict
operations.
Needs and Sources of Data
A FiAF reference data set, which contained either
complete statistics or averaged values of the quantities in question, was available for denition of
the simulation parameters. It included data from
actual ight operations and aircraft maintenance during time periods of one to six years. Throughout
the model construction, FiAF project-team members
cooperated with the authors on the model development; discussions included the general principles of
ight operations, aircraft maintenance, and different
modeling solutions. In addition, we convened two
expert panels. The rst included FiAF maintenance
personnel. The second included two senior maintenance professionals from a DE-level repair shop. In
open discussions, the experts provided their views on
specic input data and modeling assumptions.
We determined some of the model parameters (e.g.,
the parameters for the duration of daily ight and
maintenance activities, the number of maintenance
personnel in the maintenance facilities, periodicmaintenance intervals, and transfer delays of aircraft)
easily from the reference data set. Because the actual
data on spare parts and material inventories are classied, we did not consider material handling. Therefore, subsequent simulations do not consider material
handling delays.
We needed to estimate the parameters for other
items of input data from statistical data or extract
them based on the opinions of subject matter experts.
These items included:
Probability density function (p.d.f.) for the times
between failures;
Probabilities of sustaining each type of damage
during a single ight mission;
P.d.f. for the duration of each type of periodic
maintenance, failure repair, and damage repair;
Maximum size of the crew participating in each
type of periodic maintenance, failure repair, and damage repair;
P.d.f. for the times between ight missions;
P.d.f. for the duration of a mission.

193

Distribution Selection and Parameters for

Periodic Maintenance
The reference data included values for the mean and
standard deviation of the duration of type I periodic
maintenance. Because type I maintenance consisted
of relatively straightforward tasks, we could not consider durations longer than the mean duration to be
more likely than short ones. Therefore, we chose a
symmetric triangular distribution as the model.
We collected the maintenance statistics of the durations of maintenance types IIVI from the IN-level
repair shop of the Air Force Academy, which is the
FiAF primary training unit. This repair shop handles both IN- and DE-level periodic maintenance,
unlike the repair shops in the air commands. Based
on statistical tests and on the histograms of the data
samples, we determined that the maintenance durations should be modeled with a distribution that has
a longer tail on the right side. The subject matter
experts in both panels agreed with this conclusion.
For type II and IV maintenance durations, three families of distributions, Weibull, Beta, and Gamma, provided the best t according to the Chi-square and
Kolmogorov-Smirnov tests (Law and Kelton 2000).
We ultimately chose the Gamma distribution, which
also seemed suitable for maintenance types III, V, and
VI, because the different types of periodic maintenance have many similar tasks.
In choosing the parameter values for the distributions (Table 2), we rst considered type II
maintenance.
We calculated the initial values for the parameters
as maximum likelihood estimates based on the statistical data, and presented the resulting distribution
to the subject matter experts on the expert panels
and within the project team. They assessed how well
this distribution represented maintenance in the overall maintenance organization. Based on their feedback, we adjusted the value of the scale parameter
and the constant in the distribution expression representing the minimum maintenance duration upwards
in the nal choice of parameters; we left the shape
parameter unchanged. We also generalized the shape
parameter in type II maintenance to all other maintenance types from III to VI. Finally, we selected the
scale parameters and minimum maintenance durations such that the ratios of the standard deviation

Mattila, Virtanen, and Raivio: Improving Maintenance Decision Making in the FiAF Through Simulation

194

Interfaces 38(3), pp. 187201, 2008 INFORMS

Maintenance
type

Facility

Crew size

Duration (maintenance man-hours)

OR-level

II

IN-level

III
IV
V
VI

IN-level
DE-level
DE-level
DE-level

4
5
5
6

Tria
8 38 68 (Triangularly distributed

with a minimum value of 8 hours,
mode of 38 hours, and maximum of
68 hours)
200 + Gamma
2 50 (Gamma
distributed with shape parameter
equal to 2 and scale parameter
equal to 50)
500 + Gamma
2 125
1,300 + Gamma
2 300
1,500 + Gamma
2 300
2,000 + Gamma
2 500

Table 2: We determined the crew sizes and the p.d.f.s of the maintenance
durations for periodic maintenance using statistical data and expert opinion. The assignment of maintenance types to maintenance facilities was
readily available in the reference data.

and distribution mean remained approximately the

same as for type II maintenance.
No data were available on the sizes of maintenance
crews that actually perform the maintenance. Therefore, we determined the crew sizes with the help of
the subject matter experts.
Distributions and Parameters for Failure Repairs
We used two failure types for modeling all failures
(Table 3).
The rst type represents the failures that are commonly repaired at the OR-level, and the second the
failures that are repaired at the IN-level. Because
detailed knowledge on failure statistics was not
available, we assumed times between failures to be
exponentially distributed. The mean times between
failures were directly available from the reference
data set. For durations of both types of failure repairs,
the reference data included the mean and standard

Failure
type

Facility

OR-level

IN-level

Time between
failures (ight hours)

Crew
size

Duration
(maintenance
man-hours)

Exp(18.6) (Exponentially
distributed with a
mean of 18.6 hours)
Exp(43.3)

4 + gamma
2 1

78 + gamma
2 11

Table 3: The parameters for failure repairs included the assignment to

maintenance facility, time between failures, crew size, and duration.

deviation. We chose the Gamma distribution to represent the repair durations. We further set the shape
parameters of the distribution equal to those of the
periodic maintenance. We selected scale parameters
and minimum maintenance durations so that the
ratios of the mean and standard deviation remained
the same as in the distributions for periodic maintenance because both types of maintenance involve
similar tasks and are performed by the same repair
shops. Again, we selected crew sizes based on expert
opinion.
Flight Mission Characteristics
Finally, we derived the parameters for the ight operations from the statistics of all the Air Force Academy
ight missions during one year. Based on the reference data, we could model times between ight missions using an exponential distribution with a mean
of 30 minutes. Flight duration, on the other hand, follows a normal distribution with a mean of 45 minutes
and standard deviation of 12. We assumed that a single aircraft is required in each mission.

Model Validation
We validated the simulation model by comparing its
output with actual performance data. We also conducted a sensitivity analysis of the impact of input
data to key performance measures of the model and
let subject matter experts assess the underlying modeling assumptions and simulation results.
Comparison to Actual Performance Data
We chose to compare the actual and simulated aircraft availabilities because availability is the key
performance measure in actual maintenance-related
decision making. The three-month moving average of
availability during a period of four years was available for the validation. The simulation model contained 51 aircraft divided among three air commands
operating in one class 1 air base. We initialized the
accumulated usage hours of the aircraft with a set of
values that was available but that did not relate to
the situation in the data. We used a warm-up period
of six months to erase the results from the initial
transient phase and to reach the steady state of the
simulation. We compared the simulated availabilities

Mattila, Virtanen, and Raivio: Improving Maintenance Decision Making in the FiAF Through Simulation
Interfaces 38(3), pp. 187201, 2008 INFORMS

1.0
Simulated
Actual

Aircraft availability

0.9

0.8

0.7

0.6

0.5

Time (years)
Figure 6: The simulated availabilities of 10 independent replications are
very close to the actual Hawk Mk51 availability during a four-year time
period. The gure is based on the three-month moving averages of both
actual and simulated availabilities.

from 10 independent simulation replications with the

actual availability data (Figure 6).
The average availability that the model predicted
was approximately 0.72; however, the data showed
an average availability of approximately 0.70. Some
of the difference is because the input data did not
consider material handling as we discussed above. In
addition, the simulation did not seem to reproduce
a drop in the actual availability just after the second
year because of additional modication work that the
aircraft underwent during the time period. We did
not consider the modication work in the input data
because our purpose was to describe average ight
operations and maintenance. Otherwise, the simulation seemed to reproduce the actual availability well.
Sensitivity Analysis
We can use sensitivity analysis to assess how changes
in input data affect simulation output. The analysis
implies that the model is valid if the simulation output is affected in the same way as the actual system
would be under corresponding changes. Because sets
of reference data from a wide range of operating conditions are not generally available for such analysis,
the sensitivity results are frequently assessed subjectively by both model constructors and subject matter
experts.

195

Therefore, we conducted the sensitivity analysis by

examining which of a set of 12 input data items
affected the average aircraft availability signicantly
in the previously described simulation of the fouryear time period. We used design of experiments
(Montgomery 2001) for the analysis and devised a
2124 fractional factorial design involving 256 simulation runs to estimate the effects of the items. In
the design, we set the simulation parameters corresponding to the items to either 1 or +1 level as
Table 4 describes, but left other simulation parameters
unchanged.
We should also note that in Table 4 the number of mechanics in the maintenance facilities represents the effective amount of personnel resources, i.e.,
all mechanics are capable of performing all required
maintenance work within the facility. We also considered ight intensity in terms of the time between
ight missions, but did not include mission duration
in the design because the effects of both variables
are very similar and the exclusion of either variable
helped to limit the required number of simulation
runs. We combined failure types 1 and 2 into the overall mean time between failures for the same reason.
Table 4 lists the 95 percent condence intervals for the
changes in average availability due to the changes in
the input data items. The effect of each item is statistically signicant. We selected the 1 and +1 levels
in the design so that the 1 level would presumably
result in lower availability and the +1 level in higher
availability. Because the effects for all items are positive, the results are consistent with our initial expectations. The time between ight missions has the largest
effect because ight intensity governs the amount of
all maintenance needs. The model is also sensitive to
the number of DE-level mechanics and the durations
of type IV, V, and VI periodic maintenance that the
DE-level facility performs. The subject matter experts
expected this because many aircraft underwent complex periodic maintenance during the observed time
period. The number of OR-level mechanics and the
duration of type I periodic maintenance performed in
the OR-level facilities have the smallest effect because
the number of mechanics was high relative to the
maintenance needs. Decreasing the number of personnel from the +1 to 1 level did not congest the facilities and showed little effect on aircraft availability.

Mattila, Virtanen, and Raivio: Improving Maintenance Decision Making in the FiAF Through Simulation

196

Interfaces 38(3), pp. 187201, 2008 INFORMS

Input

1 level

Daily maintenance
OR-level
IN-level
DE-level
Time between ight missions
Duration of periodic maintenance
I
II
III
IV
V
VI
Overall mean time between failures

+1 level

Number of mechanics in maintenance facilities

13
17
5
7
13
17
22
28
Exp(26)
Exp(34)

95 percent condence
interval of the change in
average availability

00065 00011
00016 00010
00081 00009
01578 00039
02182 00038

Tria
103 418 733

220 + gamma
22 50
550 + gamma
22 125
1,430 + gamma
22 300
1,650 + gamma
22 300
2,200 + gamma
22 500

Tria
57 342 627

180 + gamma
18 50
450 + gamma
18 125
1,170 + gamma
18 300
1,350 + gamma
18 300
1,800 + gamma
18 500

00018 00007
00071 00011
00075 00014
00518 00019
00189 00019
00583 00014

11.7

14.3

00138 00018

Table 4: We conducted a sensitivity analysis by examining the effects of 12 items of input data on simulated
aircraft availability. The effect of each item was statistically signicant. Because the changes in availability were
positive for all items, the directions of the effects were also consistent with our initial expectations.

Some interaction effects of two input data items

were also signicant. The change in aircraft availability resulting from a change in one of the corresponding items depends on the level of the other item.
Therefore, we cannot interpret the effects of single
variables literally. They indicate the relative importance of the items, however. For brevity, we chose not
to present the interaction effects in this paper.
Expert Validation
In addition to assessing the results of the sensitivity analysis, subject matter experts were also involved
in other aspects of model validation. The two expert
panels discussed the underlying modeling assumptions and output of preliminary versions of the
model, and the FiAF project-team members repeatedly addressed both modeling solutions and simulation results.
In the nal phase of model construction, we
arranged two user training sessions to introduce both
the model and basic principles of the simulation
approach to FiAF maintenance designers. The training was necessary because the designers would use
the model independently at a later time. We also
saw the training as an opportunity to further validate
the model. We asked the designers to give feedback
on any of its features. Thus, they contributed to the

validation both as end users and as subject matter

experts.
Because the underlying system of ight activities
and aircraft maintenance is large and multifaceted,
we discussed many issues of wide-ranging scope with
the experts during model validation, e.g., the formation of maintenance teams and the sequence of activities during individual maintenance tasks with the
second expert team. We addressed higher-level issues
such as the nature of conict-time operating conditions or appropriate performance measures of maintenance primarily with the project team. Overall, the
need to involve experts with different backgrounds in
model construction and validation was apparent.
In meeting with the experts who were not members of the project team, the team members took
part in introducing the background and objectives
of the project. Our impression was that this greatly
helped to make the experts receptive to the simulation approach and committed them to improving
the model. In the meetings, we used the guidelines
of a structured walk-through that Law and Kelton
(2000) describe, and allowed as much time as necessary to discuss the modeling assumptions and simulation results. We also took great care to devote enough
time to introducing the basics of simulation modeling to the experts. In the user training sessions, the

Mattila, Virtanen, and Raivio: Improving Maintenance Decision Making in the FiAF Through Simulation

197

Interfaces 38(3), pp. 187201, 2008 INFORMS

designers were able to explore what the model does;

thus, they gained a far clearer view of its functionality
than they would in a standard classroom presentation. These actions allowed us to be condent that the
assessments the experts ultimately gave were based
on a sufciently detailed understanding of the model
and the simulation approach.
To summarize, the experts contributed to the validation in two ways. First, they helped us to adjust
the simulation model and its input data during model
construction. Second, they helped to conrm that the
nal model described the ight operations and aircraft maintenance with enough accuracy to make it
sufcient for practical use.

Practical Use of the Model

The simulation model provides FiAF with a quantitative analysis tool for the ight operations and
maintenance of its aircraft eet. In normal operating
conditions, the model can help maintenance designers to allocate appropriate personnel and material
resources for an exercise with high ight intensity.
While this is important to the designers, their ultimate
concern is to learn how to maximize the conict-time
operational capability of the eet. As an example of a
conict-related application of the model, we cooperated with the FiAF project team to simulate a scenario
in which we examined the aircraft periodic maintenance policy. The simulation provided information on
the number of aircraft that can be expected to be available and the maximum number of ight missions that
can be performed during the conict.
The Conict Scenario
The conict scenario we considered involved four different phases. In the rst phase, the level of readiness is increased resulting in higher ight intensity.
In phase two, the ight intensity is further increased;
each air command moves to operate from the normal main air base into four decentralized air bases
and carries out ight operations and maintenance
24 hours a day. In the third phase, there is actual
conict in the form of aerial battles and the aircraft
begin to sustain battle damage. In the fourth and nal
phase, the ight intensity is decreased as the conict
approaches an end.

In the scenario, we examined the aircraft periodic

maintenance policy. The maintenance facilities can
become congested at some point during the conict
because of the increased ight intensity and the need
for battle-damage repairs. The periodic maintenance
can then be temporarily suspended to guarantee that
a sufcient number of aircraft are available for ight
missions. We assume that the decision of whether to
suspend periodic maintenance is made at the beginning of each phase of a scenario. If the maintenance
is suspended, it will not be continued in any of
the remaining phases. However, any ongoing maintenance will be completed. We simulated four alternative policies:
(1) All periodic maintenance is suspended at the
beginning of the rst phase.
(2) All periodic maintenance is suspended at the
beginning of the second phase.
(3) All periodic maintenance is suspended at the
beginning of the third phase.
(4) The periodic maintenance is not suspended
during the scenario.
Scenario Input Data
The scenario input data are based on the previously
described set of simulation parameters. We modeled
the different phases of the conict by changing the
simulation parameters as Table 5 describes. All other
parameters remain unchanged.
The description of battle damage in the third and
fourth phases is an essential part of the simulation. Because no data were available for estimating the battle-damage parameters, we determined the
parameters with the help of the FiAF project team.
We assumed three types of damage in the scenario
(Table 6).
Number of Daily duration of
operative
ight operations Time between
Duration air bases per and maintenance ight missions Battle
Phase (days) air command
(hours)
(min.)
damage
1
2
3
4

30
30
10
30

1
4
4
4

8
24
24
24

Exp(24)
Exp(20)
Exp(20)
Exp(40)

no
no
yes
yes

Table 5: We modeled the conict scenario by changing a set of simulation

parameters according to the different phases of the conict.

Mattila, Virtanen, and Raivio: Improving Maintenance Decision Making in the FiAF Through Simulation

198

Interfaces 38(3), pp. 187201, 2008 INFORMS

Damage
type
1
2
3

Facility

Probability
during a single
ight mission

Crew
size

Duration
(maintenance
man-hours)

OR-level
IN-level
DE-level

0025
0015
001

4
4
4

2.6 + gamma
2 07

6.5 + gamma
2 18
130 + gamma
2 357

Table 6: We determined the battle-damage parameters for phases 3 and 4

with the help of FiAF project-team members.

We obtained the initial state of simulation for the

scenario as the nal state of a long simulation from a
suitable but articial initial state.
Simulation Results
We conducted 40 independent simulation replications
for each alternative policy. Figure 7 illustrates the
development of aircraft availability averaged across
the replications.
The most critical phase of the conict is the actual
combat phase, during which the availability decreases
rapidly. If policy 4 is employed, the periodic maintenance will use up the maintenance resources and
delay the battle-damage repairs. The availability consequently drops to as low as 0.4; this means that it is
very unlikely that the air commands could meet all
their operational goals. Policies 1 and 2 produce the
highest availability.

We concluded that some of the periodic maintenance must be suspended to maintain operational
capability, if maintenance resources, battle damage,
and ight intensity are as the scenario assumed. It
seems that the maintenance policy should be changed
before the actual combat phase. Although some types
of periodic maintenance can be performed in the early
phases, postponing the change of policy can prove
problematic in practice because the phase durations
are not known with certainty. We should also note
that suspending periodic maintenance affects the failure rate of the aircraft. The impact of periodic maintenance on the failure rates of aircraft is a challenging
topic that requires further research. Because no statistical data on this dependence were available and
the nature of the maintenance policy very preemptive, we kept the failure rate unchanged in the simulations. The simulation results therefore represent the
best-case benet of suspending the maintenance.
We also considered the daily number of completed
ight missions. If mission requests arrive with high
intensity, the air commands may not be able to
respond to all of them because of aircraft unavailability (Figure 8).
We averaged the results across 40 independent
replications. Based on the results, it would again be
benecial to suspend periodic maintenance at some
1.0

0.9

Daily proportion of completed flight missions

1.0
Policy 1

Aircraft availability

0.8
Policy 2

0.7
0.6

Policy 3

0.5
Policy 4

0.4
0.3
0.2
0.1
Phase 1

10

Phase 2

20

30

Phase 3 Phase 4

40

50

60

70

80

90

100

Time (days)
Figure 7: Policies 1 and 2 maintain a clearly higher aircraft availability
than policies 3 and 4; this indicates that some periodic maintenance must
be suspended during the conict.

Policy 1
0.9

Policy 2

0.8
Policy 3
Policy 4

0.7

0.6

0.5
Phase 1

0.4

10

Phase 2

20

30

Phase 3 Phase 4

40

50

60

70

80

90

100

Time (days)
Figure 8: The daily proportion of completed ight missions during the conict scenario indicate, as the availability results did, that operational
capability is best maintained with policies 1 and 2.

Mattila, Virtanen, and Raivio: Improving Maintenance Decision Making in the FiAF Through Simulation
Interfaces 38(3), pp. 187201, 2008 INFORMS

point during the conict. At the same time, the effect

of each policy on the ight operations became clearer.
The proportion of mission requests to which there
was no response during the busiest phase of the
conict was approximately 10 percent with policies
1 and 2. The ratios were 25 percent and 40 percent,
respectively, for policies 3 and 4.
The above results show that the consideration
of maintenance policies is essential to maintaining the operational capability in a conict situation.
A maintenance organization that is sized for normal conditions will have difculty handling increased
maintenance needs even if the battle damage probabilities are reasonably small. Additional simulations could assess the amount of additional resources
required during a conict, which maintenance activities should be suspended, and when they should
be suspended. Overall, the model provides valuable
information to support the decision making that is
involved in devising contingency plans for aircraft
maintenance.

Model Construction Challenges

We faced several challenges in constructing the
model. The primary one was scarcity of data. No statistical data were available for modeling elements of
the ight operations and aircraft maintenance such
as battle-damage probabilities and repair-time distributions. We found that subject matter experts from
different units and organizational levels needed to
be involved in determining the corresponding model
components. The experts provided their views on
the issues at hand, and the authors explained the
benets and drawbacks of incorporating a given modeling solution to them. In addition to being essential in determining given modeling assumptions, the
communication with the experts helped us to rene
our overall view of the ight operations and aircraft
maintenance.
Because we designed a number of components in
the model with the help of subject matter experts,
we needed to carefully assess whether the overall
model validly described the usage and maintenance
of the aircraft. We did extensive sensitivity analyses
to quantify how the output of the model would be
affected by departures from modeling assumptions

199

or input data. The analyses included the examination

of the structural assumptions associated with several
key components of the model, e.g., the logic of aircraft maintenance. We also tested the effect of changing the distributions of maintenance durations. As
Table 4 summarizes, we used an experimental design
to examine the effects of input data. We presented
results of our analyses to the experts and allowed
them to use the model. Therefore, we are condent
that we captured the views of the experts correctly in
the nal model.
Another challenge we faced was condentiality
of data. FiAF representatives could not provide the
authors with access to highly classied information.
However, some of this information was necessary to
model scenarios that FiAF ultimately wished to consider. For example, it included the contingency plans
on maintenance policies and anticipated ight intensities, battle-damage rates in various conict scenarios,
and the statistics from the normal time maintenance
of F-18s in the air commands.
To overcome this difculty, we implemented the
model such that the choice of input data fully governs
its operations logic. For example, we did not hardcode the conict-time maintenance policies. Instead,
we modeled these policies by selecting suitable values
for a set of input parameters. We could isolate condential information for separate handling by FiAF.
Naturally, determining the structure of the model
did require some discussions on condential issues
between the authors and the FiAF project team. The
team members described in general terms what the
model should be able to do. Based on their description, we implemented the corresponding model components and revised them repeatedly until the model
was satisfactory. Thus, we managed to guarantee that
the model had the right functionality for consideration of any relevant normal or conict-time scenarios
although we could not use classied information
directly.

Conclusions
The practical use of the simulation model implies
that it offered FiAF a valuable aid in improving
maintenance-related decision making. We rst introduced the model and initiated the project in FiAF

200

Mattila, Virtanen, and Raivio: Improving Maintenance Decision Making in the FiAF Through Simulation

headquarters. In the early phases, the FiAF projectteam members were the primary users. They applied
the model to produce short-term forecasts of aircraft
availability. They also used the model to analyze the
accumulation of maintenance needs in some of the
larger exercises and to assess the resource requirements of a smaller group of aircraft that participated
in a combined exercise with Air Forces from other
countries.
We delivered the model to the air commands as
well as to other units of FiAF. The units applied the
model to analyze the effects of a readjusted periodic
maintenance policy for the F-18s.
The project also served as a pilot study to advance
the application of simulation techniques in aircraft
maintenance, air base logistics, and other areas at
FiAF. For example, shortly after the completion of
the simulation model that we discussed in this paper,
FiAF cooperated with the Finnish Army on a simulation project to analyze the maintenance system of the
Armys new transport helicopters.
The model is suitable for training purposes.
Because it is GUI-based and does not require detailed
knowledge of the underlying simulation software, it
is useful in classroom demonstrations or individually
by trainees. However, users still need some time to
acquaint themselves with the model. Therefore, training has thus far been limited to situations where the
schedule allows a thorough introduction to the model
functionality. Some of the graduating students of the
Air Force Academy have applied the model for simulation analyses in their theses.
The process of constructing the simulation model
has also brought indirect benets. The subject matter
experts involved in the construction were required to
describe the organization and interaction of given elements of the maintenance system. The FiAF projectteam members and some of the other experts said that
this involvement helped them to obtain new insights
into the system. They regarded this as an additional
project benet.
In the future, FiAF will use simulation to design and
control ight operations and aircraft maintenance. Its
research directions include the simulation of smaller
elements of the maintenance system, e.g., a single
decentralized air base. We have also begun to pursue the scheduling of aircraft periodic maintenance by
using simulation-based optimization techniques.

Interfaces 38(3), pp. 187201, 2008 INFORMS

Acknowledgments

We gratefully acknowledge the help of the people at FiAF

who were involved in this project. In particular, we thank
Major Riku Lahtinen for his invaluable support to the entire
effort.

References
Balaban, H. S., R. T. Brigantic, S. A. Wright, A. F. Papatyi. 2000.
A simulation approach to estimating aircraft mission capable
rates for the United States Air Force. J. A. Joines, R. R. Barton,
K. Kang, P. A. Fishwick, eds. Proc. 2000 Winter Simulation Conf.,
Orlando, FL, 10351042.
Ciarallo, F. W., R. R. Hill, S. Mahadevan, V. Chopra, P. J. Vincent,
C. S. Allen. 2005. Building the mobility aircraft availability forecasting (MAAF) simulation model and decision support system. J. Defense Model. Simulation Appl. Methodology Tech. 2(2)
5769.
Kang, K., K. R. Gue, D. R. Eaton. 1998. Cycle time reduction for
naval aviation depots. D. J. Medeiros, E. F. Watson, J. S. Carson,
M. S. Manivannan, eds. Proc. 1998 Winter Simulation Conf.,
Washington, D.C., 907912.
Kelton, W. D., R. P. Sadowski, D. A. Sadowski. 1998. Simulation with
Arena. McGraw-Hill, Boston.
Kladitis, P. E., J. P. Worden, R. H. Searle. 2007. Estimating and modeling inherent availability of the B-52H using SIMPROCESS.
Proc. 2007 U.S. Air Force T&E Days, AIAA, Destin, FL.
Law, A. M., W. D. Kelton. 2000. Simulation Modeling and Analysis,
3rd ed. McGraw-Hill, Boston.
Montgomery, D. C. 2001. Design and Analysis of Experiments, 5th ed.
John Wiley & Sons, New York.
Pegden, C. D., R. E. Shannon, R. P. Sadowski. 1995. Introduction to Simulation Using SIMAN, 2nd ed. McGraw-Hill,
New York.
Pohl, L. M. 1991. Evaluation of F-15E availability during operational test. B. L. Nelson, W. D. Kelton, G. M. Clark, eds. Proc.
1991 Winter Simulation Conf., Phoenix, AZ, 549554.
Rodrigues, M. B., M. Karpowicz, K. Kang. 2000. A readiness analysis for the Argentine Air Force and Brazilian Navy A-4 eet
via consolidated logistics support. J. A. Joines, R. R. Barton,
K. Kang, P. A. Fishwick, eds. Proc. 2000 Winter Simulation Conf.,
Orlando, FL, 10681074.
Seppanen, M. S. 2000. Developing industrial strength simulation
models using Visual Basic for Applications (VBA). J. A. Joines,
R. R. Barton, K. Kang, P. A. Fishwick, eds. Proc. 2000 Winter
Simulation Conf., Orlando, FL, 7782.
Upadhya, K. S., N. K. Srinivasan. 2004. Availability of weapon systems with air-attack missions. J. Defense Model. Simulation Appl.
Methodology Tech. 1(2) 111121.

Major Riku Lahtinen, Armaments Division, FiAF

Headquaters, PO Box 30, 41161 Tikkakoski, Finland,
writes: I have acted as the head of the project
team of the Finnish Air Force (FiAF) and as the
primary contact between FiAF and the authors in
the project that is described in the paper Improving Maintenance Decision Making in the Finnish Air

Mattila, Virtanen, and Raivio: Improving Maintenance Decision Making in the FiAF Through Simulation
Interfaces 38(3), pp. 187201, 2008 INFORMS

Force Through Simulation. With this letter, Id like

to verify that the paper gives an accurate account of
the details of the project and the benets that it has
provided.
The Armaments Division, together with the Material Command, carry out the development of the aircraft of FiAF to meet operational and airworthiness
requirements. The division also plans and coordinates
the research efforts that are undertaken to support
the development. Our task is to guarantee that the
aircraft are safe, powerful, and properly equipped
at all times: in training, in increasing number of
international operations, as well as in all degrees of
readiness. This requires us to continuously improve
the quality of the maintenance processes. Since the
resources are not unconstrained, the quality must be
developed by considering the efciency of the processes as well.
We had ongoing collaboration with the Systems
Analysis Laboratory, Helsinki University of Technology, and asked them to propose how we could study
the effect of maintenance on aircraft availability. The
research team of the Systems Analysis Laboratory
rst conducted a pilot simulation study where the
operations of a single airbase were considered. We
regarded this pilot study as a success and decided
on requesting a model of the maintenance of the

201

entire ghter aircraft eet. The specics of the resulting model are described in the paper.
The benet of the simulation model has been
unquestionable. It has given us a sophisticated
approach to analyze things with a less labored way
than earlier. Besides the Armaments Division, other
units have beneted from the model in assessing
proposed improvements to maintenance practices.
Although the details of these analyses are mostly condential and can not be elaborated here, I can state
that they are signicant parts in the development of
aircraft maintenance in FiAF.
Another result of the project has been the emergence of fresh conversation and exchange of ideas
between different branches of aircraft maintenance.
The people with different backgrounds were exposed,
in a positive sense, to each others viewpoints during the discussions that went on in the project. I can
say with condence, that my understanding of the
different branches has improved and I truly believe
that this is the case for a number of other people.
Since these people are our most important assets, the
impact of the project has been an important one.
Our experiences from the project have been positive and we see simulation applications as an integral
part in maintenance-related decision making in the
future.