Managing Variability:

Queues and Waiting Time

1
Variability: Bad for the Process

2
Variability and Why It Is Bad for a Process

• Waiting is the most visible and annoying problem experienced by customers

• Queue will form and waiting will occur if Implied utilization > 100%:
• Expected arrival (demand) rate > the expected supply rate (capacity)
for a limited period of time
• A form of mismatch between supply and demand
• BUT can queues and waiting occur when implied utilization < 100%?
• YES. The cause is VARIABILITY (randomness) in the arrival and service
process.
• This makes waiting time and the length of the queue also unpredictable
(random).

3
 What to See in This Module

• Queueing or Waiting Line systems.

• Analytic tools to predict the waiting times and other performance
metrics for service quality
• Reducing customers waiting time by:
• Choosing appropriate capacity levels
• Redesigning the service system
• Pooling the servers to reduce variability

4
Example: A Somewhat Unrealistic Call Center

• Call Center: one employee, between 7:00am – 8:00am

• Calls arrive exactly every five minutes
• A call takes exactly four minutes to complete
• Customers are willing to wait

5
 A Somewhat Unrealistic Call Center
• Calls arrive EXACTLY every 5 minutes and it takes EXACTLY 4 minutes to answer a call
• The operator rests (idle) one minute between calls
• Then there is NO QUEUE and NO customer has to wait

Arrival Service
Patient
Time Time
1 0 4
2 5 4
3 10 4
4 15 4
5 20 4
6 25 4
7 30 4
8 35 4
9 40 4
10 45 4
11 50 4
12 55 4
6
7:00 7:10 7:20 7:30 7:40 7:50 8:00
 A More Realistic Service Process
Caller 1 Caller 3 Caller 5 Caller 7 Caller 9 Caller 11
Arrival Service
Caller Time Time Caller 2 Caller 4 Caller 6 Caller 8 Caller 10 Caller 12
1 0 5
2 7 6 Time
3 9 7 7:00 7:10 7:20 7:30 7:40 7:50 8:00
4 12 6
5 18 5
6 22 2 3
7 25 4 Number of cases
8 30 3 2

9 36 4
1
10 45 2
11 51 2
0
12 55 3 2 min. 3 min. 4 min. 5 min. 6 min. 7 min.
Service times

Data gathered at a call center

7
Variability Leads to Waiting Line and Waiting Time
Caller 1
Caller 2
Caller 3
Service time
Caller 4
Caller 5
Caller 6
Caller 7
Caller 8
Caller 9
Caller 10
Wait time
Caller 11
Caller 12

7:00 7:10 7:20 7:30 7:40 7:50 8:00

Inventory
(Callers on 5
hold)
4

0
7:00 7:10 7:20 7:30 7:40 7:50 8:00
Time

Detailed analysis of call center

8
 The Curse of Variability
• Variability hurts flow:
With buffers: we see waiting times and inventory
even though there exists excess capacity
• Variability is BAD and it does not average itself out
• New models are needed to understand these effects: Queueing Models
• Assumption: Stationary Arrival and Service Process
• An arrival process is stationary over a period of time if the number of
arrivals in any sub-interval depends only on the length of the interval and
not on when the interval starts.

9
Arrivals over the day are Non-stationary
… and the peaks
Number of customers The number of arrivals and troughs are
Per 15 minutes in a 3-hour interval predictable (they
clearly depends on occur roughly at the
160
which 3-hour interval same time each day.
140 your choose …

120

100

80

60

40

20

0
0:15

2:00

3:45

5:30

7:15

9:00

14:15

16:00

17:45

19:30

21:15

23:00
10:45

12:30

Time
What to Do With Nonstationary/Seasonal Data?
Measure the true demand data Apply Waiting Line model in each slice

Slice the data by the hour (30min, 15min)

• Within a “slice”, the data

for the times between
arrivals follows an
exponential distribution
• For Exponential
distribution: CVa = 1

11
Arrival Process and Service Process:

Coefficient of Variation

12
How to Model the Interarrival Times
• We will use two parameters to describe inter-arrival times to a queue:
• The average inter-arrival time: a
• The standard deviation of the inter-arrival times: 𝝈𝒂
• These two parameters can be estimated from the data

• What is the standard deviation?

• Roughly speaking, the standard deviation is a measure of how
variable the interarrival times are.

13
Absolute and Relative Variability
• The standard deviation is an absolute measure of variability.
• Two processes can have the same standard deviation but one seem to be
much more variable than the other:
• Below are random samples from two arrival processes that have the same
standard deviation, but the one on the left seems to have more variability.

Average = 10 Stdev = 10 Average = 100 Stdev = 10

50 120
45
100
Inter-arrival time (min)

Inter-arrival time (min)

40
35
80
30
25 60
20
15 40
10
20
5
0 0
Observation Observation

14
 Coefficient of Variation
• The coefficient of variation is a measure of the relative variability of a process –
it is the standard deviation divided by the average.
Standard deviation of interarrival time  a
• The coefficient of variation of the arrival process: CVa  
Average interarrival time a

• For Exponential distribution, the standard deviation = the mean = a

𝒂
• Hence, for Exponential distribution: CVa = = 1
𝒂

Average = 10 Stdev = 10
Average = 100 Stdev = 10
5.0
4.5
Inter-arrival time /Average

5.0
4.0 CVa = 10 /10 = 1 4.5

Inter-arrival time /Average

3.5 4.0
3.0 3.5
2.5 3.0
2.0 2.5
1.5 2.0
1.0 1.5
1.0
0.5 CVa = 10 /100 = 0.1 0.5
0.0
0.0
Observation
Observation

15
 The Service Time
• The average Service Time: p
• The standard deviation of the service time: 𝝈𝒑
Standard deviation of service times  p
• The coefficient of variation of the service time: CVp  
Average service time p

Frequency
800

600 Example:
 Standard deviation of the service times = 150 seconds.
400
 Average call time (i.e., service time) = 120 seconds.
200
 CVp = 150 / 120 = 1.25
0  For Exponential distribution: CVp = 1
Call durations (seconds)
16
 The Average Rate vs. Time
𝟏 𝟏
• Important to remember: Rate = ; Time =
𝑻𝒊𝒎𝒆 𝑹𝒂𝒕𝒆

Example:
𝒎𝒊𝒏𝒖𝒕𝒆𝒔
• Arrival Process: Average Inter-arrival Time: a=5 ;
𝒄𝒖𝒔𝒕𝒐𝒎𝒆𝒓
𝟏 𝟏 𝐜𝐮𝐬𝐭 𝟔𝟎 𝐦𝐢𝐧 𝒄𝒖𝒔𝒕𝒐𝒎𝒆𝒓𝒔
Average Arrival Rate (Flow Rate): λ or R = = × = 12
𝒂 𝟓 𝒎𝒊𝒏 𝟏 𝒉𝒓 𝒉𝒐𝒖𝒓

𝒎𝒊𝒏𝒖𝒕𝒆𝒔
• Service Process: Average Processing (service) Time: p = 4
𝒄𝒖𝒔𝒕𝒐𝒎𝒆𝒓

𝟏 𝟏 𝒄𝒖𝒔𝒕 𝟔𝟎 𝒎𝒊𝒏 𝒄𝒖𝒔𝒕𝒐𝒎𝒆𝒓𝒔

Average Service Rate (Capacity): µ= = × = 15
𝒑 𝟒 𝒎𝒊𝒏 𝟏 𝒉𝒓 𝒉𝒐𝒖𝒓

λ Flow Rate Avg. 𝑎𝑟𝑟𝑖𝑣𝑎𝑙 𝑅𝑎𝑡𝑒 Avg Service Time p 4

• Utilization: = u = = = = = = 0.8 = 80%
µ 𝐶𝑎𝑝𝑎𝑐𝑖𝑡𝑦 𝐴𝑣𝑔 𝑆𝑒𝑟𝑣𝑖𝑐𝑒 𝑅𝑎𝑡𝑒 𝐴𝑣𝑔 𝐼𝑛𝑡𝑒𝑟𝑎𝑟𝑟𝑖𝑣𝑎𝑙 𝑇𝑖𝑚𝑒 𝑎 5

17
Predicting the Average Waiting Time for
General Queue

18
 Notation Description Another Common Notation
𝟏
a Average Inter-arrival “Time”
N 𝟏ൗ
𝒂 Average Arrival Rate
𝝀

𝝀 (lambda)

o p Average Service (processing) “Time”

𝟏
𝝁

𝝁 (mu)
t 𝟏ൗ
𝒑 Average Service Rate (service capacity)

a m
u
The number of servers (resources)

Utilization = Flow rate / Service Capacity =

𝒑
m or s

(rho) 𝜌 =
𝝀
𝒂×𝒎
t Tq Average Time in the Queue Wq
𝒎×𝝁

i T
Iq
Average Time in the System (Flow Time) = Tq + p
Average Inventory (The number of customers in the queue)
W
Lq

o I
Ip
Average Inventory (The number of customers in the system)
Average number of customers (inventory) in service =
𝒑
L
r = 𝝀Τ𝝁
𝒂
n CV Coefficient of Variation =
𝑺𝒕𝒂𝒏𝒅𝒂𝒓𝒅 𝑫𝒆𝒗𝒊𝒂𝒕𝒊𝒐𝒏
𝑴𝒆𝒂𝒏
𝜎𝑎
CVa Coefficient of Variation of the inter-arrival Time =
𝑎
𝜎𝑝
CVp Coefficient of Variation of the processing (service) Time = 19
𝑝
 Modeling Variability in Flow – Queueing Models
Outflow
In-Flow: Minimum{Demand, Capacity} Processing No loss, waiting only
= Demand (arrival) rate = 1/a Buffer
(Service) This requires u < 100%
(Queue) Outflow = Inflow

Inflow (Arrival):
• Demand process is “random” Processing (Service):
• Look at the inter-arrival times • p = average processing (service) time
IA1 IA2 IA3 IA4
St-Dev (processing times)
Time • CVp = Average (processing times)
• a = average inter-arrival time
• Often Exponential/Poisson distributed:
St-Dev (inter-arrival times)
CVa = • CVp = 1
Average (inter-arrival times)
• Constant hazard rate (no memory)
• Often, Exponential/Poisson distributed: • Exponential inter-arrivals
• CVa = 1
• Constant hazard rate (no memory)
• Exponential inter-arrivals
20
 Standard Queueing Notation
• If the random variables representing arrival and service processes follow the Poisson distribution
(for arrival or service Rate) or Exponential distribution (for arrival or service Time),
 We use letter “M” to designate this assumption
 “M” stands for Markov – the famous Russian mathematician
 If we do not know the exact distribution function of the arrival or service processes, we use
the letter “G” which stands for General distribution.
• For example: M/M/1 queue
 The first “M” says that the arrival process is random and the arrival rate follows a Poisson
distribution or the inter-arrival time follows an exponential distribution
 The second “M” says that the service process is random: the service rate follows a Poisson
distribution or the service time follows an exponential distribution (for service time)
 The number “1” means that there is ONE server

21
Queueing Notation
• Another example: G/G/m queue.
• G/G/m means that we do not know the exact probability distribution functions
of the arrival and service processes.
• m is the number of servers, m=1, 2, 3, 4, …
• To calculate the waiting time of a customer in the queue, all we need to know are
the means and standard deviations of the inter-arrival time and the service time.
• The queueing formula uses the coefficient of variation of both the inter-arrival
time and the service time.
𝒔𝒕𝒂𝒏𝒅𝒂𝒓𝒅 𝒅𝒆𝒗𝒊𝒂𝒕𝒊𝒐𝒏
• Coefficient of Variations: CV =
𝒎𝒆𝒂𝒏 (𝒂𝒗𝒆𝒓𝒂𝒈𝒆)
• For exponential distribution: CV = 1

22
The Waiting Time Formula for Single Server Queue: G/G/1

Flow Rate Average Flow Time T

Inventory waiting Iq
Inflow Outflow
Increasing
Variability
Entry to system Begin Service Departure

Time in queue Tq Service Time p Theoretical Flow Time

Flow Time: T = Tq + p
Utilization 100%

Waiting Time Formula:

 CVa2  CV p2 
 utilizatio n   
Tq = Time in queue  p     
 1  utilization  2 
 
Variability factor
𝒑
Utilization: u = Utilization factor
𝒂
Service time factor 23
 Waiting Time Formula for Multi-Server Queue: G/G/m
Inventory in service Ip
Inventory
𝟏 waiting Iq 𝟏
Inflow = Outflow = 𝒂
𝒂 𝟏
Flow rate = 𝒂

Entry to system Begin Service Departure

𝒑
Utilization : u = Time in queue Tq Service Time p
𝒎×𝒂

Flow Time T = Tq + p
Waiting Time Formula for multi-server Queue:
 utilization 2( m 1)  1   CVa2  CV p2 
 Service time      
Time in queue      
 m  1  utilization  2 
    24
 Summary for G/G/m Queue:
 p  2( m 1) 1   CV 2
 CV 2
• Waiting Time: Tq     
u    a p 
 m   1  u 



2 


• Flow Time: T = Tq + p

• a = average inter-arrival time • m = The number of servers (resources)

• Coefficient of Variation of interarrival time: • p = average processing (service) time
St-Dev (inter-arrival times) • Coefficient of Variation of service time:
• CVa =
Average (inter-arrival times)
• CVp = St-Dev (processing times)
Average (processing times)

𝒑
Utilization : u =
𝒎×𝒂

25
 Example : Online Retailler
Customers send emails to a help desk of an online retailer every 2 minutes (30 emails per hour), on average, and the
standard deviation of the inter-arrival time is also 2 minutes. The online retailer has three employees answering
emails. It takes on average 4 minutes (15 emails per hour) to write a response email. The standard deviation of the
service times is 2 minutes.
Question: Estimate the average waiting time and flow time for an email to be answered.

Solution: This is a multi-server queue with 3 servers (m=3) or G/G/3 queue or M/G/3 queue
• Inter-arrival time: Average: a = 2 minutes per customer
Standard deviation of inter-arrival time = 𝝈𝒂 = 2 min. per customer
𝝈𝒂 𝟐
CVa = = =1
𝒂 𝟐
(Recall: exponentially distribution: CV = 1, and we use “M” for arrival)
• Service Time: Average: p = 4 minutes per customer
Standard deviation of service time = 𝜎𝑝 = 2 minutes
𝝈𝒑 𝟐
CVp = = = 0.5
𝒑 𝟒
𝑨𝒓𝒓𝒊𝒗𝒂𝒍 𝑹𝒂𝒕𝒆 𝟏/𝒂 𝒑 𝟒 𝟐
• Utilization : u = = 𝟏 = = = = 𝟎. 𝟔𝟔𝟔𝟔𝟕
𝑺𝒆𝒓𝒗𝒊𝒄𝒆 𝑹𝒂𝒕𝒆 (𝑪𝒂𝒑𝒂𝒄𝒊𝒕𝒚) 𝒎×𝒑 𝒎×𝒂 𝟑×𝟐 𝟑
26
Example: Online Retailer (cont.)
• Average Waiting Time:
 u 2( m 1)  1   CV 2  CV 2 
 
Tq      
p a p 
m 1 u   2 
   
 2(3 1) 1   2
 2  4
 4   0.66667  1 ( 0.5)      1.42939    1.25   1.19 minutes
   
3 1  0.66667  2  3  2 
   

• Average Flow Time (Time to go through the system):

T = Waiting time + Service time = Tq + p = 1.19 min + 4 min. = 5.19 min.

• To find the the average number of customers or emailS waiting before being served,
we need the famous Little’s formula.
27
 Applying Little’s Law

To Calculate All Other Performance Measures

28
 Little’s Formula for Queueing Models
• Recall for a Process: Inventory = Flow Rate x Flow Time : I = R x T
 p   u
2( m 1) 1   CVa2  CV p2 
Tq        
• Tq = The average waiting time in the queue  m   1  u 



2 

• T = Flow Time = The average time in the system (queue + service) = Flow Time
T = waiting time + service time = Tq + p
• Iq = The average number of customers waiting in the queue (Inventory in the queue)
• I = Inventory = The average number of customers in the system (queue + service)
𝒑
I = Iq + Ip where Ip =
𝒂
𝒑
• Average no. of customers in the service: Ip =
𝒂
𝟏
• Flow Rate = Arrival Rate: R =
𝒂
• Little’s Formula. System: I = R × T (waiting + in service)

Queue: Iq = 𝑹 × Tq (waiting only)

29
 From Previous Example : Online Retailler
Customers send emails to a help desk of an online retailer every 2 minutes (30 emails per hour), on average, and the
standard deviation of the inter-arrival time is also 2 minutes. The online retailer has three employees answering
emails. It takes on average 4 minutes (15 emails per hour) to write a response email. The standard deviation of the
service times is 2 minutes. This is a multi-server queue with 3 servers (m=3) or G/G/3 queue or M/G/3 queue

Question: How many emails would there be, on average, that have been submitted to the online
retailer but not yet answered?

Summary of results from previous calculations:

• Average interarrival time: a = 2 minutes per customer

• Average service time: p = 4 minutes per customer
• Number of servers: m=3
𝟐
• Utilization: u = = 0.666667
𝟑
• Average waiting time: Tq = 1.19 minutes
• Average flow time: T = 5.19 minutes
30
 Solution:
• Average number of customers (emails) waiting to be read:
𝟏 𝟏
• Arrival rate: R = = = 0.5 emails per minute ( = 30 emails per hour)
𝒂 𝟐 𝒄𝒖𝒔𝒕𝒐𝒎𝒆𝒓𝒔/𝒎𝒊𝒏

• Iq = 𝑹 × Tq = 0.5 emails/min × 1.19 min. = 0.6 emails

• Average number of emails in the system (waiting + being read):

I = 𝑹 × T = 0.5 emails/min. × 5.19 min = 2.6 emails; or another way,

𝒑 𝟒
• The average number of emails being read (in service): Ip = = = 2 emails
𝒂 𝟐

I = Iq + Ip = 0.6 + 2 = 2.6 emails (2 emails are being answered and 0.6 emails waiting)

• So, on average, 2.6 emails have been submitted to the online retailer but not yet answered.

31
Generating Staffing Plan for Multi-server Queues

32
Economic Implications: Staffing Decision Example
It is usually busier around lunch time. During this time of the day, customers send emails to the
help desk of an online retailer every 1.5 minutes. This means a = 1.5 minutes per email or the
arrival rate R = (1/a) = 40 emails per hour, on average. The standard deviation of the inter-
arrival time 𝜎𝑎 is also 2 minutes. It takes on average 4 minutes to write a response email. This
means p = 4 minutes per email or the service rate (1/p) = 15 emails per hour. The standard
deviation of the service times 𝜎𝑝 is 2 minutes.
Question: How many employees would be needed so that the average wait time is x minutes
or less (say, 2 minutes or less)? Here 2 minutes is called the target response time.
Solution:
Arrival: Service:
a = 1.5 minutes p = 4 minutes
𝝈𝒂 = 2 minutes 𝝈𝒑 = 2 minutes
CVa = 2 ÷ 1.5 = 1.33333 CVa = 2 ÷ 4 = 2

33
Staffing Decision Example
• Solution:
• Notice that we need a minimum of 3 employees to answer the emails. Why?
p 4
Ip    2.66667
a 1.5
2
• Ip = 2.66667 means that on average there are 2 email being answered.
3
2
• It also means that on average 2 help-desk workers are busy answering emails. So we
3
need at least 3 employees.
• Also for the queue to be stable, average input rate < average service capacity, or
utilizations < 100%.

• Input rate = 40 emails/hour; Service capacity = m × (1/p) = 3 × 15 = 45 emails/hour

𝒑 𝟒 𝒊𝒏𝒑𝒖𝒕 𝒓𝒂𝒕𝒆 𝟒𝟎
• Utilizations: u = = = 0.888889 or u = = = 0.8888889 < 1.
𝒎×𝒂 𝟑×𝟏.𝟓 𝒔𝒆𝒓𝒗𝒊𝒄𝒆 𝒄𝒂𝒑𝒂𝒄𝒊𝒕𝒚 𝟒𝟓

34
 Staffing Decision Example – cont.
• So, we’ll start with m = 3, and calculate Tq . If m = 3 does not meet the requirement, then next,
try m = 4, etc. until the requirement is met, i.e. Tq ≤ 2 minutes.
• These calculations can be easily automated using spread sheet by inputting the formula for Tq.
Use a, p, m , 𝜎𝑎 and 𝜎𝑝 as input parameters.
𝑝 𝑢 2 𝑚+1 − 1 𝐶𝑉𝑎2 +𝐶𝑉𝑝2 4 0.888889 2 3+1 − 1 1.333332 + 22
Tq = × × = × ×
𝑚 1−𝑢 2 3 1−0.888889 2

= 1.3333 × 7.2563 × 1.0139 = 9.8094 minutes.

• With m = 3, the waiting time Tq = 9.8 minutes. So 3 CSRs are NOT sufficient.
• Next we try m = 4 CSRs and repeat the calculations above.
• Using m = 4, we get Tq = 1.27 minutes. So, 4 CSRs is sufficient to meet the target waiting time.

• Notice that the waiting time Tq drops drastically by just adding one more server (CSR).

35
 Cost Analysis of Staffing Plan
At a Call Center in Wisconsin, between 8:00 am – 8:15 am time slot, calls arrive on
average every 11.39 seconds (a = 11.39 seconds per call) and follows an exponential
𝝈 𝟏𝟏.𝟑𝟗
distribution (meaning: standard deviation of ≈11.39 seconds; CVa = 𝒂 = = 1 ). It
𝒂 𝟏𝟏.𝟑𝟗
take on average 90 seconds to answer a call (p = 90 seconds per call) with standard
𝝈𝒑 𝟏𝟐𝟎
deviation of 120 seconds : CVp = = = 1.3333
𝒑 𝟗𝟎
There are 10 Customer Service Representatives working at this time slot. This is a
multi-server queue with 10 servers (m = 10) or G/G/10 queue; more specifically,
M/G/10 queue.

𝟏 𝟏
• Flow (arrival) Rate = = = 0.0878 calls per second = 5.2678 calls per minute
𝒂 𝟏𝟏.𝟑𝟗
𝒑 𝟗𝟎
• Utilization: u = = = 𝟎. 𝟕𝟗
𝒎×𝒂 𝟏𝟎×𝟏𝟏.𝟑𝟗

36
Cost Analysis of Staffing Plan – cont.

• The Average Waiting Time:

 u 2( m 1)  1   CV 2  CV 2  90  0.79 2(10 1) 1   12  (1.3333) 2 

   
Tq       
p a p   24.98 seconds
   
  10     
m 1 u 2 1  0.79 2
       

• The Flow Time: T = Tq + p = 24.98 + 90 = 114.98 seconds (this is the total time
required by a call and is used in the calculations for the cost of line charges)

• The next slide shows the waiting times when the number of CSRs are varied
between 8 and 14.

37
Determining the Number of Customer Service Representatives

• As the number of servers (CSRs) increases, the utilities decreases and so are the waiting times.
• With only 8 CSRs (utilization of 99%), the average waiting time is around 20 minutes.
• But by just adding one more CSR (9 CSRs), the waiting time drops drastically to just over one minutes.
• If the call center would like to limit the average waiting time for the callers to be less than 30
seconds, then 10 CSRs is sufficient to meet this target. In fact, it is actually slightly better; the
average waiting time is 25 seconds.
38
 More Cost Analysis of Staffing Plan
• Average salary per CSR = $10 per hour = $0.1666 per minute (paid whether busy or not)
• Variable Cost for 1-800 number = $0.05 per minute, based on T = Tq + p.
Sample calculations for m = 10:
𝟏 𝟏 𝟔𝟎 𝒔𝒆𝒄
• The flow rate = arrival rate = = × = 5.2678 calls per minute.
𝒂 𝟏𝟏.𝟑𝟗 𝒔𝒆𝒄/𝒄𝒂𝒍𝒍 𝒎𝒊𝒏
𝑻𝒐𝒕𝒂𝒍 𝑾𝒂𝒈𝒆𝒔 𝒑𝒆𝒓 𝒎𝒊𝒏𝒖𝒕𝒆 𝟏𝟎×$𝟎.𝟏𝟔𝟔𝟔
• Direct Labor Cost = = = $0.3164 per call
𝑭𝒍𝒐𝒘 𝒓𝒂𝒕𝒆 𝒑𝒆𝒓 𝒎𝒊𝒏𝒖𝒕𝒆 𝟓.𝟐𝟔𝟕𝟖
𝟏𝟏𝟒.𝟗𝟖
• Line Charges per call = T × $0.05 = minutes × $0.05 = $0.0958
𝟔𝟎
• The Total Cost per call = $0.3164 + $0.0958 = $0.4122
• Repeat the same calculations on spreadsheet and the next table shows the results for
m = 8, 9, …, 14.

39
More Cost Analysis of Staffing Plan

Notice, the lowest Total Cost per Call is when using 10 CSRs.

40
 Pooling of Servers
to Reduce Waiting Time and to
Increase Queueing Performance

41
Managerial Responses to Variability: Capacity Pooling
Two Independent Queues: Two (M/M/1 queue)
12 cust./hr

24 cust./hr

12 cust./hr

Two independent queues (McDonald’s)

• Processing time per server: p = 4 minutes per customer.
• Total arrival rate = 24 customers per hour
• Arrival rate to each queue = 12 customers per hour
• Inter-arrival time to each queue: a = 5 minutes (at each server)
• Utilization = 4/5 = 80%
• m = 1, CVa = CVp = 1
• Average waiting time per customer: Tq = 16 minutes
42
Pooled Server
Pooled Servers: One M/M/2 queues

24 cust/hour

Pooled servers: one queue, 2 servers

• Processing time per server: p = 4 minutes per server
• Total arrival rate = 24 customers per hour
• Inter-arrival time: a = 2.5 minutes
• Utilization = 4/(2× 𝟐.5) = 80%
• m=2, CVa = CVp = 1
• Average waiting time per customer: Tq = 7.24 minutes

43
 Another Pooling Example: Which system is more effective?
Pooled system:
a = 35 seconds Across these three types of systems:
One queue, four m=4
servers • Variability is the same: CVa = CVp =
1
Partially pooled • Total demand (arrival) is the same:
For each of the 2
system: 1/35 customers per second.
queues:
Two queues, two a = 35 x 2 = 70 sec.
servers in each m=2 • Activity time is the same: p = 120
queue. seconds
• Utilization is the same: p / (a x m)
Independent multi
= 85.7%
queue system: For each of the 4 • The probability a server is busy
queues:
Four queues, one (utilization) is the same = 0.857
a = 35 x 4 = 140 sec.
server with each m=1
queue.
44
Pooling Reduces the Waiting Time in Queue Considerably
a = 35, p = 120 (per server) , m = 4, CVa = 1, CVp = 1
Waiting Time: Tq = 150 seconds
Pooled
Flow Time: T = 270 seconds
system
The number of customers in service:
Ip = p/a = 120/35 = 3.43 customers

For each of the two queues:

Partially a = 70, p = 120 (per server), m = 2, CVa = 1, CVp = 1
pooled Waiting Time: Tq = 336 seconds
system Flow Time: T = 456 seconds
The number of customers in service:
Ip = 2×(p/a) = 2(120/70) = 3.43 customers

For each of the four queues:

Separate a = 140, p = 120 (per server), m = 1, CVa = 1, CVp = 1
queue Waiting Time: Tq = 720 seconds
system Flow Time: T = 840 seconds
The number of customers in service:
Ip = 4×( p/a) = 4 (120/140) = 3.43 customers
45
Pooling Reduces the Number of Customers Waiting
The number of customers in service: Ip = 3.43 customers for all 3 systems

Pooled
Iq = 4.3 Iq = 4.3
Ip = 3.4 Ip = r = 3.4
system
Inventories I = Iq + Ip = 4.3 + 3.4 = 7.7
for each
queue
Partially within the Iq = 9.6 ( 2 times the inventory in each queue )
pooled Iq = 4.8 system Ip = 3.4
system Ip = 1.7 I = Iq + Ip = 9.6 + 3.4 = 13.0
Inventories
for the total
in the system

Iq = 20.4
Separate Ip = 3.4 ( 4 times the inventory in each queue )
queue Iq = 5.1 I = Iq + Ip = 20.4 + 3.4 = 23.8
system Ip = 0.86

46
 How Does Pooling Work?
= A customer waiting = A customer in service

Pooled Separate
system queue
system

• Both systems currently have 6 customers

• In the pooled system, all servers are busy and only 2 customers are waiting
• In the separate queue system, 2 servers are idle and 4 customers are waiting
• The separate queue system is inefficient because there can be customers
waiting and idle servers at the same time.
47
 Benefits of Operations Pooling
• Reduce customer waiting without adding extra staffing (servers)
• Reduce the number of staffing (servers) while maintaining the same responsiveness
70.00
Waiting
Time Tq m=1
60.00

50.00

40.00

m=2
30.00

20.00
m=5

10.00
m=10

0.00
60% 65% 70% 75% 80% 85% 90% 95%

Utilization u

How pooling can reduce waiting time 48

 Limitations to Pooling
• Pooling may require workers to have a broader set of skills, which may require
more training and higher wages.
• At the surgical centers that specializes on cardiac surgery and kidney transplant,
pooling means the cardiac surgeons need to be skilled at kidney transplants as well.

• Pooling may disrupt the customer – server relationship:

• Patients like to see the same physician.
• Clients would like to stick with the same lawyer who is familiar with their cases

• Pooling may increase the time-in-queue for one customer class at the expense of
another:
• Removing priority security screening for first-class passengers may decrease the
average waiting time-in-queue for all passengers but will likely increase it for first-class
passengers.

49
Balancing Efficiency and Responsiveness
Responsiveness

High System improvement

Increase staff (e.g. pooling of resources)
(lower utilization)

Responsive process with

high costs

Reduce staff
Now (higher utilization)

Low cost process with low

responsiveness

Low Frontier reflecting

current process

High per Low per Efficiency

unit costs unit costs
(low utilization) (high utilization)

Balancing efficiency with responsiveness

50
Managing Waiting Line Systems: Points to Remember

• Variability is the norm, not the exception

• Even when a process is demand constrained (utilization is less

than 100%), waiting time for service can be substantial due to
variability in the arrival and/or service processes. The larger the
variability, the longer the waiting time.

• Waiting times tend to increase dramatically as the utilization of a

process approaches 100%.

51
 Points to Remember
• Pooling multiple queues can reduce the waiting time in the queue with the
same amount of labor (or use less labor to achieve the same time-in-queue).

• Use the Waiting Time Formula to

• get a quantitative and qualitative feeling of the system
• analyze specific recommendations / scenarios

• Managerial response to variability:

• understand where it comes from and eliminate what you can
• accommodate the rest by holding excess capacity

52
Why the long queues for the women’s restroom?
• Traditionally, equal space allocation for
men’s and women’s restrooms
• But men’s bathroom has more capacity
(urinals do not take much space)
• Service time (p) is longer for women than
men
• Number of Women waiting in Line
(Little’s Law):
𝟏
Iq = × 𝑻𝒒
𝒂

 2(m  1) 1   CV 2  CV 2 
 p  u  a p 
Time in queue       
m 1-u 2 
   
p  Service time
p
u  Utilization 
am
 Potty-Parity Laws
Women’s Restroom Equity Bill:

• Passed NY City Counsel May 2005.

• Requires women’s restrooms to have twice the flushing

capacity of men’s restrooms (at least a 2:1 ratio of women’s
toilets to men’s toilets & urinals)

• In the context of waiting time in the queue (Tq), this law

requires to increase m for women, which decreases the
capacity factor (p/m), and decreases utilization.

54
 Potty-Parity Laws
• Other solutions:

• Unisex restrooms (pool the capacity, or at least the toilets).

• Example: Whole Food store at Times Square NYC – both
men and women form one waiting line for one of the
identical available toilets
• Flexible partitions that can change the size of each restroom
• At the tool booths on turn pikes, capacity (or the number of
tool booths) can be adjusted by making the middle booths
able to serves both directions of the traffic.

55