HUAWEI UMTS KPI

KPI & QoE

KPI Measurement Methodology
HUAWEI UMTS KPI architecture
HUAWEI UMTS KPI (RAN)

KPI & QoE

KPIKey Performance Indicator
UMTS RAN KPI examples
Coverage
Voice Originate access ratio
PS service upload and download
average throughput
SHO success ratio
InterRAT CS HO success ratio
RTT
MOS
......

CN

KPI examples
CS
Paging success ratio
SMS success ratio
......
PS
Attach success ratio
SGSN inter/intra route update
success ratio
......

QoEQuality of Experience perception from customer in the service

Calling can not reach to ..

Speed of Download in FTP is fast
Fluent in video service
......

KPI & QoE

End-User

Dont care any KPI ,but the good and abundant service

ISP

Our target is to meet the glowing need of customers, close to their

perception ,provide the excellent radio access network
The evaluation standard of network is subscribers perception; only
close to their demand ,we can find what we need to do

Problem

Unclear ,unquantifiable QoE user layer Network

KPI technical layer

KPI & QoE

QoE KPI
UMTS
TE

MT

RAN

CN
Gateway

CN
EDGE
NODE

TE

End-to-End Service

TE/MT Local
Bearer Service

External Bearer
Service

UMTS Bearer Service

Radio Access Bearer Service

Radio Bearer
Service

RAN Access
Bearer Service

Physical Radio
Bearer Service

Physical
Bearer Service

CN Bearer
Service

Backbone
Bearer Service

QoE KPI proceeding

UMTS QoS architecture in 23.107

about subscribers perception and close to their experience

This is a necessary condition for a good KPI system.

Care

KPI & QoE

KPI Measurement Methodology
HUAWEI UMTS KPI architecture
HUAWEI UMTS KPI (RAN)

KPI Measurement Methodology

DTDrive Test
Evaluate the overall performance of network after optimization.
Originate access ratio/Voice/VP/,Call drop ratio, SHO success ratio,
InterRAT CS/PS HO success ratio......

CQTCall Quality Test

Evaluate the capacity of network elements.
Voice/VP access ratio/call drop ratio PDP activation success
ratio/Delay Throughput MOS UE2UE/PSTN Voice/VP Call setup
delay/Ping delay......

Stat.Statistics
Evaluate the integrated quality of network.
SHO ratio UL/DL interference cell ratio RRC/RAB connection setup
ratio Voice/VP/PS Call drop ratio SHO success ratio InterRAT
CS/PS HO success ratio Worst cell ratio Access refusal
ratio Congestion cell ratio......

KPI Measurement Methodology

DT/CQT/Stat lifecycle
Network lifecycle
Contract

Network

Service

Installation

Preparation

Request for
KPIs Items and Criterias

Ongoing

Commercial Launch

Acceptance
Site Coverage
Cluster Performance
End-to-end Quality
SHO Ratio
Pilot Pollution Ratio

Drive & Call Quality Test in All Network

Statistics Data in UTRAN and CN element

Network Expansion

Ongoing Optimization

KPI & QoE

KPI Measurement Methodology
HUAWEI UMTS KPI architecture
HUAWEI UMTS KPI (RAN)

HUAWEI UMTS KPI architecture

Up to the end of Aug 2006,HUAWEI have got 33 UMTS
commercial contract in the word.

HK-Sunday/PCCW
UAE-Etisalat
New Zealand-Econet Wireless New Zealand
Bulgaria-Globul
Portugal-Optimus
Poland-P4
Czech-Vodafone
Ireland-SMART
...

HUAWEI UMTS KPI architecture have succeed in tens of biding

and acceptance HUAWEI UMTS KPI architecture is going
improving,the main characteristic :

Over All KPI, CN RAN;

Different item level for different network performance standard ;
Include many item of E2E KPI , pay attention to QoE;
7 kinds of RAN KPI, integrated;
RAN DT/CQT/Stat. comprehensive;

> 10

HUAWEI UMTS KPI architecture

UMTS KPIs

Coverage

Accessability

Coverag
e Ratio

WCDMA
Availability

Soft
Handov
er
Ratio

Attach Success
Ratio

Pilot
Pollutio
n Ratio

Service Access
Success Ratio
(MO-Voice,
Video Phone,
SMS)
End-to-end
Access Success
Ratio (MTVoice, Video
Phone, SMS)

Retain ability

Call Drop
Ratio
(Voice,
Video
Phone)

Service integrity

Voice Quality
(MOS)

Mobility

Soft
Handover
Success
Ratio

Video Phone
(Synchronization
between voice
Hard
and image )
Handover
PDP
Context
Success
Service UL/DL
Drop Ratio
Ratio
BLER (Voice,
Video Phone)
Date
Inter-RAT
Transfer
Handover
PS Mean
Drop Ratio
Success
Throughput (RLC
Ratio

64K/128K/384K
/HSPA)

PDP Context
Activation
Success Ratio

> 11

Delay/Latency

Availability

Attach Setup
Time

Worst cell
ratio

PDP Context
Activation Time

Admission
refusal
ratio

Service Access
Time
End-to-end
Access Time
Packet Round Trip
Time (Initial/
Continuous Ping)
Transfer
Interruption Time

Congestion
cell ratio

HUAWEI UMTS KPI architecture

HUAWEI KPI feature (Common)
Full coverage all key index to the E2E/QoE , close to
customers experience , attentions to QoE
To every phase and scene ,HUAWEI KPI provide the special
KPI items to mapping.
Keep all KPI items flexible , comprehensive, integrated

> 12

HUAWEI UMTS KPI architecture

HUAWEI KPI feature (dedicated)
Customers oriented:
Customers every little success is our greatest success
According to different scale (e.g., amount of NodeB,
coverage area )and commercial strategy (e.g., service
application) of network ,HUAWEI KPIs are customized to fit
them

> 13

HUAWEI UMTS KPI architecture

HUAWEI KPI feature (dedicated)
Comprehensive DT/CQT item, complying with the industry rules
DT/CQT:
Coverage probability
SHO ratio
Pilot pollution ratio
RRC setup success ratio
Originate access ratio
Terminate access ratio
PDP activation success ratio

> 14

HUAWEI UMTS KPI architecture

HUAWEI KPI feature (dedicated)

Stat. indexes according to 3GPP protocol
according to 3gpp ,Huawei defines the abundant counters ,by which,
operators can customize stat. KPI with Huawei statistics tool, Nastar

e.g. :
a)
b)
c)
d)

e)
f)
g)

32.403 ,4.16 Attempted RAB establishments for PS domain

This measurement provides the number of RAB assignment attempts for PS domain. The measurement is
pegged by traffic class
CC
On receipt by the RNC of a RANAP RAB ASSIGNMENT REQUEST message for PS domain , each RAB
assignment request is added to the relevant measurement according to the traffic class requested. See TS
25.413 and TS 23.107
Four integer values
RAB.AttEstabPS.Conv
RAB.AttEstabPS.Strm
RAB.AttEstabPS.Intact
RAB.AttEstabPS.Bgrd
RNC Function
Valid for packet switched traffic
UMTS

> 15

HUAWEI UMTS KPI architecture

HUAWEI objectives for developping toolscomprehensive solution
Improve network plan
and optimization
efficiency
Save network operation
expenditure
Network
performance can be
Measurable .mon

master the trend of service

operation , configure network
rationally
simplify np , nb, m and op, to
improvewith solution by tools

monitor network
performance with multiple
data
Construct data system,
integrate monitor and
optimization analysis

Optimize whole network focus on care VIP

Care customers focus Monitor and optimize network quality in terms

of E-E
on services
Convert network optimization to performance
optimization, focus on customers experience
> 16

HUAWEI UMTS KPI architecture

HUAWEI KPI tools content
performance/confi
guration/CHR/trac
e/RTWP/
Engineering
parameter

Get test data

GENEX
Nastar
Share the
analysis
result

GENEX Probe

GENEX
Assistant

Agilent
Qualcomm
HUAWEI RNC

iManagerTM M2000

RAN autoNode B
configure
platform

RNC

Node B

Kinds of test data

HLR/AC
Node B

RNS

Radio test data

GENEX
Probe

RNC
te
data st

MSC/SSP/VLR
Node B

RNC

GENEX
U-Net
Network re-plan/plan

> 17

GMSC/SSP

CN
Node B
Node BRNS

PSTN
PLMN
ISDN

KPI & QoE

KPI Measurement Methodology
HUAWEI UMTS KPI architecture
HUAWEI UMTS KPI (RAN)

HUAWEI UMTS KPI (RAN)

RAN KPI include but not limited to
CLASSES

INDICATORS

SHO success ratio

Coverage probability

Intra frequency HHO success

ratio

Retain ability

Service integrity

Mobility

Inter-frequency HHO success

ratio

SHO expense

Inter-system HO success ratio

Pilot pollution ratio

......

UL interference cell ratio

Call setup delay

......

PDP activation delay

RRC setup success ratio for

registration

UE2UE user plane delay

RRC setup success ratio

Accessibility

INDICATORS

Call Drop ratio

SHO ratio
Coverage

CLASSES

Delay

UE2PSTN user plane delay

RAB setup success ratio

PING packet RTD delay

Originate access ratio

Service intermit delay

Terminate access ratio

Flow delay

PDP activation success ratio

......

......

Worst cell ratio

CS call drop ratio

Access refusal ratio

......

Availability

Congestion cell ratio

UL BLER worsening ratio

......

DL BLER worsening ratio

UL average throughput
DL average throughput
MOS
......

Remarks
1. KPI measurement methodology include Stat. or DT/CQT
2. Some KPI include sub-item based on service
3. Some KPI include sub-item based on originate call or
terminate call

HUAWEI UMTS KPI (RAN)

Coverage
Call drop ratio (DT)
Abnormal release num/ Call successful setup num *100

Coverage probability (DT)

(CPICH Ec/Io dB & the UL transmit power of planned whole
coverage services< dBm & the total test miles of reverse forward
RSCPdBm/ total miles of test route*100

SHO ratio(DT)
A2 A3 / ( A1 +A2+A3), In DT A1 indicates that the
quantity of cell in active set is 1, A2 indicates that the quantity of
cell in active set is 2, A3 indicates that the quantity of cell in active
set is 3

HUAWEI UMTS KPI (RAN)

SHO expense (Stat.)
{(ACTIVESET_1_UE_MEAN +
ACTIVESET_2_2SOFTER_UE_MEAN*2 +
ACTIVESET_2_2SOFT_UE_MEAN*2 +
ACTIVESET_3_3SOFTER_UE_MEAN*3 +
ACTIVESET_3_3SOFT_UE_MEAN*3 +
ACTIVESET_3_2SOFTER_UE_MEAN*3) /
(ACTIVESET_1_UE_MEAN + ACTIVESET_2_2SOFTER_UE_MEAN +
ACTIVESET_2_2SOFT_UE_MEAN +
ACTIVESET_3_3SOFTER_UE_MEAN +
ACTIVESET_3_3SOFT_UE_MEAN +
ACTIVESET_3_2SOFTER_UE_MEAN)} -1

HUAWEI UMTS KPI (RAN)

Pilot pollution ratio (DT)
The point of the number of the CPICH which can enter the Acitve set
more than 4 /total number of drive test data*100

UL interference cell ratio (Stat.)

The number of cells which RTWP in half an hour is higher than
-98dBm / total cell number*100

HUAWEI UMTS KPI (RAN)

Accessibility

Radio paging success ratio(Stat.)

The successful num of CN paging UE / total paging num of CN*100

HUAWEI UMTS KPI (RAN)

RRC setup success ratio (Stat.)
RRC connection setup success time / RRC connection request time
received by RNC*100

RAB setup success ratio (Stat.)

The number of 'RAB setup complete' in CS domain or in PS domain
received from UE/ Receive number of CS or PS RAB setup request
which is sent by CN*100

HUAWEI UMTS KPI (RAN)

Originate access ratio(CS)(DT&CQT)
Originate access num/Originate request num*100
Originate request numquantity of Service request message
Originate access numquantity of Alerting message

Terminate access ratio (CS)(DT&CQT)

Terminate access num/Terminate request num *100
Terminate request numquantity of Service request message
Terminate successful connection numquantity of Alerting message

PDP activation success ratio (DT&CQT)

PDP activation success num /PDP activation request num *100
PDP activation request numquantity of PDP Activation Request
message
PDP activation success numquantity of PDP Activation Accept
message

HUAWEI UMTS KPI (RAN)

Retain ability
CS call drop ratio (DT&CQT)
Service call drop num /Service access num *100
Service access numquantity of Alerting message
Service call drop numquantity of drop call

PS drop call ratio (DT&CQT)

PS service call drop num /PDP success activation num
PDP success activation num quantity of PDP Activation Accept
message
PS service call drop numquantity of PDP Context lost

HUAWEI UMTS KPI (RAN)

CS/PS call drop ratio (Stat.)
(CS/PS )call drop ratio= abnormal Release num of (CS/PS) service RNC
initiate / (CS/PS) service successfully setup num *100%

HUAWEI UMTS KPI (RAN)

Service Integrity
UL/DL BLER worsening ratio(DT&CQT)
The number of point which UL/DL BLER is higher than threshold
value / UL/DL total test point *100

MOS

(Mean Opinion Score)

(DT&CQT)

Using equipment to test the score of MOS

UL/DL average throughput(DT&CQT)

Average throughput statistic from application layer

HUAWEI UMTS KPI (RAN)

Mobility
SHO Success Ratio (DT)
SHO success ratio= SHO success num/SHO attempt num*100
SHO attempt num The number of message 'Active Set Update' RNC
receive
SHO success num The number of message 'Active Set Update
Complete' sent by UE

> 29

HUAWEI UMTS KPI (RAN)

SHO success ratio (Stat.)
SHO success ratio = SHO success num/SHO judge num*100
The quantity of message 'Active Set Update Complete' RNC receive /
The quantity of message 'Active Set Update' RNC send.

HUAWEI UMTS KPI (RAN)

Intra-Frequency HO Success Ratio (DT)
RNC side statistic the quantity of physical channel reconfiguration
used in same frequency HHO RNC receive /the quantity of physical
channel reconfiguration used in same frequency HHO RNC send.

Inter-Frequency HO Success Ratio (DT)

RNC side statistic the quantity of physical channel reconfiguration
used in inter-frequency HHO RNC receive /the quantity of physical
channel reconfiguration used in inter-frequency HHO RNC send.

Inter/Intra-Frequency HO Stat

> 31

HUAWEI UMTS KPI (RAN)

Inter-RAT HO Success Ratio (Stat./Voice)
Voice inter-system HO success ratio (handover out)= The quantity of
message 'Handover Complete' UE send /The quantity of message
'Handover from UTRAN Command' UE receive *100%.CS inter-system
handover out attempt number when RNC send RELOCATION
REQUIRED message to CN.
CS domain inter-system handover out success number In the period
of CS domain inter-system handover out ,when RNC receive IU
RELEASE COMMAND message and the reason of the message is
Successful Relocation or Normal Release

Inter-RAT HO Success Ratio(DT/Voice)

The quantity of message 'Handover to UTRAN Complete' UE send /The
number of message 'HO CMD (Handover to UTRAN Command)' UE
receive *100%.

> 32

HUAWEI UMTS KPI (RAN)

Inter-RAT HO Success Ratio (DT/PS)
PS inter-system HO(out)= The sum of PS inter-system HO out
complete / The sum of PS inter-system HO out attempt * 100%
The sum of PS inter-system HO out The sum of PS inter-system
Routing Area Update Complete.
The sum of PS inter-system HO out attempt the sum of UE receive
CELL CHANGE ORDER FROM UTRAN.

Inter-RAT HO Success Ratio (Stat./PS)

PS inter-system HO success ratio (handover out)=The quantity of PS
inter-system out route update complete message 'Routing Area Update
Complete' / The quantity of UE receive 'CELL CHANGE ORDER FROM
UTRAN' message*100%.
The quantity of PS inter-system out route update complete message
'Routing Area Update Complete'
The quantity of UE receive 'CELL CHANGE ORDER FROM UTRAN'
message.

Inter-RAT HO in DT/Stat. Voice/PS

......

HUAWEI UMTS KPI (RAN)

Delay/Latency
Signal Flow Delay (DT&CQT)
The test about the main and important signal flow e.g. any HO
flow is to seeing about the system response speed ability.
specialize
SHO link add
From UE sent measure reports about 1A event to RNC receive the
measure reports of Active Set Update Complete message.
CS inter-system HO
From RNC send the message of UE HANDOVER FROM UTRAN
COMMAND to UE to receive the IU RELEASE COMMAND message......

HUAWEI UMTS KPI (RAN)

Call setup delay CS (CQT)
Signal Statistic
Call setup delay= Average (T(CC_Alerting)-T(RRC Connection Request) )

Stopwatch Statistic
Call setup delay= Average( T(hear alerting)-T(dial up out number)

specialize
Voice(UE2UE) Voice(UE2PSTN) Voice(PSTN2UE)
VP alerting (UE2UE) VP see the image (UE2UE)
......

PDP activation delay (CQT)

Call setup delay = Average (T(Activate PDP Context Accept)-T(RRC
Connection Request) )

HUAWEI UMTS KPI (RAN)

Delay of service plane and Intermit delay of service plane
(DT&CQT)
Delay of service plane
Intermit delay of PS transmission about E2E

Intermit delay of service plane

Hard Handover The Intermit time of voice VP image PS on the user
plane when Inter-system handover.

PING packet RTD delay (CQT)

The time from send out ping to receive responses
specialize
Initial Ping/Ping Idle
Ping DCH (20Bytes/100Bytes/500Bytes...)
Ping FACH

HUAWEI UMTS KPI (RAN)

Availability
It is the mark of describing the whole system quality.
specialize

Worst cell ratio

Admission refusal ratio
Congestion cell ratio
......