YTL-Samsung LTE BTS Cluster Test Procedure - v1.4

YTLC-SAMSUNG LTE Field BTS
Cluster Testing Procedure

\
Revision History
EDITION
DATE OF ISSUE
1.3
22. April, 2015
AUTHOR
REMARKS
JH Kim
JW Park
1.4
22. June, 2015
SAMSUNG Electronics Co., Ltd.
JH Kim
In Car Loss Table
Confidential and Proprietary Information
CONTENTS
Revision History
CONTENTS
Overview 4
CHAPTER 1. Cell Edge Test
12
1.1. Cell Edge User Throughput (Single User Basis) ....................................................................12
1.2
1.3
1.1.1
DL Throughput ........................................................................................................................... 12
1.1.2
UL Throughput ........................................................................................................................... 14
Network Entry Test ....................................................................................................................16

1.2.1
MS Initial Network Entry Success Rate .................................................................................... 16
1.2.2
MS Initial Network Entry Time ................................................................................................... 18
Stationary Test...........................................................................................................................20
1.3.1
DL MCS Throughput at SINR >= 0dB ...................................................................................... 20
1.3.2
DL MCS Throughput at SINR >= 5dB ...................................................................................... 22
1.3.3
DL MCS Throughput at SINR >= 15dB .................................................................................... 23
1.3.4
1.3.5
1.3.6
UL MCS Throughput at RSRP >= -125dBm ............................................................................ 27
1.3.7
1.3.8
1.3.9
UL MCS Throughput at RSRP >= -85dBm .............................................................................. 32
CHAPTER 2. Mobility Test
34
2.1. Mobility Test Coverage ..........................................................................................................34

2.1.1
RSRP ......................................................................................................................................... 34
2.1.2
SINR ........................................................................................................................................... 36
2.2. Drive Test Data Traffic............................................................................................................38

2.2.1
MS Initial Network Success Rate.............................................................................................. 38
YTLC-SAMSUNG LTE Networks Field KPI Test Results
2.2.2
MS Initial Network Entry Time ................................................................................................... 40
2.2.5
Call Drop Rate(nGBR)............................................................................................................... 42
2.2.7
Cell Reselection LTE to 3G ....................................................................................................... 44
2.2.8
Block Error Rate(BLER) ............................................................................................................ 44
2.2.9
HOSR(Intra-Frequency+Inter Frequency) ............................................................................... 45
2.2.10
Handover Latency (nGBR)........................................................................................................ 47
2.2.12
Test Data on nGBR QCI DL Throughput............................................................................... 49
2.2.13
Test Data on nGBR QCI UL Throughput............................................................................... 51
2.2.14
IMS Registration Time ............................................................................................................... 52
2.3. Mobility Test(VoLTE) .................................................................................................................53

2.3.1
VoLTE Call Setup Success Rate(GBR).................................................................................... 53
2.3.2
VoLTE Call Setup Time(GBR)................................................................................................... 56
2.3.3
VoLTE Call Drop Rate(GBR)..................................................................................................... 58
2.3.4
CSFB Call Setup Success rate ................................................................................................. 59
2.3.5
CSFB Call Setup Time .............................................................................................................. 59
2.3.6
VoLTE Handover Success Rate (GBR) ................................................................................... 60
2.3.8
VoLTE Handover Success Rate(SRVCC) ............................................................................... 62
2.3.9
VoLTE Handover Latency (GBR).............................................................................................. 62
2.3.10
Jitter for VoLTE (GBR) ............................................................................................................... 64
2.3.11
Packet Loss................................................................................................................................ 66
2.4. Mobily Test (Ping & Network Entry) ......................................................................................67

2.4.2
Ping with Packet Size of 32 Bytes nGBR.............................................................................. 67
2.4.4
PING with Packet Size of 1500 Bytes nGBR........................................................................ 68
CHAPTER 3. MOS Mobility Test
71
3.1. MobilityTest(VoLTE MOS) .........................................................................................................71

3.1.1
VoLTE Mean Opinion Score (POLQA) ..................................................................................... 71
CHAPTER 4. Appendix
73
4.1. Legend Setting ..........................................................................................................................73

4.4.1
Legend Value: TBD ................................................................................................................... 73
Overview
These tests cover a broad range of network performance characteristics
including latency, handoff, throughput, capacity, VoLTE with MOS and
reliability.
This YTLC-SAMSUNG BTS cluster test procedure provides a detail
description of each test and a brief summary of the test results.
The BTS cluster tests were conducted at drive route and fixed Point that
YTLC and SAMSUNG agreed upon mutually. All test results were extracted
from the field data comparing with the coverage on the basis of cell plan
simulation.
KPI of BTS cluster test Item

Items
1
Test Name
Stationary Test
Test Items
Bound
Acceptance Criteria
Cell Edge Test & Stationary

Test
1.1
Cell Edge User Throughput
cell edge tests only for pilot cluster
(Single User basis)

1.1.1
DL Throughput Test
>=
DL RSRP @UL 256kbps, SINR > -1dB

2.3 GHz >=1900kbps, 2.6 GHz
>=4000kbps
(only for pilot cluster)
1.1.2
UL Throughput Test
1.2
Network Entry Test
1.2.1
MS Initial Network Entry
>=
256kbps(only for pilot cluster)
>=
99(only for pilot cluster)
=<
1000ms(only for pilot cluster)
Success Rate
1.2.2
MS Initial Network Entry Time
1.3
Stationary Test
1.3.1
DL MCS
only for pilot cluster
DL MCS Throughput at SINR
>=
>= 0dB
1.3.2
DL MCS
(only for pilot cluster stationary)
>=
>= 5dB
1.3.3
DL MCS
DL MCS
>=
DL MCS
>=
UL MCS
24 Mbps (FC1), 47 Mbps (FC2)

>=
>= 30dB
1.3.6
12 Mbps (FC1),24 Mbps (FC2)

>= 25dB
1.3.5
4.8 Mbps (FC1),9 Mbps (FC2)

>= 15dB
1.3.4
2 Mbps (FC1),3 Mbps (FC2)

DL RSRP >= -125dBm
>=
88 kbps (FC1), 128 kbps (FC2)

1.3.7
UL MCS
DL RSRP >= -115dBm
>=
700 kbps (FC1), 1.5 Mbps (FC2)

1.3.8
UL MCS
DL RSRP >= -100dBm
>=

1.3.9
UL MCS
DL RSRP >= -85dBm
>=

Mobility Test
Mobility Test
2.1
Mobility Test - Coverage
2.1.1
RSRP
>=
In car conditions 90% of 256kbps UL

RSRP value will be Outdoor RSRP tested value in car with mutual agreement.
2.1.2
SINR
>=
(2.6GHz)- 90% >= -1dB , 85% >= 2dB

- Overshooting : 10%
Condition: SINR<2dB
equal or less than 10% of the measure
coverage area whereRSRP (outdoor
RSRP +3dB - tested value in car)
(2.3GHz), Same as 2.6GHz
2.2
Drive Test (Data Traffic)
2.2.1
>=
99
=<
800
=<
Success Rate
2.2.2

Time
2.2.5
Call Drop Rate (nGBR)
2.2.7
Cell Reselection LTE to 3G
2.2.8
Block Error Rate (BLER)
=<
2.2.9
HOSR (Intra-
>=
99
=<
40
TBD
Frequency+Inter Frequency)
2.2.11
Handover Latency (nGBR)
2.2.12
Data on nGBR QCI DL
(2.6GHz)Min. DL T.P 4Mbps > 90%
Throughput
DL Ave. T.P with Mobility >= 19.0 Mbps,

(2.3GHz, BW10)Min. DL T.P 1.9Mbps >
90% DL Ave. T.P with Mobility >=
10.5Mbps
2.2.13
Data on nGBR QCI UL
(2.6GHz)Min. UL T.P 512kbps > 90%
Throughput
UL Ave. T.P with Mobility >= 4.5 Mbps

(2.3GHz, BW10)Min. UL T.P 512kbps >
90%, UL Ave. T.P with Mobility >=
2.0Mbps
2.2.14
IMS Registration Time
Report only.
1 sample per cluster
2.3
Mobility Test (VoLTE)
2.3.1
VoLTE Call Setup Success
>=
99
Rate (GBR)
2.3.2
VoLTE Call Setup Time (GBR)
=<
2.3.3
VoLTE Call Drop Rate (GBR)
=<
2.3.4
CSFB Call Setup Success rate
TBD
2.3.5
CSFB Call Setup Time
TBD
2.3.6
VoLTE HOSR (Intra-Frequency
>=
99
+Intra-Frequency)
2.3.8
VoLTE HOSR (SRVCC)
TBD
2.3.9
VoLTE Handover Latency
Duplicated with Test Item 2.2.11
(GBR)
-> Report Only
2.3.10
Jitter for Volte (GBR)
Report Only.
2.3.11
Packet Loss
Report Only
A mutual understanding and
agreements as report only test item.
2.4
Mobility Test (Ping &

Network Entry)
2.4.2
Ping with Packet Size of 32

Bytes - nGBR
2.4.4
Round Trip Time RTT (ms)
=<
45
Packet Loss
=<
Round Trip Time RTT (ms)
=<
75
Packet Loss
=<
Ping with Packet Size of 1500

Bytes - nGBR
MOS Mobility
VoLTE MOS Test
Acceptance Criteria
Test
3.1
Mobility Test (VoLTE MOS)
3.1.1
VoLTE Mean Opinion Score

(POLQA)
>=
POLQA WB MOS >=
3.6 with AMR-
WB, 12.65 kbps

POLQA WB MOS >=
3.0 with AMR-
WB, 6.6 kbps as reference value.
Cell Edge User Throughput Criteria

Morphology
Download
Dense Urban
Urban
2.3 GHz >=1900kbps

2.6 GHz >=4000kbps
2.3 GHz >=1900kbps
2.6 GHz >=4000kbps
Upload
>= 256Kbps
>= 256Kbps
2.3 GHz >=1900kbps

Suburban
2.6 GHz >=4000kbps
>= 256Kbps
2.3 GHz >=1900kbps

Rural
2.6 GHz >=4000kbps
>= 256Kbps
Table 1-1 Cell Edge user Throughput criteria each morphology
Morphology
RSRP Criteria (The target indicated in Appendix D)
Dense Urban
Must be higher than 90% of RSRP -101.4dBm - In car loss
Urban
Suburban
Rural
Table 1-2 RSRP criteria each morphology (Base on 2.3G_2T2R)

Morphology
RSRP Criteria (The target indicated in Appendix D)
Dense Urban
Must be higher than 90% of RSRP -97dBm - In car loss
Urban
Suburban
Rural
Table 1-3 RSRP criteria each morphology(Base on 2.6G_4T4R)
Test
UE Band
Turn
Setting
DL Data/UL
Data
1st
Fixed(2.3)
VoLTE/Ping
2nd
Auto
DL Data/UL
Data
1st
Fixed(2.3/2.6)
VoLTE/Ping
2nd
Auto
DL Data/UL
Data
1st
Fixed(2.6)
VoLTE/Ping
2nd
Auto
DL Data/UL
Data
1st
Fixed(2.3/2.6)
VoLTE/Ping
2nd
Auto
DL Data/UL
Data
1st
Fixed(2.3/2.6)
VoLTE/Ping
2nd
Auto
DL Data/UL
Data
1st
Fixed(2.3/2.6)
VoLTE/Ping
2nd
Auto
DL Data/UL
Data
1st
Fixed(2.3/2.6)
VoLTE/Ping
2nd
Auto
Case
Cluster Configuration
Test Item
Interleave (2.3G)
Interleave(2.3G)+ Overlay (2.6G)
Overlay(2.6G)
Interleave(2.3G)/Interleave(2.3G) +
Overlay(2.6G)
Interleave(2.3G)/Overlay(2.6G)
Interleave(2.3G)/Interleave(2.3G)
+ Overlay(2.6G)/Overlay(2.6G)
Interleave(2.3G) +
Overlay(2.6G)/Overlay(2.6G)
Number of UE
Table 1-4 Cluster Configurations
UE
Source
Test Item
Test Method
RSRP/SINR
Block Error Rate (BLER)
HOSR (Intra-Frequency)
HOSR (Inter-Frequency)
UE1
Data(DL)

Data on nGBR QCI DL
Long Call
Throughput
DL MCS Throughput at SINR >= 0dB

HOSR (Intra-Frequency)
HOSR (Inter-Frequency)
UE2
Data(UL)
Data on nGBR QCI UL Throughput
Long Call
UL RSRP >= -125dBm

UL RSRP >= -115dBm
UL RSRP >= -100dBm
UL RSRP >= -85dBm
MS Initial Network Entry Success Rate
VoLTE Call Setup Success Rate (GBR)
VoLTE Call Setup Time (GBR)
VoLTE Call Drop Rate (GBR)
UE3/
UE4
VoLTE
VoLTE HOSR(Intra Frequency) UE3/4
Traffic Time/60Sec
VoLTE HOSR(Inter Frequency) UE3/4

VoLTE Handover Latency(GBR) UE3/4
Jitter for VoLTE(GBR)
IMS Registration Time
Packet Loss
MOS_UE3/4
10
Ping 32Bytes/1500Byte
Ping 32Bytes/1500Byte
Round Trip Time RTT (ms)-32 Bytes - nGBR

UE5
Ping
Packet Loss-32 Bytes - nGBR
Ping 32Bytes
Traffic Time/20 Sec
Ping 1500Bytes
Round Trip Time RTT (ms) 1500 Bytes nGBR
(Including header)
Packet Loss 1500 Bytes - nGBR
Traffic Time/20 Sec
Table 1-5 Scenario base on UE
In car loss
Loss
RSRP
8dB
(Use the In car loss for the GAP of outdoor RSRP and in car tested environment)
Table 1-6 In car loss
11
CHAPTER 1. Cell Edge Test

1.1.
Cell Edge User Throughput (Single User Basis)
1.1.1 DL Throughput
1.1.1.1 Purpose
1) To verify cell edge single user download throughput according to each morphology
2) Cell edge tests only for pilot cluster
1.1.1.2 Test Conditions and Requirements

1) 1 laptop computer and 1 device
2) 1 DM or X-CAL Auto
3) 1 GPS
4) Define the morphology type of the cluster prior to start test.
5) Morphology type will be mutually agreed by YTLC and Samsung.
6) Test will be conducted at 10 covered area locations per pilot cluster
A.
Cell Edge RSRP Condition according to morphology and in car loss

i.
Refer to Table 1-2, 1-3, 1-6
7) System Environment
A.
Test Environment
12
1.1.1.3 Test Procedure

1) Define 10 locations prior to start test.
2) Locate a UE inside vehicle [In-Car condition]
3) After checking RSRP of user device, test mobile will set up the call.( RSRP : @threshold
3dB)
4) UE downloads a big file from FTP or UDP server for 2 minutes. (The file size should be
bigger than 3Gbytes.)
5) Save the download throughput logs with RF condition and GPS information per location by
DM tool
6) Summarize test result.
1.1.1.4 Pass/Fail Criteria

1) Definition: Average download throughput for 2 minutes
KPI
Criteria
Threshold RSRP @UL 256kbps/ SINR > -1dB
2.3 GHz >=1900kbps

2.6 GHz >=4000kbps
2) Result Conditions
Average MAC Throughput values across all samples.
Unloaded network system.
Without other system interference
13
1.1.2 UL Throughput
1.1.2.1 Purpose
1) To verify cell edge single user upload throughput each morphology
2) Cell edge tests only for pilot cluster

3) 1 GPS
A.
Cell Edge RSRP Condition according to morphology and in car loss

i.
Refer to Table 1-2, 1-3
B.
Test Environment

2) Locate a UE inside vehicle [In-Car condition] or Indoor location
7) After checking RSRP of user device, test mobile will set up the call. ( RSRP @threshold
3dB)
3) UE uploads a big file from FTP or UDP server for 2 minute. (The file size should be bigger
14
than 3 Gbytes.)
4) Save the upload throughput logs with RF condition and GPS information per location by
DM tool.

1) Definition: Average upload throughput for 2 minutes
2) Measured cell edge throughput must be higher or the same with the value from link budget
corresponding to the morphology type
3) 10 results must be fulfilling below criteria
KPI
Criteria
UL Cell Edge Throughput
>=256kbps
15
1.2 Network Entry Test

1.2.1 MS Initial Network Entry Success Rate
1.2.1.1 Purpose
1) To verify network accessibility in cell edge according to each morphology
2) This test is to be performed at Pilot Cluster only

3) 1 GPS
A.
Cell Edge RSRP Condition according to morphology.

i.
Refer to Table 1-2, 1-3
A.
Test Environment
B.
Signaling Message definition
16

2) Locate a dongle inside vehicle [In-Car condition]
8) After checking RSRP of user device (RSRP @threshold 3dB), test mobile will set up the
call.
3) Establish a call with test UE at Cell edge location and Terminate the call after it is
successfully established
4) Repeat the test calls until more than 30 samples are obtained from same location in the
cluster

1) Message Definition : Attach Request to Attach Complete (NAS) on Uplink Information
Transfer
2) Success Rate % : [The number of Attach successes / The number of Attach attempts] *
100
100% of 30 consecutive successes per each sector.
17
Attempt : Attach Request
Success : UE sends Attach Complete (NAS) on Uplink Information Transfer

KPI
Criteria
>=99%
1.2.2 MS Initial Network Entry Time

1.2.2.1 Purpose
1) To verify network accessibility entry time in cell edge according to each morphology

3) 1 GPS
7) Cell Edge RSRP Condition according to morphology..
A.
Test Environment
C.
Signaling Message :
18

2) After checking RSRP of user device, test mobile will set up the call. (RSRP@ threshold
3dB)
3) Attach the UE to the network and detach until 30 consecutive success

Transfer
2) The number of Attach Setup Success within 1000ms/ The number of Attach Setup
Success
3) MS Initial Network Entry Time= 100% of 30 consecutive success per each sector.
19
KPI
Criteria
=<1000ms
Median values across all samples.
Stationary test in YTLC's cell-edge conditions.
HSS is close to P-GW with HSS access time < 10ms.
Authentication latency < 100ms.
NAS security message exchange latency < 100ms
MME processing time < 5ms.
Un-congested backhaul with latency < 2ms.
IP address is allocated in ATTACH Accept message.
Abnormal latency cause by transmission, Samsung need to provide evidence to

YTLCC. Abnormal latency should be excluded based on mutual agreed by
YTLCC
and Samsung.
1.3 Stationary Test

1.3.1 DL MCS Throughput at SINR >= 0dB
1.3.1.1 Purpose
1) To verify single user downlink throughput according to RF condition(0dB=< SINR =< 5dB)

3) 1 GPS
4) The test point should be RF conditions (0dB=< SINR =< 5dB).
A.
Test Environment
20

1) Locate the UE at a predefined place where 0dB=< SINR =< 5dB for stationary test
2) UE setup new connection with eNB.
3) Connect FTP server and start downloading predefined file of bigger than 3Gbytes size
for 2 minutes

1) Definition : More than 2Mbps/3Mbps(Average for 2 minutes)
KPI
Criteria
2 Mbps (FC1)
3 Mbps (FC2)
FC 1: 2.3GHz 10MHz config1-7, FC2 : 2.6GHz 20MHz config1-7
21

1.3.2.1 Purpose
1) To verify single user downlink throughput according to RF condition( 5=<SINR=<10dB)

3) 1 DM or X-CAL
Auto
4) 1 GPS
5) The test point should be RF conditions (5=<SINR=<10dB)
A.
Test Environment

1) Locate the UE at a predefined place where 5=<SINR=<10dB for stationary test
for 2 minutes
22

1) Definition : More than 4.8Mbps/9Mbps(Average for 2mimutes)
KPI
Criteria
4.8Mbps (FC1)
9 Mbps (FC2)

1.3.3.1 Purpose
1) To verify single user downlink throughput according to RF condition(15dB=<SINR=<20dB )

3) 1 GPS
4) The test point should be RF conditions (15dB=<SINR=<20dB)
5) Test Environment
23

1) Locate the UE at a predefined place where 15dB=<SINR=<20dB for stationary test
for 2 minutes

1) Definition : More than 12Mbps/24Mbps(Average for 2mimutes)
KPI
Criteria
12Mbps (FC1)
24Mbps (FC2)

1.3.4.1 Purpose
1) To verify single user downlink throughput according to RF condition(SINR>=25dB)
24

2) 1 DM or X-CAP Auto
3) 1 GPS
4) The test point should be RF conditions (30dB>= SINR >=25dB)..
5) Test Environment

1) Locate the UE at a predefined place where SINR is >= 25dB (25dB~30dB) for stationary
test
for 2 minutes

KPI
Criteria
24Mbps (FC1)
47Mbps (FC2)
25

1.3.5.1 Purpose
1) To verify single user downlink throughput according to RF condition(27~ 30dB)

3) 1 GPS
4) The test point should be RF conditions (SINR >=30dB).
5) Test Environment

1) Locate the UE at a predefined place where SINR is >= 27~ 30dB (If Applicable) for
stationary test
26

for 2 minutes
4) Summarize test result

KPI
Criteria
28Mbps (FC1)
60Mbps (FC2)
1.3.6 UL MCS Throughput at RSRP >= -125dBm

1.3.6.1 Purpose
1) To verify single user uplink throughput according to RF condition (-125dBm=< RSRP =< 120dBm)

3) 1 GPS
4) The test point should be RF conditions (-125dBm=< RSRP =< -120dBm).
5) Test Environment
27

1) Locate the UE at a predefined place where -125dBm=< RSRP =< -120dBm for stationary
test
3) Connect FTP server and start uploading predefined file of bigger than 1Gbytes size for 2
minutes

1) Definition : Physical Throughput : More than 88kbps/128kbps(Average for 2mimutes)
KPI
Criteria
UL MCS Throughput at RSRP >= -125dBm
88 kbps (FC1)
128 kbps (FC2)
FC1 : 2.3GHz 10MHz config1-7, UpPTS+1OFDM symbol puncturing

FC2 : 2.6GHz 20MHz config1- 7
28

1) To verify single user uplink throughput according to RF condition (-115dBm=< RSRP =<110dBm)
1.3.7.2 Test Procedure Conditions and Requirements

1) Test 1 laptop computer and 1 device
3) 1 GPS
4) The test point should be RF conditions (-115dBm=< RSRP =<-110dBm)
5) Test Environment

1) Locate the UE at a predefined place where -115dBm=< RSRP =<-110dBm for stationary
test.
29
minutes

1) Definition : More than 700kbps/1.5Mbps(Average for 2mimutes)
KPI
Criteria
700 kbps (FC1)

1.5 Mbps (FC2)

FC2 : 2.6GHz 20MHz config1-7

1.3.8.1 Purpose
1) To verify single user uplink throughput according to RF condition (-100dBm=< RSRP =< 95dBm)

3) 1 GPS
4) The test point should be RF conditions (-100dBm=< RSRP =< -95dBm)
A.
Test Environment
30

1) Locate the UE at a predefined place where -100dBm=< RSRP =< -95dBm
minutes

1) Definition : More than 5Mbps/12Mbps(Average for 2 minutes)
KPI
Criteria
5 Mbps (FC1)
12 Mbps (FC2))

FC2 : 2.6GHz 20MHz config1-7
31

1) To verify single user uplink throughput according to RF condition( RSRP >= -85dBm)

3) 1 GPS
4) The test point should be RF conditions( RSRP >= -85dBm)
5) Test Environment

1) Locate the UE at a predefined place where RSRP >=-85dBm UE setup new connection
with eNB.
minutes
32

1) Definition :More than 6Mbps/17Mbps(Average for 2mimutes)
KPI
Criteria
6 Mbps (FC1)
17 Mbps (FC2)

FC2 : 2.6GHz 20MHz config1- 7
33
CHAPTER 2. Mobility Test

2.1.
Mobility Test Coverage
2.1.1 RSRP
2.1.1.1 Purpose
1) To Verify radio coverage at predefined drive route which is service coverage of the cluster

3) Test Vehicle, Power inverter, GPS
4) Samsung will prepare drive test route and obtain YTLCs approval. (Drive route will be
mutually agreed and include multi cells.)
5) Test scenario is obtained by UE1(Refer to Table 1-5)
6) Test Environment
A.
Test Environment
34

1) The test device will be located at the starting test position.
2) Measure the RF environment (RSRP) from the UE1 of DL throughput scenario on the
predefined drive route in the service area.

1) Definition : 90 % >= UL Throughput 256K
KPI
Value
RSRP
Must be higher than 90% of RSRP which meet the UL Throughput 256K
(90 % >= UL Throughput 256K)
Bin size is 20m x 20m(Area Binning)
Calculation : 90% of RSRP which meet the UL Throughput 256K is calculated

based on binning
RSRP as per link budget design according to morphology provided by Vendor's

specification
Exclusion Bin is defined as follows

I.
In case of device RSRP does not exist
II.
Out of coverage
Only samples within the simulated coverage will be concerned
In car conditions 90% of 256kbps UL

RSRP value will be Outdoor RSRP - tested value in car with mutual agreement
35
2.1.2 SINR
2.1.2.1 Purpose
1) To Verify radio coverage at predefined drive route which is service coverage of the cluster

4) Samsung will prepare drive test route and obtain YTLCs approval.(Drive route will be
mutually agreed and include multi cells)
A.
Test Environment

1) The test device will be located at the starting test position.
2) Measure the RF environment (SINR) from the UE1 of DL throughput scenario on the
predefined drive route in the service area.
36

1) Definition : 90% >= -1dB , 85% >= 2dB @2.3GHz, 2.6GHz
KPI
Value
SINR
90% >= -1dB , 85% >= 2dB @2.3GHz, 2.6GHz,
Overshooting
10%, SINR <= 2dB@RSRP is more than (outdoor RSRP +3dB in car loss)
Calculation : SINR is calculated based on binning
RSRP as per link budget design according to morphology provided by Vendor's

specification

I.
II.
Out of coverage
RSRP : Measure coverage area where RSRP>=(outdoor RSRP +3dB- in car loss)
37
2.2.
Drive Test Data Traffic
2.2.1 MS Initial Network Success Rate
2.2.1.1 Purpose
1) To verify network accessibility Success Rate until predefined route is fully covered

5) Test scenario is obtained by UE3, UE5(Refer to Table 1-5)
6) Test Environment
7) Signaling Message :
38

2) After checking RSRP of user device, test mobile will set up the call.
3) Start drive test around the pre-defined drive route at 30-40 Kmph continuing the attach
the UE to the network and detach according to call scenario
4) Repeat the test calls until predefined route is fully covered
5) At the end of the drive route stop the logging and ensure that the log files are saved

Transfer
2) Success Rate % : [The number of Attach successes within 800ms / The number of Attach
attempts] * 100
39
KPI
Value

Success Rate
>=99%
2.2.2 MS Initial Network Entry Time
2.2.2.1 Purpose
1) To verify network accessibility entry time until predefined route is fully covered

A.
Test Environment
40

3) Start drive test around the pre-defined drive route at 30-40 Kmph continuing the
Attach the UE to the network and detach according to call scenario with UE3, UE5

5) At the end of the drive route stop the logging and ensure that the log files are
saved

Transfer
2) The number of Attach Setup Success / The number of Attach Setup Success
3) MS Initial Network Entry Time =< 800ms
41
KPI
Value
MS Initial Network Entry time
=<800ms
2.2.5 Call Drop Rate(nGBR)
2.2.5.1 Purpose
1) To verify that dropped sessions are kept to a minimum until predefined route is fully covered

5) Test scenario is obtained by UE1, UE2,(Refer to Table 1-5)
A.
Test Environment
42

3) Start drive test around the pre-defined drive route at 30-40 Kmph continuing the Attach
the UE to the network and detach according to call scenario
5) At the end of the drive route stop the logging and ensure that the log files are saved

1) Call Drop rate Definition : Rate of Abnormal Release of Data Call
A.
Ratio of Dropped abnormal Release of Data Call such as below

Abnormal Release : UE is released without rrcConnectionRelease message
i.
2) Formula : Calculate the Call drop Rate = The number of Abnormal Release for DL DATA,
UL DATA) / (Summation of the number of Calls Attempts for DL DATA, UL DATA) x100
3) No. of call drops/ No. call attempts %,
A.
The number of Call Attempts for UE1(DL Data) : Roundup(Sum(MAC DL Bytes(DL

DATA))/ 30MBytes)
B.
The number of Call Attempts for UE2(UL Data) : Roundup(Sum(MAC UL Bytes(UL

DATA))/ 5MBytes)
KPI
Value
Call Drop Rate
=< 1%
43
2.2.7 Cell Reselection LTE to 3G

TBD
2.2.8 Block Error Rate(BLER)
2.2.8.1 Purpose
1) To verify that BLER is kept to a minimum until predefined route is fully covered

A.
Test Environment
44

generate continuous DL traffic from UE 1 according to call scenario
4) Measure the downlink BLER which is an error rate of data transmission of PDSCH after
all HARQ procedures from UE1(DL DATA)
saved

1) BLER Definition : Error rate of data transmission of PDSCH after all HARQ procedures
2) BLER is calculated by DL DATA UE
KPI
Value
Residual BLER
=< 1 %
2.2.9 HOSR(Intra-Frequency+Inter Frequency)
2.2.9.1 Purpose
1) To verify that is to measure the handover success rate within the covered area

45

5) Test scenario is obtained by UE1, UE2. UE5(Refer to Table 1-5)
A.
Test Environment
Iperf/FTP
servers
EPC
eNB monitoring
tool
Test
UEs
`
Pre-determined route
Intra eNB
HO point
B.
eNB
Iperf/ftp
clients
eNB
Inter eNB
HO point
`
Intra eNB
HO point
Intra eNB
HO point
Test equipment
in vehicle
Call Flow

generate continuous DL traffic, Ping from UE 1 , UE2, UE5 according to call scenario
save
46

1) The number of Handover Success / The number of Handover Attempts * 100
2) Latency
Definition:rrcConnectionReconfiguration(HOCommand)torrcConnectionReconfigurationC
omplete (HO Complete).
3) Formula:rrcConnectionReconfiguration(HOCommand)torrcConnectionReconfigurationCo
mplete (HO Complete)
KPI
Value
>= 99 %
HOSR
2.2.10 Handover Latency (nGBR)
2.2.10.1
Purpose
1) To verify that is to measure the handover Latency within the covered area
2.2.10.2
Test Conditions and Requirements

5) Test scenario is obtained by UE1, UE2, UE5(Refer to Table 1-5)
A.
Test Environment
47
Iperf/FTP
servers
EPC
eNB monitoring
tool
Test
UEs
`
Intra eNB
HO point
B.
2.2.10.3
eNB
Iperf/ftp
clients
eNB
Inter eNB
HO point
`
Intra eNB
HO point
Intra eNB
HO point
Test equipment
in vehicle
Call Flow
Test Procedure

generate continuous DL traffic, Ping from UE 1 , UE2, UE5 according to call scenario
save
2.2.10.4
Pass/Fail Criteria
1) LatencyDefinition:rrcConnectionReconfiguration(HOCommand)torrcConnectionReconfig
48
urationComplete (HO Complete).

2) Formula :rrcConnectionReconfiguration(HOCommand)torrcConnectionReconfigurationC
omplete (HO Complete).
KPI
Value
Handover Latency
=< 40 ms
UE's cell switching time is less than 20 ms (3GPP TS 36.133).
ALL HO latencies are measured using X-Cal Analyzer.
2.2.12 Test Data on nGBR QCI DL Throughput
2.2.12.1 Purpose
1) Verify user throughput at predefined drive route

A.
Test Environment
49

generate continuous DL traffic from UE 1 according to call scenario
4) At the end of the drive route stop the logging and ensure that the log files are save
2.2.12.4 Pass Fail Criteria

1) Criteria
KPI
Value
Minimum DL data rate(2.6GHz)
4Mbps > 90%
DL Average Throughput with Mobility(2.6GHz)
>= 19.0 Mbps
Minimum DL data rate(2.3GHz)
1.9Mbps > 90%
DL Average Throughput with Mobility(2.3GHz)
>= 10.5Mbps

III. In case of device RSRP does not exist
IV. In case of Simulation RSRP does not exist (Out of coverage)
50
2.2.13 Test Data on nGBR QCI UL Throughput
2.2.13.1 Purpose
1) Verify user throughput at predefined drive route

A.
Test Environment
51

generate continuous UL traffic from UE 2 according to call scenario
2.2.13.4 Pass Fail Criteria

1) Criteria
KPI
Value
Minimum UL data rate(2.6GHz)
512kbps > 90%
UL Average Throughput with Mobility(2.6GHz)

Minimum UL data rate(2.3GHz)
>= 4.5 Mbps

512kbps > 90%
UL Average Throughput with Mobility(2.3GHz)
>= 2.0Mbps

V.
VI. In case of Simulation RSRP does not exist (Out of coverage)
2.2.14 IMS Registration Time

TBD
52
2.3.
Mobility Test(VoLTE)
2.3.1 VoLTE Call Setup Success Rate(GBR)
2.3.1.1
Purpose
1) To verify that is to measure VoLTE Call Setup Success Rate within the covered area

A.
Test Environment

A.
Test Environment
53
B.
Signal Flow
54

generate VoLTE Call between Mobile Originated(MO) and Mobile Termination(MT)
according to call scenario
save

1) VoLTE Call Setup Definition: From "SIP INVITE" message transmission (by originating UE)
to "180 RINGING" message reception (by originating UE).
2) Formula: The number of
VoLTE Setup Successes / The number of VoLTE Setup
Attempts
A.
VoLTE Setup Success : Receiving 180 RINGING" message reception by originating

UE
B.
VoLTE Setup Attempt : Sending SIP INVITE message

KPI
Value
>=99 %

I.
II.
Out of coverage
Un-congested backhaul with RTT between eNB and P-GW <= 2 ms
UE is connected to the Home IMS
IMS server is located close to the EPC.
55
2.3.2 VoLTE Call Setup Time(GBR)

2.3.2.1 Purpose
2) To Verify that is to measure VoLTE Call Setup Time within the covered area

A.
Test Environment

56
2.6.13.3
1)
Pass/Fail Criteria
VoLTE Call Setup Time Definition: From "SIP INVITE" message transmission (by
originating UE) to "180 RINGING" message reception (by originating UE).

2) Formula :Time between SIP INVITE message and 180 Ringing(by originating UE)
KPI
Value
>=99 %

IMS server is located close to the EPC
57
2.3.3 VoLTE Call Drop Rate(GBR)

2.3.3.1 Purpose
1) To Verify that is to measure VoLTE Call Drop rate within the covered area

A.
Test Environment

save
58

1) Call
Drop
rate
Definition
Abnormal
Release
UE
is
released
without
rrcConnectionReleasemsg.
A.
Ratio of Dropped abnormal Release of Data Call such as below

Abnormal Release : UE is released without rrcConnectionRelease message
ii.
2) Formula : Calculate the Call drop Rate = The number of Abnormal Release for VoLTE) /
(Summation of the number of Calls Attempts for VoLTE Call) x100
3) No. of call drops/ No. call attempts %,
A.
The number of Call Attempts for UE3(VoLTE) : The number of Call Attempts for : VoS
Call Count
KPI
Value
Call Drop Rate
=<1 %
2.3.4 CSFB Call Setup Success rate
2.3.5 CSFB Call Setup Time
59
2.3.6 VoLTE Handover Success Rate (GBR)

2.3.6.1 Purpose
1) To verify that is to measure the handover Success Rate(GBR) within the covered area

A. Test Environment
Iperf/FTP
servers
EPC
eNB monitoring
tool
Test
UEs
`
eNB
Intra eNB
HO point
B.
Iperf/ftp
clients
eNB
Inter eNB
HO point
`
Intra eNB
HO point
Intra eNB
HO point
Test equipment
in vehicle
Call Flow
60

save

1) Time between rrcConnectionReconfigurationComplete and rrcConnectionReconfiguration
(contains HO command)
KPI
Value
Handover Success Rate
>= 99 %

V.
VI. In case of Simulation RSRP does not exist (Out of coverage)
Un-loaded network conditions.
UE's cell switching time is less than 20 ms (3GPP TS 36.133
H/O latencies are measured using X-Cal Analyzer.
61
2.3.8 VoLTE Handover Success Rate(SRVCC)

TBD
2.3.9 VoLTE Handover Latency (GBR)

2.3.9.1 Purpose
2) To verify that is to measure the handover Latency(GBR) within the covered area

C. Test Environment
Iperf/FTP
servers
EPC
eNB monitoring
tool
Test
UEs
`
Intra eNB
HO point
eNB
Iperf/ftp
clients
eNB
Inter eNB
HO point
`
Intra eNB
HO point
Intra eNB
HO point
Test equipment
in vehicle
62
D.
Call Flow

save

2) Time between rrcConnectionReconfigurationComplete and rrcConnectionReconfiguration
(contains HO command)
KPI
Value
Handover Latency
=< 40 ms

VII. In case of device RSRP does not exist
VIII. In case of Simulation RSRP does not exist (Out of coverage)
63
UE's cell switching time is less than 20 ms (3GPP TS 36.133)
H/O latencies are measured using X-Cal Analyzer.
2.3.10 Jitter for VoLTE (GBR)

2.3.10.1
Purpose
1) To verify that is to measure the Packet delay variation(Jitter_GBR) within the covered
area
2.3.10.2

A.
Test Environment
64

save
6) Using the VoLTE Call scenario, Calculate the Average jitter value

1) Statistical variation of the RTP data packet inter-arrival time.
Value
KPI
Report Only
Jitter for Volte (GBR)

2) Jitter is measured in timestamp units.

IX. In case of device RSRP does not exist
X.
In case of Simulation RSRP does not exist (Out of coverage)
65
2.3.11 Packet Loss
2.3.11.1
Purpose
1) To verify that is to measure the Packet loss(GBR) within the covered area
2.3.11.2

5) Test scenario is obtained by UE3 (Refer to Table 1-5)
A.
2.4.11.3
Test Environment
Test Procedure

66
5) Using the VoLTE Call scenario, Calculate the Average Packet Loss
2.4.11.4
Pass/Fail Criteria
1) RTP Packet Loss is counted based RTP packet sequence number.

KPI
Value
Packet Loss
Report Only

XI. In case of device RSRP does not exist
XII. In case of Simulation RSRP does not exist (Out of coverage)
2.4.
Mobily Test (Ping & Network Entry)
2.4.2 Ping with Packet Size of 32 Bytes nGBR

2.4.2.1 Purpose
1) To verify that the network meets the guaranteed latency requirements of low latency
service flows on both upstream and downstream and Packet Los

67

generate Ping with Packet Size of 32 Bytes

1) RTT : "PING REQ" message transmission (by UE) to "PING RSP" message reception (by
UE).
2) Packet Loss : The number of Lost Packets / The number of Transmitted Packets

XIII. In case of device RSRP does not exist
XIV. In case of Simulation RSRP does not exist (Out of coverage)
BH latency is excluded and no fragmentation

KPI
Value
Round Trip Time RTT
=<45 ms
Packet Loss
=<1 %
2.4.4 PING with Packet Size of 1500 Bytes nGBR

2.4.4.1 Purpose
2) To verify that the network meets the guaranteed latency requirements of low latency
service flows on both upstream and downstream and Packet Los
68

A.
Test Environment

generate Ping with Packet Size of 1500 Bytes
save

1)
RTT : "PING REQ" message transmission (by UE) to "PING RSP" message reception (by
UE).
69
2) Packet Loss : The number of Lost Packets / The number of Transmitted Packets
KPI
Value
Round Trip Time RTT
=<75 ms
Packet Loss
=<1 %

I.
II.
In case of Simulation RSRP does not exist (Out of coverage)
Un-loaded network conditions
70
CHAPTER 3. MOS Mobility Test

3.1.
MobilityTest(VoLTE MOS)
3.1.1 VoLTE Mean Opinion Score (POLQA)

3.1.1.1 Purpose
1) Purpose of this test is to verify the MOS for Volte under the mobility environment

A.
Test Environment

71
5) Calculate the MOS results for MO and MT separately based on M2M test.

1) POLQA WB MOS >= 3.6 with AMR-WB, 12.65 kbps
2) POLQA WB MOS >= 3.0 with AMR-WB, 6.6 kbps as reference value.
Value
KPI
POLQA WB MOS
AMR-WB, 12.65 kbps
POLQA WB MOS
AMR-WB, 6.6 kbps
>= 3.6
>= 3.0

Un-loaded network conditions
When the feature of AMR-WB 6.6kbps is implemented, this value will be

discussed based on the tested value.
Abnormal latency cause by transmission, Samsung need to provide evidence to

YTLCC. Abnormal latency should be excluded based on mutual agreed by
YTLCC and Samsung.
72
CHAPTER 4. Appendix
4.1.
Legend Setting
4.4.1 Legend Value: TBD
4.4.1.1 RSRP
4.4.1.2 RSRQ
4.4.1.3 SINR
4.4.1.4 Tx Power UL
4.4.1.5 MAC DL Throughput
4.4.1.6 MAC UL Throughput
4.4.1.7 PDSCH Throughput
4.4.1.8 PUSCH Throughput
4.4.1.9 RANK Index
73
2010 Samsung Electronics Co., Ltd.

All rights reserved.
Information in this document is proprietary to SAMSUNG Electronics Co., Ltd.
No information contained here may be copied, translated, transcribed or
duplicated by any form without the prior written consent of SAMSUNG.
Information in this document is subject to change without notice.
74

YTL-Samsung LTE BTS Cluster Test Procedure - v1.4

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

YTL-Samsung LTE BTS Cluster Test Procedure - v1.4

Uploaded by

Copyright:

Available Formats

YTLC-SAMSUNG LTE Field BTS

Cluster Testing Procedure

22. April, 2015

22. June, 2015

SAMSUNG Electronics Co., Ltd.

In Car Loss Table

Confidential and Proprietary Information

1.1. Cell Edge User Throughput (Single User Basis) ....................................................................12

Network Entry Test ....................................................................................................................16

MS Initial Network Entry Success Rate .................................................................................... 16

MS Initial Network Entry Time ................................................................................................... 18

DL MCS Throughput at SINR >= 0dB ...................................................................................... 20

DL MCS Throughput at SINR >= 5dB ...................................................................................... 22

DL MCS Throughput at SINR >= 15dB .................................................................................... 23

DL MCS Throughput at SINR >= 25dB .................................................................................... 24

DL MCS Throughput at SINR >= 30dB .................................................................................... 26

UL MCS Throughput at RSRP >= -125dBm ............................................................................ 27

UL MCS Throughput at RSRP >= -115dBm ............................................................................ 29

UL MCS Throughput at RSRP >= -100dBm ............................................................................ 30

UL MCS Throughput at RSRP >= -85dBm .............................................................................. 32

CHAPTER 2. Mobility Test

2.1. Mobility Test Coverage ..........................................................................................................34

2.2. Drive Test Data Traffic............................................................................................................38

MS Initial Network Success Rate.............................................................................................. 38

SAMSUNG Electronics Co., Ltd.

Confidential and Proprietary Information

YTLC-SAMSUNG LTE Networks Field KPI Test Results

MS Initial Network Entry Time ................................................................................................... 40

Call Drop Rate(nGBR)............................................................................................................... 42

Cell Reselection LTE to 3G ....................................................................................................... 44

Block Error Rate(BLER) ............................................................................................................ 44

HOSR(Intra-Frequency+Inter Frequency) ............................................................................... 45

Handover Latency (nGBR)........................................................................................................ 47

Test Data on nGBR QCI DL Throughput............................................................................... 49

Test Data on nGBR QCI UL Throughput............................................................................... 51

IMS Registration Time ............................................................................................................... 52

2.3. Mobility Test(VoLTE) .................................................................................................................53

VoLTE Call Setup Success Rate(GBR).................................................................................... 53

VoLTE Call Setup Time(GBR)................................................................................................... 56

VoLTE Call Drop Rate(GBR)..................................................................................................... 58

CSFB Call Setup Success rate ................................................................................................. 59

CSFB Call Setup Time .............................................................................................................. 59

VoLTE Handover Success Rate (GBR) ................................................................................... 60

VoLTE Handover Success Rate(SRVCC) ............................................................................... 62

VoLTE Handover Latency (GBR).............................................................................................. 62

Jitter for VoLTE (GBR) ............................................................................................................... 64

2.4. Mobily Test (Ping & Network Entry) ......................................................................................67

Ping with Packet Size of 32 Bytes nGBR.............................................................................. 67

PING with Packet Size of 1500 Bytes nGBR........................................................................ 68

CHAPTER 3. MOS Mobility Test

3.1. MobilityTest(VoLTE MOS) .........................................................................................................71

VoLTE Mean Opinion Score (POLQA) ..................................................................................... 71

4.1. Legend Setting ..........................................................................................................................73

Legend Value: TBD ................................................................................................................... 73

SAMSUNG Electronics Co., Ltd.

Confidential and Proprietary Information

SAMSUNG Electronics Co., Ltd.

Confidential and Proprietary Information

YTLC-SAMSUNG LTE Networks Field KPI Test Results

KPI of BTS cluster test Item

Cell Edge Test & Stationary

Cell Edge User Throughput

cell edge tests only for pilot cluster