Radiotherapy and Oncology 91 (2009) 455–460

Contents lists available at ScienceDirect

Radiotherapy and Oncology

journal homepage: www.thegreenjournal.com

Treatment planning

The delineation of target volumes for radiotherapy of lung cancer patients

Hilke Vorwerk a,*, Gabriele Beckmann b, Michael Bremer c, Maria Degen d, Barbara Dietl e, Rainer Fietkau f,
Tammo Gsänger a, Robert Michael Hermann a, Markus Karl Alfred Herrmann a, Ulrike Höller g,
Michael van Kampen h, Wolfgang Körber i, Burkhard Maier j, Thomas Martin k, Michael Metz l,
Ronald Richter m, Birgit Siekmeyer a, Martin Steder n, Daniela Wagner a, Clemens Friedrich Hess a,
Elisabeth Weiss a,o, Hans Christiansen a
a
Department of Radiotherapy and Radiooncology, University Hospital Göttingen, Robert-Koch-Straße, Germany
b
Department of Radiotherapy and Radiooncology, University Hospital Würzburg, Josef-Schneider-Straße, Germany
c
Department of Radiotherapy and Radiooncology, Medical School Hannover, Carl-Neubergstraße, Germany
d
Department of Pneumology, Lungenfachklinik, Waldhof Elgershausen, Greifenstein, Germany
e
Department of Radiotherapy and Radiooncology, University Hospital Regensburg, Franz-Josef-Strauß-Allee, Germany
f
Department of Radiotherapy and Radiooncology, University Hospital Erlangen, Universitätsstraße, Germany
g
Department of Radiotherapy and Radiooncology, Vivantes MVZ Neukölln, Rudower Straße, Berlin, Germany
h
Department of Radiotherapy and Radiooncology, Krankenhaus Nordwest, Steinbacher Hohl, Frankfurt, Germany
i
Department of Pneumology, Ev. Krankenhaus Weende, Pappelweg, Bovenden-Lenglern, Germany
j
Department of Radiotherapy and Radiooncology, Klinikum Bayreuth, Preuschwitzer Straße, Germany
k
Department of Radiotherapy and Radiooncology, Klinikum Bremen Mitte, St. Jürgen-Straße, Germany
l
Department of Haematooncology, University Hospital Göttingen, Robert-Koch-Straße, Germany
m
Department of Radiotherapy and Radiooncology, Klinikum St. Georg, Delitzscher Straße, Leipzig, Germany
n
Department of Haematooncology, Pius-Hospital, Georgstraße, Oldenburg, Germany
o
Department of Radiation Oncology, Virginia Commonwealth University, Richmond, VA, USA

a r t i c l e i n f o a b s t r a c t

Article history: Purpose: Differences in the delineation of the gross target volume (GTV) and planning target volume
Received 23 May 2008 (PTV) in patients with non-small-cell lung cancer are considerable. The focus of this work is on the anal-
Received in revised form 5 March 2009 ysis of observer-related reasons while controlling for other variables.
Accepted 7 March 2009
Methods: In three consecutive patients, eighteen physicians from fourteen different departments delin-
Available online 30 March 2009
eated the GTV and PTV in CT-slices using a detailed instruction for target delineation. Differences in
the volumes, the delineated anatomic lymph node compartments and differences in every delineated
Keywords:
pixel of the contoured volumes in the CT-slices (pixel-by-pixel-analysis) were evaluated for different
Radiation therapy
Lung cancer
groups: ten radiation oncologists from ten departments (ROs), four haematologic oncologists and chest
Interobserver variability physicians from four departments (HOs) and five radiation oncologists from one department (RO1D).
Delineation Results: Agreement (overlap P 70% of the contoured pixels) for the GTV and PTV delineation was found
Target volume in 16.3% and 23.7% (ROs), 30.4% and 38.6% (HOs) and 32.8% and 35.9% (RO1D), respectively.
Conclusion: A large interobserver variability in the PTV and much more in the GTV delineation were
observed in spite of a detailed instruction for delineation. The variability was smallest for group ROID
where due to repeated discussions and uniform teaching a better agreement was achieved.
Ó 2009 Elsevier Ireland Ltd. All rights reserved. Radiotherapy and Oncology 91 (2009) 455–460

Patients with unresectable or medically inoperable non-small-
cell lung cancer (NSCLC) are treated with primary radiochemother-
apy [1,2]. Although this is a curative approach, these patients still
have a poor prognosis with a curative potential of 10–20% for stage
IIIA/B. Published literature suggests that tumor control is improved
with dose escalation using conformal radiation therapy [3,4].
* Corresponding author. Address: Department of Radiotherapy and Radiooncol-
ogy, University Hospital Göttingen, Robert-Koch-Straße 40, 37099 Göttingen,
Germany. Tel.: +49 551 398866; fax: +49 551 396192.
E-mail address: h.vorwerk@med.uni-goettingen.de (H. Vorwerk).
Whether adjuvant chemotherapy is able to improve survival after
primary radiochemotherapy for inoperable NSCLC is still unclear.
To answer this question, the GILT-CRT-1-study (multicenter,
open-label, randomised, phase III study of oral Vinorelbine in com-
bination with Cisplatin concurrently with radiotherapy to a total
dose of 66 Gy) randomises to either two more cycles of consolida-
tion therapy with oral Vinorelbine and Cisplatin plus Best Support-
ive Care or Best Supportive Care alone in these patients with
unresectable locally advanced NSCLC. The comparability between
different participants of multicenter studies is compromised by
contouring errors which may cause inaccurate radiation treatments

0167-8140/$ - see front matter Ó 2009 Elsevier Ireland Ltd. All rights reserved.
doi:10.1016/j.radonc.2009.03.014
456 Interobserver variability for lung cancer
that result from different RT volumes. Contouring variations are
caused by observer-, image- and instruction-related reasons.
Within the GILT-study, we wanted to evaluate the agreement of
target volume delineation by participants of the study. The target
volume definition for conformal radiotherapy may be the most
important factor for adequate tumor coverage and sufficient
sparing of critical structures [5,6].
The aim of our analysis was to examine, whether we could min-
imise the variability in GTV and PTV delineation for primary non-
small-cell lung cancer patients by using a strict presetting for
GTV definition and for the determination of the PTV (observer-re-
lated reason) with simultaneous exclusion of image- and instruc-
tion-related reasons for testing quality assurance. Such a precise
delineation protocol to homogenize delineation was demanded
by different authors [5,7]. We analysed different groups of
physicians: radiation oncologist from different departments,
haematologic oncologists and chest physicians from different
departments and radiation oncologist from one department.
Methods and materials
Delineation was performed during a workshop of the partici-
pants of the GILT-CRT-1-study. The physicians, who agreed to par-
ticipate in the workshop, could obtain all patient informations
digitally in the forefront of the meeting. At the beginning of the
meeting all participants received a folder with all patient data to
make themselves familiar with the facts. Contouring was done in
the planning rooms of one department under survey of two phys-
icists. Interaction with any physician during contouring was not
allowed.
Eighteen physicians were asked to contour the GTV and PTV for
three patients with non-small-cell lung cancer stage IIIB scheduled
for primary radiation therapy. Contouring was done by experi-
enced physicians from fourteen different departments, divided into
three groups:
RO ten senior radiation oncologists (G.B., M.B., B.D., R.F., U.H.,
M.v.K., B.M., T.M., R.R., H.V.) from ten different departments
used to 3D planning.
HO four senior haematologic oncologists/and chest physi-
cians (M.D., W.K., M.S., M.M.) from four different departments
(further noticed as haematologic oncologists).
RO1D four senior and one experienced junior radiation oncol-
ogists (R.M.H., M.H., M.N., B.S., H.V.) from one department, all
familiar with 3D planning.
Patient data were acquired from the first three men, who were
treated in the GILT-CRT-1-study (Pierre Fabre Study Code: PM
0259 CA 304 J1; EudraCT number: 2004-005135-26) in the Uni-
versity of Göttingen, Germany. Before contouring every physician
was provided with written information on every patient, includ-
ing history, clinical examination, general performance status,
bronchoscopic findings, lung function tests and computed tomog-
raphy (CT) scans with iodine contrast enhancement. The CT-scans
were also available in digital form. All data were made
anonymous.
The median age of the patients was 60 years. All patients were
planned for curative radiochemotherapy in combination with Cis-
platin and Vinorelbine and a total radiation dose of 66.0 Gy
(5  2.0 Gy per week) with reduced field according to the study
protocol. The field was reduced after 50.0 Gy utilizing a new
restaging CT according to the study protocol. For our analysis we
used the first CT before beginning of the treatment and analysed
the contouring planned for the first part of the treatment up to
50.0 Gy.
The stage and location of the tumors were as follows:
1. cT3 cN3 M0 (left central, partial atelectasis of the lower left
lobe, central and upper mediastinal lymph nodes),
2. cT3 cN3 M0 (right central, central mediastinal lymph nodes),
3. cT2 cN3 M0 (right central, central mediastinal lymph nodes).
Continuous 5 mm CT-scans (10 mm in one patient) of the treat-
ment region in the thorax were obtained with a single-slice spiral
CT scanner (Somatom Balance, Siemens Medical Systems, Forch-
heim, Germany) for all three patients without contrast enhance-
ment. We used a resolution of 0.94 mm in axial plane, a couch
shift of 7.5 mm and a field-of-view of 480 mm (130 kV, 100 mAs).
Physicians entered their contours directly at the treatment-
planning console (Eclipse, Version 6.5, Varian Medical Systems,
Palo Alto, CA, USA). Radiation physicians not familiar with this
planning system and haematologic oncologists beforehand under-
went introductions in the management of the contouring work-
space by a physicist. Particularly, it was demonstrated how to
change the window level between soft tissue and lung window
and how to measure distances. Sagittal and coronar CT reconstruc-
tion were available to allow orientation in craniocaudal direction.
All physicians had permanent technical assistance during contour-
ing by two physicists (D.W., T.G.). All physicians worked on their
own CT image-set. Contouring was performed independently with-
out information of other participants. Two physicians from the
group ROs contoured only the CT from one patient.
All physicians obtained a detailed instruction, which is provided
by the GILT-CRT-1-study protocol, for definition of the gross target
volume (GTV) and planning target volume (PTV) as follows:
Gross tumor volume:
1. extension of the primary tumor in the soft tissue and lung
window,
2. lymph nodes larger than 1.5 cm in cross diameter in the trans-
versal CT-scans,
3. lymph nodes with central hypodensity,
4. lymph nodes larger than 1.0 cm in cross diameter in the trans-
versal CT-scans, if multiple nodes are seen in the same compart-
ment (defined by Naruke et al. [8]).
It was possible to contour different structures in one volume
(for example the GTV). The limits for the different window levels
could be chosen by the participants. The structures could be con-
toured by a paint brush or pixel-by-pixel.
Every physician received a detailed anatomic sagittal picture of
the lymph node compartments according to the Naruke Classifica-
tion [8] and described in more detail by Kirikuta et al. [9]. For eas-
ier distinction of the compartments we enhanced a transversal CT-
map created by Kirikuta et al. [9] with coloured flags for all lymph
node compartments.
Planning target volume:
1. PTV is defined as GTV plus a safety margin of 1.0 cm in lateral
direction, 1.0 cm in ventrodorsal direction and 1.5 cm in cranio-
caudal direction via automatic expansion (This considers both
oncological (subclinical extension [CTV]) and geometrical
(setup and motion uncertainty) factors).
2. If infracarinal involvement is suspected, caudal rim of the PTV
should be placed at least 3 cm below the carina.
It was possible to modify the PTV contour after automatic
expansion of the GTV.
We performed a volume analysis, an anatomic analysis and a
pixel-by-pixel analysis of all contoured parts of all CT-slices similar
to that explained in more detail below for different groups of
 H. Vorwerk et al. / Radiotherapy and Oncology 91 (2009) 455–460
physicians. In the first part a comparison of ten radiation oncolo-
gists (ROs) from ten different departments with four haematologic
oncologists (HOs) from four different departments were per-
formed. In the second part the ten radiation oncologists from the
first part were compared with five radiation oncologists from one
department (RO1D).
3D-volumes of the GTV and PTV were calculated by use of the
planning system EclipseÒ. The medians, standard deviations and
ratio of maximum to minimum volume (Vmax/Vmin) were calculated
for every patient and physician. Additionally, we assessed the devi-
ation from the mean volume, calculated from the fourteen differ-
ent departments, i.e. from RO’s and HO’s.
We analysed which anatomic sites of the lymph node compart-
ments were considered to be involved. A compartment was consid-
ered as contoured by a participant when at least a part of the
compartment was contoured. A checklist was created by us which
contained the following items: upper and lower paratracheal
lymph nodes left and right, preaortic lymph nodes, subaortic
lymph nodes (A-P window), hilar lymph nodes left and right, sub-
carinal lymph nodes, bronchopulmonal lymph nodes left and right,
paraesophageal lymph nodes (below carina). This checklist was
completed for every outlined GTV and analysed for agreement be-
tween the physicians. Agreement was stated, when 70% or more of
the observers included or excluded the lymph node compartment
in their contour of the structure.
In our clinic we developed a mapping tool in order to analyse
the concordance between the observers pixel-by-pixel. All con-
toured structures were exported from EclipseÒ with Dicom format,
were converted into ASCII format and imported in MicrosoftÒ Of-
fice Excel 2003. The structures were specified with different points
on the circumference of the structure. We composed a graticule
with one cubic centimetre equivalent to 100 pixel points. Every
pixel point of the contoured circumference from the structure
was plotted as a ‘‘1” into one table map of ExcelÒ per physician, pa-
tient and z-axis of the CT-scans. We developed a source code to
provide a linear regression between the pixel points on the circum-
ference from the contoured structure, like the planning system
EclipseÒ, and filled the area between the lines with the number
Fig. 1. Picture of a map from the pixel-by-pixel analysis of the contoured structures
by 13 physicians. All structures per physician were displayed by the number 1 for
all pixels in the contoured area and afterwards added for each pixel point. For better
visualization we coloured the different areas of numbers (The darker the colour the
higher the number). Hundred pixel points are equivalent to one cubic centimetre.
457
‘‘1” using an in house source code. Along this way we received a
map of the structures in all transversal CT-scans of all patients
and physicians, pixel-by-pixel. Afterwards we added up all maps
of each group, who should be investigated, and achieved a new
map with numbers equivalent to the number of physician, who
had contoured this pixel (Fig. 1). This analysis detected differences
in the 2D-transversal CT-slices for all slices. On this way a state-
ment for the 3D-volume was possible. Therewith we calculated
the agreement between the physician groups. Agreement was sta-
ted, if 70% or more of the observers included the pixels on the
transversal CT slice in their contour of the structure.
To identify the physicians’ level of agreement, encompassing
and common volumes were calculated for every patient. The
encompassing volumes are the smallest volumes integrating all en-
tered volumes of one patient. The common volumes are the largest
volumes that are part of all GTVs or PTVs of one patient [10–12].
Results
Firstly, we analysed the delineation of the fourteen different
clinics. Especially, we compared the ROs against the HOs. Secondly,
we analysed the differences between the ROs from ten different
departments and the RO1D from one department.
The mean volume and the standard deviation of the GTV and
PTV volumes contoured by the haematologic oncologists (HO’s)
were significantly smaller than the contours of the radiation oncol-
ogists (RO’s) (Table 1) with 50–68% of the HO volumes relative to
the RO volumes. The mean volumes of both the groups of radiation
oncologists (ROs and RO1D) are comparable for the GTV and PTV
with a much smaller standard deviation by the RO1D. The volumes
delineated by the RO1D ranged from 83% to 90% of the volumes
delineated by the ROs. The deviation from the mean GTV of all pa-
tients calculated for all fourteen different physicians (ROs and
HOs), were largest considering the ROs (30.5 ± 86.2 cm3) and com-
parable considering the HOs ( 34.0 ± 75.2 cm3) and RO1D
( 1.9 ± 39.0 cm3). The differences between the three groups get
larger regarding the contoured PTV (ROs: 46.4 ± 126.4 cm3, HOs:
30.3 ± 171.88 cm3, RO1D: 0.7 ± 89.9 cm3). For ROs and RO1D,
the deviation of the PTV was similar to the deviation of the GTV,
but the standard deviation was larger. The standard deviation of
the PTV contours is larger than that of the GTV contour, especially
for the HOs.
We found a good agreement (P70% of the observers were con-
form) in 81% (92%, 92%) of the investigated lymph node compart-
ments for the ROs (HOs, RO1D). On average the ROs and RO1D
included 6.3 of all 12 lymph node compartments as defined in
methods and materials in their contours in contrast to only 4.4
of the 12 lymph node compartments by the HOs.
We compared the agreement of the contoured parts of all CT-
slices for all patients and physicians. We divided all slices of the
CT in small parts of 0.001 cm3 and compared all of them with
one another. The accurate analysis of this pixel-by-pixel mapping
procedure leads to less agreement than the volume and anatomic
analysis. Only 70% of the ROs contoured 16.3% of the encompassing
GTV volume (smallest contoured part of the slices integrating all
entered contours), 70% of the HOs and RO1D contoured 30.4%
and 32.8% of the encompassing GTV, respectively (Table 2). The
analysis of the fourteen different departments showed even fewer
agreements (12.4%). The agreement between the physicians is bet-
ter concerning the encompassing PTV volume. 70% of the ROs con-
toured 23.7% of the encompassing PTV volume in comparison to
38.6% (35.9%) by the HOs (RO1D).
The common GTV volumes were very small for the 14 depart-
ments but also for the ROs and HOs (6.2–14.9 cm3) and not much
larger for the RO1D (36.5 cm3). The smallest encompassing GTV
458 Interobserver variability for lung cancer

Table 1
Magnitude of volumes for each observer group.

Mean volume over all physicians (cm3)

GTV PTV
Patient 1 Patient 2 Patient 3 Patient 1 Patient 2 Patient 3
14 Departments 137.5 ± 98.8 157.1 ± 84.5 179.6 ± 110.5 458.0 ± 219.1 400.1 ± 118.1 555.8 ± 165.8
ROs 162.6 ± 105.6 183.7 ± 86.9 213.1 ± 116.3 531.8 ± 206.4 450.0 ± 94.1 590.1 ± 183.6
HOs 81.1 ± 45.1 97.5 ± 33.1 122.5 ± 52.8 291.8 ± 142.7 287.7 ± 84.7 487.2 ± 89.7
RO1D 132.8 ± 50.8 151.6 ± .47.8 180.9 ± 38.6 438.2 ± 104.8 393.8 ± 125.2 582.2 ± 85.5

ROs, radiation oncologists from 10 departments; HOs, haematologic oncologist/s pneumologists; RO1D, radiation oncologists from one department; GTV, gross target volume;
PTV, planning target volume.

Table 2 better conformal target coverage, improved sparing of normal tis-

Agreement of the contoured parts of the transversal CT-slices of the physicians with a
sue, and ultimately enhanced local control.
pixel-by-pixel analysis.
Several reports on the variability of GTV and CTV delineation
Agreement in 70% Common Encompassing Common/ have been published for a variety of malignancies, such as tumors
of contoured volume volume (cm3) encompassing
of the lung, breast, cervix, bladder, pancreas and prostate and of
pixels (%) (cm3) volume (%)
the head and neck region [5,10,13–18]. The principle reasons for
GTV
interobserver variability’s can be image-, instruction- and obser-
14 Departments 12.4 ± 4.3 6.2 507.5
ver-related. For our analysis, we wanted to analyse only obser-
1.2 ver-related reasons for interobserver variability.
10 ROs 16.3 ± 4.1 14.9 422.9 3.3 One major factor for interobserver variability is the different
4 HOs 30.4 ± 8.0 9.5 227.4 4.0 assessments regarding the risk of spreading of tumor [5,7]. The
5 RO1D 32.8 ± 3.7 36.5 225.0 15.8
PTV
purpose of our analysis was the difference of interpretations
14 Departments 19.3 ± 4.3 52.9 1446.2 whether a lymph node was involved or not and the extension of tu-
mor size. The first patient presented an atelectasis of the left lower
3.3 lobe, which could increase the variations. Other reasons are train-
10 ROs 23.7 ± 3.9 83.1 1124.0 7.5
ing, differences in the attention to detail in the contouring process
4 HOs 38.6 ± 6.1 70.8 402.7 15.1
5 RO1D 35.9 ± 2.2 121.1 643.5 17.4 itself and subjective interpretation of image knowledge of topo-
graphical cross-sectional anatomy.
Agreement as stated, when 70% of the physicians had contoured the part of the
The observed wide range in volume size for the GTV (22.7–
transversal CT slice.
The encompassing volumes are the smallest volumes integrating all entered vol-
486.9 cm3) and PTV (119.0–927.1 cm3) is comparable to the results
umes of one patient. The common volumes are the largest volumes that are part of observed for lung cancer [5]. The mean volume of tumors, suffer
all GTVs or PTVs of one patient. The described volumes are the sum of all three from rather large neoplasms, may cause a problem regarding inter-
patients. observer variation [5]. The maximum to minimum volume (Vmax/
ROs, radiation oncologists from 10 departments; HOs, haematologic oncologist/s
Vmin) from the GTV is larger for the fourteen departments (10.1)
pneumologists; RO1D, radiation oncologists from one department; GTV, gross tar-
get volume; PTV, planning target volume. and comparable for the different physician groups (2.3–7.9) then
described in the published literature. Van de Steene et al. found a
ratio between 1.1 and 7.6 for the Vmax/Vmin of the GTV in patients
volumes could be found contoured by the RO1D and the HOs, with lung tumor. For some other tumors a quite good agreement
whereas the encompassing volume of the 14 departments showed on target volumes was observed, especially for the prostate
more than twice as much cubic centimetre. The relation between [10,13]. Obviously, tumors that have obvious borders like the pros-
the common and encompassing volumes was less than 5% except tate can be outlined more consistently than tumors without clear
for the RO1D with 15.8%. The observations for the common and border to the neighbouring organs.
encompassing GTV volumes are nearly the same for the PTV vol- The Vmax/Vmin of the GTV (7.9) delineated by the ROs is much
umes with altogether larger volume. But the relation between larger than that of the PTV (2.4). The rigid instruction for delinea-
the common and encompassing volumes is larger for all groups tion gave the order to enlarge the GTV volume (possible automat-
of physicians, especially for the HOs. ically) with a default offset, so that we could assume, that the Vmax/
Vmin of the GTV and PTV could be of the same order, similar to that
Discussion observed in the groups of the HOs and RO1D. The ROs have varied
their PTV after automatic enlargement, especially in lateral direc-
Within the GILT-CRT-1-study (multicenter study for patients tions, although this was not part of the study protocol. Thereby
with unresectable locally advanced NSCLC), we wanted to evaluate the large differences in the GTV delineation were reduced for the
the agreement of target volume delineation by participants of this PTV. This is reflected in the lesser differences between the ROs
study. The target volume definition for conformal radiotherapy and the HOs/RO1D in the PTV contours, whereby the clinical im-
may be the most important factor for adequate tumor coverage pact of the differences in GTV delineation will be much reduced.
and sufficient sparing of critical structures [5]. Within a study pro- The accurate definition of the GTV can be assed by pathology-
tocol every effort should be taken to ensure that the contours are correlated imaging, if patients with small tumors underwent sur-
standardized and accurate with high interobserver reproducibility. gery after radiotherapy. Thereby most GTV definitions are too lar-
Contouring variations result in systematic errors of radiation treat- ger, but can also be too small [19]. We recommend to focus not
ment that are a confounding factor for the interpretation of treat- only on the GTV delineation but also more on the PTV delineation,
ment results. Besides, the comparability between different because subsequent steps use the initial GTV data. Dosimetric and
departments, who take part in multicenter studies, will be very dif- field planning decisions are based primary on the PTV and second-
ficult. More accurate delineation of the GTV and PTV may result in arily on the GTV.
 H. Vorwerk et al. / Radiotherapy and Oncology 91 (2009) 455–460
The definition of lymph nodes classified as pathologic was ex-
actly defined with P1.5 cm or many lymph nodes with P1.0 cm
in a lymph node compartment (definition by Naruke et al. [8]
and Kirikuta et al. [9]). Every participant received a detailed CT-
map of the lymph node compartments. Particularly it was demon-
strated to all physicians how to measure diameters of the lymph
nodes. This led to a good agreement in 100% and P80% of all ana-
lysed lymph node regions in 42% and 75% of all participants,
respectively. We found only disagreement in the paratracheal
medial left paraesophageal and praeaortal lymph node regions. An-
other error was due to confusion between lymph node, esophagus
and vessels. Other authors found 63% of involved lymph node re-
gions were delineated correctly by the clinicians and 22% of the
drawn lymph node regions were accepted to be false positive [5].
This correlates exactly with 75% agreement of all physicians in
our analysis. So that we must state, that strict measuring of lesions
with a rigid instruction for delineation could not notably reduce
interobserver variations in lymph node judgement in our analysis.
The pixel-by-pixel analysis compares every part of the contours
against the contours of the other physicians instead of only math-
ematical values and anatomical sites. This reflects the exact differ-
ences between the participants. The differences in the GTV
contours of the ROs and HOs (16.3% and 30.4%) were respectable
in the analysis group by group, but less compared with each other
groups (12.4%), because the HOs delineated smaller parts of the CT-
slices than the ROs but are more consistent among each other (Ta-
ble 2). The relation between these two groups was reflected in the
PTV delineation with better agreement within the groups. The
agreement between the RO1D is higher than that between the
ROs even in the GTV (32.8%) but also in the PTV contours (35.9%)
and comparable to the HOs. This was also reflected in the relation
between the common and encompassing volumes.
Our analysis showed large discrepancies regarding the delinea-
tion of the lymph node regions by the HOs. This may be caused by
less knowledge regarding the large mediastinal vessels and the po-
sition of esophagus in CT-scans by the HOs in comparison to the
ROs. On the other hand, we found small discrepancies between
the HOs regarding the delineation of the tumor. The HOs had an-
other point of view than the ROs regarding the tumor because they
are used to define the tumor expansion based on combination of
bronchoscopy and CT. This could be the reason for this good agree-
ment. The contoured volumes by the HOs are significantly smaller
than contoured by the ROs and RO1D. This could be a consequence
of the RO’s fear of missing a part of the tumor tissue and therefore
design larger contours, whereas the HOs are more afraid of making
a false positive error.
The agreement in the group of RO1D regarding the delineated
lymph node regions was superior with 92% agreement compared
to the other groups and published data with 75%. We found the
same good agreement in the GTV and also the PTV contoured by
the RO1D in all analysis in contrast to the ROs. This implicates that
the RO1D used the automatic expansion software of the contouring
program and did not correct the PTV free-hand correction
afterwards. Remarkably we found the same agreement in the
PTV delineation after free-handly correction by the ROs and only
automatically enlarged GTV by the RO1D.
We suggest, that we could detect better agreement because the
oncologists from one department are used to compare and discuss
their target volumes daily for better accordance concerning the
delineation of the tumor as well as the lymph node regions. The
interobserver differences would not be relevantly improved by
use of a restrictive instruction, but intra-institutional agreement
can diminish differences in target volume delineation.
The interobserver variabilities vary by use of different imaging
techniques. PET-CT is superior to CT in delineation of the tumor
and the lymph nodes [3,20–22], but PET was not requested in
459
GILT-study. By use of PET the variability between the participants
would probably be smaller, especially in regions of atelectasis [23].
Delineation accuracy can be improved by using CT window le-
vel information. The CT lung windows correlate better with the
pathologic size of lung adenocarcinoma than do the mediastinal
windows [24]. Grills et al. described an overestimation of the gross
pathologic size by a mean of 5.8 mm and an underestimation of the
gross tumor size plus microscopic extension by a mean of 1.2 mm
by use of the mediastinal window. All participants were instructed
to delineate the tumor expansion in the lung and mediastinal win-
dow and pathologic lymph nodes – not the whole lymph node re-
gion – to deal with this problem. Nevertheless this led to no
diminishment of the variation.
Difficulties in discrimination between tumor and atelectasis are
widely known and a major factor in interobserver variations [12].
In our analysis we could not detect higher discrepancies of the con-
tours of patient 1 (with atelectasis) in comparison to patient 2 or 3.
Possibly a microinvasive insertion of radio-opaque markers may be
helpful in diminishing interobserver variabilities [25].
Conclusions
No ‘‘golden standard” in GTV or PTV delineation can be defined
so far. We will not till then be able to make appraisals of the clin-
ical impact, if all physicians define the PTV in the same way with a
much better agreement, than is currently detected. A good training
of radiation oncologists, a good collaboration between radiation
oncologists and radiologist, dummy runs for multicenter studies,
establishment of central reviewing boards or detailed instructions
for delineation (tested in this analysis) are recommended. How-
ever, up to now these procedures have not been shown to decrease
the interobserver variability substantially [6,12,17,18,26,27].
Nevertheless, we could not demonstrate an improvement in
agreement between the physicians by use of a detailed instruction
for delineation compared to existing literature with no detailed
instructions, but we could demonstrate, that physicians from one
department agree more than physicians from different depart-
ments. We recommend online available examples and the possibil-
ity of comparison of the own delineation in order to control oneself
periodically for better agreement in contouring of target volumes.
A ‘‘golden standard” for target delineation should be developed
taking relapse patterns into account.
References
[1] Jeremic B, Machtay M. Concurrent radiochemotherapy in the treatment of
locally advanced non-small cell lung cancer. Hematol Oncol Clin North Am
2004;18:91–100.
[2] Curran Jr WJ. Therapy for stage III non-small-cell lung cancer: Bob Beamon or
Chemo-Beamo? Clin Lung Cancer 2006;8:84.
[3] Hong R, Halama J, Bova D, Sethi A, Emami B. Correlation of PET standard uptake
value and CT window-level thresholds for target delineation in CT-based
radiation treatment planning. Int J Radiat Oncol Biol Phys 2007;67:720–6.
[4] Martel MK, Ten Haken RK, Hazuka MB, et al. Estimation of tumor control
probability model parameters from 3-D dose distributions of non-small cell
lung cancer patients. Lung Cancer 1999;24:31–7.
[5] Van de Steene J, Linthout N, de Mey J, et al. Definition of gross tumor volume in
lung cancer: inter-observer variability. Radiother Oncol 2002;62:37–49.
[6] Weiss E, Hess CF. The impact of gross tumor volume (GTV) and clinical target
volume (CTV) definition on the total accuracy in radiotherapy theoretical
aspects and practical experiences. Strahlenther Onkol 2003;179:21–30.
[7] Giraud P, Elles S, Helfre S, et al. Conformal radiotherapy for lung cancer:
different delineation of the gross tumor volume (GTV) by radiologists and
radiation oncologists. Radiother Oncol 2002;62:27–36.
[8] Naruke T, Suemasu K, Ishikawa S. Lymph node mapping and curability at
various levels of metastasis in resected lung cancer. J Thorac Cardiovasc Surg
1978;76:832–9.
[9] Kiricuta IC. Selection and delineation of lymph node target volume for lung
cancer conformal radiotherapy. Proposal for standardizing terminology based
on surgical experience. Strahlenther Onkol 2001;177:410–23.
[10] Rasch C, Barillot I, Remeijer P, et al. Definition of the prostate in CT and MRI: a
multi-observer study. Int J Radiat Oncol Biol Phys 1999;43:57–66.
460 Interobserver variability for lung cancer
[11] Weltens C, Menten J, Feron M, et al. Interobserver variations in gross tumor
volume delineation of brain tumors on computed tomography and impact of
magnetic resonance imaging. Radiother Oncol 2001;60:49–59.
[12] Valley JF, Bernier J, Tercier PA, et al. Quality assurance of the EORTC
radiotherapy trial 22931 for head and neck carcinomas: the dummy run.
Radiother Oncol 1998;47:37–44.
[13] Logue JP, Sharrock CL, Cowan RA, et al. Clinical variability of target volume
description in conformal radiotherapy planning. Int J Radiat Oncol Biol Phys
1998;41:929–31.
[14] Petersen RP, Truong PT, Kader HA, et al. Target volume delineation for partial
breast radiotherapy planning: clinical characteristics associated with low
interobserver concordance. Int J Radiat Oncol Biol Phys 2007;69:41–8.
[15] Smith WL, Lewis C, Bauman G, et al. Prostate volume contouring: a 3D analysis
of segmentation using 3DTRUS, CT, and MR. Int J Radiat Oncol Biol Phys
2007;67:1238–47.
[16] Weiss E, Richter S, Krauss T, et al. Conformal radiotherapy planning of cervix
carcinoma: differences in the delineation of the clinical target volume. A
comparison between gynaecologic and radiation oncologists. Radiother Oncol
2003;67:87–95.
[17] Yamazaki H, Nishiyama K, Tanaka E, et al. Dummy run for a phase II
multi-institute trial of chemoradiotherapy for unresectable pancreatic cancer:
inter-observer variance in contour delineation. Anticancer Res 2007;27:
2965–71.
[18] Coles CE, Hoole AC, Harden SV, et al. Quantitative assessment of inter-clinician
variability of target volume delineation for medulloblastoma: quality
assurance for the SIOP PNET 4 trial protocol. Radiother Oncol 2003;69:
189–94.
[19] Stroom J, Blaauwgeers H, van Baardwijk A, et al. Feasibility of pathology-
correlated lung imaging for accurate target definition of lung tumors. Int J
Radiat Oncol Biol Phys 2007;69:267–75.
[20] van Baardwijk A, Bosmans G, Boersma L, et al. PET-CT-based auto-contouring
in non-small-cell lung cancer correlates with pathology and reduces
interobserver variability in the delineation of the primary tumor and
involved nodal volumes. Int J Radiat Oncol Biol Phys 2007;68:771–8.
[21] Fox JL, Rengan R, O’Meara W, et al. Does registration of PET and planning CT
images decrease interobserver and intraobserver variation in delineating
tumor volumes for non-small-cell lung cancer? Int J Radiat Oncol Biol Phys
2005;62:70–5.
[22] Jeong SY, Lee KS, Shin KM, et al. Efficacy of PET/CT in the characterization of
solid or partly solid solitary pulmonary nodules. Lung Cancer 2008;61:186–94.
[23] Fitton I, Steenbakkers RJ, Gilhuijs K, et al. Impact of anatomical location on
value of CT-PET co-registration for delineation of lung tumors. Int J Radiat
Oncol Biol Phys 2008;70:1403–7.
[24] Grills IS, Fitch DL, Goldstein NS, et al. Clinicopathologic analysis of microscopic
extension in lung adenocarcinoma: defining clinical target volume for
radiotherapy. Int J Radiat Oncol Biol Phys 2007;69:334–41.
[25] Kupelian PA, Forbes A, Willoughby TR, et al. Implantation and stability of
metallic fiducials within pulmonary lesions. Int J Radiat Oncol Biol Phys
2007;69:777–85.
[26] Seddon B, Bidmead M, Wilson J, Khoo V, Dearnaley D. Target volume definition
in conformal radiotherapy for prostate cancer: quality assurance in the MRC
RT-01 trial. Radiother Oncol 2000;56:73–83.
[27] Roy AE, Wells P. Volume definition in radiotherapy planning for lung cancer:
how the radiologist can help. Cancer Imaging 2006;6:116–23.