You are on page 1of 6

ORIGINAL ARTICLE

Assessing the American Board of Orthodontics


objective grading system: Digital vs plaster
dental casts
Troy R. Okunami,a Budi Kusnoto,b Ellen BeGole,c Carla A. Evans,d Cyril Sadowsky,e
and Shahrbanoo Fadavif
Chicago, Ill

Introduction: The purposes of this study were to determine whether the American Board of Orthodontics
objective grading system (ABO OGS) can be assessed accurately from digital dental casts and whether there
are statistical differences between digital and plaster dental casts in scoring the ABO OGS. Methods: Thirty
posttreatment plaster dental casts were selected and scanned by OrthoCAD (Cadent, Carlstadt, NJ) to
produce 30 corresponding digital dental casts. The plaster and digital casts were compared by using the
criteria of the ABO OGS. Because the data were ordinal, a nonparametic statistical analysis was used.
Results: The Wilcoxon test for paired samples showed significant differences between the plaster and digital
casts for occlusal contacts, occlusal relationships, and total scores (P ⬍.05). No significant differences were
found for alignment, marginal ridges, overjet, and interproximal contacts. Buccolingual inclination was not
included in this study because of inability to perform proper measurements with the OrthoCAD program.
Mean differences of points deducted ranged from .03 point for marginal ridges to 5.07 points for the total
score. The variable with the most points deducted related to occlusal relationships, and the fewest points
were deducted for interproximal contacts. Conclusions: Based on this study, the current OrthoCAD program
(version 2.2) was not adequate for scoring all parameters as required by the ABO OGS. (Am J Orthod
Dentofacial Orthop 2007;131:51-6)

T
he American Board of Orthodontics (ABO) university cases were compared with 32 ABO cases.
developed the objective grading system (OGS) The results showed that the ABO group lost fewer
to enhance the reliability of the examiners and points than the university group due to the highly
to give candidates for board certification a tool to assess selective cases previously submitted for board certifi-
the adequacy of their finished orthodontic results. Field cation. Finishing in the anterior segment and second
testing of the OGS began in 1995, and the ABO molar region was better in the ABO group. The
officially implemented its use in February 1999 in St university group had significantly better scores for root
Louis. The OGS comprises 8 criteria, 7 of which parallelism, but the ABO group had significantly better
involve critiquing plaster dental casts; the eighth in- scores for occlusal contacts and overjet.
cludes radiographic evaluation.1 Lieber et al3 recently tested the reliability and
Yang-Powers et al2 were interested in how treat- subtraction frequency of the ABO’s study model scor-
ment results of a graduate orthodontic department ing system. Thirty-six posttreatment study models were
compared with cases that had previously been submit-
scored by 4 faculty members from a graduate orthodon-
ted and passed ABO board certification. Ninety-two
tic department. For all judges, for each criterion, the
difference between the 2 scoring sessions was between
From the College of Dentistry, University of Illinois at Chicago, Chicago, Ill.
a
Resident, Department of Orthodontics.
1 and 2 subtraction points. For the overall score, the
b
Assistant clinical professor, Department of Orthodontics. differences ranged from 3 to 6 subtraction points.
c
Associate professor, Department of Orthodontics. Therefore, the data showed that some judges, on
d
Professor and head, Department of Orthodontics.
e
Professor, Department of Orthodontics. average, were much more lenient than others. The
f
Professor, Department of Pediatric Dentistry. question remains, “Is the OGS really objective?”
Reprint requests to: Budi Kusnoto, University of Illinois at Chicago, Depart-
ment of Orthodontics M/C 841, College of Dentistry, 801 S Paulina St,
With the advent of new and improved technology,
Chicago, IL 60612; e-mail, bkusno1@uic.edu. orthodontists have many options to consider. Digital
Submitted, October 2004; revised and accepted, August 2005. technology has become an integral component of many
0889-5406/$32.00
Copyright © 2007 by the American Association of Orthodontists. practices to improve quality and efficiency. The tech-
doi:10.1016/j.ajodo.2005.04.042 nology includes digital photographs, radiographs, pa-
51

Descargado para Karen vanessa Rivero Ustariz (kv.rivero@unicieo.edu.co) en University Foundation CIEO UniCIEO de ClinicalKey.es por Elsevier en febrero 14, 2019.
Para uso personal exclusivamente. No se permiten otros usos sin autorización. Copyright ©2019. Elsevier Inc. Todos los derechos reservados.
52 Okunami et al American Journal of Orthodontics and Dentofacial Orthopedics
January 2007

perless treatment records, and 3D study models. With Table I. OGS criteria and values
multiple-doctor practices, multiple practice locations, in- Criterion Score
creases in patient bases, and a need for efficient storage,
retrieval, and transference of information, orthodontists Alignment 64
have used this technology to manage their practices.4 Marginal ridges 32
Buccolingual inclination 40
Traditionally, plaster casts have been a part of the
Occlusal contacts 64
orthodontist’s armamentarium. Plaster casts are impor- Occlusal relationship 48
tant to aid in diagnosis and treatment planning and to Overjet 32
evaluate progress and outcomes of treatment. However, Interproximal contacts 60
traditional plaster casts have been criticized for the Root angulation (radiographic) 64
Total score 404
labor-intensive work, fragility, demand on storage
space, and problems of transfer and retrieval of infor-
mation. As a means of overcoming these and other
problems, 3D study models have increased in popularity.5 might be better suited to provide an objective measure of
Santoro et al6 performed a study to compare the orthodontic treatment results than the traditional plaster
accuracy of measurements made by the OrthoCAD (Ca- dental casts and the manual measuring gauge.
dent, Carlstadt, NJ) system on digital models with Tests of reliability were made before comparing the
measurements made by hand on traditional plaster 2 methods (manual vs digital) by means of extensive
models. Two examiners, working independently, re- calibration between the examiner of this study (T.R.O.)
corded tooth size, overbite, and overjet of 76 plaster and a senior ABO examiner.
and 76 digital models. Results showed a statistically Although reliability depends on the abilities of
significant difference in tooth width measurements well-calibrated examiners in accurately making or re-
between plaster and digital models, with the digital producing measurements, in this study, we emphasized
model measurements smaller than the corresponding comparing the results of the ABO OGS measured
plaster model measurements. There was also a signifi- manually and the OrthoCAD measurements made by
cant difference in overbite measurements between the 2 the examiner of this study.
methods, with all digital measurements smaller than the
manual measurements. There was no statistically sig- MATERIAL AND METHODS
nificant difference between the overjet measurements Thirty posttreatment plaster dental casts were se-
between the 2 methods. lected from the University of Illinois at Chicago De-
Zilberman et al7 studied the accuracy of measuring partment of Orthodontics. The dental casts were obtained
casts with electronic calipers vs OrthoCAD. Twenty from patients who had nonsurgical comprehensive orth-
typodont setups with artificial teeth having various odontic treatment with fixed appliances. The dental casts
malocclusions were created, and impressions were had to be in acceptable condition with all incisors,
taken of the setups. Both plaster and digital models canines, at least 1 premolar, and the first and second
were made, and tooth size, intercanine width, and molars bilaterally in both the maxillary and mandibular
intermolar width measurements were taken from the arches. No duplicates of dental casts were selected, and
typodonts, plaster, and digital models. Results showed there was no consideration about age or sex.
that all methods of measurements were highly valid and Thirty posttreatment 3D digital dental casts were
reproducible for tooth size, intercanine width, and inter- constructed from the same 30 plaster dental casts. The
molar width. Comparing the plaster and digital measure- original plaster casts were sent to OrthoCAD to be
ments, they found that the measurements on the plaster scanned by means of nondestructive scanning and
models made with electronic calipers had greater accuracy returned to the Department of Orthodontics to be used
and reproducibility than the OrthoCAD measurements. in this study. The idea was to have identical dental casts
Although there is an increase in the use of digital for comparison.
3D study models by orthodontists, there is not yet an Eight criteria were conventionally measured, and an
objective means of evaluating orthodontic treatment overall score was calculated by subtracting from the
results via the computer from a digital study model. total of 404 points available (Table I). Each criteria has
Consequently, it is important to explore programs a maximum point value based on the number of teeth
designed to objectively evaluate treatment results with and contacts possible. Because 1 criteria, root angula-
3D study models. OrthoCAD developed an ABO pro- tion, involves radiographic evaluation, this component
gram that evaluates the 7 criteria for model analysis by was not included in the study. Therefore, 7 criteria were
using digital study models. It is believed that this program individually scored, and the total score of points sub-

Descargado para Karen vanessa Rivero Ustariz (kv.rivero@unicieo.edu.co) en University Foundation CIEO UniCIEO de ClinicalKey.es por Elsevier en febrero 14, 2019.
Para uso personal exclusivamente. No se permiten otros usos sin autorización. Copyright ©2019. Elsevier Inc. Todos los derechos reservados.
American Journal of Orthodontics and Dentofacial Orthopedics Okunami et al 53
Volume 131, Number 1

Fig 1. Seven parameters of ABO OGS available in OrthoCAD.

tracted was calculated. All measurements on the plaster establish the consistency of the measurements. Ten
casts were made manually with the ABO measuring plaster models and their corresponding digital models
gauge. The OrthoCAD program was used to make were selected at random. Two plaster and 2 digital
measurements with the digital models (Fig 1). models were measured per day for 5 consecutive days.
The primary investigator (T.R.O.) was trained by a The same procedure was repeated 2 weeks later, and
former ABO examiner who was familiar with the the measurements were compared. The buccolingual
model analysis. A training session was conducted to variable of the OGS was eliminated because of an error
teach the investigator to score plaster models and in the digital model assessment program. After calibra-
establish consistency of measurements. After the initial tion, all 30 plaster and digital models were measured.
training, interexaminer calibration was conducted be- Measurements were made on 2 randomly selected
tween the investigator and the former ABO examiner. plaster and 2 digital models each day until all models
Five sets of plaster models were used, and measure- were measured.
ments were made separately by each person and com-
pared. If there was a discrepancy of more than 2 points Statistical analysis
per component, the discrepancy was corrected through All data were entered into a data file for statistical
repeated measurements. There was no official training analysis with the Statistical Package for Social Sciences
session with the ABO program of OrthoCAD. The (SPSS, Chicago, Ill). A detailed analysis was conducted
primary investigator followed OrthoCAD’s instruc- for investigator reliability and for the data collection of
tions. the 30 plaster and digital models. For investigator
An intraexaminer calibration was also conducted to reliability, both plaster and digital models were mea-

Descargado para Karen vanessa Rivero Ustariz (kv.rivero@unicieo.edu.co) en University Foundation CIEO UniCIEO de ClinicalKey.es por Elsevier en febrero 14, 2019.
Para uso personal exclusivamente. No se permiten otros usos sin autorización. Copyright ©2019. Elsevier Inc. Todos los derechos reservados.
54 Okunami et al American Journal of Orthodontics and Dentofacial Orthopedics
January 2007

Table II. Reliability of plaster model measurements Table IV. Plaster vs digital model measurements
Mean Mean
Criterion difference Wilcoxon Significance Criterion difference Wilcoxon Significance

Alignment ⫺0.40 ⫺0.921 .357 Alignment 0.23 ⫺0.955 .340


Marginal ridges ⫺0.50 ⫺1.667 .096 Marginal ridges 0.03 ⫺0.206 .837
Occlusal contacts 0.30 ⫺1.134 .257 Occlusal contacts ⫺4.53 ⫺4.365 .000*
Occlusal relationships 0.50 ⫺1.508 .132 Occlusal relationships ⫺0.50 ⫺2.276 .023*
Overjet ⫺0.30 ⫺1.000 .317 Overjet ⫺0.37 ⫺1.643 .100
Interproximal contacts 0.10 ⫺0.577 .564 Interproximal contacts ⫺0.13 ⫺1.633 .102
Total ⫺0.30 ⫺0.480 .631 Total ⫺5.07 ⫺3.732 .000*

*P ⬍.05.

Table III. Reliability of digital model measurements


Table V. Descriptive statistics for plaster models
Mean
Criterion difference Wilcoxon Significance Criterion Mean SD Median

Alignment ⫺0.40 ⫺1.260 .206 Alignment 9.77 3.559 9.50


Marginal ridges ⫺0.30 ⫺1.732 .083 Marginal ridges 4.10 2.123 4.00
Occlusal contacts 0.50 ⫺1.667 .096 Occlusal contacts 4.60 2.860 5.00
Occlusal relationships 0.40 ⫺1.633 .102 Occlusal relationships 11.00 4.698 10.50
Overjet ⫺0.10 ⫺0.265 .791 Overjet 7.50 3.655 7.00
Interproximal contacts 0.20 ⫺1.414 .157 Interproximal contacts 0.97 1.066 1.00
Total 0.30 ⫺0.604 .546 Total 37.93 11.026 38.00

sured at 2 separate times. A comparison was done


separately for plaster models and digital models, and The overall mean for the plaster models was 37.93
the Wilcoxon test was used to test for statistical points with an SD of 11.026 points and a median of
differences. The results suggest the consistency of 38.00. For each criterion, the mean deductions were
measurements for each criterion and the total score of 9.77 points for alignment, 4.10 points for marginal
the OGS by the investigator. In addition, a detailed ridges, 4.60 points for occlusal contacts, 11.00 points
analysis of the data was conducted by using the for occlusal relationships, 7.50 points for overjet, and
Wilcoxon test to determine statistically significant dif- 0.97 point for interproximal contacts. These results are
ferences between plaster and digital dental models. listed in Table V.
These findings permitted assessment of the OGS pro- The criterion that showed the highest score (most
gram designed by OrthoCAD. Because the resulting points deducted) was occlusal relationships at 11.00
ABO scoring data were ordinal, a nonparametric sta- (SD 4.698). The criterion that showed the lowest score
tistical test was used. (fewest points deducted) was interproximal contacts at
0.97 (SD 1.066). The progression of criteria from the
RESULTS highest mean score (most points deducted) to the lowest
For intraexaminer reliability, the Wilcoxon test for mean score was occlusal relationships, alignment, over-
paired samples showed no statistical differences for the jet, occlusal contacts, marginal ridges, and interproxi-
plaster models (Table II) and the digital models (Table mal contacts.
III). The mean differences for each variable and total The overall mean for the digital models was 42.93
score involving measurements taken at 2 separate times points with an SD of 11.026 points and a median of
are shown for both the plaster and the digital models. 43.50. For each criterion, the mean deductions were
The mean differences for each criterion and the total 9.53 points for alignment, 4.07 points for marginal
score were no greater than 0.05 points deducted for ridges, 9.13 points for occlusal contacts, 11.50 points
both plaster and digital model measurements. for occlusal relationships, 7.87 points for overjet, and
With regard to comparing plaster models to digital 0.83 point for interproximal contacts. These results are
models, the Wilcoxon test for paired samples showed listed in Table VI.
statistical differences for occlusal contacts, occlusal rela- The criterion that showed the highest score (most
tionships, and total score (Table IV). The mean differ- points deducted) was occlusal relationships at 11.50
ences for each variable and total score comparing the (SD 4.833). The criterion that showed the lowest score
plaster and digital models are also shown in Table IV. was interproximal contacts at 0.83 (SD 1.053). The

Descargado para Karen vanessa Rivero Ustariz (kv.rivero@unicieo.edu.co) en University Foundation CIEO UniCIEO de ClinicalKey.es por Elsevier en febrero 14, 2019.
Para uso personal exclusivamente. No se permiten otros usos sin autorización. Copyright ©2019. Elsevier Inc. Todos los derechos reservados.
American Journal of Orthodontics and Dentofacial Orthopedics Okunami et al 55
Volume 131, Number 1

Table VI. Descriptive statistics for digital models ences between the plaster and digital model measure-
ments ranged from 1 to 2 points. However, since the
Criterion Mean SD Median
occlusal-contact criterion differed by 3 to 25 points, the
Alignment 9.53 3.329 9.00 total score could no longer be a solid measure.
Marginal ridges 4.07 1.999 3.50 Another problem encountered with OrthoCAD’s
Occlusal contacts 9.13 6.107 8.50
program involved the buccolingual-inclination crite-
Occlusal relationships 11.50 4.833 11.50
Overjet 7.87 3.776 8.00 rion. The problem was that measurements were not
Interproximal contacts 0.83 1.053 0.00 taken at the proper locations. All measurements were
Total 42.93 9.563 43.50 taken along a reference plane created by 2 selected
points. In reality, measurements were supposed to be
from specified cusp tips. Because the OrthoCAD pro-
progression of criteria from the highest mean score gram performed measurements at incorrect locations,
(most points deducted) to the lowest mean score was this criterion was excluded from the study (Fig 2).
occlusal relationships, alignment, occlusal contacts, In addition to the occlusal-contact criterion and
overjet, marginal ridges, and interproximal contacts. total score, the occlusal-relationship criterion showed
For both the plaster and the digital models, the statistically significant differences between plaster and
criteria with the highest and lowest mean scores were digital models. One explanation for the discrepancy
the same. Occlusal relationships had the highest mean might have been the angulation of the plaster models
score, and interproximal contacts had the lowest mean while measurements were taken. If the plaster models
score. The progression of most to least points deducted were not viewed perpendicularly, the measurements
was the same for both the plaster and the digital might have been interpreted differently. In contrast, the
models; for the plaster models, more points were digital models could be viewed from the exact angula-
deducted for overjet than for occlusal contacts. This tion by using the predefined spots on the horizontal
was opposite to what was seen for the digital models. plane in the view-control dialog. Another possible
Alignment had the second highest score, and marginal reason for differences in occlusal relationships between
ridges had the second fewest points deducted. digital and plaster models was that this criterion had the
most points deducted. If more points were deducted, the
DISCUSSION chances of influencing the ABO OGS overall scores
When we compared the plaster and digital models, could be elevated.
we found statistically significant differences for occlu- Although the differences for occlusal relationships
sal contacts, occlusal relationships, and total score. A between plaster and digital models were statistically
major problem was encountered throughout the study significant, it might be irrelevant to whether a candidate
that explains the differences for occlusal contacts and passes the ABO examination. A total deduction of 20
total score. When we performed the occlusal-contact points is a passing score, and a deduction of 30 points
measurements, the digital images of the maxillary and is a failing score for the study model section. The range
mandibular teeth overlapped each other. The result was between 20 and 30 points deducted is a gray zone.
an occlusal-contact measurement that measured the Therefore, 0.5 mm of mean difference for occlusal
amount of vertical overlap between the images rather than relationships between plaster and digital models would
the distance that the teeth were not in contact. Therefore, not seem to have any practical consequence. Even
points were deducted unnecessarily. OrthoCAD was though some findings show statistical differences, they
made aware of the problem and tried to fix it by were too small to be noticed clinically during the grading
manipulating the images. However, instead of fixing process and thus may not be clinically significant.
the problem, another problem was created. Several Although problems were encountered with the
models no longer had occlusal contacts. When made OrthoCAD software, OrthoCAD stressed the potential
aware of this problem, OrthoCAD conceded that the benefits of the digital ABO program compared with
problem could not be fixed with the version we were conventional plaster and the measuring gauge.
using for this study. As a result, 10 digital models had OrthoCAD provides several benefits to the ABO. For
point differences between 3 and 25 points deducted for example, scores and measurements are saved as part of
the occlusal-contact criterion alone. a digital file in OrthoCAD; a user can communicate
Because the number of points deducted for occlusal files electronically; and files in OrthCAD can be used in
contacts was so substantial, the total scores calculated web sites and online lectures. OrthoCAD also provides
for the digital models were also greatly affected. For some potential benefits to candidates pursuing board
each criterion other than occlusal contacts, the differ- certification. For example, they can submit fully digital

Descargado para Karen vanessa Rivero Ustariz (kv.rivero@unicieo.edu.co) en University Foundation CIEO UniCIEO de ClinicalKey.es por Elsevier en febrero 14, 2019.
Para uso personal exclusivamente. No se permiten otros usos sin autorización. Copyright ©2019. Elsevier Inc. Todos los derechos reservados.
56 Okunami et al American Journal of Orthodontics and Dentofacial Orthopedics
January 2007

Fig 2. Buccolingual inclination as measured in OrthoCAD.

files and communicate cases with tutors and instructors models to those made of plaster and hardened polymer
more efficiently compared with plaster models; the could be of great interest. These studies and others with
casts are more objective and standard than plaster digital models will be important in the future as the use
models. However, the most important aspect of the of digital models increases in popularity.
OrthoCAD program is that it must be shown to be
either as or even more consistent and reliable than REFERENCES
manual grading of plaster models. Although the poten-
1. Casko JS, Vaden JL, Kokich VG, Damone J, James RD, Cangia-
tial benefits are desirable, the OrthoCAD program has losi TJ, Riolo ML, et al. American Board of Orthodontics
not been demonstrated to be superior to the gold objective grading system for dental casts and panoramic radio-
standard of plaster casts. graphs. Am J Orthod Dentofacial Orthop 1998;114:589-99.
Many studies can be conducted in the future with 2. Yang-Powers LC, Sadowsky C, Rosenstein S, BeGole EA. Treat-
ment outcome in a graduate orthodontic clinic using the American
the OrthoCAD program. OrthoCAD has developed and
Board of Orthodontics grading system. Am J Orthod Dentofacial
released a new version of the software. This study can Orthop 2002;122:451-5.
be repeated with the new version, which OrthoCAD 3. Lieber WS, Carlson SK, Baumrind S, Poulton DR. Clinical use of
claims corrects the problem of the buccolingual crite- the ABO scoring index: reliability and subtraction frequency.
rion. A study quantifying the time required to score Angle Orthod 2003;73:556-64.
plaster and digital models with the ABO OGS could be 4. Redmond WR. Digital models: a new diagnostic tool. J Clin
Orthod 2001;35:386-7.
conducted. Geodigm (Chanhassen, Minn) claims to be 5. Redmond WR. The digital orthodontic office: 2001. Semin Orthod
developing an ABO OGS program for grading digital 2001;7:266-3.
models. A study could be conducted to compare plaster 6. Santoro M, Galkin S, Teredesai M, Nicolay O, Cangialosi T.
and digital models with both OrthoCAD and Geodigm Comparison of measurements made on digital and plaster models.
software. Finally, OrthoCAD currently uses a process Am J Orthod Dentofacial Orthop 2003;124:101-5.
7. Zilberman O, Huggare J, Parikakis K. Evaluation of the validity of
called rapid prototyping to create a set of orthodontic tooth size and arch width measurements using conventional and
models made of hardened polymer, which is created three-dimensional virtual orthodontic models. Angle Orthod 2003;
from the digital file. A study that compares digital 73:301-6.

Descargado para Karen vanessa Rivero Ustariz (kv.rivero@unicieo.edu.co) en University Foundation CIEO UniCIEO de ClinicalKey.es por Elsevier en febrero 14, 2019.
Para uso personal exclusivamente. No se permiten otros usos sin autorización. Copyright ©2019. Elsevier Inc. Todos los derechos reservados.