Professional Documents
Culture Documents
ORIGINAL ARTICLES
Abstract
Objective: To assess reasons for discrepancies between primary care consultation measured from patient self-report and that based on
medical records.
Methods: Retrospective comparison of recalled consultation in previous 12 months among 2,414 subjects aged 501 who reported
knee pain in a population survey vs. primary care medical records. Record review included (1) all knee morbidity codes and (2) knee prob-
lems mentioned in consultation text. It was then extended to: (3) more than 12 months before survey, and (4) consultations for leg or wide-
spread problems (e.g., generalized osteoarthritis).
Results: In those who reported knee pain, recalled consultation prevalence for knee problems ‘‘in past year’’ was 33% compared with
15% based on medical records. Forty percent of those with a recalled consultation had a recorded knee problem in the same time period
(kappa 5 0.43). Expanding record search to include leg and widespread problems, and knee problems up to 40 months prior to survey,
increased ‘‘verified’’ self-reported consulters to 80%.
Conclusions: Disparity in estimates of consultation prevalence arose from inaccuracy of: (1) recall in survey responders and (2) record-
ing by general practitioners of specific problems and repeat consultations. Perceived importance of problem in a multiproblem contact and
whether it leads to an outcome (e.g., prescription) may influence recording. Implications exist for service provision projections and re-
search. Ó 2006 Elsevier Inc. All rights reserved.
Keywords: Health care surveys; Knee; Medical records; Primary health care; Utilization
Table 1
Recorded consultation by recalled consultation for knee pain (reporters of knee pain only)
Recorded consultation
12 months before survey response 18 months before survey response
Step 1 Step 2 Step 3
Self-report status Total Knee code Knee code or text Knee code Knee code or text
Report consultation 807 214 (26.5%) 324 (40.1%) 253 (31.4%) 383 (47.5%)
Report no consultation 1607 24 (1.5%) 47 (2.9%) 50 (3.1%) 100 (6.2%)
794 K. Jordan et al. / Journal of Clinical Epidemiology 59 (2006) 791–797
Table 2
Associations with recorded GP knee disorder (coded or text) consultation within 18 months (steps 1–3) for those recalling consultation (n 5 807)
Total Recorded consultation n (%) ORa 95% CI ORb 95% CI
Recalled consultation 807 383 (47%)
Male 338 143 (42%) 1.00 1.00
Female 469 240 (51%) 1.43 1.08, 1.89 1.69 1.21, 2.35
Aged 50–64 376 188 (50%) 1.00 1.00
Aged 65–74 248 117 (48%) 0.89 0.65, 1.23 0.91 0.64, 1.32
Aged 751 183 78 (43%) 0.74 0.52, 1.06 0.78 0.51, 1.21
Practice A 208 97 (47%) 1.00 1.00
Practice B 368 186 (51%) 1.17 0.83, 1.64 1.42 0.96, 2.10
Practice C 231 100 (43%) 0.87 0.60, 1.27 1.02 0.67, 1.55
No further education 698 329 (47%) 1.00 1.00
Further education 80 42 (53%) 1.24 0.78, 1.97 1.17 0.70, 1.97
Not cohabiting 264 116 (44%) 1.00 1.00
Cohabiting 537 263 (49%) 1.23 0.91, 1.65 1.15 0.81, 1.64
Not most anxious 469 237 (51%) 1.00 1.00
Most anxiousc 314 135 (43%) 0.74 0.55, 0.98 0.94 0.65, 1.34
Not most depressed 444 237 (53%) 1.00 1.00
Most depressedd 341 137 (40%) 0.59 0.44, 0.78 0.67 0.47, 0.97
Not widespread pain 621 318 (51%) 1.00 1.00
Widespread pain 186 65 (35%) 0.51 0.36, 0.72 0.58 0.39, 0.86
No previous injury 309 133 (43%) 1.00 1.00
Previous injury 468 239 (51%) 1.38 1.03, 1.84 1.48 1.07, 2.05
Not chronic pain 186 94 (51%) 1.00 1.00
Chronic paine 616 287 (47%) 0.85 0.62, 1.19 1.06 0.71, 1.57
Unilateral pain 345 188 (54%) 1.00 1.00
Bilateral pain 456 194 (43%) 0.62 0.47, 0.82 0.72 0.52, 1.00
Not severe pain/disability 229 126 (55%) 1.00 1.00
Severe pain/disability 568 252 (44%) 0.65 0.48, 0.89 0.86 0.59, 1.26
Not frequent consulter 555 276 (50%) 1.00 1.00
Frequent consulterf 252 107 (42%) 0.75 0.55, 1.01 0.82 0.59, 1.15
Abbreviations: CI, confidence interval; HADS, Hospital Anxiety and Depression scale; OR, odds ratio.
a
Unadjusted.
b
Adjusted for other presented variables.
c
Above the top tertile on HADS anxiety scale.
d
Above the top tertile on HADS depression scale.
e
Pain for more than 3 months in last 12.
f
Above the top quintile in total number of consultations in last 12 months.
the text beyond the 18-month study period detected another of these other consultations that did not refer to a knee
98 subjects who had reported a consultation. problem inside the 12-month period. Applying this estimate
In total, 61% of the 424 subjects who stated they had to the 483 subjects leaves 42% (66–24) of the 483 (n 5
consulted and were included in the Part 2 analysis had an 203) with a recorded consultation that probably included
identified consultation in steps 4–7. This increased the per- knee pain and was in the 12-month recall period. Adding
centage of all self-reported consulters with a potential re- these to the 324 people verified in steps 1 and 2 gives
corded knee disorder consultation to 80%. This was a crude estimate of the true rate of annual consultation
compared to 21% of the 420 without a recalled consultation for knee pain among subjects reporting knee pain of 22%
included in the Part 2 analysis and an estimated 26% of all (527 of 2,414). As the recalled consultation rate was
1,607 who did not recall a consultation with evidence of 33%, this suggests that around two-thirds of those who re-
a possible knee-related consultation. called a consultation (22 of 33) are likely to be referring to
There were 483 people in the study population who a definite knee disorder consultation within the 12-month
stated that they had consulted about knee pain in the 12 recall period.
months prior to the survey but who had no record of this
after steps 1 and 2 of the record review. Tables 1 and 3
show that in steps 3 to 7, 318 (66%) of the 483 were found
6. Discussion
to have consulted about things such as ‘‘leg pain’’ or con-
sulted with a knee problem outside the 12-month period. Measures of health service utilization may be derived
A broad assumption can be made that the rate of consulta- from self-reported population data or from general practice
tion in steps 3–7 in those who did not recall a consultation medical records. We have compared the prevalence of self-
for knee pain (24%) provides an estimate of the proportion reported consultation to that recorded in general practice
K. Jordan et al. / Journal of Clinical Epidemiology 59 (2006) 791–797 795
Table 3
Subjects with an identified recorded consultation in medical records
Recalled consultation Recalled no consultation
a b c
Step n Cumulative % Cumulative % na Cumulative %b Cumulative %c
1–3: knee consultation in 18-month study period 383 d 47 d 6
4: leg/lower limb consultation in 18-month study period 94 22 59 32 8 13d
at surgery
5: generalized/widespread pain consultation in 57 36 66 21 13 18d
18-month study period at surgery
6: knee consultation in 18-month study period, nonsurgery 10 38 67 0 13 18d
7: knee consultation before 18-month study period at surgery 98 61 80 34 21 26d
% with a recorded consultation 61 80 21 26d
% without a recorded consultation 39 20 79 74d
Total % 100 (n 5 424) 100 (n 5 807) 100 (n 5 420) 100 (n 5 1607)
a
Number of subjects first identified in medical records at that step.
b
Excluding those ‘‘identified’’ in steps 1–3.
c
Including those ‘‘identified’’ in steps 1–3.
d
Estimated based on all those who did not recall a consultation.
and determined that substantial discrepancy exists. Only number of physician visits in general by elderly people is
40% of self-reported consulters over 1 year could be veri- a consistent finding [11,14,15,27]. However, this bias to-
fied by examining medical records for the identical time pe- wards underreporting has been shown to increase in line
riod for knee problems. In contrast, the rate of incorrect with the number of health care visits [13,27]. By contrast,
reporting of nonconsultation in the present study was very poorer perceived health status and greater history of disease
low; 3% of those who said they had not consulted had have been linked to overreporting of number of consulta-
a knee code or text mentioned in their records. This latter tions [11,15,27]. Coexistence of other health problems
finding is similar to other studies [21,26]. may also lead to underrecording of specific problems by
Error in the reporting of consultations from subject self- the GP when presented with multiple problems at one con-
report may arise from recall bias. Inaccurate recall of infor- tact, particularly if this leads to time pressures within the
mation may occur through problems of memory. In the cur- consultation. In our study, those recalling a consultation
rent study of knee pain, evidence exists for ‘‘telescoping,’’ who had widespread pain and greater depression, according
with 7% of those recalling a consultation having a knee to the self-reported questionnaire data, had lower odds of
code or text in the 6 months prior to the 12-month recall having a recorded consultation than those who were less de-
period. Telescoping may not necessarily be a mistake on pressed or with no widespread pain. In the overall study
the part of the subject; it may represent the patient recalling sample, 57% of respondents with knee pain had pain in
something they feel is important and that they want to con- two or more other joint sites. This suggests that some pa-
vey regardless of whether it took place within the specified tients with knee pain are actually consulting about knee
time frame. The result is a ‘‘bias’’ towards overestimating pain as part of a wider pain spectrum. This is further sub-
the actual consultation frequency. stantiated by the finding that 36% of those who reported
The expansion of the search beyond 18 months suggest a consultation but without a recorded knee consultation
that another possible explanation for discrepancies is had a leg or generalized/widespread pain consultation,
underrecording of chronic knee pain, and this may also ac- compared to just 13% of those who reported no consulta-
count for some of this 7%. If patients have previously con- tion. The association of self-reported bilateral knee pain
sulted with the condition, subsequent consultations may not with no recorded evidence may relate to bilateral pain also
be recorded (particularly if treatment is not provided or being a marker for pain in several joints.
changed). These findings are consistent with a pilot study Overall, 80% of those who said they had consulted for
of 18 patients reporting knee pain that showed reasonable knee pain had a possible true knee consultation. After ad-
agreement between self–reported consultation and paper re- justment for the rates of consultation in those who did
cords when the search was extended from 12 to 36 months not recall a consultation, about two-thirds of recalled con-
[21]. Self-reported duration of the pain in our study was not sultation could be verified. However, there is likely to be
associated with higher recorded consultation in those re- a proportion who did mention knee pain in an unrelated
calling consultation; however, chronicity in the question- consultation (i.e., not for a leg or widespread pain problem)
naire was defined as 3 or more months in the past year, but where knee pain never got recorded. Or they may have
probably too short to affect GP recording. mentioned referred pain from a hip or low back problem
Previous studies have concentrated on agreement on that was only recorded as a hip or low back problem. These
number of physician visits, rather than agreement on are likely to be where knee pain played a less significant
whether any consultation took place. Underreporting of part in the consultation. We cannot estimate the size of this
796 K. Jordan et al. / Journal of Clinical Epidemiology 59 (2006) 791–797
group. The GPs may only code what they see as the major 12-month recall period suggests this is reasonable. How-
problem that is presented in the consultation (and may in- ever, coding of consultations in the practices used may be
terpret the content of the consultation differently to pa- more reliable than coding in other practices due to the au-
tients) and may only code consultations that were dit, feedback, and training in this particular network [18].
consequential in terms of action. The GP medical record This study reports moderate agreement between recalled
may be a better estimate of prevalence of knee pain consul- consultation for knee pain in older adults and recorded con-
tations where the knee pain was a major part of the consul- sultation in general practice records. Researchers and ser-
tation. Self-report may be a better estimate of any vice providers need to be aware of the biases that exist in
consultation in which knee pain was mentioned. Conse- both sources of data when estimating health care use. Pop-
quences of the health care consultation (e.g., prescription ulation-based needs assessment using self-report data will
or referral) have been found to influence level of agreement have to take into account overestimation of contact with
between self-report and medical records in a study about services. Both types of data are required to enable an accu-
cancer screening procedures [28]. rate assessment of health care use in older adults in the gen-
Similar reasons for discrepancies are likely to exist be- eral population.
tween self-reported and practice-recorded consultation for
other conditions apart from knee pain. This is particularly
true for other joint pains, but recall error by the subject Acknowledgments
and lack of specificity, underrecording consultations for
chronic problems, and underrecording of multiple problems We are very grateful to all the survey responders, the
at one contact by GPs will affect agreement between the North Staffordshire and GP Research Network, and admin-
two sources for many conditions. However, knee pain does istration staff at Primary Care Sciences Research Centre,
not have a clearly defined classification system or diagnos- Keele University. This study was funded by an NHS Exec-
tic criteria and symptoms often fluctuate, meaning the im- utive (West Midlands) New Blood Research Training Fel-
pact of the condition can vary over time. This contrasts lowship and the Haywood Rheumatism Research and
with conditions like diabetes and hypertension, where diag- Development Foundation.
nosis is more clear cut; medications and treatments are con-
sistently taken and where people are more likely to be in
contact with services. These issues may all affect patient re- References
call of diagnosis and contact with their general practitioner. [1] Kehoe R, Wu SY, Leske MC, Chylack LT. Comparing self-reported and
Some studies have reported better agreement between self- physician-reported medical history. Am J Epidemiol 1994;139:813–8.
reported diagnosis and medical record diagnosis for cardio- [2] Gross R, Bentur N, Elhayany A, Sherf M, Epstein L. The validity of
vascular diseases than for musculoskeletal conditions self-reports on chronic disease: characteristics of underreporters and
[3,9,29]. implications for the planning of services. Public Health Rev 1996;24:
167–82.
The self-reported data was derived from a population [3] Kriegsman DMW, Penninx BWJH, van Eijk JTM, Boeke AJP,
survey of patients registered at three practices in a GP re- Deeg DJH. Self-reports and general practitioner information on the
search network. The general practices are in a range of so- presence of chronic diseases in community dwelling elderlyda study
cioeconomic areas (a rural affluent town, a semirural mixed on the accuracy of patients self-reports and on determinants of inac-
deprived and affluent area, an urban deprived area) [30]. curacy. J Clin Epidemiol 1996;49:1407–17.
[4] Walker MK, Whincup PH, Shaper AG, Lennon LT, Thomson AG.
The initial survey achieved a 77% response rate in males Validation of patient recall of doctor-diagnosed heart attack and
(75% response), with the younger ages (50–64, 74% re- stroke: a postal questionnaire and record review comparison. Am J
sponse) being slightly less likely to respond. Twenty per- Epidemiol 1998;148:355–61.
cent of survey responders with knee pain were excluded, [5] Klungel OH, de Boer A, Paes AHP, Seidell JC, Bakker A. Cardiovas-
as they did not give consent for review of their medical re- cular diseases and risk factors in a population-based study in The
Netherlands: agreement between questionnaire information and med-
cords or did not complete the question on GP consultation; ical records. Neth J Med 1999;55:177–83.
females (22%) and the oldest age group (751, 24%) were [6] Lampe FC, Walker M, Lennon LT, Whincup PH, Ebrahim S. Validity
slightly more likely to be excluded at this stage. The high of a self-reported history of doctor-diagnosed angina. J Clin Epide-
inclusion rate (80%) and the small differences by gender miol 1999;52:73–81.
and age are unlikely to cause a large bias in the results, [7] Ockander M, Hierppe M, Timpka T. Patient–doctor concordance in
elderly women’s self-reported health and medical records. Methods
and although it could be hypothesized that the oldest ages Inf Med 2002;41:119–24.
may have poorer recall, age was not independently linked [8] Ivers RQ, Cumming RG, Mitchell P, Peduto AJ. The accuracy of self-
with a recorded consultation. reported fractures in older people. J Clin Epidemiol 2002;55:452–7.
For the purposes of analysis we have assumed that codes [9] Simpson CF, Boyd CM, Carlson MC, Grsiwold ME, Guralnik JM,
and text found in the medical records are correct in that Fried LP. Agreement between self-report of disease diagnoses and
medical record validation in disabled older women: factors that mod-
they accurately represent what the GPs wished to enter. ify agreement. J Am Geriatr Soc 2004;52:123–7.
The low percentage (3%) of those without a recalled con- [10] Okura Y, Urban LH, Mahoney DW, Jacobsen SJ, Rodeheffer RJ.
sultation with a knee-related code or text within the Agreement between self-report questionnaires and medical record
K. Jordan et al. / Journal of Clinical Epidemiology 59 (2006) 791–797 797
data was substantial for diabetes, hypertension, myocardial infarction [21] Jinks C, Lewis M, Ong BN, Croft P. A brief screening tool for knee
and stroke but not for heart failure. J Clin Epidemiol 2004;57: pain in primary care. I. Validity and reliability. Rheumatology 2001;
1096–103. 40:528–36.
[11] Cleary PD, Jette AM. The validity of self-reported physician utiliza- [22] Zigmund AS, Snaith RP. The hospital anxiety and depression scale.
tion measures. Med Care 1984;22:796–803. Acta Psychiatr Scand 1983;67:370.
[12] Reynoso L, Cronan T, Foster M. Are self-reports of health care utili- [23] Macfarlane GJ, Croft PR, Schollum J, Silman AJ. Widespread pain: is
zation valid? Ann Behav Med 1995;17(Suppl):S197–211. an improved classification possible? J Rheumatol 1996;23:1628–32.
[13] Roberts RO, Bergstralh EJ, Schmidt L, Jacobsen SJ. Comparison of [24] Bellamy N. WOMAC Osteoarthritis Index. A user’s guide. London,
self-reported and medical record health care utilization measures. Ontario: London Health Services Centre, McMaster University;
J Clin Epidemiol 1996;49:989–95. 1996.
[14] Wallihan DB, Stump TE, Callahan CM. Accuracy of self-reported [25] Jinks C, Jordan K, Croft P. Measuring the population impact of knee
health services use and patterns of care among urban older adults. pain and disability with the Western Ontario and McMaster Univer-
Med Care 1999;37:662–70. sities Osteoarthritis Index (WOMAC). Pain 2002;100:55–64.
[15] Bellón JA, Lardelli P, de Dios Luna J, Delgado A. Validity of self re- [26] Thomas E, Boardman HF, Ogden H, Millson DS, Croft PR. Advice
ported utilisation of primary health care services in an urban popula- and care for headaches: who seeks it, who gives it? Cephalalgia 2004;
tion in Spain. J Epidemiol Community Health 2000;54:544–51. 24:740–52.
[16] Ritter PL, Stewart AL, Kaymaz H, Sobel DS, Block DA, Lorig KR. [27] Glandon GL, Counte MA, Tancredi D. An analysis of physician uti-
Self-reports of health care utilization compared to provider records. lization by elderly persons: systematic differences between self-
J Clin Epidemiol 2001;54:136–41. report and archival information. J Gerontol 1992;47:S245–52.
[17] Jinks C, Jordan K, Ong BN, Croft P. A brief screening tool for knee [28] Gordon NP, Hiatt RA, Lampert DI. Concordance of self-reported data
pain in primary care (KNEST). 2. Results from a survey in the gen- and medical record audit for six cancer screening procedures. J Natl
eral population aged 50 and over. Rheumatology 2004;43:55–61. Cancer Inst 1993;85:566–70.
[18] Porcheret M, Hughes R, Evans D, Jordan K, Whitehurst T, Ogden H, [29] Haapanen N, Miilunpalo S, Pasanen M, Oja P, Vuori I. Agreement
et al. Data quality of general practice electronic health records: the between questionnaire data and medical records of chronic diseases
impact of a program of assessments, feedback, and training. J Am in middle-aged and elderly Finnish men and women. Am J Epidemiol
Med Inform Assoc 2004;11:78–86. 1997;145:762–9.
[19] NHS Information Authority. The clinical terms, version 3 (The Read [30] Department of the Environment, Transport and the Regions. Regen-
Codes). Birmingham: NHS Information Authority; 2000. eration research summary: indices of deprivation 2000. Available
[20] Shrout PE. Measurement reliability and agreement in psychiatry. Stat from URL: http://neighbourhood.statistics.gov.uk (accessed 17th
Methods Med Res 1998;7:301–17. March 2005).
797.e1 K. Jordan et al. / Journal of Clinical Epidemiology 59 (2006) 791–797