Abstract

Introduction

Health inequalities—disparities observed across gender, race, and socioeconomic boundaries—are an ongoing source of cost and worry in arenas of public health, healthcare delivery, and health policy. Differences in clinical decision making (CDM) have been observed for many conditions, ranging from coronary heart disease [1–5] to schizophrenia [6, 7], and for several types of clinical decisions, such as making diagnoses, ordering tests [1, 2, 5], writing prescriptions [8, 9], and assessing patient adherence [10, 11]. Historically, explanations for the sources of these gradients have focused on individual patient factors such as health care access and utilization, health-related behaviors, and home and neighborhood environments. More recently, attention has expanded from patients to include healthcare providers and doctor-patient interaction as possible sources of inequality [12–17].

A sizeable literature has developed that focuses on physicians’ social and psychological information processing during the patient-provider encounter as an important source of observed variation in CDM behavior. Previous work has evaluated how prejudice, stereotyping, and discrimination can affect providers’ assessments of patients and thereby influence treatment decisions [14, 15, 18–23]. Even unprejudiced physicians may make biased clinical decisions, however, and the concept of uncertainty is used to describe another type of unintentional, implicit bias physicians may have in the process of cognitively organizing information. Balsa and McGuire (2003, 2005), for example, use the term “statistical discrimination” to describe bias resulting from prior probabilities (epidemiologic base rates) overwhelming presenting data (individual patients) when physicians are uncertain. Empirically, previous studies have largely focused on (1) which patient characteristics lead physicians to be most biased, with particular attention to racial differences [24], (2) which patient-physician pairs lead to the greatest uncertainty, especially whether racially-matched dyads have better outcomes [25], and (3) distinguishing between types of cognitive processes driving these differences, particularly evidence of prejudice versus uncertainty [11, 24].

Implicit in this body of work is an assumption that bias in clinical decision making is primarily a function of patient characteristics, physician attributes, and how various combinations potentially influence physicians’ cognitive processing. That is, if physicians treat patients unequally, it is due to a shortcoming in the collection and processing of patient information, not in the decisions they make once they have assessed patients. We extend this line of work by examining physicians’ diagnostic certainty. We find that certainty is not only associated with the process of assessing and diagnosing patients, as has been shown in previous studies, but also exerts an independent influence on physicians’ subsequent clinical actions. The implications of these results are critical, as they imply that the “fix” for variations in CDM may lie not only with reducing physicians’ uncertainty with specific types of patients, but also in reducing variations in how they respond clinically to their own uncertainty.

Using data from a video vignette factorial experiment, we address the following research questions: (1) How certain are physicians of their diagnoses for conditions presented by two patients? (2) Which types of physicians have the highest diagnostic certainty? (3) Which types of patients elicit the highest certainty levels among physicians? (4) Are specific types of physicians more certain with specific types of patients? (5) How does certainty about diagnosis affect subsequent patient management, such as information seeking, test ordering, prescribing, lifestyle recommendations, and referrals/follow-up? (6) How do these processes vary according to the patients’ presenting conditions (coronary heart disease [CHD] vs. depression)?

Materials and Methods

We used a factorial experiment to simultaneously measure the effects of: (a) patient attributes (age, gender, race and socioeconomic status) and (b) physician characteristics (gender and years of clinical experience) on physician diagnostic certainty and subsequent medical decision making when providers are presented with identical signs and symptoms strongly suggestive of two common medical problems—coronary heart disease (CHD) and depression (study details are described in McKinlay et al 2006). A full factorial of 2⁴ = 16 combinations of patient age (55 vs. 75), gender, race (White vs. Black) and socioeconomic status (SES) (lower vs. higher socioeconomic status, depicted by current or former employment as a janitor or a teacher) was used for the video scenarios. One of the 16 combinations was shown to each physician for each medical problem (2 videos per physician). The vignette pairs were assigned such that each vignette depicted opposite social characteristics for each medical condition (for example, if the depression vignette showed an older, black, low SES female, the CHD vignette that physician was shown would be a younger, white, high SES male). This random ordering was designed to avoid confounding by the order of presentation or by the social position of the “patient.” IRB approval was obtained for the study, and signed informed consents were collected from each participating physician.

The medical conditions (CHD and depression) were selected because: a) they are among the most common and costly problems presented by older patients to primary care providers [26]; b) they represent examples of a well-defined organic medical condition and of a less-well-defined psychosocial phenomenon; and c) they admit a range of diagnostic, therapeutic and lifestyle actions. An advantage of videotapes (over written scenarios) is that potentially relevant nonverbal indicators (e.g., the “Levine fist” for CHD, or a dejected appearance for depression) can be embedded in the presentation. Scripts for the two medical problems were developed from several tape-recorded role-playing sessions with experienced clinical advisors. Patients in the CHD vignette presented with symptoms suggestive of CHD (including, for example, chest pain worsening with exertion, pain in the back between the shoulder blades, stress and elevated blood pressure). The depressed patient presented with six of the seven classic symptoms of depression (sleep disturbance, decreased interest, guilt, reduced energy, inability to concentrate, poor appetite and psychomotor retardation). Suicidal ideation was omitted as too indicative of the diagnosis [27]. Professional actors were selected for their comparability (in weight, attractiveness, etc.) and trained (under experienced physician supervision) to realistically and consistently portray a patient presenting with the signs/symptoms of disease to a primary care provider. Previous studies have used similar methods with success [8, 28–32].

After viewing the videotaped vignette, physicians were asked, “What do you think is going on with this patient?”, and for each possibility, they were asked for their level of certainty on a scale of 0–100. They were also asked how they would treat the patient in terms of asking for additional information, performing physical examinations, ordering tests, prescribing medications, giving lifestyle advice, and referring to other physicians.

The original study included data from the United States and United Kingdom (as addressed in McKinlay et al 2006), but the present analyses focus only on the US data. To be eligible for selection, physicians had to: (a) be internists or family practitioners; (b) have ≤12 years clinical experience (graduated between 1989–96) or ≥22 years experience (graduated between 1965–79) in order to get clear separation by level of experience; (c) be trained at an accredited medical school in the US (no foreign medical graduates were included); and (d) be currently working in Massachusetts as physicians more than half-time. Screening telephone calls were conducted to identify eligible subjects and an appointment was scheduled for a one-hour long in-person, one-on-one, structured interview. The required 128 interviews (16 pairs of vignettes × 2 physician genders × 2 physician levels of experience × 2 replications) were conducted over a period of nine months in 2001–2. Each physician subject was provided a modest stipend ($100) to partially offset lost revenue and to acknowledge their participation. The response rate was 64.9% (completed appointments as a percentage of eligible physicians). Interviewers were carefully trained and certified, and quality control interviews were conducted and selected tape-recorded interviews were reviewed by supervisors on a regular basis. Physicians were blind to study hypotheses.

Every study represents a balance between internal and external validity. Because they are deliberately designed to control unwanted variability, experiments in particular are necessarily conducted under special circumstances or in an “unreal” setting. Determining whether findings from this “unreal” world of experiments are applicable to the “real world” of everyday social behavior is a perennial challenge to behavioral researchers. For the present study, external validity concerns include not only whether physicians would diagnose vignette patients the same way they would actual patients, but also whether they would take the same clinical actions under experimental conditions as in their regular practice. In numerous peer-reviewed publications over a decade we [9, 31, 33] and other investigators [34–49] have presented results which show clinical vignettes work in clinical settings (i.e., they produce unconfounded estimates of the influence of systematically manipulated variables). In medical decision making, vignettes have been validated for predicting variation in the quality of preventive care [35] and in measuring the quality of physician practice [50]. In a direct comparison of vignettes, standardized patients, and chart abstraction, Peabody and colleagues [51] found that vignettes were a valid and comprehensive method for measuring quality of outpatient care, and that vignettes consistently produced results that were closer to standardized patients than chart abstraction results were. Additional studies comparing vignettes with standardized patients and other methods corroborate the result that vignettes are ecologically valid for studies of medical decision making [35, 47, 52, 53].

For present purposes, a critical benefit of vignettes is that they allow for the manipulation of several variables at once and the measurement of unconfounded effects, thereby “isolating physicians’ decision making from other factors in the environment” [53]. While standardized patients are considered especially useful for measuring real-time communication and physical examination skills [54], these topics are beyond the scope of our research questions. More importantly, such an approach would not permit exacting experimental control (absolutely identical presentations). We took four precautionary steps in an attempt to minimize possible threats to external validity (i.e., that physicians may behave differently with a videotaped patient under experimental conditions compared with real patients in an everyday clinical setting). First, considerable effort was devoted to ensuring the clinical authenticity of the videotaped presentation—that is, to develop a script and videotaped vignette that was a clinically authentic presentation of the symptoms (e.g., CHD or depression) and the range of patients presenting them (e.g., men, women, black, white, older, younger). While there may be real-life differences in how various types of patients tend to present the same conditions, our vignette development process was designed to result in a presentation that would be plausible for the range of patient characteristics presented in the study. This was achieved by basing the scripts on clinical experience, filming with experienced clinicians present, and by using professional actors/actresses. Second, the subjects (doctors) were specifically asked how typical the patient viewed on the videotape was compared with patients they encounter in everyday practice (92% considered them either very typical or reasonably typical). Third, the doctors viewed the tapes in the context of their practice day (not at a professional meeting, a course update, or in their home) so that it was likely they encountered real patients before and after they viewed the patient in the videotape. Fourth, the doctors were specifically instructed at the outset to view the patient as one of their own patients and to respond as they would typically respond in their own practice.

Analysis of variance was used to test the main effects and two-way interactions of the design variables (patient gender, race, age, and SES, and physician gender and level of experience) on the diagnostic certainty (0–100, with 0 for complete uncertainty and 100 for complete certainty). If the physician did not make a CHD or depression diagnosis, his or her certainty for these diagnoses was set to 0. The balanced factorial design allows the unconfounded estimation of all main effects and two-way interactions using analysis of variance. Furthermore, the sample size of 128 allows us to detect a difference in certainty of 15 points with 80% power (that is, a true 15 point difference in certainty between two groups will be detected 80 percent of the time at α = 0.05). Because the experiment was replicated, a pure error term with 128 degrees of freedom was used to test all effects using analysis of variance. To determine the effect of certainty on clinical decision making, logistic regression was used for dichotomous variables (e.g., whether or not an EKG was ordered) and analysis of covariance was used for continuous variables (e.g., number of days to next appointment). Each model included as explanatory variables the design variables, certainty, and the interaction of the design variables and certainty. Using backwards elimination, non-significant effects (at the 0.05 level) other than certainty were removed from the model, leaving a parsimonious model. A paired t-test was used to test the difference between the certainty for the CHD vignette and the depression vignette. We acknowledge the limitations of multiple testing, but note that we observe consistency across the two conditions, and that the results we observe at the p < .01 level are unlikely to change. To facilitate interpretation, we present actual p-values, unadjusted for multiple testing, as this approach allows readers to choose their preferred level of significance.

Results

DISCUSSION

We conducted a factorial experiment to derive unconfounded estimates of the simultaneous effects of patient and provider characteristics on the clinical decision-making process for two common diagnoses: CHD and depression. While the vast majority of physicians successfully diagnosed the conditions depicted in the vignettes, there was significant variation in their diagnostic certainty, which was the focus of our analyses. We found that diagnostic certainty was influenced by patient race and gender for CHD but not for depression, and that for both conditions the degree of diagnostic certainty directly influenced subsequent therapeutic actions.

The scope of this study required the exclusion of some topics that could provide a springboard for future research in this area. For various logistical reasons, the study does not include all the physician information that could potentially be of interest. Physician gender and level of experience were included as design factors because previous research shows that they are relevant to clinical decision making. While it would have been ideal to also vary physician race/ethnicity as a factor in the experimental design, the small numbers of such physicians available within the defined specialties, strata, and geographic area precluded this design. We therefore randomly sampled the specialties selected so that a representative sample of minority physicians would be included, thus mirroring the actual availability of racial and ethnic minority physicians available to the patient population. For similar reasons physician recruitment was random with regard to the type of healthcare setting in which a provider worked, and the interview did not collect information about the racial composition of physicians’ typical patient caseloads or the diagnostic and treatment facilities available to patients.

Due to the experimental design of this study, its primary focus was on how physicians respond to the standardized stimulus of the patient in the vignette. For this reason, as discussed above, significant resources were invested in the development of a clinically authentic vignette. Furthermore, by randomly assigning physicians to a specific combination of patient characteristics in a given vignette, any physician differences based on the above factors that influenced selection into the study would also be randomly distributed in the results and therefore not systematically bias the results. Similarly, while we do not have information about differences between those who chose to participate in the study and those who declined, the random assignment of physicians to vignettes assures that selection biases do not affect the internal validity of the experiment. This missing information detracts from the generalizability of the results. Future studies, however, could complement the present study by examining the influenced of such influences on clinical decisions.

Based on these results, we identify three sets of policy and research implications. Our findings are consistent with earlier research showing that physicians have decreased certainty with some types of patients. However, we expand this earlier work by also showing that certainty also has an independent influence on subsequent clinical actions. The robustness of certainty in these models, particularly the differential tolerance of uncertainty exhibited by physicians, implies that the policy strategy of reducing gender- or race/ethnic differences in physicians’ clinical assessments is not sufficient to reduce overall disparities. While two physicians may be trained to have the same diagnostic certainty for a given type patient (say a younger female), our results show that their responses to that certainty are different, so that the same younger female patient may not receive treatment if she is seen by a physician whose threshold for clinical action is higher than his counterpart.

Therefore, a first policy implication is that, in order to achieve policy goals concerning the reduction of disparities, we need to understand and standardize variation in physicians’ clinical responses to uncertainty (regardless of who the patient is). Such varied responses to uncertainty should be treated as a separate potential source of unequal treatment. Reaching this goal may entail deeper examination of sociological, cultural, economic, and organizational contexts that shape decision making. This type of approach is commensurate with Fennell’s [17] call to look beyond the clinical encounter and consider the larger systems in which healthcare is structured. Intervention strategies may include establishing organizational motivators (such as insurance reimbursement) to encourage physicians to take clinical action even if their certainty is low, at least for potentially life-threatening conditions such as CHD. A second, clinical, policy implication is that if physicians are aware of this source of bias, they can make a conscious decision to be more liberal in their ordering of test and prescriptions that would identify patients who are at risk—rather than relying on high certainty levels to trigger clinical action, they could consider test ordering even when their certainty is lower. This strategy may help circumvent persistent difficulties in trying to teach physicians to eliminate their differential diagnostic uncertainty.

Physicians had a higher level of certainty for their diagnoses of depression relative to CHD. While it is difficult to compare certainty across different case vignettes, our results are consistent with the concept that physicians may be hesitant to commit to a single diagnosis (e.g. CHD) at the risk of missing other life-threatening conditions. In the case of the chest pain symptoms, the underlying condition is localized to one of several anatomical systems (cardiac, pulmonary, gastrointestinal, etc.) and therefore implicates several critical alternative diagnoses (e.g., pulmonary embolism, aortic dissection). In contrast, the relatively more benign course of depression, the less localized symptoms, and the decreased risk of incorrect diagnosis compared to the case of chest pain, may explain the greater certainty physicians were able to assign to their depression vignette diagnoses. This result implies that certainty varies differentially across conditions, and therefore physician uncertainty may be more of a problem for conditions with increased risk for catastrophic outcomes or with a greater number of alternative diagnoses. As a result, a third policy-related implication is that our CHD-related strategies should also be considered for other potentially catastrophic conditions; clinically and organizationally, physicians should be encouraged to take preventive action for these conditions even when their certainty is low, recognizing that low certainty may reflect biased responses to individual patients more than true physiological differences.

Finally, these results suggest possible limitations of clinical practice guidelines. Prior studies of CDM have demonstrated that when physicians are less certain, they are more likely to invoke stereotypical information—particularly knowledge associated with patients’ social categories—to help fill the gaps in their knowledge with information that may be relevant to diagnostic and treatment decisions [19, 24, 55, 56]. This is most likely to occur when the decision is complex or ambiguous, and the information is perceived as relevant to the clinical decision [55]. While physicians may draw from cultural stereotypes to supplement portions of missing information, “evidence-based” stereotypes (e.g., base rates) may be particularly likely to influence treatment decisions because they are seen as clinically relevant, legitimate sources of information, and providers are encouraged to use them under the rubric of Bayesian decision making [55]. Following from ethnographic research showing that evidence-based guidelines may actually morph through usage into collectively constructed “mindlines,” [57] we are concerned that guidelines may have the unintended potential to amplify and legitimate existing physician bias.

Acknowledgments

Footnotes

References