The Role of Experience in an Evidence-Based Practice

Please subscribe to use our print features or to download PDF files.

So, let’s pick the key characteristics of the individual to define our patients that would make it likely that we’ll actually find some systematically conducted evidence that bears on the patient, and similarly with the intervention, comparators, and outcomes. One then goes out having constructed a question in that way, finds the best evidence, and then comes back with the issue of the applicability. So, it turns out that what we find is more or, perhaps as you point out, less applicable to the individual, and the evidence from formal studies in the literature then provides indirect evidence with respect to this individual. Yes, it may apply, but no, it may not.

Then, you need to make a decision considering the uncertainty that has been generated by these issues of applicability. So, if we’re framing it as a sequence, it is designed for the situation when the clinician is uncertain of what the best evidence available is and needs to be systematic about going about attaining that evidence and then applying it back to the patient.

Dr. Tonelli: I think we would agree that in modern medical practice that it’s an absolute necessity for optimal practice that clinicians be aware of what relevant clinical research has to say about issues pertinent to individual patients. I think the process that you describe in terms of how one might go about that is certainly one reasonable approach. My hope, again, when we talk about taking care of individual patients, is that most clinicians will be able to bring into the physician-patient relationship that information without having to go through this entire process. Maybe that’s more likely for some specialists than generalists, I don’t know, but certainly you want physicians to be able to bring that information into the deliberation. It’s an important element, but then you still need to be able to, as you note, start looking at issues around other characteristics of the individual, whether they’re pathophysiologic or things that the clinician believes based on experience may be relevant to the question at hand and try to weigh those.

I think that’s a challenging process, that aspect of clinical judgment and, in terms of emphasis, I think that the evidence-based medicine movement has emphasized the process you’ve described. Again, I’m not stating that it’s unimportant or that one need not be able to understand the clinical research, but a lot of emphasis has been on the clinical research, understanding it and critically appraising it. Very little attention has been given to how individual clinicians can appropriately make these decisions when caring for particular patients, how the other types of features factor in. As you say, you acknowledge that they’re important, but we haven’t spent a lot of time talking about how that works and how clinicians can do it better.

Dr. Guyatt: Well, I would argue that if I look back at our writings, and perhaps it’s a historical phenomenon, in the first decade of evidence-based practice, that might be true, but subsequently, we’ve written a lot about the issues of applicability and gone back to talk about the situations of patients—what sort of patient differences, what sort of intervention, what sort of outcome differences you might take into consideration with respect, for instance, to surrogate outcomes. So, maybe we still haven’t got the optimal balance, but I would also ask, who’s done better in terms of writing about the real challenges of saying, here is what the published evidence, the formal research evidence, shows. What exactly should this process be? How can we provide guides for clinicians in doing the applicability exercise, the importance of which you, in my mind, completely appropriately laid out?

Dr. Tonelli: Two things. First, I agree with you that the notion of EBM in practice has clearly evolved over time and, as you note, a very different focus over the past several years has emerged than over the initial 10 years where other forms of medical knowledge were actively deemphasized by evidence-based medicine. I agree with that evolution and I think that’s very appropriate. I think in particular there are thoughtful proponents of the EBM who, like yourself, really get this and are interested in trying to help clinicians with that difficult issue.

Second, I think there are many other people who write about this from other perspectives, from a patient-centered medicine or a person-centered medicine approach who are particularly interested in ways to incorporate patient experience, goals, and values, into the clinician’s decision-making process. Frankly, I think there hasn’t been a lot of attention on trying to define for clinicians what are legitimate reasons, for instance, to not follow a practice guideline or what are illegitimate reasons not to follow a practice guideline and trying to lay that out. I think that’s a lot of the work going forward.

You mentioned GRADE earlier, and I think in terms of guideline development, GRADE has clearly started to say look, there are things that are important that go beyond the study design in terms of the strength of a recommendation. I think clinical practice is one step further away from guideline development and adds layers of complexity that you cannot account for when you’re doing guidelines, and I hope to see that this discussion continues to move closer to the clinician. I do think that there are a variety of features of clinical research that make that research more or less compelling to clinicians and we should really be discussing which of those are legitimate and which are illegitimate and how we can help clinicians with this decision-making process. The critical care community has begun thinking about these questions.5

A related challenge for evidence-based medicine centers on helping clinicians to decide what to do with patients who would never have been eligible for a clinical trial, perhaps because of age or co-morbidities. Your thoughts?

Dr. Guyatt: Well, two things. Number one, you would have to have—as you have pointed out—a question that is foremost in individual’s minds when they look at the formal research evidence that there may be reasons not to apply it to the individual and that the patient may not exactly meet the eligibility criteria is very important. We think that there has been excessive skepticism in that when you look for subgroup effects within the body of evidence that does exist, within a meta-analysis situation, for instance, different trials will have varying eligibility criteria. There are actually very few instances of subgroup effects that have been substantiated.

So, as it turns out, there are certainly instances in which this is not true, but there are many instances in which some characteristic of the patient that kept them out of the trials will not actually influence the magnitude of the patient’s response to treatment. I think you would support this, what you need to look at is physiologic rationale and your understanding of the individual’s physiology. The trial was conducted in Caucasians, but is atherosclerosis really different in non-Caucasians so that baseline risk may differ? Is the relative effect of the treatment going to differ?

Yes, the trial enrolled people up to 65 and our patient is 66, but is the physiology of the 66-year-old really that different from the physiology of the 65-year-old and so on? So, yes, it’s important to think about, but we think clinicians would be best served by a starting point of saying, yes it is applicable, and then finding reasons to challenge that, and if there is compelling physiologic rationale, then one loses confidence and then the next thing is to say, okay, what then is the best rationale—once we have lost confidence, we’re less confident—what are the implications of this loss of confidence in the management of the patient?

Dr. Tonelli: I think I would largely agree with that. I think that with any set of clinical research, we’re always trying to see how informative that is for a particular patient and we’re forced to ask these questions. Again, throwing out the information from clinical research simply because the patient in front of us would have met some exclusion criteria does not seem appropriate. Instead, one has to ask whether the information from the research is relevant to this patient, often from a physiologic stance.

I think the other problem related to this issue is the idea that when we start to have patients who unfortunately, for instance, have a large number of comorbidities for which there are evidence-based guidelines regarding “best practice.” Such to the point that you have a patient who, if we followed the guidelines, would be on 15 or 18 medications and we start to wonder if that is really the best care because the studies often exclude patients with comorbidities and yet we know patients have multiple diseases.

I think that’s another space where the clinician is then forced to say, wait a minute, and, along with the patient’s input, what are the priorities here? What are we trying to accomplish? We’re not going to be able to follow all of these guidelines. That’s going to cause more harm than good. So, what do we opt into and what do we opt out of? I think that’s a related challenge.

Dr. Guyatt: I completely agree with that and perhaps, ironically, my perception is that it’s the leaders of the EBM movement who, in contrast to content area experts, are saying, wait a minute here, we need to consider what is the burden we are placing on the patient’s life as in the dramatic situation just depicted regarding 15 different medications taken at different times of day and lifestyle changes we want the patient to make and everything else we want the patient to do. I believe we have taken the lead in pointing out the role of values and preferences and the fact that this may not be at all in the patient’s best interest.

Dr. Tonelli: I guess the takeaway points from my perspective are that clinicians recognize that the question they’re trying to answer revolves around the individual patient they’re dealing with and that they are required to incorporate medical knowledge from clinical research, from their personal experience, and from pathophysiologic understanding into their calculation about what is likely best for the patient. The physician also bears the responsibility of bringing in the patient’s experiences, goals, and values, and the local systems features into that process. I assert that there is no hierarchy of evidence for individual clinical decisions and that physicians should not make the mistake of believing that clinical research always trumps their pathophysiologic understanding or their experience.

Dr. Guyatt: I will agree. What Dr. Tonelli has just said captures very much the way the evidence-based practice community currently thinks about these things, and when we are teaching it, one of the mantras we have for people teaching evidence-based practices is start with the patient, finish with the patient. In the interval, what is included is a journey through understanding the best available evidence, which is sometimes challenging, and we spend a lot of our time and energy guiding people on how to do that. Nevertheless, as Dr. Tonelli has just expressed, there is a necessity of starting with the patient, finishing with the patient.

The only area where there is some disagreement, and I actually don’t think this agreement is conceptual, would have to do with what is most useful. Is it useful to highlight the fact that there are situations, many situations, in which the study designs that guard against risk of bias will lead to stronger inferences than study designs that don’t guard against risk of bias and unsystematic clinical observations that don’t guard against risk of bias? Is it useful to point out to people that, in general, the situations where risk of bias is less leave us more confident than situations in which risk of bias is greater. This implies a hierarchy. We continue to think that this is a useful way of thinking about it while we would fully acknowledge that randomized trials can often be misleading, and, as I said earlier, a lot of our current energy is in pointing out those sort of situations.

Dr. Tonelli: I think that those statements summarize the discussion well. I agree that it is the individual patient who is the focus, and would also agree with one of the points from earlier on in the discussion that “beneficent” skepticism is not misleading people but is just a prompt to assess the evidence for yourself with the patient in front of you as the focus.