The Role of Experience in an Evidence-Based Practice

Please subscribe to use our print features or to download PDF files.

Dr. Guyatt: I’m still thinking that the disagreements are relatively minor. So, for certain you have to look at all the evidence and each clinician’s clinical experience is a form of evidence, and the better documented that clinical experience, the more it would be appropriate to have confidence in it and as we bring in safeguards against bias, then our confidence increases further. So, clinical experience is a form of evidence and all clinical research is the systematic application of clinical observation with safeguards against bias.

So, I think we’d agree, you have to look at all the evidence and perhaps another way of putting our hierarchy is that—you alluded to one example where you were calling for randomized trials—in many situations, we need randomized trials to resolve uncertainties. That implicitly acknowledges that in at least some instances, what we would call evidence lower in the hierarchy, you may say it’s not lower in the hierarchy but it is at least unsatisfactory in resolving the issue, we call in randomized trials to resolve such situations.

Dr. Tonelli: I agree. I think it’s very important to make sure that we frame the question before we ask how best to answer it. I do think that, as many people acknowledge, there are certain beliefs in medicine where we are completely convinced by pathophysiologic rationale, in which we don’t need randomized control trials at this point in time to answer. The value of antibiotics for severe pneumonia, for instance. There are other situations where, frankly, experience and pathophysiologic rationale is not compelling and we need to do something else to answer that question. The question in those situations is generally a very broad one. The question is, is this the best approach as a matter of routine? Is this the best intervention for a particular class of patients?

I think the other difficulty for evidence-based health care is that when we’re asking a question about a particular patient, the patient who’s in front of us, if we’re asking what's the best thing to do for Mr. Jones, that question is not going to be answerable by an appeal to a randomized controlled trial. Then, that necessitates that we bring in a variety of types of medical knowledge, and, as you’ve acknowledged and I clearly recognize, also the patient’s experiences and goals and values into that decision-making process. So, we first have to understand, what is the question. When the question is about what is best for a particular patient, a randomized controlled trial is not going to answer that question.

Dr. Guyatt: Let me understand what you’re saying. Let’s assume we had someone who clearly fits the eligibility criteria of a particular body of evidence, multiple trials appropriately summarized in a systematic review and meta-analysis that give the optimal evidence, that leave us quite confident of the desirable and undesirable consequences of the intervention. Why wouldn’t the results be applicable to the individual?

Dr. Tonelli: I think the question for the clinician is always, are they applicable and is there something about this particular individual that would make me concerned that they’re not applicable? That might be something that the study didn’t take into account or wasn’t part of the inclusion or exclusion criteria. Certainly, I think you would acknowledge that it could be that the patient doesn’t value the outcome of the trial.

So, it doesn’t make sense to apply that intervention. It may be that there’s something about what I’m able to provide in this locality that may impact the decision. So, there are still issues that need to be considered about whether those study results are going to be compelling enough to determine the care for this patient. It’s never, and I think you would agree, that one can simply directly and deductively apply the results of even a body of evidence to an individual patient.

Dr. Guyatt: I would translate what you’re saying not as you can not apply the results of a body of evidence to a particular patient but rather that before you apply the results of a body to a patient, you should make sure that in fact it’s appropriate to apply to that individual. You’ve stated it very nicely. You have a body of evidence and you start out by saying, is there something about this individual that would make it unlikely or that would make me have serious questions that the results from this body of evidence do in fact apply to the individual?

Secondly, you may then ask whether there is something about our situation and circumstance that although, if we were in the right setting, the results would apply to our patient, something about our setting means that we cannot apply the intervention as it was done in the studies or something else peculiar about the setting makes us hesitate. I agree we should always do that. And third, which you point out, again, very appropriately, I would frame it a little bit differently. You would say the results don’t apply to the patient because the patient puts a very low value on the beneficial outcomes that were identified and puts a high value on inconvenience or burden associated with the intervention.

I would argue that the results still apply to the patient. They give us our best estimate of the desirable and undesirable consequences, and the way to apply it to the patient is that we can show the results to the patient and the patient can say, that applies to me, but I am not interested now that I understand the best estimates of the desirable and undesirable consequences. The way I’d reframe what you said is you have to ask the question of whether these results can be applied to the individual because of the characteristics of the patient or the setting. Then, if the answer is yes, you have to ask, how can the results best be interpreted in the context of this particular patient’s values and preferences? The answer may frequently be yes. It applies very directly to the patient. It gives the best estimates of what we can expect in this individual and then you go about applying values and preferences.

Dr. Guyatt: I’ll give my summary, but then you might give a different account. We have established that there are instances in which randomized trials are needed to resolve issues where uncertainty exists as a result of prior evidence and clinical experience and physiologic reasoning, and we also agree that one needs to be skeptical about the results of randomized trials, and that prior knowledge bears on how we will interpret the results of randomized trials. I am more inclined to say that it is worth thinking about this as a hierarchy and Dr. Tonelli much less inclined to say it is worth thinking about as a hierarchy.

We agree that anytime you have the results of a randomized trial, applying that to an individual patient should not be done as an automatic process. Rather, it is necessary to take a serious look asking, is there some way the results are not applicable to this patient? Is there something about the setting that makes the results not applicable to the situation, and then the context in terms of the patient’s values and preferences, that the results may be interpreted very differently in treating one patient and not another because of differences in values and preferences? I’d be inclined to characterize the situation that the results of trials are often directly applicable to the patients, but that applicability should always be questioned seriously, and you are more inclined to characterize it as the results, the trials are never applicable to individuals. Dr. Tonelli, correct me where I’ve gone astray.

TABLE.Areas A, B and D: “... represent distinct forms of knowledge, differing in kind from one another. Each has particular strengths and weaknesses when applied to clinical decisions. Since they differ in kind, they cannot be ranked or placed in a hierarchy." Tonelli MR. Ann 1st Super Sanita. 2011;47(1):26–30;27.

Modified from Tonelli MR. Integrating evidence into clinical practice: an alternative to evidence-based medicine approaches. J Eval Clin Prac. 2006;12(3):248–256.

Dr. Tonelli: I never said that the results of clinical trials are never applicable or are never informative. That would be an indefensible position. What I said is that clinical research can never be directly or deductively applied to individual patients. That this has to be an active process that incorporates other forms of medical knowledge.

I think I view it a little differently than you do. In the User’s Guide you describe a sequential process as opposed to the more complex reasoning that I think occurs. The role of the individual physician is to take into account both the clinical research and anything about this particular individual that may be different in terms of physiology or some aspect of the clinician’s experience that suggests that this patient may not respond in a way that clinical evidence would suggest. I think it’s the role of the physician as well to elicit and put into context the patient’s experiences and their goals and values and understand the system’s features that are relevant to the decision. I believe that’s all one step. It’s not a sequential process.

The clinician is going to come to some conclusion about what is probably the best course of action for this individual patient. I think that there is an important distinction here, that the focus throughout the entire process is really on this particular individual. What clinical research you bring into it depends on the individual as opposed to looking at the individual and then heading out into the world of published randomized controlled trials to try to see which one is most appropriate to bring back. So, I do think we both acknowledge that clinical research is not directly applicable to the care of an individual patient. There needs to be some process, although I think we disagree a little bit on what that process looks like.

Dr. Guyatt: Seeing the way you put it there, I’m not sure that there’s anything I would disagree with, but let me clarify—at the end you said that there is a problem with taking the patient, defining the patient’s problems and then going and seeking the best available evidence. Did I misunderstand you? Is there something wrong with saying okay, here is the patient and here’s the patient’s problem? Here are the alternatives we are considering to deliver optimal care. I better be aware of the best available evidence that bears on what the effects of the intervention are likely to be in this patient.

Dr. Tonelli: No, I don’t disagree with that. I think one better be aware of what clinical research could be relevant to a particular decision. I think what I’m trying to point out is that there is this subtle distinction between looking at a patient and framing a question, a general question, that can be answered by going to the clinical research and then applying that back to the patient. Does that make sense? Because what I don’t want to do when I’m dealing with individuals is to define a general question.

The question before the clinician is still, what is best to do for this particular patient? What’s the best thing to do for Mrs. Jones? Not necessarily, what’s the best thing to do for a patient who’s had a previous myocardial infarction and has diabetes and is now presenting with worsening heart failure. That general question is more easily answerable by going to the medical literature, but it takes me one step away from what the real question is, which is, what’s the best thing to do for this particular patient?

Dr. Guyatt: See what you think of this. The way we frame it is as a way of helping clinicians go through the process and, as we have agreed, the clinician needs to find, and understand the best available evidence to apply to the individual. So, let’s assume in our paradigmatic situations, we say the clinician suspects she doesn’t know the best evidence that’s available and concludes she better find out.

If one is talking about enabling or helping or making optimal the physician’s process, we believe that it is extremely helpful to start out with a structured clinical question that is the first step in entering into the search for the optimal information. I often teach people to frame their question: for instance, a 60-year-old hypertensive, hyperlipidemic, nondiabetic man, that’s our patient. Well, if we demand only studies that include patients with all those characteristics, we know in advance we aren't going to find any studies. That’s probably not the most helpful way of doing it.