Seeking Better Information
I think we need to stop using research papers as ammunition. In my opinion, research gives us information, not answers. A single paper, or even a well-done meta-analysis, does not provide an answer. It simply gives us more information. If you ever think you’ve finally solved the orthodontic puzzle, you’re always going to be wrong. You can’t ever stop thinking.
In that light, let’s investigate Bayes’ Theorem vs. Classical Statistics. Bayes’ Theorem is quite simple. Especially when it involves only two conditions (A and B). The easiest way to interpret Bayes’ formula is to think of it as the probability of condition A, given that condition B already exists, or has occurred. For example, the probability that you will live to be as old as the average human (72 y.o.) is different based on how old you currently are. That is, if you were just born, the likelihood you will live until 72 is lower, than if you are age 70.
We can test two variables quite easily. But healthcare is not that simple. Let’s use orthodontics as an example. As the conditions (or variables) increase, it becomes harder to measure treatment effect.
Let’s say you’d like to study the efficacy of Phase I treatment. Imagine that you’d like to use your study to decide if Phase I treatment is warranted for your particular patient, Jane Swanson. You’d obviously like to study a sample that matches as many of the conditions of Jane Swanson as possible (her age, sex, height, weight, eating habits, dental history, environment, sleep habits, allergies, cooperation expectation, dental shape, jaw size, jaw width, jaw length, TMJ anatomy, muscle anatomy, crowding, spacing, incisor position, incisor angle, etc, etc, etc…). Sound simple?
OK. Next, in order to study treatment effect, we need to be sure that our sample contains all of the treatment conditions that apply to treating Jane in OUR individual practice (appliance type, appliance duration, appliance activation, appliance hours, motivation techniques, frequency of visits, patient instructions, bracket prescription, bracket type, wire sequence, bonding protocol, ligation technique, assistant’s skill-level, etc, etc, etc…). Still simple, right?
OK. Finally, all we need to do is identify those conditions that define an “effective” outcome (facial esthetics, alignment, smile esthetics, stability, need for Phase II, shorter duration of Phase II, molar position, molar rotation, incisor position, overbite, overjet, incisal angle, periodontal health, airway health, extractions needed, etc, etc, etc…).
OK. You can see where this is going. Attempting to match a sample to all of the conditions of Jane Swanson, treated in OUR practice, to OUR satisfaction, would be next to impossible. Unfortunately, we often wrongly assume that most published studies have done this condition-matching by using “random” sampling. Sadly, true condition-matching rarely, if ever, happens. It is next to impossible. Because of this, the justification for using most studies as “evidence” for how to treat Jane Swanson is often significantly flawed. Not always, but usually. In the future, the majority of our current publications will act simply as signposts, merely there to help us identify what we might look for (or measure) in Jane Swanson. But unfortunately, much of our current research gives us little information about how to treat Jane Swanson.
In order to understand how to best treat Jane Swanson, we will need to study Jane Swanson. Very thoroughly. Once we’ve done that, when we identify the next patient who closely matches Jane’s attributes (conditions), we will make better decisions about our new patient.
So, why is understanding Bayes’ theorem so important for orthodontists? Because practicing evidence-based orthodontics seems to be very important to all of us. The problem is that we are often too quick to judge p-values as evidence. Let me give you a final example: According to a recent study, when appliance “X” was used, lower incisors proclined by 5-degrees, with a statistically significant p-value of 0.01. Most orthodontists’ logical conclusion from this study would be that appliance “X” proclines lower incisors by 5-degrees. So, if I’m an orthodontist, and I don’t want to procline lower incisors by 5-degrees, I better not use applied “X”. Unfortunately, this would be a terribly uninformed and short-sighted decision.
A recent article in Scientific American reminds us, “Embedded in Bayes’ theorem is a moral message: If you aren’t scrupulous in seeking alternative explanations for your evidence, the evidence will just confirm what you already believe. Scientists often fail to heed this dictum, which helps explain why so many scientific claims turn out to be erroneous.”
Remember, CONDITIONS are crucial for the appropriate application of Bayesian principles. That means, always ask the following questions before judging conclusions: Does this apply to MY technique? Does this apply to MY patient? Does this apply to MY treatment goal? You will find that most studies rarely do. This is why I believe orthodontic research must change. Averages and p-values rarely apply to what YOU are doing in YOUR practice. Studying the details of your individual cases will allow for a much greater understanding of what it is YOU do. Sharing this information will allow all of us to benefit from the application of multiple CONDITIONAL values. If this doesn’t quite makes sense now, please stay tuned to what we’re developing at OrthoScience. It will soon become very clear!
