« Patients Have Plenty... | Main | Out of Control: What... »

Wednesday Feb 15, 2017

Why Do We Adopt Evidence-based Data at a Snail's Pace?

Branch Rickey may have been the most pioneering figure in baseball history, but his success, at one point, seemed unlikely.

After a short playing career and two lackluster stints as a manager, Rickey found his niche as an executive for the St. Louis Cardinals, Brooklyn Dodgers and Pittsburgh Pirates from 1919 to 1955. Rickey is perhaps best remembered as the man who had the courage and vision to integrate Major League Baseball, bringing Jackie Robinson to the Dodgers in 1947. But among his many other innovations, Rickey started the first farm system, a series of minor league teams that funnel talent to the major league club; invented the batting helmet; and was instrumental in getting Major League Baseball to expand to new markets in 1961.

One of his biggest, yet lesser known, advances had to do with the use of novel statistical analyses to gain an advantage. These innovations were made public in the 1950s, but they did not come into widespread use in the major leagues for more than 50 years.

With the assistance of baseball's first full-time statistician, Allan Roth, Rickey developed many new statistics, such as on-base percentage and slugging percentage. These are now ubiquitous in baseball circles as they are most strongly correlated with scoring runs (or the inverse for pitchers, in preventing them), and thus winning games.

So why did it take more than 50 years for every other team to catch on? On the face of it, it seems ludicrous. Rickey didn't keep these tools a secret. In fact, he wrote an entire article outlining the findings in Life magazine in 1954. However, there are many reasons for the slow adoption of his innovations, and we see this resistance to change in health care, too.

Andrew Balas, M.D., PhD., and Suzanne Boren, M.H.A., Ph.D., wrote that it takes, on average, 17 years for health care to adopt new evidence-based findings. (Ironically, that paper was published 17 years ago.) This seems absolutely ridiculous. Why wouldn't we want to do the best we can to help our patients? Many of the reasons are in our control, but many are not.

Here are but a sample:

Practice Inertia
Most of us practice the way we were trained. In medical school, residency and possibly fellowship, we learned certain ways to approach diagnosis and treatment of many different conditions. We often saw small, anecdotal victories or failures that reinforced these notions for us, whether based in actual scientific evidence or not. It's hard to get rid of that mentality, and it's easier to hold onto our habits than to change. But the truth is that much of what I learned in medical school is obsolete, even though I graduated just eight years ago.

Information Overload
There is absolutely no way to stay up on all of the medical advances, and that's not even considering how busy we are with our day-to-day practice. Approximately 2.5 million scientific articles are published every year. Obviously, not all of those studies are applicable to our practice, but it is still difficult to absorb all the new knowledge we need. Also, certainly not all published research is equal, and it takes a lot of time to properly evaluate whether a study is worth applying to our practice. Addressing any potential clinical workflows to implement new practices only adds to the complexity of this barrier.

Insurance or Regulation-prescribed "Quality"
In 2015, one local insurance company began sending letters to beneficiaries offering them a gift card if they received their annual Pap smear. When patients came in, I told them they did not need a Pap smear every year based on "new" guidelines (guidelines that came out three years earlier and were supported by multiple health care organizations and advocacy groups). But patients were understandably upset because they wanted their gift cards. When I contacted the insurer about its error, the company said it would stop sending the letter to my patients. But that wasn't the issue. The insurer was endorsing old recommendations that had been proven to be unnecessary and even harmful. Oftentimes, insurers and regulators fall behind the evidence, perpetuating out-of-date knowledge. It's also more difficult when different insurers act on varying levels of evidence in their metrics.


Those who are struggling with burnout and/or clinical depression are going to be less likely to make significant efforts to improve their practice. This represents a different type of inertia than that mentioned above.

All of these barriers, and others, significantly disrupt our attempts to get our patients the best care available. This has led to a greater emphasis in translational research in recent years in an effort to determine the best ways to implement new evidence-based practices. As with most issues in the complex world of health care, the situation is difficult to resolve and takes systemic reforms.

As an academic physician, I am expected to stay up on the latest research and guidelines and teach them to students and residents. One of the easiest ways that I have found to do that is through podcasts, typically listening on the ride to work. I also subscribe to services that summarize recent research findings. These can be tailored to the needs of one's practice, which it makes it even more helpful. I occasionally need to take a deeper dive into a given article or topic, but these tools provide a great starting point. For employed physicians, many larger systems can provide other opportunities to make it easier to improve our knowledge.

There also are numerous tools and resources to help us stay up-to-date through the AAFP, including journals  and CME.  The AAFP follows a rigorous process in the development of its own guidelines and in its endorsements of others so that family physicians can make decisions based on the best available evidence.

The epilogue to Branch Rickey's statistical efforts is documented in the book and movie Moneyball. Rickey's ideas have created a huge organizational and intellectual revolution in baseball in the last 10 to 15 years, with nearly all clubs now implementing the type of analyses that Rickey's Brooklyn Dodgers were doing in the 1940s. Let's hope that evidence-based medical knowledge doesn't take so long to implement.

Kyle Jones, M.D., is a faculty member at the University of Utah Family Medicine Residency Program in Salt Lake City. He is the director of primary care at the Neurobehavior HOME Program, a patient-centered medical home for those with developmental disabilities. You can follow him on Twitter @kbjones11.


AAFP has a number of areas for improvement regarding evidence basis.

1. Maintenance of Certification supported despite lack of evidence

2. Pay for Performance and derivatives supported despite lack of outcome improvement in major reviews

3. Pay for Performance supported despite the consistent evidence to lesser payment for those serving Americans left behind (where family physicians are 3 times more likely to be found)

4. AAFP should be exposing the lack of significant health outcomes improvements from a number of clinical interventions

5. AAFP should maintain a constant focus on the personal, community, social factors that are 60 - 70% of health outcomes. This also leads to a choice between outcomes improvements for most Americans or protection of the academic/clinical focus/status quo.

The above should be reflected in all media, leadership postings, and staff activities. In addition AAFP should prioritize a return to evidence basis and movement away from jump on the bandwagon acceptance of research methods and findings that were exposed as defective 100 years ago. Dr. Jha noted below has been consistent in such work.

"What is the central tendency of a distribution but a lazy generalization? The aggregate, the mean, is wrong about everyone but the few closest to the mean, yet is so revered because we mistake the aggregate for the truth. The tyranny of the aggregate is the most extraordinary tyranny of our times. The aggregate is built by people who vary, yet it imposes itself on the individuals, the very variation which creates it. It literally bites the hands that feed it." SAURABH JHA, MD (associate editor with The Health Care Blog) More at The Tyranny of Health Care Research

As a consideration AAFP should also be critical regarding promotions of various training interventions by MD DO NP PA FM and worse of all IM. There have long been claims of overall improvements for the purpose of primary care, health access, rural practice, or care where needed. But all fail due to inadequate financial design. Training interventions can only rearrange the initials and names but not change inadequate delivery capacity.

As a cushion for the claim of inability to result in increased capacity, note

1. The substantial increased cost of delivery via new regulations and certifications that can only make the financial design and capacity worse.

2. 30 - 50% greater than average population in 2600 lowest physician concentration counties with higher concentrations of elderly, and complexity. This is a massive increase in demand where 40% of Americans fall further behind by designs that fail for generalists and general specialties that are 90% of local services in these counties.

3. Fewest remaining in primary care positions, particularly family practice positions across all sources, as the financial design rewards more new specialties with more added in each specialty

Posted by Robert C. Bowman, M.D. on February 15, 2017 at 09:44 PM CST #

Kyle, excellent article!

As an academic FP as well, I emphasize to residents and students that while EBM is the standard, the evidence is only as good as what is studied and who it's studied on - and the studies often are not generalizable to all patients that we see. Often the devil is in the details (ie for the new lung cancer screening guidelines, there is a 20% absolute risk reduction than the standard, but the number needed to treat is 217, while the number needed to harm is much less). For some newer therapies it is sometimes the case that in 5-10 years they are no better than the original standard of care, but it did cause more false positives or complications, and cost more to our patients and to the system. (Some examples - routine vertebroplasty for compression fractures, stenting for asymptomatic coronary lesions, screening for lung cancer). New FDA approved medicines may be marginally better than placebo at best (ie Addyi). To me, this adds a degree of skepticism (and suspicion) to anything new, but it is nearly impossible to look into every new therapy and the history behind it... doing the best thing for each patient is not easy!

Posted by Alap Shah on February 16, 2017 at 10:12 AM CST #

The issue of a gift card for coming in for annual paps fascinates me.

First, monetary rewards have never been transformative when they have to do with human behavior - in this case patient adherence to preventative health measures. Secondly, professionals, albeit doctors or other skilled workers in the industry where the decision making process is complicated, require autonomy more so than incremental monetary bonuses to get the job done, and get it done right! I get leadership newsletters from MBA schools (Wharton's and Kellogg's) and over and over they stated that this is the case.

So why does the insurance company does it, even if it's not medicine, at least it's not "evidence-based" practice for businesses either?

It's marketing. It's comparable to a rebate on your electronics, a sale, a Walmart "rollback." Call it what you will, it's actually the patient's money, also partly subsidized, so it's taxpayers' money, that's given back to them. Since that's part of marketing operations, on the balance sheet it's part of the business expenses. In essence, the industry is using the gift card as a double tax write offs when medicine has even yet to be delivered, nor the final outcomes known (patients have to come in first, and you have to agree to do what they were incentivized to do).

The medical profession is slow to adapt EBM yes, but it, as not a profession but as an industry, regulators included, is catching up with evidence-based business practices, or really, financial schemes. This example you'd used illustrates both! EBM is use in-name only, as a catchy phrase for the patient to come in, as a punitive measure for the doctors if you don't do what the patient wants, as a EBB (evidence based business) for tax write-offs, as marketing. It also lays the final blame on the doctor for not practicing medicine, and not the business if it failed to make patient healthier in the end (as if that's also measurable).

Posted by Michael N. on February 22, 2017 at 11:19 AM CST #

Nice response, Alap. Another recent example was acceptable HbA1c for seniors that was raised recently to 7-8 as a standard of care.

So the question on this journey for the past 26 years of intense managed care is - number needed to harm before we change the guideline... Since metrics are also used as punitive measures, the harm here is also onto the practicing physicians who took it as the holy grail... at least until better evidence is available.

Just playing devil's advocate for the little guys and the patients who don't always fall within "the averages" that is EBM.

Posted by Michael N. on February 22, 2017 at 04:40 PM CST #

Ok let's see if we can all be together on EBM.

There's a lot of statistics on the flu shots this year, particularly for the elderly. This is where the biggest cost-savings are (ICUs) and the most difference as far as healthy outcomes (multi-system comorbidities). Most people are healthy and unless there is a highly virulent outbreak, so the final outcome would not have made a difference, in the individual level regardless of your flu status. A virulent flu pandemic happens only every ?100 year.

As far as population healthcare, we are essentially giving it for the herd effect. As a country, we are pretty good at that - 90 million doses per year. Pharmacists can give flu shots. We have drive through flu shots retail clinics, and free flu shot health fairs. Herd effect is supposed to limit spread. Has it, and how do you measure that as a outcome?

Here's the evidence - overall effectiveness is 55%. That's the average. The target population, the elderly, some years have been only a dismal 10%.

Let's say we spend $10 per flu shot given as a cost to the country - that's almost $1B spent for something that's only 55% (this year it's 48%) effective for a population that don't really needs it, and 10% for a population that really needs this (the elderly).

If there is a cure pill for the flu, it's probably more cost-effective than mass vaccinations of 90M of people, yearly, and the cost of enforcement since it is required by law in some states for certain professional groups.

15 years ago when Google was just becoming a beckon of the internet powerhouse, the number one hire was - I wouldn't have guessed it - "statisticians."

We as a medial profession have a long way to go...

Posted by Michael N. on February 27, 2017 at 11:31 AM CST #

Another good argument "against" metrics and EBM... they stifle "creativity, necessary for solving complex issues, and allowing leaders and managers to agree and take risks to actually "do" it.. Sometimes, too much of a good thing could be bad... the data on the Millennials are also startling on how they are more readily embracing "data."


Posted by Michael N. on March 02, 2017 at 03:02 PM CST #

You must be logged in to post a comment. Login

Sign Up

Subscribe to receive e-mail notifications when the blog is updated.


Our other AAFP News blog

Leader Voices Blog - A Forum for AAFP Leaders and Members


The opinions and views expressed here are those of the authors and do not necessarily represent or reflect the opinions and views of the American Academy of Family Physicians. This blog is not intended to provide medical, financial, or legal advice. All comments are moderated and will be removed if they violate our Terms of Use.