J. Falatko D.O.
Where could something like this be published?
One of the sad things about the medical literature is it’s boring. Sometime, many years ago, it was decided that doctors and scientists need to write their findings as faceless, soulless, academics. We don’t use the pronouns, we, I, or me. My assumption is that much of this was academic elitism. The result is nobody enjoys reading medical journals. We do it in the same manner as your kids eating their broccoli. Honestly, I'm probably fooling myself believing this is anything different.
Scientific journals are full of scientific papers. These documents contain brief introductions explaining the purpose of the study, then the design, the data, and a discussion. The discussion is the authors opportunity to add some color to their findings. Unfortunately, this part of the paper rarely has a pulse, is inaccurate/misleading, and not worth reading. When a paper is published, a fellow scientist can write in an editorial to the journal elaborating or refuting the findings. Editorials are often brief and typically half to a full page. Not much room for detail or broader contextualization. This leaves critiques of research, with commentary on significance, without a home. This must be what blogs are for.
In fact, there are very few critics of medical research out there. I would consider myself one. However, I’m the worst type of critic because I do little research myself. Like a restaurant critic that can’t cook. It’s not that a lack respect for those that execute research. I respect them very much. They are just too close to their own work to be honest with what they’ve found. I would like to do research, but who has the time?
I came across a good specimen for criticism the other day. NEJM sends out a journal watch twice a month. One of the articles they highlighted was group of studies investigating the risk of sub-clinical atrial fibrillation and stroke. Sub-clinical atrial fibrillation is an abnormal rhythm that has been detected incidentally, usually lasting less than 20 minutes, and occurs rarely. This is often detected by implantable devices such as defibrillators, pacemakers, and loop recorders.
The excerpt was interesting because there was a negative study out of Europe, a positive study out of the U.S., and a subsequent meta-analysis. The European study was the first to be published. It was promptly criticized by the American authors a few months later. The authors of the U.S. study subsequently published a meta-analysis combining “available” data on the topic. I quickly recognized this as a recipe for shenanigans.
I think I'm right.
The basic design of the studies was to randomize patients with detected subclinical atrial fibrillation, currently not on anticoagulation, to edoxaban, or apixaban, and compare it to placebo. For the purposes of this discussion, edoxaban and apixaban are essentially the same. The meta-analysis was an aggregation of the two studies above. That’s it. No other studies met criteria for inclusion.
Evidence based medicine teaches us that a meta-analysis of high-quality studies is top level of evidence. So, I will only discuss this result. The meta-analysis showed an absolute risk reduction of 1.2% with treatment. This would equate to a number needed to treat of about 84. So, you would need to prescribe anticoagulation to 84 patients with subclinical atrial fibrillation to prevent one stroke. No other benefit outcomes were statistically significant. There was an absolute risk increase of 2.0% for major bleeding equating to a NNH of about 56. Death from major bleeding was not statistically significant.
The authors concluded there was modest reduction in stroke with treatment in subclinical atrial fibrillation.
Their conclusion is not wrong. They did not lie.
The American College of Cardiology was not slow to issue a guideline giving this a 2B recommendation. This basically means there is likely benefit but shared decision making between the practitioner and patient should take place (insert retching sound here).
Shared decision making means I should explain the risks and benefits to the patient and allow them to decide. It’s a cop out. And impractical. Office visits are 20 minutes long. Informed consent is flimsy. The remainder of this text will be an explanation of that fact.
As far as the published paper, and the guideline recommendation, we would be better off without them.
Let me state my case.
There are many factors to consider when applying this to patients. Let’s use the imaginary patient Mrs. K to illustrate this. Mrs. K is a 80 yo female with a defibrillator that detected a 5 minute burst of afib three months ago. She is otherwise healthy and in her cardiologist office for a routine check. She is on a typical medicare advantage plan that has a coverage gap (donut hole), and lives on a modest retirement.
She is presented with information and given the option to begin treatment with apixaban to prevent a stroke by her cardiologist. We’ll stop here…
One of the factors at work here is the negativity bias associated with stroke. Negativity bias is the opposite of survivorship bias. Stroke can be devastating, we all know that, but it’s not always devastating. We assume since some strokes are bad, all strokes are bad. When I presented this to my class, most of them guessed that 25% of strokes result in no residual deficit. The actual number is closer to 60%. Stroke happens, patient makes full recovery, walks out of the hospital. We tend to easily forget about these patients, but it happens regularly.
For Mrs. K, what does she really care about? Does she care about the diagnosis code of stroke on her medical chart, or is she most concerned about having a disabling stroke. Probably the latter. So, right away we can cut that ARR of 1.2% down to 0.7% since most of the strokes that occurred in the study were non-disabling.
Strokes can also be treated. Roughly 35% respond to acute treatment with clot busting medication, and the strokes that do not respond can potentially be treated with thrombectomy. The number needed to treat (NNT) for eligble stroke patients treated with tissue plasminogen activator (tpa) is 19. That’s an absolute risk reduction of 20%. The NNT for mechanical thrombectomy in acute stroke is 2; one of the most robust innovations in the past decade of medicine.
For Mrs. K, factoring in the incidence of non-disabling stroke and ability to treat, we’ve cut our ARR down to 0.26%. Based on this risk reduction you can calculate a NNT of 384. So, you would need to treat 384 patients with anticoagulation to prevent 1 disabling stroke.
Treatment for stroke takes effort and money. I don’t want to belittle that. You also need access to a stroke center, so living on the Yellowstone ranch in Montana is a different situation than my next-door neighbor (the patients in these studies had access to a stroke center).
This is the net benefit.
Treatment does cost money, but Mrs. K also has to spend money on the medication, so let’s take a look at this side of the problem.
Mrs. K is 80 years old and well into her retirement. We don’t know her financial situation, but it is safe to say she is conscientious of her cash reserves. Eliquis costs about $600 per month for non-Medicare advantage plan patients. She is fortunate to be on an advantage plan. If she weren’t she’d be totally screwed. At $600 per month Mrs. K will reach the donut hole in about 4-5 months. If she takes any other medications (which she likely does) she will reach it sooner. Most medical patients that have several chronic illnesses reach the donut hole at about 4 months into the year. At that time, they are responsible to pay for roughly 30% of the drug cost. In this case $200 per month.
$200 times 8 months is roughly $1600 dollars per year in expense. Hypothetically, if every patient diagnosed with this problem had a Medicare advantage plan, it would burden the patients $614,400 to prevent 1 disabling stroke. That is just the ANNUAL cost. Remember, these things go on in perpetuity. If you don’t have a prescription supplement/advantage plan the cost would be $675 for a 30-day supply, or $8100 annually.
To phrase it differently, Mrs. K is going to pay a $1600 dollar premium to prevent an event with the odds of 1:400.
There should be no mystery as to why healthcare costs are so high. This kind of stuff is everywhere.
It doesn’t end here. If it were only as simple as explaining the numbers and the costs to Mrs. K. She is up against several other cognitive traps.
The first and most obvious is publication bias. The entire reason the study was published and received recognition. Journals want to publish positive studies (studies that show a statistical significance). Positive studies are more likely to be read, which means subscriptions and advertising revenue go up the more positive studies you publish. Researchers want to be published because it brings personal and institutional notoriety. The Americans, in this case, researchers that published were beat to the finish line by the Europeans. So, they had to discredit the other trial and publish their own meta-analysis to validate their work. Without doing so, the American study would've likely found its way to obscurity.
Second, statistical vs. clinical significance. Statistical significance is a term that basically states two study groups are different. It doesn’t say how different they are, just that they are different. The combination of the two studies into the meta-analysis generated enough power to detect a statistical difference. As stated above, how likely is this to be clinically significant, and what is the cost that will incur to implement this intervention. We understand the two groups are not the same, but how different are they?
Third is the agency problem. The agency problem is best explained by an analogy with real-estate. Your real estate agent is supposed to work for you. They work off commission. They collect 5% of the house being sold. They are numbers people. The more homes they move, the more money they make. No real-estate agent makes their annual cash-flow in one transaction. Say you are trying to sell your home for $100,000 dollars, but you get an offer for $90,000. Your agent recommends you take it. To them, it’s $500. To you, it’s $10,000. They aren’t really working for you. This is the agency problem.
The researchers involved have an agency problem. In theory, they want to better society, but ultimately, they want to publish, maintain tenure, and be granted more time and money for research. Their incentive is not aligned with Mrs. K.
What is Mrs. K’s doctor’s incentive? Her doctor wants to get through their day, on time, and survive. By survive, I mean, be able to work for as long as possible. The more times He/She puts themselves out on a limb against a guideline the more likely they are to face certain ruin at the hands of an attorney. He/She is incentivized to follow the “best” level of evidence. Mrs. K’s doctor does not have to pay for the medication, that’s Mrs. K’s job. She is likely going to follow her doctor’s advice. If some degree of shared decision making is involved, the risk will be pushed to Mrs. K. Her doctor will leave it up to her (that’s what shared decision making means).
It's unlikely anyone would be able to explain all of this to Mrs. K in a 20 min office visit. There’s no way she can be truly informed. (This is just one little insignificant blip accidentally found on a defibrillator report. Can you imagine the millions of things that run through your doctor’s mind when you come in with an issue?)
She is up against a great deal of cognitive pitfalls: 1) Law of large numbers, 2) statistical vs. clinical significance, 3) Publication bias, 4) The agency problem, 5) Negativity bias, 6) Transfer of risk, etc.
All of this is known as the Lollapalooza effect.
Bless you, if you’ve read this far. You deserve a prize.
Long story short – Apixaban is being prescribed to Mrs. K. Her friends are getting it too. Who knows how they will pay for it?
The argument against my case above is the "utility" of the treatment matters substantially more for a few individuals than the group. To illustrate, we can put 500 patients in a room with the problem and assign them a number 1 to 500. We can spin a lottery wheel with 500 numbered balls inside and draw two numbers. Numbers 174, and 18 are the winners. They did not have a massive disabling or life-ending stroke that could not be treated. Their money was money well spent. The rest are just paying another insurance premium.
One day I’ll write a post suggesting solutions to all of this, but this one is already long enough. Save that for another day.
Comentários