BioVie (BIVI) Data Post-Mortem: The Most Ridiculous Data Mining Ever?
Sure, just exclude all the sites where the treatment arm didn't do as well vs. placebo
I’ve seen a lot of data mining in my career. They could be based on disease severity, geographic location or even number of years since an underlying trauma, but I don’t think I’ve ever seen a company just disqualify entire geographic regions due to some record keeping issues and placebo doing better than expected at a handful of trial sites. That is exactly what BioVie has done.
They excluded 82% of the patients in the trial for either being at a site with some so-called “irregularities” which included some items such as placebo doing better than expected (I bet every company that has developed depression drugs in the past would have loved to be able to use this trick!). Here is the full explanation from the press release:
As the top-line efficacy data was unblinded and PK data became available, the pre-specified demographic subgroup analyses showed that patients in the identified demographic on placebo significantly improved cognitively without any intervention – a finding that cannot be explained scientifically. Furthermore, the pre-specified anomalous sites vs. others revealed a similar scientifically improbable and that these 9 sites are in the same single geographic area. It turned out that virtually all of the patients in the identified demographic group were associated with the 9 anomalous sites. Consistent with our pre-specified statistical plan, these 9 additional sites were also excluded to arrive at our Modified Intent to Treat population, which became underpowered with just 81 subjects. Out of an abundance of caution, we also referred these 9 additional sites to the FDA’s OSI. It should be noted that virtually all of the 15 sites referred to the FDA were in the same geographic area.
And since clearly excluding 82% of patients wasn’t enough for the company to get the data where they wanted it, they excluded 30% of the remainder due to the per-protocol analysis, which only included those “who completed the trial and were verified to take study drug from pharmacokinetic (PK) data.” So, at the end of the day, 87% of participants were excluded for one reason or another.
Even weirder it's unclear how many participants were actually included in their analysis. In the press release, it mentions the per protocol population was 57 patients. But then if you look at the data they actually presented, the number looks closer to 50 (50 in the CDR-SB analysis and only 47 in ADAS-Cog12):
Does that mean they had to torture the data even more after getting to 57, to exclude some more patients? Seriously, nothing adds up with this company. Worst part was that after all those machinations, they still couldn’t come close to statistical significance. The actual ITT data must be truly horrible as I’ve never seen so much datamining (which essentially excluded all sites where they didn’t like the data) in order to get to such a bad result. As you can see, people in the “suspect” geographic area, which represented over half the patients in the trial, actually had placebo doing better than the treatment arm, so on a true ITT basis, it’s possible there was zero trend in favor of the treatment arm and possibly one favoring placebo:
But the weirdness doesn’t end there. What exactly was the “demographic” that they excluded? We might get a hint from the September 26th press release:
“Furthermore, we do not need to demonstrate efficacy and statistical significance across the board or with all the pre-specified subgroups such as mild- vs. moderate-AD, Aβ positive vs. negative, Hispanics vs. non-Hispanic, insulin-resistant vs. not, etc. A win in one or more subgroups is still a big win for the patient community and the company.”
Note how they called out hispanics/non-hispanics in that press release and then make no mention of that in the below slide on the per protocol population.
So, did they just exclude hispanics en masse? It’s possible especially as so many of the sites look to have been in South Florida:
It doesn’t stop there. On November 20th, the company updated their November 9th update by eliminating EVERY SINGLE SITE except one, that being in Columbus, Ohio.
I’m sure some BioVie supporters are going to tell me about the statistician that supported their conclusion that the data was anomalous at the sites. Unfortunately, that person was Suzanne Hendrix, CEO of Pentara, a biostatistics firm. This person has very little credibility as she was also supportive of Cassava, here and here, and you know how I feel about that one.
Frank David over at pharmagellan has a list of the most common PR red flags for clinical trial results, they are:
Any top-line statement except "met [or failed to meet] primary endpoint" — There is only one acceptable way to headline a clinical trial report. Period. "Superior response", "numerical advantage", "trend toward significance", "clinically significant" — those phrases should all make you suspicious that something shady is afoot.
Misdirection toward non-primary endpoints — When I see a headline touting a positive result that doesn't explicitly say it was the primary endpoint, my immediate next stop is clinicaltrials.gov to figure out what the primary endpoint actually was. In most cases, absence of the word "primary" in the top-line report is highly suggestive that the study whiffed on its main goal.
Over-emphasis on positive results in a subset of patients — A classic biotech spin move is to focus attention on a subset of patients, usually identified after the fact, who managed to eke out a response with P<0.05. If a subgroup is highlighted in the press release's lead sentence, that's often a bad sign.
Non-standard statistical methods — It's impossible to list all the possible shenanigans here, but one-sided P values and non-ITT (intention to treat) analyses are highly irregular and usually mean the study failed using conventional analytic approaches.
The company having a non-standard headline, a focus on a very small subgroup of patients and then using per-protocol, mean that BioVie is flying a giant red flag:
There is not going to be any partnership and no, the FDA is not going to be good with this tortured data. Their future is also in doubt. As of September 30, 2023, they had $21.2 million in cash, while burning $12.2 million a quarter, so they seem to have cash until approximately March at this point. It doesn’t help their financial flexibility that they have to repay principal on a note payable, with $12.5 million due (as of September 30, 2023) to be repaid over 15 equal monthly installments. Only way this company with a worthless lead product survives is through extremely dilutive financings with onerous terms.
Please like, share and subscribe!
Stocks mentioned:
Sadly, you disqualify yourself as any source to be taken seriously by the following arguments:
- "The company excluded only data they didn't like" when, in fact, the placebo arm showed improvement of about -6 on ADAS-Cog at 6 months, which would more than twice the highest placebo response ever reported in medical history.
- "Susan Hendrix is not authoritative because of her history with SAVA". You fail to provide any evidence that Ms Hendrix, who has led Pentara since 2008 is involved in any questionable activity outside of SAVA, and even for that you only supply your own opinion as "evidence"
- "The company tried to obfuscate in their press release", when basically the first thing they wrote / said (in their CC) was that they failed stat sig.
- "They are racist and exluded Hispanics" when they neither said that, not excluded any demographic specifically. And even if those patients were exclusively Hispanic, which you can only speculate about, but provide as fact, what would that say about them as a company? Are they at fault that their CRO recruited a large number of Hispanic patients?
Virtually the only thing that has some merit in your "analysis" is the observation about discrepancies in patient numbers in the included population.