I grew up in a town that others would describe as backwards. I was told, with curt assurance, farmers can predict the weather more accurately than meteorologists. People despised scientists, especially academic ones. They shunned the rigour of experiment and preferred personal trial-and-error. They practised dowsing (a pseudoscientific method of finding water underground), lauded echinacea as a panacea, and boasted about disgreements they'd incited with their GP. When I entered science I tried to keep it to myself and was ridiculed by those who found out: “You want to cower from the real world.”

Yet I always retained admiration for their (misplaced) distrust. In a way they were right: I learned, as I worked in various regions of the world, that the academics lacked resources, everywhere I went, and they were self-promotional, error-prone, and no one was monitoring them.

With fondness I recognise in online forums today that refusal to tolerate extraneous wisdom. The difference is: they now reciprocate others' curiosity. They are saying “what's your source?” when a comment arises that might be informed by data. It fills me with contentment. They are engaging with science and demanding a lot from her, including unequivocal prescriptions: “Do masks work or not?” (They do not know that this is best answered by a large pragmatic trial - an administrative nightmare, likely a mess with insufficient quality control, contaminated with non-adherence and missing data.)

There is a crudeness to their interpretation of results, lacking the specificity of instinct that accumulates through practice (e.g., a seasoned programmer can sense where an error resides in ten thousand lines of code). I try to tell myself: perhaps they will do some good by inadvertently energising the open-science movement which the academics support only nominally. But it seems they are more likely to fracture interpretation when they point to inconsistencies they do not understand, and infer that scientists have withheld information or misled them.

In an attempt to foment a shared, coherent scepticism, and pre-empt the cynical curators of opinion (the content creators), let us lay bare the feebleness of scientific data and the prevarications that often derive from them. Because, primarily, there is a need for consistency in interpretation of results which implies a common ability to detect the limitations of data that can make science appear reticent or even contradictory.

Without any disrespect, I refer to those on the periphery as outsiders or amateurs - that is who I am talking to, my old acquaintances back home (as if they still exist). I am just like them, only I have seen behind the curtain and can say: your penchant for scepticism is sound, I'm there with you, but you must be careful who you listen to.

A striking example to get us started is the apparent waning of the estimated effect of the COVID vaccine as we moved from Pharma-funded randomised controlled trials (RCT) to so-called real-world data (RWD), i.e. post-approval, observational data that are out of the hands of Pharma. A podcaster pondering the disconnect between RCT and RWD estimates said he felt the RCT overestimated the effect, thus indicating a deficiency in the study design and begging the question: why did the regulator allow it? Another insinuated something underhanded is going on; collusion between Pharma and the regulator to yield the inflated and artificial estimate.

This misunderstanding even ended up in court precedings in Canada, i.e., in the cross examination of Celia Lourenco, Director General of Health Canada, in the constitutional challenge brought by the Leader of the People’s Party of Canada et al.:

“[Dr Lourenco] confirms that the [RCTs] are required to demonstrate that the vaccine reduces symptomatic COVID by at least 50\%. The lawyer asks Lourenco if given the waning efficacy provided by the Public Health Agency of Canada ... would [the injections] still qualify for authorization ... if that had been observed in the trial”. [The 50% threshold is nominal and used to design the study.]

Dr Lourenco’s affirmation was offered as a shocking revelation by sceptics who appear swayed by the RWD descriptor. But RCT and RWD are not addressing the same question, i.e. statistically, they are not estimating the same quantity. The purpose of the RCT is to establish efficacy and accumulate exposure to explore safety; hence emphasis on patient follow-up, adherence to a study protocol and prespecified, conservative analyses. The RWD has a different purpose entirely, and the effect may be representative yet diluted; the data suffer from quality issues, missing data and other limitations.

Also note that we do not have RWD until the drug is approved and on the market; ipso facto, RWD cannot inform approval, and the lawyer's question is nonsensical. It is the same with saftey, i.e. the safety profile of a drug is not fully articulated until it enters the market against a backdrop of concomitant medications including off-label use. Safety detection, ultimately, amounts to a type I/type II error problem (over-reacting to limitless spurious signals and failing to react swiftly to legitimate signals), and the arbitrary setting of these thresholds. (I have noticed, on a drug's Wikipedia page, the list of potential side effects expand after the patent expires.)

It is ironic that some on the political right contrast RWD with RCT to suggest a compromised approvals process (as in the case above). In 2005 conservatives in America were pushing for reform that would permit other forms of data for consideration, i.e. non-RCT data, to speed up the approvals process. In other words, they were seeking to lower the threshold of evidentiary strength required for approval, a sentiment that contradicts conservative voices today.

At the time, the Society for Clinical Trials responded to this call from Republicans with a position paper; this paper provides concise examples of non-RCT data leading us astray:

“The theory that beta-carotene could prevent lung cancer was widely accepted based on retrospective (uncontrolled) epidemiologic studies of dietary beta-carotene consumption, yet two very large cancer prevention studies in the 1990s demonstrated convincingly that beta-carotene supplementation to smokers actually increases the incidence of lung cancer, to the great surprise of the medical community. Combined estrogen and progestin therapy was widely believed to reduce the risk of coronary heart disease (CHD) in post-menopausal women, yet the Women’s Health Initiative Study showed convincingly the opposite conclusion, that the treatment actually increased risk of myocardial infarction and CHD death compared to placebo treatment.”

Conclusions extracted from RWD are suggestive: the data are messy and effects are entangled (causal inference experts would chime in here, but their methods produce wide confindence intervals). The appeal of the large-scale RCT is that it yields a primary analysis that is straightforward (a clean estimate of the effect is calculable) and easily communicated. Although there are certainly poorly designed and premature RCTs which we should hesitate to overinterpret. And even high quality RCTs are routinely misinterpreted by the amateurs in their substacks and podcasts.

Consider MidwesternDoc's substack Forgotten Side of Medicine with over 100 thousand subscribers. In a post titled The Great Ozempic Scam and The Safe Ways to Lose Weight that was retweeted by RFK Jr (generating 1.5M views), referring to the pivotal RCT of semaglutide (Ozempic), the Doc declares: “Most of the participants could not stay on the drugs for a prolonged period”, alluding to safety and compliance concerns.

But the Doc has simply misunderstood the plot (this is not surprising; the Doc is an outsider). This trial, the SELECT trial, had an interim analysis; patients were recruited in a staggered fashion; then the interim analysis occured at the pre-specified time according to the accrual of cardiovascular events; this analysis determined that the study should be terminated. This explains why the number of patients is trailing off: the follow-up time is variable, it has nothing to do with compliance. The Doc also wanted his readers to know that 8% of those on semaglutide had serious adverse events. He failed to note that the corresponding number for the control group was 12%.

Aside from RWD heralded by the amateurs during COVID, we also saw meta-analysis offered as the pinnacle of evidentiary strength. Those who swallowed Goldacre's book Bad Pharma will reiterate this point-of-view; e.g. Bret Weinstein was excited by the meta-analysis of ivermectin. More recently, Jordan Peterson described the Cochrane collaboration as “the gold standard” when Cochrane's systematic review of masking landed. Others subsequently stole Peterson's sentiment and it spread as fact. Are they right? If I have spoken positively about RCTs, surely I must have a positive opinion of a meta-analysis of RCTs?

Confidence is the hallmark of the outsider. Peterson wouldn't know, for example, that the Cochrane software contained limitations that they didn't bother to fix for years (see Senn for details), and Cochrane leaders themselves describe systematic reviewing as like “searching through rubbish” and their reviews often conclude with a call for a well-desinged RCT (why would they want to take a step down their heirarchy of evidence?).

The reality is, meta-analysis has been on the ropes since 1997 when the Lancet published a meta-analysis of 89 homeopathy trials and declared homeopathy superior to placebo. Because homeopathy is a placebo, this was not a test of homeopathy but a test of meta-analysis which reliably compounded biases across trials. This result was considered a serious blow from which retrospective meta-analysis hasn't recovered. (The problem is intractable: you cannot repair low quality data by increasing statistical sophisticaion; if you try to solve the quality and heterogeneity problem you introduce a subjectivity problem.)

When the ivermectin meta-analysis was called into question due to data quality issues (exactly as predicted by Yuri Deigin) the insiders did not flinch, while the amateurs scrambled to rerun their analyses in Cochrane's plug-and-play online tool (a toy no statistician would ever use). It was obvious to insiders at the time, and should be obvious to all now, that the EMA's position on ivermectin was not dismissive. They were spot on. Those who offer themselves as sceptics were not sufficiently sceptical of meta-analysis. Promoting scepticism on some topic only meant re-allocating their credulity elsewhere.

There is something else, finally, I would like to clear up. All the talk (from Prasad, Peterson et al.) about the movement of staff between the medicines agency and Pharma: it is never clear what these commentators are concerned about. Perhaps it will suffice to note that those in regulation look askance at their counterparts in industry, and they are equipped to scrutinise them: they have all the company's data and code, they can run their own analyses on them - the academics' peer-review process looks futile in comparison.

The allusions to an easy-ride for Pharma betray the outsiders' ignorance, e.g., the sight of a drug company approaching an advisory committee meeting, they are almost out of their minds with anxiety. (I have seen a company produce over 30 thousand outputs in preparation.) The only time I have seen a representative plead with the agency to approve a drug was when I met with the academic investigator responsible for a trial in a rare disease (the key data supporting approval). This is worth repeating, as an antidote to the bleating anti-Pharma types: an academic was pleading with us to approve based on his flimsy data (he would not want to be known as the guy who loudly touted a drug that turned out to be a dud).

Let us bear these various details in mind, and we can fortify and hone the sceptical perspective. Data are coy, suggestive, they tease. Early phase results may be hyped in academia in an effort to win funding, but there is no analogous incentive for Big Pharma: why would the company want to fool itself into investing further and progressing to Phase III by hyping phase II results? In the public sphere ambiguity begets disagreeemnt and the outsiders will try to persuade you that it is their distance (i.e. their ignorance) from the institutions they seek to reform that confirms their integrity. You must instead find honest and informed insiders (scientists, not executives) to listen to. Unlike the outsiders, the insiders are not guessing, and thus not impressionable.

Subscribe now

Sheiner (1991) regarding the Intent-to-Treat principle: “[W]hy would anybody in their right mind advocate acting as though what they meant to happen did happen when they knew for sure that it did not?”

Despite these standards, academic clinicians flaunt their disapproval of industry with relentless crowd-pleasing books. To maintain legitimacy, the critics observe Pharma from the outskirts and, thus, betray a superficial understanding of the system they seek to reform. As clinicians they also miss the subtly of statistical arguments - where scrutiny would be best applied.

The point is: there is something seedy going on in Pharma and these critics miss it. It is behind the scenes and too esoteric for them to detect, no matter how eager they are to define themselves by their anti-Pharma posturing. And there's an added irony: the corruption is introduced by the academics themselves. I.e., the academics are sanctifying and ushering in a new mindset that offers Pharma what it has always craved: “the liberation of analyses” (an actual phrase heard in Pharma), and new ambiguity in trial results.

Some background is necessary: An industry guideline stipulates that a clinical trial must have “an appropriately qualified” statistician assigned to it. Industry always interpreted this to mean an MSc, but there are too many trials and not enough MSc level statisticians, leading Industry to fund MSc courses in the late 90s. This seemed a smart move. Courses grew in number and were attracting more students who all entered industry upon graduation, as they'd agreed. But many did not stay in their employment; the value of the degrees was tarnished as they became softened; and supply still did not meet demand.

Some 20 years later Data Science develops and makes itself known to a wider audience. The term is swiftly adopted by those who wish to appear current. Talking as if Data Science is Statistics 2.0 was a simple error made by the journalists that enhanced its apparent relevance for young scientists; combined of course with the glamour of AI and machine learning (ML) and the sudden declaration that the Fathers of modern statistics were racists.

Online magazines sprung up like Towards Data Science where influencers teach statistics to data scientists minus the history and difficult theory (statistics rebranded). Masters degrees in Statistics & Data Science began to appear; statistical societies added these degrees to their lists of accredited courses that lead to certification, e.g. Chartered Statistician; hence satisfying the regulatory requirement of “appropriately qualified” and expanding the recruitment pool. Bear in mind, there is no traditional route to accreditation for the data scientists and they are an eclectic and ill-defined group.

But how to import this new crop of analysts into drug development? I.e., how to relax requirements and change habits comprehensively?

Data Science has clout, and this is where the status-chasers come in, i.e. the academics described above who celebrate their intolerance of Bad Pharma with longwinded best-sellers. In fact, that was the title of academic clinician Ben Goldacre's book: Bad Pharma. With the style of a British tabloid newspaper, it provides an out-of-date and grotesque description of Pharma that won over sideliners who wallow in cynicism. Heather Heying held it aloft on the Dark Horse podcast and declared it a “terrifying book”.

For a long time Goldacre was in evidence-based medicine (EBM). Motivated by a distrust of Pharma, EBM proponents place meta-analysis atop their heirarchy of evidence, rather than the costly, industry-run clinical trial. The EBM enterprise lingers but could be declared a failure given that leaders confess they now think “systematic reviewing [is like] searching through rubbish”, and Goldacre's own Open Trial initiative backfired when it found that academia are much more likely than industry to fail to publish trial results, and Big Pharma is better than smaller Pharma. The Open Trials X account stopped posting 5 years ago, and the EBM proponents were silent before and after the Cass review landed.

“Trials with a commercial sponsor were substantially more likely to post results than those with a non-commercial sponsor (68.1% v 11.0% ...); as were trials by a sponsor who conducted a large number of trials (77.9% v 18.4% ...).”

Goldacre pivoted to Data Science where he became inaugural Director of the Institute for Applied Data Science at Oxford which promised to “generate new data and evidence, but also make it more impactful in the world”. It is important to notice the language used by data science advocates; it is often self-promotional marketing-speak like impactful, innovation and extracting insights from big data. Also worth noting is that Pharma are pouring obscene sums into these newly established data science centres, and, needless to say, the cash-strapped academics are clamouring to inform the public about the promise of “big data”. (It turns out one can draw parallels with the greedy Pharma execs the academics decry, only the academics are not selling drugs, they are selling themselves.)

The key issue to understand is the intentional blurring of the boundaries of these distinct fields. Each nurtures a different mindset about how one thinks about, and handles, data. Statisticians treat data as sacrosanct and handle code the same way, whereas data scientists talk about cleaning data and are happy to make use of a script found online; a careless attitude that is an affront to the statistician's sensibilities. There's a departure from strict pre-specification too; the model learns, it's dynamic and ad hoc, and the blackbox nature provides contentment for those lacking clinical understanding.

Ultimately there's a claim (a misunderstanding in fact) that the clinical trial is limited in scope and we ought to exploit uncontrolled, real world data. In the absence of clinical trials the data scientists “overclaim the usefulness and applicability of [their ML tools] to solve clinical problems”. If those who lament Pharma were sincere they should have plenty to say about promoting products off of cobbled-together data sources. After all, a bad algorithm that informs patient triage can do just as much harm, if not more, than a bad drug. Instead we witness an EBM person move to data science and upend their own heirarchy of evidence, without flinching.

In every case these examples imply an opening up of methods and a simultaneous relaxing of standards. And being open-minded in this way, i.e. showing one is unconstrained by old habits, has become the thing to espouse.

Within industry, new departments of data science are formed and statisticians are rebranded. Heads of the new groups talk about data analytics and insights and the need for statisticians to be open-minded and collaborative. Some state that those who are not onboard will be left behind and are, until then, an obstacle. These higher-ups are not statisticians or data scientists or computer scientists. They are VPs, i.e. MDs and careerists transfixed by the hype and promise of data science. They do not understand any of it in detail, they only know they are excited about it. They stand under lights on stage and declare “We can do it!” Do what? Who knows.

The statistician's presence was always considered a necessary burden by them; the dogged adherence to guidelines and strict validation is considered stifling, slow and unhelpful. But the statistician knows how to speak with, and appease, the regulator whose requests are ultimately statistical in nature. Thus the leader will smile at the statistician, but counter their suggestions by promoting a higher-level understanding which usually amounts to something reprehensible (deviating from the analysis plan to diminish a safety signal, etc.).

This is not to downplay the work of the data scientists. It would be superfluous to describe the value of this burgeoning field as it is properly defined, e.g. in understanding what is happening at the protein level. It is always interesting and inspiring to hear what they are up to. But those seeking reform should be thwarted by obvious inherent problems: you cannot have big data until the drug is on the market, thus: big data cannot inform approval.

We wish to affirm only the following: Statistics and Data Science should be treated as distinct fields that interact; Pharma's instinct will always be to create a grey-area in which their marketing teams can manoeuvre; the academics and their journals abet Pharma by not noticing that what is sold as cutting-edge is often not cutting-edge, and hence it is indistinguishable from marketing; when it is cutting-edge, it is, therefore, likely tentative and awaiting verification.

Academia is leaking public trust. Industry, with its high quality randomised controlled trials and diligence, obviated the reproducibility crisis that still affects academic research today. Industry has the opportunity to recover the trust in science spent by the academics.

"I've discovered that the whole biomedical publishing system is fraudulent. It is completely based on fraud. It is completely controlled. The major medical journals dictate what is written. There is such things as ghost writers. They determine the outcome of the study, and then they write about the outcome of the study which can be completely made up by people who never participated in the study ... Big pharma is behind this. There's no question of doubt that if a paper, even if it's an [RCT] is financed by pharma, or supported by pharma, or looking at a pharmaceutical product,..... you can be guaranteed, 100%, in someway or another, it's fraudulent."

Boghossian is entertained by Marik for about 15 minutes before he lunges at his ultimate conclusion (18m15s): Doctors are informed by the medical literature and therefore: "Doctors have been misled; they have been brainwashed; mind games played with them, by this comglomerate; and they are providing false information to patients ... Most of what doctors tell you is based on fraud; doctors don't know this."

The claim elicits a laugh from Boghossian who seems equal parts amused and perturbed. When the podcast ends, and they have signed-off, Boghossian is heard to say off-mic: "Holy fuck. Jesus Christ." For nearly an hour he's been swallowing Marik's claims like an Englishman on his stag; and now he's reeling.

Watching an intoxicated and amused Boghossian override his characteristic style is intriguing; much more so than Marik's wild and unoriginal claims. But we said we would respond to Marik, so let's do that and return to Boghossian at the end of the post.

And let's merely flag that we are unlikely to hear any precise or vulnerable statements from Marik. His words will be opaque and scattershot, and because Boghossian has become impotent, we will be on our own in making sense of it. (Boghossian has around 200 thousand subscribers on youtube; this video has 13 thousand views since March 26. Presumably many tapped-out early.)

The crux of Marik's claim is that companies fix the outcome of their clinical trial. How is this possible when the study is pre-registered on clinicaltrials.gov which displays the principal investigator and study design: duration, outcome derivation, timing of analyses, etc? Marik hangs his assertion on the anonymous medical writers who "never participated in the study". (He uses the term ghost writers, either to denigrate them or out of ignorance.)

Ben Goldacre described Pharma's use of medical writers in his book Bad Pharma (throughout Chapter 6 on Marketing). Goldacre, like all the others who flog anti-pharma sentiment, has not worked in industry and his description was grotesque. But Marik, a fervent observer, is the type to be swayed by it, just like Bret Weinstein who started banging on about regulatory capture after reading Goldacre.

It was never true that medical writers "determine the outcome of the study" (see the full quote above). It is not possible to know what Marik could mean by this. The outcome of the study is an estimate of the pre-specified primary estimand calculated by the statistician - it is this estimate that determines whether the trial is a success or not. How can the medical writer affect this estimate when the statistician is the only person in the room who fully understands it?

Marik might claim that there is leeway in the Discussion section of the paper to amplify certain findings and formulate a narrative. However, a trial is a rigid tool and does not generate much ambiguity; and those who read medical journals are qualified and harbour scepticism about, e.g., the plethora of secondary endpoints (to some degree these outcomes are intended to distinguish between the company's product and a competitor's and scepticism may be warranted).

In any case, the medical writer does not draft the Discussion section: the authors do, and the journals require them to state their contribution and sign-off on it.

The medical writer performs administrative tasks, e.g., a literature search to aid the authors when drafting the Background section; proof-reading; collating feedback from co-authors; creating enhanced displays to meet the journal's standards; etc. These practices vary among companies and the larger companies are most likely to insist that authors make a genuine contribution. I.e., people like Marik are concerned about Big Pharma but they should be much more concerned about small and medium-sized Pharma, as seen recently here:

Since Zlokovic and his co-authors no longer had the original data for one of the questioned figures, the editors wrote, “[r]eaders are therefore alerted to interpret these results with caution.”

The most persuasive and accurate critique of pharma contradicts the argument Marik is making: far from enticing favourable results out of data, Pharma analyses lack flexibility and are straight-jacketed by excessive pre-specification (it is difficult to fully pre-specify an elaborate model and thus a crude model is favoured). Industry analyses exhibit tired habits, mostly for the sake of efficiency and to minimise the possibility of coding errors, and not for scientific reasons such as parsimony.

The problem for Marik is that it is the academics who are guilty of running ad hoc analyses and massaging results. Marik would know this and this makes him appear disingenuous.

Yet there is legitimate criticism that Marik skirts regarding the exaggerated contribution of the authors. Typically Pharma identifies key opinion leaders in the field, referred to as KOLs, who become the lead author and co-authors of the paper. These academic clinicians play a minor role in the design and conduct of the study (e.g. interim analyses); mostly they contribute to the interpretation of the results and putting them in context.

It is amusing, then, when the lead author is seen discussing “their research” in major news outlets after you have described the results to them. It's a symbiotic relationship with the academics getting the attention they crave and the company borrowing the credibility of the researcher. The truth is: the academics are overconfident and request stupid analyses that leave them confused among a surplus of post hoc estimates. Perhaps they exert themselves because they are too keenly aware that they are superfluous. But that is quite a different criticism than the one Marik is making.

Marik's concern regarding peer-review of the submitted paper also misses the mark. Peer reviewers are an afterthought. They seek clarification and pine for faimiliar analyses. They are non-statisticians and one must do well to defend their analyses against them. For example, clinicians are inclined to demand conditional estimates of the treatment effect given change from baseline on a potential mediator. These are dubious estimates and the trial is not designed to answer them.

Frankly, the influence wielded by reviewers is excessive: authors are eager to appease them despite their requests for analyses lacking pre-specification and not being informed by an intimate knowledge of the study data (savoured only by the company's analyst). This is a criticism of the process, but it is not the one Marik his making. His claims of corruption are vague. He should, however, appreciate the limitations inherent in peer-review when his papers are scrutinised post-publication, including retraction.

Bearing in mind Marik's confusion about the publication process, it is interesting that he says he “discovered that the ... publishing system is fraudulent”. Discovered how? The lack of originality and specificity in his claims suggests he is merely invoking popular anti-Pharma books (the term ghost writers is straight out of Goldacre, and he says he was bowled over by Robert F. Kennedy Jr's book). If he had first-hand knowledge he would surely share it.

It is a shame whistle-blowing is restricted to those who have inside knowledge when others like Marik would like to decry Pharma too.

There is something familiar and tedious about his tone in this age of podcasting. E.g., the apparent contradiction whenever he refers to “published data” to support his views (e.g., 19m25s, and elsewhere) after claiming publishing is entirely fraudulent; the all-encompassing nature of the claims: “The major medical journals dictate what is written” (these journals are very different from one another); resentment that he and others like Kirsch are not taken serioulsy; the wayward scepticism: “Most of the vaccines they've developed simply don't work”; and he signs off with the usual plea for people to find more reliable sources of information.

Boghossian, on the other hand, is an interesting character: revealing and antagonistic, with an untenable blend of liberalism and hysteria. The following post refers to a book by Palahniuk:

We reached out to Boghossian and offered to add context and to clarify Marik's many dubious claims. In response he said this would not work because BasedScience are anonymous. (This was never an issue; we could have connected on Linkedin to confirm our credentials.) He then cancelled our subscription to his X account and blanked us. (We were an OG subscriber.) Incidentally, Boghossian's book is titled How to Have Impossible Conversations. Perhaps we were shooting too low.

Marik was never an impartial expert; he was a guest on the podcast. I.e., this is not an Andrew Neil grilling, it is more like Oprah chatting with Meghan.

Marik appeared on Bret Weinstein's podcast before appearing on Boghossian's (see Weinstein's podcast titled "Pharma: Not their first rodeo"). And Chris Martenson appeared on Boghossian's podcast in February after travelling with Weinstein to observe immigrants crossing the Darien. (Martenson does not pass the smell test; just visit his website.)

There is a mutual networking, recycling of guests, collaborating and promoting going on that the listener should take into account: Marik and Boghossian are in partnership. Soon after the podcast Boghossian appeared at Marik's institution where he spoke with employees who offered more guesswork. Perhaps these were some of the “more reliable sources” Marik was alluding to.

The enduring problem for the podcasters is that the real players would not be seen dead with them.

We witness, here, how in the era of podcasting wannabe best-sellers who flaunt their disapproval of Pharma can contaminate the minds of so many with their hackneyed criticism. Susceptible podcasters revived Goldacre's out-of-date review offering themselves as truth-tellers laying bare Pharma's secrets. It is shameful for Boghossian to allow Marik to persuade his listeners not to trust their doctor. Never mind the irony that some of these podcasters (not Boghossian) get rich selling dick pills and supplements to their listeners.

And, in a further irony, the same crowd who peddle cynicism and lies blame Pharma et al. for the rise of cynicism in public opinion.

In part 4 we will consider Marik's claims about Pharma rigging clinical trials (another accusation found in the Bad Pharma styled books, most notably Marcia Angell). We hope to persuade the reader that far from Marik's claims being true, it is industry, with its quality, who can restore the public trust lost by academia.

Subscribe now

These studies, led by academic investigators, suffer methodological problems: ad hoc composite endpoints (often infused with subjective measures) are appealing due to the small sample size and low event rates, but they are crude and overestimate power at the design stage leading to equivocal results and a lack of reproducibility. Academic clinicians have pushed composite endpoints over multivariate alternatives due to ignorance and they now pervade the literature despite the limitations. (More about this in Part 4.)

Marik espousing HAT is apparently not to be compared with a drug company pitching its drug, even though Marik's reputation (according to the FLCCC website itself) coincides with its success and is reminiscent of pharma's control of the perception of its drugs; Marik can be found on youtube, in a video created by his institution, describing how he saved a woman’s life and landed on his combination therapy with vitamin C.

The reader should be aware that disputes among academics over treatment alternatives often become heated; they seem to think there's a lot at stake. Consider Milton Packer and discussion around SGLT2 inhibitors and GLP-1 agonists. These are not good-hearted discussions. They are small-scale battles with a lot of vindictive plays going on behind the scenes. E.g., reviewing a paper harshly if the data deflect from the pet theory, and, when this fails, persuading the editor to allow a rebuttal in the form of a letter-to-the-editor.

These incentives are not flagged by either Marik or Boghossian. Instead, with his blinkered view, Marik goes so far as to say: any trial funded by Pharma cannot be trusted, even when the trial is conducted by academics. We are expected to believe that academics are trustworthy up to the point that they come into contact with Pharma, then they are utterly venal. But how does Marik think they rig a multi-million dollar clinical trial? We will get to this in subsequent posts.

Sepsis is rife with hopeful claims like Marik's precisely because of the lack of breakthroughs that have been made (the “need for systematic improvement in the initial management of patients“ and "what's next?"). This draws ambitious researchers and low quality trials that generate ambiguity allowing room for claims that would be thwarted by a robust RCT. The field is especially vulnerable to academics peddling black-box AI and machine learning tools (e.g.: TREWS machine learning-based early warning system for sepsis). Suchi Saria built a company (Bayesian Health) around her sepsis detection tool and the website has the same appearance as a drug company’s.

The astute reader will wonder at this point: academics peddling tools is perfectly analogous to the drug company marketing its drugs? Almost. The only difference is that, despite calls for RCTs of the machine learning tools comparing patient outcomes, the academics do not run them and in their absence hype is nurtured: “There is ... a tendency to over-claim the potential usefulness of ML models for clinical practice … The next body of work that is required for this research discipline is the design of [RCTs]“. If you do not test against standard care in an RCT then your research paper amounts to marketing material.

Given Marik's posturing we might expect him to be bleating about unsupported claims coming from academics who are celebrated by credulous science magazines: "Algorithm That Detects Sepsis Cut Deaths by Nearly 20 Percent." It is not as though these tools cannot do harm just as drugs can. Perhaps more so as they influence patient care and triage and do not undergo proper validation and regulation.

The motivation for the discussion with Boghossian is Marik's concern about perverse incentives affecting medical research; he cannot relent we are supposed to believe. But we witness here the uselessness of his fixation. In the scenario described above pharma play no role: we noted how academic disputes affect the publication process and how lower quality studies generated by academia tarnish the collective pool of results. Yet Marik laments pharma exclusively. Why doesn't Boghossian call it out? Is it not apparent, even to an outsider, that science becomes indistinguishable from marketing when they rush to celebrate the academic’s product?

As companies praise their drugs, institutions praise their scientists. The academic sells himself: his 'lab', which he often names after himself, is defined by marketing, influence metrics, and an increasing use of social media where he touts his ambition and makes allusions to whatever is gaining traction e.g. big data. (Beguiled by the complexity of the subject, and in the absence of proper validation, it is pure marketing.) Overlay these pictures: industry employees celebrate their company when they win marketing approval and institutions celebrate their scientists when they win awards. It is the same but inverted. The academic's reputation is paramount just as the company's reputation must be protected.

Unsurprisngly, then, when Boghossian introduces us to Marik we learn that he feels unfairly treated by the medical journals who censured him due to recognised self-plagiarism. His reputation is under attack! It is remarkable that Boghossian laughs off the idea of self-plagiarism as if it is incoherent; he says you cannot steal from yourself. (Why equate theft with self-plagiarism instead of, say, forgery?)

There lingers an unannounced shift from his previous emphasis on the plagiarism scandal affecting academia - suddenly it's all a joke. He also does not flag Marik's perceived grievance as a source of resentment which would add context to his outrageous claims. (Maybe he left that for us to detect, to spare Marik the embarrassment.) This prelude feels like a hasty attempt to restore Marik's reputation before getting on with the relevant subject matter.

It was a superflous review article about his work that led to Marik's self-plagiarism. It is well known that academics enhance their publication count (and hence their reputation) by extracting as many papers as possible from the same data source. This leads to practical problems such as duplicated data in meta-analyses. How this biases results has been reported for several decades, e.g. from 1997: "Inclusion of duplicated data in meta-analysis led to a 23% overestimation of ondansetron's antiemetic efficacy." (This problem may now be excacerbated by tools used by thoughtless researchers that scrape papers for estimates for use in meta-analysis.) Often the author will not reference their own work to limit the risk that reviewers will detect the duplicated text; it is a very seedy business.

Marik is suggesting the medical literature is corrupt while muddying it with his self-promotional material. He says he wanted to "spread the word". They always speak like this, i.e. in euphemisms; we all know what is going on. It is worth keeping this maxim in mind at all times: pharma sell drugs, academics sell themselves. Once again Boghossian misses this in his eagerness to combat bad pharma's assumed pernicious influence.

From the outset, then, we have a picture of a man who is frivolous and sloppy, defiantly so, laughing off the affair. And he is a little befuddled and resentful: we are expected to infer that if he has been censured the only possible explanation is that something underhanded is going on, rather than simply: journals have standards. (Yes, this is ad hominem. There is nothing wrong with ad hominem. We are trying to understand Marik. The public expects scientists to be fastidious, pedantic even. It's a bad impression at a time when science is leaking public trust.)

Part 3, covering Marik's specific claims, will follow in some days. Hopefully in Part 3 we will also find time and space to consider: why did Boghossian fall for this? This is what we find salient. Podcasting at this moment feels precarious and untenable, yet unavoidable.

We are several medical researchers. We have worked in pharma, academia and drug regulation in five countries and three regulatory regions; we have all the degrees. We may be inclined to generalise.

We enjoy Boghossian's podcast and had no intention of drafting a response or even listening to this particular episode given the irritation it would likely cause. But Boghossian is searching for a guest capable of responding to Marik's claims and it seems unlikely he will find one. (He needs a current industry scientist if he wants an accurate picture; someone who has worked in publications and had interactions with, or worked in, the regulatory authority. These people will not speak publicly.)

There are too many false statements made by Marik that will mislead and trouble the public. We are compelled to respond.

Preamble: Important background regarding Academia pitted against Industry

The Bad Pharma caricature is enticing and touted by many academics; e.g., Angell, Goldacre, and Goetzche. To give a sense of their zeal consider this piece about the indefatigible Vinay Prasad who takes on pharma “with glee” (more about him below). Dr Paul Marik, Boghossian's podcast guest, is a retired professor and his cynicism regarding pharma puts him in good stead. But the listener is confronted with a paradox from the outset: to be a reliable critic of pharma it is essential that you have never spent time inside it.

We have read the anti-pharma books by the academics listed above and can say they often miss the mark; i.e. there is legitimate criticism that could be levelled at Big Pharma (BP) and they do not see it, for the most part. And a wonderful irony is that the dodgy stuff industry produces is presented at the academics' conferences because they are widely known to be susceptible and open, more so than the attentive experts at the regulatory agency.

The listener should detect this potential bias and pretence to authority without any commentary from Boghossian. Nevertheless, it is disappointing that Boghossian didn't approach the topic with more scpeticism given his other comments on virture signalling and contrived consensus in academia. Unfortunately, the podcasters are a single and tiresome voice on the topic of BP. Perhaps this is because academics seek the limelight while industry experts haven't the time or interest in public skirmishes. Or perhaps it is because there is a new and spurious belief that academics and Leftists are inclined to defend BP and the podcasters see themselves as necessary to counteract this trend?

Any regular listener of Jordan Peterson's podcast will know he has said, in multiple places, that he is perplexed by the Left's (Academia's) sudden apparent defence of BP. But this confuses the Left's confidence in the regulatory mechanism with trust in BP. Academics remain as dubious as ever about BP. They pit themselves against pharma in an attempt to retain talent who know they would be better treated elsewhere. This is the pitch they give, and it can be seen in the pages of their annual statements. The promise of the academics is this: unlike pharma we are scrupulous, so who do you want to align with? The ruse is in desperation because: people are leaving academia for industry in droves.

If the academics did not feel the need to advertise their opposition to BP they would celebrate the pronounced quality and reliability of the randomised controlled trial conducted by industry personnel, contrasted with their own feeble retrospective analyses of uncontrolled data. Quality is, at the end of the day, the paramount issue. In other words, the Big in Big Pharma is a good feature, and the smallness and lack of resources to be found in academic research organisations (AROs) is in no way advantageous. It is easier to behave badly when fewer people are watching, for example. Or: if you have a single statistician then independent validation of code is not even possible (nevermind that statisticians are not programmers). Many have forgotten that in 2004 when the EU tried to align standards for clinical trials with industry there was immediate push back from academics. The gap in standards remains today.

This irony makes the attacks on industry by academics especially tough to bear. There is an impulse to respond: Do you really want us to describe what we have seen in academic clinical trials? Do you really want to talk about ethics and fraud? You want to make that comparison explicit and in the open? Maybe a discussion about the lack of resources and nous in AROs is long overdue (fraud is easy in academia, you would not believe what we have seen). Maybe industry experts are sick of these error-prone status-chasers flinging muck at them. For a glimpse of their handywork take a look at this recently published trial:

To be honest, most people working in industry and drug regulation are preoccupied and these podcaster discussions do not cause a ripple in their world. They do not know who Bret Weinstein is, even though he claims to have all the answers; and if you listened to Alex Jones in the cafeteria of a regulatory agency and let his guttural voice loose not a single head would turn. The talk of censorship is exaggerated. In this crowd no one cares what the amateurs are saying. You will not find professionals on social media defending their work in pharma against attacks from outsiders; if they did it would seem unsightly. Hence the asymmetry of the attacks.

These outsiders have no bearings when confronting the literature. E.g., they will not know that some of the disagreements between academia and industry can be understood as a feature of their varying circumstances. Why do the academics place meta-analysis atop the heirarchy of evidence while industry decisions are dictated by the large and costly RCTs? (Recall the ivermectin meta-analysis run by academics which had Weinstein terribly excited. Regulators gave it little heed; they knew better.) Ponder why the academics, who have a terrible time acquiring funding, have decided that analysing other researchers' data in the form of a meta-analysis is the route to the truth. (Goldacre is especially idiotic in this regard, but he has recently pivoted from evidence-based medicine to data science. What exactly data science is and why the academics are now pushing it is a post for another time. Suffice to say, if the podcasters knew anything about what was going on in BP they would put all their attention here. But of course they seem only to think that data science is a new word for statistics.)

Thus, the reader needs to do some work if they are to avoid being caught in the rip. We would like to encourage you to read between the lines: why do the academic authors listed above target non-experts? The huge font in Angell's book and the language she uses gives this away; and Goldacre's reckless book reads like a British tabloid newspaper. Bret Weinstein held it aloft on his podcast and declared it "a terrifying book" and subsequently reiterated how common the regulatory capture inferred by Goldacre is. Experts who reviewed the book when it landed saw it as biased and uninformed. The uninitiated are easily incited, unlike seasoned professionals; this provides a useful tell.

If academics like Angell and Goldacre had a legitimate concern then why not engage industry instead of courting the public with claims of corruption? (The regulatory authority seeks input when drafting guidelines.) Consider Prof Prasad who sounds more like a football fan in the away section than a Professor announcing some deficit in an analysis. On X the Prof has called Rob Califf (head of the FDA) names that we have been careful to forget. In doing so he distances himself from industry and makes cosy with those observers who savour distrust. (Prasad has deleted all his posts on X before 2023 that mentioned Califf, e.g. those declaring him corrupt, and has blocked us on the platform.)

Further, note that Prasad is a Professor of Biostatistics and Epidemiology* and does not hold a degree in either field (he has an MPH, a common degree among MDs who find mathematics tough going). Unlike industry, in academia titles are nominal. Maybe the reader noticed, for example, academics from peripheral fields such as engineering offering their views on vaccine safety during COVID. Or, the substack maintained by Prasad on pharmaceutical drug development, despite Prasad never having worked in pharmaceutical drug development. It is absurd to everyone aside from the academic who apparently regards it as their duty to stray outside their competence.

To appreciate the audacity on display, note that when the regulatory authority received the submission from the drug company various distinct experts were required to review: clinicians, regulatory experts, statisticians, safety experts, and a pharmacologist (post-approval studies required an epidemiologist). We were involved in these discussions and can say not one of these experts believes they can stand in for a colleague if they are absent. Transposing disciplines is a characteristic of the academic only. It was fun to watch academic epidemiologists on X calling themselves biostatisticians until COVID hit, then they were epidemiologists again. (There is more clout in being a biostatistican, but the degree is more difficult to get through.)

Only the industry scientist is expected to suffer the egregious stupidity of miscellaneous pundits. Imagine Bret Weinstein, when he is walking his cat, pausing at a construction site and shouting: "Gentlemen! You're doing it all wrong!" He'd be arse over tits in a dumpster in seconds. We have started to wonder: maybe something like this is needed to give people on the outskirts reason to pause before they serve us their guesswork. Otherwise Weinstein encourages the listener to interpret our silence in his favour (from Michael Shermer's podcast):

"[I]f i'm wrong ... there should be a flood of biologists ready to expalin what I got wrong, and when that flood of biologists doesn't show up you should ask yourself the question: Why is that?"

As noted above, if pronouncements on a podcast do not elicit a reaction it is not censorship or suppression of ideas or a conspiracy or google's algorithm. It is because the experts are too consumed with work to listen to some useless agitator.

So that is what we will offer here: a lack of silence. If it sounds harsh please recognise how generous the experts have been with their silence to this point. A response to Marik’s specific claims will follow in some days’ time.

*Since this post was published Prasad changed his title to Professor of Epidemiology, Biostatistics and Medicine, i.e. to include ‘Medicine’.