PATHWAYS is a randomised controlled trial of puberty blockers in children with gender incongruence. The announcement of ethics approval incited commentary in mainstream and new media that misunderstands the clinical context and motivation for the trial and fails to recognise its real flaws, including why the trial will ultimately fail. Controversial research draws commentary from the uninitiated, including journalists, who lack good instincts which are developed through focused application.

The trial and media response

The PATHWAYS trial is a randomised, unblinded (open-label) study investigating the effect of puberty suppressing hormones (PSH) in children aged between 10 and 16 years and with gender incongruence persisting for a minimum of two years. Recruitment will commence early next year (Roxby & Holt, 2025) with the primary analysis taking place after two years follow-up and with results expected in four years. All participants will receive PSH: 226 previously unexposed children will be randomised to an immediate or delayed (one year) start to treatment; all will receive psychosocial support (Fox, 2025). (There will be an untreated, non-randomised group with statistical comparisons based on matching. Here we focus on the randomised groups.) A £10.7 million award from the National Institute for Health and Care Research provides funding for the research (NIHR, 2025). Participants will consent to long-term follow-up stretching for the duration of the funding (5.5 years).

In November, awareness of the trial spread after ethics and regulatory approval was announced (King’s College London, 2025). Blunt and misdirected criticism appeared in newspapers (McColm, 2025) and on social media (Farrow, 2025), in particular X, including from British MPs who wish to “halt” this trial of “experimental drugs” (Lowe, 2025). The Leader of the Conservative Party, Kemi Badenoch, joined the chorus of voices to stop the trial, posting: “No child is born in the wrong body” (Badenoch, 2025). Criticism tended to impose on the trial the language of gender ideology, pseudoscience and the harms to children. Some also pointed to Big Pharma chasing profits, although gender incongruence remains a rare condition (i.e., prevalence <0.05%) and use is off-label and thus exclusivity incentives for orphan drugs do not apply (companies may nevertheless attempt off-label promotion; O’Neil, 2021).

However, the trial had already hit headlines back in February 2025 when funding was confirmed, with more precise criticism and general coverage emanating from the journalists at that time (e.g. Fox, 2025, The Times, 2025, Searles, 2025). The main point was the absence of evidence supporting the use of puberty blockers in children with gender incongruence. As noted in a systematic review by Dr Hilary Cass in April 2024, in “2014 puberty blockers moved from a research-only protocol to being available in routine clinical practice”, with the Dr’s assessment 10 years later noting that the evidence in favour of PSH is weak and the risks are not well understood (Cass, 2024). Those sceptical of early treatment declared the case settled (Turner, 2024). However, Cass’ review notes that the randomised controlled trial (RCT) is the gold standard of evidence and there are no good RCTs, and Dr Cass is “really pleased” about the upcoming trial (Roxby & Holt, 2025). The closest study to an RCT was a prospective study of 44 children with no control group (Carmichael et al. 2021).

Prominence of the RCT explained

To understand this disconnect between the media and researchers (e.g., Cass) it is important to appreciate the impetus to construct a systematic review - they seek to evaluate the current state of evidence and are thus often over-emphasised and premature. I.e., there is an abundance of systematic reviews (the output of the evidence-based medicine proponents) and they should not be seen as a transmogrifier of meagre data into definitive conclusions, and a catchall analysis should not be relied on to establish efficacy. If the same systematic biases exist in each study dataset, combining them will only reinforce the biases, not cancel them out. Unfortunately, commentators often assume systematic reviews are the pinnacle of evidence, probably based on faulty logic: if the RCT is the gold standard and a systematic review collects the available RCTs, then the former transcends the latter.

A controversial systematic review would typically be seen as the justification for a trial. Consider the meta-analysis of magnesium in myocardial infarction (MI) which caused intense debate in the literature in the 90s (Teo & Yusuf et al., 1993). An association between MI rates and levels of magnesium in the water supply was observed (Woods, 1991). Trials of modest size were then reported for nearly 20 years before a meta-analysis of ten trials indicated that magnesium was efficacious (log odds ratio of mortality on magnesium relative to control was -0.49 with 95% confidence interval –0.27 to –0.73). This result reasserted the conclusions of an earlier informal review (Rasmussen et al. 1986). A large-scale randomised controlled trial (58,050 patients) followed two years later and found that magnesium was ineffective (log odds ratio 0.06 with 95% confidence interval 0.00 to 0.12). This study is referred to as ISIS-4 (ISIS-4 Collaborative Group, 1995). Or for an example in the other direction consider aspirin in MI where the evidence was weak and clinicians were not persuaded until a large RCT established the effect (ISIS-2, 1988).

This is why the lack of prior data is spelled out in the PATHWAYS trial protocol in terms of equipoise (PATHWAYS Trial, 2025) -- it provides the justification for an RCT. An RCT should be welcomed in the midst of such uncertainty and controversy because it has the ability to quash it like nothing else; it provides a straightforward analysis and thus cogent results that declare a single hypothesis the victor. However, to achieve such a clean result requires a concerted effort from a variety of experts and academia struggles to retain qualified people when competing directly with industry (Watson, 2023). The regulatory guidelines state, for example, that a trial must have a competent statistician (ICH-E9, 1998) and they are a scarce commodity. (We have written previously about how Pharma tried to solve this problem by funding statistics MSc’s but failed: Is Data Science a Scam?.)

It is worth noting that during COVID, with a high sense of urgency, the industry experts leaned on old-school methods -- the RCT -- to evaluate vaccines. Current talk of “data science” was put aside. This revealed that data science’s promise of “real-world data”, “advanced analytics” and “data insights” is mostly the marketing of software and degrees and there is very little that has seeped into regulatory statistics from this miscellaneous collection of tools. Generally speaking, you cannot have real-world data until the product is on the market and thus it cannot inform the approvals process; the black-box nature of machine learning is not palatable in clinical research or easy to validate; causal inference implemented via mediation analysis has not been widely accepted due its speculative nature, e.g., estimates showing wide confidence intervals and spurious conclusions (the proportion mediated exceeding 100%); and patient prediction, much like real world data, is hyped and remains the exclusive interest of academics and marketeers. The Data Scientists have suggested some adjustments that tinker at the edges for a gain in statistical efficiency (EMA CHMP, 2022), but this betrays their inexperience: the regulator will make you compare this model with the standard ANCOVA model and their own simulations show that the efficiency gain is lost compared with the standard baseline covariate adjustment. I.e., there is no gain. Also, when running a large RCT a small increase in efficiency is beside the point in comparison to the many other more relevant concerns. Frankly, a lot of the tools offered by the Data Scientists are just old ideas restored by an increase in computer power and a lack of inside information.

Hence the staid RCT remains unaffected by the evidence-based medicine experts and the Data Scientists who may opine that it is old hat (Schattner, 2024) and wish to supplant it in their hierarchy of evidence, or at least have some influence over it. The failed attempt to rebrand statistics as data science (which most people did not even detect) was fundamentally an attempt to open up statistics and thus open up the discussion.

Ethics and the need for clarity

Bearing all of this in mind, the claim by The Times (2025) in February that the “trial would turn children into guinea pigs” has it backwards. I.e., until the Cass review landed and the drugs were banned, children were treated as guinea pigs, and a randomised trial yielding a persuasive result would thus be needed to inform a standard of care. Currently, if many parents are doing their own reasearch, looking into treatment options, then a definitive but negative trial result could discourage access to PSH through unsafe means, and also alleviate the concerns of those who believe claims of suicide are used to manipulate parents into supporting early treatment.

The visceral response, even among the journalists, relates to the exposure of children to experimental drugs (e.g., McColm, 2025: “We know that experimenting on children is monstrous. We know it.”). At this moment, all over the world, children are entering clinical trials in rare conditions where there are limited treatment options. Sample sizes are small and data quality issues arise because the condition is rare: investigator sites are inexperienced and historical controls may be needed for comparison. Statistical methods on these data produce a range of plausible estimates of efficacy and it is possible the experimental drug is merely a placebo with a side-effect. I.e., evidence gathering is not always straightforward, especially in these circumstances.

In the case of puberty blockers, in the absence of a quality RCT, evidence is even more sloppy and ambivalent. At least it appears reasonable to say that rates of depression are higher within this cohort: transfeminine 49%; transmasculine 62% (Becerra-Culqui et al., 2018); likewise for suicide (Suarez et al. 2024). There is some evidence of an effect of puberty blockers or gender-affirming hormones on depression and suicidality (Tordoff et al. 2022) but this particular analysis is highly contested (DHHS, 2025). Alternatively, a paradoxical age-dependent risk of suicide from anti-depressants (SSRI’s) has been described leading the FDA to add a black box warning in 2004 indicating a higher risk of suicidal ideation in young people. Naturally, this has also been questioned (Friedman, 2014).

While surveying these results we must also reckon with a perverse bias seen in psychiatric disorders: based on small event rates we are inclined to speculate that a treatment incites depression as an adverse reaction (e.g. media coverage of accutaine in the mid-1990s), while promoting cynicism about treatments intended to alleviate depression (much of the commentary on SSRI’s, e.g. Angell’s book The Truth About the Drug Companies). For whatever reason, in certain conditions we tend to believe the evidence is strong when the effects are unwanted and weak when they are desired. Or maybe we believe that a null result is less likely to be affected by bias than a result away from the null?

For example, Jordan Peterson has stated that “[t]here is no such [evidence underlying pediatric gender medicine] and every physician and psychologist worthy of their designations knows it”. But then he emphasised in at least 3 separate posts in one month (1, 2, 3) a feeble analysis of uncontrolled data suggesting an increased risk of suicide after gender affirming surgery. (‘Feeble’ because propenstiy score matching, the method used, has been shown to increase “imbalance, inefficiency, model dependence, and bias” (King & Nielsen, 2019), although the authors of this research describe it as “well-documented regarding its properties for statistical inference”.) Likewise, in discussion of the PATHWAYS trial, those against the trial use prior data to point to the risk of side effects while noting prior data are too weak to suggest any benefit exists, e.g., Genspect, 2024). Thus there is a rebuke aimed at clinicians for a lack of evidence and a subsequent rebuke awaiting them when they attempt to gather evidence (via the RCT).

Any such cynicism is aided by the fact that safety is generally more difficult to establish. To make the point and preempt haphazard interpretation of adverse events, one can plot estimates of the risk ratio for each adverse event type and arrange them in ascending order, with the value 1 appearing somewhere in the middle of the sequence: if we want to consider those at one end as causative, e.g. eye disorders, then we must consider those at the other end as preventative, e.g. hair loss (no one will want to do that). It is possible to design a study around a net-benefit type composite that captures safety by incorporating, e.g., discontinuation due to adverse events (see Composite variable strategies in ICH E9(R1) addendum). However the PATHWAYS trial does not lend itself to this approach and the sample size calculation would become more uncertain: there is uncertainty around each endpoint and amalgamating endpoints compounds uncertainty (it would have to be based on simulations and require a highly skilled programmer).

Thus we are left with the hope that the PATHWAYS trial built around efficicay is senstive enough to yield a compelling result.

Brief review of the PATHWAYS trial

Given this background, equipoise etc., the most pertinent considerations for an ethics committee become study design, quality (site monitoring), the ability to meet recruitment targets, and the drop-out rate (Fogel, 2018). Quality and recruitment rates are common concerns with any trial run by the academics: academic clinicians can be indecisive with data analysis and lack resources and standards (e.g. they are unlikely to have a dedicated programming resource, the statistician then writes the code, but it takes at least two seasoned programmers to write and validate code).

Looking over the study protocol and the standard operating procedures at the King’s Health Partners Clinical Trials Office (KHP-CTO, n.d.), one is reassured quickly that these researchers are up-to-date and they know what they are doing. However, although the analysis plan remains to be drafted, based on the analyses specified in the protocol we can say the trial is not designed to deliver a compelling verdict and will very likely produce equivocal results that elicit yet more discussion and controversy.

The primary analysis is Bayesian (with uninformative priors) which implies no statistical penalty for “peeking” when running an interim analysis. Although no interim analysis is planned it may be prompted by the external monitoring committee. Under these circumstances a regulator would usually demand that the researchers evaluate the operating characteristics of the set-up, meaning they would have to run simulations a priori to evaluate the Type I and II error rates. (Despite recent claims from the FDA and new draft guidance, the regulator won’t go full Bayes -- this would open up the analysis. Consider The Times, already noted, who say based on current data “there is strong prima facie reason to think ... any supposed benefits [are] negligible”. Their priors would overwhelm the analysis. With uninformative priors Bayesian estimates can look a lot like the classical estimates, only the interpretation changes.)

Also, the estimate of the effect is assumed conservative (the “treatment policy” estimand) because it ignores, e.g., participant drop-out. But if paticipants in the delayed arm drop-out prior to expousre this seems relevant. Thus the primary estimand may be contradicted by sensitivity analyses that target a pure effect by imputing data after the participant discontinues. To make things worse, the effect estimate lacks cogency -- the difference between means of the composite score derived from the 10-point questionnaire -- and this will push emphasis onto statistical significance when the power has been depleted by the estimand strategy. They should instead consider a transformation model which can offer more power (Buri et al. 2020) and, most importantly, produce meaningful estimates of the effect that are easy to communicate (exceedance probabilities). Otherwise, an unclear result begets further exploratory analyses and additional publications, e.g. subgroup analyses which will be under-powered.

In other words, things look imprecise and open to second-guessing by commentators who can claim they would have done things differently. The academics are inclined to use non-conventional approaches such as Bayesian methods, and for the most part that is a good thing. Convention in industry is not always a result of regulatory guidelines being imposed on analyses but instead a desire for efficiency and simplicity that is so strong it can perpetuate bad methods for years (see last-observation-carried-forward). However, the downside of the more liberal mindset of the academics is that indecision produces something that appears cobbled together and vulnerable to re-interpretation, and this is regrettable when the trial is of high public interest: see the trial of ivermectin that appeared during COVID, boasting a Bayesian primary analysis with adjustments for multiple testing -- a strange juxtapostion of Bayesian leanings and strict adherence to frequentist concepts (PRINCIPLE Trial Collaborative Group, 2024).

Under the circumstances, then, we can say the analytical approach taken in PATHWAYS lacks prudence.

The media’s peripheral understanding

We are operating in an environemnt where even the journalists who are eager to report on medical research fail to understand the RCT and spread confusion. Regarding PATHWAYS, Barnes (2025) thinks: 1) The psychosocial support is a confounder (the protocol indicates that psychosocial screening is employed “to identify any unmet needs and escalate these to appropriate services.”). Barnes says: “It is unclear how this design will allow the research team to judge to what extent any observed benefits or harms are due to the medical intervention, as opposed to the other support being provided alongside”. 2) Allocation of treatment to participants is based on matching; e.g.: participants will be “paired with someone with a similar condition”. 3) When referring to the exclusion criteria for entry into the trial Barnes notes that the sample is not representative and will skew evaluation of the drug: “The study has several entry criteria, but who is excluded is perhaps equally important, and highlights some of the difficulties the trial faces. … [A] significant number of children who might potentially benefit [are] ruled out, potentially skewing fair evaluation of the drugs”. At moments in the piece it is strongly implied that the design is inherently flawed as a result.

However, these are quite common misunderstandings. 1) Clinical trials are comparative (we do not put confidence intervals on individual treatment group estimates) and standard care, in this case psychosocial support, cancels out when calculating the treatment difference, i.e. the randomisation takes care of it. Note that any placebo effect is considered part of the measurable effect of treatment due to the open-label design, including variable application of psychosocial support. 2) The PATHWAYS trial uses minimisation to allocate participants to treatments. This is a reasonable approach despite well-known criticism (Senn, 2008). Describing this dynamic allocation as “matching” is simply a misunderstanding of the method or an improper description of it that would elicit concern. Matching will be used for the non-randomised arm and tends to over-promise (already noted above). 3) This is a typical ‘outsider’ perspective of clinical trials. As already noted, trials are not representative, they are comparative. Barnes has not identified a bias (a term that relates to the treatment effect estimate) alluded to by the word “skew” which is not apt. Researchers often discuss rectruiting patients who are sensitive to treatment (“enrichment”) and you may have a run-in period to identify them; and it is typical to exlcude patients who are previously exposed. None of this is unusual and the treatment effect itself tends to be constant across subgroups (Harrell, 2019).

Barnes raises further suspicion when noting the researchers failed to answer questions, e.g.: after the two year follow-up, will “data collected from those [who continue on puberty blockers] be included in any formal analysis”. Of course they will be in the primary analysis, this is a requirement. But analysing on a subset at some point beyond this loses the randomisation and the p-value loses its interpretation. Thus it would be descriptive statistics at best.

Such misunderstandings emphasise that, although the RCT appears to be a simple, old-school tool, its precise motivation and justification is sufficiently rarefied for important books to be written about its analysis, leaving many issues unknown to the journalists. Although some journalists, almost to their credit, ignore the clinical and statistical context completely (e.g. The Times article noted above) while podcasters are already discussing the trial and producing bewildering non sequiturs, e.g. James Esses: “potentially unlimited numbers of children can sign up to this trial” (McCormack, 2025). Alongside the podcasters are the Substackers who revelled in the ivermectin RCTs during COVID: one person published nearly 30 articles on the TOGETHER trial of ivermectin in his Substack titled Do Your Own Research (Marinos, 2022), and others made accusations of fraud (Kory, 2022). This activity can be spurred on by some academics who insist on doing their own re-analysis of the data (Bryant et al. 2021).

Thus the discussion is pulled left and right by sideliners, and instead of placating them with a clean result, the PATHWAYS investigators are likely to generate a grey area in which they can manoeuvre.

Conclusion

The PATHWAYS trial should not have received ethics approval because it is unlikely to be successful and if the trial produces a negative result it will be easy to dismiss due to insufficient statistical power, quality issues and the estimand strategy. Trials have become complex and the cost of running an RCT provides strong motivation to reduce sample sizes by implementing interim analyses, meta-analysis or Bayes. But under the circumstances these analytic choices feel needlessly complicated and uncertain (e.g., the interim analysis is a mere possibility). Who knows how those who “do their own reasearch” will misinterpret such choices, or in fact how they will interpret the inevitable systematic review of the failed small-scale studies languishing in the medical journals. The need for academics to publish can lead to over-analysis of datasets and as we saw during COVID the data sleuths feed on ambiguity and their guesswork proliferates on social media.

I.e.: the PATHWAYS trial should not go ahead in its current form because it will only generate more fodder for the sideliners. The analysis plan could resolve some issues.

Give us your thoughts:

I grew up in a town that others would describe as backwards. I was told, with curt assurance, farmers can predict the weather more accurately than meteorologists. People despised scientists, especially academic ones. They shunned the rigour of experiment and preferred personal trial-and-error. They practised dowsing (a pseudoscientific method of finding water underground), lauded echinacea as a panacea, and boasted about disgreements they'd incited with their GP. When I entered science I tried to keep it to myself and was ridiculed by those who found out: “You want to cower from the real world.”

Yet I always retained admiration for their (misplaced) distrust. In a way they were right: I learned, as I worked in various regions of the world, that the academics lacked resources, everywhere I went, and they were self-promotional, error-prone, and no one was monitoring them.

With fondness I recognise in online forums today that refusal to tolerate extraneous wisdom. The difference is: they now reciprocate others' curiosity. They are saying “what's your source?” when a comment arises that might be informed by data. It fills me with contentment. They are engaging with science and demanding a lot from her, including unequivocal prescriptions: “Do masks work or not?” (They do not know that this is best answered by a large pragmatic trial - an administrative nightmare, likely a mess with insufficient quality control, contaminated with non-adherence and missing data.)

There is a crudeness to their interpretation of results, lacking the specificity of instinct that accumulates through practice (e.g., a seasoned programmer can sense where an error resides in ten thousand lines of code). I try to tell myself: perhaps they will do some good by inadvertently energising the open-science movement which the academics support only nominally. But it seems they are more likely to fracture interpretation when they point to inconsistencies they do not understand, and infer that scientists have withheld information or misled them.

In an attempt to foment a shared, coherent scepticism, and pre-empt the cynical curators of opinion (the content creators), let us lay bare the feebleness of scientific data and the prevarications that often derive from them. Because, primarily, there is a need for consistency in interpretation of results which implies a common ability to detect the limitations of data that can make science appear reticent or even contradictory.

Without any disrespect, I refer to those on the periphery as outsiders or amateurs - that is who I am talking to, my old acquaintances back home (as if they still exist). I am just like them, only I have seen behind the curtain and can say: your penchant for scepticism is sound, I'm there with you, but you must be careful who you listen to.

A striking example to get us started is the apparent waning of the estimated effect of the COVID vaccine as we moved from Pharma-funded randomised controlled trials (RCT) to so-called real-world data (RWD), i.e. post-approval, observational data that are out of the hands of Pharma. A podcaster pondering the disconnect between RCT and RWD estimates said he felt the RCT overestimated the effect, thus indicating a deficiency in the study design and begging the question: why did the regulator allow it? Another insinuated something underhanded is going on; collusion between Pharma and the regulator to yield the inflated and artificial estimate.

This misunderstanding even ended up in court precedings in Canada, i.e., in the cross examination of Celia Lourenco, Director General of Health Canada, in the constitutional challenge brought by the Leader of the People’s Party of Canada et al.:

“[Dr Lourenco] confirms that the [RCTs] are required to demonstrate that the vaccine reduces symptomatic COVID by at least 50\%. The lawyer asks Lourenco if given the waning efficacy provided by the Public Health Agency of Canada ... would [the injections] still qualify for authorization ... if that had been observed in the trial”. [The 50% threshold is nominal and used to design the study.]

Dr Lourenco’s affirmation was offered as a shocking revelation by sceptics who appear swayed by the RWD descriptor. But RCT and RWD are not addressing the same question, i.e. statistically, they are not estimating the same quantity. The purpose of the RCT is to establish efficacy and accumulate exposure to explore safety; hence emphasis on patient follow-up, adherence to a study protocol and prespecified, conservative analyses. The RWD has a different purpose entirely, and the effect may be representative yet diluted; the data suffer from quality issues, missing data and other limitations.

Also note that we do not have RWD until the drug is approved and on the market; ipso facto, RWD cannot inform approval, and the lawyer's question is nonsensical. It is the same with saftey, i.e. the safety profile of a drug is not fully articulated until it enters the market against a backdrop of concomitant medications including off-label use. Safety detection, ultimately, amounts to a type I/type II error problem (over-reacting to limitless spurious signals and failing to react swiftly to legitimate signals), and the arbitrary setting of these thresholds. (I have noticed, on a drug's Wikipedia page, the list of potential side effects expand after the patent expires.)

It is ironic that some on the political right contrast RWD with RCT to suggest a compromised approvals process (as in the case above). In 2005 conservatives in America were pushing for reform that would permit other forms of data for consideration, i.e. non-RCT data, to speed up the approvals process. In other words, they were seeking to lower the threshold of evidentiary strength required for approval, a sentiment that contradicts conservative voices today.

At the time, the Society for Clinical Trials responded to this call from Republicans with a position paper; this paper provides concise examples of non-RCT data leading us astray:

“The theory that beta-carotene could prevent lung cancer was widely accepted based on retrospective (uncontrolled) epidemiologic studies of dietary beta-carotene consumption, yet two very large cancer prevention studies in the 1990s demonstrated convincingly that beta-carotene supplementation to smokers actually increases the incidence of lung cancer, to the great surprise of the medical community. Combined estrogen and progestin therapy was widely believed to reduce the risk of coronary heart disease (CHD) in post-menopausal women, yet the Women’s Health Initiative Study showed convincingly the opposite conclusion, that the treatment actually increased risk of myocardial infarction and CHD death compared to placebo treatment.”

Conclusions extracted from RWD are suggestive: the data are messy and effects are entangled (causal inference experts would chime in here, but their methods produce wide confindence intervals). The appeal of the large-scale RCT is that it yields a primary analysis that is straightforward (a clean estimate of the effect is calculable) and easily communicated. Although there are certainly poorly designed and premature RCTs which we should hesitate to overinterpret. And even high quality RCTs are routinely misinterpreted by the amateurs in their substacks and podcasts.

Consider MidwesternDoc's substack Forgotten Side of Medicine with over 100 thousand subscribers. In a post titled The Great Ozempic Scam and The Safe Ways to Lose Weight that was retweeted by RFK Jr (generating 1.5M views), referring to the pivotal RCT of semaglutide (Ozempic), the Doc declares: “Most of the participants could not stay on the drugs for a prolonged period”, alluding to safety and compliance concerns.

But the Doc has simply misunderstood the plot (this is not surprising; the Doc is an outsider). This trial, the SELECT trial, had an interim analysis; patients were recruited in a staggered fashion; then the interim analysis occured at the pre-specified time according to the accrual of cardiovascular events; this analysis determined that the study should be terminated. This explains why the number of patients is trailing off: the follow-up time is variable, it has nothing to do with compliance. The Doc also wanted his readers to know that 8% of those on semaglutide had serious adverse events. He failed to note that the corresponding number for the control group was 12%.

Aside from RWD heralded by the amateurs during COVID, we also saw meta-analysis offered as the pinnacle of evidentiary strength. Those who swallowed Goldacre's book Bad Pharma will reiterate this point-of-view; e.g. Bret Weinstein was excited by the meta-analysis of ivermectin. More recently, Jordan Peterson described the Cochrane collaboration as “the gold standard” when Cochrane's systematic review of masking landed. Others subsequently stole Peterson's sentiment and it spread as fact. Are they right? If I have spoken positively about RCTs, surely I must have a positive opinion of a meta-analysis of RCTs?

Confidence is the hallmark of the outsider. Peterson wouldn't know, for example, that the Cochrane software contained limitations that they didn't bother to fix for years (see Senn for details), and Cochrane leaders themselves describe systematic reviewing as like “searching through rubbish” and their reviews often conclude with a call for a well-desinged RCT (why would they want to take a step down their heirarchy of evidence?).

The reality is, meta-analysis has been on the ropes since 1997 when the Lancet published a meta-analysis of 89 homeopathy trials and declared homeopathy superior to placebo. Because homeopathy is a placebo, this was not a test of homeopathy but a test of meta-analysis which reliably compounded biases across trials. This result was considered a serious blow from which retrospective meta-analysis hasn't recovered. (The problem is intractable: you cannot repair low quality data by increasing statistical sophisticaion; if you try to solve the quality and heterogeneity problem you introduce a subjectivity problem.)

When the ivermectin meta-analysis was called into question due to data quality issues (exactly as predicted by Yuri Deigin) the insiders did not flinch, while the amateurs scrambled to rerun their analyses in Cochrane's plug-and-play online tool (a toy no statistician would ever use). It was obvious to insiders at the time, and should be obvious to all now, that the EMA's position on ivermectin was not dismissive. They were spot on. Those who offer themselves as sceptics were not sufficiently sceptical of meta-analysis. Promoting scepticism on some topic only meant re-allocating their credulity elsewhere.

There is something else, finally, I would like to clear up. All the talk (from Prasad, Peterson et al.) about the movement of staff between the medicines agency and Pharma: it is never clear what these commentators are concerned about. Perhaps it will suffice to note that those in regulation look askance at their counterparts in industry, and they are equipped to scrutinise them: they have all the company's data and code, they can run their own analyses on them - the academics' peer-review process looks futile in comparison.

The allusions to an easy-ride for Pharma betray the outsiders' ignorance, e.g., the sight of a drug company approaching an advisory committee meeting, they are almost out of their minds with anxiety. (I have seen a company produce over 30 thousand outputs in preparation.) The only time I have seen a representative plead with the agency to approve a drug was when I met with the academic investigator responsible for a trial in a rare disease (the key data supporting approval). This is worth repeating, as an antidote to the bleating anti-Pharma types: an academic was pleading with us to approve based on his flimsy data (he would not want to be known as the guy who loudly touted a drug that turned out to be a dud).

Let us bear these various details in mind, and we can fortify and hone the sceptical perspective. Data are coy, suggestive, they tease. Early phase results may be hyped in academia in an effort to win funding, but there is no analogous incentive for Big Pharma: why would the company want to fool itself into investing further and progressing to Phase III by hyping phase II results? In the public sphere ambiguity begets disagreeemnt and the outsiders will try to persuade you that it is their distance (i.e. their ignorance) from the institutions they seek to reform that confirms their integrity. You must instead find honest and informed insiders (scientists, not executives) to listen to. Unlike the outsiders, the insiders are not guessing, and thus not impressionable.

Subscribe now

I’m in middle Norway. I met a foreign scientist. We were talking about solitude and the winter darkness when I asked how life was for him during COVID. He hesitated, then said something like:

“I am apolitical. I do not vote - I am never eligible - fine with me. I can hardly hear or read the local language, and if there was a queue for the vaccine then I assume I’m at the end of it. I went on as normal: I walked to my office every day. If there had been a lockdown, I would not have known. The streets had always been empty for me because of the unusual hours I work; I walk in the middle of the street - because of the ice and snow piled up on the footpath - all the way home, up the mountainside. The stores were always open and I never wore a mask; once I wore a mask: I ended up in the emergency room, I’d suffered a fall, on the ice; the doctor told me to take the mask off becuase I wouldn’t stop fidgeting with it; I’d put it on uspide down, it kept sliding down because it didn't pinch the nose. … And when I visited the post office to inquire why books I ordered from America weren't arriving the staff lamented that everything arriving from America was quarantined. Quarantined?! We are in the midst of a pandemic, they said. I had forgotten, a year into the pandemic and it was still not on my mind. I am not riled up as some are about it, it hardly affected me, I hardly noticed. I just wanted my books.”

Subscribe now

MidwesternDoc has a substack titled Forgotten Side of Medicine which the Doc says has 109k subscribers.

Initially I sniped at him (above); it feels sufficient, and who can find any time to evaluate the kooks’ output?

But now I’m looking for a distraction, and I recall the Doc. I glance at the post he’s promoting in the X thread: The Great Ozempic Scam and The Safe Ways to Lose Weight.

Spurious claims are made with sloppy language and left hanging, e.g. the FDA pushes Ozempic and Rob Califf (head of the FDA) has ‘immense conflicts of interest’. But I am sifting thru these words, hunting for a comment, this:

One thing I’ve been immensely curious about is what life is actually like behind the scenes within the pharmaceutical industry.

Okay: the Doc is a doc; he’s uninitiated, he’s ignorant - of course. And apparently he would like to hear from us (insiders). There is something else to bear in mind: within the drug industry docs are widely known to be feeble (due to their ignorance) and easily cajoled.

A case in point: the Doc is inclined to mention the opioid scandal. How could the scandal have occurred without the willing cooperation of the docs? Bad Pharma. Bad Docs? We should not wait for Goldacre to pen that book.

After confirming the Doc’s ignorance, I wanted to find something legitimate, a specific claim that can be confronted head-on. Thus: jump to the section titled The Risks and Benefits of Ozempic.

Referring to the pivotal clinical trial of semaglutide, the Doc declares: “Most of the participants could not stay on the drugs for a prolonged period.”

I can point out that the Doc has simply misunderstood this plot, but it’s more important to understand what the error says about the Doc, than the error itself.

This trial, the SELECT trial, had an interim analysis; patients are recruited in a staggered fashion; then the interim analysis occurs at the pre-specified time according to the accrual of events; the interim analysis determines that the study should be terminated (according to the stopping rule inherent in the study design).

This explains why the number of patients is trailing off: the follow-up is variable, it has nothing to do with compliance.

There is a lot more that could be said about compliance, estimands, the complexity of multiple imputation methods for handling missing data, mixed modelling for repeated measures, etc., but none of it is relevant to the point I wish to make.

The point is that the Doc does not understand trial design and he is telling his readers, one hundred and nine thousand of them, a falsehood. More to the point: the detail of the study design is freely available on clinicaltrials.gov, and the drop-out rate he implies would not be seen in a large quality trial like this (anyone with any time in drug development would know that).

The Doc continues ... He finds it unreasonable that the weight loss attributable to the drug is partially regained when the patient is taken off drug - figure below. But it is a lot to expect of a drug, no? I.e. that it continues to work when you are no longer taking it. And if the Doc were a good cynic, rather than a mediocre one, he would ponder how the company might use this result in their marketing materials.

The Doc then moves to safety and misleads the reader in a way that is incredibly hack. He displays the adverse events reported in the trial for the drug and not for the control (placebo). It is somehow more dishonest when the distortion lacks originality. E.g., we can see that 8% of those on the active drug had serious adverse events! (For the placebo group it was 12%.)

The Doc is inciting distrust and disguises his incentives by using ‘Bad Pharma’ as a counterpoint: he’s alerting us to the misdeeds of greedy Pharma.

If the reader remembers one thing they should remember that the Doc confessed he’s an outsider; we can infer from this that he has never met Califf or collaborated with him, although he insinuates that he’s corrupt; and he has never been privy to regulatory interactions or designed a trial etc.; i.e. he’s guessing.

If there were time we would like to comment more on safety. (People may not realise GLP-1 agonists have been around for many years and there have been ongoing safety discussions with the regulatory authorities.) Maybe another time.

It would be convenient to make brief posts like this one, at least it seems doable. We will title them ‘Just quickly ...’

Subscribe now

Sheiner (1991) regarding the Intent-to-Treat principle: “[W]hy would anybody in their right mind advocate acting as though what they meant to happen did happen when they knew for sure that it did not?”

Despite these standards, academic clinicians flaunt their disapproval of industry with relentless crowd-pleasing books. To maintain legitimacy, the critics observe Pharma from the outskirts and, thus, betray a superficial understanding of the system they seek to reform. As clinicians they also miss the subtly of statistical arguments - where scrutiny would be best applied.

The point is: there is something seedy going on in Pharma and these critics miss it. It is behind the scenes and too esoteric for them to detect, no matter how eager they are to define themselves by their anti-Pharma posturing. And there's an added irony: the corruption is introduced by the academics themselves. I.e., the academics are sanctifying and ushering in a new mindset that offers Pharma what it has always craved: “the liberation of analyses” (an actual phrase heard in Pharma), and new ambiguity in trial results.

Some background is necessary: An industry guideline stipulates that a clinical trial must have “an appropriately qualified” statistician assigned to it. Industry always interpreted this to mean an MSc, but there are too many trials and not enough MSc level statisticians, leading Industry to fund MSc courses in the late 90s. This seemed a smart move. Courses grew in number and were attracting more students who all entered industry upon graduation, as they'd agreed. But many did not stay in their employment; the value of the degrees was tarnished as they became softened; and supply still did not meet demand.

Some 20 years later Data Science develops and makes itself known to a wider audience. The term is swiftly adopted by those who wish to appear current. Talking as if Data Science is Statistics 2.0 was a simple error made by the journalists that enhanced its apparent relevance for young scientists; combined of course with the glamour of AI and machine learning (ML) and the sudden declaration that the Fathers of modern statistics were racists.

Online magazines sprung up like Towards Data Science where influencers teach statistics to data scientists minus the history and difficult theory (statistics rebranded). Masters degrees in Statistics & Data Science began to appear; statistical societies added these degrees to their lists of accredited courses that lead to certification, e.g. Chartered Statistician; hence satisfying the regulatory requirement of “appropriately qualified” and expanding the recruitment pool. Bear in mind, there is no traditional route to accreditation for the data scientists and they are an eclectic and ill-defined group.

But how to import this new crop of analysts into drug development? I.e., how to relax requirements and change habits comprehensively?

Data Science has clout, and this is where the status-chasers come in, i.e. the academics described above who celebrate their intolerance of Bad Pharma with longwinded best-sellers. In fact, that was the title of academic clinician Ben Goldacre's book: Bad Pharma. With the style of a British tabloid newspaper, it provides an out-of-date and grotesque description of Pharma that won over sideliners who wallow in cynicism. Heather Heying held it aloft on the Dark Horse podcast and declared it a “terrifying book”.

For a long time Goldacre was in evidence-based medicine (EBM). Motivated by a distrust of Pharma, EBM proponents place meta-analysis atop their heirarchy of evidence, rather than the costly, industry-run clinical trial. The EBM enterprise lingers but could be declared a failure given that leaders confess they now think “systematic reviewing [is like] searching through rubbish”, and Goldacre's own Open Trial initiative backfired when it found that academia are much more likely than industry to fail to publish trial results, and Big Pharma is better than smaller Pharma. The Open Trials X account stopped posting 5 years ago, and the EBM proponents were silent before and after the Cass review landed.

“Trials with a commercial sponsor were substantially more likely to post results than those with a non-commercial sponsor (68.1% v 11.0% ...); as were trials by a sponsor who conducted a large number of trials (77.9% v 18.4% ...).”

Goldacre pivoted to Data Science where he became inaugural Director of the Institute for Applied Data Science at Oxford which promised to “generate new data and evidence, but also make it more impactful in the world”. It is important to notice the language used by data science advocates; it is often self-promotional marketing-speak like impactful, innovation and extracting insights from big data. Also worth noting is that Pharma are pouring obscene sums into these newly established data science centres, and, needless to say, the cash-strapped academics are clamouring to inform the public about the promise of “big data”. (It turns out one can draw parallels with the greedy Pharma execs the academics decry, only the academics are not selling drugs, they are selling themselves.)

The key issue to understand is the intentional blurring of the boundaries of these distinct fields. Each nurtures a different mindset about how one thinks about, and handles, data. Statisticians treat data as sacrosanct and handle code the same way, whereas data scientists talk about cleaning data and are happy to make use of a script found online; a careless attitude that is an affront to the statistician's sensibilities. There's a departure from strict pre-specification too; the model learns, it's dynamic and ad hoc, and the blackbox nature provides contentment for those lacking clinical understanding.

Ultimately there's a claim (a misunderstanding in fact) that the clinical trial is limited in scope and we ought to exploit uncontrolled, real world data. In the absence of clinical trials the data scientists “overclaim the usefulness and applicability of [their ML tools] to solve clinical problems”. If those who lament Pharma were sincere they should have plenty to say about promoting products off of cobbled-together data sources. After all, a bad algorithm that informs patient triage can do just as much harm, if not more, than a bad drug. Instead we witness an EBM person move to data science and upend their own heirarchy of evidence, without flinching.

In every case these examples imply an opening up of methods and a simultaneous relaxing of standards. And being open-minded in this way, i.e. showing one is unconstrained by old habits, has become the thing to espouse.

Within industry, new departments of data science are formed and statisticians are rebranded. Heads of the new groups talk about data analytics and insights and the need for statisticians to be open-minded and collaborative. Some state that those who are not onboard will be left behind and are, until then, an obstacle. These higher-ups are not statisticians or data scientists or computer scientists. They are VPs, i.e. MDs and careerists transfixed by the hype and promise of data science. They do not understand any of it in detail, they only know they are excited about it. They stand under lights on stage and declare “We can do it!” Do what? Who knows.

The statistician's presence was always considered a necessary burden by them; the dogged adherence to guidelines and strict validation is considered stifling, slow and unhelpful. But the statistician knows how to speak with, and appease, the regulator whose requests are ultimately statistical in nature. Thus the leader will smile at the statistician, but counter their suggestions by promoting a higher-level understanding which usually amounts to something reprehensible (deviating from the analysis plan to diminish a safety signal, etc.).

This is not to downplay the work of the data scientists. It would be superfluous to describe the value of this burgeoning field as it is properly defined, e.g. in understanding what is happening at the protein level. It is always interesting and inspiring to hear what they are up to. But those seeking reform should be thwarted by obvious inherent problems: you cannot have big data until the drug is on the market, thus: big data cannot inform approval.

We wish to affirm only the following: Statistics and Data Science should be treated as distinct fields that interact; Pharma's instinct will always be to create a grey-area in which their marketing teams can manoeuvre; the academics and their journals abet Pharma by not noticing that what is sold as cutting-edge is often not cutting-edge, and hence it is indistinguishable from marketing; when it is cutting-edge, it is, therefore, likely tentative and awaiting verification.

Academia is leaking public trust. Industry, with its high quality randomised controlled trials and diligence, obviated the reproducibility crisis that still affects academic research today. Industry has the opportunity to recover the trust in science spent by the academics.

"I've discovered that the whole biomedical publishing system is fraudulent. It is completely based on fraud. It is completely controlled. The major medical journals dictate what is written. There is such things as ghost writers. They determine the outcome of the study, and then they write about the outcome of the study which can be completely made up by people who never participated in the study ... Big pharma is behind this. There's no question of doubt that if a paper, even if it's an [RCT] is financed by pharma, or supported by pharma, or looking at a pharmaceutical product,..... you can be guaranteed, 100%, in someway or another, it's fraudulent."

Boghossian is entertained by Marik for about 15 minutes before he lunges at his ultimate conclusion (18m15s): Doctors are informed by the medical literature and therefore: "Doctors have been misled; they have been brainwashed; mind games played with them, by this comglomerate; and they are providing false information to patients ... Most of what doctors tell you is based on fraud; doctors don't know this."

The claim elicits a laugh from Boghossian who seems equal parts amused and perturbed. When the podcast ends, and they have signed-off, Boghossian is heard to say off-mic: "Holy fuck. Jesus Christ." For nearly an hour he's been swallowing Marik's claims like an Englishman on his stag; and now he's reeling.

Watching an intoxicated and amused Boghossian override his characteristic style is intriguing; much more so than Marik's wild and unoriginal claims. But we said we would respond to Marik, so let's do that and return to Boghossian at the end of the post.

And let's merely flag that we are unlikely to hear any precise or vulnerable statements from Marik. His words will be opaque and scattershot, and because Boghossian has become impotent, we will be on our own in making sense of it. (Boghossian has around 200 thousand subscribers on youtube; this video has 13 thousand views since March 26. Presumably many tapped-out early.)

The crux of Marik's claim is that companies fix the outcome of their clinical trial. How is this possible when the study is pre-registered on clinicaltrials.gov which displays the principal investigator and study design: duration, outcome derivation, timing of analyses, etc? Marik hangs his assertion on the anonymous medical writers who "never participated in the study". (He uses the term ghost writers, either to denigrate them or out of ignorance.)

Ben Goldacre described Pharma's use of medical writers in his book Bad Pharma (throughout Chapter 6 on Marketing). Goldacre, like all the others who flog anti-pharma sentiment, has not worked in industry and his description was grotesque. But Marik, a fervent observer, is the type to be swayed by it, just like Bret Weinstein who started banging on about regulatory capture after reading Goldacre.

It was never true that medical writers "determine the outcome of the study" (see the full quote above). It is not possible to know what Marik could mean by this. The outcome of the study is an estimate of the pre-specified primary estimand calculated by the statistician - it is this estimate that determines whether the trial is a success or not. How can the medical writer affect this estimate when the statistician is the only person in the room who fully understands it?

Marik might claim that there is leeway in the Discussion section of the paper to amplify certain findings and formulate a narrative. However, a trial is a rigid tool and does not generate much ambiguity; and those who read medical journals are qualified and harbour scepticism about, e.g., the plethora of secondary endpoints (to some degree these outcomes are intended to distinguish between the company's product and a competitor's and scepticism may be warranted).

In any case, the medical writer does not draft the Discussion section: the authors do, and the journals require them to state their contribution and sign-off on it.

The medical writer performs administrative tasks, e.g., a literature search to aid the authors when drafting the Background section; proof-reading; collating feedback from co-authors; creating enhanced displays to meet the journal's standards; etc. These practices vary among companies and the larger companies are most likely to insist that authors make a genuine contribution. I.e., people like Marik are concerned about Big Pharma but they should be much more concerned about small and medium-sized Pharma, as seen recently here:

Since Zlokovic and his co-authors no longer had the original data for one of the questioned figures, the editors wrote, “[r]eaders are therefore alerted to interpret these results with caution.”

The most persuasive and accurate critique of pharma contradicts the argument Marik is making: far from enticing favourable results out of data, Pharma analyses lack flexibility and are straight-jacketed by excessive pre-specification (it is difficult to fully pre-specify an elaborate model and thus a crude model is favoured). Industry analyses exhibit tired habits, mostly for the sake of efficiency and to minimise the possibility of coding errors, and not for scientific reasons such as parsimony.

The problem for Marik is that it is the academics who are guilty of running ad hoc analyses and massaging results. Marik would know this and this makes him appear disingenuous.

Yet there is legitimate criticism that Marik skirts regarding the exaggerated contribution of the authors. Typically Pharma identifies key opinion leaders in the field, referred to as KOLs, who become the lead author and co-authors of the paper. These academic clinicians play a minor role in the design and conduct of the study (e.g. interim analyses); mostly they contribute to the interpretation of the results and putting them in context.

It is amusing, then, when the lead author is seen discussing “their research” in major news outlets after you have described the results to them. It's a symbiotic relationship with the academics getting the attention they crave and the company borrowing the credibility of the researcher. The truth is: the academics are overconfident and request stupid analyses that leave them confused among a surplus of post hoc estimates. Perhaps they exert themselves because they are too keenly aware that they are superfluous. But that is quite a different criticism than the one Marik is making.

Marik's concern regarding peer-review of the submitted paper also misses the mark. Peer reviewers are an afterthought. They seek clarification and pine for faimiliar analyses. They are non-statisticians and one must do well to defend their analyses against them. For example, clinicians are inclined to demand conditional estimates of the treatment effect given change from baseline on a potential mediator. These are dubious estimates and the trial is not designed to answer them.

Frankly, the influence wielded by reviewers is excessive: authors are eager to appease them despite their requests for analyses lacking pre-specification and not being informed by an intimate knowledge of the study data (savoured only by the company's analyst). This is a criticism of the process, but it is not the one Marik his making. His claims of corruption are vague. He should, however, appreciate the limitations inherent in peer-review when his papers are scrutinised post-publication, including retraction.

Bearing in mind Marik's confusion about the publication process, it is interesting that he says he “discovered that the ... publishing system is fraudulent”. Discovered how? The lack of originality and specificity in his claims suggests he is merely invoking popular anti-Pharma books (the term ghost writers is straight out of Goldacre, and he says he was bowled over by Robert F. Kennedy Jr's book). If he had first-hand knowledge he would surely share it.

It is a shame whistle-blowing is restricted to those who have inside knowledge when others like Marik would like to decry Pharma too.

There is something familiar and tedious about his tone in this age of podcasting. E.g., the apparent contradiction whenever he refers to “published data” to support his views (e.g., 19m25s, and elsewhere) after claiming publishing is entirely fraudulent; the all-encompassing nature of the claims: “The major medical journals dictate what is written” (these journals are very different from one another); resentment that he and others like Kirsch are not taken serioulsy; the wayward scepticism: “Most of the vaccines they've developed simply don't work”; and he signs off with the usual plea for people to find more reliable sources of information.

Boghossian, on the other hand, is an interesting character: revealing and antagonistic, with an untenable blend of liberalism and hysteria. The following post refers to a book by Palahniuk:

We reached out to Boghossian and offered to add context and to clarify Marik's many dubious claims. In response he said this would not work because BasedScience are anonymous. (This was never an issue; we could have connected on Linkedin to confirm our credentials.) He then cancelled our subscription to his X account and blanked us. (We were an OG subscriber.) Incidentally, Boghossian's book is titled How to Have Impossible Conversations. Perhaps we were shooting too low.

Marik was never an impartial expert; he was a guest on the podcast. I.e., this is not an Andrew Neil grilling, it is more like Oprah chatting with Meghan.

Marik appeared on Bret Weinstein's podcast before appearing on Boghossian's (see Weinstein's podcast titled "Pharma: Not their first rodeo"). And Chris Martenson appeared on Boghossian's podcast in February after travelling with Weinstein to observe immigrants crossing the Darien. (Martenson does not pass the smell test; just visit his website.)

There is a mutual networking, recycling of guests, collaborating and promoting going on that the listener should take into account: Marik and Boghossian are in partnership. Soon after the podcast Boghossian appeared at Marik's institution where he spoke with employees who offered more guesswork. Perhaps these were some of the “more reliable sources” Marik was alluding to.

The enduring problem for the podcasters is that the real players would not be seen dead with them.

We witness, here, how in the era of podcasting wannabe best-sellers who flaunt their disapproval of Pharma can contaminate the minds of so many with their hackneyed criticism. Susceptible podcasters revived Goldacre's out-of-date review offering themselves as truth-tellers laying bare Pharma's secrets. It is shameful for Boghossian to allow Marik to persuade his listeners not to trust their doctor. Never mind the irony that some of these podcasters (not Boghossian) get rich selling dick pills and supplements to their listeners.

And, in a further irony, the same crowd who peddle cynicism and lies blame Pharma et al. for the rise of cynicism in public opinion.

In part 4 we will consider Marik's claims about Pharma rigging clinical trials (another accusation found in the Bad Pharma styled books, most notably Marcia Angell). We hope to persuade the reader that far from Marik's claims being true, it is industry, with its quality, who can restore the public trust lost by academia.

Subscribe now

These studies, led by academic investigators, suffer methodological problems: ad hoc composite endpoints (often infused with subjective measures) are appealing due to the small sample size and low event rates, but they are crude and overestimate power at the design stage leading to equivocal results and a lack of reproducibility. Academic clinicians have pushed composite endpoints over multivariate alternatives due to ignorance and they now pervade the literature despite the limitations. (More about this in Part 4.)

Marik espousing HAT is apparently not to be compared with a drug company pitching its drug, even though Marik's reputation (according to the FLCCC website itself) coincides with its success and is reminiscent of pharma's control of the perception of its drugs; Marik can be found on youtube, in a video created by his institution, describing how he saved a woman’s life and landed on his combination therapy with vitamin C.

The reader should be aware that disputes among academics over treatment alternatives often become heated; they seem to think there's a lot at stake. Consider Milton Packer and discussion around SGLT2 inhibitors and GLP-1 agonists. These are not good-hearted discussions. They are small-scale battles with a lot of vindictive plays going on behind the scenes. E.g., reviewing a paper harshly if the data deflect from the pet theory, and, when this fails, persuading the editor to allow a rebuttal in the form of a letter-to-the-editor.

These incentives are not flagged by either Marik or Boghossian. Instead, with his blinkered view, Marik goes so far as to say: any trial funded by Pharma cannot be trusted, even when the trial is conducted by academics. We are expected to believe that academics are trustworthy up to the point that they come into contact with Pharma, then they are utterly venal. But how does Marik think they rig a multi-million dollar clinical trial? We will get to this in subsequent posts.

Sepsis is rife with hopeful claims like Marik's precisely because of the lack of breakthroughs that have been made (the “need for systematic improvement in the initial management of patients“ and "what's next?"). This draws ambitious researchers and low quality trials that generate ambiguity allowing room for claims that would be thwarted by a robust RCT. The field is especially vulnerable to academics peddling black-box AI and machine learning tools (e.g.: TREWS machine learning-based early warning system for sepsis). Suchi Saria built a company (Bayesian Health) around her sepsis detection tool and the website has the same appearance as a drug company’s.

The astute reader will wonder at this point: academics peddling tools is perfectly analogous to the drug company marketing its drugs? Almost. The only difference is that, despite calls for RCTs of the machine learning tools comparing patient outcomes, the academics do not run them and in their absence hype is nurtured: “There is ... a tendency to over-claim the potential usefulness of ML models for clinical practice … The next body of work that is required for this research discipline is the design of [RCTs]“. If you do not test against standard care in an RCT then your research paper amounts to marketing material.

Given Marik's posturing we might expect him to be bleating about unsupported claims coming from academics who are celebrated by credulous science magazines: "Algorithm That Detects Sepsis Cut Deaths by Nearly 20 Percent." It is not as though these tools cannot do harm just as drugs can. Perhaps more so as they influence patient care and triage and do not undergo proper validation and regulation.

The motivation for the discussion with Boghossian is Marik's concern about perverse incentives affecting medical research; he cannot relent we are supposed to believe. But we witness here the uselessness of his fixation. In the scenario described above pharma play no role: we noted how academic disputes affect the publication process and how lower quality studies generated by academia tarnish the collective pool of results. Yet Marik laments pharma exclusively. Why doesn't Boghossian call it out? Is it not apparent, even to an outsider, that science becomes indistinguishable from marketing when they rush to celebrate the academic’s product?

As companies praise their drugs, institutions praise their scientists. The academic sells himself: his 'lab', which he often names after himself, is defined by marketing, influence metrics, and an increasing use of social media where he touts his ambition and makes allusions to whatever is gaining traction e.g. big data. (Beguiled by the complexity of the subject, and in the absence of proper validation, it is pure marketing.) Overlay these pictures: industry employees celebrate their company when they win marketing approval and institutions celebrate their scientists when they win awards. It is the same but inverted. The academic's reputation is paramount just as the company's reputation must be protected.

Unsurprisngly, then, when Boghossian introduces us to Marik we learn that he feels unfairly treated by the medical journals who censured him due to recognised self-plagiarism. His reputation is under attack! It is remarkable that Boghossian laughs off the idea of self-plagiarism as if it is incoherent; he says you cannot steal from yourself. (Why equate theft with self-plagiarism instead of, say, forgery?)

There lingers an unannounced shift from his previous emphasis on the plagiarism scandal affecting academia - suddenly it's all a joke. He also does not flag Marik's perceived grievance as a source of resentment which would add context to his outrageous claims. (Maybe he left that for us to detect, to spare Marik the embarrassment.) This prelude feels like a hasty attempt to restore Marik's reputation before getting on with the relevant subject matter.

It was a superflous review article about his work that led to Marik's self-plagiarism. It is well known that academics enhance their publication count (and hence their reputation) by extracting as many papers as possible from the same data source. This leads to practical problems such as duplicated data in meta-analyses. How this biases results has been reported for several decades, e.g. from 1997: "Inclusion of duplicated data in meta-analysis led to a 23% overestimation of ondansetron's antiemetic efficacy." (This problem may now be excacerbated by tools used by thoughtless researchers that scrape papers for estimates for use in meta-analysis.) Often the author will not reference their own work to limit the risk that reviewers will detect the duplicated text; it is a very seedy business.

Marik is suggesting the medical literature is corrupt while muddying it with his self-promotional material. He says he wanted to "spread the word". They always speak like this, i.e. in euphemisms; we all know what is going on. It is worth keeping this maxim in mind at all times: pharma sell drugs, academics sell themselves. Once again Boghossian misses this in his eagerness to combat bad pharma's assumed pernicious influence.

From the outset, then, we have a picture of a man who is frivolous and sloppy, defiantly so, laughing off the affair. And he is a little befuddled and resentful: we are expected to infer that if he has been censured the only possible explanation is that something underhanded is going on, rather than simply: journals have standards. (Yes, this is ad hominem. There is nothing wrong with ad hominem. We are trying to understand Marik. The public expects scientists to be fastidious, pedantic even. It's a bad impression at a time when science is leaking public trust.)

Part 3, covering Marik's specific claims, will follow in some days. Hopefully in Part 3 we will also find time and space to consider: why did Boghossian fall for this? This is what we find salient. Podcasting at this moment feels precarious and untenable, yet unavoidable.

We are several medical researchers. We have worked in pharma, academia and drug regulation in five countries and three regulatory regions; we have all the degrees. We may be inclined to generalise.

We enjoy Boghossian's podcast and had no intention of drafting a response or even listening to this particular episode given the irritation it would likely cause. But Boghossian is searching for a guest capable of responding to Marik's claims and it seems unlikely he will find one. (He needs a current industry scientist if he wants an accurate picture; someone who has worked in publications and had interactions with, or worked in, the regulatory authority. These people will not speak publicly.)

There are too many false statements made by Marik that will mislead and trouble the public. We are compelled to respond.

Preamble: Important background regarding Academia pitted against Industry

The Bad Pharma caricature is enticing and touted by many academics; e.g., Angell, Goldacre, and Goetzche. To give a sense of their zeal consider this piece about the indefatigible Vinay Prasad who takes on pharma “with glee” (more about him below). Dr Paul Marik, Boghossian's podcast guest, is a retired professor and his cynicism regarding pharma puts him in good stead. But the listener is confronted with a paradox from the outset: to be a reliable critic of pharma it is essential that you have never spent time inside it.

We have read the anti-pharma books by the academics listed above and can say they often miss the mark; i.e. there is legitimate criticism that could be levelled at Big Pharma (BP) and they do not see it, for the most part. And a wonderful irony is that the dodgy stuff industry produces is presented at the academics' conferences because they are widely known to be susceptible and open, more so than the attentive experts at the regulatory agency.

The listener should detect this potential bias and pretence to authority without any commentary from Boghossian. Nevertheless, it is disappointing that Boghossian didn't approach the topic with more scpeticism given his other comments on virture signalling and contrived consensus in academia. Unfortunately, the podcasters are a single and tiresome voice on the topic of BP. Perhaps this is because academics seek the limelight while industry experts haven't the time or interest in public skirmishes. Or perhaps it is because there is a new and spurious belief that academics and Leftists are inclined to defend BP and the podcasters see themselves as necessary to counteract this trend?

Any regular listener of Jordan Peterson's podcast will know he has said, in multiple places, that he is perplexed by the Left's (Academia's) sudden apparent defence of BP. But this confuses the Left's confidence in the regulatory mechanism with trust in BP. Academics remain as dubious as ever about BP. They pit themselves against pharma in an attempt to retain talent who know they would be better treated elsewhere. This is the pitch they give, and it can be seen in the pages of their annual statements. The promise of the academics is this: unlike pharma we are scrupulous, so who do you want to align with? The ruse is in desperation because: people are leaving academia for industry in droves.

If the academics did not feel the need to advertise their opposition to BP they would celebrate the pronounced quality and reliability of the randomised controlled trial conducted by industry personnel, contrasted with their own feeble retrospective analyses of uncontrolled data. Quality is, at the end of the day, the paramount issue. In other words, the Big in Big Pharma is a good feature, and the smallness and lack of resources to be found in academic research organisations (AROs) is in no way advantageous. It is easier to behave badly when fewer people are watching, for example. Or: if you have a single statistician then independent validation of code is not even possible (nevermind that statisticians are not programmers). Many have forgotten that in 2004 when the EU tried to align standards for clinical trials with industry there was immediate push back from academics. The gap in standards remains today.

This irony makes the attacks on industry by academics especially tough to bear. There is an impulse to respond: Do you really want us to describe what we have seen in academic clinical trials? Do you really want to talk about ethics and fraud? You want to make that comparison explicit and in the open? Maybe a discussion about the lack of resources and nous in AROs is long overdue (fraud is easy in academia, you would not believe what we have seen). Maybe industry experts are sick of these error-prone status-chasers flinging muck at them. For a glimpse of their handywork take a look at this recently published trial:

To be honest, most people working in industry and drug regulation are preoccupied and these podcaster discussions do not cause a ripple in their world. They do not know who Bret Weinstein is, even though he claims to have all the answers; and if you listened to Alex Jones in the cafeteria of a regulatory agency and let his guttural voice loose not a single head would turn. The talk of censorship is exaggerated. In this crowd no one cares what the amateurs are saying. You will not find professionals on social media defending their work in pharma against attacks from outsiders; if they did it would seem unsightly. Hence the asymmetry of the attacks.

These outsiders have no bearings when confronting the literature. E.g., they will not know that some of the disagreements between academia and industry can be understood as a feature of their varying circumstances. Why do the academics place meta-analysis atop the heirarchy of evidence while industry decisions are dictated by the large and costly RCTs? (Recall the ivermectin meta-analysis run by academics which had Weinstein terribly excited. Regulators gave it little heed; they knew better.) Ponder why the academics, who have a terrible time acquiring funding, have decided that analysing other researchers' data in the form of a meta-analysis is the route to the truth. (Goldacre is especially idiotic in this regard, but he has recently pivoted from evidence-based medicine to data science. What exactly data science is and why the academics are now pushing it is a post for another time. Suffice to say, if the podcasters knew anything about what was going on in BP they would put all their attention here. But of course they seem only to think that data science is a new word for statistics.)

Thus, the reader needs to do some work if they are to avoid being caught in the rip. We would like to encourage you to read between the lines: why do the academic authors listed above target non-experts? The huge font in Angell's book and the language she uses gives this away; and Goldacre's reckless book reads like a British tabloid newspaper. Bret Weinstein held it aloft on his podcast and declared it "a terrifying book" and subsequently reiterated how common the regulatory capture inferred by Goldacre is. Experts who reviewed the book when it landed saw it as biased and uninformed. The uninitiated are easily incited, unlike seasoned professionals; this provides a useful tell.

If academics like Angell and Goldacre had a legitimate concern then why not engage industry instead of courting the public with claims of corruption? (The regulatory authority seeks input when drafting guidelines.) Consider Prof Prasad who sounds more like a football fan in the away section than a Professor announcing some deficit in an analysis. On X the Prof has called Rob Califf (head of the FDA) names that we have been careful to forget. In doing so he distances himself from industry and makes cosy with those observers who savour distrust. (Prasad has deleted all his posts on X before 2023 that mentioned Califf, e.g. those declaring him corrupt, and has blocked us on the platform.)

Further, note that Prasad is a Professor of Biostatistics and Epidemiology* and does not hold a degree in either field (he has an MPH, a common degree among MDs who find mathematics tough going). Unlike industry, in academia titles are nominal. Maybe the reader noticed, for example, academics from peripheral fields such as engineering offering their views on vaccine safety during COVID. Or, the substack maintained by Prasad on pharmaceutical drug development, despite Prasad never having worked in pharmaceutical drug development. It is absurd to everyone aside from the academic who apparently regards it as their duty to stray outside their competence.

To appreciate the audacity on display, note that when the regulatory authority received the submission from the drug company various distinct experts were required to review: clinicians, regulatory experts, statisticians, safety experts, and a pharmacologist (post-approval studies required an epidemiologist). We were involved in these discussions and can say not one of these experts believes they can stand in for a colleague if they are absent. Transposing disciplines is a characteristic of the academic only. It was fun to watch academic epidemiologists on X calling themselves biostatisticians until COVID hit, then they were epidemiologists again. (There is more clout in being a biostatistican, but the degree is more difficult to get through.)

Only the industry scientist is expected to suffer the egregious stupidity of miscellaneous pundits. Imagine Bret Weinstein, when he is walking his cat, pausing at a construction site and shouting: "Gentlemen! You're doing it all wrong!" He'd be arse over tits in a dumpster in seconds. We have started to wonder: maybe something like this is needed to give people on the outskirts reason to pause before they serve us their guesswork. Otherwise Weinstein encourages the listener to interpret our silence in his favour (from Michael Shermer's podcast):

"[I]f i'm wrong ... there should be a flood of biologists ready to expalin what I got wrong, and when that flood of biologists doesn't show up you should ask yourself the question: Why is that?"

As noted above, if pronouncements on a podcast do not elicit a reaction it is not censorship or suppression of ideas or a conspiracy or google's algorithm. It is because the experts are too consumed with work to listen to some useless agitator.

So that is what we will offer here: a lack of silence. If it sounds harsh please recognise how generous the experts have been with their silence to this point. A response to Marik’s specific claims will follow in some days’ time.

*Since this post was published Prasad changed his title to Professor of Epidemiology, Biostatistics and Medicine, i.e. to include ‘Medicine’.