This book tackles a pressing issue in modern science: the growing crisis of reproducibility in biomedical research. With an ever-increasing number of studies showing that many biomedical findings cannot be reproduced, Szabo examines the structural, ethical, and methodological challenges that contribute to this crisis. My review focuses on the book's critical assessment of scientific rigour, its insights into the scientific process, and the implications of Szabo’s arguments for broader scientific and educational contexts.

Szabo provides a well-researched and comprehensive examination of the reproducibility crisis, identifying key factors that contribute to unreliable research outcomes. He covers topics such as publication bias, pressures to publish positive findings, and the misuse of statistical methods. These elements are presented in an accessible yet detailed manner, allowing readers with varying degrees of scientific background to follow his arguments effectively.

One of the strengths of Szabo's approach is his clear explanation of how misuse of statistics – particularly p-hacking and data manipulation – leads to unreliable conclusions. Szabo’s analysis extends beyond technical issues to explore the ethical dilemmas faced by researchers in a competitive academic environment. He highlights how pressure to publish, secure funding, and achieve tenure can create an environment conducive to unethical practices, including selective reporting and outright fraud. Szabo emphasises the role of institutions and funding bodies in perpetuating these problems, advocating for structural reforms to prioritise quality and reliability over quantity.

Szabo's call for institutional change is compelling, as it emphasises the importance of nurturing a scientific culture grounded in integrity. The book would serve as an excellent supplementary text for courses in research ethics, as it illustrates how real-world pressures can distort scientific values. Though the focus is on biomedical research, Szabo’s points are highly relevant across scientific disciplines. His critique of reproducibility and bias is applicable to fields like psychology, environmental science, and even theoretical physics, where publication pressures and statistical misuse can similarly lead to poor-quality publications.

Szabo’s writing is straightforward and informative, though at times it can feel dense due to the technical details and extensive discussions of specific case studies. While this level of detail will be appreciated by academics and practitioners, general readers might find it challenging. Nevertheless, Szabo does make an effort to explain complex topics in an approachable way, making the book relatively accessible to a non-specialist audience with a strong interest in science.

While Szabo’s analysis is thorough and his arguments are well-supported, the book could benefit from more emphasis on potential solutions and success stories in overcoming the reproducibility crisis. Szabo’s critique is undeniably powerful, but it occasionally lacks balance, as the focus remains heavily on the issues without enough exploration of practical steps forward. A more extensive discussion of initiatives like open science practices, data-sharing policies, and educational reforms would provide readers with a constructive path toward improvement.

Unreliable is the opposite of whatever the true crime genre is.

It is about the reproducibility crisis, but not the crisis in general. It limits its treatment to the author's own specularity, biomedicine. It is not journalistic. It is not a discussion of notable science fraud or that fraud's perpetrators. It is so much Not That that the author summarizes the big events as part of a table and sparingly refers back to it.

It is a savvy choice. The author wants to look at the crisis structurally, which means jettisoning a 'few bad apples' paradigm - you know, the one that focusing on the biggest cases implies - and arguing for a persistent miasma.

Rephrased, Unreliable is a study of the structural causes to the problem of the reproducibility crisis. Individual incidents become unimportant. They are not the real problem.

This might sound like apology and rationalization. It is not.

It does mean that the 'innocent' causes of the reproducibility crisis are discussed at length. While almost the opposite of the book's point, it is a highly memorable chapter. A number of things that researchers have done to prevent reproducibility problems may instead inject them. And this provides something of a sub-theme to the book, namely the ways in which that the practice of science has ossified, where methodology in general, down to the scientific journal itself, came about for useful reasons, but have become contradictory to the process of science.

It is then the later chapters that turn the most dismal. It mirrors the first half of the book. The intentionally malignant acts, like fake papers and sketchy journals, are produced after explaining how the context of even the good ones leads to moral hazard and unintended negative consequences.

The solutions are a letdown. Another sub-theme to the book is how no one gets punished. There are sensible suggestions to address this, foremost being that this sort of verification work has to become its own sort of discipline within science. In general, retractions have to be more forceful and more meaningful, ideally alongside a sort of sabermetrics for scientists that reorients from a ranking system to one that includes other facets of the work.

The insensible solicitations are those that stand out. "This man does not follow U.S. politics," I said to my cats when reading the author's resumption of an older proposal to criminalize research fraud. No Attorney General would permit any immunologist to darken our streets again. And while I think that there is some validity in a surveillance-state approach (think in the context of all labs acting as if a chain of custody was relevant to their work) as it is described here seems creepy.

The question of audience is a problem. While the book specifically avoids being too technical (or per the text it locks that away in the end notes), it does cover a lot of the technical issues of its material, which I suspect that will be off-putting to some members of lay audience.

There are a lot of New Yorker-style cartoons throughout the text. They are primarily unfunny. The author himself, however, is pointedly hiding his own light under a bushel in terms of comedy. The appendices are mostly satire. They are funnier than all the comics.

Unreliable was not what I expected, but in a good way. The book resembles an anthropological study, with a side of memoir, but one with actionable solutions, if imperfect ones. The problem, though, may be similar to the one in Owned, where cures are situated beyond the range of systemic fixes on the scientific-procedural level, as opposed to the economic or political one.

My thanks to the author, Csaba Szabo, for writing the book, and to the publisher, Columbia University Press, for making the ARC available to me.

I feel privileged to read an advance copy of this terrifying book. Talk about a Halloween scare! Unfortunately, this horror show is all too real.

Szabó, a storied researcher and heavyweight in his field, takes the reader on a journey into the chilling depths of the biomedical research system. We learn of the good, the bad, and the ugly in the peer review system, the grant process (which, as someone working outside of the US and Western sphere, seems to be a lot different and a lot worse than what I experience), the subjective nature of science and "gut"-based research, the publish or perish paradigm and the paper mill fiascos, the replication crisis, the various forms of data mischief (from mistakes to outright fraud), and nasty competition, alongside errors in everyday, normal research that can have resounding effects.

What really distressed me was how often this happens and the response of the scientific establishment to most cases: frequent and minimal. Szabó soberly covers the rough estimates of bad science (50% based on the research and likely much more, possibly up to 70-90%) and several case studies about how those caught were handled. An example: resveratrol. Have you ever heard about red wine being great for your health, in certain small, daily doses? Turns out that these results couldn't be replicated, likely because the original data was accidentally influenced by a fluorescent probe. Now, have you heard about that? Probably not. Meanwhile, papers sometimes get retracted, and sometimes not; some questionable work may be given warning labels, while others not; researchers may get fired or quit due to ~irreconcilable differences~, or carry on with a mere slap on the wrist. I'm almost inspired to be fearlessly deviant, just thinking about what I could get away with!

I was also surprised to learn about how little money there is for biomedical research and how little of this money goes into actual research. Again, this is geared around the US context. In my neck of the woods, the institutions take about 10-20% ... but in the US, this apparently goes up to 60%. Bare minimum lab operations, including to support personnel and students, is dependent on this funding. Szabó argues for a new system that, sadly, means less scientists. I would hope for a future where more of the wealth in the world would go into science rather than reducing this, one of humanity's key endeavours. Szabó also suggests two different tracks, reminding me of engineering: research or practice. This would help offset the workload currently leveraged on everyone, regardless of their abilities or interests. I wonder how many people would opt for the non-research track, given how disliked similar systems, like teaching tracks, tend to be.

This work has limitations. Szabó focuses on the US context, with only some mentions of other contexts, and leaving a bit of confusion when the context isn't mentioned. He also fails to cover the preprint crisis (arXiv, notably) and how this has led to all kinds of academic mischief and misinformation. Frankly, that was such a stellar oversight that I can't help but belabour the point! Szabó also falls prey to unconscious bias, with actors given masculine names (Average Joe scientist) and pronouns, plus nearly every (otherwise wonderful) cartoon illustration of the various problems outlined featuring white Western men in stereotypical professorial/medical researcher garb ... even after joking about this "adage": "The official language of biomedical science is broken English." I was also hoping for a shift in language from the ableist "blind" to "anonymous" research designs. Well, I guess we can't be woke about everything. A final issue: Szabó argues for camera surveillance in all labs, as a way of preventing-slash-capturing misconduct. This also sent chills down my spine. I know his heart is in the right place, but this level of scrutiny and reduction in civil liberties deserves very careful consideration, and none can be found here.

This is essential reading for anyone, but perhaps especially for the unaware participants in biomedical research. Indeed, Szabó doesn't touch on that point: with all the data fraud going on, the public's money is surely wasted, but also the time and energy and trust of individual participants is squandered. We must do better. I'll end with this quote:

"Yes, the system is wasteful, redundant, and littered with 'scientific garbage.' But maybe it is not beyond hope."

I don't think it's beyond hope ... as long as a critical mass agrees.

This was such an interesting read. There was so much that I learned while reading it, and it was very eye opening. The amount of bias in the science world, especially in the biomedical area is just insane. I think this book was great and had lots of great points

Thank you to NetGalley, to the author, and to the publisher for this complementary ARC in exchange for my honest review!!!

As someone with years of experience in biomedical science, I've often had friends or younger colleagues tell me they're eager to jump into biomedical research. Part of me wants to share the reality—the challenges that aren’t always visible from the outside. But I usually hold back, not wanting to be the one to dampen their enthusiasm. Yet, I feel they deserve the full story. If they’re going to commit to this path, they should know what lies ahead. I really wish this book had been published before I started getting that question so I could hand it to them. It captures the raw, unvarnished sides of biomedical science.

Szabó pulls back the curtain to reveal what research life is really like: the relentless hustle for funding, the career pressures, and the fierce competition. I appreciated its straightforward honesty—it’s a reminder that going into science for the wrong reasons, whether for fame or a sense of job security, can lead to serious consequences.

This book is for everyone—the scientists, the institutions, grant agencies, publishing industries, and even the general public, who often only see the polished results of scientific work. Szabó reminds us of the twists and turns that scientists face. Discovery is tough enough, but it’s made even harder by the need for resilience, integrity, and the ability to confront uncomfortable truths. If we’re to foster a new generation of scientists—and build a public that trusts in science—they need to know what they’re up against. Only then, with eyes wide open, can they work together to create a system that truly supports both science and the people behind it.