But let’s think about that 91% failure rate for a moment. When I bring this up in presentations, I invite the audience to consider what the auto industry would look like of 91% of new car designs proved unable to roll out of the factory, or if 91% of new airliner models were unable to leave the ground
Is an utterly irrelevant comparison. For physical thing we have engineering, and the practical application of the trades and craft, trial and error.
For modern medicine, we're only just starting to come out of the wild west era. Or perhaps slightly further along than that.
levocardia•49m ago
Indeed, I suspect the failure rate of, say, new jet engine designs is rather high as well -- those failures just never get reported in a federal repository, unlike RCTs, since they never make it out of the simulator or the prototyping lab. And we have, comparatively, much better computational models of how airplanes fly than how cancer cells mutate. FWIW this clinical stat is far better than Edison's supposed lightbulb-idea failure rate!
TheOtherHobbes•31m ago
Jet engine design is iterative, and the basic principles are well-understood.
Drug design seems a lot more binary. You can find a new pathway, but drugs themselves are fairly simply molecules, and you can't iteratively 'fix bugs' the way you can in an engine or a piece of software.
The engine is a given, it's almost astronomically complicated, you know a lot less about how it works than you'd like to, and you're trying to change how it works while it's running without breaking anything, using tiny rigid parts that have to snap into place correctly and can't be bent to fit.
High failure rates aren't surprising.
levocardia•52m ago
I'm actually surprised it isn't going up over time. That is naively what you would expect as the low-hanging fruit is plucked. So the fact that it's been stable is probably a sign that scientific advances are roughly keeping pace with the (presumably) increasing challenge of finding ever more targets for drugs.
AbsurdCensor•43m ago
As someone who works in research, this isn’t surprising at all. It’s hard to find something that treats (well most often reduces symptoms) of a particular disease. It’s even more difficult to find something that is also safe at the dose level it takes to treat said disease. Are there faster ways to do this? Probably not. AI is only going to help out so much, just like automated drug screening only helped so much since its introduction in the 90s.
calf•39m ago
Why does he conclude there is no alternative? He already argued this would be strange in other disciplines. So reform it.
mlyle•29m ago
If you can think of a way that you can have a much higher chance than 9% of success in getting a useful treatment to market, you can make a boatload of money. Go ahead, reform it.
EA-3167•28m ago
How? He’s saying that he can’t think of any good ones, and in my experience that’s true of most informed people. This is one of those hard problems that needs solutions, and so far the present system is the best anyone has managed.
debo_•13m ago
There's an alternative that is slowly emerging. Many clinical trials "fail" but the drug candidate in question works really well for identifiable subsets of the participants. Right now pharma companies won't bother pursuing those drugs because they can't market it broadly. But it's still possible these drugs could help people in the future.
perpetuallunch•54m ago
But let’s think about that 91% failure rate for a moment. When I bring this up in presentations, I invite the audience to consider what the auto industry would look like of 91% of new car designs proved unable to roll out of the factory, or if 91% of new airliner models were unable to leave the ground
Is an utterly irrelevant comparison. For physical thing we have engineering, and the practical application of the trades and craft, trial and error.
For modern medicine, we're only just starting to come out of the wild west era. Or perhaps slightly further along than that.
levocardia•49m ago
TheOtherHobbes•31m ago
Drug design seems a lot more binary. You can find a new pathway, but drugs themselves are fairly simply molecules, and you can't iteratively 'fix bugs' the way you can in an engine or a piece of software.
The engine is a given, it's almost astronomically complicated, you know a lot less about how it works than you'd like to, and you're trying to change how it works while it's running without breaking anything, using tiny rigid parts that have to snap into place correctly and can't be bent to fit.
High failure rates aren't surprising.