unpopular opinion: most of what gets said here about hazard ratio is guesswork
unpopular opinion: most of what gets said here about hazard ratio is guesswork. Not a hot take, just something I have not seen said plainly here.
A confidence interval is the range of effects compatible with the data. Two trials with overlapping intervals have not disagreed, whatever their point estimates look like next to each other.
Why comparing across trials almost never works, with the specific failure modes.
Different populations: an obesity programme and a diabetes programme enrol different people with different baseline characteristics. Different endpoints: body weight change, glycaemic control and cardiovascular events are not convertible. Different durations: 68 weeks and 72 weeks are not the same, and the curves have not flattened by either.
Different analysis populations: one paper reports intention-to-treat, another emphasises completers. Different support: some trial designs include structured lifestyle contact that no member of this board receives.
Stack those and the "X beats Y" tables that circulate here are comparing five things at once and attributing the difference to the molecule.
On means, which this board treats as targets and which are nothing of the sort.
A reported mean body weight change is the centre of a distribution that in these trials is very wide. Substantial numbers of participants did much better, and substantial numbers did considerably worse while remaining on the drug and in the analysis.
Quoting the mean as an expectation therefore misleads in both directions: it makes ordinary results look like failures and it makes exceptional results look normal. If a paper publishes the distribution — and several do, in the appendix — look at that instead. It is far more informative than the number in the abstract.
That is everything I have. The rest is opinion and I have tried to keep it out.
best — the order this archive was captured in
Retitled: the original quoted a diabetes endpoint as an obesity result.
How to read one of these papers in fifteen minutes, in the order that actually helps.
Start with the registered protocol and check the primary endpoint against what is reported. Then the methods: who was included, what the comparator was, how long the randomised phase ran. Then the discontinuation numbers, which are a tolerability result and are usually in a supplementary table.
Only then the efficacy figure, and read the interval rather than the point estimate. Finish with the limitations section, which is where the authors say what they actually think.
Fifteen minutes, and you will know more than any thread summarising it.
[removed by moderator]
phase 2 finds a dose, phase 3 measures the effect
Which trial, and which arm?
registry entry, protocol, publication — three different documents
the appendix is where the interesting tables live
Yes — the interval is the finding. A point estimate with a wide interval is a hypothesis in a nice font.
This. Intention-to-treat versus completer analysis routinely moves the headline by several points.
Same. A press release is a claim about a result; the publication is the result.
read the endpoint before you read the headline
Compared myself to a trial mean for about six months before realising the trial arm had dietitian contact every fortnight.
How long was the randomised phase before any extension?
Yes. The appendix tables are where the subgroup and the adverse event detail actually live.
Open-label extensions lose their randomisation. Anybody still enrolled at week 104 is a selected group and the numbers describe that group.
Discontinuation rates are a tolerability result. A trial with a strong efficacy number and heavy discontinuation is telling you two things and people only quote one.
a mean is not a promise
a mean is not a promise
This is the distinction that would end about half the arguments on this board.
- 1Which trial, and which arm?7 comments in this branch · started by u/iman_castellanos