the STEP thing finally clicked for me and I want to write it down
the STEP thing finally clicked for me and I want to write it down. Change my mind, genuinely — I have no stake in being right about this.
On means, which this board treats as targets and which are nothing of the sort.
A reported mean body weight change is the centre of a distribution that in these trials is very wide. Substantial numbers of participants did much better, and substantial numbers did considerably worse while remaining on the drug and in the analysis.
Quoting the mean as an expectation therefore misleads in both directions: it makes ordinary results look like failures and it makes exceptional results look normal. If a paper publishes the distribution — and several do, in the appendix — look at that instead. It is far more informative than the number in the abstract.
Argued for a week about a result and then read the limitations section, which conceded most of my opponent’s point.
How to read one of these papers in fifteen minutes, in the order that actually helps.
Start with the registered protocol and check the primary endpoint against what is reported. Then the methods: who was included, what the comparator was, how long the randomised phase ran. Then the discontinuation numbers, which are a tolerability result and are usually in a supplementary table.
Only then the efficacy figure, and read the interval rather than the point estimate. Finish with the limitations section, which is where the authors say what they actually think.
Fifteen minutes, and you will know more than any thread summarising it.
If two or three other people have done the same thing we might actually learn something. Alone it is an anecdote.
best — the order this archive was captured in
Added the trial identifier to the title so this thread is findable in two years.
Yes — the interval is the finding. A point estimate with a wide interval is a hypothesis in a nice font.
The registered protocol is public. Comparing the registered primary endpoint with the reported one is a two-minute check and it is how outcome switching gets caught.
That is the 68-week readout, not the 72-week one. Different trial, different duration.
Small fix — that was the cardiovascular outcomes trial, so weight was a secondary endpoint and the population was different.
Right — trial participants get structured support. Comparing yourself to a trial mean is comparing across two different interventions.
Right — trial participants get structured support.
This is the distinction that would end about half the arguments on this board.
Push back: an open-label extension tells you about the people who stayed. That is a different question.
The press release said that; the publication says something more hedged. Worth reading both.
The press release said that; the publication says something more hedged.
Agreed. And the interval, not the point estimate, is what the trial actually established.
Intention-to-treat analyses everybody randomised regardless of what they did afterwards. Completer analyses only those who finished. The second is systematically more flattering and both are legitimate if labelled.
a press release is not a publication
This. Intention-to-treat versus completer analysis routinely moves the headline by several points.
open-label extensions are not the same evidence as the randomised phase
- 1The registered protocol is public. Comparing the registered primary endpoint…6 comments in this branch · started by u/ahmed_iyer