{"id":6690,"date":"2011-03-02T17:22:03","date_gmt":"2011-03-02T22:22:03","guid":{"rendered":"http:\/\/blogs.nejm.org\/cardioexchange\/?p=6690"},"modified":"2011-07-19T17:44:19","modified_gmt":"2011-07-19T21:44:19","slug":"a-dose-of-reality-the-challenges-of-comparing-effectiveness","status":"publish","type":"post","link":"https:\/\/blogs.nejm.org\/cardioexchange\/2011\/03\/02\/a-dose-of-reality-the-challenges-of-comparing-effectiveness\/","title":{"rendered":"A DOSE of Reality: The Challenges of Comparing Effectiveness"},"content":{"rendered":"

An ideal paper for your next journal club \u2014 \u201cDiuretic Strategies in Patients with Acute Decompensated Heart Failure\u201d<\/a> \u2014 was just published in NEJM<\/em>, by the NHLBI Heart Failure Clinical Research Network. \u00a0In this study (called DOSE), patients hospitalized with heart failure were randomized to receive different diuretic regimens based on dose and mode of administration. The authors concluded that \u201cthere were no significant differences in patients\u2019 global assessment of symptoms or in the change in renal function when diuretic therapy was administered by bolus as compared with continuous infusion or at a high dose as compared with a low dose.\u201d The editorialist<\/a> stated, \u201cSince a high-dose regimen may relieve dyspnea more quickly without adverse effects on renal function, that regimen is preferable to a low-dose regimen.\u201d<\/p>\n

There are at least five issues here that might be worthy of your attention.<\/p>\n

1. The topic.<\/strong> Diuretics were introduced in the 20th century to treat heart failure, replacing the rigid Southey\u2019s tubes that were inserted subcutaneously to drain fluid. Thiazide diuretics were introduced in 1958, and furosemide, the first loop diuretic, was approved in 1966. This agent is now standard therapy for patients with heart failure. What is remarkable \u2014 and worth some reflection \u2014 is that despite using loop diuretics for 45 years now, we still do not have essential evidence to support decisions about dosing and mode of administration. This study responds to the clamor for comparative effectiveness trials \u2014 and should give us pause about what else we are doing in clinical practice with little evidence to guide us.<\/p>\n

2. The primary outcomes.<\/strong> The primary efficacy endpoint (there was also a safety endpoint) was the patient\u2019s global assessment of symptoms, which was quantified as the area under the curve (AUC) of serial assessments from baseline to 72 hours. The authors describe the assessment as follows:<\/p>\n

Patients were asked to self assess both their general well being (PGA) and their level of dyspnea using a visual analog scale (VAS) method. For PGA, patients marked their global well being on a 10 cm vertical line, with the top labeled \u201cbest you have ever felt\u201d and the bottom labeled \u201cworst you have ever felt.\u201d For dyspnea, the labels were \u201cI am not breathless at all\u201d and \u201cI am as breathless I have ever been.\u201d The VAS was scored from 0 to 100 by measuring the distance in millimeters from the bottom of the line. The patient was unaware of the numerical value of their response.<\/p><\/blockquote>\n

Kudos to the research team for caring about patient-reported outcomes and attempting to translate those outcomes into a useful metric. However, interpreting the results is a challenge. For the comparison of bolus versus continuous infusion, the mean AUCs were 4236 and 4373, respectively (P<\/em>=0.47). For the comparison of high- versus low-dose therapy, the values were 4430 and 4171, respectively (P<\/em>=0.06). The authors considered a 600-point difference in AUC to be clinically important based on prior studies and thus concluded that there were no significant between-group differences in the primary efficacy endpoint, either statistically or clinically. Although these conclusions seem appropriate, it\u2019s difficult to know what a 600-point difference really means in terms of the patient\u2019s experience. A few examples from the authors would have been helpful.<\/p>\n

3. The power calculation.<\/strong> With only 308 patients, this study was small by the standards of most RCTs measuring patient outcomes in heart failure. The sample size calculation was based on 88% power to detect a 600-point difference in the AUC of global assessment scores \u2014 and on 88% power to detect a difference of 0.2 mg\/dL in the change in creatinine level between groups (the primary safety endpoint). I have no quibble with these calculations (though I do wonder why they picked 88%), but the small number of patients makes it difficult to do much with exploratory analyses by subgroup or different outcomes. The investigators adjusted the significance level for the primary outcomes, stating that the threshold would be a P<\/em> value of <0.025. In doing this, they treated each trial within the 2×2 factorial design as a separate study with two endpoints.<\/p>\n

One statistic that would have been useful to see in this paper is the confidence intervals for the difference in the primary endpoints, so that we could see what kind of differences cannot be excluded based on the results. Remember, the study was not designed to show that the groups were similar; it was designed to test if they were different. The conclusion, appropriately enough, was that there were no significant differences, but your questions now might be: \u201cAre the treatment groups similar? What kind of differences can be excluded?\u201d<\/p>\n

4. The secondary analysis and adjustment for multiple comparisons. <\/strong>The investigators conducted many secondary analyses and, for these, set a P<\/em> value of 0.05 as the threshold for statistical significance. Most of these endpoints did not differ between the groups, but there were some findings that bear discussion:<\/p>\n