1. Is the reason for conducting the study discussed?
Yes. Three years of treatment with rofecoxib would reduce the risk of recurrent adenomatous polyps among patients with a history of colorectal adenomas.
2. Are the study objectives clearly defined?
Yes. Limited long-term data was available for analyzing the increased risk of thrombotic events associated with the use of COX-2 inhibitors so this was designed to be a long-term study (3 years of treatment plus follow-up).
3. Is the null hypothesis clear?
No. I don’t believe there was a null hypothesis. The study was designed to test a research hypothesis.
Methodology
For each of the following questions, access how this might influence the results or affect the validity of the study
Selection bias
Have adequate measures …show more content…
been taken to prevent selection bias?
Yes. Adequate exclusion and inclusion criteria were defined. This increases both the internal and external validity.
• Is the study population adequately defined?
Yes. A total of 2586 patients with a history of colorectal adenomas underwent randomization: 1287 were assigned to receive 25 mg of rofecoxib daily, and 1299 to receive placebo. The groups were well-matched at baseline, which increases internal validity.
• How were subjects selected? What are the inclusion criteria? Are the selection procedures clearly defined?
They were selected based on disease status. Men and women over age 40, in 108 centers in 29 countries, who had at least one histologically confirmed large-bowel adenoma removed within 12 weeks before study entry and were not anticipated to need long-term NSAID therapy (with the exception of low dose aspirin) were eligible for inclusion. Subjects were randomized using a computer-derived randomization which was stratified according to the clinical center. Randomization increases the internal validity.
Experimental:
Were subjects randomly selected?
Yes, which increases the internal validity.
Did all qualified subjects have an equal chance of being admitted to the study?
No because there was a placebo run-in period, which can influence compliance bias. This affects external validity.
Are the treatment groups comparable?
The treatment groups were well-matched, as both had similar baseline characteristics. This improves internal validity.
Are pertinent patient specific data provided? (ie healthy subjects vs patients, sex, age, concurrent disease states, concurrent therapy, race, weight or other pertinent information)
Yes. Age, gender, race, cigarette use, history of comorbid conditions, cardiovascular risk, weight, height, and concurrent therapy such as low-dose aspirin and anti-hypertensive medication data were all provided. This improves internal validity.
Have adequate measures been taken to prevent classification bias?
Yes. The study had a double-dummy design (the placebo tablet appeared identical to the rofecoxib) so that investigators and subjects to prevent this from occurring. Independent clinicians were provided for adjudication of events. These measures increased internal validity.
Does the study use specific definitions for the study parameters?
Yes the authors say that thromboembolic events, components of the APTC end point and all deaths were prespecified as eligible for adjudication and that the procedure for confirming cardiovascular events was prespecified in the protocol. However they did not provide the protocol’s definitions in the article. This might reduce its internal and external validity.
How were patients classified for entrance into the study? Do they have the disease of interest? (case-control, experimental)
Patients were classified based on their age, past medical history of colorectal adenoma and their performance in a placebo run-in period. Yes, the subjects were required to have the aforementioned disease of interest. This was an experimental study design because there was an intervention (rofecoxib or placebo). This reduces its internal validity.
All study designs
Have adequate measures been taken to prevent measurement bias?
No because the authors did not specify how compliance was assessed in the run-in phase. Were subjects self-reporting their medication compliance or they hospitalized and given their doses as prescribed? This hurts both internal and external validity.
• What measures were used to evaluate the outcomes of the study?
Clinical investigators recorded the serious adverse events. Event documents were sent to the cardiac, cerebrovascular, or peripheral vascular adjudication committee. The authors say that the procedure for confirming cardiovascular events is in the protocol but did not describe it in detail.
• Are they adequately described?
They are not adequately described. The authors constantly refer to the protocol for specific details.
• Were the measures used appropriate?
The authors should have included the measures in the study article, instead of just referring to the protocol. It’s difficult to know if the measures are appropriate with this information.
• Were objective measures used?
An independent, external safety-monitoring board met with a statistician who was aware of individual patient assignments but not involved in the study. This might affect the results and internal validity if the statistician knows the assignments.
• Are the measures reproducible?
Merck removed the drug from the market so measures cannot be reproducible.
Were subjects observed for a sufficient length of time?
Three years is not sufficient time for a long-term study. The study pooled follow-up patient years together, so it’s unclear how long individual patients were followed up.
Have adequate measures been take to prevent observer bias?
Yes but the statistician should be unaware of subject assignments even if he was not involved in the study.
• Are the observers specified?
Yes the observers are the statistician, an independent committee for adjudication of events and an external safety monitoring committee.
• Have measures been taken to prevent inter-observer variation?
No. The authors do not provide any indication that this was done.
Experimental studies
Were subjects randomized?
Yes all subjects were randomized.
Are randomization procedures appropriate and clearly defined? [Allocation bias]
Yes. Randomization was computer-derived and stratified according to the clinical center and the use or nonuse of low-dose aspirin, with blocks of 2.
Are the interventions well described?
Yes. One 25-mg tablet of rofecoxib or placebo tablet once daily for three years. Maximum of 20% of patients were allowed to use low-dose aspirin.
Is the study blinded? Are blinding procedures appropriate?
The study was double-blinded except for the statistician. Not appropriate because blinding procedures should’ve been applied to everyone, even if not involved in the study.
Were specific data on drug regimens given including dose, dosage form, duration of administration, time of dose in relationship to meals?
There were no specific data on time of dose in relationship to meals which is a limitation of the study. Food and GI upset or adverse effects can cause the drug to vary from patient to patient. All other data was provided.
Were all study drugs given in appropriate doses and regimens?
The study drug was used for an off-label indication so it is unknown if a 25 mg tablet by mouth is appropriate. It is not approved for colorectal adenoma chemoprevention.
Are both groups comparable, and treated in the same manner, except for the intervention?
Yes both groups had similar baseline characteristics, had to do the run-in placebo phase to include only compliant patients and both had vital signs checked at the same weeks during follow-up.
Were the measures adequate to insure or evaluate compliance?
No. The authors did not specify exactly how compliance was assessed in the placebo run-n phase. Were patients surveyed about how much of their medicine they took? This is important to know because there is a lot of bias involved with use of surveys.
Were there any competing therapies that would have influenced the results?
Yes, other than low-dose aspirin perhaps the type of anti-hypertensive taken would have influenced the results.
If the study is a crossover trial, was the washout, period adequate between interventions?
N/A. This is not a crossover trial.
Statistical Analyses
Have the authors described the statistical analyses to be used in the study?
Yes, the authors used Kaplan-Meier estimates of cumulative event rates over time. For serious events and the APTC endpoint, Cox proportional-hazards models were used to calculate event rates and relative risks, with 95 percent confidence intervals. The relative risk of confirmed serious thrombotic adverse events according to the quartiles of change in mean arterial pressure at week 4, was analyzed because treatment-based differences occurred early and remained constant throughout the treatment period. Changes from baseline in mean arterial pressure as a time-varying covariate in a Cox proportional-hazards model in which treatment was the main effect was reviewed in another analysis.
Are the statistical tests appropriate for the type of data (nominal, ordinal, continuous)?
The data is continuous data and should be tested using parametric tests, which study differences in means of data. This study did not compare differences in the means of data so the statistical tests are inappropriate.
Is the sample size determination information included?
No. Sample size determination information was not excluded which can affect the power of the study.
Have appropriate significance levels been established?
No because for many of the events and APTC endpoints, the groups’ ranges often crossed zero, therefore the differences between them are not significant.
Is the power of the study described?
No. The power of the study was not addressed by the authors.
Overall methodology
Based on the methodology, is the study likely to have external validity?
The study is unlikely to have external validity because it involved a drug that was subsequently pulled from the market by the study’s sponsor, for its increase in deaths due to MI, stroke and other cardiovascular events. The sponsor, Merck, and the investigators probably knew the dangers beforehand so there is a sponsorship and investigator bias. Is the study sample representative of the general population?
No because the sample was overwhelmingly white race and male gender.
Were the interventions practical? (experimental)
No the intervention was not practical but rather dangerous and perhaps unethical. Merck probably had prior knowledge of rofecoxib’s cardiovascular toxicity by the time this study was done.
Results
Patients studied
Is the number of patients specified?
Yes. A total of 3260 patients were screened and 2586 were eligible for the study.
Can all patients be accounted for?
No. 14 patients from the rofecoxib group were lost to follow-up and 20 patients from the placebo group were lost to follow up.
Is the number of dropouts given? Are the reasons for dropping out described? (experimental, follow-up)
Yes. 410 from the rofecoxib group and 319 from the placebo group. The reasons for dropping out are as follows: adverse clinical events, adverse laboratory events, withdrew consent, lost to follow up and other reasons. Were sufficient numbers of patients studied?
No. 2586 seems like a relatively small number when considering the prevalence of cardiovascular disease. The power of the study was not calculated so it is difficult to make an assumption on whether the numbers were sufficient or not.
Were patient demographics presented?
Yes. Race, gender, age, height, weight, comorbid conditions and cigarette use.
Do the groups look similar based on demographics?
Yes. The placebo and rofecoxib groups were matched very well at baseline. Data presentation
Are data presented for all measurements specified in the methodology?
No. Data was not shown for the event curves of the combined end point of congestive heart failure, pulmonary edema, or cardiac failure.
Are data presented objectively?
Yes. The Kaplan-Meier graphs clearly show the difference in cardiovascular events between rofecoxib and placebo.
Are data clear and understandable?
Yes. The data was presented using graphs and tables that are both clear and understandable. They were not cluttered and data was organized by treatment groups, events and displayed confidence intervals, relative risk and differences in event rates.
Statistical Analyses
Are appropriate descriptive statistics presented? [ie measure of central tendency (median, mean, mode), spread of the data (range), variation in the data (SD)]
No. Means and standard deviations should have been presented for all of the data. The only standard deviation values presented were for the increases in systolic and diastolic blood pressure.
Are p values and confidence intervals specified?
P-values and confidence intervals were specified for each event and endpoint.
Are the inferential statistical tests applied appropriately?
No because the data was continuous data so a parametric statistical test should have been applied. Power of the study was not calculated but it should have been in order to determine if the sample size was appropriate
Are statistical analyses meaningful?
Yes. The statistical analyses showed that rofecoxib increased cardiovascular risk and ultimately led to Merck pulling it from the market.
Discussion/ Conclusions
Are the author’s conclusions appropriate based on the data presented?
No. The authors suggest that rofecoxib increases cardiovascular events only among patients with a history of colorectal adenomas. They attempt to deflect attention to other COX-2 NSAIDs such as celecoxib, in order to downplay the danger of the rofecoxib.
Are the results statistically significant?
Yes the results are statistically significant because the corresponding relative risk was 1.92 for a thrombotic event for the rofecoxib group compared to the placebo group (95 percent confidence interval, 1.19 to 3.11; P=0.008).
Are the results clinically significant?
Yes because there were more myocardial infarctions and ischemic cerebrovascular events in the rofecoxib group than in the placebo group.
Does the author discuss objectively the limitations to the study?
No the author did not specifically address limitations. Any reader of the article would have to decipher them.
Are the conclusions consistent with the purpose of the study?
No they are not. The purpose was to assess the chemopreventive effect of rofecoxib but the conclusion was ultimately that patients with a prior history of colorectal adenomas had an increased cardiovascular risk.
Can the conclusions be extrapolated to the population in general?
No. The conclusions cannot be extrapolated to the population because the study was done in a population that already had a history of colorectal adenoma.
Overall
Do the title and abstract appropriately reflect the content of the study?
The title and abstract seem somewhat biased. The authors tried to downplay the seriousness of the cardiovascular events. The discussion on naproxen was irrelevant to the study.
Does the author cite mostly primary literature? Is the article referenced appropriately?
Yes the author cited mostly primary literature and the article is referenced appropriately.
Who sponsored the study?
Merck sponsored the study.
What is the reputation of the journal? Is it peer reviewed?
NEJM has the strongest reputation of all medical journals, as it has the highest impact factor. It is peer reviewed.
Are there editorials available that discuss the article? (companion editorials or editorials that come out later)
Yes, there was an editorial in the NEJM that was very critical of the study.
In general, what are the study’s strengths and weaknesses?
The study’s strength is that it acknowledges an increase in cardiovascular events and displays it well in graphs and tables. However, the study’s blatant sponsorship and investigator bias reflects a conflict of interest. Merck withdrew rofecoxib from the market shortly after this study was published and had to have known about its effects long before then.
Does the study have internal validity?
Yes, the study has internal validity because it can be concluded that there is a cause and effect between rofecoxib use and an increase in cardiovascular risk.
Does the study have external validity? Is it relevant to your problem/situation/practice?
The study does not have external validity because of the past medical history of the patients involved. They had a history of colorectal adenoma and the study didn’t specify the severity of disease. The study is relevant to pharmacy practice because rofecoxib was subsequently pulled from the market. As pharmacists, if the risks of a drug far outweighs its benefits then it doesn’t make sense to dispense
it.