[Ip-health] Size of trials by status (S or P) - Some 2004 FDA
and Parexcel data compared
Joseph DiMasi
joseph.dimasi@tufts.edu
Thu Jul 27 05:34:01 2006
Jamie,
Now that I am back from being away for a while and have seen your new
posting, I am going to do just one more posting on clinical trial
sizes. A number of years ago I pointed out to you that the FDA approval
labels cannot and should not be used as evidence of total clinical trial
sizes. In your report from 2003 that I cited in one of my postings, you
used these kinds of numbers for earlier years, although you did
acknowledge in some manner that the numbers obtained from the FDA
documents may not tell you everything about clinical trial sizes. The
reason that I posted in the first place was that the original posting
from Mike Palmedo on 2004 approvals tried to make an unfavorable
comparison to a number in our R&D cost study using the same approach
that you had used for the 2003 piece, but without even acknowledging
this time that there might be something wrong with your numbers (not
that you should be using the numbers anyway).
There is no question in my mind that what you did underestimates total
clinical trial sizes, often by substantial margins. As you note, the
2004 PAREXEL numbers that I gave you are all larger than the numbers
that Mike Palmedo reported in his posting for the same drugs. You
wondered why. As far as I know, PAREXEL uses FDA documents as a
starting point, but they realize that they are inadequate and they make
some attempts to get additional information. Regardless of what PAREXEL
does, why your numbers are much too low is no mystery to me. It is
clear from examining a handful of cases some years ago and again for
some of the 2004 approvals what you are doing. The FDA does not report
on the sizes of all clinical trials on a drug in its approval labeling
(nor has it been their intent to do so). What you have done is to
simply go to the "Clinical Studies" sections of the labels and count up
the trial sizes that the FDA might mention there. However, what is
discussed in that section is just some of the efficacy trials that the
reviewers considered pivotal for the approved indication. They do not
discuss every clinical trial for even the approved indication, let alone
other indications that were examined. They generally discuss a few
phase III trials (not necessarily all of them). They might occasionally
mention a phase II trial, and probably never discuss phase I trials. For
this reason, what you get out of this section is clearly not everything
that there is. You can even sometimes see this from information in
other parts of the label. Let's take as examples, the two drugs that
you mention below where there are large differences between what you and
PAREXEL have (Lyrica and Enablex).
You had 1,508 as the total number of subjects in trials for Lyrica,
while PAREXEL had 9,100. 1,508 is the number that you get out of the
Clinical Studies section. There are six efficacy trials mentioned
there. However, if you go to another part of the label you will see a
statement that in all controlled and uncontrolled premarketing trials
"MORE (my emphasis) than 9000 patients have received pregabalin." There
are two things to note about this statement. One is that it by
definition gives you an underestimate of the number of subjects that
received the drug. The other thing to note is that it only mentions the
active ingredient. That is, it does not tell you how many subjects
received a placebo or an active comparator (if there was one). Perhaps
this is what PAREXEL used and 9,100 was just a typo, or perhaps they had
some additional information. However, you can use other information in
the label to go beyond even 9,100 (but still have an underestimate).
The number of subjects in the placebo arms of five of the six efficacy
trials mentioned in the Clinical Trials section are noted there. These
total 425. So now, we are up to 9,425. This is six and a quarter times
larger than the number that cptech reported. And still, we know that it
must be an underestimate. We don't know the number of subjects that
received a placebo in the sixth study mentioned in the Clinical Studies
section, there probably were subjects in other trials that received
placebo (or active comparator), and 9000 was acknowledged to be an
underestimate of the number of subjects that received the drug that was
approved.
Finally, let's look at the case of Enablex. Cptech had the total number
of subjects at 1,454, while PAREXEL had it at 8,830. Again, 1,454 is
what you get out of the Clinical Studies section for four efficacy
trials. However, elsewhere in the label you see the statement that the
safety of Enablex was evaluated in "Phase II and III controlled clinical
trials in a total of 8,830 patients." Of course, even 8,830 is an
underestimate. It does not tell you anything about phase I uncontrolled
trials. Once again, we can do better by examining other information in
the label. We are told that 6,001 of the 8,830 patients received
Enablex. We are also told that a total of 7,363 subjects were treated
with Enablex from 3.75 mg to 75 mg once daily. So, at least 1,362
subjects received the active drug outside of controlled phase II and III
trials. So now we are up to at least 10,192 subjects, or seven times
what cptech reported as coming from FDA documents.
What all of this tells me is that approach that cptech uses
systematically underrepresents the number of subjects in clinical
trials, the numbers that cptech uses are, on average, substantially
below those reported by PAREXEL, and that even the PAREXEL numbers at
least sometimes underestimate clinical trial sizes. You should not
continue to report the kinds of numbers on clinical trial sizes that you
have reported.
Joe DiMasi
James Love wrote:
> The tables and the links work best from the Blog version. Jamie
>
> http://www.cptech.org/blogs/drugdevelopment/2006/07/size-of-trials-by-
> status-s-or-p-some.html
>
> Size of trials by status (S or P) - Some 2004 FDA and Parexcel data
> compared
> James Packard Love, Friday, July 14, 2006
>
> This note is a follow-up to discussions stimualted by Michael
> Palmedo's note on 2004 FDA NME drug approvals. In particular, it
> follows discussions on ip-health by Joe DiMasi and myself on a fairly
> narrow question -- are clinical trials trials larger for Standard (S)
> FDA NME drug approvals than for Priority (S) approvals?
>
> The following table reports the size of clinical trials for 5
> priority and 8 standard FDA NME drug approvals. The products are the
> union of those reported by Michael Palmedo for 2004 FDA approvals,
> and data from Parexel. These are the data that Joe DiMasi referred to
> in his June 14 post to ip-health.
>
> Drug Rating Size, FDA letter Size, Parexcel
> Clolar P 66 138
> Lyrica P 1,508 9,100
> Prialt P 1,434 1,634
> Sensipar P 1,146 2,000
> Tarceva P 1,837 6,000
> Apidra S 2,467 4,093
> Cymbalta S 1,850 6,100
> Enablex S 1,454 8,830
> Fosrenol S 2,357 2,697
> Ketek S 2,016 5,900
> Lunesta S 2,100 2,909
> Spiriva S 2,663 3,168
> VESIcare S 3,027 3,327
>
> Below are the mean and median , for the FDA and Parexcel data,
> reported by (P) and (S) drugs.
>
> Mean-FDA Median-FDA Mean-Parexecl Median-Parexcel
>
> Standard Approvals 2,242 2,229 4,628 3,710
> Priority Approvals 1,198 1,481 3,774 2,000
> Difference 1,044 854 795 1,710
> % larger 87% 55% 23% 86%
>
>
> Two quick points. First, Parexcel reports more patients for every
> trial. Second, the number of data points is pretty small (5 P and 8 S
> drugs), so one has be careful about drawing conclusions.
>
> Joe notes that when you look at means from the Parexel data, the
> trials for Standard approvals (S) are only 23 percent larger than for
> the Priority products. Joe notes that by comparison, when looking at
> the FDA data, the mean size of the trials for Standard approvals were
> 87 percent higher, suggesting a possible bias when looking at FDA data.
>
> However, I would add, that when looking at the MEDIANS of the
> Parexcel data (for the 13 products), the differences between the size
> of standard and priority drug trials are quite pronounced. For the
> Parexcel data, the median size of trials for the Standard drugs is 86
> percent larger than the size of the median trial for the priority
> drugs -- actually higher than the 55 percent difference (in medians)
> that Michael reported, looking at FDA data for the same drugs.
>
> Ultimately, this is too small a sample to say that much. We'll take a
> look at a larger sample, and report that. But before doing so, it is
> also interesting to look at the differences between the FDA data and
> the Parexecl data. Paraexcel always reported more patients in the
> trials than did Palemedo, looking at the FDA approval letters. In
> some cases, much more. Pfizer's Lyrica, for example, was reported by
> Palmedo as 1,508, and Parexcel as 9,100. Enablex, reported by Palmedo
> as 1,454, is reported by Parexecl is 8,830. In looking further at
> this issue, we will also look closer on these differences. One person
> suggested the initial FDA approvals may not report parallel trials in
> the works for other indications (Lyrica is now approved for 3
> indications, for example). Another comment is that some of the
> "trials" reported by Parexel may be of lesser scientific importance
> (possibly having value for marketing purposes), or may be un-reported
> by the FDA other reasons. People may speculate or offer some evidence
> on these points in the comments to this note.
>
> This issue has generated some debate with Joe DiMasi, because we have
> questioned his repeated finding (in 1991 and 2001/2003) that priority
> products are more costly than standard drugs, at least in terms of
> the important area of clinical trials. Our reviews of the data, on a
> couple of different occasions, have suggested that priority drugs
> consistently have smaller clinical trials than do standard approvals
> (findings borne out here again). If priority drugs have smaller
> trials and quicker approvals, they would seem to be less expensive,
> all other things being equal. Joe's comments have been informative
> and constructive, and we will revisit the issue, incorporating both a
> broader analysis of the Parexecl data, and a closer look at the
> differences between the FDA and Parexel data, as well as other
> evidence on this topic.
>
> Finally, we remind people that neither the Parexel nor the earlier
> (2001, 2001/2003) DiMasi et all data claim to present data for all
> drug approvals. Most importantly, DiMasi has said that "It should
> also be noted that our study was based on the R&D experiences of
> major traditional pharmaceutical firms," in contrast to "small
> biotech and niche pharmaceutical firms." This is not a criticism of
> the DiMasi studies, as any analysis is going to be limited in some
> way. It is rather a reminder that some of the estimates provided by
> DiMasi are based upon particular samples that may not be
> representative of other drug development efforts. Indeed, DiMasi's
> 2001/2003 paper, which is now so widely quoted, drew important
> conclusions about relative investments in priority and non-priority
> drugs from just 10 priority products and 14 standard products (DiMasi
> 2003 page 172). His estimates of out-of-pocket outlays were also more
> than twice as high as the previous PERI study involving 117 drug
> development projects (Project Management in Pharmaceutical Industry:
> A survey of Perceived Success Factors 1995-1996, PERI), raising some
> questions about the nature of the sample he studied. To deepen the
> understanding of these issues, people have to look at more data, and
> do some modeling of their own.
>
>
> ---------------------------------
> James Love, CPTech / www.cptech.org / mailto:james.love@cptech.org /
> tel. +1.202.332.2670 / mobile +1.202.361.3040
>
> "If everyone thinks the same: No one thinks." Bill Walton
>
>
> _______________________________________________
> Ip-health mailing list
> Ip-health@lists.essential.org
> http://lists.essential.org/mailman/listinfo/ip-health
--
-----------------------------------------------
Joseph A. DiMasi, Ph.D.
Director of Economic Analysis
Tufts Center for the Study of Drug Development
Tufts University
192 South Street, Suite 550
Boston, MA 02111
tel: 617-636-2116; fax: 617-636-2425
URL: http://csdd.tufts.edu
-----------------------------------------------