[Ip-health] Costs of Clinical trials: DiMasi I and DiMasi II
compared
Joseph DiMasi
joseph.dimasi@tufts.edu
Mon, 28 Apr 2003 14:23:14 -0400
Jamie, below are some comments to an earlier posting of yours.
Joe DiMasi
>Given the data we look at, the most eye popping part of the recent DiMasi =
study
>concerns the estimates of the costs of clinical trials, which are *way* up=
over previous >numbers, which themselves were considered high.
Considered high by whom, and on what basis? You write as though this
was widely accepted wisdom. That certainly was not the conclusion of
the Office of Technology Assessment (OTA) in their 1993 report
("Pharmaceutical R&D: Costs, Risks, and Rewards") after they conducted
their own corroborative efforts. To quote from their report: "the
estimates by DiMasi and colleagues of the cash outlays required to bring
a new drug to market and the time profile of those costs provide a
reasonably accurate picture of the mean R&D cash outlays for NCEs first
tested in humans between 1970 and 1982" (OTA, 1993, p.66).
>or how the total for Phase I, II and III rose from $28.6 to $125. Note als=
o that the TB
>Alliance estimated: "A full programme of clinical development (Phase I thr=
ough Phase
>III trials) for a new anti-TB drug is estimated to cost about $26.6 millio=
n in a country
>with an established economy.
You really ought to stop trying to use the TB Alliance report to
undermine our results. The numbers are not comparable, and their report
is not inconsistent with our study. The authors did not make any
representation in the report, nor would they do so now, that their
projections are indicative of what development costs are for drug
development as a whole. I could discuss a number of comparative issues
related to the various levels of their analysis, but I will restrict my
comments here to the comparison you are trying to make above (i.e.,
differences in clinical development costs for a single lead drug
candidate).
=B7=09The development costs in the report are focused on development of a
hypothetical lead drug candidate, without specified pharmacologic or
chemical properties, for a tuberculosis INDICATION.
=B7=09The figures in our study are costs per approved DRUG, not per approve=
d
INDICATION. Many drugs are investigated for multiple indications. This
has even been the case for some drugs developed for TB, as will be
indicated below. Thus, development costs associated with a single
indication will often not give you the total costs for a drug.
=B7=09The protocol for the hypothetical drug is based on the development of
an antibiotic (rifapentine) that was approved for a TB indication in the
United States in 1998. The types of activities conducted during a
clinical development program can differ markedly by indication. Drug
development costs for antibiotic (as opposed to some antiviral)
indications are likely below average (although antibiotics are often
studied in numerous indications so that total drug costs may not differ
as much). In addition, rifapentine was developed under the accelerated
approval and orphan drug programs. While this is likely a reasonable
approach to modeling costs for a tuberculosis indication, fast-tracked
orphan drug development is not representative of drug development as a
whole. In particular, development costs for such indications would
typically be well below average.
=B7=09The protocol, which follows the "Scientific Blueprint for TB Drug
Development", is predicated on doing just one Phase III trial. While it
is quite possible to succeed this way for a drug developed for this
indication, it is very uncommon in general for the FDA to accept just
one pivotal trial. In practice, firms will also often rationally do
more than a couple of large-scale pivotal trials more or less in
parallel to cover themselves in case some trials fail to demonstrate the
desired results. In addition, the total number of trials in the
protocol is 9. This contrasts with various estimates of 30 to 68 trials
per new drug application over different periods, which further
underscores the differences to be expected when considering developing a
drug for a tuberculosis indication as opposed to an average for the full
development programs across all drugs.
=B7=09Probably the single most important difference between the parameters
used in the TB Alliance report protocol and the actual average
experience across all drugs is the number of subjects in clinical
testing. The TB protocol posits 1000 subjects for Phase III and 1368 in
total. Others have independently estimated the average number of
subjects in NDAs for new molecular entities approved from 1998-2001 to
be 5,621. This is reported on in our paper. The average is determined
including some special cases, if you will, where only several hundred
subjects were used. Adding the average number of subjects across phases
for the successes and failures, and covering a longer period, we found
an average of 5,303 for our sample. This is also reported in the
paper. Either way, we have approximately a four-fold higher value for
the number of subjects on average compared to the number posited for the
tuberculosis indication.
=B7=09There has been too little development of drugs to treat tuberculosis
over a recent enough time frame to be able to obtain reasonable
estimates for anti-TB development based on actual historical data.
Thus, the best that one can do is to model costs. The authors are to be
commended for undertaking such a detailed appraisal. However, it is
probably the rare drug that actually comes in on time and on budget when
viewed at a point some ten years from prospective approval. Drug
development is not a fixed cookie-cutter process, and most development
plans do not survive intact as originally envisioned. Drug companies
will often project future costs and returns and evaluate net present
values as is done in the report (although not often as early as
identification of a lead candidate). It is prudent for them to do so.
However, they realize that the projections are not infallible. Thus,
they regularly update their projections of expected future costs and
returns as more is learned about the drug, the markets, and the
regulatory environment. To quote a Lilly executive at a recent
conference: "Many times we don't forecast correctly, and we don't know
what's going to happen. We live in an industry where what we don't know
is substantially greater than what we do know, almost at any point in
time. So we have to deal all the time with incomplete information and
try to make the best decisions we can" (The Pink Sheet, March 24, 2003,
p.7). The authors of the TB Alliance report are aware that there are
uncertainties that can raise costs, even within the confines of the
stated development plan. I could give you a number of quotes from the
report where this is acknowledged. As noted, firms will undertake
exercises similar to what is found in the TB Alliance report. If the
resulting expected profit outcome is substantially "in the money" (to
borrow a phrase from finance), then they will take this as a strong
signal to proceed. If the outcome is substantially "out of the money",
then they likely will pull the plug on the project. Prediction errors
are not particularly important in these cases. It is in the
intermediate range where the decision becomes very difficult and so
other factors come into play.
=B7=09The points I raised above about uncertainties in drug development and
drug versus indication costs can be illustrated with a real world
example of a drug that has been investigated as a treatment for
tuberculosis. The trade press has reported on SR Pharma's
investigational drug, SRL-172, which was studied as an adjuvant to
standard tuberculosis therapy. The Lancet reported on a failed Phase
III trial that involved 1,229 subjects (23% more than the Phase III
trial size posited in the TB report). However, this was the third Phase
III tuberculosis trial for this drug that failed ("SRL-172 fails again
in tuberculosis," SCRIP, no. 2788, October 9, 2002, p.27). The number
of subjects for the other two Phase III tuberculosis trials (or for
Phase I and II trials) were not reported, but if they were similar to
the trial size for the one that was reported then the total number of
subjects for the tuberculosis indication is at least three times higher
than for the program discussed in the TB report. In addition, although
the company appeared to have given up on the indication, the trade press
article indicated that some experts expect that this might not be the
end of the road for this drug in that indication. If another Phase III
trial (and perhaps additional earlier phase trials) is conducted, then
this will obviously increase the total number of subjects
substantially. Furthermore, the trade press and commercial drug
databases indicate that SRL-172 has been investigated clinically in a
wide range of indications that include the following: non-small cell
lung cancer, small cell lung cancer, renal cancer, prostate cancer,
melanoma, HIV-associated mycobacterium avium infection, hepatitis-B
infection, asthma, atopic eczema, and seasonal allergic rhinitis. The
drug has reportedly undergone Phase I testing for all ten of these
indications, Phase II testing for the vast majority, and has reached
Phase III testing for at least two indications aside from tuberculosis.
How many subjects were and will be used, and at what cost, for all of
these other indications? I do not know, but the costs for these
indications clearly can add substantially to the total for the drug. If
this drug were in my sample, its clinical period cost would include the
aggregate of the costs for all of these indications.
=B7=09There are a few other more micro level points that I can make. The
full clinical period cost estimate per investigational drug from our
study includes, approximately $5 million, on average, for long-term
carcinogenicity and teratogenicity testing conducted concurrent with
clinical testing. However, there are other nonclinical costs that occur
after the initiation of clinical testing that are rolled into our
clinical phase estimates. These can include some of the types of
preclinical development and chemistry, manufacturing, and controls (CMC)
activities that are listed separately from the clinical trial costs in
the TB Alliance report. The report does note that some of these costs
can be incurred after the start of clinical testing. Such costs, when
they so occur, are included in our clinical estimates. That is why I
have always been careful to refer to our total clinical cost estimates
as "clinical period" costs, and not "clinical trial" costs. In
addition, our estimates will include charges for the cost of maintaining
a clinical development infrastructure.
=B7=09The unit costs for many of the clinical activities in the TB Alliance
report were obtained by using the Medicare Resource Based Relative Value
Scale (RBRVS) reimbursement schedule. This was done because the
schedule is very comprehensive. However, those charges would not
include any balance billing that providers may levy. In any event, fees
in the markets for investigator and lab services used in drug
development are not restricted by Medicare reimbursement rates, and may
well be higher. In addition, the quality of investigators used in drug
development may, on average, be higher than that of physicians as a
whole, and they may therefore command a premium for their services.
=B7=09The scenario laid out in the TB Alliance report is one in which CROs
are hired to conduct all of the tasks involved in developing a drug from
lead identification to regulatory approval. However, when a firm
contracts with a CRO to conduct an activity the firm will incur internal
costs associated with monitoring the CRO and managing the information
obtained. These costs are not included in the analysis. In addition,
the costing is essentially resource based. Specifically, it appears
likely that the costs represent the resource costs to the CROs. If a
pharmaceutical firm hired a CRO, it would also have to cover the CRO's
profits. The report notes that the Phase III cost estimate ($22.6
million) is in some sense close to the reported cost of a Phase III-like
trial conducted post-approval for rifapentine by the CDC ($25 million).
However, it is worth noting that the reported rifapentine cost is 11%
higher, and, as indicated by a table in the report, was conducted with
15% fewer patients enrolled. It is also unclear how similar in nature
and expense CDC conducted trials are to the kinds of trials that are
conducted by pharmaceutical firms.
=B7=09The last point highlights the special nature of this model of drug
development. As the rifapentine example shows, under the report's
scenario, a drug that obtained regulatory approval for tuberculosis
would almost surely be required to conduct Phase IV testing (on the
scale of a substantial Phase III trial). In essence, in this kind of
model one is pushing some costs from what would typically be incurred
pre-approval to post-approval. An increasing number of drugs are being
approved with Phase IV requirements (although reportedly some firms have
been slow to conduct the confirmatory trials in the oncology area).
However, not all drugs proceed in this fashion, and they in fact still
represent a minority of all drugs approved. If everything else were
equal, drugs that receive conditional approval would have below average
pre-approval costs. We do have a post-approval R&D cost estimate in our
new study to help account for this phenomenon, but clearly post-approval
costs are more important for some drugs than for others.
The points listed above make it apparent that the figures in the TB
Alliance report should not be used to validate or invalidate our
results. One could argue that, given the points raised above, one might
attempt to "scale-up" the figures in the TB Alliance report to get some
sense for what average drug development costs might be. Clearly, one
could do so and get numbers that are up to and beyond our figures.
However, there is so much uncertainty involved in doing so that I
wouldn't even try it. What is abundantly clear, though, is that
juxtaposing the totals given in the TB Alliance report against our study
figures is inappropriate and highly misleading.
James Love wrote:
>
> Given the data we look at, the most eye popping part of the recent
> DiMasi study concerns the estimates of the costs of clinical trials,
> which are *way* up over previous numbers, which themselves were
> considered high. Given the fact that some products like Glivec, are
> approved on the basis of small trials (about 1,000 patients), it is
> difficult to see how one can estimate the *average* cost of Phase III
> trials at $86.3 million, for example, or how the median cost of Phase I
> trials increased from $1.5 million to $13.9 million, or how the total
> for Phase I, II and III rose from $28.6 to $125. Note also that the TB
> Alliance estimated: "A full programme of clinical development (Phase I
> through Phase III trials) for a new anti-TB drug is estimated to cost
> about $26.6 million in a country with an established economy. Comparable
> studies conducted in a country with a developing economy are estimated
> to cost approximately $9.9 million."
>
> Here are the raw (not adjusted for capital costs or risks) DiMasi I and
> DiMasi II data for clinical trials compared.
>
> Average out of pocket cost for clinical trials by Phase, in 2000 dollars
>
> Phase I
> DiMasi I
> Mean $ 3.2
> Median $ 1.5
>
> DiMasi II
> Mean $15.2
> Media $13.9
> Phase II
> DiMasi I
> Mean $ 6
> Median $ 3.3
>
> DiMasi II
> Mean $ 23.5
> Median $ 17
>
> Phase II
> DiMasi I
> Mean $ 19.4
> Median $ 12
>
> DiMasi II
> Mean $ 86.3
> Median $ 62
>
> Total, Phase I, II and III
> DiMasi I
> Mean 28.6
> Median 16.7
>
> DiMasi II
> Mean 125
> Median 92.9
>
> --
> James Love, Director, Consumer Project on Technology
> http://www.cptech.org, mailto:james.love@cptech.org
> tel. +1.202.387.8030, mobile +1.202.361.3040
>
> _______________________________________________
> Ip-health mailing list
> Ip-health@lists.essential.org
> http://lists.essential.org/mailman/listinfo/ip-health
--
-----------------------------------------------
Joseph A. DiMasi, Ph.D.
Director of Economic Analysis
Tufts Center for the Study of Drug Development
Tufts University
192 South Street, Suite 550
Boston, MA 02111
tel: 617-636-2116; fax: 617-636-2425
URL: http://csdd.tufts.edu
-----------------------------------------------