[Ip-health] Bayh Dole Rights, Size of Clinical Trials, 2004 Approvals
James Love
james.love@cptech.org
Wed Jul 12 18:37:04 2006
On Jul 12, 2006, at 5:42 PM, Joseph DiMasi wrote:
> Jamie,
>
> You seem to have missed the main point of my posting. My post does
> not
> maintain that the number of subjects is not a factor in determining
> costs; the point was that it is hardly the only one, so that you
> cannot
> justify inferences about costs (or relative costs) based just on the
> number of subjects.
>
> A few other points: The numbers you report are not for all new
> FDA-approved entitites because you do not have complete information
> for
> CDER drugs and you are missing entities approved outside of CDER.
> This
> is particularly important for the earlier years, before much of what
> CBER did was switched to CDER. While it is difficult to think of
> reasons why the relative data would be biased in one way or another by
> rating, underestimation at the individual drug level is likely highly
> variable. This imparts a substantial noise component to the ratio, so
> the ratio could be fairly imprecise (less so, obviously, the larger is
> the sample size). This is not to say that priority drugs have had
> higher pre-approval subject sizes. The point was that the particular
> ratio one gets can be unreliable.
>
It is pretty consistent, for each study I have looked at.
> The way that the FDA statistics on all CDER NDA approvals have been
> used
> is, I think, highly misleading. One should generally think of line
> extensions, not as separate new drugs, but as part of the
> lifecycles of
> new active ingredients. The FDA data on NDA approvals also has much
> chaff that should be separated from the wheat. Readers can get a
> better
> sense for what I mean from a paper we have posted on the web:
> http://csdd.tufts.edu/_documents/www/Doc_231_45_735.pdf
>
> Orphan drugs overwhelmingly get priority ratings. Some of my thoughts
> on orphan drugs are covered in what you reproduced below.
>
> "Major traditional pharmaceutical firms" covers where the bulk of the
> criticism and scrutiny of the industry goes. It also almost seems
> like
> you are trying to cast some apsersion here. The fact is that this is
> was what was deemed feasible to examine. Development outside of this
> framework was thought to be too diffuse to examine. If I thought that
> we could have gathered that kind of information in a reasonable
> timeframe, we would have done so. To cover that sector
> comprehensively
> you would also have to get information for all of the start-ups
> (private, as well as public companies) that had some discovery or
> preclinical development, but that had never gotten a drug into
> clinical
> testing. It is also interesting to note that annual surveys of the
> "biotech" sector have shown that, while a small number of firms have
> prospered, the sector as a whole has yet to earn positive accounting
> profits in any year of the more than two decades of its existence.
>
I have no problem with you doing studies of the costs of drug
development for "major traditional pharmaceutical firms," even if
some of the conclusions are only based upon 10 priority and 14
standard drugs. It was a useful, and of course, much used study.
However, I do have a problem with people thinking that your study is
something it is not. For example, it is NOT a comprehensive study
of drug development, and it should probably not be use to infer costs
of orphan drug development, as it is over and over again, including
for example, in litigation over Gleevec and other drugs.
Nor do I find your results about clinical costs for priority
drugs costing MORE to be very believable, based upon lots of other
data, and it seems to be an artifact of the selection of the projects
you looked at.
However, it should not be hard to deal with the simple issue of
the clinical trials for priority products. I believe we could work
from the public (but still incomplete) data from Parexcel reports (do
you have a recent copy?). We might also collaborate on a letter to
Mark McClellen suggesting he consider requiring the FDA to disclosure
more useful data about drug development inputs so there is more
transparency of these items, including for example, data on the out
of pocket costs and timing of trials. Jamie
> Joe
>
>
>
> James Love wrote:
>
>> Joe,
>>
>> Thanks for posting the correction to the error in the 2003 paper,
>> regarding the relationship between priority and standard approvals.
>> However, according to a more recent analysis the the US FDA Center
>> for Drug Evaluation and Research (CDER) concerning new drug approvals
>> (NDAs) approved from 1990 to 2004, of the 1,284 new drug approvals
>> (NDAs) approved from 1990 to 2004, only 289, or 22.5%, were for
>> "priority" reviews, defined as a product that has "Significant
>> improvement compared to marketed products in the treatment,
>> diagnosis, or prevention of a disease." Of these, only 183 (14.3
>> percent of the total) were new molecular entities (NMEs) classified
>> as priority products.
>>
>> This would suggest an even lower share of investments in new products
>> that are considered both newer and significantly better than existing
>> products than the table from the 2003 paper, before even reaching the
>> issue of the size of trials.
>>
>> But back to my earlier point yesterday on this list, regarding the
>> size of clinical trials for priority vrs standard products, all of
>> the surveys that I have seen of the size of trials show that
>> "priority" products have much small trials. While most of these
>> studies are based upon FDA approval letters, I would be surprised if
>> you dispute the relationship. There is no reason to believe that
>> there is a bias in the FDA approval letters one way or another....
>> but in any case, you could easily check this out from other public
>> sources of data on clinical trials, such as Parexel's surveys. Fax
>> us the relevant pages from Parexel' most recent reports on this and
>> we'll do the analysis.
>>
>> The fact that you claim that priority drugs had higher costs in your
>> 2003 study raises some questions about the sample you used in your
>> 2003 study (only 10 priority products and 14 standard products, page
>> 172).
>>
>> Earlier, on this list, you indicated that products developed by
>> biotechnology and small pharmaceutical firms were under-represented
>> in your 2003 study (the one announced in 2001 by the CEO of Merck,
>> http://lists.essential.org/pipermail/ip-health/2001-December/
>> 002492.html)
>>
>> You said then: "It should also be noted that our study was based on
>> the R&D experiences of major traditional pharmaceutical
>> firms." (link to this quote below).
>>
>> When Mike looks at drug approvals, he includes everything in a given
>> year for which the FDA reports data. He does not limit his studies
>> to "major traditional pharmaceutical firms." Indeed, it appears as
>> though many of the priority products the FDA approves are not
>> reflected in your studies of drug development -- a fact that you
>> freely admit, but rarely advertise.
>>
>> So, back the point raised yesterday, do you disagree or doubt that on
>> average, FDA approved priority drugs have much smaller clinical
>> trials than do standard drugs? And if so, why wouldn't this be
>> important in terms of their costs?
>>
>> Jamie
>>
>> http://lists.essential.org/pipermail/ip-health/2001-December/
>> 002508.html
>>
>> Of course, orphan drug development is not at all representative of
>> drug
>> development as a whole. As you know, the trials are much smaller
>> than
>> is the case for most other development. So even if costs are
>> relatively
>> low for testing orphan indications, as they likely are, they tell you
>> nothing about average costs for drug development as a whole. It
>> should
>> also be noted that our study was based on the R&D experiences of
>> major
>> traditional pharmaceutical firms. The vast majority of orphan drug
>> designations are given to small biotech and niche pharmaceutical
>> firms.
>> In terms of R&D budgets, orphan drug development is generally a very
>> small part of the development programs of the major firms. For the
>> niche players it appears to be a low-cost low-return venture that
>> they
>> are willing to undertake (with a handful of well-noted exceptions,
>> orphan drug sales are typically quite low).
>>
>>
>>
>>
>> On Jul 11, 2006, at 5:50 PM, Joseph DiMasi wrote:
>>
>>> Jamie,
>>>
>>> I feel like I am in the movie "Groundhog Day." I have addressed
>>> this
>>> sort of thing with you in years past. You cannot make inferences
>>> about
>>> full R&D costs from just clinical trial size numbers (even correct
>>> numbers, which you do not have). There is also enough noise in the
>>> underestimates to make your ratio by therapeutic rating
>>> questionable. I
>>> do not wish to keep revisiting this. I will simply point readers
>>> to a
>>> commentary (which I would have thought you had seen) that I
>>> posted on
>>> WHO's CIPIH website about a year and a half ago in response to a
>>> commentary from Aidan Hollis on me-toos that deals (albeit in the
>>> context of a discussion about "me-toos") with your point:
>>> http://www.who.int/intellectualproperty/forum/HollisResponse.pdf
>>>
>>> For convenience, the relevant part on my comment is copied below:
>>>
>>>> Professor Hollis does note and rely on an analysis in Love (2003)
>>>> that
>>>> uses the FDA therapeutic ratings for new molecular entities (NMEs)
>>>> approved from 1993 through 2002 to claim that =93the share of
>>>> investments in new products that have significant improvements over
>>>> existing treatments is 20 percent, with 80 percent of the
>>>> investment
>>>> in new products spent on projects that demonstrate no significant
>>>> improvement over marketed products.=94 This percentage breakdown is
>>>> based only on two ratios. There are a number of problems with the
>>>> calculation. One of the ratios used is the percentage share of
>>>> NMEs
>>>> that were priority-rated by the FDA for NMEs approved from 1993 to
>>>> 2002. The value of the ratio is based on an arithmetic error.
>>>> Love
>>>> (2003, p.17) has a table giving the annual numbers of priority and
>>>> standard NMEs for 1993 to 2002. The annual numbers are correct and
>>>> the sum of standard NME approvals is correct. However, the sum of
>>>> priority NMEs given in the table is wrong. Love (2003) has 79
>>>> priority NMEs in total for this period when the actual number is
>>>> 120.
>>>> The correct number raises the percentage of NMEs with priority
>>>> ratings
>>>> from the 31% that Love (2003) has to 40%.[1] <#_ftn1>
>>>>
>>
>>>>
>>>> The other ratio used is a ratio of clinical trial sizes for
>>>> priority
>>>> and standard-rated drugs, where the clinical trial sizes were
>>>> obtained
>>>> from some FDA documents. These documents do not include
>>>> discussions
>>>> and data for all clinical studies conducted on investigational
>>>> drugs.
>>>> This is acknowledged in Love (2003), but it is argued there that
>>>> this
>>>> should not affect the ratio of clinical trial sizes. That is
>>>> somewhat
>>>> dubious given that the differences between clinical trial sizes
>>>> obtained in this way and actual clinical trial sizes are likely
>>>> highly
>>>> variable by drug. However, even if we put that concern aside,
>>>> there
>>>> is no reason to believe that the amounts spent during the clinical
>>>> testing period on a per subject basis is fixed across therapeutic
>>>> ratings. This is an implicit assumption in the Love (2003)
>>>> calculation. Results in DiMasi et al. (2003) suggest that costs
>>>> per
>>>> subject for priority-rated approved new drugs were 16% higher than
>>>> for
>>>> standard-rated drugs (down from 42% higher for an earlier period
>>>> with
>>>> a three-tiered rating system [DiMasi et al., 1991] with the two
>>>> higher
>>>> ratings compressed into one group). If we applied the correct
>>>> share
>>>> of NME approvals for priority drugs for the period given and a 16%
>>>> higher cost per subject for priority NMEs, then the expenditure
>>>> share
>>>> for priority NMEs increases from 20% to 44%.
>>>>
>>>>
>>>>
>>>> The estimation of such shares for approved drugs, however, is of
>>>> questionable validity and significance. Two important factors not
>>>> considered in these calculations are discovery costs and clinical
>>>> success rates. If discovery costs are higher and success rates are
>>>> lower for priority drugs, then an estimation of the type in Love
>>>> (2003) could substantially underestimate the share of total
>>>> expenditures attributed to priority drugs. Beyond these
>>>> considerations, attempts to draw normative conclusions based
>>>> just on
>>>> distinctions between spending on priority and standard drugs is
>>>> highly
>>>> questionable given results in DiMasi and Paquette (2004) that
>>>> indicate
>>>> that development of drugs that eventually get standard ratings is
>>>> often done more or less in parallel with drugs that get priority
>>>> ratings and because the likelihood of finding at least one
>>>> molecule in
>>>> a class that has an acceptable risk/benefit ratio increases with
>>>> the
>>>> number of independent development efforts.[2] <#_ftn2>
>>>>
>>>> -------------------------------------------------------------------
>>>> --
>>>> ---
>>>>
>>>> [1] <#_ftnref1> Prior to enactment of the user fee program and the
>>>> priority/standard two-tiered rating system, the share of NMEs that
>>>> were rated the equivalent of priority was 49% (A and B ratings in a
>>>> three-tiered system). For the two years subsequent to the end
>>>> of the
>>>> period used in Love (2003), the share of NMEs that received
>>>> priority
>>>> ratings increased to 43% for 2003 and 56% for 2004 (including five
>>>> biologics moved to FDA=92s CDER from CBER in a reorganization in 2003
>>>> that shifted the review of most biologics to the drug center; the
>>>> share was 52% excluding the five biologics).
>>>>
>>>> [2] <#_ftnref2> See, for example, discussions in Scherer (1966) and
>>>> Scotchmer (2004) regarding innovation in general. To put the
>>>> matter
>>>> formally in a simple context, suppose that N firms pursue different
>>>> independent approaches to developing a drug in a new, but unproven,
>>>> class, and that the probability of failure, q, is the same for all
>>>> firms. Then, the probability that at least one investigational
>>>> drug
>>>> will be found that has an acceptable risk/benefit ratio is 1-q^N .
>>>> Thus, the likelihood of at least one success increases with the
>>>> number
>>>> of firms pursuing different molecules in a class. This observation
>>>> alone does not tell one what the socially optimal number of
>>>> experimental approaches is, but we would certainly expect that in
>>>> general it is greater than one.
>>>>
>>>
>>> Joe DiMasi
>>>
>>>
>>> James Love wrote:
>>>
>>>> Joe, I was wondering if you would comment on this issue. In two of
>>>> these studies, Michael has found far fewer patients (a little more
>>>> than half) in clinical trials for FDA priority drugs, than for the
>>>> drugs the FDA classifies as "standard." This would suggest the
>>>> costs associated with R&D for "priority" products is quite a bit
>>>> lower (assuming that any product is allowed to be "below
>>>> average" in
>>>> terms of costs), since the costs of clinical trials are highly
>>>> correlated with the size of the trials.
>>>>
>>>> Jamie
>>>>
>>>>
>>>> On Jul 10, 2006, at 5:40 PM, Joseph DiMasi wrote:
>>>>
>>>>> Mike,
>>>>>
>>>>> I have been down this road on clinical trial sizes before with
>>>>> Jamie.
>>>>> First of all, the results from the study refer to a particular
>>>>> period in
>>>>> the past, not 2004. Second, and more important, the numbers that
>>>>> you
>>>>> pull out of labels or reviewer summary documents posted on the web
>>>>> need
>>>>> not come anywhere close to the number of subjects actually studied
>>>>> prior
>>>>> to a U.S. approval. The FDA labels and documents will often just
>>>>> note
>>>>> what the reviewers considered to be trials that were pivotal
>>>>> to the
>>>>> indication that was approved (i.e., often a few relevant phase III
>>>>> trials). You cannot assume that what you pick out of these
>>>>> sources
>>>>> represents the entirety of clinical testing. Third, we noted
>>>>> in our
>>>>> study data gathered independently by PAREXEL and reported for
>>>>> various
>>>>> years in their compendium of pharmaceutical industry R&D
>>>>> statistics
>>>>> that
>>>>> had a mean of 5,621 for a period relevant to our study.
>>>>>
>>>>> Joe DiMasi
>>>>>
>>>>>
>>>>>
>>>>> Mike Palmedo wrote:
>>>>>
>>>>>> http://www.cptech.org/blogs/drugdevelopment/2006/07/bayh-dole-
>>>>>> rights-size-of-clinical.html
>>>>>>
>>>>>>
>>>>>> Bayh Dole Rights, Size of Clinical Trials, 2004 Approvals
>>>>>>
>>>>>> July 10, 2006
>>>>>> Mike Palmedo
>>>>>>
>>>>>> CPTech has looked at the patents for New Molecular Entities
>>>>>> (NMEs)
>>>>>> that
>>>>>> came to the market in 2004.
>>>>>>
>>>>>> Excluding antibiotics, which do not have Bayh-Dole listings in
>>>>>> their
>>>>>> patents, the FDA approved 19 NMEs in 2004. Nine of these received
>>>>>> priority status for approval, meaning they were found to have
>>>>>> significant therapeutic gain over existing medicines. The
>>>>>> remaining 10
>>>>>> NMEs were given standard approvals.
>>>>>>
>>>>>> For the products for which data was available, I looked up the
>>>>>> number of
>>>>>> patients cited by the FDA in approving the medicines. (I had to
>>>>>> exclude
>>>>>> three of the priority NMEs approved in 2004 for which the label
>>>>>> did not
>>>>>> include the number of patients) The average (mean) number of
>>>>>> patients in
>>>>>> the clinical trials on which the FDA approvals were based was
>>>>>> 1073
>>>>>> for
>>>>>> priority drugs and 1840 for standard drugs. The median numbers of
>>>>>> patients in these clinical trials were 1290 for the priority
>>>>>> drugs
>>>>>> and
>>>>>> 2058 for the standard drugs. These figures are considerably lower
>>>>>> than
>>>>>> the average size of clinical trials used by DiMasi in his often-
>>>>>> cited
>>>>>> research on the cost of drug development =96 5,303 patients.
>>>>>>
>>>>>> The Orange Book lists 45 patents on the 19 NMEs. Three of these
>>>>>> patents
>>>>>> include clauses citing government funding and subsequent Bayh-
>>>>>> Dole
>>>>>> rights to use or license the patent. These three patents cover
>>>>>> two of
>>>>>> the nine drugs which received priority approval =96 two patents for
>>>>>> Clolar
>>>>>> (a leukemia drug sold by Genzyme) and one for Lyrica (a diabetes
>>>>>> drug
>>>>>> sold by Pfizer).
>>>>>>
>>>>>> A spreadsheet with the drugs, patents, and size of trials is
>>>>>> online here:
>>>>>> http://www.cptech.org/ip/health/rnd/2004nmes-07102006.xls
>>>>>>
>>>>>> [Posted by Mike Palmedo to Drug Development (with access) at
>>>>>> 7/10/2006
>>>>>> 03:59:00 PM]
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Ip-health mailing list
>>>>>> Ip-health@lists.essential.org
>>>>>> http://lists.essential.org/mailman/listinfo/ip-health
>>>>>>
>>>>>
>>>>> --
>>>>> -----------------------------------------------
>>>>> Joseph A. DiMasi, Ph.D.
>>>>> Director of Economic Analysis
>>>>> Tufts Center for the Study of Drug Development
>>>>> Tufts University
>>>>> 192 South Street, Suite 550
>>>>> Boston, MA 02111
>>>>> tel: 617-636-2116; fax: 617-636-2425
>>>>> URL: http://csdd.tufts.edu
>>>>> -----------------------------------------------
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Ip-health mailing list
>>>>> Ip-health@lists.essential.org
>>>>> http://lists.essential.org/mailman/listinfo/ip-health
>>>>>
>>>>
>>>> ---------------------------------
>>>> James Love, CPTech / www.cptech.org /
>>>> mailto:james.love@cptech.org /
>>>> tel. +1.202.332.2670 / mobile +1.202.361.3040
>>>>
>>>> "If everyone thinks the same: No one thinks." Bill Walton
>>>>
>>>>
>>>> _______________________________________________
>>>> Ip-health mailing list
>>>> Ip-health@lists.essential.org
>>>> http://lists.essential.org/mailman/listinfo/ip-health
>>>>
>>>
>>> --
>>> -----------------------------------------------
>>> Joseph A. DiMasi, Ph.D.
>>> Director of Economic Analysis
>>> Tufts Center for the Study of Drug Development
>>> Tufts University
>>> 192 South Street, Suite 550
>>> Boston, MA 02111
>>> tel: 617-636-2116; fax: 617-636-2425
>>> URL: http://csdd.tufts.edu
>>> -----------------------------------------------
>>>
>>> --
>>>
>>> _______________________________________________
>>> Ip-health mailing list
>>> Ip-health@lists.essential.org
>>> http://lists.essential.org/mailman/listinfo/ip-health
>>>
>>
>> ---------------------------------
>> James Love, CPTech / www.cptech.org / mailto:james.love@cptech.org /
>> tel. +1.202.332.2670 / mobile +1.202.361.3040
>>
>> "If everyone thinks the same: No one thinks." Bill Walton
>>
>>
>> _______________________________________________
>> Ip-health mailing list
>> Ip-health@lists.essential.org
>> http://lists.essential.org/mailman/listinfo/ip-health
>>
>
> --
> -----------------------------------------------
> Joseph A. DiMasi, Ph.D.
> Director of Economic Analysis
> Tufts Center for the Study of Drug Development
> Tufts University
> 192 South Street, Suite 550
> Boston, MA 02111
> tel: 617-636-2116; fax: 617-636-2425
> URL: http://csdd.tufts.edu
> -----------------------------------------------
>
>
> _______________________________________________
> Ip-health mailing list
> Ip-health@lists.essential.org
> http://lists.essential.org/mailman/listinfo/ip-health
>
---------------------------------
James Love, CPTech / www.cptech.org / mailto:james.love@cptech.org /
tel. +1.202.332.2670 / mobile +1.202.361.3040
"If everyone thinks the same: No one thinks." Bill Walton