[Ip-health] Size of trials by status (S or P) - Some 2004 FDA and Parexcel data compared
James Love
james.love@cptech.org
Wed Aug 2 11:04:02 2006
On Aug 1, 2006, at 4:30 PM, Joseph DiMasi wrote:
> Jamie,
>
> I'll make this short. You have set up a bunch of straw men and did
> not adequately address my fundamental critique. I was not critical
> of FDA data or labels, I did not in any way suggest that the
> Clinical Studies section or any other section of the label is
> meaningless, I did not suggest that looking at the nature of
> clinical trial work was not useful, and I did not in any way
> suggest that costs do not vary (both below and above average).
> What I did question was a measure that YOU, not the FDA,
> constructed and interpreted. Let me put my concern this way. You
> seem to be suggesting that what you have is a consistent measure
> of clinical trial sizes for what is "important" clinical trial
> work (which I would interpret as development that is at least
> designed to generate useful scientific information). Leaving aside
> how we should actually define in practice what is important and
> what is not, if we want to examine this then what we should be
> interested in is the total level, if you will, of "usefulness"
> associated with the clinical development effort. Clearly, by any
> reasonable reckoning, the Clinical Studies section leaves out
> reporting of n's on a lot of work that is to a variable and unknown
> extent useful to some degree (and reporting all of the relevant n's
> is not an objective of the section). So, the n's that come out of
> this section cannot give you a measure of trial size associated
> with total "usefulness."
>
> Joe DiMasi
>
I don't think I have set up any straw men. And I also don't think
we have even said very much about what conclusions one might reach
from the data, outside of a few things. The fact that the FDA,
rather than us, chooses which trials to report on the Clinical Trials
Section of the label, makes our life easier, and it is also more
objective. This involves every drug the FDA approves, rather than a
sample that you or I (or Parexel), chooses. The fact that the size
of the trials the FDA reports (on the label for clinical trials), is
consistently and considerably smaller for the priority NME approvals
than for standard NME approvals is a fact. This is the fact that
Michael has reported on a couple of occasions. Let's start with that.
Does this fact mean anything? You suggest that it means absolutely
nothing. I think it is interesting, and I suggest that one thing it
means is that the size of trials for priority drugs is, as measured
by means and medians, smaller, for trials the FDA thinks have enough
scientific importance to mention (on this part of the label). I also
think one might want to compare the number of patients (and trials)
reported by the FDA in this section of the label with the larger
number of trials that are done on a drug candidate. Part of the
motivation for this is to better understand the claims, by many
people, including big pharma executives, that some of the investment
in clinical trials have nothing to do with science. I don't see why
this issue is irrelevant to this question. I am including quotes
from BMS and Merck on this topic. (see below). I don't think they
are out to get the pharmaceutical industry. They are just explaining
what many people are saying (marketing considerations drive the size
of clinical trials), and we are trying to better understand.
One interesting comparison might be to look at the number of patients
that are in all trials, compared to the number of patients cited by
the FDA. I have just received the Parexel data (which is a partial
sample of FDA NME approvals, selected by Parexel), for 1998 to
2004. What I was thinking we would do is the following. For every
drug in the Parexel survey, report the number of patients in clinical
trials reported by Parexel, the number of patients in the trials the
FDA reports in the section of the label on clinical trials, and
whether drugs were rated priority or standard, or as orphan
products. We haven't done this yet. We could make any additions
to this anyone wants, assuming its not too much work. Maybe this
new and expanded look at the data will have a different outcome, with
regard to the relative sizes of trials. We'll report the data,
whatever it says. Jamie
---- --------
Robert Langreth, "Drug Marketing Drives Many Clinical Trials,"
Wall Street Journal, November 16, 1998.
"The FDA told us that we don't need all these trials" to get
omapatrilat approved for blood pressure, says Hubert Pouleur, Bristol-
Myers's vice president for cardiovascular clinical research. "But
there is a difference between getting a drug approved and having it
be a commercial success. A new drug will be used only if it is a
significant improvement on existing drugs, and to establish that you
need trials that aren't required for approval."
Postmarketing studies, as trials for drugs already on the market are
called, "are billowing out of control," says Eve Slater, Merck's
senior vice president for clinical testing. She decries "a total lack
of science" in some studies. But drug marketers contend they are
helpless to stop the one-upmanship. If a rival mounts a new study
aimed at backing up a sales-expanding marketing claim, "you have to
do it, too, or you are dead in the water," she says.
>
>
> James Love wrote:
>
>> Joe, since we are beating a horse to death, allow me to respond
>> to your comments.
>>
>> On Jul 31, 2006, at 6:28 PM, Joseph DiMasi wrote:
>>
>>>> These are not "CPTech numbers." They come from the FDA. The
>>>> numbers are not "estimates," but rather the numbers reported
>>>> by the FDA as the evidence the drug is safe and effective
>>>> (assuming Michael reads the FDA letters correctly).
>>>
>>>
>>> Who said anything about CPTech generating their own numbers?
>>> It is abundantly clear from my earlier messages that we have
>>> been discussing what is in FDA-approved product labels.
>>
>>
>> Thank you for that.
>>
>>> picking out n's from the Clinical Studies section and
>>> representing or implying that they mean something in particular
>>> IS something that you own, not the FDA, since the FDA has not
>>> presented and interpreted these numbers
>>
>>
>> Yes, we do look at the numbers in the clinical studies
>> section,as well as all the other data that is available. Why do
>> you think the FDA reports some trials and not others? Is this a
>> totally random meaningless act by the FDA?
>>
>>> (nor would they, I believe, accept your interpretations).
>>
>>
>> Ok. What is the interpretation of the FDA reporting clinical
>> trials on the label? Are they tying to make some type of
>> statement? Isn't it that the FDA is saying" these are the
>> important studies that establish the drug is safe (or unsafe) and
>> effective?
>>
>>>
>>>> Relative size of trials: As you know (but did not really
>>>> address below), Michael was asked to look at the "relative"
>>>> size of trials between standard and priority drug approvals.
>>>> We are interested in this issue for the following reason. We
>>>> want to know if company R&D investments are directed at
>>>> priority or non- priority medicines. Relatively smaller trials
>>>> would suggest (all other things being equal) investments in
>>>> clinical trials for priority products may be smaller than
>>>> investments in trials for "standard" drugs.
>>>
>>>
>>> I have already addressed issues related to the relative numbers
>>> of subjects in priority and standard approvals in a number of e-
>>> mails in this string. You have now added the qualifier "all
>>> other things being equal."
>>
>>
>> Joe, this is not only not new, but it was the exact phrase
>> used when I posted this originally.
>>
>> http://lists.essential.org/pipermail/ip-health/2006-July/009853.html
>> http://www.cptech.org/blogs/drugdevelopment/2006/07/size-of-trials-
>> by- status-s-or-p-some.html
>> "If priority drugs have smaller trials and quicker approvals,
>> they would seem to be less expensive, all other things being
>> equal. Joe's comments have been informative and constructive, and
>> we will revisit the issue, incorporating both a broader analysis
>> of the Parexel data, and a closer look at the differences between
>> the FDA and Parexel data, as well as other evidence on this topic."
>>
>>
>>> This makes the statement true if total trial sizes are as you
>>> say, but the problem is that all other things are not, in
>>> general, equal. Given that nearly all orphans get priority
>>> ratings and many drugs with fast-track status get priority
>>> ratings, you might well expect that pre-approval trial sizes are
>>> lower (at least for the indications at issue).
>>
>>
>> Doesn't fast-tract status suggest the drug is more important?
>>
>>> The fast-track drugs also come with post-approval commitments,
>>> which will add to trial sizes (when and if they are done).
>>
>>
>> I agree that post-approval commitments are important. Do
>> you agree that they are less risky than pre-approval investments?
>>
>>> However, this says nothing about what was actually done in these
>>> trials, infrastructure costs (which will move the numbers toward
>>> equalization), work on other indications, and discovery costs
>>> and risks in novel areas. Total costs are also going to depend
>>> on the therapeutic class distributions and it is all confounded
>>> by the fact that the bulk of development in classes is done more
>>> or less contemporaneously and often it is the first drug to win
>>> this race that gets a priority rating, while the losers tend to
>>> be given standard ratings at approval. You cannot infer relative
>>> total costs from clinical trial sizes alone in the aggregate,
>>> let alone for any particular drug.
>>
>>
>> You left out a few items that are relevant for orphans and
>> priority products -- including government funded support for R&D,
>> including trials. The NCI supported trials for 50 of 77 cancer
>> agents approved from 1950s to 1996, for example.
>>
>> You don't want to suggest that nothing ever has below average
>> costs do you? Why it is fine for Pfizer to point out the large
>> number of patients in trials to impress everyone with the cost,
>> but so un-cool for us to notice when products are approved with
>> smaller trials? Can we only tout evidence that costs are higher,
>> and never lower? I'm serious. Ok. You can claim, the trials
>> are small, but really expensive anyway, and why. That would move
>> things forward, if there is some data to support the argument.
>> Indeed, I think the issue of economies of scale in trials is
>> pretty interesting and important, particularly in light of some
>> recent coverage of the costs of large trials.
>>
>>>
>>>> --------------Michael Palmedo quote----------
>>>> "There are two parts of the FDA labels that typically include
>>>> patients in clinical trials - the Clinical Trials section and
>>>> the Adverse Reactions section. At one point I tried to
>>>> incorporate the adverse reactions section, but found that this
>>>> approach was problematic as well. For starters, it does not
>>>> include control
>>>> groups, so it too gives incomplete information. It often gives
>>>> a higher number than the Clinical Trials section, but not
>>>> always, and
>>>> it sometimes gives a lower number. Sometime it gives estimates
>>>> instead of numbers "Approximately 2,000" or "more than 1500." So if
>>>> you take the highest number off the label for one drug, and
>>>> then compare it to the highest number on other labels for other
>>>> drugs, you
>>>> are comparing:
>>>> - full numbers of patients from a specific subset clinical trials
>>>> - the number of patients in an unknown number of trials (maybe all
>>>> the trials, maybe not) minus the control groups
>>>> - vague estimates"
>>>> --------------end quote-------------
>>>
>>>
>>> Mike Palmedo is wrong that the Adverse Reactions sections never
>>> include information on control groups.
>>
>>
>> Nice addition of the word "never"
>>
>>> The example that I gave below for Enablex proves that.
>>> Information was given on the numbers of subjects in controlled
>>> and uncontrolled phase II and III trials. The other example,
>>> Lyrica, shows that this is not always the case. This and what
>>> Mike wrote just makes my point that you cannot consistently get
>>> total clinical trial sizes from the FDA labels.
>>>
>>
>> What you can get is consistent data on what the FDA reports
>> in the clinical trials section of the label.
>>
>> You seem to be very critical of the FDA data. What about
>> your secret data that PhRMA members give you? That is not
>> representative of anything in particular, is it? It was way
>> different from the PERI survey in terms of costs. You have said
>> earlier that it under- reported some types of drug development
>> projects. You don't seem shy in sharing it with the world.
>>
>>
>>>> Michael said he wanted to "keep the comparisons apple-to-apple"
>>>> using what seemed like the most important data, the trials
>>>> cited as the main basis for the FDA approval.
>>>
>>>
>>> For there to be an appropriate and useful apples-to-apples
>>> comparison, the n's in the Clinical Studies section have to
>>> consistently measure the same thing and what they measure must
>>> be meaningful. The only thing that is consistent here is that
>>> the n's that you pick out were all from the Clinical Studies
>>> section. There is no requirement that the FDA report n's for
>>> all "relevant" or "important" clinical trials in this section.
>>> The intent seems to be to discuss trials that illustrate what
>>> the efficacy effects are for the drug. You certainly cannot
>>> interpret the handful or less of phase III trials discussed in
>>> the Clinical Studies section as all that was "important" in the
>>> drug approval process. Are phase I trials not important (and
>>> while individual studies may be small, in aggregate they can be
>>> substantial [see the Enablex example below])? Are phase II
>>> trials unimportant? Are phase I-III trials for other
>>> indications not important? Simply counting the trial sizes for
>>> those trials mentioned in the Clinical Studies section is a
>>> deeply flawed measure of importance in either absolute or
>>> relative terms (because of inconsistency in what is measured and
>>> variability in what is left out that is also "important").
>>>
>>>> You are correct, if Michael presents this data, he (we) should
>>>> always mention that these data under-report the total number of
>>>> patients,
>>>> and be clear that the data is only that reported in the
>>>> Clinical Trials section of the FDA approval. With this
>>>> explanation and
>>>> context, it is interesting to compare the size (apple to apple)
>>>> of trials for priority and non-priority products.
>>>
>>>
>>> I am glad that in the future you would always mention that these
>>> numbers understate total clinical trial sizes and that they come
>>> from one section of the labels. However, I would also hope that
>>> you refrain from using these numbers in inappropriate apples-to-
>>> oranges comparisons with other numbers. The original posting
>>> from Mike quite clearly made a comparison between these Clinical
>>> Studies section n's and an estimate of average total clinical
>>> trial sizes from our cost study.
>>>
>>
>> Yes, it seems you don't want anyone to look at this data, for any
>> reason. And I notice you pretty much ignored the mountains of
>> evidence that some company investments in clinical trials are not
>> about science, but about marketing. Including, for example, the
>> things I posted earlier (see below, reposted). If you think
>> about what those articles say, you understand more why we find
>> the FDA clinical trials data interesting and relevant.
>>
>> Jamie
>>
>>
>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> From: Evidence Regarding Research and Development Investments in
>> Innovative and Non-Innovative Medicines, September 2003. (footnotes
>> renumbered)
>>
>> 5.1 Pharmacoeconomic Analysis in Clinical Research
>>
>> When commercial interests perform research, the research program is
>> influenced in many ways, including areas where marketing objectives
>> influence the choice and design of studies. There exists
>> considerable confusion and some controversy over the actual cost of
>> performing clinical trials; but there is general agreement that
>> spending on clinical trials has increased sharply in the past fifteen
>> years. In their study of 1996-1998 new drug approvals, Tufts
>> researchers Kaitin and Healy described the underlying reasons for the
>> increases in the costs for pre-approval trials as follows:
>>
>> ------
>> Advances in scientific knowledge, the greater complexity of new
>> products, an increase in the number of pharmacoeconomic studies done
>> in the pre-market phase, and industry inefficiency are contributing
>> to an escalation in the size and complexity of clinical trials, and
>> an increase in research and development costs.[4]
>> ---------
>>
>> While the first two factors are generally considered positive
>> developments, there is considerable controversy over the benefits of
>> designing trials to address pharmacoecononic and marketing issues.
>> Fred Hassan, then CEO of Pharmacia and PhRMA chair, told McKinsey
>> Quarterly,"You=92re also dealing with demands for very expensive
>> outcome trials from various regulators as well as from managed-care
>> administrators.[5] In a brochure, Demonstrating Product Potential
>> for Competitive Advantage,[6] consultant Abt Associates describes the
>> relationship between marketing and science as follows:
>>
>> ----------Abt brochure---
>> We understand that the transition from clinical development to
>> marketplace introduction is critical to the success of any
>> pharmaceutical, biotechnology, or medical device product. Our
>> expertise can enhance your plans for research before, during, and
>> after market launch. Our projects =97 which range from small-scale
>> retrospective studies to large-scale prospective registries =97 can be
>> used to generate greater understanding of the use and value of a new
>> drug, biologic, or device. This understanding in turn supports its
>> use by physicians and managed care organizations (MCOs) as well as
>> appropriate coverage of reimbursement decisions by payors =97 both
>> public and private. Abt Associates' skill and experience in post-
>> marketing evaluations can help sponsors meet the needs of their
>> marketing and sales teams. Since the 1970s, Abt Associates has
>> conducted hundreds of prospective observational studies and
>> longitudinal surveys to determine the value of healthcare products
>> and services across a wide range of therapeutic areas. We will work
>> with you to achieve the scientific credibility needed to maximize
>> your product exposure.
>> ---------------
>>
>> An undetermined amount of investment in clinical trials, both before
>> and after product approval, is related to marketing issues.[7]
>> Consider these quotes from a Wall Street Journal article on clinical
>> trials:[8]
>>
>> ------WSJ story-----
>> . . .increasingly, marketing executives are joining research teams
>> from the start, surveying doctors and consumers and tailoring trials
>> to win optimal "positioning" in the marketplace. Many of the biggest
>> trials come not in difficult-to-treat diseases such as cancer, as one
>> might expect, but in well-established therapeutic areas, such as
>> antibiotics or blood-pressure drugs, where enormous testing programs
>> are needed to ferret out small advantages over existing drugs that
>> can then be highlighted in marketing campaigns.
>>
>> "The FDA told us that we don't need all these trials" to get
>> omapatrilat approved for blood pressure, says Hubert Pouleur,
>> Bristol-
>> Myers's vice president for cardiovascular clinical research. "But
>> there is a difference between getting a drug approved and having it
>> be a commercial success. A new drug will be used only if it is a
>> significant improvement on existing drugs, and to establish that you
>> need trials that aren't required for approval."
>>
>> Postmarketing studies, as trials for drugs already on the market are
>> called, "are billowing out of control," says Eve Slater, Merck's
>> senior vice president for clinical testing. She decries "a total lack
>> of science" in some studies. But drug marketers contend they are
>> helpless to stop the one-upmanship. If a rival mounts a new study
>> aimed at backing up a sales-expanding marketing claim, "you have to
>> do it, too, or you are dead in the water," she says.
>> ---------------
>>
>> Seeding Studies
>>
>> A particularly controversial mixture between research and marketing
>> are so-called "seeding studies," which are designed to promote the
>> use of medicines. [9]
>>
>> Traditionally, once a product has been launched the company wants to
>> gain as much market share as possible to gain return on investment.
>> One way of achieving the involvement of prescribers and patients is
>> the use of 'seeding' studies. In such trials, the data is often
>> collected by sales reps rather than Contract Research Organizations
>> (CRAs) and patient recruitment is often thinly spread over a large
>> number of sites.[10] Seeding studies are generally held in very low
>> esteem, particularly if the company attempts to masquerade them as
>> Phase IV studies. Some of the characteristics of seeding studies are
>> listed below:
>>
>> Characteristics of Seeding Studies
>> Lack of control group
>>
>> Inadequate statistical power
>>
>> Involvement of a large number of study sites, each recruiting only a
>> few patients
>>
>> Inappropriate use of sales representatives
>>
>> Vague safety aim, irrelevant or inappropriate outcomes
>>
>> Short-term studies with a drug intended for long-term use
>>
>> Paid for directly by the company's marketing department
>>
>> One interesting analysis of the role of seeding studies in marketing
>> was recently published in The Netherlands by the Health Care
>> Inspectorate.[11] The 25-page report details a wide range of areas
>> where research and marketing are blended together, often apparently
>> in violation of Dutch and European regulations on ethical marketing
>> practices. Of particular interest here are some of the discussions
>> of the practice of using clinical trials as marketing practices.
>>
>> -------Netherlands analysis----
>> Expenditure on combined scientific and marketing studies are
>> sometimes included under expenditure for research and development and
>> clinical studies and not in the marketing plans.. . . All studies
>> (including Phase IV studies) must meet the criteria set out in the
>> Medical Research Involving Human Subject Act. These tests must be
>> checked by a medical ethics committee. Exposing people to non-
>> scientific studies using medicinal products is unethical.
>> Findings
>>
>> In some cases the marketing plans refer to designations which fall
>> under the phase IV study, such as value-added projects, post
>> marketing surveillance (PMS), seeding trials, phase IV study,
>> clinical trial and study. . . The objectives of the phase IV
>> studies described in the plans show that influencing prescriptions
>> for the product being promoted and building up relationships with the
>> doctor are mentioned in 48 of the 71 surveys (68%). There are no
>> specific study objectives in the remaining 23 surveys. In addition
>> to money, incentives in kind offered to doctors including
>> sphygmomanometers, hand-held computers etc. are mentioned... The
>> designations in the studies suggest that they could be described as
>> scientific studies. The fact that no specific study objectives were
>> mentioned in some of the studies leads us to assume that the
>> execution of the study is not a prime objective. The question is how
>> the medical-ethics committees have interpreted their tasks in these
>> cases. This is all the more so in those cases in which the marketing
>> plans have explicitly mentioned influencing prescriptions as an
>> objective. Presenting these forms of influencing as research can be
>> seen as socially unacceptable and unethical. It undermines the
>> public=92s trust in healthcare. The articles in the Medicinal Products
>> Advertising Decree are not legally geared towards tackling this.
>> ----------------
>>
>> Another section of the Dutch Health Care Inspectorate report
>> describes other research-related activity that have marketing
>> objectives.
>>
>> -------
>> Investigations by the Inspectorate have revealed that about 50% of
>> refresher courses for GPs are sponsored and/or organised by the
>> pharmaceutical industry. The pharmaceutical industry often
>> determines who is invited, thus creating restricted access for the
>> profession as a whole. Because the topics are generally determined by
>> the pharmaceutical industry, the course programmes are more likely to
>> be supply-oriented rather than demand-oriented. The question is
>> whether the needs of public health are served by this situation.
>> About NLG 18,000,000, or about 11% of the total [marketing] budget,
>> is spent on promotional meetings which are often not accredited.
>> These meetings are made attractive to the target group by linking
>> them to attractive locations and/or events. The objectives mentioned
>> in the plans include persuading doctors that the medicinal product in
>> question is the best, increasing the number of patients treated, and
>> encouraging or influencing doctors to participate in phase IV
>> studies.
>> -------
>>
>> [4] Kaitin Healy, page 2.
>> [5] Catherine George and J. Michael Pearson, "Riding the pharma
>> roller coaster," The McKinsey Quarterly, 2002 Number 4.
>> [6] http://www.abtassoc.com/Page.cfm?PageID=3D6300, Demonstrating
>> Product Potential for Competitive Advantage.
>> [7] For recent examples of studies designed to achieve marketing
>> purposes, see: http://www.biospace.com/news_category.cfm?
>> CategoryID=3D25&SR=3D1
>> [8] Robert Langreth, "Drug Marketing Drives Many Clinical Trials,"
>> Wall Street Journal, November 16, 1998.
>> [9] La Puma J. Physician rewards for postmarketing surveillance
>> (seeding studies) in the US. Pharmacoeconomics 1995;7(3):187-90.
>> Steven Piantadosi, Clinical Trials: A Methodologic Perspective, John
>> Wiley and Sons, New York, 1997.
>> [10] http://www.pmlive.com/archive.cfm?
>> &ArticleID=3D175&back=3D-1&print=3D1, Clinical Trials: Part Two: Marketi=
ng
>> and Clinical Trials.
>> [11] Health Care Inspectorate, Marketing plans for medicinal products
>> available on prescription only: the current situation, The Hague,
>> July 2001, second revised edition.
>>
>>
>>
>
> --
> -----------------------------------------------
> Joseph A. DiMasi, Ph.D.
> Director of Economic Analysis
> Tufts Center for the Study of Drug Development
> Tufts University
> 192 South Street, Suite 550
> Boston, MA 02111
> tel: 617-636-2116; fax: 617-636-2425
> URL: http://csdd.tufts.edu
> -----------------------------------------------
>
>
---------------------------------
James Love, CPTech / www.cptech.org / mailto:james.love@cptech.org /
tel. +1.202.332.2670 / mobile +1.202.361.3040
"If everyone thinks the same: No one thinks." Bill Walton