[Ip-health] Size of trials by status (S or P) - Some 2004 FDA
and Parexcel data compared
Joseph DiMasi
joseph.dimasi@tufts.edu
Wed Aug 2 07:13:05 2006
Jamie,
I'll make this short. You have set up a bunch of straw men and did not
adequately address my fundamental critique. I was not critical of FDA
data or labels, I did not in any way suggest that the Clinical Studies
section or any other section of the label is meaningless, I did not
suggest that looking at the nature of clinical trial work was not
useful, and I did not in any way suggest that costs do not vary (both
below and above average). What I did question was a measure that YOU,
not the FDA, constructed and interpreted. Let me put my concern this
way. You seem to be suggesting that what you have is a consistent
measure of clinical trial sizes for what is "important" clinical trial
work (which I would interpret as development that is at least designed
to generate useful scientific information). Leaving aside how we should
actually define in practice what is important and what is not, if we
want to examine this then what we should be interested in is the total
level, if you will, of "usefulness" associated with the clinical
development effort. Clearly, by any reasonable reckoning, the Clinical
Studies section leaves out reporting of n's on a lot of work that is to
a variable and unknown extent useful to some degree (and reporting all
of the relevant n's is not an objective of the section). So, the n's
that come out of this section cannot give you a measure of trial size
associated with total "usefulness."
Joe DiMasi
James Love wrote:
> Joe, since we are beating a horse to death, allow me to respond to
> your comments.
>
> On Jul 31, 2006, at 6:28 PM, Joseph DiMasi wrote:
>
>>> These are not "CPTech numbers." They come from the FDA. The
>>> numbers are not "estimates," but rather the numbers reported by
>>> the FDA as the evidence the drug is safe and effective (assuming
>>> Michael reads the FDA letters correctly).
>>
>>
>> Who said anything about CPTech generating their own numbers? It
>> is abundantly clear from my earlier messages that we have been
>> discussing what is in FDA-approved product labels.
>
>
> Thank you for that.
>
>> picking out n's from the Clinical Studies section and representing
>> or implying that they mean something in particular IS something that
>> you own, not the FDA, since the FDA has not presented and
>> interpreted these numbers
>
>
> Yes, we do look at the numbers in the clinical studies section,as
> well as all the other data that is available. Why do you think the
> FDA reports some trials and not others? Is this a totally random
> meaningless act by the FDA?
>
>> (nor would they, I believe, accept your interpretations).
>
>
> Ok. What is the interpretation of the FDA reporting clinical
> trials on the label? Are they tying to make some type of
> statement? Isn't it that the FDA is saying" these are the
> important studies that establish the drug is safe (or unsafe) and
> effective?
>
>>
>>> Relative size of trials: As you know (but did not really address
>>> below), Michael was asked to look at the "relative" size of trials
>>> between standard and priority drug approvals. We are interested
>>> in this issue for the following reason. We want to know if
>>> company R&D investments are directed at priority or non- priority
>>> medicines. Relatively smaller trials would suggest (all other
>>> things being equal) investments in clinical trials for priority
>>> products may be smaller than investments in trials for "standard"
>>> drugs.
>>
>>
>> I have already addressed issues related to the relative numbers of
>> subjects in priority and standard approvals in a number of e-mails
>> in this string. You have now added the qualifier "all other things
>> being equal."
>
>
> Joe, this is not only not new, but it was the exact phrase used
> when I posted this originally.
>
> http://lists.essential.org/pipermail/ip-health/2006-July/009853.html
> http://www.cptech.org/blogs/drugdevelopment/2006/07/size-of-trials-by-
> status-s-or-p-some.html
> "If priority drugs have smaller trials and quicker approvals, they
> would seem to be less expensive, all other things being equal. Joe's
> comments have been informative and constructive, and we will revisit
> the issue, incorporating both a broader analysis of the Parexel data,
> and a closer look at the differences between the FDA and Parexel
> data, as well as other evidence on this topic."
>
>
>> This makes the statement true if total trial sizes are as you say,
>> but the problem is that all other things are not, in general,
>> equal. Given that nearly all orphans get priority ratings and many
>> drugs with fast-track status get priority ratings, you might well
>> expect that pre-approval trial sizes are lower (at least for the
>> indications at issue).
>
>
> Doesn't fast-tract status suggest the drug is more important?
>
>> The fast-track drugs also come with post-approval commitments, which
>> will add to trial sizes (when and if they are done).
>
>
> I agree that post-approval commitments are important. Do you
> agree that they are less risky than pre-approval investments?
>
>> However, this says nothing about what was actually done in these
>> trials, infrastructure costs (which will move the numbers toward
>> equalization), work on other indications, and discovery costs and
>> risks in novel areas. Total costs are also going to depend on the
>> therapeutic class distributions and it is all confounded by the fact
>> that the bulk of development in classes is done more or less
>> contemporaneously and often it is the first drug to win this race
>> that gets a priority rating, while the losers tend to be given
>> standard ratings at approval. You cannot infer relative total costs
>> from clinical trial sizes alone in the aggregate, let alone for any
>> particular drug.
>
>
> You left out a few items that are relevant for orphans and
> priority products -- including government funded support for R&D,
> including trials. The NCI supported trials for 50 of 77 cancer
> agents approved from 1950s to 1996, for example.
>
> You don't want to suggest that nothing ever has below average
> costs do you? Why it is fine for Pfizer to point out the large
> number of patients in trials to impress everyone with the cost, but
> so un-cool for us to notice when products are approved with smaller
> trials? Can we only tout evidence that costs are higher, and never
> lower? I'm serious. Ok. You can claim, the trials are small, but
> really expensive anyway, and why. That would move things forward, if
> there is some data to support the argument. Indeed, I think the
> issue of economies of scale in trials is pretty interesting and
> important, particularly in light of some recent coverage of the costs
> of large trials.
>
>>
>>> --------------Michael Palmedo quote----------
>>> "There are two parts of the FDA labels that typically include
>>> patients in clinical trials - the Clinical Trials section and the
>>> Adverse Reactions section. At one point I tried to incorporate the
>>> adverse reactions section, but found that this approach was
>>> problematic as well. For starters, it does not include control
>>> groups, so it too gives incomplete information. It often gives a
>>> higher number than the Clinical Trials section, but not always, and
>>> it sometimes gives a lower number. Sometime it gives estimates
>>> instead of numbers "Approximately 2,000" or "more than 1500." So if
>>> you take the highest number off the label for one drug, and then
>>> compare it to the highest number on other labels for other drugs, you
>>> are comparing:
>>> - full numbers of patients from a specific subset clinical trials
>>> - the number of patients in an unknown number of trials (maybe all
>>> the trials, maybe not) minus the control groups
>>> - vague estimates"
>>> --------------end quote-------------
>>
>>
>> Mike Palmedo is wrong that the Adverse Reactions sections never
>> include information on control groups.
>
>
> Nice addition of the word "never"
>
>> The example that I gave below for Enablex proves that. Information
>> was given on the numbers of subjects in controlled and uncontrolled
>> phase II and III trials. The other example, Lyrica, shows that this
>> is not always the case. This and what Mike wrote just makes my
>> point that you cannot consistently get total clinical trial sizes
>> from the FDA labels.
>>
>
> What you can get is consistent data on what the FDA reports in
> the clinical trials section of the label.
>
> You seem to be very critical of the FDA data. What about your
> secret data that PhRMA members give you? That is not representative
> of anything in particular, is it? It was way different from the
> PERI survey in terms of costs. You have said earlier that it under-
> reported some types of drug development projects. You don't seem shy
> in sharing it with the world.
>
>
>>> Michael said he wanted to "keep the comparisons apple-to-apple"
>>> using what seemed like the most important data, the trials cited as
>>> the main basis for the FDA approval.
>>
>>
>> For there to be an appropriate and useful apples-to-apples
>> comparison, the n's in the Clinical Studies section have to
>> consistently measure the same thing and what they measure must be
>> meaningful. The only thing that is consistent here is that the n's
>> that you pick out were all from the Clinical Studies section. There
>> is no requirement that the FDA report n's for all "relevant" or
>> "important" clinical trials in this section. The intent seems to be
>> to discuss trials that illustrate what the efficacy effects are for
>> the drug. You certainly cannot interpret the handful or less of
>> phase III trials discussed in the Clinical Studies section as all
>> that was "important" in the drug approval process. Are phase I
>> trials not important (and while individual studies may be small, in
>> aggregate they can be substantial [see the Enablex example below])?
>> Are phase II trials unimportant? Are phase I-III trials for other
>> indications not important? Simply counting the trial sizes for
>> those trials mentioned in the Clinical Studies section is a deeply
>> flawed measure of importance in either absolute or relative terms
>> (because of inconsistency in what is measured and variability in
>> what is left out that is also "important").
>>
>>> You are correct, if Michael presents this data, he (we) should
>>> always mention that these data under-report the total number of
>>> patients,
>>> and be clear that the data is only that reported in the Clinical
>>> Trials section of the FDA approval. With this explanation and
>>> context, it is interesting to compare the size (apple to apple) of
>>> trials for priority and non-priority products.
>>
>>
>> I am glad that in the future you would always mention that these
>> numbers understate total clinical trial sizes and that they come
>> from one section of the labels. However, I would also hope that you
>> refrain from using these numbers in inappropriate apples-to- oranges
>> comparisons with other numbers. The original posting from Mike
>> quite clearly made a comparison between these Clinical Studies
>> section n's and an estimate of average total clinical trial sizes
>> from our cost study.
>>
>
> Yes, it seems you don't want anyone to look at this data, for any
> reason. And I notice you pretty much ignored the mountains of
> evidence that some company investments in clinical trials are not
> about science, but about marketing. Including, for example, the
> things I posted earlier (see below, reposted). If you think about
> what those articles say, you understand more why we find the FDA
> clinical trials data interesting and relevant.
>
> Jamie
>
>
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D
> From: Evidence Regarding Research and Development Investments in
> Innovative and Non-Innovative Medicines, September 2003. (footnotes
> renumbered)
>
> 5.1 Pharmacoeconomic Analysis in Clinical Research
>
> When commercial interests perform research, the research program is
> influenced in many ways, including areas where marketing objectives
> influence the choice and design of studies. There exists
> considerable confusion and some controversy over the actual cost of
> performing clinical trials; but there is general agreement that
> spending on clinical trials has increased sharply in the past fifteen
> years. In their study of 1996-1998 new drug approvals, Tufts
> researchers Kaitin and Healy described the underlying reasons for the
> increases in the costs for pre-approval trials as follows:
>
> ------
> Advances in scientific knowledge, the greater complexity of new
> products, an increase in the number of pharmacoeconomic studies done
> in the pre-market phase, and industry inefficiency are contributing
> to an escalation in the size and complexity of clinical trials, and
> an increase in research and development costs.[4]
> ---------
>
> While the first two factors are generally considered positive
> developments, there is considerable controversy over the benefits of
> designing trials to address pharmacoecononic and marketing issues.
> Fred Hassan, then CEO of Pharmacia and PhRMA chair, told McKinsey
> Quarterly,"You=92re also dealing with demands for very expensive
> outcome trials from various regulators as well as from managed-care
> administrators.[5] In a brochure, Demonstrating Product Potential
> for Competitive Advantage,[6] consultant Abt Associates describes the
> relationship between marketing and science as follows:
>
> ----------Abt brochure---
> We understand that the transition from clinical development to
> marketplace introduction is critical to the success of any
> pharmaceutical, biotechnology, or medical device product. Our
> expertise can enhance your plans for research before, during, and
> after market launch. Our projects =97 which range from small-scale
> retrospective studies to large-scale prospective registries =97 can be
> used to generate greater understanding of the use and value of a new
> drug, biologic, or device. This understanding in turn supports its
> use by physicians and managed care organizations (MCOs) as well as
> appropriate coverage of reimbursement decisions by payors =97 both
> public and private. Abt Associates' skill and experience in post-
> marketing evaluations can help sponsors meet the needs of their
> marketing and sales teams. Since the 1970s, Abt Associates has
> conducted hundreds of prospective observational studies and
> longitudinal surveys to determine the value of healthcare products
> and services across a wide range of therapeutic areas. We will work
> with you to achieve the scientific credibility needed to maximize
> your product exposure.
> ---------------
>
> An undetermined amount of investment in clinical trials, both before
> and after product approval, is related to marketing issues.[7]
> Consider these quotes from a Wall Street Journal article on clinical
> trials:[8]
>
> ------WSJ story-----
> . . .increasingly, marketing executives are joining research teams
> from the start, surveying doctors and consumers and tailoring trials
> to win optimal "positioning" in the marketplace. Many of the biggest
> trials come not in difficult-to-treat diseases such as cancer, as one
> might expect, but in well-established therapeutic areas, such as
> antibiotics or blood-pressure drugs, where enormous testing programs
> are needed to ferret out small advantages over existing drugs that
> can then be highlighted in marketing campaigns.
>
> "The FDA told us that we don't need all these trials" to get
> omapatrilat approved for blood pressure, says Hubert Pouleur, Bristol-
> Myers's vice president for cardiovascular clinical research. "But
> there is a difference between getting a drug approved and having it
> be a commercial success. A new drug will be used only if it is a
> significant improvement on existing drugs, and to establish that you
> need trials that aren't required for approval."
>
> Postmarketing studies, as trials for drugs already on the market are
> called, "are billowing out of control," says Eve Slater, Merck's
> senior vice president for clinical testing. She decries "a total lack
> of science" in some studies. But drug marketers contend they are
> helpless to stop the one-upmanship. If a rival mounts a new study
> aimed at backing up a sales-expanding marketing claim, "you have to
> do it, too, or you are dead in the water," she says.
> ---------------
>
> Seeding Studies
>
> A particularly controversial mixture between research and marketing
> are so-called "seeding studies," which are designed to promote the
> use of medicines. [9]
>
> Traditionally, once a product has been launched the company wants to
> gain as much market share as possible to gain return on investment.
> One way of achieving the involvement of prescribers and patients is
> the use of 'seeding' studies. In such trials, the data is often
> collected by sales reps rather than Contract Research Organizations
> (CRAs) and patient recruitment is often thinly spread over a large
> number of sites.[10] Seeding studies are generally held in very low
> esteem, particularly if the company attempts to masquerade them as
> Phase IV studies. Some of the characteristics of seeding studies are
> listed below:
>
> Characteristics of Seeding Studies
> Lack of control group
>
> Inadequate statistical power
>
> Involvement of a large number of study sites, each recruiting only a
> few patients
>
> Inappropriate use of sales representatives
>
> Vague safety aim, irrelevant or inappropriate outcomes
>
> Short-term studies with a drug intended for long-term use
>
> Paid for directly by the company's marketing department
>
> One interesting analysis of the role of seeding studies in marketing
> was recently published in The Netherlands by the Health Care
> Inspectorate.[11] The 25-page report details a wide range of areas
> where research and marketing are blended together, often apparently
> in violation of Dutch and European regulations on ethical marketing
> practices. Of particular interest here are some of the discussions
> of the practice of using clinical trials as marketing practices.
>
> -------Netherlands analysis----
> Expenditure on combined scientific and marketing studies are
> sometimes included under expenditure for research and development and
> clinical studies and not in the marketing plans.. . . All studies
> (including Phase IV studies) must meet the criteria set out in the
> Medical Research Involving Human Subject Act. These tests must be
> checked by a medical ethics committee. Exposing people to non-
> scientific studies using medicinal products is unethical.
> Findings
>
> In some cases the marketing plans refer to designations which fall
> under the phase IV study, such as value-added projects, post
> marketing surveillance (PMS), seeding trials, phase IV study,
> clinical trial and study. . . The objectives of the phase IV
> studies described in the plans show that influencing prescriptions
> for the product being promoted and building up relationships with the
> doctor are mentioned in 48 of the 71 surveys (68%). There are no
> specific study objectives in the remaining 23 surveys. In addition
> to money, incentives in kind offered to doctors including
> sphygmomanometers, hand-held computers etc. are mentioned... The
> designations in the studies suggest that they could be described as
> scientific studies. The fact that no specific study objectives were
> mentioned in some of the studies leads us to assume that the
> execution of the study is not a prime objective. The question is how
> the medical-ethics committees have interpreted their tasks in these
> cases. This is all the more so in those cases in which the marketing
> plans have explicitly mentioned influencing prescriptions as an
> objective. Presenting these forms of influencing as research can be
> seen as socially unacceptable and unethical. It undermines the
> public=92s trust in healthcare. The articles in the Medicinal Products
> Advertising Decree are not legally geared towards tackling this.
> ----------------
>
> Another section of the Dutch Health Care Inspectorate report
> describes other research-related activity that have marketing
> objectives.
>
> -------
> Investigations by the Inspectorate have revealed that about 50% of
> refresher courses for GPs are sponsored and/or organised by the
> pharmaceutical industry. The pharmaceutical industry often
> determines who is invited, thus creating restricted access for the
> profession as a whole. Because the topics are generally determined by
> the pharmaceutical industry, the course programmes are more likely to
> be supply-oriented rather than demand-oriented. The question is
> whether the needs of public health are served by this situation.
> About NLG 18,000,000, or about 11% of the total [marketing] budget,
> is spent on promotional meetings which are often not accredited.
> These meetings are made attractive to the target group by linking
> them to attractive locations and/or events. The objectives mentioned
> in the plans include persuading doctors that the medicinal product in
> question is the best, increasing the number of patients treated, and
> encouraging or influencing doctors to participate in phase IV studies.
> -------
>
> [4] Kaitin Healy, page 2.
> [5] Catherine George and J. Michael Pearson, "Riding the pharma
> roller coaster," The McKinsey Quarterly, 2002 Number 4.
> [6] http://www.abtassoc.com/Page.cfm?PageID=3D6300, Demonstrating
> Product Potential for Competitive Advantage.
> [7] For recent examples of studies designed to achieve marketing
> purposes, see: http://www.biospace.com/news_category.cfm?
> CategoryID=3D25&SR=3D1
> [8] Robert Langreth, "Drug Marketing Drives Many Clinical Trials,"
> Wall Street Journal, November 16, 1998.
> [9] La Puma J. Physician rewards for postmarketing surveillance
> (seeding studies) in the US. Pharmacoeconomics 1995;7(3):187-90.
> Steven Piantadosi, Clinical Trials: A Methodologic Perspective, John
> Wiley and Sons, New York, 1997.
> [10] http://www.pmlive.com/archive.cfm?
> &ArticleID=3D175&back=3D-1&print=3D1, Clinical Trials: Part Two: Marketin=
g
> and Clinical Trials.
> [11] Health Care Inspectorate, Marketing plans for medicinal products
> available on prescription only: the current situation, The Hague,
> July 2001, second revised edition.
>
>
>
--
-----------------------------------------------
Joseph A. DiMasi, Ph.D.
Director of Economic Analysis
Tufts Center for the Study of Drug Development
Tufts University
192 South Street, Suite 550
Boston, MA 02111
tel: 617-636-2116; fax: 617-636-2425
URL: http://csdd.tufts.edu
-----------------------------------------------