[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: InfoWorld Reply
Mrs . Boderick and Morrey. Thank you for your response. I too was concerned
about how this benchmark was performed. While it is certainly legitimate
to test Java across multiple hardware platforms, as well as multiple OS's,
the concern for me was how OS results *on the same hardware platform* -
namely Pentium - were skewed by the fact that the test was run at different
hardware gradations (P180 with 64M RAM vs. P166 with 48M). There is
unlikely to be an issue of portability between these two systems, but
performing the test in this way has clearly prejudiced the results in terms
of performance, in a manner not clearly discussed or justified in the
article. I would also like to point out that the discussion of the results
in the article largely focused on performance, so I can only conclude that
performance was a relevant concern.
>---------- Forwarded message ----------
>Date: Fri, 19 Dec 1997 10:54:38 -0800
>From: John Broderick <John_Broderick@infoworld.com>
>Subject: Your letter
> We realize that by testing the OSes on differing platforms we have left
>ourselves open to criticism but felt it was worth it as the main focus of
>our tests was Java's portability. The performance numbers are really
>intended to give an indication of the overall usability of Java on varied
>platform configurations. But you have given us fuel for a future article!
>Thanks for the heads up on the incorrect entry on our benchmark chart -
>2:09 is indeed the correct number.
>John Broderick and Brad Morrey
>Regarding the benchmark at:
>The entry for Netscape4.0 + Win3.1 whose total time was reported as
>read 2:09 (time I get when adding the other columns in that row)
>Why compare software on such vastly different platforms? Benchmarks on
>software must be run on the identical hardware to have clear meaning.
>if you wish to compare hardware you must run identical software.
>Even so, why is the list NOT sorted by outcome?
>This would have put Linux right next to NT.
>I assume you're are intelligent enough to note that the NT Machine is
>faster than the Linux Machine. Had the linux test been done on
>Linux would have been about 28 seconds, making it the clear winner all
>To me the big story here is that Linux out performs all the other
>This was an interesting benchmark, but it should be redone across
>and then Hardware platforms and sorted by outcome.