Re: InfoWorld Reply

[mj@creative.net]

  Mrs . Boderick and Morrey. Thank you for your response. I too was concerned
  about how this benchmark was performed.  While it is certainly legitimate
  to test Java across multiple hardware platforms, as well as multiple OS's,
  the concern for me was how OS results *on the same hardware platform* -
  namely Pentium - were skewed by the fact that the test was run at different
  hardware gradations (P180 with 64M RAM vs. P166 with 48M). There is
  unlikely to be an issue of portability between these two systems, but
  performing the test in this way has clearly prejudiced the results in terms
  of performance, in a manner not clearly discussed or justified in the
  article.  I would also like to point out that the discussion of the results
  in the article largely focused on performance, so I can only conclude that
  performance was a relevant concern.
  
  Martin Gruber
  
  >---------- Forwarded message ----------
  >Date: Fri, 19 Dec 1997 10:54:38 -0800
  >From: John Broderick <John_Broderick@infoworld.com>
  >To: zoro@tc.fluke.com
  >Subject: Your letter
  >
  >
  >Mark,
  >
  > We realize that by testing the OSes on differing platforms we have left
  >ourselves open to criticism but felt it was worth it as the main focus of
  >our tests was Java's portability. The performance numbers are really
  >intended to give an indication of the overall usability of Java on varied
  >platform configurations.  But you have given us fuel for a future article!
  >Thanks for the heads up on the incorrect entry on our benchmark chart -
  >2:09 is indeed the correct number.
  >
  >John Broderick and Brad Morrey
  >
  >You wrote:
  >
  >Regarding the benchmark at:
  >http://www.infoworld.com/cgi-bin/displayTC.pl?/970915sb7-bench1.htm
  >The entry for Netscape4.0 + Win3.1 whose total time was reported as
  >36:43 should
  >read 2:09 (time I get when adding the other columns in that row)
  >Why compare software on such vastly different platforms? Benchmarks on
  >software must be run on the identical hardware to have clear meaning.
  >Likewise,
  >if you wish to compare hardware you must run identical software.
  >Even so, why is the list NOT sorted by outcome?
  >This would have put Linux right next to NT.
  >I assume you're are intelligent enough to note that the NT Machine is
  >about 60-70%
  >faster than the Linux Machine. Had the linux test been done on
  >equivalent Hardware,
  >Linux would have been about 28 seconds, making it the clear winner all
  >around.
  >To me the big story here is that Linux out performs all the other
  >comercial OS's
  >This was an interesting benchmark, but it should be redone across
  >similar Software
  >and then Hardware platforms and sorted by outcome.
  >Mark Hinds