[Am-info] Why does NT fail to release allocated memory?

[Lewis A. Mettler]

Eric,

Eric Lee Green wrote:
> 
> On Sat, 08 Jan 2000, Lewis A. Mettler wrote:
> > This is not necessarily the case.  However, if NT does not release all
> > memory back to the available pool until program termination (regardless
> > if free() is called or not), NT is poorly designed and can no longer be
> > used.
> 
> Note that Unix displays the exact same behavior. The "free()" call in most libc
> implementations merely puts the freed block of memory into an internal "free"
> list for use by future malloc() calls rather than return it to the operating
> system. This is due to efficiency -- most programs use a fairly constant amount
> of memory, and constantly calling the kernel to handle routine malloc/free
> would be ridiculously expensive.

Maybe it is just the inability of NT to use the virtual memory as it
should.

However, the application (I wrote) that presents the problem is a
program the memory requirements of which may very quite a bit.  It
processes specific data requests that might come in over the internet
and is designed to run 24 hours a day.  During peak usage key portions
of the rather large data base are kept in memory to optimize
processing.  As the demand slacks off the updated data base is returned
to disc.  Even with disc cache the normal access procedures for standard
data bases offers insufficient performance.  Therefore the in memory
data expands and contracts significantly.  The only way to maximize
performance is to bring portions of the data base into memory lists,
process the data and allow it remain in memory until such time as the
delay return to disc makes more sense.

> 
> I would also question how you are measuring allocated memory. Note that many
> operating systems will use memory as disk caches. Thus the amount of free
> memory rarely is the same after program exit as it was before the program was
> executed. This is a common question in the Linux world by Linux newbies -- "I
> ran a program, and now 'free' shows that all my memory is being used!".  In
> addition, lingering sockets and other such internal kernel issues can result in
> memory being used temporarily.

The problem begins to surface as the allocated memory as reported by the
NT monitor approaches the real RAM.  Why NT fails to use virtual memory
is unclear.  I assume that perhaps NT can not use virtual memory for the
allocated memory requirements.  Maybe it is limited to "program use"?  I
am not sure.  But, I have experienced failure when the allocated memory
as reported by the NT Performance monitor approaches real memory.

My guess is that there are two defects with NT (4 and 3.51).  One if the
failure to release memory and the other is the failure to use virtual
memory.

> 
> This is not to say that NT doesn't leak memory -- my brother has a program that
> on NT 4 SP2 will consistently leak memory out of the network stack, for
> example. It's just saying that the behavior you're complaining about is normal
> behavior for most operating systems, and is unrelated to the leaking problem.

If the leak is caused by bad logic that is one thing.  If the OS fails
to recover released memory that is another problem.  That is a defective
OS which prevents any application from remaining in use for any reliable
period of time.

In my own application, I have it self terminate and restart after an
appropriate period of time.  Obviously this is not a program with any
user interface.  Having a program close itself out and restart would
hardly be an acceptable solution.  In my case, since it is essentially a
batch process, it gets away with it.  However, the other application
that also causes NT to fail when RAM is about used up (128 MB) is a user
operated program.

I suspect the defect in NT is causing both programs to be of limited
value.

The suggestion you made that freed memory is reusable by the same
application appears not to be the case.  Memory demands appear to
increase until NT fails.  

-- 
Lewis A. Mettler, Esq.(Attorney and Software Developer)
lmettler@LAMLaw.com
http://www.lamlaw.com/ (detailed review of the Microsoft antitrust
trial)