free - The Memory is still not returned back to OS

If free does not return the memory to OS then look at the implementatio of the free and ask the free,s developper.

"free" can be implemented in many ways, depending on the OS (if any) and the library. It is not uncommon for there to be a "cache" of heap between the application level and the operating system. Thus when the application asks for more memory (i.e., calls malloc() or a related library function), it may get memory from this cache without needing an OS call. If this cache does not have enough spare memory, the library function will ask the OS for more. When freeing memory, the application calls free() which returns the memory to the cache - the library functions will only return the memory to the OS when an entire block is unused (and it may not return it even then). Such a strategy can reduce fragmentation in the main memory, it can improve data locality, and it can allow the main OS memory manager and the library (or application-level memory manager) to have different allocation schemes.

It is also quite possible to have free() do nothing - that's a valid strategy for embedded systems which might use malloc() for convenience during initialisation, but do not allocate and free memory during the application itself.

Once we know what OS the O/P is talking about, and some more information about the particular usage, it might be possible to help.

I came across an info that claimed that free does not return the memory to the OS immediately and it returns at the end of the prorcess/ application. In the internet i found that some Memory Management techniques allow the immediate return of the freed memory to the OS and others allow only at the end of the application / process . So, i started to look for the reasons in the internet . I got some answers , but they were not very clear / conclusive, so i posted my query here.

Thx, Karthik Balaguru

That depends on the OS, and the library used by the application - again, you'll get little useful information until you tell us what OS and tools you are using.

Both are perfectly valid techniques, but are appropriate for different sorts of systems. It is the library used by the application that decides which method is used.

Your posts are not clear either - we don't know what tools you are using, or where you have found this information on the web. We also don't know what you have tried for yourself. Have you looked at the documentation and the source for your application, your library, and your OS?

s

on

d e y

ion

uce

n

This is a generic query. It is not with respect to any particular OS / Tool. I am just seeking for clarification of some info that i came across.

Karthik Balaguru

A typical example would be an OS with virtual memory. If malloc does not have a sufficiently free space in the cache, it will ask the OS for a number virtual memory pages. The amount of virtual memory must be a multiple of the page size (4 KiB for x86).

When free() is called, it will return the memory into the cache, but this is not usually returned to the OS, until the program terminates. One reason for this is that while most small allocations within a virtual memory page have been freed by free(), there may still be other active allocations on that virtual memory page, preventing any attempts to return the page to the OS.

Paul

The local heaps without deallocation is a fairly common trick supported by many compilers and OS environments. It is fast and it simplifies the memory management immensely. When you don't need the dynamic memory any longer, you just kill the whole heap. Alternatively, the heap will be released automatically when the task exits.

Vladimir Vassilevsky DSP and Mixed Signal Design Consultant

While this seems pretty off topic, the reason has nothing to do with the OS.

"free" is a C runtime library function, not an OS function. Most C runtime libraries use "heaps" or private blocks of memory they manage. When you free memory from the heap, the runtime keeps that memory around in many cases to re-use for the next malloc call. This is a performance issue.

The following is the C standard description of free. Note the phrase "made available for further allocation". It says nothing about the OS, so that "further allocation" may be limited to the process that executed the free function.

If you want to see the code for an actual malloc/free application, you can read nmalloc.zip, which is intended for DJGPP, but will probably function unchanged on most Linux, Dos, or Windows environments. See:

7.20.3.2 The free function Synopsis [#1] #include void free(void *ptr);

Description

[#2] The free function causes the space pointed to by ptr to be deallocated, that is, made available for further allocation. If ptr is a null pointer, no action occurs. Otherwise, if the argument does not match a pointer earlier returned by the calloc, malloc, or realloc function, or if the space has been deallocated by a call to free or realloc, the behavior is undefined.

Returns

[#3] The free function returns no value.

--
 [mail]: Chuck F (cbfalconer at maineline dot net) 
 [page]: 
            Try the download section.

ther

Thx for that info. Yes, that clarifies a lot. :):)

I got some more interesting discussions regarding 'free' in comp.lang.c and comp.std.c.

free :-

2f1/8510e07d4d24034d

How do I know if free() really free()ed? :-

Function: free :-

"available for further allocation"? :-

d7c14?hl=3Den

Karthik Balaguru

urther

s.

55...

d330f9

Freeing memory - will it be available immediately

0959

Thx, Karthik Balaguru

... snip ...

Probably. No guarantee.

--
 [mail]: Chuck F (cbfalconer at maineline dot net) 
 [page]: 
            Try the download section.

I don't have the original post anymore. So I answer here. Apparent from the reaction of the OP this is not the way the question is to be understood. It is a question asked from all free developers, and not about a particular free implementation.

It could be rephrased: What are the criteria for free developers to return the memory to the OS, or not?

A typical Unix application will allocate from a pool, and ask memory from the OS only if the pool runs out. On freeing, the memory is returned to the pool. The main reason is that everything involving the OS is more costly than a simple memory allocation that stays within a process. On a virtual memory OS there are hardly disadvantages to having memory allocated that is not used. In the worst case it is swapped out and is no longer a drain on the RAM memory resource. At the end of a process, the memory is returned anyway.

Sometimes an OS is lightweight and all allocations and free's directly involve the OS (e.g. in Forth if that is not too lightweigt to be called an OS).

Some care has to be taken for processes that run indefinitely. If they keep on allocating memory and don't free it (memory leak), memory runs out, wherever it is allocated from. This is a totally separate issue.

Groetjes Albert.

--

--
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- like all pyramid schemes -- ultimately falters.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

Is there any open source tool that will tell the amount of " memory freed but not returned back to the OS " ?

Thx in advans, Karthik Balaguru

n m

I understand that the memory freed depends on the type of memory management technique, then, we should be having a corresponding tool to clearly state the amount memory freed but still with the application/process and not yet returned back to the OS. But, Do normal tools take care of this or Do they convey the info based on the amount of memory freed (That is , they do not worry about the amount of memory returned to the OS and display only the memory freed) ? How about this in linux ?

Thx in advans, Karthik Balaguru

In a demand-paged virtual-memory system, malloc() and friends allocate address space, not memory directly. The address space is separate for each process, so that different processes will be using same addresses for different data.

The physical memory is connected with the address space when an access is attempted. The connection is performed in units called pages. Common page size is 4 kilobytes.

In Linux, you'll find the page maps of each process in the /proc file system. There is a directory for each process with the process ID as the directory name. The memory mapping is in the directory under name maps.

For details, get a book on Linux kernel, e.g.

Understanding the Linux Kernel, by Daniel P. Bovet and Marco Cesati.

--

Tauno Voipio
tauno voipio (at) iki fi

If you're using glibc version of malloc/free, check out the mallinfo() function. It returns a structure containing the current state of memory allocation.

On Linux, you may also use sbrk(0) to find out the extend of the program's data space.

All those functions are also available in the malldbg module for nmalloc, for DJGPP (and most Linux systems). nmalloc avoids various problems. See nmalloc.zip, at:

--
 [mail]: Chuck F (cbfalconer at maineline dot net) 
 [page]: 
            Try the download section.