10MB Chunks of anonymous memory for each thread created from shared object

Do you have a question? Post it now! No Registration Necessary

Translate This Thread From English to

Threaded View
googlegroups Contact options for registered users 's Avatar (by Gravatar) posted on
May 31, 2007, 8:43 pm
rate this thread
Reply to this post Reply Tag this thread to save it in your account Save Printer-friendly version of this post Print
I'm having a problem when creating pthreads from a shared object
library.  Every pthread that I spawn is accompanied by a 10MB chunk of
anonymous memory.

I manually set the stack size to 17k, and verified that it provides a
seg fault when I use more than I allocate.  So I think I can rule-out
the stack.

But here's the perplexing part:  If link the same code into an
executable, the 10MB chunks become 12KB.

When I link the offending code into a shared object, it doesn't matter
if I use dlopen or I dynamically link to it with g++, these chunks are
10MB.

Does anyone have any how I can find out what these anonymous memory
pages are? Once I know what they are I can figure out how to reduce
the size.

Thanks.


I'm running on:
Linux version 2.6.9-5.ELsmp ( snipped-for-privacy@decompose.build.redhat.com)
(gcc version 3.4.3 20041212 (Red Hat 3.4.3-9.EL4)) #1 SMP Wed Jan 5
19:30:39 EST 2005

This is currently running on a pentium4 box so I have tools like pmap
available. But our embedded system is an arm9.

The following is the snippet from pmap for 12 threads spawed from the
shared object:
08048000      8K r-x--  /home/brent/MAP/bin/FredInstantiatorDynamic
0804a000      4K rw---  /home/brent/MAP/bin/FredInstantiatorDynamic
091f8000    132K rw---    [ anon ]
b07d7000      4K -----    [ anon ]
b07d8000  10240K rw---    [ anon ]
b11d8000      4K -----    [ anon ]
b11d9000  10240K rw---    [ anon ]
b1bd9000      4K -----    [ anon ]
b1bda000  10240K rw---    [ anon ]
b25da000      4K -----    [ anon ]
b25db000  10240K rw---    [ anon ]
b2fdb000      4K -----    [ anon ]
b2fdc000  10240K rw---    [ anon ]
b39dc000      4K -----    [ anon ]
b39dd000  10240K rw---    [ anon ]
b43dd000      4K -----    [ anon ]
b43de000  10240K rw---    [ anon ]
b4dde000      4K -----    [ anon ]
b4ddf000  10240K rw---    [ anon ]
b57df000      4K -----    [ anon ]
b57e0000  10240K rw---    [ anon ]
b61e0000      4K -----    [ anon ]
b61e1000  10240K rw---    [ anon ]
b6be1000      4K -----    [ anon ]
b6be2000  10240K rw---    [ anon ]
b75e2000      4K -----    [ anon ]
b75e3000  10256K rw---    [ anon ]
b7ffe000      8K rw---    [ anon ]
bfe73000   1588K rw---    [ stack ]
ffffe000      4K -----    [ anon ]


When I run it from an executable, it becomes:
08048000     64K r-x--  /home/brent/MAP/bin/FredInstantiatorStatic
08058000      4K rw---  /home/brent/MAP/bin/FredInstantiatorStatic
084fd000    132K rw---    [ anon ]
b7fc7000      4K -----    [ anon ]
b7fc8000     12K rw---    [ anon ]
b7fcb000      4K -----    [ anon ]
b7fcc000     12K rw---    [ anon ]
b7fcf000      4K -----    [ anon ]
b7fd0000     12K rw---    [ anon ]
b7fd3000      4K -----    [ anon ]
b7fd4000     12K rw---    [ anon ]
b7fd7000      4K -----    [ anon ]
b7fd8000     12K rw---    [ anon ]
b7fdb000      4K -----    [ anon ]
b7fdc000     12K rw---    [ anon ]
b7fdf000      4K -----    [ anon ]
b7fe0000     32K rw---    [ anon ]
b7feb000      4K -----    [ anon ]
b7fec000     12K rw---    [ anon ]
b7fef000      4K -----    [ anon ]
b7ff0000     12K rw---    [ anon ]
b7ff3000      4K -----    [ anon ]
b7ff4000     12K rw---    [ anon ]
b7ff7000      4K -----    [ anon ]
b7ff8000     12K rw---    [ anon ]
b7ffb000      4K -----    [ anon ]
b7ffc000     16K rw---    [ anon ]
bff7f000    516K rw---    [ stack ]
ffffe000      4K -----    [ anon ]


Other Related Threads
Paul Pluzhnikov Contact options for registered users 's Avatar (by Gravatar) posted on
June 1, 2007, 12:14 am
Permalink
Reply to this post Reply Tag this thread to save it in your account Save Printer-friendly version of this post Print
Re: 10MB Chunks of anonymous memory for each thread created from shared object
snipped-for-privacy@brentandamanda.net writes:

Quoted text here. Click to load it

Run the program under gdb, set breakpoint on __mmap and __mmap64,
when hit type 'where' ?

I bet they are still coming from thread stack allocation,
probably due to a bug in libpthread ...

Cheers,
--
In order to understand recursion you must first understand recursion.
Remove /-nsp/ for email.

Other Related Threads
googlegroups Contact options for registered users 's Avatar (by Gravatar) posted on
June 1, 2007, 3:02 pm
Permalink
Reply to this post Reply Tag this thread to save it in your account Save Printer-friendly version of this post Print
Re: 10MB Chunks of anonymous memory for each thread created from shared object
Quoted text here. Click to load it

You are correct. It was the stack.

Quoted text here. Click to load it

It could definitely be in libpthread, but it could also be the g++
linker. The behavior was different whether our code that created the
threads was in a shared object or an executable.  But both times we
linked to the same libpthread

Thanks for the help


Paul Pluzhnikov Contact options for registered users 's Avatar (by Gravatar) posted on
June 2, 2007, 4:28 am
Permalink
Reply to this post Reply Tag this thread to save it in your account Save Printer-friendly version of this post Print
Re: 10MB Chunks of anonymous memory for each thread created from shared object
snipped-for-privacy@brentandamanda.net writes:

Quoted text here. Click to load it

There is no such thing as 'g++ linker'; it uses the system GNU linker.

I have reproduced the problem you observed, and it turns out to be
a bug in the way you build your shared library (or a rather obscure
interaction between your link line and symbol versioning).

First the result:

    $ gcc -shared -fPIC -o create.so create.c   # [1]
    $ gcc -pthread main.c ./create.so
    $ ./a.out
    stack size: 163840
    &p = 0xb7fee460
    in region [0xb75ef000, 0xb7ff1000) (10493952)

Above, I asked for stack size of 160K (10*PTHREAD_STACK_MIN),
but got 10MB (just like you). I am guessing that's exactly what
you did. The problem is that command [1] is incorrect.

Correct command:

    $ gcc -pthread -shared -fPIC -o create.so create.c
          ^^^^^^^^
    $ ./a.out
    stack size: 163840
    &p = 0xb7fee460
    in region [0xb7fc8000, 0xb7ff1000) (167936)

produces expected result.

The reason:

    $ gcc -shared -fPIC -o create.so create.c &&
      objdump -T ./create.so | grep pthread_create
    00000000      D  *UND*  00000000              pthread_create
    $ gcc -pthread -shared -fPIC -o create.so create.c &&
      objdump -T ./create.so | grep pthread_create
    00000000      DF *UND*  0000018c  GLIBC_2.1   pthread_create

When create.so is built correctly, it uses pthread_create@GLIBC_2.1.
When it isn't, it uses pthread_create@GLIBC_2.0:

    $ gcc -shared -fPIC -o create.so create.c
    $ LD_DEBUG=bindings ./a.out 2>&1 | grep pthread_create
         18884:     binding file ./create.so to /lib/tls/libpthread.so.0: normal
symbol `pthread_create'

    $ gcc -pthread -shared -fPIC -o create.so create.c
    $ LD_DEBUG=bindings ./a.out 2>&1 | grep pthread_create
         18981:     binding file ./create.so to /lib/tls/libpthread.so.0: normal
symbol `pthread_create' [GLIBC_2.1]

Finally, here is the code I used:

$ cat main.c
int main() { create(); return 0; }

$ cat create.c
#include <pthread.h>
#include <limits.h>
#include <stdio.h>

void *fn(void *p)
{
    char cmd[1024];
    FILE *fp;
    fprintf(stderr, "&p = %p\n", &p);
    sprintf(cmd, "cat /proc/%d/maps", getpid());
    fp = popen(cmd, "r");
    while (fgets(cmd, sizeof(cmd), fp)) {
        void *low, *hi;
        int n = sscanf(cmd, "%x-%x", &low, &hi);
        if (n) {
            if (low < (void*)&p && (void*)&p < hi) {
                fprintf(stderr, "in region [%p, %p) (%d)\n", low, hi, (char*)hi
- (char*)low);
                fclose(fp);
                break;
            }
        }
    }
    return p;
}

void create()
{
    pthread_t tid;
    size_t size = 10*PTHREAD_STACK_MIN;
    pthread_attr_t attr;
    fprintf(stderr, "stack size: %d\n", size);

    pthread_attr_init(&attr);
    pthread_attr_setstacksize(&attr, size);
    pthread_create(&tid, &attr, fn, NULL);
    pthread_join(tid, NULL);
}


Cheers,
--
In order to understand recursion you must first understand recursion.
Remove /-nsp/ for email.

Site Timeline