10MB Chunks of anonymous memory for each thread created from shared object

Run the program under gdb, set breakpoint on __mmap and __mmap64, when hit type 'where' ?

I bet they are still coming from thread stack allocation, probably due to a bug in libpthread ...

Cheers,

You are correct. It was the stack.

It could definitely be in libpthread, but it could also be the g++ linker. The behavior was different whether our code that created the threads was in a shared object or an executable. But both times we linked to the same libpthread

Thanks for the help

There is no such thing as 'g++ linker'; it uses the system GNU linker.

I have reproduced the problem you observed, and it turns out to be a bug in the way you build your shared library (or a rather obscure interaction between your link line and symbol versioning).

First the result:

$ gcc -shared -fPIC -o create.so create.c # [1] $ gcc -pthread main.c ./create.so $ ./a.out stack size: 163840 &p = 0xb7fee460 in region [0xb75ef000, 0xb7ff1000) (10493952)

Above, I asked for stack size of 160K (10*PTHREAD_STACK_MIN), but got 10MB (just like you). I am guessing that's exactly what you did. The problem is that command [1] is incorrect.

Correct command:

$ gcc -pthread -shared -fPIC -o create.so create.c ^^^^^^^^ $ ./a.out stack size: 163840 &p = 0xb7fee460 in region [0xb7fc8000, 0xb7ff1000) (167936)

produces expected result.

The reason:

$ gcc -shared -fPIC -o create.so create.c && objdump -T ./create.so | grep pthread_create 00000000 D *UND* 00000000 pthread_create $ gcc -pthread -shared -fPIC -o create.so create.c && objdump -T ./create.so | grep pthread_create 00000000 DF *UND* 0000018c GLIBC_2.1 pthread_create

When create.so is built correctly, it uses pthread_create@GLIBC_2.1. When it isn't, it uses pthread_create@GLIBC_2.0:

$ gcc -shared -fPIC -o create.so create.c $ LD_DEBUG=bindings ./a.out 2>&1 | grep pthread_create 18884: binding file ./create.so to /lib/tls/libpthread.so.0: normal symbol `pthread_create'

$ gcc -pthread -shared -fPIC -o create.so create.c $ LD_DEBUG=bindings ./a.out 2>&1 | grep pthread_create 18981: binding file ./create.so to /lib/tls/libpthread.so.0: normal symbol `pthread_create' [GLIBC_2.1]

Finally, here is the code I used:

$ cat main.c int main() { create(); return 0; }

$ cat create.c #include #include #include

void *fn(void *p) { char cmd[1024]; FILE *fp; fprintf(stderr, "&p = %p\n", &p); sprintf(cmd, "cat /proc/%d/maps", getpid()); fp = popen(cmd, "r"); while (fgets(cmd, sizeof(cmd), fp)) { void *low, *hi; int n = sscanf(cmd, "%x-%x", &low, &hi); if (n) { if (low < (void*)&p && (void*)&p < hi) { fprintf(stderr, "in region [%p, %p) (%d)\n", low, hi, (char*)hi

- (char*)low); fclose(fp); break; } } } return p; }

void create() { pthread_t tid; size_t size = 10*PTHREAD_STACK_MIN; pthread_attr_t attr; fprintf(stderr, "stack size: %d\n", size);

pthread_attr_init(&attr); pthread_attr_setstacksize(&attr, size); pthread_create(&tid, &attr, fn, NULL); pthread_join(tid, NULL); }

Cheers,