Regarding calculation of free memory - Page 3

Do you have a question? Post it now! No Registration Necessary

Translate This Thread From English to

Threaded View
Re: Regarding calculation of free memory
On 05 Oct 2005 08:42:46 -0700,


Quoted text here. Click to load it

Are you talking about Linux or Unix in general ?

At least I had to convert some sample client code for a communication
package to K&R so that the primitive compiler intended mainly for
compiling the Unix kernel cold compile it. In these systems in which
the client was intended to be used, the customer did not use these
systems for program development but only for production, so any
"modern" compiler would be missing.

Paul


Re: Regarding calculation of free memory


Quoted text here. Click to load it

I'm talking about the open source available in the late 80s onwards time
frame, so mostly Unix in general rather than Linux in particular. And some
amount was more cross-platform than that (e.g., emacs).

--
David Gay
snipped-for-privacy@acm.org

Re: Regarding calculation of free memory
Quoted text here. Click to load it

So you can in fact in the Solaris case too these days, though that
hasn't really been needed so far AFAICT.

--
    Sander

+++ Out of cheese error +++

Re: Regarding calculation of free memory
Quoted text here. Click to load it


My original comments re sbrk() and malloc() were not about Unix.
And even with various Unices, they still apply.

If your application can be smart about it's use of memory,
that can be a huge win. On any OS. Including Unix flavored ones.

--

  ... Hank

http://home.earthlink.net/~horedson
We've slightly trimmed the long signature. Click to see the full one.
Re: Regarding calculation of free memory


Quoted text here. Click to load it


You can certainly do so in Solaris (where we do ship far too many:
libc malloc, BSD malloc, SysV -lmalloc, mmap -lmapmalloc, -lmtmalloc,
watchmalloc (for debugging) and libumem).

And you can substitute your own as long as you make sure you
implement *all* routines our malloc libraries commonly export
(mallocs often define some of their functions in terms of the others;
but some implement foo() in terms of bar() and others vice versa and
this tend to lead to nasty problems if you don't roll a complete implementation)

Casper
--
Expressed in this posting are my opinions.  They are in no way related
to opinions held by my employer, Sun Microsystems.
We've slightly trimmed the long signature. Click to see the full one.
Re: Regarding calculation of free memory

|>
|> >Ahhh, now I know why I've never had any problems replacing malloc on Linux,
|> >Solaris or the Amiga's OS. I'm a VENDOR! (or maybe not?)
|>
|> You can certainly do so in Solaris (where we do ship far too many:
|> libc malloc, BSD malloc, SysV -lmalloc, mmap -lmapmalloc, -lmtmalloc,
|> watchmalloc (for debugging) and libumem).
|>
|> And you can substitute your own as long as you make sure you
|> implement *all* routines our malloc libraries commonly export
|> (mallocs often define some of their functions in terms of the others;
|> but some implement foo() in terms of bar() and others vice versa and
|> this tend to lead to nasty problems if you don't roll a complete
implementation)


cd /usr/lib
for i in *alloc*.so
do
echo +++ $i +++
nm $i | grep GLOB | grep -v UNDEF | sed 's/^.*|//' | grep -v '^_' | \
sort -u | xargs echo
done

+++ libbsdmalloc.so +++
SUNW_1.1 free malloc realloc
+++ libmalloc.so +++
SUNW_1.1 calloc free malloc realloc
+++ libmapmalloc.so +++
SUNW_0.7 SUNW_1.1 SUNWprivate_1.1 calloc cfree free mallinfo malloc
mallopt memalign realloc valloc
+++ libmtmalloc.so +++
SUNW_1.1 SUNW_1.1.1 SUNWprivate_1.1 calloc free malloc mallocctl
memalign realloc valloc

I can't remember which system it was that had an undocumented call
that I had to reverse engineer from the pseudo-assembler.


Regards,
Nick Maclaren.



Re: Regarding calculation of free memory
Quoted text here. Click to load it

How strange. I use it. In C.

Quoted text here. Click to load it

How strange. I use my own malloc(). In C.

Quoted text here. Click to load it

Why not? Gives you complete control.
You can tune your malloc() to your application.

--

  ... Hank

http://home.earthlink.net/~horedson
We've slightly trimmed the long signature. Click to see the full one.
Re: Regarding calculation of free memory
Quoted text here. Click to load it

Nick's point is that it is not a part of the C standard.
My Linux manpage for sbrk() says to #include <unistd.h>
and under the "CONFORMING TO" heading says this:

        brk  and  sbrk  are  not defined in the C Standard
        and are deliberately excluded from  the  POSIX.1
        standard  (see  paragraphs  B.1.1.1.3  and B.8.3.3).

Of course you *can* call it from C, but you can't expect
it to be there on all systems, especially "bare metal"
embedded systems.

--
Michael N. Moran           (h) 770 516 7918
5009 Old Field Ct.         (c) 678 521 5460
We've slightly trimmed the long signature. Click to see the full one.
Re: Regarding calculation of free memory
Quoted text here. Click to load it


Who cares? When programming, one uses the tools
available. Nitpicking "standard" vs. "POSIX" vs. etc.
is just a waste of time. If we need better standards,
write them. But in any case I was not thinking of
C at all, and particularly not of Unix / Linux in my
original comments. I use the "C hacker" way of talking
about things because there are not a lot of folks who
will understand when I say "Just use ER MCOR$, it
will work the same in FOR and FTN." The point is that
the application can manage memory better than the OS
can, and there is always some way for the application
to get hold of that memory to manage.

Quoted text here. Click to load it

So what?

Quoted text here. Click to load it

The programmer must know her tools. If she is writing a
banking backroom system, she will not be interested in
porting it to a Blackberry, or an access point.

--

  ... Hank

http://home.earthlink.net/~horedson
We've slightly trimmed the long signature. Click to see the full one.
Re: Regarding calculation of free memory

Quoted text here. Click to load it

Though some access points (WRT54G) already have sbrk(), and I'm sure it's
not long until Blackberries have it too ;-)

--
David Gay
snipped-for-privacy@acm.org

Re: Regarding calculation of free memory
Quoted text here. Click to load it


Yes, I would imagine my wristwatch (were I to own such a thing)
might soon implement the whole of the C library and allow me to
run my own code via WiFi or some such ;-)

--

  ... Hank

http://home.earthlink.net/~horedson
We've slightly trimmed the long signature. Click to see the full one.
Re: Regarding calculation of free memory
Quoted text here. Click to load it

If they code to that standard, it is more likely to be someone else's
code run without your permission :-)


Regards,
Nick Maclaren.

Re: Regarding calculation of free memory
Quoted text here. Click to load it

Yes.  And you can't expect it to work, and be used, the same way,
either, which is a more serious problem.  But you and I are talking
about programs where people care about portability and RAS, both
between systems and over time - and the happy hackers don't even
understand the concept.  Some of them will learn, as new versions
of even their favourite system causes their code to break; others
will not learn from experience.


Regards,
Nick Maclaren.

Re: Regarding calculation of free memory
Quoted text here. Click to load it

That is only if people insist on trying to re-use free'd memory in
unsupported ways or otherwise abuse a quite simple interface.

The C malloc() interface is quite simple:  You get memory if the
overlying system lords decide to grant it, and you can do whatever
you want with it within reason (so long as you don't make assumptions
about its underlying byte/word length/order, etc.) but you can't
make any assumptions whatsoever about being able to get it back if
your free() it or about having it be visible in any predictable
way from anyplace outside the code segment that allocated it.
By the way, the system lords may lie to you, just as your bank
is likely to do if you try to withdraw a large amount of cash all
at once, because they already loaned it out to someone else.

Replacing malloc() is quite supported in C, just as is implementing
your own version of sqrt().  That's obviously the point of any reasonable
programming language.  It's a way to convince a stupid CPU to do work
you want done.

You just can't make any underlying assumptions about the implementation
of the vendor function or your ability to interoperate with it.

The bottom line is that the computer does what you tell it to do, not
necessarily what you want.  Shouting at the deaf is seldom effective.

This endless bickering about malloc() never ceases to amaze me.
Programmers simply won't accept that you can't really shove fifty
pounds of shit into a twenty pound sack.  They also can't accept
that someone else may have stolen the booty before they got there,
or even in between the time that they measured the bag and started
shoveling.

It's Chinatown, Jake.



Re: Regarding calculation of free memory
Quoted text here. Click to load it

Well ... there *is* static allocation. Then subystems can their own
private memory allocations.

Quoted text here. Click to load it

And, indeed, in systems that must operate indefinitely, one must
not use libraries with such dependencies because any global general
purpose heap (malloc/free) will fragment unless all blocks are the
same size.

Frequently, the same systems that have such requirements also have
limited memory resources and so using a global malloc with as single
fixed size block is unacceptable because of the wasted memory in
small allocations.

As a side note, its not the malloc that fragments heaps, its the
continuous malloc/free cycle over time. For this reason, systems
that must operate continuously for an indefinite period of time
*can* use malloc/new at initialization ... as long as they don't
use free/delete ...

--
Michael N. Moran           (h) 770 516 7918
5009 Old Field Ct.         (c) 678 521 5460
We've slightly trimmed the long signature. Click to see the full one.
Re: Regarding calculation of free memory
Quoted text here. Click to load it

That is why so many serious real time application do their own
heap management. If the heap usage is simple, programming it yourself
may be not even be hard.

Quoted text here. Click to load it

Yeah, you have to avoid printf c.s. They are mostly in the user interface
though. Most real time applications are split in a real real time part
and the user interface that is not. If you adapt this, you can use
printf freely in the user interface and the problem all but disappears.


Quoted text here. Click to load it



--
--
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- like all pyramid schemes -- ultimately falters.
We've slightly trimmed the long signature. Click to see the full one.
Re: Regarding calculation of free memory
Quoted text here. Click to load it

That's a worthlessly vague statement. As far as I can see, the problem isn't
lack of research; it's lack of application of that research in the most
commonly used systems.

--

Re: Regarding calculation of free memory
Quoted text here. Click to load it

On the contrary - one of the main reasons that it hasn't been applied
is the impracticality of the research.  That was the case in 1965,
and is the case today.  A fairly good rule when you find that real
engineers (and there are some real software engineers, even today,
and used to be more) are ignoring the wonderful results that come
out of 'scientific' research is to ask whether the research is
actually addressing the whole of the problem that the engineers
are faced with.


Regards,
Nick Maclaren.

Re: Regarding calculation of free memory
Quoted text here. Click to load it

Does every research paper on memory management describe something practical? No.
Does the research that has been done on memory management collectively address
the problems that arise in real systems in a practical way, for those engineers
that are prepared to do a bit of searching to find the papers relevant to those
problems? Yes, it does. If you think otherwise, then be specific about the
problems you think are not addressed.

--

Re: Regarding calculation of free memory
Quoted text here. Click to load it

I have, many times.  You ignored the issues every time.


Regards,
Nick Maclaren.

Site Timeline