Regarding calculation of free memory

Do you have a question? Post it now! No Registration Necessary

Translate This Thread From English to

Threaded View
Gurus,
I was just wondering what could be the best possible way to calculate
free memory while our code is running in SDRAM.I have used vxworks and
it uses a approach to calculate the free memory as follows:
Fill up the entire stack with 0xeeeee and then when you start using the
stack,if at all a particular address in memory is used,the value of
0xeeeee in that location should be overwritten with real data.So if we
are able to track the first occurance of the pattern 0xeeeee we should
be able to subtract it from the entire size of memory and make a fair
assesment of the available free memory.

How ever I beleive such a algorithm has drawbacks.First drawback which
I can think is
1)What is the guarentee that the real data is not ur pattern chosen(In
vxworks case,its 0xeeeee,if 0xeeeee itself is real data,we cant find
out real size!)?

2)What should you look for to identify unique pattern?Or other way how
would you arrive at the pattern word to be used?

3)Are there any other good techniques for calculating free memory?

4)We are making software for a consumer electronics appliance.It uses a
premitive propreitory RTOS which does not give enough tools to find out
free memory.We are in a forced situation to reduce memory
consumption,so we have to get a way to calculate free available memory
first.To add to the complexity,our application uses
heap,partitions,partition with in partitions.At any point of time we
have to give the user memory utilised like most RTOS tools
provide.Since Heap is dynamic how do u arrive at a fair judgment?
If we wanted to get total free space,we need to calculate free space in
heap,free space in partition and then arrive at it.

Are any one aware of any good way to achieve it in the given case of
RTOS vendor not providing tools for memory calculations?

Looking farward for all your replys and Advanced thanks for the same.

Regards,
s.subbarayan


Re: Regarding calculation of free memory
Quoted text here. Click to load it

It sounds like your 0xeeeee method you outline is perfectly suited for
your purposes.  Even if 0xeeeee is actually part of your real data that
happens to be in the stack, consider that it's highly unlikely that
0xeeeee will fill your stack to the extent that your stack usage figure
is way off.  The point is, even if 0xeeeee is real stack data, the
information you get from hunting for unused stack will be close enough.
 Get your usage total and add some margin for safety and you're done.

Quoted text here. Click to load it

That's application specific.  Look at your operating scenarios.  Ask
yourself if the pattern you've selected is likely to be real data or
not.

Quoted text here. Click to load it

See #1 reply.

Quoted text here. Click to load it

Don't know but...  This method is commonly used and usually works very
well.  A good white-box test designed to stress the stack usage will
give you the real info you need to accurately size your RAM.

Quoted text here. Click to load it

IMO, it's like gambling when you size how much stack margin you want to
add.  If you believe your SW tests are "NASA grade" (nearly
exhaustive), you will not need to add much stack margin.  If you don't
do SW testing, it's a roll of the dice.  You probably fit in the middle
of the two extremes.

Quoted text here. Click to load it

Good way is the way you outlined for run-time stack checking.  Maybe
some CPUs feature some sort of built-in high water mark stack pointer
register.  That would be nice but I don't know if it exists.

JJS


Re: Regarding calculation of free memory

Quoted text here. Click to load it

<SNIP>

I believe the traditional value is "0xDEADBEEF" :)


--

John Devereux

Re: Regarding calculation of free memory

Quoted text here. Click to load it

Which has the same problem.  But really, if you pick a pattern that's
unlikely to turn up by coincidence, you'll come close enough to the
right answer for all practical purposes.  Don't use, say, 00000000,
but beyond that it doesn't much matter.
--
Joseph J. Pfeiffer, Jr., Ph.D.       Phone -- (505) 646-1605
Department of Computer Science       FAX   -- (505) 646-1002
We've slightly trimmed the long signature. Click to see the full one.
Re: Regarding calculation of free memory
Quoted text here. Click to load it

There is also a slight advantage to using this value because it is odd. On
many processors using an odd address for larger than byte size data will
thow an exception. As a result uninitialised data on the stack used as
pointers can be caught more often.

The bigest problem I have though is that I am a vegetarian so I prefer
0xFACEFEED.

Peter



Re: Regarding calculation of free memory
Quoted text here. Click to load it


Seen in various 16 bit codes:
#define STACKPAT 0x55aa        // Stack fill value for high water mark
checking.

--

  ... Hank

http://home.earthlink.net/~horedson
We've slightly trimmed the long signature. Click to see the full one.
Re: Regarding calculation of free memory
Quoted text here. Click to load it

<snip>

Quoted text here. Click to load it

<pedant_mode=ON>

If you want your appliance to be totally reliable, avoid using a heap at
all. Free memory calculation then becomes rather simple [1]. Logic: heaps
are (usually) non-deterministic and *will* cause eventual malloc failures
due to fragmentation [2].

[1] Excluding stack use, but that's not too hard either.
[2] Unless you allocate all memory at startup; a common trick. But then the
linker could do the same...

<pedant_mode=OFF>

Steve
http://www.fivetrees.com



Re: Regarding calculation of free memory
Quoted text here. Click to load it

< extreme_pedant_mode=ON >

Not so.  In Fortran 77, it was possible to allocate all memory statically,
but it isn't generally possible.  It can't be done in Fortran 90 or C.

Also, of the memory management schemes that are lumped under the term of
'heap' ones, there are several that have deterministic behaviour and do
not suffer from fragmentation.  The simplest one is if all allocations
are of the same size, and chains are used instead of arrays.

The whole area of memory management schemes is sadly neglected, and
much of garbage collection work is based on unrealistic assumptions,
so much of what is currently believed isn't so.

< extreme_pedant_mode=OFF >

Quoted text here. Click to load it


Regards,
Nick Maclaren.

Re: Regarding calculation of free memory
Quoted text here. Click to load it

<even_more_extreme_pedant_mode=ON>

It *can* be done in C - by avoiding malloc ;).

<even_more_extreme_pedant_mode=OFF>

Quoted text here. Click to load it

Agreed. I've used techniques such as these (effectively an application-level
memory manager) in certain situations where extreme reliability was
required, but the platform insisted on the use of its memory manager...
<sigh>...

Quoted text here. Click to load it

Completely agreed. I've often fantasized about a hardware memory manager
that remaps blocks in such a way that fragmentation cannot occur. OTOH, I've
also fantasized that one day, a majority will write good code that doesn't
leak memory and traps malloc errors properly...

Steve
http://www.fivetrees.com



Re: Regarding calculation of free memory
Quoted text here. Click to load it

I will skip the mode nesting, as things are getting ridiculous :-)

Problem one:  in C, there is no memory management beyond stack scoped
except by using malloc.  Any algorithm that requires more than that
can't be done if you don't use malloc.

Problem two:  there are many library facilities that use malloc either
explicitly or implicitly - including almost all I/O - you would have
to avoid them, too.


Regards,
Nick Maclaren.

Re: Regarding calculation of free memory
Quoted text here. Click to load it

I'm posting this from c.a.e - embedded (or OS-less) is my main area. One
tends not to need such I/O functions therein.

You're right, though - it certainly pays to be aware of the platform
requirements of any such library function.

Steve
http://www.fivetrees.com



Re: Regarding calculation of free memory
Quoted text here. Click to load it

And still less the internationalisation and wide character
facilities :-)

My experience of that area is that the worst problem is finding out
when a compiler has chosen to implement some basic facility (e.g.
a mathematical function) in a way that drags in completely
unrelated junk from the library.  But I have been usually trying
to used hosted compilers in an embedded fashion - I would expect
compilers designed for embedded use to be better.


Regards,
Nick Maclaren.

Re: Regarding calculation of free memory
Quoted text here. Click to load it

sbrk()?

Quoted text here. Click to load it

Provide your own malloc() so the I/O that needs it will get it
from your own memory pool. Since you are doing the I/O,
you can provide optimal buffering.

--

  ... Hank

http://home.earthlink.net/~horedson
We've slightly trimmed the long signature. Click to see the full one.
Re: Regarding calculation of free memory
Quoted text here. Click to load it

Not in C - in fact, I can't think of any standard that it IS in,
because it makes such extremely horrible assumptions about the
implementation.

Quoted text here. Click to load it

No.  Replacing malloc is not supported in C, and often breaks
the nastier vendor and third-party libraries.  The reason is that
the suite of malloc-related functions is not independent, and they
often use non-standard interfaces.

Yes, I have done it, over many years - but it isn't something that
I recommend to users.  Or can :-(


Regards,
Nick Maclaren.

Re: Regarding calculation of free memory
Quoted text here. Click to load it

There are still systems where you can reasonably do so - shipping
more than one possible malloc to use is a good hint for example.

Quoted text here. Click to load it

--
    Sander

+++ Out of cheese error +++

Re: Regarding calculation of free memory

|> >
|> > No.  Replacing malloc is not supported in C, and often breaks
|> > the nastier vendor and third-party libraries.  The reason is that
|> > the suite of malloc-related functions is not independent, and they
|> > often use non-standard interfaces.
|>
|> There are still systems where you can reasonably do so - shipping
|> more than one possible malloc to use is a good hint for example.

It's a hint that the VENDOR can do it - but not that the application
programmer can.  Yes, an experienced hacker can, which was the basis
of my remark that I can, but I can't advise users to.

Also, are you aware of how often third-party libraries demand a
particular one of those mallocs, either in their release notes or
because they fall over with others?


Regards,
Nick Maclaren.

Re: Regarding calculation of free memory

snipped-for-privacy@cus.cam.ac.uk (Nick Maclaren) writes:
Quoted text here. Click to load it

Ahhh, now I know why I've never had any problems replacing malloc on Linux,
Solaris or the Amiga's OS. I'm a VENDOR! (or maybe not?)

--
David Gay
snipped-for-privacy@acm.org

Re: Regarding calculation of free memory

|>
|> > |> > No.  Replacing malloc is not supported in C, and often breaks
|> > |> > the nastier vendor and third-party libraries.  The reason is that
|> > |> > the suite of malloc-related functions is not independent, and they
|> > |> > often use non-standard interfaces.
|> > |>
|> > |> There are still systems where you can reasonably do so - shipping
|> > |> more than one possible malloc to use is a good hint for example.
|> >
|> > It's a hint that the VENDOR can do it - but not that the application
|> > programmer can.  Yes, an experienced hacker can, which was the basis
|> > of my remark that I can, but I can't advise users to.
|>
|> Ahhh, now I know why I've never had any problems replacing malloc on Linux,
|> Solaris or the Amiga's OS. I'm a VENDOR! (or maybe not?)

Or perhaps you just have a slightly narrower range of experience
than I do?

I have seen failures on half a dozen Unices, including Solaris
and Linux.  Of course, as I said, it also depends on what other
software you are using in your application.


Regards,
Nick Maclaren.

Re: Regarding calculation of free memory

snipped-for-privacy@cus.cam.ac.uk (Nick Maclaren) writes:

Quoted text here. Click to load it

Well, frankly, for the Linux case, you can go and look at the source code
to check if there's any extra magic entry points or other dependencies. For
the others, it definitely pays to check linker output files to make sure
you didn't, e.g., accidentally end up with two copies of malloc. Yes,
you'll have to repeat such checks if you change libc version - that's a
price you have to pay. And if you're going to say "but I've run into
problems", I'd like to get a specific description, rather than some
nebulous "It happened to me somewhere, sometime, somehow".

--
David Gay
snipped-for-privacy@acm.org

Re: Regarding calculation of free memory
Quoted text here. Click to load it

As I said, this is (usually) within the ability of an experienced hacker.
Do you SERIOUSLY think that it is reasonable for me to tell a Fortran
programmer to find out what version of malloc a system is running
(when he did not install it) and to inspect both the source code of
that malloc and its replacement?

As far as Solaris etc. goes, you first have to find out which version
(or versions) of malloc the application is using, INCLUDING those
called by dynamic libraries.  That is unusually hard under Solaris,
because ldd doesn't do what you apparently thinks that it does, and
a fair amount of reverse engineering of the behaviour of the linker
and loader is needed.  Then you have to extract the relevant symbols,
see which library is using what, and decide if there is a clash.

Oh, and for non-standard symbols, you may have to reverse engineer
the binary to see what they do.


Regards,
Nick Maclaren.

Site Timeline