STL containers and managing memory allocation in embedded systems

Alex:

When you say 'vector v;' actually you are saying: vector v; because of the default second parameter of the vector template definition:

template class vector.....

And the member allocate() of allocator "...Throws the exception bad_alloc if the storage is unavailable. This function uses operator new(size_t)..."

Hope this help.

It's usually beyond critical --- it's inacceptable, period. An embedded system has nobody to complain to if an operation as strictly internal as a memory allocation fails. So it had better not go there at all.

If you can't afford to allow exception handling, you generally can't allow C++ style dynamic memory handling. It's as easy as that.

Primarily by not doing it.

Primarily by not using them.

--
Hans-Bernhard Broeker (broeker@physik.rwth-aachen.de)
Even if all the snow were burnt, ashes would remain.

Seems too radical.

An embedded system may use perfectly dynamic memory as long as it is able to recover from lack of memory or,at least, not enter an unstable state. Some Denial-Of-Sservice may be allowed as long as the whole system continues to work.

The main problem is that *every* allocation must be checked, wheteher with exceptions or with null pointer return.

I am now working on one embedded system, and it *must* use some kind of memory management, so as to balance resources between different parts of the program. As I am compiling with GCC, exception throwing is a definite no-no, becuase it neraly doubles the space used by code.

But everything works fine by using no-throw new operator and checking its result. BTW, I am also making some test on using STL allocators to optimize some parts of the system, spceially the multhreaded C++ kernel.

Zara

[snip]

By the way, is this check correct?

vector v; // Stuff

size_t the_size = v.size(); Foo f; v.push_back (f);

if (v.size() != (the_size + 1)) { cerr

That's a bit overkill - not all embedded systems are 100% mission-critical. I.e. you might have a system logger, a configuration deamon etcetera. Sure, it would be annoying if they failed, but the watchdog can restart them once the critical parts are done using all memory.

Of course, the parts that ARE critical will be designed such that no memory allocation ever fails.

Used to be true for 512 byte, single function embedded systems. Doesn't matter so much for 512 Megabyte systems today, where the critical functions need 200 Mb max.

Remember: even on real-time systems, being faster may be useful, and using dynamically allocated memory is a good way to achieve that (e.g. for caches)

HTH Michiel Salters

The standard behaviour when any memory allocation fails is to throw a std::bad_alloc exception, so if push_back needs to allocate but is unable, you don't get to the code below at all.

if I would like to restrict size of some objs (ex. vector) what can I do? rewrite the allocator? is there any shortcut?

Green Hills' Extended Embedded C++ compiler enables the user to cancel exception handling. In this case, can the code below work? Does anybody have such experience?

[snip]

Alex Vinokur email: alex DOT vinokur AT gmail DOT com

The standard containers do not provide any guarantees for your case. Create and use your own containers that do.

So you need at least two separate dynamic memory areas, one that can overflow, the other that can't.

And what do you realistically do when you detect this condition ? IMHO, it is a bad design if you end up in this situation.

In a single task environment you might even be able to unwind some less important allocations (provided that unwinding does not itself use dynamic memory :-), but how do you expect to do anything useful in a multitasking environment with a shared dynamic memory ?

In practice, usually the only realistic thing to do, when even a small allocation fails is to restart the system, which in several cases is a severe malfunction.

Paul

Sure, as long as your customers don't mind an ocassional unexpected reboot. ;-)

Noone dies, but my company and I gain a less than desirable reputation.

--
Michael N. Moran           (h) 770 516 7918
5009 Old Field Ct.         (c) 678 521 5460
Kennesaw, GA, USA 30144    http://mnmoran.org

"So often times it happens, that we live our lives in chains
  and we never even know we have the key."
The Eagles, "Already Gone"

The Beatles were wrong: 1 & 1 & 1 is 1

The only save way if you disable exceptions is to provide your own allocator and assert.

--
Ian Collins.

Or own allocators for STL containers?

Alex Vinokur email: alex DOT vinokur AT gmail DOT com

Why do you assume a reboot? The failure of a malloc can simply mean postponing that particular operation until later. The important thing is not to assume success. Another option is to perform the mallocs on system initiation. They can handle variable length arrays, for example, which the usual suspects cannot.

--
"If you want to post a followup via groups.google.com, don't use
 the broken "Reply" link at the bottom of the article.  Click on 
 "show options" at the top of the article, then click on the 
 "Reply" at the bottom of the article headers." - Keith Thompson
More details at: 
Also see

This may be realistic option, say, with a fixed size memory pool maintained by the application, in which you can easily implement warning limits at e.g. 90 %, but definitively not with malloc, in which you do not get the warning, until the bad thing happens.

Sure, I use "malloc" in critical applications, but never use "free" and the system runs happily for years without a reboot :-).

If the system fails to start due to lack of memory, it is better that the person who performed the reboot is present and might be able to rectify the problem (e.g. by returning to the old functional configuration) rather than having a fatal error at random time within a few months or years e.g. due to dynamic memory fragmentation.

Paul

That's the point, really. It's not that malloc/free are evil - it's memory fragmentation that's evil, and can cause (or, given time, *will* cause) malloc to fail.

What's needed, of course, is a hardware heap manager which separates logical memory blocks from real memory blocks, and actively disallows fragmentation ;).

Steve

Now you are talking about the quality of the malloc implementation, not its use. Of course you can get in trouble by blindly using features without understanding what goes on. But you usually have several courses of action available:

You can postpone some actions when the needed malloc fails.

You can keep your own track of possibly freeable candidates, do so, and try again.

You can build some sort of memory pool, control its use, and let some background process continuously defrag it. I had such a system running for over three years, until a power failure caused it to reboot.

All these things normally involve giving up some measure of portability. However optimizing memory usage can make a process feasible on the given hardware.

Fragmentation will not necessarily ever cause failure. A decent malloc will put back together any freed blocks, and once all allocated blocks are freed there is NO fragmentation left. However if your code is faulty, and you have memory leaks, that is another tale entirely.

--
"If you want to post a followup via groups.google.com, don't use
 the broken "Reply" link at the bottom of the article.  Click on 
 "show options" at the top of the article, then click on the 
 "Reply" at the bottom of the article headers." - Keith Thompson
More details at: 
Also see

[snip]

Note the quoted statement about a watchdog at the top. IMHO, watchdog == reboot.

Two problems with malloc/free and relatives, is knowing "when" or "if" the memory will become available.

*When* must be solved by polling (yuk.) *If* is unknowable in systems where heap fragmentation is possible.

I hate non-deterministic stuff in my systems.

True for any function that can return an error. In general, it's a "Good Thing" [tm] to limit these types of functions where practical.

Yup. Pre-allocation is good. Only when you use "free" and friends does heap-fragmentation become an issue.

[I'm getting that deja-vu feeling about this thread.]

I'm out-of-sync with respect to "they". I guess you mean malloc at system initiation, to which I say ... yep.

Malloc is a wonderful tool for this type of situation. Obviously, the system documentation and requirements should state the limits so that the customer won't configure a system that fails to operate unexpectedly from lack of memory.

cheers, mike

--
Michael N. Moran           (h) 770 516 7918
5009 Old Field Ct.         (c) 678 521 5460
Kennesaw, GA, USA 30144    http://mnmoran.org

"So often times it happens, that we live our lives in chains
  and we never even know we have the key."
The Eagles, "Already Gone"

The Beatles were wrong: 1 & 1 & 1 is 1

That bears repeating:

As they say, "word".

Steve