load binary modules at run time

This is the classical overlay structure. Look in the linker/loader manual for 'Overlay loading'. Google may also help.

--

Tauno Voipio

My first thought was "overlays". Used mostly one or two decades ago, I think. But may be a nice term to search for. ;-)

--
Stef    (remove caps, dashes and .invalid from e-mail address to reply by mail) 

Do clones have navels?

What sort of tools are you using? "objcopy" implies gcc. But "create one project" implies that you are using an IDE to handle the build process. My thoughts here are that gcc, ln, objcopy, and make can combine to produce magic, but you will have to learn the joys of make and Makefiles. It can be hard going, and debugging Makefiles is "fun", but it should be possible to get a build system together so that when you change a file, all dependent modules get rebuilt as necessary, but nothing else.

There are a couple of shortcuts that can make a big difference for large builds, whether you are using your own Makefile or an IDE. First, make sure you have parallel build enabled with an appropriate number of jobs for the number of cores in your PC. Second, try to use ccache to make redundant compiles a little faster.

The other thing you should probably do is decouple your modules and your base system. Arrange for your modules to have an entry point (or entry points) at a fixed address at the start of the module. This entry point may register other functions with the base system, depending on the flexibility you need. Functions from the base system that are needed by the modules should be exported as an array of pointers, and the address of that array should be passed to the module via its entry function. Once you have this sort of system in place, the base system and the modules can be compiled and linked independently - there will be no need to update everything each time something changes in the base system.

Thanks but...this is what I did..or not?

What about drawbacks?:

1 - I have to create a lot of sections 2 - when I'm working with a module, I have to exclude from build the other modules to speed up time 3 - every time I modify a module I have to recompile all the modules because system has changed..and theoretically I SHOULD retest all the modules..

Hi David, I use CrossWorks (gcc) with ARM Cortex M4, parallel compile are enabled but it would be nice to have a base system that don't change each time you mod ify a module.. Your "decoupling" approach works well regarding module's functions needed b y the base system, in fact I think that I'll maintain this approach that I tried in these days, but regarding the needs of the modules there's a probl em: low level dependencies..for example __float64_div..I have no control of these symbols.. probably the best solution could be to force duplication of these pieces of code in each module..but I'm not able to do it..

What the hell are you trying to do? Even a smallish ARM Cortex M4 has

512 KB of FLASH these days. Why are you trying to overlay?!?!?

--
Randy Yates 
Digital Signal Labs 
http://www.digitalsignallabs.com

2014-05-21 12:28, FabiusAone skrev:

has changed..and theoretically I SHOULD retest all the modules..

Why not map each module to a task ,and only run a single task at a time?

You want a message passing RTOS.

BR Ulf Samuelsson

(Please get yourself a newsreader and a newsserver, rather than the crappy Google Groups interface. Google Groups is fine for searching, but screws up the formatting of posts. Thunderbird and news.eternal-september.org are free and work on all platforms. And please don't top-post.)

Make your modules separate projects, so that they are self-contained except for accessing specific functions in the base system via a jump table. They can even have a "main" function - this can be the entry point for the module. You will end up with duplication of library code between the base system and modules, but that is unlikely to matter - as long as you follow good practices of not using floating point (especially doubles) unless they are needed, not using things like printf(), etc., then the duplicated library code is going to be fairly small. If there are some functions that you know you need and you know they cause a lot of extra code, such as printf(), you can put them in the jump table. You can't necessarily do it with gcc library functions (as distinct from newlib or libc functions) - functions starting with two underscores are compiler-private functions and may have weird calling conventions that don't work when you try to use function pointers. And if you use malloc() and free(), you have to keep them in the base system rather than having two copies.

David

5MB..for now..

How do you possibly manage to get 5 MB of code? It's easy to use up lots of space with data, and equally easy to handle loading that at runtime.

If you really need that much code, get a Cortex M4 with an external memory bus and use that.

Thanks David. I wanted to say this, but I was supposing Fabius has some very weird corner-case.

--
Randy Yates 
Digital Signal Labs 
http://www.digitalsignallabs.com

I'm sure he has good reasons for wanting this arrangement, but I think it could be worth his while taking a step back and telling us what he is looking for in the complete system. Then maybe he can get ideas here for alternative approaches that would work better.

Thanks to all of you, I'll try to explain better the situation, sorry for my english:

I'm developing a system where the user can choose a set of functionalities grouped by c modules, there are no dependencies between these modules. The user can wait 1 second to obtain the functionality but I would a fast system startup.

5 MB is the total sum of the binary modules for now, but I can load a single module on the internal SRAM without problem (I'd prefer keep secure this code.. ) from an SDcard where are stored (crypted). If I maintain only one project with all c modules thanks to the overlay method I'm able to develop this application..but I'm not happy that each time that I add o modify a module I have to recompile all the modules and the base system..I'd prefer a modular system..

I'm using a Cortex M4 (without MMU) with Crossworks (gcc).

I do not quite understand the re-compile requirement. The system is so large that it is sensible to build it from modules, and you need to re-compile only those modules that have been changed. The make utility helps automating this task.

You do have to link the overlay file again, if an overlay changes, but there is no need to change the root code, unless the calling sequence to the overlay code changes.

--

Tauno Voipio

Sounds like a classic overlay loading case.

I don't know the features of the tool chain but assuming that the linker is capable of generating pure binary modules for a specific load address, things should be quite simple. Just decide a common load address for all your 100 object modules and compile and link them independently with that load address. No need for position independent code (PIC) or for fixups during loading.

If you change one module, just recompile that module.

If your modules use some common routines, such as C-runtime routines, put these into the root of the tree. Link this root and get a symbol table file from that linking. Link all your modules against this symbol table file. If you change the tree root, you need to generate a new symbol table file and relink all your 100 modules against it.

If your module contains multiple entry point, link an indirect jump table in the beginning of each module, so you can recompile each module independently. I have used this trick, when programming a 4 to

8 KiB EPROM took at least 5 minutes each, thus making a change to one routine took only 5 minutes to reprogram instead of using an hour to reprogram a dozen EPROMS.

You should look at the documentation of your tool chain, if you can specify an absolute load address for compilation and linking and also verify that you can generate an absolute file (no relocation or fixups, no debug info) in file.

You could also search for terms like "autoload vector" "RSX-11" (with quotes), to see how autoload was implemented in the past, but of with manual overlay loading, there were far more alternatives on a PDP-11.

It looks a lot like you're using Linux. That probably means you have POSIX.

The POSIX constellation has dlopen / dlsym / dlerror / dlclose to dynamically load modules. I've only used this with .so files, but it may go beyond that. The fail cases for this are *painful* but it works.

When you do this, it might be worthwhile to get out of the IDE and use a console/shell prompt and makefiles.

You'll have to more or less hand-code dependencies somewhere, but if you're rigorous with header files, you can use "gcc -M" to generate a file full of dependencies and munge that.

You sound (downthread) like you *really* need a good regression suite derived from unit tests and integration tests.

--
Les Cargill