Flash Memory - Loading an Application of 2M size into 1M Flash memory

Some possible aids:

2) If you are using ARM then you can compile in Thumb mode to gain a reduction in size at a cost of a performance decrease.

4) Can you compress anything? In my application I have a number of big bitmaps that must be stored as constants and I use Run Length Encoding to dramatically reduce their size.

However, what you are doing is firefighting and the only long term solution is redesign the code to be more efficient or make space in the hardware to fit your new code. Look again at was has changed to bloat your code to double its size and think about what is necessary and what is not. If you are using other people's modules then this will eb a lot more difficult but their documentation may help.

Since you have an Ethernet adaptor on the board, why not just put a PC or some other device with a large mass storage on the LAN and load only those segments currently needed into Flash (or preferably RAM).

In the old days with 16 bit computers typically restricted to 64 KiB, large programs were divided into overlays stored to disk. The initialisation routines could be in one overlay branch and initially loaded into memory and when all initialisation was done, the actual application code branch was loaded on the same addresses as the initialisation code, overwriting it. Each overly branch could contain a root and several alternative branches, thus the overlay structure was actually an overlay tree.

While very simple overlays could be handled with ordinary linkers and some application code, any complex overlay structure would definitely need a linker that understands about overlay trees and can generate the autoload vectors. Overlays were used also on MS-DOS to handle larger programs that did not fit into the 512 KiB or 640 KiB physical memory of the original PC.

With the LAN, you could have a single file server and multiple embedded system with a small memory loading overlay segments from the file server over the LAN. A UDP frame is well capable of carrying 1 KiB segments over the net. Implementing UDP on a small device with Ethernet is easy and does not require much code or buffer space.

You would have to handle the situations, when the Ethernet or file server temporary becomes unavailable.

A program RAM at the embedded device would be preferable, since the Flash has a limited number of write operations.

Paul

Are you bootloading from FLASH into RAM and executing from there? If so, you need to compress the program into FLASH and uncompress it into RAM, in a similar way to how the Linux kernel bootloader works.

I'm having trouble understanding how the application grew to 2MB when your development platform only has 1 MB of flash??? Did you write an extra 1MB of code before testing any of it???

Or is this a hypothetical homework question?

You don't say whether or not you are executing in-place from the flash, or executing a copy out of RAM.

How much (spare) RAM do you have? What percentage of your application is code vs data?

Regards,

Hi ,

I'm having trouble understanding how the application grew to 2MB when your development platform only has 1 MB of flash??? Did you write an extra 1MB of code before testing any of it??? >>>>>>>>>>>>>>>>>> It has exeeded in size so that it can not be placed in Flash. I do RAM execution, debugging and development . I did not try flashing the application though many modules were getting integrated . Now the code has got almost freezed as many modules have got finished . I did try flashing now and there comes this issue. It has not exactly doubled in size. It has crossed the 1.4 M mark.

You don't say whether or not you are executing in-place from the flash, or executing a copy out of RAM. >>>>>>>>>>> Bootloader execution is done as 'executing in-place' in Flash. But the application gets executed by getting copied to RAM. This copying is done by Bootloader as in any other normal embedded systems. So, this area also needs some tricks so that the image in bits/pieces can be put together by some means and executed !!

How much (spare) RAM do you have? What percentage of your application is code vs data? >>>>>>>>>>> RAM is not an issue. 32M is there and it is working perfectly from RAM . Only Flashing is the issue and i need tricks in that regard with how to flash the big size image in bits / pieces as size has crossed the Flash memory size and execute the bits / pieces as such or by putting together in some groups with some tricks .

.XMAP file of the application has the following info w.r.t data & code for your reference : .text = 0x12C188 - 0xB0000 = 0x7C188 .rodata = 0x134CD5 - 0x12C188 = 0x8B4D .data = 0x13620A - 0x134CE0 = 0x152A .bss = 0x22AB66 - 0x136210 = 0xF4956

Regards, Karthik Balaguru

Hi ,

cost-effective.

bitmaps that must be stored as constants and I use Run Length Encoding

to dramatically reduce their size.

However, what you are doing is firefighting and the only long term solution is redesign the code to be more efficient or make space in the hardware to fit your new code. Look again at was has changed to bloat your code to double its size and think about what is necessary and what is not. If you are using other people's modules then this will eb a lot more difficult but their documentation may help.

other software tricks to manage this situation , so trying to design and incorporate it.

Regards, Karthik Balaguru

According to this, you only have about 0.5 MB of text, rodata and data combined. Is there additional stuff that goes into flash ? Or are you trying to store the bss section in flash as well ? If so, check your scripts that generate the flash image, and make sure the bss section is removed. Instead, just clear the bss area at startup.

Hi,

Are you bootloading from FLASH into RAM and executing from there?

Flash. But the application gets executed by getting copied to RAM. This copying is done by Bootloader as in any other normal embedded systems.

If so, you need to compress the program into FLASH and uncompress it into RAM, in a similar way to how the Linux kernel bootloader works.

algorithms and dump into Flash and later the bootloader code should uncompress it while copying to the RAM area for execution ? I too beleive that this is one of the good ideas . But, the compression algorithms - Need to think about it . Is there any compression algorithm available in open-source for this kind of task oriented with embedded application or any tool for this kind of activitiy ?

I think, Splitting and putting-together or vice-versa is going to be a bit easy but this also needs synchronisation and buffer management . But, the absence of compression algorithms will ease the task here. If there is some easy way oriented with this splitting & putting-together idea , kindly share with me.

Which of the above method to adopt in this situation so that it would be time-effective ? Kindly share your ideas / views regarding this.

Regards, Karthik Balaguru

Ok, so embed a decompression algorithm into your bootloader and you're good to go. Linux uses deflate (the zip/gzip algorithm), of which versions are freely available. Look at how Linux does it.

Hi,

Ok, so embed a decompression algorithm into your bootloader and you're good to go. Linux uses deflate (the zip/gzip algorithm), of which versions are freely available. Look at how Linux does it.

Need to know the size of Memory that this Deflate algorithm will eat while residing and while execution . Are you aware of it ?

Normally the compression / decompression algorithm results will be very poor if we compress Executable files . But how does this 'Deflate' work here ??? Is it successful in compressing executables / binaries with a good compression ratio ? Here i Need to compress/decompress the ELF files, BIN files .

Kindly share your views / ideas.

Tonnes of Thx in advans, Karthik Balaguru

So stop asking us to do your work for you and GO AND LOOK AT THE THINGS WE'VE ALREADY TOLD YOU.

Yes. No, I won't tell you - it's time you go and *find out*. To get you started, here's some facts from my Linux box:

# ls -l /usr/src/linux/arch/i386/boot/compressed/ [..snip..]

-rwxr-xr-x 1 root root 1041787 2006-07-22 17:13 vmlinux*

-rwxr-xr-x 1 root root 2025956 2006-07-22 17:13 vmlinux.bin*

-rw-r--r-- 1 root root 1027095 2006-07-22 17:13 vmlinux.bin.gz

The .bin file is uncompressed, the .bin.gz is compressed, and the other file has the decompressor at the start. That's for an Intel architechure machine; you'll have to "gzip" your own binary image file to find out how it works on your architecture.

Now run along and do some experiments and googling by yourself. It's p^%$ easy for any half-competent programmer to find the facts you want, so there's no reason to continue asking here.

An experienced person reading disassembled code can quite easily detect which compiler has been used to generate the code and quite reliably guess what the high level language source statements were. This clearly proves that a compiler generates various idiomatic sequences, which are easily detectable by a trained eye and are often repeated multiple times in a program and should thus be easily be compressed to one or two bytes. The same applies to frequent macro expansions in assemblers as well as any text strings (menus etc).

Since you apparently only need to go from 1.4 MiB (not 2 MiB as originally claimed) down to 1 MiB, this should not be too hard, even considering the decompression code size.

Paul

Hi, Fine, Thankyou for your ideas and feedbacks which are really very helpful.

Warm Regards, Karthik Balaguru

Paul Ke> >