ELF file: linking in large binary data

Do you have a question? Post it now! No Registration Necessary

Translate This Thread From English to

Threaded View
Hi,

I would like to include a large block (100+KB)of pre-defined binary
data into my executable.  Mainly, I will be storing static bitmap
images.
Any ideas on how to do it? and what tools are necessary?

Basically, what I would like to do is make an all-purpose resource
object file containing various binary data.  Such as the resource dlls
in MS Windows.

Btw, my cpu is an ARM and I'm using the ARM tools.

Thanks,
Will Barry

Re: ELF file: linking in large binary data
Convert your binary image to whatever data structure you
like (in human readable form, mind you), compile, link and
presto... That's what I do, whenever the use of files is
undesirable if not impossible.

BCNU

Waldemar

Quoted text here. Click to load it



Re: ELF file: linking in large binary data
Quoted text here. Click to load it

Thanks!  But the file I would like to link with the executable is
really large, like 100K+ or so.  So converting manually is out of the
question.  But then writing the program to generate the binary data
into a pre-initialized C array form would not be difficult.  This is
going to be my fallback if I can't find an easier way.  I appreciate
your reply!

Will Barry

Re: ELF file: linking in large binary data
Quoted text here. Click to load it

Why bother?  Just let the executable read the file into an array in
its own data space on startup.  Much more flexible, and no
opportunity for translation foulups.

--
Chuck F ( snipped-for-privacy@yahoo.com) ( snipped-for-privacy@worldnet.att.net)
   Available for consulting/temporary embedded and systems.
We've slightly trimmed the long signature. Click to see the full one.
Re: ELF file: linking in large binary data

Quoted text here. Click to load it

Read the file from where?  None of the embedded systems I've
ever worked on had filesystems.

Quoted text here. Click to load it

But it requires a filesystem, which is a huge amount of
overhead when all you want is an initialized data structure.

--
Grant Edwards                   grante             Yow!  Hello... IRON
                                  at               CURTAIN? Send over a
We've slightly trimmed the long signature. Click to see the full one.
Re: ELF file: linking in large binary data
Thanks for all the replies.

Quoted text here. Click to load it

That is correct, I don't have a filesystem.  So I guess the only place
to store the binary data is together with the executable.  I'm
currently playing around with the INCBIN armasm directive which Wilco
suggested.  According to the documentation, this directive links a raw
binary file into the generated object file as is, with no
interpretation. This looks like an easy way to do it.  I'm gonna try
it and see how it works...

Re: ELF file: linking in large binary data
Quoted text here. Click to load it

Most systems have some sort of rudimentary filesystem, even if it
is not recognizable as such.  Especially systems that need 100k of
raw data.  What is the system that transfers that load module into
runnable code space, and what is used to implement stdin and stdout
(if any).  If the system is doing any user interaction, it has a
file system.

--
Chuck F ( snipped-for-privacy@yahoo.com) ( snipped-for-privacy@worldnet.att.net)
   Available for consulting/temporary embedded and systems.
We've slightly trimmed the long signature. Click to see the full one.
Re: ELF file: linking in large binary data

Quoted text here. Click to load it

Huh?  I've been doing embedded systems stuff for 25 years and
have never worked on anything that had any sort of filesystem
(even rudimentary).

Quoted text here. Click to load it

Huh?  I burn my "load modules" into ROM.

Quoted text here. Click to load it

Nothing!  There is no filesystem.  There sometimes is a
function I can call to write a string to a UART, but it's not a
filesystem.

Quoted text here. Click to load it

That's complete nonsense.

--
Grant Edwards                   grante             Yow!  I will SHAVE and
                                  at               buy JELL-O and bring my
We've slightly trimmed the long signature. Click to see the full one.
Re: ELF file: linking in large binary data
Quoted text here. Click to load it
....

Lets see one system transfers and verifies 640KB into real time video
run space. Another transfers 128KB into a different form of dual
ported RAM, both use DMA.

Quoted text here. Click to load it

One system has a tracking camera and a rarely used toggle switch for
user interaction.

Another has 6 LEDs and two touch switches.

Both have a 16bit code set in the 64KB to 128KB size and NO file I/O
especially in NO way related to user I/O.

Debug has a very rudimentary Command Line Interpreter, with no file i/o
system there.

--
Paul Carpenter          | snipped-for-privacy@pcserviceselectronics.co.uk
<http://www.pcserviceselectronics.co.uk/ PC Services
We've slightly trimmed the long signature. Click to see the full one.
Re: ELF file: linking in large binary data

Quoted text here. Click to load it

It seems that you have been infected by the rotten Unix I/O ideology
for a too long time :-) :-).

So apparently any sequence of two or more bytes would be a file
system. Thus, a 16-32 bit processor with en 8 bit data bus (such as
68008) would have a file system since multiple bytes are transferred
in sequence to get a word transfer.

Why not take the idea even further and define that any sequence of
bits is a file system. Thus, even a UART sending _only_ the SYN
character at regular intervals would have a file system.
  
Quoted text here. Click to load it

On an embedded system, that systems that transfers the load module is
often a human hand moving the EPROM from the prommer to the embedded
system PROM socket.

Quoted text here. Click to load it

What is the point of having these channels, if you do not have
anywhere to connect them.

Quoted text here. Click to load it

So a system that only reads a switch (on a safety gate that shuts down
the system, if someone tries to enter a hazardous area) is a file
system ?

Paul


Re: ELF file: linking in large binary data

Quoted text here. Click to load it

We don't need to do that --- there already are enough people
(including some vintage CS professors) who go around stating that
"everything is a database", and they actually mean it.  Classic case
of new hammer syndrome, if you ask me.

--
Hans-Bernhard Broeker ( snipped-for-privacy@physik.rwth-aachen.de)
Even if all the snow were burnt, ashes would remain.

Re: ELF file: linking in large binary data
Quoted text here. Click to load it

Which doesn't make it any less true, just as every wire is a
transmission line.  The problem is to know when to apply the
simpler approximations.  Similarly, I suspect very few ships adjust
for relativaty path distortion in their navigation, although the
GPS satellites very well may do so.

--
Chuck F ( snipped-for-privacy@yahoo.com) ( snipped-for-privacy@worldnet.att.net)
   Available for consulting/temporary embedded and systems.
We've slightly trimmed the long signature. Click to see the full one.
Re: ELF file: linking in large binary data
On 14 Oct 2004 08:22:57 GMT, Hans-Bernhard Broeker

Quoted text here. Click to load it

The current version seems to be "everything is an object".

Paul


Re: ELF file: linking in large binary data
Quoted text here. Click to load it

Well, in the past my systems consisted of code that interacted with
the outside world in some manner.  They had no disk file systems,
but they did read switch contacts, A/D converters and the ilk, and
they output information to such things as leds, even displays,
lans, etc.  All those things appeared as the specialized file
system for that particular application.  The connections were
implemented as drivers for the devices.  Devices could be brought
on (and off) line (connected to the interrupt system) by opening
(and closing) files.

This was a great convenience, since I could capture a run of the
device to actual disk files, and re-run any problems indefinitely,
measure response times, etc. ad nauseam.

I suggest you stop thinking of files in such a limited manner, but
rather as the communication path between the world and your
software.  Without such the software is totally useless.  By
abstracting that interface you will have many more abilities.

--
Chuck F ( snipped-for-privacy@yahoo.com) ( snipped-for-privacy@worldnet.att.net)
   Available for consulting/temporary embedded and systems.
We've slightly trimmed the long signature. Click to see the full one.
Re: ELF file: linking in large binary data

Quoted text here. Click to load it

Such an arrangement can be very useful - that's one of the nice things about
working with a *nix system.  It is also possible to make such an abstraction
even without a *nix OS.  But no matter how it is done, it takes a lot of
overhead.  In "big" embedded systems, it is probably worth it for the
flexibility and convenience, while in "small" systems it is not.  So while
most of the systems *you* have worked with have some sort of filesystem, I
think most of the systems used regularly by people in this group have no
filesystem of any sort.




Re: ELF file: linking in large binary data
Quoted text here. Click to load it

On the contrary, the systems I am talking about were often small,
with minimal overhead.  They rarely contained more than 8k of code,
and 8k of data space, and were based on the 8080.  Their hallmark
was reliability, since they were used in medical systems.
Everything was in ROM, and the user turned on the power switch.

A new device was integrated by filling in one row of a six column
table, specifying routines for open/close, read/write,
status/control.  Simple devices could have no more than a return
instruction for a routine.  To return an error (e.g. write on
read/only) they did "stc; ret;".  A simple byte oriented serial
device could be implemented in 20 bytes of code or less.  A
buffered interrupt driven device took slightly more.

You realize, of course, that a file system does not imply an
operating system, although the reverse seems universal.

I also had the totally unstructured raw port access embodied as
non-portable system routines for the high level languages.

One instrument where the data capture etc. was very handy could be
described by:

The operator sets an identification (digiswitches), feeds a sample,
and presses an 'inject' button.  The sample disappears into the
maw.  The next sample may be injected no sooner than about 10
seconds later.  After about two minutes (a few seconds window) the
results appear in the form of several analog voltages.  These have
to be captured, computed, and the results associated with the
original ID number.  Note the pipeline.  Some particular ID numbers
constitute calibration samples, which must be analyzed and
calibrate the overall machine, using a least squares analysis and
various 'goodness' criteria.  The results appear both on a local
hard copy, and are transmitted to a central data collection system
with error protection.  If for any reason the remote collector is
unavailable results are buffered until they can be shipped.

I have only had one system that ever needed 100k or so of
initialized data, and that had to do with rapid response to
encrypted challenges.

--
Chuck F ( snipped-for-privacy@yahoo.com) ( snipped-for-privacy@worldnet.att.net)
   Available for consulting/temporary embedded and systems.
We've slightly trimmed the long signature. Click to see the full one.
Re: ELF file: linking in large binary data

Quoted text here. Click to load it

This may be a quibble, but for the sake of terminolgy
isn't this just treating everything as a stream rather
than as a file?

At MPE we call this Generic I/O - it avoids the overhead
issues you're being accused of. It's a great approach, but
I'm not sure I'd use it as intensively as you do when
performance is an issue.

Stephen
--
Stephen Pelc, snipped-for-privacy@INVALID.mpeltd.demon.co.uk
MicroProcessor Engineering Ltd - More Real, Less Time
We've slightly trimmed the long signature. Click to see the full one.
Re: ELF file: linking in large binary data

Quoted text here. Click to load it

That's a very, uh, unique way of doing things.  I've never
heard of anybody implimenting a filesystem so they can read a
pushbutton switch or turn an LED on/off before.

Quoted text here. Click to load it

I guess I don't see how being able to open/close an I/O device
makes it into a filesystem.

Quoted text here. Click to load it

So an LED is a filesystem.  You've redefined the word
filesystem so broadly that it's become meaningless.

Quoted text here. Click to load it

--
Grant Edwards                   grante             Yow!  I'm receiving a coded
                                  at               message from EUBIE BLAKE!!
We've slightly trimmed the long signature. Click to see the full one.
Re: ELF file: linking in large binary data
Quoted text here. Click to load it

Those are operations needed for a filesystem.  Believe me, this
sort of implementation makes life much easier in the future.  I am
naturally extremely lazy.

Quoted text here. Click to load it

A LED may be a device within a filesystem.  It is write-only.  It
requires very few operations to prepare for use, or to discontinue
use.  It communicates with the outside world.  It may be
virtuallized.

I suggest you look at my response to David Brown.  BTW, please
don't strip attributes for material you quote.

--
Chuck F ( snipped-for-privacy@yahoo.com) ( snipped-for-privacy@worldnet.att.net)
   Available for consulting/temporary embedded and systems.
We've slightly trimmed the long signature. Click to see the full one.
Re: ELF file: linking in large binary data

Quoted text here. Click to load it

I always use that technique, myself.  I usually do it in python or Matlab,
rather than C, because that's easier to tweak and still easy to make part
of the build process (make files).  It also has the advantage that it's
easy to perform numerical-domain transformations on the data on the way,
like quantization and rounding.

However, since you're dealing with images, it might interest you to know
that the original default PNM (portable-any-map) or pbm image
transformation library format was in fact syntactically a C array
initializer.  That might be a convenient pre-built tool for the job.  Lots
of original X-windows programs compiled all of their icons in, even though
they would most likely be running on machines with file systems, to avoid
the problem of ensuring that the path to the directory containing the
icons/images was properly configured.  So there's actually plenty of
experience out there in existing code...

I still don't understand why some object file formats (COFF?) are so
baroque about these things, though.  On one platform that I've used, the
COFF image of a constant initialized array was several orders of magnitude
larger than the original...

Cheers,

--
Andrew

Site Timeline