Relocating application architecture and compiler support - Page 2

Do you have a question? Post it now! No Registration Necessary

Translate This Thread From English to

Threaded View
Re: Relocating application architecture and compiler support


snip

Quoted text here. Click to load it

Certainly on S/360, but not on other then contemporary architectures.  All
that is needed is a protected "system base regiser" which is part of the
program's context, to which the hardware adds all program addresses.  Then
all programs can start at zero and go to whatever, and all that is needed if
the OS needs to move a program in mid execution is to change the base
register.

BTDT

--
 - Stephen Fuld
   e-mail address disguised to prevent spam



Re: Relocating application architecture and compiler support

Quoted text here. Click to load it

i guess all those 'adcons' and the expectation that appl'n programs must
anticipate interrupts were the real complication.  the base-displacement
scheme seemed clever to me, but BAL was my first language so maybe i
just didn't know any better.  didn't have much to do with Cobol, but i
remember seeing these giant programs that were written for 360 DOS and
depended on a bit of link-editor support.  feature referred to as some
kind of 'overlay'.  i vaguely recall that they could thrash away for
hours, doing the equivalent of an OS 'xctl' several times a second.

but i think the original idea must have been far-sighted for its time
and probably under-exploited, just like Gene Amdahl's TRT.  had a chance
to meet him when he was in his seventies, still working and answering
the phones when the receptionist wasn't around.  passed it up partly
because i thought he deserved more respect than listening to me ask why
cpu's didn't have double-ended stacks.  in spite of all the fooferah
that got added by the operating systems, what he and others did still
seems very clear and clean to me.  not like all that little-endian Intel
stuff which i gather has complicated everybody's life in unseen ways
just so some 8080 programs could keep working.

a boss once made us read a business article about how complicated jet
engines are when they don't really need to be if we forget about
creature comforts for passengers and municipal bylaws.  i guess 360's
were a bit like this - you had to air condition them, but they also had
heaters to warm up the core!

pc

Re: Relocating application architecture and compiler support
(snip)

Quoted text here. Click to load it

But there is one thing that OS/360 does easily that is hard in
other systems, which is what the LINK macro does.  To take
another program, load it into the same address space, execute it
(with the original still addressable) and then return.

I remember TOPS-10, where each program was loaded with virtual
origin of 0.  There was a system for sending files to the print
queue which involved writing the entire address space to disk,
running the QUEUE program with an option to execute a program
when it was done, which would then reload and continue on.

Any system with a fixed load point can't have two programs in
the same address space at the same time.

In the OS/360 case, besides doing LINK, many routines,
especially access methods, were directly addressable somewhere
in system space.  Now, there are systems which reserve parts of
the address space for system and parts for user, but that
doesn't help LINK.

-- glen


Re: Relocating application architecture and compiler support
Quoted text here. Click to load it

basically you could do link-like function in both os/360 as well as
tss/360. the primary difference was that os/360 had to read the
executable file image into memory and then run the address constant
swizzle against that were randomly sprinkled thruout the execution
image.

in the tss/360 scenario ... you just had to memory map some portion of
the virtual address space to the executable file image on disk (and
let the paging operation fetch the pages as needed) ... and all the
address constants needing swizzling were kept in a different
structure.

os/360 had a single real address space orientation so that it was
possible for multiple different processes to share the same executable
image because they all shared the same (single, real) address space.

tss/360 had multiple virtual address spaces .... and for multiple
different processes to share the same exact executable copy ... it
relied on the shared segment infrastructure.

in the os/360 scenario, all program loading was to a unique real
address ... since all processes shared the same, single real address
space (there was no chance for program address conflict ... since each
program could be assigned a unique real address as it was read into
memory).

in the tss/360 scenario, different processes might load & populate
their own virtual address space in difference sequences
... potentially creating address assignment conflict if there was a
requirement that each application have identically assigned virtual
address across all virtual address spaces.

in the os/360 scenario ... if you had two different processes,
the first LINKed app1, then app2, then app3, and finally app4 and
the second LINKed app2, then app4, then app3, and finally app1 ...
it all fell out in the wash since there was a single global
real address space.

a difference between the os/360 and tss/360 operation, is that os/360
allowed all the address constants (needing position location
swizzling) to be randomly sprinkled thruout the executable
image. tss/360 collected address constants (needing swizzling) into a
different structure.

both could do dynamic process location specific binding at program
load time.  however, in the tss/360 scenario ... different processes
could share the same exact executable image at different address
locations ... because the executable image was a separate structure
from the structure of address constants (that required location
specific swizzling).

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn /

Re: Relocating application architecture and compiler support
Quoted text here. Click to load it
So why was TSS considered so slow?  (not that OS/360 was any ball of speed).


Re: Relocating application architecture and compiler support
Quoted text here. Click to load it

lots of reasons ... long path lengths ... over complex algorithms
... bloated code size.

lets say you had os/360 (or cms) environment that did program load
... it could queue up a single i/o that read application into memory
in multiple 64k chucks ... all batched together.

tss/360 could have a large compiler laid out as 1mbyte memory mapped
file ... do a memory mapping for 256 4k pages ... and then possibly
individually page fault each 4k page one at a time. To some extent
they got over enamored with concept of one-level store and paging and
lost the fact that the purity of the concept could significantly
increase the latency if all transfers had to be serialized 4k bytes at
a time (as opposed to doing program loading batching in larger
transfer units).

kernel was really large ... tss/360 supposedly was going to have
target of 512k 360/67 ... but quite quickly grew to minimum 768k
storage because of bloated fixed kernel. at one point there was a
statement about a set of benchmarks down on a 1mbyte 360/67
uniprocessor and on a 2mbyte 360/67 two processor machine ... with the
2mbyte/2processor having 3.8 times the thruput of the single processor
benchmark. the official coment was (while the uniprocessor system was
really slow), the two processor benchmark having 3.8 times the thruput
of the single processor benchmark demonstrated how advanced the
tss/360 multiprocessor algorithms were (getting almost four times the
thruput with only twice the hardware). the actual explanation was that
the kernel was so bloated that it could hardly fit in 1mbyte
configuration .... and that in the 2mbyte configuration there was
almost enuf memory (not in use by the kernel) for running
applications.

benchmark at the university on 768k 360/67 running tss/360 .. i
believe prerelease 0.68 with four emulated users doing mix-mode
fortran edit, compile and execute ... had multi-second response for
trivial program edit line input. at the same time, on the same
hardware with cp/67 running same mix-mode fortran edit, compile and
execute had subsecond response for trivial edit line input .... but
running 30 conccurent cms users ... compared to 4 tss users (although
i had already done a lot of cp/67 performance enhancements).

there was folklore that when tss/360 was decommited and the
development group reduced from 1200 people to possibly 20 ... that a
single person now had responsibility for the tss/360 scheduler and a
large number of other modules. supposedly the person discovered that
on a pass thru the kernel ... every involved kernel module was
repeatedly calling the scheduler ... when it turn out that it was only
necessary to call the scheduler once per kernel call (rather than
every module calling the scheduler resulting in multiple scheduler
calls per kernel call). fixing that is claimed to have eliminated a
million(?) instructions per kernel call.

at some point in the tss/370 life ... it had been relatively stable
for a long time ... with a lot work over the years on performance
tweaking ... they claimed that they got the total pathlength to handle
page fault (page fault, page replacement select, schedule page read,
task switch, page read complete, task switch, etc) down to maybe five
times better than MVS ... but still five times longer than my
pathlength for equivalent sequence in vm/370.

it was in this era (late 70s) that they did the unix project for at&t
... where a kernel semantics interface was built on low-level tss/370
kernel function to support high-level unix environment. i believe this
tss/370 environment for unix was called ssup.

one of the early tss/360 features that i gave them trouble about was
the whole thing about interactive vs-a-vs batch. early 360/67 had 2301
"drum" for limited amount of high-speed paging space and the rest was
2311 (very slow) disks. the scenario went something like ... if some
finished a line in edit and hit return (on 2741) ... tss kernel would
recognize it as interactive and pull the pages that the task had been
using the previous time from 2311 into memory and then write them all
out to 2301 ... once that was done it would start the task ... which
would allow it to page fault the pages back into memory from the 2301.
when the trivial interactive task went into wait (for more terminal
input), the kernel would pull all of the tasks pages off the 2301 back
into memory and move them to 2311. This dragging of pages off 2311 int
memory and out to 2301 and then dragging the pages off 2301 into
memory and out to 2311 would occur regardless of whether there was any
contention for either 2301 space or real memory space. It was a neat,
fancy algorithm and they would do it everytime ... whether it was
necessary or not ... just because it was so neat(?).

... there is a lot more ... but that should be enuf for the moment.

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn /

Re: Relocating application architecture and compiler support

Quoted text here. Click to load it
[...]
Quoted text here. Click to load it

And this discussion dredges up memories of the one type of OS/360
executable that was relocatable on the fly: the transient (Type 3/4)
SVC.  Typical systems had multiple transient areas, and the requirements
for a type 3/4 SVC (there wasn't really any difference between the
two types) included a prohibition against any relocatable constants,
a requirement of absolute reentrancy, and a fixed (R12?) base register.

A transient SVC might start execution in one transient area, then
surrender control (for example, to do I/O) and resume execution in
a different area.  The transient SVC handler was responsible for
setting the base register to the appropriate value each time a
transient SVC was dispatched.

Joe Morris

Re: Relocating application architecture and compiler support

|>
|> A transient SVC might start execution in one transient area, then
|> surrender control (for example, to do I/O) and resume execution in
|> a different area.  The transient SVC handler was responsible for
|> setting the base register to the appropriate value each time a
|> transient SVC was dispatched.

And the author of the SVC for not saving an address of any of its
code across potential dispatching boundaries!


Regards,
Nick Maclaren.

Re: Relocating application architecture and compiler support
Quoted text here. Click to load it

what i remember from mft/mvt days was that there were (initially?)
two, 2k transient svc areas. part of the issue was that the loader
wasn't involved in reading code into the transient area ... so there
wasn't any ability to perform swizzling operation on any (relocatable)
address constants.

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn /

Re: Relocating application architecture and compiler support
Quoted text here. Click to load it

I'm convinced.  I will never understand OS/360 relocation
without getting loaded on gin swizzles.

--
Nicholas O. Lindan, Cleveland, Ohio
Consulting Engineer:  Electronics; Informatics; Photonics.
We've slightly trimmed the long signature. Click to see the full one.
Re: Relocating application architecture and compiler support
Quoted text here. Click to load it
... snip ...
Quoted text here. Click to load it

This is all OT on comp.arch.embedded.  Please set followups on any
further entries in this thread to eliminate c.a.e.

--
"If you want to post a followup via groups.google.com, don't use
 the broken "Reply" link at the bottom of the article.  Click on
We've slightly trimmed the long signature. Click to see the full one.
Re: Relocating application architecture and compiler support

Quoted text here. Click to load it

In the interests of simplicity, I left out a few things.  In the first
implementation that I know about, there were two "base registers", typicaly
one for instructions and one for data.  There was an instruction to "link"
to another chunk of code.  The typical mechanism for things like access
methods, was to keep the same data area and link to the code, replacing the
user program code temporarily.  Thus the access method code could access the
user program data just fine.  When it was done with whatever it had to do,
ir executed the instruction to return to the user code and replace the base
address with the original one.  Later models added a second set of two base
registers that allowed further flexibility for more complex schemes.  Even
later ones had more features that moved the architecture more toward a
segment scheme (with up to 16 active segments out of something like 32K
total available) that is totally independent of the (later introduced)
paging mechansim.

Quoted text here. Click to load it

Yuck!  That sounds awfull.  It seems far simpler for the program to write
the file to disk and simply pass a file name to the printer writer.  But I
am sure there were other constraints that I just don't know about.

Quoted text here. Click to load it

Actually it can, but it takes some doing, like the hardware I talked about
above, or a convention about using different parts of the address space.
With the extra hardware, it isn't much of a problem.

One extra advantage of the scheme I an taliking abouit is that there is no
need for the "address swizzling stuff" on load that was required by OS/360.
That speeded up program loads.

Quoted text here. Click to load it

Right.  See above.

--
 - Stephen Fuld
   e-mail address disguised to prevent spam



Re: Relocating application architecture and compiler support
Quoted text here. Click to load it

<snip>

Quoted text here. Click to load it
involved

Nope.  Glen is misremembering.  There was a way to "dump" your
core to disk and then our spooling feature would send it to
the print queue.  This often happened, under our batch system.
But I don't know a single person who ever did this command on
purpose :-).

Quoted text here. Click to load it

Glen was talking about a feature of the -10 in that all
files with a LPT extension, as in FOO.LPT, would automatically
be queued when the job logged out.

<snip>

/BAH

Subtract a hundred and four for e-mail.

Re: Relocating application architecture and compiler support

(snip)

Quoted text here. Click to load it

I thought there was a way for Fortran programs to send
jobs to the printer while they were still running, and that
the program would have to run QUEUE to do it.

A story I do remember is that QUEUE had an option to run a
desired program when it was finished, I thought to implement
the system previously described.  QUEUE also had the ability
to run while a user was not logged in so one could check queue
status without logging in.  Someone discovered that the
"run after" option still worked even when a user wasn't logged
in, and that it was run under a privileged account.  Submitting
the bug report, there is a box for publish/don't publish and
the person, not expecting it to happen, checked the publish box.

After DEC did publish the bug report they then had to very
quickly fix the bug that allowed people to run privileged programs
without needing to log in.  Maybe that is also an urban
legend, but the source seemed pretty reliable.

-- glen


Re: Relocating application architecture and compiler support
On Sun, 23 Jan 2005 14:08:56 -0800 in alt.folklore.computers, glen

Quoted text here. Click to load it

There were a number of similar SPRs published for various DEC OSes.
They had to be fixed quickly because of DEC's popularity in
educational establishments, where such holes were rapidly discovered
and exploited to avoid being charged for work.
In those pre-Internet days, the only people who saw the SPRs were the
system managers, admins, and programmers who got the fixes; although
in some educational situations those people could often also be
students.

--
Thanks. Take care, Brian Inglis     Calgary, Alberta, Canada

snipped-for-privacy@CSi.com     (Brian[dot]InglisSystematicSW[dot]ab[dot]ca)
We've slightly trimmed the long signature. Click to see the full one.
Re: Relocating application architecture and compiler support
Quoted text here. Click to load it

Eh?  You are confusing two things.  If that refers to physical memory,
then that is just virtual memory support.  Not a problem.

If it refers to virtual memory, it won't work for any program that
takes the address of code and needs to compare it with others.  In C,
for example, you can take the address of functions and manipulate them
ad lib.  You can't relocate them within virtual memory thereafter.

Quoted text here. Click to load it

Yes.  It is useful and can be (and has been) implemented even in
variants of Unix.  It needs no extra hardware support.

Quoted text here. Click to load it

All numbers of base registers from 1 to 3 were common, and more were
not unknown.  If I recall, I once implemented a language system with
over 3, but can't now remember exactly why.  Something like the data
for the program and run-time system being separate.


Regards,
Nick Maclaren.

Re: Relocating application architecture and compiler support

Quoted text here. Click to load it

Well, it wasn't virtual memory in the conventional sense.  There was no
paging, and you couldn't define a single program larger than the physical
memory (without doing the link type stuff).  In a sense, it was "inverted"
virtual memory, or perhaps "virtual addresses" in that wheras in virtual
memory you could have addresses larger than physical memory that were mapped
to the same physical address at different times, in this scheme, you had
multiple pieces of code with the same "virtual" address mapped to the same
physical address at different times.

Quoted text here. Click to load it

Remember, the base register was invisible to the program.  If a program
loaded an address into a register, or passed it as an argument, it was
always a program relative address.  If the address was used to address
physical memory (either via a load/store or a jump/call) the hardware added
the base register contents to it at that time.

--
 - Stephen Fuld
   e-mail address disguised to prevent spam



Re: Relocating application architecture and compiler support

(snip of S/360 and base displacement addressing)

Quoted text here. Click to load it

Did you ever write C programs for the original Macintosh?
(I didn't, either, but I read some of the manuals.)

Many routines would, instead of pointers return pointers to pointers
such that the OS could move things around when necessary.
(The 68000 didn't have an MMU.)  There were complicated rules
on the use of pointers because you had to know when the OS could
move things and make sure you didn't keep a pointer to something
that could be moved.

With OS/360 once you load an address into a register that
object can't move because the system doesn't know to change
the register.

-- glen


Re: Relocating application architecture and compiler support

Quoted text here. Click to load it

No, I never did.

Quoted text here. Click to load it

That sounds messy.  What I was talking about required none of that.
Everything worked naturally, with no special needs to know when the OS could
move things.  In fact, the OS could move things pretty much any time that
you could have an interrupt.  All pointers were "program relative" within
your program and only got converted to physical memory addresses when used
by the hardware.

Quoted text here. Click to load it

Right.  And that was the cause of much grief among OS/360 users since that
meant no swapping out capability to compact memory or to suspend a program
(unless it got loaded back to the same physical memory location).  This
caused reduced throughput as you then had things like fixed memory
partitions (MFT) and even the variable ones (MVT) had limitations.  It also
made things like interactive programming (the original TSO) a real dog
(which we have discussed here before).

--
 - Stephen Fuld
   e-mail address disguised to prevent spam



Re: Relocating application architecture and compiler support
fOn Mon, 24 Jan 2005 02:14:01 GMT in alt.folklore.computers, "Stephen

Quoted text here. Click to load it


The S/370 (perhaps also S/360) assembler supported marking code
sections as RENT (reenterable), REUS (serially reusable), and/or PIC
(position independent code); it did require a bit more work by the
programmer to avoid using any static data areas and inadvertently
introducing relocatable items, but the supporting functionality was
there.

--
Thanks. Take care, Brian Inglis     Calgary, Alberta, Canada

snipped-for-privacy@CSi.com     (Brian[dot]InglisSystematicSW[dot]ab[dot]ca)
We've slightly trimmed the long signature. Click to see the full one.

Site Timeline