Relocating application architecture and compiler support

(snip, I wrote)

Well, that was supposed to include during interrupts.

-- glen

No, I never did.

That sounds messy. What I was talking about required none of that. Everything worked naturally, with no special needs to know when the OS could move things. In fact, the OS could move things pretty much any time that you could have an interrupt. All pointers were "program relative" within your program and only got converted to physical memory addresses when used by the hardware.

Right. And that was the cause of much grief among OS/360 users since that meant no swapping out capability to compact memory or to suspend a program (unless it got loaded back to the same physical memory location). This caused reduced throughput as you then had things like fixed memory partitions (MFT) and even the variable ones (MVT) had limitations. It also made things like interactive programming (the original TSO) a real dog (which we have discussed here before).

--
 - Stephen Fuld
   e-mail address disguised to prevent spam

But not too different. It's still a good idea to have two copies of the test code.

One possibility is an interrupt driven test which incrementally steps through memory N words at a time. If the target memory location falls within the test routine itself, a second copy of the test code is invoked instead. This is easy if checking a single location at each interrupt. If checking multiple locations each time, you need to be careful with the test code boundaries.

Another possibility is simply having two identical test routines tied to different interrupts (or alternating on one interrupt), each of which can check on each other's code addresses.

George

--
for email reply remove "/" from address

There were a number of similar SPRs published for various DEC OSes. They had to be fixed quickly because of DEC's popularity in educational establishments, where such holes were rapidly discovered and exploited to avoid being charged for work. In those pre-Internet days, the only people who saw the SPRs were the system managers, admins, and programmers who got the fixes; although in some educational situations those people could often also be students.

--
Thanks. Take care, Brian Inglis 	Calgary, Alberta, Canada

Brian.Inglis@CSi.com 	(Brian[dot]Inglis{at}SystematicSW[dot]ab[dot]ca)
    fake address		use address above to reply

The S/370 (perhaps also S/360) assembler supported marking code sections as RENT (reenterable), REUS (serially reusable), and/or PIC (position independent code); it did require a bit more work by the programmer to avoid using any static data areas and inadvertently introducing relocatable items, but the supporting functionality was there.

--
Thanks. Take care, Brian Inglis 	Calgary, Alberta, Canada

Brian.Inglis@CSi.com 	(Brian[dot]Inglis{at}SystematicSW[dot]ab[dot]ca)
    fake address		use address above to reply

For the S/370 models after the 155 and 165, there was virtual memory/paging support, so such things were either eiasier or not needed. For S/360, I don't see how it could work given that the instructions had a base register number included in them and if the program was relocated, the contents of that register would have to change, but the OS doesn't know which registers to change. Can you explain in more detail how that worked?

--
 - Stephen Fuld
   e-mail address disguised to prevent spam

I haven't written any 360/370 assembler for more than 20 years, though I used to be pretty good at it, so details may be a tad rusty.

[NB: I started to do this from memory, then decided that Al Kossow could help immensely, and I was right: Cf. page 37 of

Thanks again for your preseveration efforts, Al! --rma ]

The start of any assembler program was an instruction to assign a base address to a register, followed by one or more USING statements to assign one or more base registers starting with the one in the instruction just mentioned. The convention for OS/360 and its descendants was to use register 12, as in

BALR 12,0 USING *,12

which caused the system to place the current location into 12 and then assign implied addresses against it. Additional base registers could be assigned if needed due to code size:

BALR 12,0 USING *,12 BAL 11,4095(12) LA 11,1(11) USING *+4096,11 ...

So the code was automagically relocatable, since the base register(s) were given the correct value by the system at execution time--not by the OS, but by the hardware itself.

--
Rich Alderson                                       | /"\ ASCII ribbon     |
news@alderson.users.panix.com                       | \ / campaign against |
"You get what anybody gets. You get a lifetime."    |  x  HTML mail and    |
                         --Death, of the Endless    | / \ postings         |

Versions of MS Windows prior to full virtual memory support - ie before Windows 95[*] - essentially did the same thing. The memory allocation routines would return memory "handles". To get a pointer you made another call to "lock" the handle; then the region would be fixed in memory until you unlocked at - at which time it could be moved by the OS (such as it was) to another location.

There were a couple of different varieties of memory locking so that, for example, a process could tell Windows that it expected to keep an area locked for a while and so it should be put in an area where it was less likely to cause fragmentation. This was useful for IPC; since processes were running in the same address space, you could allocate and lock a region in one process, send the address to another process in a Windows message, and then they could share the area.

All completely unsafe if there was any program misbehavior, of course, but that's what you had with Windows. These days Windows has virtual memory and other goodies and remains completely unsafe; this is called "progress".

*Windows 3.0 and Windows for Workgroups had "386 Enhanced Mode", which provided some virtual memory support, but I can't remember how far it went. It might have run programs in separate virtual-86 regions. No doubt someone here recalls the details, or cares enough to look them up.

--
Michael Wojcik                  michael.wojcik@microfocus.com

Auden often writes like Disney.  Like Disney, he knows the shape of beasts --
(& incidently he, too, might have a company of artists producing his lines) --
unlike Lawrence, he does not know what shapes or motivates these beasts.
   -- Dylan Thomas

^^^ LA 11,4095(12) Two instructions required since displacements limited to 12 bits.

This is more of a DOS-ism. Under OS all programs are called with the entry point address in register 15, so you can use ENTRY AAA AAA LR 12,15 USING AAA,12

Snipped examples. This, and the following posts, miss the point. I know that the code could be loaded anywhere in memory for execution. The point was that once it was loaded, it could not be moved to some other physical location in memory because the actual physical address was in some user registers and the OS couldn't know which ones and thus could not change them of it wanted to move the program.

--
 - Stephen Fuld
   e-mail address disguised to prevent spam

Lynn's example is not relocateable to the level you are thinking- that is, relocateable at an interrupt level. Once execution starts, virtual (not physical) memory addresses must stay put.

The relocatabilty you are talking about can be done, but it would be enormously difficult and a PIA. It would mean that all memory (address) references must be atomic. E.g.

-> read address from memory

-> do some complex calculation based on the address to get a new address

-> read memory at the new address

could not be done, because at any point between the 1st and 3rt bullet, an interrupt could have happened, rendering the calculation moot. This is much more than simply knowing which registers contain addresses- no addresses at all in memory would be allowable *in the entire application address space*.

But I don't see any value to the level of relocateabiltiy you are discussing.

Edward Wolfgram

i had a very simple objective that the executable program image on disk would be identical to the executable program image in virtual memory and that the same executable program image could appear (using virtual memory segment sharing) could appear concurrently in multiple different virtual address spaces at potentially different virtual addresses.

each active, executing context of that executable program image (in

360) could have location specific bindings (like addresses in registers) ... but the storage image of the executable program would not.

os/360 conventions peppered the program image with things called relocatable address constants ... which would be swizzled to an address/location specific value when the program was brought into real storage (or mapped into virtual address space) before actual execution began.

tss/360 accomplished the objective by collecting the relocatable address constants (needing any swizzling) into a different structure that could be private to a specific executable instance ... but wasn't part of the instruction program image.

For the stuff i did, i made due with various programming hacks to resolve things relative to content of some register .... and in 360, it is possible to establish the specific register executable instance contents with mechanisms like BALR reg,0.

this is somewhat analogous to different threads executing the same exact code tending to have their private instance execution environment .... even though the exact same common code is being executed ,,,, and extending the private instance execution environment to include instance specific location of the (same) executable code.

in both cp67 and vm370 (360, 370 and later), the virtual memory relocation mechanism allowed placement of virtual storage pages at randomly (and potentially varying) real storage locations ... transparent to the application.

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/

Yes. That is the point.

In S/360 and its contemporaries, there was no such thing as "virtual addresses".

On S/360 architecture, yes. But the architecture I described in earlier posts in this thread did it easily and naturally with no programmer attention at all.

No. Read the description of the system I talked about earlier in this thread. Briefly, all addresses visible to the program are "program relative", that is starting at address zero. The hardware automatically adds the contents of a "base register" that is invisible to the program, but part of its context, to all program addresses to get physical hardware addresses.

It is (or I should say was, as virtual memory makes it moot) extremely valuable. Say you have several interactive/time sharing users. You really want to be able to swap out a program during the "user think time", or even if it uses up its time quantum, to allow another user to use the memory. But this is very limited if, when a program is swapped out, it must be swapped back in to the same physical address as it was loaded to originally. Or the OS just wants to move a program to defragment memory in order to make more efficient use of it (and perhaps allow another program in). This can't really be done without a mechanism like the one I described.

--
 - Stephen Fuld
   e-mail address disguised to prevent spam

Different objectives have been discussed in this thread, both with and without S/370 style DAT. If one wanted to do timesharing on S/360 including relocating programs as they were swapped in and out, something more complex would be required, possibly including conventions on register use.

It was nice of S/370 to come along when it did to remove the need for that problem.

-- glen

note the original base/bound stuff that boeing used to do swapping of contiguous memory ... may have been in the microcode of some number of (360/50) machines possibly associated with emulation (1401, 7094, etc) ... as opposed to some custom microprograming that they had done.

it was possible to get custom microprogramming support on many of these machines. I think it was Lincoln Labs may have originated the search-list (SLT) instruction that was installed on some number of the

360/67s.

the boston programming center in support of the conversational programming system (CPS) ... interactive PLI and BASIC that ran under relative vanilla os/360 ... had done a custom RPQ microcode package that could be had for 360/50 ... that was performance acceleration for doing interpretive execution. I don't know the details of how CPS handled swap-in/swap-out ... it is possible that they controlled a lot more of program execution ... they could control the instruction image not having embedded location specific information and knew what registers in the private execution context needing swizzling when swap-out/swap-in sequence didn't bring back to the same swap address.

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/

had

know

point

physical

them

Execution of what? An instruction, sure. But not a program!!

You have to have them atomic if you want to be able to sleep the system and have it continue where it left off. This was one of the features of TOPS-10's SMP implementation. You could issue the commmand SYSTEM SLEEP, the monitor would tuck itself in and store its current context on disk, the operator could add and remove any hardware he wanted to, and then resume timesharing and the monitor would brush itself off and continue from the point where it was temporaily stopped. We did not reboot!!!!

The point is to be able to reconfigure on the fly without causing all programs to have to restart.

So? Not all interrupts would affect this calculation.

There is lots of value. You guys seem to be confusing OS execution, user program execution and instruction execution.

/BAH

Subtract a hundred and four for e-mail.

[...]

And this discussion dredges up memories of the one type of OS/360 executable that was relocatable on the fly: the transient (Type 3/4) SVC. Typical systems had multiple transient areas, and the requirements for a type 3/4 SVC (there wasn't really any difference between the two types) included a prohibition against any relocatable constants, a requirement of absolute reentrancy, and a fixed (R12?) base register.

A transient SVC might start execution in one transient area, then surrender control (for example, to do I/O) and resume execution in a different area. The transient SVC handler was responsible for setting the base register to the appropriate value each time a transient SVC was dispatched.

Joe Morris

In article , Joe Morris writes: |> |> A transient SVC might start execution in one transient area, then |> surrender control (for example, to do I/O) and resume execution in |> a different area. The transient SVC handler was responsible for |> setting the base register to the appropriate value each time a |> transient SVC was dispatched.

And the author of the SVC for not saving an address of any of its code across potential dispatching boundaries!

Regards, Nick Maclaren.

what i remember from mft/mvt days was that there were (initially?) two, 2k transient svc areas. part of the issue was that the loader wasn't involved in reading code into the transient area ... so there wasn't any ability to perform swizzling operation on any (relocatable) address constants.

--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/