"Boost context" for task switching on embedded Linux on ARM

Do you have a question? Post it now! No Registration Necessary

Translate This Thread From English to

Threaded View
Following on from my recent question about PThreads I have a new question a
bout Boost:context
https://www.boost.org/doc/libs/1_67_0/libs/context/doc/html/index.html

We have some legacy code(Motorola 68000 C) that we want to use that has coo
perative task switching.  We want to run it all as a single thread and keep
 its co-operative organization within that thread.  Is "Boost context" a re
asonable way to do this?  If not, do we have to figure out what registers a
nd state we have to save/ restore and hope that we get everything?

Alternatively we could run each task as a thread and enforce round robin be
haviour but that seems a bit wasteful.

Re: "Boost context" for task switching on embedded Linux on ARM
On Fri, 3 Aug 2018 05:52:01 -0700 (PDT), snipped-for-privacy@gmail.com wrote:

Quoted text here. Click to load it

Boost Context requires C++ 11. You may be facing a lot of work even to
get your old C code to compile.


Quoted text here. Click to load it

How was the tasking accomplished before?  OS?  [which one?]  Or
something in the code?


Quoted text here. Click to load it

That won't necessarily work.

In cooperative tasking, the current task runs until it explicitly
yields the CPU.  Often shared data is manipulated without locks
because the programmer knows no other task can run and interrupt the
operation.

Running such a system under a pre-emptive task switcher is going to
fail eventually regardless of the scheduling policy.


George

Re: "Boost context" for task switching on embedded Linux on ARM
On 18-08-04 10:28 , George Neuner wrote:
Quoted text here. Click to load it
    ...
Quoted text here. Click to load it

If the tasks cooperate, round-robin execution can be enforced even if  
the scheduler is "basically" pre-emptive. One just defines a mutex (with  
a waiting-task queue), and each task takes the mutex, does its job,  
releases the mutex, and repeats. If the mutex queue is FIFO, the result  
is round-robin execution of these cooperating tasks, no?

Of course no task outside this cooperating task group should access the  
unprotected, shared data.

--  
Niklas Holsti
Tidorum Ltd
We've slightly trimmed the long signature. Click to see the full one.
Re: "Boost context" for task switching on embedded Linux on ARM
On Sat, 4 Aug 2018 11:13:02 +0300, Niklas Holsti


Quoted text here. Click to load it

That will work if the system really does queue tasks waiting on the
mutex.  But some systems simply wake all waiting tasks when a shared
resource is freed - which task gets it then is happenstance.  If
taking the mutex is a free-for-all, there is a chance that some tasks
may starve and never get it.

There also is the possibility with cooperative tasking that tasks may
sometimes need to run in a particular order by directed yielding.
Emulating that under a pre-emptive system requires something like
message (token) passing.

The OP did not specify what system is being used.

Using a mutex will solve the shared data coordination problem - the
task currently holding the mutex will be preempted for rescheduling,
but because all the other tasks are waiting for the mutex, the one
holding it will be scheduled to run again.  
[However, it may not be run immediately if there are other runnable
tasks in the system not participating in the mutex.]


Quoted text here. Click to load it

Absolutely.


George

Re: "Boost context" for task switching on embedded Linux on ARM
On Sat, 04 Aug 2018 07:08:37 -0400, George Neuner

Quoted text here. Click to load it

Oh, sorry ... the OP did specify Linux.

I'd have to double check, but I think Linux does wake all processes
pending on an OS semaphore and lets them fight for it.

George

Re: "Boost context" for task switching on embedded Linux on ARM
On Saturday, August 4, 2018 at 11:22:25 PM UTC+12, George Neuner wrote:
Quoted text here. Click to load it

Thanks for the replies.  I'm using google groups - hope it's readable.

Regarding C++11 - we'll be using GCC 7.  The guy who wrote the code was C++
 aware and it won't be hard to get it to compile - a lot easier than re-wri
ting.

If we make each co-operative task a proper thread then I was thinking we wo
uld suspend the thread when it does a task switch and the task switch funct
ion would release a mutex to allow other "non co-operative code" to access  
the data being generated by the co-operative threads.  The task switch func
tion would resume the next co-operative thread in round robin fashion.  The
 legacy code just saves each task stack pointer in an array and cycles arou
nd them.

It seems that Linux provides a switch_to and context_switch function that c
an be used for co-operative threading but I've been unable to find any sour
ce code (for ARM 9 or ARM anything) to compare with what boost::context doe
s.

I was hoping to avoid putting mutexes all through the legacy code.


  

Re: "Boost context" for task switching on embedded Linux on ARM
After reading your comment about jmp_buf in the other thread I guess we can use setjmp, longjmp to save / restore context.  I'm not sure why we're not already using it.

Re: "Boost context" for task switching on embedded Linux on ARM
On Sunday, August 5, 2018 at 12:25:50 AM UTC+12, snipped-for-privacy@gmail.com wrote:
Quoted text here. Click to load it

After investigating now I see what you mean when you say "if the jmpbuf structure is documented".  So this is not very easy and not very portable.

Re: "Boost context" for task switching on embedded Linux on ARM
On Sat, 4 Aug 2018 06:01:30 -0700 (PDT), snipped-for-privacy@gmail.com
wrote:

Quoted text here. Click to load it

setjmp/longjmp were designed originally to enable abandoning an errant
computation by jumping back to state of the program that existed
before the computation began.  Using it to jump around within a single
thread is relatively easy.

Multi-tasking with it is another issue.  It's generally pretty easy to
set up a new stack, but initializing all the other CPU state for the
new "task" can be problematic.  It was easier to do with simpler CPUs.
It still can be done with a modern CPU, but it is much more difficult
to get everything right [the more so for lack of documentation].

George

Re: "Boost context" for task switching on embedded Linux on ARM
Quoted text here. Click to load it

This sounds pretty horrendous and compiler and hardware dependent.  You
are basically writing a miniature OS in your scheduler.  It helps if the
compiler docs explicitly say it's supposed to work.  Otherwise, if you
reverse engineer it for a particular compiler version, all bets are off
for the next one.

There are some very lightweight RTOS out there and it's probably saner
to just use one instead of a hack like this.

Re: "Boost context" for task switching on embedded Linux on ARM
On 18-08-06 21:21 , Paul Rubin wrote:
Quoted text here. Click to load it

Simplest robust solution I can think of is to create one semaphore for  
each of the round-robin threads, initialise all semaphores but one to  
zero, and initialise the semaphore for the first thread that you want to  
run, to one (1).

Each thread starts with "taking" its own semaphore, which means that  
only the chosen "first thread" actually starts running, and the other  
threads all block on their semaphores. When a thread wants to yield to  
some other thread, it calls a function that:

- "signals" the semaphore for the thread that should run next (how ever  
you want to choose that thread), and then

- "takes" the semaphore for the current thread (the one that is  
yielding), which blocks this thread until someone signals this semaphore.

In this way, you can implement either round-robin execution or  
"directed" yields to a specific thread, because you control which  
semaphore is signalled next.

Assuming that the base system is Linux, or some competent RTOS, I would  
never mess around with jmp_buf or some system-dependent threading libraries.

--  
Niklas Holsti
Tidorum Ltd
We've slightly trimmed the long signature. Click to see the full one.
Re: "Boost context" for task switching on embedded Linux on ARM
On Tuesday, August 7, 2018 at 7:48:33 AM UTC+12, Niklas Holsti wrote:
Quoted text here. Click to load it

Thanks for the suggestion.

Re: "Boost context" for task switching on embedded Linux on ARM
On Mon, 06 Aug 2018 11:21:23 -0700, Paul Rubin

Quoted text here. Click to load it

???  This is *userspace* thread switching - not kernel thread or
process switching.  

Do you not remember using LWT thread libraries - e.g., cthreads -
before operating systems had kernel threads.  What do you think they
did?



Quoted text here. Click to load it

Actually, the C standard says it works - at least if you read between
the lines.  7.13 guarantees that CPU state[*] sufficient to restore
the calling environment of setjmp will be preserved in the jmp_buf
structure, and that that state [modulo setjmp's return value] will be
restored by calling longjmp.

Although the standard does not address it's use in multitasking, the
description of the operation of setjmp/longjmp matches quite well the
description of a register state task switcher.


The hard part of using it for multitasking is that compiler vendors
often don't document the jmp_buf structure very well because the
standard explicitly speaks only to its use within a single thread, and
that use does not require modifying the structure data.

Generally, setting up a new stack is pretty easy.  The hard part is
figuring out how to set the saved instruction pointer so as to
properly enter your new "thread" function. [minimum you need to know
if/how your CPU autoincrements addresses.]

Once you get past that hurdle, switching "threads" is as simple as

  if (setjmp(&mystate) == 0)
    longjmp(&yourstate,1);

and "scheduling" is just picking the next jmp_buf to restore from an
array or list of jmp_bufs that represent your runnable "threads".


[*] 7.13 says explicitly that the jmp_buf does not record the state of
FPU status flags.


Quoted text here. Click to load it

I am NOT advocating use of setjmp/longjmp to implement multitasking:
getting a "thread" to start properly can be tedious and error prone
unless the compiler documentation is good.  The OP mentioned it as a
possible solution to his problem, and I was merely explaining more
about it.

However, Boost::context must be doing something extremely similar, and
likely it is simply building on top of setjmp/longjmp which already
are provided (yes, C++ has them too).


George

Re: "Boost context" for task switching on embedded Linux on ARM
On 18-08-07 22:10 , George Neuner wrote:
Quoted text here. Click to load it
    ...
Quoted text here. Click to load it
   ...
Quoted text here. Click to load it

C11 says it is undefined -- see below.

Quoted text here. Click to load it

I don't remember which version of C the OP intends to use, but the C11  
draft (n1570.pdf) contains this text in section 7.13.2.1, for longjmp:

"The longjmp function restores the environment saved by the most recent  
invocation of the setjmp macro in the same invocation of the program  
with the corresponding jmp_buf argument. If ... the invocation was from
another thread of execution ... the behavior is undefined."

--  
Niklas Holsti
Tidorum Ltd
We've slightly trimmed the long signature. Click to see the full one.
Re: "Boost context" for task switching on embedded Linux on ARM
On Tue, 7 Aug 2018 22:41:18 +0300, Niklas Holsti

Quoted text here. Click to load it

Yes.  setjmp/longjmp is not *intended* to be used for multitasking.
That has made clear already in C89/90.

However, almost every C program does things that technically are
"undefined" or "implementation defined" according to the standard. And
things get used for more than they originally were intended.

The point is not whether it's legal, but whether it works.  You can
multitask in C with setjmp/longjmp if you do it right.  Obviously its
better to use the standard approved thread API in C11, but users of
previous versions did not have that luxury: whatever they did using a
thread library or OS thread API was "undefined" wrt C.


Userspace threading in C has been done almost since the beginning: the
1st version of what later became the portable cthreads library popped
out in 1981.  The portable cthreads API (circa ~1990) was one of the
inspirations for pthreads.  

cthreads itself was based on setjmp/longjmp.

George

Re: "Boost context" for task switching on embedded Linux on ARM
On 18-08-08 07:42 , George Neuner wrote:
Quoted text here. Click to load it

True, but not desirable.

Quoted text here. Click to load it

You can multitask in C with foo/bar if you implement foo and bar  
"right". Sure, for a simple thread context setjmp/longjmp can have some  
of the required properties of foo and bar, but may also have some  
undesirable properties.

The undefinedness of a cross-thread longjmp can be far more serious  
than, say, shifting or overflowing a signed integer. A longjmp is  
intended to *abandon* the current execution point and return to a point  
*lower* (earlier) in the call stack. Some C or C++ systems might  
implement this by unwinding the stack, frame by frame, until the setjmp  
frame is reached. This will have drastic "undefinedness" if the setjmp  
refers to another stack...

It is not advisable, today, to try to misuse setjmp/longjmp for thread  
switching.

--  
Niklas Holsti
Tidorum Ltd
We've slightly trimmed the long signature. Click to see the full one.
Re: "Boost context" for task switching on embedded Linux on ARM
On Wed, 8 Aug 2018 08:21:08 +0300, Niklas Holsti

Quoted text here. Click to load it

But none do ... every implementation I have seen leaves the stack data
intact and simply resets the CPU register(s).

longjmp is NOT equivalent to throwing an exception, and, in fact, it
is dangerous to use it that way in C++.  Using longjmp will NOT cause
destructors to be called for local objects on the "abandoned" stack.
It exists in C++ for only for compatibility with C.  


Quoted text here. Click to load it

I already said that I don't advocate doing it, and that I merely was
explaining it to the OP.  

Maybe you prefer to conceal information about things you don't like,
but I prefer to educate people and let them make up their own minds.

George

Re: "Boost context" for task switching on embedded Linux on ARM
On 18-08-09 17:55 , George Neuner wrote:
Quoted text here. Click to load it

Ok, but that is not proof that "none do".

Quoted text here. Click to load it

You cannot know if it will, or won't; it's explicitly undefined in the  
C++ standard.

Quoted text here. Click to load it

We agree.


Becoming irritated, are we?

I'll stop here.

--  
Niklas Holsti
Tidorum Ltd
We've slightly trimmed the long signature. Click to see the full one.
Re: "Boost context" for task switching on embedded Linux on ARM
On Sat, 4 Aug 2018 04:49:42 -0700 (PDT), snipped-for-privacy@gmail.com
wrote:

Quoted text here. Click to load it

That's encouraging at least.


Quoted text here. Click to load it

Just a word of caution: suspending a thread from the outside can cause
problems if done at the wrong time.  It's acceptible for a thread to
suspend itself, but apart from debugging there really are no good
reasons for one thread to suspend another.

It's ok for a thread to be in a "suspended" state while waiting for
some event that makes it runnable again.  Just make sure to
distinguish the state from the action.


Quoted text here. Click to load it

Sorry, I can't you help you there - I've never used those functions.

It might help to take a look at some of the userspace thread libraries
that are available.  In particular, the old "cthreads" library [which
predates kernel threading in Unix] was designed to be portable - if
you can find it, it might give you some good ideas.

George

Re: "Boost context" for task switching on embedded Linux on ARM
On Monday, August 6, 2018 at 8:04:22 PM UTC+12, George Neuner wrote:

Quoted text here. Click to load it

ok, thanks.  I'm not sure if the code below will work correctly  - it's not as simple as I thought.  I suspect I need a supervising thread.  Maybe boost::context would be cleaner and simpler.

void task_switch()
{
   release_mutex();
   suspend_me_for_a_while();
   acquire_mutex();

   if ( ++task_id >= max_tasks ) {
      task_id = 0;
   }
   switch ( task_id ) {
   default:
   case 0 : resume_thread(0); break;
   case 1 : resume_thread(1); break;
   // ...
   }
   suspend_me();
}

Site Timeline