I am currently looking for a solution to measure the application load on my embedded system. An approach I though about was hacking the RTOS I am using and every time the idle task is scheduled I count up the time in this task for a given time frame (e.g. 1 second). Having finished that I could use the accumulated time to calculate something like a system load with:
load = ( TOTAL - ACCUMULATED ) / ( TOTAL )
The drawback with that is that I need to hack the scheduler. Are there any better options for that? Specially I am looking for options where I don't need to hack the RTOS.
Not an easy task, since 'load' is something that's very closely related to the scheduler: only the scheduler knows which tasks runs when, so only the scheduler can do the proper book-keeping to calculate the load.
A trick you could try is to add some code to a timer interrupt that peeks into the data structures of your scheduler to see which task was running at the time the ISR was entered. Sometimes it's the idle task, sometimes its another task, and sometimes it might even be your kernel or scheduler itself. You can calculate the system load from the ratio between these peeks.
Other solutions might be possible, all depending on your exact platform, of course.
The source code for the RTOS is available and in my case is FreeRTOS and our company own operating system. In both cases it is possible to modify the RTOS code although not desirable. In case of FreeRTOS the periodic timer looks simple because I could could simply compare pxCurrentTCB (the current task control block pointer) to that one of the idle task and increment a counter if it matches. After some ticks I would calculate the load from these values. Drawbacks are (in my opinion):
- It is possible to miss something. - The time is not always equal because a task might not use its full time cycle. For example if it blocks or is preempted by another ISR. - Overhead by the ISR our would you suggest using the preemptive tick interrupt?
Are there any experience values about the possible accuracy of this method?
I am sorry but I do not understand the solution. An idle task hook is already available in the OS but how can I measure the time within this task. Counting the number of times the task has been scheduled is difficult in my opinion within the task because how should he know that. Maybe it is possible by storing the last tick and repeatedly checking if it has changed. If it has changed I could increment a counter and after a certain time the load could be estimated by:
Drawbacks are that getting the tick counter from the OS disables interrupts because it must be done atomic (At least in the OS I use) and therefore delays other interrupts - although I have to admit that this time is very short it is non the less not very nice.
Maybe you could explain your suggestion in a bit more detail because it is not obvious to me.
An approximatation of the time spent in each task is not too difficult to come by, periodic sampling can be used or you can simply look at the time each context switch. A very accurate measurement is harder to come by.
When using the tick count as a measurement you are always going to come up against the limitation of the tick resolution. As you say, tasks can be swapped in and out within a tick several times even when running at 1KHz tick frequency. You could instead use a free running timer (either by reading the timer used for the capture/compare for the tick [probably not feasible], or by using a separate timer peripheral) to measure time much more accurately. Either way you are adding quite a bit to the context switch time with the extra code that is required. I have successfully used a separate timer for accurate calucaltions in the past.
You can safely add a function to read the time directly, without the critical section, provided the reading of the tick value is atomic. This is normally the case if the tick value is 32bits and you are using a 32bit architecture for example, but not if the tick value is 32bits and you are using a 16 or 8 bit architecture.
One other technique I often use to get confidence in the amount of idle time - without actually getting an accurate numerical measurement - is to use the idle hook to toggle an IO pin very rapidly. Using a scope it is then simple to visualise the behaviour, without actually getting a numeric value.
One final note. The trace visualisation utility within FreeRTOS.org can be used to take a recording of which task is running when. You can then draw graphs and perform the necessary calculations within Excel (or whatever spreadsheet you use). This requires a fair bit of RAM on your target hardware to get a trace buffer large enough to provide meaningful results. By default the utility uses the tick count so you get a resolution problem again - the trace buffer will recognise switches within the same tick period but cannot accurately measure below the tick period. You could change the default behaviour to use a separate free running timer for the logged values fairly simply.
You create a task with a priority above the systems idle task. This way this task get all time and the original system idle gets none. In this task you count while comparing the realtime clock, however this may be done. Eg count until the realtime clock changes. Or similar. You just have to calibrate the counts with two measurements. One taken when there is zero load and one when the system it tight.
IMO, the system idle task should be configurable. At least between loadmeasurements and powersave. Two standard functions.
Why not simply create a lowest-priority task that spends all its time incrementing a 32-bit variable. If you have a fast clock signal (much faster than the RTOS scheduling tick) that it can observe and synchronize to, use that and count every tick; otherwise just increment and loop, free-running. At the end of 1 second, compare the resultant count with a known zero-load count value you have previously determined if no other tasks run. If you are synchronizing to a clock signal, calculating this zero-load value is simple math. If you are just doing a free-running up-counter, determine the zero-load value by modifying your code and letting the counter run for a full second with no stopping (either explicitly disable the other tasks, or set the counting task to be highest priority, then let it count for a second to get your reference count). Then hard-code that zero-load value into your regular software. At the end of each 1-second measurement, clear the counter variable and start again.
I think the idea is to measure the "time" spent in the "idle" task not how many times it got scheduled. Since you mention the OS provides a hook to an idle task, I presume you could provide a pointer to your task which maintains a counter. The counter need not always be implemented using a timer, you could have a reasonable estimate by incrementing a counter with a delay loop.
Look up Emmbedded System Design June 2006 Jack Ganssle "eXtreme instrumenting"
One suggestion was to find an extra I/O pin, Raise it at the beginning of the idle task and dropping it at the end. Then hook the pin to a scope or analog meter.
This may measure overall system loading. But remember that in a real-time embedded system, the overall loading is only one aspect of how close you are to running out of time. Your processor can be idle for over 50% of the time and still fail to respond to an external event in a timely manner due to too many demands being placed on the CPU at the wrong time. Overall loading is fine for systems where all exteral events are buffered, but it means little on a system with real-time requriements.
I just wanted to thank everyone who proposed solutions for my problem. I found some more time today to try some of the suggestions and wanted to let you know about my solution (Basically based on the ideas from Mike Silva and others).
I created a free running counter where I know the number of ticks within a given time interval.
I created an additional task in the system which has lower priority than all the other tasks.
This tasks periodically checks if there was enough time between the last calculation and now (for averaging). If the time has passed the load is calculated by
where ulDeltaTicks is the time between the calculations and ulIdleTicks is the number of ticks successfully monitored in this task by a busy loop.
The term "enough time" refers to the variable ulRequiredTicks which in my case is calculated from the timer frequency and was chosen to match one "real" second.
The counter is monitored continuously and if changed the value of ulIdleTicks is incremented.
For reference the relevant parts of the code are shown below. The meaning of the defines should be clear from the context.
/* If enough time has elapsed try to estimate the load. */ if( ulDeltaTicks > ulRequiredTicks ) { ucLoadEstimate = ( unsigned char ) ( ( ( ulDeltaTicks - ulIdleTicks ) * 100UL ) / ulDeltaTicks ); ulIdleTicks = 0; ulLastCalcTick = ulCurrentTick; } else { /* If the timer has changed we monitored the event. Therefore * is is likely that the processor time was spent in the idle * task. */ if( ulCurrentTick != ulLastTick ) { ulIdleTicks++; ulLastTick = ulCurrentTick; } } } while( 1 )
The canonical way to do this is to set a GPIO on entry to the idle task and clear it on exit. Connect a voltmeter to the GPIO. You get a "PWM" system load indicator. Vcc=no load. 0V=saturated.
Ooog. You could then connect _that_ pin to a resistor and capacitor pair chosen to match a given measurement period (e.g. "What's my system load, averaged over ten seconds?").
Taking this one step further, you could hook the output of this integrator to an A/D input pin on the same controller for logging purposes.
Frank McKenney, McKenney Associates Richmond, Virginia / (804) 320-4887 Munged E-mail: frank uscore mckenney ayut minds pring dawt cahm (y'all)
-- Sacred cows make the best hamburger. -- Mark Twain
ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here.
All logos and trade names are the property of their respective owners.