Progress indicators (reprise)

This is the real problem with trying to do any kind of time estimation as the status bar, the inherent non-monotonicity borders on the cruel.

Why would you say that bytes transferred is bogus? It's a real physical quantity that has the advantage of "improving" monotonically. You can do like my browser does and provide a text estimate of completion time as supplemental data, but if the question is "How complete is the task" than bytes seem to map onto the concept perfectly.

--
Rob Gaddi, Highland Technology
Email address is currently out of order

Actually, showing the progress by amount, and trusting the user (me) to be smart enough to figure out that sometimes things happen fast and sometimes things happen slowly, is what I want.

Let's repeat:

Give me a monotonically increasing progress bar. If you want, give me a time estimate, too, but give me that progress bar that works the way I expect. That way _I_ can monitor the progress, and _I_ can decide how much I want to believe the time estimate (if there), and _I_ can decide whether to kill the process.

--

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" was written for you.
See details at http://www.wescottdesign.com/actfes/actfes.html

ROTFL! Yes, that's about it!

But people don't care about "number of bytes". That might tell them how much of the file has been transferred but what they really want to know is "how much longer will I have to wait before I can {use the file, disconnect the link, give up on this insanity}?" So, the number of bytes transferred really has three practical values: "none", "some" and "all". Since

*part* of a complete transfer is typically the same as *none*, the fact that you have 98% of the bytes doesn't offer much reassurance -- unless you have a high degree of confidence in the estimator!

Every time I see one of these indicators, the boilerplate warning that accompanies financial instruments whispers in the back of my mind: "Past performance is not an indication of..."

What is so frustrating is that this seems like it is an insignificant thing -- something that should be *easy* to convey to the user. Yet coming up with any useful way of presenting the data relies heavily on the user being able to interpret that data. Imagine if a car's speedometer expressed your velocity in terms like the temperature at which maple syrup would flow at

1/10000th your current speed (i.e., at 0C, it would be flowing very slowly; at 25C, considerably faster). Studying the "estimated time remaining" on most of these indicators and trying to figure out what they *really* are attempting to say (in light of wild gyrations in the predictions) requires so much effort that it renders them practically useless!

(When moving huge files around here, I'll see estimates that range from *days* right down to hours or even minutes and then back up again)

And, if you're doing something whose "progress" isn't as easily monitored as, e.g., transferring a file, then that just compounds the issue! (consider, for example, sorting a huge database... how do you report your progress through it when you don't know how much more "work" -- moving records, etc. -- lies ahead?)

IMHO there are two problems with that whole train of thought

a) this initial premise that set you off on this journey is plain and simply wrong. b) you insist on getting a result that's quite evidently impossible

There's absolutely nothing bogus about that metric, and nothing wrong with actually displaying it as-is. If you want to, print time elapsed along with it, so people who really think they need to estimate finish time can do a rough calculation of

time_elapsed * size_left / size_done

in their heads. Sometimes, this estimate, like any other, _will_ just be wrong. If that happens, it happens, and no amount of clever thinking is likely to help with that. As the saying goes: all predictions are hard --- even more so those involving the future.

The ultimate question is: what _do_ you want your indicator to indicate?

1) completion ratio 2) speed (average, current, or something in between?) 3) time elapsed 4) guesstimated time to completion (as in: 5 minutes left) 5) guesstimated finishing time (as in: will be done at around 15:37h) 6) some mixture of the above

No indicator can hope to transport all this information at once without getting wildly confusing.

And how did you do that, if not using exactly that number you just declared "completely bogus"?

Won't work. You don't get to choose how much space there is for the bar in total to consume.

That would be a considerably bigger misrepresentation of the fact (I'm avoiding the word "lie" there) than the plain completion ratio display. That contraption would visually claim that the _job_ size suddenly had gotten bigger, which it rather obviousy didn't.

What do you consider "amount"? What if you don't know what the *total* "amount" will be? E.g., consider sorting a file "in place" with limited resources (i.e., accesses to the file are expensive and you don't have any place "fast" to hold temporaries in -- you can't sort it in memory and then write it back out). When you start, all you *might* know is the size of the file (assuming your filesystem exports these things). You, as yet, have no way of knowing how many "entries" (whatever those may be) are present in the file. At the end of the first pass through the file (which might take some considerable time to parse), you will probably have answers to these first two questions. And, maybe you have developed some metrics that will help you formulate your first metric (e.g., average size of each entry is XXX, min is YYY and max is ZZZ). Now, given that you are sorting in place, any time you "move" an entry, you have to make room for it by displacing some other entri(es) [think about this... what if the entry you are moving displaces 2.573 other entries... how do you track that 0.427 entry that is yet undisplaced? I.e., this can get expensive very quickly]

Now, what's the "amount" that you report to the user? Number of entries now in their "ultimate" sorted positions within the file? Number of bytes that they represent? etc.

And, you have no idea what the remaining "effort" will be like. Perhaps the first and last entries were the only two that were out of place. Perhaps the file was already sorted to begin with. Perhaps everything is exactly wrong. Or, some combination of these.

*If* the user is familiar with the characteristics of the sorting algorithm being used, he/she maybe able to gauge the type of behavior he/she expects to see in the future.

What if the user is your neighbor's 12 year old kid? Or, your grandmother? Do we only provide usable information to people who can *infer* from it the *possible* implications?

Great! "Time spent so far:" guaranteed always to be increasing in value!

And how will you *know* if you should kill it? How will you know if you have killed it prematurely? What if you will have to complete this SOONER OR LATER? Does that affect how willing you are to abort it "in progress"?

Repeat for the other individuals I mentioned above...

A status bar represents an implicit percentage of work completed. If you haven't the foggiest of how much work there is to do, and don't want to do something crude like "There are three tasks to be performed therefore each one is 33%", then it becomes the wrong metaphor.

If you're performing an open ended task with no idea of when completion will happen then I'd suggest - / | \ - / | \ some means of simply indicating that the process hasn't keeled over and died. It's the only credible piece of information you have to offer.

And by the way, I spend each night cursing those designers who decide that, in order to simplify the code, the actual task processing and the display that the task hasn't hung up should be run in separate, non-communicating threads.

--
Rob Gaddi, Highland Technology
Email address is currently out of order

[8>>> Some preliminary thought on alternative approaches

Note that the subject of the thread is "progress indicators". Unfortunately, people most often relate this to the "bar" that accompanies file transfers as they rarely see it in any other context (have you seen a "progress indicator" displayed when you type "make world"? Wouldn't you *really* like to know how far along in *that* "process" you were at any given time??)

How you indicate progress and what metrics you use to express it can vary wildly -- even in the same application! E.g., dollars that the factory service guy is costing your company waiting while you download this file; number of dropped connections while machine was busy doing this; amount of lost production while machine was tied up; etc.

The problem is how do you provide enough of the *right* information to the user so he knows how best to plan his use of resources (time, equipment, etc.). How do *you* know when to "fish or cut bait"?

But there are other tasks between "fixed, known" and "open ended". E.g., the file sort *has* an absolute upper bound that can be determined solely from the file size and minimum record/entry size. Of course, this may be considerably higher than any *practical* "absolute upper bound" but it is knowable -- even in the case of sparse files, etc.

If you are working on an open ended task, then you should display an indication of the *rate* at which you are "working". This allows the user to make some crude assessment as to whether you *are*, actually, working and how *effectively* you are doing so (e.g., a meter on your network connection tells you whether you might want to "try again later" if throughput is low...)

There are times when this is actually a blessing! E.g., my *ss has been saved many times when I have closed browser while an FTP transfer was "in progress"!

The worst are those interfaces (often in UN*X world) where an "idiot light" task runs telling you that some other task is "still working" -- and you know, for a fact, that the idiot light task has no clue as to what the real task is doing or even if it actually *is* still working! (e.g., progress bars that grow from left to right... then disappear and repeat. Endlessly. Telling you nothing more than "this computer has power applied and is not halted")

That there is the one I was so strenuously objecting to. When the underlying process has long since died horribly, but the status indicator just keeps lying to you. One of my favorite problems with the FPGA tools from a vendor that I won't name, but rhymes with Mylinx.

--
Rob Gaddi, Highland Technology
Email address is currently out of order

IIRC, I saw it used in a Solaris install recently (and possibly even a *BSD install... "I forget" :< ) I think the situation (problem?) in these cases is they wanted to retrofit a progress indicator to some "process" that wasn't designed with the capability of "reporting progress" in mind.

There are other equally brain-dead approaches. E.g., the ftp(1) "hash" command (for many of the file sizes "common" nowadays). What good is a screen full of '#' and how does that tell me anything?? (except that my xterm(1) hasn't died)

If you provide a visualisation of the raw data, the user can make their own estimate as to the time remaining, along with a margin-of-error estimate.

The user can see the position of the bar and how fast it's moving right now, and will have some recollection of how fast it was moving earlier, whether it progressed at a steady rate or in fits and starts, and so on.

Which is why I have no interest in the software's estimate; just give me the raw numbers and I'll make my own estimate. Not only do I have more information than the program does, I also have more control (e.g. if I want a download to finish earlier, I can pause some other downloads).

Just report the total number of results found so far. If the user has a rough idea of how many results they're expecting, this provides useful information. Other than that, there isn't actually anything you /can/ do.

My pet peeve with the Windows copy dialog isn't that it provides a (typically) bogus ETA, but that it wastes time scanning the disk just so that it can provide this information. Just get on with already; it will be done when it's done.

What "result" do you think I am trying to get? (see below)

Not everyone will be able to do that "in their heads" -- or even ON PAPER! Do you make a note of when you start EVERY OPERATION that might be time consuming (so that you have an accurate value for "time_elapsed")? Yet, I'm sure *you* regularly make decisions about what things to abort, reschedule, etc.??

The *prediction* isn't the core issue. There are ways you can come up with MUCH better "predictions" (depending on the "task" at hand). This is especially true if the device that is making the predictions considers "making predictions" as one of its key design goals.

Rather, the problem is conveying information to a user that enables that user to make "informed" decisions as to how to allocate their resources (time, money, etc.) in light of these predictions. Ultimately, my goal is to be able to make those decisions ON BEHALF OF the user!

However, before you can create an algorithm to *do* something, you need to understand how *you* would do the same thing (even if you do it poorly).

Unfortunately, the only "progress indicators" we tend to come in contact with regularly are the banal "file transfer" indicators. Or, similar indicators calibrated in bogo-units ("bytes transfered", "scan-lines rendered", "files copied", etc.). None of them intuitively map onto our "experiences" in predictable ways (they aren't "natural").

E.g., I suspect you could come up with an accurate prediction of how long it would take you to mow the lawn, or shovel the driveway... *despite* the variables that can creep into those "tasks" (how high is the grass? is it damp? will I have to refuel the mower before completion? is it a *wet* snow? etc.). As organic beings, we can relate to these "metrics". At this very moment, I can *feel* what a shovelfull of wet snow feels like in my shoulders -- despite the fact that it is ~90F today. I know how big the shovel is and the innumerable times I will have to heft it to complete the entire driveway -- even though it has *never* snowed, here!

At any time during one of these tasks, I can easily evaluate how much time likely remains, whether or not it would be a good time *now* to take a break, what the cost of abandoning the task for a later time/date is likely to be, etc.

I contend that the average person is NOT equipped to make intelligent decisions INTUITIVELY about similar issues in a computing device. "If I abandon this download, now, will it require MORE or LESS time to try again later?" "If I opt NOT to recharge this device's battery now, will the remaining charge be sufficient to address my needs (whatever those are likely to be) until I *next* have an opportunity to do so?"

Obviously, I believe that said "computing device" can easily provide additional information to help that user make those decisions. And, that the device could ultimately make some of those decisions *for* the user thus improving the "usage experience".

If a task is resumable, you can apply different criteria to it than a task that must be reSTARTed. If a task can not be abandoned, the criteria are different, still.

Imagine updating the firmware in a device and *then* discovering that you don't have enough battery life left to complete the task (and now own an expensive "brick"). Ignore the quality of the "battery gauge" for the moment... are you confident that you can look at that gauge and decide whether such an update will be "risky" at this moment? Or, do you err on the side of ensuring that the battery is "100%" before attempting it? If so, your usage pattern of the device is being needlessly constrained by lack of good information (either a mistrust for the accuracy of the battery gauge or incomplete knowledge of the actual power requirements for the update process, etc.)

You'd be annoyed if the device didn't make every attempt to provide you with a "safe" means of performing this update:

"Why the hell couldn't these folks put a BATTERY GAUGE in the device so I *knew* the state of the battery before starting the update? Otherwise, I have to charge the device for a full 8 hours JUST TO BE SURE..."

"Who's the idiot that thought a 3 valued (EMPTY, PARTIAL and FULL) battery indicator was useful for *anything*?? Unless it says 'full', I have no way of knowing if the battery is likely to be EMPTY 2 seconds from now -- in the middle of my firmware update!"

"Why couldn't they tell me how much *time* is left in this battery?"

"Why couldn't they tell me if there was enough charge left in the battery to perform an update??!"

None of the above! What I want is something that helps a user decide what *he* should do with the device, *now*. I want that information to be fine-grained enough that the user can then *change* "things" and see their consequences.

E.g., sticking with the unfortunate "file transfer" activity, a user should be able to cancel/pause other bandwidth consumers and see how that affects "progress" on this "other" activity. Or, free up some other resources. Or, perhaps *use* some other resources to get some OTHER thing done (possibly letting the "file transfer" incur a decrease in performance)

Who cares "how"? The design of the estimator is not the issue.

I track the amount of fuel remaining in my car by noting the number of miles traveled -- not by glancing at the "fuel gauge".

When traveling cross country, I track the distance traveled by noting the elapsed *time* -- not by glancing at the odometer.

When copying a file to a USB disk, I equate GB to minutes -- and don't bother watching to watch the "bytes transferred".

Sure you can! That's a function of the task at hand. Not all tasks have "unbounded completion time estimates".

E.g., if I gave you a shovel that "held" half a cubic foot of "soil" and asked you to dig a hole 1 yd deep, wide, long (i.e., "54 shovelfuls") and you knew that the soil was some combination of caliche (clay -- very hard to remove) and *sand* (effortless to remove) *and* that it would take you 5 seconds to remove a shovelful of sand and 20 seconds to remove a shovelful of caliche, you *know* that this task will take you no more than 1080 seconds. Even though you don't know whether it will be closer to 270 or 1080.

You can make an educated guess as to what is likely to lie beneath the surface -- maybe 50% sand and 50% caliche (based on past experience -- since you are an expert digger!).

While you are digging, you can report how much material you have removed -- shovelfuls. This is a monotonically increasing value. It never "shrinks" -- in reality nor "in the progress indicator".

Yet, your estimate of the "work remaining" is free to change dynamically. Sure, you always know how many

*shovelfuls* remain... but, to me, the homeowner, I only want to know *WHEN* you're going to finish so I can arrange to have the shrubs brought around to be planted (I'd hate to have them sitting -- roots exposed -- in this hot sun if you are likely to encounter a large caliche deposit).

The indicator I described would work very well in this situation. The psychological cringe associated with a "shrinking" progress indicator is eliminated. The "total estimate" indication growing and shrinking gives me feedback as to the estimate of remaining time -- so, even if you may have removed 27 shovelfuls of *sand* (50% of the "volume"), I can see that you may only be "20% done" if the balance is determined to be entirely caliche!

To the user, the size of the job isn't the number of bytes.

If the transfer is not "resumable", then, in a very PRACTICAL sense, *NOTHING* has been "done" until it is ALL done! To the user, if you have removed 27 shovelfuls of sand in 135 seconds but have 27 shovelfuls of caliche to remove in the next

540 seconds, he doesn't think "Oh, we're half way done!" Half of a hole may have been dug but the "job" isn't "half done".

"It was a bigger job than I thought!"

"Why? You KNEW it was going to be a 27 cu ft hole..."

"Yeah, I just didn't expect it to take me this LONG"

I have found people are largely clueless as to these things. E.g., they'll sit staring at a stalled HTTP transfer without ever thinking to see if there is *any* network activity taking place.

People seem to be either INSANELY patient or IRRATIONALLY impatient when it comes to these things.

An *observant*, *involved* user will/might do this. But, you are now shifting that responsibility on the user when you -- a "machine" -- should be infinitely more capable of making those decisions! Or, at least providing more information *to* the user to assist them in making that decision.

E.g., why a single criteria/mechanism for reporting progress (on file transfers, in this example) regardless of the size of the file, it's "destination", "origin", etc.? Why not a smarter assessment of "progress" based on the sorts of things that you, the machine, *know* (or have OBSERVED in the past) regarding those types of transfers?

For example, copying a hierarchy via FTP has *lots* more overhead than an NFS or CIFS mount. And, "local" network traffic tends to have different characteristics than "foreign/global" traffic. And local *filesystems* behave differently than external ones.

Yet, all get lumped into a single "progress presentation".

The estimate isn't the issue. There are ways to get really

*good* estimates (depending on the task). The issue is how do you present this information to the user in a way that allows them to make "good" decisions (good being defined as "whatever is appropriate for *this* user in *this* situation) about what he wants to do.

For example, I can tell you how long it takes me to do a "make world" on most of my boxes. This is because I deliberately made a note of the time required the LAST time I did so! (i.e., a machine can just as easily be doing this every time it does something -- so *it* knows instead of requiring the *user* to "know"). So, if I need/want to "make world", I can *plan* my activities and resource allocation accordingly.

If it will tie up an "unloaded" machine for 4 hours, I might opt to defer it to "after hours". If it's resource demands will take a heavy toll on a 3D animation rendering that I want to have finished "today", then you can *bet* I'll defer it. OTOH, if I'm just writing code and that 4 hours creeps up to 4.5 hours as it competes for resources that I am using, then I may opt to tolerate a *slightly* more sluggish codewriting environment in exchange for having "world" built *before* end of business, today.

Better indicators makes it easier to come to an "informed" decision about how to allocate your "personal" resources. This (IMO), in turn, comes to a clearer understanding of

*how* you make those decisions. And, once that is understood, I think I can make the next step to designing an intelligent *agent* that can act -- or advise -- you in the future deployment of your resources.

(Imagine a piece of code "deciding" that it would be a good use of resources to start some expensive task, now, because it has learned the impact of that activity on your *likely* activities in the near future and has decided that the "cost" to you will be manageable and far outweighed by the benefit you will receive when the "expensive task" is done -- seemingly "for free" -- before you need it)

The Windows problem -- and that of most "file transfer" progress indicators -- is that the estimator is crappy. Recall that the estimator adds no functionality to the activity -- it's window dressing. So, I suspect very little effort has been spent on making it "good".

If, OTOH, it had a key role in the performance of the code/algorithm itself (e.g., like scheduling disk writes where you want to be able to estimate the position of the "heads" at any given time to minimize access delays incurred), I think more effort would have been invested there.