1. Home
  2. Embedded Programming
  3. Quick image rendering/resizing

Quick image rendering/resizing

Hi,

I'm hacking together a kiosk-style user interface for a local non-profit. As many of the users are elderly, much is done with images/pictures instead of text (small display means you can't put much text there

*and* have it remain legible to "older eyes")

The images for each "button" (no need to be precise, here) are stored in an RDBMS. Based on the operator's actions, the screen is updated with new choices (images, etc.).

Updates need to be reasonably quick lest the user grow impatient with the interface.

The kiosk is implemented as a think client -- scarce resources on the user end of the wire.

I can trade memory (in the RDBMS) and transit time (over the network) for processing power in the client (*or* server).

So, one approach is to convert JPEGs (the images originate as pictures captured from digital cameras, scanners, etc.) into PNGs, BMPs, etc. as they are being imported to the RDBMS. (i.e., a one-time event). This makes the images a bit larger (so what! space on the server can be nearly infinite) to store. And, means screen updates will potentially require more data to be sent down the wire than a more compressed encoding would require. But, that doesn't seem (on paper) to be an issue. Plenty of bandwidth.

An uncompressed format also eases the processing requirements for the client (e.g., wireless devices).

However, uncompressed formats imply a particular "resolution". So, rescaling the display on the user interface poses problems. Now the client would need to scale the image to the desired screen size (for example). For a dozen images, this can represent a significant amount of processing -- in a *short* period of time (300ms tends to be the threshold for annoyance in a user interface -- that 300ns has to encompass the round trip transit time to access the RDBMS server, the time it takes the server to fetch the images and transfer them back to the client, the time it takes the client to resize them, plus the time it takes for the client to build the new display and any other client-side processing).

One possible solution is MS's "DIB" approach. That increases the storage requirements on the server even more (not a concern at this time) and means the client needs to either specify the version it wants (in the SQL query *or* at the start of its *session* -- which would therefore prevent resizing the user interface during a session!) from the RDBMS

*or* the RDBMS must ship the entire DIB to te client and let the client extract the most appropriate image.

This doesn't work if you want to support *arbitrary* resizing.

Another approach is to have a server side filter that resizes the image as requested (in the SQL query) prior to delivering the reply. This can exploit the extra processing power available on the server -- on the assumption that clients are greater in number, simpler in capability and sporadic in access. Alternatively, the sizing can be established when the session begins so SQL queries can be unconcerned with this detail (again, means resizing is not possible during a session. Also means you have to know a priori what the relationships of the sizes of each will be *and* maintain this regardless of how the composition of the UI changes from device to device!)

Yet another approach is to resize *all* images in the RDBMS in a "session-specific" instance (VIEW) associated with that client when the session begins. This can be a *big* performance hit as it could mean hhundreds of images get resized -- even though few of them may ever be "served up". OTOH, it would give the ideal steady-state performance. And, be portable across clients.

As always, there is no clear winner. Have other folks done anything like this? Or, any *desktop* technology worth looking into for hints?

Thanks!

--don

Reply to
D Yuniskis
Loading thread data ...

D Yuniskis wibbled on Tuesday 13 April 2010 19:09

How many possible resolutions are there for each image? If it's less than say 10, why not store every needed version on the server. Or generate on the fly into a cache on the server - then only the first access to one particular resolution will be slow, all others come from cache.

Cheers

Tim

--
Tim Watts

Managers, politicians and environmentalists: Nature's carbon buffer.
Reply to
Tim Watts

Or store a few image sizes, evenly distributed logarithmically between 1 and not-quite two (i.e 1x, 1.2x, 1.7x). Decimation by two is easy, after that just choose the largest image that'll fit in the space without overflowing, and live with the fact that it's a bit clunky.

--
Tim Wescott
Control system and signal processing consulting
www.wescottdesign.com
Reply to
Tim Wescott

In my original design/proposal, I had picked out a set of (surplus) hardware (small industrial computers with touch screens, etc.) and just defined screens relative to those

*fixed* sizes.

Someone stumbled across my little stash and tossed them in the recycle bin (they were old -- 486 vintage -- so no one could *possibly* have a use for those, right??? :< )

I then came across some nice "heavy duty" LCD monitors with SAW panels. "Oooo, yummy!" Physically larger. Requires external computer (the original units had everything "built in"). Etc. More screen area (as well as higher resolution) so I redesigned the user interface to be a bit less cramped and more informative.

*Hid* those in the back of a closet. Someone decided to do some "Spring Cleaning"... :<

Recently, another aspect was added to the design (when folks aren't paying for your time, its amazing how hard it is for them to think about what they *really* want!). This will require wireless (handheld) clients in addition to the other stuff.

Having been screwed twice (plus the feeping creaturism), I figured trying to tailor the interface to the available devices was a no-win solution. And, designing out of a

*catalog* (i.e., pick what I need, tell them to buy it) was unaffordable.

So, the only real solution is one that lets the interface morph "on the fly" to suit the needs of the client (device).

Currently, that means two or three different "dedicated display" sizes. Plus "windowed" versions (i.e., run a client on a PC that can display this alongside other cruft). The windowed client probably won't be infinitely scalable -- but, fixing it at something like

640x400/480 would be too tiny on some folks's displays (1600x1200) and too *big* on others (800x600).

So, rather than *pick* some sizes and then limiting myself to those sizes in how I arrange the images on the screen, it seems better to just support scaling -- somehow.

E.g., in many cases, the user is presented with a multiple choice screen. My software looks at the

*physical* size of the screen (window), a minimum (physical) size for a "selection" (since these have to be "touched" to be selected and you need to be able to accommodate "fat fingers" as well as the possibility of tremor or other coordination problems), the resolution of the screen (to convert to dpi), and the number of possible "selections".

From this, I come up with an optimal arrangement of those selections. I.e., try to get all selections on *one* screen (since switching screens adds complexity on the user's side) while not conflicting with the goal of minimum "button" size. Barring that, come up with the smallest number of screens minimizing "unused area" on the last screen. Once you have a layout, you know how many buttons on each screen and, from that, can deduce the target size of the images that you need to render. (recall that a high resolution screen might have a "smaller" -- in terms of pixel dimensions -- than a lower resolution screen... depending on the physical size of the screen/window, etc.)

For example, if you have a 640x480 screen/window and 1-4 choices, you could represent each of them as a 320x240 image (smaller because you need borders around the image, etc). If you add a 5th choice, this would require a "second screen". If you go that route, then you need to forfeit one of the "buttons" (choices) on that first screen for a "to next screen" button. So, you have 3 choices on the first screen plus the "next" button. The second screen would have the other two choices and a "previous" button.

Because you now only have three choices on the first screen, it is more likely (assuming all choices to have equal probability) that your choice will be on the second screen. So, your average number of keystrokes increases faster.

OTOH, if you resize the screen to support, for example, nine buttons, then you can fit all of your choices on just *one* screen. When you get to 10 choices, you again have to face the decision of "resize to smaller buttons in greater number vs. more screens". I.e., 8 on the first (plus the "next" button) and 2 on the second (plus "previous") or, maybe 16 on just one screen!

The heuristics are not-intuitive! I.e., there are times when it is better to use *bigger* buttons as the number of choices increases! (trust me, I've done the math :>)

Anyway, unless you constrain the size, shape and resolution of the presentation, its really hard to pick "all" of the right sizes a priori and be done with it.

Caching results will help when *that* screen is revisited (if the number of choices haven't changed and the window hasn't been resized or the device on which it is viewed) but it may not do anything for any of the other "clients" operating at the same time. I.e., there might be little gain from the cache itself -- the gain may come from the server side processing.

I'll have to look through Sproull and Foley/Van Dam to see if there are any tricks that can be played in the representation of the images that could speed up rendering. (noting that I could do the transcoding *once* when the images are imported to the RDBMS).

(sigh) Fun project but really taxing with the constraints imposed. :<

Reply to
D Yuniskis

Yeah, I thought of that originally but the display starts to look like crap when you render it on low resolution, larger displays. I.e., they can really benefit from the interpolation available from "larger" images to help reduce the jaggies.

I *could* just burn lots of BTUs and use the 15-blade server to handle the "back end". Can you spell overkill?? :> :<

Reply to
D Yuniskis

ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.