how to figure out where a process hung

Question

I've got a problem...I have a process run from cron that hangs in an uninterruptible sleep.It reads data from a webcam, and writes to a tmpfs partition.This process runs every 15 minutes; and most of the time it will run justfine, but once in a while, it hangs.  Then cron runs it again, and ithangs again.  Pretty soon I have a bunch of hung processes that consumeall resources, and for all practical purposes my little system is dead.The frustrating thing is that it happens rarely; the process runs every 15minutes and sometimes it will run for days just fine, and then it willstart hanging.Is there some way to find out where the process is hung after it is hungup?The program is spcacat, a very simple snapshot util for webcams using thespca driver:  .  Anyone have any suggestions?  Ihave 3 weeks to get this up and running, and that doesn't give me muchtime....--Yan--   o__  ,>/'_          o__  (_)(_)        ,>/'_        o__Yan Seiner, PE  (_)(_)       ,>/'_     o__Certified Personal Trainer   (_)(_)    ,>/'_        o__Licensed Professional Engineer         (_)(_)       ,>/'_Who says engineers have to be pencil necked geeks?  (_)(_)

Michael Schnell · Accepted Answer

What does this mean ? A sleep() call needs to specify a time, so it can't "hang".

Moreover, AFAIK, a user land process only can do uninterruptible sleep (a very short nanosleep() ), if it is assigned very special attributes.

Is it possible that the process waits for some hardware event that does not occur due to defective hardware ?

-Michael

Juergen Beisert · Answer

Try to use "strace". When it hangs connect to it with "strace -p " andyou will see where it hangs (if it hangs in a system call). If this doesnot help, try with "ltrace" instead.Hope it helpsJuergen

Sebastian · Answer

Hello,Your cronjob could kill all running instances before starting a new one.That way the ressources would stay free and the system won't get problems.It is not a clean solution, but at least it can keep your ressources free.You could log if any instances are killed (instead of a clean exit) andmaybe find some event which causes the program to hang.That's an idea, but it will only help against the ressource leak of hungprocesses, not against the problem.I'd guess the program waits for something (a camera's event?), but nevergets it.Regards,Sebastian

Captain Dondo · Answer

From 'man ps':PROCESS STATE CODES       Here are the different values that the s, stat and state output       specifiers (header "STAT" or "S") will display to describe the state of       a process.       D    Uninterruptible sleep (usually IO)These processes show up as 'D', which means they cannot be killed.I am guessing that thse processes are waiting for some camera event thatnever occurs, but I have not figured out why only sometimes....The camera shares the USB bus with a GPS, which is being polled almostcontinously.  I suspect there is some bus contention which triggers this,but I have no idea where to start looking; all of the code I've looked atlooks OK so far.--Yan--   o__  ,>/'_          o__  (_)(_)        ,>/'_        o__Yan Seiner, PE  (_)(_)       ,>/'_     o__Certified Personal Trainer   (_)(_)    ,>/'_        o__Licensed Professional Engineer         (_)(_)       ,>/'_Who says engineers have to be pencil necked geeks?  (_)(_)

how to figure out where a process hung

Join the Discussion

Didn't find your answer?