Kernel configuration problem? Waking up a thread from a hardware intterupt takes too long...

I am working on an embedded system that my company has developed.  (My
company also developed the driver.)  The problem we're seeing is that
on our embedded system it takes about 100 milliseconds (standard
deviation of 0.5 milliseconds so this is very consistent) to wake up a
thread that is waiting on a hardware interrupt.

However, running the same driver and executable (exact same binaries)
on a desktop (with different kernel configuration) only takes about
half a millisecond.  We also ran tests on our embedded system with
other operating systems (embedded XP and VxWorks) and the delays take
no longer than 3 milliseconds, so we've been able to eliminate the
hardware.

We have also changed part of the kernel configuration and recompiled
the kernel.  When we re-ran the tests with the new kernel, the delays
doubled to about 200 milliseconds.  So we believe the problem is the
kernel configuration.

So my questions are: Has anyone had similar experiences?  Does anyone
have any suggestions for what kernel configuration(s) might be the
culprit(s)?

Thank you for all your help!

- Kevin Hall