Maybe sometimes the interrupt ist missed? Set an I/O-pin and reset it again in the ISR and then do an FFT on that pin. If You only see 32kHz, not ints are missed.
To my
But I would do an analysis with just "0" sound and then experiment with sinuses of various frequencies, noise and pulses.
Good luck! Rene
P.S. Can You somehow turn on the numbers on the axis, that would be good.