In the 1930s, when lowish impedance triode and high impedance pentode output stages were used - both without feedback - the loudspeaker was selected to *sound right* on a particular amplifier, although high impedance pentodes always produced a prononunced bass resonance.
My copy of the 1940 Radiotron Designer's Handbook talks about the better performance of speakers with low impedance drive as produced by triodes or negative feedback. The issue is damping of the bass resonant frequency. The moving mass resonates with the spider spring compliance and the voice coil and magnet motor when shorted provides significant damping. The energy absorbed by the motor is dissipated in the resistance of the voice coil.
High impedance drive maintains current at higher frequencies where voice coil inductance would otherwise reduce current, and so lifts treble.
At the end of the 1930s, the 0.1% THD figure became attainable using voltage feedback; and along with the low distortion, output impedance fell to much less than the loudspeaker impedance, and true voltage drive became standard, rather than just a favourable damping factor.
Since then, quality loudspeaker systems which are sold without matched amplifiers, are always designed for zero source impedance. I believe that some powered speakers have interesting feedback producing non-zero and even negative output impedances.
The tuning of bass reflex systems is badly affected by wrong drive impedance, and sealed systems boom badly because the bass resonance is much less damped. You can easily pick the difference - feed an audio gen into an amplifier, connect a speaker and sweep the gen from 200Hz down to below the bass rolloff. Now insert a 20 or 50 ohm resistor in series with the speaker, readjust the volume, and sweep again. Typically, the high-Z drive gives an enormous bass peak at around the
+20db level ! You still get quite a peak even with an 8 ohm resistor.
Another experiment is to get a loudspeaker system and tap on the woofer, listening to the *boomp* sound, with the terminals open and short circuited and 8 ohms connected. The extra damping is very evident.
If you do impedance match, you will still hear sound, and if this is not a hi-fi application, the sound can be clear and usable.