Many modern switch IC's would do fine. As long as you drive the input side of the switch with a low impedance source, (such as a fed-back op-amp), and follow the switch with a high impedance load, (such as the + input of an non-inverting op-amp stage), the distortion added by the FETs will be negligible. The way distortion arises in FET is that drops across the channel resistance modulate the effective gate-to-channel voltage, in turn affecting on-resistance. By keeping the load current thru the channel resistance small, you keep that modulation effect small.
That would work, provided you liked the isolation it provides.
Non-tricky would be to use a IC switch.