Solid state relays behave similarly to back to back to back (in parallel, but anode of one diode connected to the cathode of the other) diodes connected in series with the speaker wires. You can try this with a pair of 1N4004's or similar. There is a forward voltage drop associated with each diode and from about -0.6 to +0.6 Volts, neither diode will conduct. This is not a big deal for a 120V lamp, but audio will be sliced up.
A mechanical relay will have a small resistance. If you want to get fussy, the surface contamination on the relay contacts might form some low grade diodes and muck with the sound, but these "diodes" break down at very low voltage and the actual voltage drop is usually very low compared to the audio. As the contacts age, this contamination can build to the point where it may cause audible trouble -- requiring contact cleaning or replacement of the relay. In case you are wondering, gold plating the relay contacts is not a long term solution for speaker relays because the small arc that is generated when switching under load will quickly blow away all of the gold. On the bright side you can exploit this arc to burn away surface contamination by repeatedly opening the relay while the amplifier is operating at full power and driving a speaker.
For super high quality audio I prefer that the speaker "protection" relay be omitted, but this is a much more complicated amplifier design project because the little clicks and pops normally created while an amplifier starts up or shuts down are directly sent to the speakers and can be beyond annoying. (Think of a little 200W "click"). And, if an output transistor shorts, the speakers are at risk.
