Hi
this depends on several things

a) what ist the nominal impedance of your speakers?
b) what is the nominal impedance of your amplifier?
c) are two pairs of speakers driven parallel or serial?
If both of your speaker pairs are 8 ohm, there is no problem because almost all amplifiers are suited to drive 4 ohm speakers.
If two speaker pairs are switched serial, it will in no case do any harm to the amplifier, but the sound quality will suffer. You can check this if you only connect 1 pair of speakers, but turn on both of them. If you hear sound, the speakers are switched parallel.