Lm324 is a linear amplifier. If the input signal cannot make it reach saturation or cutoff, of course the signal will be amplified, which will appear as so-called distortion. The best way to shape a square wave is to use a gate circuit.

If you want to use 324, you must let it reach the saturation and cut-off state when inputting high and low levels, and cannot let it work in the amplified state!