Why are they called Switches and not Buttons?

In a switch you get on/off, but a button returns to the original setting.