Don't throw the baby out with the bathwater means you should take care, when trying to improve a situation by repairing or ripping out the bad parts, that you don't damage what is already good about it.

For example, if you one day suffer a fit of misdirected handyman's passion and paint your nice white picket fence a nauseating shade of pea green, you don't have to tear down the fence. Just paint it again.

There is a theory that the origin of this aphorism lies in a time not so hygenically-minded, or luxurious, as ours, when a bath was a very rare thing for the unwashed masses. The big metal tub would be filled with hot water, and the head of the house got the privilege of bathing in the clean water; following him was usually the rest of the menfolk by seniority, and then the women. By the time the women got around to washing the babies, the tub might have six months worth of blood, sweat, and tears from ten people, and the water was rank and murky. When everyone was done and it was time to empty the tub, it could be a good idea to verify that nobody had been forgotten in there before pitching the water.

Probably not literally true, but it sounds good.