Literally, a denizen of a city who abides by the laws of that city.
The OED gives some interesting definitions and derivations of the terminology used in this phrase. For instance, a "citizen" is, by one interpretation, "an inhabitant of a city or (often) of a town; esp. one possessing civic rights and privileges, a burgess or freeman of a city." The verb "to abide" is defined by the same source as "to remain in residence; to sojourn, reside, dwell." Third, "law" is given as "the body of rules, whether proceeding from formal enactment or from custom, which a particular state or community recognizes as binding on its members or subjects. (In this sense usually the law.) Also, in early use, a code or system of rules of this kind."
The conjunction of all these definitions makes for an interesting phrase. Each term includes the idea of a home or base: citizens are from a place, abide in that place, and operate under the law of that place. The city is the geographical extent of the citizen's political authority; the fact that they abide within the bounds of that city creates their political authority within that city; the law dictates their actions within that city. But the fact that they do have some authority within the city (voting, holding political office, etc.) indicates that they have some control over the law. Citizens make the law, yet they are tied by it. This seems to be related to the Italian Renaissance city-state requirement that all citizens must at some time hold office. Hm hm hmmm. Well.
But when you take into account current usage, the phrase becomes harder in tone, or sardonic. A citizen is usually assumed to be simply an inhabitant of a country, law is something in which to find loopholes, and almost no one abides peacefully and contentedly anywhere. The law-abiding citizen is thus seen not as an active person seeking improvement in their city, but as a goody two shoes always driving five miles per hour under the speed limit.