There is a distinction to be made between systematic downvoting, which is driven by spite, and consistent downvoting, which is motivated by a sense of one's duty as a peer editor.
Systematic downvoting means deliberately and methodically going through a fellow noder's writeups and giving each a negative vote, regardless of the quality one might perceive in what one reads there. It is motivated by malice; the thought of incurring experience point loss to someone personally outweighs the Everything cultural imperative to cast positive votes for quality material.
Consistent downvoting, however, means deliberately and methodically going through a fellow noder's writeups and giving each the vote you feel it deserves in terms of style, content, and appropriateness — and finding consistently that the material is of substandard quality. It is motivated by the desire to cull Everything of that which the voter feels doesn't belong.
There are many schools of thought on Everything regarding the proper use of votes and the role of experience. The Self-Punchers, including Randofu, weStLY, and Golem, prefer to be outside of the experience and voting system, which they view as confining, or inappropriate for their personal use of Everything. This is an extreme and respected form of that XP Stoicism which is popular among the eschelons of users. Systematic downvoters, as I already mentioned, are much-maligned. The vast majority of noders, however, are divided between two camps: Those who believe in using only positive votes and who feel that negative votes are entirely inappropriate to Everything, and those who believe that downvotes are occasionally the appropriate response to poor writing, perhaps to be accompanied by a private message to the offending party.
This bifurcation of users has lead to a debate regarding what should be done with systematic downvoters. Some users have proposed that all voting be tracked and abnormal behaviour flagged, either for the attention of the victim of the alleged systematic downvoting or for the attention of an editor or god.
Currently, Everything keeps track of whether or not you have voted on a particular writeup, as well as every writeup's reputation. To track the behaviour of voters would, of course, require keeping track of whether they voted a writeup up or down. It also requires that this information become available to general (non-Blockstacker) users, and is intended to serve as a tool for controlling the behaviour of others.
These measures would be required to control systematic downvoting without actually abolishing downvoting altogether. Appeals to honour, of course, are not sufficiently binding for those who would seek to delimit the abuse of downvotes. However, I believe that any observation or policing of votes would be an unacceptably oppressive measure.
Everything quality control is based on the principle of secret ballot voting. The whole point of the secret ballot is that you can't be held accountable for the vote you cast. In principle, this liberates the voter from partisan pressure and allows the voter to vote as his conscience dictates. The flipside of this, of course, is that voters without conscience are not called to task.
Downvoting can be abused, but I think upvoting is just as abused. That potential for abuse is the price we pay for having both freedom and privacy, which are both essential to our autonomy on Everything — which is, in turn, essential to our mission.