One of the things I have always wondered as a software developer is why technical writing as a profession is held in such low esteem by market forces (employers who pay low salaries to technical writers) and end users (who do not seem to base purchase decisions on documentation, and then complain that what they do get is dense, dry or unreadable). Both the economics of software production and my professional experience tell me that technical writing is horribly, tragically undervalued by companies and consumers.

One tendency to not recognize the need for good technical writing seems to come from The Myth of the Perfect Software Design. This is a fictional piece of software (or really any technology) whose metaphors and interfaces are so thoroughly simple, intuitive, and natural that the whole thing requires no technical explanation whatsoever, even to the most novice of audiences. This is almost always the software you're working on now: maybe the other things your group has rolled out have had flaws, but this next one, man, it's gonna be perfect. Novice programmers, especially the bright ones who have the curiosity and wherewithal to read books on object-oriented design techniques, Design Patterns, user interfaces, etc., tend to fall into this trap. It's not hard to see why, because most of these books make fantastic promises about the usefulness of their methods. Even Stroustrup in his original C++ book makes it sound as if he has revolutionized programming, and all of your design problems will be solved.

Two fallacies are at work here.

Fallacy #1: Good design techniques make code, and the resulting software, inherently understandable. That is, if we make things modular, reusable, inheritable, self-contained, scaleable, consistent with metaphor and all of the other good design techniques we've learned, then the end product will be immediately understandable to coders, and the resulting lucidity will even ripple up to the end-user. This simply isn't true: much of the time the reason we're using those techniques in the first place is that the problem domain (that which the software is designed to do) is so immensely huge or complicated that we are forced to use OO techniques, etc., to break its functions and interactions down into tractable pieces. Those techniques can be invaluable in and of themselves, because without them it would be another order of magnitude harder to build (if indeed it could be built at all). But the whole, which consists primarily of how those pieces interact, may still be hugely complex and require detailed explanation, both to the programmer and the user.

Fallacy #2: Consistency and metaphor in design make software self-evident to users. These things help of course, but (1) they are mostly arbitrary to begin with, and (2) the space of things that software can do is infinite, and hence most software that does something new is going to incorporate new metaphors of some kind. There's very little intuition that humans are born with; most metaphors are learned from generalization, experience and social convention. What we do have built in is predominantly based on perception and physical rules: gravity, motion, perspective, mechanics. These turn out to be lousy metaphors for interfaces anyway, so what's used in software design is arbitrary and dictated by those that got there first. An example: drag-and-drop for moving or copying an object. This may seem intuitive, but it is arbitrary when you think about it (try dragging-and-dropping a $20 bill on your desk, and see if you can get it to copy itself). Most of the things we tend to do with computers do not have physical analogues anyway: what is the equivalent for apply-the-fast-fourier-transform?

What we're left with after admitting these two fallacies is that software (or indeed most technology) is wholly incapable of standing for itself. In fact, the larger the project (either in functionality or targeted user base) the more and better technical writing it will need to have. In order for users to make use of it, even some sophisticated ones, its complex interactions and encyclopedic functionality need to be described completely and understandably, so it can be used productively. Yet employers seem to view technical writing as a cost center, like packaging. It is something that has to be included in the box (or more often on the media itself as electronic help files) in order to sell the good stuff, the software itself.

In doing so, they make the classic economic mistake of assuming that users' time has no value, no opportunity cost. Good software is insanely expensive to develop, because the labor cost is so high. Given the upfront costs, even though it has nearly zero marginal cost to produce, software is still a pretty big investment if it is produced for anything but the largest audiences. (Classic examples: Adobe Photoshop ($600), Wolfram Research's Mathematica (around $1000).) Exactly the same economics that drives that sort of pricing are the ones that make technical writing so valuable. If you are paying your graphic designer $40/hour, every hour he or she wastes trying to figure out how to do something costs you that amount in lost productivity. Fifteen hours of wasted time later (seems like a lot, but it's about 4 minutes per day over the course of a year) you have wasted the entire $600 cost of Photoshop. That is an extremely strong argument for factoring in the quality of documentation into a software purchase, but buyers never seem to consider it, or software producers believe that they don't consider it, or both.

Worse, this becomes a self-fulfilling prophecy for the industry, because technical writers are not compensated for the value they add to the overall product. The huge salary gap between writers and programmers virtually guarantees that anyone left in technical writing is doing it for the love, not the money. As such, many talented technical writers turn to programming or other pursuits, leaving the ones that don't understand the technology well. So it's not surprising at all that users find technical writing to be terrible. It usually is.

This must change.