Aeki: This is not exactly what is going on.
You are correct, many of the analysts on The Street are full of crap, but not for the reasons you describe.
In fact, quarterly earnings estimates are where analysts are at their relative best. This is because the "analyst" is doing very little "analysis" to produce these estimates. Instead, he or she is simply parroting and extrapolating what the company has been saying over the last quarter.
This is one of the biggest misunderstandings about what analysts do: they make those predictions based on what the companies tell them. They participate in analyst conference calls. They fly out to the site, walk through the factory floor and talk to the senior management. (Well, at least the buy-side analysts do this.) They pick up the phone and call their investor relations ("IR") contact (or, if they're an analyst for a major player, they call the CFO or CEO directly) and ask "how are sales going this quarter?" Companies have IR departments solely to tell their story to analysts and fund managers.
Now, there is a time lag between the last conference call they
listened in on and the earnings announcement, so there is some
projection and extrapolation involved, but in general you're
acting solely on information that they fed you. And if that information
is good, you should be at most a penny or so a share off. So if you're wrong,
in general, it's their fault.
And it is mostly the companies' fault: in the last half-decade of rapidly rising asset prices, the senior management of many U.S. corporates has learned that their stock price (in which all of them were heavily personally invested via options) would rise quicker if they fed the Street conservative estimates, and then "beat them" by a few pennies per share when they announced. This was artificial, of course, and in theory it didn't fool any of the big institutional investors (since they were all in on the conference calls, and eventually learned which companies would lowball them), but because it snowed the increasingly large group of individual investors it would more times than not cause a real rise in the price that the institutionals would want to trade on. So it became a self-fulfilling prophecy. This is the origin of the "Whisper Number", which represents the difference between what analysts suspect reflects reality vs. what the corporations have been feeding them. Welcome to Wall Street.
Of course, this also works in reverse: if things are not going well, companies have learned to misrepresent the downside a bit in the hope that sales will pick up before the Street punishes the stock. When they don't, and they announce to the downside, it's a Bad Thing, not just because they are not doing as well and investors will build it into the stock price, but also because this is de facto evidence that they have been misrepresenting their story to the analyst corps, so it calls into question everything they've been saying.
So that covers why you should not be upset with analysts. Now here's why you should.
There is a pernicious herd mentality on Wall Street, and analysts are notoriously bad about thinking for themselves. Or perhaps they do think for themselves, and then sit on their hands and just agree with whatever Mary Meecker, Abby Joseph Cohen and Henry Blodget have been saying.
This pack mentality will manifest itself in the following way: One of the principal functions of an analyst is to give a "position rating" on a stock. Different firms have different nomenclatures for this, but in general it will be something like: Sell, Hold, Market Perform, Buy, Strong Buy. If you watch analysts' historical performance on stocks, you will notice that most of them do not start upgrading the stock until it has already made substantial gains and is an unqualified success. The further the stock goes up, the more the ratings move up into "Strong Buy".
This may make sense on the face of it (ie: if the stock is soaring, the company must be doing something right, so it makes sense to upgrade them), but it is completely and utterly wrongheaded. From a value investing perspective, ratings should increase when the price goes down, not up: if XYZ corporation is a good buy at $10, it is an excellent buy at $7 and a back-up-the-truck buy at $5. But once it has risen from $10 to $50, it is most likely a sell if the industry P/E (or whatever combination of metrics you use) tell you it should be at $27. But you just go back to March of last year and look at how many "Strong Buy" ratings analysts had on things that were at $150 and are now at $1 and on the verge of de-listing. These ratings weren't based on earnings surprises or disappointments, or indeed anything rational. They were based on a damn-the-torpedoes "the trend is your friend" pack mentality that encourages analysts to go with the Street consensus and not what the actual market, sales and prospects would otherwise tell them.
And that's why analysts are full of crap.