Search Engine Optimization, often abbreviated to just SEO, is a mysterious black art which is victim of a lot of lies, guess-work, and intuition, rather than actual science.
The reason why it's difficult to get any hard-and-fast advice in the SEO field is for the very same reason that SEO exists in the first place: Businesses want to rank high in Google for popular search terms which are relevant to their business, but search engines are only interested in delivering the best possible results to their searchers. Obviously, these two can be mutually exclusive: Someone searching for [erectile dysfunction], for example, is likely to want to get neutral information about causes, cures, and anatomy, whereas I'm sure we can all think about a few companies who would rather just flog GENU1NE HERBAL REM3D1ES, or somesuchthing.
An example of what can happen if Google dislkes your site...
In November of 2005, a one of the UK’s largest lead generation companies fell victim of some major changes within Google, known as the Jagger update.
Nobody quite knows what happened, but it is suspected that Google tried a float-and-skim approach: Let all the spam results float to the top, and then remove the top results from the search engines. Something didn’t go to plan, and suddenly the Google index was chock full of spam results.
The lead generation company dropped off the first two pages of Google as a result, and immediately saw a 70% drop in consumer enquiries. The company nearly went into administration, because they made their money on a per-enquiry basis. In December 2005, update Jagger was over, and the company regained its place in the top 10 – with better rankings than ever.
Despite the 6 week drop, the company is now recovering, and is getting 25% more enquiries than before they dropped off the front page of Google for important keywords, but the short-term bottoming-out of the market resulted in the company struggling for a long time afterward.
What is SEO?
Search Engine Optimisation (SEO) is to optimize a web site for the purpose of internet search engines. More than ever, potential customers use search engines to find websites of interest. If they find your website in the first Search Engine Results Page (SERP), this can have a significant impact on the amount of traffic the site gets.
SEO, then, is essentially a series of tricks and techniques than can be used to boost a site's results in the SERPs, and the goal of good SEO is to increase the visibility of your website for keywords relevant to the business.
Why does a website need SEO?
Normally, the vast majority of website traffic is driven to web sites by search engines. There are several dozen search engines, but there are three main players who represent roughly 80% of all search engine traffic. In order of importance, they are Google, Yahoo and MSN search. If your website is performing badly in one of the Big Three, your traffic will suffer significantly.
Most pages will be available in Google if you get specific enough. The problem is that most internet users are unspecific when they are searching for something. If you do a search for [juggling ball online shopping], you are likely to find an online juggling shop. If you only search for [juggling], for example, you might not be interesting in making a purchase - you may be looking for videos, photographs, or even time management tips.
If you were running an online store selling juggling balls, you might want to rank well for the [juggling ball] search query. This is where SEO comes into action.
Why do the search engines need SEO?
Search engines are extremely good at doing their jobs – technology is continually developing and improving, and Google & co are constantly getting better results. However, there are always limits to how well the results are coming up.
One of the things that impacts a site’s results most strongly is not how relevant the site actually is, but how relevant it is perceived to be by the search engine robots. Put differently: if a site is not adequately optimized, the search engines don't know what it is about.
If we go back to the example above, it may be that the copywriter used "it", "them" and "equipment", instead of "ball", "juggling ball" and "bean bag". The former are stop words that do no good in search results. The latter are keywords, which help the search engines along in doing their job.
How a search engine works
From website to search result is a 4-step process.
Information gathering – A search engine uses an automated programme known as a 'spider'. The spider crawls the web and saves the retrieved content to a database. It also collects any links to other websites, saving them for later. When the finishes gathering the information about one web page, it will go on to the next saved link in the list, and spiders that.
Information indexing – After data has been gathered, another part of the search engine software analyses the data. Through analysis, the search engine ‘knows’ what a page is about, and can subsequently determine how relevant it is, compared to other pages about the same topic.
Information querying – As the user of a search engine, you want to find a piece of information. When you enter a string of words into a search engine, the computer programme queries their database of information, and will try to supply the most relevant information to your query.
Information presentation – Once the search engine has found a set of results that fits the search query, it needs to order it in a logical fashion, to make sure that the page you are most likely to be looking for, shows up. Preferably in the first 10 results of the SERPs.
It's the last two steps which are fiendishly complicated processes: Someone searching for [child] wants a very different type of information from someone who searches for [child prevention], [child abuse], or [prevention of child abuse]. Search engines have to cater for all possible combinations of search terms.
Web page SEO
Influencing search engine results
If you want your web page to rank well in the SERPs, you can only influence two of the aspects of how the SE works: The information gathering phase (influenced by altering your site structure), and the ranking phase (influenced by a large number of different factors).
For the information gathering phase, you will want your website to be working well, technically. This means that you have to make sure that the server is working properly, and that the spider can access the web site in an efficient manner.
Sadly, search engines are not human: They are sophisticated computer programmes. That means that they don't have eyes, and can't 'read' images, Flash graphics, and many other web technologies. This means that these technologies should be either avoided, or that alternatives should be made available, both for disabled users, and for the search engines.
Things like dynamic URLs, pages with dozens of links, pages that are far removed from the front page of a website, documents accessible only via a dropdown or search box, pages that require user authentication to be accessible, and pages using automated redirects can be stumbling blocks for search engines, preventing the page in question to be indexed properly.
Getting better rankings
Ranking better in search engines is a drawn-out process, and anybody who claims to guarantee quick results should not be trusted. The nature of search engines means that changes done to a website now will have an impact on the SERPs in 1-2 months, on average. Consistent changes may not be measurable until 4 months after a change has been implemented.
Telling the search engines what the site is about
Improving the rankings means trying to improve a search engine's understanding of what a site is about. Using a professional copywriter to go over the copy is a good idea – most good writers can create copy that uses a healthy keyword density. This means that any text that appears on the website has to be on-topic, and actually use words describing an object.
An example: "Ford had a launch today, and it is great. The blue seats are very comfortable, and the stereo is outrageously loud" reads OK to a human. A search engine, on the other hand, would not necessarily know if Ford is a person or a brand name, and none of the rest of the copy is particularly descriptive. An SEO-conscious copywriter could rewrite the same sentence to "Ford motor corporation today launched a new car, which is very good. The vehicle's blue seats are very comfortable when driving, and the auto stereo installation is of high quality".
In the second example, we have "Ford", "motor", "new car", "vehicle", "driving" and "auto". It reads (nearly) as well as the first example, but suddenly, the copy is jam-packed with words that the search engines can use to analyze and judge the content.
Factors towards SEO'ing a site
The first step toward optimising a website for the search engines is to identify the strengths and weaknesses of an existing site.
To start this process, it's worth evaluating your site, to ascertain the status quo of the areas listed below. Based on the findings, of course, you can do a series of adjustments to the site - some are likely to be quick wins, while others could be quite expensive and complicated to implement... But as with all other business decisions, it's a cost-benefit analysis...
Technical site accessibility – how well a spider can access the web site?
- Does the site have an acceptable response time? (anything above a second will affect the site negatively)
- Is the HTML and CSS compliant to relevant standards? (non-standards-compliance can confuse spiders)
- Is the length of the HTML document prohibitive of effective spidering? (The first 50k or so are weighted heavily)
- Do the site’s meta-tags adequately reflect its purpose and content? (Trying to game a search engine is likely to backfire)
- How is the content ordered inside a HTML document? Could this order be improved in order to make it more spider-friendly? (put content near the top, and navigational stuff further down)
Navigational structure - how well can a spider navigate around the site?
- Is there a site-map? (if it exists, it has to be correct and comprehensive)
- Are URLs unambiguous and descriptive? (... and each piece of content should only live in one place)
Relevance and copy content - how well can a spider identify your content? Is it relevant?
- Is the copy clear and to the point? (Unnecessary waffling is a bad idea, both for your visitors and search engine spiders)
- Does the site use proper spelling and grammar? (relevant sites are statistically more likely to have a higher level of spelling and grammar than spammy sites)
- Is the content unique and relevant?
- Does the copy use header-tags (<h1> etc) for headings and sub-headings? Do these tags strengthen the message and relevance of the copy? (makes it easier to categorise for spiders and site visitors alike)
- Do the title-tags reflect the page content? (helps spiders and visitors who bookmark)
- If there is a news section, does it have regular updates? (stagnant news sections are pointless)
- do you have enough links to your page?
- Does the site have well-placed outbound links?
- Can the site easily bait inbound links?
- Would reciprocal links benefit the overall site SEO programme?
- Are all outbound links on the site pointing to valid documents?
- Are all inbound links pointing to valid documents? If not, can they be changed or 301-redirected to relevant substitutions? (avoid "link rot")
Making dramatic changes to a site - especially tied to, say, a redesign or a restructure, can give short-term negative effects while the search engines try to figure out what the hell is going on... So it's worth making a plan for how to deal with the temporarily downside.
An example of an SEO evaluation
Optimizing Everything2 for search engines