What is here is a description of the way I believe the Bing, Google and Yahoo! search engines "think." Why is that useful? Because understanding the way search engines think can illuminate how to get good rankings in the SERPs over the longterm. The goal is not to trick the search engines into ranking your website for every keyword imaginable, but to give them what they want so you get ranked for the terms you should be ranked for.
This article comes on the wave of some rapid and intense changes in the Google algorithm and index - specifically changes known as Mayday and Caffeine.
Search Engine Engineers are Cartographers
You probably know what search engines are. But for now, try thinking of them a different way. Think of search engines as constantly updating road maps. Think of the people who search for stuff online as tourists. The brains behind the search engines think much like the brains behind tourists' street maps - they design for the lost tourist. A lost tourist is someone who has a vague or specific idea of what she wants, but doesn't know where it is. The popular search engines Google, Bing and Yahoo are all trying to guide the lost tourist to her destination.
But because they compete with each other, Google, Bing, Yahoo, and the other search engines also try to keep tourists coming back to use their own maps. So they strive to keep their visitors loyal. They must make sure that in the process of telling the tourist the best places to visit they don't:
- lead the tourist astray into dangerous areas,
- waste her time with disappointing results, or
- abandon her by failing to give any results at all.
This means the search engines must identify the good and bad neighborhoods towards which to guide their "tourists."
They also must identify which neighborhoods to leave off the map altogether (in other words, which pages to exclude from the index), and which doubtful but promising neighborhoods to merely display in tiny faded print where only a tourist desperately poring over a map for detailed instructions will be able to see it (which is how I think of bad ranking for an indexed web page - it's deep in the index, but good luck finding it).
Make Yours a Good Web Neighborhood
What is a "neighborhood" exactly? A neighborhood can be a website with its own network of connected pages. Or it can be a network of connected websites. Or it can be a network of connected pages on various websites. Each network changes over time.
Google, Bing and Yahoo are trying to figure out if each page's "network" places it in a good neighborhood for a particular visitor performing a particular query.
That doesn't mean the search engines will never send you a tourist if your page is situated in a bad neighborhood - because maybe it's an authority on some of the more obscure and targeted searches. Just as a friend might send you to a somewhat dubious neighborhood in the real world to try out a fantastic "dive" of a restaurant, because you can't get food that good anywhere else.
But each web page is called on continually - at every search, plus after every algorithm change - to "prove" that it's still in the neighborhood the searching tourist wants to visit.
It's not enough that the web page is on an adjoining side street, with lots of links to and from external websites. And it's not enough that the web page is a good-neighborhood wannabe, becoming the popular spot of the hour after a Facebook or Twitter special.
To a search engine, the page has to be exactly what the tourist is looking for. Or, if the search engine can't put its hands on the exact destination, then the best available option.
Search Engines Have a Problem, and It's Not Just Spam
A big part of a search engine's problem is filtering out spam. But another dilemma occurs when legitimate, original web pages seem to present themselves as the place to be for various search terms (keywords), but they're not really. Search engines have to figure out whether or not a page is truly relevant to a query.
Many content publishers have a different agenda. They try to rank for keywords, whether or not their pages merit the ranking. Google doesn't care for that. Put simply, the search engines don't want your page to rank if it isn't the best of all available places for the visitor to be.
The better the search engine gets at giving the visitor what he wants, the more you might find your page, which was considered valuable for certain keywords before, now considered irrelevant to them - in other words, all but spam. Not in the eyes of a human, necessarily - so don't take it personally - but in the eyes of the search engines.
The Search Engines Need Our Help
Google, Bing and Yahoo can't rank all those billions of web pages all by themselves. They're not quite intelligent enough for that. They guide content producers in best practices because they need us to tell them what our content is. That's what SEO is. Optimizing for search engines. However, they encourage us to use SEO techniques only for factors that they can't determine themselves.
For example, one search engine might say "don't use the meta-so-and-so tag anymore" because it no longer needs that tag to perform its algorithm, though it once did. And it might say "use the meta such-and-such tag" when it still has no other way of determining the information.
But Bing, Google and Yahoo then try to develop a way of determining exactly that information as fast as they can, because they must try to beat out the people trying to manipulate the algorithm to their advantage, thus making the search results less relevant.
Google has to keep changing the criteria they use in ranking, and the weight given to each criterion, in order not to be manipulated by publishers. I used to think that was poor-spirited. I mean, isn't there good honest manipulation and bad deceptive manipulation? And wasn't I a good guy?
It's not that simple. The ultimate aim of the search engines is to be able to evaluate content as well as a human being would - or better.
As the boss owning most of the market share, Google Search especially is aiming toward figuring out what content is all on its own and urging people to not to worry about gaming their rankings, except insofar as they ask for help. "Put up stuff visitors find useful," and "Use readable text, not text on images" and "create relevant anchor text" and "be part of a good neighborhood" is what they're telling us. Because they can do the rest.
We have a hard time with that, of course, because we're trying to make a living, here. We want to know how to compete. How to scramble to the top. The inside SEO tricks.
But our efforts backfire. The more manipulation that occurs and that results in "undeserving" pages getting ranked highly, the more imperative the need to get those search engine engineers honing that part of the algorithm so our tweaks and manipulations mean nothing.
Pages that Don't Deliver Will Go Down in Rankings
The practical advice to take from this is to do what the search engines say. It may look silly and naive to some, but it's the only way I can see to make a success of a web page in the long term with the existing search engines.
If you want your pages to rank well over the long term, you must figure out what the search engines are trying to figure out: who your visitors are, what they want, and how to give it to them.
Don't try to trick them. Don't deliberately withhold the information they need. Don't mislead. If you do, you may get a revenue click today, but tomorrow, Google will be figuring out how to send them to the places that do give them what they want.
So what should you do? That's always changing. Look to the search engines themselves. Read their blogs and help pages to figure out what SEO to use, because they will tell you what they are looking for. Then try it. Do exactly what they say. And while you're doing it, write valuable Web pages that offer something others that rank for those keywords don't.
Read the Yahoo Style Guide for Web Content.
Read about SEO for Bing.
Read about Optimizing for the Google Search Engine.
Deliver What's Promised
So let's say you've had a web page called "Buy Green Widgets" and you're not actually selling green widgets there, but just making money from advertising on the page, such as Google AdSense. Chances are good that over time you'll lose ranking for those articles as Google tries to give visitors searching for "buy green widgets" a place to accomplish what they're trying to accomplish - an actual purchase. Either give those visitors a better experience than the sites that aren't ranking, or give up on ranking for those keywords.
Be Fresh for a Caffeinated Google
Content "freshness" is important with Caffeine's implementation. What "freshness" means is still to be determined.
I personally take it to mean not the date of publication, but whether a search for "green widgets recall" means the recall that just happened this month rather than the recall of two years ago. So developing a neighborhood of pages linked to the latest info on the green widgets recall is better than spending hours trying to rank well for an article about the obsolete green widgets recall by backlinking. That's called squeezing juice out of a stone.
Evergreen Content Optimization
For evergreen material that's floundering in the rankings, freshness means providing what people are searching for even more precisely. A search for "Acme green widgets warranty" might previously have turned up your page, "Guide to Green Widgets." Now, you need to optimize for the the Acme brand of green widgets and specifically include information about the warranty - either on that page, or on a linked page.
Why should you have to do this if your page is already the best there is? Several reasons:
- Competition is greater - there's an explosion of content on the web, some of it providing more relevant content than yours.
- Searchers are getting more savvy about asking for what they want.
- Search engines are getting better at delivering it.
Say Bye-Bye to Obsolete Content
I believe some web content becomes obsolete over time. A particular culprit is content that performed a function that search engines now perform. An example that I noticed recently is directory articles. Before, a page called "10 Green Widget Sellers" would rank highly for a search for "green widgets sellers." Now, it might fall off the map in favor of actual sellers of green widgets.
The love-hate relationship between content producers and search engines will continue. Writers will keep trying to analyze how to optimize with Google, Bing and the rest. Google, Bing and the rest will keep trying to learn to read us better than we read them. And throughout it all, our SERP rankings will fluctuate - it's almost guaranteed.
Wonder why Google and Bing are changing constantly what they show on the SERPs? Here's my explanation. It'll be obvious to the old hats and maybe not so obvious to the newbies.