As I've learned, there are concrete reasons our pages get de-indexed or lose relevance to searches - algorithm changes, what I tend to call the "Google Shuffle," and other factors. This post is not about those specific things. Explaining the causes never really satisfies people, because what they really want to know is "How can I fix it?" (And besides, I'm not a techie. What I know about the making, breaking and changing of algorithms can be counted on one eyelash.)
This post attempts to answer the question, "How can I fix it when Google and Bing supposedly are improving their search engine - but now my wonderful page, which has been offering value all these years to visitors, is suddenly being ignored?"
My answer is, you can't fix it, except by
I'm going to say that another way, because it's important: Search engines are becoming more and more dynamic. They are updating their results faster and faster. For example, you may notice that Google Search appears to be moving away from fixed results. Fixed results means displaying Site A, Site B, and Site C always in the first three slots irrespective of who is searching and when. Instead, Google is moving towards displaying different results to different people for different searches. This is a shift toward a long tail model of search results.
To use an analogy: Ice cream. Imagine a society where this treat is only available in ice cream parlors, in a few limited flavors. You can tell someone to "go get you an ice cream" and they will always return with a cone of vanilla, chocolate, or strawberry. These are fixed results. They are not long tail results.
But over time, the technology of making ice cream at home gets better, the distribution of ingredients gets better, the infrastructure of equipment gets better, and the dispersion of recipes gets better.
Now you tell someone to "get you some ice cream," and you might get anything - an ice cream bar, an ice cream maker, an ice cream cone, or prepackaged gourmet ice cream of any of a hundred flavors, just to start. Whether or not you get the ice cream you want depends on how well you and the fetcher communicate.
Since so many options are now available, the fetcher tries hard to figure out precisely what you want, or else you'll ask someone else the next time. The fetcher makes some mistakes at first, but over time learns to take cues from you to figure out what you want.
And search engines do the same. When a search engine does it, it's called artificial intelligence. Search engines read signals via their algorithm to figure out what searchers want. And they're not very good at it yet.
Change is the Norm for Web Page Rankings
Just because your web page comes up number one today doesn't mean it will tomorrow. A web page has to prove it deserves its rankings over and over. Why? For one thing, because the pages on the web change all the time. Pages get changed; new pages get added; old pages get deleted or moved.
In case you're wondering if the web is really that big, consider that in 2008, Google added its one trillionth URL to the index. That's twelve zeros, three more than a trillion. There are less than 7 billion people in the world. That makes about 145 Web pages indexed for every person on Earth. And that was two years ago. And it doesn't include all the deep web that Google's not yet able to index. In other words - the search engines have an enormous task before them.
But a web page is also called on to prove it is worthy - or, technically, its relevance to any given query - because the search engines know for a fact that they haven't got it right yet - determining an authority page, that is. Google, Bing and Yahoo are not 100 percent confident in the status quo. And they never will be.
Search engines try to make sense of a constantly updated data stream using frequently applied algo changes. This means they continually must keep remapping the geography of the entire web world. So with every change, they must reconfigure what is what and which goes where and when.
The criteria used by Google, Bing and Yahoo to figure out all this stuff change over time as they get better at it - supposedly. Often, the search engines take a few steps backwards in the process and produce what's popularly called "garbage results."
What is "it" that the search engines are getting better at? The search engines strive to better figure out:
- what a search query means (what the user's intentions are when he performs a search) and
- the best places to send him to answer that query.
Search Engines Try, Try and Try Again: the Frustrations of Personalized Search
Search engines keep trying to get it right, but they make a lot of mistakes.
Recently, as part of their efforts to figure out what a search query means, Google Search has been working toward personalization of search results. Personalization of search results is, when you think about it, a step toward making search engines a type of AI, otherwise known as artificial intelligence.
[Edit: July 27, 2010 - Speaking of artificial intelligence, Google recently introduced implicit triggering into their SERPs for word definitions, meaning if you type in a word and Google "thinks" you want it defined, it will give you definitions even if you didn't ask for it.]
When the search engine adapts a search to what it thinks the visitor wants, based on the visitor's navigation habits, geolocation, search history, and other data tracked by Google, this can help get a user relevant results, but it also can yield results way off the mark. The search results can be irrelevant to the particular search.
Think You Know How You Rank? Maybe You Do...Maybe You Don't.Incidentally, many writers are now seeing personalized results when they check their own articles' web page position in Google Search. So when they see themselves ranking for search terms but not seeming to be getting the associated traffic, it's probably because these are not universal rankings. Their own search results are skewed by their search history, preferences, where they are located, etc.
There's the matter of trying to determine user error. Lately, for instance, Google Search has been displaying an annoying (to this writer) tendency to ask "Did you mean...XYZ" after you search for WXY, and not only that, but presumes you did, and gives you XYZ results first as a default.
That's a change from the past, when they didn't assume quite so boldly, but instead gently gave you what you asked for, then offered you the option to say, "oh, yes, right, that's what I mean."
Why the change? Presumably enough people happily clicked on the "oh, yes, you're right, Google, that's what I meant" link to give Google Search confidence that they were right in their guess far more than they were wrong. (I pesonally hope they lose that confidence soon, though, because they're wrong most of the time in our household...!)
The point is, as search engines evolve, they can seem to devolve. The SERPS degenerate into nonsense sometimes. This is not a reason to get alarmed - not unless you enjoy getting alarmed.
If you're like me, it's a reason to watch and wait and, if you detect a long-term pattern, to analyze the difference between the pages that are showing up on the first page of the SERPS and your page. That will tell you the way to go.
Search engines change - it's an obvious fact that everyone knows, but many writers don't realize what it means. They take it to mean that search engines change on whimsy and are like a force of nature that can't be predicted.
I disagree. As content writers, we're not left helpless. We simply must watch the clouds, feel the wind direction, and develop an approach to content that is adaptive.
Are self-created backlinks worth the effort? Not for me. Read about the pros and cons of building your own backlinks.
Think writing online is a temporary, fleeting opportunity? I don't. Learn why the future of writing is on the Internet.
Confused by your fluctuating earnings on your revenue share articles? Here are 20 Non-SEO Reasons Earnings Go Up and Down.