|
Five easy to make mistakes that keep search engine
robots away from your website
Many webmasters don't get high rankings on Google and other
search engines just because Google's indexing robot has difficulty
to index their web pages.
Search engine robots are very simple software programs.
If an indexing robot cannot find the content of your website
immediately, it will skip your site and go to the next link
in the list. For that reason, it is very important to make
sure that search engine robots can index your web pages without
problems.
Here are the top 5 elements that drive search engine robots
away:
Reason 1: Your robots.txt file is damaged or it
contains a typo
If search engine robots misinterpret your robots.txt file,
they might completely ignore your web pages.
Double check your robots.txt file and make sure that you
use the disallow parameter only for web pages that you
really don't want to have indexed.
Reason 2: Your URLs contain too many variables
URLs with many variables can cause problems with search
engine robots. If your URLs contain too many variables,
search engine robots might ignore your pages.
Here's Google's official statement about web pages with
many variables:
"Google indexes dynamically generated webpages, including
.asp pages, .php pages, and pages with question marks
in their URLs. However, these pages can cause problems
for our crawler and may be ignored."
Reason 3: You use session IDs in your URLs
Many search engines don't index URLs that contain session
IDs because they can lead to duplicate content problems.
If possible, avoid session IDs in your URLs. Better use
cookies to store session IDs.
Reason 4: Your web pages contain too much code
Of course, your web pages can contain JavaScript code,
CSS code and other script code that is not directly related
to your content. Visit your website with a web browser
and select "View source" or "View HTML source".
If it is difficult for you to spot the actual content
of your website then search engines might also have difficulty
to parse your pages.
Reason 5: Your website navigation causes problems
Fancy JavaScript or DHTML menus cannot be parsed by most
search engine robots. Flash or AJAX menus are even worse
when it comes to website navigation.
As mentioned above, search engine robots are very simple
programs. They can follow HTML links, all other links can
cause problems.
Optimized web
page content and good inbound
links are crucial for high search engine rankings.
However, the best content and the best links won't help
you much if search engines cannot index your pages.
Make sure that search engine spiders can index your web
pages without problems so that your web pages can get the
rankings they deserve.
|
