|
|
#1 SUN WEB SITE PROMOTION and DESIGNGet high search engine rankings and targeted traffic.Web site designers who understand marketing. Get Free Web Design and/or Internet Marketing Quote |
![]()
Design Your Site to Rank High
Search Engine Spiders and Robots Tips about Search Engine Spiders
Web Site Builder,
Free HostingFree Design Quote from Web Designer Directory
Free Internet Marketing Quote
Pay Per Click Services
More Reports Home
Search engines use spiders and robots to index web pages to add to their listings. A spider or robot is a software program that follows links throughout the internet looking for pages to add and update.
An engine can only index your site if it can find it through links at another site or if you submit your site directly to the engine. Once it finds your site it can only index your pages if it can follow links to your inside pages.
Why Some Sites Never Get Listed
Certain web design techniques blocks these spiders and robots thereby preventing your site from being indexed. Most spiders can not follow links in javascript drop down boxes. Others can not follow links found in Flash sites. Others can not or will not follow links in dynamic urls, i.e., those containing the "?" in the domain address.
Make Sure Your Site Gets Spidererd
All spiders can follow links that are traditional "a href" blue underlined link. Make sure there is at least one of these traditional types of links to every page on your site. It doesn't have to be directly from the home page but there should be a way a spider can follow through a series of pages to find every page.
