1. Select a user agent (robot) in the below. The default is "ALL User Agents", then click on the "ADD
USER AGENT" button.
2. Type in folders and/or web pages that you do not want indexed by search engines, then
click on the "ADD" button. Make sure you not add a forward slash, for example if I would like that my html folder not get indexed by search engines type the folowing in "Add directories" html/.
Repeat this step until you
have added all of the files or folders you want under the "User Agent". If
you don't want a user agent to index ANY of your site, LEAVE THE
TEXTFIELD BLANK and click on the "ADD" button below. A "/" will appear
after "Disallow:" in the code below.
Once step 2 is
complete, you may repeat steps 1 & 2 again until
you have added all of the "User Agents" you want, and files under
each that you want to "Disallow".
3. When complete,
click on "COPY CODE" to save the robots.txt
code in a text file, after you can start uploading it to your server.
Please make sure that you upload it to your root were the index is located.
Robots.txt Generator
Select a
User Agent
Add directories/files you DO NOT want
indexed
Three Examples:
1: folderName/myPage.html
2: folderName/
3: myPage.html
Details and meaning of these tags
Allow: /myDirectory/secretPage.html # Or my "secretPage.html" inside "myDirectory"
User-agent: Scooter # This is AltaVista's robot
Disallow: /somePage.asp # This means AltaVista is not allowed to index "somePage.asp", but the rest can
User-agent: Googlebot-Images # This is Google's image search robot
Disallow: /myImages/ # This means Google images is not allowed to index my directory "myImages"
Disallow: /myPage-Full-Of-Images.html # Or "myPage-Full-Of-Images.html"
User-agent: WebCrawler # This is WebCrawler's robot
Disallow: / # This means WebCrawler is not allowed to index ANY of my site
The above example is heavily commented to help you.
if you want to add comments, make sure you use a # sign in front of each comment. The robots will ignore
these comments. Below you can view a example we use ourself
# Disallow crawlers access to certain pages.
Make sure there is 1 empty space in front of and after the # sign.
Personally, I would leave out any comments.
You don't want to make a mistake that could anger any "spiders" and keep them
from indexing your site altogether.
For more detailed information on custom
optimization or search
engine submission for your company or web-site,
please contact us, by inquiry form or e-mail us
Web Hosting
SEO WATCH is known for successful seo marketing, and offers 24/7 support to our web hosting clients. If you have any questions about our seo or hosting packages please call us: +66 (0)35 231214
... Web Hosting Services
Search Engine Friendly Hosting
Choosing
a hosting company for your site is a big decision. When
you make a decision to host your web site with a web host
provider, you're actually placing your search engine rankings
and income in the hands of your hosting provider. The wrong
web hosting plan or service provider can cause big damages
to your search engine ranking and you not aware of it.
... SEO
Friendly Hosting
Online Shop Optimization
Search
engine optimization for E-commerce sites that are based
on content management systems, os Commerce, phpShop, Zen
Cart, ASP, this means: URLs and pages are dynamically generated
by the scripting language fetching all elements from the
database.
... E-Commerce
Optimization
Search Engine News
Your web site should be informative and search engine friendly. If you are the owner of a web site, make sure that there are no technical errors that prevent search engines from indexing your web pages and only hire a professional search engine optimization company to improve ranking in search results, if you cannot manage yourself.
... Search Engine News
SEO Optimization Packages
We offer many different kind of flexible seo packages to keep our customers satisfied, to be affordable for all marketing department budgets and achieve the best results for their web site.
...
SEO Optimization Packages
Successful SEO Projects
View
our clients and companies who used our seo consulting services.
... Successful SEO Projects
