Robots TXT

Get the latest Flash Player to see this video.

* HD format might not be available

Play in: Low Quality | High Quality | HD Quality *

Local Comments (0)

Nothing Found!

Youtube Comments (3)

Jul 31, 2011
WafflesRDelicious Says:
There are couple of errors in this video: 1) Allow is non-standard, a normal robots.txt will only disallow access 2) If you have a private folder, use .htaccess to restrict access to it, if you put a url to a private folder in your robots.txt everyone will be able to read your robots.txt and see where your "private" files are and look at them

Feb 2, 2011
jlwqwerty Says:
very informative thanks

Oct 17, 2010
nikhil612 Says:
Hi, what if I have multiple sitemaps. eg: one for my blog (example.com/blog/sitemap.xml) and a 2 for my site Great video btw, created my robot.txt, looking at this video.