Robots.txt file

Go down

Robots.txt file Empty Re: Robots.txt file

Post by markcooper006 on Thu Feb 16, 2012 1:30 am

hi there jessica, well this depends on the platform your are using.
if you not want a particular page to be indexed in google then instead of modifying robots.txt try a noindex tag at header. a basic robots.txt should also be there at root all the time.



__________________________
Tier 5

markcooper006

Posts : 12
Points : 2871
Reputation : 0
Join date : 2012-02-08

Back to top Go down

Robots.txt file Empty Re: Robots.txt file

Post by jackshobs on Thu Jun 21, 2012 4:18 am

Robots.txt file is a simple text file that must be placed in root directory of a website. The robots.txt file is useful in keeping your spiders from accessing parts folders and files in your hosting directory that are totally unrelated to your actual web site content.

jackshobs

Posts : 30
Points : 2759
Reputation : 0
Join date : 2012-06-15
Age : 31
Location : India

http://www.rtmlguru.com

Back to top Go down

Robots.txt file Empty Re:

Post by NileLars on Thu Oct 11, 2012 7:04 am

Robots.txt are really important for make your page private. Robots file are very important to add for every domain. You must add this file to your website.

NileLars

Posts : 15
Points : 2626
Reputation : 0
Join date : 2012-10-11
Age : 33
Location : Chicago

http://mobile.softwebsolutions.com/

Back to top Go down

Robots.txt file Empty robots file

Post by expertSEO on Mon Nov 05, 2012 1:01 pm

Hi

This file helps site owner in restricting the pages he/she does not want to crawl or index for neither as a page or update.It is done for search engines.

Thanks
ESTI

expertSEO

Posts : 10
Points : 2596
Reputation : 0
Join date : 2012-11-05

Back to top Go down

Robots.txt file Empty Robots.txt

Post by Seo gurgaon on Tue Mar 19, 2013 4:17 am

Robots.txt files are used to give an indication or command to Search Engine Robots that the site or a webpage is to follow or not to follow; index or not to index.
Seo gurgaon
Seo gurgaon

Posts : 76
Points : 2540
Reputation : 0
Join date : 2013-03-11
Age : 27
Location : Gurgaon, HR, India

http://www.seogurgaon.com/

Back to top Go down

Robots.txt file Empty Re: Robots.txt file

Post by RafaelJames on Mon May 20, 2013 11:15 pm

Robot.txt file is used in on page SEO to indicate Google spiders that which web page is to visit and which page not to visit. The pages which are restricted to be visited cannot be indexed by Google.

RafaelJames

Posts : 80
Points : 2637
Reputation : 0
Join date : 2012-12-08

Back to top Go down

Robots.txt file Empty Re: Robots.txt file

Post by jesicafrancis on Mon Dec 09, 2013 12:15 am

Dear all,
The Robot Exclusion Standard, otherwise called the Robots Exclusion Protocol or robots.txt convention, is a gathering to exhorting chipping in web crawlers and other web robots about gaining entrance to all or part of a site which is generally openly perceptible. Robots are regularly utilized via web indexes to arrange and document sites, or by webmasters to edit source code.

jesicafrancis

Posts : 13
Points : 2209
Reputation : 0
Join date : 2013-12-02
Age : 31
Location : Chennai, India

http://www.agriya.com/

Back to top Go down

Robots.txt file Empty Re: Robots.txt file

Post by kumar29 on Mon Apr 14, 2014 7:55 am

The Robots.txt File lets you control what search engine spiders should and shouldn’t crawl on your website. It’s a file that many WordPress users forget to add. For those who do, it can be unclear what folders should and shouldn’t be blocked.

kumar29

Posts : 24
Points : 2535
Reputation : 0
Join date : 2013-01-25
Age : 31
Location : India

http://www.pulsegym.in

Back to top Go down

Robots.txt file Empty Re: Robots.txt file

Post by JuanaLott on Tue Apr 29, 2014 1:59 am

Robot.txt file is helpful for you if you want to hide any contents from search engines. Like any personal data you have on your website, but you don't want to show them another world the use robot.txt. Mention your data on reboot.txt then searches engine will not crawl it.

JuanaLott

Posts : 6
Points : 2052
Reputation : 0
Join date : 2014-04-29

http://www.greencircleagency.com/

Back to top Go down

Robots.txt file Empty Re: Robots.txt file

Post by Sponsored content


Sponsored content


Back to top Go down

Back to top


 
Permissions in this forum:
You cannot reply to topics in this forum