Sidebar Sidebar
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

stop search engines from indexing certain pages

A

Ad Pro

macrumors member
Original poster
Jul 4, 2007
86
0
UK
I want only the home page of my site to be indexed by google (and other search engines) so that when people search for the site they always reach the homepage first.

What is the best way to achieve this? I have read about robot.txt files... can anyone give me further details on how to do this, or link me to any good tutorials please.

thanks in advance.
 
xUKHCx

xUKHCx

Administrator emeritus
Jan 15, 2006
12,583
9
The Kop
Here is google's guide for creating a robots.txt file

http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=40360

And googles robot.txt file http://www.google.com/robots.txt


If you want google to show the homepage over and above your other pages it is worthwhile creating and submitting a sitemap with your homepage priority 1.0 and the others with a lower priority.

You can use a program such as the one linked below to generate site maps.

http://www.apple.com/downloads/macosx/internet_utilities/igoomapxmlsitemapgeneratorandeditor.html

If you have a google account it is worthwhile setting up google webmaster tools with your site.

Or at least that is what I learnt in my recent excursion into getting a website up and running.
 
angelwatt

angelwatt

Moderator emeritus
Aug 16, 2005
7,852
9
USA
The mentioned are good ways for doing this, but I would tell you to consider that what you're wanting is kind of bad. Why? Because when people do a search, they want the link to go to a page with what their looking for, not the home page of the site that has the answer. There's no telling how long it would take a visitor to find where the information is located at on your site. And most people, if they don't see what they were looking for on the first page they see, they will leave without a second thought. This is a known fact (don't have a reference on hand, but I'm certain of it).
 
A

Ad Pro

macrumors member
Original poster
Jul 4, 2007
86
0
UK
xUKHCx said:
Here is google's guide for creating a robots.txt file

http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=40360

And googles robot.txt file http://www.google.com/robots.txt


If you want google to show the homepage over and above your other pages it is worthwhile creating and submitting a sitemap with your homepage priority 1.0 and the others with a lower priority.

You can use a program such as the one linked below to generate site maps.

http://www.apple.com/downloads/macosx/internet_utilities/igoomapxmlsitemapgeneratorandeditor.html

If you have a google account it is worthwhile setting up google webmaster tools with your site.

Or at least that is what I learnt in my recent excursion into getting a website up and running.
Click to expand...


Thanks xUKHCx those links are really helpful. I have also set up a google account - the webmaster tools look like they will come in useful both for my current project and future projects.
 
A

Ad Pro

macrumors member
Original poster
Jul 4, 2007
86
0
UK
angelwatt said:
The mentioned are good ways for doing this, but I would tell you to consider that what you're wanting is kind of bad. Why? Because when people do a search, they want the link to go to a page with what their looking for, not the home page of the site that has the answer. There's no telling how long it would take a visitor to find where the information is located at on your site. And most people, if they don't see what they were looking for on the first page they see, they will leave without a second thought. This is a known fact (don't have a reference on hand, but I'm certain of it).
Click to expand...


This is a really valid point, I don't think I will use the robot.txt method now, and I can see that it is important for people to find what they are looking for.

I may consider submitting a site map however, as this will give the homepage priority but still return the search results people are looking for.

Thanks for bringing that to my attention, I would have made a big mistake by hiding the other pages.
 
You must log in or register to reply here.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.
Top Bottom