stop search engines from indexing certain pages

Discussion in 'Web Design and Development' started by Ad Pro, Dec 16, 2008.

  1. Ad Pro macrumors member

    Joined:
    Jul 4, 2007
    Location:
    UK
    #1
    I want only the home page of my site to be indexed by google (and other search engines) so that when people search for the site they always reach the homepage first.

    What is the best way to achieve this? I have read about robot.txt files... can anyone give me further details on how to do this, or link me to any good tutorials please.

    thanks in advance.
     
  2. xUKHCx Administrator emeritus

    xUKHCx

    Joined:
    Jan 15, 2006
    Location:
    The Kop
    #2
    Here is google's guide for creating a robots.txt file

    http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=40360

    And googles robot.txt file http://www.google.com/robots.txt


    If you want google to show the homepage over and above your other pages it is worthwhile creating and submitting a sitemap with your homepage priority 1.0 and the others with a lower priority.

    You can use a program such as the one linked below to generate site maps.

    http://www.apple.com/downloads/macosx/internet_utilities/igoomapxmlsitemapgeneratorandeditor.html

    If you have a google account it is worthwhile setting up google webmaster tools with your site.

    Or at least that is what I learnt in my recent excursion into getting a website up and running.
     
  3. angelwatt Moderator emeritus

    angelwatt

    Joined:
    Aug 16, 2005
    Location:
    USA
    #3
    The mentioned are good ways for doing this, but I would tell you to consider that what you're wanting is kind of bad. Why? Because when people do a search, they want the link to go to a page with what their looking for, not the home page of the site that has the answer. There's no telling how long it would take a visitor to find where the information is located at on your site. And most people, if they don't see what they were looking for on the first page they see, they will leave without a second thought. This is a known fact (don't have a reference on hand, but I'm certain of it).
     
  4. Ad Pro thread starter macrumors member

    Joined:
    Jul 4, 2007
    Location:
    UK
    #4

    Thanks xUKHCx those links are really helpful. I have also set up a google account - the webmaster tools look like they will come in useful both for my current project and future projects.
     
  5. Ad Pro thread starter macrumors member

    Joined:
    Jul 4, 2007
    Location:
    UK
    #5

    This is a really valid point, I don't think I will use the robot.txt method now, and I can see that it is important for people to find what they are looking for.

    I may consider submitting a site map however, as this will give the homepage priority but still return the search results people are looking for.

    Thanks for bringing that to my attention, I would have made a big mistake by hiding the other pages.
     

Share This Page