Comments on: On Site Search SEO Tip #1

By: Amanda

Amanda — Thu, 20 Nov 2008 21:27:27 +0000

Interesting post, though I have never heard of anything like this. I do not have a search field on my blog. Should I? It sounds complicated to do all of that. I feel like I need to have a degree in order SEO properly.

By: Marcel Fuursted

Marcel Fuursted — Tue, 18 Nov 2008 17:05:02 +0000

Actually there are websites that have recieved penalty for having indexed search results from the site search. So it’s not just a good idea… Its very important to exclude the search results in robots.txt

By: albert

albert — Mon, 17 Nov 2008 22:16:30 +0000

Another item that many people over look with the search function is seeing what people coming to your site are looking for. There is a way in Google Analytics to see what it is people are searching for on your site. If a lot of people are searching for something on your site you might want to take a clue and make it easier for them to find- ie a link pointing to the product or page maybe even consider using that term to optimize other pages on your site.

By: matt

matt — Mon, 17 Nov 2008 21:27:54 +0000

“Can you use the robots.txt file to exclude that search result, or any search result on the index.php page, without also excluding your index page from the bots? The answer in this example is no.”

Another option is to use a META NOINDEX tag in the header of the search results pages, instead of robots.txt. This would allow the index page to still be indexed. The main cost in this case is that the spiders would still crawl your search listings, possibly causing load and contention problems for your servers.

By: Dave Bascom

Dave Bascom — Mon, 17 Nov 2008 18:03:01 +0000

Google has come out and said that they don’t want to index search results pages that they feel don’t add value to their SERPs

Here’s what they say:

Use robots.txt to prevent crawling of search results pages or other auto-generated pages that don’t add much value for users coming from search engines.

http://www.google.com/support/webmasters/bin/answer.py?answer=35769

It’s kind of one of those touchy areas though, because most site owners feel that their SERPs pages are feakin’ awesome and should be included in the Google search. Sly webmasters try to get around this Google guideline by changing the file or directory name to something other than “search.” Although, if there’s nothing unique about the content on those pages, they won’t get indexed very well anyway, like you said.