Skip to Content
Former Member
Sep 28, 2005 at 11:47 AM

Web crawler – Exclude content


Dear Web crawler experts

I have an issue regarding crawling of an intranet that consist of 5 iframes. I need to index the entire intranet, but I only want to index content from one frame, and still display the full page in a search result. Is this possible?

I can call the page directly by an secondary asp page like: http://intranet/info.asp?sdddsd, and call the entire page by use of http://intranet/default.asp.

I thought if I could add some tags either to tell the crawler to include or exclude the content part. I have looked at but can’t find any documentation on excluding sections under a web crawler.

Any ideas on how to solve this issue?