Skip to Content
author's profile photo Richard Hirsch
0
Richard Hirsch

Developing your own crawler

Posted on Mar 31, 2004 at 09:28 AM 20 Views

Hi,

Does anyone have any experience developing their own crawlers. We need a mechanism that searches specific Repositories and adds custom properties from external datasources to resources.

Or is there some other way to accomplish this (for example, a KM Service)

Thanks,

Dick

Add a comment
10|10000 characters needed characters exceeded

Assigned Tags

Related questions

2 Answers

  • Best Answer
    author's profile photo Paul Goetz
    Paul Goetz
    Posted on Apr 26, 2004 at 07:52 AM
    1

    Hi Dick,

    Although it is possible to develop an own crawler which would be able to do that, I'm not sure if its the thing you want to.

    Within the KM, the crawlers are separated in a "navigation layer" which handles the retrieval of resources as well as the navigation through the links, and a "result-handling layer" which does something with the resources found by the "navigation layer".

    For searching, the KM's indexmanagement uses it's own implementation of this "result-handling layer" which can not be changed by custom code.

    So, if you want to search a specific repository this can be handled by the indexmanagement (just by assigning the specific repository folder as a data source to an index).

    To add custom properties from an external datasource to resources (within a specific repository), it might be a good idea to implement a <b>PropertyReadFiler</b>.

    This PropertyReadFilter can be either configured to a repository or be registered dynamically by a global service.

    In order to have to have these properties being indexed, the have to be made "known" to the indexmanagement, which is done by configuring the appropriate information in the PropertyMetaData.

    Best regards,

    Paul

    Add a comment
    10|10000 characters needed characters exceeded

  • author's profile photo Former Member
    Former Member
    Posted on Apr 13, 2004 at 09:21 AM
    0

    Hi,

    maybe the Global Service/Property Metadata could help you.

    Regards, Josef

    Add a comment
    10|10000 characters needed characters exceeded

Before answering

You should only submit an answer when you are proposing a solution to the poster's problem. If you want the poster to clarify the question or provide more information, please leave a comment instead, requesting additional details. When answering, please include specifics, such as step-by-step instructions, context for the solution, and links to useful resources. Also, please make sure that you answer complies with our Rules of Engagement.
You must be Logged in to submit an answer.

Up to 10 attachments (including images) can be used with a maximum of 1.0 MB each and 10.5 MB total.