[
https://issues.apache.org/jira/browse/CONNECTORS-1497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16378229#comment-16378229
]
Karl Wright commented on CONNECTORS-1497:
-----------------------------------------
Thinking further: continuous crawling was designed originally for web crawling.
We really do not want to override the document schedule on document
rediscovery in that case; the current behavior is correct. It's only for
MODEL_ADD_CHANGE_DELETE where we want to do that, and people don't use that
with continuous crawling today.
Ok -- so I think if the connector model is ADD_CHANGE_DELETE, a proper
implementation should do several things:
(1) Force the recrawl interval to infinity, and/or disable its display in the
UI, for all continuous jobs based on MODEL_ADD_CHANGE_DELETE connectors.
(2) Change the behavior of seeding and document discovery through
ISeedingActivity and IProcessActivity to clear the document schedule ONLY when
the underlying connector model is MODEL_ADD_CHANGE_DELETE or
MODEL_CHAINED_ADD_CHANGE_DELETE.
I'd be happy with a suite of changes of that kind.
> Re-index seeded modified documents when the re-crawl interval is infinity and
> connector model is MODEL_ADD_CHANGE
> -------------------------------------------------------------------------------------------------------------------
>
> Key: CONNECTORS-1497
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1497
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Framework agents process
> Affects Versions: ManifoldCF 2.9.1
> Reporter: Ahmed Mahfouz
> Assignee: Karl Wright
> Priority: Major
> Attachments: CONNECTORS-1497.patch, CONNECTORS-1497.patch2,
> CONNECTORS-1497.patch3
>
>
> Trying to avoid a full scan of all documents for a better efficiency with a
> large number of documents. I tried so many different setting for the Jobs but
> I couldn't accomplish that. Especially when the repository connector model is
> MODEL_ADD_CHANGE I was expecting the modified documents seeded should be
> re-indexed immediately similar to the new seeds but I found out it uses the
> re-crawl time as the scheduled time and it waits for the full scan to get
> re-indexed. I avoided full scan by setting the re-crawl interval to infinity
> but still, my modified documents seeds were not getting indexed. After
> digging into the code for quite good time. I did some modification to the
> JobManager and it worked for me. I would like to share the change with you
> for review so I opened this ticket.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
