Re: Manifold Job process isssue
One hour is quite a lot and will wreak havoc on the document queue. Karl On Tue, Nov 9, 2021 at 7:08 AM ritika jain wrote: > I have checked, there is only one hour time difference between docker > container and docker host > > On Tue, Nov 9, 2021 at 4:41 PM Karl Wright wrote: > >> If your docker image's clock is out of sync badly with the real world, >> then System.currentTimeMillis() may give bogus values, and ManifoldCF uses >> that to manage throttling etc. I don't know if that is the correct >> explanation but it's the only thing I can think of. >> >> Karl >> >> >> On Tue, Nov 9, 2021 at 4:56 AM ritika jain >> wrote: >> >>> >>> Hi All, >>> >>> I am using window shares connector , manifoldcf 2.14 and ES as output. I >>> have configured a job to process 60k of documents, Also these documents are >>> new and do not have corresponding values in DB and ES index. >>> >>> So ideally it should process/Index the documents as soon as the job >>> starts. >>> But Manifoldcf does not process anything for many hours of job start >>> up.I have tried restarting the docker container as well. But it didn't help >>> much. Also logs only correspond to Long running queries. >>> >>> Why does the manifold behave like that? >>> >>> Thanks >>> Ritika >>> >>
Re: Manifold Job process isssue
I have checked, there is only one hour time difference between docker container and docker host On Tue, Nov 9, 2021 at 4:41 PM Karl Wright wrote: > If your docker image's clock is out of sync badly with the real world, > then System.currentTimeMillis() may give bogus values, and ManifoldCF uses > that to manage throttling etc. I don't know if that is the correct > explanation but it's the only thing I can think of. > > Karl > > > On Tue, Nov 9, 2021 at 4:56 AM ritika jain > wrote: > >> >> Hi All, >> >> I am using window shares connector , manifoldcf 2.14 and ES as output. I >> have configured a job to process 60k of documents, Also these documents are >> new and do not have corresponding values in DB and ES index. >> >> So ideally it should process/Index the documents as soon as the job >> starts. >> But Manifoldcf does not process anything for many hours of job start up.I >> have tried restarting the docker container as well. But it didn't help >> much. Also logs only correspond to Long running queries. >> >> Why does the manifold behave like that? >> >> Thanks >> Ritika >> >
Re: Manifold Job process isssue
If your docker image's clock is out of sync badly with the real world, then System.currentTimeMillis() may give bogus values, and ManifoldCF uses that to manage throttling etc. I don't know if that is the correct explanation but it's the only thing I can think of. Karl On Tue, Nov 9, 2021 at 4:56 AM ritika jain wrote: > > Hi All, > > I am using window shares connector , manifoldcf 2.14 and ES as output. I > have configured a job to process 60k of documents, Also these documents are > new and do not have corresponding values in DB and ES index. > > So ideally it should process/Index the documents as soon as the job starts. > But Manifoldcf does not process anything for many hours of job start up.I > have tried restarting the docker container as well. But it didn't help > much. Also logs only correspond to Long running queries. > > Why does the manifold behave like that? > > Thanks > Ritika >
Manifold Job process isssue
Hi All, I am using window shares connector , manifoldcf 2.14 and ES as output. I have configured a job to process 60k of documents, Also these documents are new and do not have corresponding values in DB and ES index. So ideally it should process/Index the documents as soon as the job starts. But Manifoldcf does not process anything for many hours of job start up.I have tried restarting the docker container as well. But it didn't help much. Also logs only correspond to Long running queries. Why does the manifold behave like that? Thanks Ritika
