Re: Job stucked with cleaning up status

2023-02-03 Thread Karl Wright
The shutdown procedure for ManifoldCF involves sending interruptions (or socket interruptions) to all worker threads. These then put the threads in the "terminated" state, one by one. So you should only get this if you shut down the agents process, or try to. The handling for this is correct,

Re: Job stucked with cleaning up status

2023-02-02 Thread Artem Abeleshev
Karl, good day! Thank you for the hint! It was very useful! Actually, you was right and the actual problem was about the connection. But I doesn't expect it would be so dramatic. Here is what I found using some debugging: First I have found the actual code that was responsible for the deletion

Re: Job stucked with cleaning up status

2023-01-29 Thread Karl Wright
Hi, 2.22 makes no changes to the way document deletions are processed over probably 10 previous versions of ManifoldCF. What likely is the case is that the connection to the output for the job you are cleaning up is down. When that happens, the documents are queued but the delete worker threads

Job stucked with cleaning up status

2023-01-29 Thread Artem Abeleshev
Hi, everyone! Another problem that I got sometimes. We are using ManifoldCF 2.22.1 with multiple nodes in our production. The creation of the MCF job pipeline is handled via the API calls from our service. We create jobs, repositories and output repositories. The crawler extracts documents and