R: Job stuck without message

[Bisonti Mario]

I attatched  a row that correspond to a row of one of these documents in this 
mail



I obtain the pid of:
"/bin/bash -e  
/opt/manifoldcf/multiprocess-zk-example-proprietary/start-agents.sh"
The pid is 1233

I tried to use
sudo jstack -l 1233 > /tmp/jstack_start_agent.log

but I obtain:
1233: Unable to open socket file /proc/1233/cwd/.attach_pid1233: target process 
1233 doesn't respond within 10500ms or HotSpot VM not loaded

Perhaps isn’t it the right way to obtain a thread dump?
Excuse me but I am not a Linux expert..




Da: Karl Wright 
Inviato: mercoledì 28 novembre 2018 16:36
A: user@manifoldcf.apache.org
Oggetto: Re: Job stuck without message

Another thing you could do is get a thread dump of the agents process.

Karl


On Wed, Nov 28, 2018 at 10:35 AM Karl Wright 
mailto:daddy...@gmail.com>> wrote:
Can you look into the database jobqueue table and provide a row that 
corresponds to one of these documents?

Thanks,
Karl


On Wed, Nov 28, 2018 at 10:26 AM Bisonti Mario 
mailto:mario.biso...@vimar.com>> wrote:
Hallo.
Repository has Max connection=10

In the Document Status report” I see many item with :
State=“Not yet processed”
Status=”Ready for processing”
Scheduled=01-01-1970 01:00:00.000”
Scheduled Action=”Process”




But the job no more walk..


Da: Karl Wright mailto:daddy...@gmail.com>>
Inviato: mercoledì 28 novembre 2018 16:03
A: user@manifoldcf.apache.org
Oggetto: Re: Job stuck without message

"Pipe instances are busy" occurs because you are overloading the SMB access to 
your servers.  How many connections do you have allocated for your repository 
connection?  You probably want to limit this to 2-3 if you see this error a 
lot, and it appears you do.

" Tika Server: Tika Server rejects: Tika Server rejected document with the 
following reason: Unprocessable Entity" means the document is not properly 
formed XML.  The rejection will mean the document isn't indexed, but this will 
not stop the job.

If nothing is happening and you don't know why, I'd suggest looking at the 
Document Status report to figure out what documents are not being processed and 
why.  It is quite possible they are all in the process of being retried because 
of the "Pipe instances" issue above.

Karl

On Wed, Nov 28, 2018 at 9:46 AM Bisonti Mario 
mailto:mario.biso...@vimar.com>> wrote:
Hallo Karl.
I take this ticket because,now, after I use zookeeper, my job works for 7 hours 
and now it is in hang status.
I see running but it seems hanging, no log from 1 hour

This is the last manifoldcf.log lines:


at jcifs.smb.SmbFile.open(SmbFile.java:1010) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFileOutputStream.(SmbFileOutputStream.java:142) 
~[jcifs-1.3.18.3.jar:?]
at 
jcifs.smb.TransactNamedPipeOutputStream.(TransactNamedPipeOutputStream.java:32)
 ~[jcifs-1.3.18.3.jar:?]
at 
jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) 
~[jcifs-1.3.18.3.jar:?]
at 
jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) 
~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) 
~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) 
~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) 
~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951) 
~[jcifs-1.3.18.3.jar:?]
at 
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2446)
 [mcf-jcifs-connector.jar:?]
at 
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1222)
 [mcf-jcifs-connector.jar:?]
at 
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
 [mcf-jcifs-connector.jar:?]
at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) 
[mcf-pull-agent.jar:?]
WARN 2018-11-28T14:46:21,524 (Worker thread '59') - JCIFS: Possibly transient 
exception detected on attempt 1 while getting share security: All pipe 
instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569) 
~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:669) 
~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) 
~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:993) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1010) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFileOutputStream.(SmbFileOutputStream.java:142) 
~[jcifs-1.3.18.3.jar:?]
at 

R: Job stuck without message

[Bisonti Mario]

Hallo Karl.
I take this ticket because,now, after I use zookeeper, my job works for 7 hours 
and now it is in hang status.
[cid:image001.png@01D48731.7DA5C470]
I see running but it seems hanging, no log from 1 hour

This is the last manifoldcf.log lines:


at jcifs.smb.SmbFile.open(SmbFile.java:1010) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFileOutputStream.(SmbFileOutputStream.java:142) 
~[jcifs-1.3.18.3.jar:?]
at 
jcifs.smb.TransactNamedPipeOutputStream.(TransactNamedPipeOutputStream.java:32)
 ~[jcifs-1.3.18.3.jar:?]
at 
jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) 
~[jcifs-1.3.18.3.jar:?]
at 
jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) 
~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) 
~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) 
~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) 
~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951) 
~[jcifs-1.3.18.3.jar:?]
at 
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2446)
 [mcf-jcifs-connector.jar:?]
at 
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1222)
 [mcf-jcifs-connector.jar:?]
at 
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
 [mcf-jcifs-connector.jar:?]
at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) 
[mcf-pull-agent.jar:?]
WARN 2018-11-28T14:46:21,524 (Worker thread '59') - JCIFS: Possibly transient 
exception detected on attempt 1 while getting share security: All pipe 
instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569) 
~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:669) 
~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) 
~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:993) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1010) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFileOutputStream.(SmbFileOutputStream.java:142) 
~[jcifs-1.3.18.3.jar:?]
at 
jcifs.smb.TransactNamedPipeOutputStream.(TransactNamedPipeOutputStream.java:32)
 ~[jcifs-1.3.18.3.jar:?]
at 
jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) 
~[jcifs-1.3.18.3.jar:?]
at 
jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) 
~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) 
~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) 
~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) 
~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951) 
~[jcifs-1.3.18.3.jar:?]
at 
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2446)
 [mcf-jcifs-connector.jar:?]
at 
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1222)
 [mcf-jcifs-connector.jar:?]
at 
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
 [mcf-jcifs-connector.jar:?]
at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) 
[mcf-pull-agent.jar:?]
WARN 2018-11-28T14:46:25,232 (Worker thread '45') - Tika Server: Tika Server 
rejects: Tika Server rejected document with the following reason: Unprocessable 
Entity
WARN 2018-11-28T14:46:25,277 (Worker thread '70') - Tika Server: Tika Server 
rejects: Tika Server rejected document with the following reason: Unprocessable 
Entity
WARN 2018-11-28T14:46:28,825 (Worker thread '27') - JCIFS: Possibly transient 
exception detected on attempt 1 while getting share security: All pipe 
instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569) 
~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:669) 
~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) 
~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:993) ~[jcifs-1.3.18.3.jar:?]
  

R: Job stuck without message

[Bisonti Mario]

Thanks a lot, Karl.
I created a ticket.
https://issues.apache.org/jira/browse/CONNECTORS-1554


Thanks

Mario



Da: Karl Wright 
Inviato: martedì 6 novembre 2018 14:28
A: user@manifoldcf.apache.org
Oggetto: Re: Job stuck without message

ok, can you create a ticket?  Also, I'd appreciate it if you can look at the 
simple history for one of these documents; I need to see what happened to it 
last.

Thanks,
Karl


On Tue, Nov 6, 2018 at 7:32 AM Bisonti Mario 
mailto:mario.biso...@vimar.com>> wrote:
My version is 2.11




Da: Karl Wright mailto:daddy...@gmail.com>>
Inviato: martedì 6 novembre 2018 13:07
A: user@manifoldcf.apache.org
Oggetto: Re: Job stuck without message

Thanks.
What version of ManifoldCF are you using?  We fixed a problem a while back 
having to do with documents that (because of error processing) get put into a 
"ready for processing" state which don't have any document priority set.  But 
this should have been addressed, certainly, by the most recent release and 
probably by 2.10 as well.

Karl


On Tue, Nov 6, 2018 at 5:43 AM Bisonti Mario 
mailto:mario.biso...@vimar.com>> wrote:
Hallo Karl.
When it hangs I see in the Queue status:

And in the Document Status:


Da: Karl Wright mailto:daddy...@gmail.com>>
Inviato: martedì 30 ottobre 2018 19:32
A: user@manifoldcf.apache.org
Oggetto: Re: Job stuck without message

What I am interested in now is the Document Status report for any one of the 
documents that is 'stuck'.  The next crawl time value is the critical field.  
Can you include an example?

Karl

On Tue, Oct 30, 2018, 12:36 PM Bisonti Mario 
mailto:mario.biso...@vimar.com>> wrote:
Thanks a lot, Karl.

It happens that the job starts, it works and index for an hour and after it 
frezzes, I haven’t error or waiting status in Document Queue o Simple History, 
I have only “OK” status so, I haven’t failures.

I am not able to see other log errors other from the manifoldcf.log

Solr server is ok
Tika server is ok
Agent is ok
Tomcat with ManifoldCF is ok

I could search if I could to put in info log mode for example Tika servrer or 
Solr.

Thanks..


Da: Karl Wright mailto:daddy...@gmail.com>>
Inviato: martedì 30 ottobre 2018 16:38
A: user@manifoldcf.apache.org
Oggetto: Re: Job stuck without message

Hi Mario,

Please look at the Queue Status report to determine what is waiting and why it 
is waiting.
You can also look at the Simple History to see what has been happening.  If you 
are getting 100% failures in fetching documents then you may need to address 
this because your infrastructure is unhappy.  If the failure is something that 
indicates that the document is never going to be readable, that's a different 
problem and we might need to address that in the connector.

Karl


On Tue, Oct 30, 2018 at 10:33 AM Bisonti Mario 
mailto:mario.biso...@vimar.com>> wrote:

Thanks a lot Karl

Yes, I see many docs in the docs queue but they are inactive.

Infact i see that no more docs are indexed in Solr and I see that job is with 
the same number of docs Active (35012)




Da: Karl Wright mailto:daddy...@gmail.com>>
Inviato: martedì 30 ottobre 2018 13:59
A: user@manifoldcf.apache.org
Oggetto: Re: Job stuck without message

The reason the job is "stuck" is because:

' JCIFS: Possibly transient exception detected on attempt 1 while getting share 
security: All pipe instances are busy.'

This means that ManifoldCF will retry this document for a while before it gives 
up on it.  It appears to be stuck but it is not.  You can verify that by 
looking at the Document Queue report to see what is queued and what times the 
various documents will be retried.

Karl


On Tue, Oct 30, 2018 at 5:07 AM Bisonti Mario 
mailto:mario.biso...@vimar.com>> wrote:
Hallo.

I started a job that works for some minutes, and after it stucks.

In the manifoldcf.log I see:
at 
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
 [mcf-jcifs-connector.jar:?]
at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) 
[mcf-pull-agent.jar:?]
WARN 2018-10-30T09:21:31,440 (Worker thread '2') - Tika Server: Tika Server 
rejects: Tika Server rejected document with the following reason: Unprocessable 
Entity
WARN 2018-10-30T09:21:33,502 (Worker thread '14') - Tika Server: Tika Server 
rejects: Tika Server rejected document with the following reason: Unprocessable 
Entity
WARN 2018-10-30T09:21:37,725 (Worker thread '30') - Tika Server: Tika Server 
rejects: Tika Server rejected document with the following reason: Unprocessable 
Entity
WARN 2018-10-30T09:21:44,406 (Worker thread '49') - Tika Server: Tika Server 
rejects: Tika Server rejected document with the following reason: Unprocessable 
Entity
WARN 2018-10-30T09:21:47,310 (Worker thread '15') - Tika Server: Tika Server 
rejects: Tika 

R: Job stuck without message

[Bisonti Mario]

My version is 2.11




Da: Karl Wright 
Inviato: martedì 6 novembre 2018 13:07
A: user@manifoldcf.apache.org
Oggetto: Re: Job stuck without message

Thanks.
What version of ManifoldCF are you using?  We fixed a problem a while back 
having to do with documents that (because of error processing) get put into a 
"ready for processing" state which don't have any document priority set.  But 
this should have been addressed, certainly, by the most recent release and 
probably by 2.10 as well.

Karl


On Tue, Nov 6, 2018 at 5:43 AM Bisonti Mario 
mailto:mario.biso...@vimar.com>> wrote:
Hallo Karl.
When it hangs I see in the Queue status:
[cid:image001.png@01D475C5.DCE8D300]

And in the Document Status:
[cid:image002.png@01D475C5.DCE8D300]


Da: Karl Wright mailto:daddy...@gmail.com>>
Inviato: martedì 30 ottobre 2018 19:32
A: user@manifoldcf.apache.org
Oggetto: Re: Job stuck without message

What I am interested in now is the Document Status report for any one of the 
documents that is 'stuck'.  The next crawl time value is the critical field.  
Can you include an example?

Karl

On Tue, Oct 30, 2018, 12:36 PM Bisonti Mario 
mailto:mario.biso...@vimar.com>> wrote:
Thanks a lot, Karl.

It happens that the job starts, it works and index for an hour and after it 
frezzes, I haven’t error or waiting status in Document Queue o Simple History, 
I have only “OK” status so, I haven’t failures.

I am not able to see other log errors other from the manifoldcf.log

Solr server is ok
Tika server is ok
Agent is ok
Tomcat with ManifoldCF is ok

I could search if I could to put in info log mode for example Tika servrer or 
Solr.

Thanks..


Da: Karl Wright mailto:daddy...@gmail.com>>
Inviato: martedì 30 ottobre 2018 16:38
A: user@manifoldcf.apache.org
Oggetto: Re: Job stuck without message

Hi Mario,

Please look at the Queue Status report to determine what is waiting and why it 
is waiting.
You can also look at the Simple History to see what has been happening.  If you 
are getting 100% failures in fetching documents then you may need to address 
this because your infrastructure is unhappy.  If the failure is something that 
indicates that the document is never going to be readable, that's a different 
problem and we might need to address that in the connector.

Karl


On Tue, Oct 30, 2018 at 10:33 AM Bisonti Mario 
mailto:mario.biso...@vimar.com>> wrote:

Thanks a lot Karl

Yes, I see many docs in the docs queue but they are inactive.

Infact i see that no more docs are indexed in Solr and I see that job is with 
the same number of docs Active (35012)




Da: Karl Wright mailto:daddy...@gmail.com>>
Inviato: martedì 30 ottobre 2018 13:59
A: user@manifoldcf.apache.org
Oggetto: Re: Job stuck without message

The reason the job is "stuck" is because:

' JCIFS: Possibly transient exception detected on attempt 1 while getting share 
security: All pipe instances are busy.'

This means that ManifoldCF will retry this document for a while before it gives 
up on it.  It appears to be stuck but it is not.  You can verify that by 
looking at the Document Queue report to see what is queued and what times the 
various documents will be retried.

Karl


On Tue, Oct 30, 2018 at 5:07 AM Bisonti Mario 
mailto:mario.biso...@vimar.com>> wrote:
Hallo.

I started a job that works for some minutes, and after it stucks.

In the manifoldcf.log I see:
at 
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
 [mcf-jcifs-connector.jar:?]
at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) 
[mcf-pull-agent.jar:?]
WARN 2018-10-30T09:21:31,440 (Worker thread '2') - Tika Server: Tika Server 
rejects: Tika Server rejected document with the following reason: Unprocessable 
Entity
WARN 2018-10-30T09:21:33,502 (Worker thread '14') - Tika Server: Tika Server 
rejects: Tika Server rejected document with the following reason: Unprocessable 
Entity
WARN 2018-10-30T09:21:37,725 (Worker thread '30') - Tika Server: Tika Server 
rejects: Tika Server rejected document with the following reason: Unprocessable 
Entity
WARN 2018-10-30T09:21:44,406 (Worker thread '49') - Tika Server: Tika Server 
rejects: Tika Server rejected document with the following reason: Unprocessable 
Entity
WARN 2018-10-30T09:21:47,310 (Worker thread '15') - Tika Server: Tika Server 
rejects: Tika Server rejected document with the following reason: Unprocessable 
Entity
WARN 2018-10-30T09:21:52,000 (Worker thread '27') - Tika Server: Tika Server 
rejects: Tika Server rejected document with the following reason: Unprocessable 
Entity
WARN 2018-10-30T09:21:53,526 (Worker thread '15') - Tika Server: Tika Server 
rejects: Tika Server rejected document with the following reason: Unprocessable 
Entity
WARN 2018-10-30T09:22:04,511 (Worker thread '3') - JCIFS: Possibly transient 
exception 

R: Job stuck without message

[Bisonti Mario]

Thanks a lot, Karl.

It happens that the job starts, it works and index for an hour and after it 
frezzes, I haven’t error or waiting status in Document Queue o Simple History, 
I have only “OK” status so, I haven’t failures.

I am not able to see other log errors other from the manifoldcf.log

Solr server is ok
Tika server is ok
Agent is ok
Tomcat with ManifoldCF is ok

I could search if I could to put in info log mode for example Tika servrer or 
Solr.

Thanks..


Da: Karl Wright 
Inviato: martedì 30 ottobre 2018 16:38
A: user@manifoldcf.apache.org
Oggetto: Re: Job stuck without message

Hi Mario,

Please look at the Queue Status report to determine what is waiting and why it 
is waiting.
You can also look at the Simple History to see what has been happening.  If you 
are getting 100% failures in fetching documents then you may need to address 
this because your infrastructure is unhappy.  If the failure is something that 
indicates that the document is never going to be readable, that's a different 
problem and we might need to address that in the connector.

Karl


On Tue, Oct 30, 2018 at 10:33 AM Bisonti Mario 
mailto:mario.biso...@vimar.com>> wrote:

Thanks a lot Karl

Yes, I see many docs in the docs queue but they are inactive.

Infact i see that no more docs are indexed in Solr and I see that job is with 
the same number of docs Active (35012)

[cid:image002.jpg@01D47065.DEFF7B40]



Da: Karl Wright mailto:daddy...@gmail.com>>
Inviato: martedì 30 ottobre 2018 13:59
A: user@manifoldcf.apache.org
Oggetto: Re: Job stuck without message

The reason the job is "stuck" is because:

' JCIFS: Possibly transient exception detected on attempt 1 while getting share 
security: All pipe instances are busy.'

This means that ManifoldCF will retry this document for a while before it gives 
up on it.  It appears to be stuck but it is not.  You can verify that by 
looking at the Document Queue report to see what is queued and what times the 
various documents will be retried.

Karl


On Tue, Oct 30, 2018 at 5:07 AM Bisonti Mario 
mailto:mario.biso...@vimar.com>> wrote:
Hallo.

I started a job that works for some minutes, and after it stucks.

In the manifoldcf.log I see:
at 
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
 [mcf-jcifs-connector.jar:?]
at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) 
[mcf-pull-agent.jar:?]
WARN 2018-10-30T09:21:31,440 (Worker thread '2') - Tika Server: Tika Server 
rejects: Tika Server rejected document with the following reason: Unprocessable 
Entity
WARN 2018-10-30T09:21:33,502 (Worker thread '14') - Tika Server: Tika Server 
rejects: Tika Server rejected document with the following reason: Unprocessable 
Entity
WARN 2018-10-30T09:21:37,725 (Worker thread '30') - Tika Server: Tika Server 
rejects: Tika Server rejected document with the following reason: Unprocessable 
Entity
WARN 2018-10-30T09:21:44,406 (Worker thread '49') - Tika Server: Tika Server 
rejects: Tika Server rejected document with the following reason: Unprocessable 
Entity
WARN 2018-10-30T09:21:47,310 (Worker thread '15') - Tika Server: Tika Server 
rejects: Tika Server rejected document with the following reason: Unprocessable 
Entity
WARN 2018-10-30T09:21:52,000 (Worker thread '27') - Tika Server: Tika Server 
rejects: Tika Server rejected document with the following reason: Unprocessable 
Entity
WARN 2018-10-30T09:21:53,526 (Worker thread '15') - Tika Server: Tika Server 
rejects: Tika Server rejected document with the following reason: Unprocessable 
Entity
WARN 2018-10-30T09:22:04,511 (Worker thread '3') - JCIFS: Possibly transient 
exception detected on attempt 1 while getting share security: All pipe 
instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569) 
~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:669) 
~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) 
~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:993) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1010) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFileOutputStream.(SmbFileOutputStream.java:142) 
~[jcifs-1.3.18.3.jar:?]
at 
jcifs.smb.TransactNamedPipeOutputStream.(TransactNamedPipeOutputStream.java:32)
 ~[jcifs-1.3.18.3.jar:?]
at 
jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) 
~[jcifs-1.3.18.3.jar:?]
at 
jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) 
~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) 
~[jcifs-1.3.18.3.jar:?]

R: Job stuck without message

[Bisonti Mario]

Thanks a lot Karl

Yes, I see many docs in the docs queue but they are inactive.

Infact i see that no more docs are indexed in Solr and I see that job is with 
the same number of docs Active (35012)

[cid:image002.jpg@01D47065.DEFF7B40]



Da: Karl Wright 
Inviato: martedì 30 ottobre 2018 13:59
A: user@manifoldcf.apache.org
Oggetto: Re: Job stuck without message

The reason the job is "stuck" is because:

' JCIFS: Possibly transient exception detected on attempt 1 while getting share 
security: All pipe instances are busy.'

This means that ManifoldCF will retry this document for a while before it gives 
up on it.  It appears to be stuck but it is not.  You can verify that by 
looking at the Document Queue report to see what is queued and what times the 
various documents will be retried.

Karl


On Tue, Oct 30, 2018 at 5:07 AM Bisonti Mario 
mailto:mario.biso...@vimar.com>> wrote:
Hallo.

I started a job that works for some minutes, and after it stucks.

In the manifoldcf.log I see:
at 
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
 [mcf-jcifs-connector.jar:?]
at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) 
[mcf-pull-agent.jar:?]
WARN 2018-10-30T09:21:31,440 (Worker thread '2') - Tika Server: Tika Server 
rejects: Tika Server rejected document with the following reason: Unprocessable 
Entity
WARN 2018-10-30T09:21:33,502 (Worker thread '14') - Tika Server: Tika Server 
rejects: Tika Server rejected document with the following reason: Unprocessable 
Entity
WARN 2018-10-30T09:21:37,725 (Worker thread '30') - Tika Server: Tika Server 
rejects: Tika Server rejected document with the following reason: Unprocessable 
Entity
WARN 2018-10-30T09:21:44,406 (Worker thread '49') - Tika Server: Tika Server 
rejects: Tika Server rejected document with the following reason: Unprocessable 
Entity
WARN 2018-10-30T09:21:47,310 (Worker thread '15') - Tika Server: Tika Server 
rejects: Tika Server rejected document with the following reason: Unprocessable 
Entity
WARN 2018-10-30T09:21:52,000 (Worker thread '27') - Tika Server: Tika Server 
rejects: Tika Server rejected document with the following reason: Unprocessable 
Entity
WARN 2018-10-30T09:21:53,526 (Worker thread '15') - Tika Server: Tika Server 
rejects: Tika Server rejected document with the following reason: Unprocessable 
Entity
WARN 2018-10-30T09:22:04,511 (Worker thread '3') - JCIFS: Possibly transient 
exception detected on attempt 1 while getting share security: All pipe 
instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569) 
~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:669) 
~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) 
~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:993) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1010) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFileOutputStream.(SmbFileOutputStream.java:142) 
~[jcifs-1.3.18.3.jar:?]
at 
jcifs.smb.TransactNamedPipeOutputStream.(TransactNamedPipeOutputStream.java:32)
 ~[jcifs-1.3.18.3.jar:?]
at 
jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) 
~[jcifs-1.3.18.3.jar:?]
at 
jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) 
~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) 
~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) 
~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) 
~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951) 
~[jcifs-1.3.18.3.jar:?]
at 
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438)
 [mcf-jcifs-connector.jar:?]
at 
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221)
 [mcf-jcifs-connector.jar:?]
at 
org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
 [mcf-jcifs-connector.jar:?]
at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) 
[mcf-pull-agent.jar:?]
WARN 2018-10-30T09:22:10,359 (Worker thread '27') - Tika Server: Tika Server 
rejects: Tika Server rejected document with the following reason: Unprocessable 
Entity
WARN 2018-10-30T09:22:13,932 (Worker thread '12') - Tika Server: Tika Server 
rejects: Tika Server rejected document with the following reason: Unprocessable 
Entity
WARN