Re: Interrupted while acquiring credits
"crashing the manifold" is probably running out of memory, and it is probably due to having too many worker threads and insufficient memory, not the error you found. If that error caused a problem, it would simply abort the job, not "crash" Manifold. Karl On Fri, May 14, 2021 at 4:10 AM ritika jain wrote: > It retries for 3 times and it usually crashes the manifoldcf. > > Similar ticket i observed > https://issues.apache.org/jira/browse/CONNECTORS-1633, does manifoldcf > itself capable of skipping the file that cause issue instead of aborting > the job or crashing manifold > > On Fri, May 14, 2021 at 1:34 PM Karl Wright wrote: > >> ' >> >> *JCIFS: Possibly transient exception detected on attempt 1 while getting >> share security'Yes, it is going to retry.* >> >> *Karl* >> >> On Fri, May 14, 2021 at 1:45 AM ritika jain >> wrote: >> >>> Hi, >>> I am using Windows shares connector in manifoldcf 2.14 and ElasticSearch >>> connector as Output connector and Tika and Metadata adjuster as >>> Transformation connector >>> >>> Trying to crawl the files from SMB server with 64 GB of server and Start >>> option file of manifoldcf is being given 32GB of memory >>> But many times got different errors while processing documents:- >>> *1) Access is denied* >>> *2) ... 23 more* >>> >>> >>> * WARN 2021-05-13T12:33:16,318 (Worker thread '6') - Service >>> interruption reported for job 1599130705168 connection 'Themas_Repo': >>> Timeout or other service interruption: Interrupted while acquiring >>> credits WARN 2021-05-13T12:33:17,315 (Worker thread '6') - JCIFS: Possibly >>> transient exception detected on attempt 1 while getting share security: >>> Interrupted while acquiring creditsjcifs.smb.SmbException: Interrupted >>> while acquiring credits* >>> at jcifs.smb.SmbTransportImpl.send(SmbTransportImpl.java:1530) >>> ~[jcifs-ng-2.1.2.jar:?] >>> at >>> jcifs.smb.SmbSessionImpl.sessionSetupSMB2(SmbSessionImpl.java:549) >>> ~[jcifs-ng-2.1.2.jar:?] >>> at >>> jcifs.smb.SmbSessionImpl.sessionSetup(SmbSessionImpl.java:483) >>> ~[jcifs-ng-2.1.2.jar:?] >>> at jcifs.smb.SmbSessionImpl.send(SmbSessionImpl.java:369) >>> ~[jcifs-ng-2.1.2.jar:?] >>> at jcifs.smb.SmbSessionImpl.send(SmbSessionImpl.java:347) >>> ~[jcifs-ng-2.1.2.jar:?] >>> at jcifs.smb.SmbTreeImpl.treeConnect(SmbTreeImpl.java:607) >>> ~[jcifs-ng-2.1.2.jar:?] >>> at >>> jcifs.smb.SmbTreeConnection.connectTree(SmbTreeConnection.java:609) >>> ~[jcifs-ng-2.1.2.jar:?] >>> at >>> jcifs.smb.SmbTreeConnection.connectHost(SmbTreeConnection.java:563) >>> ~[jcifs-ng-2.1.2.jar:?] >>> at >>> jcifs.smb.SmbTreeConnection.connectHost(SmbTreeConnection.java:484) >>> ~[jcifs-ng-2.1.2.jar:?] >>> at >>> jcifs.smb.SmbTreeConnection.connect(SmbTreeConnection.java:460) >>> ~[jcifs-ng-2.1.2.jar:?] >>> at >>> jcifs.smb.SmbTreeConnection.connectWrapException(SmbTreeConnection.java:421) >>> ~[jcifs-ng-2.1.2.jar:?] >>> at jcifs.smb.SmbFile.ensureTreeConnected(SmbFile.java:551) >>> ~[jcifs-ng-2.1.2.jar:?] >>> at >>> jcifs.smb.SmbPipeHandleImpl.ensureTreeConnected(SmbPipeHandleImpl.java:111) >>> ~[jcifs-ng-2.1.2.jar:?] >>> at >>> jcifs.smb.SmbPipeHandleImpl.ensureOpen(SmbPipeHandleImpl.java:166) >>> ~[jcifs-ng-2.1.2.jar:?] >>> at >>> jcifs.smb.SmbPipeHandleImpl.sendrecv(SmbPipeHandleImpl.java:250) >>> ~[jcifs-ng-2.1.2.jar:?] >>> at >>> jcifs.dcerpc.DcerpcPipeHandle.doSendReceiveFragment(DcerpcPipeHandle.java:113) >>> ~[jcifs-ng-2.1.2.jar:?] >>> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:243) >>> ~[jcifs-ng-2.1.2.jar:?] >>> at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:216) >>> ~[jcifs-ng-2.1.2.jar:?] >>> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:234) >>> ~[jcifs-ng-2.1.2.jar:?] >>> at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2337) >>> ~[jcifs-ng-2.1.2.jar:?] >>> at >>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2500) >>> [mcf-jcifs-connector.jar:2.14] >>> at >>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1261) >>> [mcf-jcifs-connector.jar:2.14] >>> at >>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:647) >>> [mcf-jcifs-connector.jar:2.14] >>> at >>> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) >>> [mcf-pull-agent.jar:?] >>> Caused by: java.io.InterruptedIOException: Interrupted while acquiring >>> credits >>> at >>> jcifs.smb.SmbTransportImpl.sendrecv(SmbTransportImpl.java:976) ~[?:?] >>> at jcifs.smb.SmbTransportImpl.send(SmbTransportImpl.java:1523) >>> ~[?:?] >>> ... 23 more >>> Caused by: java.lang.InterruptedException >>> at >>>
Re: Interrupted while acquiring credits
It retries for 3 times and it usually crashes the manifoldcf. Similar ticket i observed https://issues.apache.org/jira/browse/CONNECTORS-1633, does manifoldcf itself capable of skipping the file that cause issue instead of aborting the job or crashing manifold On Fri, May 14, 2021 at 1:34 PM Karl Wright wrote: > ' > > *JCIFS: Possibly transient exception detected on attempt 1 while getting > share security'Yes, it is going to retry.* > > *Karl* > > On Fri, May 14, 2021 at 1:45 AM ritika jain > wrote: > >> Hi, >> I am using Windows shares connector in manifoldcf 2.14 and ElasticSearch >> connector as Output connector and Tika and Metadata adjuster as >> Transformation connector >> >> Trying to crawl the files from SMB server with 64 GB of server and Start >> option file of manifoldcf is being given 32GB of memory >> But many times got different errors while processing documents:- >> *1) Access is denied* >> *2) ... 23 more* >> >> >> * WARN 2021-05-13T12:33:16,318 (Worker thread '6') - Service interruption >> reported for job 1599130705168 connection 'Themas_Repo': Timeout or other >> service interruption: Interrupted while acquiring credits WARN >> 2021-05-13T12:33:17,315 (Worker thread '6') - JCIFS: Possibly transient >> exception detected on attempt 1 while getting share security: Interrupted >> while acquiring creditsjcifs.smb.SmbException: Interrupted while acquiring >> credits* >> at jcifs.smb.SmbTransportImpl.send(SmbTransportImpl.java:1530) >> ~[jcifs-ng-2.1.2.jar:?] >> at >> jcifs.smb.SmbSessionImpl.sessionSetupSMB2(SmbSessionImpl.java:549) >> ~[jcifs-ng-2.1.2.jar:?] >> at jcifs.smb.SmbSessionImpl.sessionSetup(SmbSessionImpl.java:483) >> ~[jcifs-ng-2.1.2.jar:?] >> at jcifs.smb.SmbSessionImpl.send(SmbSessionImpl.java:369) >> ~[jcifs-ng-2.1.2.jar:?] >> at jcifs.smb.SmbSessionImpl.send(SmbSessionImpl.java:347) >> ~[jcifs-ng-2.1.2.jar:?] >> at jcifs.smb.SmbTreeImpl.treeConnect(SmbTreeImpl.java:607) >> ~[jcifs-ng-2.1.2.jar:?] >> at >> jcifs.smb.SmbTreeConnection.connectTree(SmbTreeConnection.java:609) >> ~[jcifs-ng-2.1.2.jar:?] >> at >> jcifs.smb.SmbTreeConnection.connectHost(SmbTreeConnection.java:563) >> ~[jcifs-ng-2.1.2.jar:?] >> at >> jcifs.smb.SmbTreeConnection.connectHost(SmbTreeConnection.java:484) >> ~[jcifs-ng-2.1.2.jar:?] >> at >> jcifs.smb.SmbTreeConnection.connect(SmbTreeConnection.java:460) >> ~[jcifs-ng-2.1.2.jar:?] >> at >> jcifs.smb.SmbTreeConnection.connectWrapException(SmbTreeConnection.java:421) >> ~[jcifs-ng-2.1.2.jar:?] >> at jcifs.smb.SmbFile.ensureTreeConnected(SmbFile.java:551) >> ~[jcifs-ng-2.1.2.jar:?] >> at >> jcifs.smb.SmbPipeHandleImpl.ensureTreeConnected(SmbPipeHandleImpl.java:111) >> ~[jcifs-ng-2.1.2.jar:?] >> at >> jcifs.smb.SmbPipeHandleImpl.ensureOpen(SmbPipeHandleImpl.java:166) >> ~[jcifs-ng-2.1.2.jar:?] >> at >> jcifs.smb.SmbPipeHandleImpl.sendrecv(SmbPipeHandleImpl.java:250) >> ~[jcifs-ng-2.1.2.jar:?] >> at >> jcifs.dcerpc.DcerpcPipeHandle.doSendReceiveFragment(DcerpcPipeHandle.java:113) >> ~[jcifs-ng-2.1.2.jar:?] >> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:243) >> ~[jcifs-ng-2.1.2.jar:?] >> at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:216) >> ~[jcifs-ng-2.1.2.jar:?] >> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:234) >> ~[jcifs-ng-2.1.2.jar:?] >> at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2337) >> ~[jcifs-ng-2.1.2.jar:?] >> at >> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2500) >> [mcf-jcifs-connector.jar:2.14] >> at >> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1261) >> [mcf-jcifs-connector.jar:2.14] >> at >> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:647) >> [mcf-jcifs-connector.jar:2.14] >> at >> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) >> [mcf-pull-agent.jar:?] >> Caused by: java.io.InterruptedIOException: Interrupted while acquiring >> credits >> at jcifs.smb.SmbTransportImpl.sendrecv(SmbTransportImpl.java:976) >> ~[?:?] >> at jcifs.smb.SmbTransportImpl.send(SmbTransportImpl.java:1523) >> ~[?:?] >> ... 23 more >> Caused by: java.lang.InterruptedException >> at >> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1326) >> ~[?:1.8.0_292] >> at java.util.concurrent.Semaphore.tryAcquire(Semaphore.java:582) >> ~[?:1.8.0_292] >> at jcifs.smb.SmbTransportImpl.sendrecv(SmbTransportImpl.java:959) >> ~[?:?] >> at jcifs.smb.SmbTransportImpl.send(SmbTransportImpl.java:1523) >> ~[?:?] >> ... 23 more >> WARN 2021-05-13T12:33:17,314 (Worker thread
Re: Interrupted while acquiring credits
' *JCIFS: Possibly transient exception detected on attempt 1 while getting share security'Yes, it is going to retry.* *Karl* On Fri, May 14, 2021 at 1:45 AM ritika jain wrote: > Hi, > I am using Windows shares connector in manifoldcf 2.14 and ElasticSearch > connector as Output connector and Tika and Metadata adjuster as > Transformation connector > > Trying to crawl the files from SMB server with 64 GB of server and Start > option file of manifoldcf is being given 32GB of memory > But many times got different errors while processing documents:- > *1) Access is denied* > *2) ... 23 more* > > > * WARN 2021-05-13T12:33:16,318 (Worker thread '6') - Service interruption > reported for job 1599130705168 connection 'Themas_Repo': Timeout or other > service interruption: Interrupted while acquiring credits WARN > 2021-05-13T12:33:17,315 (Worker thread '6') - JCIFS: Possibly transient > exception detected on attempt 1 while getting share security: Interrupted > while acquiring creditsjcifs.smb.SmbException: Interrupted while acquiring > credits* > at jcifs.smb.SmbTransportImpl.send(SmbTransportImpl.java:1530) > ~[jcifs-ng-2.1.2.jar:?] > at > jcifs.smb.SmbSessionImpl.sessionSetupSMB2(SmbSessionImpl.java:549) > ~[jcifs-ng-2.1.2.jar:?] > at jcifs.smb.SmbSessionImpl.sessionSetup(SmbSessionImpl.java:483) > ~[jcifs-ng-2.1.2.jar:?] > at jcifs.smb.SmbSessionImpl.send(SmbSessionImpl.java:369) > ~[jcifs-ng-2.1.2.jar:?] > at jcifs.smb.SmbSessionImpl.send(SmbSessionImpl.java:347) > ~[jcifs-ng-2.1.2.jar:?] > at jcifs.smb.SmbTreeImpl.treeConnect(SmbTreeImpl.java:607) > ~[jcifs-ng-2.1.2.jar:?] > at > jcifs.smb.SmbTreeConnection.connectTree(SmbTreeConnection.java:609) > ~[jcifs-ng-2.1.2.jar:?] > at > jcifs.smb.SmbTreeConnection.connectHost(SmbTreeConnection.java:563) > ~[jcifs-ng-2.1.2.jar:?] > at > jcifs.smb.SmbTreeConnection.connectHost(SmbTreeConnection.java:484) > ~[jcifs-ng-2.1.2.jar:?] > at jcifs.smb.SmbTreeConnection.connect(SmbTreeConnection.java:460) > ~[jcifs-ng-2.1.2.jar:?] > at > jcifs.smb.SmbTreeConnection.connectWrapException(SmbTreeConnection.java:421) > ~[jcifs-ng-2.1.2.jar:?] > at jcifs.smb.SmbFile.ensureTreeConnected(SmbFile.java:551) > ~[jcifs-ng-2.1.2.jar:?] > at > jcifs.smb.SmbPipeHandleImpl.ensureTreeConnected(SmbPipeHandleImpl.java:111) > ~[jcifs-ng-2.1.2.jar:?] > at > jcifs.smb.SmbPipeHandleImpl.ensureOpen(SmbPipeHandleImpl.java:166) > ~[jcifs-ng-2.1.2.jar:?] > at > jcifs.smb.SmbPipeHandleImpl.sendrecv(SmbPipeHandleImpl.java:250) > ~[jcifs-ng-2.1.2.jar:?] > at > jcifs.dcerpc.DcerpcPipeHandle.doSendReceiveFragment(DcerpcPipeHandle.java:113) > ~[jcifs-ng-2.1.2.jar:?] > at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:243) > ~[jcifs-ng-2.1.2.jar:?] > at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:216) > ~[jcifs-ng-2.1.2.jar:?] > at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:234) > ~[jcifs-ng-2.1.2.jar:?] > at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2337) > ~[jcifs-ng-2.1.2.jar:?] > at > org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2500) > [mcf-jcifs-connector.jar:2.14] > at > org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1261) > [mcf-jcifs-connector.jar:2.14] > at > org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:647) > [mcf-jcifs-connector.jar:2.14] > at > org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) > [mcf-pull-agent.jar:?] > Caused by: java.io.InterruptedIOException: Interrupted while acquiring > credits > at jcifs.smb.SmbTransportImpl.sendrecv(SmbTransportImpl.java:976) > ~[?:?] > at jcifs.smb.SmbTransportImpl.send(SmbTransportImpl.java:1523) > ~[?:?] > ... 23 more > Caused by: java.lang.InterruptedException > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1326) > ~[?:1.8.0_292] > at java.util.concurrent.Semaphore.tryAcquire(Semaphore.java:582) > ~[?:1.8.0_292] > at jcifs.smb.SmbTransportImpl.sendrecv(SmbTransportImpl.java:959) > ~[?:?] > at jcifs.smb.SmbTransportImpl.send(SmbTransportImpl.java:1523) > ~[?:?] > ... 23 more > WARN 2021-05-13T12:33:17,314 (Worker thread '2') - JCIFS: Possibly > transient exception detected on attempt 2 while getting share security: > Interrupted while acquiring credits > jcifs.smb.SmbException: Interrupted while acquiring credits > at jcifs.smb.SmbTransportImpl.send(SmbTransportImpl. > > Do we have such functionality that in case of any error occurs like this. > That it should skip the particular record and then continue to process > further instead of