[jira] [Commented] (CONNECTORS-1600) Add support for configuring JCIFS connector's resilience to SMB exceptions before throwing a ServiceInterruption
[ https://issues.apache.org/jira/browse/CONNECTORS-1600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824738#comment-16824738 ] Tang Huan Song commented on CONNECTORS-1600: Setting retriesRemaining=5 and totalTries=8 enabled the job to complete in a reliable fashion. It is likely that the heavy traffic subsided in the additional retries, allowing for the requests to succeed. However, I acknowledge that luck played a part in the job's successes, and I am of the opinion that changing the value doesn't alleviate the high-traffic issue, but merely works around it. Though it would be beneficial to us were the retry counts made configurable, I have no objections if you choose not to implement this. Thanks for your thoughts! > Add support for configuring JCIFS connector's resilience to SMB exceptions > before throwing a ServiceInterruption > > > Key: CONNECTORS-1600 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1600 > Project: ManifoldCF > Issue Type: Improvement > Components: JCIFS connector >Affects Versions: ManifoldCF 2.10, ManifoldCF 2.11, ManifoldCF 2.12 >Reporter: Tang Huan Song >Priority: Major > > This is a improvement request regarding the JCIFS(-ng) connector's exception > handling behavior. > After examining the JCIFS connector code, I've found that the number of > retries given consecutive identical SMB exceptions and the total number of > retries per file/request is hardcoded within the connector at > retriesRemaining=3 and totalTries=5 respectively. > Depending on the amount of traffic a file server regularly handles, the > probability of any given SMB request failing, and correspondingly the total > number of SMB request failures for a given file request will vary. As a > result, the current hardcoded values may cause ManifoldCF to abandon the job > in the event of high traffic. > I would like to suggest making these values configurable, as a connector-wide > setting modified via ManifoldCF's properties.xml or a per-connection setting > modified via the corresponding repository connection's page. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1600) Add support for configuring JCIFS connector's resilience to SMB exceptions before throwing a ServiceInterruption
[ https://issues.apache.org/jira/browse/CONNECTORS-1600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823915#comment-16823915 ] Karl Wright commented on CONNECTORS-1600: - How many times did you have it retry? > Add support for configuring JCIFS connector's resilience to SMB exceptions > before throwing a ServiceInterruption > > > Key: CONNECTORS-1600 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1600 > Project: ManifoldCF > Issue Type: Improvement > Components: JCIFS connector >Affects Versions: ManifoldCF 2.10, ManifoldCF 2.11, ManifoldCF 2.12 >Reporter: Tang Huan Song >Priority: Major > > This is a improvement request regarding the JCIFS(-ng) connector's exception > handling behavior. > After examining the JCIFS connector code, I've found that the number of > retries given consecutive identical SMB exceptions and the total number of > retries per file/request is hardcoded within the connector at > retriesRemaining=3 and totalTries=5 respectively. > Depending on the amount of traffic a file server regularly handles, the > probability of any given SMB request failing, and correspondingly the total > number of SMB request failures for a given file request will vary. As a > result, the current hardcoded values may cause ManifoldCF to abandon the job > in the event of high traffic. > I would like to suggest making these values configurable, as a connector-wide > setting modified via ManifoldCF's properties.xml or a per-connection setting > modified via the corresponding repository connection's page. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1600) Add support for configuring JCIFS connector's resilience to SMB exceptions before throwing a ServiceInterruption
[ https://issues.apache.org/jira/browse/CONNECTORS-1600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823865#comment-16823865 ] Tang Huan Song commented on CONNECTORS-1600: Thanks for your comments! Allow me to share a little about our circumstances. We were actually crawling a high-traffic file server, and according to the logs, the maximum retry threshold being reached was what caused the job to fail consistently. After raising the retry counts via recompiling, the job was able to reliably complete. This was the impetus for my suggestion. > Add support for configuring JCIFS connector's resilience to SMB exceptions > before throwing a ServiceInterruption > > > Key: CONNECTORS-1600 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1600 > Project: ManifoldCF > Issue Type: Improvement > Components: JCIFS connector >Affects Versions: ManifoldCF 2.10, ManifoldCF 2.11, ManifoldCF 2.12 >Reporter: Tang Huan Song >Priority: Major > > This is a improvement request regarding the JCIFS(-ng) connector's exception > handling behavior. > After examining the JCIFS connector code, I've found that the number of > retries given consecutive identical SMB exceptions and the total number of > retries per file/request is hardcoded within the connector at > retriesRemaining=3 and totalTries=5 respectively. > Depending on the amount of traffic a file server regularly handles, the > probability of any given SMB request failing, and correspondingly the total > number of SMB request failures for a given file request will vary. As a > result, the current hardcoded values may cause ManifoldCF to abandon the job > in the event of high traffic. > I would like to suggest making these values configurable, as a connector-wide > setting modified via ManifoldCF's properties.xml or a per-connection setting > modified via the corresponding repository connection's page. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1600) Add support for configuring JCIFS connector's resilience to SMB exceptions before throwing a ServiceInterruption
[ https://issues.apache.org/jira/browse/CONNECTORS-1600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823794#comment-16823794 ] Karl Wright commented on CONNECTORS-1600: - This is, I would suggest, overkill. It is not only a bad idea to make everything possible be configurable in the UI, but the benefit of doing so is small to non-existent. In 10 years of supporting this connector, what I've learned is that whenever people say that they just need more retries is that something else is very wrong, and the added retries do not help. If you ever run into a situation where 4 retries succeeds and 3 doesn't, please let me know. > Add support for configuring JCIFS connector's resilience to SMB exceptions > before throwing a ServiceInterruption > > > Key: CONNECTORS-1600 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1600 > Project: ManifoldCF > Issue Type: Improvement > Components: JCIFS connector >Affects Versions: ManifoldCF 2.10, ManifoldCF 2.11, ManifoldCF 2.12 >Reporter: Tang Huan Song >Priority: Major > > This is a improvement request regarding the JCIFS(-ng) connector's exception > handling behavior. > After examining the JCIFS connector code, I've found that the number of > retries given consecutive identical SMB exceptions and the total number of > retries per file/request is hardcoded within the connector at > retriesRemaining=3 and totalTries=5 respectively. > Depending on the amount of traffic a file server regularly handles, the > probability of any given SMB request failing, and correspondingly the total > number of SMB request failures for a given file request will vary. As a > result, the current hardcoded values may cause ManifoldCF to abandon the job > in the event of high traffic. > I would like to suggest making these values configurable, as a connector-wide > setting modified via ManifoldCF's properties.xml or a per-connection setting > modified via the corresponding repository connection's page. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
