[jira] [Commented] (SOLR-10962) replicationHandler's reserveCommitDuration configurable in SolrCloud mode
[ https://issues.apache.org/jira/browse/SOLR-10962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16179301#comment-16179301 ] ASF subversion and git services commented on SOLR-10962: Commit 20f1e633eff373d04aad65e8d7f13fa37194b32a in lucene-solr's branch refs/heads/branch_7x from [~cpoerschke] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=20f1e63 ] SOLR-10962: Make ReplicationHandler's commitReserveDuration configurable in SolrCloud mode. (Ramsey Haddad, Christine Poerschke, hossman) > replicationHandler's reserveCommitDuration configurable in SolrCloud mode > - > > Key: SOLR-10962 > URL: https://issues.apache.org/jira/browse/SOLR-10962 > Project: Solr > Issue Type: New Feature > Components: replication (java) >Reporter: Ramsey Haddad >Assignee: Christine Poerschke >Priority: Minor > Attachments: SOLR-10962.patch, SOLR-10962.patch, SOLR-10962.patch, > SOLR-10962.patch, SOLR-10962.patch, SOLR-10962.patch, SOLR-10962.patch > > > With SolrCloud mode, when doing replication via IndexFetcher, we occasionally > see the Fetch fail and then get restarted from scratch in cases where an > Index file is deleted after fetch manifest is computed and before the fetch > actually transfers the file. The risk of this happening can be reduced with a > higher value of reserveCommitDuration. However, the current configuration > only allows this value to be adjusted for "master" mode. This change allows > the value to also be changed when using "SolrCloud" mode. > https://lucene.apache.org/solr/guide/6_6/index-replication.html -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10962) replicationHandler's reserveCommitDuration configurable in SolrCloud mode
[ https://issues.apache.org/jira/browse/SOLR-10962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175119#comment-16175119 ] Christine Poerschke commented on SOLR-10962: Ran precommit and solr/core tests. Committed to master branch today and planning to cherry-pick to branch_7x in a couple of days. _(Any further post-commit comments and input welcome as usual. Thank you.)_ > replicationHandler's reserveCommitDuration configurable in SolrCloud mode > - > > Key: SOLR-10962 > URL: https://issues.apache.org/jira/browse/SOLR-10962 > Project: Solr > Issue Type: New Feature > Components: replication (java) >Reporter: Ramsey Haddad >Assignee: Christine Poerschke >Priority: Minor > Attachments: SOLR-10962.patch, SOLR-10962.patch, SOLR-10962.patch, > SOLR-10962.patch, SOLR-10962.patch, SOLR-10962.patch, SOLR-10962.patch > > > With SolrCloud mode, when doing replication via IndexFetcher, we occasionally > see the Fetch fail and then get restarted from scratch in cases where an > Index file is deleted after fetch manifest is computed and before the fetch > actually transfers the file. The risk of this happening can be reduced with a > higher value of reserveCommitDuration. However, the current configuration > only allows this value to be adjusted for "master" mode. This change allows > the value to also be changed when using "SolrCloud" mode. > https://lucene.apache.org/solr/guide/6_6/index-replication.html -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10962) replicationHandler's reserveCommitDuration configurable in SolrCloud mode
[ https://issues.apache.org/jira/browse/SOLR-10962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175110#comment-16175110 ] ASF subversion and git services commented on SOLR-10962: Commit 56b8ad2f081e1e7d51e7c97fcbd1f0fbfdf07c98 in lucene-solr's branch refs/heads/master from [~cpoerschke] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=56b8ad2 ] SOLR-10962: Make ReplicationHandler's commitReserveDuration configurable in SolrCloud mode. (Ramsey Haddad, Christine Poerschke, hossman) > replicationHandler's reserveCommitDuration configurable in SolrCloud mode > - > > Key: SOLR-10962 > URL: https://issues.apache.org/jira/browse/SOLR-10962 > Project: Solr > Issue Type: New Feature > Components: replication (java) >Reporter: Ramsey Haddad >Assignee: Christine Poerschke >Priority: Minor > Attachments: SOLR-10962.patch, SOLR-10962.patch, SOLR-10962.patch, > SOLR-10962.patch, SOLR-10962.patch, SOLR-10962.patch, SOLR-10962.patch > > > With SolrCloud mode, when doing replication via IndexFetcher, we occasionally > see the Fetch fail and then get restarted from scratch in cases where an > Index file is deleted after fetch manifest is computed and before the fetch > actually transfers the file. The risk of this happening can be reduced with a > higher value of reserveCommitDuration. However, the current configuration > only allows this value to be adjusted for "master" mode. This change allows > the value to also be changed when using "SolrCloud" mode. > https://lucene.apache.org/solr/guide/6_6/index-replication.html -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10962) replicationHandler's reserveCommitDuration configurable in SolrCloud mode
[
https://issues.apache.org/jira/browse/SOLR-10962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16070214#comment-16070214
]
Christine Poerschke commented on SOLR-10962:
Thanks for suggesting the
{{core.solrConfig.luceneMatchVersion.onOrAfter(Version.LUCENE_7_0_0)}} clause,
that totally makes sense.
bq. ... a well-known name in Config API to update these settings in the same
way that we can update autoCommit.maxTime etc. ...
Good idea, let's explore that.
The replication handler already has an existing top-level attribute
_maxNumberOfBackups_ - presumably that would transition over to Config API at
the same time as the (currently not top-level) _reserveCommitDuration_
attribute.
> replicationHandler's reserveCommitDuration configurable in SolrCloud mode
> -
>
> Key: SOLR-10962
> URL: https://issues.apache.org/jira/browse/SOLR-10962
> Project: Solr
> Issue Type: New Feature
> Components: replication (java)
>Reporter: Ramsey Haddad
>Priority: Minor
> Attachments: SOLR-10962.patch, SOLR-10962.patch, SOLR-10962.patch
>
>
> With SolrCloud mode, when doing replication via IndexFetcher, we occasionally
> see the Fetch fail and then get restarted from scratch in cases where an
> Index file is deleted after fetch manifest is computed and before the fetch
> actually transfers the file. The risk of this happening can be reduced with a
> higher value of reserveCommitDuration. However, the current configuration
> only allows this value to be adjusted for "master" mode. This change allows
> the value to also be changed when using "SolrCloud" mode.
> https://lucene.apache.org/solr/guide/6_6/index-replication.html
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10962) replicationHandler's reserveCommitDuration configurable in SolrCloud mode
[ https://issues.apache.org/jira/browse/SOLR-10962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16068808#comment-16068808 ] Shalin Shekhar Mangar commented on SOLR-10962: -- Although from a solrconfig.xml perspective, moving commitReserveDuration to a top-level attribute for ReplicationHandler is fine, I prefer that we do not force users to add these configurations in solrconfig.xml and instead use a well-known name in Config API to update these settings in the same way that we can update autoCommit.maxTime etc. Editing solrconfig.xml by hand should never be the answer to solve configurability in SolrCloud going forward. > replicationHandler's reserveCommitDuration configurable in SolrCloud mode > - > > Key: SOLR-10962 > URL: https://issues.apache.org/jira/browse/SOLR-10962 > Project: Solr > Issue Type: New Feature > Components: replication (java) >Reporter: Ramsey Haddad >Priority: Minor > Attachments: SOLR-10962.patch, SOLR-10962.patch, SOLR-10962.patch > > > With SolrCloud mode, when doing replication via IndexFetcher, we occasionally > see the Fetch fail and then get restarted from scratch in cases where an > Index file is deleted after fetch manifest is computed and before the fetch > actually transfers the file. The risk of this happening can be reduced with a > higher value of reserveCommitDuration. However, the current configuration > only allows this value to be adjusted for "master" mode. This change allows > the value to also be changed when using "SolrCloud" mode. > https://lucene.apache.org/solr/guide/6_6/index-replication.html -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10962) replicationHandler's reserveCommitDuration configurable in SolrCloud mode
[
https://issues.apache.org/jira/browse/SOLR-10962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16068795#comment-16068795
]
Hoss Man commented on SOLR-10962:
-
i'm not really familiar with this code -- so i'm making an assumption based on
trust in christine that the entire premise of this issue makes sense. Based on
that assumption and a quick skim of the patch, i think the overall approach is
sound, but frankly I think the {{LOG.warn("Beginning with Solr 7.0...}} line is
too weak, and should be something like...
{code}
// remove this error check & backcompat logic when Version.LUCENE_7_0_0 is
removed
Config.assertWarnOrFail(
"Begining with Solr 7.0, master."+RESERVE + " is deprecated and should now be
configured directly on the ReplicationHandler",
(null == reserve),
core.solrConfig.luceneMatchVersion.onOrAfter(Version.LUCENE_7_0_0));
{code}
that way:
* anyone starting with a clean (example) config will get an error if they try
to use the old sytnax
* anyone upgrading with an old config will just get a warning - until/unless
they change the {{}} in their solrconfig.xml t which point
they must also change this
* once LUCENE_7_0_0 is removed from the code base, this error handling will
stop compiling and we'll get a built in reminder that this special error
checking (and the back compat code wrapped around it) can be removed.
(this pattern is is entire point of assertWarnOrFail)
> replicationHandler's reserveCommitDuration configurable in SolrCloud mode
> -
>
> Key: SOLR-10962
> URL: https://issues.apache.org/jira/browse/SOLR-10962
> Project: Solr
> Issue Type: New Feature
> Components: replication (java)
>Reporter: Ramsey Haddad
>Priority: Minor
> Attachments: SOLR-10962.patch, SOLR-10962.patch, SOLR-10962.patch
>
>
> With SolrCloud mode, when doing replication via IndexFetcher, we occasionally
> see the Fetch fail and then get restarted from scratch in cases where an
> Index file is deleted after fetch manifest is computed and before the fetch
> actually transfers the file. The risk of this happening can be reduced with a
> higher value of reserveCommitDuration. However, the current configuration
> only allows this value to be adjusted for "master" mode. This change allows
> the value to also be changed when using "SolrCloud" mode.
> https://lucene.apache.org/solr/guide/6_6/index-replication.html
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10962) replicationHandler's reserveCommitDuration configurable in SolrCloud mode
[ https://issues.apache.org/jira/browse/SOLR-10962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16068662#comment-16068662 ] Christine Poerschke commented on SOLR-10962: precommit and solr core tests pass. I think this is good to go but would appreciate further pairs of eyes on this since the change concerns a long established config element. Replication was added by SOLR-561 in 2008, [~shalinmangar] - any thoughts? [~hossman] - you're always good and watchful w.r.t. config deprecation issues - do you think the proposed route to add a top-level _commitReserveDuration_ element and to deprecate the _master.commitReserveDuration_ sub-element makes sense? > replicationHandler's reserveCommitDuration configurable in SolrCloud mode > - > > Key: SOLR-10962 > URL: https://issues.apache.org/jira/browse/SOLR-10962 > Project: Solr > Issue Type: New Feature > Components: replication (java) >Reporter: Ramsey Haddad >Priority: Minor > Attachments: SOLR-10962.patch, SOLR-10962.patch, SOLR-10962.patch > > > With SolrCloud mode, when doing replication via IndexFetcher, we occasionally > see the Fetch fail and then get restarted from scratch in cases where an > Index file is deleted after fetch manifest is computed and before the fetch > actually transfers the file. The risk of this happening can be reduced with a > higher value of reserveCommitDuration. However, the current configuration > only allows this value to be adjusted for "master" mode. This change allows > the value to also be changed when using "SolrCloud" mode. > https://lucene.apache.org/solr/guide/6_6/index-replication.html -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10962) replicationHandler's reserveCommitDuration configurable in SolrCloud mode
[
https://issues.apache.org/jira/browse/SOLR-10962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16064688#comment-16064688
]
Christine Poerschke commented on SOLR-10962:
At present we can have a _master_ or a _slave_ or _master and slave_ sections
in the request handler configuration i.e.
{code}
...
...
{code}
or
{code}
...
{code}
or
{code}
...
...
...
{code}
and with the addition of a _cloud_ section the number of (valid and invalid)
combinations of sections would increase.
Instead of a new _cloud_ section
{code}
...
{code}
with (for now) just the one _commitReserveDuration_ element, how about making
_commitReserveDuration_ a top-level element i.e. in cloud mode
{code}
...
{code}
and
{code}
...
...
...
{code}
in other modes?
_master.commitReserveDuration_ would be supported but deprecated for a
transition period and eventually it would go away.
> replicationHandler's reserveCommitDuration configurable in SolrCloud mode
> -
>
> Key: SOLR-10962
> URL: https://issues.apache.org/jira/browse/SOLR-10962
> Project: Solr
> Issue Type: New Feature
> Components: replication (java)
>Reporter: Ramsey Haddad
>Priority: Minor
> Attachments: SOLR-10962.patch
>
>
> With SolrCloud mode, when doing replication via IndexFetcher, we occasionally
> see the Fetch fail and then get restarted from scratch in cases where an
> Index file is deleted after fetch manifest is computed and before the fetch
> actually transfers the file. The risk of this happening can be reduced with a
> higher value of reserveCommitDuration. However, the current configuration
> only allows this value to be adjusted for "master" mode. This change allows
> the value to also be changed when using "SolrCloud" mode.
> https://lucene.apache.org/solr/guide/6_6/index-replication.html
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
