date:20180502

[JENKINS] Lucene-Solr-7.x-MacOSX (64bit/jdk-9) - Build # 612 - Unstable!

2018-05-02 Thread Policeman Jenkins Server

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-MacOSX/612/
Java: 64bit/jdk-9 -XX:+UseCompressedOops -XX:+UseSerialGC

2 tests failed.
FAILED:  
org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration

Error Message:
last state: 
DocCollection(testSplitIntegration_collection//clusterstate.json/34)={   
"replicationFactor":"2",   "pullReplicas":"0",   
"router":{"name":"compositeId"},   "maxShardsPerNode":"2",   
"autoAddReplicas":"false",   "nrtReplicas":"2",   "tlogReplicas":"0",   
"autoCreated":"true",   "shards":{ "shard2":{   "replicas":{ 
"core_node3":{   
"core":"testSplitIntegration_collection_shard2_replica_n3",   
"leader":"true",   "SEARCHER.searcher.maxDoc":11,   
"SEARCHER.searcher.deletedDocs":0,   "INDEX.sizeInBytes":1,   
"node_name":"127.0.0.1:10012_solr",   "state":"active",   
"type":"NRT",   "SEARCHER.searcher.numDocs":11}, "core_node4":{ 
  "core":"testSplitIntegration_collection_shard2_replica_n4",   
"SEARCHER.searcher.maxDoc":11,   "SEARCHER.searcher.deletedDocs":0, 
  "INDEX.sizeInBytes":1,   "node_name":"127.0.0.1:10011_solr",  
 "state":"active",   "type":"NRT",   
"SEARCHER.searcher.numDocs":11}},   "range":"0-7fff",   
"state":"active"}, "shard1":{   "stateTimestamp":"1525411885996347050", 
  "replicas":{ "core_node1":{   
"core":"testSplitIntegration_collection_shard1_replica_n1",   
"leader":"true",   "SEARCHER.searcher.maxDoc":14,   
"SEARCHER.searcher.deletedDocs":0,   "INDEX.sizeInBytes":1,   
"node_name":"127.0.0.1:10012_solr",   "state":"active",   
"type":"NRT",   "SEARCHER.searcher.numDocs":14}, "core_node2":{ 
  "core":"testSplitIntegration_collection_shard1_replica_n2",   
"SEARCHER.searcher.maxDoc":14,   "SEARCHER.searcher.deletedDocs":0, 
  "INDEX.sizeInBytes":1,   "node_name":"127.0.0.1:10011_solr",  
 "state":"active",   "type":"NRT",   
"SEARCHER.searcher.numDocs":14}},   "range":"8000-",   
"state":"inactive"}, "shard1_1":{   "parent":"shard1",   
"stateTimestamp":"1525411885997280200",   "range":"c000-",  
 "state":"active",   "replicas":{ "core_node10":{   
"leader":"true",   
"core":"testSplitIntegration_collection_shard1_1_replica1",   
"SEARCHER.searcher.maxDoc":7,   "SEARCHER.searcher.deletedDocs":0,  
 "INDEX.sizeInBytes":1,   "node_name":"127.0.0.1:10011_solr",   
"base_url":"http://127.0.0.1:10011/solr;,   "state":"active",   
"type":"NRT",   "SEARCHER.searcher.numDocs":7}, 
"core_node9":{   
"core":"testSplitIntegration_collection_shard1_1_replica0",   
"SEARCHER.searcher.maxDoc":7,   "SEARCHER.searcher.deletedDocs":0,  
 "INDEX.sizeInBytes":1,   "node_name":"127.0.0.1:10012_solr",   
"base_url":"http://127.0.0.1:10012/solr;,   "state":"active",   
"type":"NRT",   "SEARCHER.searcher.numDocs":7}}}, "shard1_0":{  
 "parent":"shard1",   "stateTimestamp":"1525411885997146600",   
"range":"8000-bfff",   "state":"active",   "replicas":{ 
"core_node7":{   "leader":"true",   
"core":"testSplitIntegration_collection_shard1_0_replica0",   
"SEARCHER.searcher.maxDoc":7,   "SEARCHER.searcher.deletedDocs":0,  
 "INDEX.sizeInBytes":1,   "node_name":"127.0.0.1:10012_solr",   
"base_url":"http://127.0.0.1:10012/solr;,   "state":"active",   
"type":"NRT",   "SEARCHER.searcher.numDocs":7}, 
"core_node8":{   
"core":"testSplitIntegration_collection_shard1_0_replica1",   
"SEARCHER.searcher.maxDoc":7,   "SEARCHER.searcher.deletedDocs":0,  
 "INDEX.sizeInBytes":1,   "node_name":"127.0.0.1:10011_solr",   
"base_url":"http://127.0.0.1:10011/solr;,   "state":"active",   
"type":"NRT",   "SEARCHER.searcher.numDocs":7}

Stack Trace:
java.util.concurrent.TimeoutException: last state: 
DocCollection(testSplitIntegration_collection//clusterstate.json/34)={
  "replicationFactor":"2",
  "pullReplicas":"0",
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"false",
  "nrtReplicas":"2",
  "tlogReplicas":"0",
  "autoCreated":"true",
  "shards":{
"shard2":{
  "replicas":{
"core_node3":{
  "core":"testSplitIntegration_collection_shard2_replica_n3",
  "leader":"true",
  "SEARCHER.searcher.maxDoc":11,
  "SEARCHER.searcher.deletedDocs":0,
  "INDEX.sizeInBytes":1,
  "node_name":"127.0.0.1:10012_solr",

[jira] [Commented] (LUCENE-8270) Remove MatchesIterator.term()

2018-05-02 Thread David Smiley (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-8270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461915#comment-16461915
 ] 

David Smiley commented on LUCENE-8270:
--

I believe the semantics of term() should simply be the term at the particular 
match position.  That ought to be one term unless the indexed data had more 
than one term at this position and furthermore the query matched more than one 
of the terms at this position.  In this very edge case, I don't care much which 
term is returned :) 

I think the confusion here is what the underlying semantics are for a 
MatchesIterator when we have something like a phrase.  If we think of 
MatchesIterator as very similar to OffsetsEnum in the UnifiedHighlighter, then 
we iterate to each word/term.  Then term() is straight-forward, and so is a 
hypothetical getPayload().  Lets just do this?  Adding a freq() too would make 
my day :)  In this scenario there is no notion of a position span; it's a 
position-by-position view.  We could add a getPosition but wouldn't need a 
start/end.

If the matches() caller wanted to detect the "span" of say a phrase, (maybe to 
highlight it nicer as one markup tag enclosing pair?) then maybe handle that 
with some new method like spanOccurrence():int which could return which 
span/occurrence the MatchesIterator is currently at?

 

> Remove MatchesIterator.term()
> -
>
> Key: LUCENE-8270
> URL: https://issues.apache.org/jira/browse/LUCENE-8270
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Attachments: LUCENE-8270.patch
>
>
> As discussed on LUCENE-8268, we don't have a clear use-case for this yet, and 
> it's complicating adding Matches to phrase queries, so let's just remove it 
> for now.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12278) Add IgnoreLargeDocumentProcessFactory

2018-05-02 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-12278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461895#comment-16461895
 ] 

ASF subversion and git services commented on SOLR-12278:


Commit b489020da50184e9815931148ac3b2ebf138ad87 in lucene-solr's branch 
refs/heads/branch_7x from [~caomanhdat]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b489020 ]

SOLR-12278: Adding ref-guide doc for IgnoreLargeDocumentProcessorFactory


> Add IgnoreLargeDocumentProcessFactory 
> --
>
> Key: SOLR-12278
> URL: https://issues.apache.org/jira/browse/SOLR-12278
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>Assignee: Cao Manh Dat
>Priority: Major
> Fix For: 7.4
>
> Attachments: SOLR-12278.patch, SOLR-12278.patch, SOLR-12278.patch
>
>
> Solr should be able to ignore very large document, so it won't affect the 
> index as well as the tlog. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12278) Add IgnoreLargeDocumentProcessFactory

2018-05-02 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-12278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461896#comment-16461896
 ] 

ASF subversion and git services commented on SOLR-12278:


Commit 83c6c70179465017e1fbd3f99debb907c6eb1e28 in lucene-solr's branch 
refs/heads/branch_7x from [~caomanhdat]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=83c6c70 ]

SOLR-12278: Fix typo errors in ref-guide


> Add IgnoreLargeDocumentProcessFactory 
> --
>
> Key: SOLR-12278
> URL: https://issues.apache.org/jira/browse/SOLR-12278
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>Assignee: Cao Manh Dat
>Priority: Major
> Fix For: 7.4
>
> Attachments: SOLR-12278.patch, SOLR-12278.patch, SOLR-12278.patch
>
>
> Solr should be able to ignore very large document, so it won't affect the 
> index as well as the tlog. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12278) Add IgnoreLargeDocumentProcessFactory

2018-05-02 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-12278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461894#comment-16461894
 ] 

ASF subversion and git services commented on SOLR-12278:


Commit ed948efabfe54a703587fc01caeed94ce2401946 in lucene-solr's branch 
refs/heads/master from [~caomanhdat]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ed948ef ]

SOLR-12278: Fix typo errors in ref-guide


> Add IgnoreLargeDocumentProcessFactory 
> --
>
> Key: SOLR-12278
> URL: https://issues.apache.org/jira/browse/SOLR-12278
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>Assignee: Cao Manh Dat
>Priority: Major
> Fix For: 7.4
>
> Attachments: SOLR-12278.patch, SOLR-12278.patch, SOLR-12278.patch
>
>
> Solr should be able to ignore very large document, so it won't affect the 
> index as well as the tlog. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12278) Add IgnoreLargeDocumentProcessFactory

2018-05-02 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-12278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461889#comment-16461889
 ] 

ASF subversion and git services commented on SOLR-12278:


Commit 596d0dc9a6ef8633a078bf74ea377d727de4183e in lucene-solr's branch 
refs/heads/master from [~caomanhdat]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=596d0dc ]

SOLR-12278: Adding ref-guide doc for IgnoreLargeDocumentProcessorFactory


> Add IgnoreLargeDocumentProcessFactory 
> --
>
> Key: SOLR-12278
> URL: https://issues.apache.org/jira/browse/SOLR-12278
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>Assignee: Cao Manh Dat
>Priority: Major
> Fix For: 7.4
>
> Attachments: SOLR-12278.patch, SOLR-12278.patch, SOLR-12278.patch
>
>
> Solr should be able to ignore very large document, so it won't affect the 
> index as well as the tlog. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-12289) Add MDC information to collection admin requests

2018-05-02 Thread Varun Thacker (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-12289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Varun Thacker resolved SOLR-12289.
--
   Resolution: Fixed
Fix Version/s: master (8.0)
   7.4

> Add MDC information to collection admin requests
> 
>
> Key: SOLR-12289
> URL: https://issues.apache.org/jira/browse/SOLR-12289
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: SOLR-12289.patch, SOLR-12289.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12289) Add MDC information to collection admin requests

2018-05-02 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-12289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461861#comment-16461861
 ] 

ASF subversion and git services commented on SOLR-12289:


Commit e56520f057ab593a012678088cbdeb459f204ee5 in lucene-solr's branch 
refs/heads/branch_7x from [~varun_saxena]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e56520f ]

SOLR-12289: Add more MDC logging information to collection admin requests

(cherry picked from commit 84925ba)


> Add MDC information to collection admin requests
> 
>
> Key: SOLR-12289
> URL: https://issues.apache.org/jira/browse/SOLR-12289
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: SOLR-12289.patch, SOLR-12289.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12289) Add MDC information to collection admin requests

2018-05-02 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-12289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461859#comment-16461859
 ] 

ASF subversion and git services commented on SOLR-12289:


Commit 84925ba9abc7da8485f27fd52d0f80b14caad98f in lucene-solr's branch 
refs/heads/master from [~varun_saxena]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=84925ba ]

SOLR-12289: Add more MDC logging information to collection admin requests


> Add MDC information to collection admin requests
> 
>
> Key: SOLR-12289
> URL: https://issues.apache.org/jira/browse/SOLR-12289
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
>Priority: Major
> Attachments: SOLR-12289.patch, SOLR-12289.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11660) Issue while update index in collection after collection restore on Solr Cloud

2018-05-02 Thread Varun Thacker (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-11660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Varun Thacker updated SOLR-11660:
-
Fix Version/s: master (8.0)
   7.4

> Issue while update index in collection after collection restore on Solr Cloud
> -
>
> Key: SOLR-11660
> URL: https://issues.apache.org/jira/browse/SOLR-11660
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Backup/Restore, SolrCloud
>Affects Versions: 6.4.1
> Environment: CentOs7.3 Solr 6.4.1 Zookeeper 3.4.6
>Reporter: Viachaslau Kabak
>Assignee: Varun Thacker
>Priority: Critical
> Fix For: 7.4, master (8.0)
>
>
> We use to backup and restore one of our collections in solrCloud.
> if not to restore collection but to create it and fill from application - the 
> second attempt to renew application index is successful both on leader and 
> followers. But is to restore collection and then to renew index from 
> application - we have new index on follower and old index on leader.
> The Reload on collection helps to make index up-to-date on leader.
> In logs we have {{o.a.s.u.p.DistributedUpdateProcessor Ignoring commit while 
> not ACTIVE - state: BUFFERING replay: false}}
> If to restart solr service before collection update from application - index 
> on all solr servers is updated.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-11660) Issue while update index in collection after collection restore on Solr Cloud

2018-05-02 Thread Varun Thacker (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-11660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Varun Thacker resolved SOLR-11660.
--
Resolution: Duplicate

> Issue while update index in collection after collection restore on Solr Cloud
> -
>
> Key: SOLR-11660
> URL: https://issues.apache.org/jira/browse/SOLR-11660
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Backup/Restore, SolrCloud
>Affects Versions: 6.4.1
> Environment: CentOs7.3 Solr 6.4.1 Zookeeper 3.4.6
>Reporter: Viachaslau Kabak
>Assignee: Varun Thacker
>Priority: Critical
> Fix For: 7.4, master (8.0)
>
>
> We use to backup and restore one of our collections in solrCloud.
> if not to restore collection but to create it and fill from application - the 
> second attempt to renew application index is successful both on leader and 
> followers. But is to restore collection and then to renew index from 
> application - we have new index on follower and old index on leader.
> The Reload on collection helps to make index up-to-date on leader.
> In logs we have {{o.a.s.u.p.DistributedUpdateProcessor Ignoring commit while 
> not ACTIVE - state: BUFFERING replay: false}}
> If to restart solr service before collection update from application - index 
> on all solr servers is updated.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11660) Issue while update index in collection after collection restore on Solr Cloud

2018-05-02 Thread Varun Thacker (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-11660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461848#comment-16461848
 ] 

Varun Thacker commented on SOLR-11660:
--

{quote}"org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
 Solr cloud with available number of nodes:2 is insufficient for restoring a 
collection with 4 shards, total replicas per shard 1 and maxShardsPerNode -1. 
Consider increasing maxShardsPerNode value OR number of available nodes."
{quote}
Yeah that's broken. it's been on my radar to fix it.  SOLR-11807 

 

This issue is resolved by SOLR-12065 . I'll close this out. Sorry I didn't 
realize there was already an Jira for this

> Issue while update index in collection after collection restore on Solr Cloud
> -
>
> Key: SOLR-11660
> URL: https://issues.apache.org/jira/browse/SOLR-11660
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Backup/Restore, SolrCloud
>Affects Versions: 6.4.1
> Environment: CentOs7.3 Solr 6.4.1 Zookeeper 3.4.6
>Reporter: Viachaslau Kabak
>Assignee: Varun Thacker
>Priority: Critical
>
> We use to backup and restore one of our collections in solrCloud.
> if not to restore collection but to create it and fill from application - the 
> second attempt to renew application index is successful both on leader and 
> followers. But is to restore collection and then to renew index from 
> application - we have new index on follower and old index on leader.
> The Reload on collection helps to make index up-to-date on leader.
> In logs we have {{o.a.s.u.p.DistributedUpdateProcessor Ignoring commit while 
> not ACTIVE - state: BUFFERING replay: false}}
> If to restart solr service before collection update from application - index 
> on all solr servers is updated.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11660) Issue while update index in collection after collection restore on Solr Cloud

2018-05-02 Thread Shawn Heisey (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-11660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461845#comment-16461845
 ] 

Shawn Heisey commented on SOLR-11660:
-

Trying to reproduce this with version 7.3.0 on the cloud example.  I created a 
'techproducts' collection with 4 shards, 2 replicas, indexed the example docs 
to it, then followed the reproduction steps.  I got this error trying to 
restore, which is a separate issue:

"org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: 
Solr cloud with available number of nodes:2 is insufficient for restoring a 
collection with 4 shards, total replicas per shard 1 and maxShardsPerNode -1. 
Consider increasing maxShardsPerNode value OR number of available nodes."

Then I added parameters to the restore command, including replicationFactor=2, 
but when it got restored, there was only one replica.  So I'm not sure what I 
need to do in order to get two replicas.

> Issue while update index in collection after collection restore on Solr Cloud
> -
>
> Key: SOLR-11660
> URL: https://issues.apache.org/jira/browse/SOLR-11660
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Backup/Restore, SolrCloud
>Affects Versions: 6.4.1
> Environment: CentOs7.3 Solr 6.4.1 Zookeeper 3.4.6
>Reporter: Viachaslau Kabak
>Assignee: Varun Thacker
>Priority: Critical
>
> We use to backup and restore one of our collections in solrCloud.
> if not to restore collection but to create it and fill from application - the 
> second attempt to renew application index is successful both on leader and 
> followers. But is to restore collection and then to renew index from 
> application - we have new index on follower and old index on leader.
> The Reload on collection helps to make index up-to-date on leader.
> In logs we have {{o.a.s.u.p.DistributedUpdateProcessor Ignoring commit while 
> not ACTIVE - state: BUFFERING replay: false}}
> If to restart solr service before collection update from application - index 
> on all solr servers is updated.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-12305) Making buffering tlog bounded for faster recovery

2018-05-02 Thread Cao Manh Dat (JIRA)

Cao Manh Dat created SOLR-12305:
---

Summary: Making buffering tlog bounded for faster recovery
Key: SOLR-12305
URL: https://issues.apache.org/jira/browse/SOLR-12305
Project: Solr
Issue Type: Improvement
Security Level: Public (Default Security Level. Issues are Public)
Reporter: Cao Manh Dat
Assignee: Cao Manh Dat

The current recovery process has 2 main problems (pointed out by
[~shalinmangar] ) which make it may never finish.
# The replay updates process is too slow, we do it in a single-thread fashion.
Therefore if the more updates get appended at a faster rate, the replay process
will be never finished
# The buffering tlog is unbounded, we keep adding more entries to buffering
tlog and waiting for them to get replayed. If we have a way to reduce the
number of updates in buffering tlog, even when replay process is slow it will
eventually finish.

I come up with a solution for the second problem which is described on this
link:

[https://docs.google.com/document/d/14DCkYRvYnQmactyWek3nYtUVdpu_CVIA4ZBTfQigjlU/edit?usp=sharing]

In short, the document presents a modification for current recovery process
(section 3: algorithm) and also proof the correctness of the modification
(section 1 and 2). There are some pros in this approach
* Making buffering tlog bounded.
* It will automatically throttle updates from the leader, imagine this case
** We have a shard with a leader and a replica. When leader sends replica an
update.
** If the replica is healthy, the leader will have to wait for the replica to
finish process that updates before return to users. Let's call the total time
for an update is T0
** If the replica is recovering, in the current code, the replica will only
append that update to its buffering tlog (which is much faster than indexing),
so the total time for an update is T1 < T0. Therefore the rate of incoming
updates will be higher in this case.
** In above design, T1 will be subequal to T0.

1 2 >

1 - 100 of 132 matches

Mail list logo