[jira] [Commented] (SOLR-12497) Add ref guide docs for Hadoop Credential Provider based SSL/TLS store password source.
[ https://issues.apache.org/jira/browse/SOLR-12497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16687709#comment-16687709 ] Mano Kovacs commented on SOLR-12497: [~ctargett], looks great, thank you for taking the time. > Add ref guide docs for Hadoop Credential Provider based SSL/TLS store > password source. > -- > > Key: SOLR-12497 > URL: https://issues.apache.org/jira/browse/SOLR-12497 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: documentation >Affects Versions: 7.4 >Reporter: Mano Kovacs >Assignee: Cassandra Targett >Priority: Minor > Fix For: 7.6, master (8.0) > > Attachments: SOLR-12497.patch > > > Document configuration added in SOLR-10783. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8335) HdfsLockFactory does not allow core to come up after a node was killed
[
https://issues.apache.org/jira/browse/SOLR-8335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16663474#comment-16663474
]
Mano Kovacs commented on SOLR-8335:
---
[~varunthacker], [~markrmil...@gmail.com], could you help me reviewing this
jira? Thank you in advance.
> HdfsLockFactory does not allow core to come up after a node was killed
> --
>
> Key: SOLR-8335
> URL: https://issues.apache.org/jira/browse/SOLR-8335
> Project: Solr
> Issue Type: Bug
>Affects Versions: 5.0, 5.1, 5.2, 5.2.1, 5.3, 5.3.1
>Reporter: Varun Thacker
>Assignee: Mark Miller
>Priority: Major
> Attachments: SOLR-8335.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> When using HdfsLockFactory if a node gets killed instead of a graceful
> shutdown the write.lock file remains in HDFS . The next time you start the
> node the core doesn't load up because of LockObtainFailedException .
> I was able to reproduce this in all 5.x versions of Solr . The problem wasn't
> there when I tested it in 4.10.4
> Steps to reproduce this on 5.x
> 1. Create directory in HDFS : {{bin/hdfs dfs -mkdir /solr}}
> 2. Start Solr: {{bin/solr start -Dsolr.directoryFactory=HdfsDirectoryFactory
> -Dsolr.lock.type=hdfs -Dsolr.data.dir=hdfs://localhost:9000/solr
> -Dsolr.updatelog=hdfs://localhost:9000/solr}}
> 3. Create core: {{./bin/solr create -c test -n data_driven}}
> 4. Kill solr
> 5. The lock file is there in HDFS and is called {{write.lock}}
> 6. Start Solr again and you get a stack trace like this:
> {code}
> 2015-11-23 13:28:04.287 ERROR (coreLoadExecutor-6-thread-1) [ x:test]
> o.a.s.c.CoreContainer Error creating core [test]: Index locked for write for
> core 'test'. Solr now longer supports forceful unlocking via
> 'unlockOnStartup'. Please verify locks manually!
> org.apache.solr.common.SolrException: Index locked for write for core 'test'.
> Solr now longer supports forceful unlocking via 'unlockOnStartup'. Please
> verify locks manually!
> at org.apache.solr.core.SolrCore.(SolrCore.java:820)
> at org.apache.solr.core.SolrCore.(SolrCore.java:659)
> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:723)
> at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:443)
> at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:434)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:210)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.lucene.store.LockObtainFailedException: Index locked
> for write for core 'test'. Solr now longer supports forceful unlocking via
> 'unlockOnStartup'. Please verify locks manually!
> at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:528)
> at org.apache.solr.core.SolrCore.(SolrCore.java:761)
> ... 9 more
> 2015-11-23 13:28:04.289 ERROR (coreContainerWorkExecutor-2-thread-1) [ ]
> o.a.s.c.CoreContainer Error waiting for SolrCore to be created
> java.util.concurrent.ExecutionException:
> org.apache.solr.common.SolrException: Unable to create core [test]
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> at org.apache.solr.core.CoreContainer$2.run(CoreContainer.java:472)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:210)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.solr.common.SolrException: Unable to create core [test]
> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:737)
> at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:443)
> at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:434)
> ... 5 more
> Caused by: org.apache.solr.common.SolrException: Index locked for write for
> core 'test'. Solr now longer supports forceful unlocking via
> 'unlockOnStartup'. Please verify locks manually!
> at org.apache.solr.core.SolrCore.(SolrCore.java:820)
> at org.apache.solr.core.SolrCore.(SolrCore
[jira] [Commented] (SOLR-12497) Add ref guide docs for Hadoop Credential Provider based SSL/TLS store password source.
[ https://issues.apache.org/jira/browse/SOLR-12497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16651793#comment-16651793 ] Mano Kovacs commented on SOLR-12497: [~ctargett], I added a patch with docs for SOLR-10783. Could you please review it? > Add ref guide docs for Hadoop Credential Provider based SSL/TLS store > password source. > -- > > Key: SOLR-12497 > URL: https://issues.apache.org/jira/browse/SOLR-12497 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: documentation >Affects Versions: 7.4 >Reporter: Mano Kovacs >Priority: Minor > Attachments: SOLR-12497.patch > > > Document configuration added in SOLR-10783. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8335) HdfsLockFactory does not allow core to come up after a node was killed
[
https://issues.apache.org/jira/browse/SOLR-8335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16651234#comment-16651234
]
Mano Kovacs commented on SOLR-8335:
---
Review would be greatly appreciated!
https://github.com/apache/lucene-solr/pull/471
> HdfsLockFactory does not allow core to come up after a node was killed
> --
>
> Key: SOLR-8335
> URL: https://issues.apache.org/jira/browse/SOLR-8335
> Project: Solr
> Issue Type: Bug
>Affects Versions: 5.0, 5.1, 5.2, 5.2.1, 5.3, 5.3.1
>Reporter: Varun Thacker
>Assignee: Mark Miller
>Priority: Major
> Attachments: SOLR-8335.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> When using HdfsLockFactory if a node gets killed instead of a graceful
> shutdown the write.lock file remains in HDFS . The next time you start the
> node the core doesn't load up because of LockObtainFailedException .
> I was able to reproduce this in all 5.x versions of Solr . The problem wasn't
> there when I tested it in 4.10.4
> Steps to reproduce this on 5.x
> 1. Create directory in HDFS : {{bin/hdfs dfs -mkdir /solr}}
> 2. Start Solr: {{bin/solr start -Dsolr.directoryFactory=HdfsDirectoryFactory
> -Dsolr.lock.type=hdfs -Dsolr.data.dir=hdfs://localhost:9000/solr
> -Dsolr.updatelog=hdfs://localhost:9000/solr}}
> 3. Create core: {{./bin/solr create -c test -n data_driven}}
> 4. Kill solr
> 5. The lock file is there in HDFS and is called {{write.lock}}
> 6. Start Solr again and you get a stack trace like this:
> {code}
> 2015-11-23 13:28:04.287 ERROR (coreLoadExecutor-6-thread-1) [ x:test]
> o.a.s.c.CoreContainer Error creating core [test]: Index locked for write for
> core 'test'. Solr now longer supports forceful unlocking via
> 'unlockOnStartup'. Please verify locks manually!
> org.apache.solr.common.SolrException: Index locked for write for core 'test'.
> Solr now longer supports forceful unlocking via 'unlockOnStartup'. Please
> verify locks manually!
> at org.apache.solr.core.SolrCore.(SolrCore.java:820)
> at org.apache.solr.core.SolrCore.(SolrCore.java:659)
> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:723)
> at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:443)
> at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:434)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:210)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.lucene.store.LockObtainFailedException: Index locked
> for write for core 'test'. Solr now longer supports forceful unlocking via
> 'unlockOnStartup'. Please verify locks manually!
> at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:528)
> at org.apache.solr.core.SolrCore.(SolrCore.java:761)
> ... 9 more
> 2015-11-23 13:28:04.289 ERROR (coreContainerWorkExecutor-2-thread-1) [ ]
> o.a.s.c.CoreContainer Error waiting for SolrCore to be created
> java.util.concurrent.ExecutionException:
> org.apache.solr.common.SolrException: Unable to create core [test]
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> at org.apache.solr.core.CoreContainer$2.run(CoreContainer.java:472)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:210)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.solr.common.SolrException: Unable to create core [test]
> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:737)
> at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:443)
> at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:434)
> ... 5 more
> Caused by: org.apache.solr.common.SolrException: Index locked for write for
> core 'test'. Solr now longer supports forceful unlocking via
> 'unlockOnStartup'. Please verify locks manually!
> at org.apache.solr.core.SolrCore.(SolrCore.java:820)
> at org.apache.solr.core.SolrCore.(SolrCore.java:659)
>
[jira] [Commented] (SOLR-12708) Async collection actions should not hide failures
[
https://issues.apache.org/jira/browse/SOLR-12708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650196#comment-16650196
]
Mano Kovacs commented on SOLR-12708:
[~varunthacker], thank you for the explanation! I am a still a bit behind, I
think I don't understand one part.
bq. CreateShardCmd is one core admin API call . So the response is either a
success or failure. Hence the if-else block covers it.
I don't understand this part. I used this command as base, as similarly to the
restore command; create shard operation involves adding multiple replicas. I
think this addReplica command in {{CreateShardCmd}} is called multiple due to
[this
for-cycle|http://github.mtv.cloudera.com/CDH/lucene-solr/blob/cdh6.x/solr/core/src/java/org/apache/solr/cloud/api/collections/CreateShardCmd.java#L91].
I assume this is the reason why multiple failures could be catched.
bq. This way we process requests and responses are very complicated for some
reason and we should improve it in general . But do you see what I am seeing
here?
Not sure. I think it is more beneficial to see every failure, instead of the
first/last one. Especially since they are executed parallel and might have
side-effects that require cleanup.
> Async collection actions should not hide failures
> -
>
> Key: SOLR-12708
> URL: https://issues.apache.org/jira/browse/SOLR-12708
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Admin UI, Backup/Restore
>Affects Versions: 7.4
>Reporter: Mano Kovacs
>Assignee: Varun Thacker
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Async collection API may hide failures compared to sync version.
> [OverseerCollectionMessageHandler::processResponses|https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/cloud/api/collections/OverseerCollectionMessageHandler.java#L744]
> structures errors differently in the response, that hides failures from most
> evaluators. RestoreCmd did not receive, nor handle async addReplica issues.
> Sample create collection sync and async result with invalid solrconfig.xml:
> {noformat}
> {
> "responseHeader":{
> "status":0,
> "QTime":32104},
> "failure":{
> "localhost:8983_solr":"org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:Error
> from server at http://localhost:8983/solr: Error CREATEing SolrCore
> 'name4_shard1_replica_n1': Unable to create core [name4_shard1_replica_n1]
> Caused by: The content of elements must consist of well-formed character data
> or markup.",
> "localhost:8983_solr":"org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:Error
> from server at http://localhost:8983/solr: Error CREATEing SolrCore
> 'name4_shard2_replica_n2': Unable to create core [name4_shard2_replica_n2]
> Caused by: The content of elements must consist of well-formed character data
> or markup.",
> "localhost:8983_solr":"org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:Error
> from server at http://localhost:8983/solr: Error CREATEing SolrCore
> 'name4_shard1_replica_n2': Unable to create core [name4_shard1_replica_n2]
> Caused by: The content of elements must consist of well-formed character data
> or markup.",
> "localhost:8983_solr":"org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:Error
> from server at http://localhost:8983/solr: Error CREATEing SolrCore
> 'name4_shard2_replica_n1': Unable to create core [name4_shard2_replica_n1]
> Caused by: The content of elements must consist of well-formed character data
> or markup."}
> }
> {noformat}
> vs async:
> {noformat}
> {
> "responseHeader":{
> "status":0,
> "QTime":3},
> "success":{
> "localhost:8983_solr":{
> "responseHeader":{
> "status":0,
> "QTime":12}},
> "localhost:8983_solr":{
> "responseHeader":{
> "status":0,
> "QTime":3}},
> "localhost:8983_solr":{
> "responseHeader":{
> "status":0,
> "QTime":11}},
> "localhost:8983_solr":{
> "responseHeader":{
> "status":0,
> "QTime":12}}},
> "myTaskId2709146382836":{
> "responseHeader":{
> "status":0,
> "QTime":1},
> "STATUS":"failed",
> "Response":"Error CREATEing SolrCore 'name_shard2_replica_n2': Unable to
> create core [name_shard2_replica_n2] Caused by: The content of elements must
> consist of well-formed character data or markup."},
> "status":{
> "state":"completed",
> "msg":"found [myTaskId] in completed tasks"}}
> {noformat}
> Proposing adding failure node to the results, keeping backward compatible but
> correct result.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands
[jira] [Commented] (SOLR-8335) HdfsLockFactory does not allow core to come up after a node was killed
[
https://issues.apache.org/jira/browse/SOLR-8335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16631699#comment-16631699
]
Mano Kovacs commented on SOLR-8335:
---
I removed hamcrest from the patch. If I could get a review, that would be
greatly appreciated: https://github.com/apache/lucene-solr/pull/459
> HdfsLockFactory does not allow core to come up after a node was killed
> --
>
> Key: SOLR-8335
> URL: https://issues.apache.org/jira/browse/SOLR-8335
> Project: Solr
> Issue Type: Bug
>Affects Versions: 5.0, 5.1, 5.2, 5.2.1, 5.3, 5.3.1
>Reporter: Varun Thacker
>Assignee: Mark Miller
>Priority: Major
> Attachments: SOLR-8335.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When using HdfsLockFactory if a node gets killed instead of a graceful
> shutdown the write.lock file remains in HDFS . The next time you start the
> node the core doesn't load up because of LockObtainFailedException .
> I was able to reproduce this in all 5.x versions of Solr . The problem wasn't
> there when I tested it in 4.10.4
> Steps to reproduce this on 5.x
> 1. Create directory in HDFS : {{bin/hdfs dfs -mkdir /solr}}
> 2. Start Solr: {{bin/solr start -Dsolr.directoryFactory=HdfsDirectoryFactory
> -Dsolr.lock.type=hdfs -Dsolr.data.dir=hdfs://localhost:9000/solr
> -Dsolr.updatelog=hdfs://localhost:9000/solr}}
> 3. Create core: {{./bin/solr create -c test -n data_driven}}
> 4. Kill solr
> 5. The lock file is there in HDFS and is called {{write.lock}}
> 6. Start Solr again and you get a stack trace like this:
> {code}
> 2015-11-23 13:28:04.287 ERROR (coreLoadExecutor-6-thread-1) [ x:test]
> o.a.s.c.CoreContainer Error creating core [test]: Index locked for write for
> core 'test'. Solr now longer supports forceful unlocking via
> 'unlockOnStartup'. Please verify locks manually!
> org.apache.solr.common.SolrException: Index locked for write for core 'test'.
> Solr now longer supports forceful unlocking via 'unlockOnStartup'. Please
> verify locks manually!
> at org.apache.solr.core.SolrCore.(SolrCore.java:820)
> at org.apache.solr.core.SolrCore.(SolrCore.java:659)
> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:723)
> at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:443)
> at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:434)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:210)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.lucene.store.LockObtainFailedException: Index locked
> for write for core 'test'. Solr now longer supports forceful unlocking via
> 'unlockOnStartup'. Please verify locks manually!
> at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:528)
> at org.apache.solr.core.SolrCore.(SolrCore.java:761)
> ... 9 more
> 2015-11-23 13:28:04.289 ERROR (coreContainerWorkExecutor-2-thread-1) [ ]
> o.a.s.c.CoreContainer Error waiting for SolrCore to be created
> java.util.concurrent.ExecutionException:
> org.apache.solr.common.SolrException: Unable to create core [test]
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> at org.apache.solr.core.CoreContainer$2.run(CoreContainer.java:472)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:210)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.solr.common.SolrException: Unable to create core [test]
> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:737)
> at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:443)
> at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:434)
> ... 5 more
> Caused by: org.apache.solr.common.SolrException: Index locked for write for
> core 'test'. Solr now longer supports forceful unlocking via
> 'unlockOnStartup'. Please verify locks manually!
> at org.apache.solr.core.SolrCore.(SolrCore.java:820)
> at or
[jira] [Commented] (SOLR-8335) HdfsLockFactory does not allow core to come up after a node was killed
[
https://issues.apache.org/jira/browse/SOLR-8335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16630042#comment-16630042
]
Mano Kovacs commented on SOLR-8335:
---
If [~mihaly.toth] does not mind, I will take on the remaining work regarding
this patch. I rebased the patch and opened a PR. I will remove the hamcrest
dependency too.
> HdfsLockFactory does not allow core to come up after a node was killed
> --
>
> Key: SOLR-8335
> URL: https://issues.apache.org/jira/browse/SOLR-8335
> Project: Solr
> Issue Type: Bug
>Affects Versions: 5.0, 5.1, 5.2, 5.2.1, 5.3, 5.3.1
>Reporter: Varun Thacker
>Assignee: Mark Miller
>Priority: Major
> Attachments: SOLR-8335.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When using HdfsLockFactory if a node gets killed instead of a graceful
> shutdown the write.lock file remains in HDFS . The next time you start the
> node the core doesn't load up because of LockObtainFailedException .
> I was able to reproduce this in all 5.x versions of Solr . The problem wasn't
> there when I tested it in 4.10.4
> Steps to reproduce this on 5.x
> 1. Create directory in HDFS : {{bin/hdfs dfs -mkdir /solr}}
> 2. Start Solr: {{bin/solr start -Dsolr.directoryFactory=HdfsDirectoryFactory
> -Dsolr.lock.type=hdfs -Dsolr.data.dir=hdfs://localhost:9000/solr
> -Dsolr.updatelog=hdfs://localhost:9000/solr}}
> 3. Create core: {{./bin/solr create -c test -n data_driven}}
> 4. Kill solr
> 5. The lock file is there in HDFS and is called {{write.lock}}
> 6. Start Solr again and you get a stack trace like this:
> {code}
> 2015-11-23 13:28:04.287 ERROR (coreLoadExecutor-6-thread-1) [ x:test]
> o.a.s.c.CoreContainer Error creating core [test]: Index locked for write for
> core 'test'. Solr now longer supports forceful unlocking via
> 'unlockOnStartup'. Please verify locks manually!
> org.apache.solr.common.SolrException: Index locked for write for core 'test'.
> Solr now longer supports forceful unlocking via 'unlockOnStartup'. Please
> verify locks manually!
> at org.apache.solr.core.SolrCore.(SolrCore.java:820)
> at org.apache.solr.core.SolrCore.(SolrCore.java:659)
> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:723)
> at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:443)
> at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:434)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:210)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.lucene.store.LockObtainFailedException: Index locked
> for write for core 'test'. Solr now longer supports forceful unlocking via
> 'unlockOnStartup'. Please verify locks manually!
> at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:528)
> at org.apache.solr.core.SolrCore.(SolrCore.java:761)
> ... 9 more
> 2015-11-23 13:28:04.289 ERROR (coreContainerWorkExecutor-2-thread-1) [ ]
> o.a.s.c.CoreContainer Error waiting for SolrCore to be created
> java.util.concurrent.ExecutionException:
> org.apache.solr.common.SolrException: Unable to create core [test]
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> at org.apache.solr.core.CoreContainer$2.run(CoreContainer.java:472)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:210)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.solr.common.SolrException: Unable to create core [test]
> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:737)
> at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:443)
> at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:434)
> ... 5 more
> Caused by: org.apache.solr.common.SolrException: Index locked for write for
> core 'test'. Solr now longer supports forceful unlocking via
> 'unlockOnStartup'. Please verify locks manually!
> at org.apache.solr.core.SolrCore.(SolrCo
[jira] [Commented] (SOLR-12708) Async collection actions should not hide failures
[
https://issues.apache.org/jira/browse/SOLR-12708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16628739#comment-16628739
]
Mano Kovacs commented on SOLR-12708:
Hello [~varunthacker], thank you for the review!
bq. I'm curious about the 10 minute latch countdown timeout. Shouldn't we wait
forever here?
I think if we would wait forever, any downstream command that stuck or never
get result would keep this job hanging as well. I would worry about the
robustness here. This part of the code creates a bunch of empty cores (one per
shards) in parallel. Considering a larger, 200-300 shard cluster, this might
take longer than 10 minutes if the overseer queue is already behind, so 10
minutes in fact might be problematic. However, if Overseer is getting behind
much more than that, it would seriously hurt the stability of the cluster
anyway. I increase this wait for an hour, if you agree, which would leave
plenty of time for overseer to process the core creation on a relatively large
collection, but still ensures that the job is getting cancelled if one task
stucks.
bq. So here we're doing something different wrt success and failure . If the
add replica call has a failure we're adding it back to the main response but if
it's a success then we will end up skipping it ( at this point
results.get("success") will always be null ) .
I have to be honest and admit that I copied the full block from
{{CreateShardCmd.java}}. I think the code is doing the right thing there. In
both branches of the {{if}} the code checks if the main {{results}} has
success/failure node already, and creates if necessary. Then adds the
corresponding {{addResult}} field into the main one. The only difference is
that the failure recalled before the {{if}} block.
bq. Can't we do this instead which will append the results directly to the main
object? We do this for the remaining add replicas as the last step of the
restore
Then we may let the downstream call override certain other fields that might be
populated. I think isolation makes it more error-prone. I think this was Dat's
original intent as well in {{CreateShardCmd}}, but not sure.
> Async collection actions should not hide failures
> -
>
> Key: SOLR-12708
> URL: https://issues.apache.org/jira/browse/SOLR-12708
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Admin UI, Backup/Restore
>Affects Versions: 7.4
>Reporter: Mano Kovacs
>Assignee: Varun Thacker
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Async collection API may hide failures compared to sync version.
> [OverseerCollectionMessageHandler::processResponses|https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/cloud/api/collections/OverseerCollectionMessageHandler.java#L744]
> structures errors differently in the response, that hides failures from most
> evaluators. RestoreCmd did not receive, nor handle async addReplica issues.
> Sample create collection sync and async result with invalid solrconfig.xml:
> {noformat}
> {
> "responseHeader":{
> "status":0,
> "QTime":32104},
> "failure":{
> "localhost:8983_solr":"org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:Error
> from server at http://localhost:8983/solr: Error CREATEing SolrCore
> 'name4_shard1_replica_n1': Unable to create core [name4_shard1_replica_n1]
> Caused by: The content of elements must consist of well-formed character data
> or markup.",
> "localhost:8983_solr":"org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:Error
> from server at http://localhost:8983/solr: Error CREATEing SolrCore
> 'name4_shard2_replica_n2': Unable to create core [name4_shard2_replica_n2]
> Caused by: The content of elements must consist of well-formed character data
> or markup.",
> "localhost:8983_solr":"org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:Error
> from server at http://localhost:8983/solr: Error CREATEing SolrCore
> 'name4_shard1_replica_n2': Unable to create core [name4_shard1_replica_n2]
> Caused by: The content of elements must consist of well-formed character data
> or markup.",
> "localhost:8983_solr":"org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:Error
> from server at http://localhost:8983/solr: Error CREATEing SolrCore
> 'name4_shard2_replica_n1': Unable to create core [name4_shard2_replica_n1]
> Caused by: The content of elements must consist of well-formed character data
> or markup."}
> }
> {noformat}
> vs async:
> {noformat}
> {
> "responseHeader":{
> "status":0,
> "QTime":3},
> "success":{
> "localhost:8983_solr":{
> "responseHeader":{
> "status":0,
> "QTime":12}},
> "localhost:
[jira] [Commented] (SOLR-12708) Async collection actions should not hide failures
[
https://issues.apache.org/jira/browse/SOLR-12708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16620510#comment-16620510
]
Mano Kovacs commented on SOLR-12708:
Any review would be greatly appreciated.
> Async collection actions should not hide failures
> -
>
> Key: SOLR-12708
> URL: https://issues.apache.org/jira/browse/SOLR-12708
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Admin UI, Backup/Restore
>Affects Versions: 7.4
>Reporter: Mano Kovacs
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Async collection API may hide failures compared to sync version.
> [OverseerCollectionMessageHandler::processResponses|https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/cloud/api/collections/OverseerCollectionMessageHandler.java#L744]
> structures errors differently in the response, that hides failures from most
> evaluators. RestoreCmd did not receive, nor handle async addReplica issues.
> Sample create collection sync and async result with invalid solrconfig.xml:
> {noformat}
> {
> "responseHeader":{
> "status":0,
> "QTime":32104},
> "failure":{
> "localhost:8983_solr":"org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:Error
> from server at http://localhost:8983/solr: Error CREATEing SolrCore
> 'name4_shard1_replica_n1': Unable to create core [name4_shard1_replica_n1]
> Caused by: The content of elements must consist of well-formed character data
> or markup.",
> "localhost:8983_solr":"org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:Error
> from server at http://localhost:8983/solr: Error CREATEing SolrCore
> 'name4_shard2_replica_n2': Unable to create core [name4_shard2_replica_n2]
> Caused by: The content of elements must consist of well-formed character data
> or markup.",
> "localhost:8983_solr":"org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:Error
> from server at http://localhost:8983/solr: Error CREATEing SolrCore
> 'name4_shard1_replica_n2': Unable to create core [name4_shard1_replica_n2]
> Caused by: The content of elements must consist of well-formed character data
> or markup.",
> "localhost:8983_solr":"org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:Error
> from server at http://localhost:8983/solr: Error CREATEing SolrCore
> 'name4_shard2_replica_n1': Unable to create core [name4_shard2_replica_n1]
> Caused by: The content of elements must consist of well-formed character data
> or markup."}
> }
> {noformat}
> vs async:
> {noformat}
> {
> "responseHeader":{
> "status":0,
> "QTime":3},
> "success":{
> "localhost:8983_solr":{
> "responseHeader":{
> "status":0,
> "QTime":12}},
> "localhost:8983_solr":{
> "responseHeader":{
> "status":0,
> "QTime":3}},
> "localhost:8983_solr":{
> "responseHeader":{
> "status":0,
> "QTime":11}},
> "localhost:8983_solr":{
> "responseHeader":{
> "status":0,
> "QTime":12}}},
> "myTaskId2709146382836":{
> "responseHeader":{
> "status":0,
> "QTime":1},
> "STATUS":"failed",
> "Response":"Error CREATEing SolrCore 'name_shard2_replica_n2': Unable to
> create core [name_shard2_replica_n2] Caused by: The content of elements must
> consist of well-formed character data or markup."},
> "status":{
> "state":"completed",
> "msg":"found [myTaskId] in completed tasks"}}
> {noformat}
> Proposing adding failure node to the results, keeping backward compatible but
> correct result.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-12708) Async collection actions should not hide failures
Mano Kovacs created SOLR-12708:
--
Summary: Async collection actions should not hide failures
Key: SOLR-12708
URL: https://issues.apache.org/jira/browse/SOLR-12708
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Components: Admin UI, Backup/Restore
Affects Versions: 7.4
Reporter: Mano Kovacs
Async collection API may hide failures compared to sync version.
[OverseerCollectionMessageHandler::processResponses|https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/cloud/api/collections/OverseerCollectionMessageHandler.java#L744]
structures errors differently in the response, that hides failures from most
evaluators. RestoreCmd did not receive, nor handle async addReplica issues.
Sample create collection sync and async result with invalid solrconfig.xml:
{noformat}
{
"responseHeader":{
"status":0,
"QTime":32104},
"failure":{
"localhost:8983_solr":"org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:Error
from server at http://localhost:8983/solr: Error CREATEing SolrCore
'name4_shard1_replica_n1': Unable to create core [name4_shard1_replica_n1]
Caused by: The content of elements must consist of well-formed character data
or markup.",
"localhost:8983_solr":"org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:Error
from server at http://localhost:8983/solr: Error CREATEing SolrCore
'name4_shard2_replica_n2': Unable to create core [name4_shard2_replica_n2]
Caused by: The content of elements must consist of well-formed character data
or markup.",
"localhost:8983_solr":"org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:Error
from server at http://localhost:8983/solr: Error CREATEing SolrCore
'name4_shard1_replica_n2': Unable to create core [name4_shard1_replica_n2]
Caused by: The content of elements must consist of well-formed character data
or markup.",
"localhost:8983_solr":"org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:Error
from server at http://localhost:8983/solr: Error CREATEing SolrCore
'name4_shard2_replica_n1': Unable to create core [name4_shard2_replica_n1]
Caused by: The content of elements must consist of well-formed character data
or markup."}
}
{noformat}
vs async:
{noformat}
{
"responseHeader":{
"status":0,
"QTime":3},
"success":{
"localhost:8983_solr":{
"responseHeader":{
"status":0,
"QTime":12}},
"localhost:8983_solr":{
"responseHeader":{
"status":0,
"QTime":3}},
"localhost:8983_solr":{
"responseHeader":{
"status":0,
"QTime":11}},
"localhost:8983_solr":{
"responseHeader":{
"status":0,
"QTime":12}}},
"myTaskId2709146382836":{
"responseHeader":{
"status":0,
"QTime":1},
"STATUS":"failed",
"Response":"Error CREATEing SolrCore 'name_shard2_replica_n2': Unable to create
core [name_shard2_replica_n2] Caused by: The content of elements must consist
of well-formed character data or markup."},
"status":{
"state":"completed",
"msg":"found [myTaskId] in completed tasks"}}
{noformat}
Proposing adding failure node to the results, keeping backward compatible but
correct result.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10783) Using Hadoop Credential Provider as SSL/TLS store password source
[ https://issues.apache.org/jira/browse/SOLR-10783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16534584#comment-16534584 ] Mano Kovacs commented on SOLR-10783: [~jafurrer], it seems like it. The reason it is printed out is because SolrCLI also initializes SSL to talk to Solr instances and it has no log4j config. So basically that is not Solr printing it out, but SolrCLI. Sorry for missing that, I'll prepare a fix. > Using Hadoop Credential Provider as SSL/TLS store password source > - > > Key: SOLR-10783 > URL: https://issues.apache.org/jira/browse/SOLR-10783 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: security >Affects Versions: 7.0 >Reporter: Mano Kovacs >Assignee: Mark Miller >Priority: Major > Fix For: 7.4, master (8.0) > > Attachments: SOLR-10783-fix.patch, SOLR-10783.patch, > SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch, > SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch > > > As a second iteration of SOLR-10307, I propose support of hadoop credential > providers as source of SSL store passwords. > Motivation: When SOLR is used in hadoop environment, support of HCP gives > better integration and unified method to pass sensitive credentials to SOLR. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11417) Crashed leader's hanging emphemral will make restarting followers stuck in recovering
[ https://issues.apache.org/jira/browse/SOLR-11417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mano Kovacs updated SOLR-11417: --- Fix Version/s: 7.3 > Crashed leader's hanging emphemral will make restarting followers stuck in > recovering > - > > Key: SOLR-11417 > URL: https://issues.apache.org/jira/browse/SOLR-11417 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: 6.3 >Reporter: Mano Kovacs >Priority: Major > Fix For: 7.3 > > Attachments: SOLR-11417.png > > > If replicas are starting up after leader crash and within the ZK session > timeout, replicas > * will lose leader election due to hanging ephemerals > * will read stale data from ZK about current leader > * will fail recovery and stuck in recovering state > If leader is down permanently (eg. hardware failure) and all replicas are > affected, shard will not come up (see also SOLR-7065). > Tested on 6.3. See attached image for details. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-11417) Crashed leader's hanging emphemral will make restarting followers stuck in recovering
[ https://issues.apache.org/jira/browse/SOLR-11417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mano Kovacs resolved SOLR-11417. Resolution: Fixed Fixed by SOLR-12011 > Crashed leader's hanging emphemral will make restarting followers stuck in > recovering > - > > Key: SOLR-11417 > URL: https://issues.apache.org/jira/browse/SOLR-11417 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: 6.3 >Reporter: Mano Kovacs >Priority: Major > Attachments: SOLR-11417.png > > > If replicas are starting up after leader crash and within the ZK session > timeout, replicas > * will lose leader election due to hanging ephemerals > * will read stale data from ZK about current leader > * will fail recovery and stuck in recovering state > If leader is down permanently (eg. hardware failure) and all replicas are > affected, shard will not come up (see also SOLR-7065). > Tested on 6.3. See attached image for details. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10783) Using Hadoop Credential Provider as SSL/TLS store password source
[ https://issues.apache.org/jira/browse/SOLR-10783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16516857#comment-16516857 ] Mano Kovacs commented on SOLR-10783: [~ctargett], you are right, this should have docs in the Ref Guide. Filed SOLR-12497, will be working on that this week. > Using Hadoop Credential Provider as SSL/TLS store password source > - > > Key: SOLR-10783 > URL: https://issues.apache.org/jira/browse/SOLR-10783 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: security >Affects Versions: 7.0 >Reporter: Mano Kovacs >Assignee: Mark Miller >Priority: Major > Fix For: 7.4, master (8.0) > > Attachments: SOLR-10783-fix.patch, SOLR-10783.patch, > SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch, > SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch > > > As a second iteration of SOLR-10307, I propose support of hadoop credential > providers as source of SSL store passwords. > Motivation: When SOLR is used in hadoop environment, support of HCP gives > better integration and unified method to pass sensitive credentials to SOLR. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-12497) Add ref guide docs for Hadoop Credential Provider based SSL/TLS store password source.
Mano Kovacs created SOLR-12497: -- Summary: Add ref guide docs for Hadoop Credential Provider based SSL/TLS store password source. Key: SOLR-12497 URL: https://issues.apache.org/jira/browse/SOLR-12497 Project: Solr Issue Type: Improvement Security Level: Public (Default Security Level. Issues are Public) Components: documentation Affects Versions: 7.4 Reporter: Mano Kovacs Document configuration added in SOLR-10783. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10783) Using Hadoop Credential Provider as SSL/TLS store password source
[ https://issues.apache.org/jira/browse/SOLR-10783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16417179#comment-16417179 ] Mano Kovacs commented on SOLR-10783: [~markrmil...@gmail.com], thanks for the testing and review. I uploaded a new patch with the precommit hook running. Also, thanks for catching that, I did forget to remove from the background version. Fixed that too. > Using Hadoop Credential Provider as SSL/TLS store password source > - > > Key: SOLR-10783 > URL: https://issues.apache.org/jira/browse/SOLR-10783 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: security >Affects Versions: 7.0 >Reporter: Mano Kovacs >Assignee: Mark Miller >Priority: Major > Attachments: SOLR-10783-fix.patch, SOLR-10783.patch, > SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch, > SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch > > > As a second iteration of SOLR-10307, I propose support of hadoop credential > providers as source of SSL store passwords. > Motivation: When SOLR is used in hadoop environment, support of HCP gives > better integration and unified method to pass sensitive credentials to SOLR. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10783) Using Hadoop Credential Provider as SSL/TLS store password source
[ https://issues.apache.org/jira/browse/SOLR-10783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mano Kovacs updated SOLR-10783: --- Attachment: SOLR-10783.patch > Using Hadoop Credential Provider as SSL/TLS store password source > - > > Key: SOLR-10783 > URL: https://issues.apache.org/jira/browse/SOLR-10783 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: security >Affects Versions: 7.0 >Reporter: Mano Kovacs >Assignee: Mark Miller >Priority: Major > Attachments: SOLR-10783-fix.patch, SOLR-10783.patch, > SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch, > SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch > > > As a second iteration of SOLR-10307, I propose support of hadoop credential > providers as source of SSL store passwords. > Motivation: When SOLR is used in hadoop environment, support of HCP gives > better integration and unified method to pass sensitive credentials to SOLR. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10783) Using Hadoop Credential Provider as SSL/TLS store password source
[ https://issues.apache.org/jira/browse/SOLR-10783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mano Kovacs updated SOLR-10783: --- Attachment: SOLR-10783.patch > Using Hadoop Credential Provider as SSL/TLS store password source > - > > Key: SOLR-10783 > URL: https://issues.apache.org/jira/browse/SOLR-10783 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: security >Affects Versions: 7.0 >Reporter: Mano Kovacs >Assignee: Mark Miller >Priority: Major > Attachments: SOLR-10783-fix.patch, SOLR-10783.patch, > SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch, > SOLR-10783.patch, SOLR-10783.patch > > > As a second iteration of SOLR-10307, I propose support of hadoop credential > providers as source of SSL store passwords. > Motivation: When SOLR is used in hadoop environment, support of HCP gives > better integration and unified method to pass sensitive credentials to SOLR. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10783) Using Hadoop Credential Provider as SSL/TLS store password source
[ https://issues.apache.org/jira/browse/SOLR-10783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16406524#comment-16406524 ] Mano Kovacs commented on SOLR-10783: [~markrmil...@gmail.com], how can we proceed with this jira? > Using Hadoop Credential Provider as SSL/TLS store password source > - > > Key: SOLR-10783 > URL: https://issues.apache.org/jira/browse/SOLR-10783 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: security >Affects Versions: 7.0 >Reporter: Mano Kovacs >Assignee: Mark Miller >Priority: Major > Attachments: SOLR-10783-fix.patch, SOLR-10783.patch, > SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch, > SOLR-10783.patch > > > As a second iteration of SOLR-10307, I propose support of hadoop credential > providers as source of SSL store passwords. > Motivation: When SOLR is used in hadoop environment, support of HCP gives > better integration and unified method to pass sensitive credentials to SOLR. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10783) Using Hadoop Credential Provider as SSL/TLS store password source
[ https://issues.apache.org/jira/browse/SOLR-10783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mano Kovacs updated SOLR-10783: --- Attachment: SOLR-10783.patch > Using Hadoop Credential Provider as SSL/TLS store password source > - > > Key: SOLR-10783 > URL: https://issues.apache.org/jira/browse/SOLR-10783 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: security >Reporter: Mano Kovacs >Assignee: Mark Miller >Priority: Major > Attachments: SOLR-10783-fix.patch, SOLR-10783.patch, > SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch, > SOLR-10783.patch > > > As a second iteration of SOLR-10307, I propose support of hadoop credential > providers as source of SSL store passwords. > Motivation: When SOLR is used in hadoop environment, support of HCP gives > better integration and unified method to pass sensitive credentials to SOLR. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10912) Adding automatic patch validation
[
https://issues.apache.org/jira/browse/SOLR-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16327038#comment-16327038
]
Mano Kovacs commented on SOLR-10912:
Hey Steve, that would be great as I couldn't work on this lately and probably
will be busy in the next weeks.
The current change is here:
[https://github.com/apache/lucene-solr/compare/master...manokovacs:feature/SOLR-10912]
Also, I have the list with the remaining work I was planning to do and the open
questions.
{quote}Steps remaining:
* Multiple module test currently duplicates the failed test result, fixing
shortly.
* Add test to call {{-validate-source-patterns}}
* Finalize the runner script to setup Yetus, similarly to Hadoop.
* Verify the test-patch with Github PR
* Add Patch Available status for SOLR project (and update the script to look
for that). [~steve_rowe], regarding to our IRL conversation, could you help me
with adding that status in JIRA?
* Add {{Precommit-SOLR}} and {{Precommit-LUCENE}} jenkins jobs
Open questions:
1. As you can see, a patch with two log entries had 6 (flaky) test failures.
Assuming flaky tests will not go away very soon, would it still be useful to
have this test-patch?
2. I propose starting with the smallest set of tests (ie. affected modules) and
extend them later to dependent modules.
{quote}
Let me know if I can be of any help.
> Adding automatic patch validation
> -
>
> Key: SOLR-10912
> URL: https://issues.apache.org/jira/browse/SOLR-10912
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Build
>Reporter: Mano Kovacs
>Priority: Major
> Attachments: SOLR-10912.ok-patch-in-core.patch,
> SOLR-10912.sample-patch.patch, SOLR-10912.solj-contrib-facet-error.patch
>
>
> Proposing introduction of automated patch validation, similar what Hadoop or
> other Apache projects are using (see link). This would ensure that every
> patch passes a certain set of criterions before getting approved. It would
> save time for developer (faster feedback loop), save time for committers
> (less step to do manually), and would increase quality.
> Hadoop is currently using Apache Yetus to run validations, which seems to be
> a good direction to start. This jira could be the board of discussing the
> preferred solution.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-11702) Redesign current LIR implementation
[ https://issues.apache.org/jira/browse/SOLR-11702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16283455#comment-16283455 ] Mano Kovacs edited comment on SOLR-11702 at 12/8/17 12:31 PM: -- Really like this approach, [~caomanhdat]. Not just a cleaner and more robust approach, but I believe it could be an alternative solution for the problems that motivates SOLR-7065 and SOLR-7034. Correct me if I am wrong, but replica could become leader, regardless of their previous state or the number of replicas participating, as their (and others) term number would explicitly say if they are in sync or behind. Is my assumption correct? was (Author: manokovacs): Really like this approach, [~caomanhdat]. Not just a cleaner and more robust approach, but I believe it could be an alternative solution for the problems that motivates SOLR-7065. Correct me if I am wrong, but replica could become leader, regardless of their previous state or the number of replicas participating, as their (and others) term number would explicitly say if they are in sync or behind. Is my assumption correct? > Redesign current LIR implementation > --- > > Key: SOLR-11702 > URL: https://issues.apache.org/jira/browse/SOLR-11702 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Cao Manh Dat >Assignee: Cao Manh Dat > Attachments: SOLR-11702.patch > > > I recently looked into some problem related to racing between LIR and > Recovering. I would like to propose a totally new approach to solve SOLR-5495 > problem because fixing current implementation by a bandage will lead us to > other problems (we can not prove the correctness of the implementation). > Feel free to give comments/thoughts about this new scheme. > https://docs.google.com/document/d/1dM2GKMULsS45ZMuvtztVnM2m3fdUeRYNCyJorIIisEo/edit?usp=sharing -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11702) Redesign current LIR implementation
[ https://issues.apache.org/jira/browse/SOLR-11702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16283455#comment-16283455 ] Mano Kovacs commented on SOLR-11702: Really like this approach, [~caomanhdat]. Not just a cleaner and more robust approach, but I believe it could be an alternative solution for the problems that motivates SOLR-7065. Correct me if I am wrong, but replica could become leader, regardless of their previous state or the number of replicas participating, as their (and others) term number would explicitly say if they are in sync or behind. Is my assumption correct? > Redesign current LIR implementation > --- > > Key: SOLR-11702 > URL: https://issues.apache.org/jira/browse/SOLR-11702 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Cao Manh Dat >Assignee: Cao Manh Dat > Attachments: SOLR-11702.patch > > > I recently looked into some problem related to racing between LIR and > Recovering. I would like to propose a totally new approach to solve SOLR-5495 > problem because fixing current implementation by a bandage will lead us to > other problems (we can not prove the correctness of the implementation). > Feel free to give comments/thoughts about this new scheme. > https://docs.google.com/document/d/1dM2GKMULsS45ZMuvtztVnM2m3fdUeRYNCyJorIIisEo/edit?usp=sharing -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10307) Provide SSL/TLS keystore password a more secure way
[ https://issues.apache.org/jira/browse/SOLR-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16235602#comment-16235602 ] Mano Kovacs commented on SOLR-10307: [~varunthacker], thanks for the followup. I think we did not change any "api". Only the internals, how the password passed by solr.sh to the JVM that is changed. > Provide SSL/TLS keystore password a more secure way > --- > > Key: SOLR-10307 > URL: https://issues.apache.org/jira/browse/SOLR-10307 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: security >Reporter: Mano Kovacs >Assignee: Mark Miller >Priority: Major > Fix For: 6.7, 7.0 > > Attachments: SOLR-10307.2.patch, SOLR-10307.patch, SOLR-10307.patch, > SOLR-10307.patch > > > Currently the only way to pass server and client side SSL keytstore and > truststore passwords is to set specific environment variables that will be > passed as system properties, through command line parameter. > First option is to pass passwords through environment variables which gives a > better level of protection. Second option would be to use hadoop credential > provider interface to access credential store. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11435) Replicas failed to be deleted can overwrite replicas of recreated collections
[
https://issues.apache.org/jira/browse/SOLR-11435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16193186#comment-16193186
]
Mano Kovacs commented on SOLR-11435:
It would be important to understand what {{ZkController.checkStateInZk()}} does
when {{legacyCloud=false}}. Considering {{legacyCloud=false}} means the true
source of core-layout is ZK and ZK only, this method should prevents core to be
initialized if that core is not assigned to that node. Now, I see
[~varunthacker]'s patch as a hardening in the logic, meaning if the core was
moved in ZK, it should not be loaded on the old node (deleting collection), or
the other way, if the core is moved physically to another node, it should be
represented in ZK as well (AAR). So either that assumption is correct and that
fix is valid, or not correct, but then we need to clarify meaning of
{{legacyCloud=false}} and how we want to ensure that.
I am not sure how this was affecting AAR, but I assume there are scenarios when
the core is already being created but ZK does not reflect the new state of the
core (with the new baseurl). If that is the case, I would not think relaxation
of this {{ZkController.checkStateInZk()}} is the right solution. I think the
hardening just made a bug to surface.
I am preparing the test for this soon.
> Replicas failed to be deleted can overwrite replicas of recreated collections
> -
>
> Key: SOLR-11435
> URL: https://issues.apache.org/jira/browse/SOLR-11435
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 7.0
>Reporter: Mano Kovacs
>
> When a replica comes up that was deleted from ZK while it was gone, it can
> replace replicas in ZK if the collection an core names are equal.
> Reproduction:
> # Create {{collection1}} with 1 shard, 2 replicas on {{node1}} and {{node2}}
> # Shut down {{node2}}
> # Delete {{collection1}}
> # Create {{collection1}} on {{node1}} and {{node3}}
> # Start {{node2}}
> Expected:
> {{node2}} will not initialize node as it is not assigned to in ZK
> ({{legacyCloud=false}})
> Actual:
> {{node2}} will overwrite the {{baseurl}} in {{state.json}} for one of the
> replicas as the {{coreNodeName}} and the collection name will match the core
> it has.
> Note: SOLR-7248 introduced a {{baseurl}} comparison which was removed in
> SOLR-10279.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11435) Replicas failed to be deleted can overwrite replicas of recreated collections
[
https://issues.apache.org/jira/browse/SOLR-11435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16191072#comment-16191072
]
Mano Kovacs commented on SOLR-11435:
CC: [~varunthacker], [~markrmil...@gmail.com]
> Replicas failed to be deleted can overwrite replicas of recreated collections
> -
>
> Key: SOLR-11435
> URL: https://issues.apache.org/jira/browse/SOLR-11435
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 7.0
>Reporter: Mano Kovacs
>
> When a replica comes up that was deleted from ZK while it was gone, it can
> replace replicas in ZK if the collection an core names are equal.
> Reproduction:
> # Create {{collection1}} with 1 shard, 2 replicas on {{node1}} and {{node2}}
> # Shut down {{node2}}
> # Delete {{collection1}}
> # Create {{collection1}} on {{node1}} and {{node3}}
> # Start {{node2}}
> Expected:
> {{node2}} will not initialize node as it is not assigned to in ZK
> ({{legacyCloud=false}})
> Actual:
> {{node2}} will overwrite the {{baseurl}} in {{state.json}} for one of the
> replicas as the {{coreNodeName}} and the collection name will match the core
> it has.
> Note: SOLR-7248 introduced a {{baseurl}} comparison which was removed in
> SOLR-10279.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-11435) Replicas failed to be deleted can overwrite replicas of recreated collections
Mano Kovacs created SOLR-11435:
--
Summary: Replicas failed to be deleted can overwrite replicas of
recreated collections
Key: SOLR-11435
URL: https://issues.apache.org/jira/browse/SOLR-11435
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: 7.0
Reporter: Mano Kovacs
When a replica comes up that was deleted from ZK while it was gone, it can
replace replicas in ZK if the collection an core names are equal.
Reproduction:
# Create {{collection1}} with 1 shard, 2 replicas on {{node1}} and {{node2}}
# Shut down {{node2}}
# Delete {{collection1}}
# Create {{collection1}} on {{node1}} and {{node3}}
# Start {{node2}}
Expected:
{{node2}} will not initialize node as it is not assigned to in ZK
({{legacyCloud=false}})
Actual:
{{node2}} will overwrite the {{baseurl}} in {{state.json}} for one of the
replicas as the {{coreNodeName}} and the collection name will match the core it
has.
Note: SOLR-7248 introduced a {{baseurl}} comparison which was removed in
SOLR-10279.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10912) Adding automatic patch validation
[ https://issues.apache.org/jira/browse/SOLR-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16191046#comment-16191046 ] Mano Kovacs commented on SOLR-10912: Thank you [~aw], [~steve_rowe]! > Adding automatic patch validation > - > > Key: SOLR-10912 > URL: https://issues.apache.org/jira/browse/SOLR-10912 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Build >Reporter: Mano Kovacs > Attachments: SOLR-10912.ok-patch-in-core.patch, > SOLR-10912.sample-patch.patch, SOLR-10912.solj-contrib-facet-error.patch > > > Proposing introduction of automated patch validation, similar what Hadoop or > other Apache projects are using (see link). This would ensure that every > patch passes a certain set of criterions before getting approved. It would > save time for developer (faster feedback loop), save time for committers > (less step to do manually), and would increase quality. > Hadoop is currently using Apache Yetus to run validations, which seems to be > a good direction to start. This jira could be the board of discussing the > preferred solution. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11417) Crashed leader's hanging emphemral will make restarting followers stuck in recovering
[ https://issues.apache.org/jira/browse/SOLR-11417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16189758#comment-16189758 ] Mano Kovacs commented on SOLR-11417: bq. I'm not sure its preferable to have a replica disable it's own ability to be a leader - kind of defeats the repeated attempts. Agreed. Going with that approach. > Crashed leader's hanging emphemral will make restarting followers stuck in > recovering > - > > Key: SOLR-11417 > URL: https://issues.apache.org/jira/browse/SOLR-11417 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.3 >Reporter: Mano Kovacs > Attachments: SOLR-11417.png > > > If replicas are starting up after leader crash and within the ZK session > timeout, replicas > * will lose leader election due to hanging ephemerals > * will read stale data from ZK about current leader > * will fail recovery and stuck in recovering state > If leader is down permanently (eg. hardware failure) and all replicas are > affected, shard will not come up (see also SOLR-7065). > Tested on 6.3. See attached image for details. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10912) Adding automatic patch validation
[
https://issues.apache.org/jira/browse/SOLR-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16189738#comment-16189738
]
Mano Kovacs commented on SOLR-10912:
As you can see from the last comment above, the minimal setup is getting ready.
Steps remaining:
* Multiple module test currently duplicates the failed test result, fixing
shortly.
* Add test to call {{-validate-source-patterns}}
* Finalize the runner script to setup Yetus, similarly to Hadoop.
* Verify the test-patch with Github PR
* Add Patch Available status for SOLR project (and update the script to look
for that). [~steve_rowe], regarding to our IRL conversation, could you help me
with adding that status in JIRA?
* Add {{Precommit-SOLR}} and {{Precommit-LUCENE}} jenkins jobs
Open questions:
# As you can see, a patch with two log entries had 6 (flaky) test failures.
Assuming flaky tests will not go away very soon, would it still be useful to
have this test-patch?
# I propose starting with the smallest set of tests (ie. affected modules) and
extend them later to dependent modules.
> Adding automatic patch validation
> -
>
> Key: SOLR-10912
> URL: https://issues.apache.org/jira/browse/SOLR-10912
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Build
>Reporter: Mano Kovacs
> Attachments: SOLR-10912.ok-patch-in-core.patch,
> SOLR-10912.sample-patch.patch, SOLR-10912.solj-contrib-facet-error.patch
>
>
> Proposing introduction of automated patch validation, similar what Hadoop or
> other Apache projects are using (see link). This would ensure that every
> patch passes a certain set of criterions before getting approved. It would
> save time for developer (faster feedback loop), save time for committers
> (less step to do manually), and would increase quality.
> Hadoop is currently using Apache Yetus to run validations, which seems to be
> a good direction to start. This jira could be the board of discussing the
> preferred solution.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-11431) Leader candidate cannot become leader if replica responds 500 to PeerSync
Mano Kovacs created SOLR-11431:
--
Summary: Leader candidate cannot become leader if replica responds
500 to PeerSync
Key: SOLR-11431
URL: https://issues.apache.org/jira/browse/SOLR-11431
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: 7.0
Reporter: Mano Kovacs
When leader candidate does PeerSync to all replicas, to download any missing
updates, it is tolerant to failures. It uses {{cantReachIsSuccess=true}} switch
which handles connection issue, 404 and 503 as success, since replicas being
DOWN should not affect the process.
However, if a replica has disk issues, the core initialization might fail and
that results in {{500}} instead of {{503}}. I failing replica like that can
prevent any other replicas becoming the leader.
Proposing either:
* Accepting {{500}} as "cant reach" so leader candidate can go on
or
* Changing {{SolrCoreInitializationException}} to return {{503}} instead of
{{500}}
* * this might be API change, however
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10912) Adding automatic patch validation
[
https://issues.apache.org/jira/browse/SOLR-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16189539#comment-16189539
]
Mano Kovacs commented on SOLR-10912:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
00s{color} | {color:green} The patch does not contain any @author tags. {color}
|
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
00s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} Check examples refer correct lucene
version {color} | {color:green} 0m 07s{color} | {color:green} the patch passed
{color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 0m 07s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 0m 07s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check licenses {color} | {color:green}
0m 07s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 62m 41s{color}
| {color:red} core in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 11m 07s{color}
| {color:red} solrj in the patch passed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 77m 37s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | solr.cloud.CdcrBootstrapTest |
| | solr.cloud.FullSolrCloudDistribCmdsTest |
| | solr.cloud.ReplaceNodeTest |
| | solr.cloud.TestCloudRecovery |
| | solr.cloud.TestLeaderElectionWithEmptyReplica |
| | solr.cloud.TestRebalanceLeaders |
| | solr.cloud.CdcrBootstrapTest |
| | solr.cloud.FullSolrCloudDistribCmdsTest |
| | solr.cloud.ReplaceNodeTest |
| | solr.cloud.TestCloudRecovery |
| | solr.cloud.TestLeaderElectionWithEmptyReplica |
| | solr.cloud.TestRebalanceLeaders |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-10912 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12889953/SOLR-10912.ok-patch-in-core.patch
|
| Optional Tests | javac unit |
| uname | Darwin manomac.local 16.7.0 Darwin Kernel Version 16.7.0: Thu Jun 15
17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64 x86_64 |
| Build tool | ant |
| Personality |
/Users/manokovacs/repos/solr-yetus/dev-tools/test-patch/solr-yetus-personality.sh
|
| git revision | master / fd2b4f3f86 |
| Default Java | 1.8.0_101 |
| unit | artifact/patchprocess/patch-unit-solr_core.txt |
| unit | artifact/patchprocess/patch-unit-solr_solrj.txt |
| modules | C: solr/core solr/solrj U: solr |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Adding automatic patch validation
> -
>
> Key: SOLR-10912
> URL: https://issues.apache.org/jira/browse/SOLR-10912
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Build
>Reporter: Mano Kovacs
> Attachments: SOLR-10912.ok-patch-in-core.patch,
> SOLR-10912.sample-patch.patch, SOLR-10912.solj-contrib-facet-error.patch
>
>
> Proposing introduction of automated patch validation, similar what Hadoop or
> other Apache projects are using (see link). This would ensure that every
> patch passes a certain set of criterions before getting approved. It would
> save time for developer (faster feedback loop), save time for committers
> (less step to do manually), and would increase quality.
> Hadoop is currently using Apache Yetus to run validations, which seems to be
> a good direction to start. This jira could be the board of discussing the
> preferred solution.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10912) Adding automatic patch validation
[
https://issues.apache.org/jira/browse/SOLR-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16189033#comment-16189033
]
Mano Kovacs commented on SOLR-10912:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
00s{color} | {color:green} The patch does not contain any @author tags. {color}
|
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
00s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} Check examples refer correct lucene
version {color} | {color:green} 0m 06s{color} | {color:green} the patch passed
{color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 0m 06s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 0m 06s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check licenses {color} | {color:green}
0m 06s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}255m 09s{color}
| {color:red} core in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 13m 11s{color}
| {color:red} solrj in the patch failed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}272m 02s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | solr.analysis.TestWordDelimiterFilterFactory |
| | solr.cloud.CdcrBootstrapTest |
| | solr.cloud.DeleteStatusTest |
| | solr.cloud.FullSolrCloudDistribCmdsTest |
| | solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest |
| | solr.cloud.LeaderElectionIntegrationTest |
| | solr.cloud.ShardRoutingTest |
| | solr.cloud.ShardSplitTest |
| | solr.cloud.SharedFSAutoReplicaFailoverUtilsTest |
| | solr.cloud.SyncSliceTest |
| | solr.cloud.TestHdfsCloudBackupRestore |
| | solr.cloud.TestLeaderElectionWithEmptyReplica |
| | solr.cloud.TestLocalFSCloudBackupRestore |
| | solr.cloud.TestTolerantUpdateProcessorCloud |
| | solr.cloud.UnloadDistributedZkTest |
| | solr.core.TestCorePropertiesReload |
| | solr.core.TestCustomStream |
| | solr.core.TestInfoStreamLogging |
| | solr.core.TestSimpleTextCodec |
| | solr.core.TestSolrDeletionPolicy1 |
| | solr.handler.loader.JavabinLoaderTest |
| | solr.handler.TestSolrConfigHandlerCloud |
| | solr.handler.V2StandaloneTest |
| | solr.handler.XsltUpdateRequestHandlerTest |
| | solr.highlight.TestPostingsSolrHighlighter |
| | solr.internal.csv.CharBufferTest |
| | solr.internal.csv.CSVUtilsTest |
| | solr.metrics.JvmMetricsTest |
| | solr.MinimalSchemaTest |
| | solr.schema.DateFieldTest |
| | solr.schema.MultiTermTest |
| | solr.schema.PolyFieldTest |
| | solr.schema.TestBinaryField |
| | solr.schema.TestBulkSchemaConcurrent |
| | solr.schema.TestManagedSchema |
| | solr.schema.UUIDFieldTest |
| | solr.search.AnalyticsMergeStrategyTest |
| | solr.search.function.distance.DistanceFunctionTest |
| | solr.search.similarities.TestPerFieldSimilarity |
| | solr.search.TestPayloadScoreQParserPlugin |
| | solr.search.TestSolrJ |
| | solr.servlet.DirectSolrConnectionTest |
| | solr.servlet.HttpSolrCallGetCoreTest |
| | solr.spelling.SpellingQueryConverterTest |
| | solr.spelling.suggest.SuggesterTest |
| | solr.spelling.suggest.TestHighFrequencyDictionaryFactory |
| | solr.spelling.suggest.TestPhraseSuggestions |
| | solr.TestHighlightDedupGrouping |
| | solr.util.CircularListTest |
| | solr.util.FileUtilsTest |
| | solr.util.PrimUtilsTest |
| | solr.util.TestUtils |
| | solr.client.solrj.io.stream.StreamExpressionTest |
| | solr.common.cloud.TestCollectionStateWatchers |
| | solr.analysis.TestWordDelimiterFilterFactory |
| | solr.cloud.CdcrBootstrapTest |
| | solr.cloud.DeleteStatusTest |
| | solr.cloud.FullSolrCloudDistribCmdsTest |
| | solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest |
| | solr.cloud.LeaderElectionIntegrationTest |
| | solr.cloud.ShardRoutingTest |
| | solr.cloud.ShardSplitTest |
| | solr.cloud.SharedFSAutoReplicaFailoverUtilsTest |
| | solr.cloud.SyncSliceTest |
| | solr.cloud.TestHdfsCloudBackupRestore |
| | solr.cloud.TestLeaderElectionWithEmptyReplica |
| | solr.cloud.TestLocalFSCloudBackupRestore |
| | solr.cloud.TestTolerantUpdateProcessorCloud |
| | solr.cloud.UnloadDistributedZkTest |
| | solr.core.TestCorePropertiesReload |
| | solr.core.TestCustomStrea
[jira] [Updated] (SOLR-10912) Adding automatic patch validation
[ https://issues.apache.org/jira/browse/SOLR-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mano Kovacs updated SOLR-10912: --- Attachment: SOLR-10912.ok-patch-in-core.patch > Adding automatic patch validation > - > > Key: SOLR-10912 > URL: https://issues.apache.org/jira/browse/SOLR-10912 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Build >Reporter: Mano Kovacs > Attachments: SOLR-10912.ok-patch-in-core.patch, > SOLR-10912.sample-patch.patch, SOLR-10912.solj-contrib-facet-error.patch > > > Proposing introduction of automated patch validation, similar what Hadoop or > other Apache projects are using (see link). This would ensure that every > patch passes a certain set of criterions before getting approved. It would > save time for developer (faster feedback loop), save time for committers > (less step to do manually), and would increase quality. > Hadoop is currently using Apache Yetus to run validations, which seems to be > a good direction to start. This jira could be the board of discussing the > preferred solution. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-11419) Validation command for missing docs in test-patch
Mano Kovacs created SOLR-11419: -- Summary: Validation command for missing docs in test-patch Key: SOLR-11419 URL: https://issues.apache.org/jira/browse/SOLR-11419 Project: Solr Issue Type: Improvement Security Level: Public (Default Security Level. Issues are Public) Components: Tests Reporter: Mano Kovacs Based on the discussion under SOLR-10912, patches should be validated if they include documentation changes. Additionally, the tool could suggest .adoc file to be updated, based on the changed files. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10912) Adding automatic patch validation
[ https://issues.apache.org/jira/browse/SOLR-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16185565#comment-16185565 ] Mano Kovacs commented on SOLR-10912: [~ctargett], [~cpoerschke], [~mdrob], thank you for the inputs. It clearly shows how much potential there is in this tool/method. I am trying to find the MVP for this to start, so with your permission, I would like to move the topic about documentation validation to SOLR-11419. Could you help me specify the details of the validation there? Meanwhile, I am trying to get this into a usable state so we can start some test-run with it. > Adding automatic patch validation > - > > Key: SOLR-10912 > URL: https://issues.apache.org/jira/browse/SOLR-10912 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Build >Reporter: Mano Kovacs > Attachments: SOLR-10912.sample-patch.patch, > SOLR-10912.solj-contrib-facet-error.patch > > > Proposing introduction of automated patch validation, similar what Hadoop or > other Apache projects are using (see link). This would ensure that every > patch passes a certain set of criterions before getting approved. It would > save time for developer (faster feedback loop), save time for committers > (less step to do manually), and would increase quality. > Hadoop is currently using Apache Yetus to run validations, which seems to be > a good direction to start. This jira could be the board of discussing the > preferred solution. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11417) Crashed leader's hanging emphemral will make restarting followers stuck in recovering
[
https://issues.apache.org/jira/browse/SOLR-11417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16185552#comment-16185552
]
Mano Kovacs commented on SOLR-11417:
Thank you [~markrmil...@gmail.com] for the followup.
Based on your comment, I see to things to change and either would prevent this
scenario:
bq. We can always read stale data, so we have to make sure still operate
correctly in the face of that.
The {{ZkController.getLeader()}} is currently double checks the leader, but
only in ZK. If we would also make sure if the leader confirms that he is the
leader, we could make it more robust.
bq. I guess the key question is, why are we stuck in recovering? What happens
so that we cannot be leader when the ephemeral goes away? Do we overwrite the
last active stat as not ACTIVE? In that case, it seems we should try and
prevent that when a replica simply has connection problems with the leader vs
failing a sync or something.
Yes, the last active state becomes RECOVERING, so it won't participate anymore.
We might want to restart the process if the failure is due to the leader.
I will evaluate both directions and prepare a patch with at least one.
> Crashed leader's hanging emphemral will make restarting followers stuck in
> recovering
> -
>
> Key: SOLR-11417
> URL: https://issues.apache.org/jira/browse/SOLR-11417
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 6.3
>Reporter: Mano Kovacs
> Attachments: SOLR-11417.png
>
>
> If replicas are starting up after leader crash and within the ZK session
> timeout, replicas
> * will lose leader election due to hanging ephemerals
> * will read stale data from ZK about current leader
> * will fail recovery and stuck in recovering state
> If leader is down permanently (eg. hardware failure) and all replicas are
> affected, shard will not come up (see also SOLR-7065).
> Tested on 6.3. See attached image for details.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11417) Crashed leader's hanging emphemral will make restarting followers stuck in recovering
[ https://issues.apache.org/jira/browse/SOLR-11417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mano Kovacs updated SOLR-11417: --- Attachment: SOLR-11417.png > Crashed leader's hanging emphemral will make restarting followers stuck in > recovering > - > > Key: SOLR-11417 > URL: https://issues.apache.org/jira/browse/SOLR-11417 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.3 >Reporter: Mano Kovacs > Attachments: SOLR-11417.png > > > If replicas are starting up after leader crash and within the ZK session > timeout, replicas > * will lose leader election due to hanging ephemerals > * will read stale data from ZK about current leader > * will fail recovery and stuck in recovering state > If leader is down permanently (eg. hardware failure) and all replicas are > affected, shard will not come up (see also SOLR-7065). > Tested on 6.3. See attached image for details. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-11417) Crashed leader's hanging emphemral will make restarting followers stuck in recovering
Mano Kovacs created SOLR-11417: -- Summary: Crashed leader's hanging emphemral will make restarting followers stuck in recovering Key: SOLR-11417 URL: https://issues.apache.org/jira/browse/SOLR-11417 Project: Solr Issue Type: Bug Security Level: Public (Default Security Level. Issues are Public) Affects Versions: 6.3 Reporter: Mano Kovacs If replicas are starting up after leader crash and within the ZK session timeout, replicas * will lose leader election due to hanging ephemerals * will read stale data from ZK about current leader * will fail recovery and stuck in recovering state If leader is down permanently (eg. hardware failure) and all replicas are affected, shard will not come up (see also SOLR-7065). Tested on 6.3. See attached image for details. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10783) Using Hadoop Credential Provider as SSL/TLS store password source
[
https://issues.apache.org/jira/browse/SOLR-10783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16171760#comment-16171760
]
Mano Kovacs commented on SOLR-10783:
Failing tests:
{noformat}
[junit4] Tests with failures [seed: 4CBAA3E7D8BB42C]:
[junit4] - org.apache.solr.update.TestInPlaceUpdatesDistrib.test
[junit4] - org.apache.solr.core.TestDynamicLoading.testDynamicLoading
[junit4] - org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test
[junit4] -
org.apache.solr.cloud.CdcrBootstrapTest.testBootstrapWithSourceCluster
[junit4] -
org.apache.solr.cloud.CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap
[junit4] - org.apache.solr.cloud.ReplaceNodeTest.test
[junit4] - org.apache.solr.cloud.TestLeaderElectionWithEmptyReplica.test
{noformat}
> Using Hadoop Credential Provider as SSL/TLS store password source
> -
>
> Key: SOLR-10783
> URL: https://issues.apache.org/jira/browse/SOLR-10783
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: security
>Reporter: Mano Kovacs
>Assignee: Mark Miller
> Attachments: SOLR-10783-fix.patch, SOLR-10783.patch,
> SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch
>
>
> As a second iteration of SOLR-10307, I propose support of hadoop credential
> providers as source of SSL store passwords.
> Motivation: When SOLR is used in hadoop environment, support of HCP gives
> better integration and unified method to pass sensitive credentials to SOLR.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10783) Using Hadoop Credential Provider as SSL/TLS store password source
[
https://issues.apache.org/jira/browse/SOLR-10783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16171657#comment-16171657
]
Mano Kovacs commented on SOLR-10783:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
00s{color} | {color:green} The patch does not contain any @author tags. {color}
|
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
00s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} Check examples refer correct lucene
version {color} | {color:green} 0m 05s{color} | {color:green} the patch passed
{color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 0m 05s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 0m 05s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check licenses {color} | {color:green}
0m 05s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 62m 07s{color}
| {color:red} core in the patch failed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 65m 25s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-10783 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12885968/SOLR-10783.patch |
| Optional Tests | javac unit |
| uname | Darwin manomac.local 16.7.0 Darwin Kernel Version 16.7.0: Thu Jun 15
17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64 x86_64 |
| Build tool | ant |
| Personality |
/Users/manokovacs/repos/solr-yetus/dev-tools/test-patch/solr-yetus-personality.sh
|
| git revision | master / bc95209774 |
| Default Java | 1.8.0_101 |
| unit | artifact/patchprocess/patch-unit-solr_core.txt |
| modules | C: solr solr/core solr/server U: solr |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Using Hadoop Credential Provider as SSL/TLS store password source
> -
>
> Key: SOLR-10783
> URL: https://issues.apache.org/jira/browse/SOLR-10783
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: security
>Reporter: Mano Kovacs
>Assignee: Mark Miller
> Attachments: SOLR-10783-fix.patch, SOLR-10783.patch,
> SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch
>
>
> As a second iteration of SOLR-10307, I propose support of hadoop credential
> providers as source of SSL store passwords.
> Motivation: When SOLR is used in hadoop environment, support of HCP gives
> better integration and unified method to pass sensitive credentials to SOLR.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11275) Adding diagrams for AutoAddReplica into Solr Ref Guide
[ https://issues.apache.org/jira/browse/SOLR-11275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mano Kovacs updated SOLR-11275: --- Attachment: autoaddreplica.puml > Adding diagrams for AutoAddReplica into Solr Ref Guide > -- > > Key: SOLR-11275 > URL: https://issues.apache.org/jira/browse/SOLR-11275 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: documentation >Reporter: Mano Kovacs >Assignee: Cassandra Targett > Attachments: autoaddreplica.png, autoaddreplica.puml, > autoaddreplica.puml, plantuml-diagram-test.png > > > Pilot jira for adding PlantUML diagrams for documenting internals. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11275) Adding diagrams for AutoAddReplica into Solr Ref Guide
[ https://issues.apache.org/jira/browse/SOLR-11275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170781#comment-16170781 ] Mano Kovacs commented on SOLR-11275: [~ctargett], that is very good news! Looks great. [~varunthacker], sure, I'll upload a new puml. > Adding diagrams for AutoAddReplica into Solr Ref Guide > -- > > Key: SOLR-11275 > URL: https://issues.apache.org/jira/browse/SOLR-11275 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: documentation >Reporter: Mano Kovacs >Assignee: Cassandra Targett > Attachments: autoaddreplica.png, autoaddreplica.puml, > plantuml-diagram-test.png > > > Pilot jira for adding PlantUML diagrams for documenting internals. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10912) Adding automatic patch validation
[ https://issues.apache.org/jira/browse/SOLR-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16162164#comment-16162164 ] Mano Kovacs commented on SOLR-10912: The latest test-patch only tested the three module was changed (either my intentional error was not placed correctly, or we need more tests there). bq. A change in solr-core shouldn't need to re-run the solrj unit tests, but a solrj change would benefit from running the solr unit tests. Yeah, solr depends on solrj, so the direction is correct. It is matter of view to say that a module's API should be validated through it's unittests or the tests of other modules depending on it (in this case solr->solrj). But for start, if that won't cause too much load on the jenkins, we can tests rather more than less. Could we define all the transitive triggers for unittest between modules? * solrj->solr-core * lucene->solr? Any other? > Adding automatic patch validation > - > > Key: SOLR-10912 > URL: https://issues.apache.org/jira/browse/SOLR-10912 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Build >Reporter: Mano Kovacs > Attachments: SOLR-10912.sample-patch.patch, > SOLR-10912.solj-contrib-facet-error.patch > > > Proposing introduction of automated patch validation, similar what Hadoop or > other Apache projects are using (see link). This would ensure that every > patch passes a certain set of criterions before getting approved. It would > save time for developer (faster feedback loop), save time for committers > (less step to do manually), and would increase quality. > Hadoop is currently using Apache Yetus to run validations, which seems to be > a good direction to start. This jira could be the board of discussing the > preferred solution. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10912) Adding automatic patch validation
[
https://issues.apache.org/jira/browse/SOLR-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16162155#comment-16162155
]
Mano Kovacs commented on SOLR-10912:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
00s{color} | {color:green} The patch does not contain any @author tags. {color}
|
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
00s{color} | {color:red} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} Check examples refer correct lucene
version {color} | {color:green} 0m 05s{color} | {color:green} the patch passed
{color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 0m 05s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 0m 05s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check licenses {color} | {color:green}
0m 05s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
24s{color} | {color:green} facet in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 2m 46s{color}
| {color:red} analytics in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 4m 25s{color}
| {color:red} solrj in the patch failed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 14s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-10912 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12886505/SOLR-10912.solj-contrib-facet-error.patch
|
| Optional Tests | javac unit |
| uname | Darwin MunawAir.local 15.6.0 Darwin Kernel Version 15.6.0: Fri Feb 17
10:21:18 PST 2017; root:xnu-3248.60.11.4.1~1/RELEASE_X86_64 x86_64 |
| Build tool | ant |
| Personality |
/Users/munaw/repos/lucene-solr/dev-tools/test-patch/solr-yetus-personality.sh |
| git revision | master / 708ed8f |
| Default Java | 1.8.0_45 |
| unit | artifact/patchprocess/patch-unit-solr_contrib_analytics.txt |
| unit | artifact/patchprocess/patch-unit-solr_solrj.txt |
| modules | C: lucene/facet solr/contrib/analytics solr/solrj U: . |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Adding automatic patch validation
> -
>
> Key: SOLR-10912
> URL: https://issues.apache.org/jira/browse/SOLR-10912
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Build
>Reporter: Mano Kovacs
> Attachments: SOLR-10912.sample-patch.patch,
> SOLR-10912.solj-contrib-facet-error.patch
>
>
> Proposing introduction of automated patch validation, similar what Hadoop or
> other Apache projects are using (see link). This would ensure that every
> patch passes a certain set of criterions before getting approved. It would
> save time for developer (faster feedback loop), save time for committers
> (less step to do manually), and would increase quality.
> Hadoop is currently using Apache Yetus to run validations, which seems to be
> a good direction to start. This jira could be the board of discussing the
> preferred solution.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10912) Adding automatic patch validation
[ https://issues.apache.org/jira/browse/SOLR-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mano Kovacs updated SOLR-10912: --- Attachment: SOLR-10912.solj-contrib-facet-error.patch Uploading a new patch with solj, contrib module, and lucene module changes to verify unittests in changed modules only. > Adding automatic patch validation > - > > Key: SOLR-10912 > URL: https://issues.apache.org/jira/browse/SOLR-10912 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Build >Reporter: Mano Kovacs > Attachments: SOLR-10912.sample-patch.patch, > SOLR-10912.solj-contrib-facet-error.patch > > > Proposing introduction of automated patch validation, similar what Hadoop or > other Apache projects are using (see link). This would ensure that every > patch passes a certain set of criterions before getting approved. It would > save time for developer (faster feedback loop), save time for committers > (less step to do manually), and would increase quality. > Hadoop is currently using Apache Yetus to run validations, which seems to be > a good direction to start. This jira could be the board of discussing the > preferred solution. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10912) Adding automatic patch validation
[ https://issues.apache.org/jira/browse/SOLR-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16162042#comment-16162042 ] Mano Kovacs commented on SOLR-10912: Yeah, that would make sense. Do you think it would be enough just to test the module changed? > Adding automatic patch validation > - > > Key: SOLR-10912 > URL: https://issues.apache.org/jira/browse/SOLR-10912 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Build >Reporter: Mano Kovacs > Attachments: SOLR-10912.sample-patch.patch > > > Proposing introduction of automated patch validation, similar what Hadoop or > other Apache projects are using (see link). This would ensure that every > patch passes a certain set of criterions before getting approved. It would > save time for developer (faster feedback loop), save time for committers > (less step to do manually), and would increase quality. > Hadoop is currently using Apache Yetus to run validations, which seems to be > a good direction to start. This jira could be the board of discussing the > preferred solution. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10783) Using Hadoop Credential Provider as SSL/TLS store password source
[
https://issues.apache.org/jira/browse/SOLR-10783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16162038#comment-16162038
]
Mano Kovacs commented on SOLR-10783:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
00s{color} | {color:green} The patch does not contain any @author tags. {color}
|
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
00s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} Check examples refer correct lucene
version {color} | {color:green} 0m 06s{color} | {color:green} the patch passed
{color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 0m 06s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 0m 06s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check licenses {color} | {color:green}
0m 06s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 39m 25s{color}
| {color:red} solr in the patch failed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 43m 29s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | solr.cloud.CdcrVersionReplicationTest |
| | solr.cloud.DistribCursorPagingTest |
| | solr.cloud.hdfs.StressHdfsTest |
| | solr.cloud.HealthCheckHandlerTest |
| | solr.cloud.MultiThreadedOCPTest |
| | solr.cloud.RollingRestartTest |
| | solr.cloud.ShardSplitTest |
| | solr.cloud.TestCollectionAPI |
| | solr.cloud.TestLeaderElectionWithEmptyReplica |
| | solr.cloud.TestSegmentSorting |
| | solr.cloud.TestShortCircuitedRequests |
| | solr.cloud.TestTlogReplica |
| | solr.schema.TestCloudSchemaless |
| | solr.search.stats.TestDistribIDF |
| | solr.update.processor.TestNamedUpdateProcessors |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-10783 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12885968/SOLR-10783.patch |
| Optional Tests | javac unit |
| uname | Darwin MunawAir.local 15.6.0 Darwin Kernel Version 15.6.0: Fri Feb 17
10:21:18 PST 2017; root:xnu-3248.60.11.4.1~1/RELEASE_X86_64 x86_64 |
| Build tool | ant |
| Personality |
/Users/munaw/repos/lucene-solr/dev-tools/test-patch/solr-yetus-personality.sh |
| git revision | master / 708ed8f |
| Default Java | 1.8.0_45 |
| unit | artifact/patchprocess/patch-unit-._solr.txt |
| modules | C: solr solr/core solr/server U: solr |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Using Hadoop Credential Provider as SSL/TLS store password source
> -
>
> Key: SOLR-10783
> URL: https://issues.apache.org/jira/browse/SOLR-10783
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: security
>Reporter: Mano Kovacs
>Assignee: Mark Miller
> Attachments: SOLR-10783-fix.patch, SOLR-10783.patch,
> SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch
>
>
> As a second iteration of SOLR-10307, I propose support of hadoop credential
> providers as source of SSL store passwords.
> Motivation: When SOLR is used in hadoop environment, support of HCP gives
> better integration and unified method to pass sensitive credentials to SOLR.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10912) Adding automatic patch validation
[
https://issues.apache.org/jira/browse/SOLR-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16161950#comment-16161950
]
Mano Kovacs commented on SOLR-10912:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
00s{color} | {color:green} The patch does not contain any @author tags. {color}
|
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
00s{color} | {color:red} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} Check examples refer correct lucene
version {color} | {color:green} 0m 06s{color} | {color:green} the patch passed
{color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 0m 06s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 0m 06s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check licenses {color} | {color:green}
0m 06s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 9m 08s{color}
| {color:red} solr in the patch failed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 55s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | solr.cloud.AliasIntegrationTest |
| | solr.cloud.ChaosMonkeyNothingIsSafeWithPullReplicasTest |
| | solr.cloud.FullSolrCloudDistribCmdsTest |
| | solr.cloud.MoveReplicaTest |
| | solr.cloud.MultiThreadedOCPTest |
| | solr.cloud.OverseerRolesTest |
| | solr.cloud.RemoteQueryErrorTest |
| | solr.cloud.ShardRoutingTest |
| | solr.cloud.ShardSplitTest |
| | solr.cloud.TestDeleteCollectionOnDownNodes |
| | solr.cloud.TestLeaderElectionWithEmptyReplica |
| | solr.cloud.TestMiniSolrCloudClusterSSL |
| | solr.cloud.TestSolrCloudWithDelegationTokens |
| | solr.cloud.TestStressCloudBlindAtomicUpdates |
| | solr.cloud.TestTolerantUpdateProcessorRandomCloud |
| | solr.cloud.UnloadDistributedZkTest |
| | solr.core.snapshots.TestSolrCoreSnapshots |
| | solr.DistributedIntervalFacetingTest |
| | solr.handler.PingRequestHandlerTest |
| | solr.metrics.reporters.solr.SolrShardReporterTest |
| | solr.search.facet.TestJsonFacetRefinement |
| | solr.search.join.BlockJoinFacetDistribTest |
| | solr.search.join.TestCloudNestedDocsSort |
| | solr.search.mlt.CloudMLTQParserTest |
| | solr.TestDistributedGrouping |
| | solr.TestDistributedMissingSort |
| | solr.TestDistributedSearch |
| | solr.TestTolerantSearch |
| | solr.update.processor.TemplateUpdateProcessorTest |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-10912 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12885918/SOLR-10912.sample-patch.patch
|
| Optional Tests | javac unit |
| uname | Darwin MunawAir.local 15.6.0 Darwin Kernel Version 15.6.0: Fri Feb 17
10:21:18 PST 2017; root:xnu-3248.60.11.4.1~1/RELEASE_X86_64 x86_64 |
| Build tool | ant |
| Personality |
/Users/munaw/repos/lucene-solr/dev-tools/test-patch/solr-yetus-personality.sh |
| git revision | master / 708ed8f |
| Default Java | 1.8.0_45 |
| unit | artifact/patchprocess/patch-unit-._solr.txt |
| modules | C: solr/solrj U: solr/solrj |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Adding automatic patch validation
> -
>
> Key: SOLR-10912
> URL: https://issues.apache.org/jira/browse/SOLR-10912
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Build
>Reporter: Mano Kovacs
> Attachments: SOLR-10912.sample-patch.patch
>
>
> Proposing introduction of automated patch validation, similar what Hadoop or
> other Apache projects are using (see link). This would ensure that every
> patch passes a certain set of criterions before getting approved. It would
> save time for developer (faster feedback loop), save time for committers
> (less step to do manually), and would increase quality.
> Hadoop is currently using Apache Yetus to run validations, which seems to be
> a good direction to start. This jira could be the board of discussing the
> preferred solution.
--
This message was sent by At
[jira] [Commented] (SOLR-10912) Adding automatic patch validation
[
https://issues.apache.org/jira/browse/SOLR-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16161786#comment-16161786
]
Mano Kovacs commented on SOLR-10912:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
00s{color} | {color:green} The patch does not contain any @author tags. {color}
|
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
00s{color} | {color:red} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} Check examples refer correct lucene
version {color} | {color:green} 0m 04s{color} | {color:green} the patch passed
{color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 0m 04s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 0m 04s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check licenses {color} | {color:green}
0m 04s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 3m 47s{color}
| {color:red} solr in the patch failed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 7m 01s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-10912 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12885918/SOLR-10912.sample-patch.patch
|
| Optional Tests | javac unit |
| uname | Darwin MunawAir.local 15.6.0 Darwin Kernel Version 15.6.0: Fri Feb 17
10:21:18 PST 2017; root:xnu-3248.60.11.4.1~1/RELEASE_X86_64 x86_64 |
| Build tool | ant |
| Personality |
/Users/munaw/repos/lucene-solr/dev-tools/test-patch/solr-yetus-personality.sh |
| git revision | master / 31eab31 |
| Default Java | 1.8.0_45 |
| unit | artifact/patchprocess/patch-unit-._solr.txt |
| modules | C: solr/solrj U: solr/solrj |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Adding automatic patch validation
> -
>
> Key: SOLR-10912
> URL: https://issues.apache.org/jira/browse/SOLR-10912
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Build
>Reporter: Mano Kovacs
> Attachments: SOLR-10912.sample-patch.patch
>
>
> Proposing introduction of automated patch validation, similar what Hadoop or
> other Apache projects are using (see link). This would ensure that every
> patch passes a certain set of criterions before getting approved. It would
> save time for developer (faster feedback loop), save time for committers
> (less step to do manually), and would increase quality.
> Hadoop is currently using Apache Yetus to run validations, which seems to be
> a good direction to start. This jira could be the board of discussing the
> preferred solution.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10912) Adding automatic patch validation
[ https://issues.apache.org/jira/browse/SOLR-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16161753#comment-16161753 ] Mano Kovacs commented on SOLR-10912: [~mdrob], thanks for the followup :) Well, there is no automated integration yet. I just wanted to have a test-run, but eventually I used SOLR-10783 for that. When I am finished, I upload and run a test-patch for each scenario to demonstrate. > Adding automatic patch validation > - > > Key: SOLR-10912 > URL: https://issues.apache.org/jira/browse/SOLR-10912 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Build >Reporter: Mano Kovacs > Attachments: SOLR-10912.sample-patch.patch > > > Proposing introduction of automated patch validation, similar what Hadoop or > other Apache projects are using (see link). This would ensure that every > patch passes a certain set of criterions before getting approved. It would > save time for developer (faster feedback loop), save time for committers > (less step to do manually), and would increase quality. > Hadoop is currently using Apache Yetus to run validations, which seems to be > a good direction to start. This jira could be the board of discussing the > preferred solution. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10783) Using Hadoop Credential Provider as SSL/TLS store password source
[
https://issues.apache.org/jira/browse/SOLR-10783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16157999#comment-16157999
]
Mano Kovacs commented on SOLR-10783:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
00s{color} | {color:green} The patch does not contain any @author tags. {color}
|
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
00s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 60m
54s{color} | {color:green} solr in the patch passed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
08s{color} | {color:red} solr in the patch failed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 61m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-10783 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12885968/SOLR-10783.patch |
| Optional Tests | asflicense javac unit |
| uname | Darwin MunawAir.local 15.6.0 Darwin Kernel Version 15.6.0: Fri Feb 17
10:21:18 PST 2017; root:xnu-3248.60.11.4.1~1/RELEASE_X86_64 x86_64 |
| Build tool | ant |
| Personality |
/Users/munaw/repos/lucene-solr/dev-tools/test-patch/solr-yetus-personality.sh |
| git revision | master / ce29124 |
| Default Java | 1.8.0_45 |
| asflicense | artifact/patchprocess/patch-asflicense-._solr.txt |
| modules | C: solr solr/core solr/server U: solr |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Using Hadoop Credential Provider as SSL/TLS store password source
> -
>
> Key: SOLR-10783
> URL: https://issues.apache.org/jira/browse/SOLR-10783
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: security
>Reporter: Mano Kovacs
>Assignee: Mark Miller
> Attachments: SOLR-10783-fix.patch, SOLR-10783.patch,
> SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch
>
>
> As a second iteration of SOLR-10307, I propose support of hadoop credential
> providers as source of SSL store passwords.
> Motivation: When SOLR is used in hadoop environment, support of HCP gives
> better integration and unified method to pass sensitive credentials to SOLR.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10783) Using Hadoop Credential Provider as SSL/TLS store password source
[ https://issues.apache.org/jira/browse/SOLR-10783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mano Kovacs updated SOLR-10783: --- Attachment: SOLR-10783.patch Attaching new patch, merged the three older plus removing default value. > Using Hadoop Credential Provider as SSL/TLS store password source > - > > Key: SOLR-10783 > URL: https://issues.apache.org/jira/browse/SOLR-10783 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: security >Reporter: Mano Kovacs >Assignee: Mark Miller > Attachments: SOLR-10783-fix.patch, SOLR-10783.patch, > SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch > > > As a second iteration of SOLR-10307, I propose support of hadoop credential > providers as source of SSL store passwords. > Motivation: When SOLR is used in hadoop environment, support of HCP gives > better integration and unified method to pass sensitive credentials to SOLR. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10912) Adding automatic patch validation
[ https://issues.apache.org/jira/browse/SOLR-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mano Kovacs updated SOLR-10912: --- Attachment: SOLR-10912.sample-patch.patch > Adding automatic patch validation > - > > Key: SOLR-10912 > URL: https://issues.apache.org/jira/browse/SOLR-10912 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Build >Reporter: Mano Kovacs > Attachments: SOLR-10912.sample-patch.patch > > > Proposing introduction of automated patch validation, similar what Hadoop or > other Apache projects are using (see link). This would ensure that every > patch passes a certain set of criterions before getting approved. It would > save time for developer (faster feedback loop), save time for committers > (less step to do manually), and would increase quality. > Hadoop is currently using Apache Yetus to run validations, which seems to be > a good direction to start. This jira could be the board of discussing the > preferred solution. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10783) Using Hadoop Credential Provider as SSL/TLS store password source
[
https://issues.apache.org/jira/browse/SOLR-10783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16157317#comment-16157317
]
Mano Kovacs commented on SOLR-10783:
[~hossman], I agree, its a small change in size but affects the
{{jetty-ssl.xml}}, therefore most of the tests. Would it make sense to roll
back from 7x and have the fix on master?
> Using Hadoop Credential Provider as SSL/TLS store password source
> -
>
> Key: SOLR-10783
> URL: https://issues.apache.org/jira/browse/SOLR-10783
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: security
>Reporter: Mano Kovacs
>Assignee: Mark Miller
> Attachments: SOLR-10783-fix.patch, SOLR-10783.patch,
> SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch
>
>
> As a second iteration of SOLR-10307, I propose support of hadoop credential
> providers as source of SSL store passwords.
> Motivation: When SOLR is used in hadoop environment, support of HCP gives
> better integration and unified method to pass sensitive credentials to SOLR.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10783) Using Hadoop Credential Provider as SSL/TLS store password source
[ https://issues.apache.org/jira/browse/SOLR-10783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mano Kovacs updated SOLR-10783: --- Attachment: SOLR-10783-fix.patch Attaching fix with the missing constructor. Run solr test suite. > Using Hadoop Credential Provider as SSL/TLS store password source > - > > Key: SOLR-10783 > URL: https://issues.apache.org/jira/browse/SOLR-10783 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: security >Reporter: Mano Kovacs >Assignee: Mark Miller > Attachments: SOLR-10783-fix.patch, SOLR-10783.patch, > SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch > > > As a second iteration of SOLR-10307, I propose support of hadoop credential > providers as source of SSL store passwords. > Motivation: When SOLR is used in hadoop environment, support of HCP gives > better integration and unified method to pass sensitive credentials to SOLR. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10783) Using Hadoop Credential Provider as SSL/TLS store password source
[ https://issues.apache.org/jira/browse/SOLR-10783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16157008#comment-16157008 ] Mano Kovacs commented on SOLR-10783: Sorry, fixing. > Using Hadoop Credential Provider as SSL/TLS store password source > - > > Key: SOLR-10783 > URL: https://issues.apache.org/jira/browse/SOLR-10783 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: security >Reporter: Mano Kovacs >Assignee: Mark Miller > Attachments: SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch, > SOLR-10783.patch > > > As a second iteration of SOLR-10307, I propose support of hadoop credential > providers as source of SSL store passwords. > Motivation: When SOLR is used in hadoop environment, support of HCP gives > better integration and unified method to pass sensitive credentials to SOLR. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10783) Using Hadoop Credential Provider as SSL/TLS store password source
[ https://issues.apache.org/jira/browse/SOLR-10783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mano Kovacs updated SOLR-10783: --- Attachment: SOLR-10783.patch PATCH: Minor cleanup > Using Hadoop Credential Provider as SSL/TLS store password source > - > > Key: SOLR-10783 > URL: https://issues.apache.org/jira/browse/SOLR-10783 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: security >Reporter: Mano Kovacs >Assignee: Mark Miller > Attachments: SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch, > SOLR-10783.patch > > > As a second iteration of SOLR-10307, I propose support of hadoop credential > providers as source of SSL store passwords. > Motivation: When SOLR is used in hadoop environment, support of HCP gives > better integration and unified method to pass sensitive credentials to SOLR. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10783) Using Hadoop Credential Provider as SSL/TLS store password source
[
https://issues.apache.org/jira/browse/SOLR-10783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mano Kovacs updated SOLR-10783:
---
Attachment: SOLR-10783.patch
[~markrmil...@gmail.com], I refactored to separate credential providers.
* Created {{SSLCredentialProvider}} interface.
* Implemented env, sysprop, and hadoop-based providers.
* Supporting system property to configure custom credential provider
implementation.
Could you please review the patch?
> Using Hadoop Credential Provider as SSL/TLS store password source
> -
>
> Key: SOLR-10783
> URL: https://issues.apache.org/jira/browse/SOLR-10783
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: security
>Reporter: Mano Kovacs
>Assignee: Mark Miller
> Attachments: SOLR-10783.patch, SOLR-10783.patch, SOLR-10783.patch
>
>
> As a second iteration of SOLR-10307, I propose support of hadoop credential
> providers as source of SSL store passwords.
> Motivation: When SOLR is used in hadoop environment, support of HCP gives
> better integration and unified method to pass sensitive credentials to SOLR.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7942) For Geo3d paths, aggregating distance values using "+" is not adequate for squared distances
[
https://issues.apache.org/jira/browse/LUCENE-7942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145252#comment-16145252
]
Mano Kovacs commented on LUCENE-7942:
-
[~kwri...@metacarta.com], there is a validation issue in the patch at
GeoStandardPath.java#811:
{noformat}
-validate-source-patterns:
[source-patterns] tabs instead spaces:
lucene/spatial3d/src/java/org/apache/lucene/spatial3d/geom/GeoStandardPath.java
BUILD FAILED
{noformat}
Could you please replace it? (I don't have write access)
> For Geo3d paths, aggregating distance values using "+" is not adequate for
> squared distances
>
>
> Key: LUCENE-7942
> URL: https://issues.apache.org/jira/browse/LUCENE-7942
> Project: Lucene - Core
> Issue Type: Bug
> Components: modules/spatial3d
>Reporter: Karl Wright
>Assignee: Karl Wright
> Fix For: 6.7, master (8.0), 7.1
>
>
> The GeoStandardPath object aggregates distances segment by segment using
> simple addition. For some kinds of Distance computations, though, addition
> is not an adequate way to do this. The xxxSquaredDistance computations, for
> example, do not produce true squared distances but rather a distance metric
> that is a combination of both squared and linear.
> I propose adding support in Distance for aggregation, which would allow
> distance calculators to compute an accurate distance (at some computational
> cost) instead.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11275) Adding diagrams for AutoAddReplica into Solr Ref Guide
[ https://issues.apache.org/jira/browse/SOLR-11275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mano Kovacs updated SOLR-11275: --- Summary: Adding diagrams for AutoAddReplica into Solr Ref Guide (was: Adding diagrams for AutoAddReplica into the documentation) > Adding diagrams for AutoAddReplica into Solr Ref Guide > -- > > Key: SOLR-11275 > URL: https://issues.apache.org/jira/browse/SOLR-11275 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: documentation >Reporter: Mano Kovacs > Attachments: autoaddreplica.png, autoaddreplica.puml > > > Pilot jira for adding PlantUML diagrams for documenting internals. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-11275) Adding diagrams for AutoAddReplica into the documentation
Mano Kovacs created SOLR-11275: -- Summary: Adding diagrams for AutoAddReplica into the documentation Key: SOLR-11275 URL: https://issues.apache.org/jira/browse/SOLR-11275 Project: Solr Issue Type: Improvement Security Level: Public (Default Security Level. Issues are Public) Components: documentation Reporter: Mano Kovacs Attachments: autoaddreplica.png, autoaddreplica.puml Pilot jira for adding PlantUML diagrams for documenting internals. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11275) Adding diagrams for AutoAddReplica into the documentation
[ https://issues.apache.org/jira/browse/SOLR-11275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mano Kovacs updated SOLR-11275: --- Attachment: autoaddreplica.png autoaddreplica.puml > Adding diagrams for AutoAddReplica into the documentation > - > > Key: SOLR-11275 > URL: https://issues.apache.org/jira/browse/SOLR-11275 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: documentation >Reporter: Mano Kovacs > Attachments: autoaddreplica.png, autoaddreplica.puml > > > Pilot jira for adding PlantUML diagrams for documenting internals. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-9555) Leader incorrectly publishes state for replica when it puts replica into LIR.
[ https://issues.apache.org/jira/browse/SOLR-9555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mano Kovacs updated SOLR-9555: -- Attachment: lir-9555-problem.png Attaching explanation diagram of the first problem. > Leader incorrectly publishes state for replica when it puts replica into LIR. > - > > Key: SOLR-9555 > URL: https://issues.apache.org/jira/browse/SOLR-9555 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Alan Woodward > Attachments: lir-9555-problem.png, SOLR-9555.patch, SOLR-9555.patch, > SOLR-9555.patch, SOLR-9555-WIP-2.patch, SOLR-9555-WIP-3.patch, > SOLR-9555-WIP.patch > > > See > https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17888/consoleFull > for an example -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10783) Using Hadoop Credential Provider as SSL/TLS store password source
[ https://issues.apache.org/jira/browse/SOLR-10783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16076542#comment-16076542 ] Mano Kovacs commented on SOLR-10783: bq. I think we'd like the ability to use a credential provider in Alfresco/Solr, but we don't use Hadoop. I think you don't need to use Hadoop to use the credential provider API. bq. Is there a way to rework this so that we have a HadoopSSLConfiguration subclass that is used rather than special casing SSLConfigurations for different providers? Sure, I will refactor the patch. > Using Hadoop Credential Provider as SSL/TLS store password source > - > > Key: SOLR-10783 > URL: https://issues.apache.org/jira/browse/SOLR-10783 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: security >Reporter: Mano Kovacs >Assignee: Mark Miller > Attachments: SOLR-10783.patch, SOLR-10783.patch > > > As a second iteration of SOLR-10307, I propose support of hadoop credential > providers as source of SSL store passwords. > Motivation: When SOLR is used in hadoop environment, support of HCP gives > better integration and unified method to pass sensitive credentials to SOLR. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9910) Allow setting of additional jetty options in bin/solr and bin/solr.cmd
[
https://issues.apache.org/jira/browse/SOLR-9910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16068129#comment-16068129
]
Mano Kovacs commented on SOLR-9910:
---
bq. So if you cannot count on it fully except at a moment in time, it's a
perhaps pretty flakey feature to build anything on.
Should we revert the patch?
> Allow setting of additional jetty options in bin/solr and bin/solr.cmd
> --
>
> Key: SOLR-9910
> URL: https://issues.apache.org/jira/browse/SOLR-9910
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mano Kovacs
>Assignee: Mark Miller
> Fix For: master (7.0), 6.7
>
> Attachments: SOLR-9910.patch, SOLR-9910.patch
>
>
> Command line tools allow the option {{-a}} to add JVM options to start
> command. Proposing to add {{-j}} option to add additional config for jetty
> (the part after {{start.jar}}).
> Motivation: jetty can be configured with start.ini in server directory.
> Running multiple Solr instances, however, requires the configuration per
> instance that cannot share the start.ini with other instances.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10783) Using Hadoop Credential Provider as SSL/TLS store password source
[ https://issues.apache.org/jira/browse/SOLR-10783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mano Kovacs updated SOLR-10783: --- Attachment: SOLR-10783.patch > Using Hadoop Credential Provider as SSL/TLS store password source > - > > Key: SOLR-10783 > URL: https://issues.apache.org/jira/browse/SOLR-10783 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: security >Reporter: Mano Kovacs >Assignee: Mark Miller > Attachments: SOLR-10783.patch, SOLR-10783.patch > > > As a second iteration of SOLR-10307, I propose support of hadoop credential > providers as source of SSL store passwords. > Motivation: When SOLR is used in hadoop environment, support of HCP gives > better integration and unified method to pass sensitive credentials to SOLR. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10783) Using Hadoop Credential Provider as SSL/TLS store password source
[
https://issues.apache.org/jira/browse/SOLR-10783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16061184#comment-16061184
]
Mano Kovacs commented on SOLR-10783:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 05s{color}
| {color:red} SOLR-10783 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.4.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-10783 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12871693/SOLR-10783.patch |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Using Hadoop Credential Provider as SSL/TLS store password source
> -
>
> Key: SOLR-10783
> URL: https://issues.apache.org/jira/browse/SOLR-10783
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: security
>Reporter: Mano Kovacs
>Assignee: Mark Miller
> Attachments: SOLR-10783.patch
>
>
> As a second iteration of SOLR-10307, I propose support of hadoop credential
> providers as source of SSL store passwords.
> Motivation: When SOLR is used in hadoop environment, support of HCP gives
> better integration and unified method to pass sensitive credentials to SOLR.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10307) Provide SSL/TLS keystore password a more secure way
[ https://issues.apache.org/jira/browse/SOLR-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16059721#comment-16059721 ] Mano Kovacs commented on SOLR-10307: [~markrmil...@gmail.com], could you please take a look at the fix patch? The issue what [~michaelsuzuki] pointed out probably will affect others. > Provide SSL/TLS keystore password a more secure way > --- > > Key: SOLR-10307 > URL: https://issues.apache.org/jira/browse/SOLR-10307 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: security >Reporter: Mano Kovacs >Assignee: Mark Miller > Fix For: master (7.0), 6.7 > > Attachments: SOLR-10307.2.patch, SOLR-10307.patch, SOLR-10307.patch, > SOLR-10307.patch > > > Currently the only way to pass server and client side SSL keytstore and > truststore passwords is to set specific environment variables that will be > passed as system properties, through command line parameter. > First option is to pass passwords through environment variables which gives a > better level of protection. Second option would be to use hadoop credential > provider interface to access credential store. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10307) Provide SSL/TLS keystore password a more secure way
[
https://issues.apache.org/jira/browse/SOLR-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mano Kovacs updated SOLR-10307:
---
Attachment: SOLR-10307.2.patch
Attaching fix patch. Fix is on {{solr}} by exporting the environment variables.
> Provide SSL/TLS keystore password a more secure way
> ---
>
> Key: SOLR-10307
> URL: https://issues.apache.org/jira/browse/SOLR-10307
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: security
>Reporter: Mano Kovacs
>Assignee: Mark Miller
> Fix For: master (7.0), 6.7
>
> Attachments: SOLR-10307.2.patch, SOLR-10307.patch, SOLR-10307.patch,
> SOLR-10307.patch
>
>
> Currently the only way to pass server and client side SSL keytstore and
> truststore passwords is to set specific environment variables that will be
> passed as system properties, through command line parameter.
> First option is to pass passwords through environment variables which gives a
> better level of protection. Second option would be to use hadoop credential
> provider interface to access credential store.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Reopened] (SOLR-10307) Provide SSL/TLS keystore password a more secure way
[
https://issues.apache.org/jira/browse/SOLR-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mano Kovacs reopened SOLR-10307:
Reopening as the parameters in {{solr.in.sh}} are not passed properly.
> Provide SSL/TLS keystore password a more secure way
> ---
>
> Key: SOLR-10307
> URL: https://issues.apache.org/jira/browse/SOLR-10307
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: security
>Reporter: Mano Kovacs
>Assignee: Mark Miller
> Fix For: master (7.0), 6.7
>
> Attachments: SOLR-10307.patch, SOLR-10307.patch, SOLR-10307.patch
>
>
> Currently the only way to pass server and client side SSL keytstore and
> truststore passwords is to set specific environment variables that will be
> passed as system properties, through command line parameter.
> First option is to pass passwords through environment variables which gives a
> better level of protection. Second option would be to use hadoop credential
> provider interface to access credential store.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10307) Provide SSL/TLS keystore password a more secure way
[
https://issues.apache.org/jira/browse/SOLR-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16055638#comment-16055638
]
Mano Kovacs commented on SOLR-10307:
[~michaelsuzuki], thank you for the clarification. Yes, indeed it is not
working anymore according the documentation. {{SOLR_SSL_KEY_STORE_PASSWORD}}
and {{SOLR_SSL_TRUST_STORE_PASSWORD}} is being used by JVM and for that it
needs to be exported. I will upload a patch for the {{solr.in.sh}} and also an
update for the documentation. Thank you for catching this!
> Provide SSL/TLS keystore password a more secure way
> ---
>
> Key: SOLR-10307
> URL: https://issues.apache.org/jira/browse/SOLR-10307
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: security
>Reporter: Mano Kovacs
>Assignee: Mark Miller
> Fix For: master (7.0), 6.7
>
> Attachments: SOLR-10307.patch, SOLR-10307.patch, SOLR-10307.patch
>
>
> Currently the only way to pass server and client side SSL keytstore and
> truststore passwords is to set specific environment variables that will be
> passed as system properties, through command line parameter.
> First option is to pass passwords through environment variables which gives a
> better level of protection. Second option would be to use hadoop credential
> provider interface to access credential store.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10307) Provide SSL/TLS keystore password a more secure way
[
https://issues.apache.org/jira/browse/SOLR-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16054833#comment-16054833
]
Mano Kovacs commented on SOLR-10307:
Hi [~michaelsuzuki], thanks for the followup. I am trying to reproduce the
issue but I might be missing something.
bq. The only way to get this to work is by setting the environment as follow:
What way would you prefer to use instead? Could you send an example?
Here is how I run:
{noformat}
export KEYSTORE=$PWD/keystore.jks
export TRUSTSTORE=$PWD/truststore.jks
rm $KEYSTORE
rm $TRUSTSTORE
keytool -genkey -noprompt \
-alias alias1 \
-dname "CN=mydomain.com, OU=ID, O=ABC, L=John, S=Doe, C=GB" \
-ext "SAN=dns:localhost" \
-keystore $KEYSTORE \
-storepass abc123 \
-keypass abc123 \
-keyalg RSA
keytool -genkey -noprompt \
-alias alias1 \
-dname "CN=mydomain.com, OU=ID, O=ABC, L=John, S=Doe, C=GB" \
-ext "SAN=dns:localhost" \
-keystore $TRUSTSTORE \
-storepass abc456 \
-keypass abc456 \
-keyalg RSA
export SOLR_SSL_ENABLED=true
export SOLR_SSL_KEY_STORE=$KEYSTORE
export SOLR_SSL_KEY_STORE_PASSWORD=abc123
export SOLR_SSL_TRUST_STORE=$TRUSTSTORE
export SOLR_SSL_TRUST_STORE_PASSWORD=abc456
bin/solr start -c -s ./example/cloud/node1/solr -f
{noformat}
Priorly, I downloaded the {{master}} branch, ran {{ant server}} and cloud
example. It is working for me. Note that I did not uncomment anything from
{{solr.in.sh}}.
Also, SOLR-10783 will restore sysprop option as the configuration handling is
moved out from {{jetty-ssl.xml}}.
> Provide SSL/TLS keystore password a more secure way
> ---
>
> Key: SOLR-10307
> URL: https://issues.apache.org/jira/browse/SOLR-10307
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: security
>Reporter: Mano Kovacs
>Assignee: Mark Miller
> Fix For: master (7.0), 6.7
>
> Attachments: SOLR-10307.patch, SOLR-10307.patch, SOLR-10307.patch
>
>
> Currently the only way to pass server and client side SSL keytstore and
> truststore passwords is to set specific environment variables that will be
> passed as system properties, through command line parameter.
> First option is to pass passwords through environment variables which gives a
> better level of protection. Second option would be to use hadoop credential
> provider interface to access credential store.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9910) Allow setting of additional jetty options in bin/solr and bin/solr.cmd
[
https://issues.apache.org/jira/browse/SOLR-9910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053706#comment-16053706
]
Mano Kovacs commented on SOLR-9910:
---
[~shalinmangar], thank you for the feedback.
bq. Mano Kovacs, the main motivation is to be able to run multiple Solr
instances from a single Solr server directory?
One of the original motivation was that. However, some custom configuration or
debug feature is still only reachable through jetty config.
bq. As you say, we could also just leave it as it is and document it as
internal and non back compat as well. That seems like the best option to me.
I'll prepare a doc change patch.
> Allow setting of additional jetty options in bin/solr and bin/solr.cmd
> --
>
> Key: SOLR-9910
> URL: https://issues.apache.org/jira/browse/SOLR-9910
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mano Kovacs
>Assignee: Mark Miller
> Fix For: master (7.0), 6.7
>
> Attachments: SOLR-9910.patch, SOLR-9910.patch
>
>
> Command line tools allow the option {{-a}} to add JVM options to start
> command. Proposing to add {{-j}} option to add additional config for jetty
> (the part after {{start.jar}}).
> Motivation: jetty can be configured with start.ini in server directory.
> Running multiple Solr instances, however, requires the configuration per
> instance that cannot share the start.ini with other instances.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-10912) Adding automatic patch validation
Mano Kovacs created SOLR-10912: -- Summary: Adding automatic patch validation Key: SOLR-10912 URL: https://issues.apache.org/jira/browse/SOLR-10912 Project: Solr Issue Type: Improvement Security Level: Public (Default Security Level. Issues are Public) Components: Build Reporter: Mano Kovacs Proposing introduction of automated patch validation, similar what Hadoop or other Apache projects are using (see link). This would ensure that every patch passes a certain set of criterions before getting approved. It would save time for developer (faster feedback loop), save time for committers (less step to do manually), and would increase quality. Hadoop is currently using Apache Yetus to run validations, which seems to be a good direction to start. This jira could be the board of discussing the preferred solution. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10783) Using Hadoop Credential Provider as SSL/TLS store password source
[
https://issues.apache.org/jira/browse/SOLR-10783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050575#comment-16050575
]
Mano Kovacs commented on SOLR-10783:
bq. Why remove the quotes on this variable? Will that have other side affects?
{{SOLR_JETTY_CONFIG}} is solely used to pass {{--module=http(s)}} to
{{start.jar}}. It is defined in the scripts so it was not usable from the
outside. It is working both {{start.jar "--module=http"}} and {{start.jar
--module=http}}, so it was not an issue before, but I added quoted jetty config
so I removed the quotes to prevent double quotes.
> Using Hadoop Credential Provider as SSL/TLS store password source
> -
>
> Key: SOLR-10783
> URL: https://issues.apache.org/jira/browse/SOLR-10783
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: security
>Reporter: Mano Kovacs
>Assignee: Mark Miller
> Attachments: SOLR-10783.patch
>
>
> As a second iteration of SOLR-10307, I propose support of hadoop credential
> providers as source of SSL store passwords.
> Motivation: When SOLR is used in hadoop environment, support of HCP gives
> better integration and unified method to pass sensitive credentials to SOLR.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-10783) Using Hadoop Credential Provider as SSL/TLS store password source
[
https://issues.apache.org/jira/browse/SOLR-10783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050575#comment-16050575
]
Mano Kovacs edited comment on SOLR-10783 at 6/15/17 2:57 PM:
-
bq. Why remove the quotes on this variable? Will that have other side affects?
{{SOLR_JETTY_CONFIG}} is solely used to pass {{\-\-module=http(s)}} to
{{start.jar}}. It is defined in the scripts so it was not usable from the
outside. It is working both {{start.jar "--module=http"}} and {{start.jar
-\-module=http}}, so it was not an issue before, but I added quoted jetty
config so I removed the quotes to prevent double quotes.
was (Author: manokovacs):
bq. Why remove the quotes on this variable? Will that have other side affects?
{{SOLR_JETTY_CONFIG}} is solely used to pass {{--module=http(s)}} to
{{start.jar}}. It is defined in the scripts so it was not usable from the
outside. It is working both {{start.jar "--module=http"}} and {{start.jar
--module=http}}, so it was not an issue before, but I added quoted jetty config
so I removed the quotes to prevent double quotes.
> Using Hadoop Credential Provider as SSL/TLS store password source
> -
>
> Key: SOLR-10783
> URL: https://issues.apache.org/jira/browse/SOLR-10783
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: security
>Reporter: Mano Kovacs
>Assignee: Mark Miller
> Attachments: SOLR-10783.patch
>
>
> As a second iteration of SOLR-10307, I propose support of hadoop credential
> providers as source of SSL store passwords.
> Motivation: When SOLR is used in hadoop environment, support of HCP gives
> better integration and unified method to pass sensitive credentials to SOLR.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10783) Using Hadoop Credential Provider as SSL/TLS store password source
[
https://issues.apache.org/jira/browse/SOLR-10783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mano Kovacs updated SOLR-10783:
---
Attachment: SOLR-10783.patch
Attaching first patch
* extending {{SSLConfigurations}} to use Hadoop configurations
* Loading store passwords from {{SSLConfigurations}} in {{jetty-ssl.xml}}
* Introducting SOLR_HADOOP_CREDENTIAL_PROVIDER_PATH variable for credential
provider path
Tested on mac and windows.
> Using Hadoop Credential Provider as SSL/TLS store password source
> -
>
> Key: SOLR-10783
> URL: https://issues.apache.org/jira/browse/SOLR-10783
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: security
>Reporter: Mano Kovacs
> Attachments: SOLR-10783.patch
>
>
> As a second iteration of SOLR-10307, I propose support of hadoop credential
> providers as source of SSL store passwords.
> Motivation: When SOLR is used in hadoop environment, support of HCP gives
> better integration and unified method to pass sensitive credentials to SOLR.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10783) Using Hadoop Credential Provider as SSL/TLS store password source
[ https://issues.apache.org/jira/browse/SOLR-10783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mano Kovacs updated SOLR-10783: --- Summary: Using Hadoop Credential Provider as SSL/TLS store password source (was: Using Hadoop Credential Provider as SSL/TLS store password sources) > Using Hadoop Credential Provider as SSL/TLS store password source > - > > Key: SOLR-10783 > URL: https://issues.apache.org/jira/browse/SOLR-10783 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: security >Reporter: Mano Kovacs > > As a second iteration of SOLR-10307, I propose support of hadoop credential > providers as source of SSL store passwords. > Motivation: When SOLR is used in hadoop environment, support of HCP gives > better integration and unified method to pass sensitive credentials to SOLR. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10783) Using Hadoop Credential Provider as SSL/TLS store password sources
[
https://issues.apache.org/jira/browse/SOLR-10783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16031233#comment-16031233
]
Mano Kovacs commented on SOLR-10783:
My plan is using {{SSLConfigurations}} in {{jetty-ssl.xml}} to use different
source for passwords (sysprop, envvars, HCP). Uploading patch shortly.
> Using Hadoop Credential Provider as SSL/TLS store password sources
> --
>
> Key: SOLR-10783
> URL: https://issues.apache.org/jira/browse/SOLR-10783
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: security
>Reporter: Mano Kovacs
>
> As a second iteration of SOLR-10307, I propose support of hadoop credential
> providers as source of SSL store passwords.
> Motivation: When SOLR is used in hadoop environment, support of HCP gives
> better integration and unified method to pass sensitive credentials to SOLR.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-10783) Using Hadoop Credential Provider as SSL/TLS store password sources
Mano Kovacs created SOLR-10783: -- Summary: Using Hadoop Credential Provider as SSL/TLS store password sources Key: SOLR-10783 URL: https://issues.apache.org/jira/browse/SOLR-10783 Project: Solr Issue Type: Improvement Security Level: Public (Default Security Level. Issues are Public) Components: security Reporter: Mano Kovacs As a second iteration of SOLR-10307, I propose support of hadoop credential providers as source of SSL store passwords. Motivation: When SOLR is used in hadoop environment, support of HCP gives better integration and unified method to pass sensitive credentials to SOLR. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10307) Provide SSL/TLS keystore password a more secure way
[ https://issues.apache.org/jira/browse/SOLR-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16014295#comment-16014295 ] Mano Kovacs commented on SOLR-10307: Thanks, [~markrmil...@gmail.com]. I see you backported to 6x too. I mentioned above: bq. There is a non-backward compatible change in the patch. If solr is not started through the batch files (solr.sh or solr.bat), but rather by running the start.jar directly with -Dsolr.jetty.truststore.password, Jetty will not pick up the keystore and truststore passwords. The patch changes jetty-ssl.xml so that is not using the system properties anymore. I did not find a way to use both there. Should this go to the upgrade notes? Is that a concern? > Provide SSL/TLS keystore password a more secure way > --- > > Key: SOLR-10307 > URL: https://issues.apache.org/jira/browse/SOLR-10307 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: security >Reporter: Mano Kovacs >Assignee: Mark Miller > Fix For: master (7.0), 6.7 > > Attachments: SOLR-10307.patch, SOLR-10307.patch, SOLR-10307.patch > > > Currently the only way to pass server and client side SSL keytstore and > truststore passwords is to set specific environment variables that will be > passed as system properties, through command line parameter. > First option is to pass passwords through environment variables which gives a > better level of protection. Second option would be to use hadoop credential > provider interface to access credential store. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9910) Allow setting of additional jetty options in bin/solr and bin/solr.cmd
[
https://issues.apache.org/jira/browse/SOLR-9910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16008220#comment-16008220
]
Mano Kovacs commented on SOLR-9910:
---
[~markrmil...@gmail.com], is there anything I can do related to this jira?
> Allow setting of additional jetty options in bin/solr and bin/solr.cmd
> --
>
> Key: SOLR-9910
> URL: https://issues.apache.org/jira/browse/SOLR-9910
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mano Kovacs
> Attachments: SOLR-9910.patch, SOLR-9910.patch
>
>
> Command line tools allow the option {{-a}} to add JVM options to start
> command. Proposing to add {{-j}} option to add additional config for jetty
> (the part after {{start.jar}}).
> Motivation: jetty can be configured with start.ini in server directory.
> Running multiple Solr instances, however, requires the configuration per
> instance that cannot share the start.ini with other instances.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10307) Provide SSL/TLS keystore password a more secure way
[ https://issues.apache.org/jira/browse/SOLR-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16008186#comment-16008186 ] Mano Kovacs commented on SOLR-10307: [~markrmil...@gmail.com], is there anything I can help to get this jira resolved? > Provide SSL/TLS keystore password a more secure way > --- > > Key: SOLR-10307 > URL: https://issues.apache.org/jira/browse/SOLR-10307 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: security >Reporter: Mano Kovacs >Assignee: Mark Miller > Attachments: SOLR-10307.patch, SOLR-10307.patch, SOLR-10307.patch > > > Currently the only way to pass server and client side SSL keytstore and > truststore passwords is to set specific environment variables that will be > passed as system properties, through command line parameter. > First option is to pass passwords through environment variables which gives a > better level of protection. Second option would be to use hadoop credential > provider interface to access credential store. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10307) Provide SSL/TLS keystore password a more secure way
[ https://issues.apache.org/jira/browse/SOLR-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994478#comment-15994478 ] Mano Kovacs commented on SOLR-10307: Thank you for your feedback, [~shalinmangar]. You are absolutely correct, SSLConnectionSocketFactory uses sysprops only. The patch is a bridge between envvar->sysprop, so the passwords will not show up in the command. > Provide SSL/TLS keystore password a more secure way > --- > > Key: SOLR-10307 > URL: https://issues.apache.org/jira/browse/SOLR-10307 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: security >Reporter: Mano Kovacs >Assignee: Mark Miller > Attachments: SOLR-10307.patch, SOLR-10307.patch, SOLR-10307.patch > > > Currently the only way to pass server and client side SSL keytstore and > truststore passwords is to set specific environment variables that will be > passed as system properties, through command line parameter. > First option is to pass passwords through environment variables which gives a > better level of protection. Second option would be to use hadoop credential > provider interface to access credential store. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10307) Provide SSL/TLS keystore password a more secure way
[ https://issues.apache.org/jira/browse/SOLR-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15992905#comment-15992905 ] Mano Kovacs commented on SOLR-10307: [~markrmil...@gmail.com], any comment on the latest patch? > Provide SSL/TLS keystore password a more secure way > --- > > Key: SOLR-10307 > URL: https://issues.apache.org/jira/browse/SOLR-10307 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: security >Reporter: Mano Kovacs >Assignee: Mark Miller > Attachments: SOLR-10307.patch, SOLR-10307.patch, SOLR-10307.patch > > > Currently the only way to pass server and client side SSL keytstore and > truststore passwords is to set specific environment variables that will be > passed as system properties, through command line parameter. > First option is to pass passwords through environment variables which gives a > better level of protection. Second option would be to use hadoop credential > provider interface to access credential store. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10570) Changing solr.cloud.wait-for-updates-with-stale-state-pause for tests
[
https://issues.apache.org/jira/browse/SOLR-10570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mano Kovacs updated SOLR-10570:
---
Description:
{{solr.cloud.wait-for-updates-with-stale-state-pause}} is by default 2.5
seconds and it controls how long the follower waits for updates on leader (see
SOLR-7141).
SOLR-9849 changes the sysprop in the test but it does not always applies to the
tests.
was:
{{solr.cloud.wait-for-updates-with-stale-state-pause}} is by default 2.5
seconds and it controls how long the follower waits for updates on leader (see
SOLR-7141).
[~mihaly.toth] is proposing to change it to a lower value so
- tests getting faster (10% of improvement based on first measurements)
- less falkyness by timeouts on distributed tests.
> Changing solr.cloud.wait-for-updates-with-stale-state-pause for tests
> -
>
> Key: SOLR-10570
> URL: https://issues.apache.org/jira/browse/SOLR-10570
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Tests
>Reporter: Mano Kovacs
>
> {{solr.cloud.wait-for-updates-with-stale-state-pause}} is by default 2.5
> seconds and it controls how long the follower waits for updates on leader
> (see SOLR-7141).
> SOLR-9849 changes the sysprop in the test but it does not always applies to
> the tests.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Reopened] (SOLR-10570) Changing solr.cloud.wait-for-updates-with-stale-state-pause for tests
[
https://issues.apache.org/jira/browse/SOLR-10570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mano Kovacs reopened SOLR-10570:
> Changing solr.cloud.wait-for-updates-with-stale-state-pause for tests
> -
>
> Key: SOLR-10570
> URL: https://issues.apache.org/jira/browse/SOLR-10570
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Tests
>Reporter: Mano Kovacs
>
> {{solr.cloud.wait-for-updates-with-stale-state-pause}} is by default 2.5
> seconds and it controls how long the follower waits for updates on leader
> (see SOLR-7141).
> [~mihaly.toth] is proposing to change it to a lower value so
> - tests getting faster (10% of improvement based on first measurements)
> - less falkyness by timeouts on distributed tests.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7141) RecoveryStrategy: Raise time that we wait for any updates from the leader before they saw the recovery state to have finished.
[ https://issues.apache.org/jira/browse/SOLR-7141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15984679#comment-15984679 ] Mano Kovacs commented on SOLR-7141: --- Sorry, the 7 seconds was already lowered in SOLR-9848. > RecoveryStrategy: Raise time that we wait for any updates from the leader > before they saw the recovery state to have finished. > -- > > Key: SOLR-7141 > URL: https://issues.apache.org/jira/browse/SOLR-7141 > Project: Solr > Issue Type: Improvement >Reporter: Mark Miller >Assignee: Mark Miller > Fix For: 5.1, 6.0 > > Attachments: SOLR-7141.patch > > > The current wait of 3 seconds is pushing the envelope a bit. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Closed] (SOLR-10570) Changing solr.cloud.wait-for-updates-with-stale-state-pause for tests
[
https://issues.apache.org/jira/browse/SOLR-10570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mano Kovacs closed SOLR-10570.
--
Resolution: Duplicate
> Changing solr.cloud.wait-for-updates-with-stale-state-pause for tests
> -
>
> Key: SOLR-10570
> URL: https://issues.apache.org/jira/browse/SOLR-10570
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Tests
>Reporter: Mano Kovacs
>
> {{solr.cloud.wait-for-updates-with-stale-state-pause}} is by default 2.5
> seconds and it controls how long the follower waits for updates on leader
> (see SOLR-7141).
> [~mihaly.toth] is proposing to change it to a lower value so
> - tests getting faster (10% of improvement based on first measurements)
> - less falkyness by timeouts on distributed tests.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10570) Changing solr.cloud.wait-for-updates-with-stale-state-pause for tests
[
https://issues.apache.org/jira/browse/SOLR-10570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mano Kovacs updated SOLR-10570:
---
Description:
{{solr.cloud.wait-for-updates-with-stale-state-pause}} is by default 2.5
seconds and it controls how long the follower waits for updates on leader (see
SOLR-7141).
[~mihaly.toth] is proposing to change it to a lower value so
- tests getting faster (10% of improvement based on first measurements)
- less falkyness by timeouts on distributed tests.
was:
{{solr.cloud.wait-for-updates-with-stale-state-pause}} is by default 7 seconds
and it controls how long the follower waits for updates on leader (see
SOLR-7141).
[~mihaly.toth] is proposing to change it to a lower value so
- tests getting faster (10% of improvement based on first measurements)
- less falkyness by timeouts on distributed tests.
> Changing solr.cloud.wait-for-updates-with-stale-state-pause for tests
> -
>
> Key: SOLR-10570
> URL: https://issues.apache.org/jira/browse/SOLR-10570
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Tests
>Reporter: Mano Kovacs
>
> {{solr.cloud.wait-for-updates-with-stale-state-pause}} is by default 2.5
> seconds and it controls how long the follower waits for updates on leader
> (see SOLR-7141).
> [~mihaly.toth] is proposing to change it to a lower value so
> - tests getting faster (10% of improvement based on first measurements)
> - less falkyness by timeouts on distributed tests.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7141) RecoveryStrategy: Raise time that we wait for any updates from the leader before they saw the recovery state to have finished.
[ https://issues.apache.org/jira/browse/SOLR-7141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15984423#comment-15984423 ] Mano Kovacs commented on SOLR-7141: --- [~mihaly.toth], I created SOLR-10570 about your proposal. > RecoveryStrategy: Raise time that we wait for any updates from the leader > before they saw the recovery state to have finished. > -- > > Key: SOLR-7141 > URL: https://issues.apache.org/jira/browse/SOLR-7141 > Project: Solr > Issue Type: Improvement >Reporter: Mark Miller >Assignee: Mark Miller > Fix For: 5.1, 6.0 > > Attachments: SOLR-7141.patch > > > The current wait of 3 seconds is pushing the envelope a bit. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-10570) Changing solr.cloud.wait-for-updates-with-stale-state-pause for tests
Mano Kovacs created SOLR-10570:
--
Summary: Changing
solr.cloud.wait-for-updates-with-stale-state-pause for tests
Key: SOLR-10570
URL: https://issues.apache.org/jira/browse/SOLR-10570
Project: Solr
Issue Type: Improvement
Security Level: Public (Default Security Level. Issues are Public)
Components: Tests
Reporter: Mano Kovacs
{{solr.cloud.wait-for-updates-with-stale-state-pause}} is by default 7 seconds
and it controls how long the follower waits for updates on leader (see
SOLR-7141).
[~mihaly.toth] is proposing to change it to a lower value so
- tests getting faster (10% of improvement based on first measurements)
- less falkyness by timeouts on distributed tests.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7427) Recovery can miss some updates when they're neither forwarded nor present in replicated index
[
https://issues.apache.org/jira/browse/SOLR-7427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15982947#comment-15982947
]
Mano Kovacs commented on SOLR-7427:
---
We bumped into this issue recently with test timeouts and I understand the
workaround is to wait 7 seconds before submit commit command.
If I understand it correctly, the reason for this wait is to avoid updates
starting before the forwarding, but ending after the commit. As far as I
understand, those updates will not yet be forwarded to recovering replica, but
will be written into the next open segment, created after the hard commit,
therefore, won't be replicated with the full replication.
If that is the case, I am wondering, would a custom commit command help?
Assuming that this delayed-commit would wait until each update (that was
already started at the time of the command) is written out, before writing.
What I am thinking of is based on [~yo...@apache.org]'s flow described above:
1. replica tells leader it want's to recover
2. leader starts forwarding updates to replica (which the replica buffers since
it's in recovery)
3. delayed-commit command
3.1 the leader collects somehow the already running updates and blocks until
each ends (newer updates are disregarded, though)
3.2 leader executes a hard commit (so replica can replicate the current index)
4. replica starts replicating index from the last leader commit point
[~markrmil...@gmail.com], [~yo...@apache.org], could you help me verify this?
> Recovery can miss some updates when they're neither forwarded nor present in
> replicated index
> -
>
> Key: SOLR-7427
> URL: https://issues.apache.org/jira/browse/SOLR-7427
> Project: Solr
> Issue Type: Bug
> Components: SolrCloud
>Affects Versions: 4.10.4, 5.1
>Reporter: Shalin Shekhar Mangar
> Labels: consistency, difficulty-hard, impact-high
>
> According to discussion in SOLR-7141. See [~ysee...@gmail.com]'s comment at
> https://issues.apache.org/jira/browse/SOLR-7141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14501622#comment-14501622
> {quote}
> From memory, here's how it's supposed to work:
> 1. replica tells leader it want's to recover
> 2. leader starts forwarding updates to replica (which the replica buffers
> since it's in recovery)
> 3. leader executes a hard commit (so replica can replicate the current index)
> 4. replica starts replicating index from the last leader commit point
> Note that the ordering of #2 and #3 is very important. If we did #3 first
> and then #2 after, some updates won't make it into the commit and also won't
> be forwarded to the replica (and that leads to data loss).
> Now the issue: even though we do #2 first and #3 after... it's possible to
> have an unfortunately scheduled update in a different thread that started
> before we did #2, and doesn't complete until after #3, so that update was not
> forwarded, and it's also not in the replicated index. The sleep (which
> should be between steps #2 and #3) is to try and give time for this update to
> complete and make it into the index.
> It occurs to me that the lucene IndexWriter thread stealing (same issue that
> caused this: SOLR-6820) could make this much more likely than we would have
> thought.
> One possible alternative is to block updates for a commit of this type
> (replication commit). Any blocked updates would need to see that they need
> to be forwarded to the replica too (once they are unblocked) - I don't know
> if the code is currently written that way.
> {quote}
> So there is some protection against such a situation but it is based on two
> timeout values:
> # The replica stalls recovery until the leader acknowledges that it has
> indeed seen the replica in 'recovery' (via the prep recovery core admin API)
> # The replica sleeps for 7 seconds by default (configured via the
> hidden-switch "solr.cloud.wait-for-updates-with-stale-state-pause" system
> property) after prep recovery completes to give additional time for such
> updates to complete.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10307) Provide SSL/TLS keystore password a more secure way
[
https://issues.apache.org/jira/browse/SOLR-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mano Kovacs updated SOLR-10307:
---
Attachment: SOLR-10307.patch
Uploading patch #3.
[~markrmil...@gmail.com], I get your point, it is in fact too generic name. I
renamed {{Configurations}} (and related factory and test) to
{{SSLConfigurations}}. This way it can be used if the SSL credentials are taken
from different source.
> Provide SSL/TLS keystore password a more secure way
> ---
>
> Key: SOLR-10307
> URL: https://issues.apache.org/jira/browse/SOLR-10307
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: security
>Reporter: Mano Kovacs
>Assignee: Mark Miller
> Attachments: SOLR-10307.patch, SOLR-10307.patch, SOLR-10307.patch
>
>
> Currently the only way to pass server and client side SSL keytstore and
> truststore passwords is to set specific environment variables that will be
> passed as system properties, through command line parameter.
> First option is to pass passwords through environment variables which gives a
> better level of protection. Second option would be to use hadoop credential
> provider interface to access credential store.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-9910) Allow setting of additional jetty options in bin/solr and bin/solr.cmd
[
https://issues.apache.org/jira/browse/SOLR-9910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mano Kovacs updated SOLR-9910:
--
Attachment: SOLR-9910.patch
Updated, [~markrmil...@gmail.com], thank you.
> Allow setting of additional jetty options in bin/solr and bin/solr.cmd
> --
>
> Key: SOLR-9910
> URL: https://issues.apache.org/jira/browse/SOLR-9910
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mano Kovacs
> Attachments: SOLR-9910.patch, SOLR-9910.patch
>
>
> Command line tools allow the option {{-a}} to add JVM options to start
> command. Proposing to add {{-j}} option to add additional config for jetty
> (the part after {{start.jar}}).
> Motivation: jetty can be configured with start.ini in server directory.
> Running multiple Solr instances, however, requires the configuration per
> instance that cannot share the start.ini with other instances.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10307) Provide SSL/TLS keystore password a more secure way
[
https://issues.apache.org/jira/browse/SOLR-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972531#comment-15972531
]
Mano Kovacs commented on SOLR-10307:
There is a non-backward compatible change in the patch. If solr is not started
through the batch files (solr.sh or solr.bat), but rather by running the
{{start.jar}} directly with {{-Dsolr.jetty.truststore.password}}, Jetty will
not pick up the keystore and truststore passwords. The patch changes
{{jetty-ssl.xml}} so that is not using the system properties anymore. I did not
find a way to use both there. Should this go to the upgrade notes?
> Provide SSL/TLS keystore password a more secure way
> ---
>
> Key: SOLR-10307
> URL: https://issues.apache.org/jira/browse/SOLR-10307
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: security
>Reporter: Mano Kovacs
>Assignee: Mark Miller
> Attachments: SOLR-10307.patch, SOLR-10307.patch
>
>
> Currently the only way to pass server and client side SSL keytstore and
> truststore passwords is to set specific environment variables that will be
> passed as system properties, through command line parameter.
> First option is to pass passwords through environment variables which gives a
> better level of protection. Second option would be to use hadoop credential
> provider interface to access credential store.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10307) Provide SSL/TLS keystore password a more secure way
[
https://issues.apache.org/jira/browse/SOLR-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mano Kovacs updated SOLR-10307:
---
Attachment: SOLR-10307.patch
Thank you for the review, [~markrmil...@gmail.com]. Uploading second patch.
bq. Only suggestion I have is that it might be nice to add a sentence comment
to Configurations init and setupSSLStorePasswords so that you can quickly tell
what these methods do and how they should be called.
I added brief documentation for {{init}} and {{setupSSLStorePasswords}}.
bq. Might use something less generic than Configurations as well, even if it's
EnvVarConfigurations or something.
I was planning to add further configuration options to this class later
(probably something with Hadoop Credential Provider Interface), which would
suggest a more generic name. If you agree, I would use a more generic name.
> Provide SSL/TLS keystore password a more secure way
> ---
>
> Key: SOLR-10307
> URL: https://issues.apache.org/jira/browse/SOLR-10307
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: security
>Reporter: Mano Kovacs
>Assignee: Mark Miller
> Attachments: SOLR-10307.patch, SOLR-10307.patch
>
>
> Currently the only way to pass server and client side SSL keytstore and
> truststore passwords is to set specific environment variables that will be
> passed as system properties, through command line parameter.
> First option is to pass passwords through environment variables which gives a
> better level of protection. Second option would be to use hadoop credential
> provider interface to access credential store.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-10151) TestRecovery.java - use monotonic increasing version number among all the tests to avoid unintentional reordering
[
https://issues.apache.org/jira/browse/SOLR-10151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15967541#comment-15967541
]
Mano Kovacs edited comment on SOLR-10151 at 4/13/17 12:53 PM:
--
Thanks [~szantaikis] for taking the effort.
I see you are named the version generator method {{getVersionString}}. Would it
make sense for you to somehow imply that it actually changes the version
between invocations? Maybe {{incrementAndGetVersion}} or similar?
was (Author: manokovacs):
Thanks [~szantaikis] for taking the effort.
I see you are named the version generator method {{getVersionString}}. Would it
make sense for you to somehow imply that it actually changes the version
between invocations? Like {{incrementAndGetVersion}} or similar like that?
> TestRecovery.java - use monotonic increasing version number among all the
> tests to avoid unintentional reordering
> -
>
> Key: SOLR-10151
> URL: https://issues.apache.org/jira/browse/SOLR-10151
> Project: Solr
> Issue Type: Test
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mano Kovacs
>Priority: Minor
> Labels: newbie
> Attachments: SOLR_10151.patch
>
>
> {{TestRecovery}} has several tests inserting updates and deletes into a
> shared core. The tests are using fixed version number which can overlap and
> can cause issues depending on the order of the tests.
> Proposing using a monotonically incrementing counter for each test and
> changing tests that they would allocate the used versions would ensure that
> later running tests would send updates with higher version only. That would
> prevent any unintentional reordering.
> h5. Example:
> Before:
> {noformat}
> ...
> updateJ(jsonAdd(sdoc("id", "RDBQ1_1", "_version_", "1010")),
> params(DISTRIB_UPDATE_PARAM, FROM_LEADER));
> updateJ(jsonDelQ("id:RDBQ1_2"), params(DISTRIB_UPDATE_PARAM, FROM_LEADER,
> "_version_", "-1017")); // This should've arrived after the 1015th update
> updateJ(jsonAdd(sdoc("id", "RDBQ1_2", "_version_", "1015")),
> params(DISTRIB_UPDATE_PARAM, FROM_LEADER));
> updateJ(jsonAdd(sdoc("id", "RDBQ1_3", "_version_", "1020")),
> params(DISTRIB_UPDATE_PARAM, FROM_LEADER));
> ...
> {noformat}
> After:
> {noformat}
> ...
> String insVer1 = getNextVersion();
> String insVer2 = getNextVersion();
> String deleteVer = getNextVersion();
> String insVer3 = getNextVersion();
> updateJ(jsonAdd(sdoc("id", "RDBQ1_1", "_version_",insVer1)),
> params(DISTRIB_UPDATE_PARAM, FROM_LEADER));
> updateJ(jsonDelQ("id:RDBQ1_2"), params(DISTRIB_UPDATE_PARAM, FROM_LEADER,
> "_version_", "-"+deleteVer)); // This should've arrived after the 1015th
> update
> updateJ(jsonAdd(sdoc("id", "RDBQ1_2", "_version_", insVer2)),
> params(DISTRIB_UPDATE_PARAM, FROM_LEADER));
> updateJ(jsonAdd(sdoc("id", "RDBQ1_3", "_version_", insVer3)),
> params(DISTRIB_UPDATE_PARAM, FROM_LEADER));
> ...
> {noformat}
> It might increase readability as the generation of the versions happen in the
> preferred replay order.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10151) TestRecovery.java - use monotonic increasing version number among all the tests to avoid unintentional reordering
[
https://issues.apache.org/jira/browse/SOLR-10151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15967541#comment-15967541
]
Mano Kovacs commented on SOLR-10151:
Thanks [~szantaikis] for taking the effort.
I see you are named the version generator method {{getVersionString}}. Would it
make sense for you to somehow imply that it actually changes the version
between invocations? Like {{incrementAndGetVersion}} or similar like that?
> TestRecovery.java - use monotonic increasing version number among all the
> tests to avoid unintentional reordering
> -
>
> Key: SOLR-10151
> URL: https://issues.apache.org/jira/browse/SOLR-10151
> Project: Solr
> Issue Type: Test
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mano Kovacs
>Priority: Minor
> Labels: newbie
> Attachments: SOLR_10151.patch
>
>
> {{TestRecovery}} has several tests inserting updates and deletes into a
> shared core. The tests are using fixed version number which can overlap and
> can cause issues depending on the order of the tests.
> Proposing using a monotonically incrementing counter for each test and
> changing tests that they would allocate the used versions would ensure that
> later running tests would send updates with higher version only. That would
> prevent any unintentional reordering.
> h5. Example:
> Before:
> {noformat}
> ...
> updateJ(jsonAdd(sdoc("id", "RDBQ1_1", "_version_", "1010")),
> params(DISTRIB_UPDATE_PARAM, FROM_LEADER));
> updateJ(jsonDelQ("id:RDBQ1_2"), params(DISTRIB_UPDATE_PARAM, FROM_LEADER,
> "_version_", "-1017")); // This should've arrived after the 1015th update
> updateJ(jsonAdd(sdoc("id", "RDBQ1_2", "_version_", "1015")),
> params(DISTRIB_UPDATE_PARAM, FROM_LEADER));
> updateJ(jsonAdd(sdoc("id", "RDBQ1_3", "_version_", "1020")),
> params(DISTRIB_UPDATE_PARAM, FROM_LEADER));
> ...
> {noformat}
> After:
> {noformat}
> ...
> String insVer1 = getNextVersion();
> String insVer2 = getNextVersion();
> String deleteVer = getNextVersion();
> String insVer3 = getNextVersion();
> updateJ(jsonAdd(sdoc("id", "RDBQ1_1", "_version_",insVer1)),
> params(DISTRIB_UPDATE_PARAM, FROM_LEADER));
> updateJ(jsonDelQ("id:RDBQ1_2"), params(DISTRIB_UPDATE_PARAM, FROM_LEADER,
> "_version_", "-"+deleteVer)); // This should've arrived after the 1015th
> update
> updateJ(jsonAdd(sdoc("id", "RDBQ1_2", "_version_", insVer2)),
> params(DISTRIB_UPDATE_PARAM, FROM_LEADER));
> updateJ(jsonAdd(sdoc("id", "RDBQ1_3", "_version_", insVer3)),
> params(DISTRIB_UPDATE_PARAM, FROM_LEADER));
> ...
> {noformat}
> It might increase readability as the generation of the versions happen in the
> preferred replay order.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
