[jira] [Commented] (SOLR-14541) Ensure classes that implement equals implement hashCode or suppress warnings
[
https://issues.apache.org/jira/browse/SOLR-14541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17132814#comment-17132814
]
Erick Erickson commented on SOLR-14541:
---
I'm working on the warnings in autoscaling and implemented throwing an
exception for hashcode. I have a reproducible failure, looks on a brief glance
like it's confined to the test, but I'd like [~ab] to weigh in. The code is up
on my fork here:
https://github.com/ErickErickson/lucene-solr/tree/SOLR-14556
reproduce with:
{code}
./gradlew :solr:core:test --tests
"org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration"
-Ptests.seed=DA115FAE01235F25 -Ptests.file.encoding=ISO-8859-1
{code}
Here's the stack trace:
{code}
2> => java.lang.UnsupportedOperationException: TODO unimplemented
2>at
org.apache.solr.client.solrj.cloud.autoscaling.ReplicaInfo.hashCode(ReplicaInfo.java:214)
2> java.lang.UnsupportedOperationException: TODO unimplemented
2>at
org.apache.solr.client.solrj.cloud.autoscaling.ReplicaInfo.hashCode(ReplicaInfo.java:214)
~[solr-solrj-9.0.0-SNAPSHOT.jar:9.0.0-SNAPSHOT
5c41afa6a0bacfb7588acbe455834aa3e3727280 [snapshot build, details omitted]]
2>at java.util.ArrayList.hashCodeRange(ArrayList.java:626) ~[?:?]
2>at java.util.ArrayList.hashCode(ArrayList.java:613) ~[?:?]
2>at java.util.Objects.hashCode(Objects.java:116) ~[?:?]
2>at java.util.HashMap$Node.hashCode(HashMap.java:297) ~[?:?]
2>at java.util.AbstractMap.hashCode(AbstractMap.java:527) ~[?:?]
2>at
org.apache.solr.cloud.autoscaling.TriggerEvent.hashCode(TriggerEvent.java:276)
~[main/:?]
2>at
java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1012)
~[?:?]
2>at
java.util.concurrent.ConcurrentHashMap$KeySetView.add(ConcurrentHashMap.java:4644)
~[?:?]
2>at
org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration$TestSearchRateAction.process(TestSimTriggerIntegration.java:1354)
~[test/:?]
2>at
org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$add$3(ScheduledTriggers.java:328)
~[main/:?]
2>at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?]
2>at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
2>at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210)
~[solr-solrj-9.0.0-SNAPSHOT.jar:9.0.0-SNAPSHOT
5c41afa6a0bacfb7588acbe455834aa3e3727280 [snapshot build, details omitted]]
2>at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
~[?:?]
2>at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
~[?:?]
2>
{code}
> Ensure classes that implement equals implement hashCode or suppress warnings
>
>
> Key: SOLR-14541
> URL: https://issues.apache.org/jira/browse/SOLR-14541
> Project: Solr
> Issue Type: Sub-task
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
> Attachments: 0001-SOLR-14541-add-hashCode-for-some-classes.patch,
> 0002-SOLR-14541-add-hashCode-for-some-classes-in-autoscal.patch,
> 0003-SOLR-14541-add-hashCode-or-remove-equals-for-some-cl.patch
>
>
> While looking at warnings, I found that the following classes generate this
> warning:
> *overrides equals, but neither it nor any superclass overrides hashCode
> method*
> I can suppress the warning, but this has been a source of errors in the past
> so I'm reluctant to just do that blindly.
> NOTE: The Lucene one should probably be it's own Jira if it's going to have
> hashCode implemented, but here for triage.
> What I need for each method is for someone who has a clue about that
> particular code to render an opinion that we can safely suppress the warning
> or to provide a hashCode method.
> Some of these have been here for a very long time and were implemented by
> people no longer active...
> lucene/suggest/src/java/org/apache/lucene/search/spell/LuceneLevenshteinDistance.java:39
> solr/solrj/src/java/org/apache/solr/common/cloud/ZkNodeProps.java:34
> solr/solrj/src/java/org/apache/solr/common/cloud/Replica.java:26
> solr/solrj/src/java/org/apache/solr/common/cloud/DocCollection.java:49
> solr/core/src/java/org/apache/solr/cloud/rule/Rule.java:277
> solr/core/src/java/org/apache/solr/pkg/PackageAPI.java:177
> solr/core/src/java/org/apache/solr/packagemanager/SolrPackageInstance.java:31
>
> Noble Paul says it's OK to suppress warnings for these:
> solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/VersionedData.java:31
>
> solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/AutoScalingConfig.java:61
>
>
[jira] [Commented] (LUCENE-9401) ComplexPhraseQuery's toString method always omits field name
[
https://issues.apache.org/jira/browse/LUCENE-9401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17132786#comment-17132786
]
Lucene/Solr QA commented on LUCENE-9401:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 0m 20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 0m 20s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} Validate source patterns {color} |
{color:red} 0m 20s{color} | {color:red} Validate source patterns
validate-source-patterns failed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 9s{color}
| {color:red} queryparser in the patch failed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 2m 26s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | lucene.queryparser.complexPhrase.TestComplexPhraseQuery |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | LUCENE-9401 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/13005427/LUCENE-9401.patch |
| Optional Tests | compile javac unit ratsources checkforbiddenapis
validatesourcepatterns |
| uname | Linux lucene1-us-west 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24
10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-LUCENE-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / 07a203628ea |
| ant | version: Apache Ant(TM) version 1.10.5 compiled on March 28 2019 |
| Default Java | LTS |
| Validate source patterns |
https://builds.apache.org/job/PreCommit-LUCENE-Build/283/artifact/out/patch-validate-source-patterns-root.txt
|
| unit |
https://builds.apache.org/job/PreCommit-LUCENE-Build/283/artifact/out/patch-unit-lucene_queryparser.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-LUCENE-Build/283/testReport/ |
| modules | C: lucene/queryparser U: lucene/queryparser |
| Console output |
https://builds.apache.org/job/PreCommit-LUCENE-Build/283/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> ComplexPhraseQuery's toString method always omits field name
>
>
> Key: LUCENE-9401
> URL: https://issues.apache.org/jira/browse/LUCENE-9401
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/queryparser
>Affects Versions: 8.5.2
>Reporter: Thomas Hecker
>Priority: Trivial
> Attachments: LUCENE-9401.patch
>
>
> The toString(String field) method in
> org.apache.lucene.queryparser.complexPhrase.ComplexPhraseQueryParser$ComplexPhraseQuery
> should only omit the field name if query's field name is not equal to field
> name that is passed as an argument.
> Instead, the query's field name is never included in the returned String.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Assigned] (SOLR-14554) BMW algorithm isn't used when scores are requested
[
https://issues.apache.org/jira/browse/SOLR-14554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomas Eduardo Fernandez Lobbe reassigned SOLR-14554:
Assignee: Tomas Eduardo Fernandez Lobbe
> BMW algorithm isn't used when scores are requested
> --
>
> Key: SOLR-14554
> URL: https://issues.apache.org/jira/browse/SOLR-14554
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: master (9.0), 8.6
>Reporter: Tomas Eduardo Fernandez Lobbe
>Assignee: Tomas Eduardo Fernandez Lobbe
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This is because, when scores are requested, we use MultiCollector, and it
> will end up using a scorer that explicitly skips delegating calls to set the
> minimum competitive score:
> {code:java}
> @Override
> public void setScorer(Scorable scorer) throws IOException {
> if (cacheScores) {
> scorer = new ScoreCachingWrappingScorer(scorer);
> }
> scorer = new FilterScorable(scorer) {
> @Override
> public void setMinCompetitiveScore(float minScore) throws IOException
> {
> // Ignore calls to setMinCompetitiveScore so that if we wrap two
> // collectors and one of them wants to skip low-scoring hits, then
> // the other collector still sees all hits. We could try to
> reconcile
> // min scores and take the maximum min score across collectors, but
> // this is very unlikely to be helpful in practice.
> }
> };
> for (int i = 0; i < numCollectors; ++i) {
> final LeafCollector c = collectors[i];
> c.setScorer(scorer);
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-14552) Add BMW support to Rerank queries
[ https://issues.apache.org/jira/browse/SOLR-14552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomas Eduardo Fernandez Lobbe resolved SOLR-14552. -- Fix Version/s: 8.6 master (9.0) Assignee: Tomas Eduardo Fernandez Lobbe Resolution: Fixed > Add BMW support to Rerank queries > - > > Key: SOLR-14552 > URL: https://issues.apache.org/jira/browse/SOLR-14552 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Tomas Eduardo Fernandez Lobbe >Assignee: Tomas Eduardo Fernandez Lobbe >Priority: Major > Fix For: master (9.0), 8.6 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14455) Autoscaling policy for ADDREPLICA not functioning in Metric Based Triggers
[ https://issues.apache.org/jira/browse/SOLR-14455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17132763#comment-17132763 ] Erick Erickson commented on SOLR-14455: --- I mis-typed the Jira number, ignore these two commits and see SOLR-14555 > Autoscaling policy for ADDREPLICA not functioning in Metric Based Triggers > -- > > Key: SOLR-14455 > URL: https://issues.apache.org/jira/browse/SOLR-14455 > Project: Solr > Issue Type: Bug >Reporter: Sujith >Priority: Major > > The Autoscaling policy for ADDREPLICA is not functioning in Metric Based > Triggers. The "preferredOperation" was given "*ADDREPLICA*" for a sample > metric trigger and it wasnt functioning. However on the other hand, the > operation MOVEREPLICA is working as expected. I tried this in Solr version 7.5 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Issue Comment Deleted] (SOLR-14455) Autoscaling policy for ADDREPLICA not functioning in Metric Based Triggers
[ https://issues.apache.org/jira/browse/SOLR-14455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Erickson updated SOLR-14455: -- Comment: was deleted (was: Commit 07a203628ea46308612f4bb60686c313a6e4d75e in lucene-solr's branch refs/heads/master from Erick Erickson [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=07a2036 ] SOLR-14455: Fix or suppress warnings in solr/test-framework ) > Autoscaling policy for ADDREPLICA not functioning in Metric Based Triggers > -- > > Key: SOLR-14455 > URL: https://issues.apache.org/jira/browse/SOLR-14455 > Project: Solr > Issue Type: Bug >Reporter: Sujith >Priority: Major > > The Autoscaling policy for ADDREPLICA is not functioning in Metric Based > Triggers. The "preferredOperation" was given "*ADDREPLICA*" for a sample > metric trigger and it wasnt functioning. However on the other hand, the > operation MOVEREPLICA is working as expected. I tried this in Solr version 7.5 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-14555) Fix or suppress warnings in solr/test-framework
[ https://issues.apache.org/jira/browse/SOLR-14555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Erickson resolved SOLR-14555. --- Fix Version/s: 8.6 Resolution: Fixed > Fix or suppress warnings in solr/test-framework > --- > > Key: SOLR-14555 > URL: https://issues.apache.org/jira/browse/SOLR-14555 > Project: Solr > Issue Type: Sub-task >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Fix For: 8.6 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Issue Comment Deleted] (SOLR-14455) Autoscaling policy for ADDREPLICA not functioning in Metric Based Triggers
[ https://issues.apache.org/jira/browse/SOLR-14455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Erickson updated SOLR-14455: -- Comment: was deleted (was: Commit e7c625430ed916fd7d0e7e8d1cdc329093d817b4 in lucene-solr's branch refs/heads/branch_8x from Erick Erickson [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=e7c6254 ] SOLR-14455: Fix or suppress warnings in solr/test-framework ) > Autoscaling policy for ADDREPLICA not functioning in Metric Based Triggers > -- > > Key: SOLR-14455 > URL: https://issues.apache.org/jira/browse/SOLR-14455 > Project: Solr > Issue Type: Bug >Reporter: Sujith >Priority: Major > > The Autoscaling policy for ADDREPLICA is not functioning in Metric Based > Triggers. The "preferredOperation" was given "*ADDREPLICA*" for a sample > metric trigger and it wasnt functioning. However on the other hand, the > operation MOVEREPLICA is working as expected. I tried this in Solr version 7.5 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14555) Fix or suppress warnings in solr/test-framework
[ https://issues.apache.org/jira/browse/SOLR-14555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17132762#comment-17132762 ] Erick Erickson commented on SOLR-14555: --- Bah, I mis-typed the Jira again. Commit 07a203628ea46308612f4bb60686c313a6e4d75e in lucene-solr's branch refs/heads/master from Erick Erickson [ [https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=07a2036] ] SOLR-14455: Fix or suppress warnings in solr/test-framework and Commit e7c625430ed916fd7d0e7e8d1cdc329093d817b4 in lucene-solr's branch refs/heads/branch_8x from Erick Erickson [ [https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=e7c6254] ] SOLR-14455: Fix or suppress warnings in solr/test-framework > Fix or suppress warnings in solr/test-framework > --- > > Key: SOLR-14555 > URL: https://issues.apache.org/jira/browse/SOLR-14555 > Project: Solr > Issue Type: Sub-task >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14455) Autoscaling policy for ADDREPLICA not functioning in Metric Based Triggers
[ https://issues.apache.org/jira/browse/SOLR-14455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17132760#comment-17132760 ] ASF subversion and git services commented on SOLR-14455: Commit e7c625430ed916fd7d0e7e8d1cdc329093d817b4 in lucene-solr's branch refs/heads/branch_8x from Erick Erickson [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=e7c6254 ] SOLR-14455: Fix or suppress warnings in solr/test-framework > Autoscaling policy for ADDREPLICA not functioning in Metric Based Triggers > -- > > Key: SOLR-14455 > URL: https://issues.apache.org/jira/browse/SOLR-14455 > Project: Solr > Issue Type: Bug >Reporter: Sujith >Priority: Major > > The Autoscaling policy for ADDREPLICA is not functioning in Metric Based > Triggers. The "preferredOperation" was given "*ADDREPLICA*" for a sample > metric trigger and it wasnt functioning. However on the other hand, the > operation MOVEREPLICA is working as expected. I tried this in Solr version 7.5 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14455) Autoscaling policy for ADDREPLICA not functioning in Metric Based Triggers
[ https://issues.apache.org/jira/browse/SOLR-14455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17132759#comment-17132759 ] ASF subversion and git services commented on SOLR-14455: Commit 07a203628ea46308612f4bb60686c313a6e4d75e in lucene-solr's branch refs/heads/master from Erick Erickson [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=07a2036 ] SOLR-14455: Fix or suppress warnings in solr/test-framework > Autoscaling policy for ADDREPLICA not functioning in Metric Based Triggers > -- > > Key: SOLR-14455 > URL: https://issues.apache.org/jira/browse/SOLR-14455 > Project: Solr > Issue Type: Bug >Reporter: Sujith >Priority: Major > > The Autoscaling policy for ADDREPLICA is not functioning in Metric Based > Triggers. The "preferredOperation" was given "*ADDREPLICA*" for a sample > metric trigger and it wasnt functioning. However on the other hand, the > operation MOVEREPLICA is working as expected. I tried this in Solr version 7.5 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14552) Add BMW support to Rerank queries
[ https://issues.apache.org/jira/browse/SOLR-14552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17132757#comment-17132757 ] ASF subversion and git services commented on SOLR-14552: Commit 800edb05bc535900974988ac989fbfbe5dd58ef7 in lucene-solr's branch refs/heads/branch_8x from Tomas Eduardo Fernandez Lobbe [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=800edb0 ] SOLR-14552: Add BMW support to ReRank queries (#1559) > Add BMW support to Rerank queries > - > > Key: SOLR-14552 > URL: https://issues.apache.org/jira/browse/SOLR-14552 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Tomas Eduardo Fernandez Lobbe >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] tflobbe commented on pull request #1567: LUCENE-9402: Let MultiCollector handle minCompetitiveScore
tflobbe commented on pull request #1567: URL: https://github.com/apache/lucene-solr/pull/1567#issuecomment-642292175 https://github.com/apache/lucene-solr/pull/1566 contains the Solr related changes This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14552) Add BMW support to Rerank queries
[ https://issues.apache.org/jira/browse/SOLR-14552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17132749#comment-17132749 ] ASF subversion and git services commented on SOLR-14552: Commit 9728f4ef201db0b8b7bce1e909efee83593dede7 in lucene-solr's branch refs/heads/master from Tomas Eduardo Fernandez Lobbe [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=9728f4e ] SOLR-14552: Add BMW support to ReRank queries (#1559) > Add BMW support to Rerank queries > - > > Key: SOLR-14552 > URL: https://issues.apache.org/jira/browse/SOLR-14552 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Tomas Eduardo Fernandez Lobbe >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] tflobbe merged pull request #1559: SOLR-14552: Add BMW support to ReRank queries
tflobbe merged pull request #1559: URL: https://github.com/apache/lucene-solr/pull/1559 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] tflobbe commented on a change in pull request #1567: LUCENE-9402: Let MultiCollector handle minCompetitiveScore
tflobbe commented on a change in pull request #1567:
URL: https://github.com/apache/lucene-solr/pull/1567#discussion_r438420301
##
File path: lucene/core/src/java/org/apache/lucene/search/MultiCollector.java
##
@@ -143,32 +143,43 @@ public LeafCollector getLeafCollector(LeafReaderContext
context) throws IOExcept
private final boolean cacheScores;
private final LeafCollector[] collectors;
private int numCollectors;
+private final float[] minScores;
+private final boolean skipNonCompetitiveScores;
-private MultiLeafCollector(List collectors, boolean
cacheScores) {
+private MultiLeafCollector(List collectors, boolean
cacheScores, boolean skipNonCompetitive) {
this.collectors = collectors.toArray(new
LeafCollector[collectors.size()]);
this.cacheScores = cacheScores;
this.numCollectors = this.collectors.length;
+ this.skipNonCompetitiveScores = skipNonCompetitive;
+ this.minScores = new float[this.skipNonCompetitiveScores ?
this.numCollectors : 0];
}
@Override
public void setScorer(Scorable scorer) throws IOException {
if (cacheScores) {
scorer = new ScoreCachingWrappingScorer(scorer);
}
- scorer = new FilterScorable(scorer) {
-@Override
-public void setMinCompetitiveScore(float minScore) {
- // Ignore calls to setMinCompetitiveScore so that if we wrap two
- // collectors and one of them wants to skip low-scoring hits, then
- // the other collector still sees all hits. We could try to reconcile
- // min scores and take the maximum min score across collectors, but
- // this is very unlikely to be helpful in practice.
+ if (skipNonCompetitiveScores) {
+for (int i = 0; i < numCollectors; ++i) {
+ final LeafCollector c = collectors[i];
+ c.setScorer(new MinCompetitiveScoreAwareScorable(scorer, i,
minScores));
}
+ } else {
+scorer = new FilterScorable(scorer) {
+ @Override
+ public void setMinCompetitiveScore(float minScore) throws
IOException {
+// Ignore calls to setMinCompetitiveScore so that if we wrap two
+// collectors and one of them wants to skip low-scoring hits, then
+// the other collector still sees all hits. We could try to
reconcile
+// min scores and take the maximum min score across collectors, but
+// this is very unlikely to be helpful in practice.
Review comment:
I should fix this comment
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] tflobbe opened a new pull request #1567: LUCENE-9402: Let MultiCollector handle minCompetitiveScore
tflobbe opened a new pull request #1567: URL: https://github.com/apache/lucene-solr/pull/1567 Here is an idea to make MultiCollector be able to handle `minCompetitiveScore`. Looking at this comment in the code: ``` // Ignore calls to setMinCompetitiveScore so that if we wrap two // collectors and one of them wants to skip low-scoring hits, then // the other collector still sees all hits. We could try to reconcile // min scores and take the maximum min score across collectors, but // this is very unlikely to be helpful in practice. ``` This implementation tries to make it so that all collectors can see all the hits and only allow skipping if all collectors set a min competitive score. The value passed to the inner scorer is the minimum among all collectors (instead of the maximum as the comment suggests). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 9.0
[
https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17132721#comment-17132721
]
Erick Erickson commented on SOLR-12823:
---
AFAIK all of the tests are run by both build systems.
What you might be hitting is the annotations. {{CloudHttp2SolrClientTest}} for
instance is annotated with @Slow, unless you have (for ant)
{code:java}
-Dtests.slow=true
{code}
it wouldn't run.
This is not a huge deal, if you want to enable that annotation, feel free but
it'll make things, well, slow ;). By and large, nobody is worried by this kind
of glitch as long as it's addressed reasonably, either by disabling the test or
finding the root cause. Preferably the latter of course...
> remove clusterstate.json in Lucene/Solr 9.0
> ---
>
> Key: SOLR-12823
> URL: https://issues.apache.org/jira/browse/SOLR-12823
> Project: Solr
> Issue Type: Task
>Reporter: Varun Thacker
>Assignee: Mike Drob
>Priority: Major
> Fix For: master (9.0)
>
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove
> that in 9.0
> It stays empty unless you explicitly ask to create the collection with the
> old "stateFormat" and there is no reason for one to create a collection with
> the old stateFormat.
> We should also remove the "stateFormat" argument in create collection
> We should also remove MIGRATESTATEVERSION as well
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] tflobbe opened a new pull request #1566: SOLR-14554: Let Solr use WAND algorithm when scores are requested
tflobbe opened a new pull request #1566: URL: https://github.com/apache/lucene-solr/pull/1566 This PR includes a commit in Lucene, for which I'm going to create a separate PR. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc commented on a change in pull request #1565: SOLR-12823: fix test failures
murblanc commented on a change in pull request #1565:
URL: https://github.com/apache/lucene-solr/pull/1565#discussion_r438409534
##
File path: solr/solrj/src/java/org/apache/solr/common/cloud/ZkStateReader.java
##
@@ -476,27 +476,32 @@ public synchronized void
createClusterStateWatchersAndUpdate() throws KeeperExce
log.debug("Updating cluster state from ZooKeeper... ");
-// on reconnect of SolrZkClient force refresh and re-add watches.
-loadClusterProperties();
-refreshLiveNodes(new LiveNodeWatcher());
-refreshCollections();
-refreshCollectionList(new CollectionsChildWatcher());
-refreshAliases(aliasesManager);
-
-if (securityNodeListener != null) {
- addSecurityNodeWatcher(pair -> {
-ConfigData cd = new ConfigData();
-cd.data = pair.first() == null || pair.first().length == 0 ? EMPTY_MAP
: Utils.getDeepCopy((Map) fromJSON(pair.first()), 4, false);
-cd.version = pair.second() == null ? -1 : pair.second().getVersion();
-securityData = cd;
-securityNodeListener.run();
+try {
+ // on reconnect of SolrZkClient force refresh and re-add watches.
+ loadClusterProperties();
+ refreshLiveNodes(new LiveNodeWatcher());
+ refreshCollections();
+ refreshCollectionList(new CollectionsChildWatcher());
+ refreshAliases(aliasesManager);
+
+ if (securityNodeListener != null) {
+addSecurityNodeWatcher(pair -> {
+ ConfigData cd = new ConfigData();
+ cd.data = pair.first() == null || pair.first().length == 0 ?
EMPTY_MAP : Utils.getDeepCopy((Map) fromJSON(pair.first()), 4, false);
+ cd.version = pair.second() == null ? -1 : pair.second().getVersion();
+ securityData = cd;
+ securityNodeListener.run();
+});
+securityData = getSecurityProps(true);
+ }
+
+ collectionPropsObservers.forEach((k, v) -> {
+collectionPropsWatchers.computeIfAbsent(k,
PropsWatcher::new).refreshAndWatch(true);
});
- securityData = getSecurityProps(true);
+} catch (KeeperException.NoNodeException nne) {
+ throw new SolrException(ErrorCode.SERVICE_UNAVAILABLE,
+ "Cannot connect to cluster at " + zkClient.getZkServerAddress() + ":
cluster not found/not ready");
Review comment:
Could have instead picked another path other than `CLUSTER_STATE` from
those created in `ZkController.createClusterZkNodes()` and explicitly tested it
like was done before, but this seemed cleaner.
Open to change.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 9.0
[
https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17132716#comment-17132716
]
Ilan Ginzburg commented on SOLR-12823:
--
PR [https://github.com/apache/lucene-solr/pull/1565] fixes tests
{{CloudHttp2SolrClientTest}}, {{CloudSolrClientTest}} and
{{TestCloudSolrClientConnections}}.
Will fix {{TestZKPropertiesWriter}} separately, adding to this PR or creating a
new one if this one is merged (don't hesitate to merge this one).
Which ant/gradle target runs such tests that are not included in {{ant test}}?
> remove clusterstate.json in Lucene/Solr 9.0
> ---
>
> Key: SOLR-12823
> URL: https://issues.apache.org/jira/browse/SOLR-12823
> Project: Solr
> Issue Type: Task
>Reporter: Varun Thacker
>Assignee: Mike Drob
>Priority: Major
> Fix For: master (9.0)
>
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove
> that in 9.0
> It stays empty unless you explicitly ask to create the collection with the
> old "stateFormat" and there is no reason for one to create a collection with
> the old stateFormat.
> We should also remove the "stateFormat" argument in create collection
> We should also remove MIGRATESTATEVERSION as well
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (LUCENE-9402) Let MultiCollector Scorer handle minCompetitiveScore calls
Tomas Eduardo Fernandez Lobbe created LUCENE-9402:
-
Summary: Let MultiCollector Scorer handle minCompetitiveScore calls
Key: LUCENE-9402
URL: https://issues.apache.org/jira/browse/LUCENE-9402
Project: Lucene - Core
Issue Type: Improvement
Reporter: Tomas Eduardo Fernandez Lobbe
See SOLR-14554. MultiCollector creates a scorer that explicitly prevents
setting the {{minCompetitiveScore}}:
{code:java}
@Override
public void setScorer(Scorable scorer) throws IOException {
if (cacheScores) {
scorer = new ScoreCachingWrappingScorer(scorer);
}
scorer = new FilterScorable(scorer) {
@Override
public void setMinCompetitiveScore(float minScore) throws IOException {
// Ignore calls to setMinCompetitiveScore so that if we wrap two
// collectors and one of them wants to skip low-scoring hits, then
// the other collector still sees all hits. We could try to reconcile
// min scores and take the maximum min score across collectors, but
// this is very unlikely to be helpful in practice.
}
};
for (int i = 0; i < numCollectors; ++i) {
final LeafCollector c = collectors[i];
c.setScorer(scorer);
}
}
{code}
Solr uses MultiCollector when scores are requested (to collect the max score),
which means it wouldn't be able to use WAND algorithm.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc opened a new pull request #1565: SOLR-12823: fix test failures
murblanc opened a new pull request #1565: URL: https://github.com/apache/lucene-solr/pull/1565 # Description Fix test failures in CloudHttp2SolrClientTest CloudSolrClientTest TestCloudSolrClientConnections due to previous SOLR-12823 commit. # Solution ZkStateReader returns a specific SolrException when expected nodes in the cluster created during cluster initialization do not exist. # Tests Fixes the tests mentioned above. # Checklist Please review the following and check all that apply: - [X] I have reviewed the guidelines for [How to Contribute](https://wiki.apache.org/solr/HowToContribute) and my code conforms to the standards described there to the best of my ability. - [X] I have created a Jira issue and added the issue ID to my pull request title. - [X] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended) - [X] I have developed this patch against the `master` branch. - [X] I have run `ant precommit` and the appropriate test suite. - [ ] I have added tests for my changes. - [ ] I have added documentation for the [Ref Guide](https://github.com/apache/lucene-solr/tree/master/solr/solr-ref-guide) (for Solr changes only). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] rmuir commented on pull request #1557: LUCENE-9396: Improve truncation detection for points.
rmuir commented on pull request #1557: URL: https://github.com/apache/lucene-solr/pull/1557#issuecomment-642242198 Should we keep the existing check also? It does more than just check length, it validates this "footer" too (e.g. detecting invalid bits). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-14558) SolrLogPostTool should record all lines
Jason Gerlowski created SOLR-14558: -- Summary: SolrLogPostTool should record all lines Key: SOLR-14558 URL: https://issues.apache.org/jira/browse/SOLR-14558 Project: Solr Issue Type: Improvement Security Level: Public (Default Security Level. Issues are Public) Components: scripts and tools Affects Versions: master (9.0) Reporter: Jason Gerlowski Currently, SolrLogPostTool recognizes a predefined set of "types" of log messages: queries, errors, commits, etc. This makes it easy to find and explore the traffic your cluster is seeing. But it would also be cool if we also indexed all records, even if many of them are just assigned a catch-all "other" type_s value. We won't be able to parse out detailed values from the log messages the way we would for type_s=query for example, but we can still store the line and timestamp. Gives much better search over the logs than dropping down to "grep" for anything that's not one of the predefined types. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9401) ComplexPhraseQuery's toString method always omits field name
[ https://issues.apache.org/jira/browse/LUCENE-9401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Hecker updated LUCENE-9401: -- Attachment: LUCENE-9401.patch Lucene Fields: New,Patch Available (was: New) Status: Patch Available (was: Patch Available) Patch added > ComplexPhraseQuery's toString method always omits field name > > > Key: LUCENE-9401 > URL: https://issues.apache.org/jira/browse/LUCENE-9401 > Project: Lucene - Core > Issue Type: Bug > Components: core/queryparser >Affects Versions: 8.5.2 >Reporter: Thomas Hecker >Priority: Trivial > Attachments: LUCENE-9401.patch > > > The toString(String field) method in > org.apache.lucene.queryparser.complexPhrase.ComplexPhraseQueryParser$ComplexPhraseQuery > should only omit the field name if query's field name is not equal to field > name that is passed as an argument. > Instead, the query's field name is never included in the returned String. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9401) ComplexPhraseQuery's toString method always omits field name
[ https://issues.apache.org/jira/browse/LUCENE-9401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Hecker updated LUCENE-9401: -- Status: Patch Available (was: Open) > ComplexPhraseQuery's toString method always omits field name > > > Key: LUCENE-9401 > URL: https://issues.apache.org/jira/browse/LUCENE-9401 > Project: Lucene - Core > Issue Type: Bug > Components: core/queryparser >Affects Versions: 8.5.2 >Reporter: Thomas Hecker >Priority: Trivial > Attachments: LUCENE-9401.patch > > > The toString(String field) method in > org.apache.lucene.queryparser.complexPhrase.ComplexPhraseQueryParser$ComplexPhraseQuery > should only omit the field name if query's field name is not equal to field > name that is passed as an argument. > Instead, the query's field name is never included in the returned String. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 9.0
[ https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17131078#comment-17131078 ] Ilan Ginzburg commented on SOLR-12823: -- Just deleted my previous comment: environment issue led me to believe some test failures were occurring before my PR got merged. If you got a notification, please ignore it, and if you didn't, good. > remove clusterstate.json in Lucene/Solr 9.0 > --- > > Key: SOLR-12823 > URL: https://issues.apache.org/jira/browse/SOLR-12823 > Project: Solr > Issue Type: Task >Reporter: Varun Thacker >Assignee: Mike Drob >Priority: Major > Fix For: master (9.0) > > Time Spent: 4h 10m > Remaining Estimate: 0h > > clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove > that in 9.0 > It stays empty unless you explicitly ask to create the collection with the > old "stateFormat" and there is no reason for one to create a collection with > the old stateFormat. > We should also remove the "stateFormat" argument in create collection > We should also remove MIGRATESTATEVERSION as well > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Issue Comment Deleted] (SOLR-12823) remove clusterstate.json in Lucene/Solr 9.0
[
https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilan Ginzburg updated SOLR-12823:
-
Comment: was deleted
(was: Failures of:
{code:java}
CloudHttp2SolrClientTest.testWrongZkChrootTest()
CloudSolrClientTest.testWrongZkChrootTest()
TestCloudSolrClientConnections.testCloudClientCanConnectAfterClusterComesUp()
TestCloudSolrClientConnections.testCloudClientUploads()
{code}
Are all due to the same root cause: {{/aliases.json}} is not found in ZK.
I removed the large {{/clusterstate.json}} commit (reverted master to commit
54c5dd7d6d8b486df9132e637a6d7c5871ac4664 “LUCENE-9148: Move the BKD index to
its own file. (#1475)”) and I see the same failures.
I therefore believe these failures are independent of the
{{/clusterstate.json}} removal, please let me know [~mdrob] if you think
otherwise.
I will concentrate on {{TestZKPropertiesWriter.testZKPropertiesWriter()}}.
Also, which test target should I run to detect these issues ahead of time next
time? {{ant test}} didn’t run them, {{gradlew test}} didn’t either.)
> remove clusterstate.json in Lucene/Solr 9.0
> ---
>
> Key: SOLR-12823
> URL: https://issues.apache.org/jira/browse/SOLR-12823
> Project: Solr
> Issue Type: Task
>Reporter: Varun Thacker
>Assignee: Mike Drob
>Priority: Major
> Fix For: master (9.0)
>
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove
> that in 9.0
> It stays empty unless you explicitly ask to create the collection with the
> old "stateFormat" and there is no reason for one to create a collection with
> the old stateFormat.
> We should also remove the "stateFormat" argument in create collection
> We should also remove MIGRATESTATEVERSION as well
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (LUCENE-9401) ComplexPhraseQuery's toString method always omits field name
Thomas Hecker created LUCENE-9401: - Summary: ComplexPhraseQuery's toString method always omits field name Key: LUCENE-9401 URL: https://issues.apache.org/jira/browse/LUCENE-9401 Project: Lucene - Core Issue Type: Bug Components: core/queryparser Affects Versions: 8.5.2 Reporter: Thomas Hecker The toString(String field) method in org.apache.lucene.queryparser.complexPhrase.ComplexPhraseQueryParser$ComplexPhraseQuery should only omit the field name if query's field name is not equal to field name that is passed as an argument. Instead, the query's field name is never included in the returned String. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9397) UniformSplit supports encodable fields metadata
[ https://issues.apache.org/jira/browse/LUCENE-9397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130940#comment-17130940 ] Adrien Grand commented on LUCENE-9397: -- Out of curiosity, what kind of data are you attaching to fields/terms? > UniformSplit supports encodable fields metadata > --- > > Key: LUCENE-9397 > URL: https://issues.apache.org/jira/browse/LUCENE-9397 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Bruno Roustant >Assignee: Bruno Roustant >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > UniformSplit already supports custom encoding for term blocks. This is an > extension to also support encodable fields metadata. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9395) ConstantValuesSource creates more than one DoubleValues unnecessarily
[ https://issues.apache.org/jira/browse/LUCENE-9395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130930#comment-17130930 ] Tony Xu commented on LUCENE-9395: - Thanks Atri and David for taking the time to respond and test the change. I agree it's not a needle-moving change and the basic intention is to save some cpu cycles with small amount of complexity added. A more advanced use case is where the constant takes non-trivial amount of time to compute. In this case, we would like to not pay that cost if the DVS is not used. I'm preparing another patch and to lazily initialize the DVs. Let's see how that goes. > ConstantValuesSource creates more than one DoubleValues unnecessarily > -- > > Key: LUCENE-9395 > URL: https://issues.apache.org/jira/browse/LUCENE-9395 > Project: Lucene - Core > Issue Type: Improvement > Components: core/search >Affects Versions: 8.5.2 >Reporter: Tony Xu >Priority: Minor > Attachments: LUCENE-9395.patch > > > At my day job, we use ConstantValuesSource to represent default values or a > constant query-level feature by calling *_DoubleValuesSource.constant_*. I > realized under the hood the _*ConstantValuesSource.getDoubleValues*_ creates > a new _*DoubleValues*_ which simply return the specified value each time it > is called. > Unless I missed something, I don't see a risk of creating one > _*DoubleValues*_ as use it as the return value of all _*getDoubleValues**()*_ > calls given that the constant _*DoubleValues*_ doesn't maintain any state. > We can also offer the user flexibilities of how to initialize it. > 1) _*DoubleValuesSource.constant(double constant)*_ – we can eagerly > initialize an `DoubleValues` that returns the constant and make it the return > value of all _*getDoubleValues()*_ calls. > 2) _*DoubleValuesSource.constant(DoubleSupplier doubleSupplier)*_ – For lazy > evaluation if the constant takes some time to compute and user expects the > returned DVS will not be used in all code path. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 9.0
[
https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130920#comment-17130920
]
Ilan Ginzburg commented on SOLR-12823:
--
Failures of:
{code:java}
CloudHttp2SolrClientTest.testWrongZkChrootTest()
CloudSolrClientTest.testWrongZkChrootTest()
TestCloudSolrClientConnections.testCloudClientCanConnectAfterClusterComesUp()
TestCloudSolrClientConnections.testCloudClientUploads()
{code}
Are all due to the same root cause: {{/aliases.json}} is not found in ZK.
I removed the large {{/clusterstate.json}} commit (reverted master to commit
54c5dd7d6d8b486df9132e637a6d7c5871ac4664 “LUCENE-9148: Move the BKD index to
its own file. (#1475)”) and I see the same failures.
I therefore believe these failures are independent of the
{{/clusterstate.json}} removal, please let me know [~mdrob] if you think
otherwise.
I will concentrate on {{TestZKPropertiesWriter.testZKPropertiesWriter()}}.
Also, which test target should I run to detect these issues ahead of time next
time? {{ant test}} didn’t run them, {{gradlew test}} didn’t either.
> remove clusterstate.json in Lucene/Solr 9.0
> ---
>
> Key: SOLR-12823
> URL: https://issues.apache.org/jira/browse/SOLR-12823
> Project: Solr
> Issue Type: Task
>Reporter: Varun Thacker
>Assignee: Mike Drob
>Priority: Major
> Fix For: master (9.0)
>
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove
> that in 9.0
> It stays empty unless you explicitly ask to create the collection with the
> old "stateFormat" and there is no reason for one to create a collection with
> the old stateFormat.
> We should also remove the "stateFormat" argument in create collection
> We should also remove MIGRATESTATEVERSION as well
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9392) Change the visibility of o.a.l.f.FacetsConfig.DELIM_CHAR to public
[ https://issues.apache.org/jira/browse/LUCENE-9392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130910#comment-17130910 ] ASF subversion and git services commented on LUCENE-9392: - Commit 1a668183db3ff147432eefe1641f0bc334bbf9a2 in lucene-solr's branch refs/heads/branch_8x from Michael McCandless [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=1a66818 ] LUCENE-9392: make FacetsConfig.DELIM_CHAR public > Change the visibility of o.a.l.f.FacetsConfig.DELIM_CHAR to public > --- > > Key: LUCENE-9392 > URL: https://issues.apache.org/jira/browse/LUCENE-9392 > Project: Lucene - Core > Issue Type: Improvement >Affects Versions: 8.5.2 >Reporter: Ankur >Priority: Minor > Attachments: LUCENE-9392.patch > > > FacetsConfig.DELIM_CHAR is marked as private. An application that wants to > use this delimiter (in a unit test for example) is forced to re-declare it in > the application code. This can break the application if tetshe value of > DELIM_CHAR is changed in FacetsConfig -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9392) Change the visibility of o.a.l.f.FacetsConfig.DELIM_CHAR to public
[ https://issues.apache.org/jira/browse/LUCENE-9392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless updated LUCENE-9392: --- Fix Version/s: 8.6 Resolution: Fixed Status: Resolved (was: Patch Available) Thank you [~goankur]! > Change the visibility of o.a.l.f.FacetsConfig.DELIM_CHAR to public > --- > > Key: LUCENE-9392 > URL: https://issues.apache.org/jira/browse/LUCENE-9392 > Project: Lucene - Core > Issue Type: Improvement >Affects Versions: 8.5.2 >Reporter: Ankur >Priority: Minor > Fix For: 8.6 > > Attachments: LUCENE-9392.patch > > > FacetsConfig.DELIM_CHAR is marked as private. An application that wants to > use this delimiter (in a unit test for example) is forced to re-declare it in > the application code. This can break the application if tetshe value of > DELIM_CHAR is changed in FacetsConfig -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9392) Change the visibility of o.a.l.f.FacetsConfig.DELIM_CHAR to public
[ https://issues.apache.org/jira/browse/LUCENE-9392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130908#comment-17130908 ] ASF subversion and git services commented on LUCENE-9392: - Commit 138cdd758a02522ec965e9323b30e0359b986508 in lucene-solr's branch refs/heads/master from Michael McCandless [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=138cdd7 ] LUCENE-9392: make FacetsConfig.DELIM_CHAR public > Change the visibility of o.a.l.f.FacetsConfig.DELIM_CHAR to public > --- > > Key: LUCENE-9392 > URL: https://issues.apache.org/jira/browse/LUCENE-9392 > Project: Lucene - Core > Issue Type: Improvement >Affects Versions: 8.5.2 >Reporter: Ankur >Priority: Minor > Attachments: LUCENE-9392.patch > > > FacetsConfig.DELIM_CHAR is marked as private. An application that wants to > use this delimiter (in a unit test for example) is forced to re-declare it in > the application code. This can break the application if tetshe value of > DELIM_CHAR is changed in FacetsConfig -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14557) eDisMax (regression)
[
https://issues.apache.org/jira/browse/SOLR-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Khludnev updated SOLR-14557:
Description:
h2. Solr 4.5
{{/select?defType=edismax=\{!lucene}(foo)=true}}
goes like
{code}
\{!lucene}(foo)
content:foo
LuceneQParser
{code}
fine
h2. Solr 8.2
with luceneMatchVersion=4.5 following SOLR-11501 I know it's a grey zone but
it's a question of migrating existing queries.
{{/select?defType=edismax=\{!lucene}(foo)=true}}
goes like
{code}
"querystring":"\{!lucene}(foo)",
"parsedquery":"+DisjunctionMaxQuery(((Project.Address:lucene
Project.Address:foo) | (Project.OwnerType:lucene Project.OwnerType:foo)
"QParser":"ExtendedDismaxQParser",
{code}
blah...
but removing braces in 8.2 works perfectly fine
{code}
"querystring":"\{!lucene}foo",
"parsedquery":"+content:foo",
"parsedquery_toString":"+content:foo",
"QParser":"ExtendedDismaxQParser",
{code}
was:
h2. Solr 4.5
{{/select?defType=edismax={!lucene}(foo)=true}}
goes like
{code}
{!lucene}(foo)
content:foo
LuceneQParser
{code}
fine
h2. Solr 8.2
with luceneMatchVersion=4.5 following SOLR-11501 I know it's a grey zone but
it's a question of migrating existing queries.
{{/select?defType=edismax={!lucene}(foo)=true}}
goes like
{code}
"querystring":"{!lucene}(foo)",
"parsedquery":"+DisjunctionMaxQuery(((Project.Address:lucene
Project.Address:foo) | (Project.OwnerType:lucene Project.OwnerType:foo)
"QParser":"ExtendedDismaxQParser",
{code}
blah...
but removing braces in 8.2 works perfectly fine
{code}
"querystring":"{!lucene}foo",
"parsedquery":"+content:foo",
"parsedquery_toString":"+content:foo",
"QParser":"ExtendedDismaxQParser",
{code}
> eDisMax (regression)
>
>
> Key: SOLR-14557
> URL: https://issues.apache.org/jira/browse/SOLR-14557
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: query parsers
>Reporter: Mikhail Khludnev
>Priority: Major
> Labels: painful
>
> h2. Solr 4.5
> {{/select?defType=edismax=\{!lucene}(foo)=true}}
>
> goes like
> {code}
> \{!lucene}(foo)
> content:foo
> LuceneQParser
> {code}
> fine
> h2. Solr 8.2
> with luceneMatchVersion=4.5 following SOLR-11501 I know it's a grey zone but
> it's a question of migrating existing queries.
> {{/select?defType=edismax=\{!lucene}(foo)=true}}
> goes like
> {code}
> "querystring":"\{!lucene}(foo)",
> "parsedquery":"+DisjunctionMaxQuery(((Project.Address:lucene
> Project.Address:foo) | (Project.OwnerType:lucene Project.OwnerType:foo)
> "QParser":"ExtendedDismaxQParser",
> {code}
> blah...
> but removing braces in 8.2 works perfectly fine
> {code}
> "querystring":"\{!lucene}foo",
> "parsedquery":"+content:foo",
> "parsedquery_toString":"+content:foo",
> "QParser":"ExtendedDismaxQParser",
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14557) eDisMax (regression)
[
https://issues.apache.org/jira/browse/SOLR-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Khludnev updated SOLR-14557:
Description:
h2. Solr 4.5
{{/select?defType=edismax={!lucene}(foo)=true}}
goes like
{code}
{!lucene}(foo)
content:foo
LuceneQParser
{code}
fine
h2. Solr 8.2
with luceneMatchVersion=4.5 following SOLR-11501 I know it's a grey zone but
it's a question of migrating existing queries.
{{/select?defType=edismax={!lucene}(foo)=true}}
goes like
{code}
"querystring":"{!lucene}(foo)",
"parsedquery":"+DisjunctionMaxQuery(((Project.Address:lucene
Project.Address:foo) | (Project.OwnerType:lucene Project.OwnerType:foo)
"QParser":"ExtendedDismaxQParser",
{code}
blah...
but removing braces in 8.2 works perfectly fine
{code}
"querystring":"{!lucene}foo",
"parsedquery":"+content:foo",
"parsedquery_toString":"+content:foo",
"QParser":"ExtendedDismaxQParser",
{code}
was:
h2. Solr 4.5
{{/select?defType=edismax={!lucene}(foo)=true}}
goes like
{code}
{!lucene}(foo)
content:foo
LuceneQParser
{code}
fine
h2. Solr 8.2
with luceneMatchVersion=4.5 following SOLR-11501 I know it's a grey zone but
it's a question of migrating existing queries.
{{/select?defType=edismax={!lucene}(foo)=true}}
goes like
{code}
"querystring":"{!lucene}(foo)",
"parsedquery":"+DisjunctionMaxQuery(((Project.Address:lucene
Project.Address:foo) | (Project.OwnerType:lucene Project.OwnerType:foo)
"QParser":"ExtendedDismaxQParser",
{code}
blah...
but removing braces in 8.2 works perfectly fine
{code}
"querystring":"{!lucene}foo",
"parsedquery":"+content:foo",
"parsedquery_toString":"+content:foo",
"QParser":"ExtendedDismaxQParser",
{code}
> eDisMax (regression)
>
>
> Key: SOLR-14557
> URL: https://issues.apache.org/jira/browse/SOLR-14557
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: query parsers
>Reporter: Mikhail Khludnev
>Priority: Major
> Labels: painful
>
> h2. Solr 4.5
> {{/select?defType=edismax={!lucene}(foo)=true}}
> goes like
> {code}
> {!lucene}(foo)
> content:foo
> LuceneQParser
> {code}
> fine
> h2. Solr 8.2
> with luceneMatchVersion=4.5 following SOLR-11501 I know it's a grey zone but
> it's a question of migrating existing queries.
> {{/select?defType=edismax={!lucene}(foo)=true}}
> goes like
> {code}
> "querystring":"{!lucene}(foo)",
> "parsedquery":"+DisjunctionMaxQuery(((Project.Address:lucene
> Project.Address:foo) | (Project.OwnerType:lucene Project.OwnerType:foo)
>"QParser":"ExtendedDismaxQParser",
> {code}
> blah...
> but removing braces in 8.2 works perfectly fine
> {code}
> "querystring":"{!lucene}foo",
> "parsedquery":"+content:foo",
> "parsedquery_toString":"+content:foo",
> "QParser":"ExtendedDismaxQParser",
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-14557) eDisMax regression
Mikhail Khludnev created SOLR-14557:
---
Summary: eDisMax regression
Key: SOLR-14557
URL: https://issues.apache.org/jira/browse/SOLR-14557
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Components: query parsers
Reporter: Mikhail Khludnev
h2. Solr 4.5
{{/select?defType=edismax={!lucene}(foo)=true}}
goes like
{code}
{!lucene}(foo)
content:foo
LuceneQParser
{code}
fine
h2. Solr 8.2
with luceneMatchVersion=4.5 following SOLR-11501 I know it's a grey zone but
it's a question of migrating existing queries.
{{/select?defType=edismax={!lucene}(foo)=true}}
goes like
{code}
"querystring":"{!lucene}(foo)",
"parsedquery":"+DisjunctionMaxQuery(((Project.Address:lucene
Project.Address:foo) | (Project.OwnerType:lucene Project.OwnerType:foo)
"QParser":"ExtendedDismaxQParser",
{code}
blah...
but removing braces in 8.2 works perfectly fine
{code}
"querystring":"{!lucene}foo",
"parsedquery":"+content:foo",
"parsedquery_toString":"+content:foo",
"QParser":"ExtendedDismaxQParser",
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14557) eDisMax (regression)
[
https://issues.apache.org/jira/browse/SOLR-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Khludnev updated SOLR-14557:
Summary: eDisMax (regression) (was: eDisMax regression)
> eDisMax (regression)
>
>
> Key: SOLR-14557
> URL: https://issues.apache.org/jira/browse/SOLR-14557
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: query parsers
>Reporter: Mikhail Khludnev
>Priority: Major
> Labels: painful
>
> h2. Solr 4.5
> {{/select?defType=edismax={!lucene}(foo)=true}}
> goes like
> {code}
> {!lucene}(foo)
> content:foo
> LuceneQParser
> {code}
> fine
> h2. Solr 8.2
> with luceneMatchVersion=4.5 following SOLR-11501 I know it's a grey zone but
> it's a question of migrating existing queries.
> {{/select?defType=edismax={!lucene}(foo)=true}}
> goes like
> {code}
> "querystring":"{!lucene}(foo)",
> "parsedquery":"+DisjunctionMaxQuery(((Project.Address:lucene
> Project.Address:foo) | (Project.OwnerType:lucene Project.OwnerType:foo)
>"QParser":"ExtendedDismaxQParser",
> {code}
> blah...
> but removing braces in 8.2 works perfectly fine
> {code}
> "querystring":"{!lucene}foo",
> "parsedquery":"+content:foo",
> "parsedquery_toString":"+content:foo",
> "QParser":"ExtendedDismaxQParser",
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13132) Improve JSON "terms" facet performance when sorted by relatedness
[
https://issues.apache.org/jira/browse/SOLR-13132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130827#comment-17130827
]
Michael Gibney commented on SOLR-13132:
---
Just pushed a commit (5e021167fe0cf8ee212edc493c7cd5c9c00f3aa7) that adds
javadocs for the "Shim" class, and also realized that I could simplify the
class itself and clarify its intended use.
I've thought a bit more about the question of how to detect the {{allBuckets}}
slot for disabling {{allBuckets}} relatedness: I don't really have any good
answers, but a handful of thoughts:
# the "baby-bear" solution (disabling sweep when {{allBuckets==true}}) would
work, but could have significant negative performance implications.
# the only downside to the "papa-bear" solution (adding a {{SlotContext}} arg
to {{incrementCount(...)}}) would be adding baggage to the
otherwise-straightforward {{CountSlotAcc}} API; but if you're ok with that, I
do think this would work (and I'd be happy to take a crack at implementing),
and this solution would I think have very little performance overhead, since we
already have {{SlotContexts}} in the {{collect}} methods that would be calling
(directly or indirectly) {{incrementCount(...)}}.
# returning the new {{SlotAcc}} from {{SlotAcc.registerSweepingAccs()}} (or
even a separate, special-purpose {{SlotAcc}}) would be the simplest solution
(in terms of requiring few structural changes) ... but introducing a non-null
{{collectAcc}} (as opposed to setting {{collectAcc==null}}) would have the
unfortunate side-effect of undermining the "count-only" performance
optimizations in {{ByArrayUIF}} and {{ByArrayDV}}.
# If adding an extra API param somewhere carrying additional information about
the slot, doing that in {{setValues(...)}} would be nice because it would be
called fewer times. But unlike for {{incrementCount(...)}}, there's no
{{SlotContext}} already available, so I guess this approach (which I'd
suggested earlier) might be a non-starter, and I'm not even sure it would work.
# ... then there's the hackish solution I put in place. What I like least
about this isn't the reflection, but the fact that it relies on grabbing the
vestigial {{collectAccSlot}} off the {{allBucketsAcc}} that has had its own
{{collectAcc}} set to null :-| ... but, as I said, it illustrates the problem,
and does seem to work.
> Improve JSON "terms" facet performance when sorted by relatedness
> --
>
> Key: SOLR-13132
> URL: https://issues.apache.org/jira/browse/SOLR-13132
> Project: Solr
> Issue Type: Improvement
> Components: Facet Module
>Affects Versions: 7.4, master (9.0)
>Reporter: Michael Gibney
>Priority: Major
> Attachments: SOLR-13132-with-cache-01.patch,
> SOLR-13132-with-cache.patch, SOLR-13132.patch, SOLR-13132_testSweep.patch
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> When sorting buckets by {{relatedness}}, JSON "terms" facet must calculate
> {{relatedness}} for every term.
> The current implementation uses a standard uninverted approach (either
> {{docValues}} or {{UnInvertedField}}) to get facet counts over the domain
> base docSet, and then uses that initial pass as a pre-filter for a
> second-pass, inverted approach of fetching docSets for each relevant term
> (i.e., {{count > minCount}}?) and calculating intersection size of those sets
> with the domain base docSet.
> Over high-cardinality fields, the overhead of per-term docSet creation and
> set intersection operations increases request latency to the point where
> relatedness sort may not be usable in practice (for my use case, even after
> applying the patch for SOLR-13108, for a field with ~220k unique terms per
> core, QTime for high-cardinality domain docSets were, e.g.: cardinality
> 1816684=9000ms, cardinality 5032902=18000ms).
> The attached patch brings the above example QTimes down to a manageable
> ~300ms and ~250ms respectively. The approach calculates uninverted facet
> counts over domain base, foreground, and background docSets in parallel in a
> single pass. This allows us to take advantage of the efficiencies built into
> the standard uninverted {{FacetFieldProcessorByArray[DV|UIF]}}), and avoids
> the per-term docSet creation and set intersection overhead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9392) Change the visibility of o.a.l.f.FacetsConfig.DELIM_CHAR to public
[ https://issues.apache.org/jira/browse/LUCENE-9392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130818#comment-17130818 ] Michael McCandless commented on LUCENE-9392: Thanks [~goankur], patch looks good – I'll tweak the wording a bit on the new javadoc and push soon. > Change the visibility of o.a.l.f.FacetsConfig.DELIM_CHAR to public > --- > > Key: LUCENE-9392 > URL: https://issues.apache.org/jira/browse/LUCENE-9392 > Project: Lucene - Core > Issue Type: Improvement >Affects Versions: 8.5.2 >Reporter: Ankur >Priority: Minor > Attachments: LUCENE-9392.patch > > > FacetsConfig.DELIM_CHAR is marked as private. An application that wants to > use this delimiter (in a unit test for example) is forced to re-declare it in > the application code. This can break the application if tetshe value of > DELIM_CHAR is changed in FacetsConfig -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-12382) new data not seen immediately after commit() on SolrCloud collection with only TLOG and PULL replicas
[ https://issues.apache.org/jira/browse/SOLR-12382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cassandra Targett resolved SOLR-12382. -- Resolution: Information Provided (was: Invalid) Marking as resolved since comments seem to indicate it can be closed. If there is a follow-on improvement, it could be filed as a new issue. > new data not seen immediately after commit() on SolrCloud collection with > only TLOG and PULL replicas > - > > Key: SOLR-12382 > URL: https://issues.apache.org/jira/browse/SOLR-12382 > Project: Solr > Issue Type: Bug >Affects Versions: 7.3 > Environment: SolrCloud on Amazon Linux AMI 2018.03 > >Reporter: Nguyen Nguyen >Priority: Major > > On collection with TLOG/PULL replicas, queries that follow right after > commit(waitSearch:true) would NOT return newly added data until several > seconds later. > Tested same scenario on another collection with only NRT replicas and found > that it behaved as expected (query returned newly added data right after > commit(waitSearch:true) was called. > 7.3 SolrCloud with 3 shards, each shard has 2 TLOG replicas + 1 PULL replica > Commit settings > > 15000 > false > > > -1 > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14409) Existing violations allow bypassing policy rules when adding new replicas
[
https://issues.apache.org/jira/browse/SOLR-14409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cassandra Targett updated SOLR-14409:
-
Fix Version/s: 8.6
> Existing violations allow bypassing policy rules when adding new replicas
> -
>
> Key: SOLR-14409
> URL: https://issues.apache.org/jira/browse/SOLR-14409
> Project: Solr
> Issue Type: Bug
> Components: AutoScaling
>Affects Versions: master (9.0), 8.5, 8.6
>Reporter: Andrzej Bialecki
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 8.6
>
> Attachments: SOLR-14409.patch
>
>
> Steps to reproduce:
> * start with an empty cluster policy.
> * create a collection with as many replicas as there are nodes.
> * add one more replica to any node. Now this node has two replicas, all
> other nodes have one.
> * define the following cluster policy:
> {code:java}
> { 'set-cluster-policy': [ {'replica': '<2', 'shard': '#ANY', 'node': '#ANY',
> 'strict': true} ] } {code}
> This automatically creates a violation because of the existing layout.
> * try adding one more replica. This should fail because no node satisfies
> the rules (there must be at most 1 replica per node). However, the command
> succeeds and adds replica to the node that already has 2 replicas, which
> clearly violates the policy and makes matters even worse.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] s1monw commented on a change in pull request #1552: LUCENE-8962
s1monw commented on a change in pull request #1552:
URL: https://github.com/apache/lucene-solr/pull/1552#discussion_r438153575
##
File path: lucene/core/src/java/org/apache/lucene/index/IndexWriter.java
##
@@ -4483,6 +4593,7 @@ public int length() {
// Merge would produce a 0-doc segment, so we do nothing except commit
the merge to remove all the 0-doc segments that we "merged":
assert merge.info.info.maxDoc() == 0;
commitMerge(merge, mergeState);
+success = true;
Review comment:
this seems unnecessary?
##
File path: lucene/core/src/java/org/apache/lucene/index/IndexWriter.java
##
@@ -3152,6 +3154,42 @@ public final boolean flushNextBuffer() throws
IOException {
}
}
+ private MergePolicy.OneMerge
updateSegmentInfosOnMergeFinish(MergePolicy.OneMerge merge, final SegmentInfos
toCommit,
+
AtomicReference mergeLatchRef) {
+return new MergePolicy.OneMerge(merge.segments) {
+ public void mergeFinished() throws IOException {
+super.mergeFinished();
+CountDownLatch mergeAwaitLatch = mergeLatchRef.get();
+if (mergeAwaitLatch == null) {
+ // Commit thread timed out waiting for this merge and moved on. No
need to manipulate toCommit.
+ return;
+}
+if (committed) {
+ deleter.incRef(this.info.files());
Review comment:
I am trying to understand why we need to do any extra work here. What
makes this special to any other merge such that we need to do all this work. If
this needs to be done only if we include this merged segment in the commit,
can't we do it outside of this mergeFinished and only use mergeFinished to
signal which merge finished in time etc? Then we also might not need the latch
construct and can use a simple callback that we can ignore on the commit end?
##
File path: lucene/core/src/java/org/apache/lucene/index/IndexWriter.java
##
@@ -3228,15 +3268,38 @@ private long prepareCommitInternal() throws IOException
{
// sneak into the commit point:
toCommit = segmentInfos.clone();
+ if (anyChanges) {
+// Find any merges that can execute on commit (per
MergePolicy).
+MergePolicy.MergeSpecification mergeSpec =
Review comment:
I do wonder if we need all these changes here in such a fragile part of
the code. Wouldn't it be possible to simply call `maybeMerge(MergePolicy
mergePolicy, MergeTrigger trigger, int maxNumSegments)` with a MergePolicy
wrapper that does all the magic like wrapping segments etc. Then we could pick
up the callback idea from above and just wait here until all merges called
back? I think we should try to reuse most of the current infrastructure in IW
instead of special casing. There was a lot of work put into this to reduce
special casing I think we should try hard to reduce it more and try harder to
not add any.
##
File path: lucene/core/src/java/org/apache/lucene/index/IndexWriter.java
##
@@ -3257,6 +3320,52 @@ private long prepareCommitInternal() throws IOException {
} finally {
maybeCloseOnTragicEvent();
}
+
+ if (mergeAwaitLatchRef != null) {
+CountDownLatch mergeAwaitLatch = mergeAwaitLatchRef.get();
+// If we found and registered any merges above, within the flushLock,
then we want to ensure that they
+// complete execution. Note that since we released the lock, other
merges may have been scheduled. We will
+// block until the merges that we registered complete. As they
complete, they will update toCommit to
+// replace merged segments with the result of each merge.
+config.getIndexWriterEvents().beginMergeOnCommit();
+mergeScheduler.merge(mergeSource, MergeTrigger.COMMIT);
+long mergeWaitStart = System.nanoTime();
+int abandonedCount = 0;
+long waitTimeMillis = (long) (config.getMaxCommitMergeWaitSeconds() *
1000.0);
+try {
+ if (mergeAwaitLatch.await(waitTimeMillis, TimeUnit.MILLISECONDS) ==
false) {
+synchronized (this) {
+ // Need to do this in a synchronized block, to make sure none of
our commit merges are currently
+ // executing mergeFinished (since mergeFinished itself is called
from within the IndexWriter lock).
+ // After we clear the value from mergeAwaitLatchRef, the merges
we schedule will still execute as
+ // usual, but when they finish, they won't attempt to update
toCommit or modify segment reference
+ // counts.
+ mergeAwaitLatchRef.set(null);
+ for (MergePolicy.OneMerge commitMerge : commitMerges) {
+if (runningMerges.contains(commitMerge) ||
pendingMerges.contains(commitMerge)) {
+ abandonedCount++;
+}
+
[GitHub] [lucene-solr] mikemccand commented on pull request #1543: LUCENE-9378: Disable compression on binary values whose length is less than 32.
mikemccand commented on pull request #1543: URL: https://github.com/apache/lucene-solr/pull/1543#issuecomment-642055625 @jpountz I am still confused about the above benchmarks. If baseline was master with `BINARY` compression reverted, contender is this PR (which adds compression but is more careful about when to compress), how can it be that faceting got so much faster? Shouldn't this PR only add (some, non-zero) cost? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9397) UniformSplit supports encodable fields metadata
[ https://issues.apache.org/jira/browse/LUCENE-9397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130720#comment-17130720 ] Bruno Roustant commented on LUCENE-9397: PR added > UniformSplit supports encodable fields metadata > --- > > Key: LUCENE-9397 > URL: https://issues.apache.org/jira/browse/LUCENE-9397 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Bruno Roustant >Assignee: Bruno Roustant >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > UniformSplit already supports custom encoding for term blocks. This is an > extension to also support encodable fields metadata. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14550) Atomic updates with add-distinct can add duplicates
[ https://issues.apache.org/jira/browse/SOLR-14550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Munendra S N updated SOLR-14550: Fix Version/s: 8.6 Resolution: Fixed Status: Resolved (was: Patch Available) > Atomic updates with add-distinct can add duplicates > --- > > Key: SOLR-14550 > URL: https://issues.apache.org/jira/browse/SOLR-14550 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Munendra S N >Assignee: Munendra S N >Priority: Major > Fix For: 8.6 > > Attachments: SOLR-14550.patch > > > When using add-distinct operation in atomic updates, it can end up adding > duplicates to the document. > The issue was reported in the > [user-list|https://lucene.472066.n3.nabble.com/Atomic-updates-with-add-distinct-in-Solr-7-cloud-td4458736.html] > According to the report, the issue doesn't occur in solr 8 or in standalone > mode but when going through the code, I came across a bug (maybe related to > it) which can lead to duplicates -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14345) Error messages are not properly propagated with non-default response parsers
[
https://issues.apache.org/jira/browse/SOLR-14345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Munendra S N updated SOLR-14345:
Fix Version/s: 8.6
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Error messages are not properly propagated with non-default response parsers
>
>
> Key: SOLR-14345
> URL: https://issues.apache.org/jira/browse/SOLR-14345
> Project: Solr
> Issue Type: Bug
>Reporter: Munendra S N
>Assignee: Munendra S N
>Priority: Major
> Fix For: 8.6
>
> Attachments: SOLR-14345.patch, SOLR-14345.patch, SOLR-14345.patch,
> SOLR-14345.patch, SOLR-14345.patch
>
>
> Default {{ResponsParseer}} is {{BinaryResponseParser}}. when non-default
> response parser is specified in the request then, the error message is
> propagated to user. This happens in solrCloud mode.
> I came across this problem when working on adding some test which uses
> {{SolrTestCaseHS}} but similar problem exists with SolrJ client
> Also, same problem exists in both HttpSolrClient and Http2SolrClient
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] bruno-roustant opened a new pull request #1564: LUCENE-9397: UniformSplit supports encodable fields metadata.
bruno-roustant opened a new pull request #1564: URL: https://github.com/apache/lucene-solr/pull/1564 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14550) Atomic updates with add-distinct can add duplicates
[ https://issues.apache.org/jira/browse/SOLR-14550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130715#comment-17130715 ] ASF subversion and git services commented on SOLR-14550: Commit a74de7f5a2f9fe1cd3024833c51f287125e54ea5 in lucene-solr's branch refs/heads/branch_8x from Munendra S N [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=a74de7f ] SOLR-14550: fix duplicate issue in Atomic updates with add-distinct * When add-distinct value is list, it can end up adding duplicate entries > Atomic updates with add-distinct can add duplicates > --- > > Key: SOLR-14550 > URL: https://issues.apache.org/jira/browse/SOLR-14550 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Munendra S N >Assignee: Munendra S N >Priority: Major > Attachments: SOLR-14550.patch > > > When using add-distinct operation in atomic updates, it can end up adding > duplicates to the document. > The issue was reported in the > [user-list|https://lucene.472066.n3.nabble.com/Atomic-updates-with-add-distinct-in-Solr-7-cloud-td4458736.html] > According to the report, the issue doesn't occur in solr 8 or in standalone > mode but when going through the code, I came across a bug (maybe related to > it) which can lead to duplicates -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14345) Error messages are not properly propagated with non-default response parsers
[
https://issues.apache.org/jira/browse/SOLR-14345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130716#comment-17130716
]
ASF subversion and git services commented on SOLR-14345:
Commit 6c77f4603cfd6d8566d4eae7d287e727edde78ed in lucene-solr's branch
refs/heads/branch_8x from Munendra S N
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=6c77f46 ]
SOLR-14345: return correct err msg when non-binary resp parser is used
* This adds support to parse error properly in case of non-binary
resp parser but the problem still exists for noopResponseParser
> Error messages are not properly propagated with non-default response parsers
>
>
> Key: SOLR-14345
> URL: https://issues.apache.org/jira/browse/SOLR-14345
> Project: Solr
> Issue Type: Bug
>Reporter: Munendra S N
>Assignee: Munendra S N
>Priority: Major
> Attachments: SOLR-14345.patch, SOLR-14345.patch, SOLR-14345.patch,
> SOLR-14345.patch, SOLR-14345.patch
>
>
> Default {{ResponsParseer}} is {{BinaryResponseParser}}. when non-default
> response parser is specified in the request then, the error message is
> propagated to user. This happens in solrCloud mode.
> I came across this problem when working on adding some test which uses
> {{SolrTestCaseHS}} but similar problem exists with SolrJ client
> Also, same problem exists in both HttpSolrClient and Http2SolrClient
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mikemccand commented on a change in pull request #1560: LUCENE-8574: Upgrade HPPC to 0.8.2
mikemccand commented on a change in pull request #1560: URL: https://github.com/apache/lucene-solr/pull/1560#discussion_r438144513 ## File path: lucene/ivy-versions.properties ## @@ -10,7 +10,7 @@ com.carrotsearch.randomizedtesting.version = 2.7.6 /com.carrotsearch.thirdparty/simple-xml-safe = 2.7.1 -/com.carrotsearch/hppc = 0.8.1 +/com.carrotsearch/hppc = 0.8.2 Review comment: Does this then mean that Lucene's `facet` module's usage of HPPC is upgraded too? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14550) Atomic updates with add-distinct can add duplicates
[ https://issues.apache.org/jira/browse/SOLR-14550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130701#comment-17130701 ] ASF subversion and git services commented on SOLR-14550: Commit 001c4e2ddd515c47ec842efadad3e56c4e648c61 in lucene-solr's branch refs/heads/master from Munendra S N [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=001c4e2 ] SOLR-14550: fix duplicate issue in Atomic updates with add-distinct * When add-distinct value is list, it can end up adding duplicate entries > Atomic updates with add-distinct can add duplicates > --- > > Key: SOLR-14550 > URL: https://issues.apache.org/jira/browse/SOLR-14550 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Munendra S N >Assignee: Munendra S N >Priority: Major > Attachments: SOLR-14550.patch > > > When using add-distinct operation in atomic updates, it can end up adding > duplicates to the document. > The issue was reported in the > [user-list|https://lucene.472066.n3.nabble.com/Atomic-updates-with-add-distinct-in-Solr-7-cloud-td4458736.html] > According to the report, the issue doesn't occur in solr 8 or in standalone > mode but when going through the code, I came across a bug (maybe related to > it) which can lead to duplicates -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14345) Error messages are not properly propagated with non-default response parsers
[
https://issues.apache.org/jira/browse/SOLR-14345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130702#comment-17130702
]
ASF subversion and git services commented on SOLR-14345:
Commit 53040984f62eedd425c98eb6f60bc54a0e83258b in lucene-solr's branch
refs/heads/master from Munendra S N
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=5304098 ]
SOLR-14345: return correct err msg when non-binary resp parser is used
* This adds support to parse error properly in case of non-binary
resp parser but the problem still exists for noopResponseParser
> Error messages are not properly propagated with non-default response parsers
>
>
> Key: SOLR-14345
> URL: https://issues.apache.org/jira/browse/SOLR-14345
> Project: Solr
> Issue Type: Bug
>Reporter: Munendra S N
>Assignee: Munendra S N
>Priority: Major
> Attachments: SOLR-14345.patch, SOLR-14345.patch, SOLR-14345.patch,
> SOLR-14345.patch, SOLR-14345.patch
>
>
> Default {{ResponsParseer}} is {{BinaryResponseParser}}. when non-default
> response parser is specified in the request then, the error message is
> propagated to user. This happens in solrCloud mode.
> I came across this problem when working on adding some test which uses
> {{SolrTestCaseHS}} but similar problem exists with SolrJ client
> Also, same problem exists in both HttpSolrClient and Http2SolrClient
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14556) Fix or suppress warnings in solrj/cloud/autoscaling
[ https://issues.apache.org/jira/browse/SOLR-14556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Erickson updated SOLR-14556: -- Summary: Fix or suppress warnings in solrj/cloud/autoscaling (was: Fix or suppress warnings in solrj/src/java/org/apache/solr/client/solrj) > Fix or suppress warnings in solrj/cloud/autoscaling > --- > > Key: SOLR-14556 > URL: https://issues.apache.org/jira/browse/SOLR-14556 > Project: Solr > Issue Type: Sub-task >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14541) Ensure classes that implement equals implement hashCode or suppress warnings
[ https://issues.apache.org/jira/browse/SOLR-14541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130681#comment-17130681 ] Erick Erickson commented on SOLR-14541: --- Yeah, that's a good idea for at least two reasons: 1> relying on "I think" is soft. and 2> future proofing. I'll do that on anything remaining before closing this. > Ensure classes that implement equals implement hashCode or suppress warnings > > > Key: SOLR-14541 > URL: https://issues.apache.org/jira/browse/SOLR-14541 > Project: Solr > Issue Type: Sub-task >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Attachments: 0001-SOLR-14541-add-hashCode-for-some-classes.patch, > 0002-SOLR-14541-add-hashCode-for-some-classes-in-autoscal.patch, > 0003-SOLR-14541-add-hashCode-or-remove-equals-for-some-cl.patch > > > While looking at warnings, I found that the following classes generate this > warning: > *overrides equals, but neither it nor any superclass overrides hashCode > method* > I can suppress the warning, but this has been a source of errors in the past > so I'm reluctant to just do that blindly. > NOTE: The Lucene one should probably be it's own Jira if it's going to have > hashCode implemented, but here for triage. > What I need for each method is for someone who has a clue about that > particular code to render an opinion that we can safely suppress the warning > or to provide a hashCode method. > Some of these have been here for a very long time and were implemented by > people no longer active... > lucene/suggest/src/java/org/apache/lucene/search/spell/LuceneLevenshteinDistance.java:39 > solr/solrj/src/java/org/apache/solr/common/cloud/ZkNodeProps.java:34 > solr/solrj/src/java/org/apache/solr/common/cloud/Replica.java:26 > solr/solrj/src/java/org/apache/solr/common/cloud/DocCollection.java:49 > solr/core/src/java/org/apache/solr/cloud/rule/Rule.java:277 > solr/core/src/java/org/apache/solr/pkg/PackageAPI.java:177 > solr/core/src/java/org/apache/solr/packagemanager/SolrPackageInstance.java:31 > > Noble Paul says it's OK to suppress warnings for these: > solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/VersionedData.java:31 > > solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/AutoScalingConfig.java:61 > > solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/AutoScalingConfig.java:150 > > solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/AutoScalingConfig.java:252 > > solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/AutoScalingConfig.java:45 > > solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/Policy.java:73 > > solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/Preference.java:32 > > solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/ReplicaInfo.java:39 > > Joel Bernstein says it's OK to suppress warnings for these: > > solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/ReplicaCount.java:27 > > solr/solrj/src/java/org/apache/solr/client/solrj/io/stream/expr/StreamExpression.java:25 > > solr/solrj/src/java/org/apache/solr/client/solrj/io/stream/expr/StreamExpressionNamedParameter.java:23 > > solr/solrj/src/java/org/apache/solr/client/solrj/io/stream/CloudSolrStream.java:467 > > solr/solrj/src/java/org/apache/solr/client/solrj/io/stream/DeepRandomStream.java:417 > > solr/solrj/src/java/org/apache/solr/client/solrj/io/stream/expr/StreamExpressionValue.java:22 > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9394) Fix or suppress compile-time warnings
[ https://issues.apache.org/jira/browse/LUCENE-9394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130679#comment-17130679 ] Erick Erickson commented on LUCENE-9394: Adding link to the deprecations JIRA. > Fix or suppress compile-time warnings > - > > Key: LUCENE-9394 > URL: https://issues.apache.org/jira/browse/LUCENE-9394 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Michael Sokolov >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > This is a spinoff from [~erickerickson]'s efforts over in SOLR-10778 > The goal is a warning-free compilation, followed by enforcement of build > failure on warnings, with the idea of suppressing innocuous warnings to the > extent that the remaining warnings be treated as build failure. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14541) Ensure classes that implement equals implement hashCode or suppress warnings
[ https://issues.apache.org/jira/browse/SOLR-14541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gézapeti updated SOLR-14541: Attachment: 0003-SOLR-14541-add-hashCode-or-remove-equals-for-some-cl.patch 0002-SOLR-14541-add-hashCode-for-some-classes-in-autoscal.patch 0001-SOLR-14541-add-hashCode-for-some-classes.patch > Ensure classes that implement equals implement hashCode or suppress warnings > > > Key: SOLR-14541 > URL: https://issues.apache.org/jira/browse/SOLR-14541 > Project: Solr > Issue Type: Sub-task >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Attachments: 0001-SOLR-14541-add-hashCode-for-some-classes.patch, > 0002-SOLR-14541-add-hashCode-for-some-classes-in-autoscal.patch, > 0003-SOLR-14541-add-hashCode-or-remove-equals-for-some-cl.patch > > > While looking at warnings, I found that the following classes generate this > warning: > *overrides equals, but neither it nor any superclass overrides hashCode > method* > I can suppress the warning, but this has been a source of errors in the past > so I'm reluctant to just do that blindly. > NOTE: The Lucene one should probably be it's own Jira if it's going to have > hashCode implemented, but here for triage. > What I need for each method is for someone who has a clue about that > particular code to render an opinion that we can safely suppress the warning > or to provide a hashCode method. > Some of these have been here for a very long time and were implemented by > people no longer active... > lucene/suggest/src/java/org/apache/lucene/search/spell/LuceneLevenshteinDistance.java:39 > solr/solrj/src/java/org/apache/solr/common/cloud/ZkNodeProps.java:34 > solr/solrj/src/java/org/apache/solr/common/cloud/Replica.java:26 > solr/solrj/src/java/org/apache/solr/common/cloud/DocCollection.java:49 > solr/core/src/java/org/apache/solr/cloud/rule/Rule.java:277 > solr/core/src/java/org/apache/solr/pkg/PackageAPI.java:177 > solr/core/src/java/org/apache/solr/packagemanager/SolrPackageInstance.java:31 > > Noble Paul says it's OK to suppress warnings for these: > solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/VersionedData.java:31 > > solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/AutoScalingConfig.java:61 > > solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/AutoScalingConfig.java:150 > > solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/AutoScalingConfig.java:252 > > solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/AutoScalingConfig.java:45 > > solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/Policy.java:73 > > solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/Preference.java:32 > > solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/ReplicaInfo.java:39 > > Joel Bernstein says it's OK to suppress warnings for these: > > solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/ReplicaCount.java:27 > > solr/solrj/src/java/org/apache/solr/client/solrj/io/stream/expr/StreamExpression.java:25 > > solr/solrj/src/java/org/apache/solr/client/solrj/io/stream/expr/StreamExpressionNamedParameter.java:23 > > solr/solrj/src/java/org/apache/solr/client/solrj/io/stream/CloudSolrStream.java:467 > > solr/solrj/src/java/org/apache/solr/client/solrj/io/stream/DeepRandomStream.java:417 > > solr/solrj/src/java/org/apache/solr/client/solrj/io/stream/expr/StreamExpressionValue.java:22 > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14469) Removed deprecated code in solr/core (master only)
[ https://issues.apache.org/jira/browse/SOLR-14469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130678#comment-17130678 ] Erick Erickson commented on SOLR-14469: --- There's some discussion at: LUCENE-9394... > Removed deprecated code in solr/core (master only) > -- > > Key: SOLR-14469 > URL: https://issues.apache.org/jira/browse/SOLR-14469 > Project: Solr > Issue Type: Sub-task >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > > I'm currently working on getting all the warnings out of the code, so this is > something of a placeholder for a week or two. > There will be sub-tasks, please create them when you start working on a > project. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14541) Ensure classes that implement equals implement hashCode or suppress warnings
[ https://issues.apache.org/jira/browse/SOLR-14541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130677#comment-17130677 ] Gézapeti commented on SOLR-14541: - I'd +1 [~murblanc]'s suggestion. If we implement a hashcode that just returns 0 or throws an exception we could avoid a nasty bug in the future. I'm uploading 3 set of patches here: one for autoscaling, one for streaming and one for the rest. I've added trivial definitions for hashcode or removed the equals when they were no-op. > Ensure classes that implement equals implement hashCode or suppress warnings > > > Key: SOLR-14541 > URL: https://issues.apache.org/jira/browse/SOLR-14541 > Project: Solr > Issue Type: Sub-task >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > > While looking at warnings, I found that the following classes generate this > warning: > *overrides equals, but neither it nor any superclass overrides hashCode > method* > I can suppress the warning, but this has been a source of errors in the past > so I'm reluctant to just do that blindly. > NOTE: The Lucene one should probably be it's own Jira if it's going to have > hashCode implemented, but here for triage. > What I need for each method is for someone who has a clue about that > particular code to render an opinion that we can safely suppress the warning > or to provide a hashCode method. > Some of these have been here for a very long time and were implemented by > people no longer active... > lucene/suggest/src/java/org/apache/lucene/search/spell/LuceneLevenshteinDistance.java:39 > solr/solrj/src/java/org/apache/solr/common/cloud/ZkNodeProps.java:34 > solr/solrj/src/java/org/apache/solr/common/cloud/Replica.java:26 > solr/solrj/src/java/org/apache/solr/common/cloud/DocCollection.java:49 > solr/core/src/java/org/apache/solr/cloud/rule/Rule.java:277 > solr/core/src/java/org/apache/solr/pkg/PackageAPI.java:177 > solr/core/src/java/org/apache/solr/packagemanager/SolrPackageInstance.java:31 > > Noble Paul says it's OK to suppress warnings for these: > solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/VersionedData.java:31 > > solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/AutoScalingConfig.java:61 > > solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/AutoScalingConfig.java:150 > > solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/AutoScalingConfig.java:252 > > solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/AutoScalingConfig.java:45 > > solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/Policy.java:73 > > solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/Preference.java:32 > > solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/ReplicaInfo.java:39 > > Joel Bernstein says it's OK to suppress warnings for these: > > solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/ReplicaCount.java:27 > > solr/solrj/src/java/org/apache/solr/client/solrj/io/stream/expr/StreamExpression.java:25 > > solr/solrj/src/java/org/apache/solr/client/solrj/io/stream/expr/StreamExpressionNamedParameter.java:23 > > solr/solrj/src/java/org/apache/solr/client/solrj/io/stream/CloudSolrStream.java:467 > > solr/solrj/src/java/org/apache/solr/client/solrj/io/stream/DeepRandomStream.java:417 > > solr/solrj/src/java/org/apache/solr/client/solrj/io/stream/expr/StreamExpressionValue.java:22 > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9394) Fix or suppress compile-time warnings
[ https://issues.apache.org/jira/browse/LUCENE-9394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130627#comment-17130627 ] Michael Sokolov commented on LUCENE-9394: - > did you know you can add that annotation on a parameter to a method? Oh! I did not I hear you about failing on deprecations, however ... sometimes I think we may deprecate something aspirationally. We don't like the class/method, but offer no functionally equivalent replacement. Probably we should just not do that. Maybe if we had stricter enforcement we'd think harder before slapping a Deprecated tag on something... > Fix or suppress compile-time warnings > - > > Key: LUCENE-9394 > URL: https://issues.apache.org/jira/browse/LUCENE-9394 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Michael Sokolov >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > This is a spinoff from [~erickerickson]'s efforts over in SOLR-10778 > The goal is a warning-free compilation, followed by enforcement of build > failure on warnings, with the idea of suppressing innocuous warnings to the > extent that the remaining warnings be treated as build failure. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9394) Fix or suppress compile-time warnings
[ https://issues.apache.org/jira/browse/LUCENE-9394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130618#comment-17130618 ] Erick Erickson commented on LUCENE-9394: re: Changing ValueSource context to Map Without really looking at the implications, we'll have to do some surgery like this to really get rid of SuppressWarnings and I'm for it. I've seen a lot of these in Solr code. I also encourage everyone who uses IntelliJ to pay attention to the highlighted code. Some of the inspections are trivial and I want to turn them off personally. For instance, I like "if (something == false)" rather than "if (! something)" but that's a nit. IntelliJ inspections are relevant to your question though, since if this change is made many of the suppressions in the Solr code will become obsolete, and IntelliJ flags them. While on the subject_,_ I'd also advise against letting IntelliJ fix issues, or at least inspect the fix. IntelliJ fixes the _inspection_, which is different than fixing compiler warnings. We can't restrict this to a particular IDE. Hmmm, I feel another precommit check coming on... > Fix or suppress compile-time warnings > - > > Key: LUCENE-9394 > URL: https://issues.apache.org/jira/browse/LUCENE-9394 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Michael Sokolov >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > This is a spinoff from [~erickerickson]'s efforts over in SOLR-10778 > The goal is a warning-free compilation, followed by enforcement of build > failure on warnings, with the idea of suppressing innocuous warnings to the > extent that the remaining warnings be treated as build failure. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9394) Fix or suppress compile-time warnings
[ https://issues.apache.org/jira/browse/LUCENE-9394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130608#comment-17130608 ] Erick Erickson commented on LUCENE-9394: Glad you’re looking at this. If you think some of the Lucene code is “filthy with SuppressWarnings”, you should see some of the code I’m checking in ;). I’ve made a conscious decision to make it ugly (trying to suppress warnings on as small a unit as possible rather than whole classes and the like; did you know you can add that annotation on a parameter to a method?) purposely. Maybe some of it’ll be fixed by-the-by if it offends peoples sense of tidiness. You’re going deeper than I can afford to given the number of warnings in Solr, but this is where I’d eventually like to be. I’m sounding like a broken record, but my goal for the Solr code short-term is just to have a chance to stop getting _worse_. About deprecation warnings. Admittedly I haven’t started looking at deprecations yet. On the surface, though, we ought to be able to remove deprecated code that was introduced X-2 ago. Or, if the intent of the deprecation was to make something private, say, make it private and remove the annotation. As far as our internal code is concerned, why can’t we fix any deprecation usages when they’re introduced? Ideally I’d like to fail compilation on deprecation usage as a general rule. I’m sure there’d be exceptions, particularly with code we don’t control but I suspect that could be worked around. Again ideally, I’d like precommit to fail unless there were two things in deprecation annotations: 1> a note about what to use instead and 2> a version stamp for _when_ it was deprecated. If that were in place, we could start failing precommit if the current version were X+2 from the version stamp in the deprecation annotation. IDK whether that’s practical, master and the current release tree might diverge too much. I’ll be opening a JIRA to discuss what to do when I get to the point. Which ought not to be too long, I’m ripping through the Solr warnings pretty quickly since what I’m doing is so simple. I’ll look forward to your input on that JIRA now that you’ve worked through one set of changes and have more “eyes on” than I have. Erick > Fix or suppress compile-time warnings > - > > Key: LUCENE-9394 > URL: https://issues.apache.org/jira/browse/LUCENE-9394 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Michael Sokolov >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > This is a spinoff from [~erickerickson]'s efforts over in SOLR-10778 > The goal is a warning-free compilation, followed by enforcement of build > failure on warnings, with the idea of suppressing innocuous warnings to the > extent that the remaining warnings be treated as build failure. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9394) Fix or suppress compile-time warnings
[ https://issues.apache.org/jira/browse/LUCENE-9394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130600#comment-17130600 ] Michael Sokolov commented on LUCENE-9394: - So, I tinkered with changing the raw Map type used for ValueSource "context". If we change it to Map we can get rid of most warnings, but we still occasionally put things in the context map, and when we do that we have to unchecked cast it to something, so we get warnings for that. On the other hand, there would be no impact to callers. To completely eliminate warnings in Lucene code related to this API, we could change the signature to Map, which in fact is what we require, since we sometimes stick things in the context, and we also allow callers to put whatever they want in it? If we make that change, I think it would be more correct, but there would be more of a burden on users of the API who would now need to possibly change the generic Map types of context maps they create, or else deal with unchecked casts of their own. Any opinions? > Fix or suppress compile-time warnings > - > > Key: LUCENE-9394 > URL: https://issues.apache.org/jira/browse/LUCENE-9394 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Michael Sokolov >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > This is a spinoff from [~erickerickson]'s efforts over in SOLR-10778 > The goal is a warning-free compilation, followed by enforcement of build > failure on warnings, with the idea of suppressing innocuous warnings to the > extent that the remaining warnings be treated as build failure. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14469) Removed deprecated code in solr/core (master only)
[
https://issues.apache.org/jira/browse/SOLR-14469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130595#comment-17130595
]
Erick Erickson commented on SOLR-14469:
---
{quote}I'm just unsure of the value of trying to automate this best practice to
insist on it.
{quote}
Well, moral 'suasion and nagging don't seem to be helping us clean up code. We
have deprecations here from _at least_ as far back as 6.0. Failing hard would
help us cut down on cruft.
Admittedly it's a harsh cure, but what we're doing now isn't working, doing
more of the same doesn't strike me as a way to move forward.
> Removed deprecated code in solr/core (master only)
> --
>
> Key: SOLR-14469
> URL: https://issues.apache.org/jira/browse/SOLR-14469
> Project: Solr
> Issue Type: Sub-task
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
>
> I'm currently working on getting all the warnings out of the code, so this is
> something of a placeholder for a week or two.
> There will be sub-tasks, please create them when you start working on a
> project.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14347) Autoscaling placement wrong when concurrent replica placements are calculated
[
https://issues.apache.org/jira/browse/SOLR-14347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130573#comment-17130573
]
ASF subversion and git services commented on SOLR-14347:
Commit b8977d4dde560560eb17a362f3bfc03ab5705749 in lucene-solr's branch
refs/heads/branch_8x from murblanc
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=b8977d4 ]
SOLR-14347: fix cached session update to not depend on Zookeeper state
> Autoscaling placement wrong when concurrent replica placements are calculated
> -
>
> Key: SOLR-14347
> URL: https://issues.apache.org/jira/browse/SOLR-14347
> Project: Solr
> Issue Type: Bug
> Components: AutoScaling
>Affects Versions: 8.5
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Critical
> Fix For: 8.6
>
> Attachments: SOLR-14347.patch
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Steps to reproduce:
> * create a cluster of a few nodes (tested with 7 nodes)
> * define per-collection policies that distribute replicas exclusively on
> different nodes per policy
> * concurrently create a few collections, each using a different policy
> * resulting replica placement will be seriously wrong, causing many policy
> violations
> Running the same scenario but instead creating collections sequentially
> results in no violations.
> I suspect this is caused by incorrect locking level for all collection
> operations (as defined in {{CollectionParams.CollectionAction}}) that create
> new replica placements - i.e. CREATE, ADDREPLICA, MOVEREPLICA, DELETENODE,
> REPLACENODE, SPLITSHARD, RESTORE, REINDEXCOLLECTION. All of these operations
> use the policy engine to create new replica placements, and as a result they
> change the cluster state. However, currently these operations are locked (in
> {{OverseerCollectionMessageHandler.lockTask}} ) using
> {{LockLevel.COLLECTION}}. In practice this means that the lock is held only
> for the particular collection that is being modified.
> A straightforward fix for this issue is to change the locking level to
> CLUSTER (and I confirm this fixes the scenario described above). However,
> this effectively serializes all collection operations listed above, which
> will result in general slow-down of all collection operations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-14347) Autoscaling placement wrong when concurrent replica placements are calculated
[
https://issues.apache.org/jira/browse/SOLR-14347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Noble Paul resolved SOLR-14347.
---
Resolution: Fixed
> Autoscaling placement wrong when concurrent replica placements are calculated
> -
>
> Key: SOLR-14347
> URL: https://issues.apache.org/jira/browse/SOLR-14347
> Project: Solr
> Issue Type: Bug
> Components: AutoScaling
>Affects Versions: 8.5
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Critical
> Fix For: 8.6
>
> Attachments: SOLR-14347.patch
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Steps to reproduce:
> * create a cluster of a few nodes (tested with 7 nodes)
> * define per-collection policies that distribute replicas exclusively on
> different nodes per policy
> * concurrently create a few collections, each using a different policy
> * resulting replica placement will be seriously wrong, causing many policy
> violations
> Running the same scenario but instead creating collections sequentially
> results in no violations.
> I suspect this is caused by incorrect locking level for all collection
> operations (as defined in {{CollectionParams.CollectionAction}}) that create
> new replica placements - i.e. CREATE, ADDREPLICA, MOVEREPLICA, DELETENODE,
> REPLACENODE, SPLITSHARD, RESTORE, REINDEXCOLLECTION. All of these operations
> use the policy engine to create new replica placements, and as a result they
> change the cluster state. However, currently these operations are locked (in
> {{OverseerCollectionMessageHandler.lockTask}} ) using
> {{LockLevel.COLLECTION}}. In practice this means that the lock is held only
> for the particular collection that is being modified.
> A straightforward fix for this issue is to change the locking level to
> CLUSTER (and I confirm this fixes the scenario described above). However,
> this effectively serializes all collection operations listed above, which
> will result in general slow-down of all collection operations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] msokolov opened a new pull request #1563: LUCENE-9394: fix and suppress warnings
msokolov opened a new pull request #1563: URL: https://github.com/apache/lucene-solr/pull/1563 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14347) Autoscaling placement wrong when concurrent replica placements are calculated
[
https://issues.apache.org/jira/browse/SOLR-14347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130527#comment-17130527
]
ASF subversion and git services commented on SOLR-14347:
Commit d4f7c90bab3d3dd76c066aeb598544cf8250bac8 in lucene-solr's branch
refs/heads/master from murblanc
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=d4f7c90 ]
SOLR-14347: fix cached session update to not depend on Zookeeper state (#1542)
SOLR-14347: fix cached session update to not depend on Zookeeper state
> Autoscaling placement wrong when concurrent replica placements are calculated
> -
>
> Key: SOLR-14347
> URL: https://issues.apache.org/jira/browse/SOLR-14347
> Project: Solr
> Issue Type: Bug
> Components: AutoScaling
>Affects Versions: 8.5
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Critical
> Fix For: 8.6
>
> Attachments: SOLR-14347.patch
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Steps to reproduce:
> * create a cluster of a few nodes (tested with 7 nodes)
> * define per-collection policies that distribute replicas exclusively on
> different nodes per policy
> * concurrently create a few collections, each using a different policy
> * resulting replica placement will be seriously wrong, causing many policy
> violations
> Running the same scenario but instead creating collections sequentially
> results in no violations.
> I suspect this is caused by incorrect locking level for all collection
> operations (as defined in {{CollectionParams.CollectionAction}}) that create
> new replica placements - i.e. CREATE, ADDREPLICA, MOVEREPLICA, DELETENODE,
> REPLACENODE, SPLITSHARD, RESTORE, REINDEXCOLLECTION. All of these operations
> use the policy engine to create new replica placements, and as a result they
> change the cluster state. However, currently these operations are locked (in
> {{OverseerCollectionMessageHandler.lockTask}} ) using
> {{LockLevel.COLLECTION}}. In practice this means that the lock is held only
> for the particular collection that is being modified.
> A straightforward fix for this issue is to change the locking level to
> CLUSTER (and I confirm this fixes the scenario described above). However,
> this effectively serializes all collection operations listed above, which
> will result in general slow-down of all collection operations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14347) Autoscaling placement wrong when concurrent replica placements are calculated
[
https://issues.apache.org/jira/browse/SOLR-14347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130526#comment-17130526
]
ASF subversion and git services commented on SOLR-14347:
Commit d4f7c90bab3d3dd76c066aeb598544cf8250bac8 in lucene-solr's branch
refs/heads/master from murblanc
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=d4f7c90 ]
SOLR-14347: fix cached session update to not depend on Zookeeper state (#1542)
SOLR-14347: fix cached session update to not depend on Zookeeper state
> Autoscaling placement wrong when concurrent replica placements are calculated
> -
>
> Key: SOLR-14347
> URL: https://issues.apache.org/jira/browse/SOLR-14347
> Project: Solr
> Issue Type: Bug
> Components: AutoScaling
>Affects Versions: 8.5
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Critical
> Fix For: 8.6
>
> Attachments: SOLR-14347.patch
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Steps to reproduce:
> * create a cluster of a few nodes (tested with 7 nodes)
> * define per-collection policies that distribute replicas exclusively on
> different nodes per policy
> * concurrently create a few collections, each using a different policy
> * resulting replica placement will be seriously wrong, causing many policy
> violations
> Running the same scenario but instead creating collections sequentially
> results in no violations.
> I suspect this is caused by incorrect locking level for all collection
> operations (as defined in {{CollectionParams.CollectionAction}}) that create
> new replica placements - i.e. CREATE, ADDREPLICA, MOVEREPLICA, DELETENODE,
> REPLACENODE, SPLITSHARD, RESTORE, REINDEXCOLLECTION. All of these operations
> use the policy engine to create new replica placements, and as a result they
> change the cluster state. However, currently these operations are locked (in
> {{OverseerCollectionMessageHandler.lockTask}} ) using
> {{LockLevel.COLLECTION}}. In practice this means that the lock is held only
> for the particular collection that is being modified.
> A straightforward fix for this issue is to change the locking level to
> CLUSTER (and I confirm this fixes the scenario described above). However,
> this effectively serializes all collection operations listed above, which
> will result in general slow-down of all collection operations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] noblepaul merged pull request #1542: SOLR-14347: fix cached session update to not depend on Zookeeper state
noblepaul merged pull request #1542: URL: https://github.com/apache/lucene-solr/pull/1542 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] noblepaul commented on a change in pull request #1542: SOLR-14347: fix cached session update to not depend on Zookeeper state
noblepaul commented on a change in pull request #1542:
URL: https://github.com/apache/lucene-solr/pull/1542#discussion_r438043059
##
File path:
solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/Policy.java
##
@@ -644,15 +643,13 @@ public Session cloneToNewSession(SolrCloudManager
cloudManager) {
List expandedClauses = new ArrayList<>(this.expandedClauses);
List matrix = new ArrayList<>(nodes.size());
+ Map copyNodes = new HashMap<>();
+ for (Row oldRow: this.matrix) {
Review comment:
>even those that might not be needed
I believe it's a reasonable tradeoff because most of the times nodes do not
go away. Yes, we make an extra copy in those rare occasions, but it should be
OK.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc commented on pull request #1542: SOLR-14347: fix cached session update to not depend on Zookeeper state
murblanc commented on pull request #1542: URL: https://github.com/apache/lucene-solr/pull/1542#issuecomment-641927858 > @murblanc > I've fixed a few things pls review & I can merge this Added one comment. Looks good, thanks for picking this one up. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9394) Fix or suppress compile-time warnings
[
https://issues.apache.org/jira/browse/LUCENE-9394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130513#comment-17130513
]
Michael Sokolov commented on LUCENE-9394:
-
I did a round of this, will post a PR soon, but I just want to note what would
be required to get to zero warnings. For the most part, it seems innocuous to
me, but there were some changes I thought would be worth surfacing. Here's a
summary of what needs doing:
# The spatial-extras package is simply riddled with easily-fixed deprecation
warnings. Change the usage there to conform to the new API in spatial4j.
# Update usage of newer version of randomizedtesting API
# Add @Deprecated annotations to tests of deprecated classes
# In a few cases, replace usage of deprecated (internal Lucene) classes or
methods with their more modern equivalents. EG: SynonymGraphFilter for
SynonymFilter in tests where it is possible.
# Replace static imports of constants from deprecated classes with qualified
references, assigning to a locally-defined constant in some cases (eg instead
of importing WordDelimiterFilter.*, import WordDelimiter and refer to constants
as WordDelimiter.XXX).
# Replace usage of junit.framework.Assert with org.junit.Assert. Is this
controversial? I don't know the history here.
# Add SuppressWarnings("deprecation") annotations as needed for the rest of
the deprecation issues. These mostly have to with *our own* deprecations. I
don't think these warnings are providing any value. The process of removing a
deprecated class or method is likely going to be a full-sweep top-down effort
(like this one) - the warnings about usage of deprecated classes/methods
(mostly in tests) are not going to be the thing that prompts us to do that.
Additionally I addressed a number of generics warnings, fixing or suppressing
unchecked casts and/or raw types. One thing I did not address is the use of the
raw Map type in ValueSource, which causes lots of warnings. I wanted to ask
here if people think it is worth switching to use Map in the signature of
ValueSource and its subclasses? I suppose that could have a pretty big ripple
effect? But the alternative is an API that is simply filthy with
SuppressWarnings("rawtypes").
> Fix or suppress compile-time warnings
> -
>
> Key: LUCENE-9394
> URL: https://issues.apache.org/jira/browse/LUCENE-9394
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Michael Sokolov
>Priority: Major
>
> This is a spinoff from [~erickerickson]'s efforts over in SOLR-10778
> The goal is a warning-free compilation, followed by enforcement of build
> failure on warnings, with the idea of suppressing innocuous warnings to the
> extent that the remaining warnings be treated as build failure.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc commented on a change in pull request #1542: SOLR-14347: fix cached session update to not depend on Zookeeper state
murblanc commented on a change in pull request #1542:
URL: https://github.com/apache/lucene-solr/pull/1542#discussion_r438038559
##
File path:
solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/Policy.java
##
@@ -644,15 +643,13 @@ public Session cloneToNewSession(SolrCloudManager
cloudManager) {
List expandedClauses = new ArrayList<>(this.expandedClauses);
List matrix = new ArrayList<>(nodes.size());
+ Map copyNodes = new HashMap<>();
+ for (Row oldRow: this.matrix) {
Review comment:
Tradeoff with this one time matrix walk and copy is that we copy all
nodes, even those that might not be needed (because not present in parameter
`nodes`). I do like the approach though it's cleaner than the code I wrote.
Maybe keep this initial loop but just store Row references, then do the copy
below when newRow is needed? (there's already an if bloc when it's null, so
copy action can easily be placed in else bloc).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9353) Move metadata of the terms dictionary to its own file
[ https://issues.apache.org/jira/browse/LUCENE-9353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130389#comment-17130389 ] Adrien Grand commented on LUCENE-9353: -- I went a bit further on the PR and also moved metadata of the FST to the metadata file too. This helps avoid doing O(numFields) seeks when opening a reader, like we did for points at LUCENE-9148. > Move metadata of the terms dictionary to its own file > - > > Key: LUCENE-9353 > URL: https://issues.apache.org/jira/browse/LUCENE-9353 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Adrien Grand >Priority: Minor > Time Spent: 1h 40m > Remaining Estimate: 0h > > Currently opening a terms index requires jumping to the end of the terms > index and terms dictionaries to decode some metadata such as sumTtf or file > pointers where information for a given field is located. It'd be nicer to > have it in a separate file, which would also have the benefit of letting us > verify checksums for this part of the content. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] iverase commented on pull request #1562: LUCENE-9400: Tessallator might fail when several holes share the same vertex
iverase commented on pull request #1562: URL: https://github.com/apache/lucene-solr/pull/1562#issuecomment-641823144 @nknize Can you have a look? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] iverase opened a new pull request #1562: LUCENE-9400: Tessallator might fail when several holes share the same vertex
iverase opened a new pull request #1562: URL: https://github.com/apache/lucene-solr/pull/1562 Make sure we do not build illegal polygons when merging holes that share a single vertex. see https://issues.apache.org/jira/browse/LUCENE-9400 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on pull request #1560: LUCENE-8574: Upgrade HPPC to 0.8.2
dweiss commented on pull request #1560: URL: https://github.com/apache/lucene-solr/pull/1560#issuecomment-641821902 Hi Patrick. Would you add a corresponding CHANGES entry (to both Solr and Lucene, I guess) and attribution to your name? Otherwise LGTM. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (LUCENE-9400) Tessallator might fail when several holes share the same vertex
Ignacio Vera created LUCENE-9400: Summary: Tessallator might fail when several holes share the same vertex Key: LUCENE-9400 URL: https://issues.apache.org/jira/browse/LUCENE-9400 Project: Lucene - Core Issue Type: Bug Reporter: Ignacio Vera Attachments: image-2020-06-10-09-54-46-316.png We found a new case where tessellation might fail on a valid polygon. The issue shows when a polygon has several holes sharing the same vertex. In this case the merging logic for holes might fail creating an invalid polygon. For example a polygon like this one: !image-2020-06-10-09-54-46-316.png|width=260,height=266! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] noblepaul commented on pull request #1542: SOLR-14347: fix cached session update to not depend on Zookeeper state
noblepaul commented on pull request #1542: URL: https://github.com/apache/lucene-solr/pull/1542#issuecomment-641785613 @murblanc I've fixed a few things pls review & I can merge this This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9398) Read BKD tree always off-heap
[ https://issues.apache.org/jira/browse/LUCENE-9398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130124#comment-17130124 ] ASF subversion and git services commented on LUCENE-9398: - Commit c143082159912a3bab11c3c1daddf20ce7ef9abe in lucene-solr's branch refs/heads/branch_8x from Ignacio Vera [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=c143082 ] LUCENE-9398: Always keep BKD index off-heap. BKD reader does not implement Accountable any more (#1558) > Read BKD tree always off-heap > - > > Key: LUCENE-9398 > URL: https://issues.apache.org/jira/browse/LUCENE-9398 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Ignacio Vera >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > > We currently only read the BKD tree index off-heap in case of MMAP. This was > consistent to the way we were reading FST. In LUCENE-9257 we change it so now > it is always read off-heap so we can probably do the same for the BKD index > as we always read the tree sequentially. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-9398) Read BKD tree always off-heap
[ https://issues.apache.org/jira/browse/LUCENE-9398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ignacio Vera resolved LUCENE-9398. -- Fix Version/s: 8.6 Assignee: Ignacio Vera Resolution: Fixed > Read BKD tree always off-heap > - > > Key: LUCENE-9398 > URL: https://issues.apache.org/jira/browse/LUCENE-9398 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Ignacio Vera >Assignee: Ignacio Vera >Priority: Major > Fix For: 8.6 > > Time Spent: 50m > Remaining Estimate: 0h > > We currently only read the BKD tree index off-heap in case of MMAP. This was > consistent to the way we were reading FST. In LUCENE-9257 we change it so now > it is always read off-heap so we can probably do the same for the BKD index > as we always read the tree sequentially. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9398) Read BKD tree always off-heap
[ https://issues.apache.org/jira/browse/LUCENE-9398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130121#comment-17130121 ] ASF subversion and git services commented on LUCENE-9398: - Commit 37a83675a79be96dd28f1c96c9ffa9fa7129a8b4 in lucene-solr's branch refs/heads/master from Ignacio Vera [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=37a8367 ] LUCENE-9398: Always keep BKD index off-heap. BKD reader does not implement Accountable any more (#1558) > Read BKD tree always off-heap > - > > Key: LUCENE-9398 > URL: https://issues.apache.org/jira/browse/LUCENE-9398 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Ignacio Vera >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > > We currently only read the BKD tree index off-heap in case of MMAP. This was > consistent to the way we were reading FST. In LUCENE-9257 we change it so now > it is always read off-heap so we can probably do the same for the BKD index > as we always read the tree sequentially. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] iverase merged pull request #1558: LUCENE-9398: BKD tree is now read always off-heap
iverase merged pull request #1558: URL: https://github.com/apache/lucene-solr/pull/1558 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
