Re: [PR] Build: ge.gradle add tags & projectId=solr [solr]
dsmiley commented on code in PR #2264:
URL: https://github.com/apache/solr/pull/2264#discussion_r1490141634
##
gradle/ge.gradle:
##
@@ -27,6 +31,25 @@ gradleEnterprise {
obfuscation {
ipAddresses { addresses -> addresses.collect { address ->
"0.0.0.0"} }
}
+
+tag("JDK" + System.getProperty("java.version").find(/\d+/)) // major
version
+value("Java Vendor", System.getProperty("java.vendor"))
+value("Java Version", System.getProperty("java.version"))
+
+// Jenkins job name, less redundant "Solr" parts
+String jenkinsJobName = System.getenv("JOB_NAME")
+if (jenkinsJobName) { // looks like "Solr/Solr-Smoketest-9.5"
+tag(jenkinsJobName.replaceAll(/Solr\W/,"")) // like "Smoketest-9.5"
Review Comment:
Wanted to keep tags short
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Build: ge.gradle add tags & projectId=solr [solr]
dsmiley commented on code in PR #2264:
URL: https://github.com/apache/solr/pull/2264#discussion_r1490141248
##
gradle/ge.gradle:
##
@@ -27,6 +31,25 @@ gradleEnterprise {
obfuscation {
ipAddresses { addresses -> addresses.collect { address ->
"0.0.0.0"} }
}
+
+tag("JDK" + System.getProperty("java.version").find(/\d+/)) // major
version
+value("Java Vendor", System.getProperty("java.vendor"))
+value("Java Version", System.getProperty("java.version"))
+
+// Jenkins job name, less redundant "Solr" parts
+String jenkinsJobName = System.getenv("JOB_NAME")
+if (jenkinsJobName) { // looks like "Solr/Solr-Smoketest-9.5"
+tag(jenkinsJobName.replaceAll(/Solr\W/,"")) // like "Smoketest-9.5"
+}
+
+//
https://docs.github.com/en/actions/learn-github-actions/variables#default-environment-variables
+if (System.getenv("GITHUB_BASE_REF")) { // only exists for PRs
+tag("PR")
+}
+String ghWorkflowName = System.getenv("GITHUB_WORKFLOW")
+if (ghWorkflowName) {
+tag(ghWorkflowName)
Review Comment:
Yes but I want a tag. The Jenkins Job Name & GitHub workflow (similar) is a
major build scan differentiator to quickly know what we're looking at.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Build: ge.gradle add tags & projectId=solr [solr]
dsmiley commented on code in PR #2264:
URL: https://github.com/apache/solr/pull/2264#discussion_r1490140328
##
gradle/ge.gradle:
##
@@ -27,6 +31,25 @@ gradleEnterprise {
obfuscation {
ipAddresses { addresses -> addresses.collect { address ->
"0.0.0.0"} }
}
+
+tag("JDK" + System.getProperty("java.version").find(/\d+/)) // major
version
Review Comment:
Yes; this is exactly why I added it. It's a major piece of useful
information that differentiates builds.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Build: ge.gradle add tags & projectId=solr [solr]
clayburn commented on code in PR #2264:
URL: https://github.com/apache/solr/pull/2264#discussion_r1490136801
##
gradle/ge.gradle:
##
@@ -27,6 +31,25 @@ gradleEnterprise {
obfuscation {
ipAddresses { addresses -> addresses.collect { address ->
"0.0.0.0"} }
}
+
+tag("JDK" + System.getProperty("java.version").find(/\d+/)) // major
version
+value("Java Vendor", System.getProperty("java.vendor"))
+value("Java Version", System.getProperty("java.version"))
+
+// Jenkins job name, less redundant "Solr" parts
+String jenkinsJobName = System.getenv("JOB_NAME")
+if (jenkinsJobName) { // looks like "Solr/Solr-Smoketest-9.5"
+tag(jenkinsJobName.replaceAll(/Solr\W/,"")) // like "Smoketest-9.5"
+}
+
+//
https://docs.github.com/en/actions/learn-github-actions/variables#default-environment-variables
+if (System.getenv("GITHUB_BASE_REF")) { // only exists for PRs
+tag("PR")
+}
+String ghWorkflowName = System.getenv("GITHUB_WORKFLOW")
+if (ghWorkflowName) {
+tag(ghWorkflowName)
Review Comment:
The Common Custom User Data Gradle Plugin already adds this as a custom
value called `CI workflow`
##
gradle/ge.gradle:
##
@@ -27,6 +31,25 @@ gradleEnterprise {
obfuscation {
ipAddresses { addresses -> addresses.collect { address ->
"0.0.0.0"} }
}
+
+tag("JDK" + System.getProperty("java.version").find(/\d+/)) // major
version
Review Comment:
For what it's worth, these JDK details are already captured in the [build
scan](https://ge.apache.org/s/iria4ppalnbdo#infrastructure). This information
will show up in build scan comparisons as well. But capturing as tags and value
is nice if you want the ability to filter on these values.
##
gradle/ge.gradle:
##
@@ -27,6 +31,25 @@ gradleEnterprise {
obfuscation {
ipAddresses { addresses -> addresses.collect { address ->
"0.0.0.0"} }
}
+
+tag("JDK" + System.getProperty("java.version").find(/\d+/)) // major
version
+value("Java Vendor", System.getProperty("java.vendor"))
+value("Java Version", System.getProperty("java.version"))
+
+// Jenkins job name, less redundant "Solr" parts
+String jenkinsJobName = System.getenv("JOB_NAME")
+if (jenkinsJobName) { // looks like "Solr/Solr-Smoketest-9.5"
+tag(jenkinsJobName.replaceAll(/Solr\W/,"")) // like "Smoketest-9.5"
Review Comment:
The Common Custom User Data Gradle Plugin already adds this as a custom
value called `CI job` (but without the filtering you've added).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-15730) Modularize SolrJ
[
https://issues.apache.org/jira/browse/SOLR-15730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817540#comment-17817540
]
David Smiley commented on SOLR-15730:
-
For 10, might as well put all the SolrJ Jetty client stuff in the package
{{org.apache.solr.client.solrj.jetty}}; no need to add "impl" sub-package.
bq. to use org.apache.solr.solrj:solrj as the default slim client instead of
solrj-core
I don't understand what you are saying there. Are you saying we don't need a
"-core"? Major +1 to that; it's a needless suffix and would disrupt users.
> Modularize SolrJ
>
>
> Key: SOLR-15730
> URL: https://issues.apache.org/jira/browse/SOLR-15730
> Project: Solr
> Issue Type: Task
> Components: SolrJ
>Reporter: Jan Høydahl
>Priority: Major
> Attachments: Skjermbilde 2021-10-28 kl. 15.38.40.png
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Umbrella issue for breaking up SolrJ into a slim solrj-core with minimal
> dependencies as well as solrj-zk, solrj-streaming, solrj-jdbc etc as needed.
> We can move relevant other JIRAs as sub-tasks to this one to keep everything
> together.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-16503) Switch UpdateShardHandler.getDefaultHttpClient to Jetty HTTP2
[
https://issues.apache.org/jira/browse/SOLR-16503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817523#comment-17817523
]
Jan Høydahl commented on SOLR-16503:
[~ichattopadhyaya] as we are about to cut the apache-http client wire in main
branch, do we yet have some public nightly solr performance benchmarks where we
could compare from day to day as we do these changes?
> Switch UpdateShardHandler.getDefaultHttpClient to Jetty HTTP2
> -
>
> Key: SOLR-16503
> URL: https://issues.apache.org/jira/browse/SOLR-16503
> Project: Solr
> Issue Type: Sub-task
>Reporter: David Smiley
>Priority: Major
>
> Much of Solr's remaining uses of Apache HttpClient (HTTP 1) is due to
> {{org.apache.solr.update.UpdateShardHandler#getDefaultHttpClient}} which
> underlies most Solr-to-Solr connectivity. This also underlies the
> {{{}CoreContainer.getSolrClientCache{}}}. Lets switch to Jetty (HTTP 2).
>
> In SolrClientCache in particular:
> Switch use of CloudLegacySolrClient.Builder to CloudSolrClient.Builder
> Switch use of HttpSolrClient.Builder to Http2SolrClient.Builder
> Undeprecate all the methods here. They should not have been deprecated in
> the first place.
> The constructor: switch from Apache HttpClient to a Jetty HttpClient.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-8680) Distribute JDBC driver as a separate jar
[ https://issues.apache.org/jira/browse/SOLR-8680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817519#comment-17817519 ] Jan Høydahl commented on SOLR-8680: --- Now that solrj is being split in several parts, do you think it makes sense to distribute an uber-jar "solrj-all" with the main purpose of satisfying needs of JDBC users? Should it ibe a "OneJar" style jar with all dependencies embedded, or simply a mashup of solrj packages with the need to include the handful of external dependencies? If OneJar, should we shade all dependencies (non-solr pkgs) to avoid version conflicts in users' applications? Such a shaded JAR will be quite fat and add much weight to our release tarball. So perhaps consider a separate release artifact for it? > Distribute JDBC driver as a separate jar > > > Key: SOLR-8680 > URL: https://issues.apache.org/jira/browse/SOLR-8680 > Project: Solr > Issue Type: New Feature > Components: Parallel SQL >Reporter: Joel Bernstein >Priority: Major > Attachments: SOLR-8680.patch > > > Currently the JDBC driver comes included with the Solrj libraries. As the > JDBC driver matures it will be useful to distribute a separate jar which > includes all of the dependancies, rather then having to include all the Solrj > dependancies separately. This will make it much easier to install and ship > with products like JasperSoft, Spotfire and Tableau. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Resolved] (SOLR-15731) Separate out a solrj-jdbc module
[ https://issues.apache.org/jira/browse/SOLR-15731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Høydahl resolved SOLR-15731. Resolution: Won't Do > Separate out a solrj-jdbc module > > > Key: SOLR-15731 > URL: https://issues.apache.org/jira/browse/SOLR-15731 > Project: Solr > Issue Type: Sub-task > Components: Parallel SQL >Reporter: Jan Høydahl >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-15731) Separate out a solrj-jdbc module
[ https://issues.apache.org/jira/browse/SOLR-15731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817518#comment-17817518 ] Jan Høydahl commented on SOLR-15731: I'm going to close this, as the solrj-streaming is enough modularization. Then the topic of producing an uber-jar for JDBC purposes can happen elsewhere. > Separate out a solrj-jdbc module > > > Key: SOLR-15731 > URL: https://issues.apache.org/jira/browse/SOLR-15731 > Project: Solr > Issue Type: Sub-task > Components: Parallel SQL >Reporter: Jan Høydahl >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Created] (SOLR-17161) Separate out a solrj-jetty package / artifact
Jan Høydahl created SOLR-17161:
--
Summary: Separate out a solrj-jetty package / artifact
Key: SOLR-17161
URL: https://issues.apache.org/jira/browse/SOLR-17161
Project: Solr
Issue Type: Sub-task
Components: clients - java
Reporter: Jan Høydahl
Given we have a native JDK based client in SOLR-599, we can separate out all
{{Http2SolrClient}} and freiends with their jetty-client dependencies into a
separate artifact {{{}solrj-jetty{}}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-15730) Modularize SolrJ
[
https://issues.apache.org/jira/browse/SOLR-15730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817514#comment-17817514
]
Jan Høydahl commented on SOLR-15730:
See discussion in sub task SOLR-16078, where I propose a new {{solrj-jetty}}
artifact and a slim {{{}solrj-core{}}}. Continuing the discussion of overall
strategy here in the parent issue.
Separate artifacts require unique java package names, so there will inevitably
be java package name changes too, such as
{code:java}
org.apache.solr.client.solrj.impl.Http2SolrClient ->
org.apache.solr.client.solrj.jetty.impl.Http2SolrClient{code}
Or do we perhaps want to use {{org.apache.solr.solrj.jetty}} to keep package
name same as group-ID? If we need to split packages anyway, users will need to
change their imports either way.
I don't think we need a "fat" jar with everything, what usecase would that be
for? Thus, I can be convinced to use {{org.apache.solr.solrj:solrj}} as the
default slim client instead of {{{}solrj-core{}}}.
> Modularize SolrJ
>
>
> Key: SOLR-15730
> URL: https://issues.apache.org/jira/browse/SOLR-15730
> Project: Solr
> Issue Type: Task
> Components: SolrJ
>Reporter: Jan Høydahl
>Priority: Major
> Attachments: Skjermbilde 2021-10-28 kl. 15.38.40.png
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Umbrella issue for breaking up SolrJ into a slim solrj-core with minimal
> dependencies as well as solrj-zk, solrj-streaming, solrj-jdbc etc as needed.
> We can move relevant other JIRAs as sub-tasks to this one to keep everything
> together.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-16078) New solrj-core module
[
https://issues.apache.org/jira/browse/SOLR-16078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817512#comment-17817512
]
Jan Høydahl commented on SOLR-16078:
Moving this discussion to the parent issue SOLR-15730
> New solrj-core module
> -
>
> Key: SOLR-16078
> URL: https://issues.apache.org/jira/browse/SOLR-16078
> Project: Solr
> Issue Type: Sub-task
> Components: SolrJ
>Reporter: Jan Høydahl
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> We should introduce a solrj-core module that is as slim as possible wrt
> dependencies.
> A user will then add solrj-core as well as any other solj-xx modules needed
> for their use.
> By marking it {{\@lucene.experimental}} we can change the API of this
> solrj-core during 9.x as we move stuff into other modules.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Updated] (SOLR-17157) upgrade to Lucene 9.10.0
[
https://issues.apache.org/jira/browse/SOLR-17157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christine Poerschke updated SOLR-17157:
---
Description:
Creating and leaving this Jira open for (new) devs to grab.
optional steps before Lucene 9.10 is released:
* Q: what would be the benefit of these optional steps?
** A: to potentially identify 9.9 to 9.10 regressions before 9.10 is released.
* monitor the relevant {{dev@lucene}} threads e.g.
** https://lists.apache.org/thread/w7smn82q5q5qsstqbpdxm5vnfw3y9ojh - discuss
** https://lists.apache.org/thread/bwjr0vpo25dv8ydycbmo3130pgfktvf0 - feature
freeze
** https://lists.apache.org/thread/stwn4gfrs1dmxjpwdf59plfpgtvsf9v6 - voting
* locally build {{main}} Solr against the latest {{branch_9_10}} Lucene as per
[https://github.com/apache/solr/blob/main/help/dependencies.txt#L111] steps
** open a draft pull request with changes
** run the tests and report and/or investigate any failures
upgrade steps once Lucene 9.10 is released:
* [https://lucene.apache.org/core/9_10_0/changes/Changes.html]
* [https://github.com/apache/solr/blob/main/dev-docs/lucene-upgrade.md]
was:
Creating and leaving this Jira open for (new) devs to grab.
optional steps before Lucene 9.10 is released:
* Q: what would be the benefit of these optional steps?
** A: to potentially identify 9.9 to 9.10 regressions before 9.10 is released.
* monitor the relevant {{dev@lucene}} threads e.g.
*** https://lists.apache.org/thread/w7smn82q5q5qsstqbpdxm5vnfw3y9ojh - discuss
*** https://lists.apache.org/thread/bwjr0vpo25dv8ydycbmo3130pgfktvf0 - feature
freeze
*** https://lists.apache.org/thread/stwn4gfrs1dmxjpwdf59plfpgtvsf9v6 - voting
* locally build {{main}} Solr against the latest {{branch_9_10}} Lucene as per
[https://github.com/apache/solr/blob/main/help/dependencies.txt#L111] steps
** open a draft pull request with changes
** run the tests and report and/or investigate any failures
upgrade steps once Lucene 9.10 is released:
* [https://lucene.apache.org/core/9_10_0/changes/Changes.html]
* [https://github.com/apache/solr/blob/main/dev-docs/lucene-upgrade.md]
> upgrade to Lucene 9.10.0
>
>
> Key: SOLR-17157
> URL: https://issues.apache.org/jira/browse/SOLR-17157
> Project: Solr
> Issue Type: Task
>Reporter: Christine Poerschke
>Priority: Major
> Labels: newdev
>
> Creating and leaving this Jira open for (new) devs to grab.
> optional steps before Lucene 9.10 is released:
> * Q: what would be the benefit of these optional steps?
> ** A: to potentially identify 9.9 to 9.10 regressions before 9.10 is
> released.
> * monitor the relevant {{dev@lucene}} threads e.g.
> ** https://lists.apache.org/thread/w7smn82q5q5qsstqbpdxm5vnfw3y9ojh - discuss
> ** https://lists.apache.org/thread/bwjr0vpo25dv8ydycbmo3130pgfktvf0 -
> feature freeze
> ** https://lists.apache.org/thread/stwn4gfrs1dmxjpwdf59plfpgtvsf9v6 - voting
> * locally build {{main}} Solr against the latest {{branch_9_10}} Lucene as
> per [https://github.com/apache/solr/blob/main/help/dependencies.txt#L111]
> steps
> ** open a draft pull request with changes
> ** run the tests and report and/or investigate any failures
> upgrade steps once Lucene 9.10 is released:
> * [https://lucene.apache.org/core/9_10_0/changes/Changes.html]
> * [https://github.com/apache/solr/blob/main/dev-docs/lucene-upgrade.md]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Updated] (SOLR-17157) upgrade to Lucene 9.10.0
[
https://issues.apache.org/jira/browse/SOLR-17157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christine Poerschke updated SOLR-17157:
---
Description:
Creating and leaving this Jira open for (new) devs to grab.
optional steps before Lucene 9.10 is released:
* Q: what would be the benefit of these optional steps?
** A: to potentially identify 9.9 to 9.10 regressions before 9.10 is released.
* monitor the relevant {{dev@lucene}} threads e.g.
*** https://lists.apache.org/thread/w7smn82q5q5qsstqbpdxm5vnfw3y9ojh - discuss
*** https://lists.apache.org/thread/bwjr0vpo25dv8ydycbmo3130pgfktvf0 - feature
freeze
*** https://lists.apache.org/thread/stwn4gfrs1dmxjpwdf59plfpgtvsf9v6 - voting
* locally build {{main}} Solr against the latest {{branch_9_10}} Lucene as per
[https://github.com/apache/solr/blob/main/help/dependencies.txt#L111] steps
** open a draft pull request with changes
** run the tests and report and/or investigate any failures
upgrade steps once Lucene 9.10 is released:
* [https://lucene.apache.org/core/9_10_0/changes/Changes.html]
* [https://github.com/apache/solr/blob/main/dev-docs/lucene-upgrade.md]
was:
Creating and leaving this Jira open for (new) devs to grab.
optional steps before Lucene 9.10 is released:
* Q: what would be the benefit of these optional steps?
** A: to potentially identify 9.9 to 9.10 regressions before 9.10 is released.
* monitor the relevant {{dev@lucene}} threads e.g.
[https://lists.apache.org/thread/w7smn82q5q5qsstqbpdxm5vnfw3y9ojh] and
https://lists.apache.org/thread/bwjr0vpo25dv8ydycbmo3130pgfktvf0
* locally build {{main}} Solr against the latest {{branch_9_10}} Lucene as per
[https://github.com/apache/solr/blob/main/help/dependencies.txt#L111] steps
** open a draft pull request with changes
** run the tests and report and/or investigate any failures
upgrade steps once Lucene 9.10 is released:
* [https://lucene.apache.org/core/9_10_0/changes/Changes.html]
* [https://github.com/apache/solr/blob/main/dev-docs/lucene-upgrade.md]
> upgrade to Lucene 9.10.0
>
>
> Key: SOLR-17157
> URL: https://issues.apache.org/jira/browse/SOLR-17157
> Project: Solr
> Issue Type: Task
>Reporter: Christine Poerschke
>Priority: Major
> Labels: newdev
>
> Creating and leaving this Jira open for (new) devs to grab.
> optional steps before Lucene 9.10 is released:
> * Q: what would be the benefit of these optional steps?
> ** A: to potentially identify 9.9 to 9.10 regressions before 9.10 is
> released.
> * monitor the relevant {{dev@lucene}} threads e.g.
> *** https://lists.apache.org/thread/w7smn82q5q5qsstqbpdxm5vnfw3y9ojh -
> discuss
> *** https://lists.apache.org/thread/bwjr0vpo25dv8ydycbmo3130pgfktvf0 -
> feature freeze
> *** https://lists.apache.org/thread/stwn4gfrs1dmxjpwdf59plfpgtvsf9v6 - voting
> * locally build {{main}} Solr against the latest {{branch_9_10}} Lucene as
> per [https://github.com/apache/solr/blob/main/help/dependencies.txt#L111]
> steps
> ** open a draft pull request with changes
> ** run the tests and report and/or investigate any failures
> upgrade steps once Lucene 9.10 is released:
> * [https://lucene.apache.org/core/9_10_0/changes/Changes.html]
> * [https://github.com/apache/solr/blob/main/dev-docs/lucene-upgrade.md]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Suppress two `gradle dependencyCheckAggregate` false positives [solr]
HoustonPutman merged PR #2198: URL: https://github.com/apache/solr/pull/2198 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Build: ge.gradle add tags & projectId=solr [solr]
dsmiley commented on PR #2264: URL: https://github.com/apache/solr/pull/2264#issuecomment-1944395079  And the build details: https://ge.apache.org/s/7tfc5bq6jabyi -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] solr.xml: honor plugin enable=true|false [solr]
dsmiley commented on PR #2260: URL: https://github.com/apache/solr/pull/2260#issuecomment-1944387038 I forgot to push the BackupRepository case yesterday; did it now. Converting to Java streams shows very nice results there too :-) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[PR] Update core param in STATUS doc [solr]
psalagnac opened a new pull request, #2265: URL: https://github.com/apache/solr/pull/2265 This is a trivial fix in doc. I did not fill a Jira for it. Parameter `core` for `STATUS` admin request is (wrongly) required in the full description, but (correctly) optional just before. # Checklist Please review the following and check all that apply: - [x] I have reviewed the guidelines for [How to Contribute](https://github.com/apache/solr/blob/main/CONTRIBUTING.md) and my code conforms to the standards described there to the best of my ability. - [ ] I have created a Jira issue and added the issue ID to my pull request title. - [x] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended) - [x] I have developed this patch against the `main` branch. - [x] I have run `./gradlew check`. - [ ] I have added tests for my changes. - [ ] I have added documentation for the [Reference Guide](https://github.com/apache/solr/tree/main/solr/solr-ref-guide) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-16995: Configure replication behaviour for each replica type [solr]
pvcnt commented on code in PR #2207:
URL: https://github.com/apache/solr/pull/2207#discussion_r1489895570
##
solr/solrj/src/java/org/apache/solr/common/cloud/Replica.java:
##
@@ -104,27 +104,35 @@ public enum Type {
* support NRT (soft commits) and RTG. Any {@link Type#NRT} replica can
become a leader. A shard
* leader will forward updates to all active {@link Type#NRT} and {@link
Type#TLOG} replicas.
*/
-NRT(true, CollectionAdminParams.NRT_REPLICAS),
+NRT(true, true, false, CollectionAdminParams.NRT_REPLICAS),
/**
* Writes to transaction log, but not to index, uses replication. Any
{@link Type#TLOG} replica
* can become leader (by first applying all local transaction log
elements). If a replica is of
* type {@link Type#TLOG} but is also the leader, it will behave as a
{@link Type#NRT}. A shard
* leader will forward updates to all active {@link Type#NRT} and {@link
Type#TLOG} replicas.
*/
-TLOG(true, CollectionAdminParams.TLOG_REPLICAS),
+TLOG(true, true, true, CollectionAdminParams.TLOG_REPLICAS),
/**
* Doesn’t index or writes to transaction log. Just replicates from {@link
Type#NRT} or {@link
* Type#TLOG} replicas. {@link Type#PULL} replicas can’t become shard
leaders (i.e., if there
* are only pull replicas in the collection at some point, updates will
fail same as if there is
* no leaders, queries continue to work), so they don’t even participate
in elections.
*/
-PULL(false, CollectionAdminParams.PULL_REPLICAS);
+PULL(false, false, true, CollectionAdminParams.PULL_REPLICAS);
public final boolean leaderEligible;
+public final boolean requireTransactionLog;
+public final boolean replicateFromLeader;
Review Comment:
I added Javadoc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-16995: Configure replication behaviour for each replica type [solr]
pvcnt commented on code in PR #2207:
URL: https://github.com/apache/solr/pull/2207#discussion_r1489888413
##
solr/solrj/src/java/org/apache/solr/common/cloud/Replica.java:
##
@@ -104,27 +104,35 @@ public enum Type {
* support NRT (soft commits) and RTG. Any {@link Type#NRT} replica can
become a leader. A shard
* leader will forward updates to all active {@link Type#NRT} and {@link
Type#TLOG} replicas.
*/
-NRT(true, CollectionAdminParams.NRT_REPLICAS),
+NRT(true, true, false, CollectionAdminParams.NRT_REPLICAS),
/**
* Writes to transaction log, but not to index, uses replication. Any
{@link Type#TLOG} replica
* can become leader (by first applying all local transaction log
elements). If a replica is of
* type {@link Type#TLOG} but is also the leader, it will behave as a
{@link Type#NRT}. A shard
* leader will forward updates to all active {@link Type#NRT} and {@link
Type#TLOG} replicas.
*/
-TLOG(true, CollectionAdminParams.TLOG_REPLICAS),
+TLOG(true, true, true, CollectionAdminParams.TLOG_REPLICAS),
/**
* Doesn’t index or writes to transaction log. Just replicates from {@link
Type#NRT} or {@link
* Type#TLOG} replicas. {@link Type#PULL} replicas can’t become shard
leaders (i.e., if there
* are only pull replicas in the collection at some point, updates will
fail same as if there is
* no leaders, queries continue to work), so they don’t even participate
in elections.
*/
-PULL(false, CollectionAdminParams.PULL_REPLICAS);
+PULL(false, false, true, CollectionAdminParams.PULL_REPLICAS);
public final boolean leaderEligible;
+public final boolean requireTransactionLog;
+public final boolean replicateFromLeader;
Review Comment:
Yes, all replica types do replica during recovery. This property might not
be named correctly, do you have a suggestion for a better name?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-16995: Configure replication behaviour for each replica type [solr]
pvcnt commented on code in PR #2207:
URL: https://github.com/apache/solr/pull/2207#discussion_r1489886455
##
solr/core/src/java/org/apache/solr/cloud/ZkController.java:
##
@@ -2400,15 +2402,15 @@ public void rejoinShardLeaderElection(SolrParams
params) {
try (SolrCore core = cc.getCore(coreName)) {
Replica.Type replicaType =
core.getCoreDescriptor().getCloudDescriptor().getReplicaType();
-if (replicaType == Type.TLOG) {
+if (replicaType.replicateFromLeader) {
String leaderUrl =
getLeader(
core.getCoreDescriptor().getCloudDescriptor(),
cloudConfig.getLeaderVoteWait());
if (!leaderUrl.equals(ourUrl)) {
Review Comment:
We are in `rejoinShardLeaderElection`, I assumed we can ever enter in this
method for a PULL (or any non leader-eligible) replica, or can we?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Build: ge.gradle add tags & projectId=solr [solr]
dsmiley commented on PR #2256: URL: https://github.com/apache/solr/pull/2256#issuecomment-1944344576 Replaced by #2264 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Build: ge.gradle add tags & projectId=solr [solr]
dsmiley closed pull request #2256: Build: ge.gradle add tags & projectId=solr URL: https://github.com/apache/solr/pull/2256 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[PR] Build: ge.gradle add tags & projectId=solr [solr]
dsmiley opened a new pull request, #2264: URL: https://github.com/apache/solr/pull/2264 Add more Gradle Enterprise build metadata. (replacing previous #2256 ) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] NO JIRA: remove duplicate '(cmd.getFlags() & GET_SCORES) != 0' calls [solr]
gerlowskija commented on PR #2262: URL: https://github.com/apache/solr/pull/2262#issuecomment-1944317482 > No JIRA ticket or CHANGES.txt entry needed I think +1 - it'd be overkill here IMO -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-13350: Multithreaded search [solr]
cpoerschke commented on code in PR #2248:
URL: https://github.com/apache/solr/pull/2248#discussion_r1489857592
##
solr/core/src/java/org/apache/solr/search/SolrMultiCollectorManager.java:
##
@@ -0,0 +1,150 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.search;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.List;
+import org.apache.lucene.index.LeafReaderContext;
+import org.apache.lucene.search.Collector;
+import org.apache.lucene.search.CollectorManager;
+import org.apache.lucene.search.FilterScorable;
+import org.apache.lucene.search.LeafCollector;
+import org.apache.lucene.search.MultiCollector;
+import org.apache.lucene.search.Scorable;
+import org.apache.lucene.search.ScoreMode;
+
+/**
+ * A {@link CollectorManager} implements which wrap a set of {@link
CollectorManager} as {@link
+ * MultiCollector} acts for {@link Collector}.
+ */
+public class SolrMultiCollectorManager
+implements CollectorManager {
Review Comment:
Lucene has a `public class MultiCollectorManager implements
CollectorManager` i.e.
https://github.com/apache/lucene/blob/releases/lucene/9.9.2/lucene/core/src/java/org/apache/lucene/search/MultiCollectorManager.java
and I'm curious if there'd be a way to code share with that, though so far I
can't see a way there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17066) Deprecate and remove core URLs in HttpSolrClient and friends
[
https://issues.apache.org/jira/browse/SOLR-17066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817461#comment-17817461
]
ASF subversion and git services commented on SOLR-17066:
Commit a406bdd29823717cd445928ad884da62d383c8c8 in solr's branch
refs/heads/branch_9x from Jason Gerlowski
[ https://gitbox.apache.org/repos/asf?p=solr.git;h=a406bdd2982 ]
SOLR-17066: Switch ConcurrentSolrClients away from core URLs (#2254)
Providing a core URL as a SolrClient's "base URL" prevents it from
communicating with other cores or making core-agnostic API requests
(e.g. node healthcheck, list cores, etc.)
This commit migrates all usage of both "concurrent update" clients
away from core URLs.
> Deprecate and remove core URLs in HttpSolrClient and friends
>
>
> Key: SOLR-17066
> URL: https://issues.apache.org/jira/browse/SOLR-17066
> Project: Solr
> Issue Type: Improvement
> Components: SolrJ
>Reporter: Jason Gerlowski
>Priority: Major
> Time Spent: 11h 50m
> Remaining Estimate: 0h
>
> Currently, URL-driven SolrClients can consume a base URL that either ends in
> an API-version specific path ("/solr" for v1 APIs, "/api" for v2), or in the
> full path to a specific core or collection ("/solr/techproducts").
> The latter option causes a number of problems in practice. It prevents the
> client from being used for any "admin" requests or for requests to other
> cores or collections. (Short of running a regex on
> {{SolrClient.getBaseURL}}, it's hard to even tell which of these restrictions
> a given client might have.) And lastly, specifying such core/collection URL
> makes it tough mix and match v1 and v2 API requests within the same client
> (see SOLR-17044).
> We should give SolrJ users some similar way to default collection/cores
> without any of these downsides. One approach would be to extend the
> {{withDefaultCollection}} pattern currently established in
> {{CloudHttp2SolrClient.Builder}}.
> (IMO we should also revisit the division of responsibilities between
> SolrClient and SolrRequest implementations - maybe clients shouldn't,
> directly at least, be holding on to request-specific settings like the
> core/collection at all. But that's a much larger concern that we might not
> want to wade into here. See SOLR-10466 for more on this topic.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Clarify split parameter usage [solr]
radu-gheorghe commented on PR #2263: URL: https://github.com/apache/solr/pull/2263#issuecomment-1944309385 Oh, wow. That's fast :) Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17066) Deprecate and remove core URLs in HttpSolrClient and friends
[
https://issues.apache.org/jira/browse/SOLR-17066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817457#comment-17817457
]
ASF subversion and git services commented on SOLR-17066:
Commit 6afbc20ee0611e8d1b0f1c41ca1aa58ee73d9f0f in solr's branch
refs/heads/main from Jason Gerlowski
[ https://gitbox.apache.org/repos/asf?p=solr.git;h=6afbc20ee06 ]
SOLR-17066: Switch ConcurrentSolrClients away from core URLs (#2254)
Providing a core URL as a SolrClient's "base URL" prevents it from
communicating with other cores or making core-agnostic API requests
(e.g. node healthcheck, list cores, etc.)
This commit migrates all usage of both "concurrent update" clients
away from core URLs.
> Deprecate and remove core URLs in HttpSolrClient and friends
>
>
> Key: SOLR-17066
> URL: https://issues.apache.org/jira/browse/SOLR-17066
> Project: Solr
> Issue Type: Improvement
> Components: SolrJ
>Reporter: Jason Gerlowski
>Priority: Major
> Time Spent: 11h 50m
> Remaining Estimate: 0h
>
> Currently, URL-driven SolrClients can consume a base URL that either ends in
> an API-version specific path ("/solr" for v1 APIs, "/api" for v2), or in the
> full path to a specific core or collection ("/solr/techproducts").
> The latter option causes a number of problems in practice. It prevents the
> client from being used for any "admin" requests or for requests to other
> cores or collections. (Short of running a regex on
> {{SolrClient.getBaseURL}}, it's hard to even tell which of these restrictions
> a given client might have.) And lastly, specifying such core/collection URL
> makes it tough mix and match v1 and v2 API requests within the same client
> (see SOLR-17044).
> We should give SolrJ users some similar way to default collection/cores
> without any of these downsides. One approach would be to extend the
> {{withDefaultCollection}} pattern currently established in
> {{CloudHttp2SolrClient.Builder}}.
> (IMO we should also revisit the division of responsibilities between
> SolrClient and SolrRequest implementations - maybe clients shouldn't,
> directly at least, be holding on to request-specific settings like the
> core/collection at all. But that's a much larger concern that we might not
> want to wade into here. See SOLR-10466 for more on this topic.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17066: Switch ConcurrentSolrClients away from core URLs [solr]
gerlowskija merged PR #2254: URL: https://github.com/apache/solr/pull/2254 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-13350: Multithreaded search [solr]
cpoerschke commented on code in PR #2248:
URL: https://github.com/apache/solr/pull/2248#discussion_r1489810761
##
solr/core/src/java/org/apache/solr/search/SolrIndexSearcher.java:
##
@@ -1917,6 +1963,203 @@ public ScoreMode scoreMode() {
qr.setDocList(new DocSlice(0, sliceLen, ids, scores, totalHits, maxScore,
hitsRelation));
}
+ SearchResult searchCollectorManagers(
+ int len,
+ QueryCommand cmd,
+ Query query,
+ boolean needTopDocs,
+ boolean needMaxScore,
+ boolean needDocSet)
+ throws IOException {
+Collection> collectors = new
ArrayList<>();
+ScoreMode scoreMode = null;
+
+Collector[] firstCollectors = new Collector[3];
+
+if (needTopDocs) {
+
+ collectors.add(
+ new CollectorManager<>() {
+@Override
+public Collector newCollector() throws IOException {
+ @SuppressWarnings("rawtypes")
+ TopDocsCollector collector = buildTopDocsCollector(len, cmd);
+ if (firstCollectors[0] == null) {
+firstCollectors[0] = collector;
+ }
+ return collector;
+}
+
+@Override
+@SuppressWarnings("rawtypes")
+public Object reduce(Collection collectors) throws IOException {
+
+ TopDocs[] topDocs = new TopDocs[collectors.size()];
+
+ int totalHits = -1;
+ int i = 0;
+
+ Collector collector;
+ for (Object o : collectors) {
+collector = (Collector) o;
+if (collector instanceof TopDocsCollector) {
+ TopDocs td = ((TopDocsCollector) collector).topDocs(0, len);
+ assert td != null : Arrays.asList(topDocs);
+ topDocs[i++] = td;
+}
+ }
+
+ TopDocs mergedTopDocs = null;
+
+ if (topDocs.length > 0 && topDocs[0] != null) {
+if (topDocs[0] instanceof TopFieldDocs) {
+ TopFieldDocs[] topFieldDocs =
+ Arrays.copyOf(topDocs, topDocs.length,
TopFieldDocs[].class);
+ mergedTopDocs =
TopFieldDocs.merge(weightSort(cmd.getSort()), len, topFieldDocs);
+} else {
+ mergedTopDocs = TopDocs.merge(0, len, topDocs);
+}
+totalHits = (int) mergedTopDocs.totalHits.value;
+ }
+ return new TopDocsResult(mergedTopDocs, totalHits);
+}
+ });
+}
+if (needMaxScore) {
+ collectors.add(
+ new CollectorManager<>() {
+@Override
+public Collector newCollector() throws IOException {
+ MaxScoreCollector collector = new MaxScoreCollector();
+ if (firstCollectors[1] == null) {
+firstCollectors[1] = collector;
+ }
+ return collector;
+}
+
+@Override
+@SuppressWarnings("rawtypes")
+public Object reduce(Collection collectors) throws IOException {
+
+ MaxScoreCollector collector;
+ float maxScore = 0.0f;
+ for (Iterator var4 = collectors.iterator();
+ var4.hasNext();
+ maxScore = Math.max(maxScore, collector.getMaxScore())) {
+collector = (MaxScoreCollector) var4.next();
+ }
+
+ return new MaxScoreResult(maxScore);
+}
+ });
+}
+if (needDocSet) {
+ int maxDoc = rawReader.maxDoc();
+ log.error("raw read max={}", rawReader.maxDoc());
+
+ LeafSlice[] leaves = getSlices();
+ int[] docBase = new int[1];
+
+ // DocSetCollector collector = new DocSetCollector(maxDoc);
+
+ ThreadSafeBitSet bits = new ThreadSafeBitSet(14, 2);
+
+ collectors.add(
+ new CollectorManager<>() {
+@Override
+public Collector newCollector() throws IOException {
+ int numDocs = 0;
+
+ if (leaves != null) {
+LeafSlice leaf = leaves[docBase[0]++];
+
+for (LeafReaderContext reader : leaf.leaves) {
+ numDocs += reader.reader().maxDoc();
+}
+ } else {
+numDocs = maxDoc();
+ }
+ log.error("new docset collector for {} max={}", numDocs,
maxDoc());
+
+ return new ThreadSafeBitSetCollector(bits, maxDoc);
Review Comment:
Not sure yet if a `firstCollectors[2] = ...` was intended here or if not we
can reduce to `new Collector[2]` above it seems.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] SOLR-13350: Multithreaded search [solr]
cpoerschke commented on code in PR #2248:
URL: https://github.com/apache/solr/pull/2248#discussion_r1489791031
##
solr/core/src/java/org/apache/solr/search/SolrIndexSearcher.java:
##
@@ -1910,6 +1960,203 @@ public ScoreMode scoreMode() {
qr.setDocList(new DocSlice(0, sliceLen, ids, scores, totalHits, maxScore,
hitsRelation));
}
+ SearchResult searchCollectorManagers(
+ int len,
+ QueryCommand cmd,
+ Query query,
+ boolean needTopDocs,
+ boolean needMaxScore,
+ boolean needDocSet)
+ throws IOException {
+Collection> collectors = new
ArrayList<>();
+ScoreMode scoreMode = null;
+
+Collector[] firstCollectors = new Collector[3];
+
+if (needTopDocs) {
+
+ collectors.add(
+ new CollectorManager<>() {
+@Override
+public Collector newCollector() throws IOException {
+ @SuppressWarnings("rawtypes")
+ TopDocsCollector collector = buildTopDocsCollector(len, cmd);
+ if (firstCollectors[0] == null) {
+firstCollectors[0] = collector;
+ }
+ return collector;
+}
+
+@Override
+@SuppressWarnings("rawtypes")
+public Object reduce(Collection collectors) throws IOException {
+
+ TopDocs[] topDocs = new TopDocs[collectors.size()];
+
+ int totalHits = -1;
+ int i = 0;
+
+ Collector collector;
+ for (Object o : collectors) {
+collector = (Collector) o;
+if (collector instanceof TopDocsCollector) {
+ TopDocs td = ((TopDocsCollector) collector).topDocs(0, len);
+ assert td != null : Arrays.asList(topDocs);
+ topDocs[i++] = td;
+}
+ }
+
+ TopDocs mergedTopDocs = null;
+
+ if (topDocs.length > 0 && topDocs[0] != null) {
+if (topDocs[0] instanceof TopFieldDocs) {
+ TopFieldDocs[] topFieldDocs =
+ Arrays.copyOf(topDocs, topDocs.length,
TopFieldDocs[].class);
+ mergedTopDocs =
TopFieldDocs.merge(weightSort(cmd.getSort()), len, topFieldDocs);
+} else {
+ mergedTopDocs = TopDocs.merge(0, len, topDocs);
+}
+totalHits = (int) mergedTopDocs.totalHits.value;
+ }
+ return new TopDocsResult(mergedTopDocs, totalHits);
+}
+ });
+}
+if (needMaxScore) {
+ collectors.add(
+ new CollectorManager<>() {
+@Override
+public Collector newCollector() throws IOException {
+ MaxScoreCollector collector = new MaxScoreCollector();
+ if (firstCollectors[1] == null) {
+firstCollectors[1] = collector;
+ }
+ return collector;
+}
+
+@Override
+@SuppressWarnings("rawtypes")
+public Object reduce(Collection collectors) throws IOException {
+
+ MaxScoreCollector collector;
+ float maxScore = 0.0f;
+ for (Iterator var4 = collectors.iterator();
+ var4.hasNext();
+ maxScore = Math.max(maxScore, collector.getMaxScore())) {
+collector = (MaxScoreCollector) var4.next();
+ }
+
+ return new MaxScoreResult(maxScore);
+}
+ });
+}
+if (needDocSet) {
+ int maxDoc = rawReader.maxDoc();
+ log.error("raw read max={}", rawReader.maxDoc());
+
+ LeafSlice[] leaves = getSlices();
+ int[] docBase = new int[1];
+
+ // DocSetCollector collector = new DocSetCollector(maxDoc);
+
+ ThreadSafeBitSet bits = new ThreadSafeBitSet(14, 2);
+
+ collectors.add(
+ new CollectorManager<>() {
+@Override
+public Collector newCollector() throws IOException {
+ int numDocs = 0;
+
+ if (leaves != null) {
+LeafSlice leaf = leaves[docBase[0]++];
+
+for (LeafReaderContext reader : leaf.leaves) {
+ numDocs += reader.reader().maxDoc();
+}
+ } else {
+numDocs = maxDoc();
+ }
+ log.error("new docset collector for {} max={}", numDocs,
maxDoc());
+
+ return new ThreadSafeBitSetCollector(bits, maxDoc);
+}
+
+@Override
+@SuppressWarnings({"rawtypes"})
+public Object reduce(Collection collectors) throws IOException {
+
+ return new DocSetResult(
+ ((ThreadSafeBitSetCollector)
collectors.iterator().next()).getDocSet());
+}
+ });
+}
+for (Collector collector : firstCollectors) {
+ if (collector != null) {
+if (scoreMode == null) {
+ scoreMod
Re: [I] Solr Backup recurrence/schedule not enabled by helm 0.7.1 [solr-operator]
HoustonPutman commented on issue #687: URL: https://github.com/apache/solr-operator/issues/687#issuecomment-1944227204 That just configures a repository for your backups to use. You have to use a separate SolrBackup resource to actually take a backup. The Solr Helm chart does not install a SolrBackup resource currently, though it could be added as a feature. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Clarify split parameter usage [solr]
HoustonPutman commented on PR #2263: URL: https://github.com/apache/solr/pull/2263#issuecomment-1944219543 This should be reflected in the official 9.5 docs after tonight! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17141: Create CpuQueryLimit implementation [solr]
sigram commented on code in PR #2244:
URL: https://github.com/apache/solr/pull/2244#discussion_r1489784004
##
solr/core/src/java/org/apache/solr/handler/component/SearchHandler.java:
##
@@ -622,8 +622,8 @@ public void handleRequestBody(SolrQueryRequest req,
SolrQueryResponse rsp) throw
} while (nextStage != Integer.MAX_VALUE);
if (publishCpuTime) {
-rsp.getResponseHeader().add(ThreadStats.CPU_TIME, totalShardCpuTime);
-rsp.addToLog(ThreadStats.CPU_TIME, totalShardCpuTime);
+rsp.getResponseHeader().add(ThreadCpuTime.CPU_TIME, totalShardCpuTime);
Review Comment:
Do you mean for the responseHeader?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17141: Create CpuQueryLimit implementation [solr]
sigram commented on code in PR #2244:
URL: https://github.com/apache/solr/pull/2244#discussion_r1489782985
##
solr/core/src/java/org/apache/solr/handler/RequestHandlerBase.java:
##
@@ -250,17 +253,17 @@ public void handleRequest(SolrQueryRequest req,
SolrQueryResponse rsp) {
long elapsed = timer.stop();
metrics.totalTime.inc(elapsed);
- if (cpuStats != null) {
-Optional cpuTime = cpuStats.getCpuTimeMs();
+ if (publishCpuTime) {
Review Comment:
I don't think that's the case - see above comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17141: Create CpuQueryLimit implementation [solr]
sigram commented on code in PR #2244:
URL: https://github.com/apache/solr/pull/2244#discussion_r1489775946
##
solr/core/src/java/org/apache/solr/handler/component/SearchHandler.java:
##
@@ -622,8 +622,8 @@ public void handleRequestBody(SolrQueryRequest req,
SolrQueryResponse rsp) throw
} while (nextStage != Integer.MAX_VALUE);
if (publishCpuTime) {
-rsp.getResponseHeader().add(ThreadStats.CPU_TIME, totalShardCpuTime);
-rsp.addToLog(ThreadStats.CPU_TIME, totalShardCpuTime);
+rsp.getResponseHeader().add(ThreadCpuTime.CPU_TIME, totalShardCpuTime);
Review Comment:
This can never be null due to the assignment that depends on the same
condition in the if() around line 221.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-16078) New solrj-core module
[
https://issues.apache.org/jira/browse/SOLR-16078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817448#comment-17817448
]
James Dyer commented on SOLR-16078:
---
bq. Can we even get rid of jackson, or is it needed for JSON
I was thinking about this and was wondering if it would be possible, even if we
need a JSON parser, to have one solrj maven coordinate that only depended on
slf4j and nothing else. This would be for those applications running in a
JakartaEE-compliant environment. Then a separate artifact would pull in
Jackson for those users who do not have JSONB provided by the environment. I
do not know what it would take to accomplish this or if it is worth it.
> New solrj-core module
> -
>
> Key: SOLR-16078
> URL: https://issues.apache.org/jira/browse/SOLR-16078
> Project: Solr
> Issue Type: Sub-task
> Components: SolrJ
>Reporter: Jan Høydahl
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> We should introduce a solrj-core module that is as slim as possible wrt
> dependencies.
> A user will then add solrj-core as well as any other solj-xx modules needed
> for their use.
> By marking it {{\@lucene.experimental}} we can change the API of this
> solrj-core during 9.x as we move stuff into other modules.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17141: Create CpuQueryLimit implementation [solr]
sigram commented on code in PR #2244:
URL: https://github.com/apache/solr/pull/2244#discussion_r1489778608
##
solr/core/src/java/org/apache/solr/handler/RequestHandlerBase.java:
##
@@ -250,17 +253,17 @@ public void handleRequest(SolrQueryRequest req,
SolrQueryResponse rsp) {
long elapsed = timer.stop();
metrics.totalTime.inc(elapsed);
- if (cpuStats != null) {
-Optional cpuTime = cpuStats.getCpuTimeMs();
+ if (publishCpuTime) {
+Optional cpuTime = threadCpuTime.getCpuTimeMs();
if (cpuTime.isPresent()) {
// add CPU_TIME if not already added by SearchHandler
NamedList header = rsp.getResponseHeader();
Review Comment:
Yeah, I don't like this either. `ThreadCpuTimer.isSupported()` is the only
condition for this to return false and it should be enough to check it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17141: Create CpuQueryLimit implementation [solr]
sigram commented on code in PR #2244:
URL: https://github.com/apache/solr/pull/2244#discussion_r1489775946
##
solr/core/src/java/org/apache/solr/handler/component/SearchHandler.java:
##
@@ -622,8 +622,8 @@ public void handleRequestBody(SolrQueryRequest req,
SolrQueryResponse rsp) throw
} while (nextStage != Integer.MAX_VALUE);
if (publishCpuTime) {
-rsp.getResponseHeader().add(ThreadStats.CPU_TIME, totalShardCpuTime);
-rsp.addToLog(ThreadStats.CPU_TIME, totalShardCpuTime);
+rsp.getResponseHeader().add(ThreadCpuTime.CPU_TIME, totalShardCpuTime);
Review Comment:
This can never be null due to the assignment that depends on the same
condition in the if() around line 221.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17141: Create CpuQueryLimit implementation [solr]
sigram commented on code in PR #2244:
URL: https://github.com/apache/solr/pull/2244#discussion_r1489766927
##
solr/core/src/java/org/apache/solr/handler/RequestHandlerBase.java:
##
@@ -250,17 +253,17 @@ public void handleRequest(SolrQueryRequest req,
SolrQueryResponse rsp) {
long elapsed = timer.stop();
metrics.totalTime.inc(elapsed);
- if (cpuStats != null) {
-Optional cpuTime = cpuStats.getCpuTimeMs();
+ if (publishCpuTime) {
+Optional cpuTime = threadCpuTime.getCpuTimeMs();
if (cpuTime.isPresent()) {
// add CPU_TIME if not already added by SearchHandler
NamedList header = rsp.getResponseHeader();
if (header != null) {
-if (header.get(ThreadStats.CPU_TIME) == null) {
- header.add(ThreadStats.CPU_TIME, cpuTime.get());
+if (header.get(ThreadCpuTime.CPU_TIME) == null) {
Review Comment:
I believe the idea was to make sure the total CPU time is added to response
from any handler, not just the `SearchHandler` where it's calculated
differently as a sum of per-shard times.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17141: Create CpuQueryLimit implementation [solr]
sigram commented on code in PR #2244: URL: https://github.com/apache/solr/pull/2244#discussion_r1489762243 ## solr/core/src/java/org/apache/solr/handler/RequestHandlerBase.java: ## @@ -76,7 +77,7 @@ public abstract class RequestHandlerBase private PluginInfo pluginInfo; - protected boolean publishCpuTime = Boolean.getBoolean(ThreadStats.ENABLE_CPU_TIME); + protected boolean publishCpuTime = Boolean.getBoolean(ThreadCpuTime.ENABLE_CPU_TIME); Review Comment: They are not. It's true that `ThreadCpuTimer` will be created on first call to `SolrRequestInfo.getThreadCpuTimer()`, whoever first calls it (either due to this sysprop being true, or due to the `cpuAllowed` query param). However, after that the logging will happen only if the sysprop is true, again regardless of the `cpuAllowed` query param, and the limiting will happen regardless of the logging sysprop. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Clarify split parameter usage [solr]
HoustonPutman merged PR #2263: URL: https://github.com/apache/solr/pull/2263 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-16078) New solrj-core module
[
https://issues.apache.org/jira/browse/SOLR-16078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817440#comment-17817440
]
Houston Putman commented on SOLR-16078:
---
I like those options a lot.
> New solrj-core module
> -
>
> Key: SOLR-16078
> URL: https://issues.apache.org/jira/browse/SOLR-16078
> Project: Solr
> Issue Type: Sub-task
> Components: SolrJ
>Reporter: Jan Høydahl
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> We should introduce a solrj-core module that is as slim as possible wrt
> dependencies.
> A user will then add solrj-core as well as any other solj-xx modules needed
> for their use.
> By marking it {{\@lucene.experimental}} we can change the API of this
> solrj-core during 9.x as we move stuff into other modules.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17066: Switch most SolrClients away from core URLs [solr]
gerlowskija commented on PR #2254: URL: https://github.com/apache/solr/pull/2254#issuecomment-1944175236 Turns out doing this for our 'LB' clients is a little more involved than I initially thought, due to some of the additional public methods they have (e.g. `request(LBSolrClient.Req)`). I'm going to back the 'LB' changes out of this PR, and focus on only handling our "Concurrent" clients here. I'll open a separate PR for tackling the LB clients. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Created] (SOLR-17160) Bulk admin operations may fail because of max tracked requests
Pierre Salagnac created SOLR-17160:
--
Summary: Bulk admin operations may fail because of max tracked
requests
Key: SOLR-17160
URL: https://issues.apache.org/jira/browse/SOLR-17160
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Components: Backup/Restore
Affects Versions: 8.11, 9.5
Reporter: Pierre Salagnac
In {{{}CoreAdminHandler{}}}, we maintain in-memory the list of in-flight
requests and completed/failed request.
_Note they are core/replica level async requests, and not top level requests
which mostly at the collection level. Top level requests are tracked by storing
the async ID in a Zookeeper node, which is not related to this ticket._
For completed/failed requests, we only track a maximum of 100 requests by
dropping the oldest ones. The typical client in
{{CollectionHandlingUtils.waitForCoreAdminAsyncCallToComplete()}} polls status
of the submitted requests, with a retry loop until requests are completed. If
for some reason we have more than 100 requests that complete or fail on a node
before all statuses are polled by the client, the statuses are lost and the
client will fail with an unexpected error similar to:
{{Invalid status request for requestId: '{_}{_}' - 'notfound'. Retried
__ times}}
Instead of having a hard limit for the number of requests we track, we could
have time based eviction. I think it makes sense to keep status of a request
until a given timeout, and then drop it ignoring how many requests we currently
track.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-13350: Multithreaded search [solr]
cpoerschke commented on code in PR #2248:
URL: https://github.com/apache/solr/pull/2248#discussion_r1489700199
##
solr/core/src/java/org/apache/solr/search/SolrIndexSearcher.java:
##
@@ -1867,35 +1873,79 @@ public ScoreMode scoreMode() {
qr.setNextCursorMark(cmd.getCursorMark());
hitsRelation = Relation.EQUAL_TO;
} else {
- final TopDocsCollector topCollector = buildTopDocsCollector(len, cmd);
- MaxScoreCollector maxScoreCollector = null;
- Collector collector = topCollector;
- if ((cmd.getFlags() & GET_SCORES) != 0) {
-maxScoreCollector = new MaxScoreCollector();
-collector = MultiCollector.wrap(topCollector, maxScoreCollector);
+ if (log.isInfoEnabled()) {
+log.info("calling from 2, query: {}", query.getClass());
}
- ScoreMode scoreModeUsed =
- buildAndRunCollectorChain(qr, query, collector, cmd,
pf.postFilter).scoreMode();
+ MTCollectorQueryCheck allowMT = new MTCollectorQueryCheck();
Review Comment:
https://github.com/apache/solr/pull/2248/commits/c69006a4ca6625307e17fb0c272b08eebacd9a69
and
https://github.com/apache/solr/pull/2248/commits/1ec21d4274fb10eaeec2fed04093b91cf1e88c72
factored something out. Not thought much about naming in doing so and not
specifically in response to this comment but just from my code
reading/learning/reviewing perspective.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] solr.xml: honor plugin enable=true|false [solr]
cpoerschke commented on PR #2260: URL: https://github.com/apache/solr/pull/2260#issuecomment-1944079222 Wondering about having similar at lines 646 and 805 in this file. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17141: Create CpuQueryLimit implementation [solr]
gus-asf commented on code in PR #2244:
URL: https://github.com/apache/solr/pull/2244#discussion_r1489655442
##
solr/core/src/java/org/apache/solr/handler/RequestHandlerBase.java:
##
@@ -250,17 +253,17 @@ public void handleRequest(SolrQueryRequest req,
SolrQueryResponse rsp) {
long elapsed = timer.stop();
metrics.totalTime.inc(elapsed);
- if (cpuStats != null) {
-Optional cpuTime = cpuStats.getCpuTimeMs();
+ if (publishCpuTime) {
Review Comment:
It looks like the present implementation cannot avoid logging cpu time if
cpuAllowed is used?
##
solr/core/src/java/org/apache/solr/handler/component/SearchHandler.java:
##
@@ -622,8 +622,8 @@ public void handleRequestBody(SolrQueryRequest req,
SolrQueryResponse rsp) throw
} while (nextStage != Integer.MAX_VALUE);
if (publishCpuTime) {
-rsp.getResponseHeader().add(ThreadStats.CPU_TIME, totalShardCpuTime);
-rsp.addToLog(ThreadStats.CPU_TIME, totalShardCpuTime);
+rsp.getResponseHeader().add(ThreadCpuTime.CPU_TIME, totalShardCpuTime);
Review Comment:
No null check here?
##
solr/core/src/java/org/apache/solr/request/SolrRequestInfo.java:
##
@@ -236,12 +240,29 @@ private void initQueryLimits() {
* empty) {@link QueryLimits} object if it has not been created, and will
then return the same
* object on every subsequent invocation.
*
- * @return The {@code QueryLimits} object for the current requet.
+ * @return The {@code QueryLimits} object for the current request.
*/
public QueryLimits getLimits() {
+// make sure the ThreadCpuTime is always initialized
+getThreadCpuTime();
return req == null
? QueryLimits.NONE
-: (QueryLimits) req.getContext().computeIfAbsent(LIMITS_KEY, (k) ->
new QueryLimits(req));
+: (QueryLimits)
+req.getContext()
+.computeIfAbsent(LIMITS_KEY, (k) -> new QueryLimits(req,
getThreadCpuTime()));
Review Comment:
I don't like this API for QueryLimits constructor. Do we really want the
possibility that it takes time limit from the request but the CPU time tracker
from somewhere else? The `getThreadCpuTime()` call is just checking the request
here, but this API leaves room for someone to do otherwise, and seems
needlessly verbose.
##
solr/core/src/java/org/apache/solr/handler/RequestHandlerBase.java:
##
@@ -250,17 +253,17 @@ public void handleRequest(SolrQueryRequest req,
SolrQueryResponse rsp) {
long elapsed = timer.stop();
metrics.totalTime.inc(elapsed);
- if (cpuStats != null) {
-Optional cpuTime = cpuStats.getCpuTimeMs();
+ if (publishCpuTime) {
+Optional cpuTime = threadCpuTime.getCpuTimeMs();
if (cpuTime.isPresent()) {
// add CPU_TIME if not already added by SearchHandler
NamedList header = rsp.getResponseHeader();
if (header != null) {
-if (header.get(ThreadStats.CPU_TIME) == null) {
- header.add(ThreadStats.CPU_TIME, cpuTime.get());
+if (header.get(ThreadCpuTime.CPU_TIME) == null) {
Review Comment:
Is this correct? When would this get called twice for the same `rsp`? if so
it appears that we would ignore a second invocation (but the `addToLog` below
doesn't ignore it so they will differ?
##
solr/core/src/java/org/apache/solr/handler/RequestHandlerBase.java:
##
@@ -76,7 +77,7 @@ public abstract class RequestHandlerBase
private PluginInfo pluginInfo;
- protected boolean publishCpuTime =
Boolean.getBoolean(ThreadStats.ENABLE_CPU_TIME);
+ protected boolean publishCpuTime =
Boolean.getBoolean(ThreadCpuTime.ENABLE_CPU_TIME);
Review Comment:
why does this default to false? This might make sense for logging but is
strange for a query parameter. I think the logging and queryLimit aspects have
become tightly coupled...
##
solr/core/src/java/org/apache/solr/search/CpuQueryTimeLimit.java:
##
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.search;
+
+import com.google.common.annotations.VisibleForTesting;
+import java.lang.invoke.M
Re: [PR] SOLR-16995: Configure replication behaviour for each replica type [solr]
pvcnt commented on code in PR #2207:
URL: https://github.com/apache/solr/pull/2207#discussion_r1489634706
##
solr/core/src/java/org/apache/solr/update/UpdateHandler.java:
##
@@ -116,10 +116,13 @@ public UpdateHandler(SolrCore core, UpdateLog updateLog) {
parseEventListeners();
PluginInfo ulogPluginInfo =
core.getSolrConfig().getPluginInfo(UpdateLog.class.getName());
-// If this is a replica of type PULL, don't create the update log
+// If this replica doesn't have a transaction log, don't create it
Review Comment:
Indeed, I changed this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Build: ge.gradle add tags & projectId=solr [solr]
clayburn commented on PR #2256: URL: https://github.com/apache/solr/pull/2256#issuecomment-1943947802 Yes, the concept would be the same. It just deals with a few implementation specific details regarding where the build scan data is stored on the build runner, moving it to the privileged runner, and publishing it from there -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Build: ge.gradle add tags & projectId=solr [solr]
dsmiley commented on PR #2256: URL: https://github.com/apache/solr/pull/2256#issuecomment-1943944071 Aah! I will move my code to a branch then. I may need to close this PR to do that. > where the build runs in the fork, but the scan is uploaded in a separate privileged step Would this also be useful for external build machines (like Crave.io as we discussed)? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Build: ge.gradle add tags & projectId=solr [solr]
clayburn commented on PR #2256: URL: https://github.com/apache/solr/pull/2256#issuecomment-1943935338 I believe it is because these builds are coming from forks, which will not have access to `secrets.GE_ACCESS_TOKEN` as it is an organization level secret under the Apache org. This is something we hope to support in the future with a different workflow (where the build runs in the fork, but the scan is uploaded in a separate privileged step). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17141: Create CpuQueryLimit implementation [solr]
gus-asf commented on code in PR #2244:
URL: https://github.com/apache/solr/pull/2244#discussion_r1489578308
##
solr/core/src/java/org/apache/solr/search/QueryLimits.java:
##
@@ -41,11 +43,15 @@ private QueryLimits() {}
* statement will hinge on hasXXXLimit() static method attached to the
implementation class.
*
* @param req the current SolrQueryRequest.
+ * @param threadCpuTime current thread CPU time monitor.
*/
- public QueryLimits(SolrQueryRequest req) {
+ public QueryLimits(SolrQueryRequest req, ThreadCpuTime threadCpuTime) {
if (hasTimeLimit(req)) {
limits.add(new SolrQueryTimeLimit(req));
}
+if (hasCpuLimit(req)) {
+ limits.add(new CpuQueryTimeLimit(req, threadCpuTime));
Review Comment:
Actually, I like the idea of tracking the parameter names.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-13350: Multithreaded search [solr]
dsmiley commented on PR #2248: URL: https://github.com/apache/solr/pull/2248#issuecomment-1943929588 Observation: I don't think we need the concurrent DocSet builder (the one from Netflix). Couldn't we build segment level FixedBitSets (no safety issue) and then at the end combine into a master FixedBitSet so that we have our DocSet? It could even be made to be long-aligned, and thus the final aggregation is just copying longs with System.arraycopy (no doc iteration! or long shifting). The boundary long would be XOR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17141: Create CpuQueryLimit implementation [solr]
dsmiley commented on code in PR #2244:
URL: https://github.com/apache/solr/pull/2244#discussion_r1489562698
##
solr/core/src/test/org/apache/solr/search/TestCpuQueryTimeLimit.java:
##
@@ -0,0 +1,126 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.search;
+
+import java.lang.invoke.MethodHandles;
+import java.util.concurrent.TimeUnit;
+import org.apache.solr.client.solrj.SolrClient;
+import org.apache.solr.client.solrj.request.CollectionAdminRequest;
+import org.apache.solr.client.solrj.response.QueryResponse;
+import org.apache.solr.cloud.CloudUtil;
+import org.apache.solr.cloud.MiniSolrCloudCluster;
+import org.apache.solr.cloud.SolrCloudTestCase;
+import org.apache.solr.util.ThreadCpuTime;
+import org.junit.Assume;
+import org.junit.Test;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class TestCpuQueryTimeLimit extends SolrCloudTestCase {
Review Comment:
Thinking of the new SolrClientTestRule stuff; it'd be cool if basic sharded
search/indexing could be done to allow a test like the one here to switch
between Standalone & SolrCloud trivially. I'm hoping to get there but the work
isn't done yet. FYI: https://issues.apache.org/jira/browse/SOLR-11872
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17141: Create CpuQueryLimit implementation [solr]
dsmiley commented on code in PR #2244:
URL: https://github.com/apache/solr/pull/2244#discussion_r1489552446
##
solr/core/src/test/org/apache/solr/search/TestCpuQueryTimeLimit.java:
##
@@ -0,0 +1,126 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.search;
+
+import java.lang.invoke.MethodHandles;
+import java.util.concurrent.TimeUnit;
+import org.apache.solr.client.solrj.SolrClient;
+import org.apache.solr.client.solrj.request.CollectionAdminRequest;
+import org.apache.solr.client.solrj.response.QueryResponse;
+import org.apache.solr.cloud.CloudUtil;
+import org.apache.solr.cloud.MiniSolrCloudCluster;
+import org.apache.solr.cloud.SolrCloudTestCase;
+import org.apache.solr.util.ThreadCpuTime;
+import org.junit.Assume;
+import org.junit.Test;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class TestCpuQueryTimeLimit extends SolrCloudTestCase {
Review Comment:
I wouldn't choose SolrCloudTestCase merely because you are not sure of this.
Remember that distributed-search existed before SolrCloud!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[PR] Clarify split parameter usage [solr]
radu-gheorghe opened a new pull request, #2263: URL: https://github.com/apache/solr/pull/2263 It's a minor change that we tested (if you just do `split=true` it seems to apply to all fields, which is rarely what you want). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-13350: Multithreaded search [solr]
cpoerschke commented on code in PR #2248:
URL: https://github.com/apache/solr/pull/2248#discussion_r1489380477
##
solr/core/src/java/org/apache/solr/search/SolrIndexSearcher.java:
##
@@ -1874,35 +1881,74 @@ public ScoreMode scoreMode() {
qr.setNextCursorMark(cmd.getCursorMark());
hitsRelation = Relation.EQUAL_TO;
} else {
- final TopDocsCollector topCollector = buildTopDocsCollector(len, cmd);
- MaxScoreCollector maxScoreCollector = null;
- Collector collector = topCollector;
- if ((cmd.getFlags() & GET_SCORES) != 0) {
-maxScoreCollector = new MaxScoreCollector();
-collector = MultiCollector.wrap(topCollector, maxScoreCollector);
+ if (log.isInfoEnabled()) {
+log.info("calling from 2, query: {}", query.getClass());
}
- final ScoreMode scoreModeUsed =
- buildAndRunCollectorChain(qr, query, collector, cmd,
pf.postFilter).scoreMode();
+ final TopDocs topDocs;
+ if (!allowMT(pf.postFilter, cmd, query)) {
+if (log.isInfoEnabled()) {
+ log.info("skipping collector manager");
+}
+final TopDocsCollector topCollector = buildTopDocsCollector(len,
cmd);
+MaxScoreCollector maxScoreCollector = null;
+Collector collector = topCollector;
+if (needScores) {
+ maxScoreCollector = new MaxScoreCollector();
+ collector = MultiCollector.wrap(topCollector, maxScoreCollector);
+}
+final ScoreMode scoreModeUsed =
+buildAndRunCollectorChain(qr, query, collector, cmd,
pf.postFilter).scoreMode();
+
+totalHits = topCollector.getTotalHits();
+topDocs = topCollector.topDocs(0, len);
+if (scoreModeUsed == ScoreMode.COMPLETE || scoreModeUsed ==
ScoreMode.COMPLETE_NO_SCORES) {
+ hitsRelation = TotalHits.Relation.EQUAL_TO;
+} else {
+ hitsRelation = topDocs.totalHits.relation;
+}
+if (cmd.getSort() != null && cmd.getQuery() instanceof RankQuery ==
false && needScores) {
+ TopFieldCollector.populateScores(topDocs.scoreDocs, this, query);
+}
+populateNextCursorMarkFromTopDocs(qr, cmd, topDocs);
+
+maxScore =
+totalHits > 0
+? (maxScoreCollector == null ? Float.NaN :
maxScoreCollector.getMaxScore())
+: 0.0f;
+nDocsReturned = topDocs.scoreDocs.length;
- totalHits = topCollector.getTotalHits();
- final TopDocs topDocs = topCollector.topDocs(0, len);
- if (scoreModeUsed == ScoreMode.COMPLETE || scoreModeUsed ==
ScoreMode.COMPLETE_NO_SCORES) {
-hitsRelation = TotalHits.Relation.EQUAL_TO;
} else {
-hitsRelation = topDocs.totalHits.relation;
- }
- if (cmd.getSort() != null
- && cmd.getQuery() instanceof RankQuery == false
- && (cmd.getFlags() & GET_SCORES) != 0) {
-TopFieldCollector.populateScores(topDocs.scoreDocs, this, query);
+if (log.isInfoEnabled()) {
+ log.info("using CollectorManager");
+}
+final SearchResult searchResult =
+searchCollectorManagers(len, cmd, query, true, needScores, false);
+final Object[] res = searchResult.result;
+final TopDocsResult result = (TopDocsResult) res[0];
+
+totalHits = result.totalHits;
+topDocs = result.topDocs;
+
+if (res.length > 1) {
+ MaxScoreResult result2 = (MaxScoreResult) res[1];
+ maxScore = totalHits > 0 ? result2.maxScore : 0.0f;
+} else {
+ maxScore = Float.NaN;
+}
+
+populateNextCursorMarkFromTopDocs(qr, cmd, topDocs);
+if (cmd.getSort() != null && !(cmd.getQuery() instanceof RankQuery) &&
needScores) {
+ TopFieldCollector.populateScores(topDocs.scoreDocs, this, query);
+}
+nDocsReturned = topDocs.scoreDocs.length;
+ScoreMode scoreModeUsed = searchResult.scoreMode;
+if (scoreModeUsed == ScoreMode.COMPLETE || scoreModeUsed ==
ScoreMode.COMPLETE_NO_SCORES) {
+ hitsRelation = TotalHits.Relation.EQUAL_TO;
+} else {
+ hitsRelation = topDocs.totalHits.relation;
+}
}
- populateNextCursorMarkFromTopDocs(qr, cmd, topDocs);
- maxScore =
- totalHits > 0
- ? (maxScoreCollector == null ? Float.NaN :
maxScoreCollector.getMaxScore())
- : 0.0f;
- nDocsReturned = topDocs.scoreDocs.length;
ids = new int[nDocsReturned];
scores = (cmd.getFlags() & GET_SCORES) != 0 ? new float[nDocsReturned] :
null;
Review Comment:
Well spotted about reuse of `needScores` above in two places, here it can be
used too I think.
```suggestion
scores = needScores ? new float[nDocsReturned] : null;
```
I've opened #2262 to change that separately for other places in this file
too. Am happy to resolve any merge conflicts that may arise for this pull
request here.
--
This
[PR] NO JIRA: remove duplicate '(cmd.getFlags() & GET_SCORES) != 0' calls [solr]
cpoerschke opened a new pull request, #2262: URL: https://github.com/apache/solr/pull/2262 @chatman included a few as part of #2248 and looking further there's a few more, so separating them out here. No JIRA ticket or CHANGES.txt entry needed I think. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-16995: Configure replication behaviour for each replica type [solr]
pvcnt commented on code in PR #2207:
URL: https://github.com/apache/solr/pull/2207#discussion_r1489287486
##
solr/solrj/src/java/org/apache/solr/common/cloud/Replica.java:
##
@@ -104,27 +104,35 @@ public enum Type {
* support NRT (soft commits) and RTG. Any {@link Type#NRT} replica can
become a leader. A shard
* leader will forward updates to all active {@link Type#NRT} and {@link
Type#TLOG} replicas.
*/
-NRT(true, CollectionAdminParams.NRT_REPLICAS),
+NRT(true, true, false, CollectionAdminParams.NRT_REPLICAS),
/**
* Writes to transaction log, but not to index, uses replication. Any
{@link Type#TLOG} replica
* can become leader (by first applying all local transaction log
elements). If a replica is of
* type {@link Type#TLOG} but is also the leader, it will behave as a
{@link Type#NRT}. A shard
* leader will forward updates to all active {@link Type#NRT} and {@link
Type#TLOG} replicas.
*/
-TLOG(true, CollectionAdminParams.TLOG_REPLICAS),
+TLOG(true, true, true, CollectionAdminParams.TLOG_REPLICAS),
/**
* Doesn’t index or writes to transaction log. Just replicates from {@link
Type#NRT} or {@link
* Type#TLOG} replicas. {@link Type#PULL} replicas can’t become shard
leaders (i.e., if there
* are only pull replicas in the collection at some point, updates will
fail same as if there is
* no leaders, queries continue to work), so they don’t even participate
in elections.
*/
-PULL(false, CollectionAdminParams.PULL_REPLICAS);
+PULL(false, false, true, CollectionAdminParams.PULL_REPLICAS);
public final boolean leaderEligible;
+public final boolean requireTransactionLog;
+public final boolean replicateFromLeader;
public final String numReplicasPropertyName;
-Type(boolean leaderEligible, String numReplicasPropertyName) {
+Type(
+boolean leaderEligible,
+boolean requireTransactionLog,
Review Comment:
I definitely prefer to separate by semantics, instead of assuming one is
always true when the other is. Otherwise, we might as well have isNrt, isPull
and isTlog booleans. :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-16995: Configure replication behaviour for each replica type [solr]
pvcnt commented on code in PR #2207:
URL: https://github.com/apache/solr/pull/2207#discussion_r1489285471
##
solr/core/src/java/org/apache/solr/core/CoreContainer.java:
##
@@ -2050,15 +2050,13 @@ public void reload(String name, UUID coreId) {
if (docCollection != null) {
Replica replica =
docCollection.getReplica(cd.getCloudDescriptor().getCoreNodeName());
assert replica != null : cd.getCloudDescriptor().getCoreNodeName() +
" had no replica";
- if (replica.getType() == Replica.Type.TLOG) { // TODO: needed here?
+ if (replica.getType().replicateFromLeader) {
getZkController().stopReplicationFromLeader(core.getName());
-if (!cd.getCloudDescriptor().isLeader()) {
- getZkController().startReplicationFromLeader(newCore.getName(),
true);
+if (!replica.getType().leaderEligible ||
!cd.getCloudDescriptor().isLeader()) {
Review Comment:
Good point, I don't remember any good reason for doing so, so I simplified
the condition.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-16995: Configure replication behaviour for each replica type [solr]
pvcnt commented on code in PR #2207:
URL: https://github.com/apache/solr/pull/2207#discussion_r1489280963
##
solr/core/src/java/org/apache/solr/cloud/RecoveryStrategy.java:
##
@@ -583,7 +587,7 @@ public final void doSyncOrReplicateRecovery(SolrCore core)
throws Exception {
}
}
-if (replicaType == Replica.Type.TLOG) {
+if (replicaType.replicateFromLeader) {
Review Comment:
This is because this method is only called for NRT and TLOG replicas.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-16995: Configure replication behaviour for each replica type [solr]
pvcnt commented on code in PR #2207:
URL: https://github.com/apache/solr/pull/2207#discussion_r1489279539
##
solr/core/src/java/org/apache/solr/cloud/RecoveryStrategy.java:
##
@@ -417,8 +419,10 @@ private final void doReplicateOnlyRecovery(SolrCore core)
throws InterruptedExce
log.error("Error while trying to recover. core={}", coreName, e);
} finally {
if (successfulRecovery) {
- log.info("Restarting background replicate from leader process");
- zkController.startReplicationFromLeader(coreName, false);
+ if (replicaType.replicateFromLeader) {
Review Comment:
No, because this method is only called for PULL replicas. But adding this as
a safeguard nonetheless.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17141: Create CpuQueryLimit implementation [solr]
sigram commented on code in PR #2244:
URL: https://github.com/apache/solr/pull/2244#discussion_r1489255103
##
solr/core/src/test/org/apache/solr/search/TestCpuQueryTimeLimit.java:
##
@@ -0,0 +1,126 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.search;
+
+import java.lang.invoke.MethodHandles;
+import java.util.concurrent.TimeUnit;
+import org.apache.solr.client.solrj.SolrClient;
+import org.apache.solr.client.solrj.request.CollectionAdminRequest;
+import org.apache.solr.client.solrj.response.QueryResponse;
+import org.apache.solr.cloud.CloudUtil;
+import org.apache.solr.cloud.MiniSolrCloudCluster;
+import org.apache.solr.cloud.SolrCloudTestCase;
+import org.apache.solr.util.ThreadCpuTime;
+import org.junit.Assume;
+import org.junit.Test;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class TestCpuQueryTimeLimit extends SolrCloudTestCase {
Review Comment:
I'm not sure if this pseudo-cluster covers the same code paths when making
distributed calls.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SolrIndexSearcher.getDocList[AndSet]NC style: more final local variables [solr]
cpoerschke merged PR #2258: URL: https://github.com/apache/solr/pull/2258 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17141: Create CpuQueryLimit implementation [solr]
sigram commented on code in PR #2244:
URL: https://github.com/apache/solr/pull/2244#discussion_r1489156800
##
solr/core/src/java/org/apache/solr/search/QueryLimits.java:
##
@@ -41,11 +43,15 @@ private QueryLimits() {}
* statement will hinge on hasXXXLimit() static method attached to the
implementation class.
*
* @param req the current SolrQueryRequest.
+ * @param threadCpuTime current thread CPU time monitor.
*/
- public QueryLimits(SolrQueryRequest req) {
+ public QueryLimits(SolrQueryRequest req, ThreadCpuTime threadCpuTime) {
if (hasTimeLimit(req)) {
limits.add(new SolrQueryTimeLimit(req));
}
+if (hasCpuLimit(req)) {
+ limits.add(new CpuQueryTimeLimit(req, threadCpuTime));
Review Comment:
It's still not too late to rename the whole family, for example:
* `SolrQueryTimeLimit` -> `TimeAllowedLimit`
* `CpuQueryTimeLimit` -> `CpuAllowedLimit`
* `MemQueryLimit` -> `MemAllowedLimit`
WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
