[jira] [Updated] (SOLR-17274) atomic update error when using json w/ multiple modifiers
[
https://issues.apache.org/jira/browse/SOLR-17274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Calvin Smith updated SOLR-17274:
Affects Version/s: (was: 8.9)
(was: 8.10)
(was: 8.11)
> atomic update error when using json w/ multiple modifiers
> -
>
> Key: SOLR-17274
> URL: https://issues.apache.org/jira/browse/SOLR-17274
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: UpdateRequestProcessors
>Affects Versions: 9.0, 9.1, 9.2, 9.3, 9.4, 9.5, 9.6
> Environment: Observed first in 9.5.0 w/ OpenJDK 11, then reproduced
> this simple test case in 9.6.0.
>Reporter: Calvin Smith
>Priority: Major
>
> I ran into a problem doing a json atomic update that tries to both `add` and
> `remove` a value for a multivalued field in a single update. I saw it
> initially in an instance that runs 9.5.0, and reproduced a minimal example
> using Solr 9.6.0.
> {{The only fields defined in the schema are:}}
> {code:java}
> required="true" stored="true"/>
>
> {code}
> {{`id` is also present, so I'm supplying docs with
> just an `id` and a multivalued `name` field. The real setup is more complex,
> but this is a minimal test case to illustrate the problem.}}
> {{Starting with an empty index, I add the following doc to the index
> successfully:}}
> {code:java}
> {"id": "1", "name": ["John Doe", "Jane Doe"]}{code}
> {{And I can query it, seeing the expected result:}}
> {code:java}
> {
> "responseHeader":{
> "status":0,
> "QTime":23,
> "params":{
> "q":"name:*"
> }
> },
> "response":{
> "numFound":1,
> "start":0,
> "numFoundExact":true,
> "docs":[{
> "id":"1",
> "name":["John Doe","Jane Doe"],
> "_version_":1797873599884820480
> }]
> }
> }{code}
> {{Next, I send an atomic update to modify the `name` field of that document
> by removing `Jane Doe` and adding `Janet Doe`:}}
> {code:java}
> {"id": "1", "name": {"add": "Janet Doe", "remove": "Jane Doe"}}{code}
> {{This atomic update that does both an `add` and a `remove` is something that
> used to work for us under Solr 6.6, but we just noticed that it fails in our
> production 9.5 instance and in 9.6, which I just downloaded to test against.}}
> {{The error in the solr.log indicates that Solr mistakenly interprets the
> `\{"add": "Janet Doe", "remove": "Jane Doe"}` as a nested document and then
> throws an exception because our schema doesn't have the `{_}root{_}` field
> that would be expected if we were using nested docs (which we don't use).}}
> {{Here's the full stack trace from `solr.log` (version 9.6.0):}}
> {noformat}
> 2024-05-01 17:49:02.479 ERROR (qtp2059461664-30-0.0.0.0-3) [c: s: r: x:atris
> t:0.0.0.0-3] o.a.s.h.RequestHandlerBase Client exception =>
> org.apache.solr.common.SolrException: Unable to index docs with children: the
> schema must include definitions for both a uniqueKey field and the '_root_'
> field, using the exact same fieldType
> at
> org.apache.solr.update.DocumentBuilder.unexpectedNestedDocException(DocumentBuilder.java:369)
> org.apache.solr.common.SolrException: Unable to index docs with children: the
> schema must include definitions for both a uniqueKey field and the '_root_'
> field, using the exact same fieldType
> at
> org.apache.solr.update.DocumentBuilder.unexpectedNestedDocException(DocumentBuilder.java:369)
> ~[?:?]
> at
> org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:153)
> ~[?:?]
> at
> org.apache.solr.update.AddUpdateCommand.makeLuceneDocs(AddUpdateCommand.java:213)
> ~[?:?]
> at
> org.apache.solr.update.DirectUpdateHandler2.updateDocOrDocValues(DirectUpdateHandler2.java:1056)
> ~[?:?]
> at
> org.apache.solr.update.DirectUpdateHandler2.doNormalUpdate(DirectUpdateHandler2.java:421)
> ~[?:?]
> at
> org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:375)
> ~[?:?]
> at
> org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:312)
> ~[?:?]
> at
> org.apache.solr.update.processor.RunUpdateProcessorFactory$RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:76)
> ~[?:?]
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:54)
> ~[?:?]
> at
> org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:270)
> ~[?:?]
> at
> org.apache.solr.update.processor.DistributedUpdateProcessor.doVersionAdd(DistributedUpdateProcessor.java:533)
> ~[?:?]
> at
>
[jira] [Updated] (SOLR-17274) atomic update error when using json w/ multiple modifiers
[
https://issues.apache.org/jira/browse/SOLR-17274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Calvin Smith updated SOLR-17274:
Description:
I ran into a problem doing a json atomic update that tries to both `add` and
`remove` a value for a multivalued field in a single update. I saw it initially
in an instance that runs 9.5.0, and reproduced a minimal example using Solr
9.6.0.
{{The only fields defined in the schema are:}}
{code:java}
{code}
{{`id` is also present, so I'm supplying docs with just
an `id` and a multivalued `name` field. The real setup is more complex, but
this is a minimal test case to illustrate the problem.}}
{{Starting with an empty index, I add the following doc to the index
successfully:}}
{code:java}
{"id": "1", "name": ["John Doe", "Jane Doe"]}{code}
{{And I can query it, seeing the expected result:}}
{code:java}
{
"responseHeader":{
"status":0,
"QTime":23,
"params":{
"q":"name:*"
}
},
"response":{
"numFound":1,
"start":0,
"numFoundExact":true,
"docs":[{
"id":"1",
"name":["John Doe","Jane Doe"],
"_version_":1797873599884820480
}]
}
}{code}
{{Next, I send an atomic update to modify the `name` field of that document by
removing `Jane Doe` and adding `Janet Doe`:}}
{code:java}
{"id": "1", "name": {"add": "Janet Doe", "remove": "Jane Doe"}}{code}
{{This atomic update that does both an `add` and a `remove` is something that
used to work for us under Solr 6.6, but we just noticed that it fails in our
production 9.5 instance and in 9.6, which I just downloaded to test against.}}
{{The error in the solr.log indicates that Solr mistakenly interprets the
`\{"add": "Janet Doe", "remove": "Jane Doe"}` as a nested document and then
throws an exception because our schema doesn't have the `{_}root{_}` field that
would be expected if we were using nested docs (which we don't use).}}
{{Here's the full stack trace from `solr.log` (version 9.6.0):}}
{noformat}
2024-05-01 17:49:02.479 ERROR (qtp2059461664-30-0.0.0.0-3) [c: s: r: x:atris
t:0.0.0.0-3] o.a.s.h.RequestHandlerBase Client exception =>
org.apache.solr.common.SolrException: Unable to index docs with children: the
schema must include definitions for both a uniqueKey field and the '_root_'
field, using the exact same fieldType
at
org.apache.solr.update.DocumentBuilder.unexpectedNestedDocException(DocumentBuilder.java:369)
org.apache.solr.common.SolrException: Unable to index docs with children: the
schema must include definitions for both a uniqueKey field and the '_root_'
field, using the exact same fieldType
at
org.apache.solr.update.DocumentBuilder.unexpectedNestedDocException(DocumentBuilder.java:369)
~[?:?]
at
org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:153)
~[?:?]
at
org.apache.solr.update.AddUpdateCommand.makeLuceneDocs(AddUpdateCommand.java:213)
~[?:?]
at
org.apache.solr.update.DirectUpdateHandler2.updateDocOrDocValues(DirectUpdateHandler2.java:1056)
~[?:?]
at
org.apache.solr.update.DirectUpdateHandler2.doNormalUpdate(DirectUpdateHandler2.java:421)
~[?:?]
at
org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:375)
~[?:?]
at
org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:312)
~[?:?]
at
org.apache.solr.update.processor.RunUpdateProcessorFactory$RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:76)
~[?:?]
at
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:54)
~[?:?]
at
org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:270)
~[?:?]
at
org.apache.solr.update.processor.DistributedUpdateProcessor.doVersionAdd(DistributedUpdateProcessor.java:533)
~[?:?]
at
org.apache.solr.update.processor.DistributedUpdateProcessor.lambda$versionAdd$0(DistributedUpdateProcessor.java:358)
~[?:?]
at
org.apache.solr.update.VersionBucket.runWithLock(VersionBucket.java:43) ~[?:?]
at
org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:355)
~[?:?]
at
org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:236)
~[?:?]
at
org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:111)
~[?:?]
at
org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.handleAdds(JsonLoader.java:553)
~[?:?]
at
org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.processUpdate(JsonLoader.java:183)
~[?:?]
at
org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.load(JsonLoader.java:151)
~[?:?]
at
[jira] [Created] (SOLR-17274) atomic update error when using json w/ multiple modifiers
Calvin Smith created SOLR-17274:
---
Summary: atomic update error when using json w/ multiple modifiers
Key: SOLR-17274
URL: https://issues.apache.org/jira/browse/SOLR-17274
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Components: UpdateRequestProcessors
Affects Versions: 9.6, 9.4, 9.3, 9.2, 9.1, 8.11, 8.10, 8.9, 9.0, 9.5
Environment: Observed first in 9.5.0 w/ OpenJDK 11, then reproduced
this simple test case in 9.6.0.
Reporter: Calvin Smith
I ran into a problem doing a json atomic update that tries to both `add` and
`remove`
a value for a multivalued field in a single update. I saw it initially in an
instance that runs
9.5.0, and reproduced a minimal example using Solr 9.6.0.
{{The only fields defined in the schema are:}}
{{}}
{code:java}
{code}
{{`id` is also present, so I'm supplying docs with just
an `id` and a multivalued `name` field. The real setup is more complex, but
this is a minimal test case to illustrate the problem.}}
{{Starting with an empty index, I add the following doc to the index
successfully:}}
{{}}
{code:java}
{"id": "1", "name": ["John Doe", "Jane Doe"]}{code}
{{And I can query it, seeing the expected result:}}
{{}}
{code:java}
{
"responseHeader":{
"status":0,
"QTime":23,
"params":{
"q":"name:*"
}
},
"response":{
"numFound":1,
"start":0,
"numFoundExact":true,
"docs":[{
"id":"1",
"name":["John Doe","Jane Doe"],
"_version_":1797873599884820480
}]
}
}{code}
{{Next, I send an atomic update to modify the `name` field of that document by
removing `Jane Doe` and adding `Janet Doe`:}}
{{}}
{code:java}
{"id": "1", "name": {"add": "Janet Doe", "remove": "Jane Doe"}}{code}
{{This atomic update that does both an `add` and a `remove` is something that
used to work for us under Solr 6.6, but we just noticed that it fails in our
production 9.5 instance and in 9.6, which I just downloaded to test against.}}
{{The error in the solr.log indicates that Solr mistakenly interprets the
`\{"add": "Janet Doe", "remove": "Jane Doe"}` as a nested document and then
throws an exception because our schema doesn't have the `_root_` field that
would be expected if we were using nested docs (which we don't use).}}
{{Here's the full stack trace from `solr.log` (version 9.6.0):}}
{{}}
{noformat}
2024-05-01 17:49:02.479 ERROR (qtp2059461664-30-0.0.0.0-3) [c: s: r: x:atris
t:0.0.0.0-3] o.a.s.h.RequestHandlerBase Client exception =>
org.apache.solr.common.SolrException: Unable to index docs with children: the
schema must include definitions for both a uniqueKey field and the '_root_'
field, using the exact same fieldType
at
org.apache.solr.update.DocumentBuilder.unexpectedNestedDocException(DocumentBuilder.java:369)
org.apache.solr.common.SolrException: Unable to index docs with children: the
schema must include definitions for both a uniqueKey field and the '_root_'
field, using the exact same fieldType
at
org.apache.solr.update.DocumentBuilder.unexpectedNestedDocException(DocumentBuilder.java:369)
~[?:?]
at
org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:153)
~[?:?]
at
org.apache.solr.update.AddUpdateCommand.makeLuceneDocs(AddUpdateCommand.java:213)
~[?:?]
at
org.apache.solr.update.DirectUpdateHandler2.updateDocOrDocValues(DirectUpdateHandler2.java:1056)
~[?:?]
at
org.apache.solr.update.DirectUpdateHandler2.doNormalUpdate(DirectUpdateHandler2.java:421)
~[?:?]
at
org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:375)
~[?:?]
at
org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:312)
~[?:?]
at
org.apache.solr.update.processor.RunUpdateProcessorFactory$RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:76)
~[?:?]
at
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:54)
~[?:?]
at
org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:270)
~[?:?]
at
org.apache.solr.update.processor.DistributedUpdateProcessor.doVersionAdd(DistributedUpdateProcessor.java:533)
~[?:?]
at
org.apache.solr.update.processor.DistributedUpdateProcessor.lambda$versionAdd$0(DistributedUpdateProcessor.java:358)
~[?:?]
at
org.apache.solr.update.VersionBucket.runWithLock(VersionBucket.java:43) ~[?:?]
at
org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:355)
~[?:?]
at
org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:236)
~[?:?]
at
Re: [PR] SOLR-4587: integrate lucene-monitor into solr [solr]
dsmiley commented on code in PR #2382:
URL: https://github.com/apache/solr/pull/2382#discussion_r1589787252
##
solr/modules/monitor/src/java/org/apache/solr/monitor/search/ReverseQueryParserPlugin.java:
##
@@ -0,0 +1,105 @@
+/*
+ *
+ * * Licensed to the Apache Software Foundation (ASF) under one or more
+ * * contributor license agreements. See the NOTICE file distributed with
+ * * this work for additional information regarding copyright ownership.
+ * * The ASF licenses this file to You under the Apache License, Version 2.0
+ * * (the "License"); you may not use this file except in compliance with
+ * * the License. You may obtain a copy of the License at
+ * *
+ * * http://www.apache.org/licenses/LICENSE-2.0
+ * *
+ * * Unless required by applicable law or agreed to in writing, software
+ * * distributed under the License is distributed on an "AS IS" BASIS,
+ * * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * * See the License for the specific language governing permissions and
+ * * limitations under the License.
+ *
+ */
+
+package org.apache.solr.monitor.search;
+
+import static
org.apache.solr.monitor.search.PresearcherFactory.PresearcherParameters;
+
+import java.io.IOException;
+import java.util.Optional;
+import org.apache.lucene.monitor.Presearcher;
+import org.apache.lucene.util.ResourceLoader;
+import org.apache.lucene.util.ResourceLoaderAware;
+import org.apache.solr.common.params.SolrParams;
+import org.apache.solr.common.util.NamedList;
+import org.apache.solr.request.SolrQueryRequest;
+import org.apache.solr.search.QParser;
+import org.apache.solr.search.QParserPlugin;
+
+public class ReverseQueryParserPlugin extends QParserPlugin implements
ResourceLoaderAware {
+
+ public static final String NAME = "reverse";
+
+ // The default prefix is long because we don't want it to clash
+ // with a user-defined name pattern and the longest one wins.
+ // {@link
+ //
https://cwiki.apache.org/confluence/display/solr/SchemaXml#Dynamic_fields:~:text=Longer%20patterns%20will%20be%20matched%20first}
+ // In the worst case this can be overridden by the user but ideally this
never comes up.
+ private static final String DEFAULT_ALIAS_PREFIX =
+ "monitor_alias_";
+
+ private Presearcher presearcher;
+ private PresearcherParameters presearcherParameters;
+
+ @Override
+ public QParser createParser(
+ String qstr, SolrParams localParams, SolrParams params, SolrQueryRequest
req) {
+return new ReverseQueryParser(qstr, localParams, params, req, presearcher);
+ }
+
+ @Override
+ public void close() throws IOException {
+super.close();
+ }
+
+ @Override
+ public void init(NamedList args) {
+super.init(args);
+String presearcherType = (String) args.get("presearcherType");
+String termWeightorType = (String) args.get("termWeightorType");
+boolean applyFieldNameAlias =
+resolveProperty(args, "applyFieldNameAlias", Boolean.class, false);
+int numberOfPasses = resolveProperty(args, "numberOfPasses",
Integer.class, 0);
+float minWeight = resolveProperty(args, "minWeight", Float.class, 0f);
+String aliasPrefix =
+Optional.ofNullable((String)
args.get("aliasPrefix")).orElse(DEFAULT_ALIAS_PREFIX);
+presearcherParameters =
+new PresearcherParameters(
+presearcherType,
+termWeightorType,
+applyFieldNameAlias,
+numberOfPasses,
+minWeight,
+aliasPrefix);
+ }
+
+ // TODO is there a better pattern for this? NamedList::get(String key, T
default) can't be called
+ // on NamedList
Review Comment:
Ah yes, invokeSetters.
I'm not recommending it, but Solr also has a `ConfigurablePlugin` thing that
presently is unique to `ContainerPluginsRegistry` which assumes JSON to map to
plugin-specific config classes. It even goes from a Map to JSON to parsing it
again via Jackson for the sole purpose of using this mapping framework.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17263: HttpJdkSolrClient doesn't encode curly braces etc (follow-up PR) [solr]
jdyer1 commented on code in PR #2435: URL: https://github.com/apache/solr/pull/2435#discussion_r1589744476 ## solr/solrj/src/java/org/apache/solr/client/solrj/impl/HttpJdkSolrClient.java: ## @@ -238,7 +238,7 @@ private PreparedRequest prepareGet( validateGetRequest(solrRequest); reqb.GET(); decorateRequest(reqb, solrRequest); -reqb.uri(new URI(url + "?" + queryParams)); +reqb.uri(new URI(url + queryParams.toQueryString())); Review Comment: I had removed the change from lines 305 and 316 as adding the `fq` parameter you supplied the unit test only seemed to show that `GET` was broken. Likes 305 & 316 are building `POST` or `PUT` requests. On the other hand, so long as the extra ? is removed, adding `toQueryString()` there doesn't seem to break anything either. Yet again, I do not like "fixing" things if I cannot first demonstrate it is broken. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17049: Fix Replica Down on startup logic [solr]
dsmiley commented on code in PR #2432:
URL: https://github.com/apache/solr/pull/2432#discussion_r1589735301
##
solr/solrj/src/java/org/apache/solr/common/cloud/ClusterState.java:
##
@@ -181,29 +183,46 @@ public String getShardId(String nodeName, String
coreName) {
}
public String getShardId(String collectionName, String nodeName, String
coreName) {
Review Comment:
Can mark deprecated for now; don't need to remove it yet.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-4587: integrate lucene-monitor into solr [solr]
kotman12 commented on code in PR #2382:
URL: https://github.com/apache/solr/pull/2382#discussion_r1589698538
##
solr/modules/monitor/src/java/org/apache/solr/monitor/search/ReverseSearchComponent.java:
##
@@ -0,0 +1,192 @@
+/*
+ *
+ * * Licensed to the Apache Software Foundation (ASF) under one or more
+ * * contributor license agreements. See the NOTICE file distributed with
+ * * this work for additional information regarding copyright ownership.
+ * * The ASF licenses this file to You under the Apache License, Version 2.0
+ * * (the "License"); you may not use this file except in compliance with
+ * * the License. You may obtain a copy of the License at
+ * *
+ * * http://www.apache.org/licenses/LICENSE-2.0
+ * *
+ * * Unless required by applicable law or agreed to in writing, software
+ * * distributed under the License is distributed on an "AS IS" BASIS,
+ * * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * * See the License for the specific language governing permissions and
+ * * limitations under the License.
+ *
+ */
+
+package org.apache.solr.monitor.search;
+
+import static org.apache.solr.monitor.MonitorConstants.DOCUMENT_BATCH_KEY;
+import static org.apache.solr.monitor.MonitorConstants.MONITOR_DOCUMENTS_KEY;
+import static org.apache.solr.monitor.MonitorConstants.MONITOR_OUTPUT_KEY;
+import static org.apache.solr.monitor.MonitorConstants.QUERY_MATCH_TYPE_KEY;
+import static
org.apache.solr.monitor.MonitorConstants.REVERSE_SEARCH_PARAM_NAME;
+import static org.apache.solr.monitor.MonitorConstants.SOLR_MONITOR_CACHE_NAME;
+import static org.apache.solr.monitor.MonitorConstants.WRITE_TO_DOC_LIST_KEY;
+import static
org.apache.solr.monitor.search.QueryMatchResponseCodec.mergeResponses;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.concurrent.ExecutorService;
+import java.util.stream.Collectors;
+import org.apache.lucene.document.Document;
+import org.apache.lucene.monitor.DocumentBatchVisitor;
+import org.apache.lucene.search.MatchAllDocsQuery;
+import org.apache.lucene.search.Query;
+import org.apache.lucene.util.NamedThreadFactory;
+import org.apache.solr.common.SolrException;
+import org.apache.solr.common.SolrException.ErrorCode;
+import org.apache.solr.common.util.ExecutorUtil;
+import org.apache.solr.common.util.NamedList;
+import org.apache.solr.core.CloseHook;
+import org.apache.solr.core.SolrCore;
+import org.apache.solr.handler.component.ResponseBuilder;
+import org.apache.solr.handler.component.SearchComponent;
+import org.apache.solr.handler.component.ShardRequest;
+import org.apache.solr.handler.loader.JsonLoader;
+import org.apache.solr.monitor.SolrMonitorQueryDecoder;
+import org.apache.solr.monitor.cache.MonitorQueryCache;
+import org.apache.solr.monitor.cache.SharedMonitorCache;
+import org.apache.solr.request.SolrQueryRequest;
+import org.apache.solr.search.QParser;
+import org.apache.solr.search.SyntaxError;
+import org.apache.solr.update.DocumentBuilder;
+import org.apache.solr.util.plugin.SolrCoreAware;
+
+public class ReverseSearchComponent extends SearchComponent implements
SolrCoreAware {
+
+ private static final String MATCHER_THREAD_COUNT_KEY = "threadCount";
+
+ private SolrMatcherSinkFactory solrMatcherSinkFactory = new
SolrMatcherSinkFactory();
+
+ private ExecutorService executor;
+
+ @Override
+ public void init(NamedList args) {
+super.init(args);
+Object threadCount = args.get(MATCHER_THREAD_COUNT_KEY);
+if (threadCount instanceof Integer) {
+ executor =
+ ExecutorUtil.newMDCAwareFixedThreadPool(
+ (Integer) threadCount, new
NamedThreadFactory("monitor-matcher"));
+ solrMatcherSinkFactory = new SolrMatcherSinkFactory(executor);
+}
+ }
+
+ @Override
+ public void prepare(ResponseBuilder rb) {
+if (skipReverseSearch(rb)) {
+ return;
+}
+var req = rb.req;
+var documentBatch = documentBatch(req);
+req.getContext().put(DOCUMENT_BATCH_KEY, documentBatch);
+var writeToDocListRaw = req.getParams().get(WRITE_TO_DOC_LIST_KEY,
"false");
+boolean writeToDocList = Boolean.parseBoolean(writeToDocListRaw);
Review Comment:
I applied the change, but unresolving for further discussion.
So this flag may seem a bit strange without context. Basically, the proposal
has two ways of generating responses. The first is just a regular solr document
list. The more I think about it, the more I am convinced that this is all that
we _really_ need. However, if you run the tests you'll see there is a
second/optional/overlapping response section which I've called `monitor.{ .. }`
which maps to output generated by lucene-monitor's candidate matchers. This
could include things like offsets of matches. So far I let the client decide
which response sections they want.
The main
Re: [PR] SOLR-4587: integrate lucene-monitor into solr [solr]
kotman12 commented on code in PR #2382:
URL: https://github.com/apache/solr/pull/2382#discussion_r1589698538
##
solr/modules/monitor/src/java/org/apache/solr/monitor/search/ReverseSearchComponent.java:
##
@@ -0,0 +1,192 @@
+/*
+ *
+ * * Licensed to the Apache Software Foundation (ASF) under one or more
+ * * contributor license agreements. See the NOTICE file distributed with
+ * * this work for additional information regarding copyright ownership.
+ * * The ASF licenses this file to You under the Apache License, Version 2.0
+ * * (the "License"); you may not use this file except in compliance with
+ * * the License. You may obtain a copy of the License at
+ * *
+ * * http://www.apache.org/licenses/LICENSE-2.0
+ * *
+ * * Unless required by applicable law or agreed to in writing, software
+ * * distributed under the License is distributed on an "AS IS" BASIS,
+ * * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * * See the License for the specific language governing permissions and
+ * * limitations under the License.
+ *
+ */
+
+package org.apache.solr.monitor.search;
+
+import static org.apache.solr.monitor.MonitorConstants.DOCUMENT_BATCH_KEY;
+import static org.apache.solr.monitor.MonitorConstants.MONITOR_DOCUMENTS_KEY;
+import static org.apache.solr.monitor.MonitorConstants.MONITOR_OUTPUT_KEY;
+import static org.apache.solr.monitor.MonitorConstants.QUERY_MATCH_TYPE_KEY;
+import static
org.apache.solr.monitor.MonitorConstants.REVERSE_SEARCH_PARAM_NAME;
+import static org.apache.solr.monitor.MonitorConstants.SOLR_MONITOR_CACHE_NAME;
+import static org.apache.solr.monitor.MonitorConstants.WRITE_TO_DOC_LIST_KEY;
+import static
org.apache.solr.monitor.search.QueryMatchResponseCodec.mergeResponses;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.concurrent.ExecutorService;
+import java.util.stream.Collectors;
+import org.apache.lucene.document.Document;
+import org.apache.lucene.monitor.DocumentBatchVisitor;
+import org.apache.lucene.search.MatchAllDocsQuery;
+import org.apache.lucene.search.Query;
+import org.apache.lucene.util.NamedThreadFactory;
+import org.apache.solr.common.SolrException;
+import org.apache.solr.common.SolrException.ErrorCode;
+import org.apache.solr.common.util.ExecutorUtil;
+import org.apache.solr.common.util.NamedList;
+import org.apache.solr.core.CloseHook;
+import org.apache.solr.core.SolrCore;
+import org.apache.solr.handler.component.ResponseBuilder;
+import org.apache.solr.handler.component.SearchComponent;
+import org.apache.solr.handler.component.ShardRequest;
+import org.apache.solr.handler.loader.JsonLoader;
+import org.apache.solr.monitor.SolrMonitorQueryDecoder;
+import org.apache.solr.monitor.cache.MonitorQueryCache;
+import org.apache.solr.monitor.cache.SharedMonitorCache;
+import org.apache.solr.request.SolrQueryRequest;
+import org.apache.solr.search.QParser;
+import org.apache.solr.search.SyntaxError;
+import org.apache.solr.update.DocumentBuilder;
+import org.apache.solr.util.plugin.SolrCoreAware;
+
+public class ReverseSearchComponent extends SearchComponent implements
SolrCoreAware {
+
+ private static final String MATCHER_THREAD_COUNT_KEY = "threadCount";
+
+ private SolrMatcherSinkFactory solrMatcherSinkFactory = new
SolrMatcherSinkFactory();
+
+ private ExecutorService executor;
+
+ @Override
+ public void init(NamedList args) {
+super.init(args);
+Object threadCount = args.get(MATCHER_THREAD_COUNT_KEY);
+if (threadCount instanceof Integer) {
+ executor =
+ ExecutorUtil.newMDCAwareFixedThreadPool(
+ (Integer) threadCount, new
NamedThreadFactory("monitor-matcher"));
+ solrMatcherSinkFactory = new SolrMatcherSinkFactory(executor);
+}
+ }
+
+ @Override
+ public void prepare(ResponseBuilder rb) {
+if (skipReverseSearch(rb)) {
+ return;
+}
+var req = rb.req;
+var documentBatch = documentBatch(req);
+req.getContext().put(DOCUMENT_BATCH_KEY, documentBatch);
+var writeToDocListRaw = req.getParams().get(WRITE_TO_DOC_LIST_KEY,
"false");
+boolean writeToDocList = Boolean.parseBoolean(writeToDocListRaw);
Review Comment:
I applied the change, but unresolving for further discussion.
So this flag may seem a bit strange without context. Basically, the proposal
has two ways of generating responses. The first is just a regular solr document
list. The more I think about it, the more I am convinced that this is all that
we _really_ need. However, if you run the tests you'll see there is a
second/optional/overlapping response section which I've called `monitor.{ .. }`
which maps to output generated by lucene-monitor's candidate matchers. This
could include things like offsets of matches. So far I let the client decide
which response sections they want.
The main
Re: [PR] SOLR-4587: integrate lucene-monitor into solr [solr]
kotman12 commented on code in PR #2382:
URL: https://github.com/apache/solr/pull/2382#discussion_r1589699704
##
solr/modules/monitor/src/java/org/apache/solr/monitor/search/ReverseSearchComponent.java:
##
@@ -0,0 +1,192 @@
+/*
+ *
+ * * Licensed to the Apache Software Foundation (ASF) under one or more
+ * * contributor license agreements. See the NOTICE file distributed with
+ * * this work for additional information regarding copyright ownership.
+ * * The ASF licenses this file to You under the Apache License, Version 2.0
+ * * (the "License"); you may not use this file except in compliance with
+ * * the License. You may obtain a copy of the License at
+ * *
+ * * http://www.apache.org/licenses/LICENSE-2.0
+ * *
+ * * Unless required by applicable law or agreed to in writing, software
+ * * distributed under the License is distributed on an "AS IS" BASIS,
+ * * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * * See the License for the specific language governing permissions and
+ * * limitations under the License.
+ *
+ */
+
+package org.apache.solr.monitor.search;
+
+import static org.apache.solr.monitor.MonitorConstants.DOCUMENT_BATCH_KEY;
+import static org.apache.solr.monitor.MonitorConstants.MONITOR_DOCUMENTS_KEY;
+import static org.apache.solr.monitor.MonitorConstants.MONITOR_OUTPUT_KEY;
+import static org.apache.solr.monitor.MonitorConstants.QUERY_MATCH_TYPE_KEY;
+import static
org.apache.solr.monitor.MonitorConstants.REVERSE_SEARCH_PARAM_NAME;
+import static org.apache.solr.monitor.MonitorConstants.SOLR_MONITOR_CACHE_NAME;
+import static org.apache.solr.monitor.MonitorConstants.WRITE_TO_DOC_LIST_KEY;
+import static
org.apache.solr.monitor.search.QueryMatchResponseCodec.mergeResponses;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.concurrent.ExecutorService;
+import java.util.stream.Collectors;
+import org.apache.lucene.document.Document;
+import org.apache.lucene.monitor.DocumentBatchVisitor;
+import org.apache.lucene.search.MatchAllDocsQuery;
+import org.apache.lucene.search.Query;
+import org.apache.lucene.util.NamedThreadFactory;
+import org.apache.solr.common.SolrException;
+import org.apache.solr.common.SolrException.ErrorCode;
+import org.apache.solr.common.util.ExecutorUtil;
+import org.apache.solr.common.util.NamedList;
+import org.apache.solr.core.CloseHook;
+import org.apache.solr.core.SolrCore;
+import org.apache.solr.handler.component.ResponseBuilder;
+import org.apache.solr.handler.component.SearchComponent;
+import org.apache.solr.handler.component.ShardRequest;
+import org.apache.solr.handler.loader.JsonLoader;
+import org.apache.solr.monitor.SolrMonitorQueryDecoder;
+import org.apache.solr.monitor.cache.MonitorQueryCache;
+import org.apache.solr.monitor.cache.SharedMonitorCache;
+import org.apache.solr.request.SolrQueryRequest;
+import org.apache.solr.search.QParser;
+import org.apache.solr.search.SyntaxError;
+import org.apache.solr.update.DocumentBuilder;
+import org.apache.solr.util.plugin.SolrCoreAware;
+
+public class ReverseSearchComponent extends SearchComponent implements
SolrCoreAware {
+
+ private static final String MATCHER_THREAD_COUNT_KEY = "threadCount";
+
+ private SolrMatcherSinkFactory solrMatcherSinkFactory = new
SolrMatcherSinkFactory();
+
+ private ExecutorService executor;
+
+ @Override
+ public void init(NamedList args) {
+super.init(args);
+Object threadCount = args.get(MATCHER_THREAD_COUNT_KEY);
+if (threadCount instanceof Integer) {
+ executor =
+ ExecutorUtil.newMDCAwareFixedThreadPool(
+ (Integer) threadCount, new
NamedThreadFactory("monitor-matcher"));
+ solrMatcherSinkFactory = new SolrMatcherSinkFactory(executor);
+}
+ }
+
+ @Override
+ public void prepare(ResponseBuilder rb) {
+if (skipReverseSearch(rb)) {
+ return;
+}
+var req = rb.req;
+var documentBatch = documentBatch(req);
+req.getContext().put(DOCUMENT_BATCH_KEY, documentBatch);
+var writeToDocListRaw = req.getParams().get(WRITE_TO_DOC_LIST_KEY,
"false");
+boolean writeToDocList = Boolean.parseBoolean(writeToDocListRaw);
Review Comment:
Btw I am still working on your `ResearchSearchHandler` suggestion so that is
a WiP.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-4587: integrate lucene-monitor into solr [solr]
kotman12 commented on code in PR #2382:
URL: https://github.com/apache/solr/pull/2382#discussion_r1589698538
##
solr/modules/monitor/src/java/org/apache/solr/monitor/search/ReverseSearchComponent.java:
##
@@ -0,0 +1,192 @@
+/*
+ *
+ * * Licensed to the Apache Software Foundation (ASF) under one or more
+ * * contributor license agreements. See the NOTICE file distributed with
+ * * this work for additional information regarding copyright ownership.
+ * * The ASF licenses this file to You under the Apache License, Version 2.0
+ * * (the "License"); you may not use this file except in compliance with
+ * * the License. You may obtain a copy of the License at
+ * *
+ * * http://www.apache.org/licenses/LICENSE-2.0
+ * *
+ * * Unless required by applicable law or agreed to in writing, software
+ * * distributed under the License is distributed on an "AS IS" BASIS,
+ * * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * * See the License for the specific language governing permissions and
+ * * limitations under the License.
+ *
+ */
+
+package org.apache.solr.monitor.search;
+
+import static org.apache.solr.monitor.MonitorConstants.DOCUMENT_BATCH_KEY;
+import static org.apache.solr.monitor.MonitorConstants.MONITOR_DOCUMENTS_KEY;
+import static org.apache.solr.monitor.MonitorConstants.MONITOR_OUTPUT_KEY;
+import static org.apache.solr.monitor.MonitorConstants.QUERY_MATCH_TYPE_KEY;
+import static
org.apache.solr.monitor.MonitorConstants.REVERSE_SEARCH_PARAM_NAME;
+import static org.apache.solr.monitor.MonitorConstants.SOLR_MONITOR_CACHE_NAME;
+import static org.apache.solr.monitor.MonitorConstants.WRITE_TO_DOC_LIST_KEY;
+import static
org.apache.solr.monitor.search.QueryMatchResponseCodec.mergeResponses;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.concurrent.ExecutorService;
+import java.util.stream.Collectors;
+import org.apache.lucene.document.Document;
+import org.apache.lucene.monitor.DocumentBatchVisitor;
+import org.apache.lucene.search.MatchAllDocsQuery;
+import org.apache.lucene.search.Query;
+import org.apache.lucene.util.NamedThreadFactory;
+import org.apache.solr.common.SolrException;
+import org.apache.solr.common.SolrException.ErrorCode;
+import org.apache.solr.common.util.ExecutorUtil;
+import org.apache.solr.common.util.NamedList;
+import org.apache.solr.core.CloseHook;
+import org.apache.solr.core.SolrCore;
+import org.apache.solr.handler.component.ResponseBuilder;
+import org.apache.solr.handler.component.SearchComponent;
+import org.apache.solr.handler.component.ShardRequest;
+import org.apache.solr.handler.loader.JsonLoader;
+import org.apache.solr.monitor.SolrMonitorQueryDecoder;
+import org.apache.solr.monitor.cache.MonitorQueryCache;
+import org.apache.solr.monitor.cache.SharedMonitorCache;
+import org.apache.solr.request.SolrQueryRequest;
+import org.apache.solr.search.QParser;
+import org.apache.solr.search.SyntaxError;
+import org.apache.solr.update.DocumentBuilder;
+import org.apache.solr.util.plugin.SolrCoreAware;
+
+public class ReverseSearchComponent extends SearchComponent implements
SolrCoreAware {
+
+ private static final String MATCHER_THREAD_COUNT_KEY = "threadCount";
+
+ private SolrMatcherSinkFactory solrMatcherSinkFactory = new
SolrMatcherSinkFactory();
+
+ private ExecutorService executor;
+
+ @Override
+ public void init(NamedList args) {
+super.init(args);
+Object threadCount = args.get(MATCHER_THREAD_COUNT_KEY);
+if (threadCount instanceof Integer) {
+ executor =
+ ExecutorUtil.newMDCAwareFixedThreadPool(
+ (Integer) threadCount, new
NamedThreadFactory("monitor-matcher"));
+ solrMatcherSinkFactory = new SolrMatcherSinkFactory(executor);
+}
+ }
+
+ @Override
+ public void prepare(ResponseBuilder rb) {
+if (skipReverseSearch(rb)) {
+ return;
+}
+var req = rb.req;
+var documentBatch = documentBatch(req);
+req.getContext().put(DOCUMENT_BATCH_KEY, documentBatch);
+var writeToDocListRaw = req.getParams().get(WRITE_TO_DOC_LIST_KEY,
"false");
+boolean writeToDocList = Boolean.parseBoolean(writeToDocListRaw);
Review Comment:
I applied the change, but unresolving for further discussion.
So this flag may seem a bit strange without context. Basically, the proposal
has two ways of generating responses. The first is just a regular solr document
list. The more I think about it, the more I am convinced that this is all that
we _really_ need. However, if you run the tests you'll see there is a
second/optional/overlapping response section which I've called `monitor.{ .. }`
which maps to output generated by lucene-monitor's candidate matchers. This
could include things like offsets of matches. So far I let the client decide
which response sections they want.
The main
Re: [PR] SOLR-17192: Add "field-limiting" URP to catch ill-designed schemas [solr]
dsmiley commented on code in PR #2395:
URL: https://github.com/apache/solr/pull/2395#discussion_r1589670077
##
solr/core/src/java/org/apache/solr/update/processor/NumFieldLimitingUpdateRequestProcessorFactory.java:
##
@@ -0,0 +1,113 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.update.processor;
+
+import org.apache.solr.common.util.NamedList;
+import org.apache.solr.core.SolrCore;
+import org.apache.solr.request.SolrQueryRequest;
+import org.apache.solr.response.SolrQueryResponse;
+import org.apache.solr.search.SolrIndexSearcher;
+import org.apache.solr.util.plugin.SolrCoreAware;
+
+/**
+ * This factory generates an UpdateRequestProcessor which fails update
requests once a core has
+ * exceeded a configurable maximum number of fields. Meant as a safeguard to
help users notice
+ * potentially-dangerous schema design before performance and stability
problems start to occur.
+ *
+ * The URP uses the core's {@link SolrIndexSearcher} to judge the current
number of fields.
+ * Accordingly, it undercounts the number of fields in the core - missing all
fields added since the
+ * previous searcher was opened. As such, the URP's request-blocking is "best
effort" - it cannot be
+ * relied on as a precise limit on the number of fields.
+ *
+ * Additionally, the field-counting includes all documents present in the
index, including any
+ * deleted docs that haven't yet been purged via segment merging. Note that
this can differ
+ * significantly from the number of fields defined in managed-schema.xml -
especially when dynamic
+ * fields are enabled. The only way to reduce this field count is to delete
documents and wait until
+ * the deleted documents have been removed by segment merges. Users may of
course speed up this
+ * process by tweaking Solr's segment-merging, triggering an "optimize"
operation, etc.
+ *
+ * {@link NumFieldLimitingUpdateRequestProcessorFactory} accepts two
configuration parameters:
+ *
+ *
+ * maxFields - (required) The maximum number of fields
before update requests
+ * should be aborted. Once this limit has been exceeded, additional
update requests will fail
+ * until fields have been removed or the "maxFields" is increased.
+ * warnOnly - (optional) If true then the URP
logs verbose warnings
+ * about the limit being exceeded but doesn't abort update requests.
Defaults to false
+ *if not specified
+ *
+ *
+ * @since 9.6.0
+ */
+public class NumFieldLimitingUpdateRequestProcessorFactory extends
UpdateRequestProcessorFactory
+implements SolrCoreAware {
+
+ private static final String MAXIMUM_FIELDS_PARAM = "maxFields";
+ private static final String WARN_ONLY_PARAM = "warnOnly";
+
+ private NumFieldsMonitor numFieldsMonitor;
+ private int maximumFields;
+ private boolean warnOnly;
+
+ @Override
+ public void inform(final SolrCore core) {
+// register a commit callback for monitoring the number of fields in the
schema
+numFieldsMonitor = new NumFieldsMonitor(core);
+core.getUpdateHandler().registerCommitCallback(numFieldsMonitor);
+core.registerNewSearcherListener(numFieldsMonitor);
+ }
+
+ @Override
+ public void init(NamedList args) {
+warnOnly = args.indexOf(WARN_ONLY_PARAM, 0) > 0 ?
args.getBooleanArg(WARN_ONLY_PARAM) : false;
+
+if (args.indexOf(MAXIMUM_FIELDS_PARAM, 0) < 0) {
+ throw new IllegalArgumentException(
+ "The "
+ + MAXIMUM_FIELDS_PARAM
+ + " parameter is required for "
+ + getClass().getName()
+ + ", but no value was provided.");
+}
+final Object rawMaxFields = args.get(MAXIMUM_FIELDS_PARAM);
+if (rawMaxFields == null || !(rawMaxFields instanceof Integer)) {
+ throw new IllegalArgumentException(
+ MAXIMUM_FIELDS_PARAM + " must be configured as a non-null ");
+}
+maximumFields = (Integer) rawMaxFields;
+if (maximumFields <= 0) {
+ throw new IllegalArgumentException(MAXIMUM_FIELDS_PARAM + " must be a
positive integer");
+}
+ }
+
+ @Override
+ public UpdateRequestProcessor getInstance(
+ SolrQueryRequest req, SolrQueryResponse rsp, UpdateRequestProcessor
next) {
+
+
[jira] [Commented] (SOLR-16155) DocumentBuilder should not include field values in error messages
[ https://issues.apache.org/jira/browse/SOLR-16155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843332#comment-17843332 ] Jerry Chung commented on SOLR-16155: I still see this issue even with Solr 9.0 > DocumentBuilder should not include field values in error messages > - > > Key: SOLR-16155 > URL: https://issues.apache.org/jira/browse/SOLR-16155 > Project: Solr > Issue Type: Bug > Components: logging >Affects Versions: 8.11.1 >Reporter: Jerry Chung >Assignee: Jan Høydahl >Priority: Critical > Time Spent: 1h 20m > Remaining Estimate: 0h > > org.apache.solr.update.DocumentBuilder throws exceptions with field values on > errors, which could reveal customer's sensitive data on log files and UI. > Field values shouldn't be included in the error messages. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17192: Add "field-limiting" URP to catch ill-designed schemas [solr]
dsmiley commented on code in PR #2395:
URL: https://github.com/apache/solr/pull/2395#discussion_r1589657516
##
solr/core/src/java/org/apache/solr/update/processor/NumFieldLimitingUpdateRequestProcessorFactory.java:
##
@@ -0,0 +1,113 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.update.processor;
+
+import org.apache.solr.common.util.NamedList;
+import org.apache.solr.core.SolrCore;
+import org.apache.solr.request.SolrQueryRequest;
+import org.apache.solr.response.SolrQueryResponse;
+import org.apache.solr.search.SolrIndexSearcher;
+import org.apache.solr.util.plugin.SolrCoreAware;
+
+/**
+ * This factory generates an UpdateRequestProcessor which fails update
requests once a core has
+ * exceeded a configurable maximum number of fields. Meant as a safeguard to
help users notice
+ * potentially-dangerous schema design before performance and stability
problems start to occur.
+ *
+ * The URP uses the core's {@link SolrIndexSearcher} to judge the current
number of fields.
+ * Accordingly, it undercounts the number of fields in the core - missing all
fields added since the
+ * previous searcher was opened. As such, the URP's request-blocking is "best
effort" - it cannot be
+ * relied on as a precise limit on the number of fields.
+ *
+ * Additionally, the field-counting includes all documents present in the
index, including any
+ * deleted docs that haven't yet been purged via segment merging. Note that
this can differ
+ * significantly from the number of fields defined in managed-schema.xml -
especially when dynamic
+ * fields are enabled. The only way to reduce this field count is to delete
documents and wait until
+ * the deleted documents have been removed by segment merges. Users may of
course speed up this
+ * process by tweaking Solr's segment-merging, triggering an "optimize"
operation, etc.
+ *
+ * {@link NumFieldLimitingUpdateRequestProcessorFactory} accepts two
configuration parameters:
+ *
+ *
+ * maxFields - (required) The maximum number of fields
before update requests
+ * should be aborted. Once this limit has been exceeded, additional
update requests will fail
+ * until fields have been removed or the "maxFields" is increased.
+ * warnOnly - (optional) If true then the URP
logs verbose warnings
+ * about the limit being exceeded but doesn't abort update requests.
Defaults to false
+ *if not specified
+ *
+ *
+ * @since 9.6.0
+ */
+public class NumFieldLimitingUpdateRequestProcessorFactory extends
UpdateRequestProcessorFactory
+implements SolrCoreAware {
+
+ private static final String MAXIMUM_FIELDS_PARAM = "maxFields";
+ private static final String WARN_ONLY_PARAM = "warnOnly";
+
+ private NumFieldsMonitor numFieldsMonitor;
+ private int maximumFields;
+ private boolean warnOnly;
+
+ @Override
+ public void inform(final SolrCore core) {
+// register a commit callback for monitoring the number of fields in the
schema
+numFieldsMonitor = new NumFieldsMonitor(core);
+core.getUpdateHandler().registerCommitCallback(numFieldsMonitor);
+core.registerNewSearcherListener(numFieldsMonitor);
+ }
+
+ @Override
+ public void init(NamedList args) {
+warnOnly = args.indexOf(WARN_ONLY_PARAM, 0) > 0 ?
args.getBooleanArg(WARN_ONLY_PARAM) : false;
+
+if (args.indexOf(MAXIMUM_FIELDS_PARAM, 0) < 0) {
+ throw new IllegalArgumentException(
+ "The "
+ + MAXIMUM_FIELDS_PARAM
+ + " parameter is required for "
+ + getClass().getName()
+ + ", but no value was provided.");
+}
+final Object rawMaxFields = args.get(MAXIMUM_FIELDS_PARAM);
+if (rawMaxFields == null || !(rawMaxFields instanceof Integer)) {
+ throw new IllegalArgumentException(
+ MAXIMUM_FIELDS_PARAM + " must be configured as a non-null ");
+}
+maximumFields = (Integer) rawMaxFields;
+if (maximumFields <= 0) {
+ throw new IllegalArgumentException(MAXIMUM_FIELDS_PARAM + " must be a
positive integer");
+}
+ }
+
+ @Override
+ public UpdateRequestProcessor getInstance(
+ SolrQueryRequest req, SolrQueryResponse rsp, UpdateRequestProcessor
next) {
+
+
Re: [PR] SOLR-17192: Add "field-limiting" URP to catch ill-designed schemas [solr]
dsmiley commented on code in PR #2395:
URL: https://github.com/apache/solr/pull/2395#discussion_r1589647578
##
solr/core/src/java/org/apache/solr/update/processor/NumFieldsMonitor.java:
##
@@ -0,0 +1,56 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.update.processor;
+
+import org.apache.solr.core.AbstractSolrEventListener;
+import org.apache.solr.core.SolrCore;
+import org.apache.solr.search.SolrIndexSearcher;
+import org.apache.solr.util.RefCounted;
+
+public class NumFieldsMonitor extends AbstractSolrEventListener {
+
+ private int numFields;
+
+ public NumFieldsMonitor(SolrCore core) {
+super(core);
+this.numFields = -1;
+ }
+
+ @Override
+ public void postCommit() {
+RefCounted indexSearcher = null;
+try {
+ indexSearcher = getCore().getSearcher();
Review Comment:
Maybe; Java exception handling is such a PITA I didn't want to defeat
the lambda convenience if the lambda threw an IOException which is rather
common.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17192: Add "field-limiting" URP to catch ill-designed schemas [solr]
dsmiley commented on code in PR #2395:
URL: https://github.com/apache/solr/pull/2395#discussion_r1589645861
##
solr/core/src/test-files/solr/configsets/cloud-minimal-field-limiting/conf/solrconfig.xml:
##
@@ -0,0 +1,73 @@
+
+
+
+
+
+
+
+
+ ${solr.data.dir:}
+
+
+
+
+ ${tests.luceneMatchVersion:LATEST}
+
+
+
+ ${solr.commitwithin.softcommit:true}
+
+
+
+
+
+
+ explicit
+ true
+ text
+
+
+
+
+${solr.test.maxFields:1234}
+
Review Comment:
Having no-config vs config doesn't conceptually change where the
element can be placed.
ex:
``
vs
```
```
Can be in the same places.
I went to the ref guide's [page on
URPs](https://solr.apache.org/guide/solr/latest/configuration-guide/update-request-processors.html#custom-update-request-processor-chain)
and observe the first example shows a chain with several URPs, one of which
has config. I think it's natural to place URPs inline in the chain.
Separating it out is for composability if there are multiple chains and URPs
and you want to avoid possible repetition.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Updated] (SOLR-17270) Umbrella: Per Replica State as default
[ https://issues.apache.org/jira/browse/SOLR-17270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Smiley updated SOLR-17270: Component/s: SolrCloud > Umbrella: Per Replica State as default > -- > > Key: SOLR-17270 > URL: https://issues.apache.org/jira/browse/SOLR-17270 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Reporter: Justin Sweeney >Priority: Major > Labels: Umbrella > > The concept of Per Replica State (PRS) as a part of state management in Solr > has existed since 8.8.0. While PRS is in use in some places, there is still > work to be done to get it to a place where we can move to PRS as the default > behavior in Solr. > > This umbrella ticket will encapsulate the work required to get to this point > and enable PRS as the default behavior for Solr 10. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17248) Refactor ZK related SolrCli tools to separate SolrZkClient and CloudSolrClient instantiation/usage
[ https://issues.apache.org/jira/browse/SOLR-17248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843286#comment-17843286 ] ASF subversion and git services commented on SOLR-17248: Commit dd64a74db3625e7c3ad024996187e666b5158e3f in solr's branch refs/heads/branch_9x from Eric Pugh [ https://gitbox.apache.org/repos/asf?p=solr.git;h=dd64a74db36 ] Backporting SOLR-17248 to branch_9x (#2439) * SOLR-17248: Refactor ZK related SolrCli tools to separate SolrZkClient and CloudSolrClient instantiation/usage to fix branch_9x. Co-authored-by: Lamine <104593675+lamine...@users.noreply.github.com> Co-authored-by: Lamine Idjeraoui > Refactor ZK related SolrCli tools to separate SolrZkClient and > CloudSolrClient instantiation/usage > -- > > Key: SOLR-17248 > URL: https://issues.apache.org/jira/browse/SOLR-17248 > Project: Solr > Issue Type: Improvement >Reporter: Lamine >Assignee: Eric Pugh >Priority: Minor > Time Spent: 6h 10m > Remaining Estimate: 0h > > Many CLI tool-related classes currently duplicate SolrZkClient & > _CloudSolrClient_ instantiation, with the same code appearing 12 times across > 8 different classes. It would be beneficial to extract this instantiation and > reuse the code for easy maintainability > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Backporting SOLR-17248 to branch_9x [solr]
epugh merged PR #2439: URL: https://github.com/apache/solr/pull/2439 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-4587: integrate lucene-monitor into solr [solr]
cpoerschke commented on code in PR #2382:
URL: https://github.com/apache/solr/pull/2382#discussion_r1589485628
##
solr/modules/monitor/src/java/org/apache/solr/monitor/search/ReverseSearchComponent.java:
##
@@ -0,0 +1,192 @@
+/*
+ *
+ * * Licensed to the Apache Software Foundation (ASF) under one or more
+ * * contributor license agreements. See the NOTICE file distributed with
+ * * this work for additional information regarding copyright ownership.
+ * * The ASF licenses this file to You under the Apache License, Version 2.0
+ * * (the "License"); you may not use this file except in compliance with
+ * * the License. You may obtain a copy of the License at
+ * *
+ * * http://www.apache.org/licenses/LICENSE-2.0
+ * *
+ * * Unless required by applicable law or agreed to in writing, software
+ * * distributed under the License is distributed on an "AS IS" BASIS,
+ * * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * * See the License for the specific language governing permissions and
+ * * limitations under the License.
+ *
+ */
+
+package org.apache.solr.monitor.search;
+
+import static org.apache.solr.monitor.MonitorConstants.DOCUMENT_BATCH_KEY;
+import static org.apache.solr.monitor.MonitorConstants.MONITOR_DOCUMENTS_KEY;
+import static org.apache.solr.monitor.MonitorConstants.MONITOR_OUTPUT_KEY;
+import static org.apache.solr.monitor.MonitorConstants.QUERY_MATCH_TYPE_KEY;
+import static
org.apache.solr.monitor.MonitorConstants.REVERSE_SEARCH_PARAM_NAME;
+import static org.apache.solr.monitor.MonitorConstants.SOLR_MONITOR_CACHE_NAME;
+import static org.apache.solr.monitor.MonitorConstants.WRITE_TO_DOC_LIST_KEY;
+import static
org.apache.solr.monitor.search.QueryMatchResponseCodec.mergeResponses;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.concurrent.ExecutorService;
+import java.util.stream.Collectors;
+import org.apache.lucene.document.Document;
+import org.apache.lucene.monitor.DocumentBatchVisitor;
+import org.apache.lucene.search.MatchAllDocsQuery;
+import org.apache.lucene.search.Query;
+import org.apache.lucene.util.NamedThreadFactory;
+import org.apache.solr.common.SolrException;
+import org.apache.solr.common.SolrException.ErrorCode;
+import org.apache.solr.common.util.ExecutorUtil;
+import org.apache.solr.common.util.NamedList;
+import org.apache.solr.core.CloseHook;
+import org.apache.solr.core.SolrCore;
+import org.apache.solr.handler.component.ResponseBuilder;
+import org.apache.solr.handler.component.SearchComponent;
+import org.apache.solr.handler.component.ShardRequest;
+import org.apache.solr.handler.loader.JsonLoader;
+import org.apache.solr.monitor.SolrMonitorQueryDecoder;
+import org.apache.solr.monitor.cache.MonitorQueryCache;
+import org.apache.solr.monitor.cache.SharedMonitorCache;
+import org.apache.solr.request.SolrQueryRequest;
+import org.apache.solr.search.QParser;
+import org.apache.solr.search.SyntaxError;
+import org.apache.solr.update.DocumentBuilder;
+import org.apache.solr.util.plugin.SolrCoreAware;
+
+public class ReverseSearchComponent extends SearchComponent implements
SolrCoreAware {
+
+ private static final String MATCHER_THREAD_COUNT_KEY = "threadCount";
+
+ private SolrMatcherSinkFactory solrMatcherSinkFactory = new
SolrMatcherSinkFactory();
+
+ private ExecutorService executor;
+
+ @Override
+ public void init(NamedList args) {
+super.init(args);
+Object threadCount = args.get(MATCHER_THREAD_COUNT_KEY);
+if (threadCount instanceof Integer) {
+ executor =
+ ExecutorUtil.newMDCAwareFixedThreadPool(
+ (Integer) threadCount, new
NamedThreadFactory("monitor-matcher"));
+ solrMatcherSinkFactory = new SolrMatcherSinkFactory(executor);
+}
+ }
+
+ @Override
+ public void prepare(ResponseBuilder rb) {
+if (skipReverseSearch(rb)) {
+ return;
+}
+var req = rb.req;
+var documentBatch = documentBatch(req);
+req.getContext().put(DOCUMENT_BATCH_KEY, documentBatch);
+var writeToDocListRaw = req.getParams().get(WRITE_TO_DOC_LIST_KEY,
"false");
+boolean writeToDocList = Boolean.parseBoolean(writeToDocListRaw);
Review Comment:
Could this be combined like this?
```suggestion
boolean writeToDocList = req.getParams().getBool(WRITE_TO_DOC_LIST_KEY,
false);
```
(And just noting (to self) to continue to browse from here w.r.t. use of the
option in the code and from the user's perspective.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (SOLR-16866) CachingDirectoryFactory Throwing Error When Closing
[
https://issues.apache.org/jira/browse/SOLR-16866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843281#comment-17843281
]
Chris M. Hostetter commented on SOLR-16866:
---
The {{CachingDirectoryFactoryTest.reorderingTest}} method added by this issue
appears to be failing on many (every?) Windows jenkins build since it was
committed.
A recent main failure...
{noformat}
org.apache.solr.core.CachingDirectoryFactoryTest > reorderingTest FAILED
java.lang.AssertionError
at
__randomizedtesting.SeedInfo.seed([85529424332B55D9:FDFFFBBE80E21597]:0)
at app//org.junit.Assert.fail(Assert.java:87)
at app//org.junit.Assert.assertTrue(Assert.java:42)
at app//org.junit.Assert.assertTrue(Assert.java:53)
at
app//org.apache.solr.core.CachingDirectoryFactoryTest.reorderingTest(CachingDirectoryFactoryTest.java:97)
at
java.base@20.0.2/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
{noformat}
Line 97...
{code}
assertTrue(pathA.toFile().exists()); // we know there are subdirs that
should prevent removal
{code}
> CachingDirectoryFactory Throwing Error When Closing
> ---
>
> Key: SOLR-16866
> URL: https://issues.apache.org/jira/browse/SOLR-16866
> Project: Solr
> Issue Type: Bug
>Affects Versions: 9.2.1
>Reporter: Stefan Pieper
>Assignee: Michael Gibney
>Priority: Major
> Labels: core
> Fix For: 9.6
>
> Attachments: solr.blueprint_gnylctsgemcz_users.log
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> Observed occasional ERROR log entries when {{CachingDirectryFactory#close()}}
> is called. The respective snippet from the log is this:
> {noformat}
> 2023-07-06 08:43:37.618 DEBUG (qtp212011969-1801) [
> blueprint_gnylctsgemcz_users] o.a.s.c.CachingDirectoryFactory Removing
> directory after core close: /var/solr/data/blueprint_gnylctsgemcz_users/data
> 2023-07-06 08:43:37.620 DEBUG (qtp212011969-1801) [
> blueprint_gnylctsgemcz_users] o.a.s.c.CachingDirectoryFactory Removing
> directory after core close:
> /var/solr/data/blueprint_gnylctsgemcz_users/data/index
> 2023-07-06 08:43:37.620 ERROR (qtp212011969-1801) [
> blueprint_gnylctsgemcz_users] o.a.s.c.CachingDirectoryFactory Error removing
> directory => java.nio.file.NoSuchFileException:
> /var/solr/data/blueprint_gnylctsgemcz_users/data/index
> java.nio.file.NoSuchFileException:
> /var/solr/data/blueprint_gnylctsgemcz_users/data/index
> {noformat}
> It seems like the order of directory removal is causing this (first parent
> dir _data_, then sub-dir _data/index_). Entries should be re-arranged or just
> the parent dir be handled.
> Full log with entries dealing with the respective core is attached.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Created] (SOLR-17271) PerReplicaState: Shard leader elections still impact state.json
Justin Sweeney created SOLR-17271: - Summary: PerReplicaState: Shard leader elections still impact state.json Key: SOLR-17271 URL: https://issues.apache.org/jira/browse/SOLR-17271 Project: Solr Issue Type: Bug Security Level: Public (Default Security Level. Issues are Public) Reporter: Justin Sweeney >From testing it has been found that shard leader elections when there are >multiple replicas still touches state.json, even when PerReplicaState (PRS) is >enabled. With PRS, leader should be contained in the per replica state so we >should not have to modify state.json. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Created] (SOLR-17270) Umbrella: Per Replica State as default
Justin Sweeney created SOLR-17270: - Summary: Umbrella: Per Replica State as default Key: SOLR-17270 URL: https://issues.apache.org/jira/browse/SOLR-17270 Project: Solr Issue Type: Improvement Security Level: Public (Default Security Level. Issues are Public) Reporter: Justin Sweeney The concept of Per Replica State (PRS) as a part of state management in Solr has existed since 8.8.0. While PRS is in use in some places, there is still work to be done to get it to a place where we can move to PRS as the default behavior in Solr. This umbrella ticket will encapsulate the work required to get to this point and enable PRS as the default behavior for Solr 10. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Remove the Before SolrCloud section, and expand the supported feature… [solr]
epugh commented on PR #1138: URL: https://github.com/apache/solr/pull/1138#issuecomment-2093362791 If you are looking for an idea... We want to rename "User Managed" to "standalone" through the ref guide ;-). And move the SolrCloud examples above the standalone examples instead of below them ;-) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Remove the Before SolrCloud section, and expand the supported feature… [solr]
epugh commented on PR #1138: URL: https://github.com/apache/solr/pull/1138#issuecomment-2093356329 @jebnix thanks for this, and don't forget to ping me on any outstanding PR's you might have ;-) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Remove the Before SolrCloud section, and expand the supported feature… [solr]
epugh merged PR #1138: URL: https://github.com/apache/solr/pull/1138 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[PR] Backporting SOLR-17248 to branch_9x [solr]
epugh opened a new pull request, #2439: URL: https://github.com/apache/solr/pull/2439 https://issues.apache.org/jira/browse/SOLR-17248 I've broken the build on 9x while backporting, and this is to fix it. I am opening a PR targeting `branch_9x` to make sure everything passes the tests before merging. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17248) Refactor ZK related SolrCli tools to separate SolrZkClient and CloudSolrClient instantiation/usage
[ https://issues.apache.org/jira/browse/SOLR-17248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843269#comment-17843269 ] ASF subversion and git services commented on SOLR-17248: Commit 57156cf014d9ea4d8b89700ef4ae9f608255ec69 in solr's branch refs/heads/backporting_SOLR-17248 from Eric Pugh [ https://gitbox.apache.org/repos/asf?p=solr.git;h=57156cf014d ] backporting SOLR-17248 > Refactor ZK related SolrCli tools to separate SolrZkClient and > CloudSolrClient instantiation/usage > -- > > Key: SOLR-17248 > URL: https://issues.apache.org/jira/browse/SOLR-17248 > Project: Solr > Issue Type: Improvement >Reporter: Lamine >Assignee: Eric Pugh >Priority: Minor > Time Spent: 5h 50m > Remaining Estimate: 0h > > Many CLI tool-related classes currently duplicate SolrZkClient & > _CloudSolrClient_ instantiation, with the same code appearing 12 times across > 8 different classes. It would be beneficial to extract this instantiation and > reuse the code for easy maintainability > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17248) Refactor ZK related SolrCli tools to separate SolrZkClient and CloudSolrClient instantiation/usage
[ https://issues.apache.org/jira/browse/SOLR-17248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843268#comment-17843268 ] ASF subversion and git services commented on SOLR-17248: Commit 23877e0867045eda06f3ee6388f318630c6d937b in solr's branch refs/heads/backporting_SOLR-17248 from Lamine [ https://gitbox.apache.org/repos/asf?p=solr.git;h=23877e08670 ] SOLR-17248: Refactor ZK related SolrCli tools to separate SolrZkClient and CloudSolrClient instantiation/usage (#2417) - Co-authored-by: Lamine Idjeraoui Co-authored-by: Eric Pugh > Refactor ZK related SolrCli tools to separate SolrZkClient and > CloudSolrClient instantiation/usage > -- > > Key: SOLR-17248 > URL: https://issues.apache.org/jira/browse/SOLR-17248 > Project: Solr > Issue Type: Improvement >Reporter: Lamine >Assignee: Eric Pugh >Priority: Minor > Time Spent: 5h 50m > Remaining Estimate: 0h > > Many CLI tool-related classes currently duplicate SolrZkClient & > _CloudSolrClient_ instantiation, with the same code appearing 12 times across > 8 different classes. It would be beneficial to extract this instantiation and > reuse the code for easy maintainability > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-4587: integrate lucene-monitor into solr [solr]
cpoerschke commented on code in PR #2382:
URL: https://github.com/apache/solr/pull/2382#discussion_r1589415896
##
solr/modules/monitor/src/java/org/apache/solr/monitor/search/ReverseQueryParserPlugin.java:
##
@@ -0,0 +1,105 @@
+/*
+ *
+ * * Licensed to the Apache Software Foundation (ASF) under one or more
+ * * contributor license agreements. See the NOTICE file distributed with
+ * * this work for additional information regarding copyright ownership.
+ * * The ASF licenses this file to You under the Apache License, Version 2.0
+ * * (the "License"); you may not use this file except in compliance with
+ * * the License. You may obtain a copy of the License at
+ * *
+ * * http://www.apache.org/licenses/LICENSE-2.0
+ * *
+ * * Unless required by applicable law or agreed to in writing, software
+ * * distributed under the License is distributed on an "AS IS" BASIS,
+ * * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * * See the License for the specific language governing permissions and
+ * * limitations under the License.
+ *
+ */
+
+package org.apache.solr.monitor.search;
+
+import static
org.apache.solr.monitor.search.PresearcherFactory.PresearcherParameters;
+
+import java.io.IOException;
+import java.util.Optional;
+import org.apache.lucene.monitor.Presearcher;
+import org.apache.lucene.util.ResourceLoader;
+import org.apache.lucene.util.ResourceLoaderAware;
+import org.apache.solr.common.params.SolrParams;
+import org.apache.solr.common.util.NamedList;
+import org.apache.solr.request.SolrQueryRequest;
+import org.apache.solr.search.QParser;
+import org.apache.solr.search.QParserPlugin;
+
+public class ReverseQueryParserPlugin extends QParserPlugin implements
ResourceLoaderAware {
+
+ public static final String NAME = "reverse";
+
+ // The default prefix is long because we don't want it to clash
+ // with a user-defined name pattern and the longest one wins.
+ // {@link
+ //
https://cwiki.apache.org/confluence/display/solr/SchemaXml#Dynamic_fields:~:text=Longer%20patterns%20will%20be%20matched%20first}
+ // In the worst case this can be overridden by the user but ideally this
never comes up.
+ private static final String DEFAULT_ALIAS_PREFIX =
+ "monitor_alias_";
+
+ private Presearcher presearcher;
+ private PresearcherParameters presearcherParameters;
+
+ @Override
+ public QParser createParser(
+ String qstr, SolrParams localParams, SolrParams params, SolrQueryRequest
req) {
+return new ReverseQueryParser(qstr, localParams, params, req, presearcher);
+ }
+
+ @Override
+ public void close() throws IOException {
+super.close();
+ }
+
+ @Override
+ public void init(NamedList args) {
+super.init(args);
+String presearcherType = (String) args.get("presearcherType");
+String termWeightorType = (String) args.get("termWeightorType");
+boolean applyFieldNameAlias =
+resolveProperty(args, "applyFieldNameAlias", Boolean.class, false);
+int numberOfPasses = resolveProperty(args, "numberOfPasses",
Integer.class, 0);
+float minWeight = resolveProperty(args, "minWeight", Float.class, 0f);
+String aliasPrefix =
+Optional.ofNullable((String)
args.get("aliasPrefix")).orElse(DEFAULT_ALIAS_PREFIX);
+presearcherParameters =
+new PresearcherParameters(
+presearcherType,
+termWeightorType,
+applyFieldNameAlias,
+numberOfPasses,
+minWeight,
+aliasPrefix);
+ }
+
+ // TODO is there a better pattern for this? NamedList::get(String key, T
default) can't be called
+ // on NamedList
Review Comment:
I don't know if it qualifies as a pattern or not but one approach could be
via the `SolrPluginUtils.invokeSetters` mechanism e.g. instead of
```
MultipassTermFilteredPresearcher
true
4
```
there could be
```
org.apache.solr.monitor.search.MultipassTermFilteredPresearcherFactory
true
4
```
and the
`org.apache.solr.monitor.search.MultipassTermFilteredPresearcherFactory`
element is turned into an object `pf` via the resource loader.
Then the remaining `NamedList` elements would (somehow) be stripped off the
`presearcher.` prefix giving
```
true
4
```
conceptually as the named list `nl` and then a
```
SolrPluginUtils.invokeSetters(pf, nl);
```
call would apply all the arguments in the list to the factory.
This may seem a bit unusual at a first glance! The approach removes the need
for calling code to 'know' the exact parameter names and parameter types and
default values, the available parameters are simply the ones for which the
factory has setters. Also it is not-silent about misconfiguration e.g. if the
code explicitly looks for `numberOfPasses` then any misconfiguration of (say)
`numPasses` could go unnoticed whereas the presence of a
Re: [PR] SOLR-17088: Fix custom solr.xml usage in tests with extra jettys [solr]
HoustonPutman commented on code in PR #2437: URL: https://github.com/apache/solr/pull/2437#discussion_r1589407140 ## solr/test-framework/src/java/org/apache/solr/cloud/MiniSolrCloudCluster.java: ## @@ -276,6 +277,7 @@ public MiniSolrCloudCluster( Objects.requireNonNull(securityJson); this.baseDir = Objects.requireNonNull(baseDir); this.jettyConfig = Objects.requireNonNull(jettyConfig); +this.solrXml = solrXml == null ? DEFAULT_CLOUD_SOLR_XML : solrXml; Review Comment: Yeah, I didn't see the other usages of this method with null. So I moved the defaulting here instead. Thanks for the review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17192: Add "field-limiting" URP to catch ill-designed schemas [solr]
gerlowskija commented on code in PR #2395:
URL: https://github.com/apache/solr/pull/2395#discussion_r1589189040
##
solr/core/src/java/org/apache/solr/update/processor/NumFieldLimitingUpdateRequestProcessorFactory.java:
##
@@ -0,0 +1,113 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.update.processor;
+
+import org.apache.solr.common.util.NamedList;
+import org.apache.solr.core.SolrCore;
+import org.apache.solr.request.SolrQueryRequest;
+import org.apache.solr.response.SolrQueryResponse;
+import org.apache.solr.search.SolrIndexSearcher;
+import org.apache.solr.util.plugin.SolrCoreAware;
+
+/**
+ * This factory generates an UpdateRequestProcessor which fails update
requests once a core has
+ * exceeded a configurable maximum number of fields. Meant as a safeguard to
help users notice
+ * potentially-dangerous schema design before performance and stability
problems start to occur.
+ *
+ * The URP uses the core's {@link SolrIndexSearcher} to judge the current
number of fields.
+ * Accordingly, it undercounts the number of fields in the core - missing all
fields added since the
+ * previous searcher was opened. As such, the URP's request-blocking is "best
effort" - it cannot be
+ * relied on as a precise limit on the number of fields.
+ *
+ * Additionally, the field-counting includes all documents present in the
index, including any
+ * deleted docs that haven't yet been purged via segment merging. Note that
this can differ
+ * significantly from the number of fields defined in managed-schema.xml -
especially when dynamic
+ * fields are enabled. The only way to reduce this field count is to delete
documents and wait until
+ * the deleted documents have been removed by segment merges. Users may of
course speed up this
+ * process by tweaking Solr's segment-merging, triggering an "optimize"
operation, etc.
+ *
+ * {@link NumFieldLimitingUpdateRequestProcessorFactory} accepts two
configuration parameters:
+ *
+ *
+ * maxFields - (required) The maximum number of fields
before update requests
+ * should be aborted. Once this limit has been exceeded, additional
update requests will fail
+ * until fields have been removed or the "maxFields" is increased.
+ * warnOnly - (optional) If true then the URP
logs verbose warnings
+ * about the limit being exceeded but doesn't abort update requests.
Defaults to false
+ *if not specified
+ *
+ *
+ * @since 9.6.0
+ */
+public class NumFieldLimitingUpdateRequestProcessorFactory extends
UpdateRequestProcessorFactory
+implements SolrCoreAware {
+
+ private static final String MAXIMUM_FIELDS_PARAM = "maxFields";
+ private static final String WARN_ONLY_PARAM = "warnOnly";
+
+ private NumFieldsMonitor numFieldsMonitor;
+ private int maximumFields;
+ private boolean warnOnly;
+
+ @Override
+ public void inform(final SolrCore core) {
+// register a commit callback for monitoring the number of fields in the
schema
+numFieldsMonitor = new NumFieldsMonitor(core);
+core.getUpdateHandler().registerCommitCallback(numFieldsMonitor);
+core.registerNewSearcherListener(numFieldsMonitor);
+ }
+
+ @Override
+ public void init(NamedList args) {
+warnOnly = args.indexOf(WARN_ONLY_PARAM, 0) > 0 ?
args.getBooleanArg(WARN_ONLY_PARAM) : false;
+
+if (args.indexOf(MAXIMUM_FIELDS_PARAM, 0) < 0) {
+ throw new IllegalArgumentException(
+ "The "
+ + MAXIMUM_FIELDS_PARAM
+ + " parameter is required for "
+ + getClass().getName()
+ + ", but no value was provided.");
+}
+final Object rawMaxFields = args.get(MAXIMUM_FIELDS_PARAM);
+if (rawMaxFields == null || !(rawMaxFields instanceof Integer)) {
+ throw new IllegalArgumentException(
+ MAXIMUM_FIELDS_PARAM + " must be configured as a non-null ");
+}
+maximumFields = (Integer) rawMaxFields;
+if (maximumFields <= 0) {
+ throw new IllegalArgumentException(MAXIMUM_FIELDS_PARAM + " must be a
positive integer");
+}
+ }
+
+ @Override
+ public UpdateRequestProcessor getInstance(
+ SolrQueryRequest req, SolrQueryResponse rsp, UpdateRequestProcessor
next) {
+
Re: [PR] SOLR-17192: Add "field-limiting" URP to catch ill-designed schemas [solr]
gerlowskija commented on code in PR #2395:
URL: https://github.com/apache/solr/pull/2395#discussion_r1589268556
##
solr/core/src/java/org/apache/solr/update/processor/NumFieldsMonitor.java:
##
@@ -0,0 +1,56 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.update.processor;
+
+import org.apache.solr.core.AbstractSolrEventListener;
+import org.apache.solr.core.SolrCore;
+import org.apache.solr.search.SolrIndexSearcher;
+import org.apache.solr.util.RefCounted;
+
+public class NumFieldsMonitor extends AbstractSolrEventListener {
+
+ private int numFields;
+
+ public NumFieldsMonitor(SolrCore core) {
+super(core);
+this.numFields = -1;
+ }
+
+ @Override
+ public void postCommit() {
+RefCounted indexSearcher = null;
+try {
+ indexSearcher = getCore().getSearcher();
Review Comment:
Is there a reason that `withSearcher` throws an IOException? Would it be
worth having one of those methods that consumes a vanilla Function instead of a
`IOFunction`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17192: Add "field-limiting" URP to catch ill-designed schemas [solr]
gerlowskija commented on PR #2395: URL: https://github.com/apache/solr/pull/2395#issuecomment-2093110049 Thanks for the re-review @dsmiley . Pending feedback from others I'm going to stick with the NumFieldMonitor approach (more details [here](https://github.com/apache/solr/pull/2395#discussion_r1589189040)), but I've addressed the rest of your suggestions. Lmk what you think! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17192: Add "field-limiting" URP to catch ill-designed schemas [solr]
gerlowskija commented on code in PR #2395:
URL: https://github.com/apache/solr/pull/2395#discussion_r1589215019
##
solr/core/src/java/org/apache/solr/update/processor/NumFieldsMonitor.java:
##
@@ -0,0 +1,56 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.update.processor;
+
+import org.apache.solr.core.AbstractSolrEventListener;
+import org.apache.solr.core.SolrCore;
+import org.apache.solr.search.SolrIndexSearcher;
+import org.apache.solr.util.RefCounted;
+
+public class NumFieldsMonitor extends AbstractSolrEventListener {
Review Comment:
> I suppose you didn't like my proposal that would eliminate the need for
this event listener
Yes, in short. I agree it's a matter of opinion, but I prefer the
particular complexity and benefits that NumFieldsMonitor offers to that of the
direct-SIS or weak-reference approach. See my reply [on the factory
above](https://github.com/apache/solr/pull/2395/files#diff-6da5de1a5fc66295d066909ad976a15ad31e8411ed71d7909e8ed0d14f3cac78R103)
for a bit more detail/explanation.
##
solr/core/src/test/org/apache/solr/update/processor/NumFieldLimitingUpdateRequestProcessorIntegrationTest.java:
##
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.update.processor;
+
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.UUID;
+import java.util.concurrent.TimeUnit;
+import org.apache.solr.client.solrj.request.CollectionAdminRequest;
+import org.apache.solr.cloud.SolrCloudTestCase;
+import org.apache.solr.common.SolrInputDocument;
+import org.hamcrest.Matchers;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+public class NumFieldLimitingUpdateRequestProcessorIntegrationTest extends
SolrCloudTestCase {
+
+ private static String SINGLE_SHARD_COLL_NAME = "singleShardColl";
Review Comment:
I think this is copy/paste from a test that used both single and multi-shard
collections. I've updated the name to be more generic.
##
solr/core/src/test-files/solr/configsets/cloud-minimal-field-limiting/conf/solrconfig.xml:
##
@@ -0,0 +1,73 @@
+
+
+
+
+
+
+
+
+ ${solr.data.dir:}
+
+
+
+
+ ${tests.luceneMatchVersion:LATEST}
+
+
+
+ ${solr.commitwithin.softcommit:true}
+
+
+
+
+
+
+ explicit
+ true
+ text
+
+
+
+
+${solr.test.maxFields:1234}
+
Review Comment:
Tbh I wasn't aware that you could declare processors (that require config)
directly *in* a processor chain. I can't find an example of that. Looking
through several other solrconfig.xml files for URPs that require
configuration...they all look similar to what I've done here afaict.
(See the [_default configset Solr
ships](https://github.com/apache/solr/blob/main/solr/server/solr/configsets/_default/conf/solrconfig.xml#L900-L944)
and the [default configset in the 'core' modules
tests](https://github.com/apache/solr/blob/main/solr/core/src/test-files/solr/configsets/_default/conf/solrconfig.xml#L61-L81)...both
look similar to the example here)
Maybe I'm misunderstanding you or missing something though - can you point
to an example of how you'd prefer to see this declared, and a reason it's
preferable?
##
solr/solr-ref-guide/modules/configuration-guide/pages/update-request-processors.adoc:
##
@@ -337,6 +337,12 @@ Documents processed prior to the offender
Re: [PR] SOLR-17088: Fix custom solr.xml usage in tests with extra jettys [solr]
cpoerschke commented on code in PR #2437: URL: https://github.com/apache/solr/pull/2437#discussion_r1589231122 ## solr/test-framework/src/java/org/apache/solr/cloud/MiniSolrCloudCluster.java: ## @@ -276,6 +277,7 @@ public MiniSolrCloudCluster( Objects.requireNonNull(securityJson); this.baseDir = Objects.requireNonNull(baseDir); this.jettyConfig = Objects.requireNonNull(jettyConfig); +this.solrXml = solrXml == null ? DEFAULT_CLOUD_SOLR_XML : solrXml; Review Comment: Wondering if the similar logic at lines 491-493/492-494 can be removed now then? edit: or rather not removed but amended i.e. use passed in `solrXml` if non-null and `this.solrXml` otherwise -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17088: Fix custom solr.xml usage in tests with extra jettys [solr]
cpoerschke commented on code in PR #2437: URL: https://github.com/apache/solr/pull/2437#discussion_r1589231122 ## solr/test-framework/src/java/org/apache/solr/cloud/MiniSolrCloudCluster.java: ## @@ -276,6 +277,7 @@ public MiniSolrCloudCluster( Objects.requireNonNull(securityJson); this.baseDir = Objects.requireNonNull(baseDir); this.jettyConfig = Objects.requireNonNull(jettyConfig); +this.solrXml = solrXml == null ? DEFAULT_CLOUD_SOLR_XML : solrXml; Review Comment: Wondering if the similar logic at lines 491-493/492-494 can be removed now then? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [I] Add support to set capabilities on Solr cloud container's security context [solr-operator]
janhoy commented on issue #489: URL: https://github.com/apache/solr-operator/issues/489#issuecomment-2092905796 > Hi all! a new version was recently released (April 12, 2024, Apache Solr Operator™ v0.8.1 available, there is no solution to our problem) is there any news on our problem? @ollixy , @pareekdevanshu, @mmoscher, @aaronsuns , @AyzekTime This is an open source project, and we rely on contributions. If your day-job has a need for this and are willing to sponsor such a feature, then the best way forward is to arrange so that you can contribute a PR directly, and we'll help get it in to the next version. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Remove encryption log prefix. [solr-sandbox]
bruno-roustant merged PR #104: URL: https://github.com/apache/solr-sandbox/pull/104 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [I] Add support to set capabilities on Solr cloud container's security context [solr-operator]
AyzekTime commented on issue #489: URL: https://github.com/apache/solr-operator/issues/489#issuecomment-2092685981 Hi all! a new version was recently released (April 12, 2024, Apache Solr Operator™ v0.8.1 available, there is no solution to our problem) is there any news on our problem? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Updated] (SOLR-17268) Solr 9.3-9.6 (JacksonJsonWriter) does not support "json.nl=arrntv"
[
https://issues.apache.org/jira/browse/SOLR-17268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ramsey Haddad updated SOLR-17268:
-
Description:
Asking for facets and using "json.nl=arrntv" results in the exception:
{code:java}
arrntv namedListStyle must only be used with
ArrayOfNameTypeValueJSONWriter{code}
* Code inspection shows that ArrayOfNameTypeValueJSONWriter is supported by the
older JSONResponseWriter and not by the newer JacksonJsonWriter.
* Running a facet query with "json.nl=arrntv" works for 9.2.0 and 9.2.1
* Running a facet query with "json.nl=arrntv" fails (Exception/500) for 9.3.0
and 9.6.0
* For 9.3.0, using
{code:java}
{code}
in solrconfig.xml is a successful workaround.
was:
Asking for facets and using "json.nl=arrntv" results in the exception:
{code:java}
arrntv namedListStyle must only be used with
ArrayOfNameTypeValueJSONWriter{code}
* Code inspection shows that ArrayOfNameTypeValueJSONWriter is supported by the
older JSONResponseWriter and not by the newer JacksonJsonWriter.
* Running a facet query with "json.nl=arrntv" works for 9.2.0 and 9.2.1
* Running a facet query with "json.nl=arrntv" fails (Exception/500) for 9.3.0
* For 9.3.0, using
{code:java}
{code}
in solrconfig.xml is a successful workaround.
> Solr 9.3-9.6 (JacksonJsonWriter) does not support "json.nl=arrntv"
> --
>
> Key: SOLR-17268
> URL: https://issues.apache.org/jira/browse/SOLR-17268
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Response Writers
>Affects Versions: 9.3, 9.4, 9.4.1, 9.6, 9.5.0
>Reporter: Ramsey Haddad
>Priority: Minor
>
> Asking for facets and using "json.nl=arrntv" results in the exception:
> {code:java}
> arrntv namedListStyle must only be used with
> ArrayOfNameTypeValueJSONWriter{code}
> * Code inspection shows that ArrayOfNameTypeValueJSONWriter is supported by
> the older JSONResponseWriter and not by the newer JacksonJsonWriter.
> * Running a facet query with "json.nl=arrntv" works for 9.2.0 and 9.2.1
> * Running a facet query with "json.nl=arrntv" fails (Exception/500) for 9.3.0
> and 9.6.0
> * For 9.3.0, using
> {code:java}
>
> {code}
> in solrconfig.xml is a successful workaround.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Remove the Before SolrCloud section, and expand the supported feature… [solr]
janhoy commented on PR #1138: URL: https://github.com/apache/solr/pull/1138#issuecomment-2092592369 I have not reviewed the other pages related to cluster types, but trust you that this might be redundant. It's a pity that doc improvements get stalled. Feel free to land it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17248) Refactor ZK related SolrCli tools to separate SolrZkClient and CloudSolrClient instantiation/usage
[ https://issues.apache.org/jira/browse/SOLR-17248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843147#comment-17843147 ] ASF subversion and git services commented on SOLR-17248: Commit defe79b3c3f022eb678a02ceecaebcbbbeb446fb in solr's branch refs/heads/main from Lamine [ https://gitbox.apache.org/repos/asf?p=solr.git;h=defe79b3c3f ] SOLR-17248: Refactor ZK related SolrCli tools to separate SolrZkClient and CloudSolrClient instantiation/usage (#2417) - Co-authored-by: Lamine Idjeraoui Co-authored-by: Eric Pugh > Refactor ZK related SolrCli tools to separate SolrZkClient and > CloudSolrClient instantiation/usage > -- > > Key: SOLR-17248 > URL: https://issues.apache.org/jira/browse/SOLR-17248 > Project: Solr > Issue Type: Improvement >Reporter: Lamine >Assignee: Eric Pugh >Priority: Minor > Time Spent: 5h 50m > Remaining Estimate: 0h > > Many CLI tool-related classes currently duplicate SolrZkClient & > _CloudSolrClient_ instantiation, with the same code appearing 12 times across > 8 different classes. It would be beneficial to extract this instantiation and > reuse the code for easy maintainability > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17248: Refactor ZK related SolrCli tools to separate SolrZkClient and CloudSolrClient instantiation/usage [solr]
epugh merged PR #2417: URL: https://github.com/apache/solr/pull/2417 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-17248: Refactor ZK related SolrCli tools to separate SolrZkClient and CloudSolrClient instantiation/usage [solr]
epugh commented on PR #2417: URL: https://github.com/apache/solr/pull/2417#issuecomment-2092493146 > > @laminelam I just merged in the latest, notice the compressor stuff for `ZkCpTool`? I really wish that all happened INSIDE of the zkclient tool.. not when you create it. oh well... Can you just quickly review my commits and then I think we are good to go! > > LGTM, although not familiar with "_withStateFileCompression_". Are you switching intentionally from "_solrhome_" to "_solr-home_"? ZKCli uses "_solrhome_" instead. withStateFileCompression is a feature for compressing state.json that only existed in ZKCli, so when I deprecated ZKCLI (gone now in `main`) I had to bring it across. and yeah, solrhome becomes solr-home in SolrCLI... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] Remove the Before SolrCloud section, and expand the supported feature… [solr]
epugh commented on PR #1138: URL: https://github.com/apache/solr/pull/1138#issuecomment-2092488123 > The old "User Managed" mode is still supported (https://solr.apache.org/guide/solr/latest/deployment-guide/user-managed-distributed-search.html), so I would not remove it. Guess it's there to give a reason why SolrCloud was introudced. > > But perhaps it should be reworded and make it clear that the feature still exists? "Before SolrCloud, solr only supported Distributed Search through (link)." So I am re-reading things... And we start out with a section "Solr Cluster Types" that talks about the various options. Then you get the section on "User-Managed Clusters" and then "SolrCloud Clusters".I feel like if you are this page, you don't need this content, because the differences is covered in "Solr Cluster Types", we are just derailing someone at this point... Thoughts? Does that change your mind on merging this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-16291 : Decay function queries gauss,linear,exponential [solr]
epugh commented on PR #939: URL: https://github.com/apache/solr/pull/939#issuecomment-2092480847 Hi all, this seems like a really nice addition to Solr... Assuming @danrosher is still up for adding the Ref Guide docs, this seems like a good capablity.Not sure how strongly you feel @risdenk on this versus the positive indication from @janhoy ? Be curious if there are some specificis that could be made to get this to a LGTM ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
