[jira] [Updated] (SOLR-13195) NPE caused by distributed queries when no shards are configured
[
https://issues.apache.org/jira/browse/SOLR-13195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Munendra S N updated SOLR-13195:
Component/s: (was: Facet Module)
FacetComponent
> NPE caused by distributed queries when no shards are configured
> ---
>
> Key: SOLR-13195
> URL: https://issues.apache.org/jira/browse/SOLR-13195
> Project: Solr
> Issue Type: Bug
> Components: FacetComponent
>Affects Versions: master (9.0)
> Environment: h1. Steps to reproduce
> * Use a Linux machine.
> * Build commit {{ea2c8ba}} of Solr as described in the section below.
> * Build the films collection as described below.
> * Start the server using the command {{./bin/solr start -f -p 8983 -s
> /tmp/home}}
> * Request the URL given in the bug description.
> h1. Compiling the server
> {noformat}
> git clone https://github.com/apache/lucene-solr
> cd lucene-solr
> git checkout ea2c8ba
> ant compile
> cd solr
> ant server
> {noformat}
> h1. Building the collection
> We followed [Exercise
> 2|http://lucene.apache.org/solr/guide/7_5/solr-tutorial.html#exercise-2] from
> the [Solr
> Tutorial|http://lucene.apache.org/solr/guide/7_5/solr-tutorial.html]. The
> attached file ({{home.zip}}) gives the contents of folder {{/tmp/home}} that
> you will obtain by following the steps below:
> {noformat}
> mkdir -p /tmp/home
> echo '' >
> /tmp/home/solr.xml
> {noformat}
> In one terminal start a Solr instance in foreground:
> {noformat}
> ./bin/solr start -f -p 8983 -s /tmp/home
> {noformat}
> In another terminal, create a collection of movies, with no shards and no
> replication, and initialize it:
> {noformat}
> bin/solr create -c films
> curl -X POST -H 'Content-type:application/json' --data-binary '{"add-field":
> {"name":"name", "type":"text_general", "multiValued":false, "stored":true}}'
> http://localhost:8983/solr/films/schema
> curl -X POST -H 'Content-type:application/json' --data-binary
> '{"add-copy-field" : {"source":"*","dest":"_text_"}}'
> http://localhost:8983/solr/films/schema
> ./bin/post -c films example/films/films.json
> {noformat}
>Reporter: Johannes Kloos
>Priority: Major
> Labels: diffblue, newdev
>
> Requesting the following URL causes Solr to return an HTTP 500 error response:
> {noformat}
> http://localhost:8983/solr/films/select?distrib=true&facet=on&facet.field=genre
> {noformat}
> The error response seems to be caused by the following uncaught exception:
> {noformat}
> null:java.lang.NullPointerException
> at
> org.apache.solr.handler.component.FacetComponent$DistribFieldFacet.(FacetComponent.java:1432)
> at
> org.apache.solr.handler.component.FacetComponent$FacetInfo.parse(FacetComponent.java:1266)
> at
> org.apache.solr.handler.component.FacetComponent.modifyRequest(FacetComponent.java:528)
> at
> org.apache.solr.handler.component.ResponseBuilder.addRequest(ResponseBuilder.java:162)
> at
> org.apache.solr.handler.component.QueryComponent.createMainQuery(QueryComponent.java:755)
> at
> org.apache.solr.handler.component.QueryComponent.regularDistributedProcess(QueryComponent.java:546)
> at
> org.apache.solr.handler.component.QueryComponent.distributedProcess(QueryComponent.java:499)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:348)
> {noformat}
> Since the query contains “distrib=true”, it is handled by calling the
> distributedProcess methods of the search pipeline. At no point in the call
> chain that leads to the above exception is there any check if the
> ResponseBuilder rb used to handle the request actually has its shared
> property configured. Nevertheless, the DistribFieldFact constructor tries to
> access rb.shards.length, but rb.shared is null in this example.
> We found this bug using [Diffblue Microservices
> Testing|https://www.diffblue.com/labs/?utm_source=solr-br]. Find more
> information on this [fuzz testing
> campaign|https://www.diffblue.com/blog/2018/12/19/diffblue-microservice-testing-a-sneak-peek-at-our-early-product-and-results?utm_source=solr-br].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-7879) Null pointer exception when
[ https://issues.apache.org/jira/browse/SOLR-7879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Munendra S N resolved SOLR-7879. Resolution: Fixed > Null pointer exception when > > > Key: SOLR-7879 > URL: https://issues.apache.org/jira/browse/SOLR-7879 > Project: Solr > Issue Type: Bug > Components: Facet Module, search >Affects Versions: 5.2.1 > Environment: Oracle jdk1.8 >Reporter: Gary Yang >Priority: Major > Labels: faceting, jsonapi, subfacet > > This can be reproduce by certain inputs, but I can't find a pattern in the > inputs that cause this error: > Caused by: > org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error > from server at http://192.168.1.47:8983/solr/core1: > java.lang.NullPointerException > at > org.apache.solr.search.facet.FacetFieldProcessorFCBase.findTopSlots(FacetField.java:301) > at > org.apache.solr.search.facet.FacetFieldProcessorFCBase.getFieldCacheCounts(FacetField.java:254) > at > org.apache.solr.search.facet.FacetFieldProcessorFCBase.process(FacetField.java:218) > at > org.apache.solr.search.facet.FacetProcessor.processSubs(FacetRequest.java:267) > at > org.apache.solr.search.facet.FacetFieldProcessorNumeric.calcFacets(FacetFieldProcessorNumeric.java:472) > at > org.apache.solr.search.facet.FacetFieldProcessorNumeric.process(FacetFieldProcessorNumeric.java:151) > at > org.apache.solr.search.facet.FacetProcessor.processSubs(FacetRequest.java:267) > at > org.apache.solr.search.facet.FacetProcessor.fillBucket(FacetRequest.java:354) > at > org.apache.solr.search.facet.FacetQueryProcessor.process(FacetQuery.java:57) > at org.apache.solr.search.facet.FacetModule.process(FacetModule.java:87) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:255) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2064) > at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:654) > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:450) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) > at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) > at > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) > at > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) > at org.eclipse.jetty.server.Server.handle(Server.java:497) > at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310) > at > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) > at > org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540) > at > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) > at > org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) > at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-7879) Null pointer exception when
[
https://issues.apache.org/jira/browse/SOLR-7879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001828#comment-17001828
]
Munendra S N commented on SOLR-7879:
The syntax is wrong here. Correct syntax is shared below. custom sort should be
computed as part of subfacet and then used for sorting. NPEs are fixes in
SOLR-13022
{code:java}
{
"uniq_viewerNum": "unique(viewer_ID)",
"powerPointName": {
"type": "terms",
"field": "powerPointName",
"limit": 1,
"sort": {
"lastUpdatedTime": "desc"
},
"facet": {
"lastUpdatedTime": "max(powerPoint_lastUpdatedTime)"
}
}
}
{code}
Closing this
> Null pointer exception when
>
>
> Key: SOLR-7879
> URL: https://issues.apache.org/jira/browse/SOLR-7879
> Project: Solr
> Issue Type: Bug
> Components: Facet Module, search
>Affects Versions: 5.2.1
> Environment: Oracle jdk1.8
>Reporter: Gary Yang
>Priority: Major
> Labels: faceting, jsonapi, subfacet
>
> This can be reproduce by certain inputs, but I can't find a pattern in the
> inputs that cause this error:
> Caused by:
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error
> from server at http://192.168.1.47:8983/solr/core1:
> java.lang.NullPointerException
> at
> org.apache.solr.search.facet.FacetFieldProcessorFCBase.findTopSlots(FacetField.java:301)
> at
> org.apache.solr.search.facet.FacetFieldProcessorFCBase.getFieldCacheCounts(FacetField.java:254)
> at
> org.apache.solr.search.facet.FacetFieldProcessorFCBase.process(FacetField.java:218)
> at
> org.apache.solr.search.facet.FacetProcessor.processSubs(FacetRequest.java:267)
> at
> org.apache.solr.search.facet.FacetFieldProcessorNumeric.calcFacets(FacetFieldProcessorNumeric.java:472)
> at
> org.apache.solr.search.facet.FacetFieldProcessorNumeric.process(FacetFieldProcessorNumeric.java:151)
> at
> org.apache.solr.search.facet.FacetProcessor.processSubs(FacetRequest.java:267)
> at
> org.apache.solr.search.facet.FacetProcessor.fillBucket(FacetRequest.java:354)
> at
> org.apache.solr.search.facet.FacetQueryProcessor.process(FacetQuery.java:57)
> at org.apache.solr.search.facet.FacetModule.process(FacetModule.java:87)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:255)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2064)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:654)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:450)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
> at org.eclipse.jetty.server.Server.handle(Server.java:497)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
> at
> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
> at java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
[jira] [Commented] (SOLR-13101) Shared storage support in SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-13101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001807#comment-17001807 ] Noble Paul commented on SOLR-13101: --- Is there any reason why wee can't release this as a third party plugin using the new package manager instead of pushing this to Solr codebase? > Shared storage support in SolrCloud > --- > > Key: SOLR-13101 > URL: https://issues.apache.org/jira/browse/SOLR-13101 > Project: Solr > Issue Type: New Feature > Components: SolrCloud >Reporter: Yonik Seeley >Priority: Major > Time Spent: 7h 50m > Remaining Estimate: 0h > > Solr should have first-class support for shared storage (blob/object stores > like S3, google cloud storage, etc. and shared filesystems like HDFS, NFS, > etc). > The key component will likely be a new replica type for shared storage. It > would have many of the benefits of the current "pull" replicas (not indexing > on all replicas, all shards identical with no shards getting out-of-sync, > etc), but would have additional benefits: > - Any shard could become leader (the blob store always has the index) > - Better elasticity scaling down >- durability not linked to number of replcias.. a single replica could be > common for write workloads >- could drop to 0 replicas for a shard when not needed (blob store always > has index) > - Allow for higher performance write workloads by skipping the transaction > log >- don't pay for what you don't need >- a commit will be necessary to flush to stable storage (blob store) > - A lot of the complexity and failure modes go away > An additional component a Directory implementation that will work well with > blob stores. We probably want one that treats local disk as a cache since > the latency to remote storage is so large. I think there are still some > "locking" issues to be solved here (ensuring that more than one writer to the > same index won't corrupt it). This should probably be pulled out into a > different JIRA issue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14125) plugins must be hot loadable
[ https://issues.apache.org/jira/browse/SOLR-14125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001805#comment-17001805 ] Noble Paul commented on SOLR-14125: --- I've added testcases. I shall merge this soon > plugins must be hot loadable > -- > > Key: SOLR-14125 > URL: https://issues.apache.org/jira/browse/SOLR-14125 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Noble Paul >Priority: Major > Labels: packagemanager > Time Spent: 20m > Remaining Estimate: 0h > > Today, the class is loaded at the core load time an a stong reference is > held. Package loading need it to be lazily loaded -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] noblepaul commented on issue #1108: SOLR-14125 : Streaming expressions to be loadable from packages
noblepaul commented on issue #1108: SOLR-14125 : Streaming expressions to be loadable from packages URL: https://github.com/apache/lucene-solr/pull/1108#issuecomment-568223643 It's in a good shape. reviews are welcome This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-8596) The replacement of comments is a bug, in "UserDictionary.java"
[
https://issues.apache.org/jira/browse/LUCENE-8596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Sokolov resolved LUCENE-8596.
-
Fix Version/s: trunk
Resolution: Fixed
This fix won't be backported to 8.x branch since it changes comment processing
in kuromoji user dictionaries and could cause breakage
> The replacement of comments is a bug, in "UserDictionary.java"
> --
>
> Key: LUCENE-8596
> URL: https://issues.apache.org/jira/browse/LUCENE-8596
> Project: Lucene - Core
> Issue Type: Bug
> Components: modules/analysis
>Reporter: miyaharas
>Priority: Major
> Fix For: trunk
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> [https://github.com/apache/lucene-solr/blob/1d85cd783863f75cea133fb9c452302214165a4d/lucene/analysis/kuromoji/src/java/org/apache/lucene/analysis/ja/dict/UserDictionary.java#L68]
>
> hi
> I think that this is bug.
> I think the following is correct
> {code:java}
> line = line.replaceAll ("^ #. * $", "");
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] rmuir commented on issue #1110: SOLR-14138: enable request log via environ var
rmuir commented on issue #1110: SOLR-14138: enable request log via environ var URL: https://github.com/apache/lucene-solr/pull/1110#issuecomment-568220195 ok, i fixed it to respect SOLR_LOGS_DIR as well, and moved it to the new non-deprecated way. I think its in much better shape. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14138) Fix commented-out RequestLog in jetty.xml to use non-deprecated class
[
https://issues.apache.org/jira/browse/SOLR-14138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001797#comment-17001797
]
ASF subversion and git services commented on SOLR-14138:
Commit c3731a7c085172d87109df6a3866419d17d47eac in lucene-solr's branch
refs/heads/jira/SOLR-14138 from Robert Muir
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=c3731a7 ]
SOLR-14138: remove deprecated class and respect SOLR_LOGS_DIR
> Fix commented-out RequestLog in jetty.xml to use non-deprecated class
> -
>
> Key: SOLR-14138
> URL: https://issues.apache.org/jira/browse/SOLR-14138
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Robert Muir
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently the jetty request logging is disabled (commented out).
> But it can be useful, e.g. since it uses a standard logging format and there
> are tools to analyze it by default. Also it can be used to detect some
> attacks not otherwise logged anywhere else, since they don't make it to solr
> servlet: requests blocked at the jetty level (invalid/malformed requests,
> ones filtered by jetty IP filtering, etc).
> We should switch it from the deprecated NCSARequestLog class, instead to use
> the CustomRequestLog with either NCSA_FORMAT or EXTENDED_NCSA_FORMAT.
> {quote}
> Deprecated.
> use CustomRequestLog given format string
> CustomRequestLog.EXTENDED_NCSA_FORMAT with a RequestLogWriter
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] rmuir opened a new pull request #1110: SOLR-14138: enable request log via environ var
rmuir opened a new pull request #1110: SOLR-14138: enable request log via environ var URL: https://github.com/apache/lucene-solr/pull/1110 Today you have to edit XML to enable the jetty request log. This makes it work via environment variable. TODO: * get this thing off the non-deprecated classes. * plumb SOLR_LOGS_DIR to the config so it respects that (today it uses hardcoded `logs/` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14138) Fix commented-out RequestLog in jetty.xml to use non-deprecated class
[
https://issues.apache.org/jira/browse/SOLR-14138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001792#comment-17001792
]
ASF subversion and git services commented on SOLR-14138:
Commit b2db7a282b1bf0f9e3c1a8ad5440d98032d8111f in lucene-solr's branch
refs/heads/jira/SOLR-14138 from Robert Muir
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=b2db7a2 ]
SOLR-14138: enable request log via environ var
> Fix commented-out RequestLog in jetty.xml to use non-deprecated class
> -
>
> Key: SOLR-14138
> URL: https://issues.apache.org/jira/browse/SOLR-14138
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Robert Muir
>Priority: Major
>
> Currently the jetty request logging is disabled (commented out).
> But it can be useful, e.g. since it uses a standard logging format and there
> are tools to analyze it by default. Also it can be used to detect some
> attacks not otherwise logged anywhere else, since they don't make it to solr
> servlet: requests blocked at the jetty level (invalid/malformed requests,
> ones filtered by jetty IP filtering, etc).
> We should switch it from the deprecated NCSARequestLog class, instead to use
> the CustomRequestLog with either NCSA_FORMAT or EXTENDED_NCSA_FORMAT.
> {quote}
> Deprecated.
> use CustomRequestLog given format string
> CustomRequestLog.EXTENDED_NCSA_FORMAT with a RequestLogWriter
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14125) plugins must be hot loadable
[ https://issues.apache.org/jira/browse/SOLR-14125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001788#comment-17001788 ] Noble Paul commented on SOLR-14125: --- [~jbernste] your comments are welcome > plugins must be hot loadable > -- > > Key: SOLR-14125 > URL: https://issues.apache.org/jira/browse/SOLR-14125 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Noble Paul >Priority: Major > Labels: packagemanager > Time Spent: 10m > Remaining Estimate: 0h > > Today, the class is loaded at the core load time an a stong reference is > held. Package loading need it to be lazily loaded -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Assigned] (SOLR-13808) Query DSL should let to cache filter
[
https://issues.apache.org/jira/browse/SOLR-13808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Khludnev reassigned SOLR-13808:
---
Assignee: Mikhail Khludnev
> Query DSL should let to cache filter
>
>
> Key: SOLR-13808
> URL: https://issues.apache.org/jira/browse/SOLR-13808
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mikhail Khludnev
>Assignee: Mikhail Khludnev
>Priority: Major
> Attachments: SOLR-13808.patch
>
>
> Query DSL let to express Lucene BQ's filter
>
> {code:java}
> { query: {bool: { filter: {term: {f:name,query:"foo bar"}}} }}{code}
> However, it might easily catch the need in caching it in filter cache. This
> might rely on ExtensibleQuery and QParser:
> {code:java}
> { query: {bool: { filter: {term: {f:name,query:"foo bar", cache:true}}} }}
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13808) Query DSL should let to cache filter
[
https://issues.apache.org/jira/browse/SOLR-13808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001777#comment-17001777
]
Mikhail Khludnev commented on SOLR-13808:
-
ok. I think I'll push it next week.
> Query DSL should let to cache filter
>
>
> Key: SOLR-13808
> URL: https://issues.apache.org/jira/browse/SOLR-13808
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mikhail Khludnev
>Priority: Major
> Attachments: SOLR-13808.patch
>
>
> Query DSL let to express Lucene BQ's filter
>
> {code:java}
> { query: {bool: { filter: {term: {f:name,query:"foo bar"}}} }}{code}
> However, it might easily catch the need in caching it in filter cache. This
> might rely on ExtensibleQuery and QParser:
> {code:java}
> { query: {bool: { filter: {term: {f:name,query:"foo bar", cache:true}}} }}
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14138) Fix commented-out RequestLog in jetty.xml to use non-deprecated class
[
https://issues.apache.org/jira/browse/SOLR-14138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001776#comment-17001776
]
Robert Muir commented on SOLR-14138:
i can investigate it. i would like to make it easier since these logs contain
ip address and are easy to process. Maybe its not easy to conditionalize with
the current start.jar approach, but maybe there is a simple way. At the least
lets get off the deprecated class.
> Fix commented-out RequestLog in jetty.xml to use non-deprecated class
> -
>
> Key: SOLR-14138
> URL: https://issues.apache.org/jira/browse/SOLR-14138
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Robert Muir
>Priority: Major
>
> Currently the jetty request logging is disabled (commented out).
> But it can be useful, e.g. since it uses a standard logging format and there
> are tools to analyze it by default. Also it can be used to detect some
> attacks not otherwise logged anywhere else, since they don't make it to solr
> servlet: requests blocked at the jetty level (invalid/malformed requests,
> ones filtered by jetty IP filtering, etc).
> We should switch it from the deprecated NCSARequestLog class, instead to use
> the CustomRequestLog with either NCSA_FORMAT or EXTENDED_NCSA_FORMAT.
> {quote}
> Deprecated.
> use CustomRequestLog given format string
> CustomRequestLog.EXTENDED_NCSA_FORMAT with a RequestLogWriter
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] asfgit merged pull request #520: LUCENE-8596: The replacement of comments is a bug, in "UserDictionary.java"
asfgit merged pull request #520: LUCENE-8596: The replacement of comments is a bug, in "UserDictionary.java" URL: https://github.com/apache/lucene-solr/pull/520 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8596) The replacement of comments is a bug, in "UserDictionary.java"
[
https://issues.apache.org/jira/browse/LUCENE-8596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001775#comment-17001775
]
ASF subversion and git services commented on LUCENE-8596:
-
Commit 93309e9728210b6db413361ae4d83382408b3627 in lucene-solr's branch
refs/heads/master from Michael Sokolov
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=93309e9 ]
LUCENE-8596: Treat hash mark as comment only at beginning of line in kuromoji
user dictionary. Via Masaru Hasegawa and Satoshi Kato
> The replacement of comments is a bug, in "UserDictionary.java"
> --
>
> Key: LUCENE-8596
> URL: https://issues.apache.org/jira/browse/LUCENE-8596
> Project: Lucene - Core
> Issue Type: Bug
> Components: modules/analysis
>Reporter: miyaharas
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> [https://github.com/apache/lucene-solr/blob/1d85cd783863f75cea133fb9c452302214165a4d/lucene/analysis/kuromoji/src/java/org/apache/lucene/analysis/ja/dict/UserDictionary.java#L68]
>
> hi
> I think that this is bug.
> I think the following is correct
> {code:java}
> line = line.replaceAll ("^ #. * $", "");
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14138) Fix commented-out RequestLog in jetty.xml to use non-deprecated class
[
https://issues.apache.org/jira/browse/SOLR-14138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001774#comment-17001774
]
Jan Høydahl commented on SOLR-14138:
+1
I recently missed some Jetty logging when debugging SSL handshake problems and
having jetty logs around would help.
Is there a way we can control enabling jetty request log through an env var
instead of having to edit xml?
> Fix commented-out RequestLog in jetty.xml to use non-deprecated class
> -
>
> Key: SOLR-14138
> URL: https://issues.apache.org/jira/browse/SOLR-14138
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Robert Muir
>Priority: Major
>
> Currently the jetty request logging is disabled (commented out).
> But it can be useful, e.g. since it uses a standard logging format and there
> are tools to analyze it by default. Also it can be used to detect some
> attacks not otherwise logged anywhere else, since they don't make it to solr
> servlet: requests blocked at the jetty level (invalid/malformed requests,
> ones filtered by jetty IP filtering, etc).
> We should switch it from the deprecated NCSARequestLog class, instead to use
> the CustomRequestLog with either NCSA_FORMAT or EXTENDED_NCSA_FORMAT.
> {quote}
> Deprecated.
> use CustomRequestLog given format string
> CustomRequestLog.EXTENDED_NCSA_FORMAT with a RequestLogWriter
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mkhludnev commented on a change in pull request #1034: SOLR-13863: payload query function now handles string encoded payload field (delimited_payloads_string)
mkhludnev commented on a change in pull request #1034: SOLR-13863: payload
query function now handles string encoded payload field
(delimited_payloads_string)
URL: https://github.com/apache/lucene-solr/pull/1034#discussion_r360661332
##
File path: lucene/core/src/java/org/apache/lucene/index/PostingsEnum.java
##
@@ -67,6 +67,56 @@ public static boolean featureRequested(int flags, short
feature) {
protected PostingsEnum() {
}
+ /**
+ * Dummy PostingsEnum object.
+ */
+ public static PostingsEnum DUMMY_INSTANCE = new PostingsEnum() {
Review comment:
Please look at org.apache.lucene.search.DocIdSetIterator.empty()
There's subtle exhausting logic which needs to be supported and you can
follow naming as well.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mkhludnev commented on a change in pull request #1034: SOLR-13863: payload query function now handles string encoded payload field (delimited_payloads_string)
mkhludnev commented on a change in pull request #1034: SOLR-13863: payload
query function now handles string encoded payload field
(delimited_payloads_string)
URL: https://github.com/apache/lucene-solr/pull/1034#discussion_r360661472
##
File path: solr/core/src/java/org/apache/solr/search/PayloadValueSource.java
##
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.search;
+
+import org.apache.lucene.queries.function.ValueSource;
+import org.apache.lucene.util.BytesRef;
+
+public abstract class PayloadValueSource extends ValueSource {
+ protected final String field;
+ protected final String val;
+ protected final String indexedField;
+ protected final BytesRef indexedBytes;
+ protected final ValueSource defaultValueSource;
+
+ public PayloadValueSource(String field, String val, String indexedField,
BytesRef indexedBytes, ValueSource defaultValueSource) {
+this.field = field;
+this.val = val;
+this.indexedField = indexedField;
+this.indexedBytes = indexedBytes;
+this.defaultValueSource = defaultValueSource;
+ }
+
+// public abstract SortField getSortField(boolean reverse);
+
+ // TODO: should this be formalized at the ValueSource level? Seems to be
the convention
+ public abstract String name();
+
+ @Override
+ public abstract String description();
+
+ @Override
+ public boolean equals(Object o) {
+if (this == o) return true;
+if (o == null || getClass() != o.getClass()) return false;
+
+PayloadValueSource that = (PayloadValueSource) o;
+
+if (!indexedField.equals(that.indexedField)) return false;
+if (indexedBytes != null ? !indexedBytes.equals(that.indexedBytes) :
that.indexedBytes != null) return false;
+return defaultValueSource.equals(that.defaultValueSource);
+
+ }
+
+ @Override
+ public int hashCode() {
Review comment:
How it will be supported by descendants?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mkhludnev commented on a change in pull request #1034: SOLR-13863: payload query function now handles string encoded payload field (delimited_payloads_string)
mkhludnev commented on a change in pull request #1034: SOLR-13863: payload
query function now handles string encoded payload field
(delimited_payloads_string)
URL: https://github.com/apache/lucene-solr/pull/1034#discussion_r360662836
##
File path: solr/core/src/java/org/apache/solr/search/ValueSourceParser.java
##
@@ -722,13 +722,29 @@ public ValueSource parse(FunctionQParser fp) throws
SyntaxError {
TInfo tinfo = parseTerm(fp); // would have made this parser a new
separate class and registered it, but this handy method is private :/
-ValueSource defaultValueSource;
-if (fp.hasMoreArguments()) {
- defaultValueSource = fp.parseValueSource();
-} else {
- defaultValueSource = new ConstValueSource(0.0f);
+IndexSchema schema = fp.getReq().getCore().getLatestSchema();
+final FieldType fieldType = schema.getFieldType(tinfo.field);
+
+final String payloadEncoder =
PayloadUtils.getPayloadEncoder(fieldType);
+if (payloadEncoder.equals("identity")) {
+
+ ValueSource defaultValueSource = (fp.hasMoreArguments()) ?
fp.parseValueSource() : new LiteralValueSource("");
+
+ if (fp.hasMoreArguments()) {
+ // functions are not supported with strings
+ throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
"Invalid payload function: " + fp.parseArg());
Review comment:
Please explain why it;s invalid in exception message. It seems like in/mas
isn't supported so far.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mkhludnev commented on a change in pull request #1034: SOLR-13863: payload query function now handles string encoded payload field (delimited_payloads_string)
mkhludnev commented on a change in pull request #1034: SOLR-13863: payload
query function now handles string encoded payload field
(delimited_payloads_string)
URL: https://github.com/apache/lucene-solr/pull/1034#discussion_r360661577
##
File path:
solr/core/src/java/org/apache/solr/search/StringPayloadValueSource.java
##
@@ -0,0 +1,271 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.search;
+
+import java.io.IOException;
+import java.util.Map;
+
+import org.apache.lucene.index.LeafReaderContext;
+import org.apache.lucene.index.PostingsEnum;
+import org.apache.lucene.index.Terms;
+import org.apache.lucene.index.TermsEnum;
+import org.apache.lucene.queries.function.FunctionValues;
+import org.apache.lucene.queries.function.ValueSource;
+import org.apache.lucene.queries.function.docvalues.StrDocValues;
+import org.apache.lucene.search.DocIdSetIterator;
+import org.apache.lucene.search.FieldComparator;
+import org.apache.lucene.search.FieldComparatorSource;
+import org.apache.lucene.search.IndexSearcher;
+import org.apache.lucene.search.SimpleFieldComparator;
+import org.apache.lucene.search.SortField;
+import org.apache.lucene.util.BytesRef;
+
+public class StringPayloadValueSource extends PayloadValueSource {
+
+ public StringPayloadValueSource(String field, String val, String
indexedField, BytesRef indexedBytes, ValueSource defaultValueSource) {
+super(field, val, indexedField, indexedBytes, defaultValueSource);
+ }
+
+ public SortField getSortField(boolean reverse) {
+return new StringPayloadValueSourceSortField(reverse);
+ }
+
+ @Override
+ public FunctionValues getValues(Map context, LeafReaderContext
readerContext) throws IOException {
+
+final Terms terms = readerContext.reader().terms(indexedField);
+
+FunctionValues defaultValues = defaultValueSource.getValues(context,
readerContext);
+
+// copied the bulk of this from TFValueSource - TODO: this is a very
repeated pattern - base-class this advance logic stuff?
+return new StrDocValues(this) {
+ PostingsEnum docs;
+ int atDoc;
+ int lastDocRequested = -1;
+ String docValue = null;
+
+ {
Review comment:
@eribeiro , to invoke reset() on instance construction.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mkhludnev commented on a change in pull request #1034: SOLR-13863: payload query function now handles string encoded payload field (delimited_payloads_string)
mkhludnev commented on a change in pull request #1034: SOLR-13863: payload
query function now handles string encoded payload field
(delimited_payloads_string)
URL: https://github.com/apache/lucene-solr/pull/1034#discussion_r360662950
##
File path:
solr/core/src/test/org/apache/solr/search/function/TestFunctionQuery.java
##
@@ -569,6 +570,57 @@ public void testPayloadFunction() {
assertQ(req("fl","*,score","q", "{!func}payload(vals_dpf,A)",
CommonParams.DEBUG, "true"), "//float[@name='score']='1.0'");
}
+ @Test
+ public void testStringPayloadFunction() {
+clearIndex();
+
+SolrInputDocument doc = new SolrInputDocument();
+doc.setField("id", "1");
+doc.setField("vals_dpss", new String[]{"A|USD", "C|EUR", "SORT|A"});
+assertU(adoc(doc));
+
+doc = new SolrInputDocument();
+doc.setField("id", "2");
+doc.setField("vals_dpss", new String[]{"A|usd", "C|eur", "SORT|B"});
+assertU(adoc(doc));
+
+doc = new SolrInputDocument();
+doc.setField("id", "3");
+doc.setField("vals_dpss", new String[]{"A|Usd", "C|Eur", "SORT|C"});
+assertU(adoc(doc));
+
+assertU(commit());
+
+assertQ(req("q", "id:1", "fl", "*,score,field_test:payload(vals_dpss,A)"),
"//str[@name='field_test']='USD'");
+assertQ(req("q", "id:2", "fl", "*,score,field_test:payload(vals_dpss,C)"),
"//str[@name='field_test']='eur'");
+assertQ(req("q", "*:*",
+ "rows", "1",
+ "sort", "payload(vals_dpss,C) asc",
+ "fl", "*,score,field_test:payload(vals_dpss,C)"),
"//str[@name='field_test']='EUR'");
Review comment:
I don't see how sort order is asserted.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8596) The replacement of comments is a bug, in "UserDictionary.java"
[
https://issues.apache.org/jira/browse/LUCENE-8596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001766#comment-17001766
]
Michael Sokolov commented on LUCENE-8596:
-
See also LUCENE-7635
> The replacement of comments is a bug, in "UserDictionary.java"
> --
>
> Key: LUCENE-8596
> URL: https://issues.apache.org/jira/browse/LUCENE-8596
> Project: Lucene - Core
> Issue Type: Bug
> Components: modules/analysis
>Reporter: miyaharas
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> [https://github.com/apache/lucene-solr/blob/1d85cd783863f75cea133fb9c452302214165a4d/lucene/analysis/kuromoji/src/java/org/apache/lucene/analysis/ja/dict/UserDictionary.java#L68]
>
> hi
> I think that this is bug.
> I think the following is correct
> {code:java}
> line = line.replaceAll ("^ #. * $", "");
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-7635) Kuromoji fails if user dictionary contains #
[
https://issues.apache.org/jira/browse/LUCENE-7635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Sokolov resolved LUCENE-7635.
-
Resolution: Duplicate
This same issue came up again recently in LUCENE-8596; we'll address it there
since this patch can no longer be applied, but I'll mention the OP in the commit
> Kuromoji fails if user dictionary contains #
>
>
> Key: LUCENE-7635
> URL: https://issues.apache.org/jira/browse/LUCENE-7635
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Masaru Hasegawa
>Priority: Major
> Attachments: LUCENE-7635.patch
>
>
> If user dictionary contains entries like:
> {code}
> withsharp#,withsharp#,withsharp#,カスタム名詞
> {code}
> It fails to create dictionary throwing
> java.lang.ArrayIndexOutOfBoundsException.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-14138) Fix commented-out RequestLog in jetty.xml to use non-deprecated class
Robert Muir created SOLR-14138:
--
Summary: Fix commented-out RequestLog in jetty.xml to use
non-deprecated class
Key: SOLR-14138
URL: https://issues.apache.org/jira/browse/SOLR-14138
Project: Solr
Issue Type: Improvement
Security Level: Public (Default Security Level. Issues are Public)
Reporter: Robert Muir
Currently the jetty request logging is disabled (commented out).
But it can be useful, e.g. since it uses a standard logging format and there
are tools to analyze it by default. Also it can be used to detect some attacks
not otherwise logged anywhere else, since they don't make it to solr servlet:
requests blocked at the jetty level (invalid/malformed requests, ones filtered
by jetty IP filtering, etc).
We should switch it from the deprecated NCSARequestLog class, instead to use
the CustomRequestLog with either NCSA_FORMAT or EXTENDED_NCSA_FORMAT.
{quote}
Deprecated.
use CustomRequestLog given format string CustomRequestLog.EXTENDED_NCSA_FORMAT
with a RequestLogWriter
{quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-13985) bind to localhost by default
[
https://issues.apache.org/jira/browse/SOLR-13985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Gerlowski updated SOLR-13985:
---
Attachment: SOLR-13985.patch
Status: Open (was: Open)
Uploading a patch to address some of the feedback given so far. This mostly
fixes some duplication in the docs and some bugs in the Windows batch script.
bq. [re: docs duplication] Alternatively it should be possible to tag the
paragraphs in securing-solr in some way so that you can include them with a
reference in taking-solr-to-production
This is the path I ended up going down. Worked out pretty nicely.
bq. should it be instead IF DEFINED
Yep, fixed.
*On Naming*
Jan pointed out that we could come up with a better name than
{{SOLR_JETTY_HOST}}. David pointed out that we have a smattering of other
properties with at least some overlap ("host" in solr.xml, for one), and that
we should aim for consistency in how these are set, and better documentation
around what each does.
These are both good points, and confusing pieces of config that could benefit
from straightening out. I straightened out the "jetty.host" vs
"solr.jetty.host" http/https inconsistency. But in the interest of not letting
the perfect get in the way of the good, I'd rather move that
investigation/untangling for a separate jira. I'll file a follow-up jira to
re-examine the names and documentation around these host/port config values,
and try to make progress on it over the holidays. I'm not sure I have enough
time to fully untangle these additional settings and put together fixes, but
I'll try to put together at least a good writeup so someone else can tackle it
if I'm not able to.
*Moving Forward*
I just finished testing this. It looks good on both Windows and Linux. Any
last qualms or votes against merging this? It's a pretty high impact change
for users, but it addresses a real security need and the documentation makes it
pretty clear how to change the default when necessary. I'll aim to commit
after the holidays if there's no additional concerns/feedback. I might send
out a dev mail to let others know this is happening as well.
> bind to localhost by default
>
>
> Key: SOLR-13985
> URL: https://issues.apache.org/jira/browse/SOLR-13985
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Robert Muir
>Assignee: Jason Gerlowski
>Priority: Major
> Attachments: SOLR-13985.patch, SOLR-13985.patch, SOLR-13985.patch,
> SOLR-13985.patch
>
>
> Currently solr binds to all interfaces by default.
> The default should be safer, so that e.g. the user is not exposed to the
> internet until they make an explicit step to do so.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13709) Race condition on core reload while core is still loading?
[
https://issues.apache.org/jira/browse/SOLR-13709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001746#comment-17001746
]
Erick Erickson commented on SOLR-13709:
---
[~hossman] any thoughts?
I've finally gotten back to this and think I'm getting close. The short form is
that I think the test calls delete collection before prior operations are
complete.
During a successful run, I see:
* a bunch of reloads
* a bunch of unloads
- The reloads seem OK, they're being run from
{code}SolrCore.getConfListener{code}.
- The unload is coming when the collection is being deleted near the end of the
test.
AFAIK, the only way the CoreDescriptor could be disappearing form the various
lists is through unload and that's only being called by deleteCollection in
this test.
So what my next best guess is is that the reloads aren't complete by the time
the test gets to the delete collection call and that's where the race condition
is coming from.
I'm putting some more debugging, I want to see all thread dumps when the NPE
occurs. I'm wondering if the sequencing in from the Overseer is part of the
problem, but that's a guess.
> Race condition on core reload while core is still loading?
> --
>
> Key: SOLR-13709
> URL: https://issues.apache.org/jira/browse/SOLR-13709
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Chris M. Hostetter
>Assignee: Erick Erickson
>Priority: Major
> Attachments: apache_Lucene-Solr-Tests-8.x_449.log.txt
>
>
> A recent jenkins failure from {{TestSolrCLIRunExample}} seems to suggest that
> there may be a race condition when attempting to re-load a SolrCore while the
> core is currently in the process of (re)loading that can leave the SolrCore
> in an unusable state.
> Details to follow...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14137) Boosting by date (and perhaps others) shows a steady decline 6.6->8.3
[
https://issues.apache.org/jira/browse/SOLR-14137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Erick Erickson updated SOLR-14137:
--
Attachment: Screen Shot 2019-12-19 at 3.09.37 PM.png
Screen Shot 2019-12-19 at 3.31.16 PM.png
Status: Open (was: Open)
Here's a couple of screenshots of a bit of profiling I did from 6.6 and 8.3.
The problem here is that they claim the time is spent in DisiPriorityQueue in
both, which hasn't changed. So the difference must be higher up in the
process
I'll dig a bit more sometime over the holidays.
> Boosting by date (and perhaps others) shows a steady decline 6.6->8.3
> -
>
> Key: SOLR-14137
> URL: https://issues.apache.org/jira/browse/SOLR-14137
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Erick Erickson
>Priority: Major
> Attachments: Screen Shot 2019-12-19 at 2.35.41 PM.png, Screen Shot
> 2019-12-19 at 3.09.37 PM.png, Screen Shot 2019-12-19 at 3.31.16 PM.png
>
>
> Moving a user's list discussion over here.
> {color:#00}The very short form is that from Solr 6.6.1 to Solr 8.3.1, the
> throughput for date boosting in my tests dropped by 40+%{color}
> {color:#00}I’ve been hearing about slowdowns in successive Solr releases
> with boost functions, so I dug into it a bit. The test setup is just a
> boost-by-date with an additional big OR clause of 100 random words so I’d be
> sure to hit a bunch of docs. I figured that if there were few hits, the
> signal would be lost in the noise, but I didn’t look at the actual hit
> counts.{color}
> {color:#00}I saw several Solr JIRAs about this subject, but they were
> slightly different, although quite possibly the same underlying issue. So I
> tried to get this down to a very specific form of a query.{color}
> {color:#00}I’ve also seen some cases in the wild where the response was
> proportional to the number of segments, thus my optimize experiments.{color}
> {color:#00}Here are the results, explanation below. O stands for
> optimized to one segment. I spot checked pdate against 7x and 8x and they
> weren’t significantly different performance wise from tdate. All have
> docValues enabled. I ran these against a multiValued=“false” field. All the
> tests pegged all my CPUs. Jmeter is being run on a different machine than
> Solr. Only one Solr was running for any test.{color}
> {color:#00}Solr version queries/min {color}
> {color:#00}6.6.1 3,400 {color}
> {color:#00}6.6.1 O 4,800 {color}
> {color:#00}7.1 2,800 {color}
> {color:#00}7.1 O 4,200 {color}
> {color:#00}7.7.1 2,400 {color}
> {color:#00}7.7.1 O 3,500 {color}
> {color:#00}8.3.1 2,000 {color}
> {color:#00}8.3.1 O 2,600 {color}
> {color:#00}The tests I’ve been running just index 20M docs into a single
> core, then run the exact same 10,000 queries against them from jmeter with 24
> threads. Spot checks showed no hits on the queryResultCache.{color}
> {color:#00}A query looks like this: {color}
> {color:#00}rows=0&\{!boost b=recip(ms(NOW,
> INSERT_FIELD_HERE),3.16e-11,1,1)}text_txt:(campaigners OR adjourned OR
> anyplace…97 more random words){color}
> {color:#00}There is no faceting. No grouping. No sorting.{color}
> {color:#00}I fill in INSERT_FIELD_HERE through jmeter magic. I’m running
> the exact same queries for every test.{color}
> {color:#00}One wildcard is that I did regenerate the index for each major
> revision, and the chose random words from the same list of words, as well as
> random times (bounded in the same range though) so the docs are not
> completely identical. The index was in the native format for that major
> version even if slightly different between versions. I ran the test once,
> then ran it again after optimizing the index.{color}
> {color:#00}I haven’t dug any farther, if anyone’s interested I can throw
> a profiler at, say, 8.3 and see what I can see, although I’m not going to
> have time to dive into this any time soon. I’d be glad to run some tests
> though. I saved the queries and the indexes so running a test would only
> take a few minutes.{color}
> {color:#00}While I concentrated on date fields, the docs have date, int,
> and long fields, both docValues=true and docValues=false, each variant with
> multiValued=true and multiValued=false and both Trie and Point (where
> possible) variants as well as a pretty simple text field.{color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
[jira] [Commented] (SOLR-14137) Boosting by date (and perhaps others) shows a steady decline 6.6->8.3
[
https://issues.apache.org/jira/browse/SOLR-14137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001742#comment-17001742
]
Erick Erickson commented on SOLR-14137:
---
Likely the same underlying cause?
> Boosting by date (and perhaps others) shows a steady decline 6.6->8.3
> -
>
> Key: SOLR-14137
> URL: https://issues.apache.org/jira/browse/SOLR-14137
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Erick Erickson
>Priority: Major
> Attachments: Screen Shot 2019-12-19 at 2.35.41 PM.png
>
>
> Moving a user's list discussion over here.
> {color:#00}The very short form is that from Solr 6.6.1 to Solr 8.3.1, the
> throughput for date boosting in my tests dropped by 40+%{color}
> {color:#00}I’ve been hearing about slowdowns in successive Solr releases
> with boost functions, so I dug into it a bit. The test setup is just a
> boost-by-date with an additional big OR clause of 100 random words so I’d be
> sure to hit a bunch of docs. I figured that if there were few hits, the
> signal would be lost in the noise, but I didn’t look at the actual hit
> counts.{color}
> {color:#00}I saw several Solr JIRAs about this subject, but they were
> slightly different, although quite possibly the same underlying issue. So I
> tried to get this down to a very specific form of a query.{color}
> {color:#00}I’ve also seen some cases in the wild where the response was
> proportional to the number of segments, thus my optimize experiments.{color}
> {color:#00}Here are the results, explanation below. O stands for
> optimized to one segment. I spot checked pdate against 7x and 8x and they
> weren’t significantly different performance wise from tdate. All have
> docValues enabled. I ran these against a multiValued=“false” field. All the
> tests pegged all my CPUs. Jmeter is being run on a different machine than
> Solr. Only one Solr was running for any test.{color}
> {color:#00}Solr version queries/min {color}
> {color:#00}6.6.1 3,400 {color}
> {color:#00}6.6.1 O 4,800 {color}
> {color:#00}7.1 2,800 {color}
> {color:#00}7.1 O 4,200 {color}
> {color:#00}7.7.1 2,400 {color}
> {color:#00}7.7.1 O 3,500 {color}
> {color:#00}8.3.1 2,000 {color}
> {color:#00}8.3.1 O 2,600 {color}
> {color:#00}The tests I’ve been running just index 20M docs into a single
> core, then run the exact same 10,000 queries against them from jmeter with 24
> threads. Spot checks showed no hits on the queryResultCache.{color}
> {color:#00}A query looks like this: {color}
> {color:#00}rows=0&\{!boost b=recip(ms(NOW,
> INSERT_FIELD_HERE),3.16e-11,1,1)}text_txt:(campaigners OR adjourned OR
> anyplace…97 more random words){color}
> {color:#00}There is no faceting. No grouping. No sorting.{color}
> {color:#00}I fill in INSERT_FIELD_HERE through jmeter magic. I’m running
> the exact same queries for every test.{color}
> {color:#00}One wildcard is that I did regenerate the index for each major
> revision, and the chose random words from the same list of words, as well as
> random times (bounded in the same range though) so the docs are not
> completely identical. The index was in the native format for that major
> version even if slightly different between versions. I ran the test once,
> then ran it again after optimizing the index.{color}
> {color:#00}I haven’t dug any farther, if anyone’s interested I can throw
> a profiler at, say, 8.3 and see what I can see, although I’m not going to
> have time to dive into this any time soon. I’d be glad to run some tests
> though. I saved the queries and the indexes so running a test would only
> take a few minutes.{color}
> {color:#00}While I concentrated on date fields, the docs have date, int,
> and long fields, both docValues=true and docValues=false, each variant with
> multiValued=true and multiValued=false and both Trie and Point (where
> possible) variants as well as a pretty simple text field.{color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14137) Boosting by date (and perhaps others) shows a steady decline 6.6->8.3
[
https://issues.apache.org/jira/browse/SOLR-14137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001739#comment-17001739
]
Erick Erickson commented on SOLR-14137:
---
>From Joel Bernstein:
Here is a quantile plot that shows the performance slowdown of the boost query,
from Ericks test runs.
{color:#00}The x-axis is the percentiles and the y-axis is the QTime for
the different Solr versions at each percentile. The median QTime's are shown in
the tooltip.{color}
{color:#00}(I'll attach){color}
> Boosting by date (and perhaps others) shows a steady decline 6.6->8.3
> -
>
> Key: SOLR-14137
> URL: https://issues.apache.org/jira/browse/SOLR-14137
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Erick Erickson
>Priority: Major
> Attachments: Screen Shot 2019-12-19 at 2.35.41 PM.png
>
>
> Moving a user's list discussion over here.
> {color:#00}The very short form is that from Solr 6.6.1 to Solr 8.3.1, the
> throughput for date boosting in my tests dropped by 40+%{color}
> {color:#00}I’ve been hearing about slowdowns in successive Solr releases
> with boost functions, so I dug into it a bit. The test setup is just a
> boost-by-date with an additional big OR clause of 100 random words so I’d be
> sure to hit a bunch of docs. I figured that if there were few hits, the
> signal would be lost in the noise, but I didn’t look at the actual hit
> counts.{color}
> {color:#00}I saw several Solr JIRAs about this subject, but they were
> slightly different, although quite possibly the same underlying issue. So I
> tried to get this down to a very specific form of a query.{color}
> {color:#00}I’ve also seen some cases in the wild where the response was
> proportional to the number of segments, thus my optimize experiments.{color}
> {color:#00}Here are the results, explanation below. O stands for
> optimized to one segment. I spot checked pdate against 7x and 8x and they
> weren’t significantly different performance wise from tdate. All have
> docValues enabled. I ran these against a multiValued=“false” field. All the
> tests pegged all my CPUs. Jmeter is being run on a different machine than
> Solr. Only one Solr was running for any test.{color}
> {color:#00}Solr version queries/min {color}
> {color:#00}6.6.1 3,400 {color}
> {color:#00}6.6.1 O 4,800 {color}
> {color:#00}7.1 2,800 {color}
> {color:#00}7.1 O 4,200 {color}
> {color:#00}7.7.1 2,400 {color}
> {color:#00}7.7.1 O 3,500 {color}
> {color:#00}8.3.1 2,000 {color}
> {color:#00}8.3.1 O 2,600 {color}
> {color:#00}The tests I’ve been running just index 20M docs into a single
> core, then run the exact same 10,000 queries against them from jmeter with 24
> threads. Spot checks showed no hits on the queryResultCache.{color}
> {color:#00}A query looks like this: {color}
> {color:#00}rows=0&\{!boost b=recip(ms(NOW,
> INSERT_FIELD_HERE),3.16e-11,1,1)}text_txt:(campaigners OR adjourned OR
> anyplace…97 more random words){color}
> {color:#00}There is no faceting. No grouping. No sorting.{color}
> {color:#00}I fill in INSERT_FIELD_HERE through jmeter magic. I’m running
> the exact same queries for every test.{color}
> {color:#00}One wildcard is that I did regenerate the index for each major
> revision, and the chose random words from the same list of words, as well as
> random times (bounded in the same range though) so the docs are not
> completely identical. The index was in the native format for that major
> version even if slightly different between versions. I ran the test once,
> then ran it again after optimizing the index.{color}
> {color:#00}I haven’t dug any farther, if anyone’s interested I can throw
> a profiler at, say, 8.3 and see what I can see, although I’m not going to
> have time to dive into this any time soon. I’d be glad to run some tests
> though. I saved the queries and the indexes so running a test would only
> take a few minutes.{color}
> {color:#00}While I concentrated on date fields, the docs have date, int,
> and long fields, both docValues=true and docValues=false, each variant with
> multiValued=true and multiValued=false and both Trie and Point (where
> possible) variants as well as a pretty simple text field.{color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional comman
[jira] [Updated] (SOLR-14137) Boosting by date (and perhaps others) shows a steady decline 6.6->8.3
[
https://issues.apache.org/jira/browse/SOLR-14137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Erick Erickson updated SOLR-14137:
--
Attachment: Screen Shot 2019-12-19 at 2.35.41 PM.png
Status: Open (was: Open)
Joel's screenshot
> Boosting by date (and perhaps others) shows a steady decline 6.6->8.3
> -
>
> Key: SOLR-14137
> URL: https://issues.apache.org/jira/browse/SOLR-14137
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Erick Erickson
>Priority: Major
> Attachments: Screen Shot 2019-12-19 at 2.35.41 PM.png
>
>
> Moving a user's list discussion over here.
> {color:#00}The very short form is that from Solr 6.6.1 to Solr 8.3.1, the
> throughput for date boosting in my tests dropped by 40+%{color}
> {color:#00}I’ve been hearing about slowdowns in successive Solr releases
> with boost functions, so I dug into it a bit. The test setup is just a
> boost-by-date with an additional big OR clause of 100 random words so I’d be
> sure to hit a bunch of docs. I figured that if there were few hits, the
> signal would be lost in the noise, but I didn’t look at the actual hit
> counts.{color}
> {color:#00}I saw several Solr JIRAs about this subject, but they were
> slightly different, although quite possibly the same underlying issue. So I
> tried to get this down to a very specific form of a query.{color}
> {color:#00}I’ve also seen some cases in the wild where the response was
> proportional to the number of segments, thus my optimize experiments.{color}
> {color:#00}Here are the results, explanation below. O stands for
> optimized to one segment. I spot checked pdate against 7x and 8x and they
> weren’t significantly different performance wise from tdate. All have
> docValues enabled. I ran these against a multiValued=“false” field. All the
> tests pegged all my CPUs. Jmeter is being run on a different machine than
> Solr. Only one Solr was running for any test.{color}
> {color:#00}Solr version queries/min {color}
> {color:#00}6.6.1 3,400 {color}
> {color:#00}6.6.1 O 4,800 {color}
> {color:#00}7.1 2,800 {color}
> {color:#00}7.1 O 4,200 {color}
> {color:#00}7.7.1 2,400 {color}
> {color:#00}7.7.1 O 3,500 {color}
> {color:#00}8.3.1 2,000 {color}
> {color:#00}8.3.1 O 2,600 {color}
> {color:#00}The tests I’ve been running just index 20M docs into a single
> core, then run the exact same 10,000 queries against them from jmeter with 24
> threads. Spot checks showed no hits on the queryResultCache.{color}
> {color:#00}A query looks like this: {color}
> {color:#00}rows=0&\{!boost b=recip(ms(NOW,
> INSERT_FIELD_HERE),3.16e-11,1,1)}text_txt:(campaigners OR adjourned OR
> anyplace…97 more random words){color}
> {color:#00}There is no faceting. No grouping. No sorting.{color}
> {color:#00}I fill in INSERT_FIELD_HERE through jmeter magic. I’m running
> the exact same queries for every test.{color}
> {color:#00}One wildcard is that I did regenerate the index for each major
> revision, and the chose random words from the same list of words, as well as
> random times (bounded in the same range though) so the docs are not
> completely identical. The index was in the native format for that major
> version even if slightly different between versions. I ran the test once,
> then ran it again after optimizing the index.{color}
> {color:#00}I haven’t dug any farther, if anyone’s interested I can throw
> a profiler at, say, 8.3 and see what I can see, although I’m not going to
> have time to dive into this any time soon. I’d be glad to run some tests
> though. I saved the queries and the indexes so running a test would only
> take a few minutes.{color}
> {color:#00}While I concentrated on date fields, the docs have date, int,
> and long fields, both docValues=true and docValues=false, each variant with
> multiValued=true and multiValued=false and both Trie and Point (where
> possible) variants as well as a pretty simple text field.{color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14137) Boosting by date (and perhaps others) shows a steady decline 6.6->8.3
[
https://issues.apache.org/jira/browse/SOLR-14137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001738#comment-17001738
]
Erick Erickson commented on SOLR-14137:
---
>From David Smiley:
{color:#00}This unfolding story shows us why we need nightly benchmarks of
Solr -- {color}SOLR-10317
> Boosting by date (and perhaps others) shows a steady decline 6.6->8.3
> -
>
> Key: SOLR-14137
> URL: https://issues.apache.org/jira/browse/SOLR-14137
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Erick Erickson
>Priority: Major
>
> Moving a user's list discussion over here.
> {color:#00}The very short form is that from Solr 6.6.1 to Solr 8.3.1, the
> throughput for date boosting in my tests dropped by 40+%{color}
> {color:#00}I’ve been hearing about slowdowns in successive Solr releases
> with boost functions, so I dug into it a bit. The test setup is just a
> boost-by-date with an additional big OR clause of 100 random words so I’d be
> sure to hit a bunch of docs. I figured that if there were few hits, the
> signal would be lost in the noise, but I didn’t look at the actual hit
> counts.{color}
> {color:#00}I saw several Solr JIRAs about this subject, but they were
> slightly different, although quite possibly the same underlying issue. So I
> tried to get this down to a very specific form of a query.{color}
> {color:#00}I’ve also seen some cases in the wild where the response was
> proportional to the number of segments, thus my optimize experiments.{color}
> {color:#00}Here are the results, explanation below. O stands for
> optimized to one segment. I spot checked pdate against 7x and 8x and they
> weren’t significantly different performance wise from tdate. All have
> docValues enabled. I ran these against a multiValued=“false” field. All the
> tests pegged all my CPUs. Jmeter is being run on a different machine than
> Solr. Only one Solr was running for any test.{color}
> {color:#00}Solr version queries/min {color}
> {color:#00}6.6.1 3,400 {color}
> {color:#00}6.6.1 O 4,800 {color}
> {color:#00}7.1 2,800 {color}
> {color:#00}7.1 O 4,200 {color}
> {color:#00}7.7.1 2,400 {color}
> {color:#00}7.7.1 O 3,500 {color}
> {color:#00}8.3.1 2,000 {color}
> {color:#00}8.3.1 O 2,600 {color}
> {color:#00}The tests I’ve been running just index 20M docs into a single
> core, then run the exact same 10,000 queries against them from jmeter with 24
> threads. Spot checks showed no hits on the queryResultCache.{color}
> {color:#00}A query looks like this: {color}
> {color:#00}rows=0&\{!boost b=recip(ms(NOW,
> INSERT_FIELD_HERE),3.16e-11,1,1)}text_txt:(campaigners OR adjourned OR
> anyplace…97 more random words){color}
> {color:#00}There is no faceting. No grouping. No sorting.{color}
> {color:#00}I fill in INSERT_FIELD_HERE through jmeter magic. I’m running
> the exact same queries for every test.{color}
> {color:#00}One wildcard is that I did regenerate the index for each major
> revision, and the chose random words from the same list of words, as well as
> random times (bounded in the same range though) so the docs are not
> completely identical. The index was in the native format for that major
> version even if slightly different between versions. I ran the test once,
> then ran it again after optimizing the index.{color}
> {color:#00}I haven’t dug any farther, if anyone’s interested I can throw
> a profiler at, say, 8.3 and see what I can see, although I’m not going to
> have time to dive into this any time soon. I’d be glad to run some tests
> though. I saved the queries and the indexes so running a test would only
> take a few minutes.{color}
> {color:#00}While I concentrated on date fields, the docs have date, int,
> and long fields, both docValues=true and docValues=false, each variant with
> multiValued=true and multiValued=false and both Trie and Point (where
> possible) variants as well as a pretty simple text field.{color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-14137) Boosting by date (and perhaps others) shows a steady decline 6.6->8.3
Erick Erickson created SOLR-14137:
-
Summary: Boosting by date (and perhaps others) shows a steady
decline 6.6->8.3
Key: SOLR-14137
URL: https://issues.apache.org/jira/browse/SOLR-14137
Project: Solr
Issue Type: Improvement
Security Level: Public (Default Security Level. Issues are Public)
Reporter: Erick Erickson
Moving a user's list discussion over here.
{color:#00}The very short form is that from Solr 6.6.1 to Solr 8.3.1, the
throughput for date boosting in my tests dropped by 40+%{color}
{color:#00}I’ve been hearing about slowdowns in successive Solr releases
with boost functions, so I dug into it a bit. The test setup is just a
boost-by-date with an additional big OR clause of 100 random words so I’d be
sure to hit a bunch of docs. I figured that if there were few hits, the signal
would be lost in the noise, but I didn’t look at the actual hit counts.{color}
{color:#00}I saw several Solr JIRAs about this subject, but they were
slightly different, although quite possibly the same underlying issue. So I
tried to get this down to a very specific form of a query.{color}
{color:#00}I’ve also seen some cases in the wild where the response was
proportional to the number of segments, thus my optimize experiments.{color}
{color:#00}Here are the results, explanation below. O stands for optimized
to one segment. I spot checked pdate against 7x and 8x and they weren’t
significantly different performance wise from tdate. All have docValues
enabled. I ran these against a multiValued=“false” field. All the tests pegged
all my CPUs. Jmeter is being run on a different machine than Solr. Only one
Solr was running for any test.{color}
{color:#00}Solr version queries/min {color}
{color:#00}6.6.1 3,400 {color}
{color:#00}6.6.1 O 4,800 {color}
{color:#00}7.1 2,800 {color}
{color:#00}7.1 O 4,200 {color}
{color:#00}7.7.1 2,400 {color}
{color:#00}7.7.1 O 3,500 {color}
{color:#00}8.3.1 2,000 {color}
{color:#00}8.3.1 O 2,600 {color}
{color:#00}The tests I’ve been running just index 20M docs into a single
core, then run the exact same 10,000 queries against them from jmeter with 24
threads. Spot checks showed no hits on the queryResultCache.{color}
{color:#00}A query looks like this: {color}
{color:#00}rows=0&\{!boost b=recip(ms(NOW,
INSERT_FIELD_HERE),3.16e-11,1,1)}text_txt:(campaigners OR adjourned OR
anyplace…97 more random words){color}
{color:#00}There is no faceting. No grouping. No sorting.{color}
{color:#00}I fill in INSERT_FIELD_HERE through jmeter magic. I’m running
the exact same queries for every test.{color}
{color:#00}One wildcard is that I did regenerate the index for each major
revision, and the chose random words from the same list of words, as well as
random times (bounded in the same range though) so the docs are not completely
identical. The index was in the native format for that major version even if
slightly different between versions. I ran the test once, then ran it again
after optimizing the index.{color}
{color:#00}I haven’t dug any farther, if anyone’s interested I can throw a
profiler at, say, 8.3 and see what I can see, although I’m not going to have
time to dive into this any time soon. I’d be glad to run some tests though. I
saved the queries and the indexes so running a test would only take a few
minutes.{color}
{color:#00}While I concentrated on date fields, the docs have date, int,
and long fields, both docValues=true and docValues=false, each variant with
multiValued=true and multiValued=false and both Trie and Point (where possible)
variants as well as a pretty simple text field.{color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] rmuir commented on a change in pull request #1082: SOLR-13984: add (experimental, disabled by default) security manager support
rmuir commented on a change in pull request #1082: SOLR-13984: add
(experimental, disabled by default) security manager support
URL: https://github.com/apache/lucene-solr/pull/1082#discussion_r360654988
##
File path: solr/bin/solr
##
@@ -2071,6 +2071,17 @@ else
REMOTE_JMX_OPTS=()
fi
+# Enable java security manager (limiting filesystem access and other things)
+# Don't cache DNS lookups forever, set the value back to the original JDK
default.
+if [ "$SOLR_SECURITY_MANAGER_ENABLED" == "true" ]; then
+ SECURITY_MANAGER_OPTS=('-Djava.security.manager' \
+"-Djava.security.policy=${SOLR_SERVER_DIR}/etc/security.policy" \
+'-Dsun.net.inetaddr.ttl=30' \
Review comment:
@shalinmangar i added a commit with a custom `java.security.properties`
file, to override this via the `networkaddress.cache.ttl`, setting it back to
the default of 30 (same as when no security manager is active). It does the
same thing as before, just in a more roundabout way...
If users want to mess with java security properties, they must edit java
security properties files either way: no variables can be supported since they
are not system properties and can't be set with `-D`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13984) Solr should run inside a SecurityManager
[
https://issues.apache.org/jira/browse/SOLR-13984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001727#comment-17001727
]
ASF subversion and git services commented on SOLR-13984:
Commit 420400063479a27bb24878abd2f86d9a50415535 in lucene-solr's branch
refs/heads/jira/SOLR-13984 from Robert Muir
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=4204000 ]
SOLR-13984: use a custom security properties file so nobody freaks out about
sun.net.inetaddr.ttl
> Solr should run inside a SecurityManager
>
>
> Key: SOLR-13984
> URL: https://issues.apache.org/jira/browse/SOLR-13984
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Ishan Chattopadhyaya
>Priority: Major
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> To reduce the effect of attacks, esp. RCE, Solr should run inside a
> SecurityManager.
> Quoting Uwe here:
> {quote}
> The correct way to fix all issues we have seen the last time is very simple:
> LET'S RUN SOLR INSIDE A SECURITY MANAGER IN PRODUCTION (like in tests).
> Elasticsearch is doing this, so please please let's do this instead. But this
> requires to finally get rid of the webapplication and start.jar and add our
> own bootstrapping (like in tests) that configure Jetty and Security Manager
> from our own org.apache.solr.bootstrap.Main.java (or similar).
> {quote}
> https://jira.apache.org/jira/browse/SOLR-12316?focusedCommentId=16465038&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16465038
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13101) Shared storage support in SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-13101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001716#comment-17001716 ] ASF subversion and git services commented on SOLR-13101: Commit 036ce539525aa9c0eb0270d94e4105cd1a0313e9 in lucene-solr's branch refs/heads/jira/SOLR-13101 from Bilal Waheed [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=036ce53 ] SOLR-13101: Concurrency tests for SHARED collection. (#1081) * SOLR-13101: Concurrency tests for SHARED collection. -Concurrency tests for SHARED collection. -On core container shutdown, shutdown the background blob delete manager too. -Use correct casing for SharedCoreStage enum. -Added INDEXING_BATCH_FINISHED to indicate the end of a batch (BLOB_PUSH_FINISHED does not help if push itself runs into error) * -CR feedback. > Shared storage support in SolrCloud > --- > > Key: SOLR-13101 > URL: https://issues.apache.org/jira/browse/SOLR-13101 > Project: Solr > Issue Type: New Feature > Components: SolrCloud >Reporter: Yonik Seeley >Priority: Major > Time Spent: 7h 50m > Remaining Estimate: 0h > > Solr should have first-class support for shared storage (blob/object stores > like S3, google cloud storage, etc. and shared filesystems like HDFS, NFS, > etc). > The key component will likely be a new replica type for shared storage. It > would have many of the benefits of the current "pull" replicas (not indexing > on all replicas, all shards identical with no shards getting out-of-sync, > etc), but would have additional benefits: > - Any shard could become leader (the blob store always has the index) > - Better elasticity scaling down >- durability not linked to number of replcias.. a single replica could be > common for write workloads >- could drop to 0 replicas for a shard when not needed (blob store always > has index) > - Allow for higher performance write workloads by skipping the transaction > log >- don't pay for what you don't need >- a commit will be necessary to flush to stable storage (blob store) > - A lot of the complexity and failure modes go away > An additional component a Directory implementation that will work well with > blob stores. We probably want one that treats local disk as a cache since > the latency to remote storage is so large. I think there are still some > "locking" issues to be solved here (ensuring that more than one writer to the > same index won't corrupt it). This should probably be pulled out into a > different JIRA issue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13101) Shared storage support in SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-13101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001717#comment-17001717 ] ASF subversion and git services commented on SOLR-13101: Commit 036ce539525aa9c0eb0270d94e4105cd1a0313e9 in lucene-solr's branch refs/heads/jira/SOLR-13101 from Bilal Waheed [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=036ce53 ] SOLR-13101: Concurrency tests for SHARED collection. (#1081) * SOLR-13101: Concurrency tests for SHARED collection. -Concurrency tests for SHARED collection. -On core container shutdown, shutdown the background blob delete manager too. -Use correct casing for SharedCoreStage enum. -Added INDEXING_BATCH_FINISHED to indicate the end of a batch (BLOB_PUSH_FINISHED does not help if push itself runs into error) * -CR feedback. > Shared storage support in SolrCloud > --- > > Key: SOLR-13101 > URL: https://issues.apache.org/jira/browse/SOLR-13101 > Project: Solr > Issue Type: New Feature > Components: SolrCloud >Reporter: Yonik Seeley >Priority: Major > Time Spent: 7h 50m > Remaining Estimate: 0h > > Solr should have first-class support for shared storage (blob/object stores > like S3, google cloud storage, etc. and shared filesystems like HDFS, NFS, > etc). > The key component will likely be a new replica type for shared storage. It > would have many of the benefits of the current "pull" replicas (not indexing > on all replicas, all shards identical with no shards getting out-of-sync, > etc), but would have additional benefits: > - Any shard could become leader (the blob store always has the index) > - Better elasticity scaling down >- durability not linked to number of replcias.. a single replica could be > common for write workloads >- could drop to 0 replicas for a shard when not needed (blob store always > has index) > - Allow for higher performance write workloads by skipping the transaction > log >- don't pay for what you don't need >- a commit will be necessary to flush to stable storage (blob store) > - A lot of the complexity and failure modes go away > An additional component a Directory implementation that will work well with > blob stores. We probably want one that treats local disk as a cache since > the latency to remote storage is so large. I think there are still some > "locking" issues to be solved here (ensuring that more than one writer to the > same index won't corrupt it). This should probably be pulled out into a > different JIRA issue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] yonik merged pull request #1081: SOLR-13101: Concurrency tests for SHARED collection.
yonik merged pull request #1081: SOLR-13101: Concurrency tests for SHARED collection. URL: https://github.com/apache/lucene-solr/pull/1081 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14056) Solr admin api's are not authenticated after enabling blockUnknown parameter in jwt authentication.
[
https://issues.apache.org/jira/browse/SOLR-14056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lakhan Gupta updated SOLR-14056:
Description:
Hi,
I am enabling jwt authentication using solr 8.1.1 in standalone mode. below is
my security.json file.
{code:java}
{{ "authentication":{ "blockUnknown": true, "class":"solr.JWTAuthPlugin",
"jwk":{ "kty":"oct", "use":"sig", "kid":"k1",
"k":"8347834radajklajdlkajskdjaskldjkasljdklasjdklaskdjaslkdjklasjdlkasjkldaskldjlkasdasjdlkasjdlkdsl",
"alg":"HS256"}, "aud":"solr"}, "authorization":{
"class":"solr.RuleBasedAuthorizationPlugin", "permissions":[ {
"name":"all", "path":"/*", "role":"admin" } ],
"user-role":{ "solr":"admin" } }}
{code}
as you can see in above code, blockunknown parameter is enabled because of that
my solr admin/info/system api is not authenticated.
I've read in 8.1.1 documentation as blockunknown parameter block unknown
request. so that, my admin/info/system api causing problem. I need an urgent
help!
Really appreciate if someone can give me a quick solution.
Thanks
Laksh Gupta
was:
Hi,
I am enabling jwt authentication using solr 8.1.1 in standalone mode. below is
my security.json file.
{code:java}
{{ "authentication":{ "blockUnknown": true, "class":"solr.JWTAuthPlugin",
"jwk":{ "kty":"oct", "use":"sig", "kid":"k1",
"k":"7A02618BE6943C22FD81CAB9F6FCF063B6E1732C3614BC3ACA6032B6B3215CAF0D28A34FD423423CA3AC34BEA27D3F79",
"alg":"HS256"}, "aud":"solr"}, "authorization":{
"class":"solr.RuleBasedAuthorizationPlugin", "permissions":[ {
"name":"all", "path":"/*", "role":"admin" } ],
"user-role":{ "solr":"admin" } }}
{code}
as you can see in above code, blockunknown parameter is enabled because of that
my solr admin/info/system api is not authenticated.
I've read in 8.1.1 documentation as blockunknown parameter block unknown
request. so that, my admin/info/system api causing problem. I need an urgent
help!
Really appreciate if someone can give me a quick solution.
Thanks
Laksh Gupta
> Solr admin api's are not authenticated after enabling blockUnknown parameter
> in jwt authentication.
> ---
>
> Key: SOLR-14056
> URL: https://issues.apache.org/jira/browse/SOLR-14056
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Authentication, Authorization
>Affects Versions: 8.1.1
> Environment: solr stand
>Reporter: Lakhan Gupta
>Priority: Critical
> Labels: beginner
>
> Hi,
> I am enabling jwt authentication using solr 8.1.1 in standalone mode. below
> is my security.json file.
> {code:java}
> {{ "authentication":{ "blockUnknown": true, "class":"solr.JWTAuthPlugin",
> "jwk":{ "kty":"oct", "use":"sig", "kid":"k1",
> "k":"8347834radajklajdlkajskdjaskldjkasljdklasjdklaskdjaslkdjklasjdlkasjkldaskldjlkasdasjdlkasjdlkdsl",
> "alg":"HS256"}, "aud":"solr"}, "authorization":{
> "class":"solr.RuleBasedAuthorizationPlugin", "permissions":[ {
> "name":"all", "path":"/*", "role":"admin" } ],
> "user-role":{ "solr":"admin" } }}
> {code}
> as you can see in above code, blockunknown parameter is enabled because of
> that my solr admin/info/system api is not authenticated.
> I've read in 8.1.1 documentation as blockunknown parameter block unknown
> request. so that, my admin/info/system api causing problem. I need an urgent
> help!
>
> Really appreciate if someone can give me a quick solution.
>
> Thanks
> Laksh Gupta
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14130) Add postlogs command line tool for indexing Solr logs
[ https://issues.apache.org/jira/browse/SOLR-14130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Bernstein updated SOLR-14130: -- Attachment: (was: Screen Shot 2019-12-19 at 8.48.08 PM.png) > Add postlogs command line tool for indexing Solr logs > - > > Key: SOLR-14130 > URL: https://issues.apache.org/jira/browse/SOLR-14130 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Joel Bernstein >Assignee: Joel Bernstein >Priority: Major > Attachments: SOLR-14130.patch, Screen Shot 2019-12-19 at 2.04.41 > PM.png, Screen Shot 2019-12-19 at 2.16.01 PM.png, Screen Shot 2019-12-19 at > 2.35.41 PM.png, Screen Shot 2019-12-21 at 8.46.51 AM.png > > > This ticket adds a simple command line tool for posting Solr logs to a solr > index. The tool works with the out of the box Solr log format. Still a work > in progress but currently indexes: > * queries > * updates > * commits > * new searchers > * errors - including stack traces > Attached are some sample visualizations using Solr Streaming Expressions and > Math Expressions after the data has been loaded. The visualizations show: > time series, scatter plots, histograms and quantile plots, but really this is > just scratching the surface of the visualizations that can be done with the > Solr logs. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14130) Add postlogs command line tool for indexing Solr logs
[ https://issues.apache.org/jira/browse/SOLR-14130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Bernstein updated SOLR-14130: -- Attachment: Screen Shot 2019-12-21 at 8.46.51 AM.png > Add postlogs command line tool for indexing Solr logs > - > > Key: SOLR-14130 > URL: https://issues.apache.org/jira/browse/SOLR-14130 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Joel Bernstein >Assignee: Joel Bernstein >Priority: Major > Attachments: SOLR-14130.patch, Screen Shot 2019-12-19 at 2.04.41 > PM.png, Screen Shot 2019-12-19 at 2.16.01 PM.png, Screen Shot 2019-12-19 at > 2.35.41 PM.png, Screen Shot 2019-12-19 at 8.48.08 PM.png, Screen Shot > 2019-12-21 at 8.46.51 AM.png > > > This ticket adds a simple command line tool for posting Solr logs to a solr > index. The tool works with the out of the box Solr log format. Still a work > in progress but currently indexes: > * queries > * updates > * commits > * new searchers > * errors - including stack traces > Attached are some sample visualizations using Solr Streaming Expressions and > Math Expressions after the data has been loaded. The visualizations show: > time series, scatter plots, histograms and quantile plots, but really this is > just scratching the surface of the visualizations that can be done with the > Solr logs. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] rmuir commented on a change in pull request #1082: SOLR-13984: add (experimental, disabled by default) security manager support
rmuir commented on a change in pull request #1082: SOLR-13984: add
(experimental, disabled by default) security manager support
URL: https://github.com/apache/lucene-solr/pull/1082#discussion_r360649127
##
File path: solr/bin/solr
##
@@ -2071,6 +2071,17 @@ else
REMOTE_JMX_OPTS=()
fi
+# Enable java security manager (limiting filesystem access and other things)
+# Don't cache DNS lookups forever, set the value back to the original JDK
default.
+if [ "$SOLR_SECURITY_MANAGER_ENABLED" == "true" ]; then
+ SECURITY_MANAGER_OPTS=('-Djava.security.manager' \
+"-Djava.security.policy=${SOLR_SERVER_DIR}/etc/security.policy" \
+'-Dsun.net.inetaddr.ttl=30' \
Review comment:
> Shouldn't this be configurable by the user? Perhaps we keep this default
but make it configurable with a SOLR_NETWORKADDRESS_CACHE_TTL variable.
It is not configurable today: I'm not trying to add that feature here. I am
just preventing dns from cached indefinitely.
> Although most (all?) JVMs still honor the sun.net.inetaddr.ttl but it is
officially deprecated so we should use -Dnetworkaddress.cache.ttl
nope: `networkaddress.cache.ttl` is not a system property at all.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] rmuir commented on a change in pull request #1082: SOLR-13984: add (experimental, disabled by default) security manager support
rmuir commented on a change in pull request #1082: SOLR-13984: add
(experimental, disabled by default) security manager support
URL: https://github.com/apache/lucene-solr/pull/1082#discussion_r360649127
##
File path: solr/bin/solr
##
@@ -2071,6 +2071,17 @@ else
REMOTE_JMX_OPTS=()
fi
+# Enable java security manager (limiting filesystem access and other things)
+# Don't cache DNS lookups forever, set the value back to the original JDK
default.
+if [ "$SOLR_SECURITY_MANAGER_ENABLED" == "true" ]; then
+ SECURITY_MANAGER_OPTS=('-Djava.security.manager' \
+"-Djava.security.policy=${SOLR_SERVER_DIR}/etc/security.policy" \
+'-Dsun.net.inetaddr.ttl=30' \
Review comment:
{quote}
Shouldn't this be configurable by the user? Perhaps we keep this default but
make it configurable with a SOLR_NETWORKADDRESS_CACHE_TTL variable.
{quote}
It is not configurable today: I'm not trying to add that feature here. I am
just preventing dns from cached indefinitely.
{quote}
Although most (all?) JVMs still honor the sun.net.inetaddr.ttl but it is
officially deprecated so we should use -Dnetworkaddress.cache.ttl
{quote}
nope: `networkaddress.cache.ttl` is not a system property at all.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14115) Deprecate zkcli and bin/solr zk support
[ https://issues.apache.org/jira/browse/SOLR-14115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001696#comment-17001696 ] Erick Erickson commented on SOLR-14115: --- Ah, ok. bin/solr doesn't use the HTTP APIs, it uses SolrZkClient. Sounds like we should keep it that way. Oh, actually I did restrict one thing. You can't recursively remove everything, this fails: "bin/solr zk rm -r /". > Deprecate zkcli and bin/solr zk support > --- > > Key: SOLR-14115 > URL: https://issues.apache.org/jira/browse/SOLR-14115 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: scripts and tools >Reporter: Erick Erickson >Priority: Major > > I think it's a valid argument that these have outlived their usefulness and > we should remove them and have APIs to do what Solr requires. Especially if > we can find and point to a third-party visual ZK tool for _changing_ > arbitrary data in ZK. Zookeeper 3.5.5 has the admin server which has a UI > (although I don't see how to change data in ZK with it. Haven't looked very > much). > While we're ripping stuff out of Solr, are these candidates? It would break > my heart to rip ZK support out from bin/solr, but all good things must come > to an end. Why do we maintain three (zkcli, bin/solr and the APIs) ways of > doing the same thing? > Mark put the zkcli stuff in before we had APIs to do what Solr needs to do > with ZK, mainly uploading configsets at the time. I put the zk support in > bin/solr also before the APIs existed because I thought having to learn our > custom wrapper for ZK was yet another orphan bit of code laying around. All > before we had things like the configsets API. > Personally, I'd prefer removing zkcli rather than bin/solr, but that's > because I originated the bin/solr code ;) > This occurred when reading SOLR-14109, I'm not entirely sure what I _really_ > think about it. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-8876) Morphlines fails with "No command builder registered for ..." when using Java 9 due to morphline "importCommands" config option attempting to resolve classname globs
[
https://issues.apache.org/jira/browse/SOLR-8876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001692#comment-17001692
]
Erick Erickson commented on SOLR-8876:
--
See SOLR-9221. This is no longer part of Solr.
> Morphlines fails with "No command builder registered for ..." when using Java
> 9 due to morphline "importCommands" config option attempting to resolve
> classname globs
> -
>
> Key: SOLR-8876
> URL: https://issues.apache.org/jira/browse/SOLR-8876
> Project: Solr
> Issue Type: Bug
> Components: contrib - MapReduce, contrib - morphlines-cell, contrib
> - morphlines-core
>Reporter: Uwe Schindler
>Assignee: Chris M. Hostetter
>Priority: Major
> Labels: Java9
> Fix For: 6.5, 7.0
>
> Attachments: SOLR-8876.patch
>
>
> When running Solr in java9, and using the morphlines contrib(s) users may
> encounter vague errors such as...
> {noformat}
> No command builder registered for COMMAND_NAME
> {noformat}
> This error comes directly from the morphlines code, and relates to the use of
> wildcards in the {{importCommands}} declaration of of morphlines {{\*.conf}}
> files used -- for example...
> {noformat}
> importCommands : ["org.kitesdk.**", "org.apache.solr.**"]
> {noformat}
> Using wildcards like {{\*}} and {{\*\*}} in morphline's {{importCommands}}
> config options do not work in java9 due to changes in the underlying JVM
> classloader.
> This issue is tracked up stream in:
> https://github.com/kite-sdk/kite/issues/469
>
> *WORK AROUND*
> The workaround is to only use fully qualified command class names in
> {{importCommands}} declaration, one for each distinct command used in that
> {{conf}} file.
> Example:
> {noformat}
> # Old config, does not work in java9
> # importCommands : ["org.kitesdk.**", "org.apache.solr.**"]
> # replaced with...
> # using globs (foo.bar.* or foo.bar.**) will not work in Java9 due to
> classpath scanning limitations
> # so we enumarate every command (builder) we know this config uses below.
> (see SOLR-8876)
> importCommands : ["org.kitesdk.morphline.stdlib.LogDebugBuilder",
>
> "org.apache.solr.morphlines.solr.SanitizeUnknownSolrFieldsBuilder",
> "org.apache.solr.morphlines.solr.LoadSolrBuilder"]
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13618) The core admin API should fail to create a core if the instanceDir is not under SOLR_HOME or a child or ancestor of instanceDir contains core.properties
[ https://issues.apache.org/jira/browse/SOLR-13618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001691#comment-17001691 ] Erick Erickson commented on SOLR-13618: --- Vincenzo: Sure, give a whirl. Short form: when the core admin API instanceDir method is called (note, core admin API, _not_ collection API) a couple of things should happen if the instanceDir property is specified: 1> make sure instanceDir is a descendant of SOLR_HOME 2> Insure that no _other_ core.properties files exist above or below it, i.e. in any directory between instanceDir and SOLR_HOME, or any directory below SOLR_HOME. You're probably looking at CoreContainer.create... > The core admin API should fail to create a core if the instanceDir is not > under SOLR_HOME or a child or ancestor of instanceDir contains core.properties > > > Key: SOLR-13618 > URL: https://issues.apache.org/jira/browse/SOLR-13618 > Project: Solr > Issue Type: Bug >Reporter: Talvinder Matharu >Priority: Major > > When new 'instanceDir' is set outside the SOLR_HOME directory then core will > fail to reload after restart as the server only discovers cores within > SOLR_HOME, looking for a 'core.properties'. > So what we ideally want is to check for all the “core.properties” defined > within all 'instanceDir' paths, which may exist outside the SOLR_HOME dir. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-13618) The core admin API should fail to create a core if the instanceDir is not under SOLR_HOME or a child or ancestor of instanceDir contains core.properties
[ https://issues.apache.org/jira/browse/SOLR-13618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Erickson updated SOLR-13618: -- Summary: The core admin API should fail to create a core if the instanceDir is not under SOLR_HOME or a child or ancestor of instanceDir contains core.properties (was: The core admin API should fail if the instanceDir is not under SOLR_HOME or a child or ancestor of instanceDir contains core.properties) > The core admin API should fail to create a core if the instanceDir is not > under SOLR_HOME or a child or ancestor of instanceDir contains core.properties > > > Key: SOLR-13618 > URL: https://issues.apache.org/jira/browse/SOLR-13618 > Project: Solr > Issue Type: Bug >Reporter: Talvinder Matharu >Priority: Major > > When new 'instanceDir' is set outside the SOLR_HOME directory then core will > fail to reload after restart as the server only discovers cores within > SOLR_HOME, looking for a 'core.properties'. > So what we ideally want is to check for all the “core.properties” defined > within all 'instanceDir' paths, which may exist outside the SOLR_HOME dir. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] shalinmangar commented on a change in pull request #1082: SOLR-13984: add (experimental, disabled by default) security manager support
shalinmangar commented on a change in pull request #1082: SOLR-13984: add
(experimental, disabled by default) security manager support
URL: https://github.com/apache/lucene-solr/pull/1082#discussion_r360646424
##
File path: solr/bin/solr
##
@@ -2071,6 +2071,17 @@ else
REMOTE_JMX_OPTS=()
fi
+# Enable java security manager (limiting filesystem access and other things)
+# Don't cache DNS lookups forever, set the value back to the original JDK
default.
+if [ "$SOLR_SECURITY_MANAGER_ENABLED" == "true" ]; then
+ SECURITY_MANAGER_OPTS=('-Djava.security.manager' \
+"-Djava.security.policy=${SOLR_SERVER_DIR}/etc/security.policy" \
+'-Dsun.net.inetaddr.ttl=30' \
Review comment:
Shouldn't this be configurable by the user? Perhaps we keep this default but
make it configurable with a `SOLR_NETWORKADDRESS_CACHE_TTL` variable.
Although most (all?) JVMs still honor the `sun.net.inetaddr.ttl` but it is
officially deprecated so we should use `-Dnetworkaddress.cache.ttl`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14120) Solr Admin UI breaks when using IE 11
[
https://issues.apache.org/jira/browse/SOLR-14120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001686#comment-17001686
]
Lucene/Solr QA commented on SOLR-14120:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} Validate source patterns {color} |
{color:green} 0m 2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:black}{color} | {color:black} {color} | {color:black} 0m 30s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-14120 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12989279/SOLR-14120.patch |
| Optional Tests | validatesourcepatterns |
| uname | Linux lucene1-us-west 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24
10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / c4f68bdab91 |
| ant | version: Apache Ant(TM) version 1.10.5 compiled on March 28 2019 |
| modules | C: solr solr/webapp U: solr |
| Console output |
https://builds.apache.org/job/PreCommit-SOLR-Build/637/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Solr Admin UI breaks when using IE 11
> -
>
> Key: SOLR-14120
> URL: https://issues.apache.org/jira/browse/SOLR-14120
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Admin UI
>Affects Versions: master (9.0), 8.3.1
> Environment: Operation System: Microsoft Windows 10 Enterprise
> (German)
> Version: 10.0.17763 Build 17763
> Browser: Internet Explorer 11
> Version: 11.864.17763
> Update: 11.0.160
>Reporter: Jakob Furrer
>Priority: Minor
> Attachments: SOLR-14120.patch, SOLR-14120.patch, SOLR-14120.patch,
> Solr-8.3.1_ IE11_AdminUI_Error.png
>
>
> A customer of mine is restricted to use no other browser than Internet
> Explorer to access the Solr Administration User Interface.
> He reported that the website does not display any content in the main area
> when any button of the menu on the left hand side is clicked.
> I can replicate that behavior on my machine.
> Symptoms
> -
> When the Admin UI is loaded, the menu on the left hand side is visible but no
> content is ever displayed in the main area.
> Any menu item (e.g. "Dashboard", "logging", "Java Properties" etc.) can be
> clicked but no content appears.
> An error with a stacktrace is shown in the JavaScript console.
> History
> -
> I can confirm, that some month ago the Admin UI worked well under IE.
> However, I have not researched with what update of IE the problems exactly
> started.
> Analysis
> -
> The displayed stacktrace indicates that Internet Explorer does not support
> the method "startsWith" (and subsequently, the same is true for the method
> "includes").
> This behavior is documented:
> https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/startsWith#Browser_compatibility
> https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/includes#Browser_compatibility
> https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/includes#Browser_compatibility
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-13618) The core admin API should fail if the instanceDir is not under SOLR_HOME or a child or ancestor of instanceDir contains core.properties
[ https://issues.apache.org/jira/browse/SOLR-13618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001675#comment-17001675 ] Vincenzo D'Amore edited comment on SOLR-13618 at 12/21/19 11:32 AM: Hi, I have some experience with Solr and would like to contribute to the community. I think being able to fix this. What is not clear, is exactly what to do. I think to double check the instanceDir is within SOLR_HOME when someone calls the core admin API, in case is outside raise an informative error message. On the other hand I didn't get what to do in the other case when a child or ancestor of instanceDir contains core.properties. was (Author: v.dam...@gmail.com): Hi, I have some experience with Solr and would like to contribute to the community. I think being able to fix this. What is not clear, is exactly what to do. I think to double check the instanceDir is within SOLR_HOME, when someone calls the core admin API, in case is outside raise an informative error message. On the other hand I didn't get what to do in the other case when a child or ancestor of instanceDir contains core.properties. > The core admin API should fail if the instanceDir is not under SOLR_HOME or a > child or ancestor of instanceDir contains core.properties > --- > > Key: SOLR-13618 > URL: https://issues.apache.org/jira/browse/SOLR-13618 > Project: Solr > Issue Type: Bug >Reporter: Talvinder Matharu >Priority: Major > > When new 'instanceDir' is set outside the SOLR_HOME directory then core will > fail to reload after restart as the server only discovers cores within > SOLR_HOME, looking for a 'core.properties'. > So what we ideally want is to check for all the “core.properties” defined > within all 'instanceDir' paths, which may exist outside the SOLR_HOME dir. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13618) The core admin API should fail if the instanceDir is not under SOLR_HOME or a child or ancestor of instanceDir contains core.properties
[ https://issues.apache.org/jira/browse/SOLR-13618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001675#comment-17001675 ] Vincenzo D'Amore commented on SOLR-13618: - Hi, I have some experience with Solr and would like to contribute to the community. I think being able to fix this. What is not clear, is exactly what to do. I think to double check the instanceDir is within SOLR_HOME, when someone calls the core admin API, in case is outside raise an informative error message. On the other hand I didn't get what to do in the other case when a child or ancestor of instanceDir contains core.properties. > The core admin API should fail if the instanceDir is not under SOLR_HOME or a > child or ancestor of instanceDir contains core.properties > --- > > Key: SOLR-13618 > URL: https://issues.apache.org/jira/browse/SOLR-13618 > Project: Solr > Issue Type: Bug >Reporter: Talvinder Matharu >Priority: Major > > When new 'instanceDir' is set outside the SOLR_HOME directory then core will > fail to reload after restart as the server only discovers cores within > SOLR_HOME, looking for a 'core.properties'. > So what we ideally want is to check for all the “core.properties” defined > within all 'instanceDir' paths, which may exist outside the SOLR_HOME dir. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] bruno-roustant commented on a change in pull request #1103: LUCENE-9102: add maxQueryLength option to DirectSpellChecker
bruno-roustant commented on a change in pull request #1103: LUCENE-9102: add
maxQueryLength option to DirectSpellChecker
URL: https://github.com/apache/lucene-solr/pull/1103#discussion_r360642971
##
File path:
lucene/suggest/src/java/org/apache/lucene/search/spell/DirectSpellChecker.java
##
@@ -195,9 +197,27 @@ public int getMinQueryLength() {
* metric.
*/
public void setMinQueryLength(int minQueryLength) {
+if (minQueryLength > this.maxQueryLength)
Review comment:
Actually I think we don't need these checks because whatever the value for
minQueryLength and maxQueryLength, the logic is always the same, no surprise.
So personally I would remove these checks, but I'm ok to keep them, your
choice.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-6209) Database connections lost during data import
[ https://issues.apache.org/jira/browse/SOLR-6209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001662#comment-17001662 ] sdhalex commented on SOLR-6209: --- has this bug been fixed in the newest version? Or need we still do the same workaround to avoid this bug now ? Thx. [~sanket.thakkar007] [~arafalov] > Database connections lost during data import > > > Key: SOLR-6209 > URL: https://issues.apache.org/jira/browse/SOLR-6209 > Project: Solr > Issue Type: Bug > Environment: OS: Windows 7 > RAM: 8 GB >Reporter: SANKET >Priority: Major > Labels: solr > Original Estimate: 30h > Remaining Estimate: 30h > > Follow the steps to generate the error: > 1. Configure the large amount of data (around 4 GB or more than 50 millions > of records) > 2. Give proper data-config.xml file for indexing the data from remote > database server. > 3. During indexing the data into solr from SQL SERVER 2010, at the half way > unplug the network cable and see the status in solr. > e.g. > localhost:8083/solr/core1/dataimport?command=status > or > localhost:8083/solr/core1/dataimport > 4. Pass few seconds then again plug back the cable. > 5. You can clearly see that there is just only "Time Elapsed" parameter > increase. "Total Rows Fetched" & "Total Documents Processed" remains same for > infinite time. > 6. You can regenerate this for small data also. > 7. Work around is you need to restart the solr. (But this is not good > solution) > Note: > This is very important issue because, so many organizations not using this > valuable products just because of the this database infinite connection issue. > Solution can be: Forcefully abort the data indexing or provide mechanism for > forcefully abort the indexing. > Hope you guys knows that abort command is also not working. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Issue Comment Deleted] (SOLR-8876) Morphlines fails with "No command builder registered for ..." when using Java 9 due to morphline "importCommands" config option attempting to resolve classna
[
https://issues.apache.org/jira/browse/SOLR-8876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sdhalex updated SOLR-8876:
--
Comment: was deleted
(was: [~sdhalex] gogogo)
> Morphlines fails with "No command builder registered for ..." when using Java
> 9 due to morphline "importCommands" config option attempting to resolve
> classname globs
> -
>
> Key: SOLR-8876
> URL: https://issues.apache.org/jira/browse/SOLR-8876
> Project: Solr
> Issue Type: Bug
> Components: contrib - MapReduce, contrib - morphlines-cell, contrib
> - morphlines-core
>Reporter: Uwe Schindler
>Assignee: Chris M. Hostetter
>Priority: Major
> Labels: Java9
> Fix For: 6.5, 7.0
>
> Attachments: SOLR-8876.patch
>
>
> When running Solr in java9, and using the morphlines contrib(s) users may
> encounter vague errors such as...
> {noformat}
> No command builder registered for COMMAND_NAME
> {noformat}
> This error comes directly from the morphlines code, and relates to the use of
> wildcards in the {{importCommands}} declaration of of morphlines {{\*.conf}}
> files used -- for example...
> {noformat}
> importCommands : ["org.kitesdk.**", "org.apache.solr.**"]
> {noformat}
> Using wildcards like {{\*}} and {{\*\*}} in morphline's {{importCommands}}
> config options do not work in java9 due to changes in the underlying JVM
> classloader.
> This issue is tracked up stream in:
> https://github.com/kite-sdk/kite/issues/469
>
> *WORK AROUND*
> The workaround is to only use fully qualified command class names in
> {{importCommands}} declaration, one for each distinct command used in that
> {{conf}} file.
> Example:
> {noformat}
> # Old config, does not work in java9
> # importCommands : ["org.kitesdk.**", "org.apache.solr.**"]
> # replaced with...
> # using globs (foo.bar.* or foo.bar.**) will not work in Java9 due to
> classpath scanning limitations
> # so we enumarate every command (builder) we know this config uses below.
> (see SOLR-8876)
> importCommands : ["org.kitesdk.morphline.stdlib.LogDebugBuilder",
>
> "org.apache.solr.morphlines.solr.SanitizeUnknownSolrFieldsBuilder",
> "org.apache.solr.morphlines.solr.LoadSolrBuilder"]
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-8876) Morphlines fails with "No command builder registered for ..." when using Java 9 due to morphline "importCommands" config option attempting to resolve classname globs
[
https://issues.apache.org/jira/browse/SOLR-8876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001655#comment-17001655
]
sdhalex commented on SOLR-8876:
---
[~sdhalex] gogogo
> Morphlines fails with "No command builder registered for ..." when using Java
> 9 due to morphline "importCommands" config option attempting to resolve
> classname globs
> -
>
> Key: SOLR-8876
> URL: https://issues.apache.org/jira/browse/SOLR-8876
> Project: Solr
> Issue Type: Bug
> Components: contrib - MapReduce, contrib - morphlines-cell, contrib
> - morphlines-core
>Reporter: Uwe Schindler
>Assignee: Chris M. Hostetter
>Priority: Major
> Labels: Java9
> Fix For: 6.5, 7.0
>
> Attachments: SOLR-8876.patch
>
>
> When running Solr in java9, and using the morphlines contrib(s) users may
> encounter vague errors such as...
> {noformat}
> No command builder registered for COMMAND_NAME
> {noformat}
> This error comes directly from the morphlines code, and relates to the use of
> wildcards in the {{importCommands}} declaration of of morphlines {{\*.conf}}
> files used -- for example...
> {noformat}
> importCommands : ["org.kitesdk.**", "org.apache.solr.**"]
> {noformat}
> Using wildcards like {{\*}} and {{\*\*}} in morphline's {{importCommands}}
> config options do not work in java9 due to changes in the underlying JVM
> classloader.
> This issue is tracked up stream in:
> https://github.com/kite-sdk/kite/issues/469
>
> *WORK AROUND*
> The workaround is to only use fully qualified command class names in
> {{importCommands}} declaration, one for each distinct command used in that
> {{conf}} file.
> Example:
> {noformat}
> # Old config, does not work in java9
> # importCommands : ["org.kitesdk.**", "org.apache.solr.**"]
> # replaced with...
> # using globs (foo.bar.* or foo.bar.**) will not work in Java9 due to
> classpath scanning limitations
> # so we enumarate every command (builder) we know this config uses below.
> (see SOLR-8876)
> importCommands : ["org.kitesdk.morphline.stdlib.LogDebugBuilder",
>
> "org.apache.solr.morphlines.solr.SanitizeUnknownSolrFieldsBuilder",
> "org.apache.solr.morphlines.solr.LoadSolrBuilder"]
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-8876) Morphlines fails with "No command builder registered for ..." when using Java 9 due to morphline "importCommands" config option attempting to resolve classname globs
[
https://issues.apache.org/jira/browse/SOLR-8876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001654#comment-17001654
]
sdhalex commented on SOLR-8876:
---
Excuse me, has this bug been fixed in the newest version? Or need we still do
the same workaround to avoid this bug now ? Thx. [~hossman][~uschindler]
> Morphlines fails with "No command builder registered for ..." when using Java
> 9 due to morphline "importCommands" config option attempting to resolve
> classname globs
> -
>
> Key: SOLR-8876
> URL: https://issues.apache.org/jira/browse/SOLR-8876
> Project: Solr
> Issue Type: Bug
> Components: contrib - MapReduce, contrib - morphlines-cell, contrib
> - morphlines-core
>Reporter: Uwe Schindler
>Assignee: Chris M. Hostetter
>Priority: Major
> Labels: Java9
> Fix For: 6.5, 7.0
>
> Attachments: SOLR-8876.patch
>
>
> When running Solr in java9, and using the morphlines contrib(s) users may
> encounter vague errors such as...
> {noformat}
> No command builder registered for COMMAND_NAME
> {noformat}
> This error comes directly from the morphlines code, and relates to the use of
> wildcards in the {{importCommands}} declaration of of morphlines {{\*.conf}}
> files used -- for example...
> {noformat}
> importCommands : ["org.kitesdk.**", "org.apache.solr.**"]
> {noformat}
> Using wildcards like {{\*}} and {{\*\*}} in morphline's {{importCommands}}
> config options do not work in java9 due to changes in the underlying JVM
> classloader.
> This issue is tracked up stream in:
> https://github.com/kite-sdk/kite/issues/469
>
> *WORK AROUND*
> The workaround is to only use fully qualified command class names in
> {{importCommands}} declaration, one for each distinct command used in that
> {{conf}} file.
> Example:
> {noformat}
> # Old config, does not work in java9
> # importCommands : ["org.kitesdk.**", "org.apache.solr.**"]
> # replaced with...
> # using globs (foo.bar.* or foo.bar.**) will not work in Java9 due to
> classpath scanning limitations
> # so we enumarate every command (builder) we know this config uses below.
> (see SOLR-8876)
> importCommands : ["org.kitesdk.morphline.stdlib.LogDebugBuilder",
>
> "org.apache.solr.morphlines.solr.SanitizeUnknownSolrFieldsBuilder",
> "org.apache.solr.morphlines.solr.LoadSolrBuilder"]
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14095) Remove serialization and/or support serialization filtering
[ https://issues.apache.org/jira/browse/SOLR-14095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001635#comment-17001635 ] Ishan Chattopadhyaya commented on SOLR-14095: - bq. Noble Paul, Ishan Chattopadhyaya, I took another look today. I still feel there is no way to use Json here unless we build a full serialization that retains types correctly. I feel Javabin here is less risky compared to Json, so I want to take that route and I want to do it soon to unblock the security manager work. +1. Thanks Tomas. > Remove serialization and/or support serialization filtering > --- > > Key: SOLR-14095 > URL: https://issues.apache.org/jira/browse/SOLR-14095 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Robert Muir >Priority: Major > Attachments: SOLR-14095-json.patch, json-nl.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Removing the use of serialization is greatly preferred. > But if serialization over the wire must really happen, then we must use JDK's > serialization filtering capability to prevent havoc. > https://docs.oracle.com/javase/10/core/serialization-filtering1.htm#JSCOR-GUID-3ECB288D-E5BD-4412-892F-E9BB11D4C98A -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
