[jira] [Updated] (SOLR-11084) Issue with starting script with solr.home (-s) == solr
[
https://issues.apache.org/jira/browse/SOLR-11084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-11084:
Attachment: SOLR-11084.patch
Does this looks good. I have extended Neil's patch. Added the help text.
{code}
-s Sets the solr.solr.home system property; Solr will create core
directories under
this directory. This allows you to run multiple Solr
instances on the same host
while reusing the same server directory set using the -d
parameter. If set, the
specified directory should contain a solr.xml file, unless
solr.xml exists in Zookeeper.
This parameter is ignored when running examples (-e), as the
solr.solr.home depends
on which example is run. The default value is server/solr. If
passed relative dir,
validation with current dir will be done, before trying
default server/
{code}
> Issue with starting script with solr.home (-s) == solr
> --
>
> Key: SOLR-11084
> URL: https://issues.apache.org/jira/browse/SOLR-11084
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 6.6
> Environment: OS X 10.12
>Reporter: Neil Ireson
> Attachments: SOLR-11084.patch, SOLR-11084.patch
>
>
> I've just hit an issue when starting solr using the script. All works well
> when I use:
> /solr-6.6.0/bin/solr start -p 9090 -m 4g -s data/solr
> However if I cd into the data directory and try:
> /solr-6.6.0/bin/solr start -p 9090 -m 4g -s solr
> Then I get no cores loaded and the "solr.solr.home" shown in the UI as my
> installation solr (i.e. /solr-6.6.0/server/solr)
> I thought that "solr" may be a reserved word so I changed the name of my
> folder to "test-solr" and this worked.
> I'm unsure if this is an undocumented feature (at least I couldn't find any
> reference to it) or a bug.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11084) Issue with starting script with solr.home (-s) == solr
[
https://issues.apache.org/jira/browse/SOLR-11084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16093127#comment-16093127
]
Amrit Sarkar commented on SOLR-11084:
-
[~neilireson]
Regarding documenting {{-s solr}} in the docs. If the patch goes through, I
don't a strong reason to put this bit of information in official docs. Your
thoughts? [~erickerickson].
> Issue with starting script with solr.home (-s) == solr
> --
>
> Key: SOLR-11084
> URL: https://issues.apache.org/jira/browse/SOLR-11084
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 6.6
> Environment: OS X 10.12
>Reporter: Neil Ireson
> Attachments: SOLR-11084.patch
>
>
> I've just hit an issue when starting solr using the script. All works well
> when I use:
> /solr-6.6.0/bin/solr start -p 9090 -m 4g -s data/solr
> However if I cd into the data directory and try:
> /solr-6.6.0/bin/solr start -p 9090 -m 4g -s solr
> Then I get no cores loaded and the "solr.solr.home" shown in the UI as my
> installation solr (i.e. /solr-6.6.0/server/solr)
> I thought that "solr" may be a reserved word so I changed the name of my
> folder to "test-solr" and this worked.
> I'm unsure if this is an undocumented feature (at least I couldn't find any
> reference to it) or a bug.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-11084) Issue with starting script with solr.home (-s) == solr
[
https://issues.apache.org/jira/browse/SOLR-11084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091201#comment-16091201
]
Amrit Sarkar edited comment on SOLR-11084 at 7/18/17 6:55 AM:
--
[~neilireson],
Absolutely makes much more sense. Before falling back to default
{{SOLR_SERVER_DIR}} it should verify with the current dir, {{pwd}}. I will cook
up a patch with above changes and upload in while, feel free to do it anyway.
was (Author: sarkaramr...@gmail.com):
[~neilireson],
Absolutely makes much more sense. Before falling back to default
{{SOLR_SERVER_DIR}} it should verify with the current dir, {{pwd}}. I will cook
up a patch with above changes and upload in short while.
> Issue with starting script with solr.home (-s) == solr
> --
>
> Key: SOLR-11084
> URL: https://issues.apache.org/jira/browse/SOLR-11084
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 6.6
> Environment: OS X 10.12
>Reporter: Neil Ireson
>
> I've just hit an issue when starting solr using the script. All works well
> when I use:
> /solr-6.6.0/bin/solr start -p 9090 -m 4g -s data/solr
> However if I cd into the data directory and try:
> /solr-6.6.0/bin/solr start -p 9090 -m 4g -s solr
> Then I get no cores loaded and the "solr.solr.home" shown in the UI as my
> installation solr (i.e. /solr-6.6.0/server/solr)
> I thought that "solr" may be a reserved word so I changed the name of my
> folder to "test-solr" and this worked.
> I'm unsure if this is an undocumented feature (at least I couldn't find any
> reference to it) or a bug.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11084) Issue with starting script with solr.home (-s) == solr
[
https://issues.apache.org/jira/browse/SOLR-11084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091201#comment-16091201
]
Amrit Sarkar commented on SOLR-11084:
-
[~neilireson],
Absolutely makes much more sense. Before falling back to default
{{SOLR_SERVER_DIR}} it should verify with the current dir, {{pwd}}. I will cook
up a patch with above changes and upload in short while.
> Issue with starting script with solr.home (-s) == solr
> --
>
> Key: SOLR-11084
> URL: https://issues.apache.org/jira/browse/SOLR-11084
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 6.6
> Environment: OS X 10.12
>Reporter: Neil Ireson
>
> I've just hit an issue when starting solr using the script. All works well
> when I use:
> /solr-6.6.0/bin/solr start -p 9090 -m 4g -s data/solr
> However if I cd into the data directory and try:
> /solr-6.6.0/bin/solr start -p 9090 -m 4g -s solr
> Then I get no cores loaded and the "solr.solr.home" shown in the UI as my
> installation solr (i.e. /solr-6.6.0/server/solr)
> I thought that "solr" may be a reserved word so I changed the name of my
> folder to "test-solr" and this worked.
> I'm unsure if this is an undocumented feature (at least I couldn't find any
> reference to it) or a bug.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11084) Issue with starting script with solr.home (-s) == solr
[
https://issues.apache.org/jira/browse/SOLR-11084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091141#comment-16091141
]
Amrit Sarkar commented on SOLR-11084:
-
[~ctargett], positively. I will see how to make the patch with {{adoc}} and
post soon.
> Issue with starting script with solr.home (-s) == solr
> --
>
> Key: SOLR-11084
> URL: https://issues.apache.org/jira/browse/SOLR-11084
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 6.6
> Environment: OS X 10.12
>Reporter: Neil Ireson
>
> I've just hit an issue when starting solr using the script. All works well
> when I use:
> /solr-6.6.0/bin/solr start -p 9090 -m 4g -s data/solr
> However if I cd into the data directory and try:
> /solr-6.6.0/bin/solr start -p 9090 -m 4g -s solr
> Then I get no cores loaded and the "solr.solr.home" shown in the UI as my
> installation solr (i.e. /solr-6.6.0/server/solr)
> I thought that "solr" may be a reserved word so I changed the name of my
> folder to "test-solr" and this worked.
> I'm unsure if this is an undocumented feature (at least I couldn't find any
> reference to it) or a bug.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11084) Issue with starting script with solr.home (-s) == solr
[
https://issues.apache.org/jira/browse/SOLR-11084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16090458#comment-16090458
]
Amrit Sarkar commented on SOLR-11084:
-
Neil,
This is undocumented actually, and we should comment on the docs for this to
get included. Here's what's happening ::
solr.sh (solr script) :: line no:: 1334 (master branch)
{code}
if [[ $SOLR_HOME != /* ]] && [[ -d "$SOLR_SERVER_DIR/$SOLR_HOME" ]]; then
SOLR_HOME="$SOLR_SERVER_DIR/$SOLR_HOME"
SOLR_PID_DIR="$SOLR_HOME"
{code}
default value of {{SOLR_SERVER_DIR}} is {SOLR_DIR/server/}}
and when we pass: {{SOLR_HOME}} as {{solr}}; it makes a perfect directory
{{SOLR_DIR/server/solr}} which is legit and it gets accepted.
> Issue with starting script with solr.home (-s) == solr
> --
>
> Key: SOLR-11084
> URL: https://issues.apache.org/jira/browse/SOLR-11084
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 6.6
> Environment: OS X 10.12
>Reporter: Neil Ireson
>
> I've just hit an issue when starting solr using the script. All works well
> when I use:
> /solr-6.6.0/bin/solr start -p 9090 -m 4g -s data/solr
> However if I cd into the data directory and try:
> /solr-6.6.0/bin/solr start -p 9090 -m 4g -s solr
> Then I get no cores loaded and the "solr.solr.home" shown in the UI as my
> installation solr (i.e. /solr-6.6.0/server/solr)
> I thought that "solr" may be a reserved word so I changed the name of my
> folder to "test-solr" and this worked.
> I'm unsure if this is an undocumented feature (at least I couldn't find any
> reference to it) or a bug.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10679) Solr CDCR cannot be configured to use Aliases for replication
[
https://issues.apache.org/jira/browse/SOLR-10679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16089706#comment-16089706
]
Amrit Sarkar commented on SOLR-10679:
-
[~WebHomer],
Even aliases on {{target}} cluster is not working, I looked into the code and
may have understood why it isn't happening. It is very similar to other cases
where the {{state.json}} for a collection cannot be retrieved from ZK as we are
referring to alias. I am not sure how much effort is required to make it work.
> Solr CDCR cannot be configured to use Aliases for replication
> -
>
> Key: SOLR-10679
> URL: https://issues.apache.org/jira/browse/SOLR-10679
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Affects Versions: 6.2
>Reporter: Webster Homer
>
> My company uses Solr aliases to limit the configuration changes that we need
> to support.
> The CDCR configuration seems to accept an alias for either the source or
> target collections, and no errors show up in the log, but no data is
> replicated if the source or target is an alias and not an actual collection.
> I see that aliases are not even mentioned in the CDCR documentation. It seems
> to me this should either work or throw an error.
> It should be documented one way or another
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8953) CDCR leader detector uses core name instead of core node name
[
https://issues.apache.org/jira/browse/SOLR-8953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16089587#comment-16089587
]
Amrit Sarkar commented on SOLR-8953:
Just skimming,
{code}
CdcrLeaderStateManager.this.setAmILeader(props.get("core").equals(core.getName()));
{code}
The core name is in format: {{collection_shardX_replicaY}}, {{core node name}}
is saved in ZK state info for that particular collection but SolrCore doesn't
have any node name field or a mapping. Not sure what to replace
{{core.getName())}} to equate if we utilise the core node name.
> CDCR leader detector uses core name instead of core node name
> -
>
> Key: SOLR-8953
> URL: https://issues.apache.org/jira/browse/SOLR-8953
> Project: Solr
> Issue Type: Bug
> Components: CDCR, SolrCloud
>Affects Versions: 6.0
>Reporter: Shalin Shekhar Mangar
> Fix For: 6.2, 7.0
>
>
> CdcrLeaderStateManager should use core node name instead of core name to
> determine if current node is the leader otherwise when all replicas have the
> same name (e.g. "collection1") then it will cause cdcr processes to start on
> every replica.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10679) Solr CDCR cannot be configured to use Aliases for replication
[
https://issues.apache.org/jira/browse/SOLR-10679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16089550#comment-16089550
]
Amrit Sarkar commented on SOLR-10679:
-
[~WebHomer],
Internally {{CdcrReplicator}} forwards the request to target collection in
target cluster via conventional UpdateRequest, which supports alas. Alias
should definitely work for {{target}} collections in CDCR, need to confirm the
same for {{source}}. Check in the logs once, {{Forwarded X updates to target
collection Y}} in source and {{/update}} in target.
> Solr CDCR cannot be configured to use Aliases for replication
> -
>
> Key: SOLR-10679
> URL: https://issues.apache.org/jira/browse/SOLR-10679
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Affects Versions: 6.2
>Reporter: Webster Homer
>
> My company uses Solr aliases to limit the configuration changes that we need
> to support.
> The CDCR configuration seems to accept an alias for either the source or
> target collections, and no errors show up in the log, but no data is
> replicated if the source or target is an alias and not an actual collection.
> I see that aliases are not even mentioned in the CDCR documentation. It seems
> to me this should either work or throw an error.
> It should be documented one way or another
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9394) CDCR: Exception on target site while using deleteById to delete a document
[ https://issues.apache.org/jira/browse/SOLR-9394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16089541#comment-16089541 ] Amrit Sarkar commented on SOLR-9394: I am not able to replicate the issue on my end. Can you provide sample doc for the same. > CDCR: Exception on target site while using deleteById to delete a document > -- > > Key: SOLR-9394 > URL: https://issues.apache.org/jira/browse/SOLR-9394 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Affects Versions: 6.1 >Reporter: BHARATH K VENKATARAMANA >Priority: Critical > > Deleting a document on the main site by using deleteById solrj method is > causing the below exception on the target site, even though the document is > deleted correctly on the main site. But if we use deleteByQuery, it works > fine. In the solr schema.xml the unique key is the "id" field and we have it > as long, if we change that to string and then deleteById works. > Error stacktrace on the target site SOLR node leader:- > 2016-08-06 08:09:21.091 ERROR (qtp472654579-2699) [c:collection s:shard1 > r:core_node3 x:collection] o.a.s.h.RequestHandlerBase > org.apache.solr.common.SolrException: Invalid Number: ^A^@^@^@^@^@^L^K0W > at > org.apache.solr.schema.TrieField.readableToIndexed(TrieField.java:537) > at > org.apache.solr.update.DeleteUpdateCommand.getIndexedId(DeleteUpdateCommand.java:65) > at > org.apache.solr.update.processor.DistributedUpdateProcessor.versionDelete(DistributedUpdateProcessor.java:1495) > at > org.apache.solr.update.processor.CdcrUpdateProcessor.versionDelete(CdcrUpdateProcessor.java:85) > at > org.apache.solr.update.processor.DistributedUpdateProcessor.processDelete(DistributedUpdateProcessor.java:1154) > at > org.apache.solr.handler.loader.JavabinLoader.delete(JavabinLoader.java:151) > at > org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDocs(JavabinLoader.java:112) > at > org.apache.solr.handler.loader.JavabinLoader.load(JavabinLoader.java:54) > at > org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:97) > at > org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:69) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2036) > at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:657) > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:464) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160) > at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) > at > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213) > at > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) > at org.eclipse.jetty.server.Server.handle(Server.java:518) > at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308) > at > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244) > at > org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) > at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) > at > org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) > at > org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246) >
[jira] [Comment Edited] (SOLR-9483) Add SolrJ support for the modify collection API
[
https://issues.apache.org/jira/browse/SOLR-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088934#comment-16088934
]
Amrit Sarkar edited comment on SOLR-9483 at 7/16/17 2:04 PM:
-
I extended and improved little the pulled request changes by
[~susheel2...@gmail.com] and put it in a patch like we are suppose to:
{code}
modified:
solrj/src/java/org/apache/solr/client/solrj/request/CollectionAdminRequest.java
modified:
solrj/src/test/org/apache/solr/client/solrj/CollectionAdminRequestRequiredParamsTest.java
{code}
Patch uploaded.
was (Author: sarkaramr...@gmail.com):
I extended and improved a bit the pulled request changes by
[~susheel2...@gmail.com] and put it in a patch like we are suppose to:
{code}
modified:
solrj/src/java/org/apache/solr/client/solrj/request/CollectionAdminRequest.java
modified:
solrj/src/test/org/apache/solr/client/solrj/CollectionAdminRequestRequiredParamsTest.java
{code}
Patch uploaded.
> Add SolrJ support for the modify collection API
> ---
>
> Key: SOLR-9483
> URL: https://issues.apache.org/jira/browse/SOLR-9483
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: clients - java, SolrCloud, SolrJ
>Reporter: Shalin Shekhar Mangar
> Labels: difficulty-easy, newdev
> Fix For: 7.0, 6.7
>
> Attachments: SOLR-9483.patch
>
>
> SolrJ currently does not have a method corresponding to the modify collection
> API. There should be a Modify class inside CollectionAdminRequest and a
> simple method to change all parameters supported by the modify API.
> Link to modify API documentation:
> https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-modifycoll
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-9483) Add SolrJ support for the modify collection API
[ https://issues.apache.org/jira/browse/SOLR-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amrit Sarkar updated SOLR-9483: --- Attachment: SOLR-9483.patch > Add SolrJ support for the modify collection API > --- > > Key: SOLR-9483 > URL: https://issues.apache.org/jira/browse/SOLR-9483 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: clients - java, SolrCloud, SolrJ >Reporter: Shalin Shekhar Mangar > Labels: difficulty-easy, newdev > Fix For: 7.0, 6.7 > > Attachments: SOLR-9483.patch > > > SolrJ currently does not have a method corresponding to the modify collection > API. There should be a Modify class inside CollectionAdminRequest and a > simple method to change all parameters supported by the modify API. > Link to modify API documentation: > https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-modifycoll -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9483) Add SolrJ support for the modify collection API
[
https://issues.apache.org/jira/browse/SOLR-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088934#comment-16088934
]
Amrit Sarkar commented on SOLR-9483:
I extended and improved a bit the pulled request changes by
[~susheel2...@gmail.com] and put it in a patch like we are suppose to:
{code}
modified:
solrj/src/java/org/apache/solr/client/solrj/request/CollectionAdminRequest.java
modified:
solrj/src/test/org/apache/solr/client/solrj/CollectionAdminRequestRequiredParamsTest.java
{code}
Patch uploaded.
> Add SolrJ support for the modify collection API
> ---
>
> Key: SOLR-9483
> URL: https://issues.apache.org/jira/browse/SOLR-9483
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: clients - java, SolrCloud, SolrJ
>Reporter: Shalin Shekhar Mangar
> Labels: difficulty-easy, newdev
> Fix For: 7.0, 6.7
>
>
> SolrJ currently does not have a method corresponding to the modify collection
> API. There should be a Modify class inside CollectionAdminRequest and a
> simple method to change all parameters supported by the modify API.
> Link to modify API documentation:
> https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-modifycoll
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9272) Auto resolve zkHost for bin/solr zk for running Solr
[
https://issues.apache.org/jira/browse/SOLR-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088228#comment-16088228
]
Amrit Sarkar commented on SOLR-9272:
[~janhoy],
{{StatusTool}} implements abstract method {{runImpl}} of SolrCli.ToolBase of
return type {{void}}. In order to fetch {{cloud/Zookeeper}} we have to change
the base abstract method return type and corresponding all its classes which
implements it.
StatusTool and all other classes which extends ToolBase, {{echo}} the output
and returns {{void}}. I would like to drive this to conclusion; hence seeking
suggestion whether it is a good idea to change base abstract method's return
type or it is too much change for too little convenience.
> Auto resolve zkHost for bin/solr zk for running Solr
>
>
> Key: SOLR-9272
> URL: https://issues.apache.org/jira/browse/SOLR-9272
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: scripts and tools
>Affects Versions: 6.2
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
> Labels: newdev
>
> Spinoff from SOLR-9194:
> We can skip requiring {{-z}} for {{bin/solr zk}} for a Solr that is already
> running. We can optionally accept the {{-p}} parameter instead, and with that
> use StatusTool to fetch the {{cloud/ZooKeeper}} property from there. It's
> easier to remember solr port than zk string.
> Example:
> {noformat}
> bin/solr start -c -p 9090
> bin/solr zk ls / -p 9090
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11069) LASTPROCESSEDVERSION for CDCR is flawed when buffering is enabled
[
https://issues.apache.org/jira/browse/SOLR-11069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088095#comment-16088095
]
Amrit Sarkar commented on SOLR-11069:
-
{quote}
I just saw a case where restarting the source cluster triggered bootstrap.
{quote}
When leader of shard of source collection goes down and non-leader is selected,
it triggers {{bootstrap}} due to above stated reason, LPV set to {{-1}}.
> LASTPROCESSEDVERSION for CDCR is flawed when buffering is enabled
> -
>
> Key: SOLR-11069
> URL: https://issues.apache.org/jira/browse/SOLR-11069
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Affects Versions: 7.0
>Reporter: Amrit Sarkar
>
> {{LASTPROCESSEDVERSION}} (a.b.v. LPV) action for CDCR breaks down due to
> poorly initialised and maintained buffer log for either source or target
> cluster core nodes.
> If buffer is enabled for cores of either source or target cluster, it return
> {{-1}}, *irrespective of number of entries in tlog read by the {{leader}}*
> node of each shard of respective collection of respective cluster. Once
> disabled, it starts telling us the correct LPV for each core.
> Due to the same flawed behavior, Update Log Synchroniser may doesn't work
> properly as expected, i.e. provides correct seek to the {{non-leader}} nodes
> to advance at. I am not sure whether this is an intended behavior for sync
> but it surely doesn't feel right.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-9483) Add SolrJ support for the modify collection API
[ https://issues.apache.org/jira/browse/SOLR-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087831#comment-16087831 ] Amrit Sarkar edited comment on SOLR-9483 at 7/14/17 7:15 PM: - [~shalinmangar] [~ichattopadhyaya] [~elyograg], Anyone working on this? If not, I can stretch this to conclusion. was (Author: sarkaramr...@gmail.com): [~shalinmangar] [~ichattopadhyaya] [~elyograg], Anyone working on this? If not, I can stretch this to conclusion. - Amrit Sarkar > Add SolrJ support for the modify collection API > --- > > Key: SOLR-9483 > URL: https://issues.apache.org/jira/browse/SOLR-9483 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: clients - java, SolrCloud, SolrJ >Reporter: Shalin Shekhar Mangar > Labels: difficulty-easy, newdev > Fix For: 7.0, 6.7 > > > SolrJ currently does not have a method corresponding to the modify collection > API. There should be a Modify class inside CollectionAdminRequest and a > simple method to change all parameters supported by the modify API. > Link to modify API documentation: > https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-modifycoll -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9483) Add SolrJ support for the modify collection API
[ https://issues.apache.org/jira/browse/SOLR-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087831#comment-16087831 ] Amrit Sarkar commented on SOLR-9483: [~shalinmangar] [~ichattopadhyaya] [~elyograg], Anyone working on this? If not, I can stretch this to conclusion. - Amrit Sarkar > Add SolrJ support for the modify collection API > --- > > Key: SOLR-9483 > URL: https://issues.apache.org/jira/browse/SOLR-9483 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: clients - java, SolrCloud, SolrJ >Reporter: Shalin Shekhar Mangar > Labels: difficulty-easy, newdev > Fix For: 7.0, 6.7 > > > SolrJ currently does not have a method corresponding to the modify collection > API. There should be a Modify class inside CollectionAdminRequest and a > simple method to change all parameters supported by the modify API. > Link to modify API documentation: > https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-modifycoll -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11069) LASTPROCESSEDVERSION for CDCR is flawed when buffering is enabled
[
https://issues.apache.org/jira/browse/SOLR-11069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087665#comment-16087665
]
Amrit Sarkar commented on SOLR-11069:
-
Continuing with how LPV was never tested robust:
The only bit where the LPV mentioned in the tests is in
{{CdcrRequestHandlerTest}}
{code}
// replication never started, lastProcessedVersion should be -1 for both
shards
rsp =
invokeCdcrAction(shardToLeaderJetty.get(SOURCE_COLLECTION).get(SHARD1),
CdcrParams.CdcrAction.LASTPROCESSEDVERSION);
long lastVersion = (Long) rsp.get(CdcrParams.LAST_PROCESSED_VERSION);
assertEquals(-1l, lastVersion);
rsp =
invokeCdcrAction(shardToLeaderJetty.get(SOURCE_COLLECTION).get(SHARD2),
CdcrParams.CdcrAction.LASTPROCESSEDVERSION);
lastVersion = (Long) rsp.get(CdcrParams.LAST_PROCESSED_VERSION);
assertEquals(-1l, lastVersion);
{code}
LPV > -1 or what LPV value (which should > 1 atleast) can be when leader reads
some entries from tlogs is never tested anywhere or at least I cannot find it.
> LASTPROCESSEDVERSION for CDCR is flawed when buffering is enabled
> -
>
> Key: SOLR-11069
> URL: https://issues.apache.org/jira/browse/SOLR-11069
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Affects Versions: 7.0
>Reporter: Amrit Sarkar
>
> {{LASTPROCESSEDVERSION}} (a.b.v. LPV) action for CDCR breaks down due to
> poorly initialised and maintained buffer log for either source or target
> cluster core nodes.
> If buffer is enabled for cores of either source or target cluster, it return
> {{-1}}, *irrespective of number of entries in tlog read by the {{leader}}*
> node of each shard of respective collection of respective cluster. Once
> disabled, it starts telling us the correct LPV for each core.
> Due to the same flawed behavior, Update Log Synchroniser may doesn't work
> properly as expected, i.e. provides correct seek to the {{non-leader}} nodes
> to advance at. I am not sure whether this is an intended behavior for sync
> but it surely doesn't feel right.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-11069) LASTPROCESSEDVERSION for CDCR is flawed when buffering is enabled
[
https://issues.apache.org/jira/browse/SOLR-11069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087100#comment-16087100
]
Amrit Sarkar edited comment on SOLR-11069 at 7/14/17 2:55 PM:
--
Regarding {{updateLogSynchronizer}} ::
Everytime we call {{DISABLEBUFFER}} or {{ENABLEBUFFER}},
CdcrBufferManager::stateUpdate gets invoked::
{code}
@Override
public synchronized void stateUpdate() {
CdcrUpdateLog ulog = (CdcrUpdateLog) core.getUpdateHandler().getUpdateLog();
// If I am not the leader, I should always buffer my updates
if (!leaderStateManager.amILeader()) {
ulog.enableBuffer();
return;
}
// If I am the leader, I should buffer my updates only if buffer is enabled
else if
(bufferStateManager.getState().equals(CdcrParams.BufferState.ENABLED)) {
ulog.enableBuffer();
return;
}
// otherwise, disable the buffer
ulog.disableBuffer();
}
{code}
The non-leader nodes are by-defaulted are always buffer enabled ::
{code}
if (!leaderStateManager.amILeader()) {
ulog.enableBuffer();
return;
}
{code}
though LPV always calculated on leader but it has serious drawbacks explained
later:
in CdcrUpdateLogSynchronizer:: run :: if buffering is {{enabled}} ::
{code}
// if we received -1, it means that the log reader on the leader has not yet
started to read log entries
// do nothing
if (lastVersion == -1) {
return;
}
try {
CdcrUpdateLog ulog = (CdcrUpdateLog)
core.getUpdateHandler().getUpdateLog();
if (ulog.isBuffering()) {
log.debug("Advancing replica buffering tlog reader to {} @ {}:{}",
lastVersion, collection, shardId);
ulog.getBufferToggle().seek(lastVersion);
}
}
{code}
It always returns on {{lastVersion == -1}} and look at the comment {{if we
received -1, it means that the log reader on the leader has not yet started to
read log entries}}, that's misleading.
As the {{lastVersion}} is not +ve, the seek for the corresponding non-leader
nodes are never set to appropriate LPV.
Now if the leader goes down, and some non-leader becomes the leader himself,
the LPV is not set properly resulting in improper sync and I have no idea how
the impact will be in that case.
Also, as for non-leader nodes buffer is always on, if in the future it becomes
the leader itself, even if we have disabled buffer for the source collection
cluster, the status and its action will be {{buffer enabled}}. Again, not sure
of the impact, need to look closely.
was (Author: sarkaramr...@gmail.com):
Regarding {{updateLogSynchronizer}} ::
Everytime we call {{DISABLEBUFFER}} or {{ENABLEBUFFER}},
CdcrBufferManager::stateUpdate gets invoked::
{code}
@Override
public synchronized void stateUpdate() {
CdcrUpdateLog ulog = (CdcrUpdateLog) core.getUpdateHandler().getUpdateLog();
// If I am not the leader, I should always buffer my updates
if (!leaderStateManager.amILeader()) {
ulog.enableBuffer();
return;
}
// If I am the leader, I should buffer my updates only if buffer is enabled
else if
(bufferStateManager.getState().equals(CdcrParams.BufferState.ENABLED)) {
ulog.enableBuffer();
return;
}
// otherwise, disable the buffer
ulog.disableBuffer();
}
{code}
The non-leader nodes are by-defaulted are always buffer enabled ::
{code}
if (!leaderStateManager.amILeader()) {
ulog.enableBuffer();
return;
}
{code}
though LPV always calculated on leader but it has serious drawbacks explained
later:
in CdcrUpdateLogSynchronizer:: run :: if buffering is {enabled} ::
{code}
// if we received -1, it means that the log reader on the leader has not yet
started to read log entries
// do nothing
if (lastVersion == -1) {
return;
}
try {
CdcrUpdateLog ulog = (CdcrUpdateLog)
core.getUpdateHandler().getUpdateLog();
if (ulog.isBuffering()) {
log.debug("Advancing replica buffering tlog reader to {} @ {}:{}",
lastVersion, collection, shardId);
ulog.getBufferToggle().seek(lastVersion);
}
}
{code}
It always returns on {lastVersion == -1} and look at the comment {{if we
received -1, it means that the log reader on the leader has not yet started to
read log entries}}, that's misleading.
As the {{lastVersion}} is not +ve, the seek for the corresponding non-leader
nodes are never set to appropriate LPV.
Now if the leader goes down, and some non-leader becomes the leader himself,
the LPV is not set properly resulting in improper sync and I have no idea how
the impact will be in that case.
Also, as for non-leader nodes buffer is always on, if in the future it becomes
the leader itself, even if we have disabled buffer for the source collection
cluster, the status and its action will be {{buffer enabled}}.
[jira] [Updated] (SOLR-11003) Enabling bi-directional CDCR active-active clusters
[
https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-11003:
Attachment: SOLR-11003.patch
Latest patch uploaded:
{code}
modified: core/src/java/org/apache/solr/handler/CdcrReplicator.java
modified: core/src/java/org/apache/solr/update/CdcrTransactionLog.java
new file: core/src/java/org/apache/solr/update/TLogCommonUtils.java
modified: core/src/java/org/apache/solr/update/TransactionLog.java
new file:
core/src/test-files/solr/configsets/cdcr-cluster1/conf/schema.xml
new file:
core/src/test-files/solr/configsets/cdcr-cluster1/conf/solrconfig.xml
new file:
core/src/test-files/solr/configsets/cdcr-cluster2/conf/schema.xml
new file:
core/src/test-files/solr/configsets/cdcr-cluster2/conf/solrconfig.xml
new file:
core/src/test/org/apache/solr/cloud/CdcrBidirectionalTest.java
{code}
TLogCommonUtils.java included contains common code of TLog and CdcrTLog and
wrapped it under one class. Neat. Looking forward to more feedback and
suggestions.
> Enabling bi-directional CDCR active-active clusters
> ---
>
> Key: SOLR-11003
> URL: https://issues.apache.org/jira/browse/SOLR-11003
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Reporter: Amrit Sarkar
>Assignee: Varun Thacker
> Attachments: sample-configs.zip, SOLR-11003.patch, SOLR-11003.patch,
> SOLR-11003-tlogutils.patch
>
>
> The latest version of Solr CDCR across collections / clusters is in
> active-passive format, where we can index into source collection and the
> updates gets forwarded to the passive one and vice-versa is not supported.
> https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html
> https://issues.apache.org/jira/browse/SOLR-6273
> We are try to get a design ready to index in both collections and the
> updates gets reflected across the collections in real-time.
> ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB =>
> ClusterACollectionA.
> The best use-case would be to we keep indexing in ClusterACollectionA which
> forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets
> down, we point the indexer and searcher application to ClusterBCollectionB.
> Once ClusterACollectionA is up, depending on updates count, they will be
> bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and
> keep indexing on the ClusterBCollectionB.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11003) Enabling bi-directional CDCR active-active clusters
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amrit Sarkar updated SOLR-11003: Attachment: (was: cdcr-bi-tuils.patch) > Enabling bi-directional CDCR active-active clusters > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker > Attachments: sample-configs.zip, SOLR-11003.patch, > SOLR-11003-tlogutils.patch > > > The latest version of Solr CDCR across collections / clusters is in > active-passive format, where we can index into source collection and the > updates gets forwarded to the passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both collections and the > updates gets reflected across the collections in real-time. > ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB => > ClusterACollectionA. > The best use-case would be to we keep indexing in ClusterACollectionA which > forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets > down, we point the indexer and searcher application to ClusterBCollectionB. > Once ClusterACollectionA is up, depending on updates count, they will be > bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and > keep indexing on the ClusterBCollectionB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11069) LASTPROCESSEDVERSION for CDCR is flawed when buffering is enabled
[
https://issues.apache.org/jira/browse/SOLR-11069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087100#comment-16087100
]
Amrit Sarkar commented on SOLR-11069:
-
Regarding {{updateLogSynchronizer}} ::
Everytime we call {{DISABLEBUFFER}} or {{ENABLEBUFFER}},
CdcrBufferManager::stateUpdate gets invoked::
{code}
@Override
public synchronized void stateUpdate() {
CdcrUpdateLog ulog = (CdcrUpdateLog) core.getUpdateHandler().getUpdateLog();
// If I am not the leader, I should always buffer my updates
if (!leaderStateManager.amILeader()) {
ulog.enableBuffer();
return;
}
// If I am the leader, I should buffer my updates only if buffer is enabled
else if
(bufferStateManager.getState().equals(CdcrParams.BufferState.ENABLED)) {
ulog.enableBuffer();
return;
}
// otherwise, disable the buffer
ulog.disableBuffer();
}
{code}
The non-leader nodes are by-defaulted are always buffer enabled ::
{code}
if (!leaderStateManager.amILeader()) {
ulog.enableBuffer();
return;
}
{code}
though LPV always calculated on leader but it has serious drawbacks explained
later:
in CdcrUpdateLogSynchronizer:: run :: if buffering is {enabled} ::
{code}
// if we received -1, it means that the log reader on the leader has not yet
started to read log entries
// do nothing
if (lastVersion == -1) {
return;
}
try {
CdcrUpdateLog ulog = (CdcrUpdateLog)
core.getUpdateHandler().getUpdateLog();
if (ulog.isBuffering()) {
log.debug("Advancing replica buffering tlog reader to {} @ {}:{}",
lastVersion, collection, shardId);
ulog.getBufferToggle().seek(lastVersion);
}
}
{code}
It always returns on {lastVersion == -1} and look at the comment {{if we
received -1, it means that the log reader on the leader has not yet started to
read log entries}}, that's misleading.
As the {{lastVersion}} is not +ve, the seek for the corresponding non-leader
nodes are never set to appropriate LPV.
Now if the leader goes down, and some non-leader becomes the leader himself,
the LPV is not set properly resulting in improper sync and I have no idea how
the impact will be in that case.
Also, as for non-leader nodes buffer is always on, if in the future it becomes
the leader itself, even if we have disabled buffer for the source collection
cluster, the status and its action will be {{buffer enabled}}. Again, not sure
of the impact, need to look closely.
> LASTPROCESSEDVERSION for CDCR is flawed when buffering is enabled
> -
>
> Key: SOLR-11069
> URL: https://issues.apache.org/jira/browse/SOLR-11069
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Affects Versions: 7.0
>Reporter: Amrit Sarkar
>
> {{LASTPROCESSEDVERSION}} (a.b.v. LPV) action for CDCR breaks down due to
> poorly initialised and maintained buffer log for either source or target
> cluster core nodes.
> If buffer is enabled for cores of either source or target cluster, it return
> {{-1}}, *irrespective of number of entries in tlog read by the {{leader}}*
> node of each shard of respective collection of respective cluster. Once
> disabled, it starts telling us the correct LPV for each core.
> Due to the same flawed behavior, Update Log Synchroniser may doesn't work
> properly as expected, i.e. provides correct seek to the {{non-leader}} nodes
> to advance at. I am not sure whether this is an intended behavior for sync
> but it surely doesn't feel right.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11069) LASTPROCESSEDVERSION for CDCR is flawed when buffering is enabled
[
https://issues.apache.org/jira/browse/SOLR-11069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087076#comment-16087076
]
Amrit Sarkar commented on SOLR-11069:
-
So when we enable buffering in CDCR, {{buffertoggle}} gets initialised via
{{newLogReader()}} where ::
{code}
return new CdcrLogReader(new ArrayList(logs), tlog);
{code}
{code}
private CdcrLogReader(List tlogs, TransactionLog tlog) {
this.tlogs = new LinkedBlockingDeque<>();
this.tlogs.addAll(tlogs);
if (tlog != null) this.tlogs.push(tlog); // ensure that the tlog being
written is pushed
// Register the pointer in the parent UpdateLog
pointer = new CdcrLogPointer();
logPointers.put(this, pointer);
// If the reader is initialised while the updates log is empty, do nothing
if ((currentTlog = this.tlogs.peekLast()) != null) {
tlogReader = currentTlog.getReader(0);
pointer.set(currentTlog.tlogFile);
numRecordsReadInCurrentTlog = 0;
log.debug("Init new tlog reader for {} - tlogReader = {}",
currentTlog.tlogFile, tlogReader);
}
}
{code}
{{lastVersion}} and {{nextToLastVersion}} initialised as {{-1}} and never
changed / modified / updated ever. The recent logs are added into {{tlogs}} and
current tlog is maintained though.
Now LPV is calculated as: CdcrRequestHandler::handleLastProcessedVersionAction
{code}
for (CdcrReplicatorState state : replicatorManager.getReplicatorStates()) {
long version = Long.MAX_VALUE;
if (state.getLogReader() != null) {
version = state.getLogReader().getLastVersion();
}
lastProcessedVersion = Math.min(lastProcessedVersion, version);
}
// next check the log reader of the buffer
CdcrUpdateLog.CdcrLogReader bufferLogReader = ((CdcrUpdateLog)
core.getUpdateHandler().getUpdateLog()).getBufferToggle();
if (bufferLogReader != null) {
lastProcessedVersion = Math.min(lastProcessedVersion,
bufferLogReader.getLastVersion());
}
{code}
bufferLogReader.getLastVersion() is calculated {{-1}} and LPV outputs {{-1}}.
> LASTPROCESSEDVERSION for CDCR is flawed when buffering is enabled
> -
>
> Key: SOLR-11069
> URL: https://issues.apache.org/jira/browse/SOLR-11069
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Affects Versions: 7.0
>Reporter: Amrit Sarkar
>
> {{LASTPROCESSEDVERSION}} (a.b.v. LPV) action for CDCR breaks down due to
> poorly initialised and maintained buffer log for either source or target
> cluster core nodes.
> If buffer is enabled for cores of either source or target cluster, it return
> {{-1}}, *irrespective of number of entries in tlog read by the {{leader}}*
> node of each shard of respective collection of respective cluster. Once
> disabled, it starts telling us the correct LPV for each core.
> Due to the same flawed behavior, Update Log Synchroniser may doesn't work
> properly as expected, i.e. provides correct seek to the {{non-leader}} nodes
> to advance at. I am not sure whether this is an intended behavior for sync
> but it surely doesn't feel right.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11069) LASTPROCESSEDVERSION for CDCR is flawed when buffering is enabled
[
https://issues.apache.org/jira/browse/SOLR-11069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-11069:
Summary: LASTPROCESSEDVERSION for CDCR is flawed when buffering is enabled
(was: LASTPROCESSEDVERSION for CDCR (both source and target) is flawed when
buffering is enabled)
> LASTPROCESSEDVERSION for CDCR is flawed when buffering is enabled
> -
>
> Key: SOLR-11069
> URL: https://issues.apache.org/jira/browse/SOLR-11069
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Affects Versions: 7.0
>Reporter: Amrit Sarkar
>
> {{LASTPROCESSEDVERSION}} (a.b.v. LPV) action for CDCR breaks down due to
> poorly initialised and maintained buffer log for either source or target
> cluster core nodes.
> If buffer is enabled for cores of either source or target cluster, it return
> {{-1}}, *irrespective of number of entries in tlog read by the {{leader}}*
> node of each shard of respective collection of respective cluster. Once
> disabled, it starts telling us the correct LPV for each core.
> Due to the same flawed behavior, Update Log Synchroniser may doesn't work
> properly as expected, i.e. provides correct seek to the {{non-leader}} nodes
> to advance at. I am not sure whether this is an intended behavior for sync
> but it surely doesn't feel right.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11069) LASTPROCESSEDVERSION for CDCR (both source and target) is flawed when buffering is enabled
[
https://issues.apache.org/jira/browse/SOLR-11069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-11069:
Description:
{{LASTPROCESSEDVERSION}} (a.b.v. LPV) action for CDCR breaks down due to poorly
initialised and maintained buffer log for either source or target cluster core
nodes.
If buffer is enabled for cores of either source or target cluster, it return
{{-1}}, *irrespective of number of entries in tlog read by the {{leader}}* node
of each shard of respective collection of respective cluster. Once disabled, it
starts telling us the correct LPV for each core.
Due to the same flawed behavior, Update Log Synchroniser may doesn't work
properly as expected, i.e. provides correct seek to the {{non-leader}} nodes to
advance at. I am not sure whether this is an intended behavior for sync but it
surely doesn't feel right.
was:
{{LASTPROCESSEDVERSION}} (a.b.v. LPV) action for CDCR breaks down terribly due
to poorly initialised and maintained buffer log for source cluster core nodes.
If buffer is enabled for cores of either source or target cluster, it return
{{-1}}, *irrespective of number of entries in tlog read by the {{leader}}* node
of each shard of respective collection of respective cluster. Once disabled, it
starts telling us the correct LPV for each core.
Due to the same flawed behavior, Update Log Synchroniser may doesn't work
properly as expected, i.e. provides correct seek to the {{non-leader}} nodes to
advance at. I am not sure whether this is an intended behavior for sync but it
surely doesn't feel right.
> LASTPROCESSEDVERSION for CDCR (both source and target) is flawed when
> buffering is enabled
> --
>
> Key: SOLR-11069
> URL: https://issues.apache.org/jira/browse/SOLR-11069
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Affects Versions: 7.0
>Reporter: Amrit Sarkar
>
> {{LASTPROCESSEDVERSION}} (a.b.v. LPV) action for CDCR breaks down due to
> poorly initialised and maintained buffer log for either source or target
> cluster core nodes.
> If buffer is enabled for cores of either source or target cluster, it return
> {{-1}}, *irrespective of number of entries in tlog read by the {{leader}}*
> node of each shard of respective collection of respective cluster. Once
> disabled, it starts telling us the correct LPV for each core.
> Due to the same flawed behavior, Update Log Synchroniser may doesn't work
> properly as expected, i.e. provides correct seek to the {{non-leader}} nodes
> to advance at. I am not sure whether this is an intended behavior for sync
> but it surely doesn't feel right.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-11069) LASTPROCESSEDVERSION for CDCR (both source and target) is flawed when buffering is enabled
Amrit Sarkar created SOLR-11069:
---
Summary: LASTPROCESSEDVERSION for CDCR (both source and target) is
flawed when buffering is enabled
Key: SOLR-11069
URL: https://issues.apache.org/jira/browse/SOLR-11069
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Components: CDCR
Affects Versions: 7.0
Reporter: Amrit Sarkar
{{LASTPROCESSEDVERSION}} (a.b.v. LPV) action for CDCR breaks down terribly due
to poorly initialised and maintained buffer log for source cluster core nodes.
If buffer is enabled for cores of either source or target cluster, it return
{{-1}}, *irrespective of number of entries in tlog read by the {{leader}}* node
of each shard of respective collection of respective cluster. Once disabled, it
starts telling us the correct LPV for each core.
Due to the same flawed behavior, Update Log Synchroniser may doesn't work
properly as expected, i.e. provides correct seek to the {{non-leader}} nodes to
advance at. I am not sure whether this is an intended behavior for sync but it
surely doesn't feel right.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11003) Enabling bi-directional CDCR active-active clusters
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amrit Sarkar updated SOLR-11003: Attachment: cdcr-bi-tuils.patch > Enabling bi-directional CDCR active-active clusters > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker > Attachments: cdcr-bi-tuils.patch, sample-configs.zip, > SOLR-11003.patch, SOLR-11003-tlogutils.patch > > > The latest version of Solr CDCR across collections / clusters is in > active-passive format, where we can index into source collection and the > updates gets forwarded to the passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both collections and the > updates gets reflected across the collections in real-time. > ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB => > ClusterACollectionA. > The best use-case would be to we keep indexing in ClusterACollectionA which > forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets > down, we point the indexer and searcher application to ClusterBCollectionB. > Once ClusterACollectionA is up, depending on updates count, they will be > bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and > keep indexing on the ClusterBCollectionB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11003) Enabling bi-directional CDCR active-active clusters
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amrit Sarkar updated SOLR-11003: Attachment: (was: SOLR-11003.patch) > Enabling bi-directional CDCR active-active clusters > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker > Attachments: cdcr-bi-tuils.patch, sample-configs.zip, > SOLR-11003.patch, SOLR-11003-tlogutils.patch > > > The latest version of Solr CDCR across collections / clusters is in > active-passive format, where we can index into source collection and the > updates gets forwarded to the passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both collections and the > updates gets reflected across the collections in real-time. > ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB => > ClusterACollectionA. > The best use-case would be to we keep indexing in ClusterACollectionA which > forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets > down, we point the indexer and searcher application to ClusterBCollectionB. > Once ClusterACollectionA is up, depending on updates count, they will be > bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and > keep indexing on the ClusterBCollectionB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11003) Enabling bi-directional CDCR active-active clusters
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amrit Sarkar updated SOLR-11003: Attachment: (was: SOLR-11003-tlogutils.patch) > Enabling bi-directional CDCR active-active clusters > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker > Attachments: cdcr-bi-tuils.patch, sample-configs.zip, > SOLR-11003.patch, SOLR-11003-tlogutils.patch > > > The latest version of Solr CDCR across collections / clusters is in > active-passive format, where we can index into source collection and the > updates gets forwarded to the passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both collections and the > updates gets reflected across the collections in real-time. > ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB => > ClusterACollectionA. > The best use-case would be to we keep indexing in ClusterACollectionA which > forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets > down, we point the indexer and searcher application to ClusterBCollectionB. > Once ClusterACollectionA is up, depending on updates count, they will be > bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and > keep indexing on the ClusterBCollectionB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Closed] (SOLR-11040) CdcrBootstrapTest failing on 'master' branch
[
https://issues.apache.org/jira/browse/SOLR-11040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar closed SOLR-11040.
---
Resolution: Invalid
> CdcrBootstrapTest failing on 'master' branch
>
>
> Key: SOLR-11040
> URL: https://issues.apache.org/jira/browse/SOLR-11040
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Affects Versions: 6.6
>Reporter: Amrit Sarkar
>Priority: Critical
> Attachments: cdcr-bs-test-error
>
>
> {{CdcrBootstrapTest}} is constantly failing for all seeds on master branch
> both in terminal and IDE.
> {code}
> 86599 ERROR (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:54919_solr
> x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node1)
> [n:127.0.0.1:54919_solr c:cdcr-target s:shard1 r:core_node1
> x:cdcr-target_shard1_replica_n1] o.a.s.h.ReplicationHandler Index fetch
> failed :org.apache.solr.common.SolrException: Index fetch failed :
> at
> org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:655)
> at
> org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:332)
> at
> org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:419)
> at
> org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:757)
> at
> org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:712)
> at
> com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at
> org.apache.solr.handler.IndexFetcher.openNewSearcherAndUpdateCommitPoint(IndexFetcher.java:888)
> at
> org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:632)
> ... 10 more
> {code}
> Attached detailed logs:
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11040) CdcrBootstrapTest failing on 'master' branch
[
https://issues.apache.org/jira/browse/SOLR-11040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16080769#comment-16080769
]
Amrit Sarkar commented on SOLR-11040:
-
Yeah. [~ichattopadhyaya] [~varunthacker], I did `ant clean` at top level, git
clone fresh code, blew up the ivy cache. No luck. I will close this JIRA as it
is looking like machine specific issue. Thank you for looking into it.
> CdcrBootstrapTest failing on 'master' branch
>
>
> Key: SOLR-11040
> URL: https://issues.apache.org/jira/browse/SOLR-11040
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Affects Versions: 6.6
>Reporter: Amrit Sarkar
>Priority: Critical
> Attachments: cdcr-bs-test-error
>
>
> {{CdcrBootstrapTest}} is constantly failing for all seeds on master branch
> both in terminal and IDE.
> {code}
> 86599 ERROR (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:54919_solr
> x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node1)
> [n:127.0.0.1:54919_solr c:cdcr-target s:shard1 r:core_node1
> x:cdcr-target_shard1_replica_n1] o.a.s.h.ReplicationHandler Index fetch
> failed :org.apache.solr.common.SolrException: Index fetch failed :
> at
> org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:655)
> at
> org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:332)
> at
> org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:419)
> at
> org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:757)
> at
> org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:712)
> at
> com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at
> org.apache.solr.handler.IndexFetcher.openNewSearcherAndUpdateCommitPoint(IndexFetcher.java:888)
> at
> org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:632)
> ... 10 more
> {code}
> Attached detailed logs:
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-10263) Different SpellcheckComponents should have their own suggestMode
[
https://issues.apache.org/jira/browse/SOLR-10263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16080001#comment-16080001
]
Amrit Sarkar edited comment on SOLR-10263 at 7/10/17 8:06 AM:
--
[~asingh2411]
Can you put the changes as patch to the master branch on this JIRA? It will be
easier to accommodate.
Amrit Sarkar
was (Author: sarkaramr...@gmail.com):
[~asingh2411]m
Can you put the changes as patch to the master branch on this JIRA? It will be
easier to accommodate.
- Amrit Sarkar
> Different SpellcheckComponents should have their own suggestMode
>
>
> Key: SOLR-10263
> URL: https://issues.apache.org/jira/browse/SOLR-10263
> Project: Solr
> Issue Type: Wish
> Security Level: Public(Default Security Level. Issues are Public)
> Components: spellchecker
>Reporter: Abhishek Kumar Singh
>Priority: Minor
>
> As of now, common spellcheck options are applied to all the
> SpellCheckComponents.
> This can create problem in the following case:-
> It may be the case that we want *DirectSolrSpellChecker* to ALWAYS_SUGGEST
> spellcheck suggestions.
> But we may want *WordBreakSpellChecker* to suggest only if the token is not
> in the index (for relevance or performance reasons)
> (SUGGEST_WHEN_NOT_IN_INDEX) .
> *UPDATE : * Recently, we also figured out that, for
> {{WordBreakSolrSpellChecker}} also, both - The {{WordBreak}} and {{WordJoin}}
> should also have different suggestModes.
> We faced this problem in our case, wherein, Most of the WordJoin cases are
> those where the words individually are valid tokens, but what the users are
> looking for is actually a combination (wordjoin) of the two tokens.
> For example:-
> *gold mine sunglasses* : Here, both *gold* and *mine* are valid tokens. But
> the actual product being looked for is *goldmine sunglasses* , where
> *goldmine* is a brand.
> In such cases, we should recommend {{didYouMean:goldmine sunglasses}} . But
> this wont be possible because we had set {{SUGGEST_WHEN_NOT_IN_INDEX}} for
> {{WordBreakSolrSpellChecker}} (of which, WordJoin is a part) .
> For this, we should have separate suggestModes for both `wordJoin` as well as
> `wordBreak`.
> Related changes have been done at Latest PR. :
> https://github.com/apache/lucene-solr/pull/218.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10263) Different SpellcheckComponents should have their own suggestMode
[
https://issues.apache.org/jira/browse/SOLR-10263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16080001#comment-16080001
]
Amrit Sarkar commented on SOLR-10263:
-
[~asingh2411]m
Can you put the changes as patch to the master branch on this JIRA? It will be
easier to accommodate.
- Amrit Sarkar
> Different SpellcheckComponents should have their own suggestMode
>
>
> Key: SOLR-10263
> URL: https://issues.apache.org/jira/browse/SOLR-10263
> Project: Solr
> Issue Type: Wish
> Security Level: Public(Default Security Level. Issues are Public)
> Components: spellchecker
>Reporter: Abhishek Kumar Singh
>Priority: Minor
>
> As of now, common spellcheck options are applied to all the
> SpellCheckComponents.
> This can create problem in the following case:-
> It may be the case that we want *DirectSolrSpellChecker* to ALWAYS_SUGGEST
> spellcheck suggestions.
> But we may want *WordBreakSpellChecker* to suggest only if the token is not
> in the index (for relevance or performance reasons)
> (SUGGEST_WHEN_NOT_IN_INDEX) .
> *UPDATE : * Recently, we also figured out that, for
> {{WordBreakSolrSpellChecker}} also, both - The {{WordBreak}} and {{WordJoin}}
> should also have different suggestModes.
> We faced this problem in our case, wherein, Most of the WordJoin cases are
> those where the words individually are valid tokens, but what the users are
> looking for is actually a combination (wordjoin) of the two tokens.
> For example:-
> *gold mine sunglasses* : Here, both *gold* and *mine* are valid tokens. But
> the actual product being looked for is *goldmine sunglasses* , where
> *goldmine* is a brand.
> In such cases, we should recommend {{didYouMean:goldmine sunglasses}} . But
> this wont be possible because we had set {{SUGGEST_WHEN_NOT_IN_INDEX}} for
> {{WordBreakSolrSpellChecker}} (of which, WordJoin is a part) .
> For this, we should have separate suggestModes for both `wordJoin` as well as
> `wordBreak`.
> Related changes have been done at Latest PR. :
> https://github.com/apache/lucene-solr/pull/218.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11040) CdcrBootstrapTest failing on 'master' branch
[
https://issues.apache.org/jira/browse/SOLR-11040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-11040:
Attachment: cdcr-bs-test-error
> CdcrBootstrapTest failing on 'master' branch
>
>
> Key: SOLR-11040
> URL: https://issues.apache.org/jira/browse/SOLR-11040
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Affects Versions: 6.6
>Reporter: Amrit Sarkar
>Priority: Critical
> Attachments: cdcr-bs-test-error
>
>
> {{CdcrBootstrapTest}} is constantly failing for all seeds on master branch
> both in terminal and IDE.
> {code}
> 86599 ERROR (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:54919_solr
> x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node1)
> [n:127.0.0.1:54919_solr c:cdcr-target s:shard1 r:core_node1
> x:cdcr-target_shard1_replica_n1] o.a.s.h.ReplicationHandler Index fetch
> failed :org.apache.solr.common.SolrException: Index fetch failed :
> at
> org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:655)
> at
> org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:332)
> at
> org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:419)
> at
> org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:757)
> at
> org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:712)
> at
> com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at
> org.apache.solr.handler.IndexFetcher.openNewSearcherAndUpdateCommitPoint(IndexFetcher.java:888)
> at
> org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:632)
> ... 10 more
> {code}
> Attached detailed logs:
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-11040) CdcrBootstrapTest failing on 'master' branch
Amrit Sarkar created SOLR-11040:
---
Summary: CdcrBootstrapTest failing on 'master' branch
Key: SOLR-11040
URL: https://issues.apache.org/jira/browse/SOLR-11040
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Components: CDCR
Affects Versions: 6.6
Reporter: Amrit Sarkar
Priority: Critical
{{CdcrBootstrapTest}} is constantly failing for all seeds on master branch both
in terminal and IDE.
{code}
86599 ERROR (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:54919_solr
x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node1)
[n:127.0.0.1:54919_solr c:cdcr-target s:shard1 r:core_node1
x:cdcr-target_shard1_replica_n1] o.a.s.h.ReplicationHandler Index fetch failed
:org.apache.solr.common.SolrException: Index fetch failed :
at
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:655)
at
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:332)
at
org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:419)
at
org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:757)
at
org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:712)
at
com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at
org.apache.solr.handler.IndexFetcher.openNewSearcherAndUpdateCommitPoint(IndexFetcher.java:888)
at
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:632)
... 10 more
{code}
Attached detailed logs:
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11003) Enabling bi-directional CDCR active-active clusters
[
https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-11003:
Attachment: SOLR-11003-tlogutils.patch
Fixed bits in SOLR-11003-tlogutils.patch. Looking forward to the resolution of
{{CdcrBootstrapTest}} test failures and we can take forward
SOLR-11003-tlogutils.patch as the standard patch for this use-case.
> Enabling bi-directional CDCR active-active clusters
> ---
>
> Key: SOLR-11003
> URL: https://issues.apache.org/jira/browse/SOLR-11003
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Reporter: Amrit Sarkar
>Assignee: Varun Thacker
> Attachments: sample-configs.zip, SOLR-11003.patch, SOLR-11003.patch,
> SOLR-11003-tlogutils.patch, SOLR-11003-tlogutils.patch
>
>
> The latest version of Solr CDCR across collections / clusters is in
> active-passive format, where we can index into source collection and the
> updates gets forwarded to the passive one and vice-versa is not supported.
> https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html
> https://issues.apache.org/jira/browse/SOLR-6273
> We are try to get a design ready to index in both collections and the
> updates gets reflected across the collections in real-time.
> ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB =>
> ClusterACollectionA.
> The best use-case would be to we keep indexing in ClusterACollectionA which
> forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets
> down, we point the indexer and searcher application to ClusterBCollectionB.
> Once ClusterACollectionA is up, depending on updates count, they will be
> bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and
> keep indexing on the ClusterBCollectionB.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-11003) Enabling bi-directional CDCR active-active clusters
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079185#comment-16079185 ] Amrit Sarkar edited comment on SOLR-11003 at 7/8/17 4:54 PM: - heads up :: CdcrBootstrapTest is failing due to missing index on both the patches, looking into it. seems it is failing in the master branch anyway :( was (Author: sarkaramr...@gmail.com): heads up :: CdcrBootstrapTest is failing due to missing index on both the patches, looking into it. > Enabling bi-directional CDCR active-active clusters > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker > Attachments: sample-configs.zip, SOLR-11003.patch, SOLR-11003.patch, > SOLR-11003-tlogutils.patch > > > The latest version of Solr CDCR across collections / clusters is in > active-passive format, where we can index into source collection and the > updates gets forwarded to the passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both collections and the > updates gets reflected across the collections in real-time. > ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB => > ClusterACollectionA. > The best use-case would be to we keep indexing in ClusterACollectionA which > forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets > down, we point the indexer and searcher application to ClusterBCollectionB. > Once ClusterACollectionA is up, depending on updates count, they will be > bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and > keep indexing on the ClusterBCollectionB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11003) Enabling bi-directional CDCR active-active clusters
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079185#comment-16079185 ] Amrit Sarkar commented on SOLR-11003: - heads up :: CdcrBootstrapTest is failing due to missing index on both the patches, looking into it. > Enabling bi-directional CDCR active-active clusters > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker > Attachments: sample-configs.zip, SOLR-11003.patch, SOLR-11003.patch, > SOLR-11003-tlogutils.patch > > > The latest version of Solr CDCR across collections / clusters is in > active-passive format, where we can index into source collection and the > updates gets forwarded to the passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both collections and the > updates gets reflected across the collections in real-time. > ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB => > ClusterACollectionA. > The best use-case would be to we keep indexing in ClusterACollectionA which > forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets > down, we point the indexer and searcher application to ClusterBCollectionB. > Once ClusterACollectionA is up, depending on updates count, they will be > bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and > keep indexing on the ClusterBCollectionB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11003) Enabling bi-directional CDCR active-active clusters
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amrit Sarkar updated SOLR-11003: Attachment: SOLR-11003-tlogutils.patch > Enabling bi-directional CDCR active-active clusters > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker > Attachments: sample-configs.zip, SOLR-11003.patch, SOLR-11003.patch, > SOLR-11003-tlogutils.patch > > > The latest version of Solr CDCR across collections / clusters is in > active-passive format, where we can index into source collection and the > updates gets forwarded to the passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both collections and the > updates gets reflected across the collections in real-time. > ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB => > ClusterACollectionA. > The best use-case would be to we keep indexing in ClusterACollectionA which > forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets > down, we point the indexer and searcher application to ClusterBCollectionB. > Once ClusterACollectionA is up, depending on updates count, they will be > bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and > keep indexing on the ClusterBCollectionB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11003) Enabling bi-directional CDCR active-active clusters
[
https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079183#comment-16079183
]
Amrit Sarkar commented on SOLR-11003:
-
[~varunthacker], thank you for the review and feedback,
{quote}
Can we also test for in place updates ( I didn't see a anything triggering
that code path )
{quote}
I think IN_PLACE updates are not forwarded to target clusters / supported in
CDCR, as those updates are not recognised in CDCRReplicator::processUpdate and
are never forwarded. I still put the boolean flag in place for IN_PLACE updates
for near future. I don't know the reason why it is excluded. See this:
{code}
private UpdateRequest processUpdate(Object o, UpdateRequest req) {
// should currently be a List
List entry = (List) o;
int operationAndFlags = (Integer) entry.get(0);
int oper = operationAndFlags & UpdateLog.OPERATION_MASK;
long version = (Long) entry.get(1);
// record the operation in the benchmark timer
state.getBenchmarkTimer().incrementCounter(oper);
switch (oper) {
case UpdateLog.ADD: {
// the version is already attached to the document
SolrInputDocument sdoc = (SolrInputDocument) entry.get(entry.size() -
1);
req.add(sdoc);
return req;
}
case UpdateLog.DELETE: {
byte[] idBytes = (byte[]) entry.get(2);
req.deleteById(new String(idBytes, Charset.forName("UTF-8")));
req.setParam(VERSION_FIELD, Long.toString(version));
return req;
}
case UpdateLog.DELETE_BY_QUERY: {
String query = (String) entry.get(2);
req.deleteByQuery(query);
req.setParam(VERSION_FIELD, Long.toString(version));
return req;
}
case UpdateLog.COMMIT: {
return null;
}
default:
throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Unknown
Operation! " + oper);
}
}
{code}
{quote}
Also it wouldn't hurt to add atomic updates to the test. Suitable comments.
{quote}
Patch updated. SOLR-11003.patch with the above recommendations.
Also I have uploaded SOLR-11003-tlogutils.patch (NOT TESTED 100%), where we are
eliminating duplicate code and putting the common in TLogCommonUtils. This
solution is much cleaner and effective. I will update once I get {{ant test}}
and {{ant test -Dtests.nightly=true}} run successfully for {{solr}} sub-project.
> Enabling bi-directional CDCR active-active clusters
> ---
>
> Key: SOLR-11003
> URL: https://issues.apache.org/jira/browse/SOLR-11003
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Reporter: Amrit Sarkar
>Assignee: Varun Thacker
> Attachments: sample-configs.zip, SOLR-11003.patch, SOLR-11003.patch
>
>
> The latest version of Solr CDCR across collections / clusters is in
> active-passive format, where we can index into source collection and the
> updates gets forwarded to the passive one and vice-versa is not supported.
> https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html
> https://issues.apache.org/jira/browse/SOLR-6273
> We are try to get a design ready to index in both collections and the
> updates gets reflected across the collections in real-time.
> ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB =>
> ClusterACollectionA.
> The best use-case would be to we keep indexing in ClusterACollectionA which
> forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets
> down, we point the indexer and searcher application to ClusterBCollectionB.
> Once ClusterACollectionA is up, depending on updates count, they will be
> bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and
> keep indexing on the ClusterBCollectionB.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11003) Enabling bi-directional CDCR active-active clusters
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amrit Sarkar updated SOLR-11003: Attachment: SOLR-11003.patch > Enabling bi-directional CDCR active-active clusters > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker > Attachments: sample-configs.zip, SOLR-11003.patch, SOLR-11003.patch > > > The latest version of Solr CDCR across collections / clusters is in > active-passive format, where we can index into source collection and the > updates gets forwarded to the passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both collections and the > updates gets reflected across the collections in real-time. > ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB => > ClusterACollectionA. > The best use-case would be to we keep indexing in ClusterACollectionA which > forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets > down, we point the indexer and searcher application to ClusterBCollectionB. > Once ClusterACollectionA is up, depending on updates count, they will be > bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and > keep indexing on the ClusterBCollectionB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11003) Enabling bi-directional CDCR active-active clusters
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amrit Sarkar updated SOLR-11003: Attachment: sample-configs.zip > Enabling bi-directional CDCR active-active clusters > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker > Attachments: sample-configs.zip, SOLR-11003.patch > > > The latest version of Solr CDCR across collections / clusters is in > active-passive format, where we can index into source collection and the > updates gets forwarded to the passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both collections and the > updates gets reflected across the collections in real-time. > ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB => > ClusterACollectionA. > The best use-case would be to we keep indexing in ClusterACollectionA which > forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets > down, we point the indexer and searcher application to ClusterBCollectionB. > Once ClusterACollectionA is up, depending on updates count, they will be > bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and > keep indexing on the ClusterBCollectionB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11003) Enabling bi-directional CDCR active-active clusters
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amrit Sarkar updated SOLR-11003: Attachment: (was: sample-configs.zip) > Enabling bi-directional CDCR active-active clusters > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker > Attachments: sample-configs.zip, SOLR-11003.patch > > > The latest version of Solr CDCR across collections / clusters is in > active-passive format, where we can index into source collection and the > updates gets forwarded to the passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both collections and the > updates gets reflected across the collections in real-time. > ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB => > ClusterACollectionA. > The best use-case would be to we keep indexing in ClusterACollectionA which > forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets > down, we point the indexer and searcher application to ClusterBCollectionB. > Once ClusterACollectionA is up, depending on updates count, they will be > bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and > keep indexing on the ClusterBCollectionB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11003) Enabling bi-directional CDCR active-active clusters
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amrit Sarkar updated SOLR-11003: Attachment: sample-configs.zip > Enabling bi-directional CDCR active-active clusters > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar >Assignee: Varun Thacker > Attachments: sample-configs.zip, SOLR-11003.patch > > > The latest version of Solr CDCR across collections / clusters is in > active-passive format, where we can index into source collection and the > updates gets forwarded to the passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both collections and the > updates gets reflected across the collections in real-time. > ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB => > ClusterACollectionA. > The best use-case would be to we keep indexing in ClusterACollectionA which > forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets > down, we point the indexer and searcher application to ClusterBCollectionB. > Once ClusterACollectionA is up, depending on updates count, they will be > bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and > keep indexing on the ClusterBCollectionB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11003) Enabling bi-directional CDCR active-active clusters
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amrit Sarkar updated SOLR-11003: Attachment: SOLR-11003.patch > Enabling bi-directional CDCR active-active clusters > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar > Attachments: SOLR-11003.patch > > > The latest version of Solr CDCR across collections / clusters is in > active-passive format, where we can index into source collection and the > updates gets forwarded to the passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both collections and the > updates gets reflected across the collections in real-time. > ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB => > ClusterACollectionA. > The best use-case would be to we keep indexing in ClusterACollectionA which > forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets > down, we point the indexer and searcher application to ClusterBCollectionB. > Once ClusterACollectionA is up, depending on updates count, they will be > bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and > keep indexing on the ClusterBCollectionB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11003) Enabling bi-directional CDCR active-active clusters
[
https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16073586#comment-16073586
]
Amrit Sarkar commented on SOLR-11003:
-
Patch uploaded:
{code}
modified:
solr/core/src/java/org/apache/solr/handler/CdcrReplicator.java
modified:
solr/core/src/java/org/apache/solr/update/CdcrTransactionLog.java
modified:
solr/core/src/java/org/apache/solr/update/TransactionLog.java
new file:
solr/core/src/test-files/solr/configsets/cdcr-cluster1/conf/schema.xml
new file:
solr/core/src/test-files/solr/configsets/cdcr-cluster1/conf/solrconfig.xml
new file:
solr/core/src/test-files/solr/configsets/cdcr-cluster2/conf/schema.xml
new file:
solr/core/src/test-files/solr/configsets/cdcr-cluster2/conf/solrconfig.xml
new file:
solr/core/src/test/org/apache/solr/cloud/CdcrBidirectionalTest.java
{code}
Added testclass CdcrBidirectionalTest where two active clusters are talking to
each other runtime.
The write operations in TransactionLog are repeated in CdcrTransactionLog to
accommodate the extra entry for each update. Repeated code! henceforth planning
to have TLogCommonUtils / UpdateLogCommonUtils to put the common code for both
classes' methods.
Eagerly looking forward to feedback, suggestions and improvements.
> Enabling bi-directional CDCR active-active clusters
> ---
>
> Key: SOLR-11003
> URL: https://issues.apache.org/jira/browse/SOLR-11003
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Reporter: Amrit Sarkar
>
> The latest version of Solr CDCR across collections / clusters is in
> active-passive format, where we can index into source collection and the
> updates gets forwarded to the passive one and vice-versa is not supported.
> https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html
> https://issues.apache.org/jira/browse/SOLR-6273
> We are try to get a design ready to index in both collections and the
> updates gets reflected across the collections in real-time.
> ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB =>
> ClusterACollectionA.
> The best use-case would be to we keep indexing in ClusterACollectionA which
> forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets
> down, we point the indexer and searcher application to ClusterBCollectionB.
> Once ClusterACollectionA is up, depending on updates count, they will be
> bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and
> keep indexing on the ClusterBCollectionB.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11003) Enabling bi-directional CDCR active-active clusters
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amrit Sarkar updated SOLR-11003: Description: The latest version of Solr CDCR across collections / clusters is in active-passive format, where we can index into source collection and the updates gets forwarded to the passive one and vice-versa is not supported. https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html https://issues.apache.org/jira/browse/SOLR-6273 We are try to get a design ready to index in both collections and the updates gets reflected across the collections in real-time. ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB => ClusterACollectionA. The best use-case would be to we keep indexing in ClusterACollectionA which forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets down, we point the indexer and searcher application to ClusterBCollectionB. Once ClusterACollectionA is up, depending on updates count, they will be bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and keep indexing on the ClusterBCollectionB. was: The latest version of Solr CDCR across collections / clusters is in active-passive format, where we can index into source collection and the updates gets forwarded to the passive one and vice-versa is not supported. https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html https://issues.apache.org/jira/browse/SOLR-6273 We are try to get a design ready to index in both collections and the updates gets reflected across the collections in real-time. ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB => ClusterACollectionA. The best use-case would be to we keep indexing in ClusterACollectionA which forwarded the updates to ClusterBCollectionB. If ClusterACollectionA gets down, we point the indexer and searcher application to ClusterBCollectionB. Once ClusterACollectionA is up, depending updates count, they will be bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB. > Enabling bi-directional CDCR active-active clusters > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar > > The latest version of Solr CDCR across collections / clusters is in > active-passive format, where we can index into source collection and the > updates gets forwarded to the passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both collections and the > updates gets reflected across the collections in real-time. > ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB => > ClusterACollectionA. > The best use-case would be to we keep indexing in ClusterACollectionA which > forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets > down, we point the indexer and searcher application to ClusterBCollectionB. > Once ClusterACollectionA is up, depending on updates count, they will be > bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and > keep indexing on the ClusterBCollectionB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-11003) Enabling bi-directional CDCR active-active clusters
[
https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16073575#comment-16073575
]
Amrit Sarkar edited comment on SOLR-11003 at 7/4/17 12:27 PM:
--
Backward-compat::
tlogs entries index are definite for each operation: identifier(0) and
version(1) common to all.
*DELETE:*
identifier: delete
update version
payload
if you see UpdateLog: doReplay: 1801
{code}
case UpdateLog.DELETE: {
recoveryInfo.deletes++;
byte[] idBytes = (byte[]) entry.get(2);
DeleteUpdateCommand cmd = new DeleteUpdateCommand(req);
cmd.setIndexedId(new BytesRef(idBytes));
cmd.setVersion(version);
cmd.setFlags(UpdateCommand.REPLAY |
UpdateCommand.IGNORE_AUTOCOMMIT);
if (debug) log.debug("delete " + cmd);
proc.processDelete(cmd);
break;
}
{code}
hardcoded, the position of payload(2) is hardcoded. See CdcrReplicator, same.
So we can put anything on next index(3).
*DELETE_BY_QUERY:*
Ditto same!!
*ADD:*
indentifier: add or inplace
version
payload
*IN_PLACE_ADD:*
indentifier: add or inplace
version
previous pointer
previous version
payload
UpdateLog:: 1916, since our current code handles both adds in one manner, it
assumes the last index of the entries is our payload:
{code}
public static AddUpdateCommand
convertTlogEntryToAddUpdateCommand(SolrQueryRequest req, List entry,
int
operation, long version) {
assert operation == UpdateLog.ADD || operation == UpdateLog.UPDATE_INPLACE;
SolrInputDocument sdoc = (SolrInputDocument) entry.get(entry.size()-1);
AddUpdateCommand cmd = new AddUpdateCommand(req);
cmd.solrDoc = sdoc;
cmd.setVersion(version);
if (operation == UPDATE_INPLACE) {
long prevVersion = (Long) entry.get(UpdateLog.PREV_VERSION_IDX);
cmd.prevVersion = prevVersion;
}
return cmd;
}
{code}
So our window of adding something is the index just before the last. entry.size
- 2. Please see CdcrReplicator for the same.
was (Author: sarkaramr...@gmail.com):
Backward-compat::
tlogs entries index are definite for each operation: identifier(0) and
version(1) common to all.
*DELETE:*
identifier: delete
update version
payload
if you see UpdateLog: doReplay: 1801
{code}
case UpdateLog.DELETE: {
recoveryInfo.deletes++;
byte[] idBytes = (byte[]) entry.get(2);
DeleteUpdateCommand cmd = new DeleteUpdateCommand(req);
cmd.setIndexedId(new BytesRef(idBytes));
cmd.setVersion(version);
cmd.setFlags(UpdateCommand.REPLAY |
UpdateCommand.IGNORE_AUTOCOMMIT);
if (debug) log.debug("delete " + cmd);
proc.processDelete(cmd);
break;
}
{code}
hardcoded, the position of payload(2) is hardcoded. See CdcrReplicator, same.
So we can put anything on next index(3).
*DELETE_BY_QUERY:*
Ditto same!!
*ADD:*
indentifier: add or inplace
version
payload
*IN_PLACE_ADD:*
indentifier: add or inplace
version
previous pointer
previous version
payload
UpdateLog:: 1916, since our current code handles both adds in one manner, it
assumes the last index of the entries is our payload:
{code}
public static AddUpdateCommand
convertTlogEntryToAddUpdateCommand(SolrQueryRequest req, List entry,
int
operation, long version) {
assert operation == UpdateLog.ADD || operation == UpdateLog.UPDATE_INPLACE;
SolrInputDocument sdoc = (SolrInputDocument) entry.get(entry.size()-1);
AddUpdateCommand cmd = new AddUpdateCommand(req);
cmd.solrDoc = sdoc;
cmd.setVersion(version);
if (operation == UPDATE_INPLACE) {
long prevVersion = (Long) entry.get(UpdateLog.PREV_VERSION_IDX);
cmd.prevVersion = prevVersion;
}
return cmd;
}
{code}
So our window of adding something is the index just before the last. entry.size
- 2. Please see CdcrReplictor for the same.
> Enabling bi-directional CDCR active-active clusters
> ---
>
> Key: SOLR-11003
> URL: https://issues.apache.org/jira/browse/SOLR-11003
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Reporter: Amrit Sarkar
>
> The latest version of Solr CDCR across collections / clusters is in
> active-passive format, where we can index into source collection and the
> updates gets forwarded to the passive one and vice-versa is not supported.
> https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html
> https://issues.apache.org/jira/browse/SOLR-6273
> We are
[jira] [Commented] (SOLR-11003) Enabling bi-directional CDCR active-active clusters
[
https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16073575#comment-16073575
]
Amrit Sarkar commented on SOLR-11003:
-
Backward-compat::
tlogs entries index are definite for each operation: identifier(0) and
version(1) common to all.
*DELETE:*
identifier: delete
update version
payload
if you see UpdateLog: doReplay: 1801
{code}
case UpdateLog.DELETE: {
recoveryInfo.deletes++;
byte[] idBytes = (byte[]) entry.get(2);
DeleteUpdateCommand cmd = new DeleteUpdateCommand(req);
cmd.setIndexedId(new BytesRef(idBytes));
cmd.setVersion(version);
cmd.setFlags(UpdateCommand.REPLAY |
UpdateCommand.IGNORE_AUTOCOMMIT);
if (debug) log.debug("delete " + cmd);
proc.processDelete(cmd);
break;
}
{code}
hardcoded, the position of payload(2) is hardcoded. See CdcrReplicator, same.
So we can put anything on next index(3).
*DELETE_BY_QUERY:*
Ditto same!!
*ADD:*
indentifier: add or inplace
version
payload
*IN_PLACE_ADD:*
indentifier: add or inplace
version
previous pointer
previous version
payload
UpdateLog:: 1916, since our current code handles both adds in one manner, it
assumes the last index of the entries is our payload:
{code}
public static AddUpdateCommand
convertTlogEntryToAddUpdateCommand(SolrQueryRequest req, List entry,
int
operation, long version) {
assert operation == UpdateLog.ADD || operation == UpdateLog.UPDATE_INPLACE;
SolrInputDocument sdoc = (SolrInputDocument) entry.get(entry.size()-1);
AddUpdateCommand cmd = new AddUpdateCommand(req);
cmd.solrDoc = sdoc;
cmd.setVersion(version);
if (operation == UPDATE_INPLACE) {
long prevVersion = (Long) entry.get(UpdateLog.PREV_VERSION_IDX);
cmd.prevVersion = prevVersion;
}
return cmd;
}
{code}
So our window of adding something is the index just before the last. entry.size
- 2. Please see CdcrReplictor for the same.
> Enabling bi-directional CDCR active-active clusters
> ---
>
> Key: SOLR-11003
> URL: https://issues.apache.org/jira/browse/SOLR-11003
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Reporter: Amrit Sarkar
>
> The latest version of Solr CDCR across collections / clusters is in
> active-passive format, where we can index into source collection and the
> updates gets forwarded to the passive one and vice-versa is not supported.
> https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html
> https://issues.apache.org/jira/browse/SOLR-6273
> We are try to get a design ready to index in both collections and the
> updates gets reflected across the collections in real-time.
> ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB =>
> ClusterACollectionA.
> The best use-case would be to we keep indexing in ClusterACollectionA which
> forwarded the updates to ClusterBCollectionB. If ClusterACollectionA gets
> down, we point the indexer and searcher application to ClusterBCollectionB.
> Once ClusterACollectionA is up, depending updates count, they will be
> bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11003) Enabling bi-directional CDCR active-active clusters
[
https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16073572#comment-16073572
]
Amrit Sarkar commented on SOLR-11003:
-
*Proposed design =>*
*Configuration:*
Configure both collections as source and point to their targets. Sample
configurations listed below:
_Cluster2:_
{code}
${cdcr.cluster1.zkHost}
cdcr-cluster2
cdcr-cluster1
1
1000
1000
1000
{code}
_Cluster1:_
{code}
${cdcr.cluster2.zkHost}
cdcr-cluster1
cdcr-cluster2
1
1000
1000
1000
{code}
*Replication Improvement:*
CdcrReplicator replays the tlogs, creates new UpdateRequests and forwards them
to target collection. *The plan is to persist information in the tlogs' each
update entries telling the collection this update is received as a CDCR forward
and it should be not be forwarded further.*
Each CDCR update forward contains the param key "cdcr.update" and value empty.
Once the target collection received that particular update, before writing it
down to transaction log, we check whether the UpdateCommand -> SolrRequest ->
contains params key "cdcr.update", if there is, apart from the entries listed
for each update, we add a flag, say isCdcr. "true" and write it to tlog. When
CdcrReplicator reads updates from the tlog, and if the flag is present / set,
skip that update for forwarding and read next.
We had to be very careful of where to put the flag so that we don't break the
backward compatibility and hence we added at the last index for DELETE and
DELETEBYQUERY while second last index for ADD. Reason listed below.
> Enabling bi-directional CDCR active-active clusters
> ---
>
> Key: SOLR-11003
> URL: https://issues.apache.org/jira/browse/SOLR-11003
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Reporter: Amrit Sarkar
>
> The latest version of Solr CDCR across collections / clusters is in
> active-passive format, where we can index into source collection and the
> updates gets forwarded to the passive one and vice-versa is not supported.
> https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html
> https://issues.apache.org/jira/browse/SOLR-6273
> We are try to get a design ready to index in both collections and the
> updates gets reflected across the collections in real-time.
> ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB =>
> ClusterACollectionA.
> The best use-case would be to we keep indexing in ClusterACollectionA which
> forwarded the updates to ClusterBCollectionB. If ClusterACollectionA gets
> down, we point the indexer and searcher application to ClusterBCollectionB.
> Once ClusterACollectionA is up, depending updates count, they will be
> bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11003) Enabling bi-directional CDCR active-active clusters
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amrit Sarkar updated SOLR-11003: Description: The latest version of Solr CDCR across collections / clusters is in active-passive format, where we can index into source collection and the updates gets forwarded to the passive one and vice-versa is not supported. https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html https://issues.apache.org/jira/browse/SOLR-6273 We are try to get a design ready to index in both collections and the updates gets reflected across the collections in real-time. ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB => ClusterACollectionA. The best use-case would be to we keep indexing in ClusterACollectionA which forwarded the updates to ClusterBCollectionB. If ClusterACollectionA gets down, we point the indexer and searcher application to ClusterBCollectionB. Once ClusterACollectionA is up, depending updates count, they will be bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB. was: The latest version of Solr CDCR across clusters is in active-passive format, where we can index into source cluster and the updates gets forwarded to the passive one and vice-versa is not supported. https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html https://issues.apache.org/jira/browse/SOLR-6273 We are try to get a design ready to index in both clusters and the updates gets reflected across the clusters in real-time. ClusterA => ClusterB | ClusterB => ClusterA. The best use-case would be to we keep indexing in ClusterA which forwarded the updates to ClusterB. If ClusterA gets down, we point the indexer and searcher application to ClusterB. Once ClusterA is up, depending updates count, they will be bootstrapped or forwarded to ClusterA from ClusterB. > Enabling bi-directional CDCR active-active clusters > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar > > The latest version of Solr CDCR across collections / clusters is in > active-passive format, where we can index into source collection and the > updates gets forwarded to the passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both collections and the > updates gets reflected across the collections in real-time. > ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB => > ClusterACollectionA. > The best use-case would be to we keep indexing in ClusterACollectionA which > forwarded the updates to ClusterBCollectionB. If ClusterACollectionA gets > down, we point the indexer and searcher application to ClusterBCollectionB. > Once ClusterACollectionA is up, depending updates count, they will be > bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11003) Enabling bi-directional CDCR active-active clusters
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amrit Sarkar updated SOLR-11003: Description: The latest version of Solr CDCR across clusters is in active-passive format, where we can index into source cluster and the updates gets forwarded to the passive one and vice-versa is not supported. https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html https://issues.apache.org/jira/browse/SOLR-6273 We are try to get a design ready to index in both clusters and the updates gets reflected across the clusters in real-time. ClusterA => ClusterB | ClusterB => ClusterA. The best use-case would be to we keep indexing in ClusterA which forwarded the updates to ClusterB. If ClusterA gets down, we point the indexer and searcher application to ClusterB. Once ClusterA is up, depending updates count, they will be bootstrapped or forwarded to ClusterA from ClusterB. was: The latest version of Solr CDCR across cluster in active-passive format, where we can index into source cluster and the updates gets forwarded to the passive one and vice-versa is not supported. https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html https://issues.apache.org/jira/browse/SOLR-6273 We are try to get a design ready to index in both clusters and the updates gets reflected across the clusters in real-time. ClusterA => ClusterB | ClusterB => ClusterA. The best use-case would be to we keep indexing in ClusterA which forwarded the updates to ClusterB. If ClusterA gets down, we point the indexer and searcher application to ClusterB. Once ClusterA is up, depending updates count, they will be bootstrapped or forwarded to ClusterA from ClusterB. > Enabling bi-directional CDCR active-active clusters > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar > > The latest version of Solr CDCR across clusters is in active-passive format, > where we can index into source cluster and the updates gets forwarded to the > passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both clusters and the updates > gets reflected across the clusters in real-time. ClusterA => ClusterB | > ClusterB => ClusterA. > The best use-case would be to we keep indexing in ClusterA which forwarded > the updates to ClusterB. If ClusterA gets down, we point the indexer and > searcher application to ClusterB. Once ClusterA is up, depending updates > count, they will be bootstrapped or forwarded to ClusterA from ClusterB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-11003) Enabling bi-directional CDCR active-active clusters
Amrit Sarkar created SOLR-11003: --- Summary: Enabling bi-directional CDCR active-active clusters Key: SOLR-11003 URL: https://issues.apache.org/jira/browse/SOLR-11003 Project: Solr Issue Type: Improvement Security Level: Public (Default Security Level. Issues are Public) Components: CDCR Reporter: Amrit Sarkar The latest version of Solr CDCR across cluster in active-passive format, where we can index into source cluster and the updates gets forwarded to the passive one and vice-versa is not supported. https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html https://issues.apache.org/jira/browse/SOLR-6273 We are try to get a design ready to index in both clusters and the updates gets reflected across the clusters in real-time. ClusterA => ClusterB | ClusterB => ClusterA. The best use-case would be to we keep indexing in ClusterA which forwarded the updates to ClusterB. If ClusterA gets down, we point the indexer and searcher application to ClusterB. Once ClusterA is up, depending updates count, they will be bootstrapped or forwarded to ClusterA from ClusterB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5822) Create a markdown formatted README file for Lucene/Solr
[
https://issues.apache.org/jira/browse/LUCENE-5822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071580#comment-16071580
]
Amrit Sarkar commented on LUCENE-5822:
--
[~mdrob] I can see {{dev-tools/scripts/.smokeTestRelease.py.swp}} checked in
too along with the py-script. Is that intentional?
> Create a markdown formatted README file for Lucene/Solr
> ---
>
> Key: LUCENE-5822
> URL: https://issues.apache.org/jira/browse/LUCENE-5822
> Project: Lucene - Core
> Issue Type: Wish
>Reporter: Shalin Shekhar Mangar
>Assignee: Mike Drob
> Fix For: master (7.0)
>
> Attachments: LUCENE-5822-addemdum.patch, LUCENE-5822.patch,
> LUCENE-5822.patch
>
>
> We have a minimal plain text readme file right now. Github is very popular
> these days and we are even accepting pull requests from there. I think we
> should add a proper markdown formatted README file which not only talks about
> the features of Lucene/Solr but also include a short tutorial on both Lucene
> and Solr.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10866) Make TimestampUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10866:
Attachment: SOLR-10866.patch
patch modified SOLR-9565.patch through.
> Make TimestampUpdateProcessorFactory as Runtime URP; take params(s) with
> request
>
>
> Key: SOLR-10866
> URL: https://issues.apache.org/jira/browse/SOLR-10866
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10866.patch, SOLR-10866.patch, SOLR-10866.patch,
> SOLR-10866.patch, SOLR-10866.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> TimestampUpdateProcessorFactory will be able to execute by sample curl like
> below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=Timestamp=timestamp_tdt=true
> --data-binary { "id" : "1" , "title_s" : "titleA" }
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10866) Make TimestampUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10866:
Description:
We are trying to get rid of processor definitions in SolrConfig for all URPs
and take parameters in the request itself.
TimestampUpdateProcessorFactory will be able to execute by sample curl like
below:
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=timestamp=timestamp_tdt=true
--data-binary { "id" : "1" , "title_s" : "titleA" }
{code}
Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
was:
We are trying to get rid of processor definitions in SolrConfig for all URPs
and take parameters in the request itself.
TimestampUpdateProcessorFactory will be able to execute by sample curl like
below:
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=Timestamp=timestamp_tdt=true
--data-binary { "id" : "1" , "title_s" : "titleA" }
{code}
Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
> Make TimestampUpdateProcessorFactory as Runtime URP; take params(s) with
> request
>
>
> Key: SOLR-10866
> URL: https://issues.apache.org/jira/browse/SOLR-10866
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10866.patch, SOLR-10866.patch, SOLR-10866.patch,
> SOLR-10866.patch, SOLR-10866.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> TimestampUpdateProcessorFactory will be able to execute by sample curl like
> below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=timestamp=timestamp_tdt=true
> --data-binary { "id" : "1" , "title_s" : "titleA" }
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10866) Make TimestampUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10866:
Description:
We are trying to get rid of processor definitions in SolrConfig for all URPs
and take parameters in the request itself.
TimestampUpdateProcessorFactory will be able to execute by sample curl like
below:
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=timestamp=timestamp_tdt=true
--data-binary { "id" : "1" , "title_s" : "titleA" }
{code}
Configuration for TimestampUpdateProcessorFactory in solrconfig.xml is optional.
was:
We are trying to get rid of processor definitions in SolrConfig for all URPs
and take parameters in the request itself.
TimestampUpdateProcessorFactory will be able to execute by sample curl like
below:
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=timestamp=timestamp_tdt=true
--data-binary { "id" : "1" , "title_s" : "titleA" }
{code}
Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
> Make TimestampUpdateProcessorFactory as Runtime URP; take params(s) with
> request
>
>
> Key: SOLR-10866
> URL: https://issues.apache.org/jira/browse/SOLR-10866
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10866.patch, SOLR-10866.patch, SOLR-10866.patch,
> SOLR-10866.patch, SOLR-10866.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> TimestampUpdateProcessorFactory will be able to execute by sample curl like
> below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=timestamp=timestamp_tdt=true
> --data-binary { "id" : "1" , "title_s" : "titleA" }
> {code}
> Configuration for TimestampUpdateProcessorFactory in solrconfig.xml is
> optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10858) Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10858:
Attachment: SOLR-10858.patch
Patch updated with SOLR-9565 design.
> Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
> ---
>
> Key: SOLR-10858
> URL: https://issues.apache.org/jira/browse/SOLR-10858
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch,
> SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch,
> SOLR-10858.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9565) Make every UpdateRequestProcessor available implicitly
[
https://issues.apache.org/jira/browse/SOLR-9565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16050219#comment-16050219
]
Amrit Sarkar commented on SOLR-9565:
just skimming;
Shouldn't support for {{processor=X}}, X in XURPFactory, continue? AtomicURP
can be invoked both via {{processor=atomic}}, simplified and
{{processor=Atomic}}, already supported. I saw complaints on StatelessScriptURP
where {{stateless}} is needless, {{processor=script}} can be handy there, along
with existing URPs shipped.
This will mean 3rd party URPs need not to do anything special out of order and
can use the prefix to invoke their respective processor.
> Make every UpdateRequestProcessor available implicitly
> --
>
> Key: SOLR-9565
> URL: https://issues.apache.org/jira/browse/SOLR-9565
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Noble Paul
> Attachments: SOLR-9565.patch
>
>
> Now that we can 'construct' the URP chains through request parameters, we
> should make all the URPs available automatically. The next challenge is to
> make them read the configuration from request parameters as well
> to access {{HTMLStripFieldUpdateProcessorFactory}} the parameter could be
> {{processor=HTMLStripField}} (The UpdateProcessorFactory part is
> automatically appended )
> The next step is to make the URPs accept request parameters instead of just
> configuration parameters e.g:
> {{processor=HTMLStripField=}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-9565) Make every UpdateRequestProcessor available implicitly
[
https://issues.apache.org/jira/browse/SOLR-9565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16050184#comment-16050184
]
Amrit Sarkar edited comment on SOLR-9565 at 6/15/17 8:55 AM:
-
{quote}
this patch introduces a set of predefined set of URPs only (template and
atomic). User defined URPs wil have to be explicitly registered his config API
{quote}
Once this patch is committed, we will try to add the the remaining URPs to the
{{simpler well defined names list}} one by one starting with UUID-URPF:
SOLR-10858.
was (Author: sarkaramr...@gmail.com):
{quote}
this patch introduces a set of predefined set of URPs only (template and
atomic). User defined URPs wil have to be explicitly registered his config API
{quote}
Once this patch is committed, we will try to add the the remaining URPs to the
{{simpler well defined names list}} one by one starting with UUID-URPF.
> Make every UpdateRequestProcessor available implicitly
> --
>
> Key: SOLR-9565
> URL: https://issues.apache.org/jira/browse/SOLR-9565
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Noble Paul
> Attachments: SOLR-9565.patch
>
>
> Now that we can 'construct' the URP chains through request parameters, we
> should make all the URPs available automatically. The next challenge is to
> make them read the configuration from request parameters as well
> to access {{HTMLStripFieldUpdateProcessorFactory}} the parameter could be
> {{processor=HTMLStripField}} (The UpdateProcessorFactory part is
> automatically appended )
> The next step is to make the URPs accept request parameters instead of just
> configuration parameters e.g:
> {{processor=HTMLStripField=}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9565) Make every UpdateRequestProcessor available implicitly
[
https://issues.apache.org/jira/browse/SOLR-9565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16050184#comment-16050184
]
Amrit Sarkar commented on SOLR-9565:
{quote}
this patch introduces a set of predefined set of URPs only (template and
atomic). User defined URPs wil have to be explicitly registered his config API
{quote}
Once this patch is committed, we will try to add the the remaining URPs to the
{{simpler well defined names list}} one by one starting with UUID-URPF.
> Make every UpdateRequestProcessor available implicitly
> --
>
> Key: SOLR-9565
> URL: https://issues.apache.org/jira/browse/SOLR-9565
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Noble Paul
> Attachments: SOLR-9565.patch
>
>
> Now that we can 'construct' the URP chains through request parameters, we
> should make all the URPs available automatically. The next challenge is to
> make them read the configuration from request parameters as well
> to access {{HTMLStripFieldUpdateProcessorFactory}} the parameter could be
> {{processor=HTMLStripField}} (The UpdateProcessorFactory part is
> automatically appended )
> The next step is to make the URPs accept request parameters instead of just
> configuration parameters e.g:
> {{processor=HTMLStripField=}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10858) Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10858:
Attachment: SOLR-10858.patch
Patch updated since SOLR-10876 is resolved. Sample curl as above:
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
--data-binary {"title": "titleA"}
{code}
> Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
> ---
>
> Key: SOLR-10858
> URL: https://issues.apache.org/jira/browse/SOLR-10858
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch,
> SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-10858) Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16047876#comment-16047876
]
Amrit Sarkar edited comment on SOLR-10858 at 6/14/17 10:30 AM:
---
Patch uploaded with prefix for input parameter as X, where X is from
{{XUpdateProcessorFactory}}, eg. {{UUID}} for UUIDUpdateProcessorFactory.
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
--data-binary {"title": "titleA"}
{code}
Will update the description if no objections.
was (Author: sarkaramr...@gmail.com):
Patch uploaded with prefix for input parameter as X, where X is from
{{XUUIDUpdateProcessorFactory}}, eg. {{UUID}} for UUIDUpdateProcessorFactory.
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
--data-binary {"title": "titleA"}
{code}
Will update the description if no objections.
> Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
> ---
>
> Key: SOLR-10858
> URL: https://issues.apache.org/jira/browse/SOLR-10858
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch,
> SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10865) parameter processor=Template doesn't invoke TemplateUpdateProcessorFactory.
[
https://issues.apache.org/jira/browse/SOLR-10865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048896#comment-16048896
]
Amrit Sarkar commented on SOLR-10865:
-
Resolving this as SOLR-10876 is resolved.
> parameter processor=Template doesn't invoke TemplateUpdateProcessorFactory.
> ---
>
> Key: SOLR-10865
> URL: https://issues.apache.org/jira/browse/SOLR-10865
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Affects Versions: master (7.0)
>Reporter: Amrit Sarkar
> Attachments: SOLR-10865.patch, SOLR-10865.patch, SOLR-10865.patch
>
>
> Ref docs::
> {quote}
> The TemplateUpdateProcessorFactory can be used to add new fields to documents
> based on a template pattern.
> This can be used directly in a request without any configuration. To enable
> this processor, use the parameter processor=Template.
> {quote}
> Sample curl::
> {code}
> curl -X POST -H 'Content-Type:application/json'
> 'http://localhost:8983/solr/gettingstarted/update/json/docs?processor=Template=fullName_s:AmritSarkar=true'
> --data-binary '{"id": 1,"title": "titleA"}'
> {code}
> I am receiving exception::
> {code}
>
>
> 400 name="QTime">3 name="error-class">org.apache.solr.common.SolrException name="root-error-class">org.apache.solr.common.SolrException name="msg">No such processor Template400
>
> {code}
> {code}
> ERROR - 2017-06-10 07:39:51.598; [c:gettingstarted s:shard2 r:core_node1
> x:gettingstarted_shard2_replica_n1] org.apache.solr.common.SolrException;
> org.apache.solr.common.SolrException: No such processor Template
> at
> org.apache.solr.update.processor.UpdateRequestProcessorChain.getReqProcessors(UpdateRequestProcessorChain.java:286)
> at
> org.apache.solr.update.processor.UpdateRequestProcessorChain.constructChain(UpdateRequestProcessorChain.java:235)
> {code}
> I looked into how the testclass has been created::
> TemplateUpdateProcessorTest::
> {code}
> public void testSimple() throws Exception {
> AddUpdateCommand cmd = new AddUpdateCommand(new
> LocalSolrQueryRequest(null,
> new ModifiableSolrParams()
> .add("processor", "Template")
> .add("Template.field", "id:${firstName}_${lastName}")
> .add("Template.field", "another:${lastName}_${firstName}")
> .add("Template.field", "missing:${lastName}_${unKnown}")
> ));
> cmd.solrDoc = new SolrInputDocument();
> cmd.solrDoc.addField("firstName", "Tom");
> cmd.solrDoc.addField("lastName", "Cruise");
> new TemplateUpdateProcessorFactory().getInstance(cmd.getReq(), new
> SolrQueryResponse(), null).processAdd(cmd);
> assertEquals("Tom_Cruise", cmd.solrDoc.getFieldValue("id"));
> assertEquals("Cruise_Tom", cmd.solrDoc.getFieldValue("another"));
> assertEquals("Cruise_", cmd.solrDoc.getFieldValue("missing"));
> }
> {code}
> *There is no test to check whether _processor=Template_ works or not*,
> TemplateUpdateProcessorFactory() object is EXPLICITLY created to getInstance
> and do processing on the updates.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-10865) parameter processor=Template doesn't invoke TemplateUpdateProcessorFactory.
[
https://issues.apache.org/jira/browse/SOLR-10865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar resolved SOLR-10865.
-
Resolution: Fixed
> parameter processor=Template doesn't invoke TemplateUpdateProcessorFactory.
> ---
>
> Key: SOLR-10865
> URL: https://issues.apache.org/jira/browse/SOLR-10865
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Affects Versions: master (7.0)
>Reporter: Amrit Sarkar
> Attachments: SOLR-10865.patch, SOLR-10865.patch, SOLR-10865.patch
>
>
> Ref docs::
> {quote}
> The TemplateUpdateProcessorFactory can be used to add new fields to documents
> based on a template pattern.
> This can be used directly in a request without any configuration. To enable
> this processor, use the parameter processor=Template.
> {quote}
> Sample curl::
> {code}
> curl -X POST -H 'Content-Type:application/json'
> 'http://localhost:8983/solr/gettingstarted/update/json/docs?processor=Template=fullName_s:AmritSarkar=true'
> --data-binary '{"id": 1,"title": "titleA"}'
> {code}
> I am receiving exception::
> {code}
>
>
> 400 name="QTime">3 name="error-class">org.apache.solr.common.SolrException name="root-error-class">org.apache.solr.common.SolrException name="msg">No such processor Template400
>
> {code}
> {code}
> ERROR - 2017-06-10 07:39:51.598; [c:gettingstarted s:shard2 r:core_node1
> x:gettingstarted_shard2_replica_n1] org.apache.solr.common.SolrException;
> org.apache.solr.common.SolrException: No such processor Template
> at
> org.apache.solr.update.processor.UpdateRequestProcessorChain.getReqProcessors(UpdateRequestProcessorChain.java:286)
> at
> org.apache.solr.update.processor.UpdateRequestProcessorChain.constructChain(UpdateRequestProcessorChain.java:235)
> {code}
> I looked into how the testclass has been created::
> TemplateUpdateProcessorTest::
> {code}
> public void testSimple() throws Exception {
> AddUpdateCommand cmd = new AddUpdateCommand(new
> LocalSolrQueryRequest(null,
> new ModifiableSolrParams()
> .add("processor", "Template")
> .add("Template.field", "id:${firstName}_${lastName}")
> .add("Template.field", "another:${lastName}_${firstName}")
> .add("Template.field", "missing:${lastName}_${unKnown}")
> ));
> cmd.solrDoc = new SolrInputDocument();
> cmd.solrDoc.addField("firstName", "Tom");
> cmd.solrDoc.addField("lastName", "Cruise");
> new TemplateUpdateProcessorFactory().getInstance(cmd.getReq(), new
> SolrQueryResponse(), null).processAdd(cmd);
> assertEquals("Tom_Cruise", cmd.solrDoc.getFieldValue("id"));
> assertEquals("Cruise_Tom", cmd.solrDoc.getFieldValue("another"));
> assertEquals("Cruise_", cmd.solrDoc.getFieldValue("missing"));
> }
> {code}
> *There is no test to check whether _processor=Template_ works or not*,
> TemplateUpdateProcessorFactory() object is EXPLICITLY created to getInstance
> and do processing on the updates.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10866) Make TimestampUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10866:
Issue Type: Sub-task (was: Improvement)
Parent: SOLR-9565
> Make TimestampUpdateProcessorFactory as Runtime URP; take params(s) with
> request
>
>
> Key: SOLR-10866
> URL: https://issues.apache.org/jira/browse/SOLR-10866
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10866.patch, SOLR-10866.patch, SOLR-10866.patch,
> SOLR-10866.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> TimestampUpdateProcessorFactory will be able to execute by sample curl like
> below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=Timestamp=timestamp_tdt=true
> --data-binary { "id" : "1" , "title_s" : "titleA" }
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10867) Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10867:
Issue Type: Sub-task (was: Improvement)
Parent: SOLR-9565
> Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with
> request
> -
>
> Key: SOLR-10867
> URL: https://issues.apache.org/jira/browse/SOLR-10867
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: error_test, SOLR-10867.patch, SOLR-10867.patch,
> SOLR-10867.patch, SOLR-10867.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> ClassificationUpdateProcessorFactory will be able to execute by sample curl
> like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=Classification=url_s=training=true
> --data-binary { "id" : "1" , "url_s" : "http://www.example.com/subroot; }
> {code}
> All the param(s) for this URP available can be passed as request handler
> param(s).
> Configuration for ClassificationUpdateProcessorFactory in solrconfig.xml is
> optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10869) Make StatelessScriptUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10869:
Issue Type: Sub-task (was: Improvement)
Parent: SOLR-9565
> Make StatelessScriptUpdateProcessorFactory as Runtime URP; take params(s)
> with request
> --
>
> Key: SOLR-10869
> URL: https://issues.apache.org/jira/browse/SOLR-10869
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> StatelessScriptUpdateProcessorFactory will be able to execute by sample curl
> like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=StatelessScript=1.js=2.js=3.js=keyA:valueA=keyB:valueB=keyC:valueC=
> rhino=true --data-binary { "id" : "1" , "title_s" : "title_random" }
> {code}
> All the param(s) for this URP available can be passed as request handler
> param(s). The scripts will be executed in the order the parameters are
> received.
> Configuration for StatelessScriptUpdateProcessorFactory in solrconfig.xml is
> optional. Backcompat is intact.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10858) Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10858:
Issue Type: Sub-task (was: Improvement)
Parent: SOLR-9565
> Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
> ---
>
> Key: SOLR-10858
> URL: https://issues.apache.org/jira/browse/SOLR-10858
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch,
> SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10859) Make URLClassifyProcessor as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10859:
Issue Type: Sub-task (was: Improvement)
Parent: SOLR-9565
> Make URLClassifyProcessor as Runtime URP; take params(s) with request
> -
>
> Key: SOLR-10859
> URL: https://issues.apache.org/jira/browse/SOLR-10859
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10859.patch, SOLR-10859.patch, SOLR-10859.patch,
> SOLR-10859.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> URLClassifyProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=URLClassify=fieldA=fieldB=fieldC=fieldD=fieldE=fieldF=fieldG=true
> --data-binary {"id" : "1", "fieldA" : "valueA"}
> {code}
> Configuration for URLClassifyProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10866) Make TimestampUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10866:
Attachment: SOLR-10866.patch
> Make TimestampUpdateProcessorFactory as Runtime URP; take params(s) with
> request
>
>
> Key: SOLR-10866
> URL: https://issues.apache.org/jira/browse/SOLR-10866
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10866.patch, SOLR-10866.patch, SOLR-10866.patch,
> SOLR-10866.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> TimestampUpdateProcessorFactory will be able to execute by sample curl like
> below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=Timestamp=timestamp_tdt=true
> --data-binary { "id" : "1" , "title_s" : "titleA" }
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10859) Make URLClassifyProcessor as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10859:
Attachment: SOLR-10859.patch
> Make URLClassifyProcessor as Runtime URP; take params(s) with request
> -
>
> Key: SOLR-10859
> URL: https://issues.apache.org/jira/browse/SOLR-10859
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10859.patch, SOLR-10859.patch, SOLR-10859.patch,
> SOLR-10859.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> URLClassifyProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=URLClassify=fieldA=fieldB=fieldC=fieldD=fieldE=fieldF=fieldG=true
> --data-binary {"id" : "1", "fieldA" : "valueA"}
> {code}
> Configuration for URLClassifyProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10859) Make URLClassifyProcessor as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048302#comment-16048302
]
Amrit Sarkar commented on SOLR-10859:
-
In order to make this work, we have to rename the processor factory from:
{{URLClassifyProcessorFactory}} to {{URLClassifyUpdateProcessorFactory}}.
> Make URLClassifyProcessor as Runtime URP; take params(s) with request
> -
>
> Key: SOLR-10859
> URL: https://issues.apache.org/jira/browse/SOLR-10859
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10859.patch, SOLR-10859.patch, SOLR-10859.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> URLClassifyProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=URLClassify=fieldA=fieldB=fieldC=fieldD=fieldE=fieldF=fieldG=true
> --data-binary {"id" : "1", "fieldA" : "valueA"}
> {code}
> Configuration for URLClassifyProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10858) Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10858:
Attachment: SOLR-10858.patch
> Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
> ---
>
> Key: SOLR-10858
> URL: https://issues.apache.org/jira/browse/SOLR-10858
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch,
> SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10858) Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10858:
Attachment: (was: SOLR-10858.patch)
> Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
> ---
>
> Key: SOLR-10858
> URL: https://issues.apache.org/jira/browse/SOLR-10858
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch,
> SOLR-10858.patch, SOLR-10858.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10858) Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10858:
Attachment: SOLR-10858.patch
Patch uploaded with prefix for input parameter as X, where X is from
{{XUUIDUpdateProcessorFactory}}, e,g. {{UUID}} for UUIDUpdateProcessorFactory.
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
--data-binary {"title": "titleA"}
{code}
Will update the description if no objections.
> Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
> ---
>
> Key: SOLR-10858
> URL: https://issues.apache.org/jira/browse/SOLR-10858
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch,
> SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-10858) Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16047876#comment-16047876
]
Amrit Sarkar edited comment on SOLR-10858 at 6/13/17 1:17 PM:
--
Patch uploaded with prefix for input parameter as X, where X is from
{{XUUIDUpdateProcessorFactory}}, eg. {{UUID}} for UUIDUpdateProcessorFactory.
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
--data-binary {"title": "titleA"}
{code}
Will update the description if no objections.
was (Author: sarkaramr...@gmail.com):
Patch uploaded with prefix for input parameter as X, where X is from
{{XUUIDUpdateProcessorFactory}}, e,g. {{UUID}} for UUIDUpdateProcessorFactory.
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
--data-binary {"title": "titleA"}
{code}
Will update the description if no objections.
> Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
> ---
>
> Key: SOLR-10858
> URL: https://issues.apache.org/jira/browse/SOLR-10858
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch,
> SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10858) Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10858:
Attachment: SOLR-10858.patch
Improved tests and URPChain plugin fixed. Patch uploaded.
> Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
> ---
>
> Key: SOLR-10858
> URL: https://issues.apache.org/jira/browse/SOLR-10858
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch,
> SOLR-10858.patch, SOLR-10858.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10859) Make URLClassifyProcessor as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10859:
Attachment: SOLR-10859.patch
Patch uploaded. Test added. Comments added.
> Make URLClassifyProcessor as Runtime URP; take params(s) with request
> -
>
> Key: SOLR-10859
> URL: https://issues.apache.org/jira/browse/SOLR-10859
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10859.patch, SOLR-10859.patch, SOLR-10859.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> URLClassifyProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=URLClassify=fieldA=fieldB=fieldC=fieldD=fieldE=fieldF=fieldG=true
> --data-binary {"id" : "1", "fieldA" : "valueA"}
> {code}
> Configuration for URLClassifyProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10865) parameter processor=Template doesn't invoke TemplateUpdateProcessorFactory.
[
https://issues.apache.org/jira/browse/SOLR-10865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10865:
Attachment: SOLR-10865.patch
The URPFactories are not allowed to Lazy Initialized and hence inform(Solrcore
...) never gets called for URPs which are not defined in SolrConfig. The one
mentioned in config, get through their inform(...) at core creation.
If we are making URPs free of config, we need to allow processors to be
initialized lazy (after core creation) and instead of using flag runtimeLib
flag to get LazyPluginHolder, we can pass _startup=true_, justifiable and neat.
Attached patch for the same. Some minor corrections, blank spaces / lines here
and there.
> parameter processor=Template doesn't invoke TemplateUpdateProcessorFactory.
> ---
>
> Key: SOLR-10865
> URL: https://issues.apache.org/jira/browse/SOLR-10865
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Affects Versions: master (7.0)
>Reporter: Amrit Sarkar
> Attachments: SOLR-10865.patch, SOLR-10865.patch, SOLR-10865.patch
>
>
> Ref docs::
> {quote}
> The TemplateUpdateProcessorFactory can be used to add new fields to documents
> based on a template pattern.
> This can be used directly in a request without any configuration. To enable
> this processor, use the parameter processor=Template.
> {quote}
> Sample curl::
> {code}
> curl -X POST -H 'Content-Type:application/json'
> 'http://localhost:8983/solr/gettingstarted/update/json/docs?processor=Template=fullName_s:AmritSarkar=true'
> --data-binary '{"id": 1,"title": "titleA"}'
> {code}
> I am receiving exception::
> {code}
>
>
> 400 name="QTime">3 name="error-class">org.apache.solr.common.SolrException name="root-error-class">org.apache.solr.common.SolrException name="msg">No such processor Template400
>
> {code}
> {code}
> ERROR - 2017-06-10 07:39:51.598; [c:gettingstarted s:shard2 r:core_node1
> x:gettingstarted_shard2_replica_n1] org.apache.solr.common.SolrException;
> org.apache.solr.common.SolrException: No such processor Template
> at
> org.apache.solr.update.processor.UpdateRequestProcessorChain.getReqProcessors(UpdateRequestProcessorChain.java:286)
> at
> org.apache.solr.update.processor.UpdateRequestProcessorChain.constructChain(UpdateRequestProcessorChain.java:235)
> {code}
> I looked into how the testclass has been created::
> TemplateUpdateProcessorTest::
> {code}
> public void testSimple() throws Exception {
> AddUpdateCommand cmd = new AddUpdateCommand(new
> LocalSolrQueryRequest(null,
> new ModifiableSolrParams()
> .add("processor", "Template")
> .add("Template.field", "id:${firstName}_${lastName}")
> .add("Template.field", "another:${lastName}_${firstName}")
> .add("Template.field", "missing:${lastName}_${unKnown}")
> ));
> cmd.solrDoc = new SolrInputDocument();
> cmd.solrDoc.addField("firstName", "Tom");
> cmd.solrDoc.addField("lastName", "Cruise");
> new TemplateUpdateProcessorFactory().getInstance(cmd.getReq(), new
> SolrQueryResponse(), null).processAdd(cmd);
> assertEquals("Tom_Cruise", cmd.solrDoc.getFieldValue("id"));
> assertEquals("Cruise_Tom", cmd.solrDoc.getFieldValue("another"));
> assertEquals("Cruise_", cmd.solrDoc.getFieldValue("missing"));
> }
> {code}
> *There is no test to check whether _processor=Template_ works or not*,
> TemplateUpdateProcessorFactory() object is EXPLICITLY created to getInstance
> and do processing on the updates.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10865) parameter processor=Template doesn't invoke TemplateUpdateProcessorFactory.
[
https://issues.apache.org/jira/browse/SOLR-10865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16046648#comment-16046648
]
Amrit Sarkar commented on SOLR-10865:
-
[~shalinmangar] agreed. This is not working after *SOLR-9530* where certain
adjustments made in *UpdateRequestProcessorChain.java* to make sure inform(...)
works for AtomicURP (related to _LazyPluginHolder_ in PluginBag.java_). I
didn't make the changes myself but I see what are you saying and we may have
gone wrong there.
patch snippet from SOLR-9530 =>
{code}
diff --git
a/solr/core/src/java/org/apache/solr/update/processor/UpdateRequestProcessorChain.java
b/solr/core/src/java/org/apache/solr/update/processor/UpdateRequestProcessorChain.java
index 0ed626cb0f..05d1a5ae3f 100644
---
a/solr/core/src/java/org/apache/solr/update/processor/UpdateRequestProcessorChain.java
+++
b/solr/core/src/java/org/apache/solr/update/processor/UpdateRequestProcessorChain.java
@@ -28,6 +28,8 @@ import org.apache.solr.common.params.MapSolrParams;
import org.apache.solr.common.params.SolrParams;
import org.apache.solr.common.util.NamedList;
import org.apache.solr.common.util.StrUtils;
+import org.apache.solr.common.util.Utils;
+import org.apache.solr.core.PluginBag;
import org.apache.solr.core.PluginInfo;
import org.apache.solr.core.SolrCore;
import org.apache.solr.request.SolrQueryRequest;
@@ -271,9 +273,13 @@ public final class UpdateRequestProcessorChain implements
PluginInfoInitialized
UpdateRequestProcessorFactory p = core.getUpdateProcessors().get(s);
if (p == null) {
try {
- p = core.createInstance(s + "UpdateProcessorFactory",
UpdateRequestProcessorFactory.class,
- "updateProcessor", null, core.getMemClassLoader());
- core.getUpdateProcessors().put(s, p);
+ PluginInfo pluginInfo = new PluginInfo("updateProcessor",
+ Utils.makeMap("name", s,
+ "class", s + "UpdateProcessorFactory",
+ "runtimeLib", "true"));
+
+ PluginBag.PluginHolder pluginHolder =
core.getUpdateProcessors().createPlugin(pluginInfo);
+ core.getUpdateProcessors().put(s, p = pluginHolder.get());
} catch (SolrException e) {
}
if (p == null)
{code}
I don't recall the exact reason behind this, but will surely look into it.
> parameter processor=Template doesn't invoke TemplateUpdateProcessorFactory.
> ---
>
> Key: SOLR-10865
> URL: https://issues.apache.org/jira/browse/SOLR-10865
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Affects Versions: master (7.0)
>Reporter: Amrit Sarkar
> Attachments: SOLR-10865.patch, SOLR-10865.patch
>
>
> Ref docs::
> {quote}
> The TemplateUpdateProcessorFactory can be used to add new fields to documents
> based on a template pattern.
> This can be used directly in a request without any configuration. To enable
> this processor, use the parameter processor=Template.
> {quote}
> Sample curl::
> {code}
> curl -X POST -H 'Content-Type:application/json'
> 'http://localhost:8983/solr/gettingstarted/update/json/docs?processor=Template=fullName_s:AmritSarkar=true'
> --data-binary '{"id": 1,"title": "titleA"}'
> {code}
> I am receiving exception::
> {code}
>
>
> 400 name="QTime">3 name="error-class">org.apache.solr.common.SolrException name="root-error-class">org.apache.solr.common.SolrException name="msg">No such processor Template400
>
> {code}
> {code}
> ERROR - 2017-06-10 07:39:51.598; [c:gettingstarted s:shard2 r:core_node1
> x:gettingstarted_shard2_replica_n1] org.apache.solr.common.SolrException;
> org.apache.solr.common.SolrException: No such processor Template
> at
> org.apache.solr.update.processor.UpdateRequestProcessorChain.getReqProcessors(UpdateRequestProcessorChain.java:286)
> at
> org.apache.solr.update.processor.UpdateRequestProcessorChain.constructChain(UpdateRequestProcessorChain.java:235)
> {code}
> I looked into how the testclass has been created::
> TemplateUpdateProcessorTest::
> {code}
> public void testSimple() throws Exception {
> AddUpdateCommand cmd = new AddUpdateCommand(new
> LocalSolrQueryRequest(null,
> new ModifiableSolrParams()
> .add("processor", "Template")
> .add("Template.field", "id:${firstName}_${lastName}")
> .add("Template.field", "another:${lastName}_${firstName}")
> .add("Template.field", "missing:${lastName}_${unKnown}")
> ));
> cmd.solrDoc = new SolrInputDocument();
> cmd.solrDoc.addField("firstName", "Tom");
> cmd.solrDoc.addField("lastName", "Cruise");
> new TemplateUpdateProcessorFactory().getInstance(cmd.getReq(), new
> SolrQueryResponse(),
[jira] [Comment Edited] (SOLR-10867) Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16046599#comment-16046599
]
Amrit Sarkar edited comment on SOLR-10867 at 6/12/17 2:11 PM:
--
Shalin,
Sorry I uploaded old test-error-log and patch, uploaded new ones, object leaks
etc. I will take some more time to understand what different I am doing from
rest of the test cases in the project. Thank you for your help.
{code}
[junit4] 2> 28893 ERROR (coreCloseExecutor-12-thread-1) [
x:collection1] o.a.s.c.CachingDirectoryFactory Timeout waiting for all
directory ref counts to be released - gave up waiting on
[jira] [Updated] (SOLR-10867) Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10867:
Attachment: (was: error)
> Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with
> request
> -
>
> Key: SOLR-10867
> URL: https://issues.apache.org/jira/browse/SOLR-10867
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: error_test, SOLR-10867.patch, SOLR-10867.patch,
> SOLR-10867.patch, SOLR-10867.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> ClassificationUpdateProcessorFactory will be able to execute by sample curl
> like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=Classification=url_s=training=true
> --data-binary { "id" : "1" , "url_s" : "http://www.example.com/subroot; }
> {code}
> All the param(s) for this URP available can be passed as request handler
> param(s).
> Configuration for ClassificationUpdateProcessorFactory in solrconfig.xml is
> optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10867) Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10867:
Attachment: SOLR-10867.patch
error_test
Shalin,
Sorry I uploaded old test-error-log and patch, uploaded new ones, object leaks
etc. I will take some more time to understand what different I am doing from
rest of the test cases in the project. Thank you for your help.
> Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with
> request
> -
>
> Key: SOLR-10867
> URL: https://issues.apache.org/jira/browse/SOLR-10867
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: error, error_test, SOLR-10867.patch, SOLR-10867.patch,
> SOLR-10867.patch, SOLR-10867.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> ClassificationUpdateProcessorFactory will be able to execute by sample curl
> like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=Classification=url_s=training=true
> --data-binary { "id" : "1" , "url_s" : "http://www.example.com/subroot; }
> {code}
> All the param(s) for this URP available can be passed as request handler
> param(s).
> Configuration for ClassificationUpdateProcessorFactory in solrconfig.xml is
> optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10867) Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10867:
Attachment: SOLR-10867.patch
error
Thanks [~shalinmangar] for the correction, I have to be more careful, Well
that rectified thread leaks rightfully as only one core is being created in the
main thread, the temp index dir is still creating issue while getting removed
at core destruction.
Update patch and error:
{code}
[junit4] 2> 28935 ERROR (coreCloseExecutor-12-thread-1) [x:collection1]
o.a.s.c.CachingDirectoryFactory Timeout waiting for all directory ref counts to
be released - gave up waiting on
[jira] [Updated] (SOLR-10865) parameter processor=Template doesn't invoke TemplateUpdateProcessorFactory.
[
https://issues.apache.org/jira/browse/SOLR-10865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10865:
Attachment: SOLR-10865.patch
Also it would be best if we specify this bit of information =>
enable *enable.runtime.lib* while starting up solr nodes to support:
processor=X to invoke XUpdateRequestProcessor.
Patch uploaded with slight adjustments.
> parameter processor=Template doesn't invoke TemplateUpdateProcessorFactory.
> ---
>
> Key: SOLR-10865
> URL: https://issues.apache.org/jira/browse/SOLR-10865
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Affects Versions: master (7.0)
>Reporter: Amrit Sarkar
> Attachments: SOLR-10865.patch, SOLR-10865.patch
>
>
> Ref docs::
> {quote}
> The TemplateUpdateProcessorFactory can be used to add new fields to documents
> based on a template pattern.
> This can be used directly in a request without any configuration. To enable
> this processor, use the parameter processor=Template.
> {quote}
> Sample curl::
> {code}
> curl -X POST -H 'Content-Type:application/json'
> 'http://localhost:8983/solr/gettingstarted/update/json/docs?processor=Template=fullName_s:AmritSarkar=true'
> --data-binary '{"id": 1,"title": "titleA"}'
> {code}
> I am receiving exception::
> {code}
>
>
> 400 name="QTime">3 name="error-class">org.apache.solr.common.SolrException name="root-error-class">org.apache.solr.common.SolrException name="msg">No such processor Template400
>
> {code}
> {code}
> ERROR - 2017-06-10 07:39:51.598; [c:gettingstarted s:shard2 r:core_node1
> x:gettingstarted_shard2_replica_n1] org.apache.solr.common.SolrException;
> org.apache.solr.common.SolrException: No such processor Template
> at
> org.apache.solr.update.processor.UpdateRequestProcessorChain.getReqProcessors(UpdateRequestProcessorChain.java:286)
> at
> org.apache.solr.update.processor.UpdateRequestProcessorChain.constructChain(UpdateRequestProcessorChain.java:235)
> {code}
> I looked into how the testclass has been created::
> TemplateUpdateProcessorTest::
> {code}
> public void testSimple() throws Exception {
> AddUpdateCommand cmd = new AddUpdateCommand(new
> LocalSolrQueryRequest(null,
> new ModifiableSolrParams()
> .add("processor", "Template")
> .add("Template.field", "id:${firstName}_${lastName}")
> .add("Template.field", "another:${lastName}_${firstName}")
> .add("Template.field", "missing:${lastName}_${unKnown}")
> ));
> cmd.solrDoc = new SolrInputDocument();
> cmd.solrDoc.addField("firstName", "Tom");
> cmd.solrDoc.addField("lastName", "Cruise");
> new TemplateUpdateProcessorFactory().getInstance(cmd.getReq(), new
> SolrQueryResponse(), null).processAdd(cmd);
> assertEquals("Tom_Cruise", cmd.solrDoc.getFieldValue("id"));
> assertEquals("Cruise_Tom", cmd.solrDoc.getFieldValue("another"));
> assertEquals("Cruise_", cmd.solrDoc.getFieldValue("missing"));
> }
> {code}
> *There is no test to check whether _processor=Template_ works or not*,
> TemplateUpdateProcessorFactory() object is EXPLICITLY created to getInstance
> and do processing on the updates.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10865) parameter processor=Template doesn't invoke TemplateUpdateProcessorFactory.
[
https://issues.apache.org/jira/browse/SOLR-10865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10865:
Attachment: SOLR-10865.patch
Figured out what the problem is =>
To leverage the _processor=Template_ or _processor=Atomic_ or _processor = X_,
where X + "UpdateRequestProcessor", must and must have system property
*-Denable.runtime.lib=true*
Now when we start a SolrCloud with default settings, I don't think we have
*enable.runtime.lib* enabled in our configs. And the error we receive at the
end of the request is:
{code}
2017-06-12 11:05:55.517 WARN (qtp846947180-13) [c:gettingstarted s:shard1
r:core_node1 x:gettingstarted_shard1_replica_n1] o.a.s.c.PluginBag runtime
library loading is not enabled, start Solr with -Denable.runtime.lib=true
org.apache.solr.common.SolrException; org.apache.solr.common.SolrException: No
such processor Template
{code}
absolutely misleading. I guess improving the error message will make the user
make the necessary changes to make it work. A nested message is better to fix
this.
Patch attached with appropriate error or exception, which will help one figure
out what to fix, where to fix when we see "No processor".
> parameter processor=Template doesn't invoke TemplateUpdateProcessorFactory.
> ---
>
> Key: SOLR-10865
> URL: https://issues.apache.org/jira/browse/SOLR-10865
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Affects Versions: master (7.0)
>Reporter: Amrit Sarkar
> Attachments: SOLR-10865.patch
>
>
> Ref docs::
> {quote}
> The TemplateUpdateProcessorFactory can be used to add new fields to documents
> based on a template pattern.
> This can be used directly in a request without any configuration. To enable
> this processor, use the parameter processor=Template.
> {quote}
> Sample curl::
> {code}
> curl -X POST -H 'Content-Type:application/json'
> 'http://localhost:8983/solr/gettingstarted/update/json/docs?processor=Template=fullName_s:AmritSarkar=true'
> --data-binary '{"id": 1,"title": "titleA"}'
> {code}
> I am receiving exception::
> {code}
>
>
> 400 name="QTime">3 name="error-class">org.apache.solr.common.SolrException name="root-error-class">org.apache.solr.common.SolrException name="msg">No such processor Template400
>
> {code}
> {code}
> ERROR - 2017-06-10 07:39:51.598; [c:gettingstarted s:shard2 r:core_node1
> x:gettingstarted_shard2_replica_n1] org.apache.solr.common.SolrException;
> org.apache.solr.common.SolrException: No such processor Template
> at
> org.apache.solr.update.processor.UpdateRequestProcessorChain.getReqProcessors(UpdateRequestProcessorChain.java:286)
> at
> org.apache.solr.update.processor.UpdateRequestProcessorChain.constructChain(UpdateRequestProcessorChain.java:235)
> {code}
> I looked into how the testclass has been created::
> TemplateUpdateProcessorTest::
> {code}
> public void testSimple() throws Exception {
> AddUpdateCommand cmd = new AddUpdateCommand(new
> LocalSolrQueryRequest(null,
> new ModifiableSolrParams()
> .add("processor", "Template")
> .add("Template.field", "id:${firstName}_${lastName}")
> .add("Template.field", "another:${lastName}_${firstName}")
> .add("Template.field", "missing:${lastName}_${unKnown}")
> ));
> cmd.solrDoc = new SolrInputDocument();
> cmd.solrDoc.addField("firstName", "Tom");
> cmd.solrDoc.addField("lastName", "Cruise");
> new TemplateUpdateProcessorFactory().getInstance(cmd.getReq(), new
> SolrQueryResponse(), null).processAdd(cmd);
> assertEquals("Tom_Cruise", cmd.solrDoc.getFieldValue("id"));
> assertEquals("Cruise_Tom", cmd.solrDoc.getFieldValue("another"));
> assertEquals("Cruise_", cmd.solrDoc.getFieldValue("missing"));
> }
> {code}
> *There is no test to check whether _processor=Template_ works or not*,
> TemplateUpdateProcessorFactory() object is EXPLICITLY created to getInstance
> and do processing on the updates.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10734) Multithreaded test/support for AtomicURP broken
[
https://issues.apache.org/jira/browse/SOLR-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10734:
Attachment: SOLR-10734.patch
Patch attached with following changes:
* Max Attempts bumped up from 5 to 25.
* Threads lowered to 10 from 100.
As the above theory posted above stands out, the following changes will make
the tests work.
> Multithreaded test/support for AtomicURP broken
> ---
>
> Key: SOLR-10734
> URL: https://issues.apache.org/jira/browse/SOLR-10734
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Ishan Chattopadhyaya
> Attachments: log-snippet, Screen Shot 2017-05-31 at 4.50.23 PM.png,
> SOLR-10734.patch
>
>
> The multithreaded test doesn't actually start the threads, but only invokes
> the run directly. The join afterwards doesn't do anything, hence.
> {code}
> diff --git
> a/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
>
> b/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
> index f3f833d..10b7770 100644
> ---
> a/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
> +++
> b/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
> @@ -238,7 +238,7 @@ public class AtomicUpdateProcessorFactoryTest extends
> SolrTestCaseJ4 {
>}
> }
>};
> - t.run();
> + t.run(); // red flag, shouldn't this be t.start?
>threads.add(t);
>finalCount += index; //int_i
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10858) Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10858:
Attachment: SOLR-10858.patch
> Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
> ---
>
> Key: SOLR-10858
> URL: https://issues.apache.org/jira/browse/SOLR-10858
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch,
> SOLR-10858.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10867) Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10867:
Attachment: SOLR-10867.patch
Got everything in order, revamped all the tests but running into temp_index &
cache directories error while deleting and thread leaks all over the place.
error attached ::
{code}
[junit4] 2> 40694 INFO (coreCloseExecutor-34-thread-1) [
x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore
org.apache.solr.core.SolrCore@3db54c55
[junit4] 2> 40705 INFO (coreCloseExecutor-34-thread-1) [
x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for
registry=solr.core.collection1, tag=1035291733
[junit4] 2> 53230 ERROR (coreCloseExecutor-34-thread-1) [
x:collection1] o.a.s.c.CachingDirectoryFactory Timeout waiting for all
directory ref counts to be released - gave up waiting on
[jira] [Updated] (SOLR-10866) Make TimestampUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10866:
Attachment: SOLR-10866.patch
> Make TimestampUpdateProcessorFactory as Runtime URP; take params(s) with
> request
>
>
> Key: SOLR-10866
> URL: https://issues.apache.org/jira/browse/SOLR-10866
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10866.patch, SOLR-10866.patch, SOLR-10866.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> TimestampUpdateProcessorFactory will be able to execute by sample curl like
> below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=Timestamp=timestamp_tdt=true
> --data-binary { "id" : "1" , "title_s" : "titleA" }
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10858) Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10858:
Attachment: SOLR-10858.patch
Support multiple instances of processor with different configurations from
single factory object.
> Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
> ---
>
> Key: SOLR-10858
> URL: https://issues.apache.org/jira/browse/SOLR-10858
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10859) Make URLClassifyProcessor as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10859:
Attachment: SOLR-10859.patch
Updated patch with first-level tests.
> Make URLClassifyProcessor as Runtime URP; take params(s) with request
> -
>
> Key: SOLR-10859
> URL: https://issues.apache.org/jira/browse/SOLR-10859
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10859.patch, SOLR-10859.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> URLClassifyProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=URLClassify=fieldA=fieldB=fieldC=fieldD=fieldE=fieldF=fieldG=true
> --data-binary {"id" : "1", "fieldA" : "valueA"}
> {code}
> Configuration for URLClassifyProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10869) Make StatelessScriptUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045921#comment-16045921
]
Amrit Sarkar commented on SOLR-10869:
-
Thanks [~erikhatcher]
{quote}
I think the parameter prefix is best as "script" rather than "statelessscript".
"script.script" is funny though, so maybe it should be "script.file"?
{quote}
Fair. "script.file", "script.engine" and "script.params" seems neat. I will go
forward with this for now.
> Make StatelessScriptUpdateProcessorFactory as Runtime URP; take params(s)
> with request
> --
>
> Key: SOLR-10869
> URL: https://issues.apache.org/jira/browse/SOLR-10869
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> StatelessScriptUpdateProcessorFactory will be able to execute by sample curl
> like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=StatelessScript=1.js=2.js=3.js=keyA:valueA=keyB:valueB=keyC:valueC=
> rhino=true --data-binary { "id" : "1" , "title_s" : "title_random" }
> {code}
> All the param(s) for this URP available can be passed as request handler
> param(s). The scripts will be executed in the order the parameters are
> received.
> Configuration for StatelessScriptUpdateProcessorFactory in solrconfig.xml is
> optional. Backcompat is intact.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10869) Make StatelessScriptUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10869:
Description:
We are trying to get rid of processor definitions in SolrConfig for all URPs
and take parameters in the request itself.
StatelessScriptUpdateProcessorFactory will be able to execute by sample curl
like below:
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=StatelessScript=1.js=2.js=3.js=keyA:valueA=keyB:valueB=keyC:valueC=
rhino=true --data-binary { "id" : "1" , "title_s" : "title_random" }
{code}
All the param(s) for this URP available can be passed as request handler
param(s). The scripts will be executed in the order the parameters are received.
Configuration for StatelessScriptUpdateProcessorFactory in solrconfig.xml is
optional. Backcompat is intact.
was:
We are trying to get rid of processor definitions in SolrConfig for all URPs
and take parameters in the request itself.
StatelessScriptUpdateProcessorFactory will be able to execute by sample curl
like below:
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=StatelessScript=1.js=2.js=3.js=keyA:valueA=keyB:valueB=keyC:valueC=true
--data-binary { "id" : "1" , "title_s" : "title_random" }
{code}
All the param(s) for this URP available can be passed as request handler
param(s). The scripts will be executed in the order the parameters are received.
Configuration for StatelessScriptUpdateProcessorFactory in solrconfig.xml is
optional. Backcompat is intact.
> Make StatelessScriptUpdateProcessorFactory as Runtime URP; take params(s)
> with request
> --
>
> Key: SOLR-10869
> URL: https://issues.apache.org/jira/browse/SOLR-10869
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> StatelessScriptUpdateProcessorFactory will be able to execute by sample curl
> like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=StatelessScript=1.js=2.js=3.js=keyA:valueA=keyB:valueB=keyC:valueC=
> rhino=true --data-binary { "id" : "1" , "title_s" : "title_random" }
> {code}
> All the param(s) for this URP available can be passed as request handler
> param(s). The scripts will be executed in the order the parameters are
> received.
> Configuration for StatelessScriptUpdateProcessorFactory in solrconfig.xml is
> optional. Backcompat is intact.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10869) Make StatelessScriptUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045879#comment-16045879
]
Amrit Sarkar commented on SOLR-10869:
-
I have a question on the implementation of this:
There are two instances where we are initialising the script engines
*initEngines(...)*;
1. In the inform(...) via SolrCoreAware (plugin), which will be executed right
after factory.init() with empty request and response =>
{code}
SolrQueryResponse rsp = new SolrQueryResponse();
SolrQueryRequest req = new LocalSolrQueryRequest(core, new
ModifiableSolrParams());
try {
initEngines(req, rsp);
} catch (Exception e) {
String msg = "Unable to initialize scripts: " + e.getMessage();
log.error(msg, e);
throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, msg, e);
} finally {
req.close();
}
{code}
2. and then with request and response in getInstance() again =>
{code}
@Override
public UpdateRequestProcessor getInstance(SolrQueryRequest req,
SolrQueryResponse rsp, UpdateRequestProcessor next) {
List scriptEngines = null;
scriptEngines = initEngines(req, rsp);
return new ScriptUpdateProcessor(req, rsp, scriptEngines, next);
}
{code}
initEngines(...) have =>
{code}
try {
Reader scriptSrc = scriptFile.openReader(resourceLoader);
try {
engine.eval(scriptSrc);
} catch (ScriptException e) {
throw new SolrException(SolrException.ErrorCode.SERVER_ERROR,
"Unable to evaluate script: " +
scriptFile.getFileName(), e);
} finally {
IOUtils.closeQuietly(scriptSrc);
}
}
{code}
So the initEngines(...) is called at inform(...) to validate the scripts at
startup of core and there's no other motive to it?
> Make StatelessScriptUpdateProcessorFactory as Runtime URP; take params(s)
> with request
> --
>
> Key: SOLR-10869
> URL: https://issues.apache.org/jira/browse/SOLR-10869
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> StatelessScriptUpdateProcessorFactory will be able to execute by sample curl
> like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=StatelessScript=1.js=2.js=3.js=keyA:valueA=keyB:valueB=keyC:valueC=true
> --data-binary { "id" : "1" , "title_s" : "title_random" }
> {code}
> All the param(s) for this URP available can be passed as request handler
> param(s). The scripts will be executed in the order the parameters are
> received.
> Configuration for StatelessScriptUpdateProcessorFactory in solrconfig.xml is
> optional. Backcompat is intact.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-10869) Make StatelessScriptUpdateProcessorFactory as Runtime URP; take params(s) with request
Amrit Sarkar created SOLR-10869:
---
Summary: Make StatelessScriptUpdateProcessorFactory as Runtime
URP; take params(s) with request
Key: SOLR-10869
URL: https://issues.apache.org/jira/browse/SOLR-10869
Project: Solr
Issue Type: Improvement
Security Level: Public (Default Security Level. Issues are Public)
Components: update
Reporter: Amrit Sarkar
Priority: Minor
We are trying to get rid of processor definitions in SolrConfig for all URPs
and take parameters in the request itself.
StatelessScriptUpdateProcessorFactory will be able to execute by sample curl
like below:
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=StatelessScript=1.js=2.js=3.js=keyA:valueA=keyB:valueB=keyC:valueC=true
--data-binary { "id" : "1" , "title_s" : "title_random" }
{code}
All the param(s) for this URP available can be passed as request handler
param(s). The scripts will be executed in the order the parameters are received.
Configuration for StatelessScriptUpdateProcessorFactory in solrconfig.xml is
optional. Backcompat is intact.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10867) Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10867:
Description:
We are trying to get rid of processor definitions in SolrConfig for all URPs
and take parameters in the request itself.
ClassificationUpdateProcessorFactory will be able to execute by sample curl
like below:
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=Classification=url_s=training=true
--data-binary { "id" : "1" , "url_s" : "http://www.example.com/subroot; }
{code}
All the param(s) for this URP available can be passed as request handler
param(s).
Configuration for ClassificationUpdateProcessorFactory in solrconfig.xml is
optional.
was:
We are trying to get rid of processor definitions in SolrConfig for all URPs
and take parameters in the request itself.
ClassificationUpdateProcessorFactory will be able to execute by sample curl
like below:
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=Classification=url_s=training=true
--data-binary { "id" : "1" , "url_s" : "http://www.example.com/subroot; }
{code}
All the param(s) for this URP available can be passed as request handler
param(s).
Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
> Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with
> request
> -
>
> Key: SOLR-10867
> URL: https://issues.apache.org/jira/browse/SOLR-10867
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10867.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> ClassificationUpdateProcessorFactory will be able to execute by sample curl
> like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=Classification=url_s=training=true
> --data-binary { "id" : "1" , "url_s" : "http://www.example.com/subroot; }
> {code}
> All the param(s) for this URP available can be passed as request handler
> param(s).
> Configuration for ClassificationUpdateProcessorFactory in solrconfig.xml is
> optional.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10867) Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10867:
Attachment: (was: SOLR-10867.patch)
> Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with
> request
> -
>
> Key: SOLR-10867
> URL: https://issues.apache.org/jira/browse/SOLR-10867
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10867.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> ClassificationUpdateProcessorFactory will be able to execute by sample curl
> like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=Classification=url_s=training=true
> --data-binary { "id" : "1" , "url_s" : "http://www.example.com/subroot; }
> {code}
> All the param(s) for this URP available can be passed as request handler
> param(s).
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10867) Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10867:
Attachment: SOLR-10867.patch
> Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with
> request
> -
>
> Key: SOLR-10867
> URL: https://issues.apache.org/jira/browse/SOLR-10867
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10867.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> ClassificationUpdateProcessorFactory will be able to execute by sample curl
> like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=Classification=url_s=training=true
> --data-binary { "id" : "1" , "url_s" : "http://www.example.com/subroot; }
> {code}
> All the param(s) for this URP available can be passed as request handler
> param(s).
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
