[jira] [Commented] (SOLR-11159) Facet buckets count still incorrect after passing {refine:true} | SOLR-7542
[
https://issues.apache.org/jira/browse/SOLR-11159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103359#comment-16103359
]
Amrit Sarkar commented on SOLR-11159:
-
Some of my observations from debugQuery,
Here's how per shard request is made for various purposes COUNT DEC, LIMIT 2 :
1st request:
{code}
"Response":"{responseHeader={zkConnected=true,status=0,QTime=2826,params={df=_text_,distrib=false,debug=[false,
timing, track],_facet_={},fl=[id,
score],shards.purpose=1048580,start=0,fsv=true,shard.url=http://127.0.0.1:8983/solr/collection1_shard3_replica_n4/,rows=10,rid=127.0.0.1-collection1_shard3_replica_n4-1501147655592-0,version=2,q=*:*,json.facet={cat_s:{type:terms,field:cat_s,sort:\"count
desc\",limit:2,overrequest:0,refine:true}},requestPurpose=GET_TOP_IDS,NOW=1501147655590,isShard=true,wt=javabin,debugQuery=false}},response={numFound=1,start=0,maxScore=1.0,docs=[SolrDocument{id=5,
score=1.0}]},sort_values={},facets={count=1,cat_s={buckets=[{val=C,count=1}]}},debug={facet-trace={processor=FacetQueryProcessor,elapse=2644,query=null,domainSize=1,sub-facet=[{processor=FacetFieldProcessorByArrayDV,elapse=1632,field=cat_s,limit=2,numBuckets=1,domainSize=1}]},json={facet={cat_s={type=terms,
field=cat_s, sort=count desc, limit=2, overrequest=0,
refine=true}}},timing={time=2817.0,prepare={time=7.0,query={time=1.0},facet={time=0.0},facet_module={time=6.0},mlt={time=0.0},highlight={time=0.0},stats={time=0.0},expand={time=0.0},terms={time=0.0},debug={time=0.0}},process={time=2741.0,query={time=0.0},facet={time=0.0},facet_module={time=2665.0},mlt={time=0.0},highlight={time=0.0},stats={time=0.0},expand={time=0.0},terms={time=0.0},debug={time=0.0}"},
{code}
2nd request:
{code}
"Response":"{responseHeader={zkConnected=true,status=0,QTime=2828,params={df=_text_,distrib=false,debug=[false,
timing, track],_facet_={},fl=[id,
score],shards.purpose=1048580,start=0,fsv=true,shard.url=http://127.0.0.1:8983/solr/collection1_shard2_replica_n2/,rows=10,rid=127.0.0.1-collection1_shard3_replica_n4-1501147655592-0,version=2,q=*:*,json.facet={cat_s:{type:terms,field:cat_s,sort:\"count
desc\",limit:2,overrequest:0,refine:true}},requestPurpose=GET_TOP_IDS,NOW=1501147655590,isShard=true,wt=javabin,debugQuery=false}},response={numFound=7,start=0,maxScore=1.0,docs=[SolrDocument{id=2,
score=1.0}, SolrDocument{id=4, score=1.0}, SolrDocument{id=3, score=1.0},
SolrDocument{id=6, score=1.0}, SolrDocument{id=9, score=1.0},
SolrDocument{id=12, score=1.0}, SolrDocument{id=15,
score=1.0}]},sort_values={},facets={count=7,cat_s={buckets=[{val=C,count=4},
{val=A,count=1}]}},debug={facet-trace={processor=FacetQueryProcessor,elapse=2090,query=null,domainSize=7,sub-facet=[{processor=FacetFieldProcessorByArrayDV,elapse=1098,field=cat_s,limit=2,numBuckets=4,domainSize=7}]},json={facet={cat_s={type=terms,
field=cat_s, sort=count desc, limit=2, overrequest=0,
refine=true}}},timing={time=2819.0,prepare={time=619.0,query={time=0.0},facet={time=0.0},facet_module={time=615.0},mlt={time=0.0},highlight={time=0.0},stats={time=0.0},expand={time=0.0},terms={time=0.0},debug={time=0.0}},process={time=2192.0,query={time=20.0},facet={time=0.0},facet_module={time=2095.0},mlt={time=0.0},highlight={time=0.0},stats={time=0.0},expand={time=0.0},terms={time=0.0},debug={time=74.0}"}
{code}
3rd request:
{code}
"Response":"{responseHeader={zkConnected=true,status=0,QTime=3231,params={df=_text_,distrib=false,debug=[false,
timing, track],_facet_={},fl=[id,
score],shards.purpose=1048580,start=0,fsv=true,shard.url=http://127.0.0.1:8983/solr/collection1_shard1_replica_n1/,rows=10,rid=127.0.0.1-collection1_shard3_replica_n4-1501147655592-0,version=2,q=*:*,json.facet={cat_s:{type:terms,field:cat_s,sort:\"count
desc\",limit:2,overrequest:0,refine:true}},requestPurpose=GET_TOP_IDS,NOW=1501147655590,isShard=true,wt=javabin,debugQuery=false}},response={numFound=4,start=0,maxScore=1.0,docs=[SolrDocument{id=1,
score=1.0}, SolrDocument{id=8, score=1.0}, SolrDocument{id=10, score=1.0},
SolrDocument{id=0,
score=1.0}]},sort_values={},facets={count=4,cat_s={buckets=[{val=E,count=2},
{val=A,count=1}]}},debug={facet-trace={processor=FacetQueryProcessor,elapse=1519,query=null,domainSize=4,sub-facet=[{processor=FacetFieldProcessorByArrayDV,elapse=997,field=cat_s,limit=2,numBuckets=3,domainSize=4}]},json={facet={cat_s={type=terms,
field=cat_s, sort=count desc, limit=2, overrequest=0,
refine=true}}},timing={time=2258.0,prepare={time=14.0,query={time=0.0},facet={time=0.0},facet_module={time=5.0},mlt={time=0.0},highlight={time=0.0},stats={time=0.0},expand={time=0.0},terms={time=0.0},debug={time=0.0}},process={time=2162.0,query={time=0.0},facet={time=0.0},facet_module={time=1650.0},mlt={time=1.0},highlight={time=0.0},stats={time=0.0},expand={time=0.0},terms={time=86.0},debug={time=2.0}"}}
{code}
4th request: *REFINE DEF Included* SHARD-3
{code}
[jira] [Commented] (SOLR-11159) Facet buckets count missing after passing {refine:true} | SOLR-7542
[
https://issues.apache.org/jira/browse/SOLR-11159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103402#comment-16103402
]
Amrit Sarkar commented on SOLR-11159:
-
Thank you so much [~ysee...@gmail.com], I was following
{{TestJsonFacetRefinement}} and didn't understand the significance of
{{overrequest}} for refinement, my bad. Thank you for the above explanation
too, made things absolutely clear and I am receiving all buckets with correct
counts positively.
I have corrected the JIRA title and would close this JIRA as "Not a Bug". Thank
you again.
> Facet buckets count missing after passing {refine:true} | SOLR-7542
> ---
>
> Key: SOLR-11159
> URL: https://issues.apache.org/jira/browse/SOLR-11159
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Facet Module
>Reporter: Amrit Sarkar
> Attachments: COUNT_DESC_LIMIT_2, COUNT_DESC_LIMIT_3, DOCS
>
>
> I was experimenting / analysing the new *Refinement* feature in JSON Facet
> Apis introduced in SOLR-7452. Passing {{refine:true}} with the facet
> definition.
> I am listing down the test-scenarios along with test-data:
> 3 sharded collection on 3 nodes
> node/shard: bucketVal - count
> 8987: C - 1
> 8983: C - 4 D - 1 E - 1 A - 1
> 8985: E - 2 A - 1 D - 1
> Total: BUCKETS
> C - 5 E - 3 D - 2 A - 2
> It is giving accurate results for COUNT ASC, LIMIT 1 - 4
> {code}
> curl http://localhost:8983/solr/collection1/select -d
> 'q=*:*={cat_s:{type:terms,field:cat_s,sort:"count
> asc",limit:1,overrequest:0,refine:true}}=json=true'
> {code}
> {code}
> "facets":{
> "count":12,
> "cat_s":{
> "buckets":[{
> "val":"A",
> "count":2}]}}}
> {code}
> {code}
> curl http://localhost:8983/solr/collection1/select -d
> 'q=*:*={cat_s:{type:terms,field:cat_s,sort:"count
> asc",limit:2,overrequest:0,refine:true}}=json=true'
> {code}
> {code}
> "facets":{
> "count":12,
> "cat_s":{
> "buckets":[{
> "val":"A",
> "count":2},
> {
> "val":"D",
> "count":2}]}}}
> {code}
> *BUT, COUNT DESC, LIMIT 2 and 3*
> {code}
> curl http://localhost:8983/solr/collection1/select -d
> 'q=*:*={cat_s:{type:terms,field:cat_s,sort:"count
> desc",limit:2,overrequest:0,refine:true}}=json=true'
> {code}
> {code}
> "facets":{
> "count":12,
> "cat_s":{
> "buckets":[{
> "val":"C",
> "count":5},
> {
> "val":"A",
> "count":2}]}}}
> {code}
> {code}
> curl http://localhost:8983/solr/collection1/select -d
> 'q=*:*={cat_s:{type:terms,field:cat_s,sort:"count
> desc",limit:3,overrequest:0,refine:true}}=json=true'
> {code}
> {code}
> "facets":{
> "count":12,
> "cat_s":{
> "buckets":[{
> "val":"C",
> "count":5},
> {
> "val":"A",
> "count":2},
> {
> "val":"D",
> "count":2}]}}}
> {code}
> *bucketVal {{E}} and its count {{3}} is not in facet response* Pardon me if I
> am missing some configuration or this behavior is right / justified. Ideally
> we should see bucketVal E and its count 3.
> I am attaching Index DOCS, debugQuery for COUNT DESC, LIMIT 2 and LIMIT 3.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-11159) Facet buckets count missing after passing {refine:true} | SOLR-7542
[
https://issues.apache.org/jira/browse/SOLR-11159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103402#comment-16103402
]
Amrit Sarkar edited comment on SOLR-11159 at 7/27/17 4:12 PM:
--
Thank you so much [~ysee...@gmail.com], I was following
{{TestJsonFacetRefinement}} and didn't understand the significance of
{{overrequest}} for refinement, my bad. Thank you for the above explanation
too, made things absolutely clear and I am receiving all buckets with correct
counts positively.
I have corrected the JIRA title and would close this as "Not a Bug". Thank you
again.
was (Author: sarkaramr...@gmail.com):
Thank you so much [~ysee...@gmail.com], I was following
{{TestJsonFacetRefinement}} and didn't understand the significance of
{{overrequest}} for refinement, my bad. Thank you for the above explanation
too, made things absolutely clear and I am receiving all buckets with correct
counts positively.
I have corrected the JIRA title and would close this JIRA as "Not a Bug". Thank
you again.
> Facet buckets count missing after passing {refine:true} | SOLR-7542
> ---
>
> Key: SOLR-11159
> URL: https://issues.apache.org/jira/browse/SOLR-11159
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Facet Module
>Reporter: Amrit Sarkar
> Attachments: COUNT_DESC_LIMIT_2, COUNT_DESC_LIMIT_3, DOCS
>
>
> I was experimenting / analysing the new *Refinement* feature in JSON Facet
> Apis introduced in SOLR-7452. Passing {{refine:true}} with the facet
> definition.
> I am listing down the test-scenarios along with test-data:
> 3 sharded collection on 3 nodes
> node/shard: bucketVal - count
> 8987: C - 1
> 8983: C - 4 D - 1 E - 1 A - 1
> 8985: E - 2 A - 1 D - 1
> Total: BUCKETS
> C - 5 E - 3 D - 2 A - 2
> It is giving accurate results for COUNT ASC, LIMIT 1 - 4
> {code}
> curl http://localhost:8983/solr/collection1/select -d
> 'q=*:*={cat_s:{type:terms,field:cat_s,sort:"count
> asc",limit:1,overrequest:0,refine:true}}=json=true'
> {code}
> {code}
> "facets":{
> "count":12,
> "cat_s":{
> "buckets":[{
> "val":"A",
> "count":2}]}}}
> {code}
> {code}
> curl http://localhost:8983/solr/collection1/select -d
> 'q=*:*={cat_s:{type:terms,field:cat_s,sort:"count
> asc",limit:2,overrequest:0,refine:true}}=json=true'
> {code}
> {code}
> "facets":{
> "count":12,
> "cat_s":{
> "buckets":[{
> "val":"A",
> "count":2},
> {
> "val":"D",
> "count":2}]}}}
> {code}
> *BUT, COUNT DESC, LIMIT 2 and 3*
> {code}
> curl http://localhost:8983/solr/collection1/select -d
> 'q=*:*={cat_s:{type:terms,field:cat_s,sort:"count
> desc",limit:2,overrequest:0,refine:true}}=json=true'
> {code}
> {code}
> "facets":{
> "count":12,
> "cat_s":{
> "buckets":[{
> "val":"C",
> "count":5},
> {
> "val":"A",
> "count":2}]}}}
> {code}
> {code}
> curl http://localhost:8983/solr/collection1/select -d
> 'q=*:*={cat_s:{type:terms,field:cat_s,sort:"count
> desc",limit:3,overrequest:0,refine:true}}=json=true'
> {code}
> {code}
> "facets":{
> "count":12,
> "cat_s":{
> "buckets":[{
> "val":"C",
> "count":5},
> {
> "val":"A",
> "count":2},
> {
> "val":"D",
> "count":2}]}}}
> {code}
> *bucketVal {{E}} and its count {{3}} is not in facet response* Pardon me if I
> am missing some configuration or this behavior is right / justified. Ideally
> we should see bucketVal E and its count 3.
> I am attaching Index DOCS, debugQuery for COUNT DESC, LIMIT 2 and LIMIT 3.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11159) Facet buckets count missing after passing {refine:true} | SOLR-7542
[
https://issues.apache.org/jira/browse/SOLR-11159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-11159:
Summary: Facet buckets count missing after passing {refine:true} |
SOLR-7542 (was: Facet buckets count still incorrect after passing
{refine:true} | SOLR-7542)
> Facet buckets count missing after passing {refine:true} | SOLR-7542
> ---
>
> Key: SOLR-11159
> URL: https://issues.apache.org/jira/browse/SOLR-11159
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Facet Module
>Reporter: Amrit Sarkar
> Attachments: COUNT_DESC_LIMIT_2, COUNT_DESC_LIMIT_3, DOCS
>
>
> I was experimenting / analysing the new *Refinement* feature in JSON Facet
> Apis introduced in SOLR-7452. Passing {{refine:true}} with the facet
> definition.
> I am listing down the test-scenarios along with test-data:
> 3 sharded collection on 3 nodes
> node/shard: bucketVal - count
> 8987: C - 1
> 8983: C - 4 D - 1 E - 1 A - 1
> 8985: E - 2 A - 1 D - 1
> Total: BUCKETS
> C - 5 E - 3 D - 2 A - 2
> It is giving accurate results for COUNT ASC, LIMIT 1 - 4
> {code}
> curl http://localhost:8983/solr/collection1/select -d
> 'q=*:*={cat_s:{type:terms,field:cat_s,sort:"count
> asc",limit:1,overrequest:0,refine:true}}=json=true'
> {code}
> {code}
> "facets":{
> "count":12,
> "cat_s":{
> "buckets":[{
> "val":"A",
> "count":2}]}}}
> {code}
> {code}
> curl http://localhost:8983/solr/collection1/select -d
> 'q=*:*={cat_s:{type:terms,field:cat_s,sort:"count
> asc",limit:2,overrequest:0,refine:true}}=json=true'
> {code}
> {code}
> "facets":{
> "count":12,
> "cat_s":{
> "buckets":[{
> "val":"A",
> "count":2},
> {
> "val":"D",
> "count":2}]}}}
> {code}
> *BUT, COUNT DESC, LIMIT 2 and 3*
> {code}
> curl http://localhost:8983/solr/collection1/select -d
> 'q=*:*={cat_s:{type:terms,field:cat_s,sort:"count
> desc",limit:2,overrequest:0,refine:true}}=json=true'
> {code}
> {code}
> "facets":{
> "count":12,
> "cat_s":{
> "buckets":[{
> "val":"C",
> "count":5},
> {
> "val":"A",
> "count":2}]}}}
> {code}
> {code}
> curl http://localhost:8983/solr/collection1/select -d
> 'q=*:*={cat_s:{type:terms,field:cat_s,sort:"count
> desc",limit:3,overrequest:0,refine:true}}=json=true'
> {code}
> {code}
> "facets":{
> "count":12,
> "cat_s":{
> "buckets":[{
> "val":"C",
> "count":5},
> {
> "val":"A",
> "count":2},
> {
> "val":"D",
> "count":2}]}}}
> {code}
> *bucketVal {{E}} and its count {{3}} is not in facet response* Pardon me if I
> am missing some configuration or this behavior is right / justified. Ideally
> we should see bucketVal E and its count 3.
> I am attaching Index DOCS, debugQuery for COUNT DESC, LIMIT 2 and LIMIT 3.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11159) Facet buckets count still incorrect after passing {refine:true} | SOLR-7542
[
https://issues.apache.org/jira/browse/SOLR-11159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-11159:
Summary: Facet buckets count still incorrect after passing {refine:true} |
SOLR-7542 (was: Facet buckets count still incorrect even after passing
{refine:true} | SOLR-7542)
> Facet buckets count still incorrect after passing {refine:true} | SOLR-7542
> ---
>
> Key: SOLR-11159
> URL: https://issues.apache.org/jira/browse/SOLR-11159
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Facet Module
>Reporter: Amrit Sarkar
>
> I was experimenting / analysing the new *Refinement* feature in JSON Facet
> Apis introduced in SOLR-7452. Passing {{refine:true}} with the facet
> definition.
> I am listing down the test-scenarios along with test-data:
> 3 sharded collection on 3 nodes
> node/shard: bucketVal - count
> 8987: C - 1
> 8983: C - 4 D - 1 E - 1 A - 1
> 8985: E - 2 A - 1 D - 1
> Total: BUCKETS
> C - 5 E - 3 D - 2 A - 2
> It is giving accurate results for COUNT ASC, LIMIT 1 - 4
> {code}
> curl http://localhost:8983/solr/collection1/select -d
> 'q=*:*={cat_s:{type:terms,field:cat_s,sort:"count
> asc",limit:1,overrequest:0,refine:true}}=json=true'
> {code}
> {code}
> "facets":{
> "count":12,
> "cat_s":{
> "buckets":[{
> "val":"A",
> "count":2}]}}}
> {code}
> {code}
> curl http://localhost:8983/solr/collection1/select -d
> 'q=*:*={cat_s:{type:terms,field:cat_s,sort:"count
> asc",limit:2,overrequest:0,refine:true}}=json=true'
> {code}
> {code}
> "facets":{
> "count":12,
> "cat_s":{
> "buckets":[{
> "val":"A",
> "count":2},
> {
> "val":"D",
> "count":2}]}}}
> {code}
> *BUT, COUNT DESC, LIMIT 2 and 3*
> {code}
> curl http://localhost:8983/solr/collection1/select -d
> 'q=*:*={cat_s:{type:terms,field:cat_s,sort:"count
> desc",limit:2,overrequest:0,refine:true}}=json=true'
> {code}
> {code}
> "facets":{
> "count":12,
> "cat_s":{
> "buckets":[{
> "val":"C",
> "count":5},
> {
> "val":"A",
> "count":2}]}}}
> {code}
> {code}
> curl http://localhost:8983/solr/collection1/select -d
> 'q=*:*={cat_s:{type:terms,field:cat_s,sort:"count
> desc",limit:3,overrequest:0,refine:true}}=json=true'
> {code}
> {code}
> "facets":{
> "count":12,
> "cat_s":{
> "buckets":[{
> "val":"C",
> "count":5},
> {
> "val":"A",
> "count":2},
> {
> "val":"D",
> "count":2}]}}}
> {code}
> *bucketVal {{E}} and its count {{3}} is not in facet response* Pardon me if I
> am missing some configuration or this behavior is right / justified. Ideally
> we should see bucketVal E and its count 3.
> I am attaching Index DOCS, debugQuery for COUNT DESC, LIMIT 2 and LIMIT 3.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-11159) Facet buckets count still incorrect even after passing {refine:true} | SOLR-7542
Amrit Sarkar created SOLR-11159:
---
Summary: Facet buckets count still incorrect even after passing
{refine:true} | SOLR-7542
Key: SOLR-11159
URL: https://issues.apache.org/jira/browse/SOLR-11159
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Components: Facet Module
Reporter: Amrit Sarkar
I was experimenting / analysing the new *Refinement* feature in JSON Facet Apis
introduced in SOLR-7452. Passing {{refine:true}} with the facet definition.
I am listing down the test-scenarios along with test-data:
3 sharded collection on 3 nodes
node/shard: bucketVal - count
8987: C - 1
8983: C - 4 D - 1 E - 1 A - 1
8985: E - 2 A - 1 D - 1
Total: BUCKETS
C - 5 E - 3 D - 2 A - 2
It is giving accurate results for COUNT ASC, LIMIT 1 - 4
{code}
curl http://localhost:8983/solr/collection1/select -d
'q=*:*={cat_s:{type:terms,field:cat_s,sort:"count
asc",limit:1,overrequest:0,refine:true}}=json=true'
{code}
{code}
"facets":{
"count":12,
"cat_s":{
"buckets":[{
"val":"A",
"count":2}]}}}
{code}
{code}
curl http://localhost:8983/solr/collection1/select -d
'q=*:*={cat_s:{type:terms,field:cat_s,sort:"count
asc",limit:2,overrequest:0,refine:true}}=json=true'
{code}
{code}
"facets":{
"count":12,
"cat_s":{
"buckets":[{
"val":"A",
"count":2},
{
"val":"D",
"count":2}]}}}
{code}
*BUT, COUNT DESC, LIMIT 2 and 3*
{code}
curl http://localhost:8983/solr/collection1/select -d
'q=*:*={cat_s:{type:terms,field:cat_s,sort:"count
desc",limit:2,overrequest:0,refine:true}}=json=true'
{code}
{code}
"facets":{
"count":12,
"cat_s":{
"buckets":[{
"val":"C",
"count":5},
{
"val":"A",
"count":2}]}}}
{code}
{code}
curl http://localhost:8983/solr/collection1/select -d
'q=*:*={cat_s:{type:terms,field:cat_s,sort:"count
desc",limit:3,overrequest:0,refine:true}}=json=true'
{code}
{code}
"facets":{
"count":12,
"cat_s":{
"buckets":[{
"val":"C",
"count":5},
{
"val":"A",
"count":2},
{
"val":"D",
"count":2}]}}}
{code}
*bucketVal {{E}} and its count {{3}} is not in facet response* Pardon me if I
am missing some configuration or this behavior is right / justified. Ideally we
should see bucketVal E and its count 3.
I am attaching Index DOCS, debugQuery for COUNT DESC, LIMIT 2 and LIMIT 3.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11159) Facet buckets count still incorrect after passing {refine:true} | SOLR-7542
[
https://issues.apache.org/jira/browse/SOLR-11159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-11159:
Attachment: COUNT_DESC_LIMIT_2
COUNT_DESC_LIMIT_3
DOCS
> Facet buckets count still incorrect after passing {refine:true} | SOLR-7542
> ---
>
> Key: SOLR-11159
> URL: https://issues.apache.org/jira/browse/SOLR-11159
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Facet Module
>Reporter: Amrit Sarkar
> Attachments: COUNT_DESC_LIMIT_2, COUNT_DESC_LIMIT_3, DOCS
>
>
> I was experimenting / analysing the new *Refinement* feature in JSON Facet
> Apis introduced in SOLR-7452. Passing {{refine:true}} with the facet
> definition.
> I am listing down the test-scenarios along with test-data:
> 3 sharded collection on 3 nodes
> node/shard: bucketVal - count
> 8987: C - 1
> 8983: C - 4 D - 1 E - 1 A - 1
> 8985: E - 2 A - 1 D - 1
> Total: BUCKETS
> C - 5 E - 3 D - 2 A - 2
> It is giving accurate results for COUNT ASC, LIMIT 1 - 4
> {code}
> curl http://localhost:8983/solr/collection1/select -d
> 'q=*:*={cat_s:{type:terms,field:cat_s,sort:"count
> asc",limit:1,overrequest:0,refine:true}}=json=true'
> {code}
> {code}
> "facets":{
> "count":12,
> "cat_s":{
> "buckets":[{
> "val":"A",
> "count":2}]}}}
> {code}
> {code}
> curl http://localhost:8983/solr/collection1/select -d
> 'q=*:*={cat_s:{type:terms,field:cat_s,sort:"count
> asc",limit:2,overrequest:0,refine:true}}=json=true'
> {code}
> {code}
> "facets":{
> "count":12,
> "cat_s":{
> "buckets":[{
> "val":"A",
> "count":2},
> {
> "val":"D",
> "count":2}]}}}
> {code}
> *BUT, COUNT DESC, LIMIT 2 and 3*
> {code}
> curl http://localhost:8983/solr/collection1/select -d
> 'q=*:*={cat_s:{type:terms,field:cat_s,sort:"count
> desc",limit:2,overrequest:0,refine:true}}=json=true'
> {code}
> {code}
> "facets":{
> "count":12,
> "cat_s":{
> "buckets":[{
> "val":"C",
> "count":5},
> {
> "val":"A",
> "count":2}]}}}
> {code}
> {code}
> curl http://localhost:8983/solr/collection1/select -d
> 'q=*:*={cat_s:{type:terms,field:cat_s,sort:"count
> desc",limit:3,overrequest:0,refine:true}}=json=true'
> {code}
> {code}
> "facets":{
> "count":12,
> "cat_s":{
> "buckets":[{
> "val":"C",
> "count":5},
> {
> "val":"A",
> "count":2},
> {
> "val":"D",
> "count":2}]}}}
> {code}
> *bucketVal {{E}} and its count {{3}} is not in facet response* Pardon me if I
> am missing some configuration or this behavior is right / justified. Ideally
> we should see bucketVal E and its count 3.
> I am attaching Index DOCS, debugQuery for COUNT DESC, LIMIT 2 and LIMIT 3.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10734) Multithreaded test/support for AtomicURP broken
[
https://issues.apache.org/jira/browse/SOLR-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102167#comment-16102167
]
Amrit Sarkar commented on SOLR-10734:
-
[~noble.paul] [~ichattopadhyaya],
Let me know if we need to do anything else apart from the above to rectify the
test class.
> Multithreaded test/support for AtomicURP broken
> ---
>
> Key: SOLR-10734
> URL: https://issues.apache.org/jira/browse/SOLR-10734
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Ishan Chattopadhyaya
> Attachments: log-snippet, Screen Shot 2017-05-31 at 4.50.23 PM.png,
> SOLR-10734.patch
>
>
> The multithreaded test doesn't actually start the threads, but only invokes
> the run directly. The join afterwards doesn't do anything, hence.
> {code}
> diff --git
> a/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
>
> b/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
> index f3f833d..10b7770 100644
> ---
> a/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
> +++
> b/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
> @@ -238,7 +238,7 @@ public class AtomicUpdateProcessorFactoryTest extends
> SolrTestCaseJ4 {
>}
> }
>};
> - t.run();
> + t.run(); // red flag, shouldn't this be t.start?
>threads.add(t);
>finalCount += index; //int_i
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11084) Issue with starting script with solr.home (-s) == solr
[
https://issues.apache.org/jira/browse/SOLR-11084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16093127#comment-16093127
]
Amrit Sarkar commented on SOLR-11084:
-
[~neilireson]
Regarding documenting {{-s solr}} in the docs. If the patch goes through, I
don't a strong reason to put this bit of information in official docs. Your
thoughts? [~erickerickson].
> Issue with starting script with solr.home (-s) == solr
> --
>
> Key: SOLR-11084
> URL: https://issues.apache.org/jira/browse/SOLR-11084
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 6.6
> Environment: OS X 10.12
>Reporter: Neil Ireson
> Attachments: SOLR-11084.patch
>
>
> I've just hit an issue when starting solr using the script. All works well
> when I use:
> /solr-6.6.0/bin/solr start -p 9090 -m 4g -s data/solr
> However if I cd into the data directory and try:
> /solr-6.6.0/bin/solr start -p 9090 -m 4g -s solr
> Then I get no cores loaded and the "solr.solr.home" shown in the UI as my
> installation solr (i.e. /solr-6.6.0/server/solr)
> I thought that "solr" may be a reserved word so I changed the name of my
> folder to "test-solr" and this worked.
> I'm unsure if this is an undocumented feature (at least I couldn't find any
> reference to it) or a bug.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5822) Create a markdown formatted README file for Lucene/Solr
[
https://issues.apache.org/jira/browse/LUCENE-5822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071580#comment-16071580
]
Amrit Sarkar commented on LUCENE-5822:
--
[~mdrob] I can see {{dev-tools/scripts/.smokeTestRelease.py.swp}} checked in
too along with the py-script. Is that intentional?
> Create a markdown formatted README file for Lucene/Solr
> ---
>
> Key: LUCENE-5822
> URL: https://issues.apache.org/jira/browse/LUCENE-5822
> Project: Lucene - Core
> Issue Type: Wish
>Reporter: Shalin Shekhar Mangar
>Assignee: Mike Drob
> Fix For: master (7.0)
>
> Attachments: LUCENE-5822-addemdum.patch, LUCENE-5822.patch,
> LUCENE-5822.patch
>
>
> We have a minimal plain text readme file right now. Github is very popular
> these days and we are even accepting pull requests from there. I think we
> should add a proper markdown formatted README file which not only talks about
> the features of Lucene/Solr but also include a short tutorial on both Lucene
> and Solr.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11003) Enabling bi-directional CDCR active-active clusters
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amrit Sarkar updated SOLR-11003: Description: The latest version of Solr CDCR across clusters is in active-passive format, where we can index into source cluster and the updates gets forwarded to the passive one and vice-versa is not supported. https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html https://issues.apache.org/jira/browse/SOLR-6273 We are try to get a design ready to index in both clusters and the updates gets reflected across the clusters in real-time. ClusterA => ClusterB | ClusterB => ClusterA. The best use-case would be to we keep indexing in ClusterA which forwarded the updates to ClusterB. If ClusterA gets down, we point the indexer and searcher application to ClusterB. Once ClusterA is up, depending updates count, they will be bootstrapped or forwarded to ClusterA from ClusterB. was: The latest version of Solr CDCR across cluster in active-passive format, where we can index into source cluster and the updates gets forwarded to the passive one and vice-versa is not supported. https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html https://issues.apache.org/jira/browse/SOLR-6273 We are try to get a design ready to index in both clusters and the updates gets reflected across the clusters in real-time. ClusterA => ClusterB | ClusterB => ClusterA. The best use-case would be to we keep indexing in ClusterA which forwarded the updates to ClusterB. If ClusterA gets down, we point the indexer and searcher application to ClusterB. Once ClusterA is up, depending updates count, they will be bootstrapped or forwarded to ClusterA from ClusterB. > Enabling bi-directional CDCR active-active clusters > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar > > The latest version of Solr CDCR across clusters is in active-passive format, > where we can index into source cluster and the updates gets forwarded to the > passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both clusters and the updates > gets reflected across the clusters in real-time. ClusterA => ClusterB | > ClusterB => ClusterA. > The best use-case would be to we keep indexing in ClusterA which forwarded > the updates to ClusterB. If ClusterA gets down, we point the indexer and > searcher application to ClusterB. Once ClusterA is up, depending updates > count, they will be bootstrapped or forwarded to ClusterA from ClusterB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-11003) Enabling bi-directional CDCR active-active clusters
[
https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16073575#comment-16073575
]
Amrit Sarkar edited comment on SOLR-11003 at 7/4/17 12:27 PM:
--
Backward-compat::
tlogs entries index are definite for each operation: identifier(0) and
version(1) common to all.
*DELETE:*
identifier: delete
update version
payload
if you see UpdateLog: doReplay: 1801
{code}
case UpdateLog.DELETE: {
recoveryInfo.deletes++;
byte[] idBytes = (byte[]) entry.get(2);
DeleteUpdateCommand cmd = new DeleteUpdateCommand(req);
cmd.setIndexedId(new BytesRef(idBytes));
cmd.setVersion(version);
cmd.setFlags(UpdateCommand.REPLAY |
UpdateCommand.IGNORE_AUTOCOMMIT);
if (debug) log.debug("delete " + cmd);
proc.processDelete(cmd);
break;
}
{code}
hardcoded, the position of payload(2) is hardcoded. See CdcrReplicator, same.
So we can put anything on next index(3).
*DELETE_BY_QUERY:*
Ditto same!!
*ADD:*
indentifier: add or inplace
version
payload
*IN_PLACE_ADD:*
indentifier: add or inplace
version
previous pointer
previous version
payload
UpdateLog:: 1916, since our current code handles both adds in one manner, it
assumes the last index of the entries is our payload:
{code}
public static AddUpdateCommand
convertTlogEntryToAddUpdateCommand(SolrQueryRequest req, List entry,
int
operation, long version) {
assert operation == UpdateLog.ADD || operation == UpdateLog.UPDATE_INPLACE;
SolrInputDocument sdoc = (SolrInputDocument) entry.get(entry.size()-1);
AddUpdateCommand cmd = new AddUpdateCommand(req);
cmd.solrDoc = sdoc;
cmd.setVersion(version);
if (operation == UPDATE_INPLACE) {
long prevVersion = (Long) entry.get(UpdateLog.PREV_VERSION_IDX);
cmd.prevVersion = prevVersion;
}
return cmd;
}
{code}
So our window of adding something is the index just before the last. entry.size
- 2. Please see CdcrReplicator for the same.
was (Author: sarkaramr...@gmail.com):
Backward-compat::
tlogs entries index are definite for each operation: identifier(0) and
version(1) common to all.
*DELETE:*
identifier: delete
update version
payload
if you see UpdateLog: doReplay: 1801
{code}
case UpdateLog.DELETE: {
recoveryInfo.deletes++;
byte[] idBytes = (byte[]) entry.get(2);
DeleteUpdateCommand cmd = new DeleteUpdateCommand(req);
cmd.setIndexedId(new BytesRef(idBytes));
cmd.setVersion(version);
cmd.setFlags(UpdateCommand.REPLAY |
UpdateCommand.IGNORE_AUTOCOMMIT);
if (debug) log.debug("delete " + cmd);
proc.processDelete(cmd);
break;
}
{code}
hardcoded, the position of payload(2) is hardcoded. See CdcrReplicator, same.
So we can put anything on next index(3).
*DELETE_BY_QUERY:*
Ditto same!!
*ADD:*
indentifier: add or inplace
version
payload
*IN_PLACE_ADD:*
indentifier: add or inplace
version
previous pointer
previous version
payload
UpdateLog:: 1916, since our current code handles both adds in one manner, it
assumes the last index of the entries is our payload:
{code}
public static AddUpdateCommand
convertTlogEntryToAddUpdateCommand(SolrQueryRequest req, List entry,
int
operation, long version) {
assert operation == UpdateLog.ADD || operation == UpdateLog.UPDATE_INPLACE;
SolrInputDocument sdoc = (SolrInputDocument) entry.get(entry.size()-1);
AddUpdateCommand cmd = new AddUpdateCommand(req);
cmd.solrDoc = sdoc;
cmd.setVersion(version);
if (operation == UPDATE_INPLACE) {
long prevVersion = (Long) entry.get(UpdateLog.PREV_VERSION_IDX);
cmd.prevVersion = prevVersion;
}
return cmd;
}
{code}
So our window of adding something is the index just before the last. entry.size
- 2. Please see CdcrReplictor for the same.
> Enabling bi-directional CDCR active-active clusters
> ---
>
> Key: SOLR-11003
> URL: https://issues.apache.org/jira/browse/SOLR-11003
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Reporter: Amrit Sarkar
>
> The latest version of Solr CDCR across collections / clusters is in
> active-passive format, where we can index into source collection and the
> updates gets forwarded to the passive one and vice-versa is not supported.
> https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html
> https://issues.apache.org/jira/browse/SOLR-6273
> We are
[jira] [Commented] (SOLR-11003) Enabling bi-directional CDCR active-active clusters
[
https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16073586#comment-16073586
]
Amrit Sarkar commented on SOLR-11003:
-
Patch uploaded:
{code}
modified:
solr/core/src/java/org/apache/solr/handler/CdcrReplicator.java
modified:
solr/core/src/java/org/apache/solr/update/CdcrTransactionLog.java
modified:
solr/core/src/java/org/apache/solr/update/TransactionLog.java
new file:
solr/core/src/test-files/solr/configsets/cdcr-cluster1/conf/schema.xml
new file:
solr/core/src/test-files/solr/configsets/cdcr-cluster1/conf/solrconfig.xml
new file:
solr/core/src/test-files/solr/configsets/cdcr-cluster2/conf/schema.xml
new file:
solr/core/src/test-files/solr/configsets/cdcr-cluster2/conf/solrconfig.xml
new file:
solr/core/src/test/org/apache/solr/cloud/CdcrBidirectionalTest.java
{code}
Added testclass CdcrBidirectionalTest where two active clusters are talking to
each other runtime.
The write operations in TransactionLog are repeated in CdcrTransactionLog to
accommodate the extra entry for each update. Repeated code! henceforth planning
to have TLogCommonUtils / UpdateLogCommonUtils to put the common code for both
classes' methods.
Eagerly looking forward to feedback, suggestions and improvements.
> Enabling bi-directional CDCR active-active clusters
> ---
>
> Key: SOLR-11003
> URL: https://issues.apache.org/jira/browse/SOLR-11003
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Reporter: Amrit Sarkar
>
> The latest version of Solr CDCR across collections / clusters is in
> active-passive format, where we can index into source collection and the
> updates gets forwarded to the passive one and vice-versa is not supported.
> https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html
> https://issues.apache.org/jira/browse/SOLR-6273
> We are try to get a design ready to index in both collections and the
> updates gets reflected across the collections in real-time.
> ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB =>
> ClusterACollectionA.
> The best use-case would be to we keep indexing in ClusterACollectionA which
> forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets
> down, we point the indexer and searcher application to ClusterBCollectionB.
> Once ClusterACollectionA is up, depending on updates count, they will be
> bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and
> keep indexing on the ClusterBCollectionB.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-11003) Enabling bi-directional CDCR active-active clusters
Amrit Sarkar created SOLR-11003: --- Summary: Enabling bi-directional CDCR active-active clusters Key: SOLR-11003 URL: https://issues.apache.org/jira/browse/SOLR-11003 Project: Solr Issue Type: Improvement Security Level: Public (Default Security Level. Issues are Public) Components: CDCR Reporter: Amrit Sarkar The latest version of Solr CDCR across cluster in active-passive format, where we can index into source cluster and the updates gets forwarded to the passive one and vice-versa is not supported. https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html https://issues.apache.org/jira/browse/SOLR-6273 We are try to get a design ready to index in both clusters and the updates gets reflected across the clusters in real-time. ClusterA => ClusterB | ClusterB => ClusterA. The best use-case would be to we keep indexing in ClusterA which forwarded the updates to ClusterB. If ClusterA gets down, we point the indexer and searcher application to ClusterB. Once ClusterA is up, depending updates count, they will be bootstrapped or forwarded to ClusterA from ClusterB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11003) Enabling bi-directional CDCR active-active clusters
[
https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16073575#comment-16073575
]
Amrit Sarkar commented on SOLR-11003:
-
Backward-compat::
tlogs entries index are definite for each operation: identifier(0) and
version(1) common to all.
*DELETE:*
identifier: delete
update version
payload
if you see UpdateLog: doReplay: 1801
{code}
case UpdateLog.DELETE: {
recoveryInfo.deletes++;
byte[] idBytes = (byte[]) entry.get(2);
DeleteUpdateCommand cmd = new DeleteUpdateCommand(req);
cmd.setIndexedId(new BytesRef(idBytes));
cmd.setVersion(version);
cmd.setFlags(UpdateCommand.REPLAY |
UpdateCommand.IGNORE_AUTOCOMMIT);
if (debug) log.debug("delete " + cmd);
proc.processDelete(cmd);
break;
}
{code}
hardcoded, the position of payload(2) is hardcoded. See CdcrReplicator, same.
So we can put anything on next index(3).
*DELETE_BY_QUERY:*
Ditto same!!
*ADD:*
indentifier: add or inplace
version
payload
*IN_PLACE_ADD:*
indentifier: add or inplace
version
previous pointer
previous version
payload
UpdateLog:: 1916, since our current code handles both adds in one manner, it
assumes the last index of the entries is our payload:
{code}
public static AddUpdateCommand
convertTlogEntryToAddUpdateCommand(SolrQueryRequest req, List entry,
int
operation, long version) {
assert operation == UpdateLog.ADD || operation == UpdateLog.UPDATE_INPLACE;
SolrInputDocument sdoc = (SolrInputDocument) entry.get(entry.size()-1);
AddUpdateCommand cmd = new AddUpdateCommand(req);
cmd.solrDoc = sdoc;
cmd.setVersion(version);
if (operation == UPDATE_INPLACE) {
long prevVersion = (Long) entry.get(UpdateLog.PREV_VERSION_IDX);
cmd.prevVersion = prevVersion;
}
return cmd;
}
{code}
So our window of adding something is the index just before the last. entry.size
- 2. Please see CdcrReplictor for the same.
> Enabling bi-directional CDCR active-active clusters
> ---
>
> Key: SOLR-11003
> URL: https://issues.apache.org/jira/browse/SOLR-11003
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Reporter: Amrit Sarkar
>
> The latest version of Solr CDCR across collections / clusters is in
> active-passive format, where we can index into source collection and the
> updates gets forwarded to the passive one and vice-versa is not supported.
> https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html
> https://issues.apache.org/jira/browse/SOLR-6273
> We are try to get a design ready to index in both collections and the
> updates gets reflected across the collections in real-time.
> ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB =>
> ClusterACollectionA.
> The best use-case would be to we keep indexing in ClusterACollectionA which
> forwarded the updates to ClusterBCollectionB. If ClusterACollectionA gets
> down, we point the indexer and searcher application to ClusterBCollectionB.
> Once ClusterACollectionA is up, depending updates count, they will be
> bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11003) Enabling bi-directional CDCR active-active clusters
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amrit Sarkar updated SOLR-11003: Attachment: SOLR-11003.patch > Enabling bi-directional CDCR active-active clusters > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar > Attachments: SOLR-11003.patch > > > The latest version of Solr CDCR across collections / clusters is in > active-passive format, where we can index into source collection and the > updates gets forwarded to the passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both collections and the > updates gets reflected across the collections in real-time. > ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB => > ClusterACollectionA. > The best use-case would be to we keep indexing in ClusterACollectionA which > forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets > down, we point the indexer and searcher application to ClusterBCollectionB. > Once ClusterACollectionA is up, depending on updates count, they will be > bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and > keep indexing on the ClusterBCollectionB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11003) Enabling bi-directional CDCR active-active clusters
[
https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16073572#comment-16073572
]
Amrit Sarkar commented on SOLR-11003:
-
*Proposed design =>*
*Configuration:*
Configure both collections as source and point to their targets. Sample
configurations listed below:
_Cluster2:_
{code}
${cdcr.cluster1.zkHost}
cdcr-cluster2
cdcr-cluster1
1
1000
1000
1000
{code}
_Cluster1:_
{code}
${cdcr.cluster2.zkHost}
cdcr-cluster1
cdcr-cluster2
1
1000
1000
1000
{code}
*Replication Improvement:*
CdcrReplicator replays the tlogs, creates new UpdateRequests and forwards them
to target collection. *The plan is to persist information in the tlogs' each
update entries telling the collection this update is received as a CDCR forward
and it should be not be forwarded further.*
Each CDCR update forward contains the param key "cdcr.update" and value empty.
Once the target collection received that particular update, before writing it
down to transaction log, we check whether the UpdateCommand -> SolrRequest ->
contains params key "cdcr.update", if there is, apart from the entries listed
for each update, we add a flag, say isCdcr. "true" and write it to tlog. When
CdcrReplicator reads updates from the tlog, and if the flag is present / set,
skip that update for forwarding and read next.
We had to be very careful of where to put the flag so that we don't break the
backward compatibility and hence we added at the last index for DELETE and
DELETEBYQUERY while second last index for ADD. Reason listed below.
> Enabling bi-directional CDCR active-active clusters
> ---
>
> Key: SOLR-11003
> URL: https://issues.apache.org/jira/browse/SOLR-11003
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Reporter: Amrit Sarkar
>
> The latest version of Solr CDCR across collections / clusters is in
> active-passive format, where we can index into source collection and the
> updates gets forwarded to the passive one and vice-versa is not supported.
> https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html
> https://issues.apache.org/jira/browse/SOLR-6273
> We are try to get a design ready to index in both collections and the
> updates gets reflected across the collections in real-time.
> ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB =>
> ClusterACollectionA.
> The best use-case would be to we keep indexing in ClusterACollectionA which
> forwarded the updates to ClusterBCollectionB. If ClusterACollectionA gets
> down, we point the indexer and searcher application to ClusterBCollectionB.
> Once ClusterACollectionA is up, depending updates count, they will be
> bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11003) Enabling bi-directional CDCR active-active clusters
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amrit Sarkar updated SOLR-11003: Description: The latest version of Solr CDCR across collections / clusters is in active-passive format, where we can index into source collection and the updates gets forwarded to the passive one and vice-versa is not supported. https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html https://issues.apache.org/jira/browse/SOLR-6273 We are try to get a design ready to index in both collections and the updates gets reflected across the collections in real-time. ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB => ClusterACollectionA. The best use-case would be to we keep indexing in ClusterACollectionA which forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets down, we point the indexer and searcher application to ClusterBCollectionB. Once ClusterACollectionA is up, depending on updates count, they will be bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and keep indexing on the ClusterBCollectionB. was: The latest version of Solr CDCR across collections / clusters is in active-passive format, where we can index into source collection and the updates gets forwarded to the passive one and vice-versa is not supported. https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html https://issues.apache.org/jira/browse/SOLR-6273 We are try to get a design ready to index in both collections and the updates gets reflected across the collections in real-time. ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB => ClusterACollectionA. The best use-case would be to we keep indexing in ClusterACollectionA which forwarded the updates to ClusterBCollectionB. If ClusterACollectionA gets down, we point the indexer and searcher application to ClusterBCollectionB. Once ClusterACollectionA is up, depending updates count, they will be bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB. > Enabling bi-directional CDCR active-active clusters > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar > > The latest version of Solr CDCR across collections / clusters is in > active-passive format, where we can index into source collection and the > updates gets forwarded to the passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both collections and the > updates gets reflected across the collections in real-time. > ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB => > ClusterACollectionA. > The best use-case would be to we keep indexing in ClusterACollectionA which > forwards the updates to ClusterBCollectionB. If ClusterACollectionA gets > down, we point the indexer and searcher application to ClusterBCollectionB. > Once ClusterACollectionA is up, depending on updates count, they will be > bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB and > keep indexing on the ClusterBCollectionB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11003) Enabling bi-directional CDCR active-active clusters
[ https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amrit Sarkar updated SOLR-11003: Description: The latest version of Solr CDCR across collections / clusters is in active-passive format, where we can index into source collection and the updates gets forwarded to the passive one and vice-versa is not supported. https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html https://issues.apache.org/jira/browse/SOLR-6273 We are try to get a design ready to index in both collections and the updates gets reflected across the collections in real-time. ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB => ClusterACollectionA. The best use-case would be to we keep indexing in ClusterACollectionA which forwarded the updates to ClusterBCollectionB. If ClusterACollectionA gets down, we point the indexer and searcher application to ClusterBCollectionB. Once ClusterACollectionA is up, depending updates count, they will be bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB. was: The latest version of Solr CDCR across clusters is in active-passive format, where we can index into source cluster and the updates gets forwarded to the passive one and vice-versa is not supported. https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html https://issues.apache.org/jira/browse/SOLR-6273 We are try to get a design ready to index in both clusters and the updates gets reflected across the clusters in real-time. ClusterA => ClusterB | ClusterB => ClusterA. The best use-case would be to we keep indexing in ClusterA which forwarded the updates to ClusterB. If ClusterA gets down, we point the indexer and searcher application to ClusterB. Once ClusterA is up, depending updates count, they will be bootstrapped or forwarded to ClusterA from ClusterB. > Enabling bi-directional CDCR active-active clusters > --- > > Key: SOLR-11003 > URL: https://issues.apache.org/jira/browse/SOLR-11003 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: CDCR >Reporter: Amrit Sarkar > > The latest version of Solr CDCR across collections / clusters is in > active-passive format, where we can index into source collection and the > updates gets forwarded to the passive one and vice-versa is not supported. > https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html > https://issues.apache.org/jira/browse/SOLR-6273 > We are try to get a design ready to index in both collections and the > updates gets reflected across the collections in real-time. > ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB => > ClusterACollectionA. > The best use-case would be to we keep indexing in ClusterACollectionA which > forwarded the updates to ClusterBCollectionB. If ClusterACollectionA gets > down, we point the indexer and searcher application to ClusterBCollectionB. > Once ClusterACollectionA is up, depending updates count, they will be > bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-9530) Add an Atomic Update Processor
[
https://issues.apache.org/jira/browse/SOLR-9530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-9530:
---
Attachment: SOLR-9530.patch
Thank you Erick for the above suggestions,
I digged more and used *commitWithin* set it to 1 ms and put a sleep of 500 ms
and I am passing through the tests without reload. This is obviously not a good
manner to write a test as it will be machine subjective. Seeking suggestions if
there is a better way or should I replace with above suggestion.
New patch uploaded.
> Add an Atomic Update Processor
> ---
>
> Key: SOLR-9530
> URL: https://issues.apache.org/jira/browse/SOLR-9530
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Varun Thacker
> Attachments: SOLR-9530.patch, SOLR-9530.patch, SOLR-9530.patch,
> SOLR-9530.patch, SOLR-9530.patch, SOLR-9530.patch, SOLR-9530.patch,
> SOLR-9530.patch, SOLR-9530.patch, SOLR-9530.patch, SOLR-9530.patch
>
>
> I'd like to explore the idea of adding a new update processor to help ingest
> partial updates.
> Example use-case - There are two datasets with a common id field. How can I
> merge both of them at index time?
> Proposed Solution:
> {code}
>
>
> add
>
>
>
>
> {code}
> So the first JSON dump could be ingested against
> {{http://localhost:8983/solr/gettingstarted/update/json}}
> And then the second JSON could be ingested against
> {{http://localhost:8983/solr/gettingstarted/update/json?processor=atomic}}
> The Atomic Update Processor could support all the atomic update operations
> currently supported.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10209) UI: Convert all Collections api calls to async requests, add new features/buttons
[
https://issues.apache.org/jira/browse/SOLR-10209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15985027#comment-15985027
]
Amrit Sarkar commented on SOLR-10209:
-
Been a while posted any update on this.
With the existing UI code, it is little challenging incorporating a new message
box, floating preferred, to show current status, as we intend to refresh the
entire scope at the end of each request. I intend to change that specially for
heavy APIs and post full and final patch within a week.
> UI: Convert all Collections api calls to async requests, add new
> features/buttons
> -
>
> Key: SOLR-10209
> URL: https://issues.apache.org/jira/browse/SOLR-10209
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Admin UI
>Reporter: Amrit Sarkar
> Attachments: SOLR-10209.patch, SOLR-10209.patch, SOLR-10209.patch,
> SOLR-10209-v1.patch
>
>
> We are having discussion on multiple jiras for requests for Collections apis
> from UI and how to improve them:
> -SOLR-9818: Solr admin UI rapidly retries any request(s) if it loses
> connection with the server-
> -SOLR-10146: Admin UI: Button to delete a shard-
> SOLR-10201: Add Collection "creates collection", "Connection to Solr lost",
> when replicationFactor>1
> Proposal =>
> *Phase 1:*
> Convert all Collections api calls to async requests and utilise REQUESTSTATUS
> to fetch the information. There will be performance hit, but the requests
> will be safe and sound. A progress bar will be added for request status.
> {noformat}
> > submit the async request
> if (the initial call failed or there was no status to be found)
> { report an error and suggest the user look check their system before
> resubmitting the request. Bail out in this case, no retries, no attempt to
> drive on. }
> else
> { put up a progress indicator while periodically checking the status,
> Continue spinning until we can report the final status. }
> {noformat}
> *Phase 2:*
> Add new buttons/features to collections.html
> a) "Split" shard
> -b) "Delete" shard-
> c) "Backup" collection
> d) "Restore" collection
> Open to suggestions and feedbacks on this.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-9530) Add an Atomic Update Processor
[
https://issues.apache.org/jira/browse/SOLR-9530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-9530:
---
Attachment: commit()-doesn't-work.png
assertU(...)-works.png
SOLR-9530.patch
Figured out. _commit()_ itself doesn't work but *assertU(commit())*.
I have attached screenshots for the same, looked into assertU(..)
implementation, don't know what special is there. Thank you Erick for looking
into it.
Updated patch, Noble I think we got everything correct this time. Kindly review
whenever you get time.
> Add an Atomic Update Processor
> ---
>
> Key: SOLR-9530
> URL: https://issues.apache.org/jira/browse/SOLR-9530
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Varun Thacker
> Attachments: assertU(...)-works.png, commit()-doesn't-work.png,
> SOLR-9530.patch, SOLR-9530.patch, SOLR-9530.patch, SOLR-9530.patch,
> SOLR-9530.patch, SOLR-9530.patch, SOLR-9530.patch, SOLR-9530.patch,
> SOLR-9530.patch, SOLR-9530.patch, SOLR-9530.patch, SOLR-9530.patch
>
>
> I'd like to explore the idea of adding a new update processor to help ingest
> partial updates.
> Example use-case - There are two datasets with a common id field. How can I
> merge both of them at index time?
> Proposed Solution:
> {code}
>
>
> add
>
>
>
>
> {code}
> So the first JSON dump could be ingested against
> {{http://localhost:8983/solr/gettingstarted/update/json}}
> And then the second JSON could be ingested against
> {{http://localhost:8983/solr/gettingstarted/update/json?processor=atomic}}
> The Atomic Update Processor could support all the atomic update operations
> currently supported.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9530) Add an Atomic Update Processor
[
https://issues.apache.org/jira/browse/SOLR-9530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15985035#comment-15985035
]
Amrit Sarkar commented on SOLR-9530:
Yeah it worked. If I put _commitwithin_ anything > 0 say X, and put sleep for Y
>> X, I don't need to reload or issue commit() and it is working as expected.
Well I also put ModifiableSolrParam, "commit", "true" like we do in a curl
update. That too is not getting picked for explicit commit by the test.
{code}
AddUpdateCommand cmd = new AddUpdateCommand(new
LocalSolrQueryRequest(h.getCore(),
new ModifiableSolrParams()
.add("processor", "Atomic")
.add("Atomic.cat", "delete")
.add("commit","true")
));
{code}
{code}
factory.getInstance(cmd.getReq(), new SolrQueryResponse(),
new DistributedUpdateProcessor(cmd.getReq(), new
SolrQueryResponse(),
new RunUpdateProcessor(cmd.getReq(),
null))).processAdd(cmd);
{code}
I like the stop-gap loop, retries are better, will incorporate this. Thank you.
> Add an Atomic Update Processor
> ---
>
> Key: SOLR-9530
> URL: https://issues.apache.org/jira/browse/SOLR-9530
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Varun Thacker
> Attachments: SOLR-9530.patch, SOLR-9530.patch, SOLR-9530.patch,
> SOLR-9530.patch, SOLR-9530.patch, SOLR-9530.patch, SOLR-9530.patch,
> SOLR-9530.patch, SOLR-9530.patch, SOLR-9530.patch, SOLR-9530.patch
>
>
> I'd like to explore the idea of adding a new update processor to help ingest
> partial updates.
> Example use-case - There are two datasets with a common id field. How can I
> merge both of them at index time?
> Proposed Solution:
> {code}
>
>
> add
>
>
>
>
> {code}
> So the first JSON dump could be ingested against
> {{http://localhost:8983/solr/gettingstarted/update/json}}
> And then the second JSON could be ingested against
> {{http://localhost:8983/solr/gettingstarted/update/json?processor=atomic}}
> The Atomic Update Processor could support all the atomic update operations
> currently supported.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-9530) Add an Atomic Update Processor
[
https://issues.apache.org/jira/browse/SOLR-9530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-9530:
---
Attachment: SOLR-9530.patch
Been a while posted a patch with the updates.
Following up with Noble's patch, corrected test methodsl
Small concern:
I have a "nocommit" in one of the test methods where we need to issue explicit
commit. I don't know how to do that via SolrTestCaseJ4. the "commit()" method
doesn't work.
> Add an Atomic Update Processor
> ---
>
> Key: SOLR-9530
> URL: https://issues.apache.org/jira/browse/SOLR-9530
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Varun Thacker
> Attachments: SOLR-9530.patch, SOLR-9530.patch, SOLR-9530.patch,
> SOLR-9530.patch, SOLR-9530.patch, SOLR-9530.patch, SOLR-9530.patch,
> SOLR-9530.patch, SOLR-9530.patch, SOLR-9530.patch
>
>
> I'd like to explore the idea of adding a new update processor to help ingest
> partial updates.
> Example use-case - There are two datasets with a common id field. How can I
> merge both of them at index time?
> Proposed Solution:
> {code}
>
>
> add
>
>
>
>
> {code}
> So the first JSON dump could be ingested against
> {{http://localhost:8983/solr/gettingstarted/update/json}}
> And then the second JSON could be ingested against
> {{http://localhost:8983/solr/gettingstarted/update/json?processor=atomic}}
> The Atomic Update Processor could support all the atomic update operations
> currently supported.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11200) provide a config to enable disable ConcurrentMergeSchedule.doAutoIOThrottle
[
https://issues.apache.org/jira/browse/SOLR-11200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-11200:
Attachment: SOLR-11200.patch
[~varunthacker] [~niqbal],
{{isAutoIOThrottle}} will be a suitable name? Patch attached. I will finish the
tests too if we agree on the param name.
> provide a config to enable disable ConcurrentMergeSchedule.doAutoIOThrottle
> ---
>
> Key: SOLR-11200
> URL: https://issues.apache.org/jira/browse/SOLR-11200
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Nawab Zada Asad iqbal
>Priority: Minor
> Attachments: SOLR-11200.patch
>
>
> This config can be useful while bulk indexing. Lucene introduced it
> https://issues.apache.org/jira/browse/LUCENE-6119 .
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11200) provide a config to enable disable ConcurrentMergeSchedule.doAutoIOThrottle
[
https://issues.apache.org/jira/browse/SOLR-11200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115348#comment-16115348
]
Amrit Sarkar commented on SOLR-11200:
-
Ah! I don't this will serve our purpose in bulk indexing, logs ::
{code}
mergeScheduler=ConcurrentMergeScheduler: maxThreadCount=5, maxMergeCount=15,
ioThrottle=false
2017-08-05 09:14:03.005 INFO (qtp1205044462-19) [c:collection1 s:shard1
r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream
[MergeScheduler][qtp1205044462-19]: updateMergeThreads ioThrottle=false
targetMBPerSec=10240.0 MB/sec
mergeScheduler=ConcurrentMergeScheduler: maxThreadCount=5, maxMergeCount=15,
ioThrottle=false
2017-08-05 09:15:51.196 INFO (qtp1205044462-69) [c:collection1 s:shard1
r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream
[MergeScheduler][qtp1205044462-69]: updateMergeThreads ioThrottle=false
targetMBPerSec=20.0 MB/sec
2017-08-05 09:15:56.711 INFO (Lucene Merge Thread #0) [c:collection1 s:shard1
r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream
[MergeScheduler][Lucene Merge Thread #0]: updateMergeThreads ioThrottle=false
targetMBPerSec=20.0 MB/sec
2017-08-05 09:16:10.752 INFO (qtp1205044462-17) [c:collection1 s:shard1
r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream
[MergeScheduler][qtp1205044462-17]: updateMergeThreads ioThrottle=false
targetMBPerSec=20.0 MB/sec
2017-08-05 09:16:18.229 INFO (Lucene Merge Thread #1) [c:collection1 s:shard1
r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream
[MergeScheduler][Lucene Merge Thread #1]: updateMergeThreads ioThrottle=false
targetMBPerSec=20.0 MB/sec
2017-08-05 09:16:26.516 INFO (qtp1205044462-69) [c:collection1 s:shard1
r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream
[MergeScheduler][qtp1205044462-69]: updateMergeThreads ioThrottle=false
targetMBPerSec=20.0 MB/sec
2017-08-05 09:16:35.551 INFO (Lucene Merge Thread #2) [c:collection1 s:shard1
r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream
[MergeScheduler][Lucene Merge Thread #2]: updateMergeThreads ioThrottle=false
targetMBPerSec=20.0 MB/sec
2017-08-05 09:16:38.580 INFO (qtp1205044462-18) [c:collection1 s:shard1
r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream
[MergeScheduler][qtp1205044462-18]: updateMergeThreads ioThrottle=false
targetMBPerSec=20.0 MB/sec
2017-08-05 09:16:49.397 INFO (Lucene Merge Thread #3) [c:collection1 s:shard1
r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream
[MergeScheduler][Lucene Merge Thread #3]: updateMergeThreads ioThrottle=false
targetMBPerSec=20.0 MB/sec
2017-08-05 09:16:56.630 INFO (qtp1205044462-15) [c:collection1 s:shard1
r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream
[MergeScheduler][qtp1205044462-15]: updateMergeThreads ioThrottle=false
targetMBPerSec=20.0 MB/sec
{code}
See the {{targetMBPerSec}} is initialised to {{10gbps}}, but then it falls back
to default {{20mbps}}, instead of maintaining at 10gbps. Maybe
{{SolrIndexConfig#buildMergeSchedule}} is not the right place to do it. I will
look more.
> provide a config to enable disable ConcurrentMergeSchedule.doAutoIOThrottle
> ---
>
> Key: SOLR-11200
> URL: https://issues.apache.org/jira/browse/SOLR-11200
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Nawab Zada Asad iqbal
>Priority: Minor
> Attachments: SOLR-11200.patch, SOLR-11200.patch
>
>
> This config can be useful while bulk indexing. Lucene introduced it
> https://issues.apache.org/jira/browse/LUCENE-6119 .
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-11200) provide a config to enable disable ConcurrentMergeSchedule.doAutoIOThrottle
[
https://issues.apache.org/jira/browse/SOLR-11200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-11200:
Attachment: SOLR-11200.patch
Patch attached ::
{code}
modified:
solr/core/src/java/org/apache/solr/update/SolrIndexConfig.java
new file:
solr/core/src/test-files/solr/collection1/conf/solrconfig-concurrentmergescheduler.xml
modified:
solr/core/src/test/org/apache/solr/update/SolrIndexConfigTest.java
{code}
Had to create {{solrconfig-concurrentmergescheduler.xml}} as other solr-configs
in test are getting used in other tests.
> provide a config to enable disable ConcurrentMergeSchedule.doAutoIOThrottle
> ---
>
> Key: SOLR-11200
> URL: https://issues.apache.org/jira/browse/SOLR-11200
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Nawab Zada Asad iqbal
>Priority: Minor
> Attachments: SOLR-11200.patch, SOLR-11200.patch
>
>
> This config can be useful while bulk indexing. Lucene introduced it
> https://issues.apache.org/jira/browse/LUCENE-6119 .
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11200) provide a config to enable disable ConcurrentMergeSchedule.doAutoIOThrottle
[
https://issues.apache.org/jira/browse/SOLR-11200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117144#comment-16117144
]
Amrit Sarkar commented on SOLR-11200:
-
The above patch works just fine, the logging isn't obvious which created
significant confusion ::
{code}
2017-08-07 17:18:18.203 INFO (Lucene Merge Thread #1) [c:collection1 s:shard1
r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream
[MergeScheduler][Lucene Merge Thread #1]: updateMergeThreads ioThrottle=false
targetMBPerSec=20.0 MB/sec
merge thread Lucene Merge Thread #0 estSize=3.2 MB (written=2.0 MB)
runTime=156.3s (stopped=0.0s, paused=0.0s) rate=unlimited
leave running at Infinity MB/sec
{code}
Even if the targetMBPersec=20, the merges are happening at {{rate=unlimited}},
maximum possible disk write speed.
{code}
merge thread Lucene Merge Thread #0 estSize=29.4 MB (written=0.0 MB)
runTime=0.0s (stopped=0.0s, paused=0.0s) rate=unlimited
merge thread Lucene Merge Thread #1 estSize=77.6 MB (written=0.0 MB)
runTime=0.0s (stopped=0.0s, paused=0.0s) rate=unlimited
merge thread Lucene Merge Thread #2 estSize=86.6 MB (written=0.0 MB)
runTime=0.0s (stopped=0.0s, paused=0.0s) rate=unlimited
merge thread Lucene Merge Thread #1 estSize=77.6 MB (written=76.1 MB)
runTime=10.0s (stopped=0.0s, paused=0.0s) rate=unlimited
merge thread Lucene Merge Thread #2 estSize=86.6 MB (written=1.0 MB)
runTime=-0.0s (stopped=0.0s, paused=0.0s) rate=unlimited
merge thread Lucene Merge Thread #3 estSize=133.9 MB (written=0.0 MB)
runTime=0.0s (stopped=0.0s, paused=0.0s) rate=unlimited
merge thread Lucene Merge Thread #3 estSize=133.9 MB (written=132.3 MB)
runTime=12.8s (stopped=0.0s, paused=0.0s) rate=unlimited
merge thread Lucene Merge Thread #4 estSize=71.9 MB (written=0.0 MB)
runTime=0.0s (stopped=0.0s, paused=0.0s) rate=unlimited
merge thread Lucene Merge Thread #4 estSize=71.9 MB (written=0.0 MB)
runTime=0.0s (stopped=0.0s, paused=0.0s) rate=unlimited
merge thread Lucene Merge Thread #5 estSize=82.0 MB (written=0.0 MB)
runTime=0.0s (stopped=0.0s, paused=0.0s) rate=unlimited
merge thread Lucene Merge Thread #6 estSize=92.5 MB (written=0.0 MB)
runTime=0.0s (stopped=0.0s, paused=0.0s) rate=unlimited
merge thread Lucene Merge Thread #7 estSize=128.2 MB (written=0.0 MB)
runTime=0.0s (stopped=0.0s, paused=0.0s) rate=unlimited
merge thread Lucene Merge Thread #7 estSize=128.2 MB (written=117.2 MB)
runTime=12.2s (stopped=0.0s, paused=0.0s) rate=unlimited
merge thread Lucene Merge Thread #8 estSize=66.7 MB (written=0.0 MB)
runTime=0.0s (stopped=0.0s, paused=0.0s) rate=unlimited
merge thread Lucene Merge Thread #8 estSize=66.7 MB (written=21.1 MB)
runTime=0.8s (stopped=0.0s, paused=0.0s) rate=unlimited
merge thread Lucene Merge Thread #9 estSize=206.2 MB (written=0.0 MB)
runTime=0.0s (stopped=0.0s, paused=0.0s) rate=unlimited
merge thread Lucene Merge Thread #8 estSize=66.7 MB (written=65.1 MB)
runTime=9.2s (stopped=0.0s, paused=0.0s) rate=unlimited
merge thread Lucene Merge Thread #9 estSize=206.2 MB (written=0.0 MB)
runTime=0.0s (stopped=0.0s, paused=0.0s) rate=unlimited
merge thread Lucene Merge Thread #9 estSize=206.2 MB (written=191.5 MB)
runTime=15.3s (stopped=0.0s, paused=0.0s) rate=unlimited
merge thread Lucene Merge Thread #10 estSize=146.7 MB (written=0.0 MB)
runTime=0.0s (stopped=0.0s, paused=0.0s) rate=unlimited
merge thread Lucene Merge Thread #10 estSize=146.7 MB (written=47.3 MB)
runTime=1.9s (stopped=0.0s, paused=0.0s) rate=unlimited
merge thread Lucene Merge Thread #11 estSize=280.9 MB (written=0.0 MB)
runTime=0.0s (stopped=0.0s, paused=0.0s) rate=unlimited
merge thread Lucene Merge Thread #10 estSize=146.7 MB (written=143.3 MB)
runTime=17.2s (stopped=0.0s, paused=0.0s) rate=unlimited
merge thread Lucene Merge Thread #11 estSize=280.9 MB (written=1.0 MB)
runTime=0.3s (stopped=0.0s, paused=0.0s) rate=unlimited
merge thread Lucene Merge Thread #11 estSize=280.9 MB (written=143.3 MB)
runTime=12.0s (stopped=0.0s, paused=0.0s) rate=unlimited
merge thread Lucene Merge Thread #12 estSize=100.8 MB (written=0.0 MB)
runTime=0.0s (stopped=0.0s, paused=0.0s) rate=unlimited
merge thread Lucene Merge Thread #11 estSize=280.9 MB (written=208.3 MB)
runTime=24.4s (stopped=0.0s, paused=0.0s) rate=unlimited
merge thread Lucene Merge Thread #11 estSize=280.9 MB (written=235.3 MB)
runTime=28.2s (stopped=0.0s, paused=0.0s) rate=unlimited
merge thread Lucene Merge Thread #13 estSize=193.6 MB (written=0.0 MB)
runTime=0.0s (stopped=0.0s, paused=0.0s) rate=unlimited
merge thread Lucene Merge Thread #13 estSize=193.6 MB (written=79.3 MB)
runTime=4.7s (stopped=0.0s, paused=0.0s) rate=unlimited
merge thread Lucene Merge Thread #13 estSize=193.6 MB (written=155.4 MB)
runTime=16.1s (stopped=0.0s, paused=0.0s) rate=unlimited
merge thread Lucene Merge Thread #14 estSize=35.5 MB (written=0.0 MB)
runTime=0.0s (stopped=0.0s, paused=0.0s) rate=unlimited
merge thread
[jira] [Comment Edited] (SOLR-11200) provide a config to enable disable ConcurrentMergeSchedule.doAutoIOThrottle
[
https://issues.apache.org/jira/browse/SOLR-11200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115348#comment-16115348
]
Amrit Sarkar edited comment on SOLR-11200 at 8/7/17 4:10 PM:
-
Ah! I don't think this will serve our purpose in bulk indexing, logs ::
{code}
mergeScheduler=ConcurrentMergeScheduler: maxThreadCount=5, maxMergeCount=15,
ioThrottle=false
2017-08-05 09:14:03.005 INFO (qtp1205044462-19) [c:collection1 s:shard1
r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream
[MergeScheduler][qtp1205044462-19]: updateMergeThreads ioThrottle=false
targetMBPerSec=10240.0 MB/sec
mergeScheduler=ConcurrentMergeScheduler: maxThreadCount=5, maxMergeCount=15,
ioThrottle=false
2017-08-05 09:15:51.196 INFO (qtp1205044462-69) [c:collection1 s:shard1
r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream
[MergeScheduler][qtp1205044462-69]: updateMergeThreads ioThrottle=false
targetMBPerSec=20.0 MB/sec
2017-08-05 09:15:56.711 INFO (Lucene Merge Thread #0) [c:collection1 s:shard1
r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream
[MergeScheduler][Lucene Merge Thread #0]: updateMergeThreads ioThrottle=false
targetMBPerSec=20.0 MB/sec
2017-08-05 09:16:10.752 INFO (qtp1205044462-17) [c:collection1 s:shard1
r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream
[MergeScheduler][qtp1205044462-17]: updateMergeThreads ioThrottle=false
targetMBPerSec=20.0 MB/sec
2017-08-05 09:16:18.229 INFO (Lucene Merge Thread #1) [c:collection1 s:shard1
r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream
[MergeScheduler][Lucene Merge Thread #1]: updateMergeThreads ioThrottle=false
targetMBPerSec=20.0 MB/sec
2017-08-05 09:16:26.516 INFO (qtp1205044462-69) [c:collection1 s:shard1
r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream
[MergeScheduler][qtp1205044462-69]: updateMergeThreads ioThrottle=false
targetMBPerSec=20.0 MB/sec
2017-08-05 09:16:35.551 INFO (Lucene Merge Thread #2) [c:collection1 s:shard1
r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream
[MergeScheduler][Lucene Merge Thread #2]: updateMergeThreads ioThrottle=false
targetMBPerSec=20.0 MB/sec
2017-08-05 09:16:38.580 INFO (qtp1205044462-18) [c:collection1 s:shard1
r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream
[MergeScheduler][qtp1205044462-18]: updateMergeThreads ioThrottle=false
targetMBPerSec=20.0 MB/sec
2017-08-05 09:16:49.397 INFO (Lucene Merge Thread #3) [c:collection1 s:shard1
r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream
[MergeScheduler][Lucene Merge Thread #3]: updateMergeThreads ioThrottle=false
targetMBPerSec=20.0 MB/sec
2017-08-05 09:16:56.630 INFO (qtp1205044462-15) [c:collection1 s:shard1
r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream
[MergeScheduler][qtp1205044462-15]: updateMergeThreads ioThrottle=false
targetMBPerSec=20.0 MB/sec
{code}
See the {{targetMBPerSec}} is initialised to {{10gbps}}, but then it falls back
to default {{20mbps}}, instead of maintaining at 10gbps. Maybe
{{SolrIndexConfig#buildMergeSchedule}} is not the right place to do it. I will
look more.
was (Author: sarkaramr...@gmail.com):
Ah! I don't this will serve our purpose in bulk indexing, logs ::
{code}
mergeScheduler=ConcurrentMergeScheduler: maxThreadCount=5, maxMergeCount=15,
ioThrottle=false
2017-08-05 09:14:03.005 INFO (qtp1205044462-19) [c:collection1 s:shard1
r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream
[MergeScheduler][qtp1205044462-19]: updateMergeThreads ioThrottle=false
targetMBPerSec=10240.0 MB/sec
mergeScheduler=ConcurrentMergeScheduler: maxThreadCount=5, maxMergeCount=15,
ioThrottle=false
2017-08-05 09:15:51.196 INFO (qtp1205044462-69) [c:collection1 s:shard1
r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream
[MergeScheduler][qtp1205044462-69]: updateMergeThreads ioThrottle=false
targetMBPerSec=20.0 MB/sec
2017-08-05 09:15:56.711 INFO (Lucene Merge Thread #0) [c:collection1 s:shard1
r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream
[MergeScheduler][Lucene Merge Thread #0]: updateMergeThreads ioThrottle=false
targetMBPerSec=20.0 MB/sec
2017-08-05 09:16:10.752 INFO (qtp1205044462-17) [c:collection1 s:shard1
r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream
[MergeScheduler][qtp1205044462-17]: updateMergeThreads ioThrottle=false
targetMBPerSec=20.0 MB/sec
2017-08-05 09:16:18.229 INFO (Lucene Merge Thread #1) [c:collection1 s:shard1
r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream
[MergeScheduler][Lucene Merge Thread #1]: updateMergeThreads ioThrottle=false
targetMBPerSec=20.0 MB/sec
2017-08-05 09:16:26.516 INFO (qtp1205044462-69) [c:collection1 s:shard1
r:core_node2
[jira] [Created] (SOLR-11278) CdcrBootstrapTest.testBootstrapWithSourceCluster failing in branch_6_6
Amrit Sarkar created SOLR-11278:
---
Summary: CdcrBootstrapTest.testBootstrapWithSourceCluster failing
in branch_6_6
Key: SOLR-11278
URL: https://issues.apache.org/jira/browse/SOLR-11278
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Components: CDCR
Affects Versions: 6.6.1
Reporter: Amrit Sarkar
I ran beast for 10 rounds:
ant beast -Dtestcase=CdcrBootstrapTest -Dtests.multiplier=2 -Dtests.slow=true
-Dtests.locale=vi -Dtests.timezone=Asia/Yekaterinburg -Dtests.asserts=true
-Dtests.file.encoding=US-ASCII -Dbeast.iters=10
and seeing following failure:
{code}
[beaster] [01:37:16.282] FAILURE 153s |
CdcrBootstrapTest.testBootstrapWithSourceCluster <<<
[beaster]> Throwable #1: java.lang.AssertionError: Document mismatch on
target after sync expected:<2000> but was:<1000>
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10564) NPE in QueryComponent when RTG
[
https://issues.apache.org/jira/browse/SOLR-10564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10564:
Attachment: SOLR-10564.patch
Added a very basic level test in
SolrCloudExampleTest::testLoadDocsIntoGettingStartedCollection()
Didn't want to add or subtract much so went on with and added relevant lines.
Requesting someone to kindly review as this is a legit bug.
> NPE in QueryComponent when RTG
> --
>
> Key: SOLR-10564
> URL: https://issues.apache.org/jira/browse/SOLR-10564
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 6.5
>Reporter: Markus Jelsma
> Fix For: master (7.0)
>
> Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png,
> screenshot-4.png, screenshot-5.png, SOLR-10564.patch, SOLR-10564.patch
>
>
> The following URL:
> {code}
> /get?fl=queries,prob_*,view_score,feedback_score=
> {code}
> Kindly returns the document.
> This once, however:
> {code}
> /select?qt=/get=queries,prob_*,view_score,feedback_score=
> {code}
> throws:
> {code}
> 2017-04-25 10:23:26.222 ERROR (qtp1873653341-28693) [c:documents s:shard1
> r:core_node3 x:documents_shard1_replica1] o.a.s.s.HttpSolrCall
> null:java.lang.NullPointerException
> at
> org.apache.solr.handler.component.QueryComponent.unmarshalSortValues(QueryComponent.java:1226)
> at
> org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:1077)
> at
> org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:777)
> at
> org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:756)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:428)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2440)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:723)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:529)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:298)
> {code}
> This is thrown when i do it manually, but the error does not appear when Solr
> issues those same queries under the hood.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-7705) Allow CharTokenizer-derived tokenizers and KeywordTokenizer to configure the max token length
[ https://issues.apache.org/jira/browse/LUCENE-7705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amrit Sarkar updated LUCENE-7705: - Attachment: LUCENE-7705 > Allow CharTokenizer-derived tokenizers and KeywordTokenizer to configure the > max token length > - > > Key: LUCENE-7705 > URL: https://issues.apache.org/jira/browse/LUCENE-7705 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Amrit Sarkar >Assignee: Erick Erickson >Priority: Minor > Attachments: LUCENE-7705, LUCENE-7705.patch, LUCENE-7705.patch, > LUCENE-7705.patch, LUCENE-7705.patch, LUCENE-7705.patch, LUCENE-7705.patch, > LUCENE-7705.patch > > > SOLR-10186 > [~erickerickson]: Is there a good reason that we hard-code a 256 character > limit for the CharTokenizer? In order to change this limit it requires that > people copy/paste the incrementToken into some new class since incrementToken > is final. > KeywordTokenizer can easily change the default (which is also 256 bytes), but > to do so requires code rather than being able to configure it in the schema. > For KeywordTokenizer, this is Solr-only. For the CharTokenizer classes > (WhitespaceTokenizer, UnicodeWhitespaceTokenizer and LetterTokenizer) > (Factories) it would take adding a c'tor to the base class in Lucene and > using it in the factory. > Any objections? -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-7705) Allow CharTokenizer-derived tokenizers and KeywordTokenizer to configure the max token length
[
https://issues.apache.org/jira/browse/LUCENE-7705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated LUCENE-7705:
-
Attachment: LUCENE-7705
Yes Erick, I saw the "ant precommit" errors, tab instead of whitespaces, got it.
I am still seeing this:
{code}
[junit4] Tests with failures [seed: C3F5B66314F27B5E]:
[junit4] -
org.apache.solr.util.TestMaxTokenLenTokenizer.testSingleFieldSameAnalyzers
{code}
{code}
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=TestMaxTokenLenTokenizer -Dtests.method=testSingleFieldSameAnalyzers
-Dtests.seed=C3F5B66314F27B5E -Dtests.slow=true -Dtests.locale=fr-CA
-Dtests.timezone=Asia/Qatar -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.10s |
TestMaxTokenLenTokenizer.testSingleFieldSameAnalyzers <<<
[junit4]> Throwable #1: java.lang.RuntimeException: Exception during
query
[junit4]>at
__randomizedtesting.SeedInfo.seed([C3F5B66314F27B5E:A927890C4C11AB91]:0)
[junit4]>at
org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:896)
[junit4]>at
org.apache.solr.util.TestMaxTokenLenTokenizer.testSingleFieldSameAnalyzers(TestMaxTokenLenTokenizer.java:104)
[junit4]>at java.lang.Thread.run(Thread.java:745)
[junit4]> Caused by: java.lang.RuntimeException: REQUEST FAILED:
xpath=//result[@numFound=1]
[junit4]>xml response was:
[junit4]>
[junit4]> 011
[junit4]>
[junit4]>request was:q=letter0:lett=xml
[junit4]>at
org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:889)
[junit4]>... 40 more
{code}
But if it working for you, I am good.
You didn't include the newly created files again in the latest patch, I have
posted a new one with "precommit" sorted and included all the files.
> Allow CharTokenizer-derived tokenizers and KeywordTokenizer to configure the
> max token length
> -
>
> Key: LUCENE-7705
> URL: https://issues.apache.org/jira/browse/LUCENE-7705
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Amrit Sarkar
>Assignee: Erick Erickson
>Priority: Minor
> Attachments: LUCENE-7705, LUCENE-7705.patch, LUCENE-7705.patch,
> LUCENE-7705.patch, LUCENE-7705.patch, LUCENE-7705.patch, LUCENE-7705.patch,
> LUCENE-7705.patch
>
>
> SOLR-10186
> [~erickerickson]: Is there a good reason that we hard-code a 256 character
> limit for the CharTokenizer? In order to change this limit it requires that
> people copy/paste the incrementToken into some new class since incrementToken
> is final.
> KeywordTokenizer can easily change the default (which is also 256 bytes), but
> to do so requires code rather than being able to configure it in the schema.
> For KeywordTokenizer, this is Solr-only. For the CharTokenizer classes
> (WhitespaceTokenizer, UnicodeWhitespaceTokenizer and LetterTokenizer)
> (Factories) it would take adding a c'tor to the base class in Lucene and
> using it in the factory.
> Any objections?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-7705) Allow CharTokenizer-derived tokenizers and KeywordTokenizer to configure the max token length
[ https://issues.apache.org/jira/browse/LUCENE-7705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amrit Sarkar updated LUCENE-7705: - Attachment: (was: LUCENE-7705) > Allow CharTokenizer-derived tokenizers and KeywordTokenizer to configure the > max token length > - > > Key: LUCENE-7705 > URL: https://issues.apache.org/jira/browse/LUCENE-7705 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Amrit Sarkar >Assignee: Erick Erickson >Priority: Minor > Attachments: LUCENE-7705.patch, LUCENE-7705.patch, LUCENE-7705.patch, > LUCENE-7705.patch, LUCENE-7705.patch, LUCENE-7705.patch, LUCENE-7705.patch > > > SOLR-10186 > [~erickerickson]: Is there a good reason that we hard-code a 256 character > limit for the CharTokenizer? In order to change this limit it requires that > people copy/paste the incrementToken into some new class since incrementToken > is final. > KeywordTokenizer can easily change the default (which is also 256 bytes), but > to do so requires code rather than being able to configure it in the schema. > For KeywordTokenizer, this is Solr-only. For the CharTokenizer classes > (WhitespaceTokenizer, UnicodeWhitespaceTokenizer and LetterTokenizer) > (Factories) it would take adding a c'tor to the base class in Lucene and > using it in the factory. > Any objections? -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-7705) Allow CharTokenizer-derived tokenizers and KeywordTokenizer to configure the max token length
[ https://issues.apache.org/jira/browse/LUCENE-7705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amrit Sarkar updated LUCENE-7705: - Attachment: LUCENE-7705.patch Erick, I have absolutely no idea how I uploaded false/incomplete patch. I certainly understand the above and incorporated two different configurations to show the difference at first place, refined the same as per your latest comments. There is one serious issue I am facing for a day, all the tests passes on IntelliJ IDE but the same gets failed when I do _"ant -Dtestcase=TestMaxTokenLenTokenizer test"_. I don't know what to do with that. The latest patch you uploaded is incomplete as the newly created files are not part of the patch. I have worked on the previous one (dated: 27-07-17). You may need to change the pre-commit changes on the latest patch. Sorry for the hiccup. > Allow CharTokenizer-derived tokenizers and KeywordTokenizer to configure the > max token length > - > > Key: LUCENE-7705 > URL: https://issues.apache.org/jira/browse/LUCENE-7705 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Amrit Sarkar >Assignee: Erick Erickson >Priority: Minor > Attachments: LUCENE-7705.patch, LUCENE-7705.patch, LUCENE-7705.patch, > LUCENE-7705.patch, LUCENE-7705.patch, LUCENE-7705.patch > > > SOLR-10186 > [~erickerickson]: Is there a good reason that we hard-code a 256 character > limit for the CharTokenizer? In order to change this limit it requires that > people copy/paste the incrementToken into some new class since incrementToken > is final. > KeywordTokenizer can easily change the default (which is also 256 bytes), but > to do so requires code rather than being able to configure it in the schema. > For KeywordTokenizer, this is Solr-only. For the CharTokenizer classes > (WhitespaceTokenizer, UnicodeWhitespaceTokenizer and LetterTokenizer) > (Factories) it would take adding a c'tor to the base class in Lucene and > using it in the factory. > Any objections? -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7705) Allow CharTokenizer-derived tokenizers and KeywordTokenizer to configure the max token length
[ https://issues.apache.org/jira/browse/LUCENE-7705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004169#comment-16004169 ] Amrit Sarkar commented on LUCENE-7705: -- Erick, we got everything right this time. > Allow CharTokenizer-derived tokenizers and KeywordTokenizer to configure the > max token length > - > > Key: LUCENE-7705 > URL: https://issues.apache.org/jira/browse/LUCENE-7705 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Amrit Sarkar >Assignee: Erick Erickson >Priority: Minor > Attachments: LUCENE-7705, LUCENE-7705.patch, LUCENE-7705.patch, > LUCENE-7705.patch, LUCENE-7705.patch, LUCENE-7705.patch, LUCENE-7705.patch, > LUCENE-7705.patch > > > SOLR-10186 > [~erickerickson]: Is there a good reason that we hard-code a 256 character > limit for the CharTokenizer? In order to change this limit it requires that > people copy/paste the incrementToken into some new class since incrementToken > is final. > KeywordTokenizer can easily change the default (which is also 256 bytes), but > to do so requires code rather than being able to configure it in the schema. > For KeywordTokenizer, this is Solr-only. For the CharTokenizer classes > (WhitespaceTokenizer, UnicodeWhitespaceTokenizer and LetterTokenizer) > (Factories) it would take adding a c'tor to the base class in Lucene and > using it in the factory. > Any objections? -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10867) Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10867:
Attachment: SOLR-10867.patch
error_test
Shalin,
Sorry I uploaded old test-error-log and patch, uploaded new ones, object leaks
etc. I will take some more time to understand what different I am doing from
rest of the test cases in the project. Thank you for your help.
> Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with
> request
> -
>
> Key: SOLR-10867
> URL: https://issues.apache.org/jira/browse/SOLR-10867
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: error, error_test, SOLR-10867.patch, SOLR-10867.patch,
> SOLR-10867.patch, SOLR-10867.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> ClassificationUpdateProcessorFactory will be able to execute by sample curl
> like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=Classification=url_s=training=true
> --data-binary { "id" : "1" , "url_s" : "http://www.example.com/subroot; }
> {code}
> All the param(s) for this URP available can be passed as request handler
> param(s).
> Configuration for ClassificationUpdateProcessorFactory in solrconfig.xml is
> optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-10867) Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16046599#comment-16046599
]
Amrit Sarkar edited comment on SOLR-10867 at 6/12/17 2:11 PM:
--
Shalin,
Sorry I uploaded old test-error-log and patch, uploaded new ones, object leaks
etc. I will take some more time to understand what different I am doing from
rest of the test cases in the project. Thank you for your help.
{code}
[junit4] 2> 28893 ERROR (coreCloseExecutor-12-thread-1) [
x:collection1] o.a.s.c.CachingDirectoryFactory Timeout waiting for all
directory ref counts to be released - gave up waiting on
[jira] [Updated] (SOLR-10867) Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10867:
Attachment: (was: error)
> Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with
> request
> -
>
> Key: SOLR-10867
> URL: https://issues.apache.org/jira/browse/SOLR-10867
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: error_test, SOLR-10867.patch, SOLR-10867.patch,
> SOLR-10867.patch, SOLR-10867.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> ClassificationUpdateProcessorFactory will be able to execute by sample curl
> like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=Classification=url_s=training=true
> --data-binary { "id" : "1" , "url_s" : "http://www.example.com/subroot; }
> {code}
> All the param(s) for this URP available can be passed as request handler
> param(s).
> Configuration for ClassificationUpdateProcessorFactory in solrconfig.xml is
> optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10867) Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10867:
Attachment: SOLR-10867.patch
error
Thanks [~shalinmangar] for the correction, I have to be more careful, Well
that rectified thread leaks rightfully as only one core is being created in the
main thread, the temp index dir is still creating issue while getting removed
at core destruction.
Update patch and error:
{code}
[junit4] 2> 28935 ERROR (coreCloseExecutor-12-thread-1) [x:collection1]
o.a.s.c.CachingDirectoryFactory Timeout waiting for all directory ref counts to
be released - gave up waiting on
[jira] [Updated] (SOLR-10865) parameter processor=Template doesn't invoke TemplateUpdateProcessorFactory.
[
https://issues.apache.org/jira/browse/SOLR-10865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10865:
Attachment: SOLR-10865.patch
Figured out what the problem is =>
To leverage the _processor=Template_ or _processor=Atomic_ or _processor = X_,
where X + "UpdateRequestProcessor", must and must have system property
*-Denable.runtime.lib=true*
Now when we start a SolrCloud with default settings, I don't think we have
*enable.runtime.lib* enabled in our configs. And the error we receive at the
end of the request is:
{code}
2017-06-12 11:05:55.517 WARN (qtp846947180-13) [c:gettingstarted s:shard1
r:core_node1 x:gettingstarted_shard1_replica_n1] o.a.s.c.PluginBag runtime
library loading is not enabled, start Solr with -Denable.runtime.lib=true
org.apache.solr.common.SolrException; org.apache.solr.common.SolrException: No
such processor Template
{code}
absolutely misleading. I guess improving the error message will make the user
make the necessary changes to make it work. A nested message is better to fix
this.
Patch attached with appropriate error or exception, which will help one figure
out what to fix, where to fix when we see "No processor".
> parameter processor=Template doesn't invoke TemplateUpdateProcessorFactory.
> ---
>
> Key: SOLR-10865
> URL: https://issues.apache.org/jira/browse/SOLR-10865
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Affects Versions: master (7.0)
>Reporter: Amrit Sarkar
> Attachments: SOLR-10865.patch
>
>
> Ref docs::
> {quote}
> The TemplateUpdateProcessorFactory can be used to add new fields to documents
> based on a template pattern.
> This can be used directly in a request without any configuration. To enable
> this processor, use the parameter processor=Template.
> {quote}
> Sample curl::
> {code}
> curl -X POST -H 'Content-Type:application/json'
> 'http://localhost:8983/solr/gettingstarted/update/json/docs?processor=Template=fullName_s:AmritSarkar=true'
> --data-binary '{"id": 1,"title": "titleA"}'
> {code}
> I am receiving exception::
> {code}
>
>
> 400 name="QTime">3 name="error-class">org.apache.solr.common.SolrException name="root-error-class">org.apache.solr.common.SolrException name="msg">No such processor Template400
>
> {code}
> {code}
> ERROR - 2017-06-10 07:39:51.598; [c:gettingstarted s:shard2 r:core_node1
> x:gettingstarted_shard2_replica_n1] org.apache.solr.common.SolrException;
> org.apache.solr.common.SolrException: No such processor Template
> at
> org.apache.solr.update.processor.UpdateRequestProcessorChain.getReqProcessors(UpdateRequestProcessorChain.java:286)
> at
> org.apache.solr.update.processor.UpdateRequestProcessorChain.constructChain(UpdateRequestProcessorChain.java:235)
> {code}
> I looked into how the testclass has been created::
> TemplateUpdateProcessorTest::
> {code}
> public void testSimple() throws Exception {
> AddUpdateCommand cmd = new AddUpdateCommand(new
> LocalSolrQueryRequest(null,
> new ModifiableSolrParams()
> .add("processor", "Template")
> .add("Template.field", "id:${firstName}_${lastName}")
> .add("Template.field", "another:${lastName}_${firstName}")
> .add("Template.field", "missing:${lastName}_${unKnown}")
> ));
> cmd.solrDoc = new SolrInputDocument();
> cmd.solrDoc.addField("firstName", "Tom");
> cmd.solrDoc.addField("lastName", "Cruise");
> new TemplateUpdateProcessorFactory().getInstance(cmd.getReq(), new
> SolrQueryResponse(), null).processAdd(cmd);
> assertEquals("Tom_Cruise", cmd.solrDoc.getFieldValue("id"));
> assertEquals("Cruise_Tom", cmd.solrDoc.getFieldValue("another"));
> assertEquals("Cruise_", cmd.solrDoc.getFieldValue("missing"));
> }
> {code}
> *There is no test to check whether _processor=Template_ works or not*,
> TemplateUpdateProcessorFactory() object is EXPLICITLY created to getInstance
> and do processing on the updates.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10865) parameter processor=Template doesn't invoke TemplateUpdateProcessorFactory.
[
https://issues.apache.org/jira/browse/SOLR-10865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16046648#comment-16046648
]
Amrit Sarkar commented on SOLR-10865:
-
[~shalinmangar] agreed. This is not working after *SOLR-9530* where certain
adjustments made in *UpdateRequestProcessorChain.java* to make sure inform(...)
works for AtomicURP (related to _LazyPluginHolder_ in PluginBag.java_). I
didn't make the changes myself but I see what are you saying and we may have
gone wrong there.
patch snippet from SOLR-9530 =>
{code}
diff --git
a/solr/core/src/java/org/apache/solr/update/processor/UpdateRequestProcessorChain.java
b/solr/core/src/java/org/apache/solr/update/processor/UpdateRequestProcessorChain.java
index 0ed626cb0f..05d1a5ae3f 100644
---
a/solr/core/src/java/org/apache/solr/update/processor/UpdateRequestProcessorChain.java
+++
b/solr/core/src/java/org/apache/solr/update/processor/UpdateRequestProcessorChain.java
@@ -28,6 +28,8 @@ import org.apache.solr.common.params.MapSolrParams;
import org.apache.solr.common.params.SolrParams;
import org.apache.solr.common.util.NamedList;
import org.apache.solr.common.util.StrUtils;
+import org.apache.solr.common.util.Utils;
+import org.apache.solr.core.PluginBag;
import org.apache.solr.core.PluginInfo;
import org.apache.solr.core.SolrCore;
import org.apache.solr.request.SolrQueryRequest;
@@ -271,9 +273,13 @@ public final class UpdateRequestProcessorChain implements
PluginInfoInitialized
UpdateRequestProcessorFactory p = core.getUpdateProcessors().get(s);
if (p == null) {
try {
- p = core.createInstance(s + "UpdateProcessorFactory",
UpdateRequestProcessorFactory.class,
- "updateProcessor", null, core.getMemClassLoader());
- core.getUpdateProcessors().put(s, p);
+ PluginInfo pluginInfo = new PluginInfo("updateProcessor",
+ Utils.makeMap("name", s,
+ "class", s + "UpdateProcessorFactory",
+ "runtimeLib", "true"));
+
+ PluginBag.PluginHolder pluginHolder =
core.getUpdateProcessors().createPlugin(pluginInfo);
+ core.getUpdateProcessors().put(s, p = pluginHolder.get());
} catch (SolrException e) {
}
if (p == null)
{code}
I don't recall the exact reason behind this, but will surely look into it.
> parameter processor=Template doesn't invoke TemplateUpdateProcessorFactory.
> ---
>
> Key: SOLR-10865
> URL: https://issues.apache.org/jira/browse/SOLR-10865
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Affects Versions: master (7.0)
>Reporter: Amrit Sarkar
> Attachments: SOLR-10865.patch, SOLR-10865.patch
>
>
> Ref docs::
> {quote}
> The TemplateUpdateProcessorFactory can be used to add new fields to documents
> based on a template pattern.
> This can be used directly in a request without any configuration. To enable
> this processor, use the parameter processor=Template.
> {quote}
> Sample curl::
> {code}
> curl -X POST -H 'Content-Type:application/json'
> 'http://localhost:8983/solr/gettingstarted/update/json/docs?processor=Template=fullName_s:AmritSarkar=true'
> --data-binary '{"id": 1,"title": "titleA"}'
> {code}
> I am receiving exception::
> {code}
>
>
> 400 name="QTime">3 name="error-class">org.apache.solr.common.SolrException name="root-error-class">org.apache.solr.common.SolrException name="msg">No such processor Template400
>
> {code}
> {code}
> ERROR - 2017-06-10 07:39:51.598; [c:gettingstarted s:shard2 r:core_node1
> x:gettingstarted_shard2_replica_n1] org.apache.solr.common.SolrException;
> org.apache.solr.common.SolrException: No such processor Template
> at
> org.apache.solr.update.processor.UpdateRequestProcessorChain.getReqProcessors(UpdateRequestProcessorChain.java:286)
> at
> org.apache.solr.update.processor.UpdateRequestProcessorChain.constructChain(UpdateRequestProcessorChain.java:235)
> {code}
> I looked into how the testclass has been created::
> TemplateUpdateProcessorTest::
> {code}
> public void testSimple() throws Exception {
> AddUpdateCommand cmd = new AddUpdateCommand(new
> LocalSolrQueryRequest(null,
> new ModifiableSolrParams()
> .add("processor", "Template")
> .add("Template.field", "id:${firstName}_${lastName}")
> .add("Template.field", "another:${lastName}_${firstName}")
> .add("Template.field", "missing:${lastName}_${unKnown}")
> ));
> cmd.solrDoc = new SolrInputDocument();
> cmd.solrDoc.addField("firstName", "Tom");
> cmd.solrDoc.addField("lastName", "Cruise");
> new TemplateUpdateProcessorFactory().getInstance(cmd.getReq(), new
> SolrQueryResponse(),
[jira] [Updated] (SOLR-10865) parameter processor=Template doesn't invoke TemplateUpdateProcessorFactory.
[
https://issues.apache.org/jira/browse/SOLR-10865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10865:
Attachment: SOLR-10865.patch
Also it would be best if we specify this bit of information =>
enable *enable.runtime.lib* while starting up solr nodes to support:
processor=X to invoke XUpdateRequestProcessor.
Patch uploaded with slight adjustments.
> parameter processor=Template doesn't invoke TemplateUpdateProcessorFactory.
> ---
>
> Key: SOLR-10865
> URL: https://issues.apache.org/jira/browse/SOLR-10865
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Affects Versions: master (7.0)
>Reporter: Amrit Sarkar
> Attachments: SOLR-10865.patch, SOLR-10865.patch
>
>
> Ref docs::
> {quote}
> The TemplateUpdateProcessorFactory can be used to add new fields to documents
> based on a template pattern.
> This can be used directly in a request without any configuration. To enable
> this processor, use the parameter processor=Template.
> {quote}
> Sample curl::
> {code}
> curl -X POST -H 'Content-Type:application/json'
> 'http://localhost:8983/solr/gettingstarted/update/json/docs?processor=Template=fullName_s:AmritSarkar=true'
> --data-binary '{"id": 1,"title": "titleA"}'
> {code}
> I am receiving exception::
> {code}
>
>
> 400 name="QTime">3 name="error-class">org.apache.solr.common.SolrException name="root-error-class">org.apache.solr.common.SolrException name="msg">No such processor Template400
>
> {code}
> {code}
> ERROR - 2017-06-10 07:39:51.598; [c:gettingstarted s:shard2 r:core_node1
> x:gettingstarted_shard2_replica_n1] org.apache.solr.common.SolrException;
> org.apache.solr.common.SolrException: No such processor Template
> at
> org.apache.solr.update.processor.UpdateRequestProcessorChain.getReqProcessors(UpdateRequestProcessorChain.java:286)
> at
> org.apache.solr.update.processor.UpdateRequestProcessorChain.constructChain(UpdateRequestProcessorChain.java:235)
> {code}
> I looked into how the testclass has been created::
> TemplateUpdateProcessorTest::
> {code}
> public void testSimple() throws Exception {
> AddUpdateCommand cmd = new AddUpdateCommand(new
> LocalSolrQueryRequest(null,
> new ModifiableSolrParams()
> .add("processor", "Template")
> .add("Template.field", "id:${firstName}_${lastName}")
> .add("Template.field", "another:${lastName}_${firstName}")
> .add("Template.field", "missing:${lastName}_${unKnown}")
> ));
> cmd.solrDoc = new SolrInputDocument();
> cmd.solrDoc.addField("firstName", "Tom");
> cmd.solrDoc.addField("lastName", "Cruise");
> new TemplateUpdateProcessorFactory().getInstance(cmd.getReq(), new
> SolrQueryResponse(), null).processAdd(cmd);
> assertEquals("Tom_Cruise", cmd.solrDoc.getFieldValue("id"));
> assertEquals("Cruise_Tom", cmd.solrDoc.getFieldValue("another"));
> assertEquals("Cruise_", cmd.solrDoc.getFieldValue("missing"));
> }
> {code}
> *There is no test to check whether _processor=Template_ works or not*,
> TemplateUpdateProcessorFactory() object is EXPLICITLY created to getInstance
> and do processing on the updates.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10865) parameter processor=Template doesn't invoke TemplateUpdateProcessorFactory.
[
https://issues.apache.org/jira/browse/SOLR-10865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10865:
Attachment: SOLR-10865.patch
The URPFactories are not allowed to Lazy Initialized and hence inform(Solrcore
...) never gets called for URPs which are not defined in SolrConfig. The one
mentioned in config, get through their inform(...) at core creation.
If we are making URPs free of config, we need to allow processors to be
initialized lazy (after core creation) and instead of using flag runtimeLib
flag to get LazyPluginHolder, we can pass _startup=true_, justifiable and neat.
Attached patch for the same. Some minor corrections, blank spaces / lines here
and there.
> parameter processor=Template doesn't invoke TemplateUpdateProcessorFactory.
> ---
>
> Key: SOLR-10865
> URL: https://issues.apache.org/jira/browse/SOLR-10865
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Affects Versions: master (7.0)
>Reporter: Amrit Sarkar
> Attachments: SOLR-10865.patch, SOLR-10865.patch, SOLR-10865.patch
>
>
> Ref docs::
> {quote}
> The TemplateUpdateProcessorFactory can be used to add new fields to documents
> based on a template pattern.
> This can be used directly in a request without any configuration. To enable
> this processor, use the parameter processor=Template.
> {quote}
> Sample curl::
> {code}
> curl -X POST -H 'Content-Type:application/json'
> 'http://localhost:8983/solr/gettingstarted/update/json/docs?processor=Template=fullName_s:AmritSarkar=true'
> --data-binary '{"id": 1,"title": "titleA"}'
> {code}
> I am receiving exception::
> {code}
>
>
> 400 name="QTime">3 name="error-class">org.apache.solr.common.SolrException name="root-error-class">org.apache.solr.common.SolrException name="msg">No such processor Template400
>
> {code}
> {code}
> ERROR - 2017-06-10 07:39:51.598; [c:gettingstarted s:shard2 r:core_node1
> x:gettingstarted_shard2_replica_n1] org.apache.solr.common.SolrException;
> org.apache.solr.common.SolrException: No such processor Template
> at
> org.apache.solr.update.processor.UpdateRequestProcessorChain.getReqProcessors(UpdateRequestProcessorChain.java:286)
> at
> org.apache.solr.update.processor.UpdateRequestProcessorChain.constructChain(UpdateRequestProcessorChain.java:235)
> {code}
> I looked into how the testclass has been created::
> TemplateUpdateProcessorTest::
> {code}
> public void testSimple() throws Exception {
> AddUpdateCommand cmd = new AddUpdateCommand(new
> LocalSolrQueryRequest(null,
> new ModifiableSolrParams()
> .add("processor", "Template")
> .add("Template.field", "id:${firstName}_${lastName}")
> .add("Template.field", "another:${lastName}_${firstName}")
> .add("Template.field", "missing:${lastName}_${unKnown}")
> ));
> cmd.solrDoc = new SolrInputDocument();
> cmd.solrDoc.addField("firstName", "Tom");
> cmd.solrDoc.addField("lastName", "Cruise");
> new TemplateUpdateProcessorFactory().getInstance(cmd.getReq(), new
> SolrQueryResponse(), null).processAdd(cmd);
> assertEquals("Tom_Cruise", cmd.solrDoc.getFieldValue("id"));
> assertEquals("Cruise_Tom", cmd.solrDoc.getFieldValue("another"));
> assertEquals("Cruise_", cmd.solrDoc.getFieldValue("missing"));
> }
> {code}
> *There is no test to check whether _processor=Template_ works or not*,
> TemplateUpdateProcessorFactory() object is EXPLICITLY created to getInstance
> and do processing on the updates.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10858) Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10858:
Attachment: SOLR-10858.patch
Patch updated with SOLR-9565 design.
> Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
> ---
>
> Key: SOLR-10858
> URL: https://issues.apache.org/jira/browse/SOLR-10858
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch,
> SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch,
> SOLR-10858.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10866) Make TimestampUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10866:
Description:
We are trying to get rid of processor definitions in SolrConfig for all URPs
and take parameters in the request itself.
TimestampUpdateProcessorFactory will be able to execute by sample curl like
below:
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=timestamp=timestamp_tdt=true
--data-binary { "id" : "1" , "title_s" : "titleA" }
{code}
Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
was:
We are trying to get rid of processor definitions in SolrConfig for all URPs
and take parameters in the request itself.
TimestampUpdateProcessorFactory will be able to execute by sample curl like
below:
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=Timestamp=timestamp_tdt=true
--data-binary { "id" : "1" , "title_s" : "titleA" }
{code}
Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
> Make TimestampUpdateProcessorFactory as Runtime URP; take params(s) with
> request
>
>
> Key: SOLR-10866
> URL: https://issues.apache.org/jira/browse/SOLR-10866
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10866.patch, SOLR-10866.patch, SOLR-10866.patch,
> SOLR-10866.patch, SOLR-10866.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> TimestampUpdateProcessorFactory will be able to execute by sample curl like
> below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=timestamp=timestamp_tdt=true
> --data-binary { "id" : "1" , "title_s" : "titleA" }
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10866) Make TimestampUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10866:
Description:
We are trying to get rid of processor definitions in SolrConfig for all URPs
and take parameters in the request itself.
TimestampUpdateProcessorFactory will be able to execute by sample curl like
below:
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=timestamp=timestamp_tdt=true
--data-binary { "id" : "1" , "title_s" : "titleA" }
{code}
Configuration for TimestampUpdateProcessorFactory in solrconfig.xml is optional.
was:
We are trying to get rid of processor definitions in SolrConfig for all URPs
and take parameters in the request itself.
TimestampUpdateProcessorFactory will be able to execute by sample curl like
below:
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=timestamp=timestamp_tdt=true
--data-binary { "id" : "1" , "title_s" : "titleA" }
{code}
Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
> Make TimestampUpdateProcessorFactory as Runtime URP; take params(s) with
> request
>
>
> Key: SOLR-10866
> URL: https://issues.apache.org/jira/browse/SOLR-10866
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10866.patch, SOLR-10866.patch, SOLR-10866.patch,
> SOLR-10866.patch, SOLR-10866.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> TimestampUpdateProcessorFactory will be able to execute by sample curl like
> below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=timestamp=timestamp_tdt=true
> --data-binary { "id" : "1" , "title_s" : "titleA" }
> {code}
> Configuration for TimestampUpdateProcessorFactory in solrconfig.xml is
> optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10866) Make TimestampUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10866:
Attachment: SOLR-10866.patch
patch modified SOLR-9565.patch through.
> Make TimestampUpdateProcessorFactory as Runtime URP; take params(s) with
> request
>
>
> Key: SOLR-10866
> URL: https://issues.apache.org/jira/browse/SOLR-10866
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10866.patch, SOLR-10866.patch, SOLR-10866.patch,
> SOLR-10866.patch, SOLR-10866.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> TimestampUpdateProcessorFactory will be able to execute by sample curl like
> below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=Timestamp=timestamp_tdt=true
> --data-binary { "id" : "1" , "title_s" : "titleA" }
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10734) Multithreaded test/support for AtomicURP broken
[
https://issues.apache.org/jira/browse/SOLR-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16022500#comment-16022500
]
Amrit Sarkar commented on SOLR-10734:
-
Ishan, thank you for sharing the concern,
I replaced the t.run() with t.start(), and the test failed for me once for this
seed:
ant test -Dtestcase=AtomicUpdateProcessorFactoryTest
-Dtests.method=testMultipleThreads -Dtests.seed=C2A82A0AA119D39F
-Dtests.slow=true -Dtests.locale=ar-KW -Dtests.timezone=Etc/GMT+11
-Dtests.asserts=true -Dtests.file.encoding=UTF-8
It failed multiple retires and eventually fails for version conflict but I am
not able to replicate the same ever since for any seed and the test is getting
passed successfully.
Can you share the seed or the log snippet from the failure.
> Multithreaded test/support for AtomicURP broken
> ---
>
> Key: SOLR-10734
> URL: https://issues.apache.org/jira/browse/SOLR-10734
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Ishan Chattopadhyaya
>
> The multithreaded test doesn't actually start the threads, but only invokes
> the run directly. The join afterwards doesn't do anything, hence.
> {code}
> diff --git
> a/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
>
> b/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
> index f3f833d..10b7770 100644
> ---
> a/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
> +++
> b/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
> @@ -238,7 +238,7 @@ public class AtomicUpdateProcessorFactoryTest extends
> SolrTestCaseJ4 {
>}
> }
>};
> - t.run();
> + t.run(); // red flag, shouldn't this be t.start?
>threads.add(t);
>finalCount += index; //int_i
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-10734) Multithreaded test/support for AtomicURP broken
[
https://issues.apache.org/jira/browse/SOLR-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16022500#comment-16022500
]
Amrit Sarkar edited comment on SOLR-10734 at 5/24/17 8:17 AM:
--
Ishan, thank you for sharing the concern,
I replaced the t.run() with t.start(), and the test failed for me once for this
seed:
ant test -Dtestcase=AtomicUpdateProcessorFactoryTest
-Dtests.method=testMultipleThreads -Dtests.seed=C2A82A0AA119D39F
-Dtests.slow=true -Dtests.locale=ar-KW -Dtests.timezone=Etc/GMT+11
-Dtests.asserts=true -Dtests.file.encoding=UTF-8
It failed multiple retires and eventually fails for version conflict but I am
not able to replicate the same ever since for any seed and the test is getting
passed successfully.
I am sure the test is not full proofed.
Can you share the seed or the log snippet from the failure.
was (Author: sarkaramr...@gmail.com):
Ishan, thank you for sharing the concern,
I replaced the t.run() with t.start(), and the test failed for me once for this
seed:
ant test -Dtestcase=AtomicUpdateProcessorFactoryTest
-Dtests.method=testMultipleThreads -Dtests.seed=C2A82A0AA119D39F
-Dtests.slow=true -Dtests.locale=ar-KW -Dtests.timezone=Etc/GMT+11
-Dtests.asserts=true -Dtests.file.encoding=UTF-8
It failed multiple retires and eventually fails for version conflict but I am
not able to replicate the same ever since for any seed and the test is getting
passed successfully.
Can you share the seed or the log snippet from the failure.
> Multithreaded test/support for AtomicURP broken
> ---
>
> Key: SOLR-10734
> URL: https://issues.apache.org/jira/browse/SOLR-10734
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Ishan Chattopadhyaya
>
> The multithreaded test doesn't actually start the threads, but only invokes
> the run directly. The join afterwards doesn't do anything, hence.
> {code}
> diff --git
> a/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
>
> b/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
> index f3f833d..10b7770 100644
> ---
> a/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
> +++
> b/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
> @@ -238,7 +238,7 @@ public class AtomicUpdateProcessorFactoryTest extends
> SolrTestCaseJ4 {
>}
> }
>};
> - t.run();
> + t.run(); // red flag, shouldn't this be t.start?
>threads.add(t);
>finalCount += index; //int_i
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-10734) Multithreaded test/support for AtomicURP broken
[
https://issues.apache.org/jira/browse/SOLR-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16022500#comment-16022500
]
Amrit Sarkar edited comment on SOLR-10734 at 5/24/17 8:19 AM:
--
Ishan, thank you for sharing the concern,
I replaced the t.run() with t.start(), and the test failed for me once for this
seed:
ant test -Dtestcase=AtomicUpdateProcessorFactoryTest
-Dtests.method=testMultipleThreads -Dtests.seed=C2A82A0AA119D39F
-Dtests.slow=true -Dtests.locale=ar-KW -Dtests.timezone=Etc/GMT+11
-Dtests.asserts=true -Dtests.file.encoding=UTF-8
It failed multiple retires and eventually fails for version conflict but I am
not able to replicate the same ever since for any seed and the test is getting
passed successfully.
I am sure the test is not full proofed.
was (Author: sarkaramr...@gmail.com):
Ishan, thank you for sharing the concern,
I replaced the t.run() with t.start(), and the test failed for me once for this
seed:
ant test -Dtestcase=AtomicUpdateProcessorFactoryTest
-Dtests.method=testMultipleThreads -Dtests.seed=C2A82A0AA119D39F
-Dtests.slow=true -Dtests.locale=ar-KW -Dtests.timezone=Etc/GMT+11
-Dtests.asserts=true -Dtests.file.encoding=UTF-8
It failed multiple retires and eventually fails for version conflict but I am
not able to replicate the same ever since for any seed and the test is getting
passed successfully.
I am sure the test is not full proofed.
Can you share the seed or the log snippet from the failure.
> Multithreaded test/support for AtomicURP broken
> ---
>
> Key: SOLR-10734
> URL: https://issues.apache.org/jira/browse/SOLR-10734
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Ishan Chattopadhyaya
> Attachments: log-snippet
>
>
> The multithreaded test doesn't actually start the threads, but only invokes
> the run directly. The join afterwards doesn't do anything, hence.
> {code}
> diff --git
> a/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
>
> b/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
> index f3f833d..10b7770 100644
> ---
> a/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
> +++
> b/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
> @@ -238,7 +238,7 @@ public class AtomicUpdateProcessorFactoryTest extends
> SolrTestCaseJ4 {
>}
> }
>};
> - t.run();
> + t.run(); // red flag, shouldn't this be t.start?
>threads.add(t);
>finalCount += index; //int_i
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10734) Multithreaded test/support for AtomicURP broken
[
https://issues.apache.org/jira/browse/SOLR-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10734:
Attachment: log-snippet
> Multithreaded test/support for AtomicURP broken
> ---
>
> Key: SOLR-10734
> URL: https://issues.apache.org/jira/browse/SOLR-10734
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Ishan Chattopadhyaya
> Attachments: log-snippet
>
>
> The multithreaded test doesn't actually start the threads, but only invokes
> the run directly. The join afterwards doesn't do anything, hence.
> {code}
> diff --git
> a/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
>
> b/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
> index f3f833d..10b7770 100644
> ---
> a/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
> +++
> b/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
> @@ -238,7 +238,7 @@ public class AtomicUpdateProcessorFactoryTest extends
> SolrTestCaseJ4 {
>}
> }
>};
> - t.run();
> + t.run(); // red flag, shouldn't this be t.start?
>threads.add(t);
>finalCount += index; //int_i
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-7705) Allow CharTokenizer-derived tokenizers and KeywordTokenizer to configure the max token length
[ https://issues.apache.org/jira/browse/LUCENE-7705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amrit Sarkar updated LUCENE-7705: - Attachment: LUCENE-7705.patch Following discussion on LUCENE-7857 and Erick's pointers, introduced _autoGeneratePhraseQueries="false"_ on the all the FieldType definitions for this test case. all the test scenarios are successfully executed on both *master* and *branch_6x*. Patch attached. > Allow CharTokenizer-derived tokenizers and KeywordTokenizer to configure the > max token length > - > > Key: LUCENE-7705 > URL: https://issues.apache.org/jira/browse/LUCENE-7705 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Amrit Sarkar >Assignee: Erick Erickson >Priority: Minor > Fix For: master (7.0), 6.7 > > Attachments: LUCENE-7705, LUCENE-7705.patch, LUCENE-7705.patch, > LUCENE-7705.patch, LUCENE-7705.patch, LUCENE-7705.patch, LUCENE-7705.patch, > LUCENE-7705.patch, LUCENE-7705.patch, LUCENE-7705.patch, LUCENE-7705.patch > > > SOLR-10186 > [~erickerickson]: Is there a good reason that we hard-code a 256 character > limit for the CharTokenizer? In order to change this limit it requires that > people copy/paste the incrementToken into some new class since incrementToken > is final. > KeywordTokenizer can easily change the default (which is also 256 bytes), but > to do so requires code rather than being able to configure it in the schema. > For KeywordTokenizer, this is Solr-only. For the CharTokenizer classes > (WhitespaceTokenizer, UnicodeWhitespaceTokenizer and LetterTokenizer) > (Factories) it would take adding a c'tor to the base class in Lucene and > using it in the factory. > Any objections? -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10734) Multithreaded test/support for AtomicURP broken
[
https://issues.apache.org/jira/browse/SOLR-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16031011#comment-16031011
]
Amrit Sarkar commented on SOLR-10734:
-
The testmethod::testMultipleThreads() is faulty, the multithreaded support is
intact until a certain limit.
If I bring down the number of threads spawned b/w range 10-20, it works
swiftly. As I debugged through the threads, there are multiple threads
executing at the same time and some threads (10-20%) are not able to catch up
ever with the version_id for maximum number of attempts.
expected_is: X, actual_is: Y and when it fetches the current version from
_vinfo_, it becomes Z (updated by some other thread), always behind.
For MAX_ATTEMPTS=100, out of total 1000 threads, if I spawn, around 200-350
threads are failing to execute resulting in failure of poorly written test
scenarios. But some threads do get indexed after multiple retries and populates
the document successfully.
e.g.
{code}
[jira] [Comment Edited] (SOLR-10734) Multithreaded test/support for AtomicURP broken
[
https://issues.apache.org/jira/browse/SOLR-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16031011#comment-16031011
]
Amrit Sarkar edited comment on SOLR-10734 at 5/31/17 11:32 AM:
---
The testmethod::testMultipleThreads() is faulty, the multithreaded support is
intact until a certain limit.
If I bring down the number of threads spawned b/w range 10-20, it works
swiftly. As I debugged through the threads, there are multiple threads
executing at the same time and some threads (10-20%) are not able to catch up
ever with the version_id for maximum number of attempts.
expected_is: X, actual_is: Y and when it fetches the current version from
_vinfo_, it becomes Z (updated by some other thread), always behind. PF
screenshot.
For MAX_ATTEMPTS=100, out of total 1000 threads, if I spawn, around 200-350
threads are failing to execute resulting in failure of poorly written test
scenarios. But some threads do get indexed after multiple retries and populates
the document successfully.
e.g.
{code}
[jira] [Updated] (SOLR-10734) Multithreaded test/support for AtomicURP broken
[
https://issues.apache.org/jira/browse/SOLR-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10734:
Attachment: Screen Shot 2017-05-31 at 4.50.23 PM.png
> Multithreaded test/support for AtomicURP broken
> ---
>
> Key: SOLR-10734
> URL: https://issues.apache.org/jira/browse/SOLR-10734
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Ishan Chattopadhyaya
> Attachments: log-snippet, Screen Shot 2017-05-31 at 4.50.23 PM.png
>
>
> The multithreaded test doesn't actually start the threads, but only invokes
> the run directly. The join afterwards doesn't do anything, hence.
> {code}
> diff --git
> a/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
>
> b/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
> index f3f833d..10b7770 100644
> ---
> a/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
> +++
> b/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
> @@ -238,7 +238,7 @@ public class AtomicUpdateProcessorFactoryTest extends
> SolrTestCaseJ4 {
>}
> }
>};
> - t.run();
> + t.run(); // red flag, shouldn't this be t.start?
>threads.add(t);
>finalCount += index; //int_i
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-10734) Multithreaded test/support for AtomicURP broken
[
https://issues.apache.org/jira/browse/SOLR-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16031011#comment-16031011
]
Amrit Sarkar edited comment on SOLR-10734 at 5/31/17 11:42 AM:
---
The testmethod::testMultipleThreads() is faulty, the multithreaded support is
intact until a certain limit.
If I bring down the number of threads spawned b/w range 10-20, it works
swiftly. As I debugged through the threads, there are multiple threads
executing at the same time and some threads (10-20%) are not able to catch up
ever with the version_id for maximum number of attempts.
expected_is: X, actual_is: Y and when it fetches the current version from
_vinfo_, it becomes Z (updated by some other thread), always behind. PF
screenshot.
For MAX_ATTEMPTS=100, out of total 1000 threads, if I spawn, around 200-350
threads are failing to execute resulting in failure of poorly written test
scenarios. But some threads do get indexed after multiple retries and populates
the document successfully.
e.g.
{code}
[jira] [Created] (SOLR-10859) Make URLClassifyProcessor as Runtime URP; take params(s) in Request not in Solrconfig
Amrit Sarkar created SOLR-10859:
---
Summary: Make URLClassifyProcessor as Runtime URP; take params(s)
in Request not in Solrconfig
Key: SOLR-10859
URL: https://issues.apache.org/jira/browse/SOLR-10859
Project: Solr
Issue Type: Improvement
Security Level: Public (Default Security Level. Issues are Public)
Components: update
Reporter: Amrit Sarkar
Priority: Minor
As discussed with Noble Paul, we are trying to get rid of processor definitions
in SolrConfig for all URPs and take parameters in the request itself.
URLClassifyProcessorFactory will be able to execute by sample curl like below:
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=URLClassify=fieldA=fieldB=fieldC=fieldD=fieldE=fieldF=fieldG=true
--data-binary {"id" : "1", "fieldA" : "valueA"}
{code}
No configuration required for URLClassifyProcessorFactory in solrconfig.xml.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10858) Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) in Request not in Solrconfig
[
https://issues.apache.org/jira/browse/SOLR-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10858:
Description:
As discussed with [~noble.paul], we are trying to get rid of processor
definitions in SolrConfig for all URPs and take parameters in the request
itself.
UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
--data-binary {"title": "titleA"}
{code}
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=UUID=true
--data-binary {"id":"1","title": "titleA"}
{code}
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
--data-binary {"id":"1","title": "titleA"}
{code}
No configuration required for UUIDUpdateProcessorFactory in solrconfig.xml.
was:
As discussed with [~noble.paul], we are trying to get rid of processor
definitions in SolrConfig for all URPs and take parameters in the request
itself.
UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
--data-binary {"title": "titleA"}
{code}
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=UUID=true
--data-binary {"id":"1","title": "titleA"}
{code}
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
--data-binary {"id":"1","title": "titleA"}
{code}
> Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) in Request not
> in Solrconfig
> ---
>
> Key: SOLR-10858
> URL: https://issues.apache.org/jira/browse/SOLR-10858
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10858.patch
>
>
> As discussed with [~noble.paul], we are trying to get rid of processor
> definitions in SolrConfig for all URPs and take parameters in the request
> itself.
> UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> No configuration required for UUIDUpdateProcessorFactory in solrconfig.xml.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10858) Convert UUIDUpdateProcessorFactory as Runtime URP; take params(s) in Request not in Solrconfig
[
https://issues.apache.org/jira/browse/SOLR-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10858:
Description:
As discussed with [~noble.paul], we are trying to get rid of processor
definitions in SolrConfig for all URPs and take parameters in the request
itself.
UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
--data-binary {"title": "titleA"}
{code}
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=UUID=true
--data-binary {"id":"1","title": "titleA"}
{code}
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
--data-binary {"id":"1","title": "titleA"}
{code}
was:
As discussed with [~noble.paul], we are trying to get rid of processor
definitions in SolrConfig for all URPs and take parameters in the request
itself.
UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
{code}
> Convert UUIDUpdateProcessorFactory as Runtime URP; take params(s) in Request
> not in Solrconfig
> --
>
> Key: SOLR-10858
> URL: https://issues.apache.org/jira/browse/SOLR-10858
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>
> As discussed with [~noble.paul], we are trying to get rid of processor
> definitions in SolrConfig for all URPs and take parameters in the request
> itself.
> UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10858) Convert UUIDUpdateProcessorFactory as Runtime URP; take params(s) in Request not in Solrconfig
[
https://issues.apache.org/jira/browse/SOLR-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10858:
Description:
As discussed with [~noble.paul], we are trying to get rid of processor
definitions in SolrConfig for all URPs and take parameters in the request
itself.
UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
{code}
was:
As discussed with [~noble.paul], we are trying to get rid of processor
definitions in SolrConfig for all URPs and take parameters in the request
itself.
UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs
?processor=UUID;ampersand;UUID.fieldName=id;ampersand;commit=true
{code}
> Convert UUIDUpdateProcessorFactory as Runtime URP; take params(s) in Request
> not in Solrconfig
> --
>
> Key: SOLR-10858
> URL: https://issues.apache.org/jira/browse/SOLR-10858
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>
> As discussed with [~noble.paul], we are trying to get rid of processor
> definitions in SolrConfig for all URPs and take parameters in the request
> itself.
> UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-10858) Convert UUIDUpdateProcessorFactory as Runtime URP; take params(s) in Request not in Solrconfig
Amrit Sarkar created SOLR-10858:
---
Summary: Convert UUIDUpdateProcessorFactory as Runtime URP; take
params(s) in Request not in Solrconfig
Key: SOLR-10858
URL: https://issues.apache.org/jira/browse/SOLR-10858
Project: Solr
Issue Type: Improvement
Security Level: Public (Default Security Level. Issues are Public)
Components: update
Reporter: Amrit Sarkar
As discussed with [~noble.paul], we are trying to get rid of processor
definitions in SolrConfig for all URPs and take parameters in the request
itself.
UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs
?processor=UUID;ampersand;UUID.fieldName=id;ampersand;commit=true
{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10858) Convert UUIDUpdateProcessorFactory as Runtime URP; take params(s) in Request not in Solrconfig
[
https://issues.apache.org/jira/browse/SOLR-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10858:
Attachment: SOLR-10858.patch
Patch attached.
Revamped _UUIDUpdateProcessorFallbackTest.java_ according to the changes.
> Convert UUIDUpdateProcessorFactory as Runtime URP; take params(s) in Request
> not in Solrconfig
> --
>
> Key: SOLR-10858
> URL: https://issues.apache.org/jira/browse/SOLR-10858
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
> Attachments: SOLR-10858.patch
>
>
> As discussed with [~noble.paul], we are trying to get rid of processor
> definitions in SolrConfig for all URPs and take parameters in the request
> itself.
> UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10858) Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) in Request not in Solrconfig
[
https://issues.apache.org/jira/browse/SOLR-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10858:
Summary: Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) in
Request not in Solrconfig (was: Convert UUIDUpdateProcessorFactory as Runtime
URP; take params(s) in Request not in Solrconfig)
> Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) in Request not
> in Solrconfig
> ---
>
> Key: SOLR-10858
> URL: https://issues.apache.org/jira/browse/SOLR-10858
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
> Attachments: SOLR-10858.patch
>
>
> As discussed with [~noble.paul], we are trying to get rid of processor
> definitions in SolrConfig for all URPs and take parameters in the request
> itself.
> UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10858) Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) in Request not in Solrconfig
[
https://issues.apache.org/jira/browse/SOLR-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10858:
Priority: Minor (was: Major)
> Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) in Request not
> in Solrconfig
> ---
>
> Key: SOLR-10858
> URL: https://issues.apache.org/jira/browse/SOLR-10858
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10858.patch
>
>
> As discussed with [~noble.paul], we are trying to get rid of processor
> definitions in SolrConfig for all URPs and take parameters in the request
> itself.
> UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10858) Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10858:
Attachment: SOLR-10858.patch
Improved tests and URPChain plugin fixed. Patch uploaded.
> Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
> ---
>
> Key: SOLR-10858
> URL: https://issues.apache.org/jira/browse/SOLR-10858
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch,
> SOLR-10858.patch, SOLR-10858.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10858) Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10858:
Attachment: SOLR-10858.patch
> Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
> ---
>
> Key: SOLR-10858
> URL: https://issues.apache.org/jira/browse/SOLR-10858
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch,
> SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10858) Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10858:
Attachment: SOLR-10858.patch
Patch uploaded with prefix for input parameter as X, where X is from
{{XUUIDUpdateProcessorFactory}}, e,g. {{UUID}} for UUIDUpdateProcessorFactory.
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
--data-binary {"title": "titleA"}
{code}
Will update the description if no objections.
> Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
> ---
>
> Key: SOLR-10858
> URL: https://issues.apache.org/jira/browse/SOLR-10858
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch,
> SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-10858) Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16047876#comment-16047876
]
Amrit Sarkar edited comment on SOLR-10858 at 6/13/17 1:17 PM:
--
Patch uploaded with prefix for input parameter as X, where X is from
{{XUUIDUpdateProcessorFactory}}, eg. {{UUID}} for UUIDUpdateProcessorFactory.
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
--data-binary {"title": "titleA"}
{code}
Will update the description if no objections.
was (Author: sarkaramr...@gmail.com):
Patch uploaded with prefix for input parameter as X, where X is from
{{XUUIDUpdateProcessorFactory}}, e,g. {{UUID}} for UUIDUpdateProcessorFactory.
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
--data-binary {"title": "titleA"}
{code}
Will update the description if no objections.
> Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
> ---
>
> Key: SOLR-10858
> URL: https://issues.apache.org/jira/browse/SOLR-10858
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch,
> SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10858) Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10858:
Attachment: (was: SOLR-10858.patch)
> Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
> ---
>
> Key: SOLR-10858
> URL: https://issues.apache.org/jira/browse/SOLR-10858
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch,
> SOLR-10858.patch, SOLR-10858.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10866) Make TimestampUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10866:
Issue Type: Sub-task (was: Improvement)
Parent: SOLR-9565
> Make TimestampUpdateProcessorFactory as Runtime URP; take params(s) with
> request
>
>
> Key: SOLR-10866
> URL: https://issues.apache.org/jira/browse/SOLR-10866
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10866.patch, SOLR-10866.patch, SOLR-10866.patch,
> SOLR-10866.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> TimestampUpdateProcessorFactory will be able to execute by sample curl like
> below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=Timestamp=timestamp_tdt=true
> --data-binary { "id" : "1" , "title_s" : "titleA" }
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10867) Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10867:
Issue Type: Sub-task (was: Improvement)
Parent: SOLR-9565
> Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with
> request
> -
>
> Key: SOLR-10867
> URL: https://issues.apache.org/jira/browse/SOLR-10867
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: error_test, SOLR-10867.patch, SOLR-10867.patch,
> SOLR-10867.patch, SOLR-10867.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> ClassificationUpdateProcessorFactory will be able to execute by sample curl
> like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=Classification=url_s=training=true
> --data-binary { "id" : "1" , "url_s" : "http://www.example.com/subroot; }
> {code}
> All the param(s) for this URP available can be passed as request handler
> param(s).
> Configuration for ClassificationUpdateProcessorFactory in solrconfig.xml is
> optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10869) Make StatelessScriptUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10869:
Issue Type: Sub-task (was: Improvement)
Parent: SOLR-9565
> Make StatelessScriptUpdateProcessorFactory as Runtime URP; take params(s)
> with request
> --
>
> Key: SOLR-10869
> URL: https://issues.apache.org/jira/browse/SOLR-10869
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> StatelessScriptUpdateProcessorFactory will be able to execute by sample curl
> like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=StatelessScript=1.js=2.js=3.js=keyA:valueA=keyB:valueB=keyC:valueC=
> rhino=true --data-binary { "id" : "1" , "title_s" : "title_random" }
> {code}
> All the param(s) for this URP available can be passed as request handler
> param(s). The scripts will be executed in the order the parameters are
> received.
> Configuration for StatelessScriptUpdateProcessorFactory in solrconfig.xml is
> optional. Backcompat is intact.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10859) Make URLClassifyProcessor as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048302#comment-16048302
]
Amrit Sarkar commented on SOLR-10859:
-
In order to make this work, we have to rename the processor factory from:
{{URLClassifyProcessorFactory}} to {{URLClassifyUpdateProcessorFactory}}.
> Make URLClassifyProcessor as Runtime URP; take params(s) with request
> -
>
> Key: SOLR-10859
> URL: https://issues.apache.org/jira/browse/SOLR-10859
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10859.patch, SOLR-10859.patch, SOLR-10859.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> URLClassifyProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=URLClassify=fieldA=fieldB=fieldC=fieldD=fieldE=fieldF=fieldG=true
> --data-binary {"id" : "1", "fieldA" : "valueA"}
> {code}
> Configuration for URLClassifyProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10859) Make URLClassifyProcessor as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10859:
Attachment: SOLR-10859.patch
> Make URLClassifyProcessor as Runtime URP; take params(s) with request
> -
>
> Key: SOLR-10859
> URL: https://issues.apache.org/jira/browse/SOLR-10859
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10859.patch, SOLR-10859.patch, SOLR-10859.patch,
> SOLR-10859.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> URLClassifyProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=URLClassify=fieldA=fieldB=fieldC=fieldD=fieldE=fieldF=fieldG=true
> --data-binary {"id" : "1", "fieldA" : "valueA"}
> {code}
> Configuration for URLClassifyProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10859) Make URLClassifyProcessor as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10859:
Issue Type: Sub-task (was: Improvement)
Parent: SOLR-9565
> Make URLClassifyProcessor as Runtime URP; take params(s) with request
> -
>
> Key: SOLR-10859
> URL: https://issues.apache.org/jira/browse/SOLR-10859
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10859.patch, SOLR-10859.patch, SOLR-10859.patch,
> SOLR-10859.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> URLClassifyProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=URLClassify=fieldA=fieldB=fieldC=fieldD=fieldE=fieldF=fieldG=true
> --data-binary {"id" : "1", "fieldA" : "valueA"}
> {code}
> Configuration for URLClassifyProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10866) Make TimestampUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10866:
Attachment: SOLR-10866.patch
> Make TimestampUpdateProcessorFactory as Runtime URP; take params(s) with
> request
>
>
> Key: SOLR-10866
> URL: https://issues.apache.org/jira/browse/SOLR-10866
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10866.patch, SOLR-10866.patch, SOLR-10866.patch,
> SOLR-10866.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> TimestampUpdateProcessorFactory will be able to execute by sample curl like
> below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=Timestamp=timestamp_tdt=true
> --data-binary { "id" : "1" , "title_s" : "titleA" }
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10858) Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10858:
Issue Type: Sub-task (was: Improvement)
Parent: SOLR-9565
> Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
> ---
>
> Key: SOLR-10858
> URL: https://issues.apache.org/jira/browse/SOLR-10858
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch,
> SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-9565) Make every UpdateRequestProcessor available implicitly
[
https://issues.apache.org/jira/browse/SOLR-9565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16050184#comment-16050184
]
Amrit Sarkar edited comment on SOLR-9565 at 6/15/17 8:55 AM:
-
{quote}
this patch introduces a set of predefined set of URPs only (template and
atomic). User defined URPs wil have to be explicitly registered his config API
{quote}
Once this patch is committed, we will try to add the the remaining URPs to the
{{simpler well defined names list}} one by one starting with UUID-URPF:
SOLR-10858.
was (Author: sarkaramr...@gmail.com):
{quote}
this patch introduces a set of predefined set of URPs only (template and
atomic). User defined URPs wil have to be explicitly registered his config API
{quote}
Once this patch is committed, we will try to add the the remaining URPs to the
{{simpler well defined names list}} one by one starting with UUID-URPF.
> Make every UpdateRequestProcessor available implicitly
> --
>
> Key: SOLR-9565
> URL: https://issues.apache.org/jira/browse/SOLR-9565
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Noble Paul
> Attachments: SOLR-9565.patch
>
>
> Now that we can 'construct' the URP chains through request parameters, we
> should make all the URPs available automatically. The next challenge is to
> make them read the configuration from request parameters as well
> to access {{HTMLStripFieldUpdateProcessorFactory}} the parameter could be
> {{processor=HTMLStripField}} (The UpdateProcessorFactory part is
> automatically appended )
> The next step is to make the URPs accept request parameters instead of just
> configuration parameters e.g:
> {{processor=HTMLStripField=}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9565) Make every UpdateRequestProcessor available implicitly
[
https://issues.apache.org/jira/browse/SOLR-9565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16050219#comment-16050219
]
Amrit Sarkar commented on SOLR-9565:
just skimming;
Shouldn't support for {{processor=X}}, X in XURPFactory, continue? AtomicURP
can be invoked both via {{processor=atomic}}, simplified and
{{processor=Atomic}}, already supported. I saw complaints on StatelessScriptURP
where {{stateless}} is needless, {{processor=script}} can be handy there, along
with existing URPs shipped.
This will mean 3rd party URPs need not to do anything special out of order and
can use the prefix to invoke their respective processor.
> Make every UpdateRequestProcessor available implicitly
> --
>
> Key: SOLR-9565
> URL: https://issues.apache.org/jira/browse/SOLR-9565
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Noble Paul
> Attachments: SOLR-9565.patch
>
>
> Now that we can 'construct' the URP chains through request parameters, we
> should make all the URPs available automatically. The next challenge is to
> make them read the configuration from request parameters as well
> to access {{HTMLStripFieldUpdateProcessorFactory}} the parameter could be
> {{processor=HTMLStripField}} (The UpdateProcessorFactory part is
> automatically appended )
> The next step is to make the URPs accept request parameters instead of just
> configuration parameters e.g:
> {{processor=HTMLStripField=}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9565) Make every UpdateRequestProcessor available implicitly
[
https://issues.apache.org/jira/browse/SOLR-9565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16050184#comment-16050184
]
Amrit Sarkar commented on SOLR-9565:
{quote}
this patch introduces a set of predefined set of URPs only (template and
atomic). User defined URPs wil have to be explicitly registered his config API
{quote}
Once this patch is committed, we will try to add the the remaining URPs to the
{{simpler well defined names list}} one by one starting with UUID-URPF.
> Make every UpdateRequestProcessor available implicitly
> --
>
> Key: SOLR-9565
> URL: https://issues.apache.org/jira/browse/SOLR-9565
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Noble Paul
> Attachments: SOLR-9565.patch
>
>
> Now that we can 'construct' the URP chains through request parameters, we
> should make all the URPs available automatically. The next challenge is to
> make them read the configuration from request parameters as well
> to access {{HTMLStripFieldUpdateProcessorFactory}} the parameter could be
> {{processor=HTMLStripField}} (The UpdateProcessorFactory part is
> automatically appended )
> The next step is to make the URPs accept request parameters instead of just
> configuration parameters e.g:
> {{processor=HTMLStripField=}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-10866) Make TimestampUpdateProcessorFactory as Runtime URP; take params(s) with request
Amrit Sarkar created SOLR-10866:
---
Summary: Make TimestampUpdateProcessorFactory as Runtime URP; take
params(s) with request
Key: SOLR-10866
URL: https://issues.apache.org/jira/browse/SOLR-10866
Project: Solr
Issue Type: Improvement
Security Level: Public (Default Security Level. Issues are Public)
Components: update
Reporter: Amrit Sarkar
Priority: Minor
We are trying to get rid of processor definitions in SolrConfig for all URPs
and take parameters in the request itself.
TimestampUpdateProcessorFactory will be able to execute by sample curl like
below:
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=Timestamp=timestamp_tdt=true
--data-binary { "id" : "1" , "title_s" : "titleA" }
{code}
Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10866) Make TimestampUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10866:
Attachment: SOLR-10866.patch
Comments Improved in the patch.
> Make TimestampUpdateProcessorFactory as Runtime URP; take params(s) with
> request
>
>
> Key: SOLR-10866
> URL: https://issues.apache.org/jira/browse/SOLR-10866
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10866.patch, SOLR-10866.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> TimestampUpdateProcessorFactory will be able to execute by sample curl like
> below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=Timestamp=timestamp_tdt=true
> --data-binary { "id" : "1" , "title_s" : "titleA" }
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10859) Make URLClassifyProcessor as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10859:
Attachment: SOLR-10859.patch
Initial patch uploaded. There is no *URLClassifyProcessorFactoryTest* class in
the project right now, need to create one and test the config and request
parameters accordingly.
> Make URLClassifyProcessor as Runtime URP; take params(s) with request
> -
>
> Key: SOLR-10859
> URL: https://issues.apache.org/jira/browse/SOLR-10859
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10859.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> URLClassifyProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=URLClassify=fieldA=fieldB=fieldC=fieldD=fieldE=fieldF=fieldG=true
> --data-binary {"id" : "1", "fieldA" : "valueA"}
> {code}
> Configuration for URLClassifyProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-10866) Make TimestampUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045523#comment-16045523
]
Amrit Sarkar edited comment on SOLR-10866 at 6/10/17 1:15 PM:
--
Patch attached. Test class for TimestampUpdateProcessorFactory is not there in
the project. Seeking advice whether to create one or we can by-pass this.
was (Author: sarkaramr...@gmail.com):
Patch attached. There is no test class for TimestampUpdateProcessorFactory is
present. Seeking advice whether to create one or we can by-pass this.
> Make TimestampUpdateProcessorFactory as Runtime URP; take params(s) with
> request
>
>
> Key: SOLR-10866
> URL: https://issues.apache.org/jira/browse/SOLR-10866
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10866.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> TimestampUpdateProcessorFactory will be able to execute by sample curl like
> below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=Timestamp=timestamp_tdt=true
> --data-binary { "id" : "1" , "title_s" : "titleA" }
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10866) Make TimestampUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10866:
Attachment: SOLR-10866.patch
Patch attached. There is no test class for TimestampUpdateProcessorFactory is
present. Seeking advice whether to create one or we can by-pass this.
> Make TimestampUpdateProcessorFactory as Runtime URP; take params(s) with
> request
>
>
> Key: SOLR-10866
> URL: https://issues.apache.org/jira/browse/SOLR-10866
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10866.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> TimestampUpdateProcessorFactory will be able to execute by sample curl like
> below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=Timestamp=timestamp_tdt=true
> --data-binary { "id" : "1" , "title_s" : "titleA" }
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10858) Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10858:
Attachment: SOLR-10858.patch
{quote}
I wonder if there should be a larger discussion (not expressly for this
particular URP) about this overall. For example, what should the conventions be
on parameter naming? . ? Should abbreviated URP
names be initial lowercase (thus "uuid" not "UUID"). I think so.
{quote}
Seems neat. If we agree on keeping lowercase , I will make
changes to AtomicURP and TemplateURP too: SOLR-9530.
{quote}
And is there a way to have an URP registered multiple times with different
configurations each? Perhaps up to some factories.
{quote}
With request handler parameters being supported in URPs, factory's config
param(s) can be overridden at runtime resulting creation of multiple instances
of URP with a new configuration each time, right?
New patch uploaded with above suggestions and backward compatibility enabled.
> Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
> ---
>
> Key: SOLR-10858
> URL: https://issues.apache.org/jira/browse/SOLR-10858
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10858.patch, SOLR-10858.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10858) Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10858:
Description:
We are trying to get rid of processor definitions in SolrConfig for all URPs
and take parameters in the request itself.
UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
--data-binary {"title": "titleA"}
{code}
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=UUID=true
--data-binary {"id":"1","title": "titleA"}
{code}
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
--data-binary {"id":"1","title": "titleA"}
{code}
Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
was:
We are trying to get rid of processor definitions in SolrConfig for all URPs
and take parameters in the request itself.
UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
--data-binary {"title": "titleA"}
{code}
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=UUID=true
--data-binary {"id":"1","title": "titleA"}
{code}
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
--data-binary {"id":"1","title": "titleA"}
{code}
Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
> Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
> ---
>
> Key: SOLR-10858
> URL: https://issues.apache.org/jira/browse/SOLR-10858
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10858.patch, SOLR-10858.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-10865) parameter processor=Template doesn't invoke TemplateUpdateProcessorFactory.
Amrit Sarkar created SOLR-10865:
---
Summary: parameter processor=Template doesn't invoke
TemplateUpdateProcessorFactory.
Key: SOLR-10865
URL: https://issues.apache.org/jira/browse/SOLR-10865
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Components: update
Affects Versions: master (7.0)
Reporter: Amrit Sarkar
Ref docs::
{quote}
The TemplateUpdateProcessorFactory can be used to add new fields to documents
based on a template pattern.
This can be used directly in a request without any configuration. To enable
this processor, use the parameter processor=Template.
{quote}
Sample curl::
{code}
curl -X POST -H 'Content-Type:application/json'
'http://localhost:8983/solr/gettingstarted/update/json/docs?processor=Template=fullName_s:AmritSarkar=true'
--data-binary '{"id": 1,"title": "titleA"}'
{code}
I am receiving exception::
{code}
4003org.apache.solr.common.SolrExceptionorg.apache.solr.common.SolrExceptionNo such processor Template400
{code}
{code}
ERROR - 2017-06-10 07:39:51.598; [c:gettingstarted s:shard2 r:core_node1
x:gettingstarted_shard2_replica_n1] org.apache.solr.common.SolrException;
org.apache.solr.common.SolrException: No such processor Template
at
org.apache.solr.update.processor.UpdateRequestProcessorChain.getReqProcessors(UpdateRequestProcessorChain.java:286)
at
org.apache.solr.update.processor.UpdateRequestProcessorChain.constructChain(UpdateRequestProcessorChain.java:235)
{code}
I looked into how the testclass has been created::
TemplateUpdateProcessorTest::
{code}
public void testSimple() throws Exception {
AddUpdateCommand cmd = new AddUpdateCommand(new LocalSolrQueryRequest(null,
new ModifiableSolrParams()
.add("processor", "Template")
.add("Template.field", "id:${firstName}_${lastName}")
.add("Template.field", "another:${lastName}_${firstName}")
.add("Template.field", "missing:${lastName}_${unKnown}")
));
cmd.solrDoc = new SolrInputDocument();
cmd.solrDoc.addField("firstName", "Tom");
cmd.solrDoc.addField("lastName", "Cruise");
new TemplateUpdateProcessorFactory().getInstance(cmd.getReq(), new
SolrQueryResponse(), null).processAdd(cmd);
assertEquals("Tom_Cruise", cmd.solrDoc.getFieldValue("id"));
assertEquals("Cruise_Tom", cmd.solrDoc.getFieldValue("another"));
assertEquals("Cruise_", cmd.solrDoc.getFieldValue("missing"));
}
{code}
*There is no test to check whether _processor=Template_ works or not*,
TemplateUpdateProcessorFactory() object is EXPLICITLY created to getInstance
and do processing on the updates.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10859) Make URLClassifyProcessor as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10859:
Description:
We are trying to get rid of processor definitions in SolrConfig for all URPs
and take parameters in the request itself.
URLClassifyProcessorFactory will be able to execute by sample curl like below:
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=URLClassify=fieldA=fieldB=fieldC=fieldD=fieldE=fieldF=fieldG=true
--data-binary {"id" : "1", "fieldA" : "valueA"}
{code}
Configuration for URLClassifyProcessorFactory in solrconfig.xml is optional.
was:
We are trying to get rid of processor definitions in SolrConfig for all URPs
and take parameters in the request itself.
URLClassifyProcessorFactory will be able to execute by sample curl like below:
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=URLClassify=fieldA=fieldB=fieldC=fieldD=fieldE=fieldF=fieldG=true
--data-binary {"id" : "1", "fieldA" : "valueA"}
{code}
Configuration for URLClassifyProcessorFactory in solrconfig.xml is optional.
> Make URLClassifyProcessor as Runtime URP; take params(s) with request
> -
>
> Key: SOLR-10859
> URL: https://issues.apache.org/jira/browse/SOLR-10859
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> URLClassifyProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=URLClassify=fieldA=fieldB=fieldC=fieldD=fieldE=fieldF=fieldG=true
> --data-binary {"id" : "1", "fieldA" : "valueA"}
> {code}
> Configuration for URLClassifyProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10867) Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10867:
Attachment: SOLR-10867.patch
Got everything in order, revamped all the tests but running into temp_index &
cache directories error while deleting and thread leaks all over the place.
error attached ::
{code}
[junit4] 2> 40694 INFO (coreCloseExecutor-34-thread-1) [
x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore
org.apache.solr.core.SolrCore@3db54c55
[junit4] 2> 40705 INFO (coreCloseExecutor-34-thread-1) [
x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for
registry=solr.core.collection1, tag=1035291733
[junit4] 2> 53230 ERROR (coreCloseExecutor-34-thread-1) [
x:collection1] o.a.s.c.CachingDirectoryFactory Timeout waiting for all
directory ref counts to be released - gave up waiting on
[jira] [Updated] (SOLR-10867) Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10867:
Attachment: SOLR-10867.patch
Initial patch uploaded.
ClassificationUpdateProcessorFactoryTest is failing miserably right now as all
the test methods verify _init_ for the config param(s). Will fix them and
update shortly.
> Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with
> request
> -
>
> Key: SOLR-10867
> URL: https://issues.apache.org/jira/browse/SOLR-10867
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10867.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> ClassificationUpdateProcessorFactory will be able to execute by sample curl
> like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=Classification=url_s=training=true
> --data-binary { "id" : "1" , "url_s" : "http://www.example.com/subroot; }
> {code}
> All the param(s) for this URP available can be passed as request handler
> param(s).
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10867) Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10867:
Attachment: SOLR-10867.patch
> Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with
> request
> -
>
> Key: SOLR-10867
> URL: https://issues.apache.org/jira/browse/SOLR-10867
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10867.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> ClassificationUpdateProcessorFactory will be able to execute by sample curl
> like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=Classification=url_s=training=true
> --data-binary { "id" : "1" , "url_s" : "http://www.example.com/subroot; }
> {code}
> All the param(s) for this URP available can be passed as request handler
> param(s).
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10867) Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10867:
Attachment: (was: SOLR-10867.patch)
> Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with
> request
> -
>
> Key: SOLR-10867
> URL: https://issues.apache.org/jira/browse/SOLR-10867
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> ClassificationUpdateProcessorFactory will be able to execute by sample curl
> like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=Classification=url_s=training=true
> --data-binary { "id" : "1" , "url_s" : "http://www.example.com/subroot; }
> {code}
> All the param(s) for this URP available can be passed as request handler
> param(s).
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-10867) Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with request
Amrit Sarkar created SOLR-10867:
---
Summary: Make ClassificationUpdateProcessorFactory as Runtime URP;
take params(s) with request
Key: SOLR-10867
URL: https://issues.apache.org/jira/browse/SOLR-10867
Project: Solr
Issue Type: Improvement
Security Level: Public (Default Security Level. Issues are Public)
Components: update
Reporter: Amrit Sarkar
Priority: Minor
We are trying to get rid of processor definitions in SolrConfig for all URPs
and take parameters in the request itself.
ClassificationUpdateProcessorFactory will be able to execute by sample curl
like below:
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=Classification=url_s=training=true
--data-binary { "id" : "1" , "url_s" : "http://www.example.com/subroot; }
{code}
All the param(s) for this URP available can be passed as request handler
param(s).
Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10867) Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10867:
Attachment: (was: SOLR-10867.patch)
> Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with
> request
> -
>
> Key: SOLR-10867
> URL: https://issues.apache.org/jira/browse/SOLR-10867
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10867.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> ClassificationUpdateProcessorFactory will be able to execute by sample curl
> like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=Classification=url_s=training=true
> --data-binary { "id" : "1" , "url_s" : "http://www.example.com/subroot; }
> {code}
> All the param(s) for this URP available can be passed as request handler
> param(s).
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10867) Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10867:
Attachment: SOLR-10867.patch
> Make ClassificationUpdateProcessorFactory as Runtime URP; take params(s) with
> request
> -
>
> Key: SOLR-10867
> URL: https://issues.apache.org/jira/browse/SOLR-10867
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10867.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> ClassificationUpdateProcessorFactory will be able to execute by sample curl
> like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=Classification=url_s=training=true
> --data-binary { "id" : "1" , "url_s" : "http://www.example.com/subroot; }
> {code}
> All the param(s) for this URP available can be passed as request handler
> param(s).
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10734) Multithreaded test/support for AtomicURP broken
[
https://issues.apache.org/jira/browse/SOLR-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10734:
Attachment: SOLR-10734.patch
Patch attached with following changes:
* Max Attempts bumped up from 5 to 25.
* Threads lowered to 10 from 100.
As the above theory posted above stands out, the following changes will make
the tests work.
> Multithreaded test/support for AtomicURP broken
> ---
>
> Key: SOLR-10734
> URL: https://issues.apache.org/jira/browse/SOLR-10734
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Ishan Chattopadhyaya
> Attachments: log-snippet, Screen Shot 2017-05-31 at 4.50.23 PM.png,
> SOLR-10734.patch
>
>
> The multithreaded test doesn't actually start the threads, but only invokes
> the run directly. The join afterwards doesn't do anything, hence.
> {code}
> diff --git
> a/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
>
> b/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
> index f3f833d..10b7770 100644
> ---
> a/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
> +++
> b/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
> @@ -238,7 +238,7 @@ public class AtomicUpdateProcessorFactoryTest extends
> SolrTestCaseJ4 {
>}
> }
>};
> - t.run();
> + t.run(); // red flag, shouldn't this be t.start?
>threads.add(t);
>finalCount += index; //int_i
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10858) Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10858:
Attachment: SOLR-10858.patch
> Make UUIDUpdateProcessorFactory as Runtime URP; take params(s) with request
> ---
>
> Key: SOLR-10858
> URL: https://issues.apache.org/jira/browse/SOLR-10858
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
> Attachments: SOLR-10858.patch, SOLR-10858.patch, SOLR-10858.patch,
> SOLR-10858.patch
>
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> UUIDUpdateProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=UUID=id=true
> --data-binary {"id":"1","title": "titleA"}
> {code}
> Configuration for UUIDUpdateProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10859) Make URLClassifyProcessor as Runtime URP; take params(s) with request
[
https://issues.apache.org/jira/browse/SOLR-10859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-10859:
Description:
We are trying to get rid of processor definitions in SolrConfig for all URPs
and take parameters in the request itself.
URLClassifyProcessorFactory will be able to execute by sample curl like below:
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=URLClassify=fieldA=fieldB=fieldC=fieldD=fieldE=fieldF=fieldG=true
--data-binary {"id" : "1", "fieldA" : "valueA"}
{code}
Configuration for URLClassifyProcessorFactory in solrconfig.xml is optional.
was:
As discussed with Noble Paul, we are trying to get rid of processor definitions
in SolrConfig for all URPs and take parameters in the request itself.
URLClassifyProcessorFactory will be able to execute by sample curl like below:
{code}
curl -X POST -H Content-Type: application/json
http://localhost:8983/solr/test/update/json/docs?processor=URLClassify=fieldA=fieldB=fieldC=fieldD=fieldE=fieldF=fieldG=true
--data-binary {"id" : "1", "fieldA" : "valueA"}
{code}
Configuration for URLClassifyProcessorFactory in solrconfig.xml is optional.
> Make URLClassifyProcessor as Runtime URP; take params(s) with request
> -
>
> Key: SOLR-10859
> URL: https://issues.apache.org/jira/browse/SOLR-10859
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: update
>Reporter: Amrit Sarkar
>Priority: Minor
>
> We are trying to get rid of processor definitions in SolrConfig for all URPs
> and take parameters in the request itself.
> URLClassifyProcessorFactory will be able to execute by sample curl like below:
> {code}
> curl -X POST -H Content-Type: application/json
> http://localhost:8983/solr/test/update/json/docs?processor=URLClassify=fieldA=fieldB=fieldC=fieldD=fieldE=fieldF=fieldG=true
> --data-binary {"id" : "1", "fieldA" : "valueA"}
> {code}
> Configuration for URLClassifyProcessorFactory in solrconfig.xml is optional.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
