date:20140116

[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_51) - Build # 9124 - Still Failing!

2014-01-16 Thread Policeman Jenkins Server

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9124/
Java: 32bit/jdk1.7.0_51 -client -XX:+UseSerialGC

All tests passed

Build Log:
[...truncated 30421 lines...]
-check-forbidden-base:
[forbidden-apis] Reading bundled API signatures: jdk-unsafe-1.7
[forbidden-apis] Reading bundled API signatures: jdk-deprecated-1.7
[forbidden-apis] Reading bundled API signatures: commons-io-unsafe-2.1
[forbidden-apis] Reading API signatures: 
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/tools/forbiddenApis/base.txt
[forbidden-apis] Reading API signatures: 
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/tools/forbiddenApis/servlet-api.txt
[forbidden-apis] Loading classes to check...
[forbidden-apis] Scanning for API signatures and dependencies...
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.cloud.OverseerRolesTest 
(OverseerRolesTest.java:160)
[forbidden-apis] Scanned 2039 (and 1337 related) class file(s) for forbidden 
API invocations (in 1.24s), 1 error(s).

BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:453: The following 
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:64: The following 
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:271: The 
following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:474: 
Check for forbidden API calls failed, see log.

Total time: 55 minutes 28 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.7.0_51 -client -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-4.x-MacOSX (64bit/jdk1.6.0) - Build # 1204 - Still Failing!

2014-01-16 Thread Policeman Jenkins Server

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-MacOSX/1204/
Java: 64bit/jdk1.6.0 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

All tests passed

Build Log:
[...truncated 16273 lines...]
   [junit4] JVM J0: stderr was not empty, see: 
/Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/contrib/solr-map-reduce/test/temp/junit4-J0-20140117_063000_798.syserr
   [junit4] >>> JVM J0: stderr (verbatim) 
   [junit4] 2014-01-17 06:30:36.678 java[4191:af0f] Unable to load realm info 
from SCDynamicStore
   [junit4] <<< JVM J0: EOF 

[...truncated 13723 lines...]
-check-forbidden-base:
[forbidden-apis] Reading bundled API signatures: jdk-unsafe-1.6
[forbidden-apis] Reading bundled API signatures: jdk-deprecated-1.6
[forbidden-apis] Reading bundled API signatures: commons-io-unsafe-2.1
[forbidden-apis] Reading API signatures: 
/Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/lucene/tools/forbiddenApis/base.txt
[forbidden-apis] Reading API signatures: 
/Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/lucene/tools/forbiddenApis/servlet-api.txt
[forbidden-apis] Loading classes to check...
[forbidden-apis] Scanning for API signatures and dependencies...
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.cloud.OverseerRolesTest 
(OverseerRolesTest.java:160)
[forbidden-apis] Scanned 1906 (and 1311 related) class file(s) for forbidden 
API invocations (in 8.49s), 1 error(s).

BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/build.xml:459: The following 
error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/build.xml:64: The following 
error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build.xml:271: The 
following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/common-build.xml:474: 
Check for forbidden API calls failed, see log.

Total time: 111 minutes 39 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.6.0 -XX:+UseCompressedOops 
-XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-4.x-Linux (64bit/jdk1.7.0_60-ea-b02) - Build # 9022 - Still Failing!

2014-01-16 Thread Policeman Jenkins Server

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/9022/
Java: 64bit/jdk1.7.0_60-ea-b02 -XX:+UseCompressedOops -XX:+UseG1GC

All tests passed

Build Log:
[...truncated 30674 lines...]
-check-forbidden-base:
[forbidden-apis] Reading bundled API signatures: jdk-unsafe-1.6
[forbidden-apis] Reading bundled API signatures: jdk-deprecated-1.6
[forbidden-apis] Reading bundled API signatures: commons-io-unsafe-2.1
[forbidden-apis] Reading API signatures: 
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/tools/forbiddenApis/base.txt
[forbidden-apis] Reading API signatures: 
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/tools/forbiddenApis/servlet-api.txt
[forbidden-apis] Loading classes to check...
[forbidden-apis] Scanning for API signatures and dependencies...
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.cloud.OverseerRolesTest 
(OverseerRolesTest.java:160)
[forbidden-apis] Scanned 1906 (and 1314 related) class file(s) for forbidden 
API invocations (in 1.34s), 1 error(s).

BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:459: The following 
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:64: The following 
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:271: The 
following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:474: 
Check for forbidden API calls failed, see log.

Total time: 56 minutes 20 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0_60-ea-b02 -XX:+UseCompressedOops 
-XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread Shawn Heisey (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874434#comment-13874434
 ] 

Shawn Heisey commented on SOLR-4260:


This might be old news by now, but I noticed it while updating my test system, 
so I'm reporting it.

The lucene_solr_4_6 branch fails to compile with these fixes committed.  One of 
the changes removes the import for RemoteSolrException from SolrCmdDistributor, 
but the doRetries method still uses this exception.  That method is very 
different in 4.6 than it is in branch_4x.  Everything's good on branch_4x.  
Re-adding the import fixes the problem, but the discrepancy between the two 
branches needs some investigation.

The specific code that fails to compile with the removed import seems to have 
been initially added to trunk by revision 1545464 (2013/11/25) and removed from 
trunk by revision 1546670 (2013/11/29).  It was then re-added to 
lucene_solr_4_6 by revision 1554122 (2013/12/29).


> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7, 4.6.1
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, SOLR-4260.patch, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4583 - Still Failing

2014-01-16 Thread Apache Jenkins Server

Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4583/

All tests passed

Build Log:
[...truncated 30676 lines...]
-check-forbidden-base:
[forbidden-apis] Reading bundled API signatures: jdk-unsafe-1.7
[forbidden-apis] Reading bundled API signatures: jdk-deprecated-1.7
[forbidden-apis] Reading bundled API signatures: commons-io-unsafe-2.1
[forbidden-apis] Reading API signatures: 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/tools/forbiddenApis/base.txt
[forbidden-apis] Reading API signatures: 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/tools/forbiddenApis/servlet-api.txt
[forbidden-apis] Loading classes to check...
[forbidden-apis] Scanning for API signatures and dependencies...
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.cloud.OverseerRolesTest 
(OverseerRolesTest.java:160)
[forbidden-apis] Scanned 2039 (and 1337 related) class file(s) for forbidden 
API invocations (in 1.67s), 1 error(s).

BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:453:
 The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:64:
 The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:271:
 The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:474:
 Check for forbidden API calls failed, see log.

Total time: 109 minutes 12 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.7.0) - Build # 1232 - Still Failing!

2014-01-16 Thread Policeman Jenkins Server

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/1232/
Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseG1GC

All tests passed

Build Log:
[...truncated 16726 lines...]
   [junit4] JVM J0: stderr was not empty, see: 
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/contrib/solr-map-reduce/test/temp/junit4-J0-20140117_043639_794.syserr
   [junit4] >>> JVM J0: stderr (verbatim) 
   [junit4] 2014-01-17 04:37:18.865 java[324:6f07] Unable to load realm info 
from SCDynamicStore
   [junit4] <<< JVM J0: EOF 

[...truncated 13655 lines...]
-check-forbidden-base:
[forbidden-apis] Reading bundled API signatures: jdk-unsafe-1.7
[forbidden-apis] Reading bundled API signatures: jdk-deprecated-1.7
[forbidden-apis] Reading bundled API signatures: commons-io-unsafe-2.1
[forbidden-apis] Reading API signatures: 
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/tools/forbiddenApis/base.txt
[forbidden-apis] Reading API signatures: 
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/tools/forbiddenApis/servlet-api.txt
[forbidden-apis] Loading classes to check...
[forbidden-apis] Scanning for API signatures and dependencies...
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.cloud.OverseerRolesTest 
(OverseerRolesTest.java:160)
[forbidden-apis] Scanned 2039 (and 1337 related) class file(s) for forbidden 
API invocations (in 3.78s), 1 error(s).

BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:453: The following 
error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:64: The following 
error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:271: The 
following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:474: 
Check for forbidden API calls failed, see log.

Total time: 97 minutes 31 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-4.x-Windows (64bit/jdk1.8.0-ea-b123) - Build # 3605 - Still Failing!

2014-01-16 Thread Policeman Jenkins Server

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Windows/3605/
Java: 64bit/jdk1.8.0-ea-b123 -XX:+UseCompressedOops -XX:+UseSerialGC

All tests passed

Build Log:
[...truncated 30883 lines...]
-check-forbidden-base:
[forbidden-apis] Reading bundled API signatures: jdk-unsafe-1.6
[forbidden-apis] Reading bundled API signatures: jdk-deprecated-1.6
[forbidden-apis] Reading bundled API signatures: commons-io-unsafe-2.1
[forbidden-apis] Reading API signatures: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\lucene\tools\forbiddenApis\base.txt
[forbidden-apis] Reading API signatures: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\lucene\tools\forbiddenApis\servlet-api.txt
[forbidden-apis] Loading classes to check...
[forbidden-apis] Scanning for API signatures and dependencies...
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.cloud.OverseerRolesTest 
(OverseerRolesTest.java:161)
[forbidden-apis] Scanned 1906 (and 1316 related) class file(s) for forbidden 
API invocations (in 2.42s), 1 error(s).

BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\build.xml:459: The 
following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\build.xml:64: The 
following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build.xml:271: The 
following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\common-build.xml:474:
 Check for forbidden API calls failed, see log.

Total time: 74 minutes 48 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.8.0-ea-b123 -XX:+UseCompressedOops 
-XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-MAVEN] Lucene-Solr-Maven-4.x #565: POMs out of sync

2014-01-16 Thread Apache Jenkins Server

Build: https://builds.apache.org/job/Lucene-Solr-Maven-4.x/565/

All tests passed

Build Log:
[...truncated 50040 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Maven-4.x/build.xml:482: 
The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Maven-4.x/build.xml:176: 
The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Maven-4.x/extra-targets.xml:77:
 Java returned: 1

Total time: 69 minutes 39 seconds
Build step 'Invoke Ant' marked build as failure
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread Mark Miller (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874364#comment-13874364
 ] 

Mark Miller commented on SOLR-4260:
---

This is a fine fix for SolrCloud, especially for 4.6.1 - but there may be a 
better general fix hidden still - what seems to happen is that we have docs 
that enter the queue that don't spawn a runner. The current fix means docs can 
be added that will sit in the queue until you call blockUntilFinished.

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7, 4.6.1
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, SOLR-4260.patch, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874352#comment-13874352
 ] 

ASF subversion and git services commented on SOLR-4260:
---

Commit 1558998 from [~markrmil...@gmail.com] in branch 
'dev/branches/lucene_solr_4_6'
[ https://svn.apache.org/r1558998 ]

SOLR-4260: If in blockUntilFinished and there are no Runners running and the 
queue is not empty, start a new Runner.

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7, 4.6.1
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, SOLR-4260.patch, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread Mark Miller (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874351#comment-13874351
 ] 

Mark Miller commented on SOLR-4260:
---

Committed something for that.

As a separate issue, it seems to me that CUSS#shutdown should probably call 
blockUntilFinished as it's first order of business.

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7, 4.6.1
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, SOLR-4260.patch, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874348#comment-13874348
 ] 

ASF subversion and git services commented on SOLR-4260:
---

Commit 1558997 from [~markrmil...@gmail.com] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1558997 ]

SOLR-4260: If in blockUntilFinished and there are no Runners running and the 
queue is not empty, start a new Runner.

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7, 4.6.1
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, SOLR-4260.patch, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874345#comment-13874345
 ] 

ASF subversion and git services commented on SOLR-4260:
---

Commit 1558996 from [~markrmil...@gmail.com] in branch 'dev/trunk'
[ https://svn.apache.org/r1558996 ]

SOLR-4260: If in blockUntilFinished and there are no Runners running and the 
queue is not empty, start a new Runner.

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7, 4.6.1
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, SOLR-4260.patch, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread Mark Miller (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874340#comment-13874340
 ] 

Mark Miller commented on SOLR-4260:
---

ChaosMonkeyNothingIsSafeTest is exposing an issue now with 
ConcurrentUpdateSolrServer - it looks like it's getting stuck in 
blockUntilFinished because the queue is not empty and no runners are being 
spawned to empty it.

It may be that NPE that would occurred before in this case just kept the docs 
from being lost 'silently', and this is closer to the actual bug?

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7, 4.6.1
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, SOLR-4260.patch, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5401) Field.StringTokenStream#end() does not call super.end()

2014-01-16 Thread Mark Miller (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-5401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated LUCENE-5401:


Fix Version/s: 4.7
   5.0

> Field.StringTokenStream#end() does not call super.end()
> ---
>
> Key: LUCENE-5401
> URL: https://issues.apache.org/jira/browse/LUCENE-5401
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/other
>Affects Versions: 4.6
>Reporter: Michael Busch
>Assignee: Michael Busch
>Priority: Minor
> Fix For: 5.0, 4.7, 4.6.1
>
> Attachments: lucene-5401.patch
>
>
> Field.StringTokenStream#end() currently does not call super.end(). This 
> prevents resetting the PositionIncrementAttribute to 0 in end(), which can 
> lead to wrong positions in the index under certain conditions.
> I added a test to TestDocument which indexes two Fields with the same name, 
> String values, indexed=true, tokenized=false and 
> IndexOptions.DOCS_AND_FREQS_AND_POSITIONS. Without the fix the test fails. 
> The first token gets the correct position 0, but the second token gets 
> position 2 instead of 1. The reason is that in DocInverterPerField line 176 
> (which is just after the call to end()) we increment the position a second 
> time, because end() didn't reset the increment to 0.
> All tests pass with the fix.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874323#comment-13874323
 ] 

ASF subversion and git services commented on SOLR-4260:
---

Commit 1558988 from [~markrmil...@gmail.com] in branch 
'dev/branches/lucene_solr_4_6'
[ https://svn.apache.org/r1558988 ]

SOLR-4260: Guard against NPE.

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7, 4.6.1
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, SOLR-4260.patch, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0-ea-b123) - Build # 9122 - Still Failing!

2014-01-16 Thread Policeman Jenkins Server

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9122/
Java: 32bit/jdk1.8.0-ea-b123 -client -XX:+UseSerialGC

All tests passed

Build Log:
[...truncated 30561 lines...]
-check-forbidden-base:
[forbidden-apis] Reading bundled API signatures: jdk-unsafe-1.7
[forbidden-apis] Reading bundled API signatures: jdk-deprecated-1.7
[forbidden-apis] Reading bundled API signatures: commons-io-unsafe-2.1
[forbidden-apis] Reading API signatures: 
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/tools/forbiddenApis/base.txt
[forbidden-apis] Reading API signatures: 
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/tools/forbiddenApis/servlet-api.txt
[forbidden-apis] Loading classes to check...
[forbidden-apis] Scanning for API signatures and dependencies...
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.cloud.OverseerRolesTest 
(OverseerRolesTest.java:161)
[forbidden-apis] Scanned 2039 (and 1339 related) class file(s) for forbidden 
API invocations (in 0.90s), 1 error(s).

BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:453: The following 
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:64: The following 
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:271: The 
following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:474: 
Check for forbidden API calls failed, see log.

Total time: 52 minutes 56 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.8.0-ea-b123 -client -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874321#comment-13874321
 ] 

ASF subversion and git services commented on SOLR-4260:
---

Commit 1558986 from [~markrmil...@gmail.com] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1558986 ]

SOLR-4260: Guard against NPE.

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7, 4.6.1
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, SOLR-4260.patch, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874320#comment-13874320
 ] 

ASF subversion and git services commented on SOLR-4260:
---

Commit 1558985 from [~markrmil...@gmail.com] in branch 'dev/trunk'
[ https://svn.apache.org/r1558985 ]

SOLR-4260: Guard against NPE.

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7, 4.6.1
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, SOLR-4260.patch, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874317#comment-13874317
 ] 

ASF subversion and git services commented on SOLR-4260:
---

Commit 1558983 from [~markrmil...@gmail.com] in branch 
'dev/branches/lucene_solr_4_6'
[ https://svn.apache.org/r1558983 ]

SOLR-4260: Add name to CHANGES

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7, 4.6.1
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, SOLR-4260.patch, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Lucene / Solr 4.6.1

2014-01-16 Thread Mark Miller

We have made some great progress on SOLR-4260 in the meantime, so I'm going
to hold the first RC just until tomorrow to make sure we have time to
validate the fix for it.


On Thu, Jan 16, 2014 at 2:34 PM, Michael Busch  wrote:

> Yes, I committed LUCENE-5401. Thanks for waiting!
>
>
> On 1/16/14 11:05 AM, Simon Willnauer wrote:
>
>> seems like we are good to go
>>
>> simon
>>
>> On Thu, Jan 16, 2014 at 1:59 PM, Simon Willnauer
>>  wrote:
>>
>>> mark, we may wait for
>>> https://issues.apache.org/jira/browse/LUCENE-5401 to be committed and
>>> merged?
>>>
>>> simon
>>>
>>> On Thu, Jan 16, 2014 at 7:34 AM, Mark Miller 
>>> wrote:
>>>
 Whoops - just built this rc with ant 1.9.2 and smoke tester still wants
 just
 1.8. I'll start another build tonight and send the vote thread in the
 morning.

 - Mark


 On Wed, Jan 15, 2014 at 3:14 PM, Simon Willnauer <
 simon.willna...@gmail.com>
 wrote:

> +1
>
> On Wed, Jan 15, 2014 at 8:02 PM, Mark Miller 
> wrote:
>
>> Unless there is an objection, I’m going to try and make a first RC
>> tonight.
>>
>> - Mark
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>>  
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

 --
 - Mark

>>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>>
>>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>


-- 
- Mark

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread Mark Miller (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874311#comment-13874311
 ] 

Mark Miller commented on SOLR-4260:
---

bq. The conditions in this statement have changed and I think made it possible 
for the null pointer to appear.

Ah, nice - thanks. I had already made some changes so couldn't line up the src 
lines - thought you meant the line that was the culprit was the one that the 
NPE came from.

I'll take a closer look.

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7, 4.6.1
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, SOLR-4260.patch, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874314#comment-13874314
 ] 

ASF subversion and git services commented on SOLR-4260:
---

Commit 1558982 from [~markrmil...@gmail.com] in branch 
'dev/branches/lucene_solr_4_6'
[ https://svn.apache.org/r1558982 ]

SOLR-4260: ConcurrentUpdateSolrServer#blockUntilFinished can return before all 
previously added updates have finished. This could cause distributed updates 
meant for replicas to be lost.

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7, 4.6.1
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, SOLR-4260.patch, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874305#comment-13874305
 ] 

ASF subversion and git services commented on SOLR-4260:
---

Commit 1558981 from [~markrmil...@gmail.com] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1558981 ]

SOLR-4260: Add name to CHANGES

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7, 4.6.1
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, SOLR-4260.patch, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread Mark Miller (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-4260:
--

Fix Version/s: 4.6.1

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7, 4.6.1
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, SOLR-4260.patch, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5560) Enable LocalParams without escaping the query

2014-01-16 Thread Ryan Cutter (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874304#comment-13874304
 ] 

Ryan Cutter commented on SOLR-5560:
---

Haha, that makes much more sense, thanks.  I'll take a longer look.

> Enable LocalParams without escaping the query
> -
>
> Key: SOLR-5560
> URL: https://issues.apache.org/jira/browse/SOLR-5560
> Project: Solr
>  Issue Type: Bug
>  Components: query parsers
>Affects Versions: 4.6
>Reporter: Isaac Hebsh
> Fix For: 4.7, 4.6.1
>
> Attachments: SOLR-5560.patch
>
>
> This query should be a legit syntax:
> http://localhost:8983/solr/collection1/select?debugQuery=true&defType=lucene&df=id&q=TERM1
>  AND {!lucene df=text}(TERM2 TERM3 "TERM4 TERM5")
> currently it isn't, because the LocalParams can be specified on a single term 
> only.
> [~billnbell] thinks it is a bug.
> From the mailing list:
> {quote}
> We want to set a LocalParam on a nested query. When quering with "v" inline 
> parameter, it works fine:
> http://localhost:8983/solr/collection1/select?debugQuery=true&defType=lucene&df=id&q=TERM1
>  AND {!lucene df=text v="TERM2 TERM3 \"TERM4 TERM5\""}
> the parsedquery_toString is
> +id:TERM1 +(text:term2 text:term3 text:"term4 term5")
> Query using the "_query_" also works fine:
> http://localhost:8983/solr/collection1/select?debugQuery=true&defType=lucene&df=id&q=TERM1
>  AND _query_:"{!lucene df=text}TERM2 TERM3 \"TERM4 TERM5\""
> (parsedquery is exactly the same).
> Obviously, there is the option of external parameter ({... 
> v=$nestedq}&nestedq=...)
> This is a good solution, but it is not practical, when having a lot of such 
> nested queries.
> BUT, when trying to put the nested query in place, it yields syntax error:
> http://localhost:8983/solr/collection1/select?debugQuery=true&defType=lucene&df=id&q=TERM1
>  AND {!lucene df=text}(TERM2 TERM3 "TERM4 TERM5")
> org.apache.solr.search.SyntaxError: Cannot parse '(TERM2'
> The previous options are less preferred, because the escaping that should be 
> made on the nested query.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread Mark Miller (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874301#comment-13874301
 ] 

Mark Miller commented on SOLR-4260:
---

Well, this is important for 4.6.1 - given Potter's feedback, in it goes. Please 
help test and review this guys. Especially around this possible NPE.

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, SOLR-4260.patch, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874302#comment-13874302
 ] 

ASF subversion and git services commented on SOLR-4260:
---

Commit 1558979 from [~markrmil...@gmail.com] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1558979 ]

SOLR-4260: ConcurrentUpdateSolrServer#blockUntilFinished can return before all 
previously added updates have finished. This could cause distributed updates 
meant for replicas to be lost.

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, SOLR-4260.patch, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874303#comment-13874303
 ] 

ASF subversion and git services commented on SOLR-4260:
---

Commit 1558980 from [~markrmil...@gmail.com] in branch 'dev/trunk'
[ https://svn.apache.org/r1558980 ]

SOLR-4260: Add name to CHANGES

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, SOLR-4260.patch, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874300#comment-13874300
 ] 

ASF subversion and git services commented on SOLR-4260:
---

Commit 1558978 from [~markrmil...@gmail.com] in branch 'dev/trunk'
[ https://svn.apache.org/r1558978 ]

SOLR-4260: ConcurrentUpdateSolrServer#blockUntilFinished can return before all 
previously added updates have finished. This could cause distributed updates 
meant for replicas to be lost.

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, SOLR-4260.patch, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5560) Enable LocalParams without escaping the query

2014-01-16 Thread Mark Miller (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874295#comment-13874295
 ] 

Mark Miller commented on SOLR-5560:
---

bq. Not much is documented and states are unremarkable ints. 

It's generated javacc code, so I think we actually have to move your logic up a 
level into the .jj file that the parser is generated from.

> Enable LocalParams without escaping the query
> -
>
> Key: SOLR-5560
> URL: https://issues.apache.org/jira/browse/SOLR-5560
> Project: Solr
>  Issue Type: Bug
>  Components: query parsers
>Affects Versions: 4.6
>Reporter: Isaac Hebsh
> Fix For: 4.7, 4.6.1
>
> Attachments: SOLR-5560.patch
>
>
> This query should be a legit syntax:
> http://localhost:8983/solr/collection1/select?debugQuery=true&defType=lucene&df=id&q=TERM1
>  AND {!lucene df=text}(TERM2 TERM3 "TERM4 TERM5")
> currently it isn't, because the LocalParams can be specified on a single term 
> only.
> [~billnbell] thinks it is a bug.
> From the mailing list:
> {quote}
> We want to set a LocalParam on a nested query. When quering with "v" inline 
> parameter, it works fine:
> http://localhost:8983/solr/collection1/select?debugQuery=true&defType=lucene&df=id&q=TERM1
>  AND {!lucene df=text v="TERM2 TERM3 \"TERM4 TERM5\""}
> the parsedquery_toString is
> +id:TERM1 +(text:term2 text:term3 text:"term4 term5")
> Query using the "_query_" also works fine:
> http://localhost:8983/solr/collection1/select?debugQuery=true&defType=lucene&df=id&q=TERM1
>  AND _query_:"{!lucene df=text}TERM2 TERM3 \"TERM4 TERM5\""
> (parsedquery is exactly the same).
> Obviously, there is the option of external parameter ({... 
> v=$nestedq}&nestedq=...)
> This is a good solution, but it is not practical, when having a lot of such 
> nested queries.
> BUT, when trying to put the nested query in place, it yields syntax error:
> http://localhost:8983/solr/collection1/select?debugQuery=true&defType=lucene&df=id&q=TERM1
>  AND {!lucene df=text}(TERM2 TERM3 "TERM4 TERM5")
> org.apache.solr.search.SyntaxError: Cannot parse '(TERM2'
> The previous options are less preferred, because the escaping that should be 
> made on the nested query.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread Joel Bernstein (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874294#comment-13874294
 ] 

Joel Bernstein commented on SOLR-4260:
--

It's actually the runner that is null:
{code}
runner.runnerLock.lock();
{code}

The conditions in this statement have changed and I think made it possible for 
the null pointer to appear.
{code}
if ((runner == null && queue.isEmpty()) || scheduler.isTerminated())
{code}

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, SOLR-4260.patch, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-trunk-Windows (32bit/jdk1.8.0-ea-b123) - Build # 3682 - Still Failing!

2014-01-16 Thread Policeman Jenkins Server

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/3682/
Java: 32bit/jdk1.8.0-ea-b123 -client -XX:+UseParallelGC

All tests passed

Build Log:
[...truncated 30493 lines...]
-check-forbidden-base:
[forbidden-apis] Reading bundled API signatures: jdk-unsafe-1.7
[forbidden-apis] Reading bundled API signatures: jdk-deprecated-1.7
[forbidden-apis] Reading bundled API signatures: commons-io-unsafe-2.1
[forbidden-apis] Reading API signatures: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\tools\forbiddenApis\base.txt
[forbidden-apis] Reading API signatures: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\tools\forbiddenApis\servlet-api.txt
[forbidden-apis] Loading classes to check...
[forbidden-apis] Scanning for API signatures and dependencies...
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.cloud.OverseerRolesTest 
(OverseerRolesTest.java:161)
[forbidden-apis] Scanned 2039 (and 1339 related) class file(s) for forbidden 
API invocations (in 2.45s), 1 error(s).

BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:453: The 
following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:64: The 
following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build.xml:271: 
The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\common-build.xml:474:
 Check for forbidden API calls failed, see log.

Total time: 79 minutes 55 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.8.0-ea-b123 -client -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 1865 - Still Failing

2014-01-16 Thread Apache Jenkins Server

Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/1865/

All tests passed

Build Log:
[...truncated 30668 lines...]
-check-forbidden-base:
[forbidden-apis] Reading bundled API signatures: jdk-unsafe-1.6
[forbidden-apis] Reading bundled API signatures: jdk-deprecated-1.6
[forbidden-apis] Reading bundled API signatures: commons-io-unsafe-2.1
[forbidden-apis] Reading API signatures: 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/tools/forbiddenApis/base.txt
[forbidden-apis] Reading API signatures: 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/tools/forbiddenApis/servlet-api.txt
[forbidden-apis] Loading classes to check...
[forbidden-apis] Scanning for API signatures and dependencies...
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.cloud.OverseerRolesTest 
(OverseerRolesTest.java:160)
[forbidden-apis] Scanned 1906 (and 1314 related) class file(s) for forbidden 
API invocations (in 1.71s), 1 error(s).

BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:459:
 The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:64:
 The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:271:
 The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/common-build.xml:474:
 Check for forbidden API calls failed, see log.

Total time: 97 minutes 20 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread Mark Miller (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874270#comment-13874270
 ] 

Mark Miller commented on SOLR-4260:
---

Strange Joel - queue and scheduler are both final and set in the constructor.

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, SOLR-4260.patch, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread Joel Bernstein (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874204#comment-13874204
 ] 

Joel Bernstein edited comment on SOLR-4260 at 1/17/14 12:34 AM:


In the code snippet below it looks like this line is the culprit:

{code}
if ((runner == null && queue.isEmpty()) || scheduler.isTerminated())
{code}

{code}
 public synchronized void blockUntilFinished(boolean waitForEmptyQueue) {
lock = new CountDownLatch(1);
try {
  // Wait until no runners are running
  for (;;) {
Runner runner;
synchronized (runners) {
  runner = runners.peek();
}
if (waitForEmptyQueue) {
  if ((runner == null && queue.isEmpty()) || scheduler.isTerminated())
break;
} else {
  if (runner == null || scheduler.isTerminated())
break;
}
runner.runnerLock.lock();
runner.runnerLock.unlock();
  }
} finally {
  lock.countDown();
  lock = null;
}
  }
{code}


was (Author: joel.bernstein):
In the code snippet below it looks like runners.peek() can return null and 
cause the exception:

{code}
 public synchronized void blockUntilFinished(boolean waitForEmptyQueue) {
lock = new CountDownLatch(1);
try {
  // Wait until no runners are running
  for (;;) {
Runner runner;
synchronized (runners) {
  runner = runners.peek();
}
if (waitForEmptyQueue) {
  if ((runner == null && queue.isEmpty()) || scheduler.isTerminated())
break;
} else {
  if (runner == null || scheduler.isTerminated())
break;
}
runner.runnerLock.lock();
runner.runnerLock.unlock();
  }
} finally {
  lock.countDown();
  lock = null;
}
  }
{code}

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, SOLR-4260.patch, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread Joel Bernstein (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874204#comment-13874204
 ] 

Joel Bernstein edited comment on SOLR-4260 at 1/17/14 12:30 AM:


In the code snippet below it looks like runners.peek() can return null and 
cause the exception:

{code}
 public synchronized void blockUntilFinished(boolean waitForEmptyQueue) {
lock = new CountDownLatch(1);
try {
  // Wait until no runners are running
  for (;;) {
Runner runner;
synchronized (runners) {
  runner = runners.peek();
}
if (waitForEmptyQueue) {
  if ((runner == null && queue.isEmpty()) || scheduler.isTerminated())
break;
} else {
  if (runner == null || scheduler.isTerminated())
break;
}
runner.runnerLock.lock();
runner.runnerLock.unlock();
  }
} finally {
  lock.countDown();
  lock = null;
}
  }
{code}


was (Author: joel.bernstein):
In the code snippet below it looks runners.peek() can return null and cause the 
exception:

{code}
 public synchronized void blockUntilFinished(boolean waitForEmptyQueue) {
lock = new CountDownLatch(1);
try {
  // Wait until no runners are running
  for (;;) {
Runner runner;
synchronized (runners) {
  runner = runners.peek();
}
if (waitForEmptyQueue) {
  if ((runner == null && queue.isEmpty()) || scheduler.isTerminated())
break;
} else {
  if (runner == null || scheduler.isTerminated())
break;
}
runner.runnerLock.lock();
runner.runnerLock.unlock();
  }
} finally {
  lock.countDown();
  lock = null;
}
  }
{code}

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, SOLR-4260.patch, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread Joel Bernstein (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874204#comment-13874204
 ] 

Joel Bernstein edited comment on SOLR-4260 at 1/17/14 12:23 AM:


In the code snippet below it looks runners.peek() can return null and cause the 
exception:

{code}
 public synchronized void blockUntilFinished(boolean waitForEmptyQueue) {
lock = new CountDownLatch(1);
try {
  // Wait until no runners are running
  for (;;) {
Runner runner;
synchronized (runners) {
  runner = runners.peek();
}
if (waitForEmptyQueue) {
  if ((runner == null && queue.isEmpty()) || scheduler.isTerminated())
break;
} else {
  if (runner == null || scheduler.isTerminated())
break;
}
runner.runnerLock.lock();
runner.runnerLock.unlock();
  }
} finally {
  lock.countDown();
  lock = null;
}
  }
{code}


was (Author: joel.bernstein):
In code snippet below it looks runners.peek() can return null and cause the 
exception:

{code}
 public synchronized void blockUntilFinished(boolean waitForEmptyQueue) {
lock = new CountDownLatch(1);
try {
  // Wait until no runners are running
  for (;;) {
Runner runner;
synchronized (runners) {
  runner = runners.peek();
}
if (waitForEmptyQueue) {
  if ((runner == null && queue.isEmpty()) || scheduler.isTerminated())
break;
} else {
  if (runner == null || scheduler.isTerminated())
break;
}
runner.runnerLock.lock();
runner.runnerLock.unlock();
  }
} finally {
  lock.countDown();
  lock = null;
}
  }
{code}

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, SOLR-4260.patch, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread Joel Bernstein (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874204#comment-13874204
 ] 

Joel Bernstein commented on SOLR-4260:
--

In code snippet below it looks runners.peek() can return null and cause the 
exception:

{code}
 public synchronized void blockUntilFinished(boolean waitForEmptyQueue) {
lock = new CountDownLatch(1);
try {
  // Wait until no runners are running
  for (;;) {
Runner runner;
synchronized (runners) {
  runner = runners.peek();
}
if (waitForEmptyQueue) {
  if ((runner == null && queue.isEmpty()) || scheduler.isTerminated())
break;
} else {
  if (runner == null || scheduler.isTerminated())
break;
}
runner.runnerLock.lock();
runner.runnerLock.unlock();
  }
} finally {
  lock.countDown();
  lock = null;
}
  }
{code}

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, SOLR-4260.patch, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread Joel Bernstein (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874194#comment-13874194
 ] 

Joel Bernstein commented on SOLR-4260:
--

I installed the patch and ran it. 

I'm getting some intermittent null pointers:

1578995 [qtp433857665-17] ERROR org.apache.solr.servlet.SolrDispatchFilter  – 
null:java.lang.NullPointerException
at 
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrServer.blockUntilFinished(ConcurrentUpdateSolrServer.java:401)
at 
org.apache.solr.update.StreamingSolrServers.blockUntilFinished(StreamingSolrServers.java:99)
at 
org.apache.solr.update.SolrCmdDistributor.finish(SolrCmdDistributor.java:69)
at 
org.apache.solr.update.processor.DistributedUpdateProcessor.doFinish(DistributedUpdateProcessor.java:606)
at 
org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:1449)
at 
org.apache.solr.update.processor.LogUpdateProcessor.finish(LogUpdateProcessorFactory.java:179)
at 
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:83)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1915)
at 
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:764)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:203)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, SOLR-4260.patch, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5639) Return type parameter 'wt' is completely ignored when url is http encoded

2014-01-16 Thread Suneeta Mall (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-5639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Suneeta Mall updated SOLR-5639:
---

Description: 
Querying solr with 'wt' parameter formats the result type as requested which 
works fine except when url is http encoded.

For example: 
http://localhost:8983/solr/suggest?q=Status:ac&wt=json&indent=true
the response I get is :




  0
  1


  

  5
  7
  9
  
acknowledged
ack
actual
actually
access
  

Status:acknowledged
  



whereas the correct response should be:

{
  "responseHeader":{
    "status":0,
    "QTime":1},
  "spellcheck":{
    "suggestions":[
      "ac",{
        "numFound":5,
        "startOffset":7,
        "endOffset":9,
        "suggestion":["acknowledged",
          "ack",
          "actual",
          "actually",
          "access"]},
      "collation","Status:acknowledged"]}}


This causes severe problem when solr is integrated with GWT client where 
embedded script often encode url as per http encoding and ends up failing with 
timeout exception. Specially noticeable solr JSONP queries. for example: 
http://localhost:8983/solr/suggest?q=Status:ac&wt=json&indent=true&json.wrf=xyz
 when it returns xml instead of json. 

 Noticeably other arguments works perfectly fine for example: 
http://localhost:8983/solr/suggest?q=Status:ac&wt=json&indent=true.

  was:
Querying solr with 'wt' parameter formats the result type as requested which 
works fine except when url is http encoded.

For example: 
http://localhost:8983/solr/suggest?q=Status:ac&wt=json&indent=true
the response I get is :




  0
  1


  

  5
  7
  9
  
acknowledged
ack
actual
actually
access
  

Status:acknowledged
  



whereas the correct response should be:

{
  "responseHeader":{
    "status":0,
    "QTime":1},
  "spellcheck":{
    "suggestions":[
      "ac",{
        "numFound":5,
        "startOffset":7,
        "endOffset":9,
        "suggestion":["acknowledged",
          "ack",
          "actual",
          "actually",
          "access"]},
      "collation","Status:acknowledged"]}}


This causes severe problem when solr is integrated with GWT client where 
embedded script often encode url as per http encoding and ends up failing with 
timeout exception. 

 Noticeably other arguments works perfectly fine for example: 
http://localhost:8983/solr/suggest?q=Status:ac&wt=json&indent=true.


> Return type parameter 'wt' is completely ignored when url is http encoded
> -
>
> Key: SOLR-5639
> URL: https://issues.apache.org/jira/browse/SOLR-5639
> Project: Solr
>  Issue Type: Bug
>  Components: query parsers, SearchComponents - other
>Affects Versions: 4.5.1
> Environment: Ubuntu 13.04, 
> Browser Chrome 
>Reporter: Suneeta Mall
>
> Querying solr with 'wt' parameter formats the result type as requested which 
> works fine except when url is http encoded.
> For example: 
> http://localhost:8983/solr/suggest?q=Status:ac&wt=json&indent=true
> the response I get is :
> 
> 
> 
>   0
>   1
> 
> 
>   
> 
>   5
>   7
>   9
>   
> acknowledged
> ack
> actual
> actually
> access
>   
> 
> Status:acknowledged
>   
> 
> 
> whereas the correct response should be:
> {
>   "responseHeader":{
>     "status":0,
>     "QTime":1},
>   "spellcheck":{
>     "suggestions":[
>       "ac",{
>         "numFound":5,
>         "startOffset":7,
>         "endOffset":9,
>         "suggestion":["acknowledged",
>           "ack",
>           "actual",
>           "actually",
>           "access"]},
>       "collation","Status:acknowledged"]}}
> This causes severe problem when solr is integrated with GWT client where 
> embedded script often encode url as per http encoding and ends up failing 
> with timeout exception. Specially noticeable solr JSONP queries. for example: 
> http://localhost:8983/solr/suggest?q=Status:ac&wt=json&indent=true&json.wrf=xyz
>  when it returns xml instead of json. 
>  Noticeably other arguments works perfectly fine for example: 
> http://localhost:8983/solr/suggest?q=Status:ac&wt=json&indent=true.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5639) Return type parameter 'wt' is completely ignored when url is http encoded

2014-01-16 Thread Suneeta Mall (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-5639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Suneeta Mall updated SOLR-5639:
---

Description: 
Querying solr with 'wt' parameter formats the result type as requested which 
works fine except when url is http encoded.

For example: 
http://localhost:8983/solr/suggest?q=Status:ac&wt=json&indent=true
the response I get is :




  0
  1


  

  5
  7
  9
  
acknowledged
ack
actual
actually
access
  

Status:acknowledged
  



whereas the correct response should be:

{
  "responseHeader":{
    "status":0,
    "QTime":1},
  "spellcheck":{
    "suggestions":[
      "ac",{
        "numFound":5,
        "startOffset":7,
        "endOffset":9,
        "suggestion":["acknowledged",
          "ack",
          "actual",
          "actually",
          "access"]},
      "collation","Status:acknowledged"]}}


This causes severe problem when solr is integrated with GWT client where 
embedded script often encode url as per http encoding and ends up failing with 
timeout exception. 

 Noticeably other arguments works perfectly fine for example: 
http://localhost:8983/solr/suggest?q=Status:ac&wt=json&indent=true.

  was:
Querying solr with wt parameter formats the result type as requested which 
works fine except when url is http encoded. 

For example: 
http://localhost:8983/solr/suggest?q=Status:ac&wt=json&indent=true
the response I get is :





  0
  1


  

  5
  7
  9
  
acknowledged
ack
actual
actually
access
  

Status:acknowledged
  



whereas the correct response should be:




{
  "responseHeader":{
    "status":0,
    "QTime":1},
  "spellcheck":{
    "suggestions":[
      "ac",{
        "numFound":5,
        "startOffset":7,
        "endOffset":9,
        "suggestion":["acknowledged",
          "ack",
          "actual",
          "actually",
          "access"]},
      "collation","Status:acknowledged"]}}


This causes severe problem when solr is integrated with GWT client where 
embedded script often encode url as per http encoding and ends up failing with 
timeout exception. 



> Return type parameter 'wt' is completely ignored when url is http encoded
> -
>
> Key: SOLR-5639
> URL: https://issues.apache.org/jira/browse/SOLR-5639
> Project: Solr
>  Issue Type: Bug
>  Components: query parsers, SearchComponents - other
>Affects Versions: 4.5.1
> Environment: Ubuntu 13.04, 
> Browser Chrome 
>Reporter: Suneeta Mall
>
> Querying solr with 'wt' parameter formats the result type as requested which 
> works fine except when url is http encoded.
> For example: 
> http://localhost:8983/solr/suggest?q=Status:ac&wt=json&indent=true
> the response I get is :
> 
> 
> 
>   0
>   1
> 
> 
>   
> 
>   5
>   7
>   9
>   
> acknowledged
> ack
> actual
> actually
> access
>   
> 
> Status:acknowledged
>   
> 
> 
> whereas the correct response should be:
> {
>   "responseHeader":{
>     "status":0,
>     "QTime":1},
>   "spellcheck":{
>     "suggestions":[
>       "ac",{
>         "numFound":5,
>         "startOffset":7,
>         "endOffset":9,
>         "suggestion":["acknowledged",
>           "ack",
>           "actual",
>           "actually",
>           "access"]},
>       "collation","Status:acknowledged"]}}
> This causes severe problem when solr is integrated with GWT client where 
> embedded script often encode url as per http encoding and ends up failing 
> with timeout exception. 
>  Noticeably other arguments works perfectly fine for example: 
> http://localhost:8983/solr/suggest?q=Status:ac&wt=json&indent=true.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5639) Return type parameter 'wt' is completely ignored when url is http encoded

2014-01-16 Thread Suneeta Mall (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-5639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Suneeta Mall updated SOLR-5639:
---

Summary: Return type parameter 'wt' is completely ignored when url is http 
encoded  (was: Return type parameter is completely ignored when url is http 
encoded)

> Return type parameter 'wt' is completely ignored when url is http encoded
> -
>
> Key: SOLR-5639
> URL: https://issues.apache.org/jira/browse/SOLR-5639
> Project: Solr
>  Issue Type: Bug
>  Components: query parsers, SearchComponents - other
>Affects Versions: 4.5.1
> Environment: Ubuntu 13.04, 
> Browser Chrome 
>Reporter: Suneeta Mall
>
> Querying solr with wt parameter formats the result type as requested which 
> works fine except when url is http encoded. 
> For example: 
> http://localhost:8983/solr/suggest?q=Status:ac&wt=json&indent=true
> the response I get is :
> 
> 
> 
>   0
>   1
> 
> 
>   
> 
>   5
>   7
>   9
>   
> acknowledged
> ack
> actual
> actually
> access
>   
> 
> Status:acknowledged
>   
> 
> 
> whereas the correct response should be:
> {
>   "responseHeader":{
>     "status":0,
>     "QTime":1},
>   "spellcheck":{
>     "suggestions":[
>       "ac",{
>         "numFound":5,
>         "startOffset":7,
>         "endOffset":9,
>         "suggestion":["acknowledged",
>           "ack",
>           "actual",
>           "actually",
>           "access"]},
>       "collation","Status:acknowledged"]}}
> This causes severe problem when solr is integrated with GWT client where 
> embedded script often encode url as per http encoding and ends up failing 
> with timeout exception. 



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5639) Return type parameter is completely ignored when url is http encoded

2014-01-16 Thread Suneeta Mall (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-5639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Suneeta Mall updated SOLR-5639:
---

Description: 
Querying solr with wt parameter formats the result type as requested which 
works fine except when url is http encoded. 

For example: 
http://localhost:8983/solr/suggest?q=Status:ac&wt=json&indent=true
the response I get is :





  0
  1


  

  5
  7
  9
  
acknowledged
ack
actual
actually
access
  

Status:acknowledged
  



whereas the correct response should be:




{
  "responseHeader":{
    "status":0,
    "QTime":1},
  "spellcheck":{
    "suggestions":[
      "ac",{
        "numFound":5,
        "startOffset":7,
        "endOffset":9,
        "suggestion":["acknowledged",
          "ack",
          "actual",
          "actually",
          "access"]},
      "collation","Status:acknowledged"]}}


This causes severe problem when solr is integrated with GWT client where 
embedded script often encode url as per http encoding and ends up failing with 
timeout exception. 


  was:
Querying solr with wt parameter formats the result type as requested which 
works fine except when url is http encoded. 

For example: 
http://localhost:8983/solr/suggest?q=Status:ac&wt=json&indent=true
the response I get is :





  0
  1


  

  5
  7
  9
  
acknowledged
ack
actual
actually
access
  

Status:acknowledged
  



whereas the correct response should be:




> Return type parameter is completely ignored when url is http encoded
> 
>
> Key: SOLR-5639
> URL: https://issues.apache.org/jira/browse/SOLR-5639
> Project: Solr
>  Issue Type: Bug
>  Components: query parsers, SearchComponents - other
>Affects Versions: 4.5.1
> Environment: Ubuntu 13.04, 
> Browser Chrome 
>Reporter: Suneeta Mall
>
> Querying solr with wt parameter formats the result type as requested which 
> works fine except when url is http encoded. 
> For example: 
> http://localhost:8983/solr/suggest?q=Status:ac&wt=json&indent=true
> the response I get is :
> 
> 
> 
>   0
>   1
> 
> 
>   
> 
>   5
>   7
>   9
>   
> acknowledged
> ack
> actual
> actually
> access
>   
> 
> Status:acknowledged
>   
> 
> 
> whereas the correct response should be:
> {
>   "responseHeader":{
>     "status":0,
>     "QTime":1},
>   "spellcheck":{
>     "suggestions":[
>       "ac",{
>         "numFound":5,
>         "startOffset":7,
>         "endOffset":9,
>         "suggestion":["acknowledged",
>           "ack",
>           "actual",
>           "actually",
>           "access"]},
>       "collation","Status:acknowledged"]}}
> This causes severe problem when solr is integrated with GWT client where 
> embedded script often encode url as per http encoding and ends up failing 
> with timeout exception. 



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-5639) Return type parameter is completely ignored when url is http encoded

2014-01-16 Thread Suneeta Mall (JIRA)

Suneeta Mall created SOLR-5639:
--

 Summary: Return type parameter is completely ignored when url is 
http encoded
 Key: SOLR-5639
 URL: https://issues.apache.org/jira/browse/SOLR-5639
 Project: Solr
  Issue Type: Bug
  Components: query parsers, SearchComponents - other
Affects Versions: 4.5.1
 Environment: Ubuntu 13.04, 
Browser Chrome 
Reporter: Suneeta Mall


Querying solr with wt parameter formats the result type as requested which 
works fine except when url is http encoded. 

For example: 
http://localhost:8983/solr/suggest?q=Status:ac&wt=json&indent=true
the response I get is :





  0
  1


  

  5
  7
  9
  
acknowledged
ack
actual
actually
access
  

Status:acknowledged
  



whereas the correct response should be:





--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5638) Collection creation partially works, but results in unusable configuration due to missing config in ZK

2014-01-16 Thread Nathan Neulinger (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874134#comment-13874134
 ] 

Nathan Neulinger commented on SOLR-5638:


Alternatively/additionally - solr really should be checking for validity of the 
requested create. If you ask for a configName, and it doesn't exist - error out 
then instead of proceeding with the create that is guaranteed to fail as a 
whole.


Procedure to reproduce: do a collection create for a config name that doesn't 
exist in ZK.


> Collection creation partially works, but results in unusable configuration 
> due to missing config in ZK
> --
>
> Key: SOLR-5638
> URL: https://issues.apache.org/jira/browse/SOLR-5638
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Affects Versions: 4.6
>Reporter: Nathan Neulinger
>
> Need help properly recovering from 'collection gets created without config 
> being defined'.
> Right now, if you submit a collection create and the config is missing, it 
> will proceed with partially creating cores, but then the cores fail to load. 
> This requires manual intervention on the server to fix unless you pick a new 
> colllection name:
> What's worse - if you retry the create a second time, it will usually try to 
> create the replicas in the opposite order, resulting in TWO broken cores on 
> each box, one for each attempted replica. 
> beta1-newarch_hive1_v12_shard1_replica1: 
> org.apache.solr.common.cloud.ZooKeeperException:org.apache.solr.common.cloud.ZooKeeperException:
>  Specified config does not exist in ZooKeeper:hivepoint-unknown
> beta1-newarch_hive1_v12_shard1_replica2: 
> org.apache.solr.common.cloud.ZooKeeperException:org.apache.solr.common.cloud.ZooKeeperException:
>  Specified config does not exist in ZooKeeper:hivepoint-unknown
> I already know how to clear this up manually, but this is something where 
> solr is allowing a condition in external service to result in a 
> corrupted/partial configuration. 
> I can see an easy option for resolving this as a workaround - allow a 
> collection CREATE operation to specify "reuseCores"  - i.e. allow it to use 
> an existing core of the proper name if it already exists. 
> Right now you wind up getting:
> org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error 
> CREATEing SolrCore 'beta1-newarch_hive1_v12_shard1_replica1': Could not 
> create a new core in solr/beta1-newarch_hive1_v12_shard1_replica1/as another 
> core is already defined there
> org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error 
> CREATEing SolrCore 'beta1-newarch_hive1_v12_shard1_replica2': Could not 
> create a new core in solr/beta1-newarch_hive1_v12_shard1_replica2/as another 
> core is already defined there



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-5638) Collection creation partially works, but results in unusable configuration due to missing config in ZK

2014-01-16 Thread Nathan Neulinger (JIRA)

Nathan Neulinger created SOLR-5638:
--

 Summary: Collection creation partially works, but results in 
unusable configuration due to missing config in ZK
 Key: SOLR-5638
 URL: https://issues.apache.org/jira/browse/SOLR-5638
 Project: Solr
  Issue Type: Bug
  Components: SolrCloud
Affects Versions: 4.6
Reporter: Nathan Neulinger


Need help properly recovering from 'collection gets created without config 
being defined'.

Right now, if you submit a collection create and the config is missing, it will 
proceed with partially creating cores, but then the cores fail to load. 

This requires manual intervention on the server to fix unless you pick a new 
colllection name:

What's worse - if you retry the create a second time, it will usually try to 
create the replicas in the opposite order, resulting in TWO broken cores on 
each box, one for each attempted replica. 

beta1-newarch_hive1_v12_shard1_replica1: 
org.apache.solr.common.cloud.ZooKeeperException:org.apache.solr.common.cloud.ZooKeeperException:
 Specified config does not exist in ZooKeeper:hivepoint-unknown
beta1-newarch_hive1_v12_shard1_replica2: 
org.apache.solr.common.cloud.ZooKeeperException:org.apache.solr.common.cloud.ZooKeeperException:
 Specified config does not exist in ZooKeeper:hivepoint-unknown


I already know how to clear this up manually, but this is something where solr 
is allowing a condition in external service to result in a corrupted/partial 
configuration. 

I can see an easy option for resolving this as a workaround - allow a 
collection CREATE operation to specify "reuseCores"  - i.e. allow it to use an 
existing core of the proper name if it already exists. 

Right now you wind up getting:

org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error 
CREATEing SolrCore 'beta1-newarch_hive1_v12_shard1_replica1': Could not create 
a new core in solr/beta1-newarch_hive1_v12_shard1_replica1/as another core is 
already defined there

org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error 
CREATEing SolrCore 'beta1-newarch_hive1_v12_shard1_replica2': Could not create 
a new core in solr/beta1-newarch_hive1_v12_shard1_replica2/as another core is 
already defined there





--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread Timothy Potter (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874120#comment-13874120
 ] 

Timothy Potter commented on SOLR-4260:
--

Did another couple of million docs in an oversharded env. 24 replicas on 6 
nodes (m1.mediums so I didn't want to overload them too much) ... still looking 
good.

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, SOLR-4260.patch, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread Timothy Potter (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874098#comment-13874098
 ] 

Timothy Potter commented on SOLR-4260:
--

So far so good, Mark! I applied the patch to latest rev of branch_4x and have 
indexed about 3M docs without hitting the issue, before the patch, I would see 
this issue within a few minutes. So jury is still out and I'll keep stress 
testing it, but looks promising. Nice work!

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, SOLR-4260.patch, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5463) Provide cursor/token based "searchAfter" support that works with arbitrary sorting (ie: "deep paging")

2014-01-16 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874094#comment-13874094
 ] 

ASF subversion and git services commented on SOLR-5463:
---

Commit 1558945 from hoss...@apache.org in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1558945 ]

SOLR-5463: more details in case of spooky 'walk already seen' errors (merge 
r1558939)

> Provide cursor/token based "searchAfter" support that works with arbitrary 
> sorting (ie: "deep paging")
> --
>
> Key: SOLR-5463
> URL: https://issues.apache.org/jira/browse/SOLR-5463
> Project: Solr
>  Issue Type: New Feature
>Reporter: Hoss Man
>Assignee: Hoss Man
> Fix For: 5.0, 4.7
>
> Attachments: SOLR-5463-randomized-faceting-test.patch, 
> SOLR-5463.patch, SOLR-5463.patch, SOLR-5463.patch, SOLR-5463.patch, 
> SOLR-5463.patch, SOLR-5463.patch, SOLR-5463.patch, 
> SOLR-5463__straw_man.patch, SOLR-5463__straw_man.patch, 
> SOLR-5463__straw_man.patch, SOLR-5463__straw_man.patch, 
> SOLR-5463__straw_man.patch, SOLR-5463__straw_man.patch, 
> SOLR-5463__straw_man.patch, SOLR-5463__straw_man.patch, 
> SOLR-5463__straw_man.patch, SOLR-5463__straw_man.patch, 
> SOLR-5463__straw_man__MissingStringLastComparatorSource.patch
>
>
> I'd like to revist a solution to the problem of "deep paging" in Solr, 
> leveraging an HTTP based API similar to how IndexSearcher.searchAfter works 
> at the lucene level: require the clients to provide back a token indicating 
> the sort values of the last document seen on the previous "page".  This is 
> similar to the "cursor" model I've seen in several other REST APIs that 
> support "pagnation" over a large sets of results (notable the twitter API and 
> it's "since_id" param) except that we'll want something that works with 
> arbitrary multi-level sort critera that can be either ascending or descending.
> SOLR-1726 laid some initial ground work here and was commited quite a while 
> ago, but the key bit of argument parsing to leverage it was commented out due 
> to some problems (see comments in that issue).  It's also somewhat out of 
> date at this point: at the time it was commited, IndexSearcher only supported 
> searchAfter for simple scores, not arbitrary field sorts; and the params 
> added in SOLR-1726 suffer from this limitation as well.
> ---
> I think it would make sense to start fresh with a new issue with a focus on 
> ensuring that we have deep paging which:
> * supports arbitrary field sorts in addition to sorting by score
> * works in distributed mode
> {panel:title=Basic Usage}
> * send a request with {{sort=X&start=0&rows=N&cursorMark=*}}
> ** sort can be anything, but must include the uniqueKey field (as a tie 
> breaker) 
> ** "N" can be any number you want per page
> ** start must be "0"
> ** "\*" denotes you want to use a cursor starting at the beginning mark
> * parse the response body and extract the (String) {{nextCursorMark}} value
> * Replace the "\*" value in your initial request params with the 
> {{nextCursorMark}} value from the response in the subsequent request
> * repeat until the {{nextCursorMark}} value stops changing, or you have 
> collected as many docs as you need
> {panel}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5463) Provide cursor/token based "searchAfter" support that works with arbitrary sorting (ie: "deep paging")

2014-01-16 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874071#comment-13874071
 ] 

ASF subversion and git services commented on SOLR-5463:
---

Commit 1558939 from hoss...@apache.org in branch 'dev/trunk'
[ https://svn.apache.org/r1558939 ]

SOLR-5463: more details in case of spooky 'walk already seen' errors

> Provide cursor/token based "searchAfter" support that works with arbitrary 
> sorting (ie: "deep paging")
> --
>
> Key: SOLR-5463
> URL: https://issues.apache.org/jira/browse/SOLR-5463
> Project: Solr
>  Issue Type: New Feature
>Reporter: Hoss Man
>Assignee: Hoss Man
> Fix For: 5.0, 4.7
>
> Attachments: SOLR-5463-randomized-faceting-test.patch, 
> SOLR-5463.patch, SOLR-5463.patch, SOLR-5463.patch, SOLR-5463.patch, 
> SOLR-5463.patch, SOLR-5463.patch, SOLR-5463.patch, 
> SOLR-5463__straw_man.patch, SOLR-5463__straw_man.patch, 
> SOLR-5463__straw_man.patch, SOLR-5463__straw_man.patch, 
> SOLR-5463__straw_man.patch, SOLR-5463__straw_man.patch, 
> SOLR-5463__straw_man.patch, SOLR-5463__straw_man.patch, 
> SOLR-5463__straw_man.patch, SOLR-5463__straw_man.patch, 
> SOLR-5463__straw_man__MissingStringLastComparatorSource.patch
>
>
> I'd like to revist a solution to the problem of "deep paging" in Solr, 
> leveraging an HTTP based API similar to how IndexSearcher.searchAfter works 
> at the lucene level: require the clients to provide back a token indicating 
> the sort values of the last document seen on the previous "page".  This is 
> similar to the "cursor" model I've seen in several other REST APIs that 
> support "pagnation" over a large sets of results (notable the twitter API and 
> it's "since_id" param) except that we'll want something that works with 
> arbitrary multi-level sort critera that can be either ascending or descending.
> SOLR-1726 laid some initial ground work here and was commited quite a while 
> ago, but the key bit of argument parsing to leverage it was commented out due 
> to some problems (see comments in that issue).  It's also somewhat out of 
> date at this point: at the time it was commited, IndexSearcher only supported 
> searchAfter for simple scores, not arbitrary field sorts; and the params 
> added in SOLR-1726 suffer from this limitation as well.
> ---
> I think it would make sense to start fresh with a new issue with a focus on 
> ensuring that we have deep paging which:
> * supports arbitrary field sorts in addition to sorting by score
> * works in distributed mode
> {panel:title=Basic Usage}
> * send a request with {{sort=X&start=0&rows=N&cursorMark=*}}
> ** sort can be anything, but must include the uniqueKey field (as a tie 
> breaker) 
> ** "N" can be any number you want per page
> ** start must be "0"
> ** "\*" denotes you want to use a cursor starting at the beginning mark
> * parse the response body and extract the (String) {{nextCursorMark}} value
> * Replace the "\*" value in your initial request params with the 
> {{nextCursorMark}} value from the response in the subsequent request
> * repeat until the {{nextCursorMark}} value stops changing, or you have 
> collected as many docs as you need
> {panel}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-4.x-MacOSX (64bit/jdk1.6.0) - Build # 1200 - Failure!

2014-01-16 Thread Chris Hostetter


: REGRESSION:  org.apache.solr.cloud.DistribCursorPagingTest.testDistribSearch
: 
: Error Message:
: walk already seen: 39

I can't reproduce this seed (or this type of failure with a bunch of 
looping on other seeds) and sarowe couldn't repro this seed on his mac 
either.

I can't even think of a reason why this type of failure might ever happen 
... but i'm going to commit some additional logic in the test to try and 
fail with more detail if it ever happens again.


-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5636) SolrRequestParsers does some xpath lookups on every request.

2014-01-16 Thread Erik Hatcher (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874055#comment-13874055
 ] 

Erik Hatcher commented on SOLR-5636:


any numbers/ideas on how much faster this makes Solr requests?

> SolrRequestParsers does some xpath lookups on every request.
> 
>
> Key: SOLR-5636
> URL: https://issues.apache.org/jira/browse/SOLR-5636
> Project: Solr
>  Issue Type: Bug
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Minor
> Fix For: 5.0, 4.7
>
> Attachments: SOLR-5636.patch
>
>
> This seems a bit wasteful for one, but also, under heavy load, with lots of 
> cores on a node, I've seen this xpath parsing randomly fail with weird 
> nullpointer exceptions. Perhaps depends on the xml parser you end up using. 
> Anyway, it's easy to work around and avoid the parsing everytime 
> solrdispatchfilter is hit by just doing it up front once.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread Mark Miller (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874031#comment-13874031
 ] 

Mark Miller commented on SOLR-4260:
---

That's all I have come up with so far - though I'm not even completely sold on 
it. Because we are using CUSS with a single thread, all the previous doc adds 
should have hit the request method and so a Runner should be going for them if 
necessary.

It's all pretty tricky logic to understand clearly though.

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, SOLR-4260.patch, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5477) Async execution of OverseerCollectionProcessor tasks

2014-01-16 Thread Anshum Gupta (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874003#comment-13874003
 ] 

Anshum Gupta commented on SOLR-5477:


Thanks all of you. I'll go with approach#2 in that case.

I think more than zk being a scalability bottleneck, not wanting to delegate 
all of the communication etc. to zk could be a stronger reason to not take 
approach#1.

> Async execution of OverseerCollectionProcessor tasks
> 
>
> Key: SOLR-5477
> URL: https://issues.apache.org/jira/browse/SOLR-5477
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrCloud
>Reporter: Noble Paul
>Assignee: Anshum Gupta
> Attachments: SOLR-5477-CoreAdminStatus.patch, SOLR-5477.patch
>
>
> Typical collection admin commands are long running and it is very common to 
> have the requests get timed out.  It is more of a problem if the cluster is 
> very large.Add an option to run these commands asynchronously
> add an extra param async=true for all collection commands
> the task is written to ZK and the caller is returned a task id. 
> as separate collection admin command will be added to poll the status of the 
> task
> command=status&id=7657668909
> if id is not passed all running async tasks should be listed
> A separate queue is created to store in-process tasks . After the tasks are 
> completed the queue entry is removed. OverSeerColectionProcessor will perform 
> these tasks in multiple threads



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread Mark Miller (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-4260:
--

Attachment: SOLR-4260.patch

Patch attached that does the above.

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, SOLR-4260.patch, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread Mark Miller (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873964#comment-13873964
 ] 

Mark Miller commented on SOLR-4260:
---

For a long time, I've wanted to try putting in a check that the queue is empty 
as well for blockUntilFinished when we use it in this case - I just need a test 
that sees this so I can check if it works :)

Without that, it seems there is a window where we can bail before we are done 
sending everything in the queue. Shutdown doesn't help much, because it can't 
even wait for the executor to shutdown in this case.

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread Timothy Potter (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873951#comment-13873951
 ] 

Timothy Potter commented on SOLR-4260:
--

Added some more logging on the leader ... as a bit of context, the replica 
received doc with ID 41029 and then 41041 and didn't receive 41033 and 41038 in 
between ... here's the log on the leader of activity between 41029 and then 
41041.

2014-01-16 16:03:02,523 [updateExecutor-1-thread-1] INFO  
solrj.impl.ConcurrentUpdateSolrServer  - sent docs to 
[http://ec2-107-21-55-0.compute-1.amazonaws.com:8985/solr/test3_shard3_replica1]
 , 41003, 41
005, 41007, 41010, 41014, 41015, 41026, 41029
2014-01-16 16:03:02,527 [qtp417447538-16] INFO  handler.loader.JavabinLoader  - 
test3_shard3_replica2 add: 41033
2014-01-16 16:03:02,527 [qtp417447538-16] INFO  
update.processor.DistributedUpdateProcessor  - doLocalAdd 41033
2014-01-16 16:03:02,527 [qtp417447538-16] INFO  
solrj.impl.ConcurrentUpdateSolrServer  - test3_shard3_replica2 queued (to: 
http://ec2-107-21-55-0.compute-1.amazonaws.com:8985/solr/test3_shard3_replica1):
 41033
2014-01-16 16:03:02,528 [qtp417447538-16] INFO  handler.loader.JavabinLoader  - 
test3_shard3_replica2 add: 41038
2014-01-16 16:03:02,528 [qtp417447538-16] INFO  
update.processor.DistributedUpdateProcessor  - doLocalAdd 41038
2014-01-16 16:03:02,528 [qtp417447538-16] INFO  
solrj.impl.ConcurrentUpdateSolrServer  - test3_shard3_replica2 queued (to: 
http://ec2-107-21-55-0.compute-1.amazonaws.com:8985/solr/test3_shard3_replica1):
 41038
2014-01-16 16:03:02,559 [qtp417447538-16] INFO  
solrj.impl.ConcurrentUpdateSolrServer  - blockUntilFinished starting 
http://ec2-107-21-55-0.compute-1.amazonaws.com:8985/solr/test3_shard3_replica1
2014-01-16 16:03:02,559 [qtp417447538-16] INFO  
solrj.impl.ConcurrentUpdateSolrServer  - blockUntilFinished is done for 
http://ec2-107-21-55-0.compute-1.amazonaws.com:8985/solr/test3_shard3_replica1
2014-01-16 16:03:02,559 [qtp417447538-16] INFO  
solrj.impl.ConcurrentUpdateSolrServer  - shutting down CUSS for 
http://ec2-107-21-55-0.compute-1.amazonaws.com:8985/solr/test3_shard3_replica1
2014-01-16 16:03:02,559 [qtp417447538-16] INFO  
solrj.impl.ConcurrentUpdateSolrServer  - shut down CUSS for 
http://ec2-107-21-55-0.compute-1.amazonaws.com:8985/solr/test3_shard3_replica1

Not quite sure what this means but I think you're hunch about 
blockUntilFinished being involved is getting warmer

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-4.x-Windows (32bit/jdk1.8.0-ea-b123) - Build # 3604 - Still Failing!

2014-01-16 Thread Policeman Jenkins Server

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Windows/3604/
Java: 32bit/jdk1.8.0-ea-b123 -server -XX:+UseParallelGC

All tests passed

Build Log:
[...truncated 30794 lines...]
-check-forbidden-base:
[forbidden-apis] Reading bundled API signatures: jdk-unsafe-1.6
[forbidden-apis] Reading bundled API signatures: jdk-deprecated-1.6
[forbidden-apis] Reading bundled API signatures: commons-io-unsafe-2.1
[forbidden-apis] Reading API signatures: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\lucene\tools\forbiddenApis\base.txt
[forbidden-apis] Reading API signatures: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\lucene\tools\forbiddenApis\servlet-api.txt
[forbidden-apis] Loading classes to check...
[forbidden-apis] Scanning for API signatures and dependencies...
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.cloud.OverseerRolesTest 
(OverseerRolesTest.java:161)
[forbidden-apis] Scanned 1906 (and 1316 related) class file(s) for forbidden 
API invocations (in 2.21s), 1 error(s).

BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\build.xml:459: The 
following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\build.xml:64: The 
following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build.xml:271: The 
following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\common-build.xml:474:
 Check for forbidden API calls failed, see log.

Total time: 79 minutes 34 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.8.0-ea-b123 -server -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread Timothy Potter (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873888#comment-13873888
 ] 

Timothy Potter edited comment on SOLR-4260 at 1/16/14 8:34 PM:
---

bq. So strange - it would be a different CUSS instance used for each server

right, I was just mentioning that I did check to make sure there wasn't a bug 
in the routing logic or anything like that but I now see that was silly because 
it wouldn't be able to go to the other replicas because the message was on the 
correct queue ;-)

agreed on the need for a unit test to reproduce this and am working on the same.


was (Author: tim.potter):
bq. So strange - it would be a different CUSS instance used for each server

right, I was just mentioning that I did check to make sure there wasn't a bug 
in the routing logic or anything like that ...

agreed on the need for a unit test to reproduce this and am working on the same.

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread Timothy Potter (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873888#comment-13873888
 ] 

Timothy Potter commented on SOLR-4260:
--

bq. So strange - it would be a different CUSS instance used for each server

right, I was just mentioning that I did check to make sure there wasn't a bug 
in the routing logic or anything like that ...

agreed on the need for a unit test to reproduce this and am working on the same.

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Span Not Queries

2014-01-16 Thread Gopal Agarwal

Sounds perfect. Hopefully one of the committer picks this up and adds this
to 4.7.

Will keep checking the updates...


On Fri, Jan 17, 2014 at 1:17 AM, Allison, Timothy B. wrote:

>  And don’t forget analysis! J
>
>
>
> The code is non-trivial, and it will take a generous committer to help me
> get it into shape for committing.  Once I push my mods to jira (end of next
> week), you should be able to compile it and run it at least for dev/testing
> to confirm that it meets your needs.
>
>
>
> *From:* Gopal Agarwal [mailto:gopal.agarw...@gmail.com]
> *Sent:* Thursday, January 16, 2014 1:21 PM
> *To:* dev@lucene.apache.org
> *Subject:* Re: Span Not Queries
>
>
>
> Thanks Tim. This exactly fits my requirements of recursion, SpanNot and
> ComplexParser combination with Boolean Parser.
>
>
>
> Since I would end up doing the exact same changes to my QueryParserBase
> class, I would be locked with the current version of SOLR for forseeable
> future.
>
>
>
> Can you comment on when is the possible release if it gets reviewed by
> next week?
>
>
>
>
>
> On Thu, Jan 16, 2014 at 11:06 PM, Allison, Timothy B. 
> wrote:
>
> Apologies for the self-promotion…LUCENE-5205 and its Solr cousin
> (SOLR-5410) might help.  I’m hoping to post updates to both by the end of
> next week.  Then, if a committer would be willing to review and add these
> to Lucene/Solr, you should be good to go.
>
>
>
> Take a look at the description for LUCENE-5205and see if that capability
> will meet your needs.  Thank you.
>
>
>
>   Best,
>
>
>
>  Tim
>
>
>
> *From:* Gopal Agarwal [mailto:gopal.agarw...@gmail.com]
> *Sent:* Thursday, January 16, 2014 4:10 AM
> *To:* dev@lucene.apache.org
> *Subject:* Fwd: Span Not Queries
>
>
>
> Please help me out with earlier query.
>
>
>
> In short:
>
> 1. Can we change the QueryParser.jj file to identify the SpanNot query as
> a boolean clause?
>
>
>
> 2. Can we use ComplexPhraseQuery Parser to support SpanOR and SpanNOT
> queries also?
>
>
>
> For further explanation, following are the examples.
>
>
>
> On Tue, Oct 15, 2013 at 11:27 PM, Ankit Kumar 
> wrote:
>
> *I have a business use case in which i need to use Span Not and
> other ordered proximity queries . And they can be nested upto any level
> A Boolean inside a ordered query or ordered query inside a Boolean
>  . Currently i am thinking of changing the QuerParser.jj file to identify
> the SpanNot query and use Complex Phrase Query Parser of Lucene for parsing
> complex queries . Can you suggest better way of achieving this.*
>
> *Following are the list of additions that i need to do in SOLR.*
>
> *1. Span NOT Operator*  .
>
> 2.Adding Recursive and Range Proximity
>
>   *Recursive Proximity *is a proximity query within a proximity query
>
> Ex:   “ “income tax”~5   statement” ~4  The recursion can be up to any
> level.
>
> * Range Proximity*: Currently we can only define number as a range we
> want interval as a range .
>
> Ex: “profit income”~3,5,  “United America”~-5,4
>
>
>
> 3. Complex  Queries
>
> A complex query is a query formed with a combination of Boolean operators
> or proximity queries or range queries or any possible combination of these.
>
> Ex:“(income AND tax) statement”~4
>
>   “ “income tax”~4  (statement OR period) ”~3
>
>   (“ income” SPAN NOT  “income tax” ) source ~3,5
>
>  Can anyone suggest us some way of achieving these 3 functionalities in
> SOLR
>  ???
>
>
> On Tue, Oct 15, 2013 at 10:15 PM, Jack Krupansky  >wrote:
>
>
> > Nope. But the LucidWorks Search product query parser does support SpanNot
> > if you use their BEFORE, AFTER, and NEAR span operators.
> >
> > See:
>
> > http://docs.lucidworks.com/**display/lweug/Proximity+**Operations<
> http://docs.lucidworks.com/display/lweug/Proximity+Operations>
>
> >
> > For example: "George BEFORE:2 Bush NOT H" to match George 
> Bush,
> > but not George H. W. Bush.
> >
> > What is your specific use case?
> >
> > -- Jack Krupansky
> >
> > -Original Message- From: Ankit Kumar
> > Sent: Tuesday, October 15, 2013 3:58 AM
> > To: solr-u...@lucene.apache.org
> > Subject: Span Not Queries
> >
> >
> > I need to add Span Not queries in solr . Ther's a parser Surround Query
> > Parser  i went through this (
>
> > http://lucene.472066.n3.**nabble.com/Surround-query-**
> > parser-not-working-td4075066.**html<
> http://lucene.472066.n3.nabble.com/Surround-query-parser-not-working-td4075066.html
> >
>
> > )
> > to discover that surround query parser does not analyze text
> >
> > Does DisMaxQueryParser supports SpanNot Queries ??
> >
>
>
>
>
>
>
>

[jira] [Comment Edited] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread Mark Miller (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873882#comment-13873882
 ] 

Mark Miller edited comment on SOLR-4260 at 1/16/14 8:17 PM:


I have various theories, but without a test that fails, it's hard to test out 
anything - so I've been putting most of my efforts into a unit test that can 
get this, but it's been surprisingly difficult for me to trigger in a test.


was (Author: markrmil...@gmail.com):
I have various theories, but with a test that fails, it's hard to test out 
anything - so I've been putting most of my efforts into a unit test that can 
get this, but it's been surprisingly difficult for me to trigger in a test.

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread Mark Miller (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873882#comment-13873882
 ] 

Mark Miller commented on SOLR-4260:
---

I have various theories, but with a test that fails, it's hard to test out 
anything - so I've been putting most of my efforts into a unit test that can 
get this, but it's been surprisingly difficult for me to trigger in a test.

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread Joel Bernstein (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873874#comment-13873874
 ] 

Joel Bernstein commented on SOLR-4260:
--

That was a blind alley, a faulty test was causing the effect I described above.


> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b123) - Build # 9017 - Still Failing!

2014-01-16 Thread Policeman Jenkins Server

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/9017/
Java: 32bit/jdk1.8.0-ea-b123 -client -XX:+UseG1GC

All tests passed

Build Log:
[...truncated 30746 lines...]
-check-forbidden-base:
[forbidden-apis] Reading bundled API signatures: jdk-unsafe-1.6
[forbidden-apis] Reading bundled API signatures: jdk-deprecated-1.6
[forbidden-apis] Reading bundled API signatures: commons-io-unsafe-2.1
[forbidden-apis] Reading API signatures: 
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/tools/forbiddenApis/base.txt
[forbidden-apis] Reading API signatures: 
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/tools/forbiddenApis/servlet-api.txt
[forbidden-apis] Loading classes to check...
[forbidden-apis] Scanning for API signatures and dependencies...
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.cloud.OverseerRolesTest 
(OverseerRolesTest.java:161)
[forbidden-apis] Scanned 1906 (and 1316 related) class file(s) for forbidden 
API invocations (in 1.28s), 1 error(s).

BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:459: The following 
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:64: The following 
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:271: The 
following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:474: 
Check for forbidden API calls failed, see log.

Total time: 50 minutes 30 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.8.0-ea-b123 -client -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread Mark Miller (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873868#comment-13873868
 ] 

Mark Miller commented on SOLR-4260:
---

bq.  I've checked the logs on all the other replicas and the docs didn't go 
there either.

So strange - it would be a different CUSS instance used for each server

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread Shikhar Bhushan (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873851#comment-13873851
 ] 

Shikhar Bhushan commented on SOLR-4260:
---

This may be unrelated - I have not done much digging or looked at the full 
context, but was just looking at CUSS out of curiosity.

Why do we flush() the OutputStream, but then write() on stuff like ending tags? 
Shouldn't the flush be after all those writes()'s?

https://github.com/apache/lucene-solr/blob/lucene_solr_4_6/solr/solrj/src/java/org/apache/solr/client/solrj/impl/ConcurrentUpdateSolrServer.java#L205

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5477) Async execution of OverseerCollectionProcessor tasks

2014-01-16 Thread Mark Miller (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873843#comment-13873843
 ] 

Mark Miller commented on SOLR-5477:
---

Given the numbers published for ZooKeeper, even with 10,000 cores all watching, 
I highly doubt it would be a *huge* scalability issue to have them all watching 
a queue.

Approach #2 still looks like the right approach though. Nice and simple.

> Async execution of OverseerCollectionProcessor tasks
> 
>
> Key: SOLR-5477
> URL: https://issues.apache.org/jira/browse/SOLR-5477
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrCloud
>Reporter: Noble Paul
>Assignee: Anshum Gupta
> Attachments: SOLR-5477-CoreAdminStatus.patch, SOLR-5477.patch
>
>
> Typical collection admin commands are long running and it is very common to 
> have the requests get timed out.  It is more of a problem if the cluster is 
> very large.Add an option to run these commands asynchronously
> add an extra param async=true for all collection commands
> the task is written to ZK and the caller is returned a task id. 
> as separate collection admin command will be added to poll the status of the 
> task
> command=status&id=7657668909
> if id is not passed all running async tasks should be listed
> A separate queue is created to store in-process tasks . After the tasks are 
> completed the queue entry is removed. OverSeerColectionProcessor will perform 
> these tasks in multiple threads



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread Timothy Potter (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873841#comment-13873841
 ] 

Timothy Potter commented on SOLR-4260:
--

I was able to reproduce this issue on EC2 without any over-sharding (on latest 
rev on branch_4x) ... basically 6 Solr nodes with 3 shards and RF=2, i.e. each 
replica gets its own Solr instance. Here's the output from my client app that 
traps the inconsistency:

>>
Found 1 shards with mis-matched doc counts.
At January 16, 2014 12:18:08 PM MST
shard2: {

http://ec2-54-236-245-61.compute-1.amazonaws.com:8985/solr/test_shard2_replica2/
 = 62984 LEADER

http://ec2-107-21-55-0.compute-1.amazonaws.com:8985/solr/test_shard2_replica1/ 
= 62980 diff:4
}
Details:
shard2
>> finished querying leader, found 62984 documents (62984)
>> finished querying 
>> http://ec2-107-21-55-0.compute-1.amazonaws.com:8985/solr/test_shard2_replica1/,
>>  found 62980 documents
Doc [182866] not found in replica: 182866test-12573452420.926573630.5259114828332452this is a 
test1457415570117885953
Doc [182859] not found in replica: 182859test9913669090.53117160.10846350752086309this is a 
test1457415570117885952
Doc [182872] not found in replica: 182872test8245128970.8303660.6560223698806142this is a 
test1457415570117885954
Doc [182876] not found in replica: 182876test-16578314730.48779650.9214420679315872this is a 
test1457415570117885955
Sending hard commit after mis-match and then will wait for user to handle it ...
<<

So four missing docs: 182866, 182859, 182872, 182876

Now I'm thinking this might be in the ConcurrentUpdateSolrServer logic. I added 
some detailed logging to show when JavabinLoader unmarshals a doc and when it 
is offered on the CUSS queue (to be sent to the replica). On the leader, here's 
the log around some messages that were lost:

2014-01-16 14:16:37,534 [qtp417447538-17] INFO  handler.loader.JavabinLoader  - 
test_shard2_replica2 add: 182857
2014-01-16 14:16:37,534 [qtp417447538-17] INFO  
solrj.impl.ConcurrentUpdateSolrServer  - test_shard2_replica2 queued: 182857
/
2014-01-16 14:16:37,552 [qtp417447538-17] INFO  handler.loader.JavabinLoader  - 
test_shard2_replica2 add: 182859
2014-01-16 14:16:37,552 [qtp417447538-17] INFO  
solrj.impl.ConcurrentUpdateSolrServer  - test_shard2_replica2 queued: 182859
2014-01-16 14:16:37,552 [qtp417447538-17] INFO  handler.loader.JavabinLoader  - 
test_shard2_replica2 add: 182866
2014-01-16 14:16:37,552 [qtp417447538-17] INFO  
solrj.impl.ConcurrentUpdateSolrServer  - test_shard2_replica2 queued: 182866
2014-01-16 14:16:37,552 [qtp417447538-17] INFO  handler.loader.JavabinLoader  - 
test_shard2_replica2 add: 182872
2014-01-16 14:16:37,552 [qtp417447538-17] INFO  
solrj.impl.ConcurrentUpdateSolrServer  - test_shard2_replica2 queued: 182872
2014-01-16 14:16:37,552 [qtp417447538-17] INFO  handler.loader.JavabinLoader  - 
test_shard2_replica2 add: 182876
2014-01-16 14:16:37,552 [qtp417447538-17] INFO  
solrj.impl.ConcurrentUpdateSolrServer  - test_shard2_replica2 queued: 182876
2014-01-16 14:16:37,558 [qtp417447538-17] INFO  
update.processor.LogUpdateProcessor  - [test_shard2_replica2] webapp=/solr 
path=/update params={wt=javabin&version=2} {add=[182704 (1457415570048679936), 
182710 (1457415570049728512), 182711 (1457415570049728513), 182717 
(1457415570056019968), 182720 (1457415570056019969), 182722 
(1457415570057068544), 182723 (1457415570057068545), 182724 
(1457415570058117120), 182730 (1457415570058117121), 182735 
(1457415570059165696), ... (61 adds)]} 0 72
/
2014-01-16 14:16:37,764 [qtp417447538-17] INFO  handler.loader.JavabinLoader  - 
test_shard2_replica2 add: 182880
2014-01-16 14:16:37,764 [qtp417447538-17] INFO  
solrj.impl.ConcurrentUpdateSolrServer  - test_shard2_replica2 queued: 182880
 

As you can see, the leader received doc with ID:182859 at 2014-01-16 
14:16:37,552 and the queued it on the CUSS queue to be sent to the replica. On 
the replica, the log shows it receiving 182857 and then 182880 ... the 4 
missing docs (182866, 182859, 182872, 182876) were definitely queued in CUSS on 
the leader. I've checked the logs on all the other replicas and the docs didn't 
go there either.


2014-01-16 14:16:37,292 [qtp417447538-14] INFO  handler.loader.JavabinLoader  - 
test_shard2_replica1 add: 182857
2014-01-16 14:16:37,293 [qtp417447538-14] INFO  
update.processor.LogUpdateProcessor  - [test_shard2_replica1] webapp=/solr 
path=/update 
params={distrib.from=http://ec2-54-236-245-61.compute-1.amazonaws.com:8985/solr/test_shard2_replica2/&update.distrib=FROMLEADER&wt=javabin&version=2}
 {add=[182841 (1457415570096914432), 182842 (1457415570096914433), 182843 
(1457415570096914434), 182844 (1457415570096914435), 182846 
(1457415570097963008), 182848 (1457415570097963009), 182850 
(1457415570099011584), 182854 (145741557009

RE: Span Not Queries

2014-01-16 Thread Allison, Timothy B.

And don't forget analysis! :)

The code is non-trivial, and it will take a generous committer to help me get 
it into shape for committing.  Once I push my mods to jira (end of next week), 
you should be able to compile it and run it at least for dev/testing to confirm 
that it meets your needs.

From: Gopal Agarwal [mailto:gopal.agarw...@gmail.com]
Sent: Thursday, January 16, 2014 1:21 PM
To: dev@lucene.apache.org
Subject: Re: Span Not Queries

Thanks Tim. This exactly fits my requirements of recursion, SpanNot and 
ComplexParser combination with Boolean Parser.

Since I would end up doing the exact same changes to my QueryParserBase class, 
I would be locked with the current version of SOLR for forseeable future.

Can you comment on when is the possible release if it gets reviewed by next 
week?

On Thu, Jan 16, 2014 at 11:06 PM, Allison, Timothy B. 
mailto:talli...@mitre.org>> wrote:
Apologies for the self-promotion...LUCENE-5205 and its Solr cousin (SOLR-5410) 
might help.  I'm hoping to post updates to both by the end of next week.  Then, 
if a committer would be willing to review and add these to Lucene/Solr, you 
should be good to go.

Take a look at the description for LUCENE-5205and see if that capability will 
meet your needs.  Thank you.

  Best,

 Tim

From: Gopal Agarwal 
[mailto:gopal.agarw...@gmail.com]
Sent: Thursday, January 16, 2014 4:10 AM
To: dev@lucene.apache.org
Subject: Fwd: Span Not Queries

Please help me out with earlier query.

In short:
1. Can we change the QueryParser.jj file to identify the SpanNot query as a 
boolean clause?

2. Can we use ComplexPhraseQuery Parser to support SpanOR and SpanNOT queries 
also?

For further explanation, following are the examples.

On Tue, Oct 15, 2013 at 11:27 PM, Ankit Kumar 
mailto:ankitthemight...@gmail.com>> wrote:
*I have a business use case in which i need to use Span Not and
other ordered proximity queries . And they can be nested upto any level
A Boolean inside a ordered query or ordered query inside a Boolean
 . Currently i am thinking of changing the QuerParser.jj file to identify
the SpanNot query and use Complex Phrase Query Parser of Lucene for parsing
complex queries . Can you suggest better way of achieving this.*

*Following are the list of additions that i need to do in SOLR.*

*1. Span NOT Operator*  .

2.Adding Recursive and Range Proximity

  *Recursive Proximity *is a proximity query within a proximity query

Ex:   " "income tax"~5   statement" ~4  The recursion can be up to any
level.

* Range Proximity*: Currently we can only define number as a range we
want interval as a range .

Ex: "profit income"~3,5,  "United America"~-5,4

3. Complex  Queries

A complex query is a query formed with a combination of Boolean operators
or proximity queries or range queries or any possible combination of these.

Ex:"(income AND tax) statement"~4

  " "income tax"~4  (statement OR period) "~3

  (" income" SPAN NOT  "income tax" ) source ~3,5

 Can anyone suggest us some way of achieving these 3 functionalities in SOLR
 ???

On Tue, Oct 15, 2013 at 10:15 PM, Jack Krupansky 
mailto:j...@basetechnology.com>>wrote:

> Nope. But the LucidWorks Search product query parser does support SpanNot
> if you use their BEFORE, AFTER, and NEAR span operators.
>
> See:
> http://docs.lucidworks.com/**display/lweug/Proximity+**Operations
>
> For example: "George BEFORE:2 Bush NOT H" to match George  Bush,
> but not George H. W. Bush.
>
> What is your specific use case?
>
> -- Jack Krupansky
>
> -Original Message- From: Ankit Kumar
> Sent: Tuesday, October 15, 2013 3:58 AM
> To: solr-u...@lucene.apache.org
> Subject: Span Not Queries
>
>
> I need to add Span Not queries in solr . Ther's a parser Surround Query
> Parser  i went through this (
> http://lucene.472066.n3.**nabble.com/Surround-query-**
> parser-not-working-td4075066.**html
> )
> to discover that surround query parser does not analyze text
>
> Does DisMaxQueryParser supports SpanNot Queries ??
>

Re: Lucene / Solr 4.6.1

2014-01-16 Thread Michael Busch


Yes, I committed LUCENE-5401. Thanks for waiting!

On 1/16/14 11:05 AM, Simon Willnauer wrote:

seems like we are good to go

simon

On Thu, Jan 16, 2014 at 1:59 PM, Simon Willnauer
 wrote:

mark, we may wait for
https://issues.apache.org/jira/browse/LUCENE-5401 to be committed and
merged?

simon

On Thu, Jan 16, 2014 at 7:34 AM, Mark Miller  wrote:

Whoops - just built this rc with ant 1.9.2 and smoke tester still wants just
1.8. I'll start another build tonight and send the vote thread in the
morning.

- Mark


On Wed, Jan 15, 2014 at 3:14 PM, Simon Willnauer 
wrote:

+1

On Wed, Jan 15, 2014 at 8:02 PM, Mark Miller 
wrote:

Unless there is an objection, I’m going to try and make a first RC
tonight.

- Mark
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org




--
- Mark

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org





-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0-ea-b123) - Build # 9118 - Still Failing!

2014-01-16 Thread Policeman Jenkins Server

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9118/
Java: 64bit/jdk1.8.0-ea-b123 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

All tests passed

Build Log:
[...truncated 30666 lines...]
-check-forbidden-base:
[forbidden-apis] Reading bundled API signatures: jdk-unsafe-1.7
[forbidden-apis] Reading bundled API signatures: jdk-deprecated-1.7
[forbidden-apis] Reading bundled API signatures: commons-io-unsafe-2.1
[forbidden-apis] Reading API signatures: 
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/tools/forbiddenApis/base.txt
[forbidden-apis] Reading API signatures: 
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/tools/forbiddenApis/servlet-api.txt
[forbidden-apis] Loading classes to check...
[forbidden-apis] Scanning for API signatures and dependencies...
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.cloud.OverseerRolesTest 
(OverseerRolesTest.java:161)
[forbidden-apis] Scanned 2039 (and 1339 related) class file(s) for forbidden 
API invocations (in 1.00s), 1 error(s).

BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:453: The following 
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:64: The following 
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:271: The 
following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:474: 
Check for forbidden API calls failed, see log.

Total time: 55 minutes 13 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.8.0-ea-b123 -XX:-UseCompressedOops 
-XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread Joel Bernstein (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873788#comment-13873788
 ] 

Joel Bernstein commented on SOLR-4260:
--

I'm betting it's something in the streaming. This afternoon I'm going to put 
some debugging in to see if the docs being flushed by the commit were already 
written to the stream. My bet is that they were, and that the commit is pushing 
them all the way through to the replica. 

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Lucene / Solr 4.6.1

2014-01-16 Thread Simon Willnauer

seems like we are good to go

simon

On Thu, Jan 16, 2014 at 1:59 PM, Simon Willnauer
 wrote:
> mark, we may wait for
> https://issues.apache.org/jira/browse/LUCENE-5401 to be committed and
> merged?
>
> simon
>
> On Thu, Jan 16, 2014 at 7:34 AM, Mark Miller  wrote:
>> Whoops - just built this rc with ant 1.9.2 and smoke tester still wants just
>> 1.8. I'll start another build tonight and send the vote thread in the
>> morning.
>>
>> - Mark
>>
>>
>> On Wed, Jan 15, 2014 at 3:14 PM, Simon Willnauer 
>> wrote:
>>>
>>> +1
>>>
>>> On Wed, Jan 15, 2014 at 8:02 PM, Mark Miller 
>>> wrote:
>>> > Unless there is an objection, I’m going to try and make a first RC
>>> > tonight.
>>> >
>>> > - Mark
>>> > -
>>> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> > For additional commands, e-mail: dev-h...@lucene.apache.org
>>> >
>>>
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>>
>>
>>
>>
>> --
>> - Mark

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-5399) PagingFieldCollector is very slow with String fields

2014-01-16 Thread Michael McCandless (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-5399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless resolved LUCENE-5399.


   Resolution: Fixed
Fix Version/s: 4.7
   5.0

> PagingFieldCollector is very slow with String fields
> 
>
> Key: LUCENE-5399
> URL: https://issues.apache.org/jira/browse/LUCENE-5399
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/search
>Reporter: Robert Muir
> Fix For: 5.0, 4.7
>
> Attachments: LUCENE-5399.patch, LUCENE-5399.patch, LUCENE-5399.patch, 
> LUCENE-5399.patch, LUCENE-5399.patch, LUCENE-5399.patch, LUCENE-5399.patch, 
> LUCENE-5399.patch, LUCENE-5399.patch
>
>
> PagingFieldCollector (sort comparator) is significantly slower with string 
> fields, because of how its "seen on a previous page" works: it calls 
> compareDocToValue(int doc, T t) first to check this. (its the only user of 
> this method)
> This is very slow with String, because no ordinals are used. so each document 
> must lookup ord, then lookup bytes, then compare bytes.
> I think maybe we should replace this method with an 'after' slot, and just 
> have compareDocToAfter or something.
> Otherwise we could use a hack-patch like the one i will upload (i did this 
> just to test the performance, although tests do pass).



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5399) PagingFieldCollector is very slow with String fields

2014-01-16 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873770#comment-13873770
 ] 

ASF subversion and git services commented on LUCENE-5399:
-

Commit 1558887 from [~mikemccand] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1558887 ]

LUCENE-5399: add missingFirst/last support when sorting by Type.STRING; speed 
up deep paging; fix solr's distributed group sort for certain field types

> PagingFieldCollector is very slow with String fields
> 
>
> Key: LUCENE-5399
> URL: https://issues.apache.org/jira/browse/LUCENE-5399
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/search
>Reporter: Robert Muir
> Attachments: LUCENE-5399.patch, LUCENE-5399.patch, LUCENE-5399.patch, 
> LUCENE-5399.patch, LUCENE-5399.patch, LUCENE-5399.patch, LUCENE-5399.patch, 
> LUCENE-5399.patch, LUCENE-5399.patch
>
>
> PagingFieldCollector (sort comparator) is significantly slower with string 
> fields, because of how its "seen on a previous page" works: it calls 
> compareDocToValue(int doc, T t) first to check this. (its the only user of 
> this method)
> This is very slow with String, because no ordinals are used. so each document 
> must lookup ord, then lookup bytes, then compare bytes.
> I think maybe we should replace this method with an 'after' slot, and just 
> have compareDocToAfter or something.
> Otherwise we could use a hack-patch like the one i will upload (i did this 
> just to test the performance, although tests do pass).



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5637) Per-request cache statistics

2014-01-16 Thread Shikhar Bhushan (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-5637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shikhar Bhushan updated SOLR-5637:
--

Attachment: SOLR-5367.patch

first cut of patch attached for feedback.

it needs to be made to work in the distributed case from {{DebugComponent}} and 
i'm still figuring things out for that. pointers appreciated :)

> Per-request cache statistics
> 
>
> Key: SOLR-5637
> URL: https://issues.apache.org/jira/browse/SOLR-5637
> Project: Solr
>  Issue Type: New Feature
>Reporter: Shikhar Bhushan
>Priority: Minor
> Attachments: SOLR-5367.patch
>
>
> We have found it very useful to have information on the number of cache hits 
> and misses for key Solr caches (filterCache, documentCache, etc.) at the 
> request level.
> This is currently implemented in our codebase using custom {{SolrCache}} 
> implementations.
> I am working on moving to maintaining stats in the {{SolrRequestInfo}} 
> thread-local, and adding hooks in get() methods of SolrCache implementations. 
> This will be glued up using the {{DebugComponent}} and can be requested using 
> a "debug.cache" parameter.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-5637) Per-request cache statistics

2014-01-16 Thread Shikhar Bhushan (JIRA)

Shikhar Bhushan created SOLR-5637:
-

 Summary: Per-request cache statistics
 Key: SOLR-5637
 URL: https://issues.apache.org/jira/browse/SOLR-5637
 Project: Solr
  Issue Type: New Feature
Reporter: Shikhar Bhushan
Priority: Minor


We have found it very useful to have information on the number of cache hits 
and misses for key Solr caches (filterCache, documentCache, etc.) at the 
request level.

This is currently implemented in our codebase using custom {{SolrCache}} 
implementations.

I am working on moving to maintaining stats in the {{SolrRequestInfo}} 
thread-local, and adding hooks in get() methods of SolrCache implementations. 
This will be glued up using the {{DebugComponent}} and can be requested using a 
"debug.cache" parameter.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5401) Field.StringTokenStream#end() does not call super.end()

2014-01-16 Thread Michael Busch (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873746#comment-13873746
 ] 

Michael Busch commented on LUCENE-5401:
---

Thanks, guys! I backported to 4.6.1 and just committed. (feels good after a 
looong time :) )

> Field.StringTokenStream#end() does not call super.end()
> ---
>
> Key: LUCENE-5401
> URL: https://issues.apache.org/jira/browse/LUCENE-5401
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/other
>Affects Versions: 4.6
>Reporter: Michael Busch
>Assignee: Michael Busch
>Priority: Minor
> Fix For: 4.6.1
>
> Attachments: lucene-5401.patch
>
>
> Field.StringTokenStream#end() currently does not call super.end(). This 
> prevents resetting the PositionIncrementAttribute to 0 in end(), which can 
> lead to wrong positions in the index under certain conditions.
> I added a test to TestDocument which indexes two Fields with the same name, 
> String values, indexed=true, tokenized=false and 
> IndexOptions.DOCS_AND_FREQS_AND_POSITIONS. Without the fix the test fails. 
> The first token gets the correct position 0, but the second token gets 
> position 2 instead of 1. The reason is that in DocInverterPerField line 176 
> (which is just after the call to end()) we increment the position a second 
> time, because end() didn't reset the increment to 0.
> All tests pass with the fix.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-5401) Field.StringTokenStream#end() does not call super.end()

2014-01-16 Thread Michael Busch (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-5401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Busch resolved LUCENE-5401.
---

Resolution: Fixed

> Field.StringTokenStream#end() does not call super.end()
> ---
>
> Key: LUCENE-5401
> URL: https://issues.apache.org/jira/browse/LUCENE-5401
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/other
>Affects Versions: 4.6
>Reporter: Michael Busch
>Assignee: Michael Busch
>Priority: Minor
> Fix For: 4.6.1
>
> Attachments: lucene-5401.patch
>
>
> Field.StringTokenStream#end() currently does not call super.end(). This 
> prevents resetting the PositionIncrementAttribute to 0 in end(), which can 
> lead to wrong positions in the index under certain conditions.
> I added a test to TestDocument which indexes two Fields with the same name, 
> String values, indexed=true, tokenized=false and 
> IndexOptions.DOCS_AND_FREQS_AND_POSITIONS. Without the fix the test fails. 
> The first token gets the correct position 0, but the second token gets 
> position 2 instead of 1. The reason is that in DocInverterPerField line 176 
> (which is just after the call to end()) we increment the position a second 
> time, because end() didn't reset the increment to 0.
> All tests pass with the fix.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5401) Field.StringTokenStream#end() does not call super.end()

2014-01-16 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873733#comment-13873733
 ] 

ASF subversion and git services commented on LUCENE-5401:
-

Commit 1558876 from [~buschmic] in branch 'dev/trunk'
[ https://svn.apache.org/r1558876 ]

LUCENE-5401: Field.StringTokenStream#end() calls super.end() now.

> Field.StringTokenStream#end() does not call super.end()
> ---
>
> Key: LUCENE-5401
> URL: https://issues.apache.org/jira/browse/LUCENE-5401
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/other
>Affects Versions: 4.6
>Reporter: Michael Busch
>Assignee: Michael Busch
>Priority: Minor
> Fix For: 4.6.1
>
> Attachments: lucene-5401.patch
>
>
> Field.StringTokenStream#end() currently does not call super.end(). This 
> prevents resetting the PositionIncrementAttribute to 0 in end(), which can 
> lead to wrong positions in the index under certain conditions.
> I added a test to TestDocument which indexes two Fields with the same name, 
> String values, indexed=true, tokenized=false and 
> IndexOptions.DOCS_AND_FREQS_AND_POSITIONS. Without the fix the test fails. 
> The first token gets the correct position 0, but the second token gets 
> position 2 instead of 1. The reason is that in DocInverterPerField line 176 
> (which is just after the call to end()) we increment the position a second 
> time, because end() didn't reset the increment to 0.
> All tests pass with the fix.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5401) Field.StringTokenStream#end() does not call super.end()

2014-01-16 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873734#comment-13873734
 ] 

ASF subversion and git services commented on LUCENE-5401:
-

Commit 1558877 from [~buschmic] in branch 'dev/branches/lucene_solr_4_6'
[ https://svn.apache.org/r1558877 ]

LUCENE-5401: Field.StringTokenStream#end() calls super.end() now.

> Field.StringTokenStream#end() does not call super.end()
> ---
>
> Key: LUCENE-5401
> URL: https://issues.apache.org/jira/browse/LUCENE-5401
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/other
>Affects Versions: 4.6
>Reporter: Michael Busch
>Assignee: Michael Busch
>Priority: Minor
> Fix For: 4.6.1
>
> Attachments: lucene-5401.patch
>
>
> Field.StringTokenStream#end() currently does not call super.end(). This 
> prevents resetting the PositionIncrementAttribute to 0 in end(), which can 
> lead to wrong positions in the index under certain conditions.
> I added a test to TestDocument which indexes two Fields with the same name, 
> String values, indexed=true, tokenized=false and 
> IndexOptions.DOCS_AND_FREQS_AND_POSITIONS. Without the fix the test fails. 
> The first token gets the correct position 0, but the second token gets 
> position 2 instead of 1. The reason is that in DocInverterPerField line 176 
> (which is just after the call to end()) we increment the position a second 
> time, because end() didn't reset the increment to 0.
> All tests pass with the fix.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5401) Field.StringTokenStream#end() does not call super.end()

2014-01-16 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873735#comment-13873735
 ] 

ASF subversion and git services commented on LUCENE-5401:
-

Commit 1558878 from [~buschmic] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1558878 ]

LUCENE-5401: Field.StringTokenStream#end() calls super.end() now.

> Field.StringTokenStream#end() does not call super.end()
> ---
>
> Key: LUCENE-5401
> URL: https://issues.apache.org/jira/browse/LUCENE-5401
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/other
>Affects Versions: 4.6
>Reporter: Michael Busch
>Assignee: Michael Busch
>Priority: Minor
> Fix For: 4.6.1
>
> Attachments: lucene-5401.patch
>
>
> Field.StringTokenStream#end() currently does not call super.end(). This 
> prevents resetting the PositionIncrementAttribute to 0 in end(), which can 
> lead to wrong positions in the index under certain conditions.
> I added a test to TestDocument which indexes two Fields with the same name, 
> String values, indexed=true, tokenized=false and 
> IndexOptions.DOCS_AND_FREQS_AND_POSITIONS. Without the fix the test fails. 
> The first token gets the correct position 0, but the second token gets 
> position 2 instead of 1. The reason is that in DocInverterPerField line 176 
> (which is just after the call to end()) we increment the position a second 
> time, because end() didn't reset the increment to 0.
> All tests pass with the fix.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Span Not Queries

2014-01-16 Thread Gopal Agarwal

Thanks Tim. This exactly fits my requirements of recursion, SpanNot and
ComplexParser combination with Boolean Parser.

Since I would end up doing the exact same changes to my QueryParserBase
class, I would be locked with the current version of SOLR for forseeable
future.

Can you comment on when is the possible release if it gets reviewed by next
week?



On Thu, Jan 16, 2014 at 11:06 PM, Allison, Timothy B. wrote:

>  Apologies for the self-promotion…LUCENE-5205 and its Solr cousin
> (SOLR-5410) might help.  I’m hoping to post updates to both by the end of
> next week.  Then, if a committer would be willing to review and add these
> to Lucene/Solr, you should be good to go.
>
>
>
> Take a look at the description for LUCENE-5205and see if that capability
> will meet your needs.  Thank you.
>
>
>
>   Best,
>
>
>
>  Tim
>
>
>
> *From:* Gopal Agarwal [mailto:gopal.agarw...@gmail.com]
> *Sent:* Thursday, January 16, 2014 4:10 AM
> *To:* dev@lucene.apache.org
> *Subject:* Fwd: Span Not Queries
>
>
>
> Please help me out with earlier query.
>
>
>
> In short:
>
> 1. Can we change the QueryParser.jj file to identify the SpanNot query as
> a boolean clause?
>
>
>
> 2. Can we use ComplexPhraseQuery Parser to support SpanOR and SpanNOT
> queries also?
>
>
>
> For further explanation, following are the examples.
>
>
>
> On Tue, Oct 15, 2013 at 11:27 PM, Ankit Kumar 
> wrote:
>
> *I have a business use case in which i need to use Span Not and
> other ordered proximity queries . And they can be nested upto any level
> A Boolean inside a ordered query or ordered query inside a Boolean
>  . Currently i am thinking of changing the QuerParser.jj file to identify
> the SpanNot query and use Complex Phrase Query Parser of Lucene for parsing
> complex queries . Can you suggest better way of achieving this.*
>
> *Following are the list of additions that i need to do in SOLR.*
>
> *1. Span NOT Operator*  .
>
> 2.Adding Recursive and Range Proximity
>
>   *Recursive Proximity *is a proximity query within a proximity query
>
> Ex:   “ “income tax”~5   statement” ~4  The recursion can be up to any
> level.
>
> * Range Proximity*: Currently we can only define number as a range we
> want interval as a range .
>
> Ex: “profit income”~3,5,  “United America”~-5,4
>
>
>
> 3. Complex  Queries
>
> A complex query is a query formed with a combination of Boolean operators
> or proximity queries or range queries or any possible combination of these.
>
> Ex:“(income AND tax) statement”~4
>
>   “ “income tax”~4  (statement OR period) ”~3
>
>   (“ income” SPAN NOT  “income tax” ) source ~3,5
>
>  Can anyone suggest us some way of achieving these 3 functionalities in
> SOLR
>  ???
>
>
> On Tue, Oct 15, 2013 at 10:15 PM, Jack Krupansky  >wrote:
>
>
> > Nope. But the LucidWorks Search product query parser does support SpanNot
> > if you use their BEFORE, AFTER, and NEAR span operators.
> >
> > See:
>
> > http://docs.lucidworks.com/**display/lweug/Proximity+**Operations<
> http://docs.lucidworks.com/display/lweug/Proximity+Operations>
>
> >
> > For example: "George BEFORE:2 Bush NOT H" to match George 
> Bush,
> > but not George H. W. Bush.
> >
> > What is your specific use case?
> >
> > -- Jack Krupansky
> >
> > -Original Message- From: Ankit Kumar
> > Sent: Tuesday, October 15, 2013 3:58 AM
> > To: solr-u...@lucene.apache.org
> > Subject: Span Not Queries
> >
> >
> > I need to add Span Not queries in solr . Ther's a parser Surround Query
> > Parser  i went through this (
>
> > http://lucene.472066.n3.**nabble.com/Surround-query-**
> > parser-not-working-td4075066.**html<
> http://lucene.472066.n3.nabble.com/Surround-query-parser-not-working-td4075066.html
> >
>
> > )
> > to discover that surround query parser does not analyze text
> >
> > Does DisMaxQueryParser supports SpanNot Queries ??
> >
>
>
>
>
>

[JENKINS] Lucene-Solr-4.x-MacOSX (64bit/jdk1.7.0) - Build # 1202 - Still Failing!

2014-01-16 Thread Policeman Jenkins Server

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-MacOSX/1202/
Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

All tests passed

Build Log:
[...truncated 16971 lines...]
   [junit4] JVM J0: stderr was not empty, see: 
/Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/contrib/solr-map-reduce/test/temp/junit4-J0-20140116_175421_138.syserr
   [junit4] >>> JVM J0: stderr (verbatim) 
   [junit4] 2014-01-16 17:55:05.620 java[1559:6703] Unable to load realm info 
from SCDynamicStore
   [junit4] <<< JVM J0: EOF 

[...truncated 13730 lines...]
-check-forbidden-base:
[forbidden-apis] Reading bundled API signatures: jdk-unsafe-1.6
[forbidden-apis] Reading bundled API signatures: jdk-deprecated-1.6
[forbidden-apis] Reading bundled API signatures: commons-io-unsafe-2.1
[forbidden-apis] Reading API signatures: 
/Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/lucene/tools/forbiddenApis/base.txt
[forbidden-apis] Reading API signatures: 
/Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/lucene/tools/forbiddenApis/servlet-api.txt
[forbidden-apis] Loading classes to check...
[forbidden-apis] Scanning for API signatures and dependencies...
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.cloud.OverseerCollectionProcessor 
(OverseerCollectionProcessor.java:355)
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.cloud.OverseerCollectionProcessor 
(OverseerCollectionProcessor.java:355)
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.cloud.OverseerCollectionProcessor 
(OverseerCollectionProcessor.java:391)
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.cloud.OverseerCollectionProcessor 
(OverseerCollectionProcessor.java:394)
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.handler.admin.CollectionsHandler 
(CollectionsHandler.java:201)
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.handler.admin.CollectionsHandler 
(CollectionsHandler.java:205)
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.cloud.OverseerRolesTest 
(OverseerRolesTest.java:160)
[forbidden-apis] Scanned 1910 (and 1314 related) class file(s) for forbidden 
API invocations (in 4.00s), 7 error(s).

BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/build.xml:459: The following 
error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/build.xml:64: The following 
error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build.xml:271: The 
following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/common-build.xml:474: 
Check for forbidden API calls failed, see log.

Total time: 103 minutes 47 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops 
-XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread Mark Miller (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873657#comment-13873657
 ] 

Mark Miller commented on SOLR-4260:
---

I spent some time a while back trying to find a fault in 
ConcurrentSolrServer#blockUntilFinished - didn't uncover anything yet though.

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5399) PagingFieldCollector is very slow with String fields

2014-01-16 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873647#comment-13873647
 ] 

ASF subversion and git services commented on LUCENE-5399:
-

Commit 1558865 from [~mikemccand] in branch 'dev/trunk'
[ https://svn.apache.org/r1558865 ]

LUCENE-5399: add missingFirst/last support when sorting by Type.STRING; speed 
up deep paging; fix solr's distributed group sort for certain field types

> PagingFieldCollector is very slow with String fields
> 
>
> Key: LUCENE-5399
> URL: https://issues.apache.org/jira/browse/LUCENE-5399
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/search
>Reporter: Robert Muir
> Attachments: LUCENE-5399.patch, LUCENE-5399.patch, LUCENE-5399.patch, 
> LUCENE-5399.patch, LUCENE-5399.patch, LUCENE-5399.patch, LUCENE-5399.patch, 
> LUCENE-5399.patch, LUCENE-5399.patch
>
>
> PagingFieldCollector (sort comparator) is significantly slower with string 
> fields, because of how its "seen on a previous page" works: it calls 
> compareDocToValue(int doc, T t) first to check this. (its the only user of 
> this method)
> This is very slow with String, because no ordinals are used. so each document 
> must lookup ord, then lookup bytes, then compare bytes.
> I think maybe we should replace this method with an 'after' slot, and just 
> have compareDocToAfter or something.
> Otherwise we could use a hack-patch like the one i will upload (i did this 
> just to test the performance, although tests do pass).



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: Span Not Queries

2014-01-16 Thread Allison, Timothy B.

Apologies for the self-promotion...LUCENE-5205 and its Solr cousin (SOLR-5410) 
might help.  I'm hoping to post updates to both by the end of next week.  Then, 
if a committer would be willing to review and add these to Lucene/Solr, you 
should be good to go.

Take a look at the description for LUCENE-5205and see if that capability will 
meet your needs.  Thank you.

  Best,

 Tim

From: Gopal Agarwal [mailto:gopal.agarw...@gmail.com]
Sent: Thursday, January 16, 2014 4:10 AM
To: dev@lucene.apache.org
Subject: Fwd: Span Not Queries

Please help me out with earlier query.

In short:
1. Can we change the QueryParser.jj file to identify the SpanNot query as a 
boolean clause?

2. Can we use ComplexPhraseQuery Parser to support SpanOR and SpanNOT queries 
also?

For further explanation, following are the examples.

On Tue, Oct 15, 2013 at 11:27 PM, Ankit Kumar 
mailto:ankitthemight...@gmail.com>> wrote:
*I have a business use case in which i need to use Span Not and
other ordered proximity queries . And they can be nested upto any level
A Boolean inside a ordered query or ordered query inside a Boolean
 . Currently i am thinking of changing the QuerParser.jj file to identify
the SpanNot query and use Complex Phrase Query Parser of Lucene for parsing
complex queries . Can you suggest better way of achieving this.*

*Following are the list of additions that i need to do in SOLR.*

*1. Span NOT Operator*  .

2.Adding Recursive and Range Proximity

  *Recursive Proximity *is a proximity query within a proximity query

Ex:   " "income tax"~5   statement" ~4  The recursion can be up to any
level.

* Range Proximity*: Currently we can only define number as a range we
want interval as a range .

Ex: "profit income"~3,5,  "United America"~-5,4

3. Complex  Queries

A complex query is a query formed with a combination of Boolean operators
or proximity queries or range queries or any possible combination of these.

Ex:"(income AND tax) statement"~4

  " "income tax"~4  (statement OR period) "~3

  (" income" SPAN NOT  "income tax" ) source ~3,5

 Can anyone suggest us some way of achieving these 3 functionalities in SOLR
 ???

On Tue, Oct 15, 2013 at 10:15 PM, Jack Krupansky 
mailto:j...@basetechnology.com>>wrote:

> Nope. But the LucidWorks Search product query parser does support SpanNot
> if you use their BEFORE, AFTER, and NEAR span operators.
>
> See:
> http://docs.lucidworks.com/**display/lweug/Proximity+**Operations
>
> For example: "George BEFORE:2 Bush NOT H" to match George  Bush,
> but not George H. W. Bush.
>
> What is your specific use case?
>
> -- Jack Krupansky
>
> -Original Message- From: Ankit Kumar
> Sent: Tuesday, October 15, 2013 3:58 AM
> To: solr-u...@lucene.apache.org
> Subject: Span Not Queries
>
>
> I need to add Span Not queries in solr . Ther's a parser Surround Query
> Parser  i went through this (
> http://lucene.472066.n3.**nabble.com/Surround-query-**
> parser-not-working-td4075066.**html
> )
> to discover that surround query parser does not analyze text
>
> Does DisMaxQueryParser supports SpanNot Queries ??
>

[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica

2014-01-16 Thread Joel Bernstein (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873638#comment-13873638
 ] 

Joel Bernstein commented on SOLR-4260:
--

The commit behavior is interesting. I'm seeing docs flushing from the leader to 
replica following a manual hard commit issued long after indexing has stopped. 
That means somewhere along the way docs are buffered and waiting for an event 
to flush them to the replica. I haven't figured out just yet where the 
buffering is occurring but I'm trying to track it down. 

> Inconsistent numDocs between leader and replica
> ---
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
> Environment: 5.0.0.2013.01.04.15.31.51
>Reporter: Markus Jelsma
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 5.0, 4.7
>
> Attachments: 192.168.20.102-replica1.png, 
> 192.168.20.104-replica2.png, clusterstate.png, 
> demo_shard1_replicas_out_of_sync.tgz
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using 
> CloudSolrServer we see inconsistencies between the leader and replica for 
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have 
> a small deviation in then number of documents. The leader and slave deviate 
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my 
> attention, there were small IDF differences for exactly the same record 
> causing a record to shift positions in the result set. During those tests no 
> records were indexed. Consecutive catch all queries also return different 
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor 
> of two and frequently reindex using a fresh build from trunk. I've not seen 
> this issue for quite some time until a few days ago.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_60-ea-b02) - Build # 9117 - Still Failing!

2014-01-16 Thread Policeman Jenkins Server

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9117/
Java: 32bit/jdk1.7.0_60-ea-b02 -client -XX:+UseSerialGC

All tests passed

Build Log:
[...truncated 30543 lines...]
-check-forbidden-base:
[forbidden-apis] Reading bundled API signatures: jdk-unsafe-1.7
[forbidden-apis] Reading bundled API signatures: jdk-deprecated-1.7
[forbidden-apis] Reading bundled API signatures: commons-io-unsafe-2.1
[forbidden-apis] Reading API signatures: 
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/tools/forbiddenApis/base.txt
[forbidden-apis] Reading API signatures: 
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/tools/forbiddenApis/servlet-api.txt
[forbidden-apis] Loading classes to check...
[forbidden-apis] Scanning for API signatures and dependencies...
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.cloud.OverseerRolesTest 
(OverseerRolesTest.java:160)
[forbidden-apis] Scanned 2043 (and 1337 related) class file(s) for forbidden 
API invocations (in 1.53s), 1 error(s).

BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:453: The following 
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:64: The following 
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:271: The 
following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:474: 
Check for forbidden API calls failed, see log.

Total time: 57 minutes 43 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.7.0_60-ea-b02 -client -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

lucene-solr pull request: clarified the Sort(Sortfield...) constructor

2014-01-16 Thread s4ke

GitHub user s4ke opened a pull request:

https://github.com/apache/lucene-solr/pull/20

clarified the Sort(Sortfield...) constructor

I found that the documentation in the Constructor and the setSort method 
were misleading, so I clarified them.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/s4ke/lucene-solr trunk

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/lucene-solr/pull/20.patch


commit b0090dd66530595dbd272220cefd96939e2c794d
Author: Martin Braun 
Date:   2014-01-16T16:51:36Z

clarified the Sort(Sortfield...) constructor




-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5477) Async execution of OverseerCollectionProcessor tasks

2014-01-16 Thread Noble Paul (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873583#comment-13873583
 ] 

Noble Paul commented on SOLR-5477:
--

Other than Overseer no other nodes watch queues. A core watching a queue will 
be a huge scalability issue 

> Async execution of OverseerCollectionProcessor tasks
> 
>
> Key: SOLR-5477
> URL: https://issues.apache.org/jira/browse/SOLR-5477
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrCloud
>Reporter: Noble Paul
>Assignee: Anshum Gupta
> Attachments: SOLR-5477-CoreAdminStatus.patch, SOLR-5477.patch
>
>
> Typical collection admin commands are long running and it is very common to 
> have the requests get timed out.  It is more of a problem if the cluster is 
> very large.Add an option to run these commands asynchronously
> add an extra param async=true for all collection commands
> the task is written to ZK and the caller is returned a task id. 
> as separate collection admin command will be added to poll the status of the 
> task
> command=status&id=7657668909
> if id is not passed all running async tasks should be listed
> A separate queue is created to store in-process tasks . After the tasks are 
> completed the queue entry is removed. OverSeerColectionProcessor will perform 
> these tasks in multiple threads



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5476) Overseer Role for nodes

2014-01-16 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873563#comment-13873563
 ] 

ASF subversion and git services commented on SOLR-5476:
---

Commit 1558847 from [~noble.paul] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1558847 ]

SOLR-5476 removing forbidden API usage

> Overseer Role for nodes
> ---
>
> Key: SOLR-5476
> URL: https://issues.apache.org/jira/browse/SOLR-5476
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrCloud
>Reporter: Noble Paul
>Assignee: Noble Paul
> Fix For: 5.0, 4.7
>
> Attachments: SOLR-5476.patch, SOLR-5476.patch, SOLR-5476.patch, 
> SOLR-5476.patch, SOLR-5476.patch
>
>
> In a very large cluster the Overseer is likely to be overloaded.If the same 
> node is a serving a few other shards it can lead to OverSeer getting slowed 
> down due to GC pauses , or simply too much of work  . If the cluster is 
> really large , it is possible to dedicate high end h/w for OverSeers
> It works as a new collection admin command
> command=addrole&role=overseer&node=192.168.1.5:8983_solr
> This results in the creation of a entry in the /roles.json in ZK which would 
> look like the following
> {code:javascript}
> {
> "overseer" : ["192.168.1.5:8983_solr"]
> }
> {code}
> If a node is designated for overseer it gets preference over others when 
> overseer election takes place. If no designated servers are available another 
> random node would become the Overseer.
> Later on, if one of the designated nodes are brought up ,it would take over 
> the Overseer role from the current Overseer to become the Overseer of the 
> system



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5476) Overseer Role for nodes

2014-01-16 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873562#comment-13873562
 ] 

ASF subversion and git services commented on SOLR-5476:
---

Commit 1558846 from [~noble.paul] in branch 'dev/trunk'
[ https://svn.apache.org/r1558846 ]

SOLR-5476 removing forbidden API usage

> Overseer Role for nodes
> ---
>
> Key: SOLR-5476
> URL: https://issues.apache.org/jira/browse/SOLR-5476
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrCloud
>Reporter: Noble Paul
>Assignee: Noble Paul
> Fix For: 5.0, 4.7
>
> Attachments: SOLR-5476.patch, SOLR-5476.patch, SOLR-5476.patch, 
> SOLR-5476.patch, SOLR-5476.patch
>
>
> In a very large cluster the Overseer is likely to be overloaded.If the same 
> node is a serving a few other shards it can lead to OverSeer getting slowed 
> down due to GC pauses , or simply too much of work  . If the cluster is 
> really large , it is possible to dedicate high end h/w for OverSeers
> It works as a new collection admin command
> command=addrole&role=overseer&node=192.168.1.5:8983_solr
> This results in the creation of a entry in the /roles.json in ZK which would 
> look like the following
> {code:javascript}
> {
> "overseer" : ["192.168.1.5:8983_solr"]
> }
> {code}
> If a node is designated for overseer it gets preference over others when 
> overseer election takes place. If no designated servers are available another 
> random node would become the Overseer.
> Later on, if one of the designated nodes are brought up ,it would take over 
> the Overseer role from the current Overseer to become the Overseer of the 
> system



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-5477) Async execution of OverseerCollectionProcessor tasks

2014-01-16 Thread Shalin Shekhar Mangar (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873554#comment-13873554
 ] 

Shalin Shekhar Mangar edited comment on SOLR-5477 at 1/16/14 4:21 PM:
--

{quote}
Questions:
This move would mean having a lot more clients talk to and write to zk. Does 
this approach make sense as far as the intended direction of SolrCloud is 
concerned?
Any suggestions/concerns about scalability of zk as far as having multiple 
updates coming into zk is concerned.
{quote}

# I don't think we need to use a ZooKeeper queue for communication. The only 
reason where that is required is when we need fail over. That is desired for 
the overseer actions but not required for core admin actions. For example, I 
don't think you should be able to submit a core admin action when the target 
node is not up.
# This approach only works with a cloud aware client such as SolrJ i.e. you 
can't submit an async core admin action with HTTP. This in itself is a strong 
reason to warrant a -1 from me.
# This a very intrusive change compared to other approaches. I think the 
benefits you have outlined can be achieved in simpler ways (see my comments on 
approach #2 below)

{quote}
Approach #2:
Continue accepting the request like right now, but just :
Get the call to return immediately
Use zk to only track/store the status (persistence). The request status calls 
still comes to the core and the status is fetched from zk by the core instead 
of the client being intelligent and talking directly to zk.
{quote}
This is much more acceptable to me. Clients should not have to worry about ZK 
to submit a request.

{quote}
This approach is certainly less intrusive but then also doesn't come with the 
benefit of having the client just watch over a particular zk node for task 
state change etc.
{quote}
In my mind, the proposal to have the client watch nodes for task state change 
is orthogonal to the way the task is invoked. There is no reason why the 
request can't be made via HTTP _and_ the response from core container can't 
contain the request id (is this the complete zookeeper node name?) which can be 
used by a client to setup a watch directly if it so desires. In any case a HTTP 
based status/polling API must exist.


was (Author: shalinmangar):
{quote}
Questions:
This move would mean having a lot more clients talk to and write to zk. Does 
this approach make sense as far as the intended direction of SolrCloud is 
concerned?
Any suggestions/concerns about scalability of zk as far as having multiple 
updates coming into zk is concerned.
{quote}

# I don't think we need to use a ZooKeeper queue for communication. The only 
reason where that is required is when we need fail over. That is desired for 
the overseer actions but not required for core admin actions. For example, I 
don't think you should be able to submit a core admin action when the target 
node is not up.
# This approach only works with a cloud aware client such as SolrJ i.e. you 
can't submit an async core admin action with HTTP. This in itself is a strong 
reason to warrant a -1 from me.
# This a very intrusive change compared to other approaches. I think the 
benefits you have outlined can be achieved in simpler ways (see my comments on 
approach #2 below)

{quote}
Approach #2:
Continue accepting the request like right now, but just :
Get the call to return immediately
Use zk to only track/store the status (persistence). The request status calls 
still comes to the core and the status is fetched from zk by the core instead 
of the client being intelligent and talking directly to zk.
{quote}
This is much more acceptable to me. Clients should not have to worry about ZK 
to submit a request.

{quote}
This approach is certainly less intrusive but then also doesn't come with the 
benefit of having the client just watch over a particular zk node for task 
state change etc.
{quote}
In my mind, the proposal to have the client watch nodes for task state change 
is orthogonal to the way the task is invoked. There is no reason why the 
request can be made via HTTP _and_ the response from core container can't 
contain the request id (is this the complete zookeeper node name?) which can be 
used by a client to setup a watch directly if it so desires. In any case a HTTP 
based status/polling API must exist.

> Async execution of OverseerCollectionProcessor tasks
> 
>
> Key: SOLR-5477
> URL: https://issues.apache.org/jira/browse/SOLR-5477
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrCloud
>Reporter: Noble Paul
>Assignee: Anshum Gupta
> Attachments: SOLR-5477-CoreAdminStatus.patch, SOLR-5477.patch
>
>
> Typical collection admin commands are long running and it is very common to 
> have the r

[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.7.0) - Build # 1230 - Still Failing!

2014-01-16 Thread Policeman Jenkins Server

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/1230/
Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseParallelGC

All tests passed

Build Log:
[...truncated 16772 lines...]
   [junit4] JVM J0: stderr was not empty, see: 
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/contrib/solr-map-reduce/test/temp/junit4-J0-20140116_160853_638.syserr
   [junit4] >>> JVM J0: stderr (verbatim) 
   [junit4] 2014-01-16 16:09:34.114 java[681:4b23] Unable to load realm info 
from SCDynamicStore
   [junit4] <<< JVM J0: EOF 

[...truncated 13655 lines...]
-check-forbidden-base:
[forbidden-apis] Reading bundled API signatures: jdk-unsafe-1.7
[forbidden-apis] Reading bundled API signatures: jdk-deprecated-1.7
[forbidden-apis] Reading bundled API signatures: commons-io-unsafe-2.1
[forbidden-apis] Reading API signatures: 
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/tools/forbiddenApis/base.txt
[forbidden-apis] Reading API signatures: 
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/tools/forbiddenApis/servlet-api.txt
[forbidden-apis] Loading classes to check...
[forbidden-apis] Scanning for API signatures and dependencies...
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.cloud.OverseerCollectionProcessor 
(OverseerCollectionProcessor.java:355)
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.cloud.OverseerCollectionProcessor 
(OverseerCollectionProcessor.java:355)
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.cloud.OverseerCollectionProcessor 
(OverseerCollectionProcessor.java:391)
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.cloud.OverseerCollectionProcessor 
(OverseerCollectionProcessor.java:394)
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.handler.admin.CollectionsHandler 
(CollectionsHandler.java:201)
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.handler.admin.CollectionsHandler 
(CollectionsHandler.java:205)
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.cloud.OverseerRolesTest 
(OverseerRolesTest.java:160)
[forbidden-apis] Scanned 2043 (and 1337 related) class file(s) for forbidden 
API invocations (in 4.40s), 7 error(s).

BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:453: The following 
error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:64: The following 
error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:271: The 
following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:474: 
Check for forbidden API calls failed, see log.

Total time: 99 minutes 15 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5477) Async execution of OverseerCollectionProcessor tasks

2014-01-16 Thread Shalin Shekhar Mangar (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873554#comment-13873554
 ] 

Shalin Shekhar Mangar commented on SOLR-5477:
-

{quote}
Questions:
This move would mean having a lot more clients talk to and write to zk. Does 
this approach make sense as far as the intended direction of SolrCloud is 
concerned?
Any suggestions/concerns about scalability of zk as far as having multiple 
updates coming into zk is concerned.
{quote}

# I don't think we need to use a ZooKeeper queue for communication. The only 
reason where that is required is when we need fail over. That is desired for 
the overseer actions but not required for core admin actions. For example, I 
don't think you should be able to submit a core admin action when the target 
node is not up.
# This approach only works with a cloud aware client such as SolrJ i.e. you 
can't submit an async core admin action with HTTP. This in itself is a strong 
reason to warrant a -1 from me.
# This a very intrusive change compared to other approaches. I think the 
benefits you have outlined can be achieved in simpler ways (see my comments on 
approach #2 below)

{quote}
Approach #2:
Continue accepting the request like right now, but just :
Get the call to return immediately
Use zk to only track/store the status (persistence). The request status calls 
still comes to the core and the status is fetched from zk by the core instead 
of the client being intelligent and talking directly to zk.
{quote}
This is much more acceptable to me. Clients should not have to worry about ZK 
to submit a request.

{quote}
This approach is certainly less intrusive but then also doesn't come with the 
benefit of having the client just watch over a particular zk node for task 
state change etc.
{quote}
In my mind, the proposal to have the client watch nodes for task state change 
is orthogonal to the way the task is invoked. There is no reason why the 
request can be made via HTTP _and_ the response from core container can't 
contain the request id (is this the complete zookeeper node name?) which can be 
used by a client to setup a watch directly if it so desires. In any case a HTTP 
based status/polling API must exist.

> Async execution of OverseerCollectionProcessor tasks
> 
>
> Key: SOLR-5477
> URL: https://issues.apache.org/jira/browse/SOLR-5477
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrCloud
>Reporter: Noble Paul
>Assignee: Anshum Gupta
> Attachments: SOLR-5477-CoreAdminStatus.patch, SOLR-5477.patch
>
>
> Typical collection admin commands are long running and it is very common to 
> have the requests get timed out.  It is more of a problem if the cluster is 
> very large.Add an option to run these commands asynchronously
> add an extra param async=true for all collection commands
> the task is written to ZK and the caller is returned a task id. 
> as separate collection admin command will be added to poll the status of the 
> task
> command=status&id=7657668909
> if id is not passed all running async tasks should be listed
> A separate queue is created to store in-process tasks . After the tasks are 
> completed the queue entry is removed. OverSeerColectionProcessor will perform 
> these tasks in multiple threads



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5401) Field.StringTokenStream#end() does not call super.end()

2014-01-16 Thread Mark Miller (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873539#comment-13873539
 ] 

Mark Miller commented on LUCENE-5401:
-

+1, let's pop this in for 4.6.1.

> Field.StringTokenStream#end() does not call super.end()
> ---
>
> Key: LUCENE-5401
> URL: https://issues.apache.org/jira/browse/LUCENE-5401
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/other
>Affects Versions: 4.6
>Reporter: Michael Busch
>Assignee: Michael Busch
>Priority: Minor
> Fix For: 4.6.1
>
> Attachments: lucene-5401.patch
>
>
> Field.StringTokenStream#end() currently does not call super.end(). This 
> prevents resetting the PositionIncrementAttribute to 0 in end(), which can 
> lead to wrong positions in the index under certain conditions.
> I added a test to TestDocument which indexes two Fields with the same name, 
> String values, indexed=true, tokenized=false and 
> IndexOptions.DOCS_AND_FREQS_AND_POSITIONS. Without the fix the test fails. 
> The first token gets the correct position 0, but the second token gets 
> position 2 instead of 1. The reason is that in DocInverterPerField line 176 
> (which is just after the call to end()) we increment the position a second 
> time, because end() didn't reset the increment to 0.
> All tests pass with the fix.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0-ea-b123) - Build # 9116 - Still Failing!

2014-01-16 Thread Policeman Jenkins Server

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9116/
Java: 64bit/jdk1.8.0-ea-b123 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

All tests passed

Build Log:
[...truncated 30570 lines...]
-check-forbidden-base:
[forbidden-apis] Reading bundled API signatures: jdk-unsafe-1.7
[forbidden-apis] Reading bundled API signatures: jdk-deprecated-1.7
[forbidden-apis] Reading bundled API signatures: commons-io-unsafe-2.1
[forbidden-apis] Reading API signatures: 
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/tools/forbiddenApis/base.txt
[forbidden-apis] Reading API signatures: 
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/tools/forbiddenApis/servlet-api.txt
[forbidden-apis] Loading classes to check...
[forbidden-apis] Scanning for API signatures and dependencies...
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.handler.admin.CollectionsHandler 
(CollectionsHandler.java:201)
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.handler.admin.CollectionsHandler 
(CollectionsHandler.java:205)
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.cloud.OverseerRolesTest 
(OverseerRolesTest.java:161)
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.cloud.OverseerCollectionProcessor 
(OverseerCollectionProcessor.java:355)
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.cloud.OverseerCollectionProcessor 
(OverseerCollectionProcessor.java:355)
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.cloud.OverseerCollectionProcessor 
(OverseerCollectionProcessor.java:391)
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() 
[Uses default locale]
[forbidden-apis]   in org.apache.solr.cloud.OverseerCollectionProcessor 
(OverseerCollectionProcessor.java:394)
[forbidden-apis] Scanned 2043 (and 1339 related) class file(s) for forbidden 
API invocations (in 0.98s), 7 error(s).

BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:453: The following 
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:64: The following 
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:271: The 
following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:474: 
Check for forbidden API calls failed, see log.

Total time: 52 minutes 36 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.8.0-ea-b123 -XX:+UseCompressedOops 
-XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

1 2 >

1 - 100 of 124 matches

Mail list logo