[JENKINS] Lucene-Solr-6.1-Windows (32bit/jdk1.8.0_92) - Build # 4 - Still Failing!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.1-Windows/4/
Java: 32bit/jdk1.8.0_92 -client -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.core.TestDynamicLoading.testDynamicLoading

Error Message:
Could not get expected value  'X val changed' for path 'x' full output: {   
"responseHeader":{ "status":0, "QTime":0},   "params":{"wt":"json"},   
"context":{ "webapp":"/t_nds/jx", "path":"/test1", 
"httpMethod":"GET"},   
"class":"org.apache.solr.core.BlobStoreTestRequestHandler",   "x":null},  from 
server:  null

Stack Trace:
java.lang.AssertionError: Could not get expected value  'X val changed' for 
path 'x' full output: {
  "responseHeader":{
"status":0,
"QTime":0},
  "params":{"wt":"json"},
  "context":{
"webapp":"/t_nds/jx",
"path":"/test1",
"httpMethod":"GET"},
  "class":"org.apache.solr.core.BlobStoreTestRequestHandler",
  "x":null},  from server:  null
at 
__randomizedtesting.SeedInfo.seed([4DE6E0798DF9C798:95ABCD2E7A246238]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.core.TestSolrConfigHandler.testForResponseElement(TestSolrConfigHandler.java:481)
at 
org.apache.solr.core.TestDynamicLoading.testDynamicLoading(TestDynamicLoading.java:250)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.j

[JENKINS] Lucene-Solr-master-Windows (32bit/jdk1.8.0_92) - Build # 5903 - Still Failing!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/5903/
Java: 32bit/jdk1.8.0_92 -server -XX:+UseParallelGC

1 tests failed.
FAILED:  
org.apache.solr.cloud.TestTolerantUpdateProcessorCloud.testVariousAddsViaShard1LeaderClient

Error Message:
GC overhead limit exceeded

Stack Trace:
java.lang.OutOfMemoryError: GC overhead limit exceeded
at 
__randomizedtesting.SeedInfo.seed([9CDED135A6D57336:2FABE816ABCBFF3D]:0)
at java.util.Arrays.copyOfRange(Arrays.java:3664)
at java.lang.String.(String.java:207)
at java.lang.String.substring(String.java:1969)
at java.lang.String.subSequence(String.java:2003)
at java.util.regex.Pattern.split(Pattern.java:1233)
at java.util.regex.Pattern.split(Pattern.java:1273)
at 
sun.security.util.AlgorithmDecomposer.decompose(AlgorithmDecomposer.java:56)
at 
sun.security.ssl.SSLAlgorithmDecomposer.decomposes(SSLAlgorithmDecomposer.java:149)
at 
sun.security.ssl.SSLAlgorithmDecomposer.decompose(SSLAlgorithmDecomposer.java:222)
at 
sun.security.ssl.SSLAlgorithmDecomposer.decompose(SSLAlgorithmDecomposer.java:243)
at 
sun.security.util.AbstractAlgorithmConstraints.checkAlgorithm(AbstractAlgorithmConstraints.java:104)
at 
sun.security.util.DisabledAlgorithmConstraints.permits(DisabledAlgorithmConstraints.java:91)
at 
sun.security.ssl.SSLAlgorithmConstraints.permits(SSLAlgorithmConstraints.java:144)
at 
sun.security.ssl.Handshaker.getActiveCipherSuites(Handshaker.java:642)
at sun.security.ssl.Handshaker.activate(Handshaker.java:509)
at 
sun.security.ssl.SSLSocketImpl.kickstartHandshake(SSLSocketImpl.java:1482)
at 
sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1351)
at 
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403)
at 
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387)
at 
org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:394)
at 
org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:353)
at 
org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:134)
at 
org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
at 
org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
at 
org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
at 
org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
at 
org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at 
org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:511)




Build Log:
[...truncated 12085 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestTolerantUpdateProcessorCloud
   [junit4]   2> Creating dataDir: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestTolerantUpdateProcessorCloud_9CDED135A6D57336-001\init-core-data-001
   [junit4]   2> 2848259 INFO  
(SUITE-TestTolerantUpdateProcessorCloud-seed#[9CDED135A6D57336]-worker) [] 
o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: 
@org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 2848263 INFO  
(SUITE-TestTolerantUpdateProcessorCloud-seed#[9CDED135A6D57336]-worker) [] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2848263 INFO  (Thread-6051) [] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2848263 INFO  (Thread-6051) [] o.a.s.c.ZkTestServer 
Starting server
   [junit4]   2> 2848368 INFO  
(SUITE-TestTolerantUpdateProcessorCloud-seed#[9CDED135A6D57336]-worker) [] 
o.a.s.c.ZkTestServer start zk server on port:50115
   [junit4]   2> 2848369 INFO  
(SUITE-TestTolerantUpdateProcessorCloud-seed#[9CDED135A6D57336]-worker) [] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2848369 INFO  
(SUITE-TestTolerantUpdateProcessorCloud-seed#[9CDED135A6D57336]-worker) [] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2848377 INFO  (zkCallback-2684-thread-1) [] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@97fd07 name:ZooKeeperConnection 
Watcher:127.0.0.1:50115 got event WatchedEvent state:SyncConnected type:N

[jira] [Commented] (SOLR-9185) Solr's "Lucene"/standard query parser should not split on whitespace before sending terms to analysis

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15325683#comment-15325683
 ] 

Yonik Seeley commented on SOLR-9185:


bq. The current behavior might be counter to a new user's expectations, but if 
we just change how it works by default, a lot of existing users might suddenly 
be surprised by very different behavior from Solr 

+1, it's been this way since the beginning of Lucene.  It's not a bug (esp 
since in the past multiple terms were treated as a phrase query, and still are 
depending on configuration - think word delimiter filter).

bq. I suppose we could add luceneMatchVersion-sensitive code 

That's a very blunt hammer ;-)
It would be nice if we could selectively keep the old behavior, but I'm not 
sure how difficult that would be w/o duplicating the whole parser.

> Solr's "Lucene"/standard query parser should not split on whitespace before 
> sending terms to analysis
> -
>
> Key: SOLR-9185
> URL: https://issues.apache.org/jira/browse/SOLR-9185
> Project: Solr
>  Issue Type: Bug
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>
> Copied from LUCENE-2605:
> The queryparser parses input on whitespace, and sends each whitespace 
> separated term to its own independent token stream.
> This breaks the following at query-time, because they can't see across 
> whitespace boundaries:
> n-gram analysis
> shingles
> synonyms (especially multi-word for whitespace-separated languages)
> languages where a 'word' can contain whitespace (e.g. vietnamese)
> Its also rather unexpected, as users think their 
> charfilters/tokenizers/tokenfilters will do the same thing at index and 
> querytime, but
> in many cases they can't. Instead, preferably the queryparser would parse 
> around only real 'operators'.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9205) Parse schema in LukeResponse

[Fengtan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Fengtan updated SOLR-9205:
--
Attachment: SOLR-9205.patch

Suggested patch.

> Parse schema in LukeResponse
> 
>
> Key: SOLR-9205
> URL: https://issues.apache.org/jira/browse/SOLR-9205
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrJ
>Reporter: Fengtan
>Priority: Minor
> Attachments: SOLR-9205.patch
>
>
> LukeRequestHandler (/admin/luke) lists schema flags using two fields named 
> "schema" and "flags".
> For instance on my local machine 
> http://localhost:8983/solr/collection1/admin/luke returns something like this:
> {code:xml}
> 
>   string
>   I-S-OF-l
> 
> {code}
> And http://localhost:8983/solr/collection1/admin/luke?show=schema returns 
> something like this:
> {code:xml}
> 
>   string
>   I-S-OF-l
> 
> {code}
> However, when processing a LukeRequest in SolrJ, only the "flags" field is 
> parsed into a Set of FieldFlag objects. The "schema" field is left as a 
> String, and as a result is hard to process by client applications who do not 
> know how to parse "I-S-OF-l".
> Here is an example that illustrates the problem:
> {code}
> public class MyClass {
>   public static void main(String[] args) throws Exception {
> SolrClient client = new 
> HttpSolrClient("http://localhost:8983/solr/collection1";);
> LukeRequest request = new LukeRequest();
> LukeResponse response = request.process(client);
> for (LukeResponse.FieldInfo field:response.getFieldInfo().values()) {
>   System.out.println(field.getSchema());
>   // field.getSchema() returns "I-S-OF--" (i.e. a String) which 
> is not much meaningful for SolrJ applications
>   // Ideally field.getSchema() should return something like "[INDEXED, 
> STORED, OMIT_NORMS, OMIT_TF]" (i.e. a EnumSet) which is meaningful 
> for SolrJ applications
> }
>   }
> }
> {code}
> It is probably fine to parse both fields the same way in SolrJ since 
> LukeRequestHandler populates them [the same 
> way|https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/handler/admin/LukeRequestHandler.java#L288].



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9205) Parse schema in LukeResponse

[Fengtan (JIRA)]

Fengtan created SOLR-9205:
-

 Summary: Parse schema in LukeResponse
 Key: SOLR-9205
 URL: https://issues.apache.org/jira/browse/SOLR-9205
 Project: Solr
  Issue Type: Improvement
  Components: SolrJ
Reporter: Fengtan
Priority: Minor


LukeRequestHandler (/admin/luke) lists schema flags using two fields named 
"schema" and "flags".

For instance on my local machine 
http://localhost:8983/solr/collection1/admin/luke returns something like this:
{code:xml}

  string
  I-S-OF-l

{code}
And http://localhost:8983/solr/collection1/admin/luke?show=schema returns 
something like this:
{code:xml}

  string
  I-S-OF-l

{code}

However, when processing a LukeRequest in SolrJ, only the "flags" field is 
parsed into a Set of FieldFlag objects. The "schema" field is left as a String, 
and as a result is hard to process by client applications who do not know how 
to parse "I-S-OF-l".

Here is an example that illustrates the problem:
{code}
public class MyClass {
  public static void main(String[] args) throws Exception {
SolrClient client = new 
HttpSolrClient("http://localhost:8983/solr/collection1";);
LukeRequest request = new LukeRequest();
LukeResponse response = request.process(client);
for (LukeResponse.FieldInfo field:response.getFieldInfo().values()) {
  System.out.println(field.getSchema());
  // field.getSchema() returns "I-S-OF--" (i.e. a String) which is 
not much meaningful for SolrJ applications
  // Ideally field.getSchema() should return something like "[INDEXED, 
STORED, OMIT_NORMS, OMIT_TF]" (i.e. a EnumSet) which is meaningful 
for SolrJ applications
}
  }
}
{code}

It is probably fine to parse both fields the same way in SolrJ since 
LukeRequestHandler populates them [the same 
way|https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/handler/admin/LukeRequestHandler.java#L288].



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8812) ExtendedDismaxQParser (edismax) ignores Boolean OR when q.op=AND and mm is not explicitly set

[Greg Pendlebury (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15325638#comment-15325638
 ] 

Greg Pendlebury commented on SOLR-8812:
---

Sounds great. Add my thanks to the those you've already received.

> ExtendedDismaxQParser (edismax) ignores Boolean OR when q.op=AND and mm is 
> not explicitly set
> -
>
> Key: SOLR-8812
> URL: https://issues.apache.org/jira/browse/SOLR-8812
> Project: Solr
>  Issue Type: Bug
>  Components: query parsers
>Affects Versions: 5.5
>Reporter: Ryan Steinberg
>Assignee: Steve Rowe
> Fix For: 5.6, 6.1, 5.5.2, master (7.0), 6.0.2
>
> Attachments: SOLR-8812-barbie.patch, SOLR-8812.patch, 
> SOLR-8812.patch, SOLR-8812.patch
>
>
> The edismax parser ignores Boolean OR in queries when q.op=AND. This behavior 
> is new to Solr 5.5.0 and an unexpected major change.
> Example:
>   "q": "id:12345 OR zz",
>   "defType": "edismax",
>   "q.op": "AND",
> where "12345" is a known document ID and "zz" is a string NOT present 
> in my data
> Version 5.5.0 produces zero results:
> "rawquerystring": "id:12345 OR zz",
> "querystring": "id:12345 OR zz",
> "parsedquery": "(+((id:12345 
> DisjunctionMaxQuery((text:zz)))~2))/no_coord",
> "parsedquery_toString": "+((id:12345 (text:zz))~2)",
> "explain": {},
> "QParser": "ExtendedDismaxQParser"
> Version 5.4.0 produces one result as expected
>   "rawquerystring": "id:12345 OR zz",
> "querystring": "id:12345 OR zz",
> "parsedquery": "(+(id:12345 
> DisjunctionMaxQuery((text:zz/no_coord",
> "parsedquery_toString": "+(id:12345 (text:zz))"
> "explain": {},
> "QParser": "ExtendedDismaxQParser"



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[damien kamerman (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15325626#comment-15325626
 ] 

damien kamerman commented on SOLR-7191:
---

This fits with what I've seen on solr 4/5. The cores register on an
unlimited thread pool. The patch I did was to limit the thread pool and
register in order.



> Improve stability and startup performance of SolrCloud with thousands of 
> collections
> 
>
> Key: SOLR-7191
> URL: https://issues.apache.org/jira/browse/SOLR-7191
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Affects Versions: 5.0
>Reporter: Shawn Heisey
>Assignee: Shalin Shekhar Mangar
>  Labels: performance, scalability
> Attachments: SOLR-7191.patch, SOLR-7191.patch, SOLR-7191.patch, 
> SOLR-7191.patch, lots-of-zkstatereader-updates-branch_5x.log
>
>
> A user on the mailing list with thousands of collections (5000 on 4.10.3, 
> 4000 on 5.0) is having severe problems with getting Solr to restart.
> I tried as hard as I could to duplicate the user setup, but I ran into many 
> problems myself even before I was able to get 4000 collections created on a 
> 5.0 example cloud setup.  Restarting Solr takes a very long time, and it is 
> not very stable once it's up and running.
> This kind of setup is very much pushing the envelope on SolrCloud performance 
> and scalability.  It doesn't help that I'm running both Solr nodes on one 
> machine (I started with 'bin/solr -e cloud') and that ZK is embedded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15325575#comment-15325575
 ] 

Erick Erickson commented on SOLR-7191:
--

I had to chase after this for a while, so I'm recording results 
of some testing for posterity.

> Setup: 4 Solr JVMs, 8G each (64G total RAM on the machine).
> Create 100 4x4 collections (i.e. 4 replicas, 4 shards each). 1,600 total 
> shards
  > Note that the cluster is fine at this point, everything's green.
> No data indexed at all.
> Shut all Solr instances down.
> Bring up a Solr on a different box. I did this to eliminate the chance
  that the Overseer was somehow involved since it is now on the machine
  with no replicas. I don't think this matters much though.
> Bring up one JVM.
> Wait for all the nodes on that JVM to come up. Now every shard has a leader,
  and the collections are all green, 3 of 4 replicas for each shard are
  "gone" of course, but it's a functioning cluster.
> Bring up the next JVM: Kabloooey. Very shortly you'll start to see OOM
  errors on the _second_ JVM but not the first.
  > The numbers of threads on the first JVM are about 1,200. On the second,
they go over 2,000. Whether this would drop back down or not
is an open question.
  > So I tried playing with -Xss to drop the size of the stack on the threads
and even dropping by half didn't help.
  > Expanding the memory on the second JVM to 32G didn't help
  > I tried increasing the processes to no avail (ulimit -u) on a hint
that there was a wonky effect there somehow.
  > Especially disconcerting is the fact that this node was running fine
when the collections were _created_, it just can't get past restart.
  > Changing coreLoadThreads even down to 2 did not seem to help.
  > At no point does the reported memory consumption via jConsole or top
show even getting close to the allocated JVM limits.
> I'd like to be able to just start all 4 JVMs at once, but didn't get
  that far.
> If one tries to start additional JVMs anyway, there's a lot of thrashing
  around, replicas go into recovery, go out of recovery, are permanently down 
etc.
  Of course with OOMs it's unclear what _should_ happen.
> The OOM killer script apparently does NOT get triggered, I think the OOM
  is swallowed, perhaps in Zookeeper client code. Note that if the OOM
  killer script _did_ get fired there'd the second & greater JVMs would
  ust die.
> Error is OOM: Unable to create new native thread.
> Here's a stack trace, there are a _lot_ of these...

ERROR - 2016-06-11 00:05:36.806; [   ] 
org.apache.zookeeper.ClientCnxn$EventThread; Error while calling watcher 
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:714)
at 
java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950)
at 
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:214)
at 
java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112)
at 
org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:266)
at 
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)


> Improve stability and startup performance of SolrCloud with thousands of 
> collections
> 
>
> Key: SOLR-7191
> URL: https://issues.apache.org/jira/browse/SOLR-7191
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Affects Versions: 5.0
>Reporter: Shawn Heisey
>Assignee: Shalin Shekhar Mangar
>  Labels: performance, scalability
> Attachments: SOLR-7191.patch, SOLR-7191.patch, SOLR-7191.patch, 
> SOLR-7191.patch, lots-of-zkstatereader-updates-branch_5x.log
>
>
> A user on the mailing list with thousands of collections (5000 on 4.10.3, 
> 4000 on 5.0) is having severe problems with getting Solr to restart.
> I tried as hard as I could to duplicate the user setup, but I ran into many 
> problems myself even before I was able to get 4000 collections created on a 
> 5.0 example cloud setup.  Restarting Solr takes a very long time, and it is 
> not very stable once it's up and running.
> This kind of setup is very much pushing the envelope on SolrCloud performance 
> and scalability.  It doesn't help that I'm running both Solr nodes on one 
> machine (I started with 'bin/solr -e cloud') and that ZK is embedded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (LUCENE-7333) BaseDirectoryTestCase may create temporary files with names not accepted by Windows (e.g. com1, con,...)

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15325495#comment-15325495
 ] 

Michael McCandless commented on LUCENE-7333:


+1, thanks [~thetaphi]!

> BaseDirectoryTestCase may create temporary files with names not accepted by 
> Windows (e.g. com1, con,...)
> 
>
> Key: LUCENE-7333
> URL: https://issues.apache.org/jira/browse/LUCENE-7333
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/test-framework
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
> Attachments: LUCENE-7333.patch, LUCENE-7333.patch
>
>
> BaseDirectoryTestCase may randomly create files with "special names", which 
> are not allowed by certain operating systems, e.g. Windows.
> See [https://msdn.microsoft.com/en-us/library/aa365247.aspx] for more info.
> This is the issue we have seen:
> {noformat}
> java.security.AccessControlException: access denied ("java.io.FilePermission" 
> "C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\lucene\build\core\test\J0\temp\lucene.store.TestMmapDirectory_96FC6F2D45B76809-001\tempDir-007\con"
>  "write")
>   at 
> __randomizedtesting.SeedInfo.seed([96FC6F2D45B76809:DE018553DD164AB]:0)
>   at 
> java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
>   at 
> java.security.AccessController.checkPermission(AccessController.java:884)
>   at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
>   at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
>   at sun.nio.fs.WindowsChannelFactory.open(WindowsChannelFactory.java:295)
>   at 
> sun.nio.fs.WindowsChannelFactory.newFileChannel(WindowsChannelFactory.java:162)
>   at 
> sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:225)
>   at 
> java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434)
>   at 
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
>   at 
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
>   at 
> org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
>   at 
> org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
>   at 
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
>   at 
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
>   at java.nio.file.Files.newOutputStream(Files.java:216)
>   at 
> org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:408)
>   at 
> org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:404)
>   at 
> org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:253)
>   at 
> org.apache.lucene.store.BaseDirectoryTestCase.testPendingDeletions(BaseDirectoryTestCase.java:1215)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
>   at 
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
>   at 
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>   at 
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>   at 
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>   at 
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>   at 
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>   at 
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
>   at 
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
>   at 
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.jav

[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 89 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/89/

2 tests failed.
FAILED:  org.apache.solr.schema.TestManagedSchemaAPI.test

Error Message:
Error from server at http://127.0.0.1:49683/solr/testschemaapi_shard1_replica2: 
ERROR: [doc=2] unknown field 'myNewField1'

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from 
server at http://127.0.0.1:49683/solr/testschemaapi_shard1_replica2: ERROR: 
[doc=2] unknown field 'myNewField1'
at 
__randomizedtesting.SeedInfo.seed([666A32AC31F0DA8A:EE3E0D769F0CB772]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:697)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1109)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:998)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:934)
at 
org.apache.solr.schema.TestManagedSchemaAPI.testAddFieldAndDocument(TestManagedSchemaAPI.java:86)
at 
org.apache.solr.schema.TestManagedSchemaAPI.test(TestManagedSchemaAPI.java:55)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)

[jira] [Commented] (SOLR-9185) Solr's "Lucene"/standard query parser should not split on whitespace before sending terms to analysis

[Mary Jo Sminkey (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15325445#comment-15325445
 ] 

Mary Jo Sminkey commented on SOLR-9185:
---

Agree that in a case like this, an incremental approach is often best, where 
the new behavior is available via a setting, etc. while warning people that 
existing method is decremented and will be replaced in the next (or a future) 
major update. 

> Solr's "Lucene"/standard query parser should not split on whitespace before 
> sending terms to analysis
> -
>
> Key: SOLR-9185
> URL: https://issues.apache.org/jira/browse/SOLR-9185
> Project: Solr
>  Issue Type: Bug
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>
> Copied from LUCENE-2605:
> The queryparser parses input on whitespace, and sends each whitespace 
> separated term to its own independent token stream.
> This breaks the following at query-time, because they can't see across 
> whitespace boundaries:
> n-gram analysis
> shingles
> synonyms (especially multi-word for whitespace-separated languages)
> languages where a 'word' can contain whitespace (e.g. vietnamese)
> Its also rather unexpected, as users think their 
> charfilters/tokenizers/tokenfilters will do the same thing at index and 
> querytime, but
> in many cases they can't. Instead, preferably the queryparser would parse 
> around only real 'operators'.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9185) Solr's "Lucene"/standard query parser should not split on whitespace before sending terms to analysis

[Shawn Heisey (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15325410#comment-15325410
 ] 

Shawn Heisey commented on SOLR-9185:


The current behavior might be counter to a new user's expectations, but if we 
just change how it works by default, a lot of existing users might suddenly be 
surprised by very different behavior from Solr after a minor version upgrade 
where no config changes were made.  In many cases the new behavior might be 
preferred, but I don't think we can assume that.

> Solr's "Lucene"/standard query parser should not split on whitespace before 
> sending terms to analysis
> -
>
> Key: SOLR-9185
> URL: https://issues.apache.org/jira/browse/SOLR-9185
> Project: Solr
>  Issue Type: Bug
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>
> Copied from LUCENE-2605:
> The queryparser parses input on whitespace, and sends each whitespace 
> separated term to its own independent token stream.
> This breaks the following at query-time, because they can't see across 
> whitespace boundaries:
> n-gram analysis
> shingles
> synonyms (especially multi-word for whitespace-separated languages)
> languages where a 'word' can contain whitespace (e.g. vietnamese)
> Its also rather unexpected, as users think their 
> charfilters/tokenizers/tokenfilters will do the same thing at index and 
> querytime, but
> in many cases they can't. Instead, preferably the queryparser would parse 
> around only real 'operators'.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7333) BaseDirectoryTestCase may create temporary files with names not accepted by Windows (e.g. com1, con,...)

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-7333:
--
Attachment: LUCENE-7333.patch

More realistic variant. I think that's ready; will commit tomorrow morning.

> BaseDirectoryTestCase may create temporary files with names not accepted by 
> Windows (e.g. com1, con,...)
> 
>
> Key: LUCENE-7333
> URL: https://issues.apache.org/jira/browse/LUCENE-7333
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/test-framework
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
> Attachments: LUCENE-7333.patch, LUCENE-7333.patch
>
>
> BaseDirectoryTestCase may randomly create files with "special names", which 
> are not allowed by certain operating systems, e.g. Windows.
> See [https://msdn.microsoft.com/en-us/library/aa365247.aspx] for more info.
> This is the issue we have seen:
> {noformat}
> java.security.AccessControlException: access denied ("java.io.FilePermission" 
> "C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\lucene\build\core\test\J0\temp\lucene.store.TestMmapDirectory_96FC6F2D45B76809-001\tempDir-007\con"
>  "write")
>   at 
> __randomizedtesting.SeedInfo.seed([96FC6F2D45B76809:DE018553DD164AB]:0)
>   at 
> java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
>   at 
> java.security.AccessController.checkPermission(AccessController.java:884)
>   at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
>   at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
>   at sun.nio.fs.WindowsChannelFactory.open(WindowsChannelFactory.java:295)
>   at 
> sun.nio.fs.WindowsChannelFactory.newFileChannel(WindowsChannelFactory.java:162)
>   at 
> sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:225)
>   at 
> java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434)
>   at 
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
>   at 
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
>   at 
> org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
>   at 
> org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
>   at 
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
>   at 
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
>   at java.nio.file.Files.newOutputStream(Files.java:216)
>   at 
> org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:408)
>   at 
> org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:404)
>   at 
> org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:253)
>   at 
> org.apache.lucene.store.BaseDirectoryTestCase.testPendingDeletions(BaseDirectoryTestCase.java:1215)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
>   at 
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
>   at 
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>   at 
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>   at 
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>   at 
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>   at 
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>   at 
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
>   at 
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
>   at 
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(Th

[JENKINS-EA] Lucene-Solr-6.1-Linux (32bit/jdk-9-ea+122) - Build # 12 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.1-Linux/12/
Java: 32bit/jdk-9-ea+122 -server -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.handler.TestReqParamsAPI.test

Error Message:
Could not get expected value  'CY val' for path 'response/params/y/c' full 
output: {   "responseHeader":{ "status":0, "QTime":0},   "response":{   
  "znodeVersion":0, "params":{"x":{ "a":"A val", "b":"B 
val", "":{"v":0},  from server:  https://127.0.0.1:33179/collection1

Stack Trace:
java.lang.AssertionError: Could not get expected value  'CY val' for path 
'response/params/y/c' full output: {
  "responseHeader":{
"status":0,
"QTime":0},
  "response":{
"znodeVersion":0,
"params":{"x":{
"a":"A val",
"b":"B val",
"":{"v":0},  from server:  https://127.0.0.1:33179/collection1
at 
__randomizedtesting.SeedInfo.seed([5D3E34003561A35:8D87DC9AADAA77CD]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.core.TestSolrConfigHandler.testForResponseElement(TestSolrConfigHandler.java:481)
at 
org.apache.solr.handler.TestReqParamsAPI.testReqParams(TestReqParamsAPI.java:160)
at 
org.apache.solr.handler.TestReqParamsAPI.test(TestReqParamsAPI.java:62)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-ea/Native 
Method)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-ea/NativeMethodAccessorImpl.java:62)
at 
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-ea/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@9-ea/Method.java:531)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:3

[jira] [Updated] (LUCENE-7333) BaseDirectoryTestCase may create temporary files with names not accepted by Windows (e.g. com1, con,...)

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-7333:
--
Description: 
BaseDirectoryTestCase may randomly create files with "special names", which are 
not allowed by certain operating systems, e.g. Windows.

See [https://msdn.microsoft.com/en-us/library/aa365247.aspx] for more info.

This is the issue we have seen:

{noformat}
java.security.AccessControlException: access denied ("java.io.FilePermission" 
"C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\lucene\build\core\test\J0\temp\lucene.store.TestMmapDirectory_96FC6F2D45B76809-001\tempDir-007\con"
 "write")
at 
__randomizedtesting.SeedInfo.seed([96FC6F2D45B76809:DE018553DD164AB]:0)
at 
java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
at 
java.security.AccessController.checkPermission(AccessController.java:884)
at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
at sun.nio.fs.WindowsChannelFactory.open(WindowsChannelFactory.java:295)
at 
sun.nio.fs.WindowsChannelFactory.newFileChannel(WindowsChannelFactory.java:162)
at 
sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:225)
at 
java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434)
at 
org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
at 
org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
at 
org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
at 
org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
at 
org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
at 
org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
at java.nio.file.Files.newOutputStream(Files.java:216)
at 
org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:408)
at 
org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:404)
at 
org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:253)
at 
org.apache.lucene.store.BaseDirectoryTestCase.testPendingDeletions(BaseDirectoryTestCase.java:1215)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41

[jira] [Updated] (LUCENE-7333) BaseDirectoryTestCase may create temporary files with names not accepted by Windows (e.g. com1, con,...)

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-7333:
--
Summary: BaseDirectoryTestCase may create temporary files with names not 
accepted by Windows (e.g. com1, con,...)  (was: Test framework may create 
temporary files with names not accepted by Windows (e.g. com1, con,...))

> BaseDirectoryTestCase may create temporary files with names not accepted by 
> Windows (e.g. com1, con,...)
> 
>
> Key: LUCENE-7333
> URL: https://issues.apache.org/jira/browse/LUCENE-7333
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/test-framework
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
> Attachments: LUCENE-7333.patch
>
>
> The test framework can randomly create files with "special names", which are 
> not allowed by certain operating systems, e.g. Windows.
> See [https://msdn.microsoft.com/en-us/library/aa365247.aspx] for more info.
> This is the issue we have seen:
> {noformat}
> java.security.AccessControlException: access denied ("java.io.FilePermission" 
> "C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\lucene\build\core\test\J0\temp\lucene.store.TestMmapDirectory_96FC6F2D45B76809-001\tempDir-007\con"
>  "write")
>   at 
> __randomizedtesting.SeedInfo.seed([96FC6F2D45B76809:DE018553DD164AB]:0)
>   at 
> java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
>   at 
> java.security.AccessController.checkPermission(AccessController.java:884)
>   at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
>   at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
>   at sun.nio.fs.WindowsChannelFactory.open(WindowsChannelFactory.java:295)
>   at 
> sun.nio.fs.WindowsChannelFactory.newFileChannel(WindowsChannelFactory.java:162)
>   at 
> sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:225)
>   at 
> java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434)
>   at 
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
>   at 
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
>   at 
> org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
>   at 
> org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
>   at 
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
>   at 
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
>   at java.nio.file.Files.newOutputStream(Files.java:216)
>   at 
> org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:408)
>   at 
> org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:404)
>   at 
> org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:253)
>   at 
> org.apache.lucene.store.BaseDirectoryTestCase.testPendingDeletions(BaseDirectoryTestCase.java:1215)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
>   at 
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
>   at 
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>   at 
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>   at 
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>   at 
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>   at 
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>   at 
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
>   at 
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakContr

[jira] [Updated] (LUCENE-7333) Test framework may create temporary files with names not accepted by Windows (e.g. com1, con,...)

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-7333:
--
Attachment: LUCENE-7333.patch

Simple patch to fix this test only. I did not find any other places where a 
file name was created by randomSimpleString().

> Test framework may create temporary files with names not accepted by Windows 
> (e.g. com1, con,...)
> -
>
> Key: LUCENE-7333
> URL: https://issues.apache.org/jira/browse/LUCENE-7333
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/test-framework
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
> Attachments: LUCENE-7333.patch
>
>
> The test framework can randomly create files with "special names", which are 
> not allowed by certain operating systems, e.g. Windows.
> See [https://msdn.microsoft.com/en-us/library/aa365247.aspx] for more info.
> This is the issue we have seen:
> {noformat}
> java.security.AccessControlException: access denied ("java.io.FilePermission" 
> "C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\lucene\build\core\test\J0\temp\lucene.store.TestMmapDirectory_96FC6F2D45B76809-001\tempDir-007\con"
>  "write")
>   at 
> __randomizedtesting.SeedInfo.seed([96FC6F2D45B76809:DE018553DD164AB]:0)
>   at 
> java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
>   at 
> java.security.AccessController.checkPermission(AccessController.java:884)
>   at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
>   at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
>   at sun.nio.fs.WindowsChannelFactory.open(WindowsChannelFactory.java:295)
>   at 
> sun.nio.fs.WindowsChannelFactory.newFileChannel(WindowsChannelFactory.java:162)
>   at 
> sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:225)
>   at 
> java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434)
>   at 
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
>   at 
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
>   at 
> org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
>   at 
> org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
>   at 
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
>   at 
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
>   at java.nio.file.Files.newOutputStream(Files.java:216)
>   at 
> org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:408)
>   at 
> org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:404)
>   at 
> org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:253)
>   at 
> org.apache.lucene.store.BaseDirectoryTestCase.testPendingDeletions(BaseDirectoryTestCase.java:1215)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
>   at 
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
>   at 
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>   at 
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>   at 
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>   at 
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>   at 
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>   at 
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
>   at 
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
>   at 
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3

[jira] [Commented] (LUCENE-7333) Test framework may create temporary files with names not accepted by Windows (e.g. com1, con,...)

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15325312#comment-15325312
 ] 

Uwe Schindler commented on LUCENE-7333:
---

The createTempDir functions work fine, because they retry on IOException. So 
its only this test. I will fix by adding a new TestUtil function to create a 
random filename instead of just randomSimpleString(): randomFileName()

> Test framework may create temporary files with names not accepted by Windows 
> (e.g. com1, con,...)
> -
>
> Key: LUCENE-7333
> URL: https://issues.apache.org/jira/browse/LUCENE-7333
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/test-framework
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>
> The test framework can randomly create files with "special names", which are 
> not allowed by certain operating systems, e.g. Windows.
> See [https://msdn.microsoft.com/en-us/library/aa365247.aspx] for more info.
> This is the issue we have seen:
> {noformat}
> java.security.AccessControlException: access denied ("java.io.FilePermission" 
> "C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\lucene\build\core\test\J0\temp\lucene.store.TestMmapDirectory_96FC6F2D45B76809-001\tempDir-007\con"
>  "write")
>   at 
> __randomizedtesting.SeedInfo.seed([96FC6F2D45B76809:DE018553DD164AB]:0)
>   at 
> java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
>   at 
> java.security.AccessController.checkPermission(AccessController.java:884)
>   at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
>   at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
>   at sun.nio.fs.WindowsChannelFactory.open(WindowsChannelFactory.java:295)
>   at 
> sun.nio.fs.WindowsChannelFactory.newFileChannel(WindowsChannelFactory.java:162)
>   at 
> sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:225)
>   at 
> java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434)
>   at 
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
>   at 
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
>   at 
> org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
>   at 
> org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
>   at 
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
>   at 
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
>   at java.nio.file.Files.newOutputStream(Files.java:216)
>   at 
> org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:408)
>   at 
> org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:404)
>   at 
> org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:253)
>   at 
> org.apache.lucene.store.BaseDirectoryTestCase.testPendingDeletions(BaseDirectoryTestCase.java:1215)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
>   at 
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
>   at 
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>   at 
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>   at 
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>   at 
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>   at 
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>   at 
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
>   at 
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakCon

[jira] [Commented] (LUCENE-7333) Test framework may create temporary files with names not accepted by Windows (e.g. com1, con,...)

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15325279#comment-15325279
 ] 

Uwe Schindler commented on LUCENE-7333:
---

The problem in this test is that it creates a file using randomSimpleString(). 
It is not using the temp dir / temp file functions. Not so easy to solve.

> Test framework may create temporary files with names not accepted by Windows 
> (e.g. com1, con,...)
> -
>
> Key: LUCENE-7333
> URL: https://issues.apache.org/jira/browse/LUCENE-7333
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/test-framework
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>
> The test framework can randomly create files with "special names", which are 
> not allowed by certain operating systems, e.g. Windows.
> See [https://msdn.microsoft.com/en-us/library/aa365247.aspx] for more info.
> This is the issue we have seen:
> {noformat}
> java.security.AccessControlException: access denied ("java.io.FilePermission" 
> "C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\lucene\build\core\test\J0\temp\lucene.store.TestMmapDirectory_96FC6F2D45B76809-001\tempDir-007\con"
>  "write")
>   at 
> __randomizedtesting.SeedInfo.seed([96FC6F2D45B76809:DE018553DD164AB]:0)
>   at 
> java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
>   at 
> java.security.AccessController.checkPermission(AccessController.java:884)
>   at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
>   at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
>   at sun.nio.fs.WindowsChannelFactory.open(WindowsChannelFactory.java:295)
>   at 
> sun.nio.fs.WindowsChannelFactory.newFileChannel(WindowsChannelFactory.java:162)
>   at 
> sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:225)
>   at 
> java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434)
>   at 
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
>   at 
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
>   at 
> org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
>   at 
> org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
>   at 
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
>   at 
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
>   at java.nio.file.Files.newOutputStream(Files.java:216)
>   at 
> org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:408)
>   at 
> org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:404)
>   at 
> org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:253)
>   at 
> org.apache.lucene.store.BaseDirectoryTestCase.testPendingDeletions(BaseDirectoryTestCase.java:1215)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
>   at 
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
>   at 
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>   at 
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>   at 
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>   at 
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>   at 
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>   at 
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
>   at 
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
>   at 
> com.carrotsearch.randomizedtesting.ThreadLea

Re: [JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_92) - Build # 239 - Still Failing!

[Michael McCandless]

Thanks Uwe.

Mike McCandless

http://blog.mikemccandless.com

On Fri, Jun 10, 2016 at 4:28 PM, Uwe Schindler  wrote:

> OK. Will do and work on a solution!
>
> I will also check for similar issues with Linux or Mäc.
>
> Uwe
>
> Am 10. Juni 2016 22:23:28 MESZ, schrieb Michael McCandless <
> luc...@mikemccandless.com>:
> >Let's open an issue?  It's a confusing failure :)  I had no clue "con"
> >was
> >not allowed on windows.
> >
> >Mike McCandless
> >
> >http://blog.mikemccandless.com
> >
> >On Fri, Jun 10, 2016 at 1:28 PM, Uwe Schindler  wrote:
> >
> >> Here is the documentation:
> >>
> >>
> >>
> >> https://msdn.microsoft.com/en-us/library/aa365247.aspx
> >>
> >>
> >>
> >> We could either fix the issue by adding the list of illegal names to
> >the
> >> random name generator, or we just let it be an let it fail sometimes
> >J
> >>
> >> Should we open issue?
> >>
> >>
> >>
> >> Uwe
> >>
> >>
> >>
> >> -
> >>
> >> Uwe Schindler
> >>
> >> H.-H.-Meier-Allee 63, D-28213 Bremen
> >>
> >> http://www.thetaphi.de
> >>
> >> eMail: u...@thetaphi.de
> >>
> >>
> >>
> >> *From:* Uwe Schindler [mailto:u...@thetaphi.de]
> >> *Sent:* Friday, June 10, 2016 7:26 PM
> >>
> >> *To:* dev@lucene.apache.org
> >> *Subject:* RE: [JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_92)
> >-
> >> Build # 239 - Still Failing!
> >>
> >>
> >>
> >> Hi,
> >>
> >>
> >>
> >> Of course!
> >>
> >>
> >>
> >> It is only strange that this error comes from the AccessController
> >and not
> >> the operating system… So either the FilePermission code “knows” about
> >this
> >> special names on windows, or it is a side effect while calculating
> >canonic
> >> name of the file.
> >>
> >>
> >>
> >> Uwe
> >>
> >>
> >>
> >> -
> >>
> >> Uwe Schindler
> >>
> >> H.-H.-Meier-Allee 63, D-28213 Bremen
> >>
> >> http://www.thetaphi.de
> >>
> >> eMail: u...@thetaphi.de
> >>
> >>
> >>
> >> *From:* Dawid Weiss [mailto:dawid.we...@gmail.com
> >]
> >>
> >> *Sent:* Friday, June 10, 2016 6:59 PM
> >> *To:* dev@lucene.apache.org
> >> *Subject:* RE: [JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_92)
> >-
> >> Build # 239 - Still Failing!
> >>
> >>
> >>
> >>
> >> I think I do. Con is a special filename. You cannot use it on
> >windows.
> >>
> >> On Jun 10, 2016 5:50 PM, "Uwe Schindler"  wrote:
> >>
> >> Hi,
> >>
> >>
> >>
> >> This is indeed very strange. I have no idea!
> >>
> >>
> >>
> >> Uwe
> >>
> >>
> >>
> >> -
> >>
> >> Uwe Schindler
> >>
> >> H.-H.-Meier-Allee 63, D-28213 Bremen
> >>
> >> http://www.thetaphi.de
> >>
> >> eMail: u...@thetaphi.de
> >>
> >>
> >>
> >> *From:* Michael McCandless [mailto:luc...@mikemccandless.com]
> >> *Sent:* Friday, June 10, 2016 5:45 PM
> >> *To:* Policeman Jenkins Server 
> >> *Cc:* Adrien Grand ; Gregory Chanan <
> >> gcha...@cloudera.com>; dragonsi...@apache.org; Lucene/Solr dev <
> >> dev@lucene.apache.org>
> >> *Subject:* Re: [JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_92)
> >-
> >> Build # 239 - Still Failing!
> >>
> >>
> >>
> >> Hmm, why was access denied here?
> >>
> >>
> >> Mike McCandless
> >>
> >> http://blog.mikemccandless.com
> >>
> >>
> >>
> >> On Thu, Jun 9, 2016 at 5:51 PM, Policeman Jenkins Server <
> >> jenk...@thetaphi.de> wrote:
> >>
> >> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Windows/239/
> >> Java: 32bit/jdk1.8.0_92 -client -XX:+UseConcMarkSweepGC
> >>
> >> 2 tests failed.
> >> FAILED:
> >org.apache.lucene.store.TestMmapDirectory.testPendingDeletions
> >>
> >> Error Message:
> >> access denied ("java.io.FilePermission"
> >>
>
> >"C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\lucene\build\core\test\J0\temp\lucene.store.TestMmapDirectory_96FC6F2D45B76809-001\tempDir-007\con"
> >> "write")
> >>
> >> Stack Trace:
> >> java.security.AccessControlException: access denied
> >> ("java.io.FilePermission"
> >>
>
> >"C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\lucene\build\core\test\J0\temp\lucene.store.TestMmapDirectory_96FC6F2D45B76809-001\tempDir-007\con"
> >> "write")
> >> at
> >>
> >__randomizedtesting.SeedInfo.seed([96FC6F2D45B76809:DE018553DD164AB]:0)
> >> at
> >>
>
> >java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> >> at
> >>
> >java.security.AccessController.checkPermission(AccessController.java:884)
> >> at
> >> java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
> >> at
> >java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
> >> at
> >> sun.nio.fs.WindowsChannelFactory.open(WindowsChannelFactory.java:295)
> >> at
> >>
>
> >sun.nio.fs.WindowsChannelFactory.newFileChannel(WindowsChannelFactory.java:162)
> >> at
> >>
>
> >sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:225)
> >> at
> >>
>
> >java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434)
> >> at
> >>
>
> >org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
> >>   

[jira] [Created] (LUCENE-7333) Test framework may create temporary files with names not accepted by Windows (e.g. com1, con,...)

[Uwe Schindler (JIRA)]

Uwe Schindler created LUCENE-7333:
-

 Summary: Test framework may create temporary files with names not 
accepted by Windows (e.g. com1, con,...)
 Key: LUCENE-7333
 URL: https://issues.apache.org/jira/browse/LUCENE-7333
 Project: Lucene - Core
  Issue Type: Bug
  Components: modules/test-framework
Reporter: Uwe Schindler
Assignee: Uwe Schindler


The test framework can randomly create files with "special names", which are 
not allowed by certain operating systems, e.g. Windows.

See [https://msdn.microsoft.com/en-us/library/aa365247.aspx] for more info.

This is the issue we have seen:

{noformat}
java.security.AccessControlException: access denied ("java.io.FilePermission" 
"C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\lucene\build\core\test\J0\temp\lucene.store.TestMmapDirectory_96FC6F2D45B76809-001\tempDir-007\con"
 "write")
at 
__randomizedtesting.SeedInfo.seed([96FC6F2D45B76809:DE018553DD164AB]:0)
at 
java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
at 
java.security.AccessController.checkPermission(AccessController.java:884)
at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
at sun.nio.fs.WindowsChannelFactory.open(WindowsChannelFactory.java:295)
at 
sun.nio.fs.WindowsChannelFactory.newFileChannel(WindowsChannelFactory.java:162)
at 
sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:225)
at 
java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434)
at 
org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
at 
org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
at 
org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
at 
org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
at 
org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
at 
org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
at java.nio.file.Files.newOutputStream(Files.java:216)
at 
org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:408)
at 
org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:404)
at 
org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:253)
at 
org.apache.lucene.store.BaseDirectoryTestCase.testPendingDeletions(BaseDirectoryTestCase.java:1215)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
org.apache.lucene.util.AbstractBe

RE: [JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_92) - Build # 239 - Still Failing!

[Uwe Schindler]

https://issues.apache.org/jira/browse/LUCENE-7333

-
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de

> -Original Message-
> From: Uwe Schindler [mailto:u...@thetaphi.de]
> Sent: Friday, June 10, 2016 10:28 PM
> To: dev@lucene.apache.org
> Subject: Re: [JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_92) - Build #
> 239 - Still Failing!
> 
> OK. Will do and work on a solution!
> 
> I will also check for similar issues with Linux or Mäc.
> 
> Uwe
> 
> Am 10. Juni 2016 22:23:28 MESZ, schrieb Michael McCandless
> :
> >Let's open an issue?  It's a confusing failure :)  I had no clue "con"
> >was
> >not allowed on windows.
> >
> >Mike McCandless
> >
> >http://blog.mikemccandless.com
> >
> >On Fri, Jun 10, 2016 at 1:28 PM, Uwe Schindler  wrote:
> >
> >> Here is the documentation:
> >>
> >>
> >>
> >> https://msdn.microsoft.com/en-us/library/aa365247.aspx
> >>
> >>
> >>
> >> We could either fix the issue by adding the list of illegal names to
> >the
> >> random name generator, or we just let it be an let it fail sometimes
> >J
> >>
> >> Should we open issue?
> >>
> >>
> >>
> >> Uwe
> >>
> >>
> >>
> >> -
> >>
> >> Uwe Schindler
> >>
> >> H.-H.-Meier-Allee 63, D-28213 Bremen
> >>
> >> http://www.thetaphi.de
> >>
> >> eMail: u...@thetaphi.de
> >>
> >>
> >>
> >> *From:* Uwe Schindler [mailto:u...@thetaphi.de]
> >> *Sent:* Friday, June 10, 2016 7:26 PM
> >>
> >> *To:* dev@lucene.apache.org
> >> *Subject:* RE: [JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_92)
> >-
> >> Build # 239 - Still Failing!
> >>
> >>
> >>
> >> Hi,
> >>
> >>
> >>
> >> Of course!
> >>
> >>
> >>
> >> It is only strange that this error comes from the AccessController
> >and not
> >> the operating system… So either the FilePermission code “knows” about
> >this
> >> special names on windows, or it is a side effect while calculating
> >canonic
> >> name of the file.
> >>
> >>
> >>
> >> Uwe
> >>
> >>
> >>
> >> -
> >>
> >> Uwe Schindler
> >>
> >> H.-H.-Meier-Allee 63, D-28213 Bremen
> >>
> >> http://www.thetaphi.de
> >>
> >> eMail: u...@thetaphi.de
> >>
> >>
> >>
> >> *From:* Dawid Weiss [mailto:dawid.we...@gmail.com
> >]
> >>
> >> *Sent:* Friday, June 10, 2016 6:59 PM
> >> *To:* dev@lucene.apache.org
> >> *Subject:* RE: [JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_92)
> >-
> >> Build # 239 - Still Failing!
> >>
> >>
> >>
> >>
> >> I think I do. Con is a special filename. You cannot use it on
> >windows.
> >>
> >> On Jun 10, 2016 5:50 PM, "Uwe Schindler"  wrote:
> >>
> >> Hi,
> >>
> >>
> >>
> >> This is indeed very strange. I have no idea!
> >>
> >>
> >>
> >> Uwe
> >>
> >>
> >>
> >> -
> >>
> >> Uwe Schindler
> >>
> >> H.-H.-Meier-Allee 63, D-28213 Bremen
> >>
> >> http://www.thetaphi.de
> >>
> >> eMail: u...@thetaphi.de
> >>
> >>
> >>
> >> *From:* Michael McCandless [mailto:luc...@mikemccandless.com]
> >> *Sent:* Friday, June 10, 2016 5:45 PM
> >> *To:* Policeman Jenkins Server 
> >> *Cc:* Adrien Grand ; Gregory Chanan <
> >> gcha...@cloudera.com>; dragonsi...@apache.org; Lucene/Solr dev <
> >> dev@lucene.apache.org>
> >> *Subject:* Re: [JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_92)
> >-
> >> Build # 239 - Still Failing!
> >>
> >>
> >>
> >> Hmm, why was access denied here?
> >>
> >>
> >> Mike McCandless
> >>
> >> http://blog.mikemccandless.com
> >>
> >>
> >>
> >> On Thu, Jun 9, 2016 at 5:51 PM, Policeman Jenkins Server <
> >> jenk...@thetaphi.de> wrote:
> >>
> >> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Windows/239/
> >> Java: 32bit/jdk1.8.0_92 -client -XX:+UseConcMarkSweepGC
> >>
> >> 2 tests failed.
> >> FAILED:
> >org.apache.lucene.store.TestMmapDirectory.testPendingDeletions
> >>
> >> Error Message:
> >> access denied ("java.io.FilePermission"
> >>
> >"C:\Users\jenkins\workspace\Lucene-Solr-6.x-
> Windows\lucene\build\core\test\J0\temp\lucene.store.TestMmapDirectory
> _96FC6F2D45B76809-001\tempDir-007\con"
> >> "write")
> >>
> >> Stack Trace:
> >> java.security.AccessControlException: access denied
> >> ("java.io.FilePermission"
> >>
> >"C:\Users\jenkins\workspace\Lucene-Solr-6.x-
> Windows\lucene\build\core\test\J0\temp\lucene.store.TestMmapDirectory
> _96FC6F2D45B76809-001\tempDir-007\con"
> >> "write")
> >> at
> >>
> >__randomizedtesting.SeedInfo.seed([96FC6F2D45B76809:DE018553DD164A
> B]:0)
> >> at
> >>
> >java.security.AccessControlContext.checkPermission(AccessControlContext
> .java:472)
> >> at
> >>
> >java.security.AccessController.checkPermission(AccessController.java:884)
> >> at
> >> java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
> >> at
> >java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
> >> at
> >>
> sun.nio.fs.WindowsChannelFactory.open(WindowsChannelFactory.java:295)
> >> at
> >>
> >sun.nio.fs.WindowsChannelFactory.newFileChannel(WindowsChannelFact
> ory.java:162)
> >> at
> >>
> >sun.nio.fs.WindowsFileSystemProvider.newByteChannel(Windo

Re: [JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_92) - Build # 239 - Still Failing!

[Uwe Schindler]

OK. Will do and work on a solution!

I will also check for similar issues with Linux or Mäc.

Uwe

Am 10. Juni 2016 22:23:28 MESZ, schrieb Michael McCandless 
:
>Let's open an issue?  It's a confusing failure :)  I had no clue "con"
>was
>not allowed on windows.
>
>Mike McCandless
>
>http://blog.mikemccandless.com
>
>On Fri, Jun 10, 2016 at 1:28 PM, Uwe Schindler  wrote:
>
>> Here is the documentation:
>>
>>
>>
>> https://msdn.microsoft.com/en-us/library/aa365247.aspx
>>
>>
>>
>> We could either fix the issue by adding the list of illegal names to
>the
>> random name generator, or we just let it be an let it fail sometimes
>J
>>
>> Should we open issue?
>>
>>
>>
>> Uwe
>>
>>
>>
>> -
>>
>> Uwe Schindler
>>
>> H.-H.-Meier-Allee 63, D-28213 Bremen
>>
>> http://www.thetaphi.de
>>
>> eMail: u...@thetaphi.de
>>
>>
>>
>> *From:* Uwe Schindler [mailto:u...@thetaphi.de]
>> *Sent:* Friday, June 10, 2016 7:26 PM
>>
>> *To:* dev@lucene.apache.org
>> *Subject:* RE: [JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_92)
>-
>> Build # 239 - Still Failing!
>>
>>
>>
>> Hi,
>>
>>
>>
>> Of course!
>>
>>
>>
>> It is only strange that this error comes from the AccessController
>and not
>> the operating system… So either the FilePermission code “knows” about
>this
>> special names on windows, or it is a side effect while calculating
>canonic
>> name of the file.
>>
>>
>>
>> Uwe
>>
>>
>>
>> -
>>
>> Uwe Schindler
>>
>> H.-H.-Meier-Allee 63, D-28213 Bremen
>>
>> http://www.thetaphi.de
>>
>> eMail: u...@thetaphi.de
>>
>>
>>
>> *From:* Dawid Weiss [mailto:dawid.we...@gmail.com
>]
>>
>> *Sent:* Friday, June 10, 2016 6:59 PM
>> *To:* dev@lucene.apache.org
>> *Subject:* RE: [JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_92)
>-
>> Build # 239 - Still Failing!
>>
>>
>>
>>
>> I think I do. Con is a special filename. You cannot use it on
>windows.
>>
>> On Jun 10, 2016 5:50 PM, "Uwe Schindler"  wrote:
>>
>> Hi,
>>
>>
>>
>> This is indeed very strange. I have no idea!
>>
>>
>>
>> Uwe
>>
>>
>>
>> -
>>
>> Uwe Schindler
>>
>> H.-H.-Meier-Allee 63, D-28213 Bremen
>>
>> http://www.thetaphi.de
>>
>> eMail: u...@thetaphi.de
>>
>>
>>
>> *From:* Michael McCandless [mailto:luc...@mikemccandless.com]
>> *Sent:* Friday, June 10, 2016 5:45 PM
>> *To:* Policeman Jenkins Server 
>> *Cc:* Adrien Grand ; Gregory Chanan <
>> gcha...@cloudera.com>; dragonsi...@apache.org; Lucene/Solr dev <
>> dev@lucene.apache.org>
>> *Subject:* Re: [JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_92)
>-
>> Build # 239 - Still Failing!
>>
>>
>>
>> Hmm, why was access denied here?
>>
>>
>> Mike McCandless
>>
>> http://blog.mikemccandless.com
>>
>>
>>
>> On Thu, Jun 9, 2016 at 5:51 PM, Policeman Jenkins Server <
>> jenk...@thetaphi.de> wrote:
>>
>> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Windows/239/
>> Java: 32bit/jdk1.8.0_92 -client -XX:+UseConcMarkSweepGC
>>
>> 2 tests failed.
>> FAILED: 
>org.apache.lucene.store.TestMmapDirectory.testPendingDeletions
>>
>> Error Message:
>> access denied ("java.io.FilePermission"
>>
>"C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\lucene\build\core\test\J0\temp\lucene.store.TestMmapDirectory_96FC6F2D45B76809-001\tempDir-007\con"
>> "write")
>>
>> Stack Trace:
>> java.security.AccessControlException: access denied
>> ("java.io.FilePermission"
>>
>"C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\lucene\build\core\test\J0\temp\lucene.store.TestMmapDirectory_96FC6F2D45B76809-001\tempDir-007\con"
>> "write")
>> at
>>
>__randomizedtesting.SeedInfo.seed([96FC6F2D45B76809:DE018553DD164AB]:0)
>> at
>>
>java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
>> at
>>
>java.security.AccessController.checkPermission(AccessController.java:884)
>> at
>> java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
>> at
>java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
>> at
>> sun.nio.fs.WindowsChannelFactory.open(WindowsChannelFactory.java:295)
>> at
>>
>sun.nio.fs.WindowsChannelFactory.newFileChannel(WindowsChannelFactory.java:162)
>> at
>>
>sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:225)
>> at
>>
>java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434)
>> at
>>
>org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
>> at
>>
>org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
>> at
>>
>org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
>> at
>>
>org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
>> at
>>
>org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
>> at
>>
>org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
>

Re: [JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_92) - Build # 239 - Still Failing!

[Michael McCandless]

Let's open an issue?  It's a confusing failure :)  I had no clue "con" was
not allowed on windows.

Mike McCandless

http://blog.mikemccandless.com

On Fri, Jun 10, 2016 at 1:28 PM, Uwe Schindler  wrote:

> Here is the documentation:
>
>
>
> https://msdn.microsoft.com/en-us/library/aa365247.aspx
>
>
>
> We could either fix the issue by adding the list of illegal names to the
> random name generator, or we just let it be an let it fail sometimes J
>
> Should we open issue?
>
>
>
> Uwe
>
>
>
> -
>
> Uwe Schindler
>
> H.-H.-Meier-Allee 63, D-28213 Bremen
>
> http://www.thetaphi.de
>
> eMail: u...@thetaphi.de
>
>
>
> *From:* Uwe Schindler [mailto:u...@thetaphi.de]
> *Sent:* Friday, June 10, 2016 7:26 PM
>
> *To:* dev@lucene.apache.org
> *Subject:* RE: [JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_92) -
> Build # 239 - Still Failing!
>
>
>
> Hi,
>
>
>
> Of course!
>
>
>
> It is only strange that this error comes from the AccessController and not
> the operating system… So either the FilePermission code “knows” about this
> special names on windows, or it is a side effect while calculating canonic
> name of the file.
>
>
>
> Uwe
>
>
>
> -
>
> Uwe Schindler
>
> H.-H.-Meier-Allee 63, D-28213 Bremen
>
> http://www.thetaphi.de
>
> eMail: u...@thetaphi.de
>
>
>
> *From:* Dawid Weiss [mailto:dawid.we...@gmail.com ]
>
> *Sent:* Friday, June 10, 2016 6:59 PM
> *To:* dev@lucene.apache.org
> *Subject:* RE: [JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_92) -
> Build # 239 - Still Failing!
>
>
>
>
> I think I do. Con is a special filename. You cannot use it on windows.
>
> On Jun 10, 2016 5:50 PM, "Uwe Schindler"  wrote:
>
> Hi,
>
>
>
> This is indeed very strange. I have no idea!
>
>
>
> Uwe
>
>
>
> -
>
> Uwe Schindler
>
> H.-H.-Meier-Allee 63, D-28213 Bremen
>
> http://www.thetaphi.de
>
> eMail: u...@thetaphi.de
>
>
>
> *From:* Michael McCandless [mailto:luc...@mikemccandless.com]
> *Sent:* Friday, June 10, 2016 5:45 PM
> *To:* Policeman Jenkins Server 
> *Cc:* Adrien Grand ; Gregory Chanan <
> gcha...@cloudera.com>; dragonsi...@apache.org; Lucene/Solr dev <
> dev@lucene.apache.org>
> *Subject:* Re: [JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_92) -
> Build # 239 - Still Failing!
>
>
>
> Hmm, why was access denied here?
>
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
>
>
> On Thu, Jun 9, 2016 at 5:51 PM, Policeman Jenkins Server <
> jenk...@thetaphi.de> wrote:
>
> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Windows/239/
> Java: 32bit/jdk1.8.0_92 -client -XX:+UseConcMarkSweepGC
>
> 2 tests failed.
> FAILED:  org.apache.lucene.store.TestMmapDirectory.testPendingDeletions
>
> Error Message:
> access denied ("java.io.FilePermission"
> "C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\lucene\build\core\test\J0\temp\lucene.store.TestMmapDirectory_96FC6F2D45B76809-001\tempDir-007\con"
> "write")
>
> Stack Trace:
> java.security.AccessControlException: access denied
> ("java.io.FilePermission"
> "C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\lucene\build\core\test\J0\temp\lucene.store.TestMmapDirectory_96FC6F2D45B76809-001\tempDir-007\con"
> "write")
> at
> __randomizedtesting.SeedInfo.seed([96FC6F2D45B76809:DE018553DD164AB]:0)
> at
> java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> at
> java.security.AccessController.checkPermission(AccessController.java:884)
> at
> java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
> at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
> at
> sun.nio.fs.WindowsChannelFactory.open(WindowsChannelFactory.java:295)
> at
> sun.nio.fs.WindowsChannelFactory.newFileChannel(WindowsChannelFactory.java:162)
> at
> sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:225)
> at
> java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434)
> at
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
> at
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
> at
> org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
> at
> org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
> at
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
> at
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
> at java.nio.file.Files.newOutputStream(Files.java:216)
> at
> org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:408)
> at
> org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:404)
> at
> org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:253)
> at
> org.apache.lucene.store.BaseDirec

[jira] [Updated] (LUCENE-5931) DirectoryReader.openIfChanged(oldReader, commit) incorrectly assumes given commit point has deletes/field updates

[Michael McCandless (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-5931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless updated LUCENE-5931:
---
Fix Version/s: 6.2
   master (7.0)

> DirectoryReader.openIfChanged(oldReader, commit) incorrectly assumes given 
> commit point has deletes/field updates
> -
>
> Key: LUCENE-5931
> URL: https://issues.apache.org/jira/browse/LUCENE-5931
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 4.6.1
>Reporter: Vitaly Funstein
>Assignee: Michael McCandless
>Priority: Critical
> Fix For: master (7.0), 6.2
>
> Attachments: CommitReuseTest.java, LUCENE-5931.patch, 
> LUCENE-5931.patch, LUCENE-5931.patch, LUCENE-5931.patch, LUCENE-5931.patch
>
>
> {{StandardDirectoryReader}} assumes that the segments from commit point have 
> deletes, when they may not, yet the original SegmentReader for the segment 
> that we are trying to reuse does. This is evident when running attached JUnit 
> test case with asserts enabled (default): 
> {noformat}
> java.lang.AssertionError
>   at 
> org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:188)
>   at 
> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:326)
>   at 
> org.apache.lucene.index.StandardDirectoryReader$2.doBody(StandardDirectoryReader.java:320)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:702)
>   at 
> org.apache.lucene.index.StandardDirectoryReader.doOpenFromCommit(StandardDirectoryReader.java:315)
>   at 
> org.apache.lucene.index.StandardDirectoryReader.doOpenNoWriter(StandardDirectoryReader.java:311)
>   at 
> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:262)
>   at 
> org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:183)
> {noformat}
> or, if asserts are disabled then it falls through into NPE:
> {noformat}
> java.lang.NullPointerException
>   at java.io.File.(File.java:305)
>   at 
> org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:80)
>   at 
> org.apache.lucene.codecs.lucene40.BitVector.(BitVector.java:327)
>   at 
> org.apache.lucene.codecs.lucene40.Lucene40LiveDocsFormat.readLiveDocs(Lucene40LiveDocsFormat.java:90)
>   at org.apache.lucene.index.SegmentReader.(SegmentReader.java:131)
>   at 
> org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:194)
>   at 
> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:326)
>   at 
> org.apache.lucene.index.StandardDirectoryReader$2.doBody(StandardDirectoryReader.java:320)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:702)
>   at 
> org.apache.lucene.index.StandardDirectoryReader.doOpenFromCommit(StandardDirectoryReader.java:315)
>   at 
> org.apache.lucene.index.StandardDirectoryReader.doOpenNoWriter(StandardDirectoryReader.java:311)
>   at 
> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:262)
>   at 
> org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:183)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5931) DirectoryReader.openIfChanged(oldReader, commit) incorrectly assumes given commit point has deletes/field updates

[Michael McCandless (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-5931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless updated LUCENE-5931:
---
Attachment: LUCENE-5931.patch

Woops, this almost dropped past the event horizon of my TODO list.

I modernized the patch, and was able to improve how effective its check is, by 
switching to comparing segment IDs (a very good check that the segments changed 
on disk) vs what the patch used to do, comparing maxDoc.

> DirectoryReader.openIfChanged(oldReader, commit) incorrectly assumes given 
> commit point has deletes/field updates
> -
>
> Key: LUCENE-5931
> URL: https://issues.apache.org/jira/browse/LUCENE-5931
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 4.6.1
>Reporter: Vitaly Funstein
>Assignee: Michael McCandless
>Priority: Critical
> Fix For: master (7.0), 6.2
>
> Attachments: CommitReuseTest.java, LUCENE-5931.patch, 
> LUCENE-5931.patch, LUCENE-5931.patch, LUCENE-5931.patch, LUCENE-5931.patch
>
>
> {{StandardDirectoryReader}} assumes that the segments from commit point have 
> deletes, when they may not, yet the original SegmentReader for the segment 
> that we are trying to reuse does. This is evident when running attached JUnit 
> test case with asserts enabled (default): 
> {noformat}
> java.lang.AssertionError
>   at 
> org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:188)
>   at 
> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:326)
>   at 
> org.apache.lucene.index.StandardDirectoryReader$2.doBody(StandardDirectoryReader.java:320)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:702)
>   at 
> org.apache.lucene.index.StandardDirectoryReader.doOpenFromCommit(StandardDirectoryReader.java:315)
>   at 
> org.apache.lucene.index.StandardDirectoryReader.doOpenNoWriter(StandardDirectoryReader.java:311)
>   at 
> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:262)
>   at 
> org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:183)
> {noformat}
> or, if asserts are disabled then it falls through into NPE:
> {noformat}
> java.lang.NullPointerException
>   at java.io.File.(File.java:305)
>   at 
> org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:80)
>   at 
> org.apache.lucene.codecs.lucene40.BitVector.(BitVector.java:327)
>   at 
> org.apache.lucene.codecs.lucene40.Lucene40LiveDocsFormat.readLiveDocs(Lucene40LiveDocsFormat.java:90)
>   at org.apache.lucene.index.SegmentReader.(SegmentReader.java:131)
>   at 
> org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:194)
>   at 
> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:326)
>   at 
> org.apache.lucene.index.StandardDirectoryReader$2.doBody(StandardDirectoryReader.java:320)
>   at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:702)
>   at 
> org.apache.lucene.index.StandardDirectoryReader.doOpenFromCommit(StandardDirectoryReader.java:315)
>   at 
> org.apache.lucene.index.StandardDirectoryReader.doOpenNoWriter(StandardDirectoryReader.java:311)
>   at 
> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:262)
>   at 
> org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:183)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7317) Remove auto prefix terms

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15325192#comment-15325192
 ] 

Michael McCandless commented on LUCENE-7317:


+1.  How much easier it is to remove than it was to add!!

> Remove auto prefix terms
> 
>
> Key: LUCENE-7317
> URL: https://issues.apache.org/jira/browse/LUCENE-7317
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Adrien Grand
>Priority: Minor
> Attachments: LUCENE-7317.patch
>
>
> This was mostly superseded by the new points API so should we remove 
> auto-prefix terms?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7325) GeoPointInBBoxQuery no longer includes boundaries?

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15325166#comment-15325166
 ] 

Michael McCandless commented on LUCENE-7325:


Thanks [~nknize].

> GeoPointInBBoxQuery no longer includes boundaries?
> --
>
> Key: LUCENE-7325
> URL: https://issues.apache.org/jira/browse/LUCENE-7325
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 6.1
>Reporter: Michael McCandless
>Priority: Blocker
> Attachments: LUCENE-7325.patch, LUCENE-7325.patch
>
>
> {{GeoPointInBBoxQuery}} is supposed to include boundaries, and it does in 5.x 
> and 6.0, but in 6.1 something broke.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-Solaris (64bit/jdk1.8.0) - Build # 190 - Still Failing!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Solaris/190/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC

3 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.lucene.search.TestBoolean2

Error Message:
this writer hit an unrecoverable error; cannot commit

Stack Trace:
java.lang.IllegalStateException: this writer hit an unrecoverable error; cannot 
commit
at __randomizedtesting.SeedInfo.seed([A6A15104CA0B0101]:0)
at 
org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2793)
at 
org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2989)
at org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:1086)
at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1129)
at org.apache.lucene.util.IOUtils.close(IOUtils.java:89)
at org.apache.lucene.util.IOUtils.close(IOUtils.java:76)
at 
org.apache.lucene.index.RandomIndexWriter.close(RandomIndexWriter.java:405)
at 
org.apache.lucene.search.TestBoolean2.beforeClass(TestBoolean2.java:123)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError: Java heap space
at org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:78)
at org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:51)
at 
org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:164)
at 
org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:150)
at 
org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:141)
at 
org.apache.lucene.store.RateLimitedIndexOutput.writeBytes(RateLimitedIndexOutput.java:73)
at org.apache.lucene.store.DataOutput.copyBytes(DataOutput.java:278)
at 
org.apache.lucene.codecs.simpletext.SimpleTextCompoundFormat.write(SimpleTextCompoundFormat.java:183)
at 
org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:4681)
at 
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4156)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3679)
at 
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
at 
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)


FAILED:  junit.framework.TestSuite.org.apache.lucene.search.TestBoolean2

Error Message:


Stack Trace:
java.lang.NullPointerException
at __randomizedtesting.SeedInfo.seed([A6A15104CA0B0101]:0)
at 
org.apache.lucene.search.TestBoolean2.afterClass(TestBoolean2.java:147)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce

[jira] [Commented] (LUCENE-7315) Flexible "standard" query parser parses on whitespace

[Steve Rowe (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15325156#comment-15325156
 ] 

Steve Rowe commented on LUCENE-7315:


Yes.

> Flexible "standard" query parser parses on whitespace
> -
>
> Key: LUCENE-7315
> URL: https://issues.apache.org/jira/browse/LUCENE-7315
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/queryparser
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>
> Copied from LUCENE-2605:
> The queryparser parses input on whitespace, and sends each whitespace 
> separated term to its own independent token stream.
> This breaks the following at query-time, because they can't see across 
> whitespace boundaries:
> n-gram analysis
> shingles
> synonyms (especially multi-word for whitespace-separated languages)
> languages where a 'word' can contain whitespace (e.g. vietnamese)
> Its also rather unexpected, as users think their 
> charfilters/tokenizers/tokenfilters will do the same thing at index and 
> querytime, but in many cases they can't. Instead, preferably the queryparser 
> would parse around only real 'operators'.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-7138) TestMinShouldMatch2 assertion failure

[Michael McCandless (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless resolved LUCENE-7138.

Resolution: Duplicate

Dup of LUCENE-7132

> TestMinShouldMatch2 assertion failure
> -
>
> Key: LUCENE-7138
> URL: https://issues.apache.org/jira/browse/LUCENE-7138
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
>
> I randomly hit this test failure:
> {noformat}
> ant test  -Dtestcase=TestMinShouldMatch2 -Dtests.seed=D712D2CC986DDBA 
> -Dtests.multiplier=3 -Dtests.nightly=true -Dtests.locale=ar-BH 
> -Dtests.timezone=Etc/GMT-9 -Dtests.asserts=true 
> -Dtests.file.encoding=ISO-8859-1
> {noformat}
> {noformat}
>[junit4]> Throwable #1: java.lang.AssertionError: 
> expected:<0.559075117111206> but was:<0.06988438963890076>
>[junit4]>  at 
> __randomizedtesting.SeedInfo.seed([D712D2CC986DDBA:F2A0B1A99368A5DB]:0)
>[junit4]>  at 
> org.apache.lucene.search.TestMinShouldMatch2.assertNext(TestMinShouldMatch2.java:161)
>[junit4]>  at 
> org.apache.lucene.search.TestMinShouldMatch2.testNextVaryingNumberOfTerms(TestMinShouldMatch2.java:278)
>[junit4]>  at java.lang.Thread.run(Thread.java:745)
>[junit4]   2> NOTE: test params are: codec=Asserting(Lucene60): 
> {field=Lucene50(blocksize=128)}, 
> docValues:{dv=DocValuesFormat(name=Lucene54)}, maxPointsInLeafNode=367, 
> maxMBSortInHeap=4.737709757956011, 
> sim=RandomSimilarity(queryNorm=true,coord=yes): {}, locale=ar-BH, 
> timezone=Etc/GMT-9
>[junit4]   2> NOTE: Linux 3.13.0-71-generic amd64/Oracle Corporation 
> 1.8.0_60 (64-bit)/cpus=8,threads=1,free=441776504,total=516947968
>[junit4]   2> NOTE: All tests run in this JVM: [TestMinShouldMatch2]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7315) Flexible "standard" query parser parses on whitespace

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15325150#comment-15325150
 ] 

Michael McCandless commented on LUCENE-7315:


OK I see: this issue is about making the same fixes in LUCENE-2605, which was 
for the classic query parser, to the flexible query parser.

> Flexible "standard" query parser parses on whitespace
> -
>
> Key: LUCENE-7315
> URL: https://issues.apache.org/jira/browse/LUCENE-7315
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/queryparser
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>
> Copied from LUCENE-2605:
> The queryparser parses input on whitespace, and sends each whitespace 
> separated term to its own independent token stream.
> This breaks the following at query-time, because they can't see across 
> whitespace boundaries:
> n-gram analysis
> shingles
> synonyms (especially multi-word for whitespace-separated languages)
> languages where a 'word' can contain whitespace (e.g. vietnamese)
> Its also rather unexpected, as users think their 
> charfilters/tokenizers/tokenfilters will do the same thing at index and 
> querytime, but in many cases they can't. Instead, preferably the queryparser 
> would parse around only real 'operators'.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7315) Flexible "standard" query parser parses on whitespace

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15325147#comment-15325147
 ] 

Michael McCandless commented on LUCENE-7315:


How does this issue differ from LUCENE-2605?

> Flexible "standard" query parser parses on whitespace
> -
>
> Key: LUCENE-7315
> URL: https://issues.apache.org/jira/browse/LUCENE-7315
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/queryparser
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>
> Copied from LUCENE-2605:
> The queryparser parses input on whitespace, and sends each whitespace 
> separated term to its own independent token stream.
> This breaks the following at query-time, because they can't see across 
> whitespace boundaries:
> n-gram analysis
> shingles
> synonyms (especially multi-word for whitespace-separated languages)
> languages where a 'word' can contain whitespace (e.g. vietnamese)
> Its also rather unexpected, as users think their 
> charfilters/tokenizers/tokenfilters will do the same thing at index and 
> querytime, but in many cases they can't. Instead, preferably the queryparser 
> would parse around only real 'operators'.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 639 - Still Failing!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/639/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.update.HardAutoCommitTest.testCommitWithin

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
at 
__randomizedtesting.SeedInfo.seed([C556AE78FEB1CE6E:7F84C1007D9F207B]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:780)
at 
org.apache.solr.update.HardAutoCommitTest.testCommitWithin(HardAutoCommitTest.java:99)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: 
xpath=//result[@numFound=1]
xml response was: 

00


request was:q=id:529&qt=standard&start=0&rows=20&version=2.2
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:773)
... 40 more




Build Log:
[...truncated 11515 lines...]
   [junit4] Suite

[jira] [Commented] (SOLR-8744) Overseer operations need more fine grained mutual exclusion

[Scott Blum (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15325119#comment-15325119
 ] 

Scott Blum commented on SOLR-8744:
--

That said, if you'd feel more comfortable committing your patch as-is, I can 
toss my tweaks down after.

> Overseer operations need more fine grained mutual exclusion
> ---
>
> Key: SOLR-8744
> URL: https://issues.apache.org/jira/browse/SOLR-8744
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Affects Versions: 5.4.1
>Reporter: Scott Blum
>Assignee: Noble Paul
>Priority: Blocker
>  Labels: sharding, solrcloud
> Fix For: 6.1
>
> Attachments: SOLR-8744.patch, SOLR-8744.patch, SOLR-8744.patch, 
> SOLR-8744.patch, SOLR-8744.patch, SOLR-8744.patch, SOLR-8744.patch, 
> SOLR-8744.patch, SOLR-8744.patch, SmileyLockTree.java, SmileyLockTree.java
>
>
> SplitShard creates a mutex over the whole collection, but, in practice, this 
> is a big scaling problem.  Multiple split shard operations could happen at 
> the time time, as long as different shards are being split.  In practice, 
> those shards often reside on different machines, so there's no I/O bottleneck 
> in those cases, just the mutex in Overseer forcing the operations to be done 
> serially.
> Given that a single split can take many minutes on a large collection, this 
> is a bottleneck at scale.
> Here is the proposed new design
> There are various Collection operations performed at Overseer. They may need 
> exclusive access at various levels. Each operation must define the Access 
> level at which the access is required. Access level is an enum. 
> CLUSTER(0)
> COLLECTION(1)
> SHARD(2)
> REPLICA(3)
> The Overseer node maintains a tree of these locks. The lock tree would look 
> as follows. The tree can be created lazily as and when tasks come up.
> {code}
> Legend: 
> C1, C2 -> Collections
> S1, S2 -> Shards 
> R1,R2,R3,R4 -> Replicas
>  Cluster
> /   \
>/ \ 
>   C1  C2
>  / \ /   \ 
> /   \   / \  
>S1   S2  S1 S2
> R1, R2  R3.R4  R1,R2   R3,R4
> {code}
> When the overseer receives a message, it tries to acquire the appropriate 
> lock from the tree. For example, if an operation needs a lock at a Collection 
> level and it needs to operate on Collection C1, the node C1 and all child 
> nodes of C1 must be free. 
> h2.Lock acquiring logic
> Each operation would start from the root of the tree (Level 0 -> Cluster) and 
> start moving down depending upon the operation. After it reaches the right 
> node, it checks if all the children are free from a lock.  If it fails to 
> acquire a lock, it remains in the work queue. A scheduler thread waits for 
> notification from the current set of tasks . Every task would do a 
> {{notify()}} on the monitor of  the scheduler thread. The thread would start 
> from the head of the queue and check all tasks to see if that task is able to 
> acquire the right lock. If yes, it is executed, if not, the task is left in 
> the work queue.  
> When a new task arrives in the work queue, the schedulerthread wakes and just 
> try to schedule that task.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-7374) Backup/Restore should provide a param for specifying the directory implementation it should use

[Hrishikesh Gadre (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-7374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hrishikesh Gadre updated SOLR-7374:
---
Attachment: SOLR-7374.patch

[~markrmil...@gmail.com] Please find the updated patch which resolves the 
TestReplicationHandlerBackup failure.

> Backup/Restore should provide a param for specifying the directory 
> implementation it should use
> ---
>
> Key: SOLR-7374
> URL: https://issues.apache.org/jira/browse/SOLR-7374
> Project: Solr
>  Issue Type: Bug
>Reporter: Varun Thacker
>Assignee: Mark Miller
> Fix For: 5.2, 6.0
>
> Attachments: SOLR-7374.patch, SOLR-7374.patch, SOLR-7374.patch, 
> SOLR-7374.patch
>
>
> Currently when we create a backup we use SimpleFSDirectory to write the 
> backup indexes. Similarly during a restore we open the index using 
> FSDirectory.open . 
> We should provide a param called {{directoryImpl}} or {{type}} which will be 
> used to specify the Directory implementation to backup the index. 
> Likewise during a restore you would need to specify the directory impl which 
> was used during backup so that the index can be opened correctly.
> This param will address the problem that currently if a user is running Solr 
> on HDFS there is no way to use the backup/restore functionality as the 
> directory is hardcoded.
> With this one could be running Solr on a local FS but backup the index on 
> HDFS etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-7332) Better propagate two-phase iterators in conjunctions

[Adrien Grand (JIRA)]

Adrien Grand created LUCENE-7332:


 Summary: Better propagate two-phase iterators in conjunctions
 Key: LUCENE-7332
 URL: https://issues.apache.org/jira/browse/LUCENE-7332
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Adrien Grand
Assignee: Adrien Grand
Priority: Minor


I suspect it is very common to have a single two-phase iterator to check in a 
query tree. Yet, even in that case, the way the API works currently forces 
ConjunctionDISI to wrap it anyway in order to propagate it to the higher level. 
I'd like to explore changing the API so that the top-level scorer would 
directly call the two-phase iterator instead of going through multiple wrappers.

Then we might be able to add the same RandomAccessStratogy optimization as 
FilteredQuery had in case a two-phase iterator supports random-access and is 
expected to match many documents.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-MacOSX (64bit/jdk1.8.0) - Build # 198 - Still Failing!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-MacOSX/198/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

All tests passed

Build Log:
[...truncated 12155 lines...]
   [junit4] JVM J1: stdout was not empty, see: 
/Users/jenkins/workspace/Lucene-Solr-6.x-MacOSX/solr/build/solr-core/test/temp/junit4-J1-20160610_170032_836.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) 
   [junit4] #
   [junit4] # A fatal error has been detected by the Java Runtime Environment:
   [junit4] #
   [junit4] #  Internal Error (sharedRuntime.cpp:873), pid=75233, tid=71491
   [junit4] #  guarantee(nm != NULL) failed: must have containing nmethod for 
implicit division-by-zero exceptions
   [junit4] #
   [junit4] # JRE version: Java(TM) SE Runtime Environment (8.0_72-b15) (build 
1.8.0_72-b15)
   [junit4] # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.72-b15 mixed mode 
bsd-amd64 )
   [junit4] # Failed to write core dump. Core dumps have been disabled. To 
enable core dumping, try "ulimit -c unlimited" before starting Java again
   [junit4] #
   [junit4] # An error report file with more information is saved as:
   [junit4] # 
/Users/jenkins/workspace/Lucene-Solr-6.x-MacOSX/solr/build/solr-core/test/J1/hs_err_pid75233.log
   [junit4] #
   [junit4] # If you would like to submit a bug report, please visit:
   [junit4] #   http://bugreport.java.com/bugreport/crash.jsp
   [junit4] #
   [junit4] <<< JVM J1: EOF 

[...truncated 313 lines...]
   [junit4] ERROR: JVM J1 ended with an exception, command line: 
/Library/Java/JavaVirtualMachines/jdk1.8.0_72.jdk/Contents/Home/jre/bin/java 
-XX:-UseCompressedOops -XX:+UseConcMarkSweepGC -XX:+HeapDumpOnOutOfMemoryError 
-XX:HeapDumpPath=/Users/jenkins/workspace/Lucene-Solr-6.x-MacOSX/heapdumps -ea 
-esa -Dtests.prefix=tests -Dtests.seed=84EF2E6B2A42C57F -Xmx512M -Dtests.iters= 
-Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random 
-Dtests.postingsformat=random -Dtests.docvaluesformat=random 
-Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random 
-Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=6.2.0 
-Dtests.cleanthreads=perClass 
-Djava.util.logging.config.file=/Users/jenkins/workspace/Lucene-Solr-6.x-MacOSX/lucene/tools/junit4/logging.properties
 -Dtests.nightly=false -Dtests.weekly=false -Dtests.monster=false 
-Dtests.slow=true -Dtests.asserts=true -Dtests.multiplier=1 -DtempDir=./temp 
-Djava.io.tmpdir=./temp 
-Djunit4.tempDir=/Users/jenkins/workspace/Lucene-Solr-6.x-MacOSX/solr/build/solr-core/test/temp
 -Dcommon.dir=/Users/jenkins/workspace/Lucene-Solr-6.x-MacOSX/lucene 
-Dclover.db.dir=/Users/jenkins/workspace/Lucene-Solr-6.x-MacOSX/lucene/build/clover/db
 
-Djava.security.policy=/Users/jenkins/workspace/Lucene-Solr-6.x-MacOSX/lucene/tools/junit4/solr-tests.policy
 -Dtests.LUCENE_VERSION=6.2.0 -Djetty.testMode=1 -Djetty.insecurerandom=1 
-Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory 
-Djava.awt.headless=true -Djdk.map.althashing.threshold=0 
-Djunit4.childvm.cwd=/Users/jenkins/workspace/Lucene-Solr-6.x-MacOSX/solr/build/solr-core/test/J1
 -Djunit4.childvm.id=1 -Djunit4.childvm.count=2 -Dtests.leaveTemporary=false 
-Dtests.filterstacks=true -Dtests.disableHdfs=true 
-Djava.security.manager=org.apache.lucene.util.TestSecurityManager -classpath 
/Users/jenkins/workspace/Lucene-Solr-6.x-MacOSX/solr/build/solr-core/classes/test:/Users/jenkins/workspace/Lucene-Solr-6.x-MacOSX/solr/build/solr-test-framework/classes/java:/Users/jenkins/workspace/Lucene-Solr-6.x-MacOSX/solr/test-framework/lib/junit4-ant-2.3.4.jar:/Users/jenkins/workspace/Lucene-Solr-6.x-MacOSX/solr/core/src/test-files:/Users/jenkins/workspace/Lucene-Solr-6.x-MacOSX/lucene/build/test-framework/classes/java:/Users/jenkins/workspace/Lucene-Solr-6.x-MacOSX/lucene/build/codecs/classes/java:/Users/jenkins/workspace/Lucene-Solr-6.x-MacOSX/solr/build/solr-solrj/classes/java:/Users/jenkins/workspace/Lucene-Solr-6.x-MacOSX/solr/build/solr-core/classes/java:/Users/jenkins/workspace/Lucene-Solr-6.x-MacOSX/lucene/build/analysis/common/lucene-analyzers-common-6.2.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-6.x-MacOSX/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-6.2.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-6.x-MacOSX/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-6.2.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-6.x-MacOSX/lucene/build/codecs/lucene-codecs-6.2.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-6.x-MacOSX/lucene/build/backward-codecs/lucene-backward-codecs-6.2.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-6.x-MacOSX/lucene/build/highlighter/lucene-highlighter-6.2.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-6.x-MacOSX/lucene/build/memory/lucene-memory-6.2.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-6.x-MacOSX/lucene/build/misc/lucene-misc-6.2.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-6.x-MacOSX/lucene/build/spatial-extras/luce

[jira] [Commented] (LUCENE-7331) GeoPointTestQuery should use GeoTestUtil instead of GeoPointTestUtil

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324951#comment-15324951
 ] 

ASF subversion and git services commented on LUCENE-7331:
-

Commit f767855da30e8d8b070b7566cb6eebb29af63334 in lucene-solr's branch 
refs/heads/master from [~nknize]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=f767855 ]

LUCENE-7331: Remove GeoPointTestUtil from TestGeoPointQuery.


> GeoPointTestQuery should use GeoTestUtil instead of GeoPointTestUtil
> 
>
> Key: LUCENE-7331
> URL: https://issues.apache.org/jira/browse/LUCENE-7331
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: 6.1
>Reporter: Nicholas Knize
> Attachments: LUCENE-7331.patch
>
>
> Initially discussed in LUCENE-7166, and revisited in LUCENE-7325, 
> {{TestGeoPointQuery}} should use the RNG provided by {{GeoTestUtil}} instead 
> of the {{GeoPointTestUtil}} hack.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-7291) HeatmapFacetCounter bug with dateline and large non-point shapes

[David Smiley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley resolved LUCENE-7291.
--
   Resolution: Fixed
Fix Version/s: 6.1

> HeatmapFacetCounter bug with dateline and large non-point shapes
> 
>
> Key: LUCENE-7291
> URL: https://issues.apache.org/jira/browse/LUCENE-7291
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/spatial-extras
>Reporter: David Smiley
>Assignee: David Smiley
> Fix For: 6.1
>
> Attachments: LUCENE_7291.patch
>
>
> Jenkins found a test failure today.
> This reproduces for me (master, java 8):
> ant test  -Dtestcase=HeatmapFacetCounterTest -Dtests.method=testRandom 
> -Dtests.seed=3EC907D1784B6F23 -Dtests.multiplier=2 -Dtests.nightly=true 
> -Dtests.slow=true 
> -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt 
> -Dtests.locale=is-IS -Dtests.timezone=Europe/Tirane -Dtests.asserts=true 
> -Dtests.file.encoding=UTF-8
> {noformat}
> java.lang.AssertionError: 
> Expected :1
> Actual   :0
>  
>   at 
> __randomizedtesting.SeedInfo.seed([3EC907D1784B6F23:A3439C5F68FEAB94]:0)
>   at org.junit.Assert.fail(Assert.java:93)
>   at org.junit.Assert.failNotEquals(Assert.java:647)
>   at org.junit.Assert.assertEquals(Assert.java:128)
>   at org.junit.Assert.assertEquals(Assert.java:472)
>   at org.junit.Assert.assertEquals(Assert.java:456)
>   at 
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.validateHeatmapResult(HeatmapFacetCounterTest.java:226)
>   at 
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:193)
>   at 
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:206)
>   at 
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.testRandom(HeatmapFacetCounterTest.java:172)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7291) HeatmapFacetCounter bug with dateline and large non-point shapes

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324937#comment-15324937
 ] 

ASF subversion and git services commented on LUCENE-7291:
-

Commit 6372c0b4042ec2c8d94e50e5c2b9c1df469414e2 in lucene-solr's branch 
refs/heads/branch_6_1 from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=6372c0b ]

LUCENE-7291: Fix spatial HeatmapFacetCounter bug with dateline and large 
non-point shapes
(cherry picked from commit 7520d79)


> HeatmapFacetCounter bug with dateline and large non-point shapes
> 
>
> Key: LUCENE-7291
> URL: https://issues.apache.org/jira/browse/LUCENE-7291
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/spatial-extras
>Reporter: David Smiley
>Assignee: David Smiley
> Attachments: LUCENE_7291.patch
>
>
> Jenkins found a test failure today.
> This reproduces for me (master, java 8):
> ant test  -Dtestcase=HeatmapFacetCounterTest -Dtests.method=testRandom 
> -Dtests.seed=3EC907D1784B6F23 -Dtests.multiplier=2 -Dtests.nightly=true 
> -Dtests.slow=true 
> -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt 
> -Dtests.locale=is-IS -Dtests.timezone=Europe/Tirane -Dtests.asserts=true 
> -Dtests.file.encoding=UTF-8
> {noformat}
> java.lang.AssertionError: 
> Expected :1
> Actual   :0
>  
>   at 
> __randomizedtesting.SeedInfo.seed([3EC907D1784B6F23:A3439C5F68FEAB94]:0)
>   at org.junit.Assert.fail(Assert.java:93)
>   at org.junit.Assert.failNotEquals(Assert.java:647)
>   at org.junit.Assert.assertEquals(Assert.java:128)
>   at org.junit.Assert.assertEquals(Assert.java:472)
>   at org.junit.Assert.assertEquals(Assert.java:456)
>   at 
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.validateHeatmapResult(HeatmapFacetCounterTest.java:226)
>   at 
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:193)
>   at 
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:206)
>   at 
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.testRandom(HeatmapFacetCounterTest.java:172)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-7331) GeoPointTestQuery should use GeoTestUtil instead of GeoPointTestUtil

[Nicholas Knize (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicholas Knize resolved LUCENE-7331.

Resolution: Fixed

> GeoPointTestQuery should use GeoTestUtil instead of GeoPointTestUtil
> 
>
> Key: LUCENE-7331
> URL: https://issues.apache.org/jira/browse/LUCENE-7331
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: 6.1
>Reporter: Nicholas Knize
> Attachments: LUCENE-7331.patch
>
>
> Initially discussed in LUCENE-7166, and revisited in LUCENE-7325, 
> {{TestGeoPointQuery}} should use the RNG provided by {{GeoTestUtil}} instead 
> of the {{GeoPointTestUtil}} hack.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7291) HeatmapFacetCounter bug with dateline and large non-point shapes

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324929#comment-15324929
 ] 

ASF subversion and git services commented on LUCENE-7291:
-

Commit 7520d79e040c16c5ab666f1ad28c8665fb0ceb40 in lucene-solr's branch 
refs/heads/branch_6x from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=7520d79 ]

LUCENE-7291: Fix spatial HeatmapFacetCounter bug with dateline and large 
non-point shapes
(cherry picked from commit b33d717)


> HeatmapFacetCounter bug with dateline and large non-point shapes
> 
>
> Key: LUCENE-7291
> URL: https://issues.apache.org/jira/browse/LUCENE-7291
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/spatial-extras
>Reporter: David Smiley
>Assignee: David Smiley
> Attachments: LUCENE_7291.patch
>
>
> Jenkins found a test failure today.
> This reproduces for me (master, java 8):
> ant test  -Dtestcase=HeatmapFacetCounterTest -Dtests.method=testRandom 
> -Dtests.seed=3EC907D1784B6F23 -Dtests.multiplier=2 -Dtests.nightly=true 
> -Dtests.slow=true 
> -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt 
> -Dtests.locale=is-IS -Dtests.timezone=Europe/Tirane -Dtests.asserts=true 
> -Dtests.file.encoding=UTF-8
> {noformat}
> java.lang.AssertionError: 
> Expected :1
> Actual   :0
>  
>   at 
> __randomizedtesting.SeedInfo.seed([3EC907D1784B6F23:A3439C5F68FEAB94]:0)
>   at org.junit.Assert.fail(Assert.java:93)
>   at org.junit.Assert.failNotEquals(Assert.java:647)
>   at org.junit.Assert.assertEquals(Assert.java:128)
>   at org.junit.Assert.assertEquals(Assert.java:472)
>   at org.junit.Assert.assertEquals(Assert.java:456)
>   at 
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.validateHeatmapResult(HeatmapFacetCounterTest.java:226)
>   at 
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:193)
>   at 
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:206)
>   at 
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.testRandom(HeatmapFacetCounterTest.java:172)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8744) Overseer operations need more fine grained mutual exclusion

[Scott Blum (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324928#comment-15324928
 ] 

Scott Blum commented on SOLR-8744:
--

Attached a slightly modified patchfile with the changes I suggested.
Otherwise LGTM

> Overseer operations need more fine grained mutual exclusion
> ---
>
> Key: SOLR-8744
> URL: https://issues.apache.org/jira/browse/SOLR-8744
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Affects Versions: 5.4.1
>Reporter: Scott Blum
>Assignee: Noble Paul
>Priority: Blocker
>  Labels: sharding, solrcloud
> Fix For: 6.1
>
> Attachments: SOLR-8744.patch, SOLR-8744.patch, SOLR-8744.patch, 
> SOLR-8744.patch, SOLR-8744.patch, SOLR-8744.patch, SOLR-8744.patch, 
> SOLR-8744.patch, SOLR-8744.patch, SmileyLockTree.java, SmileyLockTree.java
>
>
> SplitShard creates a mutex over the whole collection, but, in practice, this 
> is a big scaling problem.  Multiple split shard operations could happen at 
> the time time, as long as different shards are being split.  In practice, 
> those shards often reside on different machines, so there's no I/O bottleneck 
> in those cases, just the mutex in Overseer forcing the operations to be done 
> serially.
> Given that a single split can take many minutes on a large collection, this 
> is a bottleneck at scale.
> Here is the proposed new design
> There are various Collection operations performed at Overseer. They may need 
> exclusive access at various levels. Each operation must define the Access 
> level at which the access is required. Access level is an enum. 
> CLUSTER(0)
> COLLECTION(1)
> SHARD(2)
> REPLICA(3)
> The Overseer node maintains a tree of these locks. The lock tree would look 
> as follows. The tree can be created lazily as and when tasks come up.
> {code}
> Legend: 
> C1, C2 -> Collections
> S1, S2 -> Shards 
> R1,R2,R3,R4 -> Replicas
>  Cluster
> /   \
>/ \ 
>   C1  C2
>  / \ /   \ 
> /   \   / \  
>S1   S2  S1 S2
> R1, R2  R3.R4  R1,R2   R3,R4
> {code}
> When the overseer receives a message, it tries to acquire the appropriate 
> lock from the tree. For example, if an operation needs a lock at a Collection 
> level and it needs to operate on Collection C1, the node C1 and all child 
> nodes of C1 must be free. 
> h2.Lock acquiring logic
> Each operation would start from the root of the tree (Level 0 -> Cluster) and 
> start moving down depending upon the operation. After it reaches the right 
> node, it checks if all the children are free from a lock.  If it fails to 
> acquire a lock, it remains in the work queue. A scheduler thread waits for 
> notification from the current set of tasks . Every task would do a 
> {{notify()}} on the monitor of  the scheduler thread. The thread would start 
> from the head of the queue and check all tasks to see if that task is able to 
> acquire the right lock. If yes, it is executed, if not, the task is left in 
> the work queue.  
> When a new task arrives in the work queue, the schedulerthread wakes and just 
> try to schedule that task.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8744) Overseer operations need more fine grained mutual exclusion

[Scott Blum (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Blum updated SOLR-8744:
-
Attachment: SOLR-8744.patch

> Overseer operations need more fine grained mutual exclusion
> ---
>
> Key: SOLR-8744
> URL: https://issues.apache.org/jira/browse/SOLR-8744
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Affects Versions: 5.4.1
>Reporter: Scott Blum
>Assignee: Noble Paul
>Priority: Blocker
>  Labels: sharding, solrcloud
> Fix For: 6.1
>
> Attachments: SOLR-8744.patch, SOLR-8744.patch, SOLR-8744.patch, 
> SOLR-8744.patch, SOLR-8744.patch, SOLR-8744.patch, SOLR-8744.patch, 
> SOLR-8744.patch, SOLR-8744.patch, SmileyLockTree.java, SmileyLockTree.java
>
>
> SplitShard creates a mutex over the whole collection, but, in practice, this 
> is a big scaling problem.  Multiple split shard operations could happen at 
> the time time, as long as different shards are being split.  In practice, 
> those shards often reside on different machines, so there's no I/O bottleneck 
> in those cases, just the mutex in Overseer forcing the operations to be done 
> serially.
> Given that a single split can take many minutes on a large collection, this 
> is a bottleneck at scale.
> Here is the proposed new design
> There are various Collection operations performed at Overseer. They may need 
> exclusive access at various levels. Each operation must define the Access 
> level at which the access is required. Access level is an enum. 
> CLUSTER(0)
> COLLECTION(1)
> SHARD(2)
> REPLICA(3)
> The Overseer node maintains a tree of these locks. The lock tree would look 
> as follows. The tree can be created lazily as and when tasks come up.
> {code}
> Legend: 
> C1, C2 -> Collections
> S1, S2 -> Shards 
> R1,R2,R3,R4 -> Replicas
>  Cluster
> /   \
>/ \ 
>   C1  C2
>  / \ /   \ 
> /   \   / \  
>S1   S2  S1 S2
> R1, R2  R3.R4  R1,R2   R3,R4
> {code}
> When the overseer receives a message, it tries to acquire the appropriate 
> lock from the tree. For example, if an operation needs a lock at a Collection 
> level and it needs to operate on Collection C1, the node C1 and all child 
> nodes of C1 must be free. 
> h2.Lock acquiring logic
> Each operation would start from the root of the tree (Level 0 -> Cluster) and 
> start moving down depending upon the operation. After it reaches the right 
> node, it checks if all the children are free from a lock.  If it fails to 
> acquire a lock, it remains in the work queue. A scheduler thread waits for 
> notification from the current set of tasks . Every task would do a 
> {{notify()}} on the monitor of  the scheduler thread. The thread would start 
> from the head of the queue and check all tasks to see if that task is able to 
> acquire the right lock. If yes, it is executed, if not, the task is left in 
> the work queue.  
> When a new task arrives in the work queue, the schedulerthread wakes and just 
> try to schedule that task.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7331) GeoPointTestQuery should use GeoTestUtil instead of GeoPointTestUtil

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324926#comment-15324926
 ] 

ASF subversion and git services commented on LUCENE-7331:
-

Commit 1bbac6bbd896c110d08656f79fef3ce6d7828d6b in lucene-solr's branch 
refs/heads/branch_6_1 from [~nknize]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1bbac6b ]

LUCENE-7331: Remove GeoPointTestUtil from TestGeoPointQuery.


> GeoPointTestQuery should use GeoTestUtil instead of GeoPointTestUtil
> 
>
> Key: LUCENE-7331
> URL: https://issues.apache.org/jira/browse/LUCENE-7331
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: 6.1
>Reporter: Nicholas Knize
> Attachments: LUCENE-7331.patch
>
>
> Initially discussed in LUCENE-7166, and revisited in LUCENE-7325, 
> {{TestGeoPointQuery}} should use the RNG provided by {{GeoTestUtil}} instead 
> of the {{GeoPointTestUtil}} hack.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7291) HeatmapFacetCounter bug with dateline and large non-point shapes

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324915#comment-15324915
 ] 

ASF subversion and git services commented on LUCENE-7291:
-

Commit b33d7176aa3624df2de1708b17919f20d034872f in lucene-solr's branch 
refs/heads/master from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b33d717 ]

LUCENE-7291: Fix spatial HeatmapFacetCounter bug with dateline and large 
non-point shapes


> HeatmapFacetCounter bug with dateline and large non-point shapes
> 
>
> Key: LUCENE-7291
> URL: https://issues.apache.org/jira/browse/LUCENE-7291
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/spatial-extras
>Reporter: David Smiley
>Assignee: David Smiley
> Attachments: LUCENE_7291.patch
>
>
> Jenkins found a test failure today.
> This reproduces for me (master, java 8):
> ant test  -Dtestcase=HeatmapFacetCounterTest -Dtests.method=testRandom 
> -Dtests.seed=3EC907D1784B6F23 -Dtests.multiplier=2 -Dtests.nightly=true 
> -Dtests.slow=true 
> -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt 
> -Dtests.locale=is-IS -Dtests.timezone=Europe/Tirane -Dtests.asserts=true 
> -Dtests.file.encoding=UTF-8
> {noformat}
> java.lang.AssertionError: 
> Expected :1
> Actual   :0
>  
>   at 
> __randomizedtesting.SeedInfo.seed([3EC907D1784B6F23:A3439C5F68FEAB94]:0)
>   at org.junit.Assert.fail(Assert.java:93)
>   at org.junit.Assert.failNotEquals(Assert.java:647)
>   at org.junit.Assert.assertEquals(Assert.java:128)
>   at org.junit.Assert.assertEquals(Assert.java:472)
>   at org.junit.Assert.assertEquals(Assert.java:456)
>   at 
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.validateHeatmapResult(HeatmapFacetCounterTest.java:226)
>   at 
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:193)
>   at 
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:206)
>   at 
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.testRandom(HeatmapFacetCounterTest.java:172)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-7325) GeoPointInBBoxQuery no longer includes boundaries?

[Adrien Grand (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adrien Grand resolved LUCENE-7325.
--
Resolution: Not A Bug

Agreed: closing. Thanks for having looked into it!

> GeoPointInBBoxQuery no longer includes boundaries?
> --
>
> Key: LUCENE-7325
> URL: https://issues.apache.org/jira/browse/LUCENE-7325
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 6.1
>Reporter: Michael McCandless
>Priority: Blocker
> Attachments: LUCENE-7325.patch, LUCENE-7325.patch
>
>
> {{GeoPointInBBoxQuery}} is supposed to include boundaries, and it does in 5.x 
> and 6.0, but in 6.1 something broke.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7331) GeoPointTestQuery should use GeoTestUtil instead of GeoPointTestUtil

[Adrien Grand (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324901#comment-15324901
 ] 

Adrien Grand commented on LUCENE-7331:
--

+1

> GeoPointTestQuery should use GeoTestUtil instead of GeoPointTestUtil
> 
>
> Key: LUCENE-7331
> URL: https://issues.apache.org/jira/browse/LUCENE-7331
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: 6.1
>Reporter: Nicholas Knize
> Attachments: LUCENE-7331.patch
>
>
> Initially discussed in LUCENE-7166, and revisited in LUCENE-7325, 
> {{TestGeoPointQuery}} should use the RNG provided by {{GeoTestUtil}} instead 
> of the {{GeoPointTestUtil}} hack.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Lucene/Solr 6.1.0

[Adrien Grand]

Thanks David.

Le ven. 10 juin 2016 à 19:28, David Smiley  a
écrit :

> I'm getting LUCENE-7291 done now too.  A rare heatmap faceting bug.
>
> On Fri, Jun 10, 2016 at 10:38 AM Adrien Grand  wrote:
>
>> There are still two blockers for 6.1, so I will wait until next week to
>> build a first RC:
>>  - https://issues.apache.org/jira/browse/SOLR-8744: Noble, is my
>> understanding correct that this one is almost ready?
>>  - https://issues.apache.org/jira/browse/LUCENE-7325: There is an
>> ongoing discussion about whether this is a blocker, and what needs to be
>> done but hopefully this will be sorted out soon.
>>
>> Le jeu. 9 juin 2016 à 08:18, Adrien Grand  a écrit :
>>
>>> Awesome, thanks Steve!
>>>
>>> Le jeu. 9 juin 2016 à 01:45, Steve Rowe  a écrit :
>>>
 Hi Adrien,

 I remembered that I never fixed up the CHANGES.txt files on master and
 branch_6x after the 6.0.1 release to move the 6.0.1 changes out of the
 6.1.0 section into the 6.0.1 section, so I went and did that, and
 backported to branch_6_1.

 I set up the ASF Jenkins jobs for 6.1 according to <
 http://wiki.apache.org/lucene-java/JenkinsReleaseBuilds>.

 I also set up the Policeman Jenkins jobs for 6.1 by cloning the 6.0
 Linux and Windows jobs[1], switching to */branch_6_1, enabling them, then
 triggering them as post-build actions from the corresponding master jobs.
 Uwe, apologies if I screwed this up :) - please fix if so!

 [1] I didn’t set up OSX and Solaris jobs because I didn’t see those for
 the 5.5 and 6.0 jobs.

 --
 Steve
 www.lucidworks.com

 > On Jun 8, 2016, at 8:38 AM, Adrien Grand  wrote:
 >
 >
 > I just created the 6.1.0 branch. Can someone help me with setting up
 Apache and Policeman Jenkins to test this branch?
 >
 > I still plan on cutting a first RC on Friday (in two days), please
 let me know if you know about issues that should be included in 6.1.0.


 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org

 --
> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
> http://www.solrenterprisesearchserver.com
>

[jira] [Commented] (SOLR-8744) Overseer operations need more fine grained mutual exclusion

[Scott Blum (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324896#comment-15324896
 ] 

Scott Blum commented on SOLR-8744:
--

Also, add blockedTasks to printTrackingMaps()?

> Overseer operations need more fine grained mutual exclusion
> ---
>
> Key: SOLR-8744
> URL: https://issues.apache.org/jira/browse/SOLR-8744
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Affects Versions: 5.4.1
>Reporter: Scott Blum
>Assignee: Noble Paul
>Priority: Blocker
>  Labels: sharding, solrcloud
> Fix For: 6.1
>
> Attachments: SOLR-8744.patch, SOLR-8744.patch, SOLR-8744.patch, 
> SOLR-8744.patch, SOLR-8744.patch, SOLR-8744.patch, SOLR-8744.patch, 
> SOLR-8744.patch, SmileyLockTree.java, SmileyLockTree.java
>
>
> SplitShard creates a mutex over the whole collection, but, in practice, this 
> is a big scaling problem.  Multiple split shard operations could happen at 
> the time time, as long as different shards are being split.  In practice, 
> those shards often reside on different machines, so there's no I/O bottleneck 
> in those cases, just the mutex in Overseer forcing the operations to be done 
> serially.
> Given that a single split can take many minutes on a large collection, this 
> is a bottleneck at scale.
> Here is the proposed new design
> There are various Collection operations performed at Overseer. They may need 
> exclusive access at various levels. Each operation must define the Access 
> level at which the access is required. Access level is an enum. 
> CLUSTER(0)
> COLLECTION(1)
> SHARD(2)
> REPLICA(3)
> The Overseer node maintains a tree of these locks. The lock tree would look 
> as follows. The tree can be created lazily as and when tasks come up.
> {code}
> Legend: 
> C1, C2 -> Collections
> S1, S2 -> Shards 
> R1,R2,R3,R4 -> Replicas
>  Cluster
> /   \
>/ \ 
>   C1  C2
>  / \ /   \ 
> /   \   / \  
>S1   S2  S1 S2
> R1, R2  R3.R4  R1,R2   R3,R4
> {code}
> When the overseer receives a message, it tries to acquire the appropriate 
> lock from the tree. For example, if an operation needs a lock at a Collection 
> level and it needs to operate on Collection C1, the node C1 and all child 
> nodes of C1 must be free. 
> h2.Lock acquiring logic
> Each operation would start from the root of the tree (Level 0 -> Cluster) and 
> start moving down depending upon the operation. After it reaches the right 
> node, it checks if all the children are free from a lock.  If it fails to 
> acquire a lock, it remains in the work queue. A scheduler thread waits for 
> notification from the current set of tasks . Every task would do a 
> {{notify()}} on the monitor of  the scheduler thread. The thread would start 
> from the head of the queue and check all tasks to see if that task is able to 
> acquire the right lock. If yes, it is executed, if not, the task is left in 
> the work queue.  
> When a new task arrives in the work queue, the schedulerthread wakes and just 
> try to schedule that task.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8744) Overseer operations need more fine grained mutual exclusion

[Scott Blum (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324891#comment-15324891
 ] 

Scott Blum commented on SOLR-8744:
--

LG.  I only have one suggestion left, to formulate the "fetch" section like 
this:

{code}
  ArrayList heads = new ArrayList<>(blockedTasks.size() + 
MAX_PARALLEL_TASKS);
  heads.addAll(blockedTasks.values());

  // If we have enough items in the blocked tasks already, it makes
  // no sense to read more items from the work queue. it makes sense
  // to clear out at least a few items in the queue before we read more 
items
  if (heads.size() < MAX_BLOCKED_TASKS) {
//instead of reading MAX_PARALLEL_TASKS items always, we should 
only fetch as much as we can execute
int toFetch = Math.min(MAX_BLOCKED_TASKS - heads.size(), 
MAX_PARALLEL_TASKS - runningTasks.size());
List newTasks = workQueue.peekTopN(toFetch, 
excludedTasks, 2000L);
log.debug("Got {} tasks from work-queue : [{}]", newTasks.size(), 
newTasks.toString());
heads.addAll(newTasks);
  } else {
Thread.sleep(1000);
  }

  if (isClosed) break;

  if (heads.isEmpty()) {
continue;
  }

  blockedTasks.clear(); // clear it now; may get refilled below.
{code}

This prevents two problems:

1) The log.debug message "Got {} tasks from work-queue" won't keep reporting 
blockedTasks as if they were freshly fetched.
2) When the blockedTask map gets completely full, the Thread.sleep() will 
prevent a free-spin (the only other real pause in the loop is peekTopN)

> Overseer operations need more fine grained mutual exclusion
> ---
>
> Key: SOLR-8744
> URL: https://issues.apache.org/jira/browse/SOLR-8744
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Affects Versions: 5.4.1
>Reporter: Scott Blum
>Assignee: Noble Paul
>Priority: Blocker
>  Labels: sharding, solrcloud
> Fix For: 6.1
>
> Attachments: SOLR-8744.patch, SOLR-8744.patch, SOLR-8744.patch, 
> SOLR-8744.patch, SOLR-8744.patch, SOLR-8744.patch, SOLR-8744.patch, 
> SOLR-8744.patch, SmileyLockTree.java, SmileyLockTree.java
>
>
> SplitShard creates a mutex over the whole collection, but, in practice, this 
> is a big scaling problem.  Multiple split shard operations could happen at 
> the time time, as long as different shards are being split.  In practice, 
> those shards often reside on different machines, so there's no I/O bottleneck 
> in those cases, just the mutex in Overseer forcing the operations to be done 
> serially.
> Given that a single split can take many minutes on a large collection, this 
> is a bottleneck at scale.
> Here is the proposed new design
> There are various Collection operations performed at Overseer. They may need 
> exclusive access at various levels. Each operation must define the Access 
> level at which the access is required. Access level is an enum. 
> CLUSTER(0)
> COLLECTION(1)
> SHARD(2)
> REPLICA(3)
> The Overseer node maintains a tree of these locks. The lock tree would look 
> as follows. The tree can be created lazily as and when tasks come up.
> {code}
> Legend: 
> C1, C2 -> Collections
> S1, S2 -> Shards 
> R1,R2,R3,R4 -> Replicas
>  Cluster
> /   \
>/ \ 
>   C1  C2
>  / \ /   \ 
> /   \   / \  
>S1   S2  S1 S2
> R1, R2  R3.R4  R1,R2   R3,R4
> {code}
> When the overseer receives a message, it tries to acquire the appropriate 
> lock from the tree. For example, if an operation needs a lock at a Collection 
> level and it needs to operate on Collection C1, the node C1 and all child 
> nodes of C1 must be free. 
> h2.Lock acquiring logic
> Each operation would start from the root of the tree (Level 0 -> Cluster) and 
> start moving down depending upon the operation. After it reaches the right 
> node, it checks if all the children are free from a lock.  If it fails to 
> acquire a lock, it remains in the work queue. A scheduler thread waits for 
> notification from the current set of tasks . Every task would do a 
> {{notify()}} on the monitor of  the scheduler thread. The thread would start 
> from the head of the queue and check all tasks to see if that task is able to 
> acquire the right lock. If yes, it is executed, if not, the task is left in 
> the work queue.  
> When a new task arrives in the work queue, the schedulerthread wakes and just 
> try to schedule that task.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-

[jira] [Commented] (LUCENE-7325) GeoPointInBBoxQuery no longer includes boundaries?

[Nicholas Knize (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324886#comment-15324886
 ] 

Nicholas Knize commented on LUCENE-7325:


I moved the bug fix to LUCENE-7331 since its not technically related to this 
issue. I think we can go ahead and close this as a non-issue?

> GeoPointInBBoxQuery no longer includes boundaries?
> --
>
> Key: LUCENE-7325
> URL: https://issues.apache.org/jira/browse/LUCENE-7325
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 6.1
>Reporter: Michael McCandless
>Priority: Blocker
> Attachments: LUCENE-7325.patch, LUCENE-7325.patch
>
>
> {{GeoPointInBBoxQuery}} is supposed to include boundaries, and it does in 5.x 
> and 6.0, but in 6.1 something broke.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7291) HeatmapFacetCounter bug with dateline and large non-point shapes

[David Smiley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley updated LUCENE-7291:
-
Summary: HeatmapFacetCounter bug with dateline and large non-point shapes  
(was: HeatmapFacetCounterTest.testRandom failure; random)

No I didn't; I already caught that and will commit that unchanged.

> HeatmapFacetCounter bug with dateline and large non-point shapes
> 
>
> Key: LUCENE-7291
> URL: https://issues.apache.org/jira/browse/LUCENE-7291
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/spatial-extras
>Reporter: David Smiley
>Assignee: David Smiley
> Attachments: LUCENE_7291.patch
>
>
> Jenkins found a test failure today.
> This reproduces for me (master, java 8):
> ant test  -Dtestcase=HeatmapFacetCounterTest -Dtests.method=testRandom 
> -Dtests.seed=3EC907D1784B6F23 -Dtests.multiplier=2 -Dtests.nightly=true 
> -Dtests.slow=true 
> -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt 
> -Dtests.locale=is-IS -Dtests.timezone=Europe/Tirane -Dtests.asserts=true 
> -Dtests.file.encoding=UTF-8
> {noformat}
> java.lang.AssertionError: 
> Expected :1
> Actual   :0
>  
>   at 
> __randomizedtesting.SeedInfo.seed([3EC907D1784B6F23:A3439C5F68FEAB94]:0)
>   at org.junit.Assert.fail(Assert.java:93)
>   at org.junit.Assert.failNotEquals(Assert.java:647)
>   at org.junit.Assert.assertEquals(Assert.java:128)
>   at org.junit.Assert.assertEquals(Assert.java:472)
>   at org.junit.Assert.assertEquals(Assert.java:456)
>   at 
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.validateHeatmapResult(HeatmapFacetCounterTest.java:226)
>   at 
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:193)
>   at 
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:206)
>   at 
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.testRandom(HeatmapFacetCounterTest.java:172)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: [JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_92) - Build # 239 - Still Failing!

[Uwe Schindler]

Here is the documentation:

 

https://msdn.microsoft.com/en-us/library/aa365247.aspx

 

We could either fix the issue by adding the list of illegal names to the random 
name generator, or we just let it be an let it fail sometimes :)

Should we open issue?

 

Uwe

 

-

Uwe Schindler

H.-H.-Meier-Allee 63, D-28213 Bremen

  http://www.thetaphi.de

eMail: u...@thetaphi.de

 

From: Uwe Schindler [mailto:u...@thetaphi.de] 
Sent: Friday, June 10, 2016 7:26 PM
To: dev@lucene.apache.org
Subject: RE: [JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_92) - Build # 
239 - Still Failing!

 

Hi,

 

Of course!

 

It is only strange that this error comes from the AccessController and not the 
operating system… So either the FilePermission code “knows” about this special 
names on windows, or it is a side effect while calculating canonic name of the 
file.

 

Uwe

 

-

Uwe Schindler

H.-H.-Meier-Allee 63, D-28213 Bremen

  http://www.thetaphi.de

eMail:   u...@thetaphi.de

 

From: Dawid Weiss [mailto:dawid.we...@gmail.com] 
Sent: Friday, June 10, 2016 6:59 PM
To: dev@lucene.apache.org  
Subject: RE: [JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_92) - Build # 
239 - Still Failing!

 


I think I do. Con is a special filename. You cannot use it on windows.

On Jun 10, 2016 5:50 PM, "Uwe Schindler" <  
u...@thetaphi.de> wrote:

Hi,

 

This is indeed very strange. I have no idea!

 

Uwe

 

-

Uwe Schindler

H.-H.-Meier-Allee 63, D-28213 Bremen

  http://www.thetaphi.de

eMail:   u...@thetaphi.de

 

From: Michael McCandless [mailto:  
luc...@mikemccandless.com] 
Sent: Friday, June 10, 2016 5:45 PM
To: Policeman Jenkins Server <  jenk...@thetaphi.de>
Cc: Adrien Grand <  jpou...@gmail.com>; Gregory 
Chanan <  gcha...@cloudera.com>;  
 dragonsi...@apache.org; Lucene/Solr dev < 
 dev@lucene.apache.org>
Subject: Re: [JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_92) - Build # 
239 - Still Failing!

 

Hmm, why was access denied here?




Mike McCandless

  http://blog.mikemccandless.com

 

On Thu, Jun 9, 2016 at 5:51 PM, Policeman Jenkins Server < 
 jenk...@thetaphi.de> wrote:

Build:   
http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Windows/239/
Java: 32bit/jdk1.8.0_92 -client -XX:+UseConcMarkSweepGC

2 tests failed.
FAILED:  org.apache.lucene.store.TestMmapDirectory.testPendingDeletions

Error Message:
access denied ("java.io.FilePermission" 
"C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\lucene\build\core\test\J0\temp\lucene.store.TestMmapDirectory_96FC6F2D45B76809-001\tempDir-007\con"
 "write")

Stack Trace:
java.security.AccessControlException: access denied ("java.io.FilePermission" 
"C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\lucene\build\core\test\J0\temp\lucene.store.TestMmapDirectory_96FC6F2D45B76809-001\tempDir-007\con"
 "write")
at 
__randomizedtesting.SeedInfo.seed([96FC6F2D45B76809:DE018553DD164AB]:0)
at 
java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
at 
java.security.AccessController.checkPermission(AccessController.java:884)
at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
at sun.nio.fs.WindowsChannelFactory.open(WindowsChannelFactory.java:295)
at 
sun.nio.fs.WindowsChannelFactory.newFileChannel(WindowsChannelFactory.java:162)
at 
sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:225)
at 
java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434)
at 
org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
at 
org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
at 
org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
at 
org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
at 
org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
at 
org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
at java.nio.file.Files.newOutputStream(Files.java:216)
at 
org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:408)
at 
org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:404)
at 
org.apache.lucene.s

[jira] [Updated] (LUCENE-7331) GeoPointTestQuery should use GeoTestUtil instead of GeoPointTestUtil

[Nicholas Knize (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicholas Knize updated LUCENE-7331:
---
Attachment: LUCENE-7331.patch

Simple patch:

* removes {{GeoPointTestUtil}} from {{TestGeoPointQuery}}
* fixes a range corner case in {{GeoPointPrefixTermsEnum}}
* adds an explicit test for the corner case

> GeoPointTestQuery should use GeoTestUtil instead of GeoPointTestUtil
> 
>
> Key: LUCENE-7331
> URL: https://issues.apache.org/jira/browse/LUCENE-7331
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: 6.1
>Reporter: Nicholas Knize
> Attachments: LUCENE-7331.patch
>
>
> Initially discussed in LUCENE-7166, and revisited in LUCENE-7325, 
> {{TestGeoPointQuery}} should use the RNG provided by {{GeoTestUtil}} instead 
> of the {{GeoPointTestUtil}} hack.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Lucene/Solr 6.1.0

[David Smiley]

I'm getting LUCENE-7291 done now too.  A rare heatmap faceting bug.

On Fri, Jun 10, 2016 at 10:38 AM Adrien Grand  wrote:

> There are still two blockers for 6.1, so I will wait until next week to
> build a first RC:
>  - https://issues.apache.org/jira/browse/SOLR-8744: Noble, is my
> understanding correct that this one is almost ready?
>  - https://issues.apache.org/jira/browse/LUCENE-7325: There is an ongoing
> discussion about whether this is a blocker, and what needs to be done but
> hopefully this will be sorted out soon.
>
> Le jeu. 9 juin 2016 à 08:18, Adrien Grand  a écrit :
>
>> Awesome, thanks Steve!
>>
>> Le jeu. 9 juin 2016 à 01:45, Steve Rowe  a écrit :
>>
>>> Hi Adrien,
>>>
>>> I remembered that I never fixed up the CHANGES.txt files on master and
>>> branch_6x after the 6.0.1 release to move the 6.0.1 changes out of the
>>> 6.1.0 section into the 6.0.1 section, so I went and did that, and
>>> backported to branch_6_1.
>>>
>>> I set up the ASF Jenkins jobs for 6.1 according to <
>>> http://wiki.apache.org/lucene-java/JenkinsReleaseBuilds>.
>>>
>>> I also set up the Policeman Jenkins jobs for 6.1 by cloning the 6.0
>>> Linux and Windows jobs[1], switching to */branch_6_1, enabling them, then
>>> triggering them as post-build actions from the corresponding master jobs.
>>> Uwe, apologies if I screwed this up :) - please fix if so!
>>>
>>> [1] I didn’t set up OSX and Solaris jobs because I didn’t see those for
>>> the 5.5 and 6.0 jobs.
>>>
>>> --
>>> Steve
>>> www.lucidworks.com
>>>
>>> > On Jun 8, 2016, at 8:38 AM, Adrien Grand  wrote:
>>> >
>>> >
>>> > I just created the 6.1.0 branch. Can someone help me with setting up
>>> Apache and Policeman Jenkins to test this branch?
>>> >
>>> > I still plan on cutting a first RC on Friday (in two days), please let
>>> me know if you know about issues that should be included in 6.1.0.
>>>
>>>
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>>
>>> --
Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
http://www.solrenterprisesearchserver.com

[jira] [Commented] (LUCENE-7291) HeatmapFacetCounterTest.testRandom failure; random

[Adrien Grand (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324879#comment-15324879
 ] 

Adrien Grand commented on LUCENE-7291:
--

+1

Did you comment out the "@Repeat(iterations = 20)" on purpose?

> HeatmapFacetCounterTest.testRandom failure; random
> --
>
> Key: LUCENE-7291
> URL: https://issues.apache.org/jira/browse/LUCENE-7291
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/spatial-extras
>Reporter: David Smiley
>Assignee: David Smiley
> Attachments: LUCENE_7291.patch
>
>
> Jenkins found a test failure today.
> This reproduces for me (master, java 8):
> ant test  -Dtestcase=HeatmapFacetCounterTest -Dtests.method=testRandom 
> -Dtests.seed=3EC907D1784B6F23 -Dtests.multiplier=2 -Dtests.nightly=true 
> -Dtests.slow=true 
> -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt 
> -Dtests.locale=is-IS -Dtests.timezone=Europe/Tirane -Dtests.asserts=true 
> -Dtests.file.encoding=UTF-8
> {noformat}
> java.lang.AssertionError: 
> Expected :1
> Actual   :0
>  
>   at 
> __randomizedtesting.SeedInfo.seed([3EC907D1784B6F23:A3439C5F68FEAB94]:0)
>   at org.junit.Assert.fail(Assert.java:93)
>   at org.junit.Assert.failNotEquals(Assert.java:647)
>   at org.junit.Assert.assertEquals(Assert.java:128)
>   at org.junit.Assert.assertEquals(Assert.java:472)
>   at org.junit.Assert.assertEquals(Assert.java:456)
>   at 
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.validateHeatmapResult(HeatmapFacetCounterTest.java:226)
>   at 
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:193)
>   at 
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:206)
>   at 
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.testRandom(HeatmapFacetCounterTest.java:172)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7331) GeoPointTestQuery should use GeoTestUtil instead of GeoPointTestUtil

[Nicholas Knize (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicholas Knize updated LUCENE-7331:
---
Affects Version/s: 6.1

> GeoPointTestQuery should use GeoTestUtil instead of GeoPointTestUtil
> 
>
> Key: LUCENE-7331
> URL: https://issues.apache.org/jira/browse/LUCENE-7331
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: 6.1
>Reporter: Nicholas Knize
>
> Initially discussed in LUCENE-7166, and revisited in LUCENE-7325, 
> {{TestGeoPointQuery}} should use the RNG provided by {{GeoTestUtil}} instead 
> of the {{GeoPointTestUtil}} hack.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: [JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_92) - Build # 239 - Still Failing!

[Uwe Schindler]

Hi,

 

Of course!

 

It is only strange that this error comes from the AccessController and not the 
operating system… So either the FilePermission code “knows” about this special 
names on windows, or it is a side effect while calculating canonic name of the 
file.

 

Uwe

 

-

Uwe Schindler

H.-H.-Meier-Allee 63, D-28213 Bremen

  http://www.thetaphi.de

eMail: u...@thetaphi.de

 

From: Dawid Weiss [mailto:dawid.we...@gmail.com] 
Sent: Friday, June 10, 2016 6:59 PM
To: dev@lucene.apache.org
Subject: RE: [JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_92) - Build # 
239 - Still Failing!

 


I think I do. Con is a special filename. You cannot use it on windows.

On Jun 10, 2016 5:50 PM, "Uwe Schindler" <  
u...@thetaphi.de> wrote:

Hi,

 

This is indeed very strange. I have no idea!

 

Uwe

 

-

Uwe Schindler

H.-H.-Meier-Allee 63, D-28213 Bremen

  http://www.thetaphi.de

eMail:   u...@thetaphi.de

 

From: Michael McCandless [mailto:  
luc...@mikemccandless.com] 
Sent: Friday, June 10, 2016 5:45 PM
To: Policeman Jenkins Server <  jenk...@thetaphi.de>
Cc: Adrien Grand <  jpou...@gmail.com>; Gregory 
Chanan <  gcha...@cloudera.com>;  
 dragonsi...@apache.org; Lucene/Solr dev < 
 dev@lucene.apache.org>
Subject: Re: [JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_92) - Build # 
239 - Still Failing!

 

Hmm, why was access denied here?




Mike McCandless

  http://blog.mikemccandless.com

 

On Thu, Jun 9, 2016 at 5:51 PM, Policeman Jenkins Server < 
 jenk...@thetaphi.de> wrote:

Build:   
http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Windows/239/
Java: 32bit/jdk1.8.0_92 -client -XX:+UseConcMarkSweepGC

2 tests failed.
FAILED:  org.apache.lucene.store.TestMmapDirectory.testPendingDeletions

Error Message:
access denied ("java.io.FilePermission" 
"C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\lucene\build\core\test\J0\temp\lucene.store.TestMmapDirectory_96FC6F2D45B76809-001\tempDir-007\con"
 "write")

Stack Trace:
java.security.AccessControlException: access denied ("java.io.FilePermission" 
"C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\lucene\build\core\test\J0\temp\lucene.store.TestMmapDirectory_96FC6F2D45B76809-001\tempDir-007\con"
 "write")
at 
__randomizedtesting.SeedInfo.seed([96FC6F2D45B76809:DE018553DD164AB]:0)
at 
java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
at 
java.security.AccessController.checkPermission(AccessController.java:884)
at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
at sun.nio.fs.WindowsChannelFactory.open(WindowsChannelFactory.java:295)
at 
sun.nio.fs.WindowsChannelFactory.newFileChannel(WindowsChannelFactory.java:162)
at 
sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:225)
at 
java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434)
at 
org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
at 
org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
at 
org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
at 
org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
at 
org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
at 
org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
at java.nio.file.Files.newOutputStream(Files.java:216)
at 
org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:408)
at 
org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:404)
at 
org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:253)
at 
org.apache.lucene.store.BaseDirectoryTestCase.testPendingDeletions(BaseDirectoryTestCase.java:1215)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:87

[jira] [Updated] (LUCENE-7291) HeatmapFacetCounterTest.testRandom failure; random

[David Smiley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley updated LUCENE-7291:
-
Attachment: LUCENE_7291.patch

This is a real bug; the patch I'm attaching fixes it and adds a test for the 
input that reproduced this circumstance.

A faulty count accumulation can happen in this circumstance:
* indexed data includes non-point shapes
* heatmap region crosses the dateline
* one of the indexed shapes is a good deal larger than the heatmap region such 
that it has leaf cells larger than the target heatmap cell size (so-called 
"ancestors" in the code).

I'll commit this to master, 6x, and 6.1

> HeatmapFacetCounterTest.testRandom failure; random
> --
>
> Key: LUCENE-7291
> URL: https://issues.apache.org/jira/browse/LUCENE-7291
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/spatial-extras
>Reporter: David Smiley
>Assignee: David Smiley
> Attachments: LUCENE_7291.patch
>
>
> Jenkins found a test failure today.
> This reproduces for me (master, java 8):
> ant test  -Dtestcase=HeatmapFacetCounterTest -Dtests.method=testRandom 
> -Dtests.seed=3EC907D1784B6F23 -Dtests.multiplier=2 -Dtests.nightly=true 
> -Dtests.slow=true 
> -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt 
> -Dtests.locale=is-IS -Dtests.timezone=Europe/Tirane -Dtests.asserts=true 
> -Dtests.file.encoding=UTF-8
> {noformat}
> java.lang.AssertionError: 
> Expected :1
> Actual   :0
>  
>   at 
> __randomizedtesting.SeedInfo.seed([3EC907D1784B6F23:A3439C5F68FEAB94]:0)
>   at org.junit.Assert.fail(Assert.java:93)
>   at org.junit.Assert.failNotEquals(Assert.java:647)
>   at org.junit.Assert.assertEquals(Assert.java:128)
>   at org.junit.Assert.assertEquals(Assert.java:472)
>   at org.junit.Assert.assertEquals(Assert.java:456)
>   at 
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.validateHeatmapResult(HeatmapFacetCounterTest.java:226)
>   at 
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:193)
>   at 
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:206)
>   at 
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.testRandom(HeatmapFacetCounterTest.java:172)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7330) Speed up conjunctions

[Adrien Grand (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adrien Grand updated LUCENE-7330:
-
Attachment: LUCENE-7330.patch

Here is a patch. It speeds up conjunctions thanks to 2 changes:

First it removes the 'if (doc == NO_MORE_DOCS) return NO_MORE_DOCS;' at the top 
of doNext(). This was needed because TwoPhaseConjunctionDISI extended 
ConjunctionDISI and it is illegal to call TwoPhaseIterator.matches() on 
NO_MORE_DOCS. I had to refactor a bit how the two-phase iterator is exposed but 
I don't think it makes things more complicated.

Second, it adds a special case for the second least costly iterator so that we 
do not have to check whether it is already on the same document as the 'lead'. 
If you look at the impl of doNext, we currently have to protect the call to 
'other.advance()' under a 'if (other.docID() < doc)', but we can actually avoid 
it for the 2nd least costly iterator without changing the order in which 
iterators are invoked.

luceneutil reports the following numbers on wikimedium10m, there seems to be a 
noticeable gain for conjunction-based queries (And*, *Span* and *Phrase):

{noformat}
TaskQPS baseline  StdDev   QPS patch  StdDev
Pct diff
OrHighNotLow  128.17  (9.1%)  126.04  (8.6%)   
-1.7% ( -17% -   17%)
  OrHighHigh   14.75  (6.5%)   14.54  (5.8%)   
-1.4% ( -12% -   11%)
   OrHighMed   66.53  (6.2%)   65.65  (5.8%)   
-1.3% ( -12% -   11%)
   OrHighLow   85.42  (7.3%)   84.51  (6.7%)   
-1.1% ( -14% -   13%)
  Fuzzy1   68.08 (10.9%)   67.37 (10.2%)   
-1.0% ( -19% -   22%)
OrHighNotMed  133.66  (8.5%)  132.33  (7.5%)   
-1.0% ( -15% -   16%)
   OrHighNotHigh   64.83  (4.6%)   64.36  (4.3%)   
-0.7% (  -9% -8%)
OrNotHighLow 1150.80  (3.1%) 1144.91  (3.4%)   
-0.5% (  -6% -6%)
  Fuzzy2   61.60 (22.2%)   61.31 (14.0%)   
-0.5% ( -30% -   46%)
   OrNotHighHigh   22.30  (2.7%)   22.23  (2.6%)   
-0.3% (  -5% -5%)
OrNotHighMed  155.90  (2.4%)  155.74  (2.7%)   
-0.1% (  -5% -5%)
 Respell   94.52  (1.9%)   94.69  (1.9%)
0.2% (  -3% -4%)
Wildcard   66.04  (4.6%)   66.50  (4.4%)
0.7% (  -7% -   10%)
 Prefix3  104.62  (4.7%)  105.54  (4.3%)
0.9% (  -7% -   10%)
HighTerm   98.37  (5.3%)   99.65  (4.5%)
1.3% (  -8% -   11%)
  AndHighLow  612.09  (3.0%)  620.90  (2.6%)
1.4% (  -3% -7%)
 MedTerm  237.97  (4.9%)  241.93  (4.4%)
1.7% (  -7% -   11%)
  IntNRQ   18.72  (9.4%)   19.05  (7.7%)
1.7% ( -13% -   20%)
 LowSloppyPhrase  108.80  (1.7%)  111.16  (2.2%)
2.2% (  -1% -6%)
   MedPhrase  100.85  (2.2%)  103.08  (2.1%)
2.2% (  -2% -6%)
 MedSpanNear   71.08  (2.2%)   73.09  (2.2%)
2.8% (  -1% -7%)
 LowTerm  623.38  (9.5%)  641.55  (7.7%)
2.9% ( -12% -   22%)
  HighPhrase   35.36  (3.2%)   36.42  (3.0%)
3.0% (  -3% -9%)
 LowSpanNear   92.47  (2.9%)   95.41  (2.8%)
3.2% (  -2% -9%)
HighSloppyPhrase   31.99  (4.9%)   33.09  (4.8%)
3.5% (  -5% -   13%)
  AndHighMed  223.42  (1.6%)  231.21  (1.9%)
3.5% (   0% -7%)
 MedSloppyPhrase   43.07  (2.5%)   45.13  (2.2%)
4.8% (   0% -9%)
HighSpanNear   28.57  (2.9%)   29.95  (3.6%)
4.8% (  -1% -   11%)
 AndHighHigh   74.55  (1.0%)   78.39  (1.6%)
5.2% (   2% -7%)
   LowPhrase   19.97  (2.5%)   21.04  (2.9%)
5.4% (   0% -   10%)
{noformat}

> Speed up conjunctions
> -
>
> Key: LUCENE-7330
> URL: https://issues.apache.org/jira/browse/LUCENE-7330
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Adrien Grand
>Assignee: Adrien Grand
>Priority: Minor
> Attachments: LUCENE-7330.patch
>
>
> I am digging into some performance regressions between 4.x and 5.x which seem 
> to be due to how we always run conjunctions with ConjunctionDISI now while 
> 4.x had FilteredQuery, which was optimized for the case that there are only 
> two clauses or that one of the clause supports random access. I'd like to 
> explore the former in 

[jira] [Updated] (SOLR-9204) Improve performance of getting directory size with hdfs.

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-9204:
--
Attachment: SOLR-9204.patch

> Improve performance of getting directory size with hdfs.
> 
>
> Key: SOLR-9204
> URL: https://issues.apache.org/jira/browse/SOLR-9204
> Project: Solr
>  Issue Type: Improvement
>  Components: hdfs
>Reporter: Mark Miller
>Assignee: Mark Miller
> Attachments: SOLR-9204.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-7331) GeoPointTestQuery should use GeoTestUtil instead of GeoPointTestUtil

[Nicholas Knize (JIRA)]

Nicholas Knize created LUCENE-7331:
--

 Summary: GeoPointTestQuery should use GeoTestUtil instead of 
GeoPointTestUtil
 Key: LUCENE-7331
 URL: https://issues.apache.org/jira/browse/LUCENE-7331
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Nicholas Knize


Initially discussed in LUCENE-7166, and revisited in LUCENE-7325, 
{{TestGeoPointQuery}} should use the RNG provided by {{GeoTestUtil}} instead of 
the {{GeoPointTestUtil}} hack.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9204) Improve performance of getting directory size with hdfs.

[Mark Miller (JIRA)]

Mark Miller created SOLR-9204:
-

 Summary: Improve performance of getting directory size with hdfs.
 Key: SOLR-9204
 URL: https://issues.apache.org/jira/browse/SOLR-9204
 Project: Solr
  Issue Type: Improvement
  Components: hdfs
Reporter: Mark Miller
Assignee: Mark Miller






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: [JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_92) - Build # 239 - Still Failing!

[Dawid Weiss]

I think I do. Con is a special filename. You cannot use it on windows.
On Jun 10, 2016 5:50 PM, "Uwe Schindler"  wrote:

> Hi,
>
>
>
> This is indeed very strange. I have no idea!
>
>
>
> Uwe
>
>
>
> -
>
> Uwe Schindler
>
> H.-H.-Meier-Allee 63, D-28213 Bremen
>
> http://www.thetaphi.de
>
> eMail: u...@thetaphi.de
>
>
>
> *From:* Michael McCandless [mailto:luc...@mikemccandless.com]
> *Sent:* Friday, June 10, 2016 5:45 PM
> *To:* Policeman Jenkins Server 
> *Cc:* Adrien Grand ; Gregory Chanan <
> gcha...@cloudera.com>; dragonsi...@apache.org; Lucene/Solr dev <
> dev@lucene.apache.org>
> *Subject:* Re: [JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_92) -
> Build # 239 - Still Failing!
>
>
>
> Hmm, why was access denied here?
>
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
>
>
> On Thu, Jun 9, 2016 at 5:51 PM, Policeman Jenkins Server <
> jenk...@thetaphi.de> wrote:
>
> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Windows/239/
> Java: 32bit/jdk1.8.0_92 -client -XX:+UseConcMarkSweepGC
>
> 2 tests failed.
> FAILED:  org.apache.lucene.store.TestMmapDirectory.testPendingDeletions
>
> Error Message:
> access denied ("java.io.FilePermission"
> "C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\lucene\build\core\test\J0\temp\lucene.store.TestMmapDirectory_96FC6F2D45B76809-001\tempDir-007\con"
> "write")
>
> Stack Trace:
> java.security.AccessControlException: access denied
> ("java.io.FilePermission"
> "C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\lucene\build\core\test\J0\temp\lucene.store.TestMmapDirectory_96FC6F2D45B76809-001\tempDir-007\con"
> "write")
> at
> __randomizedtesting.SeedInfo.seed([96FC6F2D45B76809:DE018553DD164AB]:0)
> at
> java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> at
> java.security.AccessController.checkPermission(AccessController.java:884)
> at
> java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
> at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
> at
> sun.nio.fs.WindowsChannelFactory.open(WindowsChannelFactory.java:295)
> at
> sun.nio.fs.WindowsChannelFactory.newFileChannel(WindowsChannelFactory.java:162)
> at
> sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:225)
> at
> java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434)
> at
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
> at
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
> at
> org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
> at
> org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
> at
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
> at
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
> at java.nio.file.Files.newOutputStream(Files.java:216)
> at
> org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:408)
> at
> org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:404)
> at
> org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:253)
> at
> org.apache.lucene.store.BaseDirectoryTestCase.testPendingDeletions(BaseDirectoryTestCase.java:1215)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
> at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.Thread

[jira] [Resolved] (LUCENE-7328) Remove LegacyNumericEncoding from GeoPointField

[Nicholas Knize (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicholas Knize resolved LUCENE-7328.

Resolution: Fixed

> Remove LegacyNumericEncoding from GeoPointField
> ---
>
> Key: LUCENE-7328
> URL: https://issues.apache.org/jira/browse/LUCENE-7328
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: master (7.0)
>Reporter: Nicholas Knize
> Attachments: LUCENE-7328.patch
>
>
> 6.0 deprecated legacy {{NumericEncoding}} in {{GeoPointField}} this issue 
> completely removes the encoding for 7.0.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-7330) Speed up conjunctions

[Adrien Grand (JIRA)]

Adrien Grand created LUCENE-7330:


 Summary: Speed up conjunctions
 Key: LUCENE-7330
 URL: https://issues.apache.org/jira/browse/LUCENE-7330
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Adrien Grand
Assignee: Adrien Grand
Priority: Minor


I am digging into some performance regressions between 4.x and 5.x which seem 
to be due to how we always run conjunctions with ConjunctionDISI now while 4.x 
had FilteredQuery, which was optimized for the case that there are only two 
clauses or that one of the clause supports random access. I'd like to explore 
the former in this issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-6.x - Build # 267 - Failure

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-6.x/267/

3 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestCoreDiscovery

Error Message:
ObjectTracker found 4 object(s) that were not released!!! 
[MockDirectoryWrapper, SolrCore, MDCAwareThreadPoolExecutor, 
MockDirectoryWrapper]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 4 object(s) that were not 
released!!! [MockDirectoryWrapper, SolrCore, MDCAwareThreadPoolExecutor, 
MockDirectoryWrapper]
at __randomizedtesting.SeedInfo.seed([1AF6C4EB5D7F7245]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at 
org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:257)
at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestCoreDiscovery

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.core.TestCoreDiscovery: 
1) Thread[id=9100, name=searcherExecutor-4527-thread-1, state=WAITING, 
group=TGRP-TestCoreDiscovery] at sun.misc.Unsafe.park(Native Method)
 at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)   
  at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
 at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) 
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.core.TestCoreDiscovery: 
   1) Thread[id=9100, name=searcherExecutor-4527-thread-1, state=WAITING, 
group=TGRP-TestCoreDiscovery]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([1AF6C4EB5D7F7245]:

[jira] [Commented] (LUCENE-7328) Remove LegacyNumericEncoding from GeoPointField

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324767#comment-15324767
 ] 

ASF subversion and git services commented on LUCENE-7328:
-

Commit e4db256d3f2723f6d56a9734d4c820f70455f52b in lucene-solr's branch 
refs/heads/master from [~nknize]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e4db256 ]

LUCENE-7328: Remove LegacyNumericEncoding from GeoPointField


> Remove LegacyNumericEncoding from GeoPointField
> ---
>
> Key: LUCENE-7328
> URL: https://issues.apache.org/jira/browse/LUCENE-7328
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: master (7.0)
>Reporter: Nicholas Knize
> Attachments: LUCENE-7328.patch
>
>
> 6.0 deprecated legacy {{NumericEncoding}} in {{GeoPointField}} this issue 
> completely removes the encoding for 7.0.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7329) Simplify CharacterUtils

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324749#comment-15324749
 ] 

Uwe Schindler commented on LUCENE-7329:
---

+1

I stumbled on that one, too.

> Simplify CharacterUtils
> ---
>
> Key: LUCENE-7329
> URL: https://issues.apache.org/jira/browse/LUCENE-7329
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Adrien Grand
>Priority: Minor
> Attachments: LUCENE-7329.patch
>
>
> This class has abstractions for the Java 4 and 5 ways of dealing with 
> characters, but we now only use the Java 5 implementation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3330 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3330/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.cloud.ShardSplitTest.test

Error Message:
Wrong doc count on shard1_0. See SOLR-5309 expected:<114> but was:<113>

Stack Trace:
java.lang.AssertionError: Wrong doc count on shard1_0. See SOLR-5309 
expected:<114> but was:<113>
at 
__randomizedtesting.SeedInfo.seed([1256EA9592F0E250:9A02D54F3C0C8FA8]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at 
org.apache.solr.cloud.ShardSplitTest.checkDocCountsAndShardStates(ShardSplitTest.java:462)
at 
org.apache.solr.cloud.ShardSplitTest.splitByUniqueKeyTest(ShardSplitTest.java:245)
at org.apache.solr.cloud.ShardSplitTest.test(ShardSplitTest.java:82)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apac

[jira] [Commented] (LUCENE-7329) Simplify CharacterUtils

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324742#comment-15324742
 ] 

Michael McCandless commented on LUCENE-7329:


+1

> Simplify CharacterUtils
> ---
>
> Key: LUCENE-7329
> URL: https://issues.apache.org/jira/browse/LUCENE-7329
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Adrien Grand
>Priority: Minor
> Attachments: LUCENE-7329.patch
>
>
> This class has abstractions for the Java 4 and 5 ways of dealing with 
> characters, but we now only use the Java 5 implementation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9185) Solr's "Lucene"/standard query parser should not split on whitespace before sending terms to analysis

[Steve Rowe (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324740#comment-15324740
 ] 

Steve Rowe commented on SOLR-9185:
--

bq. I think we need an option that turns the whitespace split off.

I disagree.  I think the current behavior is counter to users' expectations, so 
we should just get rid of it.

I suppose we could add luceneMatchVersion-sensitive code and include both 
versions, but yuck, I'd much rather not do that.

bq. I think the default behavior in 6.x should remain unchanged. We can change 
the default in master.

I disagree.  I think we should change the default behavior ASAP.

bq. The implementation might take a while to become bulletproof. I suspect that 
the query parser code relies heavily on the current behavior and that things 
will break in unexpected ways when changing that behavior.

Here I agree.  (e)dismax and other parsers that are based on the Solr clone of 
the Lucene QP will need work before this change can be released.

> Solr's "Lucene"/standard query parser should not split on whitespace before 
> sending terms to analysis
> -
>
> Key: SOLR-9185
> URL: https://issues.apache.org/jira/browse/SOLR-9185
> Project: Solr
>  Issue Type: Bug
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>
> Copied from LUCENE-2605:
> The queryparser parses input on whitespace, and sends each whitespace 
> separated term to its own independent token stream.
> This breaks the following at query-time, because they can't see across 
> whitespace boundaries:
> n-gram analysis
> shingles
> synonyms (especially multi-word for whitespace-separated languages)
> languages where a 'word' can contain whitespace (e.g. vietnamese)
> Its also rather unexpected, as users think their 
> charfilters/tokenizers/tokenfilters will do the same thing at index and 
> querytime, but
> in many cases they can't. Instead, preferably the queryparser would parse 
> around only real 'operators'.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-7327) Geo3d quantization test failure

[Michael McCandless (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless resolved LUCENE-7327.

Resolution: Fixed

> Geo3d quantization test failure
> ---
>
> Key: LUCENE-7327
> URL: https://issues.apache.org/jira/browse/LUCENE-7327
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Adrien Grand
> Fix For: master (7.0), 6.2
>
>
> Here is a reproducible TestGeo3DPoint.testQuantization failure on branch_6x:
> {noformat}
>[junit4] Suite: org.apache.lucene.spatial3d.TestGeo3DPoint
>[junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestGeo3DPoint 
> -Dtests.method=testQuantization -Dtests.seed=ABC4B9A9A4DC7369 
> -Dtests.slow=true -Dtests.locale=es-AR 
> -Dtests.timezone=America/Argentina/Cordoba -Dtests.asserts=true 
> -Dtests.file.encoding=UTF-8
>[junit4] ERROR   0.24s | TestGeo3DPoint.testQuantization <<<
>[junit4]> Throwable #1: java.lang.IllegalArgumentException: 
> value=1.0011189825039197 is out-of-bounds (greater than WGS84's 
> planetMax=1.0011188539924791)
>[junit4]>  at 
> __randomizedtesting.SeedInfo.seed([ABC4B9A9A4DC7369:9D2D96BFB23D6E4B]:0)
>[junit4]>  at 
> org.apache.lucene.spatial3d.Geo3DUtil.encodeValue(Geo3DUtil.java:53)
>[junit4]>  at 
> org.apache.lucene.spatial3d.TestGeo3DPoint.testQuantization(TestGeo3DPoint.java:1222)
>[junit4]>  at java.lang.Thread.run(Thread.java:745)
>[junit4]   2> NOTE: test params are: codec=Asserting(Lucene60): {}, 
> docValues:{}, maxPointsInLeafNode=20, maxMBSortInHeap=7.62052343111666, 
> sim=RandomSimilarity(queryNorm=false,coord=crazy): {}, locale=es-AR, 
> timezone=America/Argentina/Cordoba
>[junit4]   2> NOTE: Linux 3.13.0-79-generic amd64/Oracle Corporation 
> 1.8.0_66-ea (64-bit)/cpus=8,threads=1,free=275893960,total=319291392
>[junit4]   2> NOTE: All tests run in this JVM: [TestGeo3DPoint]
>[junit4] Completed [1/1 (1!)] in 0.55s, 1 test, 1 error <<< FAILURES!
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-6968) LSH Filter

[Tommaso Teofili (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-6968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tommaso Teofili updated LUCENE-6968:

Attachment: LUCENE-6968.6.patch

minor fixes to the last patch, apart from that I cannot see any error, even 
with Japanese locale explicitly set
{noformat}
ant clean test -Dtests.multiplier=100 -Dtestcase=MinHashFilterTest 
-Dtests.locale=ja
{noformat}

[~andyhind] do you have the seed of a failed execution ?


> LSH Filter
> --
>
> Key: LUCENE-6968
> URL: https://issues.apache.org/jira/browse/LUCENE-6968
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Cao Manh Dat
>Assignee: Tommaso Teofili
> Attachments: LUCENE-6968.4.patch, LUCENE-6968.5.patch, 
> LUCENE-6968.6.patch, LUCENE-6968.patch, LUCENE-6968.patch, LUCENE-6968.patch
>
>
> I'm planning to implement LSH. Which support query like this
> {quote}
> Find similar documents that have 0.8 or higher similar score with a given 
> document. Similarity measurement can be cosine, jaccard, euclid..
> {quote}
> For example. Given following corpus
> {quote}
> 1. Solr is an open source search engine based on Lucene
> 2. Solr is an open source enterprise search engine based on Lucene
> 3. Solr is an popular open source enterprise search engine based on Lucene
> 4. Apache Lucene is a high-performance, full-featured text search engine 
> library written entirely in Java
> {quote}
> We wanna find documents that have 0.6 score in jaccard measurement with this 
> doc
> {quote}
> Solr is an open source search engine
> {quote}
> It will return only docs 1,2 and 3 (MoreLikeThis will also return doc 4)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9185) Solr's "Lucene"/standard query parser should not split on whitespace before sending terms to analysis

[Shawn Heisey (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324714#comment-15324714
 ] 

Shawn Heisey commented on SOLR-9185:


I have often wondered whether things would work better if that behavior were 
changed.

I think we need an option that turns the whitespace split off.  An alternate 
implementation path is a completely new query parser in addition to the 
existing parser.  Perhaps it could be named "solr".

Whatever implementation is chosen, I think the default behavior in 6.x should 
remain unchanged.  We can change the default in master.

The implementation might take a while to become bulletproof.  I suspect that 
the query parser code relies heavily on the current behavior and that things 
will break in unexpected ways when changing that behavior. 

> Solr's "Lucene"/standard query parser should not split on whitespace before 
> sending terms to analysis
> -
>
> Key: SOLR-9185
> URL: https://issues.apache.org/jira/browse/SOLR-9185
> Project: Solr
>  Issue Type: Bug
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>
> Copied from LUCENE-2605:
> The queryparser parses input on whitespace, and sends each whitespace 
> separated term to its own independent token stream.
> This breaks the following at query-time, because they can't see across 
> whitespace boundaries:
> n-gram analysis
> shingles
> synonyms (especially multi-word for whitespace-separated languages)
> languages where a 'word' can contain whitespace (e.g. vietnamese)
> Its also rather unexpected, as users think their 
> charfilters/tokenizers/tokenfilters will do the same thing at index and 
> querytime, but
> in many cases they can't. Instead, preferably the queryparser would parse 
> around only real 'operators'.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7325) GeoPointInBBoxQuery no longer includes boundaries?

[Nicholas Knize (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324700#comment-15324700
 ] 

Nicholas Knize commented on LUCENE-7325:


{noformat}
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestGeoPointQuery 
-Dtests.method=testRandomMedium -Dtests.seed=3254B65F0C8C1F93 -Dtests.slow=true 
-Dtests.locale=th-TH -Dtests.timezone=Etc/GMT -Dtests.asserts=true 
-Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.35s J0 | TestGeoPointQuery.testRandomMedium <<<
   [junit4]> Throwable #1: java.lang.IllegalArgumentException: Illegal 
shift value, must be 32..63; got shift=0
{noformat}

> GeoPointInBBoxQuery no longer includes boundaries?
> --
>
> Key: LUCENE-7325
> URL: https://issues.apache.org/jira/browse/LUCENE-7325
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 6.1
>Reporter: Michael McCandless
>Priority: Blocker
> Attachments: LUCENE-7325.patch, LUCENE-7325.patch
>
>
> {{GeoPointInBBoxQuery}} is supposed to include boundaries, and it does in 5.x 
> and 6.0, but in 6.1 something broke.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7326) TestSimpleTextPostingsFormat.testInvertedWrite() failure: An SPI class of type PostingsFormat with name 'SimpleText' does not exist

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324704#comment-15324704
 ] 

Michael McCandless commented on LUCENE-7326:


Thanks for confirming [~steve_rowe]!

> TestSimpleTextPostingsFormat.testInvertedWrite() failure: An SPI class of 
> type PostingsFormat with name 'SimpleText' does not exist
> ---
>
> Key: LUCENE-7326
> URL: https://issues.apache.org/jira/browse/LUCENE-7326
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Steve Rowe
> Fix For: master (7.0), 6.2
>
>
> My Jenkins found a master seed that reproduces on branch_6x but not on 
> branch_6_1:
> {noformat}
> Checking out Revision 963206969eddca6ec743f5f0901e0abdfeacd3cc 
> (refs/remotes/origin/master)
> [...]
>   2> NOTE: reproduce with: ant test  -Dtestcase=TestSimpleTextPostingsFormat 
> -Dtests.method=testInvertedWrite -Dtests.seed=7555A4CF724BDB74 
> -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true 
> -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt 
> -Dtests.locale=es -Dtests.timezone=Australia/Brisbane -Dtests.asserts=true 
> -Dtests.file.encoding=UTF-8
> [23:09:22.113] ERROR   0.23s J3 | 
> TestSimpleTextPostingsFormat.testInvertedWrite <<<
>> Throwable #1: java.lang.IllegalArgumentException: An SPI class of type 
> org.apache.lucene.codecs.PostingsFormat with name 'SimpleText' does not 
> exist.  You need to add the corresponding JAR file supporting this SPI to 
> your classpath.  The current classpath supports the following names: 
> [MockRandom, RAMOnly, LuceneFixedGap, LuceneVarGapFixedInterval, 
> LuceneVarGapDocFreqInterval, TestBloomFilteredLucenePostings, Asserting, 
> BlockTreeOrds, BloomFilter, Direct, FSTOrd50, FST50, Memory, AutoPrefix, 
> Lucene50]
>>at 
> __randomizedtesting.SeedInfo.seed([7555A4CF724BDB74:6C21E9040C1050A8]:0)
>>at 
> org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:116)
>>at 
> org.apache.lucene.codecs.PostingsFormat.forName(PostingsFormat.java:112)
>>at 
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.(PerFieldPostingsFormat.java:258)
>>at 
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
>>at 
> org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:106)
>>at org.apache.lucene.index.SegmentReader.(SegmentReader.java:67)
>>at 
> org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:61)
>>at 
> org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:53)
>>at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:675)
>>at 
> org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:76)
>>at 
> org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:63)
>>at 
> org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:383)
>>at 
> org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:313)
>>at 
> org.apache.lucene.index.BasePostingsFormatTestCase.testInvertedWrite(BasePostingsFormatTestCase.java:519)
> [...]
>   2> NOTE: test params are: codec=Asserting(Lucene62): {}, docValues:{}, 
> maxPointsInLeafNode=1657, maxMBSortInHeap=5.09627287608914, 
> sim=RandomSimilarity(queryNorm=true,coord=crazy): {f_DOCS_AND_FREQS=DFR 
> I(ne)Z(0.3), field=DFR I(ne)L3(800.0), 
> f_DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS=DFI(Saturated), 
> f_DOCS_AND_FREQS_AND_POSITIONS=IB SPL-L1, titleTokenized=IB LL-L3(800.0), 
> body=DFR I(F)L1, f_DOCS=DFR I(n)L2}, locale=es, timezone=Australia/Brisbane
>   2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation 1.8.0_77 
> (64-bit)/cpus=16,threads=1,free=257107688,total=486539264
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7326) TestSimpleTextPostingsFormat.testInvertedWrite() failure: An SPI class of type PostingsFormat with name 'SimpleText' does not exist

[Steve Rowe (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324691#comment-15324691
 ] 

Steve Rowe commented on LUCENE-7326:


With [~mikemccand]'s commit, neither seed reproduces for me on either branch.

> TestSimpleTextPostingsFormat.testInvertedWrite() failure: An SPI class of 
> type PostingsFormat with name 'SimpleText' does not exist
> ---
>
> Key: LUCENE-7326
> URL: https://issues.apache.org/jira/browse/LUCENE-7326
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Steve Rowe
> Fix For: master (7.0), 6.2
>
>
> My Jenkins found a master seed that reproduces on branch_6x but not on 
> branch_6_1:
> {noformat}
> Checking out Revision 963206969eddca6ec743f5f0901e0abdfeacd3cc 
> (refs/remotes/origin/master)
> [...]
>   2> NOTE: reproduce with: ant test  -Dtestcase=TestSimpleTextPostingsFormat 
> -Dtests.method=testInvertedWrite -Dtests.seed=7555A4CF724BDB74 
> -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true 
> -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt 
> -Dtests.locale=es -Dtests.timezone=Australia/Brisbane -Dtests.asserts=true 
> -Dtests.file.encoding=UTF-8
> [23:09:22.113] ERROR   0.23s J3 | 
> TestSimpleTextPostingsFormat.testInvertedWrite <<<
>> Throwable #1: java.lang.IllegalArgumentException: An SPI class of type 
> org.apache.lucene.codecs.PostingsFormat with name 'SimpleText' does not 
> exist.  You need to add the corresponding JAR file supporting this SPI to 
> your classpath.  The current classpath supports the following names: 
> [MockRandom, RAMOnly, LuceneFixedGap, LuceneVarGapFixedInterval, 
> LuceneVarGapDocFreqInterval, TestBloomFilteredLucenePostings, Asserting, 
> BlockTreeOrds, BloomFilter, Direct, FSTOrd50, FST50, Memory, AutoPrefix, 
> Lucene50]
>>at 
> __randomizedtesting.SeedInfo.seed([7555A4CF724BDB74:6C21E9040C1050A8]:0)
>>at 
> org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:116)
>>at 
> org.apache.lucene.codecs.PostingsFormat.forName(PostingsFormat.java:112)
>>at 
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.(PerFieldPostingsFormat.java:258)
>>at 
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
>>at 
> org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:106)
>>at org.apache.lucene.index.SegmentReader.(SegmentReader.java:67)
>>at 
> org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:61)
>>at 
> org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:53)
>>at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:675)
>>at 
> org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:76)
>>at 
> org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:63)
>>at 
> org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:383)
>>at 
> org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:313)
>>at 
> org.apache.lucene.index.BasePostingsFormatTestCase.testInvertedWrite(BasePostingsFormatTestCase.java:519)
> [...]
>   2> NOTE: test params are: codec=Asserting(Lucene62): {}, docValues:{}, 
> maxPointsInLeafNode=1657, maxMBSortInHeap=5.09627287608914, 
> sim=RandomSimilarity(queryNorm=true,coord=crazy): {f_DOCS_AND_FREQS=DFR 
> I(ne)Z(0.3), field=DFR I(ne)L3(800.0), 
> f_DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS=DFI(Saturated), 
> f_DOCS_AND_FREQS_AND_POSITIONS=IB SPL-L1, titleTokenized=IB LL-L3(800.0), 
> body=DFR I(F)L1, f_DOCS=DFR I(n)L2}, locale=es, timezone=Australia/Brisbane
>   2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation 1.8.0_77 
> (64-bit)/cpus=16,threads=1,free=257107688,total=486539264
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7326) TestSimpleTextPostingsFormat.testInvertedWrite() failure: An SPI class of type PostingsFormat with name 'SimpleText' does not exist

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324680#comment-15324680
 ] 

ASF subversion and git services commented on LUCENE-7326:
-

Commit 816fae9622d9719fd38a5381a7029383e54d2e77 in lucene-solr's branch 
refs/heads/master from Mike McCandless
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=816fae9 ]

LUCENE-7326: don't use postings format by name in this test


> TestSimpleTextPostingsFormat.testInvertedWrite() failure: An SPI class of 
> type PostingsFormat with name 'SimpleText' does not exist
> ---
>
> Key: LUCENE-7326
> URL: https://issues.apache.org/jira/browse/LUCENE-7326
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Steve Rowe
> Fix For: master (7.0), 6.2
>
>
> My Jenkins found a master seed that reproduces on branch_6x but not on 
> branch_6_1:
> {noformat}
> Checking out Revision 963206969eddca6ec743f5f0901e0abdfeacd3cc 
> (refs/remotes/origin/master)
> [...]
>   2> NOTE: reproduce with: ant test  -Dtestcase=TestSimpleTextPostingsFormat 
> -Dtests.method=testInvertedWrite -Dtests.seed=7555A4CF724BDB74 
> -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true 
> -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt 
> -Dtests.locale=es -Dtests.timezone=Australia/Brisbane -Dtests.asserts=true 
> -Dtests.file.encoding=UTF-8
> [23:09:22.113] ERROR   0.23s J3 | 
> TestSimpleTextPostingsFormat.testInvertedWrite <<<
>> Throwable #1: java.lang.IllegalArgumentException: An SPI class of type 
> org.apache.lucene.codecs.PostingsFormat with name 'SimpleText' does not 
> exist.  You need to add the corresponding JAR file supporting this SPI to 
> your classpath.  The current classpath supports the following names: 
> [MockRandom, RAMOnly, LuceneFixedGap, LuceneVarGapFixedInterval, 
> LuceneVarGapDocFreqInterval, TestBloomFilteredLucenePostings, Asserting, 
> BlockTreeOrds, BloomFilter, Direct, FSTOrd50, FST50, Memory, AutoPrefix, 
> Lucene50]
>>at 
> __randomizedtesting.SeedInfo.seed([7555A4CF724BDB74:6C21E9040C1050A8]:0)
>>at 
> org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:116)
>>at 
> org.apache.lucene.codecs.PostingsFormat.forName(PostingsFormat.java:112)
>>at 
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.(PerFieldPostingsFormat.java:258)
>>at 
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
>>at 
> org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:106)
>>at org.apache.lucene.index.SegmentReader.(SegmentReader.java:67)
>>at 
> org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:61)
>>at 
> org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:53)
>>at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:675)
>>at 
> org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:76)
>>at 
> org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:63)
>>at 
> org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:383)
>>at 
> org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:313)
>>at 
> org.apache.lucene.index.BasePostingsFormatTestCase.testInvertedWrite(BasePostingsFormatTestCase.java:519)
> [...]
>   2> NOTE: test params are: codec=Asserting(Lucene62): {}, docValues:{}, 
> maxPointsInLeafNode=1657, maxMBSortInHeap=5.09627287608914, 
> sim=RandomSimilarity(queryNorm=true,coord=crazy): {f_DOCS_AND_FREQS=DFR 
> I(ne)Z(0.3), field=DFR I(ne)L3(800.0), 
> f_DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS=DFI(Saturated), 
> f_DOCS_AND_FREQS_AND_POSITIONS=IB SPL-L1, titleTokenized=IB LL-L3(800.0), 
> body=DFR I(F)L1, f_DOCS=DFR I(n)L2}, locale=es, timezone=Australia/Brisbane
>   2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation 1.8.0_77 
> (64-bit)/cpus=16,threads=1,free=257107688,total=486539264
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-7326) TestSimpleTextPostingsFormat.testInvertedWrite() failure: An SPI class of type PostingsFormat with name 'SimpleText' does not exist

[Michael McCandless (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless resolved LUCENE-7326.

   Resolution: Fixed
Fix Version/s: 6.2
   master (7.0)

> TestSimpleTextPostingsFormat.testInvertedWrite() failure: An SPI class of 
> type PostingsFormat with name 'SimpleText' does not exist
> ---
>
> Key: LUCENE-7326
> URL: https://issues.apache.org/jira/browse/LUCENE-7326
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Steve Rowe
> Fix For: master (7.0), 6.2
>
>
> My Jenkins found a master seed that reproduces on branch_6x but not on 
> branch_6_1:
> {noformat}
> Checking out Revision 963206969eddca6ec743f5f0901e0abdfeacd3cc 
> (refs/remotes/origin/master)
> [...]
>   2> NOTE: reproduce with: ant test  -Dtestcase=TestSimpleTextPostingsFormat 
> -Dtests.method=testInvertedWrite -Dtests.seed=7555A4CF724BDB74 
> -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true 
> -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt 
> -Dtests.locale=es -Dtests.timezone=Australia/Brisbane -Dtests.asserts=true 
> -Dtests.file.encoding=UTF-8
> [23:09:22.113] ERROR   0.23s J3 | 
> TestSimpleTextPostingsFormat.testInvertedWrite <<<
>> Throwable #1: java.lang.IllegalArgumentException: An SPI class of type 
> org.apache.lucene.codecs.PostingsFormat with name 'SimpleText' does not 
> exist.  You need to add the corresponding JAR file supporting this SPI to 
> your classpath.  The current classpath supports the following names: 
> [MockRandom, RAMOnly, LuceneFixedGap, LuceneVarGapFixedInterval, 
> LuceneVarGapDocFreqInterval, TestBloomFilteredLucenePostings, Asserting, 
> BlockTreeOrds, BloomFilter, Direct, FSTOrd50, FST50, Memory, AutoPrefix, 
> Lucene50]
>>at 
> __randomizedtesting.SeedInfo.seed([7555A4CF724BDB74:6C21E9040C1050A8]:0)
>>at 
> org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:116)
>>at 
> org.apache.lucene.codecs.PostingsFormat.forName(PostingsFormat.java:112)
>>at 
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.(PerFieldPostingsFormat.java:258)
>>at 
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
>>at 
> org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:106)
>>at org.apache.lucene.index.SegmentReader.(SegmentReader.java:67)
>>at 
> org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:61)
>>at 
> org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:53)
>>at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:675)
>>at 
> org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:76)
>>at 
> org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:63)
>>at 
> org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:383)
>>at 
> org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:313)
>>at 
> org.apache.lucene.index.BasePostingsFormatTestCase.testInvertedWrite(BasePostingsFormatTestCase.java:519)
> [...]
>   2> NOTE: test params are: codec=Asserting(Lucene62): {}, docValues:{}, 
> maxPointsInLeafNode=1657, maxMBSortInHeap=5.09627287608914, 
> sim=RandomSimilarity(queryNorm=true,coord=crazy): {f_DOCS_AND_FREQS=DFR 
> I(ne)Z(0.3), field=DFR I(ne)L3(800.0), 
> f_DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS=DFI(Saturated), 
> f_DOCS_AND_FREQS_AND_POSITIONS=IB SPL-L1, titleTokenized=IB LL-L3(800.0), 
> body=DFR I(F)L1, f_DOCS=DFR I(n)L2}, locale=es, timezone=Australia/Brisbane
>   2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation 1.8.0_77 
> (64-bit)/cpus=16,threads=1,free=257107688,total=486539264
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7326) TestSimpleTextPostingsFormat.testInvertedWrite() failure: An SPI class of type PostingsFormat with name 'SimpleText' does not exist

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324678#comment-15324678
 ] 

ASF subversion and git services commented on LUCENE-7326:
-

Commit e98aa2266a7da7b574325b828879f79e6cb66826 in lucene-solr's branch 
refs/heads/branch_6x from Mike McCandless
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e98aa22 ]

LUCENE-7326: don't use postings format by name in this test


> TestSimpleTextPostingsFormat.testInvertedWrite() failure: An SPI class of 
> type PostingsFormat with name 'SimpleText' does not exist
> ---
>
> Key: LUCENE-7326
> URL: https://issues.apache.org/jira/browse/LUCENE-7326
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Steve Rowe
>
> My Jenkins found a master seed that reproduces on branch_6x but not on 
> branch_6_1:
> {noformat}
> Checking out Revision 963206969eddca6ec743f5f0901e0abdfeacd3cc 
> (refs/remotes/origin/master)
> [...]
>   2> NOTE: reproduce with: ant test  -Dtestcase=TestSimpleTextPostingsFormat 
> -Dtests.method=testInvertedWrite -Dtests.seed=7555A4CF724BDB74 
> -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true 
> -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt 
> -Dtests.locale=es -Dtests.timezone=Australia/Brisbane -Dtests.asserts=true 
> -Dtests.file.encoding=UTF-8
> [23:09:22.113] ERROR   0.23s J3 | 
> TestSimpleTextPostingsFormat.testInvertedWrite <<<
>> Throwable #1: java.lang.IllegalArgumentException: An SPI class of type 
> org.apache.lucene.codecs.PostingsFormat with name 'SimpleText' does not 
> exist.  You need to add the corresponding JAR file supporting this SPI to 
> your classpath.  The current classpath supports the following names: 
> [MockRandom, RAMOnly, LuceneFixedGap, LuceneVarGapFixedInterval, 
> LuceneVarGapDocFreqInterval, TestBloomFilteredLucenePostings, Asserting, 
> BlockTreeOrds, BloomFilter, Direct, FSTOrd50, FST50, Memory, AutoPrefix, 
> Lucene50]
>>at 
> __randomizedtesting.SeedInfo.seed([7555A4CF724BDB74:6C21E9040C1050A8]:0)
>>at 
> org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:116)
>>at 
> org.apache.lucene.codecs.PostingsFormat.forName(PostingsFormat.java:112)
>>at 
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.(PerFieldPostingsFormat.java:258)
>>at 
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
>>at 
> org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:106)
>>at org.apache.lucene.index.SegmentReader.(SegmentReader.java:67)
>>at 
> org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:61)
>>at 
> org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:53)
>>at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:675)
>>at 
> org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:76)
>>at 
> org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:63)
>>at 
> org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:383)
>>at 
> org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:313)
>>at 
> org.apache.lucene.index.BasePostingsFormatTestCase.testInvertedWrite(BasePostingsFormatTestCase.java:519)
> [...]
>   2> NOTE: test params are: codec=Asserting(Lucene62): {}, docValues:{}, 
> maxPointsInLeafNode=1657, maxMBSortInHeap=5.09627287608914, 
> sim=RandomSimilarity(queryNorm=true,coord=crazy): {f_DOCS_AND_FREQS=DFR 
> I(ne)Z(0.3), field=DFR I(ne)L3(800.0), 
> f_DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS=DFI(Saturated), 
> f_DOCS_AND_FREQS_AND_POSITIONS=IB SPL-L1, titleTokenized=IB LL-L3(800.0), 
> body=DFR I(F)L1, f_DOCS=DFR I(n)L2}, locale=es, timezone=Australia/Brisbane
>   2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation 1.8.0_77 
> (64-bit)/cpus=16,threads=1,free=257107688,total=486539264
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7326) TestSimpleTextPostingsFormat.testInvertedWrite() failure: An SPI class of type PostingsFormat with name 'SimpleText' does not exist

[Steve Rowe (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324674#comment-15324674
 ] 

Steve Rowe commented on LUCENE-7326:


My Jenkins found a branch_6x seed that reproduces on master for the same method 
failure but a different cause:

{noformat}
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=TestSimpleTextPostingsFormat -Dtests.method=testInvertedWrite 
-Dtests.seed=7EEFAF93C5A99DEA -Dtests.multiplier=2 -Dtests.nightly=true 
-Dtests.slow=true 
-Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt 
-Dtests.locale=zh-TW -Dtests.timezone=Asia/Kabul -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.29s J3 | TestSimpleTextPostingsFormat.testInvertedWrite 
<<<
   [junit4]> Throwable #1: org.apache.lucene.index.CorruptIndexException: 
compound sub-files must have a valid codec header and footer: codec header 
mismatch: actual header=1718183276 vs expected header=1071082519 
(resource=BufferedChecksumIndexInput(MockIndexInputWrapper(MMapIndexInput(path="/var/lib/jenkins/jobs/Lucene-Solr-Nightly-6.x/workspace/lucene/build/codecs/test/J3/temp/lucene.codecs.simpletext.TestSimpleTextPostingsFormat_7EEFAF93C5A99DEA-001/index-MMapDirectory-002/_0_SimpleText_0.pst"
   [junit4]>at 
__randomizedtesting.SeedInfo.seed([7EEFAF93C5A99DEA:679BE258BBF21636]:0)
   [junit4]>at 
org.apache.lucene.codecs.CodecUtil.verifyAndCopyIndexHeader(CodecUtil.java:287)
   [junit4]>at 
org.apache.lucene.codecs.lucene50.Lucene50CompoundFormat.write(Lucene50CompoundFormat.java:96)
   [junit4]>at 
org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:4681)
   [junit4]>at 
org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:496)
   [junit4]>at 
org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:460)
   [junit4]>at 
org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:502)
   [junit4]>at 
org.apache.lucene.index.DocumentsWriter.postUpdate(DocumentsWriter.java:376)
   [junit4]>at 
org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:429)
   [junit4]>at 
org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1333)
   [junit4]>at 
org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1312)
   [junit4]>at 
org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:140)
   [junit4]>at 
org.apache.lucene.index.BasePostingsFormatTestCase.testInvertedWrite(BasePostingsFormatTestCase.java:515)
   [junit4]>at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: leaving temporary files on disk at: 
/var/lib/jenkins/jobs/Lucene-Solr-Nightly-6.x/workspace/lucene/build/codecs/test/J3/temp/lucene.codecs.simpletext.TestSimpleTextPostingsFormat_7EEFAF93C5A99DEA-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene60): {}, 
docValues:{}, maxPointsInLeafNode=617, maxMBSortInHeap=6.3814037394847265, 
sim=ClassicSimilarity, locale=zh-TW, timezone=Asia/Kabul
   [junit4]   2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation 
1.8.0_77 (64-bit)/cpus=16,threads=1,free=222363136,total=450887680
   [junit4]   2> NOTE: All tests run in this JVM: [TestMemoryPostingsFormat, 
TestSimpleTextCompoundFormat, TestSimpleTextPostingsFormat]
{noformat}

> TestSimpleTextPostingsFormat.testInvertedWrite() failure: An SPI class of 
> type PostingsFormat with name 'SimpleText' does not exist
> ---
>
> Key: LUCENE-7326
> URL: https://issues.apache.org/jira/browse/LUCENE-7326
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Steve Rowe
>
> My Jenkins found a master seed that reproduces on branch_6x but not on 
> branch_6_1:
> {noformat}
> Checking out Revision 963206969eddca6ec743f5f0901e0abdfeacd3cc 
> (refs/remotes/origin/master)
> [...]
>   2> NOTE: reproduce with: ant test  -Dtestcase=TestSimpleTextPostingsFormat 
> -Dtests.method=testInvertedWrite -Dtests.seed=7555A4CF724BDB74 
> -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true 
> -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt 
> -Dtests.locale=es -Dtests.timezone=Australia/Brisbane -Dtests.asserts=true 
> -Dtests.file.encoding=UTF-8
> [23:09:22.113] ERROR   0.23s J3 | 
> TestSimpleTextPostingsFormat.testInvertedWrite <<<
>> Throwable #1: java.lang.IllegalArgumentException: An SPI class of type 
> org.apache.lucene.codecs.PostingsFormat with name 'SimpleText' does not 
> exist.  You need to add the corresponding JAR file supporting this SPI to 
> yo

[jira] [Commented] (LUCENE-6590) Explore different ways to apply boosts

[Upayavira (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324676#comment-15324676
 ] 

Upayavira commented on LUCENE-6590:
---

Here's an example for 4.10 and the same query against 5.5 - note, it is a 
different doc though:

{code}
4.10 score 
  "2937439": {
"match": true,
"value": 5.5993805,
"description": "weight(description:obama in 394012)
[DefaultSimilarity], result of:",
"details": [
  {
"match": true,
"value": 5.5993805,
"description": "fieldWeight in 394012, product of:",
"details": [
  {
"match": true,
"value": 1,
"description": "tf(freq=1.0), with freq of:",
"details": [
  {
"match": true,
"value": 1,
"description": "termFreq=1.0"
  }
]
  },
  {
"match": true,
"value": 5.5993805,
"description": "idf(docFreq=56010, maxDocs=5568765)"
  },
  {
"match": true,
"value": 1,
"description": "fieldNorm(doc=394012)"
  }
]
  }
]
5.5 score 
  "2502281":{
"match":true,
"value":28.51136,
"description":"weight(description:obama in 43472) [], result
of:",
"details":[{
"match":true,
"value":28.51136,
"description":"score(doc=43472,freq=1.0), product of:",
"details":[{
"match":true,
"value":5.339603,
"description":"queryWeight, product of:",
"details":[{
"match":true,
"value":5.339603,
"description":"idf(docFreq=31905,
maxDocs=2446459)"},
  {
"match":true,
"value":1.0,
"description":"queryNorm"}]},
  {
"match":true,
"value":5.339603,
"description":"fieldWeight in 43472, product of:",
"details":[{
"match":true,
"value":1.0,
"description":"tf(freq=1.0), with freq of:",
"details":[{
"match":true,
"value":1.0,
"description":"termFreq=1.0"}]},
  {
"match":true,
"value":5.339603,
"description":"idf(docFreq=31905,
maxDocs=2446459)"},
  {
"match":true,
"value":1.0,
"description":"fieldNorm(doc=43472)"}]}]}]},
{code}

> Explore different ways to apply boosts
> --
>
> Key: LUCENE-6590
> URL: https://issues.apache.org/jira/browse/LUCENE-6590
> Project: Lucene - Core
>  Issue Type: Wish
>Reporter: Adrien Grand
>Priority: Minor
> Fix For: 5.4
>
> Attachments: LUCENE-6590.patch, LUCENE-6590.patch, LUCENE-6590.patch, 
> LUCENE-6590.patch, LUCENE-6590.patch, LUCENE-6590.patch, LUCENE-6590.patch
>
>
> Follow-up from LUCENE-6570: the fact that all queries are mutable in order to 
> allow for applying a boost raises issues since it makes queries bad cache 
> keys since their hashcode can change anytime. We could just document that 
> queries should never be modified after they have gone through IndexSearcher 
> but it would be even better if the API made queries impossible to mutate at 
> all.
> I think there are two main options:
>  - either replace "void setBoost(boost)" with something like "Query 
> withBoost(boost)" which would return a clone that has a different boost
>  - or move boost handling outside of Query, for instance we could have a 
> (immutable) query impl that would be dedicated to applying boosts, that 
> queries that need to change boosts at rewrite time (such as BooleanQuery) 
> would use as a wrapper.
> The latter idea is from Robert and I like it a lot given how often I either 
> introduced or found a bug which was due to the boost parameter being ignored. 
> Maybe there are other options, but I think this is worth exploring.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8993) New UI can't show DIH

[Upayavira (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324668#comment-15324668
 ] 

Upayavira commented on SOLR-8993:
-

I have a serious lack of spare time at the moment, and on top of that, I've 
never used the DIH.

Please feel free to try and provide a patch, in the meantime.

> New UI can't show DIH
> -
>
> Key: SOLR-8993
> URL: https://issues.apache.org/jira/browse/SOLR-8993
> Project: Solr
>  Issue Type: Bug
>  Components: UI
>Affects Versions: 6.0
>Reporter: Buğra Yıldırım
>Priority: Critical
> Attachments: SOLR-8993.patch, SOLR6.png, screenshot-1.png, 
> screenshot-2.png, screenshot-3.png, screenshot-4.png, screenshot-5.png, 
> screenshot-6.png
>
>
> New UI can't show DIHs when more than one DIH exists. I switch to old UI for 
> importing from database.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1038 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1038/

1 tests failed.
FAILED:  
org.apache.lucene.codecs.simpletext.TestSimpleTextPostingsFormat.testInvertedWrite

Error Message:
compound sub-files must have a valid codec header and footer: codec header 
mismatch: actual header=1718183276 vs expected header=1071082519 
(resource=BufferedChecksumIndexInput(MockIndexInputWrapper(RAMInputStream(name=_0_SimpleText_0.pst

Stack Trace:
org.apache.lucene.index.CorruptIndexException: compound sub-files must have a 
valid codec header and footer: codec header mismatch: actual header=1718183276 
vs expected header=1071082519 
(resource=BufferedChecksumIndexInput(MockIndexInputWrapper(RAMInputStream(name=_0_SimpleText_0.pst
at 
__randomizedtesting.SeedInfo.seed([9122A898E1FB4FCA:8856E5539FA0C416]:0)
at 
org.apache.lucene.codecs.CodecUtil.verifyAndCopyIndexHeader(CodecUtil.java:287)
at 
org.apache.lucene.codecs.lucene50.Lucene50CompoundFormat.write(Lucene50CompoundFormat.java:96)
at 
org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:4860)
at 
org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:516)
at 
org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:480)
at 
org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:526)
at 
org.apache.lucene.index.DocumentsWriter.postUpdate(DocumentsWriter.java:390)
at 
org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:486)
at 
org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1562)
at 
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1306)
at 
org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:171)
at 
org.apache.lucene.index.BasePostingsFormatTestCase.testInvertedWrite(BasePostingsFormatTestCase.java:515)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdap

RE: [JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_92) - Build # 239 - Still Failing!

[Uwe Schindler]

Hi,

 

This is indeed very strange. I have no idea!

 

Uwe

 

-

Uwe Schindler

H.-H.-Meier-Allee 63, D-28213 Bremen

  http://www.thetaphi.de

eMail: u...@thetaphi.de

 

From: Michael McCandless [mailto:luc...@mikemccandless.com] 
Sent: Friday, June 10, 2016 5:45 PM
To: Policeman Jenkins Server 
Cc: Adrien Grand ; Gregory Chanan ; 
dragonsi...@apache.org; Lucene/Solr dev 
Subject: Re: [JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_92) - Build # 
239 - Still Failing!

 

Hmm, why was access denied here?




Mike McCandless

http://blog.mikemccandless.com

 

On Thu, Jun 9, 2016 at 5:51 PM, Policeman Jenkins Server mailto:jenk...@thetaphi.de> > wrote:

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Windows/239/
Java: 32bit/jdk1.8.0_92 -client -XX:+UseConcMarkSweepGC

2 tests failed.
FAILED:  org.apache.lucene.store.TestMmapDirectory.testPendingDeletions

Error Message:
access denied ("java.io.FilePermission" 
"C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\lucene\build\core\test\J0\temp\lucene.store.TestMmapDirectory_96FC6F2D45B76809-001\tempDir-007\con"
 "write")

Stack Trace:
java.security.AccessControlException: access denied ("java.io.FilePermission" 
"C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\lucene\build\core\test\J0\temp\lucene.store.TestMmapDirectory_96FC6F2D45B76809-001\tempDir-007\con"
 "write")
at 
__randomizedtesting.SeedInfo.seed([96FC6F2D45B76809:DE018553DD164AB]:0)
at 
java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
at 
java.security.AccessController.checkPermission(AccessController.java:884)
at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
at sun.nio.fs.WindowsChannelFactory.open(WindowsChannelFactory.java:295)
at 
sun.nio.fs.WindowsChannelFactory.newFileChannel(WindowsChannelFactory.java:162)
at 
sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:225)
at 
java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434)
at 
org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
at 
org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
at 
org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
at 
org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
at 
org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
at 
org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
at java.nio.file.Files.newOutputStream(Files.java:216)
at 
org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:408)
at 
org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:404)
at 
org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:253)
at 
org.apache.lucene.store.BaseDirectoryTestCase.testPendingDeletions(BaseDirectoryTestCase.java:1215)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.Random

[jira] [Commented] (SOLR-8993) New UI can't show DIH

[JIRA]

[ 
https://issues.apache.org/jira/browse/SOLR-8993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324661#comment-15324661
 ] 

Buğra Yıldırım commented on SOLR-8993:
--

solr version 6.0.1 has been released but the problem continues. 

> New UI can't show DIH
> -
>
> Key: SOLR-8993
> URL: https://issues.apache.org/jira/browse/SOLR-8993
> Project: Solr
>  Issue Type: Bug
>  Components: UI
>Affects Versions: 6.0
>Reporter: Buğra Yıldırım
>Priority: Critical
> Attachments: SOLR-8993.patch, SOLR6.png, screenshot-1.png, 
> screenshot-2.png, screenshot-3.png, screenshot-4.png, screenshot-5.png, 
> screenshot-6.png
>
>
> New UI can't show DIHs when more than one DIH exists. I switch to old UI for 
> importing from database.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_92) - Build # 239 - Still Failing!

[Michael McCandless]

Hmm, why was access denied here?

Mike McCandless

http://blog.mikemccandless.com

On Thu, Jun 9, 2016 at 5:51 PM, Policeman Jenkins Server <
jenk...@thetaphi.de> wrote:

> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Windows/239/
> Java: 32bit/jdk1.8.0_92 -client -XX:+UseConcMarkSweepGC
>
> 2 tests failed.
> FAILED:  org.apache.lucene.store.TestMmapDirectory.testPendingDeletions
>
> Error Message:
> access denied ("java.io.FilePermission"
> "C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\lucene\build\core\test\J0\temp\lucene.store.TestMmapDirectory_96FC6F2D45B76809-001\tempDir-007\con"
> "write")
>
> Stack Trace:
> java.security.AccessControlException: access denied
> ("java.io.FilePermission"
> "C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\lucene\build\core\test\J0\temp\lucene.store.TestMmapDirectory_96FC6F2D45B76809-001\tempDir-007\con"
> "write")
> at
> __randomizedtesting.SeedInfo.seed([96FC6F2D45B76809:DE018553DD164AB]:0)
> at
> java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> at
> java.security.AccessController.checkPermission(AccessController.java:884)
> at
> java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
> at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
> at
> sun.nio.fs.WindowsChannelFactory.open(WindowsChannelFactory.java:295)
> at
> sun.nio.fs.WindowsChannelFactory.newFileChannel(WindowsChannelFactory.java:162)
> at
> sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:225)
> at
> java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434)
> at
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
> at
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
> at
> org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
> at
> org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
> at
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
> at
> org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
> at java.nio.file.Files.newOutputStream(Files.java:216)
> at
> org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:408)
> at
> org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:404)
> at
> org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:253)
> at
> org.apache.lucene.store.BaseDirectoryTestCase.testPendingDeletions(BaseDirectoryTestCase.java:1215)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
> at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
> at
> org.ap

[jira] [Commented] (LUCENE-7325) GeoPointInBBoxQuery no longer includes boundaries?

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324658#comment-15324658
 ] 

Michael McCandless commented on LUCENE-7325:


bq.  It was uncovered after Robert suggested removing GeoPointTestUtil. After 
that bug is fixed all tests pass w/ GeoTestUtil.

OK, thanks for the explanation.  When that bug strikes what exception do you 
see?

> GeoPointInBBoxQuery no longer includes boundaries?
> --
>
> Key: LUCENE-7325
> URL: https://issues.apache.org/jira/browse/LUCENE-7325
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 6.1
>Reporter: Michael McCandless
>Priority: Blocker
> Attachments: LUCENE-7325.patch, LUCENE-7325.patch
>
>
> {{GeoPointInBBoxQuery}} is supposed to include boundaries, and it does in 5.x 
> and 6.0, but in 6.1 something broke.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-7325) GeoPointInBBoxQuery no longer includes boundaries?

[Nicholas Knize (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324654#comment-15324654
 ] 

Nicholas Knize edited comment on LUCENE-7325 at 6/10/16 3:41 PM:
-

Thanks [~mikemccand]!

bq. Can you add an explicit test case showing that bug?

sure thing. It was uncovered after Robert suggested removing 
{{GeoPointTestUtil}}. After that bug is fixed all tests pass w/ {{GeoTestUtil}}.

bq. I see no comment there?

Sorry, the first issue comment in LUCENE-7166.

bq. But, today, GeoPointField is still using 62 bit encoding right?

That's correct.




was (Author: nknize):
Thanks [~mikemccand]!

bq. Can you add an explicit test case showing that bug?

sure thing. It was uncovered after Robert suggested removing 
{{GeoPointTestUtil}}. After that bug is fixed all tests pass w/ {{GeoTestUtil}}.

bq. I see no comment there?

Sorry, the first issue comment in LUCENE-7166.

But, today, GeoPointField is still using 62 bit encoding right?

That's correct.



> GeoPointInBBoxQuery no longer includes boundaries?
> --
>
> Key: LUCENE-7325
> URL: https://issues.apache.org/jira/browse/LUCENE-7325
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 6.1
>Reporter: Michael McCandless
>Priority: Blocker
> Attachments: LUCENE-7325.patch, LUCENE-7325.patch
>
>
> {{GeoPointInBBoxQuery}} is supposed to include boundaries, and it does in 5.x 
> and 6.0, but in 6.1 something broke.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-7325) GeoPointInBBoxQuery no longer includes boundaries?

[Nicholas Knize (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324654#comment-15324654
 ] 

Nicholas Knize edited comment on LUCENE-7325 at 6/10/16 3:41 PM:
-

Thanks [~mikemccand]!

bq. Can you add an explicit test case showing that bug?

sure thing. It was uncovered after Robert suggested removing 
{{GeoPointTestUtil}}. After that bug is fixed all tests pass w/ {{GeoTestUtil}}.

bq. I see no comment there?

Sorry, the first issue comment in LUCENE-7166.

But, today, GeoPointField is still using 62 bit encoding right?

That's correct.




was (Author: nknize):
Thanks [~mikemccand]!

bq. Can you add an explicit test case showing that bug?

sure thing. It was uncovered after Robert suggested removing 
{{GeoPointTestUtil}}. After that bug is fixed all tests pass w/ {{GeoTestUtil}}.

bq. I see no comment there?

Sorry, the first issue comment in LUCENE-7166.





> GeoPointInBBoxQuery no longer includes boundaries?
> --
>
> Key: LUCENE-7325
> URL: https://issues.apache.org/jira/browse/LUCENE-7325
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 6.1
>Reporter: Michael McCandless
>Priority: Blocker
> Attachments: LUCENE-7325.patch, LUCENE-7325.patch
>
>
> {{GeoPointInBBoxQuery}} is supposed to include boundaries, and it does in 5.x 
> and 6.0, but in 6.1 something broke.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7325) GeoPointInBBoxQuery no longer includes boundaries?

[Nicholas Knize (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324654#comment-15324654
 ] 

Nicholas Knize commented on LUCENE-7325:


Thanks [~mikemccand]!

bq. Can you add an explicit test case showing that bug?

sure thing. It was uncovered after Robert suggested removing 
{{GeoPointTestUtil}}. After that bug is fixed all tests pass w/ {{GeoTestUtil}}.

bq. I see no comment there?

Sorry, the first issue comment in LUCENE-7166.





> GeoPointInBBoxQuery no longer includes boundaries?
> --
>
> Key: LUCENE-7325
> URL: https://issues.apache.org/jira/browse/LUCENE-7325
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 6.1
>Reporter: Michael McCandless
>Priority: Blocker
> Attachments: LUCENE-7325.patch, LUCENE-7325.patch
>
>
> {{GeoPointInBBoxQuery}} is supposed to include boundaries, and it does in 5.x 
> and 6.0, but in 6.1 something broke.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7287) New lemma-tizer plugin for ukrainian language.

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324651#comment-15324651
 ] 

Michael McCandless commented on LUCENE-7287:


Thanks for the detailed analysis [~arysin]!  On where the dictionary lives, I 
think option 2 is good?

On #4, whichever is best for you!

> New lemma-tizer plugin for ukrainian language.
> --
>
> Key: LUCENE-7287
> URL: https://issues.apache.org/jira/browse/LUCENE-7287
> Project: Lucene - Core
>  Issue Type: New Feature
>  Components: modules/analysis
>Reporter: Dmytro Hambal
>Priority: Minor
>  Labels: analysis, language, plugin
>
> Hi all,
> I wonder whether you are interested in supporting a plugin which provides a 
> mapping between ukrainian word forms and their lemmas. Some tests and docs go 
> out-of-the-box =) .
> https://github.com/mrgambal/elasticsearch-ukrainian-lemmatizer
> It's really simple but still works and generates some value for its users.
> More: https://github.com/elastic/elasticsearch/issues/18303



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7326) TestSimpleTextPostingsFormat.testInvertedWrite() failure: An SPI class of type PostingsFormat with name 'SimpleText' does not exist

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324639#comment-15324639
 ] 

Michael McCandless commented on LUCENE-7326:


Woops, I'll dig ... I think maybe I can fix the test to use {{FilterCodec}} so 
we can keep testing {{SimpleText}}'s postings format here ... I'll try.  Thanks 
[~steve_rowe] and [~jpountz]!

> TestSimpleTextPostingsFormat.testInvertedWrite() failure: An SPI class of 
> type PostingsFormat with name 'SimpleText' does not exist
> ---
>
> Key: LUCENE-7326
> URL: https://issues.apache.org/jira/browse/LUCENE-7326
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Steve Rowe
>
> My Jenkins found a master seed that reproduces on branch_6x but not on 
> branch_6_1:
> {noformat}
> Checking out Revision 963206969eddca6ec743f5f0901e0abdfeacd3cc 
> (refs/remotes/origin/master)
> [...]
>   2> NOTE: reproduce with: ant test  -Dtestcase=TestSimpleTextPostingsFormat 
> -Dtests.method=testInvertedWrite -Dtests.seed=7555A4CF724BDB74 
> -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true 
> -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt 
> -Dtests.locale=es -Dtests.timezone=Australia/Brisbane -Dtests.asserts=true 
> -Dtests.file.encoding=UTF-8
> [23:09:22.113] ERROR   0.23s J3 | 
> TestSimpleTextPostingsFormat.testInvertedWrite <<<
>> Throwable #1: java.lang.IllegalArgumentException: An SPI class of type 
> org.apache.lucene.codecs.PostingsFormat with name 'SimpleText' does not 
> exist.  You need to add the corresponding JAR file supporting this SPI to 
> your classpath.  The current classpath supports the following names: 
> [MockRandom, RAMOnly, LuceneFixedGap, LuceneVarGapFixedInterval, 
> LuceneVarGapDocFreqInterval, TestBloomFilteredLucenePostings, Asserting, 
> BlockTreeOrds, BloomFilter, Direct, FSTOrd50, FST50, Memory, AutoPrefix, 
> Lucene50]
>>at 
> __randomizedtesting.SeedInfo.seed([7555A4CF724BDB74:6C21E9040C1050A8]:0)
>>at 
> org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:116)
>>at 
> org.apache.lucene.codecs.PostingsFormat.forName(PostingsFormat.java:112)
>>at 
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.(PerFieldPostingsFormat.java:258)
>>at 
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
>>at 
> org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:106)
>>at org.apache.lucene.index.SegmentReader.(SegmentReader.java:67)
>>at 
> org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:61)
>>at 
> org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:53)
>>at 
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:675)
>>at 
> org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:76)
>>at 
> org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:63)
>>at 
> org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:383)
>>at 
> org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:313)
>>at 
> org.apache.lucene.index.BasePostingsFormatTestCase.testInvertedWrite(BasePostingsFormatTestCase.java:519)
> [...]
>   2> NOTE: test params are: codec=Asserting(Lucene62): {}, docValues:{}, 
> maxPointsInLeafNode=1657, maxMBSortInHeap=5.09627287608914, 
> sim=RandomSimilarity(queryNorm=true,coord=crazy): {f_DOCS_AND_FREQS=DFR 
> I(ne)Z(0.3), field=DFR I(ne)L3(800.0), 
> f_DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS=DFI(Saturated), 
> f_DOCS_AND_FREQS_AND_POSITIONS=IB SPL-L1, titleTokenized=IB LL-L3(800.0), 
> body=DFR I(F)L1, f_DOCS=DFR I(n)L2}, locale=es, timezone=Australia/Brisbane
>   2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation 1.8.0_77 
> (64-bit)/cpus=16,threads=1,free=257107688,total=486539264
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8812) ExtendedDismaxQParser (edismax) ignores Boolean OR when q.op=AND and mm is not explicitly set

[Steve Rowe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Rowe updated SOLR-8812:
-
Fix Version/s: master (7.0)
   5.6

> ExtendedDismaxQParser (edismax) ignores Boolean OR when q.op=AND and mm is 
> not explicitly set
> -
>
> Key: SOLR-8812
> URL: https://issues.apache.org/jira/browse/SOLR-8812
> Project: Solr
>  Issue Type: Bug
>  Components: query parsers
>Affects Versions: 5.5
>Reporter: Ryan Steinberg
>Assignee: Steve Rowe
> Fix For: 5.6, 6.1, 5.5.2, master (7.0), 6.0.2
>
> Attachments: SOLR-8812-barbie.patch, SOLR-8812.patch, 
> SOLR-8812.patch, SOLR-8812.patch
>
>
> The edismax parser ignores Boolean OR in queries when q.op=AND. This behavior 
> is new to Solr 5.5.0 and an unexpected major change.
> Example:
>   "q": "id:12345 OR zz",
>   "defType": "edismax",
>   "q.op": "AND",
> where "12345" is a known document ID and "zz" is a string NOT present 
> in my data
> Version 5.5.0 produces zero results:
> "rawquerystring": "id:12345 OR zz",
> "querystring": "id:12345 OR zz",
> "parsedquery": "(+((id:12345 
> DisjunctionMaxQuery((text:zz)))~2))/no_coord",
> "parsedquery_toString": "+((id:12345 (text:zz))~2)",
> "explain": {},
> "QParser": "ExtendedDismaxQParser"
> Version 5.4.0 produces one result as expected
>   "rawquerystring": "id:12345 OR zz",
> "querystring": "id:12345 OR zz",
> "parsedquery": "(+(id:12345 
> DisjunctionMaxQuery((text:zz/no_coord",
> "parsedquery_toString": "+(id:12345 (text:zz))"
> "explain": {},
> "QParser": "ExtendedDismaxQParser"



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9203) Cross Data Center Replication for existing Indexes

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324620#comment-15324620
 ] 

Erick Erickson commented on SOLR-9203:
--

First, you'll probably get better/faster responses if you raise these kinds of 
questions on the user's list first, then we can raise JIRAs for specific issues.

The first bit about the index out of bounds certainly seems like something that 
could be improved. 

Not quite sure I understand the second point. CdcrUpdateLog class is explicitly 
shown in both the Source and Target examples in the Configuration section. The 
First Time section is meant to illustrated specific points, not repeat the 
entire configuration discussion. I've added a note to that effect. Or is it 
something else?





> Cross Data Center Replication for existing Indexes
> --
>
> Key: SOLR-9203
> URL: https://issues.apache.org/jira/browse/SOLR-9203
> Project: Solr
>  Issue Type: Bug
>  Components: CDCR
>Affects Versions: 6.0
>Reporter: Mike Rocke
>
> Hi all,
> After several attempts I am unable to get CDCR working for a solr core with 
> existing documents.
> My first attempt failed due to incompatibility with the tlog file names,
> Caused by: java.lang.StringIndexOutOfBoundsException: String index out of 
> range: -1
> at java.lang.String.substring(String.java:1967)
> at 
> org.apache.solr.update.CdcrUpdateLog.getLastLogId(CdcrUpdateLog.java:154)
> at org.apache.solr.update.UpdateLog.init(UpdateLog.java:282)
> at org.apache.solr.update.CdcrUpdateLog.init(CdcrUpdateLog.java:77)
> at org.apache.solr.update.UpdateHandler.(UpdateHandler.java:137)
> This is going from having an update handler of 
> 
>   /somedir/
> 
> to 
> 
>   /somedir
> 
> I believe this is due to the file patterns differing for the default update 
> log class and the CdcrUpdateLog class. 
> I then continued to follow 
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=62687462#CrossDataCenterReplication(CDCR)-FirstTime
> which does not describe having the CdcrUpdateLog class, to which solr will 
> now start erroring with 
> Caused by: org.apache.solr.common.SolrException: Solr instance is not 
> configured with the cdcr update log.
>   at org.apache.solr.core.SolrCore.(SolrCore.java:771)
>   at org.apache.solr.core.SolrCore.(SolrCore.java:642)
>   at org.apache.solr.core.CoreContainer.create(CoreContainer.java:817)
>   at org.apache.solr.core.CoreContainer.access$000(CoreContainer.java:88)
>   at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:468)
>   at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:459)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   ... 1 more
> Caused by: org.apache.solr.common.SolrException: Solr instance is not 
> configured with the cdcr update log.
>   at 
> org.apache.solr.handler.CdcrRequestHandler.inform(CdcrRequestHandler.java:223)
>   at 
> org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:675)
>   at org.apache.solr.core.SolrCore.(SolrCore.java:751)
>   ... 10 more
> I may have completely missunderstood but please advise?
> Thanks
> Mike



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7328) Remove LegacyNumericEncoding from GeoPointField

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324616#comment-15324616
 ] 

Michael McCandless commented on LUCENE-7328:


+1

> Remove LegacyNumericEncoding from GeoPointField
> ---
>
> Key: LUCENE-7328
> URL: https://issues.apache.org/jira/browse/LUCENE-7328
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: master (7.0)
>Reporter: Nicholas Knize
> Attachments: LUCENE-7328.patch
>
>
> 6.0 deprecated legacy {{NumericEncoding}} in {{GeoPointField}} this issue 
> completely removes the encoding for 7.0.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8812) ExtendedDismaxQParser (edismax) ignores Boolean OR when q.op=AND and mm is not explicitly set

[Steve Rowe (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324602#comment-15324602
 ] 

Steve Rowe commented on SOLR-8812:
--

bq. Sounds (tentatively) ok to me. I was quite concerned when you said it puts 
things back to pre-SOLR-2649 functionality, but from looking at what got 
committed it seems that q.op=OR is no longer hardcoded in setDefaultOperator() 
(which was fixed in SOLR-2649). I haven't executed anything, but this seems 
like a good step with regards to mm handling.

Greg, what I committed is behaviorally the same as your patch - check the unit 
tests: I included all of the ones you wrote, but added more to cover all 
permutations of operator and {{q.op}}.  

What I meant by "put things back" was that I attempted to minimize the diff 
between what I committed and the pre-SOLR-2649 code, so I put your 
{{foundOperators()}} back in the same location that it was when it was named 
{{doMinMatched()}} - your patch put it in a different location in the source 
file.

Thanks for the work you did on this issue!

> ExtendedDismaxQParser (edismax) ignores Boolean OR when q.op=AND and mm is 
> not explicitly set
> -
>
> Key: SOLR-8812
> URL: https://issues.apache.org/jira/browse/SOLR-8812
> Project: Solr
>  Issue Type: Bug
>  Components: query parsers
>Affects Versions: 5.5
>Reporter: Ryan Steinberg
>Assignee: Steve Rowe
> Fix For: 6.1, 5.5.2, 6.0.2
>
> Attachments: SOLR-8812-barbie.patch, SOLR-8812.patch, 
> SOLR-8812.patch, SOLR-8812.patch
>
>
> The edismax parser ignores Boolean OR in queries when q.op=AND. This behavior 
> is new to Solr 5.5.0 and an unexpected major change.
> Example:
>   "q": "id:12345 OR zz",
>   "defType": "edismax",
>   "q.op": "AND",
> where "12345" is a known document ID and "zz" is a string NOT present 
> in my data
> Version 5.5.0 produces zero results:
> "rawquerystring": "id:12345 OR zz",
> "querystring": "id:12345 OR zz",
> "parsedquery": "(+((id:12345 
> DisjunctionMaxQuery((text:zz)))~2))/no_coord",
> "parsedquery_toString": "+((id:12345 (text:zz))~2)",
> "explain": {},
> "QParser": "ExtendedDismaxQParser"
> Version 5.4.0 produces one result as expected
>   "rawquerystring": "id:12345 OR zz",
> "querystring": "id:12345 OR zz",
> "parsedquery": "(+(id:12345 
> DisjunctionMaxQuery((text:zz/no_coord",
> "parsedquery_toString": "+(id:12345 (text:zz))"
> "explain": {},
> "QParser": "ExtendedDismaxQParser"



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org