date:20140820


[ 
https://issues.apache.org/jira/browse/LUCENE-5890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14103544#comment-14103544
 ] 

Uwe Schindler commented on LUCENE-5890:
---

I added JDK 1.9.0 b26 in 32 and 64 bits to the Linux Policeman Jenkins. One 
successful build already:

- http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/10933/

This one failed, but with one well-known error (TestManagedResourceStorage):

- http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/11059/
- http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/11058/

 Add Java9 previews to Policeman Jenkins
 ---

 Key: LUCENE-5890
 URL: https://issues.apache.org/jira/browse/LUCENE-5890
 Project: Lucene - Core
  Issue Type: Task
  Components: general/test
 Environment: Policeman Jenkins
Reporter: Uwe Schindler
Assignee: Uwe Schindler
  Labels: Java9

 Since we solved several issues on our side (missing constants, build target 
 constraints, a test failure) and one bug in OpenJDK9 (fixed lowercasing bug 
 in JDK9 preview build 25) we are able to run our ant jenkins target also on 
 Java 9.
 This issue is here to track problems occuring with randomly also building on 
 Java 9.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-5890) Add Java9 previews to Policeman Jenkins


 [ 
https://issues.apache.org/jira/browse/LUCENE-5890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler resolved LUCENE-5890.
---

Resolution: Fixed

I will add Windows and Mac builds later, once JDK9 is proceeding. For now Linux 
is good to find API and similar issues.

 Add Java9 previews to Policeman Jenkins
 ---

 Key: LUCENE-5890
 URL: https://issues.apache.org/jira/browse/LUCENE-5890
 Project: Lucene - Core
  Issue Type: Task
  Components: general/test
 Environment: Policeman Jenkins
Reporter: Uwe Schindler
Assignee: Uwe Schindler
  Labels: Java9

 Since we solved several issues on our side (missing constants, build target 
 constraints, a test failure) and one bug in OpenJDK9 (fixed lowercasing bug 
 in JDK9 preview build 25) we are able to run our ant jenkins target also on 
 Java 9.
 This issue is here to track problems occuring with randomly also building on 
 Java 9.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6314) Multi-threaded facet counts differ when SolrCloud has 1 shard

2014-08-20 Thread Vamsee Yarlagadda (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-6314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14103564#comment-14103564
 ] 

Vamsee Yarlagadda commented on SOLR-6314:
-

The patch looks good to me. 
Thanks [~erickerickson].

 Multi-threaded facet counts differ when SolrCloud has 1 shard
 --

 Key: SOLR-6314
 URL: https://issues.apache.org/jira/browse/SOLR-6314
 Project: Solr
  Issue Type: Bug
  Components: SearchComponents - other, SolrCloud
Affects Versions: 5.0
Reporter: Vamsee Yarlagadda
Assignee: Erick Erickson
 Fix For: 5.0, 4.10

 Attachments: SOLR-6314.patch, SOLR-6314.patch, SOLR-6314.patch


 I am trying to work with multi-threaded faceting on SolrCloud and in the 
 process i was hit by some issues.
 I am currently running the below upstream test on different SolrCloud 
 configurations and i am getting a different result set per configuration.
 https://github.com/apache/lucene-solr/blob/trunk/solr/core/src/test/org/apache/solr/request/TestFaceting.java#L654
 Setup:
 - *Indexed 50 docs into SolrCloud.*
 - *If the SolrCloud has only 1 shard, the facet field query has the below 
 output (which matches with the expected upstream test output - # facet fields 
 ~ 50).*
 {code}
 $ curl  
 http://localhost:8983/solr/collection1/select?facet=truefl=idindent=trueq=id%3A*facet.limit=-1facet.threads=1000facet.field=f0_wsfacet.field=f0_wsfacet.field=f0_wsfacet.field=f0_wsfacet.field=f0_wsfacet.field=f1_wsfacet.field=f1_wsfacet.field=f1_wsfacet.field=f1_wsfacet.field=f1_wsfacet.field=f2_wsfacet.field=f2_wsfacet.field=f2_wsfacet.field=f2_wsfacet.field=f2_wsfacet.field=f3_wsfacet.field=f3_wsfacet.field=f3_wsfacet.field=f3_wsfacet.field=f3_wsfacet.field=f4_wsfacet.field=f4_wsfacet.field=f4_wsfacet.field=f4_wsfacet.field=f4_wsfacet.field=f5_wsfacet.field=f5_wsfacet.field=f5_wsfacet.field=f5_wsfacet.field=f5_wsfacet.field=f6_wsfacet.field=f6_wsfacet.field=f6_wsfacet.field=f6_wsfacet.field=f6_wsfacet.field=f7_wsfacet.field=f7_wsfacet.field=f7_wsfacet.field=f7_wsfacet.field=f7_wsfacet.field=f8_wsfacet.field=f8_wsfacet.field=f8_wsfacet.field=f8_wsfacet.field=f8_wsfacet.field=f9_wsfacet.field=f9_wsfacet.field=f9_wsfacet.field=f9_wsfacet.field=f9_wsrows=1wt=xml;
 ?xml version=1.0 encoding=UTF-8?
 response
 lst name=responseHeader
   int name=status0/int
   int name=QTime21/int
   lst name=params
 str name=facettrue/str
 str name=flid/str
 str name=indenttrue/str
 str name=qid:*/str
 str name=facet.limit-1/str
 str name=facet.threads1000/str
 arr name=facet.field
   strf0_ws/str
   strf0_ws/str
   strf0_ws/str
   strf0_ws/str
   strf0_ws/str
   strf1_ws/str
   strf1_ws/str
   strf1_ws/str
   strf1_ws/str
   strf1_ws/str
   strf2_ws/str
   strf2_ws/str
   strf2_ws/str
   strf2_ws/str
   strf2_ws/str
   strf3_ws/str
   strf3_ws/str
   strf3_ws/str
   strf3_ws/str
   strf3_ws/str
   strf4_ws/str
   strf4_ws/str
   strf4_ws/str
   strf4_ws/str
   strf4_ws/str
   strf5_ws/str
   strf5_ws/str
   strf5_ws/str
   strf5_ws/str
   strf5_ws/str
   strf6_ws/str
   strf6_ws/str
   strf6_ws/str
   strf6_ws/str
   strf6_ws/str
   strf7_ws/str
   strf7_ws/str
   strf7_ws/str
   strf7_ws/str
   strf7_ws/str
   strf8_ws/str
   strf8_ws/str
   strf8_ws/str
   strf8_ws/str
   strf8_ws/str
   strf9_ws/str
   strf9_ws/str
   strf9_ws/str
   strf9_ws/str
   strf9_ws/str
 /arr
 str name=wtxml/str
 str name=rows1/str
   /lst
 /lst
 result name=response numFound=50 start=0
   doc
 float name=id0.0/float/doc
 /result
 lst name=facet_counts
   lst name=facet_queries/
   lst name=facet_fields
 lst name=f0_ws
   int name=zero_125/int
   int name=zero_225/int
 /lst
 lst name=f0_ws
   int name=zero_125/int
   int name=zero_225/int
 /lst
 lst name=f0_ws
   int name=zero_125/int
   int name=zero_225/int
 /lst
 lst name=f0_ws
   int name=zero_125/int
   int name=zero_225/int
 /lst
 lst name=f0_ws
   int name=zero_125/int
   int name=zero_225/int
 /lst
 lst name=f1_ws
   int name=one_133/int
   int name=one_317/int
 /lst
 lst name=f1_ws
   int name=one_133/int
   int name=one_317/int
 /lst
 lst name=f1_ws
   int name=one_133/int
   int name=one_317/int
 /lst
 lst name=f1_ws
   int name=one_133/int
   int name=one_317/int
 /lst
 lst name=f1_ws
   int name=one_133/int
   int name=one_317/int
 /lst
 lst name=f2_ws
   int name=two_137/int
   int name=two_413/int
 /lst
 lst name=f2_ws

[jira] [Commented] (SOLR-5683) Documentation of Suggester V2

2014-08-20 Thread Varun Thacker (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14103562#comment-14103562
]

Varun Thacker commented on SOLR-5683:
-

First draft at documenting the suggesters - This covers all the documentation
wrt suggesters under Issues from CHANGES.txt that were never doc'ed as part of
their release: in the
https://cwiki.apache.org/confluence/display/solr/Internal+-+TODO+List link.

bq. DocumentDictionaryFactory – user can specify suggestion field along with
optional weight and payload fields from their search index.

Looking at the code of DocumentDictionaryFactory the weight field is not
optional.

field - The field from which the suggesters dictionary will be populated.
weightField - The field from which the suggestions weight will be populated.
This should be a numeric field. Suggestions will be sorted based on the value
as this is the sole criteria for relevance.
payloadField - Accompanying payload for each suggestion that gets built.
suggestAnalyzerFieldType - Specify the analyzer to be used for the suggester.
The index analyzer of this fieldType will be used to build the suggest
dictionary and the query analyzer will be used during querying.

Config (index time) options:
name - Name of suggester. This is optional if you have only one suggester
defined.
sourceLocation - External file location for file-based suggesters only.
lookupImpl - Type of lookup to use whose default is JaspellLookupFactory. A
table below lists all the various lookup implementations present.
dictionaryImpl - The type of dictionary to be used when building the suggester.
The default is FileDictionaryFactory for a file-based suggester and it defaults
to HighFrequencyDictionaryFactory otherwise.
storeDir - Location to store the dictionary on disk.
buildOnCommit - Command to build suggester automatically after every commit
that is called. Useful if you want to keep the suggester in sync with your
latest data.
buildOnOptimize - Command to build suggester automatically after every optimize
that is called. Useful if you want to keep the suggester in sync with your
latest data.

Query time options:
suggest.dictionary - name of suggester to use
suggest.count - number of suggestions to return
suggest.q - query to use for lookup
suggest.build - command to build the suggester
suggest.reload - command to reload the suggester
buildAll – command to build all suggesters in the component
reloadAll – command to reload all suggesters in the component

Lookup Implementation Options -
- AnalyzingLookupFactory: Suggester that first analyzes the incoming text and
adds the analyzed form to a weighted FST, and then does the same thing at
lookup time.
suggestAnalyzerFieldType - The analyzer used at query-time and
build-time to analyze suggestions.
exactMatchFirst - If true the exact suggestions are returned first,
even if they are prefixes of other strings in the FST have larger weights.
Default is true.
preserveSep - If true then a separator between tokens is preserved.
This means that suggestions are sensitive to tokenization (e.g. baseball is
different from base ball. Default is true.
preservePositionIncrements - Whether the suggester should preserve
position increments. What this means is that token filters which leave gaps
(for example when StopFilter matches a stopword) the position would be
respected when building the suggester. The default is false.

- FuzzyLookupFactory: This is a suggester which is an extension of the
AnalyzingSuggester but is fuzzy in nature. The similarity is measured by the
Levenshtein algorithm.
exactMatchFirst - If true the exact suggestions are returned first,
even if they are prefixes of other strings in the FST have larger weights.
Default is true.
preserveSep - If true then a separator between tokens is preserved.
This means that suggestions are sensitive to tokenization (e.g. baseball is
different from base ball. Default is true.
maxSurfaceFormsPerAnalyzedForm - Maximum number of surface forms to
keep for a single analyzed form. When there are too many surface forms we
discard the lowest weighted ones.
maxGraphExpansions - When building the FST (index-time), we add each
path through the tokenstream graph as an individual entry. This places an
upper-bound on how many expansions will be added for a single suggestion. The
default is -1 which means there is no limit.
preservePositionIncrements - Whether the suggester should preserve
position increments. What this means is that token filters

Re: IndexReader.removeReaderClosedListener

2014-08-20 Thread Michael McCandless

Hmm, you should call this method before closing the IndexReader, to
remove a listener you previously added.

Mike McCandless

http://blog.mikemccandless.com


On Tue, Aug 19, 2014 at 11:27 AM, Phil Herold herold.p...@gmail.com wrote:
 The implementation of this final method (latest 4.9 code) calls
 ensureOpen(), which fails, since the reader is closed. As a result, this
 method doesn't work. It seems as if this is therefore a potential memory
 leak,  not being able to remove the listener. Or am I missing something?

 --
 Phil

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5893) FreeTextSuggester can now use Files.createTempDirectory()


[ 
https://issues.apache.org/jira/browse/LUCENE-5893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14103584#comment-14103584
 ] 

Michael McCandless commented on LUCENE-5893:


Thanks [~varunthacker] this looks good, except I think we no longer need to add 
the random.nextInt(Integer.MAX_VALUE) part?  Ie, this API will find a unique 
name for us from the prefix we provide?

 FreeTextSuggester can now use Files.createTempDirectory()
 -

 Key: LUCENE-5893
 URL: https://issues.apache.org/jira/browse/LUCENE-5893
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Varun Thacker
Priority: Trivial
 Attachments: LUCENE-5893.patch


 Came across the TODO in the code and now it's possible to use 
 Files.createTempDirectory since 4x is also on Java 7. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-4x-Linux-Java7-64-test-only - Build # 29124 - Failure!

2014-08-20 Thread Michael McCandless

Thanks Ryan!

Mike McCandless

http://blog.mikemccandless.com


On Mon, Aug 18, 2014 at 1:21 PM, Ryan Ernst r...@iernst.net wrote:
 I pushed a fix.  This was a bug in waitForMerges: it needs to allow
 the schedule to run one last time to clear pending merges.

 On Mon, Aug 18, 2014 at 8:43 AM, Ryan Ernst r...@iernst.net wrote:
 I'm looking into this.

 On Mon, Aug 18, 2014 at 7:59 AM,  buil...@flonkings.com wrote:
 Build: builds.flonkings.com/job/Lucene-4x-Linux-Java7-64-test-only/29124/

 2 tests failed.
 FAILED:  
 junit.framework.TestSuite.org.apache.lucene.index.TestIndexWriterExceptions2

 Error Message:
 Suite timeout exceeded (= 720 msec).

 Stack Trace:
 java.lang.Exception: Suite timeout exceeded (= 720 msec).
 at __randomizedtesting.SeedInfo.seed([21F5C86C935992D1]:0)


 REGRESSION:  org.apache.lucene.index.TestIndexWriterExceptions2.testBasics

 Error Message:
 Test abandoned because suite timeout was reached.

 Stack Trace:
 java.lang.Exception: Test abandoned because suite timeout was reached.
 at __randomizedtesting.SeedInfo.seed([21F5C86C935992D1]:0)




 Build Log:
 [...truncated 1756 lines...]
[junit4] Suite: org.apache.lucene.index.TestIndexWriterExceptions2
[junit4]   2 ?g?. 18, 2014 4:58:04 PM 
 com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4]   2 WARNING: Suite execution timed out: 
 org.apache.lucene.index.TestIndexWriterExceptions2
[junit4]   2  jstack at approximately timeout time 
[junit4]   2 
 TEST-TestIndexWriterExceptions2.testBasics-seed#[21F5C86C935992D1] ID=515 
 TIMED_WAITING on org.apache.lucene.index.IndexWriter@b711810
[junit4]   2at java.lang.Object.wait(Native Method)
[junit4]   2- timed waiting on 
 org.apache.lucene.index.IndexWriter@b711810
[junit4]   2at 
 org.apache.lucene.index.IndexWriter.doWait(IndexWriter.java:4434)
[junit4]   2at 
 org.apache.lucene.index.IndexWriter.waitForMerges(IndexWriter.java:2384)
[junit4]   2at 
 org.apache.lucene.index.IndexWriter.finishMerges(IndexWriter.java:2368)
[junit4]   2at 
 org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:910)
[junit4]   2at 
 org.apache.lucene.index.IndexWriter.close(IndexWriter.java:982)
[junit4]   2at 
 org.apache.lucene.index.IndexWriter.close(IndexWriter.java:952)
[junit4]   2at 
 org.apache.lucene.index.TestIndexWriterExceptions2.testBasics(TestIndexWriterExceptions2.java:206)
[junit4]   2at 
 sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4]   2at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4]   2at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4]   2at java.lang.reflect.Method.invoke(Method.java:606)
[junit4]   2at 
 com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
[junit4]   2at 
 com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
[junit4]   2at 
 com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4]   2at 
 com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
[junit4]   2at 
 org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4]   2at 
 org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4]   2at 
 org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4]   2at 
 com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4]   2at 
 org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4]   2at 
 org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[junit4]   2at 
 org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4]   2at 
 com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4]   2at 
 com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
[junit4]   2at 
 com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
[junit4]   2at 
 com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
[junit4]   2at 
 com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
[junit4]   2at

[jira] [Commented] (SOLR-6392) If run Solr having two collections configured but only one config delivered to Zookeeper causes that config is applied for all collections

2014-08-20 Thread Daniel Collins (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-6392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14103598#comment-14103598
 ] 

Daniel Collins commented on SOLR-6392:
--

By deliver config, I am assuming you mean upload that config to zk? Do you 
use the zkcli.sh script for that or do you do it via some other means?

What I think you are saying is:

# You have 2 collections which should be using independent configurations (both 
stored in ZK).
# If you change config1 (and restart Solr), that takes effect (in collection1 
or both?)
# If you change config2 (and restart Solr), there is no apparent effect?

First question is then, are you sure both collections are using different 
configs, or have they somehow both picked up the same config?  How did you set 
them up, and how did you define which config each collection uses?
There used to be a fall-back approach in Solr, if you started a core but 
didn't tell it to use any config from ZK AND there was only 1 possible config 
in ZK, the Solr guessed that was what you meant and set up the links.

I would guess that might what's happened here, so both your collections are 
actually using config1.

Check in ZK, under the /collections/collectionName node there should be an 
element called configName which maps to the configuration in ZK.
If that is wrong, you need to correct that, which is the -linkconfig option 
in zkcli.sh

But as Erick says, this would be better discussed on the list.

 If run Solr having two collections configured but only one config delivered 
 to Zookeeper causes that config is applied for all collections
 --

 Key: SOLR-6392
 URL: https://issues.apache.org/jira/browse/SOLR-6392
 Project: Solr
  Issue Type: Bug
Affects Versions: 4.4
Reporter: Ilya Meleshkov

 I have simplest Solr cloud configured locally. Thus I have single Solr and 
 Zookeeper nodes. 
 Steps to reproduce an error:
 # have stopped Solr+ZK with two collections
 # run ZK
 # deliver config to one collection only
 # run Solr - Solr running without any complains or errors
 # deliver config to second collection - doesn't have an effect
 But if I deliver configs for both collections before start Solr - it work 
 perfectly.
 So I would say that Solr should fail with meaningful error if there is no 
 config for some collection.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5893) FreeTextSuggester can now use Files.createTempDirectory()

2014-08-20 Thread Varun Thacker (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-5893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Varun Thacker updated LUCENE-5893:
--

Attachment: LUCENE-5893.patch

You're right :) New patch fixes it.

 FreeTextSuggester can now use Files.createTempDirectory()
 -

 Key: LUCENE-5893
 URL: https://issues.apache.org/jira/browse/LUCENE-5893
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Varun Thacker
Priority: Trivial
 Attachments: LUCENE-5893.patch, LUCENE-5893.patch


 Came across the TODO in the code and now it's possible to use 
 Files.createTempDirectory since 4x is also on Java 7. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: IndexReader.removeReaderClosedListener

2014-08-20 Thread Uwe Schindler

Hi,

In my opinion, removing the listener before close looks wrong. Because you 
would not get the listener events for the explicit close of the DirectoryReader.

I would just remove the ensureOpen(). We should open issue.

Uwe
-
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de


 -Original Message-
 From: Michael McCandless [mailto:luc...@mikemccandless.com]
 Sent: Wednesday, August 20, 2014 9:53 AM
 To: Lucene/Solr dev; Phil Herold
 Subject: Re: IndexReader.removeReaderClosedListener
 
 Hmm, you should call this method before closing the IndexReader, to
 remove a listener you previously added.
 
 Mike McCandless
 
 http://blog.mikemccandless.com
 
 
 On Tue, Aug 19, 2014 at 11:27 AM, Phil Herold herold.p...@gmail.com
 wrote:
  The implementation of this final method (latest 4.9 code) calls
  ensureOpen(), which fails, since the reader is closed. As a result,
  this method doesn't work. It seems as if this is therefore a potential
  memory leak,  not being able to remove the listener. Or am I missing
 something?
 
  --
  Phil
 
 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional
 commands, e-mail: dev-h...@lucene.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5699) Lucene classification score calculation normalize and return lists


[ 
https://issues.apache.org/jira/browse/LUCENE-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14103642#comment-14103642
 ] 

Tommaso Teofili commented on LUCENE-5699:
-

thanks Gergő, the latest patch looks good.

 Lucene classification score calculation normalize and return lists
 --

 Key: LUCENE-5699
 URL: https://issues.apache.org/jira/browse/LUCENE-5699
 Project: Lucene - Core
  Issue Type: Sub-task
  Components: modules/classification
Reporter: Gergő Törcsvári
Assignee: Tommaso Teofili
 Attachments: 06-06-5699.patch, 0730.patch, 0803-base.patch, 
 0810-base.patch


 Now the classifiers can return only the best matching classes. If somebody 
 want it to use more complex tasks he need to modify these classes for get 
 second and third results too. If it is possible to return a list and it is 
 not a lot resource why we dont do that? (We iterate a list so also.)
 The Bayes classifier get too small return values, and there were a bug with 
 the zero floats. It was fixed with logarithmic. It would be nice to scale the 
 class scores sum vlue to one, and then we coud compare two documents return 
 score and relevance. (If we dont do this the wordcount in the test documents 
 affected the result score.)
 With bulletpoints:
 * In the Bayes classification normalized score values, and return with result 
 lists.
 * In the KNN classifier possibility to return a result list.
 * Make the ClassificationResult Comparable for list sorting.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5699) Lucene classification score calculation normalize and return lists

2014-08-20 Thread ASF subversion and git services (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tommaso Teofili updated LUCENE-5699:


Labels: gsoc2014  (was: )

 Lucene classification score calculation normalize and return lists
 --

 Key: LUCENE-5699
 URL: https://issues.apache.org/jira/browse/LUCENE-5699
 Project: Lucene - Core
  Issue Type: Sub-task
  Components: modules/classification
Reporter: Gergő Törcsvári
Assignee: Tommaso Teofili
  Labels: gsoc2014
 Fix For: 5.0

 Attachments: 06-06-5699.patch, 0730.patch, 0803-base.patch, 
 0810-base.patch


 Now the classifiers can return only the best matching classes. If somebody 
 want it to use more complex tasks he need to modify these classes for get 
 second and third results too. If it is possible to return a list and it is 
 not a lot resource why we dont do that? (We iterate a list so also.)
 The Bayes classifier get too small return values, and there were a bug with 
 the zero floats. It was fixed with logarithmic. It would be nice to scale the 
 class scores sum vlue to one, and then we coud compare two documents return 
 score and relevance. (If we dont do this the wordcount in the test documents 
 affected the result score.)
 With bulletpoints:
 * In the Bayes classification normalized score values, and return with result 
 lists.
 * In the KNN classifier possibility to return a result list.
 * Make the ClassificationResult Comparable for list sorting.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5699) Lucene classification score calculation normalize and return lists


[ 
https://issues.apache.org/jira/browse/LUCENE-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14103644#comment-14103644
 ] 

ASF subversion and git services commented on LUCENE-5699:
-

Commit 1619053 from [~teofili] in branch 'dev/trunk'
[ https://svn.apache.org/r1619053 ]

LUCENE-5699 - patch from Gergő Törcsvári for normalized score and return lists 
in classification

 Lucene classification score calculation normalize and return lists
 --

 Key: LUCENE-5699
 URL: https://issues.apache.org/jira/browse/LUCENE-5699
 Project: Lucene - Core
  Issue Type: Sub-task
  Components: modules/classification
Reporter: Gergő Törcsvári
Assignee: Tommaso Teofili
  Labels: gsoc2014
 Fix For: 5.0

 Attachments: 06-06-5699.patch, 0730.patch, 0803-base.patch, 
 0810-base.patch


 Now the classifiers can return only the best matching classes. If somebody 
 want it to use more complex tasks he need to modify these classes for get 
 second and third results too. If it is possible to return a list and it is 
 not a lot resource why we dont do that? (We iterate a list so also.)
 The Bayes classifier get too small return values, and there were a bug with 
 the zero floats. It was fixed with logarithmic. It would be nice to scale the 
 class scores sum vlue to one, and then we coud compare two documents return 
 score and relevance. (If we dont do this the wordcount in the test documents 
 affected the result score.)
 With bulletpoints:
 * In the Bayes classification normalized score values, and return with result 
 lists.
 * In the KNN classifier possibility to return a result list.
 * Make the ClassificationResult Comparable for list sorting.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5699) Lucene classification score calculation normalize and return lists


 [ 
https://issues.apache.org/jira/browse/LUCENE-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tommaso Teofili updated LUCENE-5699:


Fix Version/s: 5.0

 Lucene classification score calculation normalize and return lists
 --

 Key: LUCENE-5699
 URL: https://issues.apache.org/jira/browse/LUCENE-5699
 Project: Lucene - Core
  Issue Type: Sub-task
  Components: modules/classification
Reporter: Gergő Törcsvári
Assignee: Tommaso Teofili
  Labels: gsoc2014
 Fix For: 5.0

 Attachments: 06-06-5699.patch, 0730.patch, 0803-base.patch, 
 0810-base.patch


 Now the classifiers can return only the best matching classes. If somebody 
 want it to use more complex tasks he need to modify these classes for get 
 second and third results too. If it is possible to return a list and it is 
 not a lot resource why we dont do that? (We iterate a list so also.)
 The Bayes classifier get too small return values, and there were a bug with 
 the zero floats. It was fixed with logarithmic. It would be nice to scale the 
 class scores sum vlue to one, and then we coud compare two documents return 
 score and relevance. (If we dont do this the wordcount in the test documents 
 affected the result score.)
 With bulletpoints:
 * In the Bayes classification normalized score values, and return with result 
 lists.
 * In the KNN classifier possibility to return a result list.
 * Make the ClassificationResult Comparable for list sorting.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-5699) Lucene classification score calculation normalize and return lists


 [ 
https://issues.apache.org/jira/browse/LUCENE-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tommaso Teofili resolved LUCENE-5699.
-

Resolution: Fixed

 Lucene classification score calculation normalize and return lists
 --

 Key: LUCENE-5699
 URL: https://issues.apache.org/jira/browse/LUCENE-5699
 Project: Lucene - Core
  Issue Type: Sub-task
  Components: modules/classification
Reporter: Gergő Törcsvári
Assignee: Tommaso Teofili
  Labels: gsoc2014
 Fix For: 5.0

 Attachments: 06-06-5699.patch, 0730.patch, 0803-base.patch, 
 0810-base.patch


 Now the classifiers can return only the best matching classes. If somebody 
 want it to use more complex tasks he need to modify these classes for get 
 second and third results too. If it is possible to return a list and it is 
 not a lot resource why we dont do that? (We iterate a list so also.)
 The Bayes classifier get too small return values, and there were a bug with 
 the zero floats. It was fixed with logarithmic. It would be nice to scale the 
 class scores sum vlue to one, and then we coud compare two documents return 
 score and relevance. (If we dont do this the wordcount in the test documents 
 affected the result score.)
 With bulletpoints:
 * In the Bayes classification normalized score values, and return with result 
 lists.
 * In the KNN classifier possibility to return a result list.
 * Make the ClassificationResult Comparable for list sorting.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5893) FreeTextSuggester can now use Files.createTempDirectory()

2014-08-20 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14103683#comment-14103683
 ] 

Michael McCandless commented on LUCENE-5893:


Thanks [~varunthacker] that looks great, I'll commit shortly!

 FreeTextSuggester can now use Files.createTempDirectory()
 -

 Key: LUCENE-5893
 URL: https://issues.apache.org/jira/browse/LUCENE-5893
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Varun Thacker
Priority: Trivial
 Attachments: LUCENE-5893.patch, LUCENE-5893.patch


 Came across the TODO in the code and now it's possible to use 
 Files.createTempDirectory since 4x is also on Java 7. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5893) FreeTextSuggester can now use Files.createTempDirectory()


[ 
https://issues.apache.org/jira/browse/LUCENE-5893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14103685#comment-14103685
 ] 

ASF subversion and git services commented on LUCENE-5893:
-

Commit 1619057 from [~mikemccand] in branch 'dev/trunk'
[ https://svn.apache.org/r1619057 ]

LUCENE-5893: use Files.createTempDirectory

 FreeTextSuggester can now use Files.createTempDirectory()
 -

 Key: LUCENE-5893
 URL: https://issues.apache.org/jira/browse/LUCENE-5893
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Varun Thacker
Priority: Trivial
 Attachments: LUCENE-5893.patch, LUCENE-5893.patch


 Came across the TODO in the code and now it's possible to use 
 Files.createTempDirectory since 4x is also on Java 7. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5893) FreeTextSuggester can now use Files.createTempDirectory()

2014-08-20 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14103689#comment-14103689
 ] 

ASF subversion and git services commented on LUCENE-5893:
-

Commit 1619058 from [~mikemccand] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1619058 ]

LUCENE-5893: use Files.createTempDirectory

 FreeTextSuggester can now use Files.createTempDirectory()
 -

 Key: LUCENE-5893
 URL: https://issues.apache.org/jira/browse/LUCENE-5893
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Varun Thacker
Priority: Trivial
 Fix For: 5.0, 4.10

 Attachments: LUCENE-5893.patch, LUCENE-5893.patch


 Came across the TODO in the code and now it's possible to use 
 Files.createTempDirectory since 4x is also on Java 7. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-5893) FreeTextSuggester can now use Files.createTempDirectory()


 [ 
https://issues.apache.org/jira/browse/LUCENE-5893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless resolved LUCENE-5893.


   Resolution: Fixed
Fix Version/s: 4.10
   5.0

 FreeTextSuggester can now use Files.createTempDirectory()
 -

 Key: LUCENE-5893
 URL: https://issues.apache.org/jira/browse/LUCENE-5893
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Varun Thacker
Priority: Trivial
 Fix For: 5.0, 4.10

 Attachments: LUCENE-5893.patch, LUCENE-5893.patch


 Came across the TODO in the code and now it's possible to use 
 Files.createTempDirectory since 4x is also on Java 7. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0_11) - Build # 11061 - Failure!

2014-08-20 Thread Policeman Jenkins Server

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/11061/
Java: 64bit/jdk1.8.0_11 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

4 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.rest.TestManagedResourceStorage

Error Message:
SolrCore.getOpenCount()==2

Stack Trace:
java.lang.RuntimeException: SolrCore.getOpenCount()==2
at __randomizedtesting.SeedInfo.seed([2789FF79311FC265]:0)
at org.apache.solr.util.TestHarness.close(TestHarness.java:332)
at org.apache.solr.SolrTestCaseJ4.deleteCore(SolrTestCaseJ4.java:633)
at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:184)
at sun.reflect.GeneratedMethodAccessor30.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:790)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)


FAILED:  
junit.framework.TestSuite.org.apache.solr.rest.TestManagedResourceStorage

Error Message:
Clean up static fields (in @AfterClass?), your test seems to hang on to 
approximately 12,491,320 bytes (threshold is 10,485,760). Field reference sizes 
(counted individually):   - 13,850,960 bytes, protected static 
org.apache.solr.core.SolrConfig org.apache.solr.SolrTestCaseJ4.solrConfig   - 
13,298,184 bytes, protected static 
org.apache.solr.util.TestHarness$LocalRequestFactory 
org.apache.solr.SolrTestCaseJ4.lrf   - 13,297,920 bytes, protected static 
org.apache.solr.util.TestHarness org.apache.solr.SolrTestCaseJ4.h   - 448 
bytes, private static java.util.regex.Pattern 
org.apache.solr.SolrTestCaseJ4.nonEscapedSingleQuotePattern   - 312 bytes, 
private static java.util.regex.Pattern 
org.apache.solr.SolrTestCaseJ4.escapedSingleQuotePattern   - 296 bytes, public 
static org.junit.rules.TestRule org.apache.solr.SolrTestCaseJ4.solrClassRules   
- 264 bytes, public static java.io.File 
org.apache.solr.cloud.AbstractZkTestCase.SOLRHOME   - 216 bytes, protected 
static java.lang.String org.apache.solr.SolrTestCaseJ4.testSolrHome   - 144 
bytes, private static java.lang.String 
org.apache.solr.SolrTestCaseJ4.factoryProp   - 88 bytes, protected static 
java.lang.String org.apache.solr.SolrTestCaseJ4.configString   - 80 bytes, 
private static java.lang.String org.apache.solr.SolrTestCaseJ4.coreName   - 80 
bytes, protected static java.lang.String 
org.apache.solr.SolrTestCaseJ4.schemaString

Stack Trace:
junit.framework.AssertionFailedError: Clean up static fields (in @AfterClass?), 
your test seems to hang on to approximately 12,491,320 bytes (threshold is 
10,485,760). Field reference sizes (counted individually):
  - 13,850,960 bytes, protected static org.apache.solr.core.SolrConfig 
org.apache.solr.SolrTestCaseJ4.solrConfig
  - 13,298,184 bytes, protected static 
org.apache.solr.util.TestHarness$LocalRequestFactory 
org.apache.solr.SolrTestCaseJ4.lrf
  - 13,297,920 bytes, protected static

Re: FW: [jira] [Commented] (LUCENE-5886) current ecj-javadoc-lint crashes on SharedFSAutoReplicaFailoverUtilsTest.java

2014-08-20 Thread Balchandra Vaidya



Hi Uwe,

As you might have already noticed, Java SE 8u20 has been
released and is available from
http://www.oracle.com/technetwork/java/javase/downloads/index.html

Thanks
Balchandra


On 08/18/14 11:48 AM, Balchandra Vaidya wrote:


Hi Uwe,

On 08/15/14 11:34 PM, Uwe Schindler wrote:

Hi Rory,

I opened https://issues.apache.org/jira/browse/LUCENE-5890 to track 
any problems. I will install JDK 9 and update the other JDKs during 
the next week.
Is there any release date for Java 8 update 20? If yes, I could 
combine the updates, because it always causes downtime of virtual 
machines.

8u20 is expected to be released soon.
http://openjdk.java.net/projects/jdk8u/releases/8u20.html

We will inform you as soon as the release go out.


Kind regards,
Balchandra




Uwe

-
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de



-Original Message-
From: Rory O'Donnell Oracle, Dublin Ireland
[mailto:rory.odonn...@oracle.com]
Sent: Friday, August 15, 2014 7:20 PM
To: Uwe Schindler; 'Balchandra Vaidya'; 'Dalibor Topic'
Cc: dev@lucene.apache.org
Subject: Re: FW: [jira] [Commented] (LUCENE-5886) current 
ecj-javadoc-lint

crashes on SharedFSAutoReplicaFailoverUtilsTest.java

Thanks for the update Uwe!
On 15/08/2014 17:49, Uwe Schindler wrote:

Hi Rory,

FYI, the JDK 9 b26 build seems to work now with Lucene. I have not yet
completed the tests (no Solr up to now, only Lucene), so we might 
add it as

build JDK to the Policeman Jenkins server soon!

As you see in the attached issue mail

(https://issues.apache.org/jira/browse/LUCENE-5886), I will add Java 9
support to our build files (and a new Constant accessible from Lucene
classes), so some conditionals in the ANT build works correct (we do 
some

checks only on specific JVMs).

Uwe

-
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de



--
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org





-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-trunk-Linux (64bit/ibm-j9-jdk7) - Build # 11062 - Still Failing!

2014-08-20 Thread Policeman Jenkins Server

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/11062/
Java: 64bit/ibm-j9-jdk7 
-Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}

2 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.lucene.search.TestSimpleSearchEquivalence

Error Message:


Stack Trace:
java.lang.NullPointerException
at __randomizedtesting.SeedInfo.seed([A798244826442DBA]:0)
at 
org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer$NormMap.add(Lucene49NormsConsumer.java:226)
at 
org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer.addNumericField(Lucene49NormsConsumer.java:95)
at 
org.apache.lucene.codecs.DocValuesConsumer.mergeNumericField(DocValuesConsumer.java:129)
at 
org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:253)
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:131)
at 
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3995)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3590)
at 
org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1883)
at 
org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4584)
at 
org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:697)
at 
org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4609)
at 
org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4601)
at 
org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1391)
at 
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1105)
at 
org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:143)
at 
org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:104)
at 
org.apache.lucene.search.SearchEquivalenceTestBase.beforeClass(SearchEquivalenceTestBase.java:74)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:94)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:619)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:767)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:853)


FAILED:  
junit.framework.TestSuite.org.apache.lucene.search.TestSimpleSearchEquivalence

Error Message:


Stack Trace:
java.lang.NullPointerException
at __randomizedtesting.SeedInfo.seed([A798244826442DBA]:0)
at 
org.apache.lucene.search.SearchEquivalenceTestBase.afterClass(SearchEquivalenceTestBase.java:96)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:94)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:619)
at

[jira] [Updated] (LUCENE-5889) AnalyzingInfixSuggester should expose commit()

2014-08-20 Thread Varun Thacker (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-5889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Varun Thacker updated LUCENE-5889:
--

Attachment: LUCENE-5889.patch

Patch which exposes commit.

I also added param to the constructor 'commitOnBuild' . I felt it was a good 
option to have and especially useful in Solr where we just expose the build() 
method for all suggesters. 

[~mikemccand] - Also regarding your comments on the user list regarding the 
NPE's in add(), update()  I think it would be more convenient for a user to 
keep calling add() in a loop and then call commit() to build his suggester. It 
saves the user the hassle of creating a Iterator. If you think otherwise I 
could change it to IllegalStateException.


 AnalyzingInfixSuggester should expose commit()
 --

 Key: LUCENE-5889
 URL: https://issues.apache.org/jira/browse/LUCENE-5889
 Project: Lucene - Core
  Issue Type: Improvement
  Components: modules/spellchecker
Reporter: Mike Sokolov
 Attachments: LUCENE-5889.patch


 There is no way short of close() for a user of AnalyzingInfixSuggester to 
 cause it to commit() its underlying index: only refresh() is provided.  But 
 callers might want to ensure the index is flushed to disk without closing.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: IndexReader.removeReaderClosedListener

2014-08-20 Thread Phil Herold

Bingo. ensureOpen() should not be called from this method. Thanks.

--
Phil


On Wed, Aug 20, 2014 at 4:55 AM, Uwe Schindler u...@thetaphi.de wrote:

 Hi,

 In my opinion, removing the listener before close looks wrong. Because you
 would not get the listener events for the explicit close of the
 DirectoryReader.

 I would just remove the ensureOpen(). We should open issue.

 Uwe
 -
 Uwe Schindler
 H.-H.-Meier-Allee 63, D-28213 Bremen
 http://www.thetaphi.de
 eMail: u...@thetaphi.de


  -Original Message-
  From: Michael McCandless [mailto:luc...@mikemccandless.com]
  Sent: Wednesday, August 20, 2014 9:53 AM
  To: Lucene/Solr dev; Phil Herold
  Subject: Re: IndexReader.removeReaderClosedListener
 
  Hmm, you should call this method before closing the IndexReader, to
  remove a listener you previously added.
 
  Mike McCandless
 
  http://blog.mikemccandless.com
 
 
  On Tue, Aug 19, 2014 at 11:27 AM, Phil Herold herold.p...@gmail.com
  wrote:
   The implementation of this final method (latest 4.9 code) calls
   ensureOpen(), which fails, since the reader is closed. As a result,
   this method doesn't work. It seems as if this is therefore a potential
   memory leak,  not being able to remove the listener. Or am I missing
  something?
  
   --
   Phil
 
  -
  To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional
  commands, e-mail: dev-h...@lucene.apache.org


 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org

Re: IndexReader.removeReaderClosedListener

2014-08-20 Thread Robert Muir

I don't agree. Operations like this shouldnt be performed on closed readers.

the ensureOpen is correct.

On Wed, Aug 20, 2014 at 8:27 AM, Phil Herold pher...@nc.rr.com wrote:
 Bingo. ensureOpen() should not be called from this method. Thanks.

 --
 Phil



 On Wed, Aug 20, 2014 at 4:55 AM, Uwe Schindler u...@thetaphi.de wrote:

 Hi,

 In my opinion, removing the listener before close looks wrong. Because you
 would not get the listener events for the explicit close of the
 DirectoryReader.

 I would just remove the ensureOpen(). We should open issue.

 Uwe
 -
 Uwe Schindler
 H.-H.-Meier-Allee 63, D-28213 Bremen
 http://www.thetaphi.de
 eMail: u...@thetaphi.de


  -Original Message-
  From: Michael McCandless [mailto:luc...@mikemccandless.com]
  Sent: Wednesday, August 20, 2014 9:53 AM
  To: Lucene/Solr dev; Phil Herold
  Subject: Re: IndexReader.removeReaderClosedListener
 
  Hmm, you should call this method before closing the IndexReader, to
  remove a listener you previously added.
 
  Mike McCandless
 
  http://blog.mikemccandless.com
 
 
  On Tue, Aug 19, 2014 at 11:27 AM, Phil Herold herold.p...@gmail.com
  wrote:
   The implementation of this final method (latest 4.9 code) calls
   ensureOpen(), which fails, since the reader is closed. As a result,
   this method doesn't work. It seems as if this is therefore a potential
   memory leak,  not being able to remove the listener. Or am I missing
  something?
  
   --
   Phil
 
  -
  To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional
  commands, e-mail: dev-h...@lucene.apache.org


 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-6394) Managed Synonyms should support deleting all synonyms or replacing a single one

2014-08-20 Thread Mathias H. (JIRA)

Mathias H. created SOLR-6394:


 Summary: Managed Synonyms should support deleting all synonyms or 
replacing a single one
 Key: SOLR-6394
 URL: https://issues.apache.org/jira/browse/SOLR-6394
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.9
Reporter: Mathias H.
Priority: Minor


Currently it is only possible to add synonyms and deleting single synonyms. If 
you need to delete all synonyms you have to get the list and sending an delete 
request to every single synonym. Also you can't overwrite a synonym only append 
it.

It would be more convenient to have additional possibilities:

Deleting all synonyms
curl -XDELETE 
http://localhost:8983/solr/collection1/schema/analysis/synonyms/english/

Overwriting a single synonym 
curl -XPUT 
http://localhost:8983/solr/collection1/schema/analysis/synonyms/english/apple

Add a synonym / append to an existing synonym
curl -XPOST 
http://localhost:8983/solr/collection1/schema/analysis/synonyms/english/apple





--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-6394) Managed Synonyms should support deleting all synonyms or replacing a single one

2014-08-20 Thread Mathias H. (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-6394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mathias H. updated SOLR-6394:
-

Description: 
Currently it is only possible to add synonyms and deleting single synonyms. If 
you need to delete all synonyms you have to get the list and then sending an 
delete request to every single synonym. Also you can't overwrite a synonym only 
append it.

It would be more convenient to have additional possibilities:

Deleting all synonyms
curl -XDELETE 
http://localhost:8983/solr/collection1/schema/analysis/synonyms/english/

Overwriting a single synonym 
curl -XPUT 
http://localhost:8983/solr/collection1/schema/analysis/synonyms/english/apple

Add a synonym / append to an existing synonym
curl -XPOST 
http://localhost:8983/solr/collection1/schema/analysis/synonyms/english/apple



  was:
Currently it is only possible to add synonyms and deleting single synonyms. If 
you need to delete all synonyms you have to get the list and sending an delete 
request to every single synonym. Also you can't overwrite a synonym only append 
it.

It would be more convenient to have additional possibilities:

Deleting all synonyms
curl -XDELETE 
http://localhost:8983/solr/collection1/schema/analysis/synonyms/english/

Overwriting a single synonym 
curl -XPUT 
http://localhost:8983/solr/collection1/schema/analysis/synonyms/english/apple

Add a synonym / append to an existing synonym
curl -XPOST 
http://localhost:8983/solr/collection1/schema/analysis/synonyms/english/apple




 Managed Synonyms should support deleting all synonyms or replacing a single 
 one
 ---

 Key: SOLR-6394
 URL: https://issues.apache.org/jira/browse/SOLR-6394
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.9
Reporter: Mathias H.
Priority: Minor
  Labels: managed, rest, synonyms

 Currently it is only possible to add synonyms and deleting single synonyms. 
 If you need to delete all synonyms you have to get the list and then sending 
 an delete request to every single synonym. Also you can't overwrite a synonym 
 only append it.
 It would be more convenient to have additional possibilities:
 Deleting all synonyms
 curl -XDELETE 
 http://localhost:8983/solr/collection1/schema/analysis/synonyms/english/
 Overwriting a single synonym 
 curl -XPUT 
 http://localhost:8983/solr/collection1/schema/analysis/synonyms/english/apple
 Add a synonym / append to an existing synonym
 curl -XPOST 
 http://localhost:8983/solr/collection1/schema/analysis/synonyms/english/apple



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-6394) Managed Synonyms should support deleting all synonyms or replacing a single one

2014-08-20 Thread Mathias H. (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-6394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mathias H. updated SOLR-6394:
-

Description: 
Currently it is only possible to add synonyms and deleting single synonyms. If 
you need to delete all synonyms you have to get the list and then sending an 
delete request to every single synonym. Also you can't overwrite a synonym but 
only append it.

It would be more convenient to have additional possibilities:

Deleting all synonyms
curl -XDELETE 
http://localhost:8983/solr/collection1/schema/analysis/synonyms/english/

Overwriting a single synonym 
curl -XPUT 
http://localhost:8983/solr/collection1/schema/analysis/synonyms/english/apple

Add a synonym / append to an existing synonym
curl -XPOST 
http://localhost:8983/solr/collection1/schema/analysis/synonyms/english/apple



  was:
Currently it is only possible to add synonyms and deleting single synonyms. If 
you need to delete all synonyms you have to get the list and then sending an 
delete request to every single synonym. Also you can't overwrite a synonym only 
append it.

It would be more convenient to have additional possibilities:

Deleting all synonyms
curl -XDELETE 
http://localhost:8983/solr/collection1/schema/analysis/synonyms/english/

Overwriting a single synonym 
curl -XPUT 
http://localhost:8983/solr/collection1/schema/analysis/synonyms/english/apple

Add a synonym / append to an existing synonym
curl -XPOST 
http://localhost:8983/solr/collection1/schema/analysis/synonyms/english/apple




 Managed Synonyms should support deleting all synonyms or replacing a single 
 one
 ---

 Key: SOLR-6394
 URL: https://issues.apache.org/jira/browse/SOLR-6394
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.9
Reporter: Mathias H.
Priority: Minor
  Labels: managed, rest, synonyms

 Currently it is only possible to add synonyms and deleting single synonyms. 
 If you need to delete all synonyms you have to get the list and then sending 
 an delete request to every single synonym. Also you can't overwrite a synonym 
 but only append it.
 It would be more convenient to have additional possibilities:
 Deleting all synonyms
 curl -XDELETE 
 http://localhost:8983/solr/collection1/schema/analysis/synonyms/english/
 Overwriting a single synonym 
 curl -XPUT 
 http://localhost:8983/solr/collection1/schema/analysis/synonyms/english/apple
 Add a synonym / append to an existing synonym
 curl -XPOST 
 http://localhost:8983/solr/collection1/schema/analysis/synonyms/english/apple



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2014-08-20 Thread Bragadeesh (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14103900#comment-14103900
 ] 

Bragadeesh commented on SOLR-4586:
--

I bumped through this issue recently. Is this something planned for a release 
sooner ? 

 Increase default maxBooleanClauses
 --

 Key: SOLR-4586
 URL: https://issues.apache.org/jira/browse/SOLR-4586
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.2
 Environment: 4.3-SNAPSHOT 1456767M - ncindex - 2013-03-15 13:11:50
Reporter: Shawn Heisey
 Attachments: SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch, 
 SOLR-4586.patch, SOLR-4586.patch, SOLR-4586_verify_maxClauses.patch


 In the #solr IRC channel, I mentioned the maxBooleanClauses limitation to 
 someone asking a question about queries.  Mark Miller told me that 
 maxBooleanClauses no longer applies, that the limitation was removed from 
 Lucene sometime in the 3.x series.  The config still shows up in the example 
 even in the just-released 4.2.
 Checking through the source code, I found that the config option is parsed 
 and the value stored in objects, but does not actually seem to be used by 
 anything.  I removed every trace of it that I could find, and all tests still 
 pass.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-5895) Add per-segment and per-commit id to help replication

Michael McCandless created LUCENE-5895:
--

 Summary: Add per-segment and per-commit id to help replication
 Key: LUCENE-5895
 URL: https://issues.apache.org/jira/browse/LUCENE-5895
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Michael McCandless
Assignee: Michael McCandless
 Fix For: 5.0, 4.10
 Attachments: LUCENE-5895.patch

It would be useful if Lucene recorded a unique id for each segment written and 
each commit point.  This way, file-based replicators can use this to know 
whether the segment/commit they are looking at on a source machine and dest 
machine are in fact that same.

I know this would have been very useful when I was playing with NRT replication 
(LUCENE-5438).



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5895) Add per-segment and per-commit id to help replication


 [ 
https://issues.apache.org/jira/browse/LUCENE-5895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless updated LUCENE-5895:
---

Attachment: LUCENE-5895.patch

Initial patch ...

 Add per-segment and per-commit id to help replication
 -

 Key: LUCENE-5895
 URL: https://issues.apache.org/jira/browse/LUCENE-5895
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Michael McCandless
Assignee: Michael McCandless
 Fix For: 5.0, 4.10

 Attachments: LUCENE-5895.patch


 It would be useful if Lucene recorded a unique id for each segment written 
 and each commit point.  This way, file-based replicators can use this to 
 know whether the segment/commit they are looking at on a source machine and 
 dest machine are in fact that same.
 I know this would have been very useful when I was playing with NRT 
 replication (LUCENE-5438).



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5895) Add per-segment and per-commit id to help replication


[ 
https://issues.apache.org/jira/browse/LUCENE-5895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14103939#comment-14103939
 ] 

Uwe Schindler commented on LUCENE-5895:
---

Why not use Java's UUID class? 
http://docs.oracle.com/javase/7/docs/api/java/util/UUID.html - especially: 
http://docs.oracle.com/javase/7/docs/api/java/util/UUID.html#randomUUID()

 Add per-segment and per-commit id to help replication
 -

 Key: LUCENE-5895
 URL: https://issues.apache.org/jira/browse/LUCENE-5895
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Michael McCandless
Assignee: Michael McCandless
 Fix For: 5.0, 4.10

 Attachments: LUCENE-5895.patch


 It would be useful if Lucene recorded a unique id for each segment written 
 and each commit point.  This way, file-based replicators can use this to 
 know whether the segment/commit they are looking at on a source machine and 
 dest machine are in fact that same.
 I know this would have been very useful when I was playing with NRT 
 replication (LUCENE-5438).



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-5895) Add per-segment and per-commit id to help replication


[ 
https://issues.apache.org/jira/browse/LUCENE-5895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14103939#comment-14103939
 ] 

Uwe Schindler edited comment on LUCENE-5895 at 8/20/14 2:35 PM:


Why not use Java's UUID class? 
[http://docs.oracle.com/javase/7/docs/api/java/util/UUID.html] - especially: 
[http://docs.oracle.com/javase/7/docs/api/java/util/UUID.html#randomUUID()]


was (Author: thetaphi):
Why not use Java's UUID class? 
http://docs.oracle.com/javase/7/docs/api/java/util/UUID.html - especially: 
http://docs.oracle.com/javase/7/docs/api/java/util/UUID.html#randomUUID()

 Add per-segment and per-commit id to help replication
 -

 Key: LUCENE-5895
 URL: https://issues.apache.org/jira/browse/LUCENE-5895
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Michael McCandless
Assignee: Michael McCandless
 Fix For: 5.0, 4.10

 Attachments: LUCENE-5895.patch


 It would be useful if Lucene recorded a unique id for each segment written 
 and each commit point.  This way, file-based replicators can use this to 
 know whether the segment/commit they are looking at on a source machine and 
 dest machine are in fact that same.
 I know this would have been very useful when I was playing with NRT 
 replication (LUCENE-5438).



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: IndexReader.removeReaderClosedListener

2014-08-20 Thread Phil Herold

Fine. I'm sure you've got a good reason for this. But it is totally
un-intuitive, and introduces a potential memory leak, IMO. The
implementation of this pattern is definitely different than just about any
other library I can think of.

Given that having a listener for index closed doesn't really solve the
issue I was having that I was hoping it would, it doesn't much matter to me
at this point.

Thanks.

--
Phil


On Wed, Aug 20, 2014 at 8:37 AM, Robert Muir rcm...@gmail.com wrote:

 I don't agree. Operations like this shouldnt be performed on closed
 readers.

 the ensureOpen is correct.

 On Wed, Aug 20, 2014 at 8:27 AM, Phil Herold pher...@nc.rr.com wrote:
  Bingo. ensureOpen() should not be called from this method. Thanks.
 
  --
  Phil
 
 
 
  On Wed, Aug 20, 2014 at 4:55 AM, Uwe Schindler u...@thetaphi.de wrote:
 
  Hi,
 
  In my opinion, removing the listener before close looks wrong. Because
 you
  would not get the listener events for the explicit close of the
  DirectoryReader.
 
  I would just remove the ensureOpen(). We should open issue.
 
  Uwe
  -
  Uwe Schindler
  H.-H.-Meier-Allee 63, D-28213 Bremen
  http://www.thetaphi.de
  eMail: u...@thetaphi.de
 
 
   -Original Message-
   From: Michael McCandless [mailto:luc...@mikemccandless.com]
   Sent: Wednesday, August 20, 2014 9:53 AM
   To: Lucene/Solr dev; Phil Herold
   Subject: Re: IndexReader.removeReaderClosedListener
  
   Hmm, you should call this method before closing the IndexReader, to
   remove a listener you previously added.
  
   Mike McCandless
  
   http://blog.mikemccandless.com
  
  
   On Tue, Aug 19, 2014 at 11:27 AM, Phil Herold herold.p...@gmail.com
   wrote:
The implementation of this final method (latest 4.9 code) calls
ensureOpen(), which fails, since the reader is closed. As a result,
this method doesn't work. It seems as if this is therefore a
 potential
memory leak,  not being able to remove the listener. Or am I missing
   something?
   
--
Phil
  
   -
   To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For
 additional
   commands, e-mail: dev-h...@lucene.apache.org
 
 
  -
  To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
  For additional commands, e-mail: dev-h...@lucene.apache.org
 
 

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org

Re: IndexReader.removeReaderClosedListener

2014-08-20 Thread Shai Erera

I don't understand something -- you add a ReaderClosedListener to get a
notification when IR.close() is called (actually, when it's actually being
closed). Why removing the listener after the reader has been closed? Don't
apps usually nullify their IR member after it's been closed?

Shai


On Wed, Aug 20, 2014 at 5:51 PM, Phil Herold pher...@nc.rr.com wrote:

 Fine. I'm sure you've got a good reason for this. But it is totally
 un-intuitive, and introduces a potential memory leak, IMO. The
 implementation of this pattern is definitely different than just about any
 other library I can think of.

 Given that having a listener for index closed doesn't really solve the
 issue I was having that I was hoping it would, it doesn't much matter to me
 at this point.

 Thanks.

 --
 Phil


 On Wed, Aug 20, 2014 at 8:37 AM, Robert Muir rcm...@gmail.com wrote:

 I don't agree. Operations like this shouldnt be performed on closed
 readers.

 the ensureOpen is correct.

 On Wed, Aug 20, 2014 at 8:27 AM, Phil Herold pher...@nc.rr.com wrote:
  Bingo. ensureOpen() should not be called from this method. Thanks.
 
  --
  Phil
 
 
 
  On Wed, Aug 20, 2014 at 4:55 AM, Uwe Schindler u...@thetaphi.de wrote:
 
  Hi,
 
  In my opinion, removing the listener before close looks wrong. Because
 you
  would not get the listener events for the explicit close of the
  DirectoryReader.
 
  I would just remove the ensureOpen(). We should open issue.
 
  Uwe
  -
  Uwe Schindler
  H.-H.-Meier-Allee 63, D-28213 Bremen
  http://www.thetaphi.de
  eMail: u...@thetaphi.de
 
 
   -Original Message-
   From: Michael McCandless [mailto:luc...@mikemccandless.com]
   Sent: Wednesday, August 20, 2014 9:53 AM
   To: Lucene/Solr dev; Phil Herold
   Subject: Re: IndexReader.removeReaderClosedListener
  
   Hmm, you should call this method before closing the IndexReader, to
   remove a listener you previously added.
  
   Mike McCandless
  
   http://blog.mikemccandless.com
  
  
   On Tue, Aug 19, 2014 at 11:27 AM, Phil Herold herold.p...@gmail.com
 
   wrote:
The implementation of this final method (latest 4.9 code) calls
ensureOpen(), which fails, since the reader is closed. As a result,
this method doesn't work. It seems as if this is therefore a
 potential
memory leak,  not being able to remove the listener. Or am I
 missing
   something?
   
--
Phil
  
   -
   To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For
 additional
   commands, e-mail: dev-h...@lucene.apache.org
 
 
  -
  To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
  For additional commands, e-mail: dev-h...@lucene.apache.org
 
 

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4527) Atomic updates when running distributed seem broken.

2014-08-20 Thread JIRA


[ 
https://issues.apache.org/jira/browse/SOLR-4527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14103978#comment-14103978
 ] 

José Joaquín commented on SOLR-4527:


The NullPointerException returned by a real time get when a collection with 
more than a shard is requested was fixed in 4.7.1 (maybe before). But it's 
happening again in version 4.9.

 Atomic updates when running distributed seem broken.
 

 Key: SOLR-4527
 URL: https://issues.apache.org/jira/browse/SOLR-4527
 Project: Solr
  Issue Type: Bug
  Components: SolrCloud, update
Affects Versions: 4.1
Reporter: mike st. john
 Fix For: 4.9, 5.0


 When using solrcloud as a nosql solution,  i've run into the issue where i've 
 sent some atomic updates and i'm receiving an error  missing required 
 field:  implying that this is an add instead of an update.  when i add 
 distrib=false to the url and send the doc to the index where it resides, the 
 update is applied.
 Possibly related...when i try and do a real time get for the id,  its 
 throwing an NPE
  trace:java.lang.NullPointerException\n\tat 
 org.apache.solr.handler.component.RealTimeGetComponent.createSubRequests(RealTimeGetComponent.java:368)\n\tat
  
 org.apache.solr.handler.component.RealTimeGetComponent.distributedProcess(RealTimeGetComponent.java:325)\n\tat
  
 org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:244)\n\tat
  
 org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)\n\tat
  org.apache.solr.core.SolrCore.execute(SolrCore.java:1808)\n\tat 
 org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:583)\n\tat
  
 org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:293)\n\tat
  
 org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)\n\tat
  
 org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)\n\tat
  
 org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)\n\tat
  
 org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)\n\tat
  
 org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)\n\tat
  
 org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)\n\tat
  
 org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)\n\tat
  
 org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)\n\tat
  
 org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)\n\tat
  
 org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)\n\tat
  
 org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)\n\tat
  
 org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)\n\tat
  
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)\n\tat
  
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)\n\tat
  java.lang.Thread.run(Thread.java:679)\n,
 code:500}}
 the command will succeed , if i use the url the doc exists on and add 
 distrib=false to the end.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: IndexReader.removeReaderClosedListener

2014-08-20 Thread Uwe Schindler

Yeah,

 

I dont see the memory leak, too. After you closed the IndexReader its free for 
garbage collection. So it should no longer block the listener to be GCed.

 

Uwe

 

-

Uwe Schindler

H.-H.-Meier-Allee 63, D-28213 Bremen

 http://www.thetaphi.de/ http://www.thetaphi.de

eMail: u...@thetaphi.de

 

From: Shai Erera [mailto:ser...@gmail.com] 
Sent: Wednesday, August 20, 2014 5:10 PM
To: dev@lucene.apache.org
Subject: Re: IndexReader.removeReaderClosedListener

 

I don't understand something -- you add a ReaderClosedListener to get a 
notification when IR.close() is called (actually, when it's actually being 
closed). Why removing the listener after the reader has been closed? Don't apps 
usually nullify their IR member after it's been closed?

Shai

 

On Wed, Aug 20, 2014 at 5:51 PM, Phil Herold pher...@nc.rr.com wrote:

Fine. I'm sure you've got a good reason for this. But it is totally 
un-intuitive, and introduces a potential memory leak, IMO. The implementation 
of this pattern is definitely different than just about any other library I can 
think of. 

 

Given that having a listener for index closed doesn't really solve the issue I 
was having that I was hoping it would, it doesn't much matter to me at this 
point.

 

Thanks.

 

--

Phil 

 

On Wed, Aug 20, 2014 at 8:37 AM, Robert Muir rcm...@gmail.com wrote:

I don't agree. Operations like this shouldnt be performed on closed readers.

the ensureOpen is correct.


On Wed, Aug 20, 2014 at 8:27 AM, Phil Herold pher...@nc.rr.com wrote:
 Bingo. ensureOpen() should not be called from this method. Thanks.

 --
 Phil



 On Wed, Aug 20, 2014 at 4:55 AM, Uwe Schindler u...@thetaphi.de wrote:

 Hi,

 In my opinion, removing the listener before close looks wrong. Because you
 would not get the listener events for the explicit close of the
 DirectoryReader.

 I would just remove the ensureOpen(). We should open issue.

 Uwe
 -
 Uwe Schindler
 H.-H.-Meier-Allee 63, D-28213 Bremen
 http://www.thetaphi.de
 eMail: u...@thetaphi.de


  -Original Message-
  From: Michael McCandless [mailto:luc...@mikemccandless.com]
  Sent: Wednesday, August 20, 2014 9:53 AM
  To: Lucene/Solr dev; Phil Herold
  Subject: Re: IndexReader.removeReaderClosedListener
 
  Hmm, you should call this method before closing the IndexReader, to
  remove a listener you previously added.
 
  Mike McCandless
 
  http://blog.mikemccandless.com
 
 
  On Tue, Aug 19, 2014 at 11:27 AM, Phil Herold herold.p...@gmail.com
  wrote:
   The implementation of this final method (latest 4.9 code) calls
   ensureOpen(), which fails, since the reader is closed. As a result,
   this method doesn't work. It seems as if this is therefore a potential
   memory leak,  not being able to remove the listener. Or am I missing
  something?
  
   --
   Phil
 
  -
  To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional
  commands, e-mail: dev-h...@lucene.apache.org


 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: IndexReader.removeReaderClosedListener

2014-08-20 Thread Phil Herold

If the IndexReader is still holding a reference to the listener in its
collection, I don't see how it could be GC'ed. This is a pretty classic
memory leak case.

--
Phil


On Wed, Aug 20, 2014 at 11:15 AM, Uwe Schindler u...@thetaphi.de wrote:

 Yeah,



 I dont see the memory leak, too. After you closed the IndexReader its free
 for garbage collection. So it should no longer block the listener to be
 GCed.



 Uwe



 -

 Uwe Schindler

 H.-H.-Meier-Allee 63, D-28213 Bremen

 http://www.thetaphi.de

 eMail: u...@thetaphi.de



 *From:* Shai Erera [mailto:ser...@gmail.com]
 *Sent:* Wednesday, August 20, 2014 5:10 PM
 *To:* dev@lucene.apache.org
 *Subject:* Re: IndexReader.removeReaderClosedListener



 I don't understand something -- you add a ReaderClosedListener to get a
 notification when IR.close() is called (actually, when it's actually being
 closed). Why removing the listener after the reader has been closed? Don't
 apps usually nullify their IR member after it's been closed?

 Shai



 On Wed, Aug 20, 2014 at 5:51 PM, Phil Herold pher...@nc.rr.com wrote:

 Fine. I'm sure you've got a good reason for this. But it is totally
 un-intuitive, and introduces a potential memory leak, IMO. The
 implementation of this pattern is definitely different than just about any
 other library I can think of.



 Given that having a listener for index closed doesn't really solve the
 issue I was having that I was hoping it would, it doesn't much matter to me
 at this point.



 Thanks.



 --

 Phil



 On Wed, Aug 20, 2014 at 8:37 AM, Robert Muir rcm...@gmail.com wrote:

 I don't agree. Operations like this shouldnt be performed on closed
 readers.

 the ensureOpen is correct.


 On Wed, Aug 20, 2014 at 8:27 AM, Phil Herold pher...@nc.rr.com wrote:
  Bingo. ensureOpen() should not be called from this method. Thanks.
 
  --
  Phil
 
 
 
  On Wed, Aug 20, 2014 at 4:55 AM, Uwe Schindler u...@thetaphi.de wrote:
 
  Hi,
 
  In my opinion, removing the listener before close looks wrong. Because
 you
  would not get the listener events for the explicit close of the
  DirectoryReader.
 
  I would just remove the ensureOpen(). We should open issue.
 
  Uwe
  -
  Uwe Schindler
  H.-H.-Meier-Allee 63, D-28213 Bremen
  http://www.thetaphi.de
  eMail: u...@thetaphi.de
 
 
   -Original Message-
   From: Michael McCandless [mailto:luc...@mikemccandless.com]
   Sent: Wednesday, August 20, 2014 9:53 AM
   To: Lucene/Solr dev; Phil Herold
   Subject: Re: IndexReader.removeReaderClosedListener
  
   Hmm, you should call this method before closing the IndexReader, to
   remove a listener you previously added.
  
   Mike McCandless
  
   http://blog.mikemccandless.com
  
  
   On Tue, Aug 19, 2014 at 11:27 AM, Phil Herold herold.p...@gmail.com
   wrote:
The implementation of this final method (latest 4.9 code) calls
ensureOpen(), which fails, since the reader is closed. As a result,
this method doesn't work. It seems as if this is therefore a
 potential
memory leak,  not being able to remove the listener. Or am I missing
   something?
   
--
Phil
  
   -
   To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For
 additional
   commands, e-mail: dev-h...@lucene.apache.org
 
 
  -
  To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
  For additional commands, e-mail: dev-h...@lucene.apache.org
 
 

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org

Re: IndexReader.removeReaderClosedListener

2014-08-20 Thread Shai Erera

But that also implies that your search app holds a reference to the closed
IndexReader ...

Shai


On Wed, Aug 20, 2014 at 6:17 PM, Phil Herold pher...@nc.rr.com wrote:

 If the IndexReader is still holding a reference to the listener in its
 collection, I don't see how it could be GC'ed. This is a pretty classic
 memory leak case.

 --
 Phil


 On Wed, Aug 20, 2014 at 11:15 AM, Uwe Schindler u...@thetaphi.de wrote:

 Yeah,



 I dont see the memory leak, too. After you closed the IndexReader its
 free for garbage collection. So it should no longer block the listener to
 be GCed.



 Uwe



 -

 Uwe Schindler

 H.-H.-Meier-Allee 63, D-28213 Bremen

 http://www.thetaphi.de

 eMail: u...@thetaphi.de



 *From:* Shai Erera [mailto:ser...@gmail.com]
 *Sent:* Wednesday, August 20, 2014 5:10 PM
 *To:* dev@lucene.apache.org
 *Subject:* Re: IndexReader.removeReaderClosedListener



 I don't understand something -- you add a ReaderClosedListener to get a
 notification when IR.close() is called (actually, when it's actually being
 closed). Why removing the listener after the reader has been closed? Don't
 apps usually nullify their IR member after it's been closed?

 Shai



 On Wed, Aug 20, 2014 at 5:51 PM, Phil Herold pher...@nc.rr.com wrote:

 Fine. I'm sure you've got a good reason for this. But it is totally
 un-intuitive, and introduces a potential memory leak, IMO. The
 implementation of this pattern is definitely different than just about any
 other library I can think of.



 Given that having a listener for index closed doesn't really solve the
 issue I was having that I was hoping it would, it doesn't much matter to me
 at this point.



 Thanks.



 --

 Phil



 On Wed, Aug 20, 2014 at 8:37 AM, Robert Muir rcm...@gmail.com wrote:

 I don't agree. Operations like this shouldnt be performed on closed
 readers.

 the ensureOpen is correct.


 On Wed, Aug 20, 2014 at 8:27 AM, Phil Herold pher...@nc.rr.com wrote:
  Bingo. ensureOpen() should not be called from this method. Thanks.
 
  --
  Phil
 
 
 
  On Wed, Aug 20, 2014 at 4:55 AM, Uwe Schindler u...@thetaphi.de wrote:
 
  Hi,
 
  In my opinion, removing the listener before close looks wrong. Because
 you
  would not get the listener events for the explicit close of the
  DirectoryReader.
 
  I would just remove the ensureOpen(). We should open issue.
 
  Uwe
  -
  Uwe Schindler
  H.-H.-Meier-Allee 63, D-28213 Bremen
  http://www.thetaphi.de
  eMail: u...@thetaphi.de
 
 
   -Original Message-
   From: Michael McCandless [mailto:luc...@mikemccandless.com]
   Sent: Wednesday, August 20, 2014 9:53 AM
   To: Lucene/Solr dev; Phil Herold
   Subject: Re: IndexReader.removeReaderClosedListener
  
   Hmm, you should call this method before closing the IndexReader, to
   remove a listener you previously added.
  
   Mike McCandless
  
   http://blog.mikemccandless.com
  
  
   On Tue, Aug 19, 2014 at 11:27 AM, Phil Herold herold.p...@gmail.com
 
   wrote:
The implementation of this final method (latest 4.9 code) calls
ensureOpen(), which fails, since the reader is closed. As a result,
this method doesn't work. It seems as if this is therefore a
 potential
memory leak,  not being able to remove the listener. Or am I
 missing
   something?
   
--
Phil
  
   -
   To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For
 additional
   commands, e-mail: dev-h...@lucene.apache.org
 
 
  -
  To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
  For additional commands, e-mail: dev-h...@lucene.apache.org
 
 

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5896) A view potential reproducible issues


 [ 
https://issues.apache.org/jira/browse/LUCENE-5896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Simon Willnauer updated LUCENE-5896:


Attachment: LUCENE-5896.patch

here are the ones I found looking over it for 2 min I bet there are more

 A view potential reproducible issues
 

 Key: LUCENE-5896
 URL: https://issues.apache.org/jira/browse/LUCENE-5896
 Project: Lucene - Core
  Issue Type: Test
  Components: general/test
Affects Versions: 4.9
Reporter: Simon Willnauer
 Fix For: 5.0, 4.10

 Attachments: LUCENE-5896.patch


 I realized that passing the same seeded random instance to LuceneTestCase# 
 newIndewWriterConfig doesn't necessarily produce the same IWC and I found a 
 bunch of issues in that class using global random rather than local random. 
 Yet, I went over the file to spot others but we might need to think about a 
 more automated way to spot those...



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-5896) A view potential reproducible issues

Simon Willnauer created LUCENE-5896:
---

 Summary: A view potential reproducible issues
 Key: LUCENE-5896
 URL: https://issues.apache.org/jira/browse/LUCENE-5896
 Project: Lucene - Core
  Issue Type: Test
  Components: general/test
Affects Versions: 4.9
Reporter: Simon Willnauer
 Fix For: 5.0, 4.10
 Attachments: LUCENE-5896.patch

I realized that passing the same seeded random instance to LuceneTestCase# 
newIndewWriterConfig doesn't necessarily produce the same IWC and I found a 
bunch of issues in that class using global random rather than local random. 
Yet, I went over the file to spot others but we might need to think about a 
more automated way to spot those...



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-6395) If the overseer queue is large, then the cloud tree view (admin UI) hangs

Timothy Potter created SOLR-6395:


 Summary: If the overseer queue is large, then the cloud tree view 
(admin UI) hangs
 Key: SOLR-6395
 URL: https://issues.apache.org/jira/browse/SOLR-6395
 Project: Solr
  Issue Type: Bug
  Components: SolrCloud, web gui
Reporter: Timothy Potter


Of course, an overseer queue that is backed up is a symptom of bigger issues, 
but  if it is, the tree view in the cloud panel becomes almost un-usable, 
presumably because the UI is trying to pull all the overseer queue child nodes? 
Be better to lazily load child nodes when the parent znode tree element is 
opened.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-6395) If the overseer queue is large, then the cloud tree view (admin UI) hangs


 [ 
https://issues.apache.org/jira/browse/SOLR-6395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Timothy Potter reassigned SOLR-6395:


Assignee: Timothy Potter

 If the overseer queue is large, then the cloud tree view (admin UI) hangs
 -

 Key: SOLR-6395
 URL: https://issues.apache.org/jira/browse/SOLR-6395
 Project: Solr
  Issue Type: Bug
  Components: SolrCloud, web gui
Reporter: Timothy Potter
Assignee: Timothy Potter

 Of course, an overseer queue that is backed up is a symptom of bigger issues, 
 but  if it is, the tree view in the cloud panel becomes almost un-usable, 
 presumably because the UI is trying to pull all the overseer queue child 
 nodes? Be better to lazily load child nodes when the parent znode tree 
 element is opened.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5896) A view potential reproducible issues


[ 
https://issues.apache.org/jira/browse/LUCENE-5896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104022#comment-14104022
 ] 

Robert Muir commented on LUCENE-5896:
-

+1

maybe in the future we can use some other strategy to at least detect or avoid 
such bugs, maybe ThreadLocalRandom?

But please fix these for now, they just stop tests from reproducing and so on.

 A view potential reproducible issues
 

 Key: LUCENE-5896
 URL: https://issues.apache.org/jira/browse/LUCENE-5896
 Project: Lucene - Core
  Issue Type: Test
  Components: general/test
Affects Versions: 4.9
Reporter: Simon Willnauer
 Fix For: 5.0, 4.10

 Attachments: LUCENE-5896.patch


 I realized that passing the same seeded random instance to LuceneTestCase# 
 newIndewWriterConfig doesn't necessarily produce the same IWC and I found a 
 bunch of issues in that class using global random rather than local random. 
 Yet, I went over the file to spot others but we might need to think about a 
 more automated way to spot those...



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5896) A view potential reproducible issues


[ 
https://issues.apache.org/jira/browse/LUCENE-5896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104028#comment-14104028
 ] 

Michael McCandless commented on LUCENE-5896:


+1, nice catch.

 A view potential reproducible issues
 

 Key: LUCENE-5896
 URL: https://issues.apache.org/jira/browse/LUCENE-5896
 Project: Lucene - Core
  Issue Type: Test
  Components: general/test
Affects Versions: 4.9
Reporter: Simon Willnauer
 Fix For: 5.0, 4.10

 Attachments: LUCENE-5896.patch


 I realized that passing the same seeded random instance to LuceneTestCase# 
 newIndewWriterConfig doesn't necessarily produce the same IWC and I found a 
 bunch of issues in that class using global random rather than local random. 
 Yet, I went over the file to spot others but we might need to think about a 
 more automated way to spot those...



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5894) refactor bulk merge logic


[ 
https://issues.apache.org/jira/browse/LUCENE-5894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104061#comment-14104061
 ] 

Michael McCandless commented on LUCENE-5894:


+1, I think this patch is nice; it's great to have merging fully under
control of the codec.  There are lots of nice improvements here:

  * SegmentMerger is much simpler

  * Merging responsibility moves to XXXConsumer, and bulk-merge optos
(and new MatchingReaders class) are now entirely codec private
(CompressingStoredFields/TVFormat)

  * Moved old writers (Lucene40StoredFields/TVsWriter) to
test-framework so compressing (current default) is the only writer
now.

  * We now need a NormsConsumer/Producer (can't reuse DVConsumer) since the
source for norms must be known in the default merge impl.

  * Factored out SegmentDocValuesProducer to hold all per-field DVPs,
updates.

  * Also separated out the classes in IW that buffer up norms in RAM
until flush from the DV classes, letting you remove
trackDocsWithField boolean...

I think this is a good cleanup!


 refactor bulk merge logic
 -

 Key: LUCENE-5894
 URL: https://issues.apache.org/jira/browse/LUCENE-5894
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Robert Muir
 Attachments: LUCENE-5894.patch


 Today its only usable really by stored fields/term vectors, has hardcoded 
 logic in SegmentMerger specific to certain impls, etc.
 It would be better if this was generalized to terms/postings/norms/docvalues 
 as well.
 Bulk merge is boring, the real idea is to allow codecs to do more: e.g. with 
 this patch they could do streaming checksum validation, or prevent the 
 loading of latent norms, or other things we cannot do today.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5683) Documentation of Suggester V2


[ 
https://issues.apache.org/jira/browse/SOLR-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104063#comment-14104063
 ] 

Cassandra Targett commented on SOLR-5683:
-

Thanks [~areek] and [~varunthacker]: I'll take a stab at getting your all this 
into the 4.10 Ref Guide.

 Documentation of Suggester V2
 -

 Key: SOLR-5683
 URL: https://issues.apache.org/jira/browse/SOLR-5683
 Project: Solr
  Issue Type: Task
  Components: SearchComponents - other
Reporter: Areek Zillur
Assignee: Cassandra Targett
 Fix For: 5.0, 4.10


 Place holder for documentation that will eventually end up in the Solr Ref 
 guide.
 
 The new Suggester Component allows Solr to fully utilize the Lucene 
 suggesters. 
 The main features are:
 - lookup pluggability (TODO: add description):
   -- AnalyzingInfixLookupFactory
   -- AnalyzingLookupFactory
   -- FuzzyLookupFactory
   -- FreeTextLookupFactory
   -- FSTLookupFactory
   -- WFSTLookupFactory
   -- TSTLookupFactory
   --  JaspellLookupFactory
- Dictionary pluggability (give users the option to choose the dictionary 
 implementation to use for their suggesters to consume)
-- Input from search index
   --- DocumentDictionaryFactory – user can specify suggestion field along 
 with optional weight and payload fields from their search index.
   --- DocumentExpressionFactory – same as DocumentDictionaryFactory but 
 allows users to specify arbitrary expression using existing numeric fields.
  --- HighFrequencyDictionaryFactory – user can specify a suggestion field 
 and specify a threshold to prune out less frequent terms.   
-- Input from external files
  --- FileDictionaryFactory – user can specify a file which contains 
 suggest entries, along with optional weights and payloads.
 Config (index time) options:
   - name - name of suggester
   - sourceLocation - external file location (for file-based suggesters)
   - lookupImpl - type of lookup to use [default JaspellLookupFactory]
   - dictionaryImpl - type of dictionary to use (lookup input) [default
 (sourceLocation == null ? HighFrequencyDictionaryFactory : 
 FileDictionaryFactory)]
   - storeDir - location to store in-memory data structure in disk
   - buildOnCommit - command to build suggester for every commit
   - buildOnOptimize - command to build suggester for every optimize
 Query time options:
   - suggest.dictionary - name of suggester to use (can occur multiple times 
 for batching suggester requests)
   - suggest.count - number of suggestions to return
   - suggest.q - query to use for lookup
   - suggest.build - command to build the suggester
   - suggest.reload - command to reload the suggester
   - buildAll – command to build all suggesters in the component
   - reloadAll – command to reload all suggesters in the component
 Example query:
 {code}
 http://localhost:8983/solr/suggest?suggest.dictionary=suggester1suggest=truesuggest.build=truesuggest.q=elec
 {code}
 Distributed query:
 {code}
 http://localhost:7574/solr/suggest?suggest.dictionary=suggester2suggest=truesuggest.build=truesuggest.q=elecshards=localhost:8983/solr,localhost:7574/solrshards.qt=/suggest
 {code}
 Response Format:
 The response format can be either XML or JSON. The typical response structure 
 is as follows:
  {code}
 {
   suggest: {
 suggester_name: {
suggest_query: { numFound:  .., suggestions: [ {term: .., weight: .., 
 payload: ..}, .. ]} 
}
 } 
 {code}
   
 Example Response:
 {code}
 {
 responseHeader: {
 status: 0,
 QTime: 3
 },
 suggest: {
 suggester1: {
 e: {
 numFound: 1,
 suggestions: [
 {
 term: electronics and computer1,
 weight: 100,
 payload: 
 }
 ]
 }
 },
 suggester2: {
 e: {
 numFound: 1,
 suggestions: [
 {
 term: electronics and computer1,
 weight: 10,
 payload: 
 }
 ]
 }
 }
 }
 }
 {code}
 Example solrconfig snippet with multiple suggester configuration:
 {code}  
   searchComponent name=suggest class=solr.SuggestComponent
 lst name=suggester
   str name=namesuggester1/str
   str name=lookupImplFuzzyLookupFactory/str  
   str name=dictionaryImplDocumentDictionaryFactory/str  
   str name=fieldcat/str
   str name=weightFieldprice/str
   str name=suggestAnalyzerFieldTypestring/str
 /lst
lst name=suggester
 str name=namesuggester2 /str
 str

[jira] [Assigned] (SOLR-5683) Documentation of Suggester V2


 [ 
https://issues.apache.org/jira/browse/SOLR-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cassandra Targett reassigned SOLR-5683:
---

Assignee: Cassandra Targett  (was: Areek Zillur)

 Documentation of Suggester V2
 -

 Key: SOLR-5683
 URL: https://issues.apache.org/jira/browse/SOLR-5683
 Project: Solr
  Issue Type: Task
  Components: SearchComponents - other
Reporter: Areek Zillur
Assignee: Cassandra Targett
 Fix For: 5.0, 4.10


 Place holder for documentation that will eventually end up in the Solr Ref 
 guide.
 
 The new Suggester Component allows Solr to fully utilize the Lucene 
 suggesters. 
 The main features are:
 - lookup pluggability (TODO: add description):
   -- AnalyzingInfixLookupFactory
   -- AnalyzingLookupFactory
   -- FuzzyLookupFactory
   -- FreeTextLookupFactory
   -- FSTLookupFactory
   -- WFSTLookupFactory
   -- TSTLookupFactory
   --  JaspellLookupFactory
- Dictionary pluggability (give users the option to choose the dictionary 
 implementation to use for their suggesters to consume)
-- Input from search index
   --- DocumentDictionaryFactory – user can specify suggestion field along 
 with optional weight and payload fields from their search index.
   --- DocumentExpressionFactory – same as DocumentDictionaryFactory but 
 allows users to specify arbitrary expression using existing numeric fields.
  --- HighFrequencyDictionaryFactory – user can specify a suggestion field 
 and specify a threshold to prune out less frequent terms.   
-- Input from external files
  --- FileDictionaryFactory – user can specify a file which contains 
 suggest entries, along with optional weights and payloads.
 Config (index time) options:
   - name - name of suggester
   - sourceLocation - external file location (for file-based suggesters)
   - lookupImpl - type of lookup to use [default JaspellLookupFactory]
   - dictionaryImpl - type of dictionary to use (lookup input) [default
 (sourceLocation == null ? HighFrequencyDictionaryFactory : 
 FileDictionaryFactory)]
   - storeDir - location to store in-memory data structure in disk
   - buildOnCommit - command to build suggester for every commit
   - buildOnOptimize - command to build suggester for every optimize
 Query time options:
   - suggest.dictionary - name of suggester to use (can occur multiple times 
 for batching suggester requests)
   - suggest.count - number of suggestions to return
   - suggest.q - query to use for lookup
   - suggest.build - command to build the suggester
   - suggest.reload - command to reload the suggester
   - buildAll – command to build all suggesters in the component
   - reloadAll – command to reload all suggesters in the component
 Example query:
 {code}
 http://localhost:8983/solr/suggest?suggest.dictionary=suggester1suggest=truesuggest.build=truesuggest.q=elec
 {code}
 Distributed query:
 {code}
 http://localhost:7574/solr/suggest?suggest.dictionary=suggester2suggest=truesuggest.build=truesuggest.q=elecshards=localhost:8983/solr,localhost:7574/solrshards.qt=/suggest
 {code}
 Response Format:
 The response format can be either XML or JSON. The typical response structure 
 is as follows:
  {code}
 {
   suggest: {
 suggester_name: {
suggest_query: { numFound:  .., suggestions: [ {term: .., weight: .., 
 payload: ..}, .. ]} 
}
 } 
 {code}
   
 Example Response:
 {code}
 {
 responseHeader: {
 status: 0,
 QTime: 3
 },
 suggest: {
 suggester1: {
 e: {
 numFound: 1,
 suggestions: [
 {
 term: electronics and computer1,
 weight: 100,
 payload: 
 }
 ]
 }
 },
 suggester2: {
 e: {
 numFound: 1,
 suggestions: [
 {
 term: electronics and computer1,
 weight: 10,
 payload: 
 }
 ]
 }
 }
 }
 }
 {code}
 Example solrconfig snippet with multiple suggester configuration:
 {code}  
   searchComponent name=suggest class=solr.SuggestComponent
 lst name=suggester
   str name=namesuggester1/str
   str name=lookupImplFuzzyLookupFactory/str  
   str name=dictionaryImplDocumentDictionaryFactory/str  
   str name=fieldcat/str
   str name=weightFieldprice/str
   str name=suggestAnalyzerFieldTypestring/str
 /lst
lst name=suggester
 str name=namesuggester2 /str
 str name=dictionaryImplDocumentExpressionDictionaryFactory/str
 str name=lookupImplFuzzyLookupFactory/str
 str

[jira] [Updated] (SOLR-5683) Documentation of Suggester V2


 [ 
https://issues.apache.org/jira/browse/SOLR-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cassandra Targett updated SOLR-5683:


Fix Version/s: (was: 4.9)
   4.10

 Documentation of Suggester V2
 -

 Key: SOLR-5683
 URL: https://issues.apache.org/jira/browse/SOLR-5683
 Project: Solr
  Issue Type: Task
  Components: SearchComponents - other
Reporter: Areek Zillur
Assignee: Cassandra Targett
 Fix For: 5.0, 4.10


 Place holder for documentation that will eventually end up in the Solr Ref 
 guide.
 
 The new Suggester Component allows Solr to fully utilize the Lucene 
 suggesters. 
 The main features are:
 - lookup pluggability (TODO: add description):
   -- AnalyzingInfixLookupFactory
   -- AnalyzingLookupFactory
   -- FuzzyLookupFactory
   -- FreeTextLookupFactory
   -- FSTLookupFactory
   -- WFSTLookupFactory
   -- TSTLookupFactory
   --  JaspellLookupFactory
- Dictionary pluggability (give users the option to choose the dictionary 
 implementation to use for their suggesters to consume)
-- Input from search index
   --- DocumentDictionaryFactory – user can specify suggestion field along 
 with optional weight and payload fields from their search index.
   --- DocumentExpressionFactory – same as DocumentDictionaryFactory but 
 allows users to specify arbitrary expression using existing numeric fields.
  --- HighFrequencyDictionaryFactory – user can specify a suggestion field 
 and specify a threshold to prune out less frequent terms.   
-- Input from external files
  --- FileDictionaryFactory – user can specify a file which contains 
 suggest entries, along with optional weights and payloads.
 Config (index time) options:
   - name - name of suggester
   - sourceLocation - external file location (for file-based suggesters)
   - lookupImpl - type of lookup to use [default JaspellLookupFactory]
   - dictionaryImpl - type of dictionary to use (lookup input) [default
 (sourceLocation == null ? HighFrequencyDictionaryFactory : 
 FileDictionaryFactory)]
   - storeDir - location to store in-memory data structure in disk
   - buildOnCommit - command to build suggester for every commit
   - buildOnOptimize - command to build suggester for every optimize
 Query time options:
   - suggest.dictionary - name of suggester to use (can occur multiple times 
 for batching suggester requests)
   - suggest.count - number of suggestions to return
   - suggest.q - query to use for lookup
   - suggest.build - command to build the suggester
   - suggest.reload - command to reload the suggester
   - buildAll – command to build all suggesters in the component
   - reloadAll – command to reload all suggesters in the component
 Example query:
 {code}
 http://localhost:8983/solr/suggest?suggest.dictionary=suggester1suggest=truesuggest.build=truesuggest.q=elec
 {code}
 Distributed query:
 {code}
 http://localhost:7574/solr/suggest?suggest.dictionary=suggester2suggest=truesuggest.build=truesuggest.q=elecshards=localhost:8983/solr,localhost:7574/solrshards.qt=/suggest
 {code}
 Response Format:
 The response format can be either XML or JSON. The typical response structure 
 is as follows:
  {code}
 {
   suggest: {
 suggester_name: {
suggest_query: { numFound:  .., suggestions: [ {term: .., weight: .., 
 payload: ..}, .. ]} 
}
 } 
 {code}
   
 Example Response:
 {code}
 {
 responseHeader: {
 status: 0,
 QTime: 3
 },
 suggest: {
 suggester1: {
 e: {
 numFound: 1,
 suggestions: [
 {
 term: electronics and computer1,
 weight: 100,
 payload: 
 }
 ]
 }
 },
 suggester2: {
 e: {
 numFound: 1,
 suggestions: [
 {
 term: electronics and computer1,
 weight: 10,
 payload: 
 }
 ]
 }
 }
 }
 }
 {code}
 Example solrconfig snippet with multiple suggester configuration:
 {code}  
   searchComponent name=suggest class=solr.SuggestComponent
 lst name=suggester
   str name=namesuggester1/str
   str name=lookupImplFuzzyLookupFactory/str  
   str name=dictionaryImplDocumentDictionaryFactory/str  
   str name=fieldcat/str
   str name=weightFieldprice/str
   str name=suggestAnalyzerFieldTypestring/str
 /lst
lst name=suggester
 str name=namesuggester2 /str
 str name=dictionaryImplDocumentExpressionDictionaryFactory/str
 str name=lookupImplFuzzyLookupFactory/str
 str

[jira] [Comment Edited] (SOLR-5683) Documentation of Suggester V2


[ 
https://issues.apache.org/jira/browse/SOLR-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104063#comment-14104063
 ] 

Cassandra Targett edited comment on SOLR-5683 at 8/20/14 3:55 PM:
--

Thanks [~areek] and [~varunthacker]: I'll take a stab at getting all this into 
the 4.10 Ref Guide.


was (Author: ctargett):
Thanks [~areek] and [~varunthacker]: I'll take a stab at getting your all this 
into the 4.10 Ref Guide.

 Documentation of Suggester V2
 -

 Key: SOLR-5683
 URL: https://issues.apache.org/jira/browse/SOLR-5683
 Project: Solr
  Issue Type: Task
  Components: SearchComponents - other
Reporter: Areek Zillur
Assignee: Cassandra Targett
 Fix For: 5.0, 4.10


 Place holder for documentation that will eventually end up in the Solr Ref 
 guide.
 
 The new Suggester Component allows Solr to fully utilize the Lucene 
 suggesters. 
 The main features are:
 - lookup pluggability (TODO: add description):
   -- AnalyzingInfixLookupFactory
   -- AnalyzingLookupFactory
   -- FuzzyLookupFactory
   -- FreeTextLookupFactory
   -- FSTLookupFactory
   -- WFSTLookupFactory
   -- TSTLookupFactory
   --  JaspellLookupFactory
- Dictionary pluggability (give users the option to choose the dictionary 
 implementation to use for their suggesters to consume)
-- Input from search index
   --- DocumentDictionaryFactory – user can specify suggestion field along 
 with optional weight and payload fields from their search index.
   --- DocumentExpressionFactory – same as DocumentDictionaryFactory but 
 allows users to specify arbitrary expression using existing numeric fields.
  --- HighFrequencyDictionaryFactory – user can specify a suggestion field 
 and specify a threshold to prune out less frequent terms.   
-- Input from external files
  --- FileDictionaryFactory – user can specify a file which contains 
 suggest entries, along with optional weights and payloads.
 Config (index time) options:
   - name - name of suggester
   - sourceLocation - external file location (for file-based suggesters)
   - lookupImpl - type of lookup to use [default JaspellLookupFactory]
   - dictionaryImpl - type of dictionary to use (lookup input) [default
 (sourceLocation == null ? HighFrequencyDictionaryFactory : 
 FileDictionaryFactory)]
   - storeDir - location to store in-memory data structure in disk
   - buildOnCommit - command to build suggester for every commit
   - buildOnOptimize - command to build suggester for every optimize
 Query time options:
   - suggest.dictionary - name of suggester to use (can occur multiple times 
 for batching suggester requests)
   - suggest.count - number of suggestions to return
   - suggest.q - query to use for lookup
   - suggest.build - command to build the suggester
   - suggest.reload - command to reload the suggester
   - buildAll – command to build all suggesters in the component
   - reloadAll – command to reload all suggesters in the component
 Example query:
 {code}
 http://localhost:8983/solr/suggest?suggest.dictionary=suggester1suggest=truesuggest.build=truesuggest.q=elec
 {code}
 Distributed query:
 {code}
 http://localhost:7574/solr/suggest?suggest.dictionary=suggester2suggest=truesuggest.build=truesuggest.q=elecshards=localhost:8983/solr,localhost:7574/solrshards.qt=/suggest
 {code}
 Response Format:
 The response format can be either XML or JSON. The typical response structure 
 is as follows:
  {code}
 {
   suggest: {
 suggester_name: {
suggest_query: { numFound:  .., suggestions: [ {term: .., weight: .., 
 payload: ..}, .. ]} 
}
 } 
 {code}
   
 Example Response:
 {code}
 {
 responseHeader: {
 status: 0,
 QTime: 3
 },
 suggest: {
 suggester1: {
 e: {
 numFound: 1,
 suggestions: [
 {
 term: electronics and computer1,
 weight: 100,
 payload: 
 }
 ]
 }
 },
 suggester2: {
 e: {
 numFound: 1,
 suggestions: [
 {
 term: electronics and computer1,
 weight: 10,
 payload: 
 }
 ]
 }
 }
 }
 }
 {code}
 Example solrconfig snippet with multiple suggester configuration:
 {code}  
   searchComponent name=suggest class=solr.SuggestComponent
 lst name=suggester
   str name=namesuggester1/str
   str name=lookupImplFuzzyLookupFactory/str  
   str name=dictionaryImplDocumentDictionaryFactory/str  
   str name=fieldcat/str
   str

[jira] [Commented] (LUCENE-5895) Add per-segment and per-commit id to help replication


[ 
https://issues.apache.org/jira/browse/LUCENE-5895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104081#comment-14104081
 ] 

Michael McCandless commented on LUCENE-5895:


I looked at UUID, and used UUID.randomUUID() in early iterations at first,
but then decided it's a poor fit here:

  * It tries to be crypographically secure, which is overkill for us:
we don't care if someone can guess what the next segment id will
be.

  * It uses SecureRandom to pull randomness, which on Linux can easily
lead to long hangs (I saw ~10 second hangs) when there's not
enough entropy being harvested.

  * It loses a few (6 of 128) bits to version/variant, which weakens
it a bit for our use case.  In particular, it's not clear how that
impacts the period, where with simple ++ (mod 2^128) the period is
clearly full.


 Add per-segment and per-commit id to help replication
 -

 Key: LUCENE-5895
 URL: https://issues.apache.org/jira/browse/LUCENE-5895
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Michael McCandless
Assignee: Michael McCandless
 Fix For: 5.0, 4.10

 Attachments: LUCENE-5895.patch


 It would be useful if Lucene recorded a unique id for each segment written 
 and each commit point.  This way, file-based replicators can use this to 
 know whether the segment/commit they are looking at on a source machine and 
 dest machine are in fact that same.
 I know this would have been very useful when I was playing with NRT 
 replication (LUCENE-5438).



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-trunk-Linux (64bit/ibm-j9-jdk7) - Build # 11062 - Still Failing!

2014-08-20 Thread Michael McCandless

J9 bug ...

Mike McCandless

http://blog.mikemccandless.com


On Wed, Aug 20, 2014 at 7:44 AM, Policeman Jenkins Server
jenk...@thetaphi.de wrote:
 Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/11062/
 Java: 64bit/ibm-j9-jdk7 
 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}

 2 tests failed.
 FAILED:  
 junit.framework.TestSuite.org.apache.lucene.search.TestSimpleSearchEquivalence

 Error Message:


 Stack Trace:
 java.lang.NullPointerException
 at __randomizedtesting.SeedInfo.seed([A798244826442DBA]:0)
 at 
 org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer$NormMap.add(Lucene49NormsConsumer.java:226)
 at 
 org.apache.lucene.codecs.lucene49.Lucene49NormsConsumer.addNumericField(Lucene49NormsConsumer.java:95)
 at 
 org.apache.lucene.codecs.DocValuesConsumer.mergeNumericField(DocValuesConsumer.java:129)
 at 
 org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:253)
 at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:131)
 at 
 org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3995)
 at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3590)
 at 
 org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
 at 
 org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1883)
 at 
 org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4584)
 at 
 org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:697)
 at 
 org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4609)
 at 
 org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4601)
 at 
 org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1391)
 at 
 org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1105)
 at 
 org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:143)
 at 
 org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:104)
 at 
 org.apache.lucene.search.SearchEquivalenceTestBase.beforeClass(SearchEquivalenceTestBase.java:74)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:94)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
 at java.lang.reflect.Method.invoke(Method.java:619)
 at 
 com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
 at 
 com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:767)
 at 
 com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
 at 
 org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
 at 
 org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
 at 
 com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
 at 
 com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
 at 
 com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
 at 
 com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
 at 
 com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
 at 
 com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
 at 
 org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
 at 
 org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
 at 
 org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
 at 
 org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
 at 
 com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
 at 
 com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
 at java.lang.Thread.run(Thread.java:853)


 FAILED:  
 junit.framework.TestSuite.org.apache.lucene.search.TestSimpleSearchEquivalence

 Error Message:


 Stack Trace:
 java.lang.NullPointerException
 at __randomizedtesting.SeedInfo.seed([A798244826442DBA]:0)
 at 
 org.apache.lucene.search.SearchEquivalenceTestBase.afterClass(SearchEquivalenceTestBase.java:96)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at

[jira] [Commented] (LUCENE-5889) AnalyzingInfixSuggester should expose commit()


[ 
https://issues.apache.org/jira/browse/LUCENE-5889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104098#comment-14104098
 ] 

Michael McCandless commented on LUCENE-5889:


Thanks Varun!

I think we shouldn't add another ctor?  (The API is experimental.).
Just add the new commitOnBuild param ...

I think it's fine to init the writer in add/update, but can you pull
that out to its own method, e.g. getWriter?  And I guess it should be
sync'd, in case multiple threads try to add/update at once?

And maybe add some tests showing that you can just use .add and then
refresh and see the suggestions?


 AnalyzingInfixSuggester should expose commit()
 --

 Key: LUCENE-5889
 URL: https://issues.apache.org/jira/browse/LUCENE-5889
 Project: Lucene - Core
  Issue Type: Improvement
  Components: modules/spellchecker
Reporter: Mike Sokolov
 Attachments: LUCENE-5889.patch


 There is no way short of close() for a user of AnalyzingInfixSuggester to 
 cause it to commit() its underlying index: only refresh() is provided.  But 
 callers might want to ensure the index is flushed to disk without closing.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5895) Add per-segment and per-commit id to help replication

2014-08-20 Thread ASF subversion and git services (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-5895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless updated LUCENE-5895:
---

Attachment: LUCENE-5895.patch

New patch: I hit a failure in the test I added when it got an older codec that 
doesn't write the id, and fixed SegmentInfo ctor to not generate its own id 
(that's too dangerous).  I also added a comment explaining why UUID isn't a 
good match for our use ...

 Add per-segment and per-commit id to help replication
 -

 Key: LUCENE-5895
 URL: https://issues.apache.org/jira/browse/LUCENE-5895
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Michael McCandless
Assignee: Michael McCandless
 Fix For: 5.0, 4.10

 Attachments: LUCENE-5895.patch, LUCENE-5895.patch


 It would be useful if Lucene recorded a unique id for each segment written 
 and each commit point.  This way, file-based replicators can use this to 
 know whether the segment/commit they are looking at on a source machine and 
 dest machine are in fact that same.
 I know this would have been very useful when I was playing with NRT 
 replication (LUCENE-5438).



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5714) Improve tests for BBoxStrategy then port to 4x.

[
https://issues.apache.org/jira/browse/LUCENE-5714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104148#comment-14104148
]

ASF subversion and git services commented on LUCENE-5714:
-

Commit 1619163 from [~dsmiley] in branch 'dev/trunk'
[ https://svn.apache.org/r1619163 ]

LUCENE-5714: TestBBoxStrategy work-around for Spatial4j Rect bug #85

Improve tests for BBoxStrategy then port to 4x.
---

Key: LUCENE-5714
URL: https://issues.apache.org/jira/browse/LUCENE-5714
Project: Lucene - Core
Issue Type: Improvement
Components: modules/spatial
Reporter: David Smiley
Assignee: David Smiley
Fix For: 5.0, 4.10

Attachments: LUCENE-5714_Enhance_BBoxStrategy.patch,
LUCENE-5714__Enhance_BBoxStrategy__more_tests,_fix_dateline_bugs,_new_AreaSimilarity_algor.patch

BBoxStrategy needs better tests before I'm comfortable seeing it in 4x.
Specifically it should use random rectangles based validation (ones that may
cross the dateline), akin to the other tests. And I think I see an
equals/hashcode bug to be fixed in there too.
One particular thing I'd like to see added is how to handle a zero-area case
for AreaSimilarity. I think an additional feature in which you declare a
minimum % area (relative to the query shape) would be good.
It should be possible for the user to combine rectangle center-point to query
shape center-point distance sorting as well. I think it is but I need to
make sure it's possible without _having_ to index a separate center point
field.
Another possibility (probably not to be addressed here) is a minimum ratio
between width/height, perhaps 10%. A long but nearly no height line should
not be massively disadvantaged relevancy-wise to an equivalently long
diagonal road that has a square bbox.

--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5714) Improve tests for BBoxStrategy then port to 4x.

2014-08-20 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104150#comment-14104150
 ] 

ASF subversion and git services commented on LUCENE-5714:
-

Commit 1619165 from [~dsmiley] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1619165 ]

LUCENE-5714: TestBBoxStrategy work-around for Spatial4j Rect bug #85

 Improve tests for BBoxStrategy then port to 4x.
 ---

 Key: LUCENE-5714
 URL: https://issues.apache.org/jira/browse/LUCENE-5714
 Project: Lucene - Core
  Issue Type: Improvement
  Components: modules/spatial
Reporter: David Smiley
Assignee: David Smiley
 Fix For: 5.0, 4.10

 Attachments: LUCENE-5714_Enhance_BBoxStrategy.patch, 
 LUCENE-5714__Enhance_BBoxStrategy__more_tests,_fix_dateline_bugs,_new_AreaSimilarity_algor.patch


 BBoxStrategy needs better tests before I'm comfortable seeing it in 4x.  
 Specifically it should use random rectangles based validation (ones that may 
 cross the dateline), akin to the other tests.  And I think I see an 
 equals/hashcode bug to be fixed in there too.
 One particular thing I'd like to see added is how to handle a zero-area case 
 for AreaSimilarity.  I think an additional feature in which you declare a 
 minimum % area (relative to the query shape) would be good.
 It should be possible for the user to combine rectangle center-point to query 
 shape center-point distance sorting as well.  I think it is but I need to 
 make sure it's possible without _having_ to index a separate center point 
 field.
 Another possibility (probably not to be addressed here) is a minimum ratio 
 between width/height, perhaps 10%.  A long but nearly no height line should 
 not be massively disadvantaged relevancy-wise to an equivalently long 
 diagonal road that has a square bbox.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6178) Deprecate Jaspell suggester

2014-08-20 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-6178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104169#comment-14104169
 ] 

ASF subversion and git services commented on SOLR-6178:
---

Commit 1619172 from hoss...@apache.org in branch 'dev/trunk'
[ https://svn.apache.org/r1619172 ]

SOLR-6178: backout deprecation until we have a diff default

 Deprecate Jaspell suggester
 ---

 Key: SOLR-6178
 URL: https://issues.apache.org/jira/browse/SOLR-6178
 Project: Solr
  Issue Type: Improvement
  Components: spellchecker
Reporter: Michael McCandless
Assignee: Uwe Schindler
 Fix For: 5.0, 4.10

 Attachments: SOLR-6178.patch


 Right now Solr defaults to Jaspell, but we've deprecated it in LUCENE-5775 
 ... and in trunk I'd like to remove it.  But first we need to fix Solr to not 
 default to it.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6178) Deprecate Jaspell suggester

2014-08-20 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-6178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104175#comment-14104175
 ] 

ASF subversion and git services commented on SOLR-6178:
---

Commit 1619173 from hoss...@apache.org in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1619173 ]

SOLR-6178: backout deprecation until we have a diff default (merge r1619172)

 Deprecate Jaspell suggester
 ---

 Key: SOLR-6178
 URL: https://issues.apache.org/jira/browse/SOLR-6178
 Project: Solr
  Issue Type: Improvement
  Components: spellchecker
Reporter: Michael McCandless
Assignee: Uwe Schindler
 Fix For: 5.0, 4.10

 Attachments: SOLR-6178.patch


 Right now Solr defaults to Jaspell, but we've deprecated it in LUCENE-5775 
 ... and in trunk I'd like to remove it.  But first we need to fix Solr to not 
 default to it.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-5897) performance bug (adversary) in StandardTokenizer

Robert Muir created LUCENE-5897:
---

 Summary: performance bug (adversary) in StandardTokenizer
 Key: LUCENE-5897
 URL: https://issues.apache.org/jira/browse/LUCENE-5897
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Robert Muir


There seem to be some conditions (I don't know how rare or what conditions) 
that cause StandardTokenizer to essentially hang on input: I havent looked hard 
yet, but as its essentially a DFA I think something wierd might be going on.

An easy way to reproduce is with 1MB of underscores, it will just hang forever.
{code}
  public void testWorthyAdversary() throws Exception {
char buffer[] = new char[1024 * 1024];
Arrays.fill(buffer, '_');
int tokenCount = 0;
Tokenizer ts = new StandardTokenizer();
ts.setReader(new StringReader(new String(buffer)));
ts.reset();
while (ts.incrementToken()) {
  tokenCount++;
}
ts.end();
ts.close();
assertEquals(0, tokenCount);
  }
{code} 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6178) Deprecate Jaspell suggester

2014-08-20 Thread Hoss Man (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-6178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104193#comment-14104193
 ] 

Hoss Man commented on SOLR-6178:


removed deprecation and CHANGES.txt entry.

updating linkage to make it clear we can't deprecate until we come up with a 
better default (SOLR-6185)

 Deprecate Jaspell suggester
 ---

 Key: SOLR-6178
 URL: https://issues.apache.org/jira/browse/SOLR-6178
 Project: Solr
  Issue Type: Improvement
  Components: spellchecker
Reporter: Michael McCandless
Assignee: Uwe Schindler
 Fix For: 5.0, 4.10

 Attachments: SOLR-6178.patch


 Right now Solr defaults to Jaspell, but we've deprecated it in LUCENE-5775 
 ... and in trunk I'd like to remove it.  But first we need to fix Solr to not 
 default to it.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5897) performance bug (adversary) in StandardTokenizer


[ 
https://issues.apache.org/jira/browse/LUCENE-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104202#comment-14104202
 ] 

Robert Muir commented on LUCENE-5897:
-

it seems stuck here:
{noformat}
TRACE 301319:

org.apache.lucene.analysis.standard.StandardTokenizerImpl.getNextToken(StandardTokenizerImpl.java:756)

org.apache.lucene.analysis.standard.StandardTokenizer.incrementToken(StandardTokenizer.java:150)

org.apache.lucene.analysis.core.TestStandardAnalyzer.testWorthyAdversary(TestStandardAnalyzer.java:286)
{noformat}

This is in generated code: so I don't yet know if its something about our 
grammar or something in jflex itself?

 performance bug (adversary) in StandardTokenizer
 --

 Key: LUCENE-5897
 URL: https://issues.apache.org/jira/browse/LUCENE-5897
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Robert Muir

 There seem to be some conditions (I don't know how rare or what conditions) 
 that cause StandardTokenizer to essentially hang on input: I havent looked 
 hard yet, but as its essentially a DFA I think something wierd might be going 
 on.
 An easy way to reproduce is with 1MB of underscores, it will just hang 
 forever.
 {code}
   public void testWorthyAdversary() throws Exception {
 char buffer[] = new char[1024 * 1024];
 Arrays.fill(buffer, '_');
 int tokenCount = 0;
 Tokenizer ts = new StandardTokenizer();
 ts.setReader(new StringReader(new String(buffer)));
 ts.reset();
 while (ts.incrementToken()) {
   tokenCount++;
 }
 ts.end();
 ts.close();
 assertEquals(0, tokenCount);
   }
 {code} 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: FW: [jira] [Commented] (LUCENE-5886) current ecj-javadoc-lint crashes on SharedFSAutoReplicaFailoverUtilsTest.java

2014-08-20 Thread Uwe Schindler

Hi Balchandra,

Thanks fort he info! I installed JDK 8u20 and JDK 7u67 on the Jenkins machines 
(Linux, Windows, OSX).
In addition, on Linux, we now also test JDK 9 build 26.

Give me a note, if we should also look at stuff like 7u80 and 8u40 previews!

Uwe

-
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de


 -Original Message-
 From: Balchandra Vaidya [mailto:balchandra.vai...@oracle.com]
 Sent: Wednesday, August 20, 2014 1:42 PM
 To: Uwe Schindler
 Cc: dev@lucene.apache.org; rory.odonn...@oracle.com; 'Dalibor Topic'
 Subject: Re: FW: [jira] [Commented] (LUCENE-5886) current ecj-javadoc-lint
 crashes on SharedFSAutoReplicaFailoverUtilsTest.java
 
 
 Hi Uwe,
 
 As you might have already noticed, Java SE 8u20 has been released and is
 available from
 http://www.oracle.com/technetwork/java/javase/downloads/index.html
 
 Thanks
 Balchandra
 
 
 On 08/18/14 11:48 AM, Balchandra Vaidya wrote:
 
  Hi Uwe,
 
  On 08/15/14 11:34 PM, Uwe Schindler wrote:
  Hi Rory,
 
  I opened https://issues.apache.org/jira/browse/LUCENE-5890 to track
  any problems. I will install JDK 9 and update the other JDKs during
  the next week.
  Is there any release date for Java 8 update 20? If yes, I could
  combine the updates, because it always causes downtime of virtual
  machines.
  8u20 is expected to be released soon.
  http://openjdk.java.net/projects/jdk8u/releases/8u20.html
 
  We will inform you as soon as the release go out.
 
 
  Kind regards,
  Balchandra
 
 
 
  Uwe
 
  -
  Uwe Schindler
  H.-H.-Meier-Allee 63, D-28213 Bremen
  http://www.thetaphi.de
  eMail: u...@thetaphi.de
 
 
  -Original Message-
  From: Rory O'Donnell Oracle, Dublin Ireland
  [mailto:rory.odonn...@oracle.com]
  Sent: Friday, August 15, 2014 7:20 PM
  To: Uwe Schindler; 'Balchandra Vaidya'; 'Dalibor Topic'
  Cc: dev@lucene.apache.org
  Subject: Re: FW: [jira] [Commented] (LUCENE-5886) current
  ecj-javadoc-lint crashes on
  SharedFSAutoReplicaFailoverUtilsTest.java
 
  Thanks for the update Uwe!
  On 15/08/2014 17:49, Uwe Schindler wrote:
  Hi Rory,
 
  FYI, the JDK 9 b26 build seems to work now with Lucene. I have not
  yet
  completed the tests (no Solr up to now, only Lucene), so we might
  add it as build JDK to the Policeman Jenkins server soon!
  As you see in the attached issue mail
  (https://issues.apache.org/jira/browse/LUCENE-5886), I will add Java
  9 support to our build files (and a new Constant accessible from
  Lucene classes), so some conditionals in the ANT build works correct
  (we do some checks only on specific JVMs).
  Uwe
 
  -
  Uwe Schindler
  H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de
  eMail: u...@thetaphi.de
 
 
  --
  Rgds,Rory O'Donnell
  Quality Engineering Manager
  Oracle EMEA , Dublin, Ireland
 
 
  -
  To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
  For additional commands, e-mail: dev-h...@lucene.apache.org
 


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6393) Improve transaction log replay speed on HDFS.


[ 
https://issues.apache.org/jira/browse/SOLR-6393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104246#comment-14104246
 ] 

Erick Erickson commented on SOLR-6393:
--

Assuming that

needs to be replayed on startup and updates are not coming in
should read
needs to be replayed on startup and updates are coming in


 Improve transaction log replay speed on HDFS.
 -

 Key: SOLR-6393
 URL: https://issues.apache.org/jira/browse/SOLR-6393
 Project: Solr
  Issue Type: Improvement
Reporter: Mark Miller
Assignee: Mark Miller
 Fix For: 5.0, 4.10

 Attachments: SOLR-6393.patch


 Replay speed is pretty slow on HDFS because we currently reopen a reader 
 between reading each update.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6393) Improve transaction log replay speed on HDFS.


[ 
https://issues.apache.org/jira/browse/SOLR-6393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104251#comment-14104251
 ] 

Mark Miller commented on SOLR-6393:
---

No, it's written right. The second paragraph addresses when updates are also 
coming in.

 Improve transaction log replay speed on HDFS.
 -

 Key: SOLR-6393
 URL: https://issues.apache.org/jira/browse/SOLR-6393
 Project: Solr
  Issue Type: Improvement
Reporter: Mark Miller
Assignee: Mark Miller
 Fix For: 5.0, 4.10

 Attachments: SOLR-6393.patch


 Replay speed is pretty slow on HDFS because we currently reopen a reader 
 between reading each update.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses


[ 
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104249#comment-14104249
 ] 

Erick Erickson commented on SOLR-4586:
--

Bragadeesh:

You can set it in Solr via a setting in solrconfig.xml, and in Lucene by the 
appropriate setter method.

This is just a default, not a hard limit.



 Increase default maxBooleanClauses
 --

 Key: SOLR-4586
 URL: https://issues.apache.org/jira/browse/SOLR-4586
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.2
 Environment: 4.3-SNAPSHOT 1456767M - ncindex - 2013-03-15 13:11:50
Reporter: Shawn Heisey
 Attachments: SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch, 
 SOLR-4586.patch, SOLR-4586.patch, SOLR-4586_verify_maxClauses.patch


 In the #solr IRC channel, I mentioned the maxBooleanClauses limitation to 
 someone asking a question about queries.  Mark Miller told me that 
 maxBooleanClauses no longer applies, that the limitation was removed from 
 Lucene sometime in the 3.x series.  The config still shows up in the example 
 even in the just-released 4.2.
 Checking through the source code, I found that the config option is parsed 
 and the value stored in objects, but does not actually seem to be used by 
 anything.  I removed every trace of it that I could find, and all tests still 
 pass.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-2894) Implement distributed pivot faceting

2014-08-20 Thread Hoss Man (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man resolved SOLR-2894.


   Resolution: Fixed
Fix Version/s: (was: 4.9)
   4.10

This has been backported to 4x for almost 72 hours w/o any sign of problems 
from jenkins.  i think it's safe to call this reslved.

A big thanks to everyone who contributed to this issue over the years, with 
code and tests and feedback and reports of problems using the various patches 
against various input ... and especially thanks to everyone for your patience 
and persistence -- it's definitely paid off.

 Implement distributed pivot faceting
 

 Key: SOLR-2894
 URL: https://issues.apache.org/jira/browse/SOLR-2894
 Project: Solr
  Issue Type: Improvement
Reporter: Erik Hatcher
Assignee: Hoss Man
 Fix For: 5.0, 4.10

 Attachments: 48.pivotfails.log.bz2, 
 SOLR-2894-mincount-minification.patch, SOLR-2894-reworked.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894_cloud_test.patch, dateToObject.patch, 
 pivot_mincount_problem.sh, pivotfail.log


 Following up on SOLR-792, pivot faceting currently only supports 
 undistributed mode.  Distributed pivot faceting needs to be implemented.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6393) Improve transaction log replay speed on HDFS.


[ 
https://issues.apache.org/jira/browse/SOLR-6393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104262#comment-14104262
 ] 

Erick Erickson commented on SOLR-6393:
--

Ah, missed that. Thanks!

 Improve transaction log replay speed on HDFS.
 -

 Key: SOLR-6393
 URL: https://issues.apache.org/jira/browse/SOLR-6393
 Project: Solr
  Issue Type: Improvement
Reporter: Mark Miller
Assignee: Mark Miller
 Fix For: 5.0, 4.10

 Attachments: SOLR-6393.patch


 Replay speed is pretty slow on HDFS because we currently reopen a reader 
 between reading each update.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: 4.10

2014-08-20 Thread Chris Hostetter


: Sure, I can wait. Please let me know once you are satisfied with its
: stability on 4x.

Thanks Ryan,

it's been on 4x for ~72 hours w/o any sign of problems -- so i'm feeling 
very satisfied.

Proceed at your lesuire.

-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4807 - Failure

2014-08-20 Thread Apache Jenkins Server

Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4807/

All tests passed

Build Log:
[...truncated 46637 lines...]
-documentation-lint:
 [echo] checking for broken html...
[jtidy] Checking for broken html (such as invalid tags)...
   [delete] Deleting directory 
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/build/jtidy_tmp
 [echo] Checking for broken links...
 [exec] 
 [exec] Crawl/parse...
 [exec] 
 [exec] Verify...
 [echo] Checking for missing docs...
 [exec] 
 [exec] 
build/docs/classification/org/apache/lucene/classification/SimpleNaiveBayesClassifier.html
 [exec]   missing Fields: analyzer
 [exec]   missing Fields: atomicReader
 [exec]   missing Fields: classFieldName
 [exec]   missing Fields: indexSearcher
 [exec]   missing Fields: query
 [exec]   missing Fields: textFieldNames
 [exec]   missing Methods: countDocsWithClass()
 [exec]   missing Methods: tokenizeDoc(java.lang.String)
 [exec] 
 [exec] Missing javadocs were found!

BUILD FAILED
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:474:
 The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:63:
 The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/build.xml:212:
 The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/build.xml:242:
 The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:2425:
 exec returned: 1

Total time: 155 minutes 30 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-trunk-Java7 #4806
Archived 1 artifacts
Archive block size is 32768
Received 0 blocks and 464 bytes
Compression is 0.0%
Took 25 ms
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5897) performance bug (adversary) in StandardTokenizer


[ 
https://issues.apache.org/jira/browse/LUCENE-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104272#comment-14104272
 ] 

Robert Muir commented on LUCENE-5897:
-

I spent some time trying to debug it, didn't get very far.

I know at least you can substitute any Extend_Num_Let for the underscore and it 
hangs. I don't yet know if other word break categories would have a similar 
issue, maybe even in other contexts.

 performance bug (adversary) in StandardTokenizer
 --

 Key: LUCENE-5897
 URL: https://issues.apache.org/jira/browse/LUCENE-5897
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Robert Muir

 There seem to be some conditions (I don't know how rare or what conditions) 
 that cause StandardTokenizer to essentially hang on input: I havent looked 
 hard yet, but as its essentially a DFA I think something wierd might be going 
 on.
 An easy way to reproduce is with 1MB of underscores, it will just hang 
 forever.
 {code}
   public void testWorthyAdversary() throws Exception {
 char buffer[] = new char[1024 * 1024];
 Arrays.fill(buffer, '_');
 int tokenCount = 0;
 Tokenizer ts = new StandardTokenizer();
 ts.setReader(new StringReader(new String(buffer)));
 ts.reset();
 while (ts.incrementToken()) {
   tokenCount++;
 }
 ts.end();
 ts.close();
 assertEquals(0, tokenCount);
   }
 {code} 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: 4.10

2014-08-20 Thread Steve Molloy

Nice to see this finally getting in, thanks for the huge effort. :)

Steve

From: Chris Hostetter [hossman_luc...@fucit.org]
Sent: August 20, 2014 2:15 PM
To: dev@lucene.apache.org
Subject: Re: 4.10

: Sure, I can wait. Please let me know once you are satisfied with its
: stability on 4x.

Thanks Ryan,

it's been on 4x for ~72 hours w/o any sign of problems -- so i'm feeling
very satisfied.

Proceed at your lesuire.

-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5897) performance bug (adversary) in StandardTokenizer

2014-08-20 Thread Steve Rowe (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104287#comment-14104287
 ] 

Steve Rowe commented on LUCENE-5897:


I'm looking into it, I think it's a bug in JFlex.  {{zzRefill()}}, which pulls 
data into and if necessary expands the buffer over which tokenization occurs, 
is being called repeatedly even though EOF has been reached.

I'm going to see if this reproduces in Lucene 4.9 - I suspect I introduced the 
bug in JFlex 1.6.  If so, the thing to do is likely revert the JFlex 1.5-1.6 
changes (LUCENE-5770), since that hasn't been released yet.

 performance bug (adversary) in StandardTokenizer
 --

 Key: LUCENE-5897
 URL: https://issues.apache.org/jira/browse/LUCENE-5897
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Robert Muir

 There seem to be some conditions (I don't know how rare or what conditions) 
 that cause StandardTokenizer to essentially hang on input: I havent looked 
 hard yet, but as its essentially a DFA I think something wierd might be going 
 on.
 An easy way to reproduce is with 1MB of underscores, it will just hang 
 forever.
 {code}
   public void testWorthyAdversary() throws Exception {
 char buffer[] = new char[1024 * 1024];
 Arrays.fill(buffer, '_');
 int tokenCount = 0;
 Tokenizer ts = new StandardTokenizer();
 ts.setReader(new StringReader(new String(buffer)));
 ts.reset();
 while (ts.incrementToken()) {
   tokenCount++;
 }
 ts.end();
 ts.close();
 assertEquals(0, tokenCount);
   }
 {code} 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5897) performance bug (adversary) in StandardTokenizer

2014-08-20 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104300#comment-14104300
 ] 

Robert Muir commented on LUCENE-5897:
-

I think it might be older? I tried 4.7 branch really quick, and it hung too.

 performance bug (adversary) in StandardTokenizer
 --

 Key: LUCENE-5897
 URL: https://issues.apache.org/jira/browse/LUCENE-5897
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Robert Muir

 There seem to be some conditions (I don't know how rare or what conditions) 
 that cause StandardTokenizer to essentially hang on input: I havent looked 
 hard yet, but as its essentially a DFA I think something wierd might be going 
 on.
 An easy way to reproduce is with 1MB of underscores, it will just hang 
 forever.
 {code}
   public void testWorthyAdversary() throws Exception {
 char buffer[] = new char[1024 * 1024];
 Arrays.fill(buffer, '_');
 int tokenCount = 0;
 Tokenizer ts = new StandardTokenizer();
 ts.setReader(new StringReader(new String(buffer)));
 ts.reset();
 while (ts.incrementToken()) {
   tokenCount++;
 }
 ts.end();
 ts.close();
 assertEquals(0, tokenCount);
   }
 {code} 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5897) performance bug (adversary) in StandardTokenizer

2014-08-20 Thread Steve Rowe (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104299#comment-14104299
 ] 

Steve Rowe commented on LUCENE-5897:


bq. I'm going to see if this reproduces in Lucene 4.9 - I suspect I introduced 
the bug in JFlex 1.6. If so, the thing to do is likely revert the JFlex 
1.5-1.6 changes (LUCENE-5770), since that hasn't been released yet.

Unfortunately, the hanging behavior occurs on 4.9 too, so reverting LUCENE-5770 
won't help.

 performance bug (adversary) in StandardTokenizer
 --

 Key: LUCENE-5897
 URL: https://issues.apache.org/jira/browse/LUCENE-5897
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Robert Muir

 There seem to be some conditions (I don't know how rare or what conditions) 
 that cause StandardTokenizer to essentially hang on input: I havent looked 
 hard yet, but as its essentially a DFA I think something wierd might be going 
 on.
 An easy way to reproduce is with 1MB of underscores, it will just hang 
 forever.
 {code}
   public void testWorthyAdversary() throws Exception {
 char buffer[] = new char[1024 * 1024];
 Arrays.fill(buffer, '_');
 int tokenCount = 0;
 Tokenizer ts = new StandardTokenizer();
 ts.setReader(new StringReader(new String(buffer)));
 ts.reset();
 while (ts.incrementToken()) {
   tokenCount++;
 }
 ts.end();
 ts.close();
 assertEquals(0, tokenCount);
   }
 {code} 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6393) Improve transaction log replay speed on HDFS.


[ 
https://issues.apache.org/jira/browse/SOLR-6393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104324#comment-14104324
 ] 

ASF subversion and git services commented on SOLR-6393:
---

Commit 1619200 from [~markrmil...@gmail.com] in branch 'dev/trunk'
[ https://svn.apache.org/r1619200 ]

SOLR-6393: TransactionLog replay performance on HDFS is very poor.

 Improve transaction log replay speed on HDFS.
 -

 Key: SOLR-6393
 URL: https://issues.apache.org/jira/browse/SOLR-6393
 Project: Solr
  Issue Type: Improvement
Reporter: Mark Miller
Assignee: Mark Miller
 Fix For: 5.0, 4.10

 Attachments: SOLR-6393.patch


 Replay speed is pretty slow on HDFS because we currently reopen a reader 
 between reading each update.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-6396) Allow the name of core.properties file used in discovery to be specified by an environment variable

Erick Erickson created SOLR-6396:


 Summary: Allow the name of core.properties file used in discovery 
to be specified by an environment variable
 Key: SOLR-6396
 URL: https://issues.apache.org/jira/browse/SOLR-6396
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.9, 5.0
Reporter: Erick Erickson
Assignee: Erick Erickson
Priority: Minor


This was brought up on the user's list. I haven't thought this through, but it 
seems reasonable.

This has some interesting implications in the core rename case, i.e. The 
unloaded props file will have the different name as well.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5897) performance bug (adversary) in StandardTokenizer

2014-08-20 Thread Steve Rowe (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104352#comment-14104352
 ] 

Steve Rowe commented on LUCENE-5897:


This looks exactly like LUCENE-5400: very slow tokenization when a rule has to 
search to end of stream for a condition that doesn't occur.

 performance bug (adversary) in StandardTokenizer
 --

 Key: LUCENE-5897
 URL: https://issues.apache.org/jira/browse/LUCENE-5897
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Robert Muir

 There seem to be some conditions (I don't know how rare or what conditions) 
 that cause StandardTokenizer to essentially hang on input: I havent looked 
 hard yet, but as its essentially a DFA I think something wierd might be going 
 on.
 An easy way to reproduce is with 1MB of underscores, it will just hang 
 forever.
 {code}
   public void testWorthyAdversary() throws Exception {
 char buffer[] = new char[1024 * 1024];
 Arrays.fill(buffer, '_');
 int tokenCount = 0;
 Tokenizer ts = new StandardTokenizer();
 ts.setReader(new StringReader(new String(buffer)));
 ts.reset();
 while (ts.incrementToken()) {
   tokenCount++;
 }
 ts.end();
 ts.close();
 assertEquals(0, tokenCount);
   }
 {code} 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6396) Allow the name of core.properties file used in discovery to be specified by an environment variable

2014-08-20 Thread Hoss Man (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-6396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104370#comment-14104370
 ] 

Hoss Man commented on SOLR-6396:


I don't think we want to make the name of {{core.properties}} be variable ... 
that way leads to madness and confusion.

the request on the user list was about being able to dynamically load a 
property file with diff values between dev  production like you could do in 
the old style solr.xml -- that doesn't mean {{core.properties}} needs to have a 
configurable name, it just means there needs to be a configurable way to load 
properties.

we already have a {{properties}} option which can be specified in 
{{core.properties}} to point to an additional external file that should also be 
loaded ... if variable substitution was in play when parsing 
{{core.properties}} then you could have something like 
{{properties=custom.$\{env\}.properties}} in {{core.properties}} ... but 
introducing variable substitution into the {{core.properties}} (which solr both 
reads  writes based on CoreAdmin calls) brings back the host of complexities 
involved when we had persistence of {{solr.xml}} as a feature, with the 
questions about persisting the original values with variables in them, vs the 
values after evaluating variables.

 Allow the name of core.properties file used in discovery to be specified by 
 an environment variable
 ---

 Key: SOLR-6396
 URL: https://issues.apache.org/jira/browse/SOLR-6396
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.9, 5.0
Reporter: Erick Erickson
Assignee: Erick Erickson
Priority: Minor

 This was brought up on the user's list. I haven't thought this through, but 
 it seems reasonable.
 This has some interesting implications in the core rename case, i.e. The 
 unloaded props file will have the different name as well.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5897) performance bug (adversary) in StandardTokenizer


[ 
https://issues.apache.org/jira/browse/LUCENE-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104379#comment-14104379
 ] 

Robert Muir commented on LUCENE-5897:
-

OK, that makes sense. Can we do something crazy during generation of jflex 
(e.g. inject a little code) to bound this to at least maxTokenLength() ?

 performance bug (adversary) in StandardTokenizer
 --

 Key: LUCENE-5897
 URL: https://issues.apache.org/jira/browse/LUCENE-5897
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Robert Muir

 There seem to be some conditions (I don't know how rare or what conditions) 
 that cause StandardTokenizer to essentially hang on input: I havent looked 
 hard yet, but as its essentially a DFA I think something wierd might be going 
 on.
 An easy way to reproduce is with 1MB of underscores, it will just hang 
 forever.
 {code}
   public void testWorthyAdversary() throws Exception {
 char buffer[] = new char[1024 * 1024];
 Arrays.fill(buffer, '_');
 int tokenCount = 0;
 Tokenizer ts = new StandardTokenizer();
 ts.setReader(new StringReader(new String(buffer)));
 ts.reset();
 while (ts.incrementToken()) {
   tokenCount++;
 }
 ts.end();
 ts.close();
 assertEquals(0, tokenCount);
   }
 {code} 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-6397) zkcli script put/putfile should allow overwriting an existing znode's data

Timothy Potter created SOLR-6397:


 Summary: zkcli script put/putfile should allow overwriting an 
existing znode's data
 Key: SOLR-6397
 URL: https://issues.apache.org/jira/browse/SOLR-6397
 Project: Solr
  Issue Type: Bug
Reporter: Timothy Potter


zkcli doesn't allow me to update a znode that already exists, perhaps using a 
-f (force) flag?

Currently, if I want to putfile for a znode that already exists, results in:

Exception in thread main 
org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = 
NodeExists for /clusterstate.json
at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
at 
org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:273)
at 
org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:270)
at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
at 
org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:270)
at org.apache.solr.cloud.ZkCLI.main(ZkCLI.java:268)


The workaround is to use ZooKeeper's command-line set command with `cat 
file`



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-6397) zkcli script put/putfile should allow overwriting an existing znode's data

2014-08-20 Thread ASF subversion and git services (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-6397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Timothy Potter updated SOLR-6397:
-

Component/s: SolrCloud

 zkcli script put/putfile should allow overwriting an existing znode's data
 --

 Key: SOLR-6397
 URL: https://issues.apache.org/jira/browse/SOLR-6397
 Project: Solr
  Issue Type: Bug
  Components: SolrCloud
Reporter: Timothy Potter

 zkcli doesn't allow me to update a znode that already exists, perhaps using a 
 -f (force) flag?
 Currently, if I want to putfile for a znode that already exists, results in:
 Exception in thread main 
 org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = 
 NodeExists for /clusterstate.json
   at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
   at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
   at 
 org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:273)
   at 
 org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:270)
   at 
 org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   at 
 org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:270)
   at org.apache.solr.cloud.ZkCLI.main(ZkCLI.java:268)
 The workaround is to use ZooKeeper's command-line set command with `cat 
 file`



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-1485) PayloadTermQuery support

2014-08-20 Thread Erik Hatcher (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-1485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104405#comment-14104405
 ] 

Erik Hatcher commented on SOLR-1485:


Is there a reason not to use SchemaSimilarityFactory as the default Similarity 
moving forward?   Relying on that would be nice, it seems.

 PayloadTermQuery support
 

 Key: SOLR-1485
 URL: https://issues.apache.org/jira/browse/SOLR-1485
 Project: Solr
  Issue Type: New Feature
  Components: search
Reporter: Erik Hatcher
Assignee: Erik Hatcher
 Fix For: 5.0

 Attachments: PayloadTermQueryPlugin.java


 Solr currently has no support for Lucene's PayloadTermQuery, yet it has 
 support for indexing payloads. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5896) A view potential reproducible issues


[ 
https://issues.apache.org/jira/browse/LUCENE-5896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104410#comment-14104410
 ] 

ASF subversion and git services commented on LUCENE-5896:
-

Commit 1619211 from [~simonw] in branch 'dev/trunk'
[ https://svn.apache.org/r1619211 ]

LUCENE-5896: Use local random instance rather than global in LuceneTestCase

 A view potential reproducible issues
 

 Key: LUCENE-5896
 URL: https://issues.apache.org/jira/browse/LUCENE-5896
 Project: Lucene - Core
  Issue Type: Test
  Components: general/test
Affects Versions: 4.9
Reporter: Simon Willnauer
 Fix For: 5.0, 4.10

 Attachments: LUCENE-5896.patch


 I realized that passing the same seeded random instance to LuceneTestCase# 
 newIndewWriterConfig doesn't necessarily produce the same IWC and I found a 
 bunch of issues in that class using global random rather than local random. 
 Yet, I went over the file to spot others but we might need to think about a 
 more automated way to spot those...



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5896) A few potential reproducible issues

2014-08-20 Thread ASF subversion and git services (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-5896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Simon Willnauer updated LUCENE-5896:


Summary: A few potential reproducible issues  (was: A view potential 
reproducible issues)

 A few potential reproducible issues
 ---

 Key: LUCENE-5896
 URL: https://issues.apache.org/jira/browse/LUCENE-5896
 Project: Lucene - Core
  Issue Type: Test
  Components: general/test
Affects Versions: 4.9
Reporter: Simon Willnauer
 Fix For: 5.0, 4.10

 Attachments: LUCENE-5896.patch


 I realized that passing the same seeded random instance to LuceneTestCase# 
 newIndewWriterConfig doesn't necessarily produce the same IWC and I found a 
 bunch of issues in that class using global random rather than local random. 
 Yet, I went over the file to spot others but we might need to think about a 
 more automated way to spot those...



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5896) A few potential reproducible issues


[ 
https://issues.apache.org/jira/browse/LUCENE-5896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104423#comment-14104423
 ] 

ASF subversion and git services commented on LUCENE-5896:
-

Commit 1619213 from [~simonw] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1619213 ]

LUCENE-5896: Use local random instance rather than global in LuceneTestCase

 A few potential reproducible issues
 ---

 Key: LUCENE-5896
 URL: https://issues.apache.org/jira/browse/LUCENE-5896
 Project: Lucene - Core
  Issue Type: Test
  Components: general/test
Affects Versions: 4.9
Reporter: Simon Willnauer
 Fix For: 5.0, 4.10

 Attachments: LUCENE-5896.patch


 I realized that passing the same seeded random instance to LuceneTestCase# 
 newIndewWriterConfig doesn't necessarily produce the same IWC and I found a 
 bunch of issues in that class using global random rather than local random. 
 Yet, I went over the file to spot others but we might need to think about a 
 more automated way to spot those...



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5896) A few potential reproducible issues

2014-08-20 Thread Dawid Weiss (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104428#comment-14104428
 ] 

Dawid Weiss commented on LUCENE-5896:
-

Every random instance in the test framework is already assigned per-thread; 
there are no races there.

I think *not* using the randomized context's random is actually a mistake (the 
only exception being tight loops where performance is the key). Once you fork a 
custom random it will escape. If you consistently use the framework's Random 
then you should always be able to reproduce the same run with tests.seed.

Unless I'm missing something.

 A few potential reproducible issues
 ---

 Key: LUCENE-5896
 URL: https://issues.apache.org/jira/browse/LUCENE-5896
 Project: Lucene - Core
  Issue Type: Test
  Components: general/test
Affects Versions: 4.9
Reporter: Simon Willnauer
 Fix For: 5.0, 4.10

 Attachments: LUCENE-5896.patch


 I realized that passing the same seeded random instance to LuceneTestCase# 
 newIndewWriterConfig doesn't necessarily produce the same IWC and I found a 
 bunch of issues in that class using global random rather than local random. 
 Yet, I went over the file to spot others but we might need to think about a 
 more automated way to spot those...



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5896) A few potential reproducible issues

2014-08-20 Thread Dawid Weiss (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104434#comment-14104434
 ] 

Dawid Weiss commented on LUCENE-5896:
-

So, to be clear -- I think this is the problem:
{code}
public static IndexWriterConfig newIndexWriterConfig(Random r, Analyzer a) {
{code}
It should just use the randomized context's randomness consistently instead of 
accepting an external Random. It would be consistent (and is not leading to any 
thread related randomization races).

 A few potential reproducible issues
 ---

 Key: LUCENE-5896
 URL: https://issues.apache.org/jira/browse/LUCENE-5896
 Project: Lucene - Core
  Issue Type: Test
  Components: general/test
Affects Versions: 4.9
Reporter: Simon Willnauer
 Fix For: 5.0, 4.10

 Attachments: LUCENE-5896.patch


 I realized that passing the same seeded random instance to LuceneTestCase# 
 newIndewWriterConfig doesn't necessarily produce the same IWC and I found a 
 bunch of issues in that class using global random rather than local random. 
 Yet, I went over the file to spot others but we might need to think about a 
 more automated way to spot those...



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5896) A few potential reproducible issues


[ 
https://issues.apache.org/jira/browse/LUCENE-5896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104441#comment-14104441
 ] 

Robert Muir commented on LUCENE-5896:
-

Dawid: well this is exactly what i mean. We got ourselves into the situation 
long before that, when there was just a huge pile of code in 
LuceneTestCase.java, and I think a lot of that stuff is just hanging around, we 
should try to think of a way to do it better...

 A few potential reproducible issues
 ---

 Key: LUCENE-5896
 URL: https://issues.apache.org/jira/browse/LUCENE-5896
 Project: Lucene - Core
  Issue Type: Test
  Components: general/test
Affects Versions: 4.9
Reporter: Simon Willnauer
 Fix For: 5.0, 4.10

 Attachments: LUCENE-5896.patch


 I realized that passing the same seeded random instance to LuceneTestCase# 
 newIndewWriterConfig doesn't necessarily produce the same IWC and I found a 
 bunch of issues in that class using global random rather than local random. 
 Yet, I went over the file to spot others but we might need to think about a 
 more automated way to spot those...



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5896) A few potential reproducibility issues

2014-08-20 Thread ASF subversion and git services (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-5896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Simon Willnauer updated LUCENE-5896:


Summary: A few potential reproducibility issues  (was: A few potential 
reproducible issues)

 A few potential reproducibility issues
 --

 Key: LUCENE-5896
 URL: https://issues.apache.org/jira/browse/LUCENE-5896
 Project: Lucene - Core
  Issue Type: Test
  Components: general/test
Affects Versions: 4.9
Reporter: Simon Willnauer
 Fix For: 5.0, 4.10

 Attachments: LUCENE-5896.patch


 I realized that passing the same seeded random instance to LuceneTestCase# 
 newIndewWriterConfig doesn't necessarily produce the same IWC and I found a 
 bunch of issues in that class using global random rather than local random. 
 Yet, I went over the file to spot others but we might need to think about a 
 more automated way to spot those...



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-SmokeRelease-4.x - Build # 185 - Still Failing

2014-08-20 Thread Apache Jenkins Server

Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-4.x/185/

No tests ran.

Build Log:
[...truncated 51542 lines...]
prepare-release-no-sign:
[mkdir] Created dir: 
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-4.x/lucene/build/fakeRelease
 [copy] Copying 431 files to 
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-4.x/lucene/build/fakeRelease/lucene
 [copy] Copying 239 files to 
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-4.x/lucene/build/fakeRelease/solr
 [exec] JAVA7_HOME is /home/jenkins/tools/java/latest1.7
 [exec] NOTE: output encoding is US-ASCII
 [exec] 
 [exec] Load release URL 
file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-4.x/lucene/build/fakeRelease/...
 [exec] 
 [exec] Test Lucene...
 [exec]   test basics...
 [exec]   get KEYS
 [exec] 0.1 MB in 0.01 sec (11.4 MB/sec)
 [exec]   check changes HTML...
 [exec]   download lucene-4.10.0-src.tgz...
 [exec] 27.7 MB in 0.04 sec (675.5 MB/sec)
 [exec] verify md5/sha1 digests
 [exec]   download lucene-4.10.0.tgz...
 [exec] 61.9 MB in 0.09 sec (689.6 MB/sec)
 [exec] verify md5/sha1 digests
 [exec]   download lucene-4.10.0.zip...
 [exec] 71.7 MB in 0.08 sec (935.2 MB/sec)
 [exec] verify md5/sha1 digests
 [exec]   unpack lucene-4.10.0.tgz...
 [exec] verify JAR metadata/identity/no javax.* or java.* classes...
 [exec] test demo with 1.7...
 [exec]   got 5793 hits for query lucene
 [exec] checkindex with 1.7...
 [exec] check Lucene's javadoc JAR
 [exec]   unpack lucene-4.10.0.zip...
 [exec] verify JAR metadata/identity/no javax.* or java.* classes...
 [exec] test demo with 1.7...
 [exec]   got 5793 hits for query lucene
 [exec] checkindex with 1.7...
 [exec] check Lucene's javadoc JAR
 [exec]   unpack lucene-4.10.0-src.tgz...
 [exec] make sure no JARs/WARs in src dist...
 [exec] run ant validate
 [exec] run tests w/ Java 7 and testArgs='-Dtests.jettyConnector=Socket 
 -Dtests.disableHdfs=true'...
 [exec] test demo with 1.7...
 [exec]   got 260 hits for query lucene
 [exec] checkindex with 1.7...
 [exec] generate javadocs w/ Java 7...
 [exec] 
 [exec] Crawl/parse...
 [exec] 
 [exec] Verify...
 [exec] 
 [exec] Test Solr...
 [exec]   test basics...
 [exec]   get KEYS
 [exec] 0.1 MB in 0.01 sec (14.0 MB/sec)
 [exec]   check changes HTML...
 [exec]   download solr-4.10.0-src.tgz...
 [exec] 33.7 MB in 0.13 sec (254.4 MB/sec)
 [exec] verify md5/sha1 digests
 [exec]   download solr-4.10.0.tgz...
 [exec] 146.7 MB in 0.64 sec (228.7 MB/sec)
 [exec] verify md5/sha1 digests
 [exec]   download solr-4.10.0.zip...
 [exec] 152.4 MB in 0.57 sec (265.8 MB/sec)
 [exec] verify md5/sha1 digests
 [exec]   unpack solr-4.10.0.tgz...
 [exec] verify JAR metadata/identity/no javax.* or java.* classes...
 [exec] unpack lucene-4.10.0.tgz...
 [exec]   **WARNING**: skipping check of 
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-4.x/lucene/build/fakeReleaseTmp/unpack/solr-4.10.0/contrib/dataimporthandler-extras/lib/javax.mail-1.5.1.jar:
 it has javax.* classes
 [exec]   **WARNING**: skipping check of 
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-4.x/lucene/build/fakeReleaseTmp/unpack/solr-4.10.0/contrib/dataimporthandler-extras/lib/activation-1.1.1.jar:
 it has javax.* classes
 [exec] verify WAR metadata/contained JAR identity/no javax.* or java.* 
classes...
 [exec] unpack lucene-4.10.0.tgz...
 [exec] copying unpacked distribution for Java 7 ...
 [exec] test solr example w/ Java 7...
 [exec]   start Solr instance 
(log=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-4.x/lucene/build/fakeReleaseTmp/unpack/solr-4.10.0-java7/solr-example.log)...
 [exec]   startup done
 [exec]   test utf8...
 [exec]   index example docs...
 [exec]   run query...
 [exec]   stop server (SIGINT)...
 [exec]   unpack solr-4.10.0.zip...
 [exec] verify JAR metadata/identity/no javax.* or java.* classes...
 [exec] unpack lucene-4.10.0.tgz...
 [exec]   **WARNING**: skipping check of 
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-4.x/lucene/build/fakeReleaseTmp/unpack/solr-4.10.0/contrib/dataimporthandler-extras/lib/activation-1.1.1.jar:
 it has javax.* classes
 [exec]   **WARNING**: skipping check of 
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-4.x/lucene/build/fakeReleaseTmp/unpack/solr-4.10.0/contrib/dataimporthandler-extras/lib/javax.mail-1.5.1.jar:
 it has javax.* classes
 [exec] verify WAR

[jira] [Commented] (SOLR-6393) Improve transaction log replay speed on HDFS.


[ 
https://issues.apache.org/jira/browse/SOLR-6393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104456#comment-14104456
 ] 

ASF subversion and git services commented on SOLR-6393:
---

Commit 1619218 from [~markrmil...@gmail.com] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1619218 ]

SOLR-6393: TransactionLog replay performance on HDFS is very poor.

 Improve transaction log replay speed on HDFS.
 -

 Key: SOLR-6393
 URL: https://issues.apache.org/jira/browse/SOLR-6393
 Project: Solr
  Issue Type: Improvement
Reporter: Mark Miller
Assignee: Mark Miller
 Fix For: 5.0, 4.10

 Attachments: SOLR-6393.patch


 Replay speed is pretty slow on HDFS because we currently reopen a reader 
 between reading each update.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5896) A few potential reproducibility issues

2014-08-20 Thread Dawid Weiss (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104452#comment-14104452
 ] 

Dawid Weiss commented on LUCENE-5896:
-

Ok, clear. I would really encourage removing Random as a parameter in favor of 
grabbing a random() locally inside a method that needs it (or use adequate 
methods from the superclass). When there are loops that do millions of 
iterations and performance becomes an issue, fork off a local variable with
{code}
Random local = new Random(randomLong());
...
{code}

Ideally, it shouldn't escape outside of the method scope.

 A few potential reproducibility issues
 --

 Key: LUCENE-5896
 URL: https://issues.apache.org/jira/browse/LUCENE-5896
 Project: Lucene - Core
  Issue Type: Test
  Components: general/test
Affects Versions: 4.9
Reporter: Simon Willnauer
 Fix For: 5.0, 4.10

 Attachments: LUCENE-5896.patch


 I realized that passing the same seeded random instance to LuceneTestCase# 
 newIndewWriterConfig doesn't necessarily produce the same IWC and I found a 
 bunch of issues in that class using global random rather than local random. 
 Yet, I went over the file to spot others but we might need to think about a 
 more automated way to spot those...



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6397) zkcli script put/putfile should allow overwriting an existing znode's data


[ 
https://issues.apache.org/jira/browse/SOLR-6397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104463#comment-14104463
 ] 

Mark Miller commented on SOLR-6397:
---

Yeah, I think it's just an oversight. I'd probably go without requiring -f 
myself.

 zkcli script put/putfile should allow overwriting an existing znode's data
 --

 Key: SOLR-6397
 URL: https://issues.apache.org/jira/browse/SOLR-6397
 Project: Solr
  Issue Type: Bug
  Components: SolrCloud
Reporter: Timothy Potter

 zkcli doesn't allow me to update a znode that already exists, perhaps using a 
 -f (force) flag?
 Currently, if I want to putfile for a znode that already exists, results in:
 Exception in thread main 
 org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = 
 NodeExists for /clusterstate.json
   at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
   at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
   at 
 org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:273)
   at 
 org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:270)
   at 
 org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   at 
 org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:270)
   at org.apache.solr.cloud.ZkCLI.main(ZkCLI.java:268)
 The workaround is to use ZooKeeper's command-line set command with `cat 
 file`



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-6396) Allow the name of core.properties file used in discovery to be specified by an environment variable


 [ 
https://issues.apache.org/jira/browse/SOLR-6396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Erick Erickson resolved SOLR-6396.
--

Resolution: Won't Fix

OK, now that Chris has thought it through I'll can the thought.

Oh my, thanks Hossman for preventing me from opening that persistence can of 
worms again.

 Allow the name of core.properties file used in discovery to be specified by 
 an environment variable
 ---

 Key: SOLR-6396
 URL: https://issues.apache.org/jira/browse/SOLR-6396
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.9, 5.0
Reporter: Erick Erickson
Assignee: Erick Erickson
Priority: Minor

 This was brought up on the user's list. I haven't thought this through, but 
 it seems reasonable.
 This has some interesting implications in the core rename case, i.e. The 
 unloaded props file will have the different name as well.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6393) Improve transaction log replay speed on HDFS.


[ 
https://issues.apache.org/jira/browse/SOLR-6393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104475#comment-14104475
 ] 

Mark Miller commented on SOLR-6393:
---

bq. Ah, missed that. Thanks!

The straight read speed should be really fast now, but when we are updating 
while reading, when we get to the end of the file we are reading, we have to 
try and open it again to see if there is anything new. The local fs impl uses 
channels and doesn't have to do this to see the latest data from the writer. So 
even when updates are also coming in, this should be a huge improvement, 
because it was previously reopening needlessly on every update it read, but we 
do still have to take that hit of opening a new reader every time we read up to 
the end of the view of the last reader.

 Improve transaction log replay speed on HDFS.
 -

 Key: SOLR-6393
 URL: https://issues.apache.org/jira/browse/SOLR-6393
 Project: Solr
  Issue Type: Improvement
Reporter: Mark Miller
Assignee: Mark Miller
 Fix For: 5.0, 4.10

 Attachments: SOLR-6393.patch


 Replay speed is pretty slow on HDFS because we currently reopen a reader 
 between reading each update.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-6393) Improve transaction log replay speed on HDFS.