date:20130129

Do you have your latest work attached to the issue? If so, I'll start working 
with it locally.

For now, can you try this experimental, test patch and see what the results are?

Index: solr/core/src/java/org/apache/solr/update/DefaultSolrCoreState.java
===
--- solr/core/src/java/org/apache/solr/update/DefaultSolrCoreState.java 
(revision 1440275)
+++ solr/core/src/java/org/apache/solr/update/DefaultSolrCoreState.java 
(working copy)
@@ -135,13 +135,24 @@
   pauseWriter = true;
   // then lets wait until its out of use
   log.info("Waiting until IndexWriter is unused... core=" + coreName);
+  
+  boolean yielded = false;
+  try {
   while (!writerFree) {
-try {
-  writerPauseLock.wait(100);
-} catch (InterruptedException e) {}
-
-if (closed) {
-  throw new RuntimeException("SolrCoreState already closed");
+  // yield the commit lock
+  core.getUpdateHandler().yieldCommitLock();
+  yielded = true;
+  try {
+writerPauseLock.wait(100);
+  } catch (InterruptedException e) {}
+  
+  if (closed) {
+throw new RuntimeException("SolrCoreState already closed");
+  }
+}
+  } finally {
+if (yielded) {
+  core.getUpdateHandler().getCommitLock();
 }
   }
 
Index: solr/core/src/java/org/apache/solr/update/UpdateHandler.java
===
--- solr/core/src/java/org/apache/solr/update/UpdateHandler.java
(revision 1440275)
+++ solr/core/src/java/org/apache/solr/update/UpdateHandler.java
(working copy)
@@ -189,4 +189,10 @@
   }
 
   public abstract void split(SplitIndexCommand cmd) throws IOException;
+
+
+  public void getCommitLock() {}
+
+
+  public void yieldCommitLock() {}
 }
Index: solr/core/src/java/org/apache/solr/update/DirectUpdateHandler2.java
===
--- solr/core/src/java/org/apache/solr/update/DirectUpdateHandler2.java 
(revision 1440275)
+++ solr/core/src/java/org/apache/solr/update/DirectUpdateHandler2.java 
(working copy)
@@ -830,4 +830,13 @@
   public CommitTracker getSoftCommitTracker() {
 return softCommitTracker;
   }
+  
+  public void getCommitLock() {
+commitLock.lock();
+  }
+
+
+  public void yieldCommitLock() {
+commitLock.unlock();
+  }
 }


On Jan 29, 2013, at 11:24 PM, Mark Miller  wrote:

> Digging into the stack traces...
> 
> This shows a thread waiting for the commit lock trying to close an index 
> writer.
> 
> There is another thread with the commit lock that is waiting for the writer 
> to be returned.
> 
> That seems to be the situation - a race around the commit lock.
> 
> Needs some thought.
> 
> - Mark
> 
> On Jan 29, 2013, at 8:31 AM, Erick Erickson  wrote:
> 
>> Java stack information for the threads listed above:
>> ===
>> "commitScheduler-42617-thread-1":
>> at 
>> org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:78)
>> - waiting to lock <78b4aa518> (a org.apache.solr.update.DefaultSolrCoreState)
>> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1359)
>> at 
>> org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:561)
>> - locked <7884ca730> (a java.lang.Object)
>> at org.apache.solr.update.CommitTracker.run(CommitTracker.java:216)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>> at 
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
>> at 
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
>> at 
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> at 
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> at java.lang.Thread.run(Thread.java:680)
>> 
>> *
>> Other thread
>> "qtp1401888126-32":
>> at sun.misc.Unsafe.park(Native Method)
>> - parking to wait for  <788d73208> (a
>> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
>> at 
>> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
>> at 
>> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:842)
>> at 
>> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1178)
>> at 
>> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:186)
>> at java.util.concurrent.locks.ReentrantLock.lock(Ree

Re: Stress test deadlocks

Digging into the stack traces...

This shows a thread waiting for the commit lock trying to close an index writer.

There is another thread with the commit lock that is waiting for the writer to 
be returned.

That seems to be the situation - a race around the commit lock.

Needs some thought.

- Mark

On Jan 29, 2013, at 8:31 AM, Erick Erickson  wrote:

> Java stack information for the threads listed above:
> ===
> "commitScheduler-42617-thread-1":
> at 
> org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:78)
> - waiting to lock <78b4aa518> (a org.apache.solr.update.DefaultSolrCoreState)
> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1359)
> at 
> org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:561)
> - locked <7884ca730> (a java.lang.Object)
> at org.apache.solr.update.CommitTracker.run(CommitTracker.java:216)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> 
> *
> Other thread
> "qtp1401888126-32":
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for  <788d73208> (a
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:842)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1178)
> at 
> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:186)
> at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262)
> at 
> org.apache.solr.update.DirectUpdateHandler2.closeWriter(DirectUpdateHandler2.java:668)
> at 
> org.apache.solr.update.DefaultSolrCoreState.closeIndexWriter(DefaultSolrCoreState.java:64)
> - locked <78b4aa518> (a org.apache.solr.update.DefaultSolrCoreState)
> at 
> org.apache.solr.update.DefaultSolrCoreState.close(DefaultSolrCoreState.java:272)
> - locked <78b4aa518> (a org.apache.solr.update.DefaultSolrCoreState)
> at org.apache.solr.core.SolrCore.decrefSolrCoreState(SolrCore.java:888)
> - locked <78b4aa518> (a org.apache.solr.update.DefaultSolrCoreState)
> at org.apache.solr.core.SolrCore.close(SolrCore.java:980)
> at org.apache.solr.core.CoreMaps.putTransientCore(CoreContainer.java:1465)
> at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:730)
> at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1137)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:190)
> at


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3904) add package level javadocs to every package


[ 
https://issues.apache.org/jira/browse/SOLR-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13566139#comment-13566139
 ] 

Commit Tag Bot commented on SOLR-3904:
--

[trunk commit] Chris M. Hostetter
http://svn.apache.org/viewvc?view=revision&revision=1440263

SOLR-3904: package.html for all packages in solr-core


> add package level javadocs to every package
> ---
>
> Key: SOLR-3904
> URL: https://issues.apache.org/jira/browse/SOLR-3904
> Project: Solr
>  Issue Type: Improvement
>  Components: documentation
>Reporter: Hoss Man
>Assignee: Hoss Man
> Fix For: 4.2, 5.0
>
> Attachments: SOLR-3904_buildxml.patch
>
>
> quoth rmuir on the mailing list...
> {quote}
> We've been working on this for the lucene side (3.6 was the first
> release where every package had docs, 4.0 will be the first where
> every class had docs, and we are now working towards
> methods/fields/ctors/enums).
> I think this would be valuable for solr too (especially solrj as a start).
> Besides users, its really useful to developers as well. Of course we
> all think our code is self-documenting, but its not always the case. a
> few extra seconds can save someone a ton of time trying to figure out
> your code.
> Additionally at least in my IDE, when things are done as javadoc
> comments then they are more easily accessible than code comments. I'm
> sure its the case for some other development environments too.
> Filling in these package.html's to at least have a one sentence
> description would be a really good start. It lets someone know where
> to go at the high-level.
> If I was brand new to solr and wanted to write a java app that uses
> solrj, i wouldn't have a clue where to start
> (https://builds.apache.org/job/Solr-Artifacts-4.x/javadoc/solr-solrj/index.html).
> 12 sentences could go a really long way.
> And for all new code, I hope we can all try harder for more complete
> javadocs. when you are working on something and its fresh in your head
> its a lot easier to do this than for someone else to come back around
> and figure it out.
> {quote}
> I'm going to try and make it a priority for me to fill in package level docs 
> as we look towards 4.1

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3904) add package level javadocs to every package


[ 
https://issues.apache.org/jira/browse/SOLR-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13566138#comment-13566138
 ] 

Commit Tag Bot commented on SOLR-3904:
--

[branch_4x commit] Chris M. Hostetter
http://svn.apache.org/viewvc?view=revision&revision=1440265

SOLR-3904: package.html for all packages in solr-core (merge r1440263)


> add package level javadocs to every package
> ---
>
> Key: SOLR-3904
> URL: https://issues.apache.org/jira/browse/SOLR-3904
> Project: Solr
>  Issue Type: Improvement
>  Components: documentation
>Reporter: Hoss Man
>Assignee: Hoss Man
> Fix For: 4.2, 5.0
>
> Attachments: SOLR-3904_buildxml.patch
>
>
> quoth rmuir on the mailing list...
> {quote}
> We've been working on this for the lucene side (3.6 was the first
> release where every package had docs, 4.0 will be the first where
> every class had docs, and we are now working towards
> methods/fields/ctors/enums).
> I think this would be valuable for solr too (especially solrj as a start).
> Besides users, its really useful to developers as well. Of course we
> all think our code is self-documenting, but its not always the case. a
> few extra seconds can save someone a ton of time trying to figure out
> your code.
> Additionally at least in my IDE, when things are done as javadoc
> comments then they are more easily accessible than code comments. I'm
> sure its the case for some other development environments too.
> Filling in these package.html's to at least have a one sentence
> description would be a really good start. It lets someone know where
> to go at the high-level.
> If I was brand new to solr and wanted to write a java app that uses
> solrj, i wouldn't have a clue where to start
> (https://builds.apache.org/job/Solr-Artifacts-4.x/javadoc/solr-solrj/index.html).
> 12 sentences could go a really long way.
> And for all new code, I hope we can all try harder for more complete
> javadocs. when you are working on something and its fresh in your head
> its a lot easier to do this than for someone else to come back around
> and figure it out.
> {quote}
> I'm going to try and make it a priority for me to fill in package level docs 
> as we look towards 4.1

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-3904) add package level javadocs to every package

2013-01-29 Thread Hoss Man (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man resolved SOLR-3904.


   Resolution: Fixed
Fix Version/s: 5.0

I finally got around finishing this...


Committed revision 1440263.
Committed revision 1440265.


> add package level javadocs to every package
> ---
>
> Key: SOLR-3904
> URL: https://issues.apache.org/jira/browse/SOLR-3904
> Project: Solr
>  Issue Type: Improvement
>  Components: documentation
>Reporter: Hoss Man
>Assignee: Hoss Man
> Fix For: 4.2, 5.0
>
> Attachments: SOLR-3904_buildxml.patch
>
>
> quoth rmuir on the mailing list...
> {quote}
> We've been working on this for the lucene side (3.6 was the first
> release where every package had docs, 4.0 will be the first where
> every class had docs, and we are now working towards
> methods/fields/ctors/enums).
> I think this would be valuable for solr too (especially solrj as a start).
> Besides users, its really useful to developers as well. Of course we
> all think our code is self-documenting, but its not always the case. a
> few extra seconds can save someone a ton of time trying to figure out
> your code.
> Additionally at least in my IDE, when things are done as javadoc
> comments then they are more easily accessible than code comments. I'm
> sure its the case for some other development environments too.
> Filling in these package.html's to at least have a one sentence
> description would be a really good start. It lets someone know where
> to go at the high-level.
> If I was brand new to solr and wanted to write a java app that uses
> solrj, i wouldn't have a clue where to start
> (https://builds.apache.org/job/Solr-Artifacts-4.x/javadoc/solr-solrj/index.html).
> 12 sentences could go a really long way.
> And for all new code, I hope we can all try harder for more complete
> javadocs. when you are working on something and its fresh in your head
> its a lot easier to do this than for someone else to come back around
> and figure it out.
> {quote}
> I'm going to try and make it a priority for me to fill in package level docs 
> as we look towards 4.1

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Stress test deadlocks

On Jan 29, 2013, at 10:26 PM, Erick Erickson  wrote:

> I'm not entirely sure what it would show though, since you don't get
> into this situation without loading and unloading cores at the same
> time, the stuff from SOLR-1028 and SOLR-4300. Which you couldn't do
> before this rapidly.

I don't follow this at all - of course you could rapidly load and unload cores 
at the same time before this patch?

If you cannot easily produce a test that causes deadlock without your patch, 
I'm less inclined to believe it's an existing issue.

Like I said, that synchronization is complicated - it's easy to mess it up if 
you don't follow how it all works.

- Mark
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Stress test deadlocks

On Jan 29, 2013, at 10:26 PM, Erick Erickson  wrote:

> Why synchronize the entire method and then synchronize everything
> except the error condition on a _different_ object?

Because they are different locks protecting different state. Def cannot be 
safely removed.

- Mark
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-trunk-Linux-Java6-64-test-only - Build # 22231 - Failure!

2013-01-29 Thread builder

Build: builds.flonkings.com/job/Lucene-trunk-Linux-Java6-64-test-only/22231/

1 tests failed.
REGRESSION:  org.apache.lucene.index.TestPayloadsOnVectors.testRandomVectors

Error Message:
first position increment must be > 0 (got 0)

Stack Trace:
java.lang.IllegalArgumentException: first position increment must be > 0 (got 0)
at 
__randomizedtesting.SeedInfo.seed([20A09D32E6B2AA4D:F25D1A29B6391B56]:0)
at 
org.apache.lucene.index.DocInverterPerField.processFields(DocInverterPerField.java:125)
at 
org.apache.lucene.index.DocFieldProcessor.processDocument(DocFieldProcessor.java:272)
at 
org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:250)
at 
org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:376)
at 
org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1482)
at 
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1157)
at 
org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:201)
at 
org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:160)
at 
org.apache.lucene.index.TestPayloadsOnVectors.testRandomVectors(TestPayloadsOnVectors.java:371)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
co

Re: Stress test deadlocks

2013-01-29 Thread Erick Erickson

First, I approach synchronization with fear and loathing because it's
so easy to screw up, so any comments deeply appreciated.

Second, the code (see below) in DefaultSolrCoreState just looks wrong.
Why synchronize the entire method and then synchronize everything
except the error condition on a _different_ object? The same thing
happens a bit further down in the newIndexWriter method and yet again
potentially in, I think, every synchronized method. That seems like
classic deadlock-vulnerable code. I really think we've been getting
away with this because the incidence of simultaneously opening and
closing cores is very low, or was before the "many cores" madness. In
fact, I'm not sure _any_ of the synchronized methods in
DefaultSolrCoreState should be synchronized.

Odd that I had that exact thought about running this against the
unmodified code an hour ago on the way back from some family business.
I'm not entirely sure what it would show though, since you don't get
into this situation without loading and unloading cores at the same
time, the stuff from SOLR-1028 and SOLR-4300. Which you couldn't do
before this rapidly. The deadlock is definitely happening when closing
a core, which generally wouldn't happen the way it's happening with
the stress test before the other changes. I can say that the deadlock
doesn't involve any of the new synchronization SOLR-4196 has
introduced, but I think is a consequence of rapidly opening/closing
cores. Now that I look more closely, the stack trace for the
deadlocked cores are _both_ in DefaultSolrCoreState. Smoking gun?

re: reviewing the patch. It spun out of control as I tried to get all
of the solr.xml purged. It's huge. For the main changes to things like
CoreContainer I don't think looking at the differences will help,
we'll have to evaluate it as new code. I'd be happy to get on a
marathon remote-review session with anyone who wants to rip things
apart. All tests pass at this point, but I'm not sure how much
that really means in terms of deadlocking And I'm particularly
uncertain of the violence I did to persisting

Final comment. With the one synchronization on the one method removed,
I've had about 11 hours if my stress test run without problems. So
this is in the right area at least. I'm about to remove the other
synchronized statement on the methods and run it overnight, will look
at the logs and report in the morning.

Here's the code pattern I just think is wrong (pseudo). It's repeated
several times...

 public synchronized RefCounted getIndexWriter(SolrCore core)
  throws IOException {

if (closed) {
  throw new RuntimeException("SolrCoreState already closed");
}

synchronized (writerPauseLock) {
 all the rest of the method
}
}

On Tue, Jan 29, 2013 at 1:32 PM, Mark Miller  wrote:
> There is a lot of complicated interplay between locks in that area of the 
> code - small changes can easily get you into trouble.
>
> Can you modify your test to run on the code before your patch? That would 
> help in telling if it's something existing or something your introducing.
>
> I still plan on looking at this issue closer this week, so perhaps I can 
> offer some more help soon.
>
> - Mark
>
> On Jan 29, 2013, at 1:26 PM, Erick Erickson  wrote:
>
>> Two runs doth not a conclusion reach, but removing "synchronized" from:
>> DefaultSolrCoreState.getIndexWriter (line 78)
>>
>> let me run for an hour, at least twice. And my stress test succeeds,
>> which fires up 15 indexing threads on 100 cores (transient core size
>> is 20), indexes documents for an hour while another 15 threads fire
>> off queries. At the end, it inspects each core to see if there are the
>> expected number of documents.
>>
>> But that's kinda a frail reed to pin my hopes on, these are
>> notoriously hard to reproduce.
>>
>> I'll set this up to run on an old machine for much longer later today.
>> Does anyone who knows that code know whether I'm playing with fire? I
>> haven't looked at the synchronization in that code to try to figure
>> out the purpose, I'm hoping someone will glance at it and say "that's
>> wrong".
>>
>> I'll dig into it later and see how much I can figure out about whether it's 
>> safe
>>
>> FWIW,
>>
>> On Tue, Jan 29, 2013 at 8:31 AM, Erick Erickson  
>> wrote:
>>> All:
>>>
>>> As part of SOLR-4196, I'm opening and closing cores at a furious rate.
>>> My tests are running for 20-40 minutes then locking up quite
>>> regularly. Of course the first place I'm looking is my recent code,
>>> since it has a bunch of synchronized blocks.
>>>
>>> The deadlock is definitely happening at a call from the new code to
>>> close a Solr core, so to really look at this anyone will need to get
>>> the patch I'll put up in a minute. The deadlock trace is below.
>>>
>>> But without going that far, I question whether it's really anything to
>>> do with new synchronizations I'm doing or whether it's just something
>>> that's been lurking for a while and I'm

[jira] [Commented] (SOLR-4383) DataImportHandler: Semantic inconsistency of column/name attribute


[ 
https://issues.apache.org/jira/browse/SOLR-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13566097#comment-13566097
 ] 

Alexandre Rafalovitch commented on SOLR-4383:
-

Also means that the following line stops import for SqlEntityProcessor:
  
And has to be 
  

Because 'column' attribute has to be always present and throws error message if 
missing.

> DataImportHandler: Semantic inconsistency of column/name attribute
> --
>
> Key: SOLR-4383
> URL: https://issues.apache.org/jira/browse/SOLR-4383
> Project: Solr
>  Issue Type: Bug
>  Components: contrib - DataImportHandler, documentation
>Affects Versions: 4.1
>Reporter: Alexandre Rafalovitch
> Fix For: 5.0
>
>
> Different DIH Entity Processor assign different meaning to 'column' 
> attribute. This can cause serious confusion to beginners but can also lead to 
> extremely hard to troubleshoot subtle bugs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-3918) Port index sorter to trunk APIs

2013-01-29 Thread David Smiley (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-3918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13566071#comment-13566071
 ] 

David Smiley commented on LUCENE-3918:
--

Oh, that's interesting.  My proposal doesn't help your early termination query 
use-case.  It does help cache-locality for spatial oriented queries, or queries 
in the vicinity of a time "recent" or whatever.

> Port index sorter to trunk APIs
> ---
>
> Key: LUCENE-3918
> URL: https://issues.apache.org/jira/browse/LUCENE-3918
> Project: Lucene - Core
>  Issue Type: Task
>  Components: modules/other
>Affects Versions: 4.0-ALPHA
>Reporter: Robert Muir
> Fix For: 4.2, 5.0
>
> Attachments: LUCENE-3918.patch
>
>
> LUCENE-2482 added an IndexSorter to 3.x, but we need to port this
> functionality to 4.0 apis.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4383) DataImportHandler: Semantic inconsistency of column/name attribute


[ 
https://issues.apache.org/jira/browse/SOLR-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13566055#comment-13566055
 ] 

Alexandre Rafalovitch commented on SOLR-4383:
-

Specifically:
1) In SqlEntityProcessor: @column is source SQL column, @name is the target 
schema field name
2) In XPathEntityProcessor: @column is the target schema field name, @xpath is 
the source XPath expression
3) In PlainTextEntityProcessor: @column is the source implicit field, @name is 
the target field name (did not test)

In Transformers: @column is target schema field name (I don't think they have 
access to actual source fields)

This causes very hard to figure out bugs like:
DIH:
  
SCHEMA:


The SqlEntityProcessor will pick up 'date' column from database 
(case-insensitive) 
and map it to 'DATE' schema name. 
DateFormatTransformer will then look for field as in @column, finds 'date' and 
tries to transform it. 
But we don't have 'date' field, we have 'DATE' field, so it does not get 
transformed.
The untransformed DATE field then gets submitted to schema and (must be also 
case-insensitive) fails to get parsed causing (nearly silently) skipped entity.



> DataImportHandler: Semantic inconsistency of column/name attribute
> --
>
> Key: SOLR-4383
> URL: https://issues.apache.org/jira/browse/SOLR-4383
> Project: Solr
>  Issue Type: Bug
>  Components: contrib - DataImportHandler, documentation
>Affects Versions: 4.1
>Reporter: Alexandre Rafalovitch
> Fix For: 5.0
>
>
> Different DIH Entity Processor assign different meaning to 'column' 
> attribute. This can cause serious confusion to beginners but can also lead to 
> extremely hard to troubleshoot subtle bugs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-4383) DataImportHandler: Semantic inconsistency of column/name attribute

Alexandre Rafalovitch created SOLR-4383:
---

 Summary: DataImportHandler: Semantic inconsistency of column/name 
attribute
 Key: SOLR-4383
 URL: https://issues.apache.org/jira/browse/SOLR-4383
 Project: Solr
  Issue Type: Bug
  Components: contrib - DataImportHandler, documentation
Affects Versions: 4.1
Reporter: Alexandre Rafalovitch
 Fix For: 5.0


Different DIH Entity Processor assign different meaning to 'column' attribute. 
This can cause serious confusion to beginners but can also lead to extremely 
hard to troubleshoot subtle bugs.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-3456) Analysis Consumers should end and close any TokenStreams


[ 
https://issues.apache.org/jira/browse/LUCENE-3456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13566009#comment-13566009
 ] 

Steve Rowe commented on LUCENE-3456:


Robert, Chris, Uwe: Can this be resolved now?

> Analysis Consumers should end and close any TokenStreams
> 
>
> Key: LUCENE-3456
> URL: https://issues.apache.org/jira/browse/LUCENE-3456
> Project: Lucene - Core
>  Issue Type: Sub-task
>Reporter: Chris Male
> Attachments: LUCENE-3456.patch, LUCENE-3456.patch
>
>
> While converting consumers over to using reusableTokenStream, I notice many 
> don't call end() or close() on TokenStreams.
> Even if they are test TSs only created once, we should follow the defined 
> usage pattern.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3967) Mapping error: langid.enforceSchema option checks source field instead of target field


[ 
https://issues.apache.org/jira/browse/SOLR-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13566007#comment-13566007
 ] 

Commit Tag Bot commented on SOLR-3967:
--

[trunk commit] Jan Høydahl
http://svn.apache.org/viewvc?view=revision&revision=1440226

SOLR-3967: langid.enforceSchema option checks source field instead of target 
field


> Mapping error: langid.enforceSchema option checks source field instead of 
> target field
> --
>
> Key: SOLR-3967
> URL: https://issues.apache.org/jira/browse/SOLR-3967
> Project: Solr
>  Issue Type: Bug
>  Components: contrib - LangId
>Affects Versions: 4.0
>Reporter: Mateusz Matela
>Assignee: Jan Høydahl
> Fix For: 4.2, 5.0
>
> Attachments: SOLR-3967.patch
>
>
> I use LangDetect update processor with a document that has "body" field. 
> LangDetect should map this field to "body_pl", "body_en" or "body_nolang". My 
> schema defines fields with language suffixes, but not "body" field. When the 
> processor runs, I get error:
> {quote}Unsuccessful field name mapping to body_nolang, field does not exist, 
> skipping mapping.{quote}
> I looked up source code and it seems there's an error in 
> {{org.apache.solr.update.processor.LanguageIdentifierUpdateProcessor.process(SolrInputDocument)}}:
> {noformat}
>   String mappedOutputField = getMappedField(fieldName, fieldLang);
>   if(enforceSchema && schema.getFieldOrNull(fieldName) == null) {
> log.warn("Unsuccessful field name mapping to {}, field does not 
> exist, skipping mapping.", mappedOutputField, fieldName);
> mappedOutputField = fieldName;
>   }
> {noformat}
> I think it should check for {{schema.getFieldOrNull(mappedOutputField)}} 
> instead.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4377) making release tarballs identical to the release tags


[ 
https://issues.apache.org/jira/browse/SOLR-4377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565984#comment-13565984
 ] 

Robert Muir commented on SOLR-4377:
---

{quote}
However, for one, I'd like to nuke the var expansion anyway
{quote}

Please!

> making release tarballs identical to the release tags
> -
>
> Key: SOLR-4377
> URL: https://issues.apache.org/jira/browse/SOLR-4377
> Project: Solr
>  Issue Type: Improvement
>  Components: scripts and tools
>Affects Versions: 4.0, 4.1
>Reporter: Roman Shaposhnik
> Fix For: 4.2, 5.0
>
> Attachments: diff.txt
>
>
> Now that we're integrating Solr with Hadoop via the Apache Bigtop project it 
> is a bit of a nuisance to our build system that the release tarballs don't 
> quite match the SVN tags. This is also something that is not quite ASF kosher 
> strictly speaking. 
> Would it be ok with a Solr community to add a comparison check between 
> release tarballs and SVN tags as part of the release process checklist?
> If you guys have a Wiki outlining how-to-release perhaps it needs to be 
> captured over there or just added to the process. Either way would be fine.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-4382) edismax, qf, multiterm analyzer, wildcard query bug

2013-01-29 Thread Ahmet Arslan (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-4382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ahmet Arslan updated SOLR-4382:
---

Attachment: SOLR-4382.patch

A failing test case

> edismax, qf, multiterm analyzer, wildcard query bug
> ---
>
> Key: SOLR-4382
> URL: https://issues.apache.org/jira/browse/SOLR-4382
> Project: Solr
>  Issue Type: Bug
>  Components: query parsers, Schema and Analysis, search
>Affects Versions: 4.0, 4.1
>Reporter: Ahmet Arslan
> Attachments: SOLR-4382.patch
>
>
> If I fire a wildcard query o*t*v*h using edismax, and add a non existed field 
> to qf parameter, i get this phrase query at the end. (with 
> autoGeneratePhraseQueries="true")
> http://localhost:8983/solr/collection1/select?q=O*t*v*h&wt=xml&debugQuery=on&defType=edismax&qf=sku%20doesNotExit
> parsedquery = (+DisjunctionMaxQuery((sku:"o t v h")))/no_coord
> parsedquery_toString = +(sku:"o t v h")
> Existing field(s) works as expected :
> http://localhost:8983/solr/collection1/select?q=O*t*v*h&wt=xml&debugQuery=on&defType=edismax&qf=sku
> yields
> (+DisjunctionMaxQuery((sku:o*t*v*h)))/no_coord
> +(sku:o*t*v*h)
> For a workaround I enabled following dynamic field :
> 
> User list link : http://search-lucene.com/m/zB104LKTRI

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-4382) edismax, qf, multiterm analyzer, wildcard query bug

2013-01-29 Thread Ahmet Arslan (JIRA)

Ahmet Arslan created SOLR-4382:
--

 Summary: edismax, qf, multiterm analyzer, wildcard query bug
 Key: SOLR-4382
 URL: https://issues.apache.org/jira/browse/SOLR-4382
 Project: Solr
  Issue Type: Bug
  Components: query parsers, Schema and Analysis, search
Affects Versions: 4.1, 4.0
Reporter: Ahmet Arslan


If I fire a wildcard query o*t*v*h using edismax, and add a non existed field 
to qf parameter, i get this phrase query at the end. (with 
autoGeneratePhraseQueries="true")

http://localhost:8983/solr/collection1/select?q=O*t*v*h&wt=xml&debugQuery=on&defType=edismax&qf=sku%20doesNotExit

parsedquery = (+DisjunctionMaxQuery((sku:"o t v h")))/no_coord
parsedquery_toString = +(sku:"o t v h")

Existing field(s) works as expected :

http://localhost:8983/solr/collection1/select?q=O*t*v*h&wt=xml&debugQuery=on&defType=edismax&qf=sku

yields

(+DisjunctionMaxQuery((sku:o*t*v*h)))/no_coord
+(sku:o*t*v*h)


For a workaround I enabled following dynamic field :



User list link : http://search-lucene.com/m/zB104LKTRI

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4734) FastVectorHighlighter Overlapping Proximity Queries Do Not Highlight

2013-01-29 Thread Ryan Lauck (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-4734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ryan Lauck updated LUCENE-4734:
---

Attachment: lucene-fvh-tests.patch

Added a patch with two test cases that reproduce the issue

> FastVectorHighlighter Overlapping Proximity Queries Do Not Highlight
> 
>
> Key: LUCENE-4734
> URL: https://issues.apache.org/jira/browse/LUCENE-4734
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/highlighter
>Affects Versions: 4.0, 4.1
>Reporter: Ryan Lauck
>  Labels: fastvectorhighlighter, highlighter
> Attachments: lucene-fvh-tests.patch
>
>
> If a proximity phrase query overlaps with any other query term it will not be 
> highlighted.
> Example Text:  A B C D E F G
> Example Queries: 
> "B E"~10 D
> (only D will be highlighted)
> "B E"~10 "C F"~10
> (neither phrase will be highlighted)
> This can be traced to the FieldPhraseList constructor's inner while loop. 
> From the first example query, the first TermInfo popped off the stack will be 
> "B". The second TermInfo will be "D" which will not be found in the submap 
> for "B E"~10 and will trigger a failed match.
> I wanted to report this issue before digging into a solution but my first 
> thought is:
> Add an additional int property to QueryPhraseMap to store the maximum 
> possible phrase width for each term based on any proximity searches it is 
> part of (defaulting to zero, in the above examples it would be 10). 
> If a term is popped off the stack that is not a part of a proximity phrase 
> being matched ( currMap.getTermMap(ti.getText()) == null ), it is added to a 
> temporary list until either the longest possible phrase is successfully 
> matched or a term is found outside the maximum possible phrase width.
> After this search is complete, any non-matching terms that were added to the 
> temporary list are pushed back onto the stack to be evaluated again and the 
> temp list is cleared.
> Hopefully this makes sense, and if nobody sees any obvious flaws I may try to 
> create a patch.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-4381) Query-time multi-word synonym expansion

2013-01-29 Thread Nolan Lawson (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-4381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nolan Lawson updated SOLR-4381:
---

Attachment: SOLR-4381.patch

> Query-time multi-word synonym expansion
> ---
>
> Key: SOLR-4381
> URL: https://issues.apache.org/jira/browse/SOLR-4381
> Project: Solr
>  Issue Type: Improvement
>Reporter: Nolan Lawson
>Priority: Minor
>  Labels: multi-word, queryparser, synonyms
> Attachments: SOLR-4381.patch
>
>
> This is an issue that seems to come up perennially.
> The [Solr 
> docs|http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.SynonymFilterFactory]
>  caution that index-time synonym expansion should be preferred to query-time 
> synonym expansion, due to the way multi-word synonyms are treated and how IDF 
> values can be boosted artificially. But query-time expansion should have huge 
> benefits, given that changes to the synonyms don't require re-indexing, the 
> index size stays the same, and the IDF values for the documents don't get 
> permanently altered.
> The proposed solution is to move the synonym expansion logic from the 
> analysis chain (either query- or index-type) and into a new QueryParser.  See 
> the attached patch for an implementation.
> The core Lucene functionality is untouched.  Instead, the EDismaxQParser is 
> extended, and synonym expansion is done on-the-fly.  Queries are parsed into 
> a lattice (i.e. all possible synonym combinations), while individual 
> components of the query are still handled by the EDismaxQParser itself.
> It's not an ideal solution by any stretch. But it's nice and self-contained, 
> so it invites experimentation and improvement.  And I think it fits in well 
> with the merry band of misfit query parsers, like {{func}} and {{frange}}.
> More details about this solution can be found in [this blog 
> post|http://nolanlawson.com/2012/10/31/better-synonym-handling-in-solr/] and 
> [the Github page for the 
> code|https://github.com/healthonnet/hon-lucene-synonyms].
> At the risk of tooting my own horn, I also think this patch sufficiently 
> fixes SOLR-3390 (highlighting problems with multi-word synonyms) and 
> LUCENE-4499 (better support for multi-word synonyms).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-4381) Query-time multi-word synonym expansion

2013-01-29 Thread Nolan Lawson (JIRA)

Nolan Lawson created SOLR-4381:
--

Summary: Query-time multi-word synonym expansion
Key: SOLR-4381
URL: https://issues.apache.org/jira/browse/SOLR-4381
Project: Solr
Issue Type: Improvement
Reporter: Nolan Lawson
Priority: Minor

This is an issue that seems to come up perennially.

The [Solr
docs|http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.SynonymFilterFactory]
caution that index-time synonym expansion should be preferred to query-time
synonym expansion, due to the way multi-word synonyms are treated and how IDF
values can be boosted artificially. But query-time expansion should have huge
benefits, given that changes to the synonyms don't require re-indexing, the
index size stays the same, and the IDF values for the documents don't get
permanently altered.

The proposed solution is to move the synonym expansion logic from the analysis
chain (either query- or index-type) and into a new QueryParser. See the
attached patch for an implementation.

The core Lucene functionality is untouched. Instead, the EDismaxQParser is
extended, and synonym expansion is done on-the-fly. Queries are parsed into a
lattice (i.e. all possible synonym combinations), while individual components
of the query are still handled by the EDismaxQParser itself.

It's not an ideal solution by any stretch. But it's nice and self-contained, so
it invites experimentation and improvement. And I think it fits in well with
the merry band of misfit query parsers, like {{func}} and {{frange}}.

More details about this solution can be found in [this blog
post|http://nolanlawson.com/2012/10/31/better-synonym-handling-in-solr/] and
[the Github page for the
code|https://github.com/healthonnet/hon-lucene-synonyms].

At the risk of tooting my own horn, I also think this patch sufficiently fixes
SOLR-3390 (highlighting problems with multi-word synonyms) and LUCENE-4499
(better support for multi-word synonyms).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4377) making release tarballs identical to the release tags


[ 
https://issues.apache.org/jira/browse/SOLR-4377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565949#comment-13565949
 ] 

Steve Rowe commented on SOLR-4377:
--

@Mark:
bq. I saw that right after the 4.1 tag, a couple quick commits went in - one 
for a spell fix in CHANGES and one a build issue - it made me unsure about 
whether we had released the tag, or a rev with the two fixes that happened 
right after. Of course I suspect we released the tag, but an automated smoke 
test would make me feel better about trusting that.

After I created the RC, I made those commits to the branch anticipating another 
RC, but I tagged the revision the RC was built from, so those commits aren't 
included there.

The {{dev-tools/scripts/buildAndPushRelease.py}} script creates the trailing 
dir name in the release staging RC URL (e.g. 
http://people.apache.org/~sarowe/staging_area/lucene-solr-4.1.0-RC1-rev1434440/),
 including the SVN revision, supplied by the {{svnversion}} program.  You can 
manually verify that the tag revision and the RC revision match once the tag 
has been created.

> making release tarballs identical to the release tags
> -
>
> Key: SOLR-4377
> URL: https://issues.apache.org/jira/browse/SOLR-4377
> Project: Solr
>  Issue Type: Improvement
>  Components: scripts and tools
>Affects Versions: 4.0, 4.1
>Reporter: Roman Shaposhnik
> Fix For: 4.2, 5.0
>
> Attachments: diff.txt
>
>
> Now that we're integrating Solr with Hadoop via the Apache Bigtop project it 
> is a bit of a nuisance to our build system that the release tarballs don't 
> quite match the SVN tags. This is also something that is not quite ASF kosher 
> strictly speaking. 
> Would it be ok with a Solr community to add a comparison check between 
> release tarballs and SVN tags as part of the release process checklist?
> If you guys have a Wiki outlining how-to-release perhaps it needs to be 
> captured over there or just added to the process. Either way would be fine.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4377) making release tarballs identical to the release tags


[ 
https://issues.apache.org/jira/browse/SOLR-4377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565933#comment-13565933
 ] 

Uwe Schindler commented on SOLR-4377:
-

I still don't think they are useful in a source release. In my opinion, bigtop 
should use the same "defaultexcludes" when checking the tag against the release 
artifact.

> making release tarballs identical to the release tags
> -
>
> Key: SOLR-4377
> URL: https://issues.apache.org/jira/browse/SOLR-4377
> Project: Solr
>  Issue Type: Improvement
>  Components: scripts and tools
>Affects Versions: 4.0, 4.1
>Reporter: Roman Shaposhnik
> Fix For: 4.2, 5.0
>
> Attachments: diff.txt
>
>
> Now that we're integrating Solr with Hadoop via the Apache Bigtop project it 
> is a bit of a nuisance to our build system that the release tarballs don't 
> quite match the SVN tags. This is also something that is not quite ASF kosher 
> strictly speaking. 
> Would it be ok with a Solr community to add a comparison check between 
> release tarballs and SVN tags as part of the release process checklist?
> If you guys have a Wiki outlining how-to-release perhaps it needs to be 
> captured over there or just added to the process. Either way would be fine.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

test failures when running under IBM J9

2013-01-29 Thread Michael McCandless

When I run Lucene core tests with IBM J9 1.6, some tests fail with
zombie threads that could not be killed, eg:

[junit4:junit4] ERROR   0.00s J2 | TestIndexWriterDelete (suite) <<<
[junit4:junit4]> Throwable #1:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked
from SUITE scope at org.apache.lucene.index.TestIndexWriterDelete:
[junit4:junit4]>1) Thread[id=102, name=Thread-44,
state=TIMED_WAITING, group=TGRP-TestIndexWriterDelete]
[junit4:junit4]> at java.lang.Object.wait(Native Method)
[junit4:junit4]> at java.lang.Object.wait(Object.java:196)
[junit4:junit4]> at java.util.Timer$TimerImpl.run(Timer.java:247)
[junit4:junit4]>at 
__randomizedtesting.SeedInfo.seed([C9014BCB129899BF]:0)
[junit4:junit4]> Throwable #2:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still
zombie threads that couldn't be terminated:
[junit4:junit4]>1) Thread[id=102, name=Thread-44,
state=TIMED_WAITING, group=TGRP-TestIndexWriterDelete]
[junit4:junit4]> at java.lang.Object.wait(Native Method)
[junit4:junit4]> at java.lang.Object.wait(Object.java:196)
[junit4:junit4]> at java.util.Timer$TimerImpl.run(Timer.java:247)
[junit4:junit4]>at 
__randomizedtesting.SeedInfo.seed([C9014BCB129899BF]:0)

It looks like this is a JVM thread ... is there an "exceptions list"
somewhere to ignore this thread?  Or some other solution?

Second, strangely, I see the "who tests the tester tests" hit real failures, eg:

[junit4:junit4] FAILURE 0.01s J2 |
TestReproduceMessage.testFailureBeforeClass <<<
[junit4:junit4]> Throwable #1: java.lang.AssertionError
[junit4:junit4]>at org.junit.Assert.fail(Assert.java:92)
[junit4:junit4]>at org.junit.Assert.assertTrue(Assert.java:43)
[junit4:junit4]>at org.junit.Assert.assertTrue(Assert.java:54)
[junit4:junit4]>at
org.apache.lucene.util.junitcompat.TestReproduceMessage.testFailureBeforeClass(TestReproduceMessage.java:173)
[junit4:junit4]>at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4]>at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
[junit4:junit4]>at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
[junit4:junit4]>at java.lang.reflect.Method.invoke(Method.java:611)
[junit4:junit4]>at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
[junit4:junit4]>at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
[junit4:junit4]>at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
[junit4:junit4]>at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
[junit4:junit4]>at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
[junit4:junit4]>at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
[junit4:junit4]>at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]>at org.junit.rules.RunRules.evaluate(RunRules.java:18)
[junit4:junit4]>at
org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
[junit4:junit4]>at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
[junit4:junit4]>at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
[junit4:junit4]>at
org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
[junit4:junit4]>at
org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
[junit4:junit4]>at
org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
[junit4:junit4]>at
org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
[junit4:junit4]>at
org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
[junit4:junit4]>at
org.junit.runners.ParentRunner.run(ParentRunner.java:300)
[junit4:junit4]>at
com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:180)
[junit4:junit4]>at
com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:275)
[junit4:junit4]>at
com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:12)

These are tests that intentionally set up a failure, and then confirm
that the test framework catches that failure ... yet for some reason
with J9 this turns into a real failure ... any ideas?

Mike McCandless

http://blog.mikemccandless.com

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4377) making release tarballs identical to the release tags


[ 
https://issues.apache.org/jira/browse/SOLR-4377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565916#comment-13565916
 ] 

Steve Rowe commented on SOLR-4377:
--

bq. They are not in the src-tarball, because the ANT  used by the 
 task excludes those file types: 
http://ant.apache.org/manual/dirtasks.html#defaultexcludes - This is intended, 
otherwise it would also contain .svn/ and similar files.

I checked in {{build/solr/svn-export/}}, where the source is put before 
packaging, and the .gitignore and .hgignore files are there, so you must be 
right Uwe.  But since this is an svn *export*, and not a *checkout*, there are 
no {{.svn/}} directories.  So we should add {{defaultexcludes="no"}} to include 
those two file.  I looked at all of the patterns in the Ant docs, and none of 
them are applicable, since this is always a fresh export, there will be no 
backup/temporary/VC files that we'll want to exclude.

> making release tarballs identical to the release tags
> -
>
> Key: SOLR-4377
> URL: https://issues.apache.org/jira/browse/SOLR-4377
> Project: Solr
>  Issue Type: Improvement
>  Components: scripts and tools
>Affects Versions: 4.0, 4.1
>Reporter: Roman Shaposhnik
> Fix For: 4.2, 5.0
>
> Attachments: diff.txt
>
>
> Now that we're integrating Solr with Hadoop via the Apache Bigtop project it 
> is a bit of a nuisance to our build system that the release tarballs don't 
> quite match the SVN tags. This is also something that is not quite ASF kosher 
> strictly speaking. 
> Would it be ok with a Solr community to add a comparison check between 
> release tarballs and SVN tags as part of the release process checklist?
> If you guys have a Wiki outlining how-to-release perhaps it needs to be 
> captured over there or just added to the process. Either way would be fine.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4377) making release tarballs identical to the release tags


[ 
https://issues.apache.org/jira/browse/SOLR-4377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565913#comment-13565913
 ] 

Uwe Schindler commented on SOLR-4377:
-

bq. However, for one, I'd like to nuke the var expansion anyway

+1, we never started that officially (whenever I saw such a comment, I nuked 
it), but nobody did it everywhere. Solr has one problem with this, as some 
description "Strings" for MBeans also contain $Id$ tags. 

We can add the pattern for those $Id$ and other tags to our precommit ANT 
check. This would also find the offenders very quick, just change the root 
build.xml to include another regex (next to nocommit,...).

> making release tarballs identical to the release tags
> -
>
> Key: SOLR-4377
> URL: https://issues.apache.org/jira/browse/SOLR-4377
> Project: Solr
>  Issue Type: Improvement
>  Components: scripts and tools
>Affects Versions: 4.0, 4.1
>Reporter: Roman Shaposhnik
> Fix For: 4.2, 5.0
>
> Attachments: diff.txt
>
>
> Now that we're integrating Solr with Hadoop via the Apache Bigtop project it 
> is a bit of a nuisance to our build system that the release tarballs don't 
> quite match the SVN tags. This is also something that is not quite ASF kosher 
> strictly speaking. 
> Would it be ok with a Solr community to add a comparison check between 
> release tarballs and SVN tags as part of the release process checklist?
> If you guys have a Wiki outlining how-to-release perhaps it needs to be 
> captured over there or just added to the process. Either way would be fine.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-4380) ReplicationHandler replicateAfter startup not showing commits after SOLR-3911

2013-01-29 Thread Gregg Donovan (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-4380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gregg Donovan updated SOLR-4380:


Description: 
In the process of upgrading to 4.1 from 3.6, I've noticed that our master 
servers do not show any commit points -- e.g. via 
http://127.0.0.1:8983/solr/collection1/replication?command=commits -- available 
until after a new commit happens. So, for static indexes, replication doesn't 
happen and for dynamic indexes, we have to wait until an incremental update of 
master for slaves to see any commits.

Tracing through the code, it looks like the change that may have effected us 
was part of [SOLR-3911|https://issues.apache.org/jira/browse/SOLR-3911], 
specifically commenting out the initialization of the newIndexWriter in the 
replicateAfterStartup block [1]:
{code:java}
// TODO: perhaps this is no longer necessary then?
// core.getUpdateHandler().newIndexWriter(true);
{code}

I'm guessing this is commented out because it is assumed that indexCommitPoint 
was going to be set by that block, but when a slave requests commits, that goes 
back to core.getDeletionPolicy().getCommits() to fetch the list of commits. If 
no indexWriter has been initialized, then, as far as I can tell, 
IndexDeletionPolicyWrapper#onInit will not have been called and there will be 
no commits available.

By uncommenting this line, I was able to see commits on startup and slaves 
began to replicate successfully.


[1] 
http://svn.apache.org/viewvc/lucene/dev/branches/branch_4x/solr/core/src/java/org/apache/solr/handler/ReplicationHandler.java?annotate=1420992&diff_format=h&pathrev=1420992#l880

  was:
In the process of upgrading to 4.1 from 3.6, I've noticed that our
master servers do not show any commit points -- e.g. via 
http://127.0.0.1:8983/solr/collection1/replication?command=commits -- available 
until after a
new commit happens. So, for static indexes, replication doesn't happen
and for dynamic indexes, we have to wait until an incremental update
of master for slaves to see any commits.

Tracing through the code, it looks like the change that may have
effected us was part of 
[SOLR-3911|https://issues.apache.org/jira/browse/SOLR-3911], specifically 
commenting out the
initialization of the newIndexWriter in the [replicateAfterStartup
block [1]:
{code:java}
// TODO: perhaps this is no longer necessary then?
// core.getUpdateHandler().newIndexWriter(true);
{code}

I'm guessing this is commented out because it is assumed that
indexCommitPoint was going to be set by that block, but when a slave
requests commits, that goes back to
core.getDeletionPolicy().getCommits() to fetch the list of commits. If
no indexWriter has been initialized, then, as far as I can tell,
IndexDeletionPolicyWrapper#onInit will not have been called and there
will be no commits available.

By uncommenting this line, I was able to see commits on startup and slaves 
began to replicate successfully.


[1] 
http://svn.apache.org/viewvc/lucene/dev/branches/branch_4x/solr/core/src/java/org/apache/solr/handler/ReplicationHandler.java?annotate=1420992&diff_format=h&pathrev=1420992#l880


> ReplicationHandler replicateAfter startup not showing commits after SOLR-3911
> -
>
> Key: SOLR-4380
> URL: https://issues.apache.org/jira/browse/SOLR-4380
> Project: Solr
>  Issue Type: Bug
>  Components: replication (java)
>Affects Versions: 4.0, 4.1
>Reporter: Gregg Donovan
>
> In the process of upgrading to 4.1 from 3.6, I've noticed that our master 
> servers do not show any commit points -- e.g. via 
> http://127.0.0.1:8983/solr/collection1/replication?command=commits -- 
> available until after a new commit happens. So, for static indexes, 
> replication doesn't happen and for dynamic indexes, we have to wait until an 
> incremental update of master for slaves to see any commits.
> Tracing through the code, it looks like the change that may have effected us 
> was part of [SOLR-3911|https://issues.apache.org/jira/browse/SOLR-3911], 
> specifically commenting out the initialization of the newIndexWriter in the 
> replicateAfterStartup block [1]:
> {code:java}
> // TODO: perhaps this is no longer necessary then?
> // core.getUpdateHandler().newIndexWriter(true);
> {code}
> I'm guessing this is commented out because it is assumed that 
> indexCommitPoint was going to be set by that block, but when a slave requests 
> commits, that goes back to core.getDeletionPolicy().getCommits() to fetch the 
> list of commits. If no indexWriter has been initialized, then, as far as I 
> can tell, IndexDeletionPolicyWrapper#onInit will not have been called and 
> there will be no commits available.
> By uncommenting this line, I was able to see commits on startup and slaves 
> began to replicate successfully.
> [1] 
> http://svn.a

[jira] [Created] (SOLR-4380) ReplicationHandler replicateAfter startup not showing commits after SOLR-3911

2013-01-29 Thread Gregg Donovan (JIRA)

Gregg Donovan created SOLR-4380:
---

 Summary: ReplicationHandler replicateAfter startup not showing 
commits after SOLR-3911
 Key: SOLR-4380
 URL: https://issues.apache.org/jira/browse/SOLR-4380
 Project: Solr
  Issue Type: Bug
  Components: replication (java)
Affects Versions: 4.1, 4.0
Reporter: Gregg Donovan


In the process of upgrading to 4.1 from 3.6, I've noticed that our
master servers do not show any commit points -- e.g. via 
http://127.0.0.1:8983/solr/collection1/replication?command=commits -- available 
until after a
new commit happens. So, for static indexes, replication doesn't happen
and for dynamic indexes, we have to wait until an incremental update
of master for slaves to see any commits.

Tracing through the code, it looks like the change that may have
effected us was part of 
[SOLR-3911|https://issues.apache.org/jira/browse/SOLR-3911], specifically 
commenting out the
initialization of the newIndexWriter in the [replicateAfterStartup
block [1]:
{code:java}
// TODO: perhaps this is no longer necessary then?
// core.getUpdateHandler().newIndexWriter(true);
{code}

I'm guessing this is commented out because it is assumed that
indexCommitPoint was going to be set by that block, but when a slave
requests commits, that goes back to
core.getDeletionPolicy().getCommits() to fetch the list of commits. If
no indexWriter has been initialized, then, as far as I can tell,
IndexDeletionPolicyWrapper#onInit will not have been called and there
will be no commits available.

By uncommenting this line, I was able to see commits on startup and slaves 
began to replicate successfully.


[1] 
http://svn.apache.org/viewvc/lucene/dev/branches/branch_4x/solr/core/src/java/org/apache/solr/handler/ReplicationHandler.java?annotate=1420992&diff_format=h&pathrev=1420992#l880

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4377) making release tarballs identical to the release tags


[ 
https://issues.apache.org/jira/browse/SOLR-4377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565904#comment-13565904
 ] 

Mark Miller commented on SOLR-4377:
---

Steve: I'd agree these differences are intentional and while I labeled them 
suspicious initially, each appears to check out as expected.

However, for one, I'd like to nuke the var expansion anyway :), and two, we 
should check with Roman about what the issue is in regards to big top 
integration and see what we can do to smooth things out if possible.

> making release tarballs identical to the release tags
> -
>
> Key: SOLR-4377
> URL: https://issues.apache.org/jira/browse/SOLR-4377
> Project: Solr
>  Issue Type: Improvement
>  Components: scripts and tools
>Affects Versions: 4.0, 4.1
>Reporter: Roman Shaposhnik
> Fix For: 4.2, 5.0
>
> Attachments: diff.txt
>
>
> Now that we're integrating Solr with Hadoop via the Apache Bigtop project it 
> is a bit of a nuisance to our build system that the release tarballs don't 
> quite match the SVN tags. This is also something that is not quite ASF kosher 
> strictly speaking. 
> Would it be ok with a Solr community to add a comparison check between 
> release tarballs and SVN tags as part of the release process checklist?
> If you guys have a Wiki outlining how-to-release perhaps it needs to be 
> captured over there or just added to the process. Either way would be fine.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4377) making release tarballs identical to the release tags


[ 
https://issues.apache.org/jira/browse/SOLR-4377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565903#comment-13565903
 ] 

Uwe Schindler commented on SOLR-4377:
-

{quote}
bq. I'm not sure why .gitignore and .hgignore aren't present, but I'd argue 
that they have no point in a source tarball outside of a VC system.

As they are part of our src tree and serve a purpose, I don't see why wouldn't 
include them.
{quote}

They are not in the src-tarball, because the ANT  used by the  
task excludes those file types: 
http://ant.apache.org/manual/dirtasks.html#defaultexcludes - This is intended, 
otherwise it would also contain .svn/ and similar files.

> making release tarballs identical to the release tags
> -
>
> Key: SOLR-4377
> URL: https://issues.apache.org/jira/browse/SOLR-4377
> Project: Solr
>  Issue Type: Improvement
>  Components: scripts and tools
>Affects Versions: 4.0, 4.1
>Reporter: Roman Shaposhnik
> Fix For: 4.2, 5.0
>
> Attachments: diff.txt
>
>
> Now that we're integrating Solr with Hadoop via the Apache Bigtop project it 
> is a bit of a nuisance to our build system that the release tarballs don't 
> quite match the SVN tags. This is also something that is not quite ASF kosher 
> strictly speaking. 
> Would it be ok with a Solr community to add a comparison check between 
> release tarballs and SVN tags as part of the release process checklist?
> If you guys have a Wiki outlining how-to-release perhaps it needs to be 
> captured over there or just added to the process. Either way would be fine.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4377) making release tarballs identical to the release tags


[ 
https://issues.apache.org/jira/browse/SOLR-4377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565901#comment-13565901
 ] 

Mark Miller commented on SOLR-4377:
---

bq. I'm not sure why .gitignore and .hgignore aren't present, but I'd argue 
that they have no point in a source tarball outside of a VC system.

As they are part of our src tree and serve a purpose, I don't see why wouldn't 
include them.

bq. keyword expansion

I'd like to just remove these. Who stands to defend them?

> making release tarballs identical to the release tags
> -
>
> Key: SOLR-4377
> URL: https://issues.apache.org/jira/browse/SOLR-4377
> Project: Solr
>  Issue Type: Improvement
>  Components: scripts and tools
>Affects Versions: 4.0, 4.1
>Reporter: Roman Shaposhnik
> Fix For: 4.2, 5.0
>
> Attachments: diff.txt
>
>
> Now that we're integrating Solr with Hadoop via the Apache Bigtop project it 
> is a bit of a nuisance to our build system that the release tarballs don't 
> quite match the SVN tags. This is also something that is not quite ASF kosher 
> strictly speaking. 
> Would it be ok with a Solr community to add a comparison check between 
> release tarballs and SVN tags as part of the release process checklist?
> If you guys have a Wiki outlining how-to-release perhaps it needs to be 
> captured over there or just added to the process. Either way would be fine.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4377) making release tarballs identical to the release tags


[ 
https://issues.apache.org/jira/browse/SOLR-4377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565897#comment-13565897
 ] 

Steve Rowe commented on SOLR-4377:
--

I'm not sure why .gitignore and .hgignore aren't present, but I'd argue that 
they have no point in a source tarball outside of a VC system.

All of the other difference (except for svn $URL$ keyword expansion differences 
as a result of branch vs. tag location) appear to be intentional - see [the 
package-src-tgz target in 
{{solr/build.xml}}|http://svn.apache.org/viewvc/lucene/dev/trunk/solr/build.xml?view=markup#l382].

> making release tarballs identical to the release tags
> -
>
> Key: SOLR-4377
> URL: https://issues.apache.org/jira/browse/SOLR-4377
> Project: Solr
>  Issue Type: Improvement
>  Components: scripts and tools
>Affects Versions: 4.0, 4.1
>Reporter: Roman Shaposhnik
> Fix For: 4.2, 5.0
>
> Attachments: diff.txt
>
>
> Now that we're integrating Solr with Hadoop via the Apache Bigtop project it 
> is a bit of a nuisance to our build system that the release tarballs don't 
> quite match the SVN tags. This is also something that is not quite ASF kosher 
> strictly speaking. 
> Would it be ok with a Solr community to add a comparison check between 
> release tarballs and SVN tags as part of the release process checklist?
> If you guys have a Wiki outlining how-to-release perhaps it needs to be 
> captured over there or just added to the process. Either way would be fine.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4377) making release tarballs identical to the release tags


[ 
https://issues.apache.org/jira/browse/SOLR-4377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565864#comment-13565864
 ] 

Uwe Schindler commented on SOLR-4377:
-

bq. generated stuff - eg our html changes stuff.

Good point. The Changes HTML is added to the source tarball and is not existent 
in SVN.

> making release tarballs identical to the release tags
> -
>
> Key: SOLR-4377
> URL: https://issues.apache.org/jira/browse/SOLR-4377
> Project: Solr
>  Issue Type: Improvement
>  Components: scripts and tools
>Affects Versions: 4.0, 4.1
>Reporter: Roman Shaposhnik
> Fix For: 4.2, 5.0
>
> Attachments: diff.txt
>
>
> Now that we're integrating Solr with Hadoop via the Apache Bigtop project it 
> is a bit of a nuisance to our build system that the release tarballs don't 
> quite match the SVN tags. This is also something that is not quite ASF kosher 
> strictly speaking. 
> Would it be ok with a Solr community to add a comparison check between 
> release tarballs and SVN tags as part of the release process checklist?
> If you guys have a Wiki outlining how-to-release perhaps it needs to be 
> captured over there or just added to the process. Either way would be fine.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4377) making release tarballs identical to the release tags


[ 
https://issues.apache.org/jira/browse/SOLR-4377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565862#comment-13565862
 ] 

Mark Miller commented on SOLR-4377:
---

bq. The release process I followed and I think Robert followed, was to tag 
after a successful VOTE.

I don't think that is a problem - as long as the tag matches what we released.

> making release tarballs identical to the release tags
> -
>
> Key: SOLR-4377
> URL: https://issues.apache.org/jira/browse/SOLR-4377
> Project: Solr
>  Issue Type: Improvement
>  Components: scripts and tools
>Affects Versions: 4.0, 4.1
>Reporter: Roman Shaposhnik
> Fix For: 4.2, 5.0
>
> Attachments: diff.txt
>
>
> Now that we're integrating Solr with Hadoop via the Apache Bigtop project it 
> is a bit of a nuisance to our build system that the release tarballs don't 
> quite match the SVN tags. This is also something that is not quite ASF kosher 
> strictly speaking. 
> Would it be ok with a Solr community to add a comparison check between 
> release tarballs and SVN tags as part of the release process checklist?
> If you guys have a Wiki outlining how-to-release perhaps it needs to be 
> captured over there or just added to the process. Either way would be fine.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4377) making release tarballs identical to the release tags


[ 
https://issues.apache.org/jira/browse/SOLR-4377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565860#comment-13565860
 ] 

Uwe Schindler commented on SOLR-4377:
-

Yes, I also tagged once the vote for the artifacts completes. The release 
artifacts are made from the branch and the revision is recorded in the vote 
mail thread. After the vote succeeds, I tagged this exact revision number. So 
the only differences can be rev nos in $Id$ tags inside source files (which we 
should remove asap, they are causing trouble all the time).

> making release tarballs identical to the release tags
> -
>
> Key: SOLR-4377
> URL: https://issues.apache.org/jira/browse/SOLR-4377
> Project: Solr
>  Issue Type: Improvement
>  Components: scripts and tools
>Affects Versions: 4.0, 4.1
>Reporter: Roman Shaposhnik
> Fix For: 4.2, 5.0
>
> Attachments: diff.txt
>
>
> Now that we're integrating Solr with Hadoop via the Apache Bigtop project it 
> is a bit of a nuisance to our build system that the release tarballs don't 
> quite match the SVN tags. This is also something that is not quite ASF kosher 
> strictly speaking. 
> Would it be ok with a Solr community to add a comparison check between 
> release tarballs and SVN tags as part of the release process checklist?
> If you guys have a Wiki outlining how-to-release perhaps it needs to be 
> captured over there or just added to the process. Either way would be fine.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4377) making release tarballs identical to the release tags


[ 
https://issues.apache.org/jira/browse/SOLR-4377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565856#comment-13565856
 ] 

Mark Miller commented on SOLR-4377:
---

The rest of the issues look like:

1. generated stuff - eg our html changes stuff.
2. hg and git ignore files at the root.

So it looks to me like those are the 3 total differences?

> making release tarballs identical to the release tags
> -
>
> Key: SOLR-4377
> URL: https://issues.apache.org/jira/browse/SOLR-4377
> Project: Solr
>  Issue Type: Improvement
>  Components: scripts and tools
>Affects Versions: 4.0, 4.1
>Reporter: Roman Shaposhnik
> Fix For: 4.2, 5.0
>
> Attachments: diff.txt
>
>
> Now that we're integrating Solr with Hadoop via the Apache Bigtop project it 
> is a bit of a nuisance to our build system that the release tarballs don't 
> quite match the SVN tags. This is also something that is not quite ASF kosher 
> strictly speaking. 
> Would it be ok with a Solr community to add a comparison check between 
> release tarballs and SVN tags as part of the release process checklist?
> If you guys have a Wiki outlining how-to-release perhaps it needs to be 
> captured over there or just added to the process. Either way would be fine.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4377) making release tarballs identical to the release tags


[ 
https://issues.apache.org/jira/browse/SOLR-4377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565853#comment-13565853
 ] 

Steve Rowe commented on SOLR-4377:
--

The release process I followed and I think Robert followed, was to tag after a 
successful VOTE.  Otherwise, the tag will be in the wrong place for some of the 
svn history.

> making release tarballs identical to the release tags
> -
>
> Key: SOLR-4377
> URL: https://issues.apache.org/jira/browse/SOLR-4377
> Project: Solr
>  Issue Type: Improvement
>  Components: scripts and tools
>Affects Versions: 4.0, 4.1
>Reporter: Roman Shaposhnik
> Fix For: 4.2, 5.0
>
> Attachments: diff.txt
>
>
> Now that we're integrating Solr with Hadoop via the Apache Bigtop project it 
> is a bit of a nuisance to our build system that the release tarballs don't 
> quite match the SVN tags. This is also something that is not quite ASF kosher 
> strictly speaking. 
> Would it be ok with a Solr community to add a comparison check between 
> release tarballs and SVN tags as part of the release process checklist?
> If you guys have a Wiki outlining how-to-release perhaps it needs to be 
> captured over there or just added to the process. Either way would be fine.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-4373) In multicore, lib directives in solrconfig.xml cause conflict and clobber directives from earlier cores


 [ 
https://issues.apache.org/jira/browse/SOLR-4373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-4373:
--

Fix Version/s: 5.0

> In multicore, lib directives in solrconfig.xml cause conflict and clobber 
> directives from earlier cores
> ---
>
> Key: SOLR-4373
> URL: https://issues.apache.org/jira/browse/SOLR-4373
> Project: Solr
>  Issue Type: Bug
>  Components: multicore
>Affects Versions: 4.1
>Reporter: Alexandre Rafalovitch
>Priority: Blocker
>  Labels: lib, multicore
> Fix For: 4.2, 5.0, 4.1.1
>
> Attachments: multicore-bug.zip
>
>
> Having lib directives in solrconfig.xml seem to wipe out/override the 
> definitions in previous cores.
> The exception (for the earlier core) is:
>   at 
> org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:177)
>   at org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:369)
>   at org.apache.solr.schema.IndexSchema.(IndexSchema.java:113)
>   at 
> org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:1000)
>   at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1033)
>   at org.apache.solr.core.CoreContainer$3.call(CoreContainer.java:629)
>   at org.apache.solr.core.CoreContainer$3.call(CoreContainer.java:624)
>   at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>   at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>   at java.lang.Thread.run(Thread.java:722)
> Caused by: org.apache.solr.common.SolrException: Plugin init failure for 
> [schema.xml] analyzer/filter: Error loading class 
> 'solr.ICUFoldingFilterFactory'
>   at 
> org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:177)
>   at 
> org.apache.solr.schema.FieldTypePluginLoader.readAnalyzer(FieldTypePluginLoader.java:377)
>   at 
> org.apache.solr.schema.FieldTypePluginLoader.create(FieldTypePluginLoader.java:95)
>   at 
> org.apache.solr.schema.FieldTypePluginLoader.create(FieldTypePluginLoader.java:43)
>   at 
> org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:151)
> The full replication case is attached.
> If the SECOND core is turned off in solr.xml, the FIRST core loads just fine.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-4377) making release tarballs identical to the release tags


 [ 
https://issues.apache.org/jira/browse/SOLR-4377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-4377:
--

Fix Version/s: 5.0
   4.2

> making release tarballs identical to the release tags
> -
>
> Key: SOLR-4377
> URL: https://issues.apache.org/jira/browse/SOLR-4377
> Project: Solr
>  Issue Type: Improvement
>  Components: scripts and tools
>Affects Versions: 4.0, 4.1
>Reporter: Roman Shaposhnik
> Fix For: 4.2, 5.0
>
> Attachments: diff.txt
>
>
> Now that we're integrating Solr with Hadoop via the Apache Bigtop project it 
> is a bit of a nuisance to our build system that the release tarballs don't 
> quite match the SVN tags. This is also something that is not quite ASF kosher 
> strictly speaking. 
> Would it be ok with a Solr community to add a comparison check between 
> release tarballs and SVN tags as part of the release process checklist?
> If you guys have a Wiki outlining how-to-release perhaps it needs to be 
> captured over there or just added to the process. Either way would be fine.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-4379) solr-core has a dependency to slf4j-jdk14 and is not binding agnostic

2013-01-29 Thread Nicolas Labrot (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicolas Labrot updated SOLR-4379:
-

Priority: Minor  (was: Major)

> solr-core has a dependency to slf4j-jdk14 and is not binding agnostic
> -
>
> Key: SOLR-4379
> URL: https://issues.apache.org/jira/browse/SOLR-4379
> Project: Solr
>  Issue Type: Improvement
>Affects Versions: 4.1
>Reporter: Nicolas Labrot
>Priority: Minor
>
> solr-core can be used as a dependency in other projects which used others 
> binding. In these cases slf4j-jdk14 must be excluded
> In my opinion it may be better to move the slf4j-jdk14 dependency from 
> solr-core to the war project. 
> solr-core will be binding agnostics.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-4379) solr-core has a dependency to slf4j-jdk14 and is not binding agnostic

2013-01-29 Thread Nicolas Labrot (JIRA)

Nicolas Labrot created SOLR-4379:


 Summary: solr-core has a dependency to slf4j-jdk14 and is not 
binding agnostic
 Key: SOLR-4379
 URL: https://issues.apache.org/jira/browse/SOLR-4379
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.1
Reporter: Nicolas Labrot


solr-core can be used as a dependency in other projects which used others 
binding. In these cases slf4j-jdk14 must be excluded

In my opinion it may be better to move the slf4j-jdk14 dependency from 
solr-core to the war project. 

solr-core will be binding agnostics.




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4377) making release tarballs identical to the release tags


[ 
https://issues.apache.org/jira/browse/SOLR-4377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565843#comment-13565843
 ] 

Mark Miller commented on SOLR-4377:
---

Most of the ones that look like the example i pasted above are because the src 
release came from the branch and not the tag it looks.

That's less worrisome than the css and other files with content differences

> making release tarballs identical to the release tags
> -
>
> Key: SOLR-4377
> URL: https://issues.apache.org/jira/browse/SOLR-4377
> Project: Solr
>  Issue Type: Improvement
>  Components: scripts and tools
>Affects Versions: 4.0, 4.1
>Reporter: Roman Shaposhnik
> Attachments: diff.txt
>
>
> Now that we're integrating Solr with Hadoop via the Apache Bigtop project it 
> is a bit of a nuisance to our build system that the release tarballs don't 
> quite match the SVN tags. This is also something that is not quite ASF kosher 
> strictly speaking. 
> Would it be ok with a Solr community to add a comparison check between 
> release tarballs and SVN tags as part of the release process checklist?
> If you guys have a Wiki outlining how-to-release perhaps it needs to be 
> captured over there or just added to the process. Either way would be fine.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4735) IBM J9 JVM bug causes test failure in Kuromoji's TestExtended


[ 
https://issues.apache.org/jira/browse/LUCENE-4735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565829#comment-13565829
 ] 

Commit Tag Bot commented on LUCENE-4735:


[branch_4x commit] Michael McCandless
http://svn.apache.org/viewvc?view=revision&revision=1440143

LUCENE-4735: workaround IBM J9 JVM bug


> IBM J9 JVM bug causes test failure in Kuromoji's TestExtended
> -
>
> Key: LUCENE-4735
> URL: https://issues.apache.org/jira/browse/LUCENE-4735
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
> Attachments: LUCENE-4735.patch, TestTreeMap2.java
>
>
> Note that this is not a Lucene bug; it's a JVM bug, but I wanted to track it 
> in Lucene as well in case others hit it.
> I noticed this test frequently fails when running under IBM's J9 JVM (1.6.0) 
> ... and I finally tracked down the root cause and made a small test case, eg 
> on trunk, rev 1439839, if you run:
> {noformat}
>   ant test -Dtestcase=TestExtendedMode -Dtestmethod=testRandomHugeStrings 
> -Dtests.seed=26D2B352E9603950
> {noformat}
> it fails with this:
> {noformat}
> [junit4:junit4]> Throwable #1: java.lang.IllegalArgumentException: 
> startOffset must be non-negative, and endOffset must be >= startOffset, 
> startOffset=4272,endOffset=4271
> [junit4:junit4]>  at 
> __randomizedtesting.SeedInfo.seed([26D2B352E9603950:BEF1D491B7168518]:0)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.tokenattributes.OffsetAttributeImpl.setOffset(OffsetAttributeImpl.java:45)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.ja.JapaneseTokenizer.incrementToken(JapaneseTokenizer.java:463)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:635)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:546)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:447)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:375)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.ja.TestExtendedMode.testRandomHugeStrings(TestExtendedMode.java:76)
> {noformat}
> I've seen other analyzer tests fail with similar exceptions.
> I dug in, and found that there's a bug in TreeMap.subMap, and it's easily 
> reproduced with a small test case, which I'll attach.  I'll also open an 
> issue with J9.
> I also found a workaround that seems to sidestep the bug for Lucene.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4377) making release tarballs identical to the release tags


[ 
https://issues.apache.org/jira/browse/SOLR-4377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565828#comment-13565828
 ] 

Mark Miller commented on SOLR-4377:
---

The incubators projects have a long history of having much stricter rules than 
other TLP's that are applied before graduating. Podlings are at the mercy of 
the incubator PMC. Other top level projects are not however.

In our case, we vote on specific artifacts bits for release, not on a tag in 
svn. We could also choose to vote on a tag, but I don't believe that any 
current policy requires that our project ensures the tag exactly matches the 
released artifacts we voted on. It's a nice quality in any case though!

Looking at that diff, this looks beyond svn subs. That does not really look 
like the issue after all. I'd have to look closer to understand what those 
diffs are all about:

{noformat}
diff -ruN 
solr-4.1.0/solr/core/src/java/org/apache/solr/highlight/RegexFragmenter.java 
lucene_solr_4_1_0/solr/core/src/java/org/apache/solr/highlight/RegexFragmenter.java
--- 
solr-4.1.0/solr/core/src/java/org/apache/solr/highlight/RegexFragmenter.java
2012-12-11 08:19:43.0 -0800
+++ 
lucene_solr_4_1_0/solr/core/src/java/org/apache/solr/highlight/RegexFragmenter.java
 2013-01-29 11:43:20.242509623 -0800
{noformat}

There is an example - all this stuff looks suspicious.

> making release tarballs identical to the release tags
> -
>
> Key: SOLR-4377
> URL: https://issues.apache.org/jira/browse/SOLR-4377
> Project: Solr
>  Issue Type: Improvement
>  Components: scripts and tools
>Affects Versions: 4.0, 4.1
>Reporter: Roman Shaposhnik
> Attachments: diff.txt
>
>
> Now that we're integrating Solr with Hadoop via the Apache Bigtop project it 
> is a bit of a nuisance to our build system that the release tarballs don't 
> quite match the SVN tags. This is also something that is not quite ASF kosher 
> strictly speaking. 
> Would it be ok with a Solr community to add a comparison check between 
> release tarballs and SVN tags as part of the release process checklist?
> If you guys have a Wiki outlining how-to-release perhaps it needs to be 
> captured over there or just added to the process. Either way would be fine.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4735) IBM J9 JVM bug causes test failure in Kuromoji's TestExtended


[ 
https://issues.apache.org/jira/browse/LUCENE-4735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565820#comment-13565820
 ] 

Commit Tag Bot commented on LUCENE-4735:


[trunk commit] Michael McCandless
http://svn.apache.org/viewvc?view=revision&revision=1440137

LUCENE-4735: workaround IBM J9 JVM bug


> IBM J9 JVM bug causes test failure in Kuromoji's TestExtended
> -
>
> Key: LUCENE-4735
> URL: https://issues.apache.org/jira/browse/LUCENE-4735
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
> Attachments: LUCENE-4735.patch, TestTreeMap2.java
>
>
> Note that this is not a Lucene bug; it's a JVM bug, but I wanted to track it 
> in Lucene as well in case others hit it.
> I noticed this test frequently fails when running under IBM's J9 JVM (1.6.0) 
> ... and I finally tracked down the root cause and made a small test case, eg 
> on trunk, rev 1439839, if you run:
> {noformat}
>   ant test -Dtestcase=TestExtendedMode -Dtestmethod=testRandomHugeStrings 
> -Dtests.seed=26D2B352E9603950
> {noformat}
> it fails with this:
> {noformat}
> [junit4:junit4]> Throwable #1: java.lang.IllegalArgumentException: 
> startOffset must be non-negative, and endOffset must be >= startOffset, 
> startOffset=4272,endOffset=4271
> [junit4:junit4]>  at 
> __randomizedtesting.SeedInfo.seed([26D2B352E9603950:BEF1D491B7168518]:0)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.tokenattributes.OffsetAttributeImpl.setOffset(OffsetAttributeImpl.java:45)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.ja.JapaneseTokenizer.incrementToken(JapaneseTokenizer.java:463)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:635)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:546)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:447)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:375)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.ja.TestExtendedMode.testRandomHugeStrings(TestExtendedMode.java:76)
> {noformat}
> I've seen other analyzer tests fail with similar exceptions.
> I dug in, and found that there's a bug in TreeMap.subMap, and it's easily 
> reproduced with a small test case, which I'll attach.  I'll also open an 
> issue with J9.
> I also found a workaround that seems to sidestep the bug for Lucene.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4735) IBM J9 JVM bug causes test failure in Kuromoji's TestExtended


[ 
https://issues.apache.org/jira/browse/LUCENE-4735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565785#comment-13565785
 ] 

Robert Muir commented on LUCENE-4735:
-

LOL

+1 to commit the workaround, its just MockCharFilter (which is not fast!)

> IBM J9 JVM bug causes test failure in Kuromoji's TestExtended
> -
>
> Key: LUCENE-4735
> URL: https://issues.apache.org/jira/browse/LUCENE-4735
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
> Attachments: LUCENE-4735.patch, TestTreeMap2.java
>
>
> Note that this is not a Lucene bug; it's a JVM bug, but I wanted to track it 
> in Lucene as well in case others hit it.
> I noticed this test frequently fails when running under IBM's J9 JVM (1.6.0) 
> ... and I finally tracked down the root cause and made a small test case, eg 
> on trunk, rev 1439839, if you run:
> {noformat}
>   ant test -Dtestcase=TestExtendedMode -Dtestmethod=testRandomHugeStrings 
> -Dtests.seed=26D2B352E9603950
> {noformat}
> it fails with this:
> {noformat}
> [junit4:junit4]> Throwable #1: java.lang.IllegalArgumentException: 
> startOffset must be non-negative, and endOffset must be >= startOffset, 
> startOffset=4272,endOffset=4271
> [junit4:junit4]>  at 
> __randomizedtesting.SeedInfo.seed([26D2B352E9603950:BEF1D491B7168518]:0)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.tokenattributes.OffsetAttributeImpl.setOffset(OffsetAttributeImpl.java:45)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.ja.JapaneseTokenizer.incrementToken(JapaneseTokenizer.java:463)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:635)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:546)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:447)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:375)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.ja.TestExtendedMode.testRandomHugeStrings(TestExtendedMode.java:76)
> {noformat}
> I've seen other analyzer tests fail with similar exceptions.
> I dug in, and found that there's a bug in TreeMap.subMap, and it's easily 
> reproduced with a small test case, which I'll attach.  I'll also open an 
> issue with J9.
> I also found a workaround that seems to sidestep the bug for Lucene.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4735) IBM J9 JVM bug causes test failure in Kuromoji's TestExtended


 [ 
https://issues.apache.org/jira/browse/LUCENE-4735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless updated LUCENE-4735:
---

Summary: IBM J9 JVM bug causes test failure in Kuromoji's TestExtended  
(was: IBM J9 JVM causes test failure in Kuromoji's TestExtended)

> IBM J9 JVM bug causes test failure in Kuromoji's TestExtended
> -
>
> Key: LUCENE-4735
> URL: https://issues.apache.org/jira/browse/LUCENE-4735
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
> Attachments: LUCENE-4735.patch, TestTreeMap2.java
>
>
> Note that this is not a Lucene bug; it's a JVM bug, but I wanted to track it 
> in Lucene as well in case others hit it.
> I noticed this test frequently fails when running under IBM's J9 JVM (1.6.0) 
> ... and I finally tracked down the root cause and made a small test case, eg 
> on trunk, rev 1439839, if you run:
> {noformat}
>   ant test -Dtestcase=TestExtendedMode -Dtestmethod=testRandomHugeStrings 
> -Dtests.seed=26D2B352E9603950
> {noformat}
> it fails with this:
> {noformat}
> [junit4:junit4]> Throwable #1: java.lang.IllegalArgumentException: 
> startOffset must be non-negative, and endOffset must be >= startOffset, 
> startOffset=4272,endOffset=4271
> [junit4:junit4]>  at 
> __randomizedtesting.SeedInfo.seed([26D2B352E9603950:BEF1D491B7168518]:0)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.tokenattributes.OffsetAttributeImpl.setOffset(OffsetAttributeImpl.java:45)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.ja.JapaneseTokenizer.incrementToken(JapaneseTokenizer.java:463)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:635)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:546)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:447)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:375)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.ja.TestExtendedMode.testRandomHugeStrings(TestExtendedMode.java:76)
> {noformat}
> I've seen other analyzer tests fail with similar exceptions.
> I dug in, and found that there's a bug in TreeMap.subMap, and it's easily 
> reproduced with a small test case, which I'll attach.  I'll also open an 
> issue with J9.
> I also found a workaround that seems to sidestep the bug for Lucene.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4735) IBM J9 JVM causes test failure in Kuromoji's TestExtended


 [ 
https://issues.apache.org/jira/browse/LUCENE-4735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless updated LUCENE-4735:
---

Attachment: TestTreeMap2.java

Simple standalone test... if you run with J9 1.6, or at least with this version:
{noformat}
Java(TM) SE Runtime Environment (build pxa6460sr9fp2ifix-2011_05(SR9 
FP2+IV03622+IV02378+IZ99243+IZ97310+IV00707))
IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux amd64-64 
jvmxa6460sr9-2011_94827 (JIT enabled, AOT enabled)
J9VM - 2011_094827
JIT  - r9_20101028_17488ifx45
GC   - 20101027_AA)
JCL  - 20110727_04
{noformat}

Then the test will print:
{noformat}
FAILED: subMap.lastKey=4545 but should be 4576
{noformat}

But with Oracle Java 1.6 it prints "OK".

> IBM J9 JVM causes test failure in Kuromoji's TestExtended
> -
>
> Key: LUCENE-4735
> URL: https://issues.apache.org/jira/browse/LUCENE-4735
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
> Attachments: LUCENE-4735.patch, TestTreeMap2.java
>
>
> Note that this is not a Lucene bug; it's a JVM bug, but I wanted to track it 
> in Lucene as well in case others hit it.
> I noticed this test frequently fails when running under IBM's J9 JVM (1.6.0) 
> ... and I finally tracked down the root cause and made a small test case, eg 
> on trunk, rev 1439839, if you run:
> {noformat}
>   ant test -Dtestcase=TestExtendedMode -Dtestmethod=testRandomHugeStrings 
> -Dtests.seed=26D2B352E9603950
> {noformat}
> it fails with this:
> {noformat}
> [junit4:junit4]> Throwable #1: java.lang.IllegalArgumentException: 
> startOffset must be non-negative, and endOffset must be >= startOffset, 
> startOffset=4272,endOffset=4271
> [junit4:junit4]>  at 
> __randomizedtesting.SeedInfo.seed([26D2B352E9603950:BEF1D491B7168518]:0)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.tokenattributes.OffsetAttributeImpl.setOffset(OffsetAttributeImpl.java:45)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.ja.JapaneseTokenizer.incrementToken(JapaneseTokenizer.java:463)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:635)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:546)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:447)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:375)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.ja.TestExtendedMode.testRandomHugeStrings(TestExtendedMode.java:76)
> {noformat}
> I've seen other analyzer tests fail with similar exceptions.
> I dug in, and found that there's a bug in TreeMap.subMap, and it's easily 
> reproduced with a small test case, which I'll attach.  I'll also open an 
> issue with J9.
> I also found a workaround that seems to sidestep the bug for Lucene.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4735) IBM J9 JVM causes test failure in Kuromoji's TestExtended


 [ 
https://issues.apache.org/jira/browse/LUCENE-4735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless updated LUCENE-4735:
---

Attachment: LUCENE-4735.patch

Patch w/ workaround for Lucene.  If you use TreeMap.lowerEntry instead of 
TreeMap.subMap.lastKey then it seems to sidestep the issue.

> IBM J9 JVM causes test failure in Kuromoji's TestExtended
> -
>
> Key: LUCENE-4735
> URL: https://issues.apache.org/jira/browse/LUCENE-4735
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
> Attachments: LUCENE-4735.patch
>
>
> Note that this is not a Lucene bug; it's a JVM bug, but I wanted to track it 
> in Lucene as well in case others hit it.
> I noticed this test frequently fails when running under IBM's J9 JVM (1.6.0) 
> ... and I finally tracked down the root cause and made a small test case, eg 
> on trunk, rev 1439839, if you run:
> {noformat}
>   ant test -Dtestcase=TestExtendedMode -Dtestmethod=testRandomHugeStrings 
> -Dtests.seed=26D2B352E9603950
> {noformat}
> it fails with this:
> {noformat}
> [junit4:junit4]> Throwable #1: java.lang.IllegalArgumentException: 
> startOffset must be non-negative, and endOffset must be >= startOffset, 
> startOffset=4272,endOffset=4271
> [junit4:junit4]>  at 
> __randomizedtesting.SeedInfo.seed([26D2B352E9603950:BEF1D491B7168518]:0)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.tokenattributes.OffsetAttributeImpl.setOffset(OffsetAttributeImpl.java:45)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.ja.JapaneseTokenizer.incrementToken(JapaneseTokenizer.java:463)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:635)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:546)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:447)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:375)
> [junit4:junit4]>  at 
> org.apache.lucene.analysis.ja.TestExtendedMode.testRandomHugeStrings(TestExtendedMode.java:76)
> {noformat}
> I've seen other analyzer tests fail with similar exceptions.
> I dug in, and found that there's a bug in TreeMap.subMap, and it's easily 
> reproduced with a small test case, which I'll attach.  I'll also open an 
> issue with J9.
> I also found a workaround that seems to sidestep the bug for Lucene.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-4735) IBM J9 JVM causes test failure in Kuromoji's TestExtended

Michael McCandless created LUCENE-4735:
--

 Summary: IBM J9 JVM causes test failure in Kuromoji's TestExtended
 Key: LUCENE-4735
 URL: https://issues.apache.org/jira/browse/LUCENE-4735
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Michael McCandless


Note that this is not a Lucene bug; it's a JVM bug, but I wanted to track it in 
Lucene as well in case others hit it.

I noticed this test frequently fails when running under IBM's J9 JVM (1.6.0) 
... and I finally tracked down the root cause and made a small test case, eg on 
trunk, rev 1439839, if you run:

{noformat}
  ant test -Dtestcase=TestExtendedMode -Dtestmethod=testRandomHugeStrings 
-Dtests.seed=26D2B352E9603950
{noformat}

it fails with this:

{noformat}
[junit4:junit4]> Throwable #1: java.lang.IllegalArgumentException: 
startOffset must be non-negative, and endOffset must be >= startOffset, 
startOffset=4272,endOffset=4271
[junit4:junit4]>at 
__randomizedtesting.SeedInfo.seed([26D2B352E9603950:BEF1D491B7168518]:0)
[junit4:junit4]>at 
org.apache.lucene.analysis.tokenattributes.OffsetAttributeImpl.setOffset(OffsetAttributeImpl.java:45)
[junit4:junit4]>at 
org.apache.lucene.analysis.ja.JapaneseTokenizer.incrementToken(JapaneseTokenizer.java:463)
[junit4:junit4]>at 
org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:635)
[junit4:junit4]>at 
org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:546)
[junit4:junit4]>at 
org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:447)
[junit4:junit4]>at 
org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:375)
[junit4:junit4]>at 
org.apache.lucene.analysis.ja.TestExtendedMode.testRandomHugeStrings(TestExtendedMode.java:76)
{noformat}

I've seen other analyzer tests fail with similar exceptions.

I dug in, and found that there's a bug in TreeMap.subMap, and it's easily 
reproduced with a small test case, which I'll attach.  I'll also open an issue 
with J9.

I also found a workaround that seems to sidestep the bug for Lucene.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4300) Possible race condition in CoreContainer.getCore() when lazily loading cores.

2013-01-29 Thread Erick Erickson (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565773#comment-13565773
 ] 

Erick Erickson commented on SOLR-4300:
--

It turns out that this code was already done for SOLR-4196, of course that's 
not checked in yet. I'll see if I can do one of these in that code.

Erick

> Possible race condition in CoreContainer.getCore() when lazily loading cores.
> -
>
> Key: SOLR-4300
> URL: https://issues.apache.org/jira/browse/SOLR-4300
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 4.1, 5.0
>Reporter: Erick Erickson
>Assignee: Steve Rowe
>Priority: Blocker
> Fix For: 4.1, 5.0
>
> Attachments: SOLR-4300.patch
>
>
> Yonik pointed out in SOLR-1028 that there is a possible race condition here, 
> he's right not to my surprise. Calling it a "blocker" for now so we make a 
> decision on it rather than let it fall through the cracks. I should be able 
> to get a patch up tonight (Sunday).
> That said, there's potential here to introduce deadlocks, is it worth rushing 
> into 4.1?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [jira] [Commented] (SOLR-4300) Possible race condition in CoreContainer.getCore() when lazily loading cores.

2013-01-29 Thread Erick Erickson

It turns out I'd already done this in some (uncommitted) code for
SOLR-4196, I'll see what I can do there.


On Tue, Jan 29, 2013 at 2:49 PM, Yonik Seeley (JIRA)  wrote:
>
> [ 
> https://issues.apache.org/jira/browse/SOLR-4300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565715#comment-13565715
>  ]
>
> Yonik Seeley commented on SOLR-4300:
> 
>
> I haven't had a chance to check out the whole patch in context... but if we 
> get a chance to revisit at any point, it would probably be nicer to use a 
> mechanism other than sleep/poll.  Either wait/notify or Future would seem to 
> fit the bill.
>
>> Possible race condition in CoreContainer.getCore() when lazily loading cores.
>> -
>>
>> Key: SOLR-4300
>> URL: https://issues.apache.org/jira/browse/SOLR-4300
>> Project: Solr
>>  Issue Type: Bug
>>Affects Versions: 4.1, 5.0
>>Reporter: Erick Erickson
>>Assignee: Steve Rowe
>>Priority: Blocker
>> Fix For: 4.1, 5.0
>>
>> Attachments: SOLR-4300.patch
>>
>>
>> Yonik pointed out in SOLR-1028 that there is a possible race condition here, 
>> he's right not to my surprise. Calling it a "blocker" for now so we make a 
>> decision on it rather than let it fall through the cracks. I should be able 
>> to get a patch up tonight (Sunday).
>> That said, there's potential here to introduce deadlocks, is it worth 
>> rushing into 4.1?
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA administrators
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-3177) Excluding tagged filter in StatsComponent

2013-01-29 Thread Nikolai Luthman (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-3177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikolai Luthman updated SOLR-3177:
--

Attachment: (was: statsfilterexclude.patch)

> Excluding tagged filter in StatsComponent
> -
>
> Key: SOLR-3177
> URL: https://issues.apache.org/jira/browse/SOLR-3177
> Project: Solr
>  Issue Type: Improvement
>  Components: SearchComponents - other
>Affects Versions: 3.5, 3.6, 4.0-ALPHA, 4.1
>Reporter: Mathias H.
>Priority: Minor
>  Labels: localparams, stats, statscomponent
> Attachments: SOLR-3177.patch
>
>
> It would be useful to exclude the effects of some "fq" params from the set of 
> documents used to compute stats -- similar to 
> how you can exclude tagged filters when generating facet counts... 
> https://wiki.apache.org/solr/SimpleFacetParameters#Tagging_and_excluding_Filters
> So that it's possible to do something like this... 
> http://localhost:8983/solr/select?fq={!tag=priceFilter}price:[1 TO 
> 20]q=*:*&stats=true&stats.field={!ex=priceFilter}price 
> If you want to create a price slider this is very useful because then you can 
> filter the price ([1 TO 20) and nevertheless get the lower and upper bound of 
> the unfiltered price (min=0, max=100):
> {noformat}
> |<-[---]-->|
> $0 $1 $20$100
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-3177) Excluding tagged filter in StatsComponent

2013-01-29 Thread Nikolai Luthman (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-3177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikolai Luthman updated SOLR-3177:
--

Attachment: SOLR-3177.patch

Thanks for the comment Jan. I've updated the patch now to work against trunk, 
and also added a testcase that tests the !ex and !key functionality.

> Excluding tagged filter in StatsComponent
> -
>
> Key: SOLR-3177
> URL: https://issues.apache.org/jira/browse/SOLR-3177
> Project: Solr
>  Issue Type: Improvement
>  Components: SearchComponents - other
>Affects Versions: 3.5, 3.6, 4.0-ALPHA, 4.1
>Reporter: Mathias H.
>Priority: Minor
>  Labels: localparams, stats, statscomponent
> Attachments: SOLR-3177.patch
>
>
> It would be useful to exclude the effects of some "fq" params from the set of 
> documents used to compute stats -- similar to 
> how you can exclude tagged filters when generating facet counts... 
> https://wiki.apache.org/solr/SimpleFacetParameters#Tagging_and_excluding_Filters
> So that it's possible to do something like this... 
> http://localhost:8983/solr/select?fq={!tag=priceFilter}price:[1 TO 
> 20]q=*:*&stats=true&stats.field={!ex=priceFilter}price 
> If you want to create a price slider this is very useful because then you can 
> filter the price ([1 TO 20) and nevertheless get the lower and upper bound of 
> the unfiltered price (min=0, max=100):
> {noformat}
> |<-[---]-->|
> $0 $1 $20$100
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4373) In multicore, lib directives in solrconfig.xml cause conflict and clobber directives from earlier cores


[ 
https://issues.apache.org/jira/browse/SOLR-4373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565744#comment-13565744
 ] 

Alexandre Rafalovitch commented on SOLR-4373:
-

I am unable to get the problem go away by using coreLoadThreads="1":
  

Where do the cores store their libraries? Why should core2 see the core1's 
structure for the library at all? Do they get ClassLoaders mixed up somehow? Is 
there a way to catch the modifications to whatever that structure is and dump 
the stack trace at modification time? Just thinking aloud. 

> In multicore, lib directives in solrconfig.xml cause conflict and clobber 
> directives from earlier cores
> ---
>
> Key: SOLR-4373
> URL: https://issues.apache.org/jira/browse/SOLR-4373
> Project: Solr
>  Issue Type: Bug
>  Components: multicore
>Affects Versions: 4.1
>Reporter: Alexandre Rafalovitch
>Priority: Blocker
>  Labels: lib, multicore
> Fix For: 4.2, 4.1.1
>
> Attachments: multicore-bug.zip
>
>
> Having lib directives in solrconfig.xml seem to wipe out/override the 
> definitions in previous cores.
> The exception (for the earlier core) is:
>   at 
> org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:177)
>   at org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:369)
>   at org.apache.solr.schema.IndexSchema.(IndexSchema.java:113)
>   at 
> org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:1000)
>   at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1033)
>   at org.apache.solr.core.CoreContainer$3.call(CoreContainer.java:629)
>   at org.apache.solr.core.CoreContainer$3.call(CoreContainer.java:624)
>   at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>   at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>   at java.lang.Thread.run(Thread.java:722)
> Caused by: org.apache.solr.common.SolrException: Plugin init failure for 
> [schema.xml] analyzer/filter: Error loading class 
> 'solr.ICUFoldingFilterFactory'
>   at 
> org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:177)
>   at 
> org.apache.solr.schema.FieldTypePluginLoader.readAnalyzer(FieldTypePluginLoader.java:377)
>   at 
> org.apache.solr.schema.FieldTypePluginLoader.create(FieldTypePluginLoader.java:95)
>   at 
> org.apache.solr.schema.FieldTypePluginLoader.create(FieldTypePluginLoader.java:43)
>   at 
> org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:151)
> The full replication case is attached.
> If the SECOND core is turned off in solr.xml, the FIRST core loads just fine.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-4734) FastVectorHighlighter Overlapping Proximity Queries Do Not Highlight

2013-01-29 Thread Ryan Lauck (JIRA)

Ryan Lauck created LUCENE-4734:
--

 Summary: FastVectorHighlighter Overlapping Proximity Queries Do 
Not Highlight
 Key: LUCENE-4734
 URL: https://issues.apache.org/jira/browse/LUCENE-4734
 Project: Lucene - Core
  Issue Type: Bug
  Components: modules/highlighter
Affects Versions: 4.1, 4.0
Reporter: Ryan Lauck


If a proximity phrase query overlaps with any other query term it will not be 
highlighted.

Example Text:  A B C D E F G

Example Queries: 

"B E"~10 D
(only D will be highlighted)

"B E"~10 "C F"~10
(neither phrase will be highlighted)


This can be traced to the FieldPhraseList constructor's inner while loop. From 
the first example query, the first TermInfo popped off the stack will be "B". 
The second TermInfo will be "D" which will not be found in the submap for "B 
E"~10 and will trigger a failed match.

I wanted to report this issue before digging into a solution but my first 
thought is:

Add an additional int property to QueryPhraseMap to store the maximum possible 
phrase width for each term based on any proximity searches it is part of 
(defaulting to zero, in the above examples it would be 10). 

If a term is popped off the stack that is not a part of a proximity phrase 
being matched ( currMap.getTermMap(ti.getText()) == null ), it is added to a 
temporary list until either the longest possible phrase is successfully matched 
or a term is found outside the maximum possible phrase width.

After this search is complete, any non-matching terms that were added to the 
temporary list are pushed back onto the stack to be evaluated again and the 
temp list is cleared.

Hopefully this makes sense, and if nobody sees any obvious flaws I may try to 
create a patch.




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-2476) FieldAnalysisRequestHandler; Missing Tokenizer- & Filter-Options


 [ 
https://issues.apache.org/jira/browse/SOLR-2476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Matheis (steffkes) resolved SOLR-2476.
-

Resolution: Duplicate
  Assignee: Stefan Matheis (steffkes)

Not sure anymore why i've created two issues .. closing this on in favor of 
SOLR-2635

> FieldAnalysisRequestHandler; Missing Tokenizer- & Filter-Options
> 
>
> Key: SOLR-2476
> URL: https://issues.apache.org/jira/browse/SOLR-2476
> Project: Solr
>  Issue Type: Improvement
>  Components: web gui
>Reporter: Stefan Matheis (steffkes)
>Assignee: Stefan Matheis (steffkes)
>Priority: Minor
> Attachments: 110424_FieldAnalysisRequestHandler.xml
>
>
> Ryan mentioned in SOLR-2399 that the new analysis page is missing some 
> important parameters -- that's based on the fact that they are actually not 
> included in the Output (sample for "ViewSonic") attached.
> To see the difference, compare the [current 
> page|http://files.mathe.is/solr-admin/04_analysis_verbose_cur.png] with the 
> [new analysis-page|http://files.mathe.is/solr-admin/04_analysis_verbose.png].
> maybe it's possible that anyone could included them there? :)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2635) FieldAnalysisRequestHandler; Expose Filter- & Tokenizer-Settings


[ 
https://issues.apache.org/jira/browse/SOLR-2635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565720#comment-13565720
 ] 

Stefan Matheis (steffkes) commented on SOLR-2635:
-

[~thetaphi‍] quick reminder on this issue, if we could solve this, the Legend 
(proposed in SOLR-4378) might be able to include that somehow

> FieldAnalysisRequestHandler; Expose Filter- & Tokenizer-Settings
> 
>
> Key: SOLR-2635
> URL: https://issues.apache.org/jira/browse/SOLR-2635
> Project: Solr
>  Issue Type: Improvement
>  Components: Schema and Analysis, web gui
>Reporter: Stefan Matheis (steffkes)
>Assignee: Uwe Schindler
>Priority: Minor
>
> The [current/old Analysis 
> Page|http://files.mathe.is/solr-admin/04_analysis_verbose_cur.png] exposes 
> the Filter- & Tokenizer-Settings -- the FieldAnalysisRequestHandler not :/
> This Information is already available on the 
> [Schema-Browser|http://files.mathe.is/solr-admin/06_schema-browser.png] 
> (through LukeRequestHandler) - so we could load this in parallel and grab the 
> required informations .. but it would be easier if we could add this 
> Information, so that we have all relevant Information at one Place.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4377) making release tarballs identical to the release tags


[ 
https://issues.apache.org/jira/browse/SOLR-4377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565719#comment-13565719
 ] 

Roman Shaposhnik commented on SOLR-4377:


bq. I'm not sure that is the case. I think it comes down to the PMC for a 
particular project and what they decide to release.

Well, as an IPMC member I can tell you that this is something we strongly 
required from the poddling release. Perhaps you're right and somehow the TLPs 
are exempt from this requirement. I'd be happy to solicit the advice of ASF 
long timers if for nothing else but to make sure we treat poddling the same way 
we are going to treat them when they graduate.

> making release tarballs identical to the release tags
> -
>
> Key: SOLR-4377
> URL: https://issues.apache.org/jira/browse/SOLR-4377
> Project: Solr
>  Issue Type: Improvement
>  Components: scripts and tools
>Affects Versions: 4.0, 4.1
>Reporter: Roman Shaposhnik
> Attachments: diff.txt
>
>
> Now that we're integrating Solr with Hadoop via the Apache Bigtop project it 
> is a bit of a nuisance to our build system that the release tarballs don't 
> quite match the SVN tags. This is also something that is not quite ASF kosher 
> strictly speaking. 
> Would it be ok with a Solr community to add a comparison check between 
> release tarballs and SVN tags as part of the release process checklist?
> If you guys have a Wiki outlining how-to-release perhaps it needs to be 
> captured over there or just added to the process. Either way would be fine.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-4377) making release tarballs identical to the release tags


 [ 
https://issues.apache.org/jira/browse/SOLR-4377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Roman Shaposhnik updated SOLR-4377:
---

Attachment: diff.txt

> making release tarballs identical to the release tags
> -
>
> Key: SOLR-4377
> URL: https://issues.apache.org/jira/browse/SOLR-4377
> Project: Solr
>  Issue Type: Improvement
>  Components: scripts and tools
>Affects Versions: 4.0, 4.1
>Reporter: Roman Shaposhnik
> Attachments: diff.txt
>
>
> Now that we're integrating Solr with Hadoop via the Apache Bigtop project it 
> is a bit of a nuisance to our build system that the release tarballs don't 
> quite match the SVN tags. This is also something that is not quite ASF kosher 
> strictly speaking. 
> Would it be ok with a Solr community to add a comparison check between 
> release tarballs and SVN tags as part of the release process checklist?
> If you guys have a Wiki outlining how-to-release perhaps it needs to be 
> captured over there or just added to the process. Either way would be fine.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4300) Possible race condition in CoreContainer.getCore() when lazily loading cores.

2013-01-29 Thread Yonik Seeley (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565715#comment-13565715
 ] 

Yonik Seeley commented on SOLR-4300:


I haven't had a chance to check out the whole patch in context... but if we get 
a chance to revisit at any point, it would probably be nicer to use a mechanism 
other than sleep/poll.  Either wait/notify or Future would seem to fit the bill.

> Possible race condition in CoreContainer.getCore() when lazily loading cores.
> -
>
> Key: SOLR-4300
> URL: https://issues.apache.org/jira/browse/SOLR-4300
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 4.1, 5.0
>Reporter: Erick Erickson
>Assignee: Steve Rowe
>Priority: Blocker
> Fix For: 4.1, 5.0
>
> Attachments: SOLR-4300.patch
>
>
> Yonik pointed out in SOLR-1028 that there is a possible race condition here, 
> he's right not to my surprise. Calling it a "blocker" for now so we make a 
> decision on it rather than let it fall through the cracks. I should be able 
> to get a patch up tonight (Sunday).
> That said, there's potential here to introduce deadlocks, is it worth rushing 
> into 4.1?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4377) making release tarballs identical to the release tags


[ 
https://issues.apache.org/jira/browse/SOLR-4377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565713#comment-13565713
 ] 

Roman Shaposhnik commented on SOLR-4377:


Here's an example from the 4.1.0 release (diff is attached):
{noformat}
$ cd /tmp
$ tar xzvf solr-4.1.0-src.tgz
$ svn co http://svn.apache.org/repos/asf/lucene/dev/tags/lucene_solr_4_1_0
$ find . -name .svn -type d -exec rm -rf {} \;
$ diff -ruN solr-4.1.0 lucene_solr_4_1_0 | grep '^diff' | wc
71 284   12019
$ diff -ruN solr-4.1.0 lucene_solr_4_1_0  > diff.txt
{noformat}

So as you can see the diff is actually quite large (71 files total). My cursory 
glance shows that most of the deltas are trivial enough to be taken care of 
during the release process. What's more important (and what I'm asking for) is 
to have
 a release policy where a diff like I've mentioned would be part of the release 
check list.

> making release tarballs identical to the release tags
> -
>
> Key: SOLR-4377
> URL: https://issues.apache.org/jira/browse/SOLR-4377
> Project: Solr
>  Issue Type: Improvement
>  Components: scripts and tools
>Affects Versions: 4.0, 4.1
>Reporter: Roman Shaposhnik
> Attachments: diff.txt
>
>
> Now that we're integrating Solr with Hadoop via the Apache Bigtop project it 
> is a bit of a nuisance to our build system that the release tarballs don't 
> quite match the SVN tags. This is also something that is not quite ASF kosher 
> strictly speaking. 
> Would it be ok with a Solr community to add a comparison check between 
> release tarballs and SVN tags as part of the release process checklist?
> If you guys have a Wiki outlining how-to-release perhaps it needs to be 
> captured over there or just added to the process. Either way would be fine.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4377) making release tarballs identical to the release tags


[ 
https://issues.apache.org/jira/browse/SOLR-4377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565703#comment-13565703
 ] 

Mark Miller commented on SOLR-4377:
---

bq. This is also something that is not quite ASF kosher strictly speaking.

I'm not sure that is the case. I think it comes down to the PMC for a 
particular project and what they decide to release.

Regardless, I agree that it's a good goal.

Before we can put in a check for it, we have to fix the current issue though. 
Eg, what are all the differences?

I suspect the biggest issue is around svn subtitution variables (those $id's 
and such). That's the first thing I saw when I took a quick look.

There where a bunch of them in test config files - I just zapped them.

The more committers that start to use Git, the less these svn properties make 
sense (if they ever did - I've never liked them and I'm glad we have been 
removing them over time).

In any event:

+1 to finding the issues that make up the differences and addressing them
+1 to adding a check about tags matching src releases

I saw that right after the 4.1 tag, a couple quick commits went in - one for a 
spell fix in CHANGES and one a build issue - it made me unsure about whether we 
had released the tag, or a rev with the two fixes that happened right after. Of 
course I suspect we released the tag, but an automated smoke test would make me 
feel better about trusting that.

> making release tarballs identical to the release tags
> -
>
> Key: SOLR-4377
> URL: https://issues.apache.org/jira/browse/SOLR-4377
> Project: Solr
>  Issue Type: Improvement
>  Components: scripts and tools
>Affects Versions: 4.0, 4.1
>Reporter: Roman Shaposhnik
>
> Now that we're integrating Solr with Hadoop via the Apache Bigtop project it 
> is a bit of a nuisance to our build system that the release tarballs don't 
> quite match the SVN tags. This is also something that is not quite ASF kosher 
> strictly speaking. 
> Would it be ok with a Solr community to add a comparison check between 
> release tarballs and SVN tags as part of the release process checklist?
> If you guys have a Wiki outlining how-to-release perhaps it needs to be 
> captured over there or just added to the process. Either way would be fine.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4377) making release tarballs identical to the release tags


[ 
https://issues.apache.org/jira/browse/SOLR-4377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565695#comment-13565695
 ] 

Steve Rowe commented on SOLR-4377:
--

Hi Roman,

I'm not sure what you mean when you say that release tarballs don't match the 
SVN tags.  Please provide details.

> making release tarballs identical to the release tags
> -
>
> Key: SOLR-4377
> URL: https://issues.apache.org/jira/browse/SOLR-4377
> Project: Solr
>  Issue Type: Improvement
>  Components: scripts and tools
>Affects Versions: 4.0, 4.1
>Reporter: Roman Shaposhnik
>
> Now that we're integrating Solr with Hadoop via the Apache Bigtop project it 
> is a bit of a nuisance to our build system that the release tarballs don't 
> quite match the SVN tags. This is also something that is not quite ASF kosher 
> strictly speaking. 
> Would it be ok with a Solr community to add a comparison check between 
> release tarballs and SVN tags as part of the release process checklist?
> If you guys have a Wiki outlining how-to-release perhaps it needs to be 
> captured over there or just added to the process. Either way would be fine.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-4378) Admin UI - Legend for Analysis Screen

Stefan Matheis (steffkes) created SOLR-4378:
---

 Summary: Admin UI - Legend for Analysis Screen
 Key: SOLR-4378
 URL: https://issues.apache.org/jira/browse/SOLR-4378
 Project: Solr
  Issue Type: Improvement
  Components: Schema and Analysis, web gui
Affects Versions: 4.0
Reporter: Stefan Matheis (steffkes)
Priority: Minor


As we did in SOLR-3915 for a Legend on the Cloud Screen, perhaps another one 
might be helpful on the Analysis Screen. We're currently using an abbreviation 
which follows the Rule "All uppercase characters from the classname", which 
ends up with something like this:

{quote}LCF : org.apache.lucene.analysis.core.LowerCaseFilter
ENGTF : org.apache.lucene.analysis.ngram.EdgeNGramTokenFilter{quote}

Actually those abbreviations already show the complete classname (including the 
package) on mouseover, but an additional legend might help especially new users 
who might not identify LCF on the first look ..

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3866) CoreAdmin SWAP and RENAME need fixed/defined when using SolrCloud


[ 
https://issues.apache.org/jira/browse/SOLR-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565687#comment-13565687
 ] 

Mark Miller commented on SOLR-3866:
---

Seems we should disable these when using SolrCloud as a first step then - that 
only takes a moment. Then work on adding further functionality.

> CoreAdmin SWAP and RENAME need fixed/defined when using SolrCloud
> -
>
> Key: SOLR-3866
> URL: https://issues.apache.org/jira/browse/SOLR-3866
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 4.0-ALPHA, 4.0-BETA, 4.0
>Reporter: Hoss Man
>
> We need to define what the expected behavior of using the CoreAdminHandler's 
> SWAP and RENAME commands is if you are running in SolrCloud mode.
> At the moment, it seems to introduce a disconnect between which "collection" 
> the SolrCore thinks it's a part of (and what appears in the persisted 
> solr.xml) vs what collection ZooKeeper thinks the SolrCore(s) that were 
> swaped/renamed are a part of.  We should either "fix" this, or document it if 
> it as actually consistent and intentional for low level controls, or disable 
> commands like SWAP/RENAME in SolrCloud mode
> https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201209.mbox/%3CCALB4QrP2GZAwLeAiy%3DfcmOLYbc5r0i9Tp1DquyPS8mMJPwCgnw%40mail.gmail.com%3E

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-4371) Admin UI - Analysis Screen shows empty result


 [ 
https://issues.apache.org/jira/browse/SOLR-4371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Matheis (steffkes) updated SOLR-4371:


Attachment: SOLR-4371.patch

> Admin UI - Analysis Screen shows empty result
> -
>
> Key: SOLR-4371
> URL: https://issues.apache.org/jira/browse/SOLR-4371
> Project: Solr
>  Issue Type: Bug
>  Components: web gui
>Affects Versions: 4.1
>Reporter: Stefan Matheis (steffkes)
>Priority: Minor
> Attachments: SOLR-4371.patch
>
>
> Poking around for an change to FieldAnalysisRequestHandler hoss mentioned 
> SOLR-4336, which documents the cause of this.
> bq. 
> /solr/collection1/analysis/field?wt=json&analysis.showmatch=true&analysis.fieldvalue=foo&analysis.query=&analysis.fieldname=cat
> the output will contain this:
> {code}"field_names": {
>   "cat": {
>   "index": [...],
>   "query": [
>   "org.apache.solr.schema.FieldType$DefaultAnalyzer$1",
>   [
>   ]
>   ]
>   }
> }{code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3866) CoreAdmin SWAP and RENAME need fixed/defined when using SolrCloud

2013-01-29 Thread David Smiley (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565664#comment-13565664
 ] 

David Smiley commented on SOLR-3866:


I wanted to use the RENAME feature so that the core could match a predictable 
name for its shard no matter which machine is hosting the shard.  This is to 
work-around the fact that pivot faceting isn't distributed and so I query each 
shard and merge, as a work-around.  Boy was using the RENAME a mistake as my 
collection was hosed, especially after I tried to rename the leader's core.  My 
attempts to delete the collection via the API failed (or at least didn't 
succeed within the few minutes I gave it).  So I wound up reconfiguring all my 
collections and related config from scratch :-(   (on Solr 4.1)

> CoreAdmin SWAP and RENAME need fixed/defined when using SolrCloud
> -
>
> Key: SOLR-3866
> URL: https://issues.apache.org/jira/browse/SOLR-3866
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 4.0-ALPHA, 4.0-BETA, 4.0
>Reporter: Hoss Man
>
> We need to define what the expected behavior of using the CoreAdminHandler's 
> SWAP and RENAME commands is if you are running in SolrCloud mode.
> At the moment, it seems to introduce a disconnect between which "collection" 
> the SolrCore thinks it's a part of (and what appears in the persisted 
> solr.xml) vs what collection ZooKeeper thinks the SolrCore(s) that were 
> swaped/renamed are a part of.  We should either "fix" this, or document it if 
> it as actually consistent and intentional for low level controls, or disable 
> commands like SWAP/RENAME in SolrCloud mode
> https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201209.mbox/%3CCALB4QrP2GZAwLeAiy%3DfcmOLYbc5r0i9Tp1DquyPS8mMJPwCgnw%40mail.gmail.com%3E

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-4377) making release tarballs identical to the release tags

Roman Shaposhnik created SOLR-4377:
--

 Summary: making release tarballs identical to the release tags
 Key: SOLR-4377
 URL: https://issues.apache.org/jira/browse/SOLR-4377
 Project: Solr
  Issue Type: Improvement
  Components: scripts and tools
Affects Versions: 4.1, 4.0
Reporter: Roman Shaposhnik


Now that we're integrating Solr with Hadoop via the Apache Bigtop project it is 
a bit of a nuisance to our build system that the release tarballs don't quite 
match the SVN tags. This is also something that is not quite ASF kosher 
strictly speaking. 

Would it be ok with a Solr community to add a comparison check between release 
tarballs and SVN tags as part of the release process checklist?

If you guys have a Wiki outlining how-to-release perhaps it needs to be 
captured over there or just added to the process. Either way would be fine.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Stress test deadlocks

There is a lot of complicated interplay between locks in that area of the code 
- small changes can easily get you into trouble.

Can you modify your test to run on the code before your patch? That would help 
in telling if it's something existing or something your introducing.

I still plan on looking at this issue closer this week, so perhaps I can offer 
some more help soon.

- Mark

On Jan 29, 2013, at 1:26 PM, Erick Erickson  wrote:

> Two runs doth not a conclusion reach, but removing "synchronized" from:
> DefaultSolrCoreState.getIndexWriter (line 78)
> 
> let me run for an hour, at least twice. And my stress test succeeds,
> which fires up 15 indexing threads on 100 cores (transient core size
> is 20), indexes documents for an hour while another 15 threads fire
> off queries. At the end, it inspects each core to see if there are the
> expected number of documents.
> 
> But that's kinda a frail reed to pin my hopes on, these are
> notoriously hard to reproduce.
> 
> I'll set this up to run on an old machine for much longer later today.
> Does anyone who knows that code know whether I'm playing with fire? I
> haven't looked at the synchronization in that code to try to figure
> out the purpose, I'm hoping someone will glance at it and say "that's
> wrong".
> 
> I'll dig into it later and see how much I can figure out about whether it's 
> safe
> 
> FWIW,
> 
> On Tue, Jan 29, 2013 at 8:31 AM, Erick Erickson  
> wrote:
>> All:
>> 
>> As part of SOLR-4196, I'm opening and closing cores at a furious rate.
>> My tests are running for 20-40 minutes then locking up quite
>> regularly. Of course the first place I'm looking is my recent code,
>> since it has a bunch of synchronized blocks.
>> 
>> The deadlock is definitely happening at a call from the new code to
>> close a Solr core, so to really look at this anyone will need to get
>> the patch I'll put up in a minute. The deadlock trace is below.
>> 
>> But without going that far, I question whether it's really anything to
>> do with new synchronizations I'm doing or whether it's just something
>> that's been lurking for a while and I'm flushing out of the woodwork.
>> One of the deadlocked threads may be called form my code, but as far
>> as I can tell none of the actual synchronization objects I'm using are
>> held. I have the full jstack output if anyone needs it...
>> 
>> Of course I'll continue looking, but at a glance I'm wondering if this
>> code has really ever been stressed this way before or whether these
>> have existed for a while. All synchronization should be approached
>> with fear and loathing IMO.
>> 
>> One thread blocks at a synchronized method, but should this method
>> really be synchronized?
>> 
>> at 
>> org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:78)
>> (here's the method)  @Override
>>  public synchronized RefCounted getIndexWriter(SolrCore core)
>>  throws IOException {
>> 
>> and a little later in the method there's:
>>synchronized (writerPauseLock) {
>>  if (core == null) {
>> 
>> 
>> 
>> and the other thread blocks at:
>> 
>> at 
>> org.apache.solr.update.DirectUpdateHandler2.closeWriter(DirectUpdateHandler2.java:668),
>> (here's the method)
>>  // IndexWriterCloser interface method - called from 
>> solrCoreState.decref(this)
>>  @Override
>>  public void closeWriter(IndexWriter writer) throws IOException {
>>boolean clearRequestInfo = false;
>>commitLock.lock(); **locking here!
>>try {
>>  SolrQueryRequest req = new LocalSolrQueryRequest(core, new
>> ModifiableSolrParams());
>>  SolrQueryResponse rsp = new SolrQueryResponse();
>>  if (SolrRequestInfo.getRequestInfo() == null) {
>>clearRequestInfo = true;
>> 
>> 
>> 
>> 
>> Java stack information for the threads listed above:
>> ===
>> "commitScheduler-42617-thread-1":
>> at 
>> org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:78)
>> - waiting to lock <78b4aa518> (a org.apache.solr.update.DefaultSolrCoreState)
>> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1359)
>> at 
>> org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:561)
>> - locked <7884ca730> (a java.lang.Object)
>> at org.apache.solr.update.CommitTracker.run(CommitTracker.java:216)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>> at 
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
>> at 
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
>> at 
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> at 
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo

[jira] [Commented] (SOLR-4284) admin UI - make core list scrollable separate from the rest of the UI


[ 
https://issues.apache.org/jira/browse/SOLR-4284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565627#comment-13565627
 ] 

Commit Tag Bot commented on SOLR-4284:
--

[branch_4x commit] Stefan Matheis
http://svn.apache.org/viewvc?view=revision&revision=1440045

SOLR-4284: Admin UI - Fix Schema-Browser-Link on Analysis-Screen (merge 
r1440044)


> admin UI - make core list scrollable separate from the rest of the UI
> -
>
> Key: SOLR-4284
> URL: https://issues.apache.org/jira/browse/SOLR-4284
> Project: Solr
>  Issue Type: Improvement
>  Components: web gui
>Affects Versions: 4.0
>Reporter: Shawn Heisey
>Assignee: Stefan Matheis (steffkes)
>Priority: Minor
> Fix For: 4.2, 5.0
>
> Attachments: collection_chooser_ie9.png, SOLR-4284.patch, 
> SOLR-4284.patch, SOLR-4284.patch, SOLR-4284.patch, SOLR-4284.patch
>
>
> If the list of cores goes beyond the bottom of the browser window, it would 
> be nice if the user could scroll through that list with the mouse wheel 
> without making the main part of the window move.  This would particularly 
> important for someone with hundreds or thousands of cores.
> I'm not sure if this is even possible.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Stress test deadlocks

2013-01-29 Thread Erick Erickson

Two runs doth not a conclusion reach, but removing "synchronized" from:
DefaultSolrCoreState.getIndexWriter (line 78)

let me run for an hour, at least twice. And my stress test succeeds,
which fires up 15 indexing threads on 100 cores (transient core size
is 20), indexes documents for an hour while another 15 threads fire
off queries. At the end, it inspects each core to see if there are the
expected number of documents.

But that's kinda a frail reed to pin my hopes on, these are
notoriously hard to reproduce.

I'll set this up to run on an old machine for much longer later today.
Does anyone who knows that code know whether I'm playing with fire? I
haven't looked at the synchronization in that code to try to figure
out the purpose, I'm hoping someone will glance at it and say "that's
wrong".

I'll dig into it later and see how much I can figure out about whether it's safe

FWIW,

On Tue, Jan 29, 2013 at 8:31 AM, Erick Erickson  wrote:
> All:
>
> As part of SOLR-4196, I'm opening and closing cores at a furious rate.
> My tests are running for 20-40 minutes then locking up quite
> regularly. Of course the first place I'm looking is my recent code,
> since it has a bunch of synchronized blocks.
>
> The deadlock is definitely happening at a call from the new code to
> close a Solr core, so to really look at this anyone will need to get
> the patch I'll put up in a minute. The deadlock trace is below.
>
> But without going that far, I question whether it's really anything to
> do with new synchronizations I'm doing or whether it's just something
> that's been lurking for a while and I'm flushing out of the woodwork.
> One of the deadlocked threads may be called form my code, but as far
> as I can tell none of the actual synchronization objects I'm using are
> held. I have the full jstack output if anyone needs it...
>
> Of course I'll continue looking, but at a glance I'm wondering if this
> code has really ever been stressed this way before or whether these
> have existed for a while. All synchronization should be approached
> with fear and loathing IMO.
>
> One thread blocks at a synchronized method, but should this method
> really be synchronized?
>
> at 
> org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:78)
> (here's the method)  @Override
>   public synchronized RefCounted getIndexWriter(SolrCore core)
>   throws IOException {
>
> and a little later in the method there's:
> synchronized (writerPauseLock) {
>   if (core == null) {
>
>
>
> and the other thread blocks at:
>
> at 
> org.apache.solr.update.DirectUpdateHandler2.closeWriter(DirectUpdateHandler2.java:668),
> (here's the method)
>   // IndexWriterCloser interface method - called from 
> solrCoreState.decref(this)
>   @Override
>   public void closeWriter(IndexWriter writer) throws IOException {
> boolean clearRequestInfo = false;
> commitLock.lock(); **locking here!
> try {
>   SolrQueryRequest req = new LocalSolrQueryRequest(core, new
> ModifiableSolrParams());
>   SolrQueryResponse rsp = new SolrQueryResponse();
>   if (SolrRequestInfo.getRequestInfo() == null) {
> clearRequestInfo = true;
>
>
>
>
> Java stack information for the threads listed above:
> ===
> "commitScheduler-42617-thread-1":
> at 
> org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:78)
> - waiting to lock <78b4aa518> (a org.apache.solr.update.DefaultSolrCoreState)
> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1359)
> at 
> org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:561)
> - locked <7884ca730> (a java.lang.Object)
> at org.apache.solr.update.CommitTracker.run(CommitTracker.java:216)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
>
> *
> Other thread
> "qtp1401888126-32":
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for  <788d73208> (a
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:842)
> at 
> java.util.con

[jira] [Commented] (SOLR-4284) admin UI - make core list scrollable separate from the rest of the UI


[ 
https://issues.apache.org/jira/browse/SOLR-4284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565622#comment-13565622
 ] 

Commit Tag Bot commented on SOLR-4284:
--

[trunk commit] Stefan Matheis
http://svn.apache.org/viewvc?view=revision&revision=1440044

SOLR-4284: Admin UI - Fix Schema-Browser-Link on Analysis-Screen


> admin UI - make core list scrollable separate from the rest of the UI
> -
>
> Key: SOLR-4284
> URL: https://issues.apache.org/jira/browse/SOLR-4284
> Project: Solr
>  Issue Type: Improvement
>  Components: web gui
>Affects Versions: 4.0
>Reporter: Shawn Heisey
>Assignee: Stefan Matheis (steffkes)
>Priority: Minor
> Fix For: 4.2, 5.0
>
> Attachments: collection_chooser_ie9.png, SOLR-4284.patch, 
> SOLR-4284.patch, SOLR-4284.patch, SOLR-4284.patch, SOLR-4284.patch
>
>
> If the list of cores goes beyond the bottom of the browser window, it would 
> be nice if the user could scroll through that list with the mouse wheel 
> without making the main part of the window move.  This would particularly 
> important for someone with hundreds or thousands of cores.
> I'm not sure if this is even possible.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4376) dih.last_index_time has bad Date.toString() format during first delta import

2013-01-29 Thread James Dyer (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565611#comment-13565611
 ] 

James Dyer commented on SOLR-4376:
--

now that I've read your edited description, I think I get it.  If you run a 
delta even though a full import has never been run before, it trips up, right?

> dih.last_index_time has bad Date.toString() format during first delta import
> 
>
> Key: SOLR-4376
> URL: https://issues.apache.org/jira/browse/SOLR-4376
> Project: Solr
>  Issue Type: Bug
>  Components: contrib - DataImportHandler
>Affects Versions: 4.1
>Reporter: Sebastien Lorber
>Priority: Minor
>
> Hi
> In:
> org.apache.solr.handler.dataimport.DocBuilder#getVariableResolver
> {code:java} 
>   private static final Date EPOCH = new Date(0);
>   if (persistedProperties.get(LAST_INDEX_TIME) != null) {
> // String added to map
> indexerNamespace.put(LAST_INDEX_TIME, 
> persistedProperties.get(LAST_INDEX_TIME));
>   } else  {
> // Date added to map
> indexerNamespace.put(LAST_INDEX_TIME, EPOCH);
>   }
> {code} 
> When LAST_INDEX_TIME is found in the data-import.properties, the value in the 
> map is a String.
> When LAST_INDEX_TIME is not found, we use timestamp = 0, but the value is a 
> Date
> When using full-import it works fine because basically we don't need this 
> LAST_INDEX_TIME. 
> When doing delta import after a full import it also works fine.
> But when doing a first delta import on a clean configuration, without any 
> data-import.properties present, I have an SQL exception because of this query:
> SELECT xxx 
> FROM BATCH_JOB_EXECUTION yyy 
> WHERE last_updated > Thu Jan 01 01:00:00 CET 1970 
> I think in any case, the value associated to the key in the map must be 
> consistent and either be String or Date, but not both. 
> Personally I would expect it to be stored as String, and the EPOCH date being 
> formatted in the exact same format the date properties are persisted in the 
> file, which is:
> org.apache.solr.handler.dataimport.SimplePropertiesWriter#dateFormat
> This doesn't have a real impact on our code but it is just that an 
> integration test "test_delta_import_when_never_indexed" was unexpectedly 
> failing while all others were ok, after a Solr 1.4 to Solr 4.1 migration. 
> Thus it seems to be a minor regression.
> Thanks

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4376) dih.last_index_time has bad Date.toString() format during first delta import

2013-01-29 Thread James Dyer (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565607#comment-13565607
 ] 

James Dyer commented on SOLR-4376:
--

could explain what is happening to your dataimport.properties file and how it 
is wrong?

> dih.last_index_time has bad Date.toString() format during first delta import
> 
>
> Key: SOLR-4376
> URL: https://issues.apache.org/jira/browse/SOLR-4376
> Project: Solr
>  Issue Type: Bug
>  Components: contrib - DataImportHandler
>Affects Versions: 4.1
>Reporter: Sebastien Lorber
>Priority: Minor
>
> Hi
> In:
> org.apache.solr.handler.dataimport.DocBuilder#getVariableResolver
> {code:java} 
>   private static final Date EPOCH = new Date(0);
>   if (persistedProperties.get(LAST_INDEX_TIME) != null) {
> // String added to map
> indexerNamespace.put(LAST_INDEX_TIME, 
> persistedProperties.get(LAST_INDEX_TIME));
>   } else  {
> // Date added to map
> indexerNamespace.put(LAST_INDEX_TIME, EPOCH);
>   }
> {code} 
> When LAST_INDEX_TIME is found in the data-import.properties, the value in the 
> map is a String.
> When LAST_INDEX_TIME is not found, we use timestamp = 0, but the value is a 
> Date
> When using full-import it works fine because basically we don't need this 
> LAST_INDEX_TIME. 
> When doing delta import after a full import it also works fine.
> But when doing a first delta import on a clean configuration, without any 
> data-import.properties present, I have an SQL exception because of this query:
> SELECT xxx 
> FROM BATCH_JOB_EXECUTION yyy 
> WHERE last_updated > Thu Jan 01 01:00:00 CET 1970 
> I think in any case, the value associated to the key in the map must be 
> consistent and either be String or Date, but not both. 
> Personally I would expect it to be stored as String, and the EPOCH date being 
> formatted in the exact same format the date properties are persisted in the 
> file, which is:
> org.apache.solr.handler.dataimport.SimplePropertiesWriter#dateFormat
> This doesn't have a real impact on our code but it is just that an 
> integration test "test_delta_import_when_never_indexed" was unexpectedly 
> failing while all others were ok, after a Solr 1.4 to Solr 4.1 migration. 
> Thus it seems to be a minor regression.
> Thanks

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4733) Make CompressingTermVectorsFormat the new default term vectors format?


[ 
https://issues.apache.org/jira/browse/LUCENE-4733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565606#comment-13565606
 ] 

Robert Muir commented on LUCENE-4733:
-

I think its a good idea! About the corner cases:

If we are going to add new tests anyway, maybe we could fashion it as a base 
class like BaseDocValuesFormat/BasePostingsFormat?
Each TVFormat we have (4.0, simpletext, 4.2, etc) would extend this base class 
and just return getCodec().

Any tests in TestTermVectors/Reader/Writer today that are testing codecs (and 
not really indexwriter tests) could be folded in.

Of course if any tests are *compressing specific they can just stay in its 
folder too.


> Make CompressingTermVectorsFormat the new default term vectors format?
> --
>
> Key: LUCENE-4733
> URL: https://issues.apache.org/jira/browse/LUCENE-4733
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Adrien Grand
>Assignee: Adrien Grand
>Priority: Trivial
> Fix For: 4.2
>
>
> In LUCENE-4599, I wrote an alternate term vectors format which has a more 
> compact format, and I think it could replace the current 
> Lucene40TermVectorsFormat for the next (4.2) release?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4715) Add OrdinalPolicy.ALL_BUT_DIMENSION

2013-01-29 Thread Shai Erera (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-4715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565599#comment-13565599
 ] 

Shai Erera commented on LUCENE-4715:


Mike, those are great results (ALL vs ALL_BUT) ... amazing what a change of a 
flag can do :).

> Add OrdinalPolicy.ALL_BUT_DIMENSION
> ---
>
> Key: LUCENE-4715
> URL: https://issues.apache.org/jira/browse/LUCENE-4715
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/facet
>Reporter: Shai Erera
>Assignee: Shai Erera
> Attachments: LUCENE-4715.patch
>
>
> With the move of OrdinalPolicy to CategoryListParams, 
> NonTopLevelOrdinalPolicy was nuked. It might be good to restore it, as 
> another enum value of OrdinalPolicy.
> It's the same like ALL_PARENTS, only doesn't add the dimension ordinal, which 
> could save space as well as computation time. It's good for when you don't 
> care about the count of Date/, but only about its children counts.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4733) Make CompressingTermVectorsFormat the new default term vectors format?


[ 
https://issues.apache.org/jira/browse/LUCENE-4733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565596#comment-13565596
 ] 

Adrien Grand commented on LUCENE-4733:
--

It would be great if someone could set up a Jenkins job to run tests with 
-Dtests.codec=Compressing to catch bugs of this new format. Otherwise, the 
following points need to be addressed (I'll take care of these):
 - write file format docs,
 - wire this format in Lucene42Codec (I'll wait for the lucene4547 to be merged 
back into trunk for this one I think),
 - write tests for corner cases (small/large number of 
fields/terms/positions/offsets/payloads, etc.),
 - find good default params (chunk size / compression alg).

Feel free to let me know if you think it is a bad idea.

> Make CompressingTermVectorsFormat the new default term vectors format?
> --
>
> Key: LUCENE-4733
> URL: https://issues.apache.org/jira/browse/LUCENE-4733
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Adrien Grand
>Assignee: Adrien Grand
>Priority: Trivial
> Fix For: 4.2
>
>
> In LUCENE-4599, I wrote an alternate term vectors format which has a more 
> compact format, and I think it could replace the current 
> Lucene40TermVectorsFormat for the next (4.2) release?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-4733) Make CompressingTermVectorsFormat the new default term vectors format?

Adrien Grand created LUCENE-4733:


 Summary: Make CompressingTermVectorsFormat the new default term 
vectors format?
 Key: LUCENE-4733
 URL: https://issues.apache.org/jira/browse/LUCENE-4733
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Adrien Grand
Assignee: Adrien Grand
Priority: Trivial
 Fix For: 4.2


In LUCENE-4599, I wrote an alternate term vectors format which has a more 
compact format, and I think it could replace the current 
Lucene40TermVectorsFormat for the next (4.2) release?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4715) Add OrdinalPolicy.ALL_BUT_DIMENSION

2013-01-29 Thread Shai Erera (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-4715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565586#comment-13565586
 ] 

Shai Erera commented on LUCENE-4715:


First, look at the patch, there's a test for that :).

The way it works is that we now do per-dimension counts rollup. That is, say 
that you index the following dimensions under the same CLP: A (NO_PARENTS), B 
(ALL_PARENTS) and C (ALL_BUT_DIMENSION). When you ask to aggregate all of them 
then:

* StandardFacetsCollector does not work with NO_PARENTS (not sure if it throws 
a hard exception now, I'll check). So your only choice is 
CountingFacetsCollector.

* CountingFacetsCollector works as follows:
** Aggregates in a FixedBitSet (one per segment) the matching documents.
** It then traverses the counting list and counts all the ordinals that it 
finds.
** Then when it computes the facet results, it goes per FacetRequest:
*** If the FR.categoryPath was indexed with NO_PARENTS ("A" in our case), it 
rolls up its ordinals only, not caring about the huge counts[]. See Mike's test 
above, this general improves the process by a bit.
*** Otherwise, there's no more rollup needed. "B" would have a count too, while 
"C" count will be 0, and only its children will be counted.

Hope that explains it.

> Add OrdinalPolicy.ALL_BUT_DIMENSION
> ---
>
> Key: LUCENE-4715
> URL: https://issues.apache.org/jira/browse/LUCENE-4715
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/facet
>Reporter: Shai Erera
>Assignee: Shai Erera
> Attachments: LUCENE-4715.patch
>
>
> With the move of OrdinalPolicy to CategoryListParams, 
> NonTopLevelOrdinalPolicy was nuked. It might be good to restore it, as 
> another enum value of OrdinalPolicy.
> It's the same like ALL_PARENTS, only doesn't add the dimension ordinal, which 
> could save space as well as computation time. It's good for when you don't 
> care about the count of Date/, but only about its children counts.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4732) Test TermsEnum.seek on term vectors


 [ 
https://issues.apache.org/jira/browse/LUCENE-4732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adrien Grand updated LUCENE-4732:
-

Attachment: LUCENE-4732.patch

New patch that incorporates Robert's improvements and tests more corner cases.

> Test TermsEnum.seek on term vectors
> ---
>
> Key: LUCENE-4732
> URL: https://issues.apache.org/jira/browse/LUCENE-4732
> Project: Lucene - Core
>  Issue Type: Test
>Affects Versions: 4.1
>Reporter: Adrien Grand
>Assignee: Adrien Grand
> Fix For: 4.2, 5.0
>
> Attachments: LUCENE-4732_dueling.patch, LUCENE-4732.patch, 
> LUCENE-4732.patch
>
>
> We don't have test cases for this method and it looks broken with both 
> Lucene40TermVectorsFormat and CompressiongTermVectorsFormat.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4715) Add OrdinalPolicy.ALL_BUT_DIMENSION

2013-01-29 Thread Gilad Barkai (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-4715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565570#comment-13565570
 ] 

Gilad Barkai commented on LUCENE-4715:
--

How can a mess be avoided when allowing different OrdinalPolicies in the same 
CLP ?
There would be ordinals which has the parents, and ordinals that dont? How can 
the collector or aggregator know which ordinals should be delt with as without 
parents and which should not?



> Add OrdinalPolicy.ALL_BUT_DIMENSION
> ---
>
> Key: LUCENE-4715
> URL: https://issues.apache.org/jira/browse/LUCENE-4715
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/facet
>Reporter: Shai Erera
>Assignee: Shai Erera
> Attachments: LUCENE-4715.patch
>
>
> With the move of OrdinalPolicy to CategoryListParams, 
> NonTopLevelOrdinalPolicy was nuked. It might be good to restore it, as 
> another enum value of OrdinalPolicy.
> It's the same like ALL_PARENTS, only doesn't add the dimension ordinal, which 
> could save space as well as computation time. It's good for when you don't 
> care about the count of Date/, but only about its children counts.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-4376) dih.last_index_time has bad Date.toString() format during first delta import

2013-01-29 Thread Sebastien Lorber (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-4376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sebastien Lorber updated SOLR-4376:
---

Description: 
Hi

In:
org.apache.solr.handler.dataimport.DocBuilder#getVariableResolver

{code:java} 
  private static final Date EPOCH = new Date(0);

  if (persistedProperties.get(LAST_INDEX_TIME) != null) {
// String added to map
indexerNamespace.put(LAST_INDEX_TIME, 
persistedProperties.get(LAST_INDEX_TIME));
  } else  {
// Date added to map
indexerNamespace.put(LAST_INDEX_TIME, EPOCH);
  }
{code} 



When LAST_INDEX_TIME is found in the data-import.properties, the value in the 
map is a String.

When LAST_INDEX_TIME is not found, we use timestamp = 0, but the value is a Date






When using full-import it works fine because basically we don't need this 
LAST_INDEX_TIME. 

When doing delta import after a full import it also works fine.

But when doing a first delta import on a clean configuration, without any 
data-import.properties present, I have an SQL exception because of this query:
SELECT xxx 
FROM BATCH_JOB_EXECUTION yyy 
WHERE last_updated > Thu Jan 01 01:00:00 CET 1970 




I think in any case, the value associated to the key in the map must be 
consistent and either be String or Date, but not both. 

Personally I would expect it to be stored as String, and the EPOCH date being 
formatted in the exact same format the date properties are persisted in the 
file, which is:
org.apache.solr.handler.dataimport.SimplePropertiesWriter#dateFormat




This doesn't have a real impact on our code but it is just that an integration 
test "test_delta_import_when_never_indexed" was unexpectedly failing while all 
others were ok, after a Solr 1.4 to Solr 4.1 migration. 
Thus it seems to be a minor regression.



Thanks


  was:
Hi

In:
org.apache.solr.handler.dataimport.DocBuilder#getVariableResolver


  if (persistedProperties.get(LAST_INDEX_TIME) != null) {
indexerNamespace.put(LAST_INDEX_TIME, 
persistedProperties.get(LAST_INDEX_TIME));
  } else  {
// set epoch
indexerNamespace.put(LAST_INDEX_TIME, EPOCH);
  }



When LAST_INDEX_TIME is found in the data-import.properties, the value in the 
map is a String.

When LAST_INDEX_TIME is not found, we use timestamp = 0, but the value is a Date






When using full-import it works fine because basically we don't need this 
LAST_INDEX_TIME. 

When doing delta import after a full import it also works fine.

But when doing a first delta import on a clean configuration, without any 
data-import.properties present, I have an SQL exception because of this query:
SELECT xxx 
FROM BATCH_JOB_EXECUTION yyy 
WHERE last_updated > Thu Jan 01 01:00:00 CET 1970 




I think in any case, the value associated to the key in the map must be 
consistent and either be String or Date, but not both. 

Personally I would expect it to be stored as String, and the EPOCH date being 
formatted in the exact same format the date properties are persisted in the 
file, which is:
org.apache.solr.handler.dataimport.SimplePropertiesWriter#dateFormat




This doesn't have a real impact on our code but it is just that an integration 
test "test_delta_import_when_never_indexed" was unexpectedly failing while all 
others were ok, after a Solr 1.4 to Solr 4.1 migration. 
Thus it seems to be a minor regression.



Thanks



> dih.last_index_time has bad Date.toString() format during first delta import
> 
>
> Key: SOLR-4376
> URL: https://issues.apache.org/jira/browse/SOLR-4376
> Project: Solr
>  Issue Type: Bug
>  Components: contrib - DataImportHandler
>Affects Versions: 4.1
>Reporter: Sebastien Lorber
>Priority: Minor
>
> Hi
> In:
> org.apache.solr.handler.dataimport.DocBuilder#getVariableResolver
> {code:java} 
>   private static final Date EPOCH = new Date(0);
>   if (persistedProperties.get(LAST_INDEX_TIME) != null) {
> // String added to map
> indexerNamespace.put(LAST_INDEX_TIME, 
> persistedProperties.get(LAST_INDEX_TIME));
>   } else  {
> // Date added to map
> indexerNamespace.put(LAST_INDEX_TIME, EPOCH);
>   }
> {code} 
> When LAST_INDEX_TIME is found in the data-import.properties, the value in the 
> map is a String.
> When LAST_INDEX_TIME is not found, we use timestamp = 0, but the value is a 
> Date
> When using full-import it works fine because basically we don't need this 
> LAST_INDEX_TIME. 
> When doing delta import after a full import it also works fine.
> But when doing a first delta import on a clean configuration, without any 
> data-import.properties present, I have an SQL exception because of this query:
> SELECT xxx 
> FROM BATCH_JOB_EXECUTION yyy 
> WHERE last_updated > Thu Jan 01

[jira] [Commented] (SOLR-4311) Luke and Core admin ajax requests shouldn't be cached in admin


[ 
https://issues.apache.org/jira/browse/SOLR-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565563#comment-13565563
 ] 

Stefan Matheis (steffkes) commented on SOLR-4311:
-

bq. It can also be a problem when you compare the count of "Num Docs" on the 
main index page (/solr/#/CORE) vs the count on the core admin page 
(/solr/#/~cores/CORE). Consider that if you first visit the main index page, 
add and commit 100 docs, and then visit core admin the doc count will be off by 
100.

This was not related to any caching headers used or not used .. just to fact 
the the UI internally didn't fetch the informations a second time and reused 
what was available from the first call. this was changed while implementing 
SOLR-4306

but maybe it's general a good idea to disable caching for ajax-requests in the 
UI .. because it's a admin UI, not intended for public usage which may result 
in high traffic and you'd like to see that latest possible information.

> Luke and Core admin ajax requests shouldn't be cached in admin
> --
>
> Key: SOLR-4311
> URL: https://issues.apache.org/jira/browse/SOLR-4311
> Project: Solr
>  Issue Type: Bug
>  Components: web gui
>Affects Versions: 4.0
>Reporter: Chris Bleakley
>Priority: Minor
>
>   Although both the luke and core admin handlers set http caching to false in 
> the response headers** I believe the Cache-Control settings are ignored 
> during ajax requests in certain browsers. This can be a problem if you're 
> refreshing admin to get the latest do count. It can also be a problem when 
> you compare the count of "Num Docs" on the main index page (/solr/#/CORE) vs 
> the count on the core admin page (/solr/#/~cores/CORE). Consider that if you 
> first visit the main index page, add and commit 100 docs, and then visit core 
> admin the doc count will be off by 100.
>   
>   If this is an issue the ajax requests can explictly set caching to false ( 
> http://api.jquery.com/jQuery.ajax/#jQuery-ajax-settings ) ... for example, 
> inserting 'cache: false,' after line 91 here: 
> https://github.com/apache/lucene-solr/blob/branch_4x/solr/webapp/web/js/scripts/dashboard.js#L91
>   
>   ** 
> https://github.com/apache/lucene-solr/blob/branch_4x/solr/core/src/java/org/apache/solr/handler/admin/LukeRequestHandler.java#L167
>   ** 
> https://github.com/apache/lucene-solr/blob/branch_4x/solr/core/src/java/org/apache/solr/handler/admin/CoreAdminHandler.java#L216
>   
>   Tested using Chrome Version 24.0.1312.52

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4364) Admin UI - Locale based number formatting


[ 
https://issues.apache.org/jira/browse/SOLR-4364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565561#comment-13565561
 ] 

Commit Tag Bot commented on SOLR-4364:
--

[branch_4x commit] Stefan Matheis
http://svn.apache.org/viewvc?view=revision&revision=1440015

SOLR-4364: Admin UI - Locale based number formatting (merge r1440014)


> Admin UI - Locale based number formatting
> -
>
> Key: SOLR-4364
> URL: https://issues.apache.org/jira/browse/SOLR-4364
> Project: Solr
>  Issue Type: Improvement
>  Components: web gui
>Affects Versions: 4.1
>Reporter: Stefan Matheis (steffkes)
>Assignee: Stefan Matheis (steffkes)
>Priority: Minor
> Fix For: 4.2, 5.0
>
> Attachments: SOLR-4364.patch
>
>
> While fixing SOLR-4225, Hoss pointed out, that a apostrophe is not the 
> typical character for number formatting in english speaking countries - we 
> decided to go with an generic whitespace as separator in the first place.
> this issue tries to "fix" this, my first idea was using the browser-language 
> to decide how to format numbers - based on the fact, that if you're using a 
> english setup, you're able to recognize english number formatting, the same 
> goes for german and other languages 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-4376) dih.last_index_time has bad Date.toString() format during first delta import

2013-01-29 Thread Sebastien Lorber (JIRA)

Sebastien Lorber created SOLR-4376:
--

 Summary: dih.last_index_time has bad Date.toString() format during 
first delta import
 Key: SOLR-4376
 URL: https://issues.apache.org/jira/browse/SOLR-4376
 Project: Solr
  Issue Type: Bug
  Components: contrib - DataImportHandler
Affects Versions: 4.1
Reporter: Sebastien Lorber
Priority: Minor


Hi

In:
org.apache.solr.handler.dataimport.DocBuilder#getVariableResolver


  if (persistedProperties.get(LAST_INDEX_TIME) != null) {
indexerNamespace.put(LAST_INDEX_TIME, 
persistedProperties.get(LAST_INDEX_TIME));
  } else  {
// set epoch
indexerNamespace.put(LAST_INDEX_TIME, EPOCH);
  }



When LAST_INDEX_TIME is found in the data-import.properties, the value in the 
map is a String.

When LAST_INDEX_TIME is not found, we use timestamp = 0, but the value is a Date






When using full-import it works fine because basically we don't need this 
LAST_INDEX_TIME. 

When doing delta import after a full import it also works fine.

But when doing a first delta import on a clean configuration, without any 
data-import.properties present, I have an SQL exception because of this query:
SELECT xxx 
FROM BATCH_JOB_EXECUTION yyy 
WHERE last_updated > Thu Jan 01 01:00:00 CET 1970 




I think in any case, the value associated to the key in the map must be 
consistent and either be String or Date, but not both. 

Personally I would expect it to be stored as String, and the EPOCH date being 
formatted in the exact same format the date properties are persisted in the 
file, which is:
org.apache.solr.handler.dataimport.SimplePropertiesWriter#dateFormat




This doesn't have a real impact on our code but it is just that an integration 
test "test_delta_import_when_never_indexed" was unexpectedly failing while all 
others were ok, after a Solr 1.4 to Solr 4.1 migration. 
Thus it seems to be a minor regression.



Thanks


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4364) Admin UI - Locale based number formatting


[ 
https://issues.apache.org/jira/browse/SOLR-4364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565556#comment-13565556
 ] 

Commit Tag Bot commented on SOLR-4364:
--

[trunk commit] Stefan Matheis
http://svn.apache.org/viewvc?view=revision&revision=1440014

SOLR-4364: Admin UI - Locale based number formatting


> Admin UI - Locale based number formatting
> -
>
> Key: SOLR-4364
> URL: https://issues.apache.org/jira/browse/SOLR-4364
> Project: Solr
>  Issue Type: Improvement
>  Components: web gui
>Affects Versions: 4.1
>Reporter: Stefan Matheis (steffkes)
>Assignee: Stefan Matheis (steffkes)
>Priority: Minor
> Fix For: 4.2, 5.0
>
> Attachments: SOLR-4364.patch
>
>
> While fixing SOLR-4225, Hoss pointed out, that a apostrophe is not the 
> typical character for number formatting in english speaking countries - we 
> decided to go with an generic whitespace as separator in the first place.
> this issue tries to "fix" this, my first idea was using the browser-language 
> to decide how to format numbers - based on the fact, that if you're using a 
> english setup, you're able to recognize english number formatting, the same 
> goes for german and other languages 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-4364) Admin UI - Locale based number formatting


 [ 
https://issues.apache.org/jira/browse/SOLR-4364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Matheis (steffkes) resolved SOLR-4364.
-

   Resolution: Implemented
Fix Version/s: 5.0
   4.2

> Admin UI - Locale based number formatting
> -
>
> Key: SOLR-4364
> URL: https://issues.apache.org/jira/browse/SOLR-4364
> Project: Solr
>  Issue Type: Improvement
>  Components: web gui
>Affects Versions: 4.1
>Reporter: Stefan Matheis (steffkes)
>Assignee: Stefan Matheis (steffkes)
>Priority: Minor
> Fix For: 4.2, 5.0
>
> Attachments: SOLR-4364.patch
>
>
> While fixing SOLR-4225, Hoss pointed out, that a apostrophe is not the 
> typical character for number formatting in english speaking countries - we 
> decided to go with an generic whitespace as separator in the first place.
> this issue tries to "fix" this, my first idea was using the browser-language 
> to decide how to format numbers - based on the fact, that if you're using a 
> english setup, you're able to recognize english number formatting, the same 
> goes for german and other languages 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4375) Add config or schema option to turn off compressed stored fields in the Lucene 4.1 index format

2013-01-29 Thread Artem Lukanin (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565548#comment-13565548
 ] 

Artem Lukanin commented on SOLR-4375:
-

Yes, sure, here are my numbers for mean response time:
Solr 4.0 with termVectors: 4,47887323943662 ms
Solr 4.1 with termVectors: 7,23474178403756 ms
Solr 4.1 without termVectors: 8,0093896713615 ms

Tested during 2130 sec every time with full-import using DIH (after clearing 
data folder) before the tests.

master-slave configuration with 20 sec for replication, NRT enabled

I will try to make a custom 4.1 codec without compression and report results

> Add config or schema option to turn off compressed stored fields in the 
> Lucene 4.1 index format
> ---
>
> Key: SOLR-4375
> URL: https://issues.apache.org/jira/browse/SOLR-4375
> Project: Solr
>  Issue Type: Improvement
>Affects Versions: 4.1
>Reporter: Shawn Heisey
> Fix For: 4.2, 5.0
>
>
> There have been performance complaints on the mailing list about the default 
> compressed stored fields in the 4.1 index format.  There should be an option 
> to turn the compression off.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4732) Test TermsEnum.seek on term vectors


 [ 
https://issues.apache.org/jira/browse/LUCENE-4732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Muir updated LUCENE-4732:


Attachment: LUCENE-4732_dueling.patch

ah thanks... here's a patch to beef this up some.

> Test TermsEnum.seek on term vectors
> ---
>
> Key: LUCENE-4732
> URL: https://issues.apache.org/jira/browse/LUCENE-4732
> Project: Lucene - Core
>  Issue Type: Test
>Affects Versions: 4.1
>Reporter: Adrien Grand
>Assignee: Adrien Grand
> Fix For: 4.2, 5.0
>
> Attachments: LUCENE-4732_dueling.patch, LUCENE-4732.patch
>
>
> We don't have test cases for this method and it looks broken with both 
> Lucene40TermVectorsFormat and CompressiongTermVectorsFormat.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4732) Test TermsEnum.seek on term vectors


[ 
https://issues.apache.org/jira/browse/LUCENE-4732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565539#comment-13565539
 ] 

Adrien Grand commented on LUCENE-4732:
--

bq. assertTermsSeeking both shuffles its terms and reuses its termsenums.

The problem is that in the "for (BytesRef b : shuffledTests)" loop, it always 
calls "leftEnum = leftTerms.iterator(leftEnum);" before seeking, so the seek 
always goes forwards, never backward.

> Test TermsEnum.seek on term vectors
> ---
>
> Key: LUCENE-4732
> URL: https://issues.apache.org/jira/browse/LUCENE-4732
> Project: Lucene - Core
>  Issue Type: Test
>Affects Versions: 4.1
>Reporter: Adrien Grand
>Assignee: Adrien Grand
> Fix For: 4.2, 5.0
>
> Attachments: LUCENE-4732.patch
>
>
> We don't have test cases for this method and it looks broken with both 
> Lucene40TermVectorsFormat and CompressiongTermVectorsFormat.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4732) Test TermsEnum.seek on term vectors


[ 
https://issues.apache.org/jira/browse/LUCENE-4732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565534#comment-13565534
 ] 

Robert Muir commented on LUCENE-4732:
-

{quote}
I just checked the code: the problem is that it always seeks from a freshly 
instantiated terms enum. 
{quote}

I still dont get it... assertTermsSeeking both shuffles its terms and reuses 
its termsenums.

> Test TermsEnum.seek on term vectors
> ---
>
> Key: LUCENE-4732
> URL: https://issues.apache.org/jira/browse/LUCENE-4732
> Project: Lucene - Core
>  Issue Type: Test
>Affects Versions: 4.1
>Reporter: Adrien Grand
>Assignee: Adrien Grand
> Fix For: 4.2, 5.0
>
> Attachments: LUCENE-4732.patch
>
>
> We don't have test cases for this method and it looks broken with both 
> Lucene40TermVectorsFormat and CompressiongTermVectorsFormat.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4732) Test TermsEnum.seek on term vectors


[ 
https://issues.apache.org/jira/browse/LUCENE-4732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565532#comment-13565532
 ] 

Adrien Grand commented on LUCENE-4732:
--

bq. What is the problem with the checks in TestDuelingCodecs?

I just checked the code: the problem is that it always seeks from a freshly 
instantiated terms enum. With Lucene 4.0, the problem appears when the current 
position is after the seeked term.

> Test TermsEnum.seek on term vectors
> ---
>
> Key: LUCENE-4732
> URL: https://issues.apache.org/jira/browse/LUCENE-4732
> Project: Lucene - Core
>  Issue Type: Test
>Affects Versions: 4.1
>Reporter: Adrien Grand
>Assignee: Adrien Grand
> Fix For: 4.2, 5.0
>
> Attachments: LUCENE-4732.patch
>
>
> We don't have test cases for this method and it looks broken with both 
> Lucene40TermVectorsFormat and CompressiongTermVectorsFormat.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4732) Test TermsEnum.seek on term vectors


[ 
https://issues.apache.org/jira/browse/LUCENE-4732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565519#comment-13565519
 ] 

Robert Muir commented on LUCENE-4732:
-

{quote}
We don't have test cases for this method
{quote}

What is the problem with the checks in TestDuelingCodecs? Are the docs it 
indexes not large enough (maybe don't ever use vectors?)


> Test TermsEnum.seek on term vectors
> ---
>
> Key: LUCENE-4732
> URL: https://issues.apache.org/jira/browse/LUCENE-4732
> Project: Lucene - Core
>  Issue Type: Test
>Affects Versions: 4.1
>Reporter: Adrien Grand
>Assignee: Adrien Grand
> Fix For: 4.2, 5.0
>
> Attachments: LUCENE-4732.patch
>
>
> We don't have test cases for this method and it looks broken with both 
> Lucene40TermVectorsFormat and CompressiongTermVectorsFormat.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4375) Add config or schema option to turn off compressed stored fields in the Lucene 4.1 index format


[ 
https://issues.apache.org/jira/browse/SOLR-4375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565518#comment-13565518
 ] 

Mark Miller commented on SOLR-4375:
---

I think before we assume compression is even to blame, someone needs to perform 
their tests without it then compare their results to disabling it. If we can 
find a few of those data points, I think a simple option to disable it becomes 
very important and is a no brainer. You shouldn't have to mess with codecs to 
get back pre 4.1 speed. Without that evidence though, who knows anything. There 
are too many variables.

> Add config or schema option to turn off compressed stored fields in the 
> Lucene 4.1 index format
> ---
>
> Key: SOLR-4375
> URL: https://issues.apache.org/jira/browse/SOLR-4375
> Project: Solr
>  Issue Type: Improvement
>Affects Versions: 4.1
>Reporter: Shawn Heisey
> Fix For: 4.2, 5.0
>
>
> There have been performance complaints on the mailing list about the default 
> compressed stored fields in the 4.1 index format.  There should be an option 
> to turn the compression off.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4732) Test TermsEnum.seek on term vectors


 [ 
https://issues.apache.org/jira/browse/LUCENE-4732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adrien Grand updated LUCENE-4732:
-

Attachment: LUCENE-4732.patch

New patch: quick test and fix for both impls.

> Test TermsEnum.seek on term vectors
> ---
>
> Key: LUCENE-4732
> URL: https://issues.apache.org/jira/browse/LUCENE-4732
> Project: Lucene - Core
>  Issue Type: Test
>Affects Versions: 4.1
>Reporter: Adrien Grand
>Assignee: Adrien Grand
> Fix For: 4.2, 5.0
>
> Attachments: LUCENE-4732.patch
>
>
> We don't have test cases for this method and it looks broken with both 
> Lucene40TermVectorsFormat and CompressiongTermVectorsFormat.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-4732) Test TermsEnum.seek on term vectors