tiny patch for java7 on os X
i installed java7 on my os X... with the following build patch
pylucene seems to work fine (tests pass etc).
I think java7 is just pickier about -source/-target both being set for
jcc. And the extensions should use the same explicit source/target (or
the build can hit classfile version problems).
Index: extensions.xml
===
--- extensions.xml (revision 1425975)
+++ extensions.xml (working copy)
@@ -16,7 +16,7 @@
target name=compile
mkdir dir=${classes.dir}/
-javac srcdir=java destdir=${classes.dir} classpathref=classpath /
+javac srcdir=java destdir=${classes.dir}
classpathref=classpath source=1.5 target=1.5 /
/target
target name=jar depends=compile
Index: jcc/setup.py
===
--- jcc/setup.py(revision 1425975)
+++ jcc/setup.py(working copy)
@@ -149,7 +149,7 @@
LFLAGS['linux2'] = LFLAGS['linux2/%s' %(machine)]
JAVAC = {
-'darwin': ['javac', '-target', '1.5'],
+'darwin': ['javac', '-source', '1.5', '-target', '1.5'],
'ipod': ['jikes', '-cp', '/usr/share/classpath/glibj.zip'],
'linux2': ['javac'],
'sunos5': ['javac'],
http://pastebin.com/raw.php?i=qHpMw9Na
Re: [VOTE] Release PyLucene 3.6.2
On Dec 26, 2012, at 10:50, Robert Muir rcm...@gmail.com wrote:
On OS X, i had to run 'make test' twice. The first time, i got a strange
error:
Installed
/Users/rmuir/pylucene/pylucene-3.6.2-1/build/test/lucene-3.6.2-py2.7-macosx-10.7-x86_64.egg
Processing dependencies for lucene==3.6.2
Searching for lucene==3.6.2
Reading http://pypi.python.org/simple/lucene/
Couldn't find index page for 'lucene' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading http://pypi.python.org/simple/
No local packages or download links found for lucene==3.6.2
error: Could not find suitable distribution for
Requirement.parse('lucene==3.6.2')
make: *** [install-test] Error 1
I just ran it again and it worked...
Very strange. Why would it go out to pypi to install unrelated packages ?
Odd. Did you run just 'make' first before running 'make test' ? (my workflow).
Andi..
On Tue, Dec 25, 2012 at 6:56 PM, Andi Vajda va...@apache.org wrote:
The PyLucene 3.6.2-1 release tracking the recent release of Apache Lucene
3.6.2 is ready.
A release candidate is available from:
http://people.apache.org/~vajda/staging_area/
A list of changes in this release can be seen at:
http://svn.apache.org/repos/asf/lucene/pylucene/branches/pylucene_3_6/CHANGES
PyLucene 3.6.2 is built with JCC 2.15 included in these release artifacts:
http://svn.apache.org/repos/asf/lucene/pylucene/trunk/jcc/CHANGES
A list of Lucene Java changes can be seen at:
http://svn.apache.org/repos/asf/lucene/dev/tags/lucene_solr_3_6_2/lucene/CHANGES.txt
Please vote to release these artifacts as PyLucene 3.6.2-1.
Thanks !
Andi..
ps: the KEYS file for PyLucene release signing is at:
http://svn.apache.org/repos/asf/lucene/pylucene/dist/KEYS
http://people.apache.org/~vajda/staging_area/KEYS
pps: here is my +1
Re: [VOTE] Release PyLucene 3.6.2
On Wed, Dec 26, 2012 at 11:14 AM, Andi Vajda va...@apache.org wrote: Very strange. Why would it go out to pypi to install unrelated packages ? Odd. Did you run just 'make' first before running 'make test' ? (my workflow). I just tried make, followed by make test, and it worked fine. So I think i must have just tried 'make test' in one shot... must be a little build thing. doesn't seem like a blocker to me, just seemed a bit odd.
RE: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.6.0_37) - Build # 3421 - Failure!
Hi Mark, No, this is a default one with default multiplier, so just ant test. What's important to reproduce: - Number of JVMs because this dictates, how many tests are run inside one JVM: -Dtests.jvms=2. - And this is 32 bit Java! - more settings like used garbage collector in build description on website Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Mark Miller [mailto:markrmil...@gmail.com] Sent: Wednesday, December 26, 2012 3:22 AM To: dev@lucene.apache.org Subject: Re: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.6.0_37) - Build # 3421 - Failure! Is this one a nightly build? I can run it and look at it closely tomorrow. - Mark On Dec 25, 2012, at 6:04 PM, Uwe Schindler u...@thetaphi.de wrote: Can we add a finally/try block that catches permgen errors and calls System.halt (not exit)? I could add another extra allowance to the security manager, disallowing exits. But we should try to find the issue in the tests, maybe Mark has an idea. We have the heap dump readily available, but I don't have the tools to inspect it. Uwe Dawid Weiss dawid.we...@cs.put.poznan.pl schrieb: the test framework crashes somehow and does not respond anymore. I think I know exactly how it crashes -- there's not much mystery about this: once the permgen is exhausted OOM errors are thrown from tests; what happens then is these errors are caught and an attempt is made to serialize these errors to the master node. Unfortunately this process involves loading some classes that are not yet loaded and, since the permgen is already exhausted, everything goes insane (the thread apparently just silently quits; there are finally blocks that are never reached). Like I said -- I'll see what I can do about it but I don't have any optimistic feelings. This is really riding a critical edge and short of preallocating static data structures I don't see any way of implementing a clean solution for the problem. Dawid To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org -- Uwe Schindler H.-H.-Meier-Allee 63, 28213 Bremen http://www.thetaphi.de - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.6.0_37) - Build # 3421 - Failure!
I think it would be nice if Mike could add permgen pool stats (mx bean) to his charts :) This way we would see the average permgen usage over time -- it's easy to spot the regression then. Something to think of for the future. D. On Wed, Dec 26, 2012 at 9:02 AM, Uwe Schindler u...@thetaphi.de wrote: Hi Mark, No, this is a default one with default multiplier, so just ant test. What's important to reproduce: - Number of JVMs because this dictates, how many tests are run inside one JVM: -Dtests.jvms=2. - And this is 32 bit Java! - more settings like used garbage collector in build description on website Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Mark Miller [mailto:markrmil...@gmail.com] Sent: Wednesday, December 26, 2012 3:22 AM To: dev@lucene.apache.org Subject: Re: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.6.0_37) - Build # 3421 - Failure! Is this one a nightly build? I can run it and look at it closely tomorrow. - Mark On Dec 25, 2012, at 6:04 PM, Uwe Schindler u...@thetaphi.de wrote: Can we add a finally/try block that catches permgen errors and calls System.halt (not exit)? I could add another extra allowance to the security manager, disallowing exits. But we should try to find the issue in the tests, maybe Mark has an idea. We have the heap dump readily available, but I don't have the tools to inspect it. Uwe Dawid Weiss dawid.we...@cs.put.poznan.pl schrieb: the test framework crashes somehow and does not respond anymore. I think I know exactly how it crashes -- there's not much mystery about this: once the permgen is exhausted OOM errors are thrown from tests; what happens then is these errors are caught and an attempt is made to serialize these errors to the master node. Unfortunately this process involves loading some classes that are not yet loaded and, since the permgen is already exhausted, everything goes insane (the thread apparently just silently quits; there are finally blocks that are never reached). Like I said -- I'll see what I can do about it but I don't have any optimistic feelings. This is really riding a critical edge and short of preallocating static data structures I don't see any way of implementing a clean solution for the problem. Dawid To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org -- Uwe Schindler H.-H.-Meier-Allee 63, 28213 Bremen http://www.thetaphi.de - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.6.0_37) - Build # 3421 - Failure!
I started jhat on the machine: http://jenkins.sd-datasolutions.de:7000/ you can inspect the heap dump with it. The Jenkins build was made sticky, so it stays alive until I delete it. It is also nice to look to the heap dump with visualvm (shipped with JDK @ jvisualvm heapfile). You should use the same bitness and version of the JDK (32bit/jdk1.6.0_37) like used for this build after downloading the heapdump: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux/3421/artifact/heapdumps/java_pid13141.hprof Unfortunately I did not find a good tool to inspect permgen heap only (it contains loaded classes and interned strings). I checked the heapdump, we have no strange classloaders involved, all classes seem to be loaded by the standard app-classloader of the JDK and there are no duplicates (same class loaded multiple times by different class loaders). So SolrResourceLoader is not the bad guy as Robert and I expected as a first guess. Interestingly the dump has milltions of java.lang.String objects (which makes me wonder, I thought Lucene 4.x does no longer use Strings? - BUT Solr, 90% of all strings look like this: http://jenkins.sd-datasolutions.de:7000/object/0xdbf3d938, contents are similar to org.apache.solr.handler:type=RequestHandlerBase,scope=metrics-scope-22344,name=numTimeouts. The parent object are some huge HashMaps of com.yammer.metrics.core.MetricName instances). When looking at the MBean mess, it looks like: The whole VM is filled with MBean statistics (20% of the total heap!!!), just for statistics. It looks like the MBean server is not shut down correctly when the Solr instance shuts down, so it sums up while running tests, every new Solr instance adds new statistics to the huge MBean maps eating all the heap (and possibly permgen, because most strings may be interned)! This is a huge leak, we should fix this (or disable the whole useless MBean shit completely, at least for tests). Was this strange, never-seen package com.yammer.metrics introduced recently related to mbeans - or is zookeeper the bad guy? Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Mark Miller [mailto:markrmil...@gmail.com] Sent: Wednesday, December 26, 2012 3:22 AM To: dev@lucene.apache.org Subject: Re: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.6.0_37) - Build # 3421 - Failure! Is this one a nightly build? I can run it and look at it closely tomorrow. - Mark On Dec 25, 2012, at 6:04 PM, Uwe Schindler u...@thetaphi.de wrote: Can we add a finally/try block that catches permgen errors and calls System.halt (not exit)? I could add another extra allowance to the security manager, disallowing exits. But we should try to find the issue in the tests, maybe Mark has an idea. We have the heap dump readily available, but I don't have the tools to inspect it. Uwe Dawid Weiss dawid.we...@cs.put.poznan.pl schrieb: the test framework crashes somehow and does not respond anymore. I think I know exactly how it crashes -- there's not much mystery about this: once the permgen is exhausted OOM errors are thrown from tests; what happens then is these errors are caught and an attempt is made to serialize these errors to the master node. Unfortunately this process involves loading some classes that are not yet loaded and, since the permgen is already exhausted, everything goes insane (the thread apparently just silently quits; there are finally blocks that are never reached). Like I said -- I'll see what I can do about it but I don't have any optimistic feelings. This is really riding a critical edge and short of preallocating static data structures I don't see any way of implementing a clean solution for the problem. Dawid To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org -- Uwe Schindler H.-H.-Meier-Allee 63, 28213 Bremen http://www.thetaphi.de - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.6.0_37) - Build # 3421 - Failure!
Hi, I did further investigation (with jvisualvm - you can use any version, also the newest one with other bitness, it can always read the heap dump - I recommend the Java 7 64bit one, its most fancy and does not itself OOM): When looking at the MBean mess, it looks like: The whole VM is filled with MBean statistics (20% of the total heap!!!), just for statistics. It looks like the MBean server is not shut down correctly when the Solr instance shuts down, so it sums up while running tests, every new Solr instance adds new statistics to the huge MBean maps eating all the heap (and possibly permgen, because most strings may be interned)! This is a huge leak, we should fix this (or disable the whole useless MBean shit completely, at least for tests). Was this strange, never-seen package com.yammer.metrics introduced recently related to mbeans - or is zookeeper the bad guy? It's much worse: the String instances are only 20% of heap, but 26% are used for the ConcurrentHashMap.Entry classes holding those references and tons of ConcurrentHashMaps and com.yammer.metrics.core instances, eating up 60% of the total heap space (only reachable object, not those to be GCed). The big question: Do we need com.yammer.metrics.core (it is metrics-core-2.1.2.jar in solr/core/lib) at all? When was it introduced? Lucene 3.6 does not have it, neither Solr 4.0. It must be introduced recently - and eats up all memory. Uwe -Original Message- From: Mark Miller [mailto:markrmil...@gmail.com] Sent: Wednesday, December 26, 2012 3:22 AM To: dev@lucene.apache.org Subject: Re: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.6.0_37) - Build # 3421 - Failure! Is this one a nightly build? I can run it and look at it closely tomorrow. - Mark On Dec 25, 2012, at 6:04 PM, Uwe Schindler u...@thetaphi.de wrote: Can we add a finally/try block that catches permgen errors and calls System.halt (not exit)? I could add another extra allowance to the security manager, disallowing exits. But we should try to find the issue in the tests, maybe Mark has an idea. We have the heap dump readily available, but I don't have the tools to inspect it. Uwe Dawid Weiss dawid.we...@cs.put.poznan.pl schrieb: the test framework crashes somehow and does not respond anymore. I think I know exactly how it crashes -- there's not much mystery about this: once the permgen is exhausted OOM errors are thrown from tests; what happens then is these errors are caught and an attempt is made to serialize these errors to the master node. Unfortunately this process involves loading some classes that are not yet loaded and, since the permgen is already exhausted, everything goes insane (the thread apparently just silently quits; there are finally blocks that are never reached). Like I said -- I'll see what I can do about it but I don't have any optimistic feelings. This is really riding a critical edge and short of preallocating static data structures I don't see any way of implementing a clean solution for the problem. Dawid To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org -- Uwe Schindler H.-H.-Meier-Allee 63, 28213 Bremen http://www.thetaphi.de - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.6.0_37) - Build # 3421 - Failure!
It was introduced by: Revision: 1403555 Author: romseygeek Date: Montag, 29. Oktober 2012 23:13:03 Message: SOLR-1972: Add extra query stats to RequestHandler Modified : /lucene/dev/trunk/solr/CHANGES.txt Modified : /lucene/dev/trunk/solr/core/ivy.xml Modified : /lucene/dev/trunk/solr/core/src/java/org/apache/solr/handler/RequestHandlerBase.java Modified : /lucene/dev/trunk/solr/core/src/test/org/apache/solr/core/RequestHandlersTest.java Modified : /lucene/dev/trunk/solr/test-framework/src/java/org/apache/solr/SolrIgnoredThreadsFilter.java This one adds com.yammer.metrics.core to Solr and causes the huge memory leak. I'll reopen the issue. - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Uwe Schindler [mailto:u...@thetaphi.de] Sent: Wednesday, December 26, 2012 1:36 PM To: dev@lucene.apache.org Subject: RE: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.6.0_37) - Build # 3421 - Failure! Hi, I did further investigation (with jvisualvm - you can use any version, also the newest one with other bitness, it can always read the heap dump - I recommend the Java 7 64bit one, its most fancy and does not itself OOM): When looking at the MBean mess, it looks like: The whole VM is filled with MBean statistics (20% of the total heap!!!), just for statistics. It looks like the MBean server is not shut down correctly when the Solr instance shuts down, so it sums up while running tests, every new Solr instance adds new statistics to the huge MBean maps eating all the heap (and possibly permgen, because most strings may be interned)! This is a huge leak, we should fix this (or disable the whole useless MBean shit completely, at least for tests). Was this strange, never-seen package com.yammer.metrics introduced recently related to mbeans - or is zookeeper the bad guy? It's much worse: the String instances are only 20% of heap, but 26% are used for the ConcurrentHashMap.Entry classes holding those references and tons of ConcurrentHashMaps and com.yammer.metrics.core instances, eating up 60% of the total heap space (only reachable object, not those to be GCed). The big question: Do we need com.yammer.metrics.core (it is metrics-core- 2.1.2.jar in solr/core/lib) at all? When was it introduced? Lucene 3.6 does not have it, neither Solr 4.0. It must be introduced recently - and eats up all memory. Uwe -Original Message- From: Mark Miller [mailto:markrmil...@gmail.com] Sent: Wednesday, December 26, 2012 3:22 AM To: dev@lucene.apache.org Subject: Re: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.6.0_37) - Build # 3421 - Failure! Is this one a nightly build? I can run it and look at it closely tomorrow. - Mark On Dec 25, 2012, at 6:04 PM, Uwe Schindler u...@thetaphi.de wrote: Can we add a finally/try block that catches permgen errors and calls System.halt (not exit)? I could add another extra allowance to the security manager, disallowing exits. But we should try to find the issue in the tests, maybe Mark has an idea. We have the heap dump readily available, but I don't have the tools to inspect it. Uwe Dawid Weiss dawid.we...@cs.put.poznan.pl schrieb: the test framework crashes somehow and does not respond anymore. I think I know exactly how it crashes -- there's not much mystery about this: once the permgen is exhausted OOM errors are thrown from tests; what happens then is these errors are caught and an attempt is made to serialize these errors to the master node. Unfortunately this process involves loading some classes that are not yet loaded and, since the permgen is already exhausted, everything goes insane (the thread apparently just silently quits; there are finally blocks that are never reached). Like I said -- I'll see what I can do about it but I don't have any optimistic feelings. This is really riding a critical edge and short of preallocating static data structures I don't see any way of implementing a clean solution for the problem. Dawid To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org -- Uwe Schindler H.-H.-Meier-Allee 63, 28213 Bremen http://www.thetaphi.de - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail:
[jira] [Reopened] (SOLR-1972) Need additional query stats in admin interface - median, 95th and 99th percentile
[ https://issues.apache.org/jira/browse/SOLR-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler reopened SOLR-1972: - This commit causes a huge memory leak in Solr: The hole heap is filled with (interned) Strings, ConcurrentHashMap.Entry, and lots of class instances from com.yammer.metrics package. This causes the the recent permgen issues in running Solr tests. We should revert this and investigate how to remove the com.yammer.metrics package dependency (or make the stats cleaner). To me it looks like every query to solr creates a new entry in those huge maps, causing them to grow and grow and grow... There is also no cleanup that shuts down the MBean servers holding all those references on core shutdown. See: http://lucene.472066.n3.nabble.com/JENKINS-Lucene-Solr-trunk-Linux-32bit-jdk1-6-0-37-Build-3421-Failure-td4029050.html Need additional query stats in admin interface - median, 95th and 99th percentile - Key: SOLR-1972 URL: https://issues.apache.org/jira/browse/SOLR-1972 Project: Solr Issue Type: Improvement Components: web gui Affects Versions: 1.4 Reporter: Shawn Heisey Assignee: Alan Woodward Priority: Minor Fix For: 4.1, 5.0 Attachments: elyograg-1972-3.2.patch, elyograg-1972-3.2.patch, elyograg-1972-trunk.patch, elyograg-1972-trunk.patch, SOLR-1972-branch3x-url_pattern.patch, SOLR-1972-branch4x.patch, SOLR-1972-branch4x.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, solr1972-metricsregistry-branch4x-failure.log, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972-url_pattern.patch I would like to see more detailed query statistics from the admin GUI. This is what you can get now: requests : 809 errors : 0 timeouts : 0 totalTime : 70053 avgTimePerRequest : 86.59209 avgRequestsPerSecond : 0.8148785 I'd like to see more data on the time per request - median, 95th percentile, 99th percentile, and any other statistical function that makes sense to include. In my environment, the first bunch of queries after startup tend to take several seconds each. I find that the average value tends to be useless until it has several thousand queries under its belt and the caches are thoroughly warmed. The statistical functions I have mentioned would quickly eliminate the influence of those initial slow queries. The system will have to store individual data about each query. I don't know if this is something Solr does already. It would be nice to have a configurable count of how many of the most recent data points are kept, to control the amount of memory the feature uses. The default value could be something like 1024 or 4096. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-1972) Need additional query stats in admin interface - median, 95th and 99th percentile
[ https://issues.apache.org/jira/browse/SOLR-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539526#comment-13539526 ] Uwe Schindler edited comment on SOLR-1972 at 12/26/12 12:50 PM: This commit causes a huge memory leak in Solr: The whole heap (60% while running tests) is filled with (interned) Strings, ConcurrentHashMap.Entry, and lots of class instances from the com.yammer.metrics package. This causes the the recent permgen issues in running Solr tests. We should revert this and investigate how to remove the com.yammer.metrics package dependency (or make the stats cleaner). To me it looks like every query to solr creates a new entry in those huge maps, causing them to grow and grow and grow... There is also no cleanup that shuts down the MBean servers holding all those references on core shutdown. See: http://lucene.472066.n3.nabble.com/JENKINS-Lucene-Solr-trunk-Linux-32bit-jdk1-6-0-37-Build-3421-Failure-td4029050.html was (Author: thetaphi): This commit causes a huge memory leak in Solr: The hole heap is filled with (interned) Strings, ConcurrentHashMap.Entry, and lots of class instances from com.yammer.metrics package. This causes the the recent permgen issues in running Solr tests. We should revert this and investigate how to remove the com.yammer.metrics package dependency (or make the stats cleaner). To me it looks like every query to solr creates a new entry in those huge maps, causing them to grow and grow and grow... There is also no cleanup that shuts down the MBean servers holding all those references on core shutdown. See: http://lucene.472066.n3.nabble.com/JENKINS-Lucene-Solr-trunk-Linux-32bit-jdk1-6-0-37-Build-3421-Failure-td4029050.html Need additional query stats in admin interface - median, 95th and 99th percentile - Key: SOLR-1972 URL: https://issues.apache.org/jira/browse/SOLR-1972 Project: Solr Issue Type: Improvement Components: web gui Affects Versions: 1.4 Reporter: Shawn Heisey Assignee: Alan Woodward Priority: Minor Fix For: 4.1, 5.0 Attachments: elyograg-1972-3.2.patch, elyograg-1972-3.2.patch, elyograg-1972-trunk.patch, elyograg-1972-trunk.patch, SOLR-1972-branch3x-url_pattern.patch, SOLR-1972-branch4x.patch, SOLR-1972-branch4x.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, solr1972-metricsregistry-branch4x-failure.log, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972-url_pattern.patch I would like to see more detailed query statistics from the admin GUI. This is what you can get now: requests : 809 errors : 0 timeouts : 0 totalTime : 70053 avgTimePerRequest : 86.59209 avgRequestsPerSecond : 0.8148785 I'd like to see more data on the time per request - median, 95th percentile, 99th percentile, and any other statistical function that makes sense to include. In my environment, the first bunch of queries after startup tend to take several seconds each. I find that the average value tends to be useless until it has several thousand queries under its belt and the caches are thoroughly warmed. The statistical functions I have mentioned would quickly eliminate the influence of those initial slow queries. The system will have to store individual data about each query. I don't know if this is something Solr does already. It would be nice to have a configurable count of how many of the most recent data points are kept, to control the amount of memory the feature uses. The default value could be something like 1024 or 4096. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.6.0_37) - Build # 3421 - Failure!
Good bug hunting, Mr Holmes! Unfortunately I did not find a good tool to inspect permgen heap only (it contains loaded classes and interned strings). I Not sure but YourKit may be able to do that (in particular if you attach it using its own agent, not the default one). (and possibly permgen, because most strings may be interned)! Just a note -- interned string pool is no longer stored in permgen as of 1.7+ so they probably don't contribute to permgen exhaustion. Still not good that there's so many of them of course ;) Dawid - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.6.0_37) - Build # 3421 - Failure!
Just a note -- interned string pool is no longer stored in permgen as of 1.7+ so they probably don't contribute to permgen exhaustion. And I meant 1.7 build plans of course; the ones using 1.6 may be affected if these strings are indeed interned. D. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4191) Exceptions thrown when /admin/mbeans is accessed during update/commit
[
https://issues.apache.org/jira/browse/SOLR-4191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539538#comment-13539538
]
Mark Smith commented on SOLR-4191:
--
This happening for me as well on Solr 4.0. Interestingly enough, I'm not doing
any writes. I've got my solr app running for the past few weeks, and have not
done any changes (no updates, no inserts, only selects). Every few days I see
this exception, so I simply kill my server and restart, and everything is fine.
Please let me know if there is any test or more info that can help.
Exceptions thrown when /admin/mbeans is accessed during update/commit
-
Key: SOLR-4191
URL: https://issues.apache.org/jira/browse/SOLR-4191
Project: Solr
Issue Type: Bug
Affects Versions: 4.1
Environment: solr-impl 4.1-SNAPSHOT 1421496 - ncindex - 2012-12-13
14:56:25
Reporter: Shawn Heisey
Fix For: 4.1, 5.0
Attachments: solr-2012-12-14[1].log
I am getting the following exceptions in quick succession in the solr log
when /admin/mbeans is accessed at the moment that an update/commit is
happening:
ERROR - 2012-12-13 18:17:01.930; org.apache.solr.common.SolrException;
null:org.eclipse.jetty.io.EofException
ERROR - 2012-12-13 18:17:01.982; org.apache.solr.common.SolrException;
null:org.eclipse.jetty.io.EofException
WARN - 2012-12-13 18:17:01.984; org.eclipse.jetty.server.Response; Committed
before 500 {msg=Broken pipe,trace=org.eclipse.jetty.io.EofException
WARN - 2012-12-13 18:17:01.984; org.eclipse.jetty.servlet.ServletHandler;
/solr/s0live/admin/mbeans
java.lang.IllegalStateException: Committed
I will attach the full solr log. Before SOLR-4135 was fixed, I got a *lot*
of those exceptions, but these were far less common. Now these appear to be
the only thing I am getting in my logs, which log4j is logging at WARN.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-4191) Exceptions thrown when /admin/mbeans is accessed during update/commit
[
https://issues.apache.org/jira/browse/SOLR-4191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539538#comment-13539538
]
Mark Smith edited comment on SOLR-4191 at 12/26/12 1:55 PM:
This happening for me as well on Solr 4.0. Interestingly enough, I'm not doing
any writes. I've got my solr app running for the past few weeks, and have not
done any changes (no updates, no inserts, only selects). Every few days I see
this exception, so I simply kill my server and restart, and everything is fine.
Please let me know if anything I can do to help.
was (Author: marksmithurbana):
This happening for me as well on Solr 4.0. Interestingly enough, I'm not
doing any writes. I've got my solr app running for the past few weeks, and
have not done any changes (no updates, no inserts, only selects). Every few
days I see this exception, so I simply kill my server and restart, and
everything is fine. Please let me know if there is any test or more info that
can help.
Exceptions thrown when /admin/mbeans is accessed during update/commit
-
Key: SOLR-4191
URL: https://issues.apache.org/jira/browse/SOLR-4191
Project: Solr
Issue Type: Bug
Affects Versions: 4.1
Environment: solr-impl 4.1-SNAPSHOT 1421496 - ncindex - 2012-12-13
14:56:25
Reporter: Shawn Heisey
Fix For: 4.1, 5.0
Attachments: solr-2012-12-14[1].log
I am getting the following exceptions in quick succession in the solr log
when /admin/mbeans is accessed at the moment that an update/commit is
happening:
ERROR - 2012-12-13 18:17:01.930; org.apache.solr.common.SolrException;
null:org.eclipse.jetty.io.EofException
ERROR - 2012-12-13 18:17:01.982; org.apache.solr.common.SolrException;
null:org.eclipse.jetty.io.EofException
WARN - 2012-12-13 18:17:01.984; org.eclipse.jetty.server.Response; Committed
before 500 {msg=Broken pipe,trace=org.eclipse.jetty.io.EofException
WARN - 2012-12-13 18:17:01.984; org.eclipse.jetty.servlet.ServletHandler;
/solr/s0live/admin/mbeans
java.lang.IllegalStateException: Committed
I will attach the full solr log. Before SOLR-4135 was fixed, I got a *lot*
of those exceptions, but these were far less common. Now these appear to be
the only thing I am getting in my logs, which log4j is logging at WARN.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Cache stats and the log files
H. Well, I guess I can argue that this shouldn't be on by default, it's just a bit startling. Thanks for finding that! On a related note, setting the root logging level in the admin UI to finest changes all of the leaves to FINE level. But no changes happen in the logging (at least nothing goes out to the console). Doesn't seem right, anyone seen anything similar? On Tue, Dec 25, 2012 at 4:23 PM, Jack Krupansky j...@basetechnology.comwrote: SOLR-3157
[jira] [Commented] (SOLR-1972) Need additional query stats in admin interface - median, 95th and 99th percentile
[ https://issues.apache.org/jira/browse/SOLR-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539544#comment-13539544 ] Shawn Heisey commented on SOLR-1972: This is really useful to me, I'm very sad to learn it's causing problems. I haven't noticed anything in my own running of branch_4x, which runs for days at a time. I'm attempting to get some heap dumps so I can compare them. Based on what I know from my own experience, I don't think this is actually a leak, but even a minimal test JVM ends up with a lot of request handlers, each of which has these additional objects. As for seeing what looks like a leak with each query, after 1024 queries, the growth in non-garbage objects should stop, because it throws away an entry to make room for the new one. I wonder if there might be an easy way to make these new statistics optional in the request handler, so they do not cause memory problems with minimal test configs. Although it's really cool to be able to see detailed query statistics on the SolrInfoMBeanHandler, it's unnecessary. If it does get removed, I guess I'll have to go back to the previous version of the patch that only has additional statistics on qtime. I depend on these statistics - the average values otherwise available are nearly useless. I will attempt a patch. No guarantees about success! Need additional query stats in admin interface - median, 95th and 99th percentile - Key: SOLR-1972 URL: https://issues.apache.org/jira/browse/SOLR-1972 Project: Solr Issue Type: Improvement Components: web gui Affects Versions: 1.4 Reporter: Shawn Heisey Assignee: Alan Woodward Priority: Minor Fix For: 4.1, 5.0 Attachments: elyograg-1972-3.2.patch, elyograg-1972-3.2.patch, elyograg-1972-trunk.patch, elyograg-1972-trunk.patch, SOLR-1972-branch3x-url_pattern.patch, SOLR-1972-branch4x.patch, SOLR-1972-branch4x.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, solr1972-metricsregistry-branch4x-failure.log, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972-url_pattern.patch I would like to see more detailed query statistics from the admin GUI. This is what you can get now: requests : 809 errors : 0 timeouts : 0 totalTime : 70053 avgTimePerRequest : 86.59209 avgRequestsPerSecond : 0.8148785 I'd like to see more data on the time per request - median, 95th percentile, 99th percentile, and any other statistical function that makes sense to include. In my environment, the first bunch of queries after startup tend to take several seconds each. I find that the average value tends to be useless until it has several thousand queries under its belt and the caches are thoroughly warmed. The statistical functions I have mentioned would quickly eliminate the influence of those initial slow queries. The system will have to store individual data about each query. I don't know if this is something Solr does already. It would be nice to have a configurable count of how many of the most recent data points are kept, to control the amount of memory the feature uses. The default value could be something like 1024 or 4096. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.6.0_37) - Build # 3421 - Failure!
On Wed, Dec 26, 2012 at 4:13 AM, Dawid Weiss dawid.we...@cs.put.poznan.pl wrote: I think it would be nice if Mike could add permgen pool stats (mx bean) to his charts :) This way we would see the average permgen usage over time -- it's easy to spot the regression then. Something to think of for the future. Hmm this is interesting! Does this just amount to running top-level ant test -Dargs=-verbose:gc -XX:+PrintGCDetails -Dtests.jvms=1 and then parsing the GC stdout for the permgen usage? Mike McCandless http://blog.mikemccandless.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1972) Need additional query stats in admin interface - median, 95th and 99th percentile
[
https://issues.apache.org/jira/browse/SOLR-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539546#comment-13539546
]
Robert Muir commented on SOLR-1972:
---
{quote}
This commit causes a huge memory leak in Solr
...
We should revert this and investigate how to remove the com.yammer.metrics
package dependency (or make the stats cleaner). To me it looks like every query
to solr creates a new entry in those huge maps, causing them to grow and grow
and grow...
{quote}
+1
Maybe the stats have to be implemented by hand or something. I don't understand
why huge maps or string interning is necessary here.
Lets back out the change and implement it in a non-leaky way.
Need additional query stats in admin interface - median, 95th and 99th
percentile
-
Key: SOLR-1972
URL: https://issues.apache.org/jira/browse/SOLR-1972
Project: Solr
Issue Type: Improvement
Components: web gui
Affects Versions: 1.4
Reporter: Shawn Heisey
Assignee: Alan Woodward
Priority: Minor
Fix For: 4.1, 5.0
Attachments: elyograg-1972-3.2.patch, elyograg-1972-3.2.patch,
elyograg-1972-trunk.patch, elyograg-1972-trunk.patch,
SOLR-1972-branch3x-url_pattern.patch, SOLR-1972-branch4x.patch,
SOLR-1972-branch4x.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch,
SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch,
SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch,
solr1972-metricsregistry-branch4x-failure.log, SOLR-1972.patch,
SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972-url_pattern.patch
I would like to see more detailed query statistics from the admin GUI. This
is what you can get now:
requests : 809
errors : 0
timeouts : 0
totalTime : 70053
avgTimePerRequest : 86.59209
avgRequestsPerSecond : 0.8148785
I'd like to see more data on the time per request - median, 95th percentile,
99th percentile, and any other statistical function that makes sense to
include. In my environment, the first bunch of queries after startup tend to
take several seconds each. I find that the average value tends to be useless
until it has several thousand queries under its belt and the caches are
thoroughly warmed. The statistical functions I have mentioned would quickly
eliminate the influence of those initial slow queries.
The system will have to store individual data about each query. I don't know
if this is something Solr does already. It would be nice to have a
configurable count of how many of the most recent data points are kept, to
control the amount of memory the feature uses. The default value could be
something like 1024 or 4096.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.6.0_37) - Build # 3421 - Failure!
On Wed, Dec 26, 2012 at 5:23 AM, Dawid Weiss dawid.we...@cs.put.poznan.pl wrote: Good bug hunting, Mr Holmes! I think he should be promoted from policeman to detective! - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-4232) Make request handler metrics optional
Shawn Heisey created SOLR-4232: -- Summary: Make request handler metrics optional Key: SOLR-4232 URL: https://issues.apache.org/jira/browse/SOLR-4232 Project: Solr Issue Type: Bug Components: web gui Affects Versions: 4.1, 5.0 Reporter: Shawn Heisey Priority: Blocker Fix For: 4.1, 5.0 Uwe Schindler noticed what looked like a memory leak caused by the addition of SOLR-1972. I don't believe it's actually a leak, but the additional memory required does appear to be causing problems for Solr test JVMs. I think this is likely because there are a LOT of request handlers defined for even a very minimal test config, each of which ends up with the new objects. This is an attempt to provide an option for turning on the new statistics only when required. For most people, this will only be required for search handlers. If this is not successful at fixing the test problems, we can remove metrics with this issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.6.0_37) - Build # 3421 - Failure!
i was thinking about adding a hook to memorypool mx bean as far as i remember it does have 'peak' memory usage for permgen and it could be charted over time. parsing gc logs is kind of hard because they will differ depending on the vm and even the gc used. also, the gc logs are dumped to process descriptors and not to system.out/err streams and the runner will complain about unrecognized process output. this isn't crucial, but i think would be nice to add at some point. On Wednesday, December 26, 2012, Michael McCandless wrote: On Wed, Dec 26, 2012 at 4:13 AM, Dawid Weiss dawid.we...@cs.put.poznan.pl javascript:; wrote: I think it would be nice if Mike could add permgen pool stats (mx bean) to his charts :) This way we would see the average permgen usage over time -- it's easy to spot the regression then. Something to think of for the future. Hmm this is interesting! Does this just amount to running top-level ant test -Dargs=-verbose:gc -XX:+PrintGCDetails -Dtests.jvms=1 and then parsing the GC stdout for the permgen usage? Mike McCandless http://blog.mikemccandless.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org javascript:; For additional commands, e-mail: dev-h...@lucene.apache.org javascript:;
[jira] [Commented] (SOLR-1972) Need additional query stats in admin interface - median, 95th and 99th percentile
[ https://issues.apache.org/jira/browse/SOLR-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539553#comment-13539553 ] Mark Miller commented on SOLR-1972: --- bq. I would revert this patch for now +1 Need additional query stats in admin interface - median, 95th and 99th percentile - Key: SOLR-1972 URL: https://issues.apache.org/jira/browse/SOLR-1972 Project: Solr Issue Type: Improvement Components: web gui Affects Versions: 1.4 Reporter: Shawn Heisey Assignee: Alan Woodward Priority: Minor Fix For: 4.1, 5.0 Attachments: elyograg-1972-3.2.patch, elyograg-1972-3.2.patch, elyograg-1972-trunk.patch, elyograg-1972-trunk.patch, SOLR-1972-branch3x-url_pattern.patch, SOLR-1972-branch4x.patch, SOLR-1972-branch4x.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, solr1972-metricsregistry-branch4x-failure.log, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972-url_pattern.patch I would like to see more detailed query statistics from the admin GUI. This is what you can get now: requests : 809 errors : 0 timeouts : 0 totalTime : 70053 avgTimePerRequest : 86.59209 avgRequestsPerSecond : 0.8148785 I'd like to see more data on the time per request - median, 95th percentile, 99th percentile, and any other statistical function that makes sense to include. In my environment, the first bunch of queries after startup tend to take several seconds each. I find that the average value tends to be useless until it has several thousand queries under its belt and the caches are thoroughly warmed. The statistical functions I have mentioned would quickly eliminate the influence of those initial slow queries. The system will have to store individual data about each query. I don't know if this is something Solr does already. It would be nice to have a configurable count of how many of the most recent data points are kept, to control the amount of memory the feature uses. The default value could be something like 1024 or 4096. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1972) Need additional query stats in admin interface - median, 95th and 99th percentile
[
https://issues.apache.org/jira/browse/SOLR-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539552#comment-13539552
]
Uwe Schindler commented on SOLR-1972:
-
bq. As for seeing what looks like a leak with each query, after 1024 queries,
the growth in non-garbage objects should stop, because it throws away an entry
to make room for the new one.
That might be correct, the problem here is a leak, that might also affect
people reloading Solr in their App-Server. The problem is that somehow on
CoreShutdown the MBeans are not destroyed. Maybe this is fine how it is
implemented, but when the Solr core shuts down, all statistics data have to be
removed and nulled out or whatever.
I would revert this patch for now, until there is a better solution. Once you
have a new patch, that does not leak those objects on core shutdowns.
The horrible thing is that all those huge maps contains stupid strings, all
starting with some solr class name (a RequestHandler), followed by an id and
various other information. There are (in the dump analyzed with visualvm)
millions of String looking like that (OQL executed on heap dump):
{code:sql}
select count(heap.objects(java.lang.String),
'it.toString().startsWith(org.apache.solr.handler)')
{code}
This looks like a huge waste!
Need additional query stats in admin interface - median, 95th and 99th
percentile
-
Key: SOLR-1972
URL: https://issues.apache.org/jira/browse/SOLR-1972
Project: Solr
Issue Type: Improvement
Components: web gui
Affects Versions: 1.4
Reporter: Shawn Heisey
Assignee: Alan Woodward
Priority: Minor
Fix For: 4.1, 5.0
Attachments: elyograg-1972-3.2.patch, elyograg-1972-3.2.patch,
elyograg-1972-trunk.patch, elyograg-1972-trunk.patch,
SOLR-1972-branch3x-url_pattern.patch, SOLR-1972-branch4x.patch,
SOLR-1972-branch4x.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch,
SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch,
SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch,
solr1972-metricsregistry-branch4x-failure.log, SOLR-1972.patch,
SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972-url_pattern.patch
I would like to see more detailed query statistics from the admin GUI. This
is what you can get now:
requests : 809
errors : 0
timeouts : 0
totalTime : 70053
avgTimePerRequest : 86.59209
avgRequestsPerSecond : 0.8148785
I'd like to see more data on the time per request - median, 95th percentile,
99th percentile, and any other statistical function that makes sense to
include. In my environment, the first bunch of queries after startup tend to
take several seconds each. I find that the average value tends to be useless
until it has several thousand queries under its belt and the caches are
thoroughly warmed. The statistical functions I have mentioned would quickly
eliminate the influence of those initial slow queries.
The system will have to store individual data about each query. I don't know
if this is something Solr does already. It would be nice to have a
configurable count of how many of the most recent data points are kept, to
control the amount of memory the feature uses. The default value could be
something like 1024 or 4096.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1972) Need additional query stats in admin interface - median, 95th and 99th percentile
[ https://issues.apache.org/jira/browse/SOLR-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539554#comment-13539554 ] Robert Muir commented on SOLR-1972: --- I wonder if the permgen leak is related to the thread leaks caused by this metrics package (see SolrIgnoredThreadsFilter.class, it ignores threads leakd by the metrics package). In general we should try to prune this list and keep it minimal... at the moment i'm pretty suspicious of this third party library though: leaking threads, interning strings, etc. Need additional query stats in admin interface - median, 95th and 99th percentile - Key: SOLR-1972 URL: https://issues.apache.org/jira/browse/SOLR-1972 Project: Solr Issue Type: Improvement Components: web gui Affects Versions: 1.4 Reporter: Shawn Heisey Assignee: Alan Woodward Priority: Minor Fix For: 4.1, 5.0 Attachments: elyograg-1972-3.2.patch, elyograg-1972-3.2.patch, elyograg-1972-trunk.patch, elyograg-1972-trunk.patch, SOLR-1972-branch3x-url_pattern.patch, SOLR-1972-branch4x.patch, SOLR-1972-branch4x.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, solr1972-metricsregistry-branch4x-failure.log, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972-url_pattern.patch I would like to see more detailed query statistics from the admin GUI. This is what you can get now: requests : 809 errors : 0 timeouts : 0 totalTime : 70053 avgTimePerRequest : 86.59209 avgRequestsPerSecond : 0.8148785 I'd like to see more data on the time per request - median, 95th percentile, 99th percentile, and any other statistical function that makes sense to include. In my environment, the first bunch of queries after startup tend to take several seconds each. I find that the average value tends to be useless until it has several thousand queries under its belt and the caches are thoroughly warmed. The statistical functions I have mentioned would quickly eliminate the influence of those initial slow queries. The system will have to store individual data about each query. I don't know if this is something Solr does already. It would be nice to have a configurable count of how many of the most recent data points are kept, to control the amount of memory the feature uses. The default value could be something like 1024 or 4096. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-1972) Need additional query stats in admin interface - median, 95th and 99th percentile
[
https://issues.apache.org/jira/browse/SOLR-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539552#comment-13539552
]
Uwe Schindler edited comment on SOLR-1972 at 12/26/12 3:14 PM:
---
bq. As for seeing what looks like a leak with each query, after 1024 queries,
the growth in non-garbage objects should stop, because it throws away an entry
to make room for the new one.
That might be correct, the problem here is a leak, that might also affect
people reloading Solr in their App-Server. The problem is that somehow on
CoreShutdown the MBeans are not destroyed. Maybe this is fine how it is
implemented, but when the Solr core shuts down, all statistics data have to be
removed and nulled out or whatever.
I would revert this patch for now, until there is a better solution. Once you
have a new patch, that does not leak those objects on core shutdowns.
The horrible thing is that all those huge maps contains stupid strings, all
starting with some solr class name (a RequestHandler), followed by an id and
various other information. There are (in the dump analyzed with visualvm)
millions of String looking like that (OQL executed on heap dump):
{code:sql}
select count(heap.objects(java.lang.String),
'it.toString().startsWith(org.apache.solr.handler)')
{code}
returns: 305064 (to just show some of the string instances hanging around)
This looks like a huge waste!
was (Author: thetaphi):
bq. As for seeing what looks like a leak with each query, after 1024
queries, the growth in non-garbage objects should stop, because it throws away
an entry to make room for the new one.
That might be correct, the problem here is a leak, that might also affect
people reloading Solr in their App-Server. The problem is that somehow on
CoreShutdown the MBeans are not destroyed. Maybe this is fine how it is
implemented, but when the Solr core shuts down, all statistics data have to be
removed and nulled out or whatever.
I would revert this patch for now, until there is a better solution. Once you
have a new patch, that does not leak those objects on core shutdowns.
The horrible thing is that all those huge maps contains stupid strings, all
starting with some solr class name (a RequestHandler), followed by an id and
various other information. There are (in the dump analyzed with visualvm)
millions of String looking like that (OQL executed on heap dump):
{code:sql}
select count(heap.objects(java.lang.String),
'it.toString().startsWith(org.apache.solr.handler)')
{code}
This looks like a huge waste!
Need additional query stats in admin interface - median, 95th and 99th
percentile
-
Key: SOLR-1972
URL: https://issues.apache.org/jira/browse/SOLR-1972
Project: Solr
Issue Type: Improvement
Components: web gui
Affects Versions: 1.4
Reporter: Shawn Heisey
Assignee: Alan Woodward
Priority: Minor
Fix For: 4.1, 5.0
Attachments: elyograg-1972-3.2.patch, elyograg-1972-3.2.patch,
elyograg-1972-trunk.patch, elyograg-1972-trunk.patch,
SOLR-1972-branch3x-url_pattern.patch, SOLR-1972-branch4x.patch,
SOLR-1972-branch4x.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch,
SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch,
SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch,
solr1972-metricsregistry-branch4x-failure.log, SOLR-1972.patch,
SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972-url_pattern.patch
I would like to see more detailed query statistics from the admin GUI. This
is what you can get now:
requests : 809
errors : 0
timeouts : 0
totalTime : 70053
avgTimePerRequest : 86.59209
avgRequestsPerSecond : 0.8148785
I'd like to see more data on the time per request - median, 95th percentile,
99th percentile, and any other statistical function that makes sense to
include. In my environment, the first bunch of queries after startup tend to
take several seconds each. I find that the average value tends to be useless
until it has several thousand queries under its belt and the caches are
thoroughly warmed. The statistical functions I have mentioned would quickly
eliminate the influence of those initial slow queries.
The system will have to store individual data about each query. I don't know
if this is something Solr does already. It would be nice to have a
configurable count of how many of the most recent data points are kept, to
control the amount of memory the feature uses. The default value could be
something like 1024 or 4096.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA
[jira] [Commented] (SOLR-1972) Need additional query stats in admin interface - median, 95th and 99th percentile
[ https://issues.apache.org/jira/browse/SOLR-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539558#comment-13539558 ] Uwe Schindler commented on SOLR-1972: - [~romseygeek] shood revert, if he is not available for the next days, I will do it. OK? (I dont want to be again the bad guy reverting other's commits) Need additional query stats in admin interface - median, 95th and 99th percentile - Key: SOLR-1972 URL: https://issues.apache.org/jira/browse/SOLR-1972 Project: Solr Issue Type: Improvement Components: web gui Affects Versions: 1.4 Reporter: Shawn Heisey Assignee: Alan Woodward Priority: Minor Fix For: 4.1, 5.0 Attachments: elyograg-1972-3.2.patch, elyograg-1972-3.2.patch, elyograg-1972-trunk.patch, elyograg-1972-trunk.patch, SOLR-1972-branch3x-url_pattern.patch, SOLR-1972-branch4x.patch, SOLR-1972-branch4x.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, solr1972-metricsregistry-branch4x-failure.log, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972-url_pattern.patch I would like to see more detailed query statistics from the admin GUI. This is what you can get now: requests : 809 errors : 0 timeouts : 0 totalTime : 70053 avgTimePerRequest : 86.59209 avgRequestsPerSecond : 0.8148785 I'd like to see more data on the time per request - median, 95th percentile, 99th percentile, and any other statistical function that makes sense to include. In my environment, the first bunch of queries after startup tend to take several seconds each. I find that the average value tends to be useless until it has several thousand queries under its belt and the caches are thoroughly warmed. The statistical functions I have mentioned would quickly eliminate the influence of those initial slow queries. The system will have to store individual data about each query. I don't know if this is something Solr does already. It would be nice to have a configurable count of how many of the most recent data points are kept, to control the amount of memory the feature uses. The default value could be something like 1024 or 4096. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.6.0_37) - Build # 3421 - Failure!
Thanks. I am still working on getting the root of the permgen issue. I have no hammer to tell visualvm to get a list of interned strings. How to identify an interned string? Having fun with OQL (Object Query Language)... Dawid's note about Java 7 no longer puts interned strings into permgen seems hardened by the fact that only Java 6 builds have the permgen issue. Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Robert Muir [mailto:rcm...@gmail.com] Sent: Wednesday, December 26, 2012 3:33 PM To: dev@lucene.apache.org Subject: Re: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.6.0_37) - Build # 3421 - Failure! On Wed, Dec 26, 2012 at 5:23 AM, Dawid Weiss dawid.we...@cs.put.poznan.pl wrote: Good bug hunting, Mr Holmes! I think he should be promoted from policeman to detective! - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.6.0_37) - Build # 3421 - Failure!
I am still working on getting the root of the permgen issue. I have no hammer to tell visualvm to get a list of interned strings. How to identify an interned string? Having fun with OQL (Object Query Language)... Can you make that dump available? I'll check with yourkit. They have licenses for open source developers too and it's a great piece of software. Dawid's note about Java 7 no longer puts interned strings into permgen seems hardened by the fact that only Java 6 builds have the permgen issue. Yeah, this is a known gotcha. In Java8 they've removed the permgen entirely (or are planning to; I didn't track progress of it recently) so it'll get simpler. Dawid - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.6.0_37) - Build # 3421 - Failure!
It is there! Just scroll up in the thread and download it... I made the jenkins run sticky: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux/3421/artifact/heapdumps/java_pid13141.hprof - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: dawid.we...@gmail.com [mailto:dawid.we...@gmail.com] On Behalf Of Dawid Weiss Sent: Wednesday, December 26, 2012 4:53 PM To: dev@lucene.apache.org Subject: Re: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.6.0_37) - Build # 3421 - Failure! I am still working on getting the root of the permgen issue. I have no hammer to tell visualvm to get a list of interned strings. How to identify an interned string? Having fun with OQL (Object Query Language)... Can you make that dump available? I'll check with yourkit. They have licenses for open source developers too and it's a great piece of software. Dawid's note about Java 7 no longer puts interned strings into permgen seems hardened by the fact that only Java 6 builds have the permgen issue. Yeah, this is a known gotcha. In Java8 they've removed the permgen entirely (or are planning to; I didn't track progress of it recently) so it'll get simpler. Dawid - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-1972) Need additional query stats in admin interface - median, 95th and 99th percentile
[ https://issues.apache.org/jira/browse/SOLR-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir updated SOLR-1972: -- Attachment: leak.patch A possible patch to the leak (I cannot run solr tests *AT ALL* on my machine, because it doesnt like my network configuration, sorry, someone else must test). The root cause here is Metrics.newXXX. we should not use this, it enrolls entries into a global static MetricsRegistry. So its better to make a throwaway one. I think we should still just revert this issue for now. Just explaining where the leak comes from. Need additional query stats in admin interface - median, 95th and 99th percentile - Key: SOLR-1972 URL: https://issues.apache.org/jira/browse/SOLR-1972 Project: Solr Issue Type: Improvement Components: web gui Affects Versions: 1.4 Reporter: Shawn Heisey Assignee: Alan Woodward Priority: Minor Fix For: 4.1, 5.0 Attachments: elyograg-1972-3.2.patch, elyograg-1972-3.2.patch, elyograg-1972-trunk.patch, elyograg-1972-trunk.patch, leak.patch, SOLR-1972-branch3x-url_pattern.patch, SOLR-1972-branch4x.patch, SOLR-1972-branch4x.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, solr1972-metricsregistry-branch4x-failure.log, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972-url_pattern.patch I would like to see more detailed query statistics from the admin GUI. This is what you can get now: requests : 809 errors : 0 timeouts : 0 totalTime : 70053 avgTimePerRequest : 86.59209 avgRequestsPerSecond : 0.8148785 I'd like to see more data on the time per request - median, 95th percentile, 99th percentile, and any other statistical function that makes sense to include. In my environment, the first bunch of queries after startup tend to take several seconds each. I find that the average value tends to be useless until it has several thousand queries under its belt and the caches are thoroughly warmed. The statistical functions I have mentioned would quickly eliminate the influence of those initial slow queries. The system will have to store individual data about each query. I don't know if this is something Solr does already. It would be nice to have a configurable count of how many of the most recent data points are kept, to control the amount of memory the feature uses. The default value could be something like 1024 or 4096. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1972) Need additional query stats in admin interface - median, 95th and 99th percentile
[ https://issues.apache.org/jira/browse/SOLR-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539575#comment-13539575 ] Shawn Heisey commented on SOLR-1972: Robert, Alan did try creating a registry for each handler. It caused OOM errors when running Solr tests. See comments on 2012/10/26. Perhaps we could create a single static registry object that gets used by every handler to create the individual objects. I'll use your patch and attempt that. Need additional query stats in admin interface - median, 95th and 99th percentile - Key: SOLR-1972 URL: https://issues.apache.org/jira/browse/SOLR-1972 Project: Solr Issue Type: Improvement Components: web gui Affects Versions: 1.4 Reporter: Shawn Heisey Assignee: Alan Woodward Priority: Minor Fix For: 4.1, 5.0 Attachments: elyograg-1972-3.2.patch, elyograg-1972-3.2.patch, elyograg-1972-trunk.patch, elyograg-1972-trunk.patch, leak.patch, SOLR-1972-branch3x-url_pattern.patch, SOLR-1972-branch4x.patch, SOLR-1972-branch4x.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, solr1972-metricsregistry-branch4x-failure.log, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972-url_pattern.patch I would like to see more detailed query statistics from the admin GUI. This is what you can get now: requests : 809 errors : 0 timeouts : 0 totalTime : 70053 avgTimePerRequest : 86.59209 avgRequestsPerSecond : 0.8148785 I'd like to see more data on the time per request - median, 95th percentile, 99th percentile, and any other statistical function that makes sense to include. In my environment, the first bunch of queries after startup tend to take several seconds each. I find that the average value tends to be useless until it has several thousand queries under its belt and the caches are thoroughly warmed. The statistical functions I have mentioned would quickly eliminate the influence of those initial slow queries. The system will have to store individual data about each query. I don't know if this is something Solr does already. It would be nice to have a configurable count of how many of the most recent data points are kept, to control the amount of memory the feature uses. The default value could be something like 1024 or 4096. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1972) Need additional query stats in admin interface - median, 95th and 99th percentile
[ https://issues.apache.org/jira/browse/SOLR-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539576#comment-13539576 ] Robert Muir commented on SOLR-1972: --- I dont think we should have a single static: its no better than whats happening right now! its the problem! But for now, again I feel the right solution is to back out the change. there are too many resource leaks here. Besides the permgen leak, this thing leaks threads like a sieve. and this is currently just being ignored. Need additional query stats in admin interface - median, 95th and 99th percentile - Key: SOLR-1972 URL: https://issues.apache.org/jira/browse/SOLR-1972 Project: Solr Issue Type: Improvement Components: web gui Affects Versions: 1.4 Reporter: Shawn Heisey Assignee: Alan Woodward Priority: Minor Fix For: 4.1, 5.0 Attachments: elyograg-1972-3.2.patch, elyograg-1972-3.2.patch, elyograg-1972-trunk.patch, elyograg-1972-trunk.patch, leak.patch, SOLR-1972-branch3x-url_pattern.patch, SOLR-1972-branch4x.patch, SOLR-1972-branch4x.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, solr1972-metricsregistry-branch4x-failure.log, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972-url_pattern.patch I would like to see more detailed query statistics from the admin GUI. This is what you can get now: requests : 809 errors : 0 timeouts : 0 totalTime : 70053 avgTimePerRequest : 86.59209 avgRequestsPerSecond : 0.8148785 I'd like to see more data on the time per request - median, 95th percentile, 99th percentile, and any other statistical function that makes sense to include. In my environment, the first bunch of queries after startup tend to take several seconds each. I find that the average value tends to be useless until it has several thousand queries under its belt and the caches are thoroughly warmed. The statistical functions I have mentioned would quickly eliminate the influence of those initial slow queries. The system will have to store individual data about each query. I don't know if this is something Solr does already. It would be nice to have a configurable count of how many of the most recent data points are kept, to control the amount of memory the feature uses. The default value could be something like 1024 or 4096. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Reopened] (SOLR-4088) ZkController baseURL only get the host name not address, will occur can't get the right URL to do recovery.
[ https://issues.apache.org/jira/browse/SOLR-4088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller reopened SOLR-4088: --- Robert just mentioned to me that this can cause a problem in tests - sometimes a host cannot be found depending on your network connectivity and setup. Makes this kind of difficult to test...but most tests go could something else - we could explicitly set the host to 127.0...or we could fail back to the previous detection method if this new one fails? Or we could do both. ZkController baseURL only get the host name not address, will occur can't get the right URL to do recovery. -- Key: SOLR-4088 URL: https://issues.apache.org/jira/browse/SOLR-4088 Project: Solr Issue Type: Bug Components: SolrCloud Affects Versions: 4.0-ALPHA, 4.0-BETA, 4.0 Environment: Linux env Reporter: Raintung Li Assignee: Mark Miller Fix For: 4.1, 5.0 Attachments: patch-4088.txt In the ZKController.getHostAddress, it gets host name. In the linux, get hostname from /etc/sysconfig/network or /etc/hostname that maybe not config for ip address. Other server can't access it(http://hostname:port/..) correctly that cause recovery fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-4088) ZkController baseURL only get the host name not address, will occur can't get the right URL to do recovery.
[ https://issues.apache.org/jira/browse/SOLR-4088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539582#comment-13539582 ] Mark Miller edited comment on SOLR-4088 at 12/26/12 4:20 PM: - Robert just mentioned to me that this can cause a problem in tests - sometimes a host cannot be found depending on your network connectivity and setup. Makes this kind of difficult to test...but most tests could go to something else - we could explicitly set the host to 127.0...or we could fail back to the previous detection method if this new one fails? Or we could do both. I'm not sure what we should do to test this specific detection method though. was (Author: markrmil...@gmail.com): Robert just mentioned to me that this can cause a problem in tests - sometimes a host cannot be found depending on your network connectivity and setup. Makes this kind of difficult to test...but most tests go could something else - we could explicitly set the host to 127.0...or we could fail back to the previous detection method if this new one fails? Or we could do both. ZkController baseURL only get the host name not address, will occur can't get the right URL to do recovery. -- Key: SOLR-4088 URL: https://issues.apache.org/jira/browse/SOLR-4088 Project: Solr Issue Type: Bug Components: SolrCloud Affects Versions: 4.0-ALPHA, 4.0-BETA, 4.0 Environment: Linux env Reporter: Raintung Li Assignee: Mark Miller Fix For: 4.1, 5.0 Attachments: patch-4088.txt In the ZKController.getHostAddress, it gets host name. In the linux, get hostname from /etc/sysconfig/network or /etc/hostname that maybe not config for ip address. Other server can't access it(http://hostname:port/..) correctly that cause recovery fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4088) ZkController baseURL only get the host name not address, will occur can't get the right URL to do recovery.
[
https://issues.apache.org/jira/browse/SOLR-4088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539583#comment-13539583
]
Robert Muir commented on SOLR-4088:
---
I hit this when running tests:
{noformat}
[junit4:junit4] Caused by: java.net.UnknownHostException: xxx.yyy.zzz:
xxx.yyy.zzz: nodename nor servname provided, or not known
[junit4:junit4]at
java.net.InetAddress.getLocalHost(InetAddress.java:1438)
[junit4:junit4]at
org.apache.solr.cloud.ZkController.getHostAddress(ZkController.java:375)
[junit4:junit4]at
org.apache.solr.cloud.ZkController.init(ZkController.java:157)
[junit4:junit4]at
org.apache.solr.core.CoreContainer.initZooKeeper(CoreContainer.java:254)
[junit4:junit4]... 44 more
{noformat}
ZkController baseURL only get the host name not address, will occur can't get
the right URL to do recovery.
--
Key: SOLR-4088
URL: https://issues.apache.org/jira/browse/SOLR-4088
Project: Solr
Issue Type: Bug
Components: SolrCloud
Affects Versions: 4.0-ALPHA, 4.0-BETA, 4.0
Environment: Linux env
Reporter: Raintung Li
Assignee: Mark Miller
Fix For: 4.1, 5.0
Attachments: patch-4088.txt
In the ZKController.getHostAddress, it gets host name. In the linux, get
hostname from /etc/sysconfig/network or /etc/hostname that maybe not config
for ip address.
Other server can't access it(http://hostname:port/..) correctly that cause
recovery fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1972) Need additional query stats in admin interface - median, 95th and 99th percentile
[ https://issues.apache.org/jira/browse/SOLR-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539586#comment-13539586 ] Uwe Schindler commented on SOLR-1972: - MetricsRegistry *is* the singleton: With VisualVM you can get the retained size (Retained size of an object is its shallow size plus the shallow sizes of the objects that are accessible, directly or indirectly, only from this object. In other words, the retained size represents the amount of memory that will be freed by the garbage collector when this object is collected.) - o holy x-mas! One instance hold references to all metrics ever created. This is the leak, together with the threads spawned in its lifetime. Whenever you reload yur Solr instance you leak more objects (which is *the* classic classloader leak). Need additional query stats in admin interface - median, 95th and 99th percentile - Key: SOLR-1972 URL: https://issues.apache.org/jira/browse/SOLR-1972 Project: Solr Issue Type: Improvement Components: web gui Affects Versions: 1.4 Reporter: Shawn Heisey Assignee: Alan Woodward Priority: Minor Fix For: 4.1, 5.0 Attachments: elyograg-1972-3.2.patch, elyograg-1972-3.2.patch, elyograg-1972-trunk.patch, elyograg-1972-trunk.patch, leak.patch, SOLR-1972-branch3x-url_pattern.patch, SOLR-1972-branch4x.patch, SOLR-1972-branch4x.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, solr1972-metricsregistry-branch4x-failure.log, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972-url_pattern.patch I would like to see more detailed query statistics from the admin GUI. This is what you can get now: requests : 809 errors : 0 timeouts : 0 totalTime : 70053 avgTimePerRequest : 86.59209 avgRequestsPerSecond : 0.8148785 I'd like to see more data on the time per request - median, 95th percentile, 99th percentile, and any other statistical function that makes sense to include. In my environment, the first bunch of queries after startup tend to take several seconds each. I find that the average value tends to be useless until it has several thousand queries under its belt and the caches are thoroughly warmed. The statistical functions I have mentioned would quickly eliminate the influence of those initial slow queries. The system will have to store individual data about each query. I don't know if this is something Solr does already. It would be nice to have a configurable count of how many of the most recent data points are kept, to control the amount of memory the feature uses. The default value could be something like 1024 or 4096. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-4088) ZkController baseURL only get the host name not address, will occur can't get the right URL to do recovery.
[ https://issues.apache.org/jira/browse/SOLR-4088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir updated SOLR-4088: -- Attachment: SOLR-4088_unknown_host.patch Here's the patch i'm testing locally. The problem is when the host is configured as null, there are heuristics to guess how the host should be advertised. basically the patch just handles the situation where the first heuristical method (resolving the hostname from the system) fails, which can happen, e.g. if DNS is not functioning. In this case we just fall through to the logic below, of iterating interfaces. ZkController baseURL only get the host name not address, will occur can't get the right URL to do recovery. -- Key: SOLR-4088 URL: https://issues.apache.org/jira/browse/SOLR-4088 Project: Solr Issue Type: Bug Components: SolrCloud Affects Versions: 4.0-ALPHA, 4.0-BETA, 4.0 Environment: Linux env Reporter: Raintung Li Assignee: Mark Miller Fix For: 4.1, 5.0 Attachments: patch-4088.txt, SOLR-4088_unknown_host.patch In the ZKController.getHostAddress, it gets host name. In the linux, get hostname from /etc/sysconfig/network or /etc/hostname that maybe not config for ip address. Other server can't access it(http://hostname:port/..) correctly that cause recovery fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-4233) ScriptEngineTest can crash junit4 test runner
Robert Muir created SOLR-4233: - Summary: ScriptEngineTest can crash junit4 test runner Key: SOLR-4233 URL: https://issues.apache.org/jira/browse/SOLR-4233 Project: Solr Issue Type: Bug Components: Tests Environment: Mac os X 10.7.5. java version 1.7.0_09 Java(TM) SE Runtime Environment (build 1.7.0_09-b05) Java HotSpot(TM) 64-Bit Server VM (build 23.5-b02, mixed mode) Reporter: Robert Muir I think it might be a jvm bug... but im just uploading information since its wierd and annoying -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4233) ScriptEngineTest can crash junit4 test runner
[
https://issues.apache.org/jira/browse/SOLR-4233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539596#comment-13539596
]
Robert Muir commented on SOLR-4233:
---
Sorry, i hit two bugs at once.
Bug #1 is ScriptEngineTest adding new sysprops:
{noformat}
junit4:junit4] Suite: org.apache.solr.update.processor.ScriptEngineTest
[junit4:junit4] IGNOR/A 0.04s J3 | ScriptEngineTest.testJRuby
[junit4:junit4] Assumption #1: got: [null], expected: each not null
[junit4:junit4] 2 NOTE: test params are: codec=Lucene41: {},
sim=DefaultSimilarity, locale=it_CH, timezone=America/Inuvik
[junit4:junit4] 2 NOTE: Mac OS X 10.7.5 x86_64/Oracle Corporation 1.7.0_09
(64-bit)/cpus=8,threads=1,free=74721592,total=263520256
[junit4:junit4] 2 NOTE: All tests run in this JVM: [TestSort,
DistributedTermsComponentTest, TestStressReorder, TestJmxIntegration,
TestRangeQuery, DocumentAnalysisRequestHandlerTest, SolrCmdDistributorTest,
ZkControllerTest, ZkSolrClientTest, TestSystemIdResolver, ScriptEngineTest]
[junit4:junit4] 2 NOTE: reproduce with: ant test
-Dtestcase=ScriptEngineTest -Dtests.seed=7BF9C084407FF39D -Dtests.slow=true
-Dtests.locale=it_CH -Dtests.timezone=America/Inuvik
-Dtests.file.encoding=ISO-8859-1
[junit4:junit4] ERROR 0.00s J3 | ScriptEngineTest (suite)
[junit4:junit4] Throwable #1: java.lang.AssertionError: System properties
invariant violated.
[junit4:junit4] New keys:
[junit4:junit4] sun.awt.enableExtraMouseButtons=true
[junit4:junit4] sun.font.fontmanager=sun.font.CFontManager
{noformat}
Bug #2 is the crazy malloc error:
{noformat}
[junit4:junit4] HEARTBEAT J2 PID(67321@localhost): 2012-12-26T08:47:15, stalled
for 63.2s at: SyncSliceTest.testDistribSearch
[junit4:junit4] JVM J0: stderr was not empty, see:
/Users/rmuir/workspace/lucene-trunk/solr/build/solr-core/test/temp/junit4-J0-20121226_084402_963.syserr
[junit4:junit4] JVM J0: stderr (verbatim)
[junit4:junit4] java(67319,0x10e6e1000) malloc: *** error for object
0x10e6cfc92: pointer being freed was not allocated
[junit4:junit4] *** set a breakpoint in malloc_error_break to debug
[junit4:junit4] JVM J0: EOF
{noformat}
ScriptEngineTest can crash junit4 test runner
-
Key: SOLR-4233
URL: https://issues.apache.org/jira/browse/SOLR-4233
Project: Solr
Issue Type: Bug
Components: Tests
Environment: Mac os X 10.7.5.
java version 1.7.0_09
Java(TM) SE Runtime Environment (build 1.7.0_09-b05)
Java HotSpot(TM) 64-Bit Server VM (build 23.5-b02, mixed mode)
Reporter: Robert Muir
I think it might be a jvm bug... but im just uploading information since its
wierd and annoying
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-4233) Test runner problems on OS X / java 7
[ https://issues.apache.org/jira/browse/SOLR-4233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir updated SOLR-4233: -- Summary: Test runner problems on OS X / java 7 (was: ScriptEngineTest can crash junit4 test runner) updating test description to make more sense. Test runner problems on OS X / java 7 - Key: SOLR-4233 URL: https://issues.apache.org/jira/browse/SOLR-4233 Project: Solr Issue Type: Bug Components: Tests Environment: Mac os X 10.7.5. java version 1.7.0_09 Java(TM) SE Runtime Environment (build 1.7.0_09-b05) Java HotSpot(TM) 64-Bit Server VM (build 23.5-b02, mixed mode) Reporter: Robert Muir I think it might be a jvm bug... but im just uploading information since its wierd and annoying -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4233) Test runner problems on OS X / java 7
[ https://issues.apache.org/jira/browse/SOLR-4233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539602#comment-13539602 ] Uwe Schindler commented on SOLR-4233: - This is the well-known @UweSays bug at https://twitter.com/UweSays/status/260487231880433664 :-) I would propose to disable all tests on MacOSX that use JSR Script engines. Test runner problems on OS X / java 7 - Key: SOLR-4233 URL: https://issues.apache.org/jira/browse/SOLR-4233 Project: Solr Issue Type: Bug Components: Tests Environment: Mac os X 10.7.5. java version 1.7.0_09 Java(TM) SE Runtime Environment (build 1.7.0_09-b05) Java HotSpot(TM) 64-Bit Server VM (build 23.5-b02, mixed mode) Reporter: Robert Muir I think it might be a jvm bug... but im just uploading information since its wierd and annoying -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4233) Test runner problems on OS X / java 7
[ https://issues.apache.org/jira/browse/SOLR-4233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539604#comment-13539604 ] Robert Muir commented on SOLR-4233: --- +1, ill add an assume here for OS X. Note: another option is to conditionally System.clearProperty in an afterclass, but these sysprops are initialized by the JDK, so that seems a little scary. Test runner problems on OS X / java 7 - Key: SOLR-4233 URL: https://issues.apache.org/jira/browse/SOLR-4233 Project: Solr Issue Type: Bug Components: Tests Environment: Mac os X 10.7.5. java version 1.7.0_09 Java(TM) SE Runtime Environment (build 1.7.0_09-b05) Java HotSpot(TM) 64-Bit Server VM (build 23.5-b02, mixed mode) Reporter: Robert Muir I think it might be a jvm bug... but im just uploading information since its wierd and annoying -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4233) Test runner problems on OS X / java 7
[ https://issues.apache.org/jira/browse/SOLR-4233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539606#comment-13539606 ] Commit Tag Bot commented on SOLR-4233: -- [trunk commit] Robert Muir http://svn.apache.org/viewvc?view=revisionrevision=1425969 SOLR-4233: throw assumption for this test on OS X: it fails due to AWT engine startup Test runner problems on OS X / java 7 - Key: SOLR-4233 URL: https://issues.apache.org/jira/browse/SOLR-4233 Project: Solr Issue Type: Bug Components: Tests Environment: Mac os X 10.7.5. java version 1.7.0_09 Java(TM) SE Runtime Environment (build 1.7.0_09-b05) Java HotSpot(TM) 64-Bit Server VM (build 23.5-b02, mixed mode) Reporter: Robert Muir I think it might be a jvm bug... but im just uploading information since its wierd and annoying -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-4233) Test runner problems on OS X / java 7
[ https://issues.apache.org/jira/browse/SOLR-4233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir resolved SOLR-4233. --- Resolution: Fixed Fix Version/s: 5.0 4.1 Test runner problems on OS X / java 7 - Key: SOLR-4233 URL: https://issues.apache.org/jira/browse/SOLR-4233 Project: Solr Issue Type: Bug Components: Tests Environment: Mac os X 10.7.5. java version 1.7.0_09 Java(TM) SE Runtime Environment (build 1.7.0_09-b05) Java HotSpot(TM) 64-Bit Server VM (build 23.5-b02, mixed mode) Reporter: Robert Muir Fix For: 4.1, 5.0 I think it might be a jvm bug... but im just uploading information since its wierd and annoying -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4233) Test runner problems on OS X / java 7
[ https://issues.apache.org/jira/browse/SOLR-4233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539608#comment-13539608 ] Commit Tag Bot commented on SOLR-4233: -- [branch_4x commit] Robert Muir http://svn.apache.org/viewvc?view=revisionrevision=1425970 SOLR-4233: throw assumption for this test on OS X: it fails due to AWT engine startup Test runner problems on OS X / java 7 - Key: SOLR-4233 URL: https://issues.apache.org/jira/browse/SOLR-4233 Project: Solr Issue Type: Bug Components: Tests Environment: Mac os X 10.7.5. java version 1.7.0_09 Java(TM) SE Runtime Environment (build 1.7.0_09-b05) Java HotSpot(TM) 64-Bit Server VM (build 23.5-b02, mixed mode) Reporter: Robert Muir Fix For: 4.1, 5.0 I think it might be a jvm bug... but im just uploading information since its wierd and annoying -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-4202) Relax rules around accepting updates when not connected to zookeeper.
[ https://issues.apache.org/jira/browse/SOLR-4202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated SOLR-4202: -- Fix Version/s: (was: 4.1) 4.2 Relax rules around accepting updates when not connected to zookeeper. - Key: SOLR-4202 URL: https://issues.apache.org/jira/browse/SOLR-4202 Project: Solr Issue Type: Improvement Components: SolrCloud Reporter: Mark Miller Assignee: Mark Miller Priority: Minor Fix For: 4.2, 5.0 We are pretty tight about this currently - I think it might be a bit nicer if we relax a little. Right now, as soon we realize we cannot talk to zookeeper, we stop accepting updates in all cases. I think it might be better if we change that a bit for a non leader. It might be nicer if it would still accept updates from the leader, but fail them. This way, there is some chance that if the problem was simply a connection loss with zookeeper, when the leader asks the replica to recover because it failed the update, it's more likely to just take a peersync to catch up. Thoughts? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-4204) Make SolrCloud tests more friendly to FreeBSD blackhole 2 environments.
[ https://issues.apache.org/jira/browse/SOLR-4204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller resolved SOLR-4204. --- Resolution: Fixed Calling this done - there are still some problem areas to track down on the freebsd runs, but my overall plan around making things reasonable in this type of env are finished. Further work is on an issue by issue basis. Make SolrCloud tests more friendly to FreeBSD blackhole 2 environments. --- Key: SOLR-4204 URL: https://issues.apache.org/jira/browse/SOLR-4204 Project: Solr Issue Type: Improvement Components: SolrCloud Reporter: Mark Miller Assignee: Mark Miller Fix For: 4.1, 5.0 Attachments: SOLR-4204.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-4203) An ephemeral directory implementation should cause the transaction log to be ignored on startup.
[ https://issues.apache.org/jira/browse/SOLR-4203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller resolved SOLR-4203. --- Resolution: Fixed An ephemeral directory implementation should cause the transaction log to be ignored on startup. Key: SOLR-4203 URL: https://issues.apache.org/jira/browse/SOLR-4203 Project: Solr Issue Type: Bug Components: SolrCloud Reporter: Mark Miller Assignee: Mark Miller Fix For: 4.1, 5.0 Attachments: SOLR-4203.patch If you are using something like ram dir, you can restart a node and if no updates have come in, it will think its up to date but be empty - we should clear the update log in these cases on startup. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4174) Optionally allow dropping the update log when merging indexes.
[ https://issues.apache.org/jira/browse/SOLR-4174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539610#comment-13539610 ] Mark Miller commented on SOLR-4174: --- On further consideration, I'm not convinced this is necessary. It seems you have two options really - only merge in indexes and don't add docs normally, or merge by adding docs normally. We can't currently make a std index merge play nice with distrib update fault tolerance and failover. Optionally allow dropping the update log when merging indexes. -- Key: SOLR-4174 URL: https://issues.apache.org/jira/browse/SOLR-4174 Project: Solr Issue Type: Bug Reporter: Mark Miller Assignee: Mark Miller Priority: Minor Fix For: 4.1, 5.0 Attachments: SOLR-4174.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-3154) SolrJ CloudServer should be leader and network aware when adding docs
[ https://issues.apache.org/jira/browse/SOLR-3154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated SOLR-3154: -- Fix Version/s: (was: 4.1) 4.2 SolrJ CloudServer should be leader and network aware when adding docs - Key: SOLR-3154 URL: https://issues.apache.org/jira/browse/SOLR-3154 Project: Solr Issue Type: Improvement Components: SolrCloud Affects Versions: 4.0-ALPHA Reporter: Grant Ingersoll Assignee: Mark Miller Priority: Minor Fix For: 4.2, 5.0 Attachments: SOLR-3154.patch It would be good when indexing if the SolrJ CloudServer was leader aware so that we could avoid doing an extra hop for the data. It would also be good if one could easily set things up based on data locality principles. This might mean that CloudServer is aware of where on the network it is and would pick leaders that are as close as possible (i.e. local, perhaps.) This would come in to play when working with tools like Hadoop or other grid computing frameworks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-4174) Optionally allow dropping the update log when merging indexes.
[ https://issues.apache.org/jira/browse/SOLR-4174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller resolved SOLR-4174. --- Resolution: Won't Fix Optionally allow dropping the update log when merging indexes. -- Key: SOLR-4174 URL: https://issues.apache.org/jira/browse/SOLR-4174 Project: Solr Issue Type: Bug Reporter: Mark Miller Assignee: Mark Miller Priority: Minor Fix For: 4.1, 5.0 Attachments: SOLR-4174.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-3935) Change the default jetty connector to be the NIO implementation.
[ https://issues.apache.org/jira/browse/SOLR-3935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated SOLR-3935: -- Fix Version/s: (was: 4.1) 4.2 Change the default jetty connector to be the NIO implementation. Key: SOLR-3935 URL: https://issues.apache.org/jira/browse/SOLR-3935 Project: Solr Issue Type: Improvement Reporter: Mark Miller Assignee: Mark Miller Priority: Minor Fix For: 4.2, 5.0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-1945) Allow @Field annotations in nested classes using DocumentObjectBinder
[ https://issues.apache.org/jira/browse/SOLR-1945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated SOLR-1945: -- Fix Version/s: (was: 4.1) Assignee: (was: Mark Miller) Allow @Field annotations in nested classes using DocumentObjectBinder - Key: SOLR-1945 URL: https://issues.apache.org/jira/browse/SOLR-1945 Project: Solr Issue Type: Improvement Reporter: Mark Miller Priority: Minor Attachments: SOLR-1945.patch see http://search.lucidimagination.com/search/document/d909d909420aeb4e/does_solrj_support_nested_annotated_beans Would be nice to be able to pass an object graph to solrj with @field annotations rather than just a top level class -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-3041) Solrs using SolrCloud feature for having shared config in ZK, might not all start successfully when started for the first time simultaneously
[ https://issues.apache.org/jira/browse/SOLR-3041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated SOLR-3041: -- Fix Version/s: (was: 4.1) Assignee: (was: Mark Miller) Solrs using SolrCloud feature for having shared config in ZK, might not all start successfully when started for the first time simultaneously - Key: SOLR-3041 URL: https://issues.apache.org/jira/browse/SOLR-3041 Project: Solr Issue Type: Bug Components: SolrCloud Affects Versions: 4.0-ALPHA Environment: Exact version: https://builds.apache.org/job/Solr-trunk/1718/artifact/artifacts/apache-solr-4.0-2011-12-28_08-33-55.tgz Reporter: Per Steffensen Original Estimate: 96h Remaining Estimate: 96h Starting Solr like this java -DzkHost=ZKs -Dbootstrap_confdir=./myproject/conf -Dcollection.configName=myproject_conf -Dsolr.solr.home=./myproject -jar start.jar When not already there (starting solr for the first time) the content of ./myproject/conf will be copied by Solr into ZK. That process does not work very well in parallel, so if the content is not there and I start several Solrs simultaneously, one or more of them might not start successfully. I see exceptions like the ones shown below, and the Solrs throwing them will not work correctly afterwards. I know that there could be different workarounds, like making sure to always start one Solr and wait for a while before starting the rest of them, but I think we should really be more robuste in these cases. Regards, Per Steffensen exception example 1 (the znode causing the problem can be different than /configs/myproject_conf/protwords.txt) org.apache.solr.common.cloud.ZooKeeperException: at org.apache.solr.core.CoreContainer.initZooKeeper(CoreContainer.java:193) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:337) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:294) at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:240) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:93) at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:713) at org.mortbay.jetty.servlet.Context.startContext(Context.java:140) at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282) at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:518) at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152) at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130) at org.mortbay.jetty.Server.doStart(Server.java:224) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.mortbay.xml.XmlConfiguration.main(XmlConfiguration.java:985) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.mortbay.start.Main.invokeMain(Main.java:194) at org.mortbay.start.Main.start(Main.java:534) at org.mortbay.start.Main.start(Main.java:441) at org.mortbay.start.Main.main(Main.java:119) Caused by: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /configs/myproject_conf/protwords.txt at org.apache.zookeeper.KeeperException.create(KeeperException.java:110) at org.apache.zookeeper.KeeperException.create(KeeperException.java:42) at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:637) at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:347) at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:308) at
[jira] [Updated] (SOLR-3721) Multiple concurrent recoveries of same shard?
[
https://issues.apache.org/jira/browse/SOLR-3721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Miller updated SOLR-3721:
--
Fix Version/s: (was: 5.0)
(was: 4.1)
Assignee: (was: Mark Miller)
Multiple concurrent recoveries of same shard?
-
Key: SOLR-3721
URL: https://issues.apache.org/jira/browse/SOLR-3721
Project: Solr
Issue Type: Bug
Components: multicore, SolrCloud
Environment: Using our own Solr release based on Apache revision
1355667 from 4.x branch. Our changes to the Solr version is our solutions to
TLT-3178 etc., and should have no effect on this issue.
Reporter: Per Steffensen
Labels: concurrency, multicore, recovery, solrcloud
Attachments: recovery_in_progress.png, recovery_start_finish.log
We run a performance/endurance test on a 7 Solr instance SolrCloud setup and
eventually Solrs lose ZK connections and go into recovery. BTW the recovery
often does not ever succeed, but we are looking into that. While doing that I
noticed that, according to logs, multiple recoveries are in progress at the
same time for the same shard. That cannot be intended and I can certainly
imagine that it will cause some problems.
It is just the logs that are wrong, did I make some mistake, or is this a
real bug?
See attached grep from log, grepping only on Finished recovery and
Starting recovery logs.
{code}
grep -B 1 Finished recovery\|Starting recovery solr9.log solr8.log
solr7.log solr6.log solr5.log solr4.log solr3.log solr2.log solr1.log
solr0.log recovery_start_finish.log
{code}
It can be hard to get an overview of the log, but I have generated a graph
showing (based alone on Started recovery and Finished recovery logs) how
many recoveries are in progress at any time for the different shards. See
attached recovery_in_progress.png. The graph is also a little hard to get an
overview of (due to the many shards) but it is clear that for several shards
there are multiple recoveries going on at the same time, and that several
recoveries never succeed.
Regards, Per Steffensen
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-2018) waitFlush=false currently doens't work
[ https://issues.apache.org/jira/browse/SOLR-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated SOLR-2018: -- Fix Version/s: (was: 4.1) 5.0 4.2 waitFlush=false currently doens't work Key: SOLR-2018 URL: https://issues.apache.org/jira/browse/SOLR-2018 Project: Solr Issue Type: Bug Affects Versions: 1.4, 1.4.1 Reporter: Hoss Man Assignee: Mark Miller Fix For: 4.2, 5.0 circling back on an old mailing list thread... http://search.lucidimagination.com/search/document/24ef53e6e66d3a41/optimize_with_waitflush_false_and_waitsearcher_false_takes_a_long_time according to miller and koji, SOlr doesn't currently pay any attention to the waitFLush option when doing a commit/optimize. Prior to the next release, we either fix this, or document that it changed (when did it change?) and rip out that code -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-2646) Integrate Solr benchmarking support into the Benchmark module
[ https://issues.apache.org/jira/browse/SOLR-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated SOLR-2646: -- Fix Version/s: (was: 4.1) Integrate Solr benchmarking support into the Benchmark module - Key: SOLR-2646 URL: https://issues.apache.org/jira/browse/SOLR-2646 Project: Solr Issue Type: New Feature Reporter: Mark Miller Assignee: Mark Miller Attachments: chart.jpg, Dev-SolrBenchmarkModule.pdf, SOLR-2646.patch, SOLR-2646.patch, SOLR-2646.patch, SOLR-2646.patch, SolrIndexingPerfHistory.pdf As part of my buzzwords Solr pef talk, I did some work to allow some Solr benchmarking with the benchmark module. I'll attach a patch with the current work I've done soon - there is still a fair amount to clean up and fix - a couple hacks or three - but it's already fairly useful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-3604) The wiki claims all comments are wiped from solr.xml on save - we should fix this.
[ https://issues.apache.org/jira/browse/SOLR-3604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated SOLR-3604: -- Fix Version/s: (was: 4.1) 5.0 4.2 The wiki claims all comments are wiped from solr.xml on save - we should fix this. -- Key: SOLR-3604 URL: https://issues.apache.org/jira/browse/SOLR-3604 Project: Solr Issue Type: Bug Reporter: Mark Miller Assignee: Mark Miller Priority: Minor Fix For: 4.2, 5.0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-2326) Replication command indexversion fails to return index version
[ https://issues.apache.org/jira/browse/SOLR-2326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated SOLR-2326: -- Fix Version/s: (was: 4.1) 5.0 4.2 Replication command indexversion fails to return index version -- Key: SOLR-2326 URL: https://issues.apache.org/jira/browse/SOLR-2326 Project: Solr Issue Type: Bug Components: replication (java) Environment: Branch 3x latest Reporter: Eric Pugh Assignee: Mark Miller Fix For: 4.2, 5.0 To test this, I took the /example/multicore/core0 solrconfig and added a simple replication handler: requestHandler name=/replication class=solr.ReplicationHandler lst name=master str name=replicateAftercommit/str str name=replicateAfterstartup/str str name=confFilesschema.xml/str /lst /requestHandler When I query the handler for details I get back the indexVersion that I expect: http://localhost:8983/solr/core0/replication?command=detailswt=jsonindent=true But when I ask for just the indexVersion I get back a 0, which prevent the slaves from pulling updates: http://localhost:8983/solr/core0/replication?command=indexversionwt=jsonindent=true -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-3071) When you change the config for a collection, all nodes in the collection should reload their SolrCore.
[ https://issues.apache.org/jira/browse/SOLR-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated SOLR-3071: -- Fix Version/s: (was: 4.1) 5.0 4.2 When you change the config for a collection, all nodes in the collection should reload their SolrCore. -- Key: SOLR-3071 URL: https://issues.apache.org/jira/browse/SOLR-3071 Project: Solr Issue Type: Improvement Components: SolrCloud Reporter: Mark Miller Assignee: Mark Miller Priority: Minor Fix For: 4.2, 5.0 This will make it much easier to make configuration updates. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-3090) When running a ZK ensemble within Solr, the info logging is too verbose.
[
https://issues.apache.org/jira/browse/SOLR-3090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Miller updated SOLR-3090:
--
Fix Version/s: (was: 4.1)
5.0
4.2
When running a ZK ensemble within Solr, the info logging is too verbose.
Key: SOLR-3090
URL: https://issues.apache.org/jira/browse/SOLR-3090
Project: Solr
Issue Type: Bug
Components: SolrCloud
Reporter: Mark Miller
Assignee: Mark Miller
Priority: Minor
Fix For: 4.2, 5.0
you gets lines and lines of..
{noformat}
Feb 2, 2012 10:55:09 PM org.apache.zookeeper.server.quorum.FastLeaderElection
printNotification
INFO: Notification: 1 (n.leader), 0 (n.zxid), 2 (n.round), LOOKING (n.state),
1 (n.sid), FOLLOWING (my state)
Feb 2, 2012 10:55:09 PM org.apache.zookeeper.server.quorum.FastLeaderElection
printNotification
INFO: Notification: 1 (n.leader), 0 (n.zxid), 2 (n.round), LOOKING (n.state),
1 (n.sid), FOLLOWING (my state)
Feb 2, 2012 10:55:09 PM org.apache.zookeeper.server.quorum.FastLeaderElection
printNotification
INFO: Notification: 1 (n.leader), 0 (n.zxid), 2 (n.round), LOOKING (n.state),
1 (n.sid), FOLLOWING (my state)
Feb 2, 2012 10:55:09 PM org.apache.zookeeper.server.quorum.FastLeaderElection
printNotification
INFO: Notification: 1 (n.leader), 0 (n.zxid), 2 (n.round), LOOKING (n.state),
1 (n.sid), FOLLOWING (my state)
Feb 2, 2012 10:55:09 PM org.apache.zookeeper.server.quorum.FastLeaderElection
printNotification
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1972) Need additional query stats in admin interface - median, 95th and 99th percentile
[ https://issues.apache.org/jira/browse/SOLR-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539623#comment-13539623 ] Shawn Heisey commented on SOLR-1972: I've been trying to find evidence of the leak, and can't. I completely trust that you guys do know what you're talking about, so this is probably just my inexperience. What I did find: Before any additional patching, looking at jconsole, I have two metrics threads. Looking at MBeans in jconsole, there are 290 metric scopes under RequestHandlerBase with names from metrics-scope-0 to metrics-scope-289. Taking Robert's patch and fixing it to create one static MetricsRegistry, the additional objects under MBeans disappear, and the number of threads named metric-* goes from 2 to 1. Through some additional debugs, I have files containing stacktraces from every time RequestHandlerBase is called, so I know it is still being called 290 times during startup. Examining those stacktraces, I have determined that each of my 16 cores has 18 request handlers in common, with two additional server-wide handlers - CollectionsHandler and CoreAdminHandler - for a total of 290. I'm attaching an archive with all the stacktraces and a summary - the third line from each stacktrace, sorted. If we made the metrics package optional on a per-handler basis, I would drop from 290 scopes to 48 - each of my 16 cores has four search handlers, but I really only need full metrics in three of them. Need additional query stats in admin interface - median, 95th and 99th percentile - Key: SOLR-1972 URL: https://issues.apache.org/jira/browse/SOLR-1972 Project: Solr Issue Type: Improvement Components: web gui Affects Versions: 1.4 Reporter: Shawn Heisey Assignee: Alan Woodward Priority: Minor Fix For: 4.1, 5.0 Attachments: elyograg-1972-3.2.patch, elyograg-1972-3.2.patch, elyograg-1972-trunk.patch, elyograg-1972-trunk.patch, leak.patch, SOLR-1972-branch3x-url_pattern.patch, SOLR-1972-branch4x.patch, SOLR-1972-branch4x.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, solr1972-metricsregistry-branch4x-failure.log, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972-url_pattern.patch I would like to see more detailed query statistics from the admin GUI. This is what you can get now: requests : 809 errors : 0 timeouts : 0 totalTime : 70053 avgTimePerRequest : 86.59209 avgRequestsPerSecond : 0.8148785 I'd like to see more data on the time per request - median, 95th percentile, 99th percentile, and any other statistical function that makes sense to include. In my environment, the first bunch of queries after startup tend to take several seconds each. I find that the average value tends to be useless until it has several thousand queries under its belt and the caches are thoroughly warmed. The statistical functions I have mentioned would quickly eliminate the influence of those initial slow queries. The system will have to store individual data about each query. I don't know if this is something Solr does already. It would be nice to have a configurable count of how many of the most recent data points are kept, to control the amount of memory the feature uses. The default value could be something like 1024 or 4096. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-1972) Need additional query stats in admin interface - median, 95th and 99th percentile
[ https://issues.apache.org/jira/browse/SOLR-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shawn Heisey updated SOLR-1972: --- Attachment: stacktraces.tar.gz Archive with debug stacktraces and a summary. Need additional query stats in admin interface - median, 95th and 99th percentile - Key: SOLR-1972 URL: https://issues.apache.org/jira/browse/SOLR-1972 Project: Solr Issue Type: Improvement Components: web gui Affects Versions: 1.4 Reporter: Shawn Heisey Assignee: Alan Woodward Priority: Minor Fix For: 4.1, 5.0 Attachments: elyograg-1972-3.2.patch, elyograg-1972-3.2.patch, elyograg-1972-trunk.patch, elyograg-1972-trunk.patch, leak.patch, SOLR-1972-branch3x-url_pattern.patch, SOLR-1972-branch4x.patch, SOLR-1972-branch4x.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, solr1972-metricsregistry-branch4x-failure.log, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972-url_pattern.patch, stacktraces.tar.gz I would like to see more detailed query statistics from the admin GUI. This is what you can get now: requests : 809 errors : 0 timeouts : 0 totalTime : 70053 avgTimePerRequest : 86.59209 avgRequestsPerSecond : 0.8148785 I'd like to see more data on the time per request - median, 95th percentile, 99th percentile, and any other statistical function that makes sense to include. In my environment, the first bunch of queries after startup tend to take several seconds each. I find that the average value tends to be useless until it has several thousand queries under its belt and the caches are thoroughly warmed. The statistical functions I have mentioned would quickly eliminate the influence of those initial slow queries. The system will have to store individual data about each query. I don't know if this is something Solr does already. It would be nice to have a configurable count of how many of the most recent data points are kept, to control the amount of memory the feature uses. The default value could be something like 1024 or 4096. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4088) ZkController baseURL only get the host name not address, will occur can't get the right URL to do recovery.
[ https://issues.apache.org/jira/browse/SOLR-4088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539625#comment-13539625 ] Commit Tag Bot commented on SOLR-4088: -- [branch_4x commit] Robert Muir http://svn.apache.org/viewvc?view=revisionrevision=1425972 SOLR-4088: when host is null, fall back to interface iteration (heuristic 2) if resolving the system hostname (heuristic 1) fails ZkController baseURL only get the host name not address, will occur can't get the right URL to do recovery. -- Key: SOLR-4088 URL: https://issues.apache.org/jira/browse/SOLR-4088 Project: Solr Issue Type: Bug Components: SolrCloud Affects Versions: 4.0-ALPHA, 4.0-BETA, 4.0 Environment: Linux env Reporter: Raintung Li Assignee: Mark Miller Fix For: 4.1, 5.0 Attachments: patch-4088.txt, SOLR-4088_unknown_host.patch In the ZKController.getHostAddress, it gets host name. In the linux, get hostname from /etc/sysconfig/network or /etc/hostname that maybe not config for ip address. Other server can't access it(http://hostname:port/..) correctly that cause recovery fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4088) ZkController baseURL only get the host name not address, will occur can't get the right URL to do recovery.
[ https://issues.apache.org/jira/browse/SOLR-4088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539626#comment-13539626 ] Commit Tag Bot commented on SOLR-4088: -- [trunk commit] Robert Muir http://svn.apache.org/viewvc?view=revisionrevision=1425971 SOLR-4088: when host is null, fall back to interface iteration (heuristic 2) if resolving the system hostname (heuristic 1) fails ZkController baseURL only get the host name not address, will occur can't get the right URL to do recovery. -- Key: SOLR-4088 URL: https://issues.apache.org/jira/browse/SOLR-4088 Project: Solr Issue Type: Bug Components: SolrCloud Affects Versions: 4.0-ALPHA, 4.0-BETA, 4.0 Environment: Linux env Reporter: Raintung Li Assignee: Mark Miller Fix For: 4.1, 5.0 Attachments: patch-4088.txt, SOLR-4088_unknown_host.patch In the ZKController.getHostAddress, it gets host name. In the linux, get hostname from /etc/sysconfig/network or /etc/hostname that maybe not config for ip address. Other server can't access it(http://hostname:port/..) correctly that cause recovery fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1972) Need additional query stats in admin interface - median, 95th and 99th percentile
[ https://issues.apache.org/jira/browse/SOLR-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539628#comment-13539628 ] Shawn Heisey commented on SOLR-1972: Some things just clicked and I think I might have figured it out, tell me if I get this wrong. The metrics objects get created during initialization, but then if you reload a core (or possibly if you reload the entire app in your container) then the old metrics objects are never garbage collected because the registry object holds onto them forever. I do see that the registry has the ability to remove metrics. To make this possible, RequestHandlerBase would probably need a close() method, and it would need to be properly called when required. I am assuming this is not a trivial undertaking? Need additional query stats in admin interface - median, 95th and 99th percentile - Key: SOLR-1972 URL: https://issues.apache.org/jira/browse/SOLR-1972 Project: Solr Issue Type: Improvement Components: web gui Affects Versions: 1.4 Reporter: Shawn Heisey Assignee: Alan Woodward Priority: Minor Fix For: 4.1, 5.0 Attachments: elyograg-1972-3.2.patch, elyograg-1972-3.2.patch, elyograg-1972-trunk.patch, elyograg-1972-trunk.patch, leak.patch, SOLR-1972-branch3x-url_pattern.patch, SOLR-1972-branch4x.patch, SOLR-1972-branch4x.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, solr1972-metricsregistry-branch4x-failure.log, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972-url_pattern.patch, stacktraces.tar.gz I would like to see more detailed query statistics from the admin GUI. This is what you can get now: requests : 809 errors : 0 timeouts : 0 totalTime : 70053 avgTimePerRequest : 86.59209 avgRequestsPerSecond : 0.8148785 I'd like to see more data on the time per request - median, 95th percentile, 99th percentile, and any other statistical function that makes sense to include. In my environment, the first bunch of queries after startup tend to take several seconds each. I find that the average value tends to be useless until it has several thousand queries under its belt and the caches are thoroughly warmed. The statistical functions I have mentioned would quickly eliminate the influence of those initial slow queries. The system will have to store individual data about each query. I don't know if this is something Solr does already. It would be nice to have a configurable count of how many of the most recent data points are kept, to control the amount of memory the feature uses. The default value could be something like 1024 or 4096. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-1972) Need additional query stats in admin interface - median, 95th and 99th percentile
[ https://issues.apache.org/jira/browse/SOLR-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shawn Heisey updated SOLR-1972: --- Attachment: leak-closeable.patch An extension of Robert's leak.patch, as a possible starting point for a complete solution to the leak problem. The patch mimics the overall memory efficiency of the current approach by changing to a static registry object instead of an implied global registry. It also implements the Closeable interface, using the now-accessible registry object. This part is incomplete, because calls to the close() method would need to be added to anyplace where a request handler object becomes garbage. I don't know where these places might be. Additional note: If we ever extend the metrics capability to things other than request handlers, we will need to make the static registry object global. Need additional query stats in admin interface - median, 95th and 99th percentile - Key: SOLR-1972 URL: https://issues.apache.org/jira/browse/SOLR-1972 Project: Solr Issue Type: Improvement Components: web gui Affects Versions: 1.4 Reporter: Shawn Heisey Assignee: Alan Woodward Priority: Minor Fix For: 4.1, 5.0 Attachments: elyograg-1972-3.2.patch, elyograg-1972-3.2.patch, elyograg-1972-trunk.patch, elyograg-1972-trunk.patch, leak-closeable.patch, leak.patch, SOLR-1972-branch3x-url_pattern.patch, SOLR-1972-branch4x.patch, SOLR-1972-branch4x.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, solr1972-metricsregistry-branch4x-failure.log, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972-url_pattern.patch, stacktraces.tar.gz I would like to see more detailed query statistics from the admin GUI. This is what you can get now: requests : 809 errors : 0 timeouts : 0 totalTime : 70053 avgTimePerRequest : 86.59209 avgRequestsPerSecond : 0.8148785 I'd like to see more data on the time per request - median, 95th percentile, 99th percentile, and any other statistical function that makes sense to include. In my environment, the first bunch of queries after startup tend to take several seconds each. I find that the average value tends to be useless until it has several thousand queries under its belt and the caches are thoroughly warmed. The statistical functions I have mentioned would quickly eliminate the influence of those initial slow queries. The system will have to store individual data about each query. I don't know if this is something Solr does already. It would be nice to have a configurable count of how many of the most recent data points are kept, to control the amount of memory the feature uses. The default value could be something like 1024 or 4096. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-4648) PostingsHighlighter should score only with within-document statistics
[ https://issues.apache.org/jira/browse/LUCENE-4648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir resolved LUCENE-4648. - Resolution: Fixed Fix Version/s: 5.0 4.1 PostingsHighlighter should score only with within-document statistics - Key: LUCENE-4648 URL: https://issues.apache.org/jira/browse/LUCENE-4648 Project: Lucene - Core Issue Type: Improvement Components: modules/other Affects Versions: 4.1 Reporter: Robert Muir Fix For: 4.1, 5.0 Attachments: LUCENE-4648.patch PostingsHighlighter (in sandbox) treats documents to be highlighted as collections of small documents (snippets). But (like the other highlighters), it uses collection-wide stats to score the snippets. It actually calls the indexsearcher method which might even result in a distributed call. I don't think its worth it to do all this. It means we have to reseek to all terms across all segments at the minimum, and it means snippets change over time and so on. I think its good enough to only use within-document stats. And it might speed up highlighting. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4648) PostingsHighlighter should score only with within-document statistics
[ https://issues.apache.org/jira/browse/LUCENE-4648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539642#comment-13539642 ] Commit Tag Bot commented on LUCENE-4648: [trunk commit] Robert Muir http://svn.apache.org/viewvc?view=revisionrevision=1425992 LUCENE-4648: use only within-doc stats for PostingsHighlighter PostingsHighlighter should score only with within-document statistics - Key: LUCENE-4648 URL: https://issues.apache.org/jira/browse/LUCENE-4648 Project: Lucene - Core Issue Type: Improvement Components: modules/other Affects Versions: 4.1 Reporter: Robert Muir Fix For: 4.1, 5.0 Attachments: LUCENE-4648.patch PostingsHighlighter (in sandbox) treats documents to be highlighted as collections of small documents (snippets). But (like the other highlighters), it uses collection-wide stats to score the snippets. It actually calls the indexsearcher method which might even result in a distributed call. I don't think its worth it to do all this. It means we have to reseek to all terms across all segments at the minimum, and it means snippets change over time and so on. I think its good enough to only use within-document stats. And it might speed up highlighting. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4648) PostingsHighlighter should score only with within-document statistics
[ https://issues.apache.org/jira/browse/LUCENE-4648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539643#comment-13539643 ] Commit Tag Bot commented on LUCENE-4648: [branch_4x commit] Robert Muir http://svn.apache.org/viewvc?view=revisionrevision=1425993 LUCENE-4648: use only within-doc stats for PostingsHighlighter PostingsHighlighter should score only with within-document statistics - Key: LUCENE-4648 URL: https://issues.apache.org/jira/browse/LUCENE-4648 Project: Lucene - Core Issue Type: Improvement Components: modules/other Affects Versions: 4.1 Reporter: Robert Muir Fix For: 4.1, 5.0 Attachments: LUCENE-4648.patch PostingsHighlighter (in sandbox) treats documents to be highlighted as collections of small documents (snippets). But (like the other highlighters), it uses collection-wide stats to score the snippets. It actually calls the indexsearcher method which might even result in a distributed call. I don't think its worth it to do all this. It means we have to reseek to all terms across all segments at the minimum, and it means snippets change over time and so on. I think its good enough to only use within-document stats. And it might speed up highlighting. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4233) Test runner problems on OS X / java 7
[ https://issues.apache.org/jira/browse/SOLR-4233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539644#comment-13539644 ] Dawid Weiss commented on SOLR-4233: --- Sorry for belated follow-up. The first error is what Uwe already clarified -- indeed an issue on a Mac version of Java. The second message you're getting is (in my opinion) exactly what I wanted to capture when I was writing junit4 task -- something totally unpexpected being written to jvm output descriptors. In this case I think it's a bug in the JVM somewhere, hard to tell where exactly. Is this the Apple JVM you're testing on? Test runner problems on OS X / java 7 - Key: SOLR-4233 URL: https://issues.apache.org/jira/browse/SOLR-4233 Project: Solr Issue Type: Bug Components: Tests Environment: Mac os X 10.7.5. java version 1.7.0_09 Java(TM) SE Runtime Environment (build 1.7.0_09-b05) Java HotSpot(TM) 64-Bit Server VM (build 23.5-b02, mixed mode) Reporter: Robert Muir Fix For: 4.1, 5.0 I think it might be a jvm bug... but im just uploading information since its wierd and annoying -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4233) Test runner problems on OS X / java 7
[ https://issues.apache.org/jira/browse/SOLR-4233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539655#comment-13539655 ] Robert Muir commented on SOLR-4233: --- The second error definitely worked as far as it didnt actually cause test failures. I thought it did, but i was confused :) I think junit4 did what it was supposed to do, something went to stderr and it just printed it, but didnt fail the test. This isnt the apple JVM actually: Its the oracle one. I downloaded it from java.sun.com Test runner problems on OS X / java 7 - Key: SOLR-4233 URL: https://issues.apache.org/jira/browse/SOLR-4233 Project: Solr Issue Type: Bug Components: Tests Environment: Mac os X 10.7.5. java version 1.7.0_09 Java(TM) SE Runtime Environment (build 1.7.0_09-b05) Java HotSpot(TM) 64-Bit Server VM (build 23.5-b02, mixed mode) Reporter: Robert Muir Fix For: 4.1, 5.0 I think it might be a jvm bug... but im just uploading information since its wierd and annoying -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4233) Test runner problems on OS X / java 7
[ https://issues.apache.org/jira/browse/SOLR-4233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539657#comment-13539657 ] Dawid Weiss commented on SOLR-4233: --- Yeah... by default it would have ended in an error but (due to popular criticism :) it is configured in Lucene to only warn about JVM output. See the description here: http://labs.carrotsearch.com/download/randomizedtesting/2.0.0/docs/junit4-ant/Tasks/junit4.html See jvmOutputAction attribute and a section of examples below, under Capturing original JVM output. Test runner problems on OS X / java 7 - Key: SOLR-4233 URL: https://issues.apache.org/jira/browse/SOLR-4233 Project: Solr Issue Type: Bug Components: Tests Environment: Mac os X 10.7.5. java version 1.7.0_09 Java(TM) SE Runtime Environment (build 1.7.0_09-b05) Java HotSpot(TM) 64-Bit Server VM (build 23.5-b02, mixed mode) Reporter: Robert Muir Fix For: 4.1, 5.0 I think it might be a jvm bug... but im just uploading information since its wierd and annoying -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-4234) ZooKeeper doesn't handle binary files
Eric Pugh created SOLR-4234: --- Summary: ZooKeeper doesn't handle binary files Key: SOLR-4234 URL: https://issues.apache.org/jira/browse/SOLR-4234 Project: Solr Issue Type: Bug Components: SolrCloud Affects Versions: 4.0 Reporter: Eric Pugh I was attempting to get the ShowFileHandler to show a .png file, and it was failing. But in non-ZK mode it worked just fine! It took a while, but it seems that we upload to zk as a text, and download as well. I've attached a unit test that demonstrates the problem, and a fix. You have to have a binary file in the conf directory to make the test work, I put solr.png in the collection1/conf/velocity directory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-4235) Solr join or subquery should support sorting (and perhaps paging)
Dan Ertman created SOLR-4235:
Summary: Solr join or subquery should support sorting (and perhaps
paging)
Key: SOLR-4235
URL: https://issues.apache.org/jira/browse/SOLR-4235
Project: Solr
Issue Type: Improvement
Affects Versions: 4.0
Reporter: Dan Ertman
Suppose a list of product documents, and a list of order documents, in the same
core. We want to retrieve the products ordered. With the new join syntax,
q={!join from=orderSku to=sku}accountNumber=x
Great. Now, we wish to retrieve the products most recently ordered, by order
date.
q={!join from=orderSku to=sku}accountNumber=xsort=orderDate desc
Nope - product doesn't have an orderDate field, no effect
q={!join from=order to=sku)_query_:accountNumber:xsort=orderDate desc
Nope. Parser approves it, but appears to have no effect. My guess is that sort
is applied after the subquery result is effectively sent to the parent.
So we wind up getting all of the orders by date desc, and bundling up their
SKUs, and POSTing that as a big OR clause. All things that join was supposed to
save us from. It would be great if there was a way to support sort on the
parent instead.
It would also be helpful if there was a way to support pagination of the
parent. For example, suppose I now want to display the date of the order along
side the product. I can join from product to order, but if they've paged the
search result I can't page the products used in the join.
I'd be perfectly content to use nested query syntax for both of these things -
that might be more flexible overall - but it doesn't appear to work.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-4234) ZooKeeper doesn't handle binary files
[ https://issues.apache.org/jira/browse/SOLR-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Pugh updated SOLR-4234: Attachment: binary_upload_download.patch Testcase and patch for binary files in ZK. ZooKeeper doesn't handle binary files - Key: SOLR-4234 URL: https://issues.apache.org/jira/browse/SOLR-4234 Project: Solr Issue Type: Bug Components: SolrCloud Affects Versions: 4.0 Reporter: Eric Pugh Attachments: binary_upload_download.patch I was attempting to get the ShowFileHandler to show a .png file, and it was failing. But in non-ZK mode it worked just fine! It took a while, but it seems that we upload to zk as a text, and download as well. I've attached a unit test that demonstrates the problem, and a fix. You have to have a binary file in the conf directory to make the test work, I put solr.png in the collection1/conf/velocity directory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4290) basic highlighter that uses postings offsets
[ https://issues.apache.org/jira/browse/LUCENE-4290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539726#comment-13539726 ] Commit Tag Bot commented on LUCENE-4290: [trunk commit] Robert Muir http://svn.apache.org/viewvc?view=revisionrevision=1426072 LUCENE-4290: add some more testing for this sandy highlighter basic highlighter that uses postings offsets Key: LUCENE-4290 URL: https://issues.apache.org/jira/browse/LUCENE-4290 Project: Lucene - Core Issue Type: New Feature Components: modules/other Reporter: Robert Muir Fix For: 4.1, 5.0 Attachments: LUCENE-4290.patch We added IndexOptions.DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS so you can efficiently compress character offsets in the postings list, but nothing yet makes use of this. Here is a simple highlighter that uses them: it doesn't have many tests or fancy features, but I think its ok for the sandbox/ (maybe with a couple more tests) Additionally I didnt do any benchmarking. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-4234) ZooKeeper doesn't handle binary files
[ https://issues.apache.org/jira/browse/SOLR-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Pugh updated SOLR-4234: Attachment: fix_show_file_handler_with_binaries.patch This is the patch to allow the ShowFileHandler to take binaries. ZooKeeper doesn't handle binary files - Key: SOLR-4234 URL: https://issues.apache.org/jira/browse/SOLR-4234 Project: Solr Issue Type: Bug Components: SolrCloud Affects Versions: 4.0 Reporter: Eric Pugh Attachments: binary_upload_download.patch, fix_show_file_handler_with_binaries.patch I was attempting to get the ShowFileHandler to show a .png file, and it was failing. But in non-ZK mode it worked just fine! It took a while, but it seems that we upload to zk as a text, and download as well. I've attached a unit test that demonstrates the problem, and a fix. You have to have a binary file in the conf directory to make the test work, I put solr.png in the collection1/conf/velocity directory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4234) ZooKeeper doesn't handle binary files
[ https://issues.apache.org/jira/browse/SOLR-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539728#comment-13539728 ] Eric Pugh commented on SOLR-4234: - Without this patch, a request like this would fail: http://localhost:8983/solr/my_search_core_shard1_replica1/admin/file?file=/velocity/header.pngcontentType=image/png ZooKeeper doesn't handle binary files - Key: SOLR-4234 URL: https://issues.apache.org/jira/browse/SOLR-4234 Project: Solr Issue Type: Bug Components: SolrCloud Affects Versions: 4.0 Reporter: Eric Pugh Attachments: binary_upload_download.patch, fix_show_file_handler_with_binaries.patch I was attempting to get the ShowFileHandler to show a .png file, and it was failing. But in non-ZK mode it worked just fine! It took a while, but it seems that we upload to zk as a text, and download as well. I've attached a unit test that demonstrates the problem, and a fix. You have to have a binary file in the conf directory to make the test work, I put solr.png in the collection1/conf/velocity directory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4290) basic highlighter that uses postings offsets
[ https://issues.apache.org/jira/browse/LUCENE-4290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539731#comment-13539731 ] Commit Tag Bot commented on LUCENE-4290: [branch_4x commit] Robert Muir http://svn.apache.org/viewvc?view=revisionrevision=1426075 LUCENE-4290: add some more testing for this sandy highlighter basic highlighter that uses postings offsets Key: LUCENE-4290 URL: https://issues.apache.org/jira/browse/LUCENE-4290 Project: Lucene - Core Issue Type: New Feature Components: modules/other Reporter: Robert Muir Fix For: 4.1, 5.0 Attachments: LUCENE-4290.patch We added IndexOptions.DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS so you can efficiently compress character offsets in the postings list, but nothing yet makes use of this. Here is a simple highlighter that uses them: it doesn't have many tests or fancy features, but I think its ok for the sandbox/ (maybe with a couple more tests) Additionally I didnt do any benchmarking. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4208) Refactor edismax query parser
[ https://issues.apache.org/jira/browse/SOLR-4208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539735#comment-13539735 ] Tomás Fernández Löbbe commented on SOLR-4208: - bq. Could you make the patch such that it's not an entire file replacement so we can more easily see what's changed exactly? I think moving the parser to a different file is a good part of the refactor. I'm attaching the file qParserDiff.txt to show how the parser changed with the refactor, but I'm keeping it separately on SOLR-4208.patch. bq. Perhaps rather (or in conjunction with) making the parser easier/cleaner to extends, we could make it open to Solr-style plugins, where the edismax parser itself is still used directly, but various overrides/extensions can be plugged in (and perhaps made query-time changeable[!] by name lookup Solr-plugin-style; see HighlightComponent for example) This is a good idea. When I started to work on my use case I initially thought on having something like the UpdateRequestProcessorChain at query time that would end with the EdismaxQParser itself (maybe a QParser that would add this component chain and use the EdismaxQParser as part of it), but that wouldn't give me enough points to extend, because the parse() method was going to be executed by the EdismaxQParser and there was no way of modifying its behavior. Maybe there is another way of add a plugin style that allows more customization? In the meantime I'm attaching a new patch with some more minor changes and some tests Refactor edismax query parser - Key: SOLR-4208 URL: https://issues.apache.org/jira/browse/SOLR-4208 Project: Solr Issue Type: Improvement Reporter: Tomás Fernández Löbbe Priority: Minor Fix For: 4.1, 5.0 Attachments: SOLR-4208.patch With successive changes, the edismax query parser has become more complex. It would be nice to refactor it to reduce code complexity, also to allow better extension and code reuse. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-4208) Refactor edismax query parser
[ https://issues.apache.org/jira/browse/SOLR-4208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomás Fernández Löbbe updated SOLR-4208: Attachment: SOLR-4208.patch qParserDiff.txt Refactor edismax query parser - Key: SOLR-4208 URL: https://issues.apache.org/jira/browse/SOLR-4208 Project: Solr Issue Type: Improvement Reporter: Tomás Fernández Löbbe Priority: Minor Fix For: 4.1, 5.0 Attachments: qParserDiff.txt, SOLR-4208.patch, SOLR-4208.patch With successive changes, the edismax query parser has become more complex. It would be nice to refactor it to reduce code complexity, also to allow better extension and code reuse. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.6.0_37) - Build # 2311 - Failure!
Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Windows/2311/ Java: 64bit/jdk1.6.0_37 -XX:+UseSerialGC All tests passed Build Log: [...truncated 29917 lines...] BUILD FAILED C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:312: The following error occurred while executing this line: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-targets.xml:120: The following files are missing svn:eol-style (or binary svn:mime-type): * lucene/sandbox/src/test/org/apache/lucene/sandbox/postingshighlight/TestPostingsHighlighterRanking.java Total time: 66 minutes 59 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts Recording test results Description set: Java: 64bit/jdk1.6.0_37 -XX:+UseSerialGC Email was triggered for: Failure Sending email for trigger: Failure - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-Tests-4.x-java7 - Build # 831 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-java7/831/ All tests passed Build Log: [...truncated 30432 lines...] BUILD FAILED /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:312: The following error occurred while executing this line: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/extra-targets.xml:120: The following files are missing svn:eol-style (or binary svn:mime-type): * lucene/sandbox/src/test/org/apache/lucene/sandbox/postingshighlight/TestPostingsHighlighterRanking.java Total time: 53 minutes 17 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts Recording test results Email was triggered for: Failure Sending email for trigger: Failure - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.7.0_09) - Build # 3428 - Failure!
Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Linux/3428/ Java: 32bit/jdk1.7.0_09 -server -XX:+UseSerialGC All tests passed Build Log: [...truncated 30415 lines...] BUILD FAILED /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:312: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:120: The following files are missing svn:eol-style (or binary svn:mime-type): * lucene/sandbox/src/test/org/apache/lucene/sandbox/postingshighlight/TestPostingsHighlighterRanking.java Total time: 34 minutes 0 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts Recording test results Description set: Java: 32bit/jdk1.7.0_09 -server -XX:+UseSerialGC Email was triggered for: Failure Sending email for trigger: Failure - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0-ea-b65) - Build # 3427 - Failure!
I committed a hack for this test. really, this test is broken. I think it should be removed. it will randomly fail. On Tue, Dec 25, 2012 at 11:59 PM, Policeman Jenkins Server jenk...@sd-datasolutions.de wrote: org.apache.solr.client.solrj.impl.BasicHttpSolrServerTest.testConnectionRefused Error Message: Test abandoned because suite timeout was reached. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1972) Need additional query stats in admin interface - median, 95th and 99th percentile
[ https://issues.apache.org/jira/browse/SOLR-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539779#comment-13539779 ] Lance Norskog commented on SOLR-1972: - The OnlineSummary class in Mahout does the calculations you want. One little class you can steal. No dependencies necessary. Need additional query stats in admin interface - median, 95th and 99th percentile - Key: SOLR-1972 URL: https://issues.apache.org/jira/browse/SOLR-1972 Project: Solr Issue Type: Improvement Components: web gui Affects Versions: 1.4 Reporter: Shawn Heisey Assignee: Alan Woodward Priority: Minor Fix For: 4.1, 5.0 Attachments: elyograg-1972-3.2.patch, elyograg-1972-3.2.patch, elyograg-1972-trunk.patch, elyograg-1972-trunk.patch, leak-closeable.patch, leak.patch, SOLR-1972-branch3x-url_pattern.patch, SOLR-1972-branch4x.patch, SOLR-1972-branch4x.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, solr1972-metricsregistry-branch4x-failure.log, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972-url_pattern.patch, stacktraces.tar.gz I would like to see more detailed query statistics from the admin GUI. This is what you can get now: requests : 809 errors : 0 timeouts : 0 totalTime : 70053 avgTimePerRequest : 86.59209 avgRequestsPerSecond : 0.8148785 I'd like to see more data on the time per request - median, 95th percentile, 99th percentile, and any other statistical function that makes sense to include. In my environment, the first bunch of queries after startup tend to take several seconds each. I find that the average value tends to be useless until it has several thousand queries under its belt and the caches are thoroughly warmed. The statistical functions I have mentioned would quickly eliminate the influence of those initial slow queries. The system will have to store individual data about each query. I don't know if this is something Solr does already. It would be nice to have a configurable count of how many of the most recent data points are kept, to control the amount of memory the feature uses. The default value could be something like 1024 or 4096. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Assigned] (SOLR-4234) ZooKeeper doesn't handle binary files
[ https://issues.apache.org/jira/browse/SOLR-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller reassigned SOLR-4234: - Assignee: Mark Miller ZooKeeper doesn't handle binary files - Key: SOLR-4234 URL: https://issues.apache.org/jira/browse/SOLR-4234 Project: Solr Issue Type: Bug Components: SolrCloud Affects Versions: 4.0 Reporter: Eric Pugh Assignee: Mark Miller Attachments: binary_upload_download.patch, fix_show_file_handler_with_binaries.patch I was attempting to get the ShowFileHandler to show a .png file, and it was failing. But in non-ZK mode it worked just fine! It took a while, but it seems that we upload to zk as a text, and download as well. I've attached a unit test that demonstrates the problem, and a fix. You have to have a binary file in the conf directory to make the test work, I put solr.png in the collection1/conf/velocity directory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-4234) ZooKeeper doesn't handle binary files
[ https://issues.apache.org/jira/browse/SOLR-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated SOLR-4234: -- Fix Version/s: 5.0 4.1 ZooKeeper doesn't handle binary files - Key: SOLR-4234 URL: https://issues.apache.org/jira/browse/SOLR-4234 Project: Solr Issue Type: Bug Components: SolrCloud Affects Versions: 4.0 Reporter: Eric Pugh Assignee: Mark Miller Fix For: 4.1, 5.0 Attachments: binary_upload_download.patch, fix_show_file_handler_with_binaries.patch I was attempting to get the ShowFileHandler to show a .png file, and it was failing. But in non-ZK mode it worked just fine! It took a while, but it seems that we upload to zk as a text, and download as well. I've attached a unit test that demonstrates the problem, and a fix. You have to have a binary file in the conf directory to make the test work, I put solr.png in the collection1/conf/velocity directory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-1972) Need additional query stats in admin interface - median, 95th and 99th percentile
[ https://issues.apache.org/jira/browse/SOLR-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shawn Heisey updated SOLR-1972: --- Fix Version/s: (was: 4.1) 4.2 Need additional query stats in admin interface - median, 95th and 99th percentile - Key: SOLR-1972 URL: https://issues.apache.org/jira/browse/SOLR-1972 Project: Solr Issue Type: Improvement Components: web gui Affects Versions: 1.4 Reporter: Shawn Heisey Assignee: Alan Woodward Priority: Minor Fix For: 4.2, 5.0 Attachments: elyograg-1972-3.2.patch, elyograg-1972-3.2.patch, elyograg-1972-trunk.patch, elyograg-1972-trunk.patch, leak-closeable.patch, leak.patch, SOLR-1972-branch3x-url_pattern.patch, SOLR-1972-branch4x.patch, SOLR-1972-branch4x.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, solr1972-metricsregistry-branch4x-failure.log, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972-url_pattern.patch, stacktraces.tar.gz I would like to see more detailed query statistics from the admin GUI. This is what you can get now: requests : 809 errors : 0 timeouts : 0 totalTime : 70053 avgTimePerRequest : 86.59209 avgRequestsPerSecond : 0.8148785 I'd like to see more data on the time per request - median, 95th percentile, 99th percentile, and any other statistical function that makes sense to include. In my environment, the first bunch of queries after startup tend to take several seconds each. I find that the average value tends to be useless until it has several thousand queries under its belt and the caches are thoroughly warmed. The statistical functions I have mentioned would quickly eliminate the influence of those initial slow queries. The system will have to store individual data about each query. I don't know if this is something Solr does already. It would be nice to have a configurable count of how many of the most recent data points are kept, to control the amount of memory the feature uses. The default value could be something like 1024 or 4096. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [VOTE] Release PyLucene 3.6.2
On Wed, 26 Dec 2012, Andi Vajda wrote: On Dec 26, 2012, at 11:58, Robert Muir rcm...@gmail.com wrote: On Wed, Dec 26, 2012 at 11:14 AM, Andi Vajda va...@apache.org wrote: Very strange. Why would it go out to pypi to install unrelated packages ? Odd. Did you run just 'make' first before running 'make test' ? (my workflow). I just tried make, followed by make test, and it worked fine. So I think i must have just tried 'make test' in one shot... must be a little build thing. doesn't seem like a blocker to me, just seemed a bit odd. I'll see if I can reproduce this. Thank you for the clarification. I did reproduce something similar running 'make test' after 'make clean'. Adding a dependency on 'all' fixed it. Andi.. Andi..
Re: tiny patch for java7 on os X
On Wed, 26 Dec 2012, Robert Muir wrote:
i installed java7 on my os X... with the following build patch
pylucene seems to work fine (tests pass etc).
I think java7 is just pickier about -source/-target both being set for
jcc. And the extensions should use the same explicit source/target (or
the build can hit classfile version problems).
Applied in rev 1426118, thank you !
Andi..
Index: extensions.xml
===
--- extensions.xml (revision 1425975)
+++ extensions.xml (working copy)
@@ -16,7 +16,7 @@
target name=compile
mkdir dir=${classes.dir}/
-javac srcdir=java destdir=${classes.dir} classpathref=classpath /
+javac srcdir=java destdir=${classes.dir}
classpathref=classpath source=1.5 target=1.5 /
/target
target name=jar depends=compile
Index: jcc/setup.py
===
--- jcc/setup.py(revision 1425975)
+++ jcc/setup.py(working copy)
@@ -149,7 +149,7 @@
LFLAGS['linux2'] = LFLAGS['linux2/%s' %(machine)]
JAVAC = {
-'darwin': ['javac', '-target', '1.5'],
+'darwin': ['javac', '-source', '1.5', '-target', '1.5'],
'ipod': ['jikes', '-cp', '/usr/share/classpath/glibj.zip'],
'linux2': ['javac'],
'sunos5': ['javac'],
http://pastebin.com/raw.php?i=qHpMw9Na
[jira] [Updated] (SOLR-1972) Need additional query stats in admin interface - median, 95th and 99th percentile
[ https://issues.apache.org/jira/browse/SOLR-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shawn Heisey updated SOLR-1972: --- Attachment: revert-SOLR-1972.patch Since I know we have to revert this to keep the 4.1 release clean, I might as well save everyone some time. Here is a patch to revert the latest metrics patch as well as every related mention of yammer and metrics that I could find. It's against 4x but applies 100% cleanly to trunk as well. All Solr tests pass on 4x. In the time that it took for the tests to run, I have realized that changing the fix version to 4.2 may have been premature - probably should have waited until a revert patch was committed. Need additional query stats in admin interface - median, 95th and 99th percentile - Key: SOLR-1972 URL: https://issues.apache.org/jira/browse/SOLR-1972 Project: Solr Issue Type: Improvement Components: web gui Affects Versions: 1.4 Reporter: Shawn Heisey Assignee: Alan Woodward Priority: Minor Fix For: 4.2, 5.0 Attachments: elyograg-1972-3.2.patch, elyograg-1972-3.2.patch, elyograg-1972-trunk.patch, elyograg-1972-trunk.patch, leak-closeable.patch, leak.patch, revert-SOLR-1972.patch, SOLR-1972-branch3x-url_pattern.patch, SOLR-1972-branch4x.patch, SOLR-1972-branch4x.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, SOLR-1972_metrics.patch, solr1972-metricsregistry-branch4x-failure.log, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch, SOLR-1972-url_pattern.patch, stacktraces.tar.gz I would like to see more detailed query statistics from the admin GUI. This is what you can get now: requests : 809 errors : 0 timeouts : 0 totalTime : 70053 avgTimePerRequest : 86.59209 avgRequestsPerSecond : 0.8148785 I'd like to see more data on the time per request - median, 95th percentile, 99th percentile, and any other statistical function that makes sense to include. In my environment, the first bunch of queries after startup tend to take several seconds each. I find that the average value tends to be useless until it has several thousand queries under its belt and the caches are thoroughly warmed. The statistical functions I have mentioned would quickly eliminate the influence of those initial slow queries. The system will have to store individual data about each query. I don't know if this is something Solr does already. It would be nice to have a configurable count of how many of the most recent data points are kept, to control the amount of memory the feature uses. The default value could be something like 1024 or 4096. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-4236) Commit issue: Can't search while add commit=true in the call URL about insert index
Raintung Li created SOLR-4236: - Summary: Commit issue: Can't search while add commit=true in the call URL about insert index Key: SOLR-4236 URL: https://issues.apache.org/jira/browse/SOLR-4236 Project: Solr Issue Type: Bug Components: SolrCloud Affects Versions: 4.0, 4.0-BETA Environment: one collection, one shard, three sever, one leader, two duplicate Reporter: Raintung Li I setup three instances for solr cloud for one same collection and shards, the cloud is one instance is shard leader and the others are replicate. Send the index request to one instance, the URL example like this. curl http://localhost:7002/solr/update?commit=true; -H Content-Type: text/xml --data-binary 'adddocfield name=idtest/field/doc/add' If send the request to the leader server, only the leader server can search this index, the replicate can't search. I close the autoSoftCommit. If request send to the replicate server, all servers can't search this index. The major problem: SolrCmdDistributor distribAdd method will batch some requests in the cache. DistributedUpdateProcessor class method processCommit will trigger the send the distribute request after the send commit request. If send the testing index's request to replicate server, replicate server will dispatch the request to leader server. But in this case, commit command will send to the other server before actually index request. The index can be searched only wait the softCommit or the other commit command coming. A litter confuse: Why commit command don't need the leader server send to duplicate server? Only receive request server send the commit to full shards server? It look like solr doesn't implement the transaction logic. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
