Re: [VOTE] Release PyLucene 4.5.0-2
+1 Having installed setuptools 1.1.6 for the previous RC, I successfully installed JCC and pylucene and got 0 failures from 'make test'. One small thing about the installation instructions: I had to run 'make test' as root because of some permissions issues (since 'make install' was run as root and apparently took ownership of some files in the unpacked distribution directory) - seems like that shouldn't be necessary. - running build_ext running install running bdist_egg running egg_info writing lucene.egg-info/PKG-INFO error: lucene.egg-info/PKG-INFO: Permission denied make: *** [install-test] Error 1 - Steve On Oct 13, 2013, at 11:04 PM, Andi Vajda va...@apache.org wrote: The PyLucene 4.5.0-2 release tracking the recent release of Apache Lucene 4.5.0 is ready. A release candidate is available from: http://people.apache.org/~vajda/staging_area/ A list of changes in this release can be seen at: http://svn.apache.org/repos/asf/lucene/pylucene/branches/pylucene_4_5/CHANGES PyLucene 4.5.0 is built with JCC 2.18 included in these release artifacts: http://svn.apache.org/repos/asf/lucene/pylucene/trunk/jcc/CHANGES A list of Lucene Java changes can be seen at: http://svn.apache.org/repos/asf/lucene/dev/tags/lucene_solr_4_5_0/lucene/CHANGES.txt Please vote to release these artifacts as PyLucene 4.5.0-2. Thanks ! Andi.. ps: the KEYS file for PyLucene release signing is at: http://svn.apache.org/repos/asf/lucene/pylucene/dist/KEYS http://people.apache.org/~vajda/staging_area/KEYS pps: here is my +1
[jira] [Closed] (SOLR-5215) Deadlock in Solr Cloud ConnectionManager
[ https://issues.apache.org/jira/browse/SOLR-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar closed SOLR-5215. --- Deadlock in Solr Cloud ConnectionManager Key: SOLR-5215 URL: https://issues.apache.org/jira/browse/SOLR-5215 Project: Solr Issue Type: Bug Components: clients - java, SolrCloud Affects Versions: 4.2.1 Environment: Linux 2.6.18-164.el5 #1 SMP Tue Aug 18 15:51:48 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux java version 1.6.0_18 Java(TM) SE Runtime Environment (build 1.6.0_18-b07) Java HotSpot(TM) 64-Bit Server VM (build 16.0-b13, mixed mode) Reporter: Ricardo Merizalde Assignee: Mark Miller Fix For: 4.5, 5.0 Attachments: SOLR-5215.patch We are constantly seeing a deadlocks in our production application servers. The problem seems to be that a thread A: - tries to process an event and acquires the ConnectionManager lock - the update callback acquires connectionUpdateLock and invokes waitForConnected - waitForConnected tries to acquire the ConnectionManager lock (which already has) - waitForConnected calls wait and release the ConnectionManager lock (but still has the connectionUpdateLock) The a thread B: - tries to process an event and acquires the ConnectionManager lock - the update call back tries to acquire connectionUpdateLock but gets blocked holding the ConnectionManager lock and preventing thread A from getting out of the wait state. Here is part of the thread dump: http-0.0.0.0-8080-82-EventThread daemon prio=10 tid=0x59965800 nid=0x3e81 waiting for monitor entry [0x57169000] java.lang.Thread.State: BLOCKED (on object monitor) at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:71) - waiting to lock 0x2aab1b0e0ce0 (a org.apache.solr.common.cloud.ConnectionManager) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495) http-0.0.0.0-8080-82-EventThread daemon prio=10 tid=0x5ad4 nid=0x3e67 waiting for monitor entry [0x4dbd4000] java.lang.Thread.State: BLOCKED (on object monitor) at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:98) - waiting to lock 0x2aab1b0e0f78 (a java.lang.Object) at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:46) at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:91) - locked 0x2aab1b0e0ce0 (a org.apache.solr.common.cloud.ConnectionManager) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495) http-0.0.0.0-8080-82-EventThread daemon prio=10 tid=0x2aac4c2f7000 nid=0x3d9a waiting for monitor entry [0x42821000] java.lang.Thread.State: BLOCKED (on object monitor) at java.lang.Object.wait(Native Method) - waiting on 0x2aab1b0e0ce0 (a org.apache.solr.common.cloud.ConnectionManager) at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:165) - locked 0x2aab1b0e0ce0 (a org.apache.solr.common.cloud.ConnectionManager) at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:98) - locked 0x2aab1b0e0f78 (a java.lang.Object) at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:46) at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:91) - locked 0x2aab1b0e0ce0 (a org.apache.solr.common.cloud.ConnectionManager) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495) Found one Java-level deadlock: = http-0.0.0.0-8080-82-EventThread: waiting to lock monitor 0x5c7694b0 (object 0x2aab1b0e0ce0, a org.apache.solr.common.cloud.ConnectionManager), which is held by http-0.0.0.0-8080-82-EventThread http-0.0.0.0-8080-82-EventThread: waiting to lock monitor 0x2aac4c314978 (object 0x2aab1b0e0f78, a java.lang.Object), which is held by http-0.0.0.0-8080-82-EventThread http-0.0.0.0-8080-82-EventThread: waiting to lock monitor 0x5c7694b0 (object 0x2aab1b0e0ce0, a org.apache.solr.common.cloud.ConnectionManager), which is held by http-0.0.0.0-8080-82-EventThread Java stack information for the threads
[jira] [Resolved] (SOLR-5215) Deadlock in Solr Cloud ConnectionManager
[ https://issues.apache.org/jira/browse/SOLR-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar resolved SOLR-5215. - Resolution: Fixed Fix Version/s: (was: 4.6) 5.0 4.5 This fix was released in 4.5 Deadlock in Solr Cloud ConnectionManager Key: SOLR-5215 URL: https://issues.apache.org/jira/browse/SOLR-5215 Project: Solr Issue Type: Bug Components: clients - java, SolrCloud Affects Versions: 4.2.1 Environment: Linux 2.6.18-164.el5 #1 SMP Tue Aug 18 15:51:48 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux java version 1.6.0_18 Java(TM) SE Runtime Environment (build 1.6.0_18-b07) Java HotSpot(TM) 64-Bit Server VM (build 16.0-b13, mixed mode) Reporter: Ricardo Merizalde Assignee: Mark Miller Fix For: 4.5, 5.0 Attachments: SOLR-5215.patch We are constantly seeing a deadlocks in our production application servers. The problem seems to be that a thread A: - tries to process an event and acquires the ConnectionManager lock - the update callback acquires connectionUpdateLock and invokes waitForConnected - waitForConnected tries to acquire the ConnectionManager lock (which already has) - waitForConnected calls wait and release the ConnectionManager lock (but still has the connectionUpdateLock) The a thread B: - tries to process an event and acquires the ConnectionManager lock - the update call back tries to acquire connectionUpdateLock but gets blocked holding the ConnectionManager lock and preventing thread A from getting out of the wait state. Here is part of the thread dump: http-0.0.0.0-8080-82-EventThread daemon prio=10 tid=0x59965800 nid=0x3e81 waiting for monitor entry [0x57169000] java.lang.Thread.State: BLOCKED (on object monitor) at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:71) - waiting to lock 0x2aab1b0e0ce0 (a org.apache.solr.common.cloud.ConnectionManager) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495) http-0.0.0.0-8080-82-EventThread daemon prio=10 tid=0x5ad4 nid=0x3e67 waiting for monitor entry [0x4dbd4000] java.lang.Thread.State: BLOCKED (on object monitor) at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:98) - waiting to lock 0x2aab1b0e0f78 (a java.lang.Object) at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:46) at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:91) - locked 0x2aab1b0e0ce0 (a org.apache.solr.common.cloud.ConnectionManager) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495) http-0.0.0.0-8080-82-EventThread daemon prio=10 tid=0x2aac4c2f7000 nid=0x3d9a waiting for monitor entry [0x42821000] java.lang.Thread.State: BLOCKED (on object monitor) at java.lang.Object.wait(Native Method) - waiting on 0x2aab1b0e0ce0 (a org.apache.solr.common.cloud.ConnectionManager) at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:165) - locked 0x2aab1b0e0ce0 (a org.apache.solr.common.cloud.ConnectionManager) at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:98) - locked 0x2aab1b0e0f78 (a java.lang.Object) at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:46) at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:91) - locked 0x2aab1b0e0ce0 (a org.apache.solr.common.cloud.ConnectionManager) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495) Found one Java-level deadlock: = http-0.0.0.0-8080-82-EventThread: waiting to lock monitor 0x5c7694b0 (object 0x2aab1b0e0ce0, a org.apache.solr.common.cloud.ConnectionManager), which is held by http-0.0.0.0-8080-82-EventThread http-0.0.0.0-8080-82-EventThread: waiting to lock monitor 0x2aac4c314978 (object 0x2aab1b0e0f78, a java.lang.Object), which is held by http-0.0.0.0-8080-82-EventThread http-0.0.0.0-8080-82-EventThread: waiting to lock monitor 0x5c7694b0 (object 0x2aab1b0e0ce0, a
[jira] [Commented] (LUCENE-5236) Use broadword bit selection in EliasFanoDecoder
[
https://issues.apache.org/jira/browse/LUCENE-5236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794015#comment-13794015
]
Adrien Grand commented on LUCENE-5236:
--
Thanks, {{ant precommit}} and tests ran successfully.
bq. fails with the error message Linting documentation HTML is not supported
on this Java version (1.7).
I don't clearly understand what triggers this message but it works for me
although I have 1.7 as well.
bq. OpenBitSet.bits2words will fail when the number of bits gets just over the
maximum int value, so I prefer not to use it for now.
You mean you don't like the fact that the cast is unchecked, right? I guess we
should fix it and use it?
bq. I am looking forward to the benchmark results on a 64 bits machine.
Here it is, and it is excellent!
http://people.apache.org/~jpountz/doc_id_sets.html
I'll commit your latest patch soon if you have no objection.
Use broadword bit selection in EliasFanoDecoder
---
Key: LUCENE-5236
URL: https://issues.apache.org/jira/browse/LUCENE-5236
Project: Lucene - Core
Issue Type: Improvement
Reporter: Paul Elschot
Assignee: Adrien Grand
Priority: Minor
Attachments: LUCENE-5236.patch, LUCENE-5236.patch, LUCENE-5236.patch,
LUCENE-5236.patch, TestDocIdSetBenchmark.java
Try and speed up decoding
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2548) Multithreaded faceting
[ https://issues.apache.org/jira/browse/SOLR-2548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794018#comment-13794018 ] alexey commented on SOLR-2548: -- We observed 4x speedup when calculating 14 facets in 6 threads for 200mln index. Thanks everybody! https://twitter.com/AlexKozhemiakin/status/389688204309196800 Multithreaded faceting -- Key: SOLR-2548 URL: https://issues.apache.org/jira/browse/SOLR-2548 Project: Solr Issue Type: Improvement Components: search Affects Versions: 3.1 Reporter: Janne Majaranta Assignee: Erick Erickson Priority: Minor Labels: facet Fix For: 4.5, 5.0 Attachments: SOLR-2548_4.2.1.patch, SOLR-2548_for_31x.patch, SOLR-2548_multithreaded_faceting,_dsmiley.patch, SOLR-2548_multithreaded_faceting,_dsmiley.patch, SOLR-2548.patch, SOLR-2548.patch, SOLR-2548.patch, SOLR-2548.patch, SOLR-2548.patch, SOLR-2548.patch Add multithreading support for faceting. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5338) Split shards by a route key
[
https://issues.apache.org/jira/browse/SOLR-5338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shalin Shekhar Mangar updated SOLR-5338:
Description:
Provide a way to split a shard using a route key such that all documents of the
specified route key end up in a single dedicated sub-shard.
Example:
Assume that collection1, shard1 has hash range [0, 20]. Also that route key
'A!' has hash range [12,15]. Then invoking:
{code}
/admin/collections?action=SPLITcollection=collection1split.key=A!
{code}
should produce three sub-shards with hash range [0,11], [12,15] and [16,20].
Specifying the source shard is not required here because the route key is
enough to figure it out. Route keys spanning more than one shards will not be
supported.
Note that the sub-shard with the hash range of the route key may also contain
documents for other route keys whose hash ranges overlap.
was:
Provide a way to split a shard using a route key such that all documents of the
specified route key end up in a single dedicated sub-shard.
Example:
Assume that collection1, shard1 has hash range [0, 20]. Also that route key
'A!' has hash range [12,15]. Then invoking:
{code}
/admin/collections?action=SPLITcollection=collection1split.key=A!
{code}
should produce three sub-shards with hash range [0,11], [12,15] and [16,20].
Specifying the source shard is not required here because the route key is
enough to figure it out. Route keys spanning more than one shards will not be
supported.
Note that the sub-shard with the hash range of the route key may also contain
documents for other route keys whose hashes collide.
Split shards by a route key
---
Key: SOLR-5338
URL: https://issues.apache.org/jira/browse/SOLR-5338
Project: Solr
Issue Type: Improvement
Components: SolrCloud
Reporter: Shalin Shekhar Mangar
Assignee: Shalin Shekhar Mangar
Fix For: 4.6, 5.0
Attachments: SOLR-5338.patch, SOLR-5338.patch, SOLR-5338.patch
Provide a way to split a shard using a route key such that all documents of
the specified route key end up in a single dedicated sub-shard.
Example:
Assume that collection1, shard1 has hash range [0, 20]. Also that route key
'A!' has hash range [12,15]. Then invoking:
{code}
/admin/collections?action=SPLITcollection=collection1split.key=A!
{code}
should produce three sub-shards with hash range [0,11], [12,15] and [16,20].
Specifying the source shard is not required here because the route key is
enough to figure it out. Route keys spanning more than one shards will not be
supported.
Note that the sub-shard with the hash range of the route key may also contain
documents for other route keys whose hash ranges overlap.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-5347) DataImportHandler 'start' parameter doesn't work unless 'rows' is also specified
Shalin Shekhar Mangar created SOLR-5347:
---
Summary: DataImportHandler 'start' parameter doesn't work unless
'rows' is also specified
Key: SOLR-5347
URL: https://issues.apache.org/jira/browse/SOLR-5347
Project: Solr
Issue Type: Bug
Components: contrib - DataImportHandler
Affects Versions: 4.5
Reporter: Shalin Shekhar Mangar
Assignee: Shalin Shekhar Mangar
Priority: Minor
Fix For: 4.6, 5.0
The 'rows' param is initialized to Long.MAX_VALUE by default. Therefore when
'start' parameter is specified and DocBuilder does the following check:
{code}
if(importStatistics.docCount.get() (reqParams.getStart() +
reqParams.getRows())) break;
{code}
The R.H.S. overflows and the condition evaluates to true. The end result is
that no documents are ever indexed.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5263) Deletes may be silently lost if an IOException is hit and later not hit (e.g., disk fills up and then frees up)
[
https://issues.apache.org/jira/browse/LUCENE-5263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794025#comment-13794025
]
ASF subversion and git services commented on LUCENE-5263:
-
Commit 1531837 from [~mikemccand] in branch 'dev/trunk'
[ https://svn.apache.org/r1531837 ]
LUCENE-5263: fix comment
Deletes may be silently lost if an IOException is hit and later not hit
(e.g., disk fills up and then frees up)
---
Key: LUCENE-5263
URL: https://issues.apache.org/jira/browse/LUCENE-5263
Project: Lucene - Core
Issue Type: Bug
Components: core/index
Reporter: Michael McCandless
Assignee: Michael McCandless
Fix For: 4.5.1, 4.6, 5.0
Attachments: LUCENE-5263.patch, LUCENE-5263.patch
This case is tricky to handle, yet I think realistic: disk fills up
temporarily, causes an exception in writeLiveDocs, and then the app
keeps using the IW instance.
Meanwhile disk later frees up again, IW is closed successfully. In
certain cases, we can silently lose deletes in this case.
I had already committed
TestIndexWriterDeletes.testNoLostDeletesOnDiskFull, and Jenkins seems
happy with it so far, but when I added fangs to the test (cutover to
RandomIndexWriter from IndexWriter, allow IOE during getReader, add
randomness to when exc is thrown, etc.), it uncovered some real/nasty
bugs:
* ReaderPool.dropAll was suppressing any exception it hit, because
{code}if (priorE != null){code} should instead be {code}if (priorE ==
null){code}
* After a merge, we have to write deletes before committing the
segment, because an exception when writing deletes means we need
to abort the merge
* Several places that were directly calling deleter.checkpoint must
also increment the changeCount else on close IW thinks there are
no changes and doesn't write a new segments file.
* closeInternal was dropping pooled readers after writing the
segments file, which would lose deletes still buffered due to a
previous exc.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5263) Deletes may be silently lost if an IOException is hit and later not hit (e.g., disk fills up and then frees up)
[
https://issues.apache.org/jira/browse/LUCENE-5263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794024#comment-13794024
]
ASF subversion and git services commented on LUCENE-5263:
-
Commit 1531836 from [~mikemccand] in branch 'dev/trunk'
[ https://svn.apache.org/r1531836 ]
LUCENE-5263: if merge hits exc when releasing merged reader then drop changes
and remove it from the pool, since the merge will be aborted
Deletes may be silently lost if an IOException is hit and later not hit
(e.g., disk fills up and then frees up)
---
Key: LUCENE-5263
URL: https://issues.apache.org/jira/browse/LUCENE-5263
Project: Lucene - Core
Issue Type: Bug
Components: core/index
Reporter: Michael McCandless
Assignee: Michael McCandless
Fix For: 4.5.1, 4.6, 5.0
Attachments: LUCENE-5263.patch, LUCENE-5263.patch
This case is tricky to handle, yet I think realistic: disk fills up
temporarily, causes an exception in writeLiveDocs, and then the app
keeps using the IW instance.
Meanwhile disk later frees up again, IW is closed successfully. In
certain cases, we can silently lose deletes in this case.
I had already committed
TestIndexWriterDeletes.testNoLostDeletesOnDiskFull, and Jenkins seems
happy with it so far, but when I added fangs to the test (cutover to
RandomIndexWriter from IndexWriter, allow IOE during getReader, add
randomness to when exc is thrown, etc.), it uncovered some real/nasty
bugs:
* ReaderPool.dropAll was suppressing any exception it hit, because
{code}if (priorE != null){code} should instead be {code}if (priorE ==
null){code}
* After a merge, we have to write deletes before committing the
segment, because an exception when writing deletes means we need
to abort the merge
* Several places that were directly calling deleter.checkpoint must
also increment the changeCount else on close IW thinks there are
no changes and doesn't write a new segments file.
* closeInternal was dropping pooled readers after writing the
segments file, which would lose deletes still buffered due to a
previous exc.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5263) Deletes may be silently lost if an IOException is hit and later not hit (e.g., disk fills up and then frees up)
[
https://issues.apache.org/jira/browse/LUCENE-5263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794026#comment-13794026
]
ASF subversion and git services commented on LUCENE-5263:
-
Commit 1531838 from [~mikemccand] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1531838 ]
LUCENE-5263: if merge hits exc when releasing merged reader then drop changes
and remove it from the pool, since the merge will be aborted
Deletes may be silently lost if an IOException is hit and later not hit
(e.g., disk fills up and then frees up)
---
Key: LUCENE-5263
URL: https://issues.apache.org/jira/browse/LUCENE-5263
Project: Lucene - Core
Issue Type: Bug
Components: core/index
Reporter: Michael McCandless
Assignee: Michael McCandless
Fix For: 4.5.1, 4.6, 5.0
Attachments: LUCENE-5263.patch, LUCENE-5263.patch
This case is tricky to handle, yet I think realistic: disk fills up
temporarily, causes an exception in writeLiveDocs, and then the app
keeps using the IW instance.
Meanwhile disk later frees up again, IW is closed successfully. In
certain cases, we can silently lose deletes in this case.
I had already committed
TestIndexWriterDeletes.testNoLostDeletesOnDiskFull, and Jenkins seems
happy with it so far, but when I added fangs to the test (cutover to
RandomIndexWriter from IndexWriter, allow IOE during getReader, add
randomness to when exc is thrown, etc.), it uncovered some real/nasty
bugs:
* ReaderPool.dropAll was suppressing any exception it hit, because
{code}if (priorE != null){code} should instead be {code}if (priorE ==
null){code}
* After a merge, we have to write deletes before committing the
segment, because an exception when writing deletes means we need
to abort the merge
* Several places that were directly calling deleter.checkpoint must
also increment the changeCount else on close IW thinks there are
no changes and doesn't write a new segments file.
* closeInternal was dropping pooled readers after writing the
segments file, which would lose deletes still buffered due to a
previous exc.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5263) Deletes may be silently lost if an IOException is hit and later not hit (e.g., disk fills up and then frees up)
[
https://issues.apache.org/jira/browse/LUCENE-5263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794027#comment-13794027
]
ASF subversion and git services commented on LUCENE-5263:
-
Commit 1531839 from [~mikemccand] in branch 'dev/branches/lucene_solr_4_5'
[ https://svn.apache.org/r1531839 ]
LUCENE-5263: if merge hits exc when releasing merged reader then drop changes
and remove it from the pool, since the merge will be aborted
Deletes may be silently lost if an IOException is hit and later not hit
(e.g., disk fills up and then frees up)
---
Key: LUCENE-5263
URL: https://issues.apache.org/jira/browse/LUCENE-5263
Project: Lucene - Core
Issue Type: Bug
Components: core/index
Reporter: Michael McCandless
Assignee: Michael McCandless
Fix For: 4.5.1, 4.6, 5.0
Attachments: LUCENE-5263.patch, LUCENE-5263.patch
This case is tricky to handle, yet I think realistic: disk fills up
temporarily, causes an exception in writeLiveDocs, and then the app
keeps using the IW instance.
Meanwhile disk later frees up again, IW is closed successfully. In
certain cases, we can silently lose deletes in this case.
I had already committed
TestIndexWriterDeletes.testNoLostDeletesOnDiskFull, and Jenkins seems
happy with it so far, but when I added fangs to the test (cutover to
RandomIndexWriter from IndexWriter, allow IOE during getReader, add
randomness to when exc is thrown, etc.), it uncovered some real/nasty
bugs:
* ReaderPool.dropAll was suppressing any exception it hit, because
{code}if (priorE != null){code} should instead be {code}if (priorE ==
null){code}
* After a merge, we have to write deletes before committing the
segment, because an exception when writing deletes means we need
to abort the merge
* Several places that were directly calling deleter.checkpoint must
also increment the changeCount else on close IW thinks there are
no changes and doesn't write a new segments file.
* closeInternal was dropping pooled readers after writing the
segments file, which would lose deletes still buffered due to a
previous exc.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-5347) DataImportHandler 'start' parameter doesn't work unless 'rows' is also specified
[
https://issues.apache.org/jira/browse/SOLR-5347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shalin Shekhar Mangar resolved SOLR-5347.
-
Resolution: Invalid
Fix Version/s: (was: 4.6)
(was: 5.0)
I was mistaken. The 'start' and 'rows' parameter are only for debug mode. They
aren't supported in non-debug mode.
DataImportHandler 'start' parameter doesn't work unless 'rows' is also
specified
Key: SOLR-5347
URL: https://issues.apache.org/jira/browse/SOLR-5347
Project: Solr
Issue Type: Bug
Components: contrib - DataImportHandler
Affects Versions: 4.5
Reporter: Shalin Shekhar Mangar
Assignee: Shalin Shekhar Mangar
Priority: Minor
The 'rows' param is initialized to Long.MAX_VALUE by default. Therefore when
'start' parameter is specified and DocBuilder does the following check:
{code}
if(importStatistics.docCount.get() (reqParams.getStart() +
reqParams.getRows())) break;
{code}
The R.H.S. overflows and the condition evaluates to true. The end result is
that no documents are ever indexed.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5280) Rename TermFreqPayloadIterator - SuggestionIterator
[ https://issues.apache.org/jira/browse/LUCENE-5280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794040#comment-13794040 ] Michael McCandless commented on LUCENE-5280: +1 for InputIterator Rename TermFreqPayloadIterator - SuggestionIterator Key: LUCENE-5280 URL: https://issues.apache.org/jira/browse/LUCENE-5280 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Fix For: 4.6, 5.0 Attachments: LUCENE-5280.patch The current name is cumbersome, and annoying to change whenever we add something to the iterator (in this case payloads). Since we are breaking it anyway in 4.6, I think we should take the opportunity to find a better name. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4956) the korean analyzer that has a korean morphological analyzer and dictionaries
[ https://issues.apache.org/jira/browse/LUCENE-4956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794046#comment-13794046 ] Christian Moen commented on LUCENE-4956: Soomyung and myself met up in Seoul today and we've merged his latest locally. I'll commit the changes to this branch when I'm back in Tokyo and Soomyung will follow up with fixing a known issue afterwards. Hopefully we can commit this to trunk very soon. the korean analyzer that has a korean morphological analyzer and dictionaries - Key: LUCENE-4956 URL: https://issues.apache.org/jira/browse/LUCENE-4956 Project: Lucene - Core Issue Type: New Feature Components: modules/analysis Affects Versions: 4.2 Reporter: SooMyung Lee Assignee: Christian Moen Labels: newbie Attachments: kr.analyzer.4x.tar, lucene-4956.patch, lucene4956.patch, LUCENE-4956.patch Korean language has specific characteristic. When developing search service with lucene solr in korean, there are some problems in searching and indexing. The korean analyer solved the problems with a korean morphological anlyzer. It consists of a korean morphological analyzer, dictionaries, a korean tokenizer and a korean filter. The korean anlyzer is made for lucene and solr. If you develop a search service with lucene in korean, It is the best idea to choose the korean analyzer. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0-ea-b106) - Build # 7872 - Failure!
I committed a fix for this one. Mike McCandless http://blog.mikemccandless.com On Sun, Oct 13, 2013 at 3:31 PM, Michael McCandless luc...@mikemccandless.com wrote: I'll dig. Mike McCandless http://blog.mikemccandless.com On Sun, Oct 13, 2013 at 3:25 PM, Policeman Jenkins Server jenk...@thetaphi.de wrote: Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/7872/ Java: 64bit/jdk1.8.0-ea-b106 -XX:-UseCompressedOops -XX:+UseParallelGC 1 tests failed. REGRESSION: org.apache.lucene.index.TestIndexWriterDelete.testNoLostDeletesOrUpdatesOnIOException Error Message: _l.fnm in dir=org.apache.lucene.store.RAMDirectory@16706731 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4fe13ade Stack Trace: java.io.FileNotFoundException: _l.fnm in dir=org.apache.lucene.store.RAMDirectory@16706731 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4fe13ade at __randomizedtesting.SeedInfo.seed([4B1A4BEDD58638FB:2823537CA70B2F36]:0) at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:530) at org.apache.lucene.codecs.lucene46.Lucene46FieldInfosReader.read(Lucene46FieldInfosReader.java:52) at org.apache.lucene.index.SegmentReader.readFieldInfos(SegmentReader.java:251) at org.apache.lucene.index.SegmentReader.init(SegmentReader.java:106) at org.apache.lucene.index.ReadersAndLiveDocs.writeLiveDocs(ReadersAndLiveDocs.java:367) at org.apache.lucene.index.IndexWriter$ReaderPool.dropAll(IndexWriter.java:497) at org.apache.lucene.index.IndexWriter.closeInternal(IndexWriter.java:1038) at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:933) at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:895) at org.apache.lucene.index.RandomIndexWriter.close(RandomIndexWriter.java:333) at org.apache.lucene.index.TestIndexWriterDelete.testNoLostDeletesOrUpdatesOnIOException(TestIndexWriterDelete.java:1363) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:491) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at
[jira] [Commented] (SOLR-2548) Multithreaded faceting
[ https://issues.apache.org/jira/browse/SOLR-2548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794064#comment-13794064 ] Erick Erickson commented on SOLR-2548: -- Alexey: Thanks for the feedback. This is one of those things that's very dependent on faceting on more then one field/query and the facets being fairly complex/expensive. But when those conditions are met, it's very noticeable. Multithreaded faceting -- Key: SOLR-2548 URL: https://issues.apache.org/jira/browse/SOLR-2548 Project: Solr Issue Type: Improvement Components: search Affects Versions: 3.1 Reporter: Janne Majaranta Assignee: Erick Erickson Priority: Minor Labels: facet Fix For: 4.5, 5.0 Attachments: SOLR-2548_4.2.1.patch, SOLR-2548_for_31x.patch, SOLR-2548_multithreaded_faceting,_dsmiley.patch, SOLR-2548_multithreaded_faceting,_dsmiley.patch, SOLR-2548.patch, SOLR-2548.patch, SOLR-2548.patch, SOLR-2548.patch, SOLR-2548.patch, SOLR-2548.patch Add multithreading support for faceting. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5282) speed up javadocs generation tasks
[
https://issues.apache.org/jira/browse/LUCENE-5282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794068#comment-13794068
]
Uwe Schindler commented on LUCENE-5282:
---
Hi, status update:
It is not as easy as expected. My idea how to fix the macro was to use the
similar approach like the javadocs-packages-file downloader. This was invoked
by antcall, so if/unless properties were respected and updated. The idea I
had was the same: use {{local}} to define property, then {{uptodate}} on
the local property to find out if source files are newer than
{{@destdir/package.html}} (see is the recommended approach to detect javadocs
timestamp, see e.g. the official ANT build.xml), and finally execute the
javadocs task with {{antcall}}, passing the {{@}}-variables as parameters to
the task. The problem why this did not work is the {{nested}} sourcefiles
property, you cannot pass xml structures as parameters to a target. I found no
was around that.
The second problem: You cannot get the source folder with the HTML files from
the macro: you only have the nested element that has the package names and
directories somewhere in them.
In my opinion the only was to fix this would be a separate uptodate task in
common-build and all module's javadocs targets depend on that one + ifunless
check.
speed up javadocs generation tasks
--
Key: LUCENE-5282
URL: https://issues.apache.org/jira/browse/LUCENE-5282
Project: Lucene - Core
Issue Type: Bug
Reporter: Robert Muir
These generate the same things over and over. I think this is due to javadocs
always rebuilding their dependencies.
Can we not add a fake timestamp file (with 'touch') somewhere like
javadocs.generated and then use 'uptodate' comparing that against the
relevant source code to determine if javadocs need regeneration?
This seems like it would be a very simple solution.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-5282) speed up javadocs generation tasks
[
https://issues.apache.org/jira/browse/LUCENE-5282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794068#comment-13794068
]
Uwe Schindler edited comment on LUCENE-5282 at 10/14/13 12:02 PM:
--
Hi, status update:
It is not as easy as expected. My idea how to fix the macro was to use the
similar approach like the javadocs-packages-file downloader. This was invoked
by antcall, so if/unless properties were respected and updated. The idea I
had was the same: use {{local}} to define property, then {{uptodate}} on
the local property to find out if source files are newer than
{{@destdir/package.html}} (this is the recommended approach to detect javadocs
timestamp, see e.g. the official ANT build.xml), and finally execute the
javadocs task with {{antcall}}, passing the {{@}}-variables as parameters to
the task. The problem why this did not work is the {{nested}} sourcefiles
property, you cannot pass xml structures as parameters to a target. I found no
was around that.
The second problem: You cannot get the source folder with the Java files from
the macro: you only have the nested element that has the package names and
directories somewhere in them.
In my opinion the only was to fix this would be a separate uptodate task in
common-build and all module's javadocs targets depend on that one + ifunless
check.
was (Author: thetaphi):
Hi, status update:
It is not as easy as expected. My idea how to fix the macro was to use the
similar approach like the javadocs-packages-file downloader. This was invoked
by antcall, so if/unless properties were respected and updated. The idea I
had was the same: use {{local}} to define property, then {{uptodate}} on
the local property to find out if source files are newer than
{{@destdir/package.html}} (see is the recommended approach to detect javadocs
timestamp, see e.g. the official ANT build.xml), and finally execute the
javadocs task with {{antcall}}, passing the {{@}}-variables as parameters to
the task. The problem why this did not work is the {{nested}} sourcefiles
property, you cannot pass xml structures as parameters to a target. I found no
was around that.
The second problem: You cannot get the source folder with the HTML files from
the macro: you only have the nested element that has the package names and
directories somewhere in them.
In my opinion the only was to fix this would be a separate uptodate task in
common-build and all module's javadocs targets depend on that one + ifunless
check.
speed up javadocs generation tasks
--
Key: LUCENE-5282
URL: https://issues.apache.org/jira/browse/LUCENE-5282
Project: Lucene - Core
Issue Type: Bug
Reporter: Robert Muir
These generate the same things over and over. I think this is due to javadocs
always rebuilding their dependencies.
Can we not add a fake timestamp file (with 'touch') somewhere like
javadocs.generated and then use 'uptodate' comparing that against the
relevant source code to determine if javadocs need regeneration?
This seems like it would be a very simple solution.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-5282) speed up javadocs generation tasks
[
https://issues.apache.org/jira/browse/LUCENE-5282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794068#comment-13794068
]
Uwe Schindler edited comment on LUCENE-5282 at 10/14/13 12:05 PM:
--
Hi, status update:
It is not as easy as expected. My idea how to fix the macro was to use the
similar approach like the javadocs-packages-file downloader. This was invoked
by antcall, so if/unless properties were respected and updated. The idea I
had was the same: use {{local}} to define property, then {{uptodate}} on
the local property to find out if source files are newer than
{{@destdir/package.html}} (this is the recommended approach to detect javadocs
timestamp, see e.g. the official ANT build.xml:
[http://svn.apache.org/viewvc/ant/core/branches/ANT_18_BRANCH/build.xml?revision=1341924view=markup],
line 1438), and finally execute the javadocs task with {{antcall}}, passing
the {{@}}-variables as parameters to the task. The problem why this did not
work is the {{nested}} sourcefiles property, you cannot pass xml structures
as parameters to a target. I found no was around that.
The second problem: You cannot get the source folder with the Java files from
the macro: you only have the nested element that has the package names and
directories somewhere in them.
In my opinion the only was to fix this would be a separate uptodate task in
common-build and all module's javadocs targets depend on that one + ifunless
check.
was (Author: thetaphi):
Hi, status update:
It is not as easy as expected. My idea how to fix the macro was to use the
similar approach like the javadocs-packages-file downloader. This was invoked
by antcall, so if/unless properties were respected and updated. The idea I
had was the same: use {{local}} to define property, then {{uptodate}} on
the local property to find out if source files are newer than
{{@destdir/package.html}} (this is the recommended approach to detect javadocs
timestamp, see e.g. the official ANT build.xml), and finally execute the
javadocs task with {{antcall}}, passing the {{@}}-variables as parameters to
the task. The problem why this did not work is the {{nested}} sourcefiles
property, you cannot pass xml structures as parameters to a target. I found no
was around that.
The second problem: You cannot get the source folder with the Java files from
the macro: you only have the nested element that has the package names and
directories somewhere in them.
In my opinion the only was to fix this would be a separate uptodate task in
common-build and all module's javadocs targets depend on that one + ifunless
check.
speed up javadocs generation tasks
--
Key: LUCENE-5282
URL: https://issues.apache.org/jira/browse/LUCENE-5282
Project: Lucene - Core
Issue Type: Bug
Reporter: Robert Muir
These generate the same things over and over. I think this is due to javadocs
always rebuilding their dependencies.
Can we not add a fake timestamp file (with 'touch') somewhere like
javadocs.generated and then use 'uptodate' comparing that against the
relevant source code to determine if javadocs need regeneration?
This seems like it would be a very simple solution.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.7.0_40) - Build # 7779 - Failure!
I was able to repro this (took 253 iters beasting with this seed) before my commit for the other build failure, but after that commit I can't reproduce it, so far (at 2708 iters now)... So I think this was a different manifestation of the same issue: we were failing to purge the merged segments entry in the ReaderPool after hitting an exception that aborts the merge. Mike McCandless http://blog.mikemccandless.com On Sun, Oct 13, 2013 at 6:54 PM, Policeman Jenkins Server jenk...@thetaphi.de wrote: Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/7779/ Java: 32bit/jdk1.7.0_40 -client -XX:+UseParallelGC 1 tests failed. REGRESSION: org.apache.lucene.index.TestIndexWriterDelete.testNoLostDeletesOnIOException Error Message: info=_1g(4.6):c2482/32:delGen=2 isn't live Stack Trace: java.lang.AssertionError: info=_1g(4.6):c2482/32:delGen=2 isn't live at __randomizedtesting.SeedInfo.seed([3F31CC1FCA9DA422:A79869D48EA5B4A0]:0) at org.apache.lucene.index.IndexWriter$ReaderPool.infoIsLive(IndexWriter.java:428) at org.apache.lucene.index.IndexWriter$ReaderPool.dropAll(IndexWriter.java:497) at org.apache.lucene.index.IndexWriter.closeInternal(IndexWriter.java:1050) at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:945) at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:907) at org.apache.lucene.index.RandomIndexWriter.close(RandomIndexWriter.java:332) at org.apache.lucene.index.TestIndexWriterDelete.testNoLostDeletesOnIOException(TestIndexWriterDelete.java:1354) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) at
[jira] [Comment Edited] (LUCENE-5282) speed up javadocs generation tasks
[
https://issues.apache.org/jira/browse/LUCENE-5282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794068#comment-13794068
]
Uwe Schindler edited comment on LUCENE-5282 at 10/14/13 12:06 PM:
--
Hi, status update:
It is not as easy as expected. My idea how to fix the macro was to use the
similar approach like the javadocs-packages-file downloader. This was invoked
by antcall, so if/unless properties were respected and updated. The idea I
had was the same: use {{local}} to define property, then {{uptodate}} on
the local property to find out if source files are newer than
{{@destdir/package.html}} (this is the recommended approach to detect javadocs
timestamp, see e.g. the official ANT build.xml:
[http://svn.apache.org/viewvc/ant/core/branches/ANT_18_BRANCH/build.xml?revision=1341924view=markup],
line 1438), and finally execute the javadocs task with {{antcall}}, passing
the {{@}}-variables as parameters to the task. The problem why this did not
work is the {{nested}} sourcefiles property, you cannot pass xml structures
as parameters to a target. I found no way around that.
The second problem: You cannot get the source folder with the Java files from
the macro: you only have the nested element that has the package names and
directories somewhere in them.
In my opinion the only was to fix this would be a separate uptodate task in
common-build and all module's javadocs targets depend on that one + ifunless
check. ANT's own build.xml shows how to do this:
[http://svn.apache.org/viewvc/ant/core/branches/ANT_18_BRANCH/build.xml?revision=1341924view=markup],
line 1450.
was (Author: thetaphi):
Hi, status update:
It is not as easy as expected. My idea how to fix the macro was to use the
similar approach like the javadocs-packages-file downloader. This was invoked
by antcall, so if/unless properties were respected and updated. The idea I
had was the same: use {{local}} to define property, then {{uptodate}} on
the local property to find out if source files are newer than
{{@destdir/package.html}} (this is the recommended approach to detect javadocs
timestamp, see e.g. the official ANT build.xml:
[http://svn.apache.org/viewvc/ant/core/branches/ANT_18_BRANCH/build.xml?revision=1341924view=markup],
line 1438), and finally execute the javadocs task with {{antcall}}, passing
the {{@}}-variables as parameters to the task. The problem why this did not
work is the {{nested}} sourcefiles property, you cannot pass xml structures
as parameters to a target. I found no was around that.
The second problem: You cannot get the source folder with the Java files from
the macro: you only have the nested element that has the package names and
directories somewhere in them.
In my opinion the only was to fix this would be a separate uptodate task in
common-build and all module's javadocs targets depend on that one + ifunless
check.
speed up javadocs generation tasks
--
Key: LUCENE-5282
URL: https://issues.apache.org/jira/browse/LUCENE-5282
Project: Lucene - Core
Issue Type: Bug
Reporter: Robert Muir
These generate the same things over and over. I think this is due to javadocs
always rebuilding their dependencies.
Can we not add a fake timestamp file (with 'touch') somewhere like
javadocs.generated and then use 'uptodate' comparing that against the
relevant source code to determine if javadocs need regeneration?
This seems like it would be a very simple solution.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Should build fail if test does not exist?
Hi If you run something like ant test -Dtestcase=TestFoo where there's no TestFoo.java, or ant test -Dtestcase=TestBar -Dtests.method=testFoo where there's TestBar.java but no testFoo() method, the build currently passes as SUCCESSFUL. Though, the report says 0 suits or 1 suits, 0 tests. I wonder if it should be a build failure? The problem I have is that you can e.g. make a very stupid typo (usually around plurals as in testSomethingBadOnIOException vs testSomethingBadOnIOExceptions) and get a false SUCCESSFUL notification. If we want to fix/change it, where does it belong - build scripts, LuceneTestCase or randomizedrunner? Shai
[jira] [Updated] (LUCENE-5248) Improve the data structure used in ReaderAndLiveDocs to hold the updates
[ https://issues.apache.org/jira/browse/LUCENE-5248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shai Erera updated LUCENE-5248: --- Attachment: LUCENE-5248.patch Patch tracks docsWithFields in a parallel bitset to the docs/values arrays. Also modifies TestIndexWriterDelete.testNoLostDeletesOrUpdatesOnIOException to fail on either writeLiveDocs or writeFieldUpdates. I think it's ready - no nocommits left. Improve the data structure used in ReaderAndLiveDocs to hold the updates Key: LUCENE-5248 URL: https://issues.apache.org/jira/browse/LUCENE-5248 Project: Lucene - Core Issue Type: Improvement Components: core/index Reporter: Shai Erera Assignee: Shai Erera Attachments: LUCENE-5248.patch, LUCENE-5248.patch, LUCENE-5248.patch, LUCENE-5248.patch, LUCENE-5248.patch Currently ReaderAndLiveDocs holds the updates in two structures: +MapString,MapInteger,Long+ Holds a mapping from each field, to all docs that were updated and their values. This structure is updated when applyDeletes is called, and needs to satisfy several requirements: # Un-ordered writes: if a field f is updated by two terms, termA and termB, in that order, and termA affects doc=100 and termB doc=2, then the updates are applied in that order, meaning we cannot rely on updates coming in order. # Same document may be updated multiple times, either by same term (e.g. several calls to IW.updateNDV) or by different terms. Last update wins. # Sequential read: when writing the updates to the Directory (fieldsConsumer), we iterate on the docs in-order and for each one check if it's updated and if not, pull its value from the current DV. # A single update may affect several million documents, therefore need to be efficient w.r.t. memory consumption. +MapInteger,MapString,Long+ Holds a mapping from a document, to all the fields that it was updated in and the updated value for each field. This is used by IW.commitMergedDeletes to apply the updates that came in while the segment was merging. The requirements this structure needs to satisfy are: # Access in doc order: this is how commitMergedDeletes works. # One-pass: we visit a document once (currently) and so if we can, it's better if we know all the fields in which it was updated. The updates are applied to the merged ReaderAndLiveDocs (where they are stored in the first structure mentioned above). Comments with proposals will follow next. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5340) Add support for named snapshots
[ https://issues.apache.org/jira/browse/SOLR-5340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794079#comment-13794079 ] Shalin Shekhar Mangar commented on SOLR-5340: - Mike, this is good idea. If you can put up a patch, I'd be happy to review and commit it. Add support for named snapshots --- Key: SOLR-5340 URL: https://issues.apache.org/jira/browse/SOLR-5340 Project: Solr Issue Type: Improvement Components: SolrCloud Affects Versions: 4.5 Reporter: Mike Schrag It would be really nice if Solr supported named snapshots. Right now if you snapshot a SolrCloud cluster, every node potentially records a slightly different timestamp. Correlating those back together to effectively restore the entire cluster to a consistent snapshot is pretty tedious. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5331) SolrCloud 4.5 bulk add errors
[
https://issues.apache.org/jira/browse/SOLR-5331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shalin Shekhar Mangar updated SOLR-5331:
Fix Version/s: 4.5.1
This is worth investigating before we release 4.5.1
SolrCloud 4.5 bulk add errors
-
Key: SOLR-5331
URL: https://issues.apache.org/jira/browse/SOLR-5331
Project: Solr
Issue Type: Bug
Components: SolrCloud
Affects Versions: 4.5
Reporter: Chris
Fix For: 4.5.1
Since Solr 4.5 bulk adding documents via SolrJ (at least) is causing errors.
// build array list of SolrInputDocuments
server.add(docs);
I've tried with CUSS (which swallows exceptions as expected) however they are
shown in the logs on server, and with CloudSolrServer which is returning the
errors as well as seeing them in the server logs.
I've tried downgrading my SolrJ to 4.4, still errors so looks like a
regression in the server code. Reverting to Solr 4.4 on server and I don't
get errors (however run into deadlock issues).
I raised this issue in IRC - NOT the mailing list, and elyorag suggested
opening a ticket, and to mention this has now been discussed in IRC.
The exceptions would indicate I'm attempting to do multiple operations in a
single request which is malformed. I am not, I am only attempting to add
documents.
Stack traces seen here:
14:57:13 ERROR SolrCmdDistributor shard update error RetryNode:
http://X.X.X.X:8080/solr/collection1_shard16_replica2/:org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:
Illegal to have multiple roots (start tag in epilog?).
shard update error RetryNode:
http://X.X.X.X:8080/solr/collection1_shard16_replica2/:org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:
Illegal to have multiple roots (start tag in epilog?).
at [row,col {unknown-source}]: [18,327]
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:424)
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at
org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:401)
at
org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:375)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
org.apache.solr.common.SolrException: Illegal to have multiple roots
(start tag in epilog?).
at [row,col {unknown-source}]: [7,6314]
at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:176)
at
org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:703)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:406)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:312)
at
Re: Should build fail if test does not exist?
I don't think this should be an error. If you make a typo you will see no tests were executed. I don't see thr benefit of failing the build. In any case, this would belong to the runner. D. On Oct 14, 2013 2:17 PM, Shai Erera ser...@gmail.com wrote: Hi If you run something like ant test -Dtestcase=TestFoo where there's no TestFoo.java, or ant test -Dtestcase=TestBar -Dtests.method=testFoo where there's TestBar.java but no testFoo() method, the build currently passes as SUCCESSFUL. Though, the report says 0 suits or 1 suits, 0 tests. I wonder if it should be a build failure? The problem I have is that you can e.g. make a very stupid typo (usually around plurals as in testSomethingBadOnIOException vs testSomethingBadOnIOExceptions) and get a false SUCCESSFUL notification. If we want to fix/change it, where does it belong - build scripts, LuceneTestCase or randomizedrunner? Shai
[jira] [Commented] (SOLR-5331) SolrCloud 4.5 bulk add errors
[
https://issues.apache.org/jira/browse/SOLR-5331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794094#comment-13794094
]
Joel Bernstein commented on SOLR-5331:
--
Agreed. I'll take a look at this today as well and see if I can recreate the
bug.
Chris, can you post more of the Solrj code, I'd like to see the entire request?
SolrCloud 4.5 bulk add errors
-
Key: SOLR-5331
URL: https://issues.apache.org/jira/browse/SOLR-5331
Project: Solr
Issue Type: Bug
Components: SolrCloud
Affects Versions: 4.5
Reporter: Chris
Fix For: 4.5.1
Since Solr 4.5 bulk adding documents via SolrJ (at least) is causing errors.
// build array list of SolrInputDocuments
server.add(docs);
I've tried with CUSS (which swallows exceptions as expected) however they are
shown in the logs on server, and with CloudSolrServer which is returning the
errors as well as seeing them in the server logs.
I've tried downgrading my SolrJ to 4.4, still errors so looks like a
regression in the server code. Reverting to Solr 4.4 on server and I don't
get errors (however run into deadlock issues).
I raised this issue in IRC - NOT the mailing list, and elyorag suggested
opening a ticket, and to mention this has now been discussed in IRC.
The exceptions would indicate I'm attempting to do multiple operations in a
single request which is malformed. I am not, I am only attempting to add
documents.
Stack traces seen here:
14:57:13 ERROR SolrCmdDistributor shard update error RetryNode:
http://X.X.X.X:8080/solr/collection1_shard16_replica2/:org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:
Illegal to have multiple roots (start tag in epilog?).
shard update error RetryNode:
http://X.X.X.X:8080/solr/collection1_shard16_replica2/:org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:
Illegal to have multiple roots (start tag in epilog?).
at [row,col {unknown-source}]: [18,327]
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:424)
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at
org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:401)
at
org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:375)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
org.apache.solr.common.SolrException: Illegal to have multiple roots
(start tag in epilog?).
at [row,col {unknown-source}]: [7,6314]
at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:176)
at
org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:703)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:406)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023)
at
Re: Should build fail if test does not exist?
I don't always read the entire log -- if a build is SUCCESSFUL, I assume it's successful. I hit this while using luceneutil/repeatLuceneTest.py -- it happily ran a couple of hundred iterations until I suspected something's wrong because the iterations finished too quickly. Indeed, if I run 'ant test -Dtestcase=TestFoo* -Dtests.iters=1000', nothing is run and then it's easy to note that in the console because I'm missing the 1000 OK lines. Maybe it should be fixed in luceneutil, I'm not sure. To fix it there would mean that we'll need to parse the test log to search for 0 suites or 0 tests, which is both fragile and only possible if you choose to log the output to a file (if it's logged to the console there's nothing you can do I believe). So I thought if the runner would fail the build, it will help more easily detect such errors. Why rely on someone to note within the hundreds of characters the framework spits at you that no tests were actually ran? What's the harm of failing the build in that case? Shai On Mon, Oct 14, 2013 at 4:00 PM, Dawid Weiss dawid.we...@gmail.com wrote: I don't think this should be an error. If you make a typo you will see no tests were executed. I don't see thr benefit of failing the build. In any case, this would belong to the runner. D. On Oct 14, 2013 2:17 PM, Shai Erera ser...@gmail.com wrote: Hi If you run something like ant test -Dtestcase=TestFoo where there's no TestFoo.java, or ant test -Dtestcase=TestBar -Dtests.method=testFoo where there's TestBar.java but no testFoo() method, the build currently passes as SUCCESSFUL. Though, the report says 0 suits or 1 suits, 0 tests. I wonder if it should be a build failure? The problem I have is that you can e.g. make a very stupid typo (usually around plurals as in testSomethingBadOnIOException vs testSomethingBadOnIOExceptions) and get a false SUCCESSFUL notification. If we want to fix/change it, where does it belong - build scripts, LuceneTestCase or randomizedrunner? Shai
Re: Should build fail if test does not exist?
On Mon, Oct 14, 2013 at 9:11 AM, Shai Erera ser...@gmail.com wrote: What's the harm of failing the build in that case? because i should be able to do this and for it to pass: cd lucene/ ant test -Dtestcase=TestIndexWriter So please, don't make this change. it would totally screw everything up - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Should build fail if test does not exist?
I see, didn't think about that usecase. Ok so let's not do it. Shai On Mon, Oct 14, 2013 at 4:27 PM, Robert Muir rcm...@gmail.com wrote: On Mon, Oct 14, 2013 at 9:11 AM, Shai Erera ser...@gmail.com wrote: What's the harm of failing the build in that case? because i should be able to do this and for it to pass: cd lucene/ ant test -Dtestcase=TestIndexWriter So please, don't make this change. it would totally screw everything up - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4327) SolrJ code review indicates potential for leaked HttpClient connections
[
https://issues.apache.org/jira/browse/SOLR-4327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794126#comment-13794126
]
David Smiley commented on SOLR-4327:
+1 Oleg. Mark, please change to catch Exception.
SolrJ code review indicates potential for leaked HttpClient connections
---
Key: SOLR-4327
URL: https://issues.apache.org/jira/browse/SOLR-4327
Project: Solr
Issue Type: Bug
Affects Versions: 4.0
Reporter: Karl Wright
Assignee: Mark Miller
Fix For: 4.5.1, 4.6, 5.0
Attachments: SOLR-4327.patch, SOLR-4327.patch
The SolrJ HttpSolrServer implementation does not seem to handle errors
properly and seems capable of leaking HttpClient connections. See the
request() method in org.apache.solr.client.solrj.impl.HttpSolrServer. The
issue is that exceptions thrown from within this method do not necessarily
consume the stream when an exception is thrown. There is a try/finally block
which reads (in part):
{code}
} finally {
if (respBody != null processor!=null) {
try {
respBody.close();
} catch (Throwable t) {} // ignore
}
}
{code}
But, in order to always guarantee consumption of the stream, it should
include:
{code}
method.abort();
{code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5345) OpenCloseCoreStressTest opens too many files (causing it to fail often)
[ https://issues.apache.org/jira/browse/SOLR-5345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794133#comment-13794133 ] Mark Miller commented on SOLR-5345: --- Nightly runs can still be an issue. We probably need to clamp down on the merge factor, etc for that one. OpenCloseCoreStressTest opens too many files (causing it to fail often) --- Key: SOLR-5345 URL: https://issues.apache.org/jira/browse/SOLR-5345 Project: Solr Issue Type: Bug Reporter: Robert Muir Assignee: Mark Miller Fix For: 4.5.1 Typically the symptom with jenkins is strange socket failures (because sockets are files too). On my machine though i got the nice too many open files from lucene because it just happened to fail that way. In addition to holding thousands of open files, this test creates 90 *megabytes* of logging output! Can we tone down the test? I havent looked at all: but it doesn't seem necessary to have huge numbers of cores/indexes to test the functionality, we should be able to do this with relatively small numbers. Additionally i dont think its good to use random merge/IW buffering parameters here, we should set ones that won't make tons of files, force the use of compound file format, etc. The massive amounts of logging should be addressed too. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4327) SolrJ code review indicates potential for leaked HttpClient connections
[
https://issues.apache.org/jira/browse/SOLR-4327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794135#comment-13794135
]
Mark Miller commented on SOLR-4327:
---
That change has nothing to do with this JIRA issue.
SolrJ code review indicates potential for leaked HttpClient connections
---
Key: SOLR-4327
URL: https://issues.apache.org/jira/browse/SOLR-4327
Project: Solr
Issue Type: Bug
Affects Versions: 4.0
Reporter: Karl Wright
Assignee: Mark Miller
Fix For: 4.5.1, 4.6, 5.0
Attachments: SOLR-4327.patch, SOLR-4327.patch
The SolrJ HttpSolrServer implementation does not seem to handle errors
properly and seems capable of leaking HttpClient connections. See the
request() method in org.apache.solr.client.solrj.impl.HttpSolrServer. The
issue is that exceptions thrown from within this method do not necessarily
consume the stream when an exception is thrown. There is a try/finally block
which reads (in part):
{code}
} finally {
if (respBody != null processor!=null) {
try {
respBody.close();
} catch (Throwable t) {} // ignore
}
}
{code}
But, in order to always guarantee consumption of the stream, it should
include:
{code}
method.abort();
{code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5331) SolrCloud 4.5 bulk add errors
[
https://issues.apache.org/jira/browse/SOLR-5331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794153#comment-13794153
]
Chris commented on SOLR-5331:
-
Joel, I have my own parser class which returns a SolrInputDocument from a
FileInputStream.
I itterate the file system and add to the arraylist, when it reaches 100, I
add...
CloudSolrServer server = new CloudSolrServer(zkString);
server.setDefaultCollection(collection1);
File dir = new File(/path/to/files);
ListSolrInputDocument docs = new ArrayListSolrInputDocument();
for(File file : dir.listFiles()) {
docs.add(MyParser.parse(new FileInputStream(file));
if(docs.size() = 100) {
server.add(docs);
docs.clear();
}
}
SolrCloud 4.5 bulk add errors
-
Key: SOLR-5331
URL: https://issues.apache.org/jira/browse/SOLR-5331
Project: Solr
Issue Type: Bug
Components: SolrCloud
Affects Versions: 4.5
Reporter: Chris
Fix For: 4.5.1
Since Solr 4.5 bulk adding documents via SolrJ (at least) is causing errors.
// build array list of SolrInputDocuments
server.add(docs);
I've tried with CUSS (which swallows exceptions as expected) however they are
shown in the logs on server, and with CloudSolrServer which is returning the
errors as well as seeing them in the server logs.
I've tried downgrading my SolrJ to 4.4, still errors so looks like a
regression in the server code. Reverting to Solr 4.4 on server and I don't
get errors (however run into deadlock issues).
I raised this issue in IRC - NOT the mailing list, and elyorag suggested
opening a ticket, and to mention this has now been discussed in IRC.
The exceptions would indicate I'm attempting to do multiple operations in a
single request which is malformed. I am not, I am only attempting to add
documents.
Stack traces seen here:
14:57:13 ERROR SolrCmdDistributor shard update error RetryNode:
http://X.X.X.X:8080/solr/collection1_shard16_replica2/:org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:
Illegal to have multiple roots (start tag in epilog?).
shard update error RetryNode:
http://X.X.X.X:8080/solr/collection1_shard16_replica2/:org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:
Illegal to have multiple roots (start tag in epilog?).
at [row,col {unknown-source}]: [18,327]
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:424)
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at
org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:401)
at
org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:375)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
org.apache.solr.common.SolrException: Illegal to have multiple roots
(start tag in epilog?).
at [row,col {unknown-source}]: [7,6314]
at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:176)
at
org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:703)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:406)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
Re: Should build fail if test does not exist?
This has actually bit me before too ... I mean, sure, I do eventually notice that it ran too quickly and so it was not in fact really SUCCESSFUL. Why would Rob's example fail? In that case, it would have in fact run TestIndexWriter, right? (Sure, other modules didn't have such a test, but the fact that one of the visited modules did have the test should mean that the overall ant run is SUCCESSFUL?). Is it just too hard with ant to make this logic be across modules? Mike McCandless http://blog.mikemccandless.com On Mon, Oct 14, 2013 at 9:41 AM, Shai Erera ser...@gmail.com wrote: I see, didn't think about that usecase. Ok so let's not do it. Shai On Mon, Oct 14, 2013 at 4:27 PM, Robert Muir rcm...@gmail.com wrote: On Mon, Oct 14, 2013 at 9:11 AM, Shai Erera ser...@gmail.com wrote: What's the harm of failing the build in that case? because i should be able to do this and for it to pass: cd lucene/ ant test -Dtestcase=TestIndexWriter So please, don't make this change. it would totally screw everything up - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5213) collections?action=SPLITSHARD parent vs. sub-shards numDocs
[ https://issues.apache.org/jira/browse/SOLR-5213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794159#comment-13794159 ] Shalin Shekhar Mangar commented on SOLR-5213: - bq. A variation of the patch i uploaded here would be to 'rescue' (and id+hash log) any documents that would have been lost otherwise e.g. always put them in the first sub-shard, they don't belong there but at least that way they are not lost and could be analysed and dealt with later on. Hmm, that is going to be difficult because we have features such as SOLR-5338. It is completely valid to have documents that do not fall into any hash range passed into SolrIndexSplitter. collections?action=SPLITSHARD parent vs. sub-shards numDocs --- Key: SOLR-5213 URL: https://issues.apache.org/jira/browse/SOLR-5213 Project: Solr Issue Type: Improvement Components: update Affects Versions: 4.4 Reporter: Christine Poerschke Assignee: Shalin Shekhar Mangar Attachments: SOLR-5213.patch The problem we saw was that splitting a shard took a long time and at the end of it the sub-shards contained fewer documents than the original shard. The root cause was eventually tracked down to the disappearing documents not falling into the hash ranges of the sub-shards. Could SolrIndexSplitter split report per-segment numDocs for parent and sub-shards, with at least a warning logged for any discrepancies (documents falling into none of the sub-shards or documents falling into several sub-shards)? Additionally, could a case be made for erroring out when discrepancies are detected i.e. not proceeding with the shard split? Either to always error or to have an verifyNumDocs=false/true optional parameter for the SPLITSHARD action. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Should build fail if test does not exist?
On Mon, Oct 14, 2013 at 10:41 AM, Michael McCandless luc...@mikemccandless.com wrote: This has actually bit me before too ... I mean, sure, I do eventually notice that it ran too quickly and so it was not in fact really SUCCESSFUL. Why would Rob's example fail? In that case, it would have in fact run TestIndexWriter, right? (Sure, other modules didn't have such a test, but the fact that one of the visited modules did have the test should mean that the overall ant run is SUCCESSFUL?). Is it just too hard with ant to make this logic be across modules? 'ant test' needs to do a lot more than the specialized python script you have to repeat one test. so I think you should modify the latter instead of trying to make the whole build system complicated. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5331) SolrCloud 4.5 bulk add errors
[
https://issues.apache.org/jira/browse/SOLR-5331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794164#comment-13794164
]
Chris commented on SOLR-5331:
-
I read about using Java bin transport in CloudSolrServer in another ticket - if
possible I would like to test this. I am running with the latest 4.6 snapshot
from Jenkins. Please advise if this can be done.
SolrCloud 4.5 bulk add errors
-
Key: SOLR-5331
URL: https://issues.apache.org/jira/browse/SOLR-5331
Project: Solr
Issue Type: Bug
Components: SolrCloud
Affects Versions: 4.5
Reporter: Chris
Fix For: 4.5.1
Since Solr 4.5 bulk adding documents via SolrJ (at least) is causing errors.
// build array list of SolrInputDocuments
server.add(docs);
I've tried with CUSS (which swallows exceptions as expected) however they are
shown in the logs on server, and with CloudSolrServer which is returning the
errors as well as seeing them in the server logs.
I've tried downgrading my SolrJ to 4.4, still errors so looks like a
regression in the server code. Reverting to Solr 4.4 on server and I don't
get errors (however run into deadlock issues).
I raised this issue in IRC - NOT the mailing list, and elyorag suggested
opening a ticket, and to mention this has now been discussed in IRC.
The exceptions would indicate I'm attempting to do multiple operations in a
single request which is malformed. I am not, I am only attempting to add
documents.
Stack traces seen here:
14:57:13 ERROR SolrCmdDistributor shard update error RetryNode:
http://X.X.X.X:8080/solr/collection1_shard16_replica2/:org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:
Illegal to have multiple roots (start tag in epilog?).
shard update error RetryNode:
http://X.X.X.X:8080/solr/collection1_shard16_replica2/:org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:
Illegal to have multiple roots (start tag in epilog?).
at [row,col {unknown-source}]: [18,327]
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:424)
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at
org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:401)
at
org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:375)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
org.apache.solr.common.SolrException: Illegal to have multiple roots
(start tag in epilog?).
at [row,col {unknown-source}]: [7,6314]
at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:176)
at
org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:703)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:406)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023)
at
Re: Should build fail if test does not exist?
I know understand why Dawid tried to make it clear that this stuff is
wildcard matching.
!-- Aliases for tests filters --
condition property=tests.class value=*.${testcase}
isset property=testcase /
/condition
Its sorta like shell expansion on the unix prompt: 'echo *' shouldnt
return non-zero because there are no files in the current directory.
thats because its very general and has a lot of use cases. On the
other hand, it makes sense that 'ls *' returns 1 in this case, because
its sole purpose is listing files.
The same can be said for your python test-repeater
On Mon, Oct 14, 2013 at 10:41 AM, Michael McCandless
luc...@mikemccandless.com wrote:
This has actually bit me before too ...
I mean, sure, I do eventually notice that it ran too quickly and so it
was not in fact really SUCCESSFUL.
Why would Rob's example fail? In that case, it would have in fact run
TestIndexWriter, right? (Sure, other modules didn't have such a test,
but the fact that one of the visited modules did have the test should
mean that the overall ant run is SUCCESSFUL?). Is it just too hard
with ant to make this logic be across modules?
Mike McCandless
http://blog.mikemccandless.com
On Mon, Oct 14, 2013 at 9:41 AM, Shai Erera ser...@gmail.com wrote:
I see, didn't think about that usecase. Ok so let's not do it.
Shai
On Mon, Oct 14, 2013 at 4:27 PM, Robert Muir rcm...@gmail.com wrote:
On Mon, Oct 14, 2013 at 9:11 AM, Shai Erera ser...@gmail.com wrote:
What's the harm of failing the build in that case?
because i should be able to do this and for it to pass:
cd lucene/
ant test -Dtestcase=TestIndexWriter
So please, don't make this change. it would totally screw everything up
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5248) Improve the data structure used in ReaderAndLiveDocs to hold the updates
[
https://issues.apache.org/jira/browse/LUCENE-5248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794185#comment-13794185
]
Robert Muir commented on LUCENE-5248:
-
{quote}
Also modifies TestIndexWriterDelete.testNoLostDeletesOrUpdatesOnIOException to
fail on either writeLiveDocs or writeFieldUpdates.
{quote}
Maybe this could be named more succinctly: like 'testExceptions' ?
Improve the data structure used in ReaderAndLiveDocs to hold the updates
Key: LUCENE-5248
URL: https://issues.apache.org/jira/browse/LUCENE-5248
Project: Lucene - Core
Issue Type: Improvement
Components: core/index
Reporter: Shai Erera
Assignee: Shai Erera
Attachments: LUCENE-5248.patch, LUCENE-5248.patch, LUCENE-5248.patch,
LUCENE-5248.patch, LUCENE-5248.patch
Currently ReaderAndLiveDocs holds the updates in two structures:
+MapString,MapInteger,Long+
Holds a mapping from each field, to all docs that were updated and their
values. This structure is updated when applyDeletes is called, and needs to
satisfy several requirements:
# Un-ordered writes: if a field f is updated by two terms, termA and termB,
in that order, and termA affects doc=100 and termB doc=2, then the updates
are applied in that order, meaning we cannot rely on updates coming in order.
# Same document may be updated multiple times, either by same term (e.g.
several calls to IW.updateNDV) or by different terms. Last update wins.
# Sequential read: when writing the updates to the Directory
(fieldsConsumer), we iterate on the docs in-order and for each one check if
it's updated and if not, pull its value from the current DV.
# A single update may affect several million documents, therefore need to be
efficient w.r.t. memory consumption.
+MapInteger,MapString,Long+
Holds a mapping from a document, to all the fields that it was updated in and
the updated value for each field. This is used by IW.commitMergedDeletes to
apply the updates that came in while the segment was merging. The
requirements this structure needs to satisfy are:
# Access in doc order: this is how commitMergedDeletes works.
# One-pass: we visit a document once (currently) and so if we can, it's
better if we know all the fields in which it was updated. The updates are
applied to the merged ReaderAndLiveDocs (where they are stored in the first
structure mentioned above).
Comments with proposals will follow next.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5274) Teach fast FastVectorHighlighter to highlight child fields with parent fields
[ https://issues.apache.org/jira/browse/LUCENE-5274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794190#comment-13794190 ] Nik Everett commented on LUCENE-5274: - I'm having a look at what I can do to pull MergedIterator into the util package and give it nice unit tests. Almost done with that and I should be able to spin another version of this patch. I'm not exactly sure of a good way to test the synonym stuff in FastVectorHighlighterTest - I don't see a mock Synonym filter. Teach fast FastVectorHighlighter to highlight child fields with parent fields --- Key: LUCENE-5274 URL: https://issues.apache.org/jira/browse/LUCENE-5274 Project: Lucene - Core Issue Type: Improvement Components: core/other Reporter: Nik Everett Assignee: Adrien Grand Priority: Minor Attachments: LUCENE-5274.patch I've been messing around with the FastVectorHighlighter and it looks like I can teach it to highlight matches on child fields. Like this query: foo:scissors foo_exact:running would highlight foo like this: emrunning/em with emscissors/em Where foo is stored WITH_POSITIONS_OFFSETS and foo_plain is an unstored copy of foo a different analyzer and its own WITH_POSITIONS_OFFSETS. This would make queries that perform weighted matches against different analyzers much more convenient to highlight. I have working code and test cases but they are hacked into Elasticsearch. I'd love to Lucene-ify if you'll take them. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5248) Improve the data structure used in ReaderAndLiveDocs to hold the updates
[ https://issues.apache.org/jira/browse/LUCENE-5248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794195#comment-13794195 ] Shai Erera commented on LUCENE-5248: We can even move the test to TestIWExceptions, but I think it has to be named properly .. maybe testNoLostDeletesOrUpdates, or just testNoLostUpdates? Improve the data structure used in ReaderAndLiveDocs to hold the updates Key: LUCENE-5248 URL: https://issues.apache.org/jira/browse/LUCENE-5248 Project: Lucene - Core Issue Type: Improvement Components: core/index Reporter: Shai Erera Assignee: Shai Erera Attachments: LUCENE-5248.patch, LUCENE-5248.patch, LUCENE-5248.patch, LUCENE-5248.patch, LUCENE-5248.patch Currently ReaderAndLiveDocs holds the updates in two structures: +MapString,MapInteger,Long+ Holds a mapping from each field, to all docs that were updated and their values. This structure is updated when applyDeletes is called, and needs to satisfy several requirements: # Un-ordered writes: if a field f is updated by two terms, termA and termB, in that order, and termA affects doc=100 and termB doc=2, then the updates are applied in that order, meaning we cannot rely on updates coming in order. # Same document may be updated multiple times, either by same term (e.g. several calls to IW.updateNDV) or by different terms. Last update wins. # Sequential read: when writing the updates to the Directory (fieldsConsumer), we iterate on the docs in-order and for each one check if it's updated and if not, pull its value from the current DV. # A single update may affect several million documents, therefore need to be efficient w.r.t. memory consumption. +MapInteger,MapString,Long+ Holds a mapping from a document, to all the fields that it was updated in and the updated value for each field. This is used by IW.commitMergedDeletes to apply the updates that came in while the segment was merging. The requirements this structure needs to satisfy are: # Access in doc order: this is how commitMergedDeletes works. # One-pass: we visit a document once (currently) and so if we can, it's better if we know all the fields in which it was updated. The updates are applied to the merged ReaderAndLiveDocs (where they are stored in the first structure mentioned above). Comments with proposals will follow next. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5274) Teach fast FastVectorHighlighter to highlight child fields with parent fields
[
https://issues.apache.org/jira/browse/LUCENE-5274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794197#comment-13794197
]
Adrien Grand commented on LUCENE-5274:
--
bq. I'm not exactly sure of a good way to test the synonym stuff in
FastVectorHighlighterTest
I usually do that by building the token streams by hand, it is quite easy, see
{{CannedTokenStream}} and {{Token}} for example.
Teach fast FastVectorHighlighter to highlight child fields with parent
fields
---
Key: LUCENE-5274
URL: https://issues.apache.org/jira/browse/LUCENE-5274
Project: Lucene - Core
Issue Type: Improvement
Components: core/other
Reporter: Nik Everett
Assignee: Adrien Grand
Priority: Minor
Attachments: LUCENE-5274.patch
I've been messing around with the FastVectorHighlighter and it looks like I
can teach it to highlight matches on child fields. Like this query:
foo:scissors foo_exact:running
would highlight foo like this:
emrunning/em with emscissors/em
Where foo is stored WITH_POSITIONS_OFFSETS and foo_plain is an unstored copy
of foo a different analyzer and its own WITH_POSITIONS_OFFSETS.
This would make queries that perform weighted matches against different
analyzers much more convenient to highlight.
I have working code and test cases but they are hacked into Elasticsearch.
I'd love to Lucene-ify if you'll take them.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Should build fail if test does not exist?
On Mon, Oct 14, 2013 at 10:48 AM, Robert Muir rcm...@gmail.com wrote: On Mon, Oct 14, 2013 at 10:41 AM, Michael McCandless luc...@mikemccandless.com wrote: This has actually bit me before too ... I mean, sure, I do eventually notice that it ran too quickly and so it was not in fact really SUCCESSFUL. Why would Rob's example fail? In that case, it would have in fact run TestIndexWriter, right? (Sure, other modules didn't have such a test, but the fact that one of the visited modules did have the test should mean that the overall ant run is SUCCESSFUL?). Is it just too hard with ant to make this logic be across modules? 'ant test' needs to do a lot more than the specialized python script you have to repeat one test. Right, I agree this is hard to fix, because of ant / randomizedtesting / our build scripts limitations. But I still think it's wrong that ant test -Dtestcase=foo -Dtestmethod=bar finishes with BUILD SUCCESSFUL when you have an accidental typo and in fact nothing ran. It's like javac declaring success when you mis-typed one of your java source files. I know and agree this is really, really hard for us to fix, but I still think it's wrong: it's so trappy. Maybe we need a new ant run-this-test-for-certain target or something. so I think you should modify the latter instead of trying to make the whole build system complicated. Yeah, I fixed luceneutil ... it's of course hackity, since I peek in the stdout for OK (0 tests) and then call that a failure. Also, luceneutil cheats since this particular beasting tool (repeatLuceneTest.py) only runs one module (you have to cd to that directory first). The distributed beasting tool (runRemoteTests.py) runs all modules though ... Mike McCandless http://blog.mikemccandless.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-5268) Cutover more postings formats to the inverted pull API
[ https://issues.apache.org/jira/browse/LUCENE-5268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless resolved LUCENE-5268. Resolution: Fixed Cutover more postings formats to the inverted pull API Key: LUCENE-5268 URL: https://issues.apache.org/jira/browse/LUCENE-5268 Project: Lucene - Core Issue Type: Improvement Components: core/index Reporter: Michael McCandless Assignee: Michael McCandless Fix For: 5.0 Attachments: LUCENE-5268.patch, LUCENE-5268.patch In LUCENE-5123, we added a new, more flexible, pull API for writing postings. This API allows the postings format to iterate the fields/terms/postings more than once, and mirrors the API for writing doc values. But that was just the first step (only SimpleText was cutover to the new API). I want to cutover more components, so we can (finally) e.g. play with different encodings depending on the term's postings, such as using a bitset for high freq DOCS_ONLY terms (LUCENE-5052). -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5268) Cutover more postings formats to the inverted pull API
[ https://issues.apache.org/jira/browse/LUCENE-5268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794231#comment-13794231 ] ASF subversion and git services commented on LUCENE-5268: - Commit 1531949 from [~mikemccand] in branch 'dev/trunk' [ https://svn.apache.org/r1531949 ] LUCENE-5268: cutover all postings formats to FieldsConsumer Cutover more postings formats to the inverted pull API Key: LUCENE-5268 URL: https://issues.apache.org/jira/browse/LUCENE-5268 Project: Lucene - Core Issue Type: Improvement Components: core/index Reporter: Michael McCandless Assignee: Michael McCandless Fix For: 5.0 Attachments: LUCENE-5268.patch, LUCENE-5268.patch In LUCENE-5123, we added a new, more flexible, pull API for writing postings. This API allows the postings format to iterate the fields/terms/postings more than once, and mirrors the API for writing doc values. But that was just the first step (only SimpleText was cutover to the new API). I want to cutover more components, so we can (finally) e.g. play with different encodings depending on the term's postings, such as using a bitset for high freq DOCS_ONLY terms (LUCENE-5052). -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-trunk-Linux-Java7-64-test-only - Build # 63029 - Failure!
Thanks Dawid: I didn't know this was always the case. Eh... if you didn't know this too then perhaps I should have made it clearer from the beginning... So, again -- anywhere you see the seed in a stack trace or in a log dump or anywhere, really, it should contain the full path to reproduce a given failure. It is hierarchical in a way that it locks down the context from top to bottom -- master seed first (this also applies to static level code, beforeclass hooks, etc., then the test context (there are no nested contexts below a test). When you're reproducing you can provide the full seed: ant -Dtests.seed=[foo]:[bar] and this would lock the static- and test- scopes. Alternatively you can provide only the master: ant -Dtests.seed=[foo] and the [bar] should be inferred from the master in an identical way (a test seed is a derivative of the master, the test's name and iteration). Dawid On Sun, Oct 13, 2013 at 10:59 PM, Robert Muir rcm...@gmail.com wrote: On Sun, Oct 13, 2013 at 4:46 PM, Dawid Weiss dawid.we...@cs.put.poznan.pl wrote: Anyway, in short -- if you see a [foo]:[bar] then [foo] is your master seed. Thanks Dawid: I didn't know this was always the case. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5280) Rename TermFreqPayloadIterator - SuggestionIterator
[ https://issues.apache.org/jira/browse/LUCENE-5280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794260#comment-13794260 ] Michael McCandless commented on LUCENE-5280: Patch looks great, I'll commit shortly. Thanks Areek! Rename TermFreqPayloadIterator - SuggestionIterator Key: LUCENE-5280 URL: https://issues.apache.org/jira/browse/LUCENE-5280 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Fix For: 4.6, 5.0 Attachments: LUCENE-5280.patch The current name is cumbersome, and annoying to change whenever we add something to the iterator (in this case payloads). Since we are breaking it anyway in 4.6, I think we should take the opportunity to find a better name. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Should build fail if test does not exist?
Exactly. If you pass a wildcard that filters out everything then you
end up without tests. This isn't an error condition (to me).
I can add an option to the runner to fail on zero executed test cases
but then Robert's example won't work. I don't think there is any
sensible way to do it across modules -- aggregation would be very
difficult because every ant call is pretty much isolated and doesn't
know anything about the past/future. It'd be extremely hacky.
If you need to filter out a test case for a particular module, cd to
that module and run your tests there -- then your command line will
end with the number of test cases actually run. If you're running from
the top level then filtering just shouldn't fail -- it's very likely
that one module or another won't contain any tests matching your
filter. You can always resort to Mike's pythacks and spawn your tests
from there.
Dawid
On Mon, Oct 14, 2013 at 5:01 PM, Robert Muir rcm...@gmail.com wrote:
I know understand why Dawid tried to make it clear that this stuff is
wildcard matching.
!-- Aliases for tests filters --
condition property=tests.class value=*.${testcase}
isset property=testcase /
/condition
Its sorta like shell expansion on the unix prompt: 'echo *' shouldnt
return non-zero because there are no files in the current directory.
thats because its very general and has a lot of use cases. On the
other hand, it makes sense that 'ls *' returns 1 in this case, because
its sole purpose is listing files.
The same can be said for your python test-repeater
On Mon, Oct 14, 2013 at 10:41 AM, Michael McCandless
luc...@mikemccandless.com wrote:
This has actually bit me before too ...
I mean, sure, I do eventually notice that it ran too quickly and so it
was not in fact really SUCCESSFUL.
Why would Rob's example fail? In that case, it would have in fact run
TestIndexWriter, right? (Sure, other modules didn't have such a test,
but the fact that one of the visited modules did have the test should
mean that the overall ant run is SUCCESSFUL?). Is it just too hard
with ant to make this logic be across modules?
Mike McCandless
http://blog.mikemccandless.com
On Mon, Oct 14, 2013 at 9:41 AM, Shai Erera ser...@gmail.com wrote:
I see, didn't think about that usecase. Ok so let's not do it.
Shai
On Mon, Oct 14, 2013 at 4:27 PM, Robert Muir rcm...@gmail.com wrote:
On Mon, Oct 14, 2013 at 9:11 AM, Shai Erera ser...@gmail.com wrote:
What's the harm of failing the build in that case?
because i should be able to do this and for it to pass:
cd lucene/
ant test -Dtestcase=TestIndexWriter
So please, don't make this change. it would totally screw everything up
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Welcome Ryan Ernst as Lucene/Solr committer
I'm pleased to announce that Ryan Ernst has accepted to join our ranks as a committer. Ryan has been working on a number of Lucene and Solr issues and recently contributed the new expressions module[1] which allows for compiling javascript expressions into SortField instances with excellent performance since it doesn't rely on a scripting engine but directly generates Java bytecode. This is a very exciting change which will be available in Lucene 4.6. Ryan, it is tradition that you introduce yourself with a brief bio. Congratulations and welcome! [1] https://issues.apache.org/jira/browse/LUCENE-5207 -- Adrien - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4956) the korean analyzer that has a korean morphological analyzer and dictionaries
[ https://issues.apache.org/jira/browse/LUCENE-4956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794294#comment-13794294 ] Uwe Schindler commented on LUCENE-4956: --- Hi, I have seen the same code at a customer and found a big bug in FileUtils and JarResources. We should fix and replace this code completely. It's not platform independent. We should fix the following (in my opinion) horrible code parts: - FileUtils: The code with getProtectionDomain is very crazy... It also will never work if the JAR file is not a local file, but maybe some other resource. Its also using APIs that are not intended for the use-case. getProtectionDomain() is for sure not to be used to get the JAR file of the classloader. - FileUtils converts the JAR file URL (from getProtectionDomain) in a wrong way to a filesystem path: We should add URL#getPath() to the forbidden APIs, it is almost always a bug!!! The code should use toURI() and the new File(uri) - JarResources is some crazy caching for resources and in combination with FileUtils its just wrong. Its also does not scale if you create an uber-jar. The idea of this class is to not always open a stream again, so it loads all resources of the JAR file to memory. This is the wrong way to do. Please remove this! We should remove both classes completely and load resources correctly with Class#getResourceAsStream. the korean analyzer that has a korean morphological analyzer and dictionaries - Key: LUCENE-4956 URL: https://issues.apache.org/jira/browse/LUCENE-4956 Project: Lucene - Core Issue Type: New Feature Components: modules/analysis Affects Versions: 4.2 Reporter: SooMyung Lee Assignee: Christian Moen Labels: newbie Attachments: kr.analyzer.4x.tar, lucene-4956.patch, lucene4956.patch, LUCENE-4956.patch Korean language has specific characteristic. When developing search service with lucene solr in korean, there are some problems in searching and indexing. The korean analyer solved the problems with a korean morphological anlyzer. It consists of a korean morphological analyzer, dictionaries, a korean tokenizer and a korean filter. The korean anlyzer is made for lucene and solr. If you develop a search service with lucene in korean, It is the best idea to choose the korean analyzer. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4992) Solr queries don't propagate Java OutOfMemoryError back to the JVM
[
https://issues.apache.org/jira/browse/SOLR-4992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794295#comment-13794295
]
Mark Miller commented on SOLR-4992:
---
We like to catch those throwables because of things like assertion errors - we
still want to close resources, etc.
One idea is to always do:
{code}
} catch (Throwable t) {
if (t instanceof OutOfMemoryError) {
throw (OutOfMemoryError)t;
}
{code}
But that is difficult to enforce over time.
Solr queries don't propagate Java OutOfMemoryError back to the JVM
--
Key: SOLR-4992
URL: https://issues.apache.org/jira/browse/SOLR-4992
Project: Solr
Issue Type: Bug
Components: search, SolrCloud, update
Affects Versions: 4.3.1
Reporter: Daniel Collins
Assignee: Mark Miller
Fix For: 4.6, 5.0
Solr (specifically SolrDispatchFilter.doFilter() but there might be other
places) handle generic java.lang.Throwable errors but that hides
OutOfMemoryError scenarios.
IndexWriter does this too but that has a specific exclusion for OOM scenarios
and handles them explicitly (stops committing and just logs to the
transaction log).
{noformat}
Example Stack trace:
2013-06-26 19:31:33,801 [qtp632640515-62] ERROR
solr.servlet.SolrDispatchFilter Q:22 -
null:java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap
space
at
org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:670)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:380)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:155)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1423)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:450)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:138)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:564)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:213)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1083)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:379)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:175)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1017)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:136)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:258)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:109)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:445)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:260)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:225)
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.run(AbstractConnection.java:358)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:596)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:527)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.OutOfMemoryError: Java heap space
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4992) Solr queries don't propagate Java OutOfMemoryError back to the JVM
[
https://issues.apache.org/jira/browse/SOLR-4992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794297#comment-13794297
]
Ramkumar Aiyengar commented on SOLR-4992:
-
Doesn't `finally` offer a cleaner way to do that?
Solr queries don't propagate Java OutOfMemoryError back to the JVM
--
Key: SOLR-4992
URL: https://issues.apache.org/jira/browse/SOLR-4992
Project: Solr
Issue Type: Bug
Components: search, SolrCloud, update
Affects Versions: 4.3.1
Reporter: Daniel Collins
Assignee: Mark Miller
Fix For: 4.6, 5.0
Solr (specifically SolrDispatchFilter.doFilter() but there might be other
places) handle generic java.lang.Throwable errors but that hides
OutOfMemoryError scenarios.
IndexWriter does this too but that has a specific exclusion for OOM scenarios
and handles them explicitly (stops committing and just logs to the
transaction log).
{noformat}
Example Stack trace:
2013-06-26 19:31:33,801 [qtp632640515-62] ERROR
solr.servlet.SolrDispatchFilter Q:22 -
null:java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap
space
at
org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:670)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:380)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:155)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1423)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:450)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:138)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:564)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:213)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1083)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:379)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:175)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1017)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:136)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:258)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:109)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:445)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:260)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:225)
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.run(AbstractConnection.java:358)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:596)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:527)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.OutOfMemoryError: Java heap space
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Welcome Ryan Ernst as Lucene/Solr committer
Welcome Ryan! Steve On Oct 14, 2013, at 1:27 PM, Adrien Grand jpou...@gmail.com wrote: I'm pleased to announce that Ryan Ernst has accepted to join our ranks as a committer. Ryan has been working on a number of Lucene and Solr issues and recently contributed the new expressions module[1] which allows for compiling javascript expressions into SortField instances with excellent performance since it doesn't rely on a scripting engine but directly generates Java bytecode. This is a very exciting change which will be available in Lucene 4.6. Ryan, it is tradition that you introduce yourself with a brief bio. Congratulations and welcome! [1] https://issues.apache.org/jira/browse/LUCENE-5207 -- Adrien - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Welcome Ryan Ernst as Lucene/Solr committer
Welcome Ryan! On Mon, Oct 14, 2013 at 7:27 PM, Adrien Grand jpou...@gmail.com wrote: I'm pleased to announce that Ryan Ernst has accepted to join our ranks as a committer. Ryan has been working on a number of Lucene and Solr issues and recently contributed the new expressions module[1] which allows for compiling javascript expressions into SortField instances with excellent performance since it doesn't rely on a scripting engine but directly generates Java bytecode. This is a very exciting change which will be available in Lucene 4.6. Ryan, it is tradition that you introduce yourself with a brief bio. Congratulations and welcome! [1] https://issues.apache.org/jira/browse/LUCENE-5207 -- Adrien - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5280) Rename TermFreqPayloadIterator - SuggestionIterator
[ https://issues.apache.org/jira/browse/LUCENE-5280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794301#comment-13794301 ] ASF subversion and git services commented on LUCENE-5280: - Commit 1531987 from [~mikemccand] in branch 'dev/trunk' [ https://svn.apache.org/r1531987 ] LUCENE-5280: rename TermFreqPayloadIterator - InputIterator Rename TermFreqPayloadIterator - SuggestionIterator Key: LUCENE-5280 URL: https://issues.apache.org/jira/browse/LUCENE-5280 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Fix For: 4.6, 5.0 Attachments: LUCENE-5280.patch The current name is cumbersome, and annoying to change whenever we add something to the iterator (in this case payloads). Since we are breaking it anyway in 4.6, I think we should take the opportunity to find a better name. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Welcome Ryan Ernst as Lucene/Solr committer
Welcome Ryan! Alan Woodward www.flax.co.uk On 14 Oct 2013, at 18:27, Adrien Grand wrote: I'm pleased to announce that Ryan Ernst has accepted to join our ranks as a committer. Ryan has been working on a number of Lucene and Solr issues and recently contributed the new expressions module[1] which allows for compiling javascript expressions into SortField instances with excellent performance since it doesn't rely on a scripting engine but directly generates Java bytecode. This is a very exciting change which will be available in Lucene 4.6. Ryan, it is tradition that you introduce yourself with a brief bio. Congratulations and welcome! [1] https://issues.apache.org/jira/browse/LUCENE-5207 -- Adrien - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: Welcome Ryan Ernst as Lucene/Solr committer
Welcome Ryan! Glad to have the one taking care of SolrResourceCorrumpter on board! Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Adrien Grand [mailto:jpou...@gmail.com] Sent: Monday, October 14, 2013 7:28 PM To: dev@lucene.apache.org Subject: Welcome Ryan Ernst as Lucene/Solr committer I'm pleased to announce that Ryan Ernst has accepted to join our ranks as a committer. Ryan has been working on a number of Lucene and Solr issues and recently contributed the new expressions module[1] which allows for compiling javascript expressions into SortField instances with excellent performance since it doesn't rely on a scripting engine but directly generates Java bytecode. This is a very exciting change which will be available in Lucene 4.6. Ryan, it is tradition that you introduce yourself with a brief bio. Congratulations and welcome! [1] https://issues.apache.org/jira/browse/LUCENE-5207 -- Adrien - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Welcome Ryan Ernst as Lucene/Solr committer
Welcome Ryan! On Mon, Oct 14, 2013 at 10:57 PM, Adrien Grand jpou...@gmail.com wrote: I'm pleased to announce that Ryan Ernst has accepted to join our ranks as a committer. Ryan has been working on a number of Lucene and Solr issues and recently contributed the new expressions module[1] which allows for compiling javascript expressions into SortField instances with excellent performance since it doesn't rely on a scripting engine but directly generates Java bytecode. This is a very exciting change which will be available in Lucene 4.6. Ryan, it is tradition that you introduce yourself with a brief bio. Congratulations and welcome! [1] https://issues.apache.org/jira/browse/LUCENE-5207 -- Adrien - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org -- Regards, Shalin Shekhar Mangar.
[JENKINS] Lucene-trunk-Linux-Java7-64-test-only - Build # 63659 - Failure!
Build: builds.flonkings.com/job/Lucene-trunk-Linux-Java7-64-test-only/63659/ 1 tests failed. REGRESSION: org.apache.lucene.index.TestTransactions.testTransactions Error Message: Stack Trace: java.lang.AssertionError at __randomizedtesting.SeedInfo.seed([D378017B136F4A0:CEA53CDFE5A21DA4]:0) at org.junit.Assert.fail(Assert.java:92) at org.junit.Assert.assertTrue(Assert.java:43) at org.junit.Assert.assertTrue(Assert.java:54) at org.apache.lucene.index.TestTransactions.testTransactions(TestTransactions.java:259) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358) at java.lang.Thread.run(Thread.java:722) Build Log: [...truncated 945 lines...] [junit4] Suite: org.apache.lucene.index.TestTransactions [junit4] 1 Thread[Thread-260,5,TGRP-TestTransactions]: exc [junit4] 1 java.io.IOException: MockDirectoryWrapper: file _5_TestBloomFilteredLucene41Postings_0.doc is still open: cannot overwrite [junit4] 1at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:452) [junit4] 1at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:44) [junit4] 1
Re: Welcome Ryan Ernst as Lucene/Solr committer
Welcome Ryan! Mike McCandless http://blog.mikemccandless.com On Mon, Oct 14, 2013 at 1:27 PM, Adrien Grand jpou...@gmail.com wrote: I'm pleased to announce that Ryan Ernst has accepted to join our ranks as a committer. Ryan has been working on a number of Lucene and Solr issues and recently contributed the new expressions module[1] which allows for compiling javascript expressions into SortField instances with excellent performance since it doesn't rely on a scripting engine but directly generates Java bytecode. This is a very exciting change which will be available in Lucene 4.6. Ryan, it is tradition that you introduce yourself with a brief bio. Congratulations and welcome! [1] https://issues.apache.org/jira/browse/LUCENE-5207 -- Adrien - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-4956) the korean analyzer that has a korean morphological analyzer and dictionaries
[ https://issues.apache.org/jira/browse/LUCENE-4956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794294#comment-13794294 ] Uwe Schindler edited comment on LUCENE-4956 at 10/14/13 5:46 PM: - Hi, I have seen the same code at a customer and found a big bug in FileUtils and JarResources. We should fix and replace this code completely. It's not platform independent. We should fix the following (in my opinion) horrible code parts: - FileUtils: The code with getProtectionDomain is very crazy... It also will never work if the JAR file is not a local file, but maybe some other resource. Its also using APIs that are not intended for the use-case. getProtectionDomain() is for sure not to be used to get the JAR file of the classloader. - FileUtils converts the JAR file URL (from getProtectionDomain) in a wrong way to a filesystem path: We should add URL#getPath() to the forbidden APIs, it is almost always a bug!!! The code should use toURI() and then new File(uri). The other methods in FileUtil are also having similar bugs or try to prevent them. The whole class *must* be removed, sorry! - JarResources is some crazy caching for resources and in combination with FileUtils its just wrong. Its also does not scale if you create an uber-jar. The idea of this class is to not always open a stream again, so it loads all resources of the JAR file to memory. This is the wrong way to do. Please remove this! We should remove both classes completely and load resources correctly with Class#getResourceAsStream. was (Author: thetaphi): Hi, I have seen the same code at a customer and found a big bug in FileUtils and JarResources. We should fix and replace this code completely. It's not platform independent. We should fix the following (in my opinion) horrible code parts: - FileUtils: The code with getProtectionDomain is very crazy... It also will never work if the JAR file is not a local file, but maybe some other resource. Its also using APIs that are not intended for the use-case. getProtectionDomain() is for sure not to be used to get the JAR file of the classloader. - FileUtils converts the JAR file URL (from getProtectionDomain) in a wrong way to a filesystem path: We should add URL#getPath() to the forbidden APIs, it is almost always a bug!!! The code should use toURI() and the new File(uri) - JarResources is some crazy caching for resources and in combination with FileUtils its just wrong. Its also does not scale if you create an uber-jar. The idea of this class is to not always open a stream again, so it loads all resources of the JAR file to memory. This is the wrong way to do. Please remove this! We should remove both classes completely and load resources correctly with Class#getResourceAsStream. the korean analyzer that has a korean morphological analyzer and dictionaries - Key: LUCENE-4956 URL: https://issues.apache.org/jira/browse/LUCENE-4956 Project: Lucene - Core Issue Type: New Feature Components: modules/analysis Affects Versions: 4.2 Reporter: SooMyung Lee Assignee: Christian Moen Labels: newbie Attachments: kr.analyzer.4x.tar, lucene-4956.patch, lucene4956.patch, LUCENE-4956.patch Korean language has specific characteristic. When developing search service with lucene solr in korean, there are some problems in searching and indexing. The korean analyer solved the problems with a korean morphological anlyzer. It consists of a korean morphological analyzer, dictionaries, a korean tokenizer and a korean filter. The korean anlyzer is made for lucene and solr. If you develop a search service with lucene in korean, It is the best idea to choose the korean analyzer. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5280) Rename TermFreqPayloadIterator - SuggestionIterator
[ https://issues.apache.org/jira/browse/LUCENE-5280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794311#comment-13794311 ] ASF subversion and git services commented on LUCENE-5280: - Commit 1532001 from [~mikemccand] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1532001 ] LUCENE-5280: rename TermFreqPayloadIterator - InputIterator Rename TermFreqPayloadIterator - SuggestionIterator Key: LUCENE-5280 URL: https://issues.apache.org/jira/browse/LUCENE-5280 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Fix For: 4.6, 5.0 Attachments: LUCENE-5280.patch The current name is cumbersome, and annoying to change whenever we add something to the iterator (in this case payloads). Since we are breaking it anyway in 4.6, I think we should take the opportunity to find a better name. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-5280) Rename TermFreqPayloadIterator - SuggestionIterator
[ https://issues.apache.org/jira/browse/LUCENE-5280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless resolved LUCENE-5280. Resolution: Fixed Thanks Areek! Rename TermFreqPayloadIterator - SuggestionIterator Key: LUCENE-5280 URL: https://issues.apache.org/jira/browse/LUCENE-5280 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Fix For: 4.6, 5.0 Attachments: LUCENE-5280.patch The current name is cumbersome, and annoying to change whenever we add something to the iterator (in this case payloads). Since we are breaking it anyway in 4.6, I think we should take the opportunity to find a better name. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-trunk-Linux-Java7-64-test-only - Build # 63659 - Failure!
I'll dig. Mike McCandless http://blog.mikemccandless.com On Mon, Oct 14, 2013 at 1:41 PM, buil...@flonkings.com wrote: Build: builds.flonkings.com/job/Lucene-trunk-Linux-Java7-64-test-only/63659/ 1 tests failed. REGRESSION: org.apache.lucene.index.TestTransactions.testTransactions Error Message: Stack Trace: java.lang.AssertionError at __randomizedtesting.SeedInfo.seed([D378017B136F4A0:CEA53CDFE5A21DA4]:0) at org.junit.Assert.fail(Assert.java:92) at org.junit.Assert.assertTrue(Assert.java:43) at org.junit.Assert.assertTrue(Assert.java:54) at org.apache.lucene.index.TestTransactions.testTransactions(TestTransactions.java:259) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358) at java.lang.Thread.run(Thread.java:722) Build Log: [...truncated 945 lines...] [junit4] Suite: org.apache.lucene.index.TestTransactions [junit4] 1 Thread[Thread-260,5,TGRP-TestTransactions]: exc [junit4] 1 java.io.IOException: MockDirectoryWrapper: file _5_TestBloomFilteredLucene41Postings_0.doc is still open: cannot overwrite [junit4] 1at
Re: Embedded zookeeper (zkRun) - SolrPort+1000?
When I first read this I thought, I am 99.9% sure that the ref guide and the wiki say Solr port +1000, but I checked again and both say that do in the place I was thinking of...but another place (the parameter reference) says Solr port +1001. So, +1 to fixing both sources to be consistent. I'll do it if you don't get to it. On Sat, Oct 12, 2013 at 8:03 PM, Shawn Heisey s...@elyograg.org wrote: The wiki and the ref guide both say that Solr's embedded zookeeper will default to SolrPort+1001 ... but that would result in a typical port of 9984. All the examples show 9983. I also checked the source code (SolrZkServer), and it does appear to actually use +1000. I'm pretty sure that changing the source code to +1001 would be a bad idea, that we actually want to update the documentation. I'm ready to update the wiki and the ref guide, but before I do so, I would just like to throw this out for sanity checking. Is that the correct path? Thanks, Shawn - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5274) Teach fast FastVectorHighlighter to highlight child fields with parent fields
[ https://issues.apache.org/jira/browse/LUCENE-5274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nik Everett updated LUCENE-5274: Attachment: (was: LUCENE-5274.patch) Teach fast FastVectorHighlighter to highlight child fields with parent fields --- Key: LUCENE-5274 URL: https://issues.apache.org/jira/browse/LUCENE-5274 Project: Lucene - Core Issue Type: Improvement Components: core/other Reporter: Nik Everett Assignee: Adrien Grand Priority: Minor Attachments: LUCENE-5274.patch I've been messing around with the FastVectorHighlighter and it looks like I can teach it to highlight matches on child fields. Like this query: foo:scissors foo_exact:running would highlight foo like this: emrunning/em with emscissors/em Where foo is stored WITH_POSITIONS_OFFSETS and foo_plain is an unstored copy of foo a different analyzer and its own WITH_POSITIONS_OFFSETS. This would make queries that perform weighted matches against different analyzers much more convenient to highlight. I have working code and test cases but they are hacked into Elasticsearch. I'd love to Lucene-ify if you'll take them. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5274) Teach fast FastVectorHighlighter to highlight child fields with parent fields
[ https://issues.apache.org/jira/browse/LUCENE-5274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nik Everett updated LUCENE-5274: Attachment: LUCENE-5274.patch Not done yet but progress: 1. Move MergedIterator to util. 2. Add a mode to it to not remove duplicates (one extra branch per call to next). 3. Add a unit test for MergedIterator. 4. Make FieldTermStack.TermInfo, FieldPhraseList.WeighterPhraseInfo, FieldPhraseList.WeightedPhraseInfo.Toffs consistent under equals, hashCode, and compareTo. I don't think any of them would make good hash keys but I fixed up hashCode because I fixed up equals. 5. Unit tests for point 4. 7. Use the non-duplicate removing mode of MergedIterator in FieldPhraseList's merge methods. 6. More tests in FastVectorHighlighterTest - mostly around exact equal matches and how they effect segment sorting. At this point this is left: 1. Unit tests for equal matches in the same FieldPhraseList. 2. Poke around with corner cases during merges. Test them in FastVectorHighlighterTest if they reflect mockable real world cases. Expand FieldPhraseListTest if they don't. 4. Remove highlighter dependency on analyzer module. Would it make sense to move PerFieldAnalyzerWrapper into core? Something else? 3. Anything else from review. Teach fast FastVectorHighlighter to highlight child fields with parent fields --- Key: LUCENE-5274 URL: https://issues.apache.org/jira/browse/LUCENE-5274 Project: Lucene - Core Issue Type: Improvement Components: core/other Reporter: Nik Everett Assignee: Adrien Grand Priority: Minor Attachments: LUCENE-5274.patch I've been messing around with the FastVectorHighlighter and it looks like I can teach it to highlight matches on child fields. Like this query: foo:scissors foo_exact:running would highlight foo like this: emrunning/em with emscissors/em Where foo is stored WITH_POSITIONS_OFFSETS and foo_plain is an unstored copy of foo a different analyzer and its own WITH_POSITIONS_OFFSETS. This would make queries that perform weighted matches against different analyzers much more convenient to highlight. I have working code and test cases but they are hacked into Elasticsearch. I'd love to Lucene-ify if you'll take them. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Welcome Ryan Ernst as Lucene/Solr committer
Welcome Ryan! On Mon, Oct 14, 2013 at 10:27 AM, Adrien Grand jpou...@gmail.com wrote: I'm pleased to announce that Ryan Ernst has accepted to join our ranks as a committer. Ryan has been working on a number of Lucene and Solr issues and recently contributed the new expressions module[1] which allows for compiling javascript expressions into SortField instances with excellent performance since it doesn't rely on a scripting engine but directly generates Java bytecode. This is a very exciting change which will be available in Lucene 4.6. Ryan, it is tradition that you introduce yourself with a brief bio. Congratulations and welcome! [1] https://issues.apache.org/jira/browse/LUCENE-5207 -- Adrien - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Welcome Ryan Ernst as Lucene/Solr committer
Congrats Ryan! -Yonik On Mon, Oct 14, 2013 at 1:27 PM, Adrien Grand jpou...@gmail.com wrote: I'm pleased to announce that Ryan Ernst has accepted to join our ranks as a committer. Ryan has been working on a number of Lucene and Solr issues and recently contributed the new expressions module[1] which allows for compiling javascript expressions into SortField instances with excellent performance since it doesn't rely on a scripting engine but directly generates Java bytecode. This is a very exciting change which will be available in Lucene 4.6. Ryan, it is tradition that you introduce yourself with a brief bio. Congratulations and welcome! [1] https://issues.apache.org/jira/browse/LUCENE-5207 -- Adrien - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Should build fail if test does not exist?
On Mon, Oct 14, 2013 at 12:46 PM, Dawid Weiss dawid.we...@cs.put.poznan.pl wrote: This isn't an error condition (to me). Wait, I think it is an error :) Yes, a hard to fix error (our test infra is complex), but still an error. Ie, if I ask ant test to run a specific test, and it finds no such test, yet declares BUILD SUCCESSFUL in the end, how can such lenient error checking be good? It's trappy. It's like javac foo.java returning successfully when foo.java doesn't exist. I'm not using wildcards when I ask it to run a test :) Sure, maybe wildcards are used under the hood, but this is an implementation detail. Maybe a simple compromise: could we make a new ant target, run-specific-test-for-certain, whose purpose was to run that test and fail if no such test was not found? I think it's fine if this only works at the module level, if ant makes it too hairy to recurse. Mike McCandless http://blog.mikemccandless.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Welcome Ryan Ernst as Lucene/Solr committer
Thanks Adrian. I grew up in Bakersfield, CA (colloquially known as the armpit of California). I escaped and went to Cal Poly for my bachelors in computer science, and after a very brief stint working on HPUX, I landed working on the Amazon search engine for A9. I especially enjoy working with compression and encodings, and hope to experiment there some more with Lucene. Thanks Ryan On Mon, Oct 14, 2013 at 10:27 AM, Adrien Grand jpou...@gmail.com wrote: I'm pleased to announce that Ryan Ernst has accepted to join our ranks as a committer. Ryan has been working on a number of Lucene and Solr issues and recently contributed the new expressions module[1] which allows for compiling javascript expressions into SortField instances with excellent performance since it doesn't rely on a scripting engine but directly generates Java bytecode. This is a very exciting change which will be available in Lucene 4.6. Ryan, it is tradition that you introduce yourself with a brief bio. Congratulations and welcome! [1] https://issues.apache.org/jira/browse/LUCENE-5207 -- Adrien - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5331) SolrCloud 4.5 bulk add errors
[
https://issues.apache.org/jira/browse/SOLR-5331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794332#comment-13794332
]
Chris commented on SOLR-5331:
-
Some updates.
I tried reducing my arraylist size check before calling server.add(docs) to 10
instead of 100, now I have no issues with my importing. Maybe there is a buffer
size, or XML length limit which I was exceeding with my (large?) posts and they
were being truncated, and causing the malformed error message?
Switching to java bin transport also solved my issue (thanks shalin) and is
faster.
SolrCloud 4.5 bulk add errors
-
Key: SOLR-5331
URL: https://issues.apache.org/jira/browse/SOLR-5331
Project: Solr
Issue Type: Bug
Components: SolrCloud
Affects Versions: 4.5
Reporter: Chris
Fix For: 4.5.1
Since Solr 4.5 bulk adding documents via SolrJ (at least) is causing errors.
// build array list of SolrInputDocuments
server.add(docs);
I've tried with CUSS (which swallows exceptions as expected) however they are
shown in the logs on server, and with CloudSolrServer which is returning the
errors as well as seeing them in the server logs.
I've tried downgrading my SolrJ to 4.4, still errors so looks like a
regression in the server code. Reverting to Solr 4.4 on server and I don't
get errors (however run into deadlock issues).
I raised this issue in IRC - NOT the mailing list, and elyorag suggested
opening a ticket, and to mention this has now been discussed in IRC.
The exceptions would indicate I'm attempting to do multiple operations in a
single request which is malformed. I am not, I am only attempting to add
documents.
Stack traces seen here:
14:57:13 ERROR SolrCmdDistributor shard update error RetryNode:
http://X.X.X.X:8080/solr/collection1_shard16_replica2/:org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:
Illegal to have multiple roots (start tag in epilog?).
shard update error RetryNode:
http://X.X.X.X:8080/solr/collection1_shard16_replica2/:org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:
Illegal to have multiple roots (start tag in epilog?).
at [row,col {unknown-source}]: [18,327]
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:424)
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at
org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:401)
at
org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:375)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
org.apache.solr.common.SolrException: Illegal to have multiple roots
(start tag in epilog?).
at [row,col {unknown-source}]: [7,6314]
at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:176)
at
org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:703)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:406)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at
Re: Should build fail if test does not exist?
On Mon, Oct 14, 2013 at 11:11 AM, Michael McCandless luc...@mikemccandless.com wrote: Maybe a simple compromise: could we make a new ant target, run-specific-test-for-certain, whose purpose was to run that test and fail if no such test was not found? I think it's fine if this only works at the module level, if ant makes it too hairy to recurse. But is that really a compromise or will it only lead to increased complexity? I can see how i would implement this now, e.g. i'd just create a fileset of build/*pattern.class and fail if it was non-empty, and then invoke 'test' target. But I'm worried the simple compromise would lead us down a slippery slope, once we opened the can of worms, someone would eventually want it to validate that you didn't typo the -Dtestmethod too right? And someone might be upset that my simple solution fails always if they run clean first (since class file doesnt exist), or that it doesnt fail if the test is @Ignored, or @Nightly and they forgot to supply -Dtests.nightly, or @BadApples or @Slow, or throws an assumption always because it suppresses Lucene3xCodec and you supply -Dtests.codec=Lucene3x, or ... (it goes on and on and on). And one could argue tehse are all traps and its trappy if we dont fix it. :) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Should build fail if test does not exist?
Wait, I think it is an error :) Yes, a hard to fix error (our test infra is complex), but still an error. It's a mistake in the filter declaration, not an error in its application. It's like javac foo.java returning successfully when foo.java doesn't exist. I don't think this is particularly matching since filters are pattern matching. It's more like: grep nonexistent input returning an error just because a pattern didn't occur in input. Like I said -- I'm really indifferent to this, I can add it. I just don't think we should cater for all the possible typos and errors one can make - this would be insane. D. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5283) Fail the build if ant test didn't execute any tests (everything filtered out).
[ https://issues.apache.org/jira/browse/LUCENE-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dawid Weiss updated LUCENE-5283: Issue Type: Wish (was: Bug) Fail the build if ant test didn't execute any tests (everything filtered out). -- Key: LUCENE-5283 URL: https://issues.apache.org/jira/browse/LUCENE-5283 Project: Lucene - Core Issue Type: Wish Reporter: Dawid Weiss Assignee: Dawid Weiss Priority: Trivial This should be an optional setting that defaults to 'false' (the build proceeds). -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-5348) Fail the build if ant test didn't execute any tests (everything filtered out).
Dawid Weiss created SOLR-5348: - Summary: Fail the build if ant test didn't execute any tests (everything filtered out). Key: SOLR-5348 URL: https://issues.apache.org/jira/browse/SOLR-5348 Project: Solr Issue Type: Bug Reporter: Dawid Weiss Assignee: Dawid Weiss Priority: Trivial This should be an optional setting that defaults to 'false' (the build proceeds). -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Moved] (LUCENE-5283) Fail the build if ant test didn't execute any tests (everything filtered out).
[ https://issues.apache.org/jira/browse/LUCENE-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dawid Weiss moved SOLR-5348 to LUCENE-5283: --- Key: LUCENE-5283 (was: SOLR-5348) Project: Lucene - Core (was: Solr) Fail the build if ant test didn't execute any tests (everything filtered out). -- Key: LUCENE-5283 URL: https://issues.apache.org/jira/browse/LUCENE-5283 Project: Lucene - Core Issue Type: Bug Reporter: Dawid Weiss Assignee: Dawid Weiss Priority: Trivial This should be an optional setting that defaults to 'false' (the build proceeds). -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Should build fail if test does not exist?
Anyway, I filed LUCENE-5283 as a wish. :) I'll add it in the spare cycle -- it'll work at the module level and you can override your local settings if you wish zero-test executions to fail your build. Dawid On Mon, Oct 14, 2013 at 8:22 PM, Dawid Weiss dawid.we...@cs.put.poznan.pl wrote: Wait, I think it is an error :) Yes, a hard to fix error (our test infra is complex), but still an error. It's a mistake in the filter declaration, not an error in its application. It's like javac foo.java returning successfully when foo.java doesn't exist. I don't think this is particularly matching since filters are pattern matching. It's more like: grep nonexistent input returning an error just because a pattern didn't occur in input. Like I said -- I'm really indifferent to this, I can add it. I just don't think we should cater for all the possible typos and errors one can make - this would be insane. D. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Should build fail if test does not exist?
On Mon, Oct 14, 2013 at 2:20 PM, Robert Muir rcm...@gmail.com wrote: On Mon, Oct 14, 2013 at 11:11 AM, Michael McCandless luc...@mikemccandless.com wrote: Maybe a simple compromise: could we make a new ant target, run-specific-test-for-certain, whose purpose was to run that test and fail if no such test was not found? I think it's fine if this only works at the module level, if ant makes it too hairy to recurse. But is that really a compromise or will it only lead to increased complexity? I can see how i would implement this now, e.g. i'd just create a fileset of build/*pattern.class and fail if it was non-empty, and then invoke 'test' target. But I'm worried the simple compromise would lead us down a slippery slope, once we opened the can of worms, someone would eventually want it to validate that you didn't typo the -Dtestmethod too right? I'm less concerned about that -- if you typo that, then all tests run, right? That becomes quickly obvious user error :) I think the other direction (BUILD SUCCESSFUL when the test did not in fact run) is much worse: it's non-obvious user error. You go away gleeful that your test passed. And someone might be upset that my simple solution fails always if they run clean first (since class file doesnt exist), or that it doesnt fail if the test is @Ignored, or @Nightly and they forgot to supply -Dtests.nightly, or @BadApples or @Slow, or throws an assumption always because it suppresses Lucene3xCodec and you supply -Dtests.codec=Lucene3x, or ... (it goes on and on and on). And one could argue tehse are all traps and its trappy if we dont fix it. :) Actually I think these would be good failures, i.e. if it was a nightly test and I did not specify -Dtests.nightly then it should fail, so I know something went wrong when I tried to run the one test. I think the restriction of you must be in the module's directory is acceptable. In fact, this would close another test trap I've hit, where I run a single test (spelled correctly!), yet the test hit an Assume, appears to pass (BUILD SUCCESSFUL) but actually did not run anything, I commit, test then fails in jenkins for a stupid reason and I'm like WTF? I swear I tested it. Net/net I think your simple solution would be great? Really I just want a run this exact test name, for sure! test target. Mike McCandless http://blog.mikemccandless.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Should build fail if test does not exist?
On Mon, Oct 14, 2013 at 2:36 PM, Michael McCandless luc...@mikemccandless.com wrote: In fact, this would close another test trap I've hit, where I run a single test (spelled correctly!), yet the test hit an Assume, appears to pass (BUILD SUCCESSFUL) but actually did not run anything, I commit, test then fails in jenkins for a stupid reason and I'm like WTF? I swear I tested it. This is the complexity i dont like though. Now assume() sometimes fails the build? - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Should build fail if test does not exist?
We differ in opinions on this one, but I don't see any point in arguing -- I'll implement it, whoever wants to use it, their free will. Net/net I think your simple solution would be great? Really I just want a run this exact test name, for sure! test target. Mind you it's not exactly like that. The condition here will be at least one test was executed; it's still a pattern so if you make a typo that runs something then it will pass as successful. Dawid - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Should build fail if test does not exist?
On Mon, Oct 14, 2013 at 2:22 PM, Dawid Weiss dawid.we...@cs.put.poznan.pl wrote: Wait, I think it is an error :) Yes, a hard to fix error (our test infra is complex), but still an error. It's a mistake in the filter declaration, not an error in its application. The difference is really an implementation detail :) I just want to run one test; that this is a 2-step process (append wildcard apply filter, execute tests that matched that filter) ... is not important to the user. It's like javac foo.java returning successfully when foo.java doesn't exist. I don't think this is particularly matching since filters are pattern matching. It's more like: grep nonexistent input returning an error just because a pattern didn't occur in input. Well, javac foo*.java does result in an error, if no files match foo*.java. Ie javac is angry that it had nothing to compile. Like I said -- I'm really indifferent to this, I can add it. I just don't think we should cater for all the possible typos and errors one can make - this would be insane. OK, thanks for opening the wish issue. I just wish more devs were able to help out on our test infra ... it shouldn't have to be you always responding to our crazy feature requests! Mike McCandless http://blog.mikemccandless.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-trunk-Linux-Java7-64-test-only - Build # 63672 - Failure!
Build: builds.flonkings.com/job/Lucene-trunk-Linux-Java7-64-test-only/63672/ 1 tests failed. REGRESSION: org.apache.lucene.index.TestIndexWriterOutOfFileDescriptors.test Error Message: file _e_TestBloomFilteredLucene41Postings_0.tim already exists; siFiles=[_e.si] Stack Trace: java.lang.AssertionError: file _e_TestBloomFilteredLucene41Postings_0.tim already exists; siFiles=[_e.si] at __randomizedtesting.SeedInfo.seed([5DFACE3F0360D45:8D8B93395ECA60BD]:0) at org.apache.lucene.index.IndexWriter.copySegmentAsIs(IndexWriter.java:2672) at org.apache.lucene.index.IndexWriter.addIndexes(IndexWriter.java:2433) at org.apache.lucene.index.TestIndexWriterOutOfFileDescriptors.test(TestIndexWriterOutOfFileDescriptors.java:76) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358) at java.lang.Thread.run(Thread.java:722) Build Log: [...truncated 1315 lines...] [junit4] Suite: org.apache.lucene.index.TestIndexWriterOutOfFileDescriptors [junit4] 2 NOTE: reproduce with: ant test -Dtestcase=TestIndexWriterOutOfFileDescriptors -Dtests.method=test -Dtests.seed=5DFACE3F0360D45 -Dtests.slow=true -Dtests.locale=es_CL -Dtests.timezone=Asia/Beirut -Dtests.file.encoding=UTF-8
[jira] [Comment Edited] (SOLR-5331) SolrCloud 4.5 bulk add errors
[
https://issues.apache.org/jira/browse/SOLR-5331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794365#comment-13794365
]
Joel Bernstein edited comment on SOLR-5331 at 10/14/13 6:52 PM:
Thanks for the update. I was working on reproducing the bug, but wasn't seeing
the error. I suspect you're right about the buffer size.
was (Author: joel.bernstein):
Thanks for the update. I was working on reproducing the bug, wasn't seeing the
error. I suspect you're right about the buffer size.
SolrCloud 4.5 bulk add errors
-
Key: SOLR-5331
URL: https://issues.apache.org/jira/browse/SOLR-5331
Project: Solr
Issue Type: Bug
Components: SolrCloud
Affects Versions: 4.5
Reporter: Chris
Fix For: 4.5.1
Since Solr 4.5 bulk adding documents via SolrJ (at least) is causing errors.
// build array list of SolrInputDocuments
server.add(docs);
I've tried with CUSS (which swallows exceptions as expected) however they are
shown in the logs on server, and with CloudSolrServer which is returning the
errors as well as seeing them in the server logs.
I've tried downgrading my SolrJ to 4.4, still errors so looks like a
regression in the server code. Reverting to Solr 4.4 on server and I don't
get errors (however run into deadlock issues).
I raised this issue in IRC - NOT the mailing list, and elyorag suggested
opening a ticket, and to mention this has now been discussed in IRC.
The exceptions would indicate I'm attempting to do multiple operations in a
single request which is malformed. I am not, I am only attempting to add
documents.
Stack traces seen here:
14:57:13 ERROR SolrCmdDistributor shard update error RetryNode:
http://X.X.X.X:8080/solr/collection1_shard16_replica2/:org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:
Illegal to have multiple roots (start tag in epilog?).
shard update error RetryNode:
http://X.X.X.X:8080/solr/collection1_shard16_replica2/:org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:
Illegal to have multiple roots (start tag in epilog?).
at [row,col {unknown-source}]: [18,327]
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:424)
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at
org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:401)
at
org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:375)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
org.apache.solr.common.SolrException: Illegal to have multiple roots
(start tag in epilog?).
at [row,col {unknown-source}]: [7,6314]
at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:176)
at
org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:703)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:406)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at
[jira] [Commented] (SOLR-5331) SolrCloud 4.5 bulk add errors
[
https://issues.apache.org/jira/browse/SOLR-5331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794365#comment-13794365
]
Joel Bernstein commented on SOLR-5331:
--
Thanks for the update. I was working on reproducing the bug, wasn't seeing the
error. I suspect you're right about the buffer size.
SolrCloud 4.5 bulk add errors
-
Key: SOLR-5331
URL: https://issues.apache.org/jira/browse/SOLR-5331
Project: Solr
Issue Type: Bug
Components: SolrCloud
Affects Versions: 4.5
Reporter: Chris
Fix For: 4.5.1
Since Solr 4.5 bulk adding documents via SolrJ (at least) is causing errors.
// build array list of SolrInputDocuments
server.add(docs);
I've tried with CUSS (which swallows exceptions as expected) however they are
shown in the logs on server, and with CloudSolrServer which is returning the
errors as well as seeing them in the server logs.
I've tried downgrading my SolrJ to 4.4, still errors so looks like a
regression in the server code. Reverting to Solr 4.4 on server and I don't
get errors (however run into deadlock issues).
I raised this issue in IRC - NOT the mailing list, and elyorag suggested
opening a ticket, and to mention this has now been discussed in IRC.
The exceptions would indicate I'm attempting to do multiple operations in a
single request which is malformed. I am not, I am only attempting to add
documents.
Stack traces seen here:
14:57:13 ERROR SolrCmdDistributor shard update error RetryNode:
http://X.X.X.X:8080/solr/collection1_shard16_replica2/:org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:
Illegal to have multiple roots (start tag in epilog?).
shard update error RetryNode:
http://X.X.X.X:8080/solr/collection1_shard16_replica2/:org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:
Illegal to have multiple roots (start tag in epilog?).
at [row,col {unknown-source}]: [18,327]
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:424)
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at
org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:401)
at
org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:375)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
org.apache.solr.common.SolrException: Illegal to have multiple roots
(start tag in epilog?).
at [row,col {unknown-source}]: [7,6314]
at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:176)
at
org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:703)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:406)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at
Re: Welcome Ryan Ernst as Lucene/Solr committer
Welcome Ryan! On 14 October 2013 20:13, Ryan Ernst r...@iernst.net wrote: Thanks Adrian. I grew up in Bakersfield, CA (colloquially known as the armpit of California). I escaped and went to Cal Poly for my bachelors in computer science, and after a very brief stint working on HPUX, I landed working on the Amazon search engine for A9. I especially enjoy working with compression and encodings, and hope to experiment there some more with Lucene. Thanks Ryan On Mon, Oct 14, 2013 at 10:27 AM, Adrien Grand jpou...@gmail.com wrote: I'm pleased to announce that Ryan Ernst has accepted to join our ranks as a committer. Ryan has been working on a number of Lucene and Solr issues and recently contributed the new expressions module[1] which allows for compiling javascript expressions into SortField instances with excellent performance since it doesn't rely on a scripting engine but directly generates Java bytecode. This is a very exciting change which will be available in Lucene 4.6. Ryan, it is tradition that you introduce yourself with a brief bio. Congratulations and welcome! [1] https://issues.apache.org/jira/browse/LUCENE-5207 -- Adrien - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org -- Met vriendelijke groet, Martijn van Groningen
Re: Welcome Ryan Ernst as Lucene/Solr committer
On 10/14/2013 11:27 AM, Adrien Grand wrote: I'm pleased to announce that Ryan Ernst has accepted to join our ranks as a committer. Congratulations and welcome! -- Shawn - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Welcome Ryan Ernst as Lucene/Solr committer
Welcome Ryan! On Mon, Oct 14, 2013 at 8:54 PM, Shawn Heisey s...@elyograg.org wrote: On 10/14/2013 11:27 AM, Adrien Grand wrote: I'm pleased to announce that Ryan Ernst has accepted to join our ranks as a committer. Congratulations and welcome! -- Shawn - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5331) SolrCloud 4.5 bulk add errors
[
https://issues.apache.org/jira/browse/SOLR-5331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794381#comment-13794381
]
Shawn Heisey commented on SOLR-5331:
Solr sets the servlet container's POST buffer size limit. This can be
controlled with the formdataUploadLimitInKB attribute on the requestParsers tag
in solrconfig.xml. It defaults to 2048, or 2 megabytes.
SolrCloud 4.5 bulk add errors
-
Key: SOLR-5331
URL: https://issues.apache.org/jira/browse/SOLR-5331
Project: Solr
Issue Type: Bug
Components: SolrCloud
Affects Versions: 4.5
Reporter: Chris
Fix For: 4.5.1
Since Solr 4.5 bulk adding documents via SolrJ (at least) is causing errors.
// build array list of SolrInputDocuments
server.add(docs);
I've tried with CUSS (which swallows exceptions as expected) however they are
shown in the logs on server, and with CloudSolrServer which is returning the
errors as well as seeing them in the server logs.
I've tried downgrading my SolrJ to 4.4, still errors so looks like a
regression in the server code. Reverting to Solr 4.4 on server and I don't
get errors (however run into deadlock issues).
I raised this issue in IRC - NOT the mailing list, and elyorag suggested
opening a ticket, and to mention this has now been discussed in IRC.
The exceptions would indicate I'm attempting to do multiple operations in a
single request which is malformed. I am not, I am only attempting to add
documents.
Stack traces seen here:
14:57:13 ERROR SolrCmdDistributor shard update error RetryNode:
http://X.X.X.X:8080/solr/collection1_shard16_replica2/:org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:
Illegal to have multiple roots (start tag in epilog?).
shard update error RetryNode:
http://X.X.X.X:8080/solr/collection1_shard16_replica2/:org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:
Illegal to have multiple roots (start tag in epilog?).
at [row,col {unknown-source}]: [18,327]
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:424)
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at
org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:401)
at
org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:375)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
org.apache.solr.common.SolrException: Illegal to have multiple roots
(start tag in epilog?).
at [row,col {unknown-source}]: [7,6314]
at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:176)
at
org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:703)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:406)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023)
at
Re: Welcome Ryan Ernst as Lucene/Solr committer
Welcome Ryan! On Oct 14, 2013 9:56 PM, Simon Willnauer simon.willna...@gmail.com wrote: Welcome Ryan! On Mon, Oct 14, 2013 at 8:54 PM, Shawn Heisey s...@elyograg.org wrote: On 10/14/2013 11:27 AM, Adrien Grand wrote: I'm pleased to announce that Ryan Ernst has accepted to join our ranks as a committer. Congratulations and welcome! -- Shawn - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Should build fail if test does not exist?
I just wish more devs were able to help out on our test infra ... it shouldn't have to be you always responding to our crazy feature requests! I don't mind that at all. I am just resistant to adding things that will complicate the ant build even more than it already is... I still don't think your comparison to javac is correct. It's closer to adding a special-case exception/warning/ confirmation to rm command just for the particular scenario of it receiving a combination of -rf / arguments -- the effects are annoying, but the scenario is rare and the code mostly dead. You need to know your tools -- we should make it more explicit that these filters are glob patterns rather than hide this even deeper. And now - back to work, comrades! Dawid - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5283) Fail the build if ant test didn't execute any tests (everything filtered out).
[ https://issues.apache.org/jira/browse/LUCENE-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794394#comment-13794394 ] Dawid Weiss commented on LUCENE-5283: - Impl. note: - add an enum flag ifNoTests=warn|fail|ignore. - add a return property(ies) which could hold the number of executed/ ignored/ failed/ erroneous tests (separate from errorproperty and failureproperty which are already covered in standard junit target); this would allow regular ant fail and logical operators combination. Fail the build if ant test didn't execute any tests (everything filtered out). -- Key: LUCENE-5283 URL: https://issues.apache.org/jira/browse/LUCENE-5283 Project: Lucene - Core Issue Type: Wish Reporter: Dawid Weiss Assignee: Dawid Weiss Priority: Trivial This should be an optional setting that defaults to 'false' (the build proceeds). -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-trunk-Linux-Java7-64-test-only - Build # 63685 - Failure!
Build: builds.flonkings.com/job/Lucene-trunk-Linux-Java7-64-test-only/63685/
1 tests failed.
REGRESSION: org.apache.lucene.index.TestIndexWriterDelete.testUpdatesOnDiskFull
Error Message:
MockDirectoryWrapper: cannot close: there are still open files:
{_7_TestBloomFilteredLucene41Postings_0.tim=1,
_7_TestBloomFilteredLucene41Postings_0.tip=1,
_7_TestBloomFilteredLucene41Postings_0.pos=1,
_7_TestBloomFilteredLucene41Postings_0.doc=1}
Stack Trace:
java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are still
open files: {_7_TestBloomFilteredLucene41Postings_0.tim=1,
_7_TestBloomFilteredLucene41Postings_0.tip=1,
_7_TestBloomFilteredLucene41Postings_0.pos=1,
_7_TestBloomFilteredLucene41Postings_0.doc=1}
at
__randomizedtesting.SeedInfo.seed([145B72EB142D4F7C:70BDB4D8B027E0DA]:0)
at
org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:623)
at
org.apache.lucene.index.TestIndexWriterDelete.doTestOperationsOnDiskFull(TestIndexWriterDelete.java:698)
at
org.apache.lucene.index.TestIndexWriterDelete.testUpdatesOnDiskFull(TestIndexWriterDelete.java:489)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at
org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at
[JENKINS] Lucene-4x-Linux-Java7-64-test-only - Build # 1 - Failure!
Build: builds.flonkings.com/job/Lucene-4x-Linux-Java7-64-test-only/1/ All tests passed Build Log: [...truncated 9772 lines...] ERROR: No artifacts found that match the file pattern heapdumps/**. Configuration error? ERROR: ‘heapdumps/**’ doesn’t match anything, but ‘**’ does. Perhaps that’s what you mean? Build step 'Archive the artifacts' changed build result to FAILURE Recording test results Email was triggered for: Failure Sending email for trigger: Failure - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-4x-Linux-Java7-64-test-only - Build # 1 - Failure!
sorry for the noise - I just added tasks for 4.x and 4.5.1 next to the trunk task simon On Mon, Oct 14, 2013 at 10:10 PM, buil...@flonkings.com wrote: Build: builds.flonkings.com/job/Lucene-4x-Linux-Java7-64-test-only/1/ All tests passed Build Log: [...truncated 9772 lines...] ERROR: No artifacts found that match the file pattern heapdumps/**. Configuration error? ERROR: ‘heapdumps/**’ doesn’t match anything, but ‘**’ does. Perhaps that’s what you mean? Build step 'Archive the artifacts' changed build result to FAILURE Recording test results Email was triggered for: Failure Sending email for trigger: Failure - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-trunk-Linux-Java7-64-test-only - Build # 63685 - Failure!
I'm pretty sure this is the same root cause as the previous failure
... I'll commit a fix (from Rob!) shortly ...
Mike McCandless
http://blog.mikemccandless.com
On Mon, Oct 14, 2013 at 3:51 PM, buil...@flonkings.com wrote:
Build: builds.flonkings.com/job/Lucene-trunk-Linux-Java7-64-test-only/63685/
1 tests failed.
REGRESSION:
org.apache.lucene.index.TestIndexWriterDelete.testUpdatesOnDiskFull
Error Message:
MockDirectoryWrapper: cannot close: there are still open files:
{_7_TestBloomFilteredLucene41Postings_0.tim=1,
_7_TestBloomFilteredLucene41Postings_0.tip=1,
_7_TestBloomFilteredLucene41Postings_0.pos=1,
_7_TestBloomFilteredLucene41Postings_0.doc=1}
Stack Trace:
java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are
still open files: {_7_TestBloomFilteredLucene41Postings_0.tim=1,
_7_TestBloomFilteredLucene41Postings_0.tip=1,
_7_TestBloomFilteredLucene41Postings_0.pos=1,
_7_TestBloomFilteredLucene41Postings_0.doc=1}
at
__randomizedtesting.SeedInfo.seed([145B72EB142D4F7C:70BDB4D8B027E0DA]:0)
at
org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:623)
at
org.apache.lucene.index.TestIndexWriterDelete.doTestOperationsOnDiskFull(TestIndexWriterDelete.java:698)
at
org.apache.lucene.index.TestIndexWriterDelete.testUpdatesOnDiskFull(TestIndexWriterDelete.java:489)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at
org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at
[jira] [Updated] (LUCENE-5274) Teach fast FastVectorHighlighter to highlight child fields with parent fields
[ https://issues.apache.org/jira/browse/LUCENE-5274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nik Everett updated LUCENE-5274: Attachment: (was: LUCENE-5274.patch) Teach fast FastVectorHighlighter to highlight child fields with parent fields --- Key: LUCENE-5274 URL: https://issues.apache.org/jira/browse/LUCENE-5274 Project: Lucene - Core Issue Type: Improvement Components: core/other Reporter: Nik Everett Assignee: Adrien Grand Priority: Minor Attachments: LUCENE-5274.patch I've been messing around with the FastVectorHighlighter and it looks like I can teach it to highlight matches on child fields. Like this query: foo:scissors foo_exact:running would highlight foo like this: emrunning/em with emscissors/em Where foo is stored WITH_POSITIONS_OFFSETS and foo_plain is an unstored copy of foo a different analyzer and its own WITH_POSITIONS_OFFSETS. This would make queries that perform weighted matches against different analyzers much more convenient to highlight. I have working code and test cases but they are hacked into Elasticsearch. I'd love to Lucene-ify if you'll take them. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5268) Cutover more postings formats to the inverted pull API
[ https://issues.apache.org/jira/browse/LUCENE-5268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794426#comment-13794426 ] ASF subversion and git services commented on LUCENE-5268: - Commit 1532060 from [~mikemccand] in branch 'dev/trunk' [ https://svn.apache.org/r1532060 ] LUCENE-5268: fix test failures: bloom must first call delegate.write, then write its own Cutover more postings formats to the inverted pull API Key: LUCENE-5268 URL: https://issues.apache.org/jira/browse/LUCENE-5268 Project: Lucene - Core Issue Type: Improvement Components: core/index Reporter: Michael McCandless Assignee: Michael McCandless Fix For: 5.0 Attachments: LUCENE-5268.patch, LUCENE-5268.patch In LUCENE-5123, we added a new, more flexible, pull API for writing postings. This API allows the postings format to iterate the fields/terms/postings more than once, and mirrors the API for writing doc values. But that was just the first step (only SimpleText was cutover to the new API). I want to cutover more components, so we can (finally) e.g. play with different encodings depending on the term's postings, such as using a bitset for high freq DOCS_ONLY terms (LUCENE-5052). -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5274) Teach fast FastVectorHighlighter to highlight child fields with parent fields
[ https://issues.apache.org/jira/browse/LUCENE-5274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nik Everett updated LUCENE-5274: Attachment: LUCENE-5274.patch Removed analyzer dependency and added tests covering synonyms. Teach fast FastVectorHighlighter to highlight child fields with parent fields --- Key: LUCENE-5274 URL: https://issues.apache.org/jira/browse/LUCENE-5274 Project: Lucene - Core Issue Type: Improvement Components: core/other Reporter: Nik Everett Assignee: Adrien Grand Priority: Minor Attachments: LUCENE-5274.patch I've been messing around with the FastVectorHighlighter and it looks like I can teach it to highlight matches on child fields. Like this query: foo:scissors foo_exact:running would highlight foo like this: emrunning/em with emscissors/em Where foo is stored WITH_POSITIONS_OFFSETS and foo_plain is an unstored copy of foo a different analyzer and its own WITH_POSITIONS_OFFSETS. This would make queries that perform weighted matches against different analyzers much more convenient to highlight. I have working code and test cases but they are hacked into Elasticsearch. I'd love to Lucene-ify if you'll take them. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
